평가자-기반 평가에서 첫인상의 역할에 대한 고찰(Adv in Health Sci Educ, 2014)

평가자-기반 평가에서 첫인상의 역할에 대한 고찰(Adv in Health Sci Educ, 2014)

Exploring the role of first impressions in rater-based assessments

Timothy J. Wood

의학은 오랜 기간 학습자의 역량을 평가할 때 선생이나 전문가의 판단에 의존해왔다. 이들을 평가자로 사용하는 것은 두 가지 요인을 반영한다. 첫째로, 좋은 의사가 되는데 필요한 스킬은 지필고사와 같은 비-평가자 기반 평가로는 쉽사리 드러나지 않는다. 둘째 요인은 어떻게 의사가 훈련되느냐와 관계되어 있다. 학습자가 임상환경에서 수행하는 능력이 관찰되상이 되며, 이는 훈련과정 중 하나이다.

Medicine has a long history of assessing the competence of learners by relying on the judgments of teacher and/or experts. This use of these people as raters is likely a reflection of two factors. First, the skills that make a good physician do not necessarily lend themselves easily to non-rater based assessments methods like written examinations. The second factor relates to how physicians are trained. Learners are observed in clinical settings as part of their training,

최근, 역량-바탕 프레임워크를 적용하여 학습자를 평가할 것이 권고되고 있다. 이 평가 프레임워크는 근무지 평가 뿐 아니라 피드백의 활용을 강조하는데, 이 둘 모두 평가자 역할의 중요성이 매우 강조된다.

More recently, there has been an increased push to adopt a competency- based framework to assess the skills of learners (Holmboe et al. 2010). This assessment framework emphasizes the use of feedback as well as workplace assessments, both of which require observation thus further highlighting the critical role of the rater.

그러나 안타깝게도, 모든 인간은 선입견과 편견을 가지고 있고, 이것이 학습자의 역량을 평가할 때 그 평가의 퀄리티에 영향을 끼친다.

Unfortunately, all humans have preconceived notions, biases and abilities that influence the quality of the judgments they make when assessing the competence of learners (Gige- renzer and Gaissmaier 2011; Hoyt 2000; Landy and Farr 1980; Saal et al. 1980, 1974; Williams et al. 2003).

의학교육에서 활용되는 평가가 가치를 가지려면(타당하고 신뢰성 있으려면), 사람들이 타인에게 점수를 매기는 인지적 프로세스를 이해하는 것이 중요하다. 실제로, 모든 평가에 있어서 이러한 종류의 정보를 수집하는 것은 chain of validity evidence의 한 부분이다.

To ensure the assessments that are used in medical education have value (i.e., are valid and reliable), it is crucial that we understand the cognitive processes behind howpeople assign scores when assessing others. In fact, the collection of this type of information is considered part of the chain of validity evidence one should collect with regards to any assessment (AERAet al. 1999; Clauser et al. 2008; Cook and Beckman 2006; Downing and Haladyna 2009).

특히 관심의 대상이 되는 것은 '첫인상', '단편적 판단', '아는바 없음' 와 같은 판단이다.

Of particular interest is the impact of judgments often referred to as ‘‘first impression’’, ‘‘thin slice’’ or ‘‘zero acquaintance’’ judgments.

첫인상

First impressions

이러한 타인에 대한 판단을 '인상'이라 부르며, 우리가 타인의 인성과 행동에 관한 정보를 인지하고 조직하고 통합하는데 도움을 주는 카테고리이다. 첫인상은 빠르게 만들어지는 인상으로서, 누군가를 만나고 5분 내에 형성된다. 첫인상은 첫인상 판단이 내려지는 시점이 매우 빠르고 제한된 정보에 따라 내려진다는 점을 감안하면 놀라울 정도로 정확하다.

These judgments about others are called impressions, which are categories that we use to help us perceive, organize and integrate information about an individual’s personality and behavior (Feldman 1981; Fiske and Neuberg 1990; Gingerich et al. 2011). First impressions are a type of impression that is made quickly, usually within 5 min of meeting someone for the first time. First impressions have been found to be surprisingly accurate given how quickly they form and the limited information on which they are based (Ambady and Rosenthal 1992; Ambady 2010; Harris and Garris 2008).

 

 

첫인상 뒤에 숨겨진 인지절차는 무엇인가?

What are the cognitive processes behind a first impression?

많은 인지적 활동(의사결정, 추론, 카테고리화, 기억) 등은 두 가지 절차에 따라 이뤄진다. 시스템1과 시스템2 프로세스이다. 일반적으로 시스템1은 빠르고, 노력이 덜 들며, 비-분석적이고, 자동적이고, 무의식적이며, 시스템2는 느리고, 노력이 들고, 분석적이고, 통제되며, 의식적이다.

Many cognitive activities including decision making, reasoning, categorization, and memory are thought to consist of two underlying processes; what have come to be known as System 1 and System 2 processes (Evans 2008; Uleman et al. 2008; Kahneman 2011). It is generally accepted that System 1 processes are rapid, effortless, non-analytic, automatic, and/or unconscious, whereas System 2 processes are slow, effortful, analytic, controlled, and/or conscious.

• 시스템1: 강아지나 고양이 라는 단어를 읽는 것 cat, dog.
• 시스템2: 이러한 단어를 읽는 것 parasito- logical, incudostapedial.

시스템1과 시스템2 모두 많은 인지활동에 활용될 수 있으며, 인지심리학과 사회적판단, 의사결정에 관한 연구에서 우리가 어떻게 이 두 가지 프로세스를 조화시키는지 이해하려고 노력해왔다.

Both System 1 and System 2 processes can be used to perform many of the cognitive activities listed above; therefore the focus of research in cognitive psychology and social judgment and decision making is to try to understand how we coordinate these two processes (see Brooks 2005; DeNisi et al. 1984; Fiske and Neuberg 1990; Jacoby 1991; Kahneman 2011; Norman 2009; Schneider and Chein 2003 for examples in these areas).

첫인상은 주로 시스템1 프로세스를 반영한다.

First impressions are thought to reflect primarily System 1 processes

만약 첫인상에 대한 이러한 가정(시스템1 프로세스)이 사실이라면, 무의식적 프로세스에 의존하는 과제에 있어서, 사람들이 자신이 그 과제를 어떻게 수행했는지를 말로 설명하는 것은 어려울 것이다.

If this assumption about first impressions is true, for tasks that rely on unconscious processes, it should be difficult for people to accurately verbalize how they performed a task

 

어린아이에게 자전거를 어떻게 타는지 설명해주는 것이 얼마나 어려운가를 생각해보라

One just has to think of how hard it is to explicitly verbalize to a child the steps needed to ride a bicycle to realize this occurs.

첫인상과 관련된 인식의 수준(level of awareness)를 본 연구는 적다. 이 중, 정확성과 자신감(accuracy and confidence)에 대한 것이 있다. 예컨대, 정확성과 자신감은 낮지만 정적 상관관계에 있다는 연구가 있음.

there have only been a few studies that have looked at the level of awareness associated with first impressions. Of the work that has been done, the focus has been on the relationship between accuracy and confidence. For example, (Smith et al. 1991) found a low but positive relation between accuracy and confidence levels.

유사하게, 정확성과 자신감의 관계는 평가자가 자신의 평가에 자신감이 전혀 없을 때 가장 높았는데, 왜냐하면 이 때 평가자는 자신이 평가하는 대상에 대해서 생각이 없었no idea기 때문이다.

Similarly, (Ames et al. 2010), the correlation between accuracy and confidence was highest for those raters with no confidence in their rating because they knew when they had no idea about a personality judgment.

비록 사람들이 다른 사람에 대해서 판단하는 것이 가져올 결과를 알고 있더라도, 사람들은 어떻게 그 판단을 내렸는지 설명하는 일을 어려워하며, 그 판단을 어떻게 내렸는가에 대한 통찰insight을 거의 가지고 있지 않다

The conclusion from both of these studies is that, although people may be aware of the outcome of forming a judgment of others, they appear to have difficulty articulating how they did it and/or have little insight into how they actually made that judgment.

Biesanz 등은 자신감과 정확성 사이에 관계가 작다는 것이 lack of awareness를 반영한다는 결과에 의문을 표했다. 이들은 사람들이 비록 스스로 첫인상 판단이 얼마나 정확한가에 대해서는 잘 모르더라도, 그들이 첫인상에 대한 판단을 내리는 시점을 인지하고 있다고 주장한다.

Recently, Biesanz et al. (2011) questioned the finding that a low relationship between confidence and accuracy in first impression judgments reflects a lack of awareness. They argued that people are aware at the time when individual first impression judgments are accurate even if they do not know how accurate their judgments are globally.

첫인상이 시스템1 프로세스를 사용함을 반영하는 또 다른 연구패턴은 이 판단이 빠르게 내려진다는 것이다.

Another pattern of results that one should expect if a first impression reflects the use of System 1 processes is that the judgments should be made quickly.

Willis and Todorov 는 사진에 있는 사람을 보고 성격에 대한 인상을 정확하게 판단할 때, 100ms만 보고서도 시간제한없이 본 것과 비슷한 정확도로 판단할 수 있음을 밝혔다.

Willis and Todorov (2006). found that people can produce as accurate an impression of the personality traits associated with a person in a photograph after 100 ms exposure as they do when viewing the same photo- graph with no time constraints.

 

Dodson 등은 OSCE의 평가자에게 피평가자를 5분 시점에서, 그리고 8분 시점에서 평가하게 했다. 5분 시점에서 평가한 결과는 8분 시점에서 평가한 결과보다 점수가 낮았으나, 둘 사이의 상관관계는 높았으며, 5분 시점이 평가로도 점수의 신뢰도는 낮아지지 않았다.

Dodson et al. (2009) asked examiners on an admissions OSCE to provide a rating of the examinee’s abilities at the 5 min mark and then again at the 8 min mark. Ratings at 5 min were lower than ratings at eight minutes, but the correlation between ratings at the two time points was high (r = 0.82–0.91) with no drop in reliability for scores at the 5 min mark.

Govaerts 등은 경험이 많은 평가자와 경험이 거의 없는 평가자에게 피평가자의 비디오를 보게 하였는데, 모든 경우에서 평가자들은 5분 내에 판단을 내릴 수 있다고 생각했다.

Govaerts et al (2011; see also Govaerts et al. 2013) asked experienced and inexperienced examiners to watch two videos of a trainee with a patient. In all conditions, therefore, the examiners thought they could judge the performance in under 5 min.

첫인상이 시스템1 프로세스를 사용한다고 가정한다면, 세 번째 특징은 판단을 내릴 때 인지적 자원의 소모가 거의 없어야 한다는 것이다. 인지심리학에서 과제의 자동화를 연구하는 한 가지 흔한 방법은 divided attention task를 사용하는 것.

A third characteristic that one would expect if first impressions reflect System 1 pro- cesses is that the judgment should require few cognitive resources to operate. In cognitive psychology, one common method used to study the automaticity of a task has been to use a divided attention task;

 

이러한 논리에 따라서, 또 다른 과제를 동시에 하게끔 하여도 초기 판단의 정확성이나 수행능력에는 차이가 없었다.

By this logic, introducing a simultaneous task will have little impact on the accuracy or performance associated with the initial judgment.

Patterson and Stockbridge 는 첫인상을 높은 인지적부하 조건에서 판단하게 한 그룹에서 오랜 시간 숙고하게 한 그룹보다 더 정확한 판단을 내렸으며, 이는 첫인상이 시스템1 프로세스를 사용한다는 것을 기대하게끔 한다.

Patterson and Stockbridge (1998). found that participants in the high cognitive load condition were more accurate in the first impression group compared to the deliberative group, a finding one would expect if a first impression judgment relied primarily on a System 1 process.

Ambady는 과목에 대한 과목 초반의 평가와 최종 평가의 상관관계가 낮음을 보고하면서, 인지부하조건에 비해서 통제조건 혹은 지연조건은 아무런 차이가 없음을 밝혔다.

Ambady found a low correlation between the initial ratings and the final course ratings in the reasons condition (r = 0.27) but no differences between the control and delay conditions compared to the cognitive load condition (r = 0.65–0.71). This pattern is what would be expected if participants were relying primarily on a System1 process to make their initial judgments.

요약하자면, 첫인상에 깔린 인지프로세스에 대한 연구를 보면 첫인상은 시스템1 프로세스에 의존하는 것으로 보이며, 왜냐하면 평가자는 흔히 자신들이 어떻게 그 인상을 형성했는지 인지하지 못하며, 그 판단은 빠르게 내려지고, 인지적 부하를 가한 조건에 민감하게 반응하지 않았기 때문이다.

In summary, research looking at the underlying cognitive processes behind first impressions would suggest that first impressions likely reflect the reliance on a System 1 process because raters are typically unaware of how they created an impression, the impressions are made quickly, and they are not sensitive to manipulations that add com- peting attentional demands.

첫인상은 얼마나 정확한가?

How accurate are first impressions?

이 질문에 대한 대답은 논란이 있다. 듀얼-프로세스 모델에 곤한 대부분의 연구는 사람들이 시스템1 프로세스에 의존하면 오류를 일으킬 확률이 높다는 것에 초점을 둔다. 시스템1 의존 시에 늘어나는 에러는, 평가자들이 (그들을 심사숙고하고 분석적으로 만들어주는 것보다) 휴리스틱, 기억 인출, 인지적 편향과 같이 에러를 유발하는 것들에 영향을 받는 경향이 높기 때문이라고 설명한다. 이러한 관점에서, 첫인상은 에러에 취약하며, 여기에 기반한 판단은 지양되어야 한다.

The answer to this question is debatable. Much of the literature on dual process models (Evans 2008; Croskerry 2009; Kahneman 2011; Tversky and Kahneman 1974) has focused on the increase in errors that occur when people rely on System 1 processes. The explanation for the increase is that when relying on System 1 processes, raters are more likely to be influenced by heuristics, memory retrieval or other cognitive biases which lead to errors compared to processes that are more deliberative and analytic. From this perspective, first impressions should be prone to errors, and judgments based on them should be avoided.

시스템1 프로세스가 시스템2 프로세스보다 더 에러를 발생시킬 가능성이 높다는 근거에도, 일부 연구자들은 이 연구결과의 일반성에 의문을 표한다. 예컨대, 임상추론 연구를 리뷰하여, 일부 연구자들은 임상문제에 대해서 (빠른 반응은 정확하나) 늦은 반응slow response은 오류를 만들어내는 상황들을 찾아내었다.

Despite evidence that System 1 processes can lead to more errors than System 2 pro- cesses, some researchers have challenged the generality of these results (Eva and Norman 2005; Gigerenzer and Gaissmaier 2011; Klein 2009). For example, (Norman 2009; also Sherbino et al. 2012), in a review of clinical reasoning studies, described situations in which errors were associated with slow responses to clinical problems, whereas fast responses were more accurate.

또 다른 연구는 실험실 세팅에서 내린 틀린 판단이 실제 상황에서는 옳은 판단이었을 수 있다는 것에 초점을 둔다. Mu¨ller-Lyer Illusion 을 시스템1과 시스템2에 따라서 해석한 예시가 있다.

Other researchers have suggested that rating-based research needs to be focused on what people can do in more naturalistic settings or with more realistic stimuli because a wrong judgment in the laboratory may be a correct judgment in the real world. This distinction is best demonstrated by considering how the Mu¨ller-Lyer Illusion is interpreted in terms of System1 and System2 processes.

즉, '착각'이 반드시 판단의 오류를 의미하는 것은 아니라는 점이다.

In other words, the illusion does not reflect an error of judgment.

정확성에 대한 또 다른 논점으로는, '정확성'이라는 것이 다양한 사회적판단 연구에서 상대적인 개념이며, 왜냐하면 무엇을 옳고 그르다고 정의내리는 황금률은 존재하지 않기 때문이다. 이보다는 정확성이 '판단'에 기초하고 있으며, 그 판단이란 '동의'혹은 '예측'에 의존한다고 보는 것이 옳다.

Another comment about accuracy is needed. Accuracy is a relative concept in many social judgment studies because a gold standard that clearly defines right or wrong does not exist. Rather, accuracy is based on a judgment, which may rely on agreement or prediction (Funder 1987; Funder and West 1993; Kenny 1993).

'동의agreement'에 있어서, 평가자 평가와 자기평가를 비교한 것이나 같은 준거로 다른 평가자의 평가와 비교한 연구 등이 있다.(self-other agreement / consensus rating)

In the case of agreement, studies usually compare ratings made by a rater to those made by the target (self-other agreement) or to a rating made by other raters using the same criteria (consensus rating).

'예측prediction'에 있어서, 동일한 혹은 다른 준거에 따라 미래의 결과를 예측하는지 보는 것이다.

In the case of prediction, the ratings are used to see if they predict a future result based on either the same or different criteria.

Funder가 주장한 바와 같이, 이 분야의 연구는 '상관관계의 크기가 아니라, 판단이 더 정확해지는지 아니면 덜 정확해지는지'를 연구해야 한다.

As argued by Funder (1987), research in this area should study circumstances in which judgments become more or less accurate rather than focus on the magnitude of the correlation.

판단의 정확성이 상대적이라는 관점에서, 첫인상에 대한 연구는 정확도에 있어서 다양한 결과를 보여주었다. Barrick 등은 짧은 라뽀 세션rapport session에 기반한 첫인상과 인터뷰 점수가 중등도의 상관관계를 가짐을 보여주었다.

In light of the argument that accuracy is relative, studies of first impressions have shown considerable range in terms of the degree of accuracy. Barrick et al. (2010) found a moderate correlation between first impression based on a short rapport session and an interview score (r = 0.42).

요약하자면, 듀얼-코드 이론가들에게 공통적인 결론은 시스템1 프로세스에 의존하는 것이 (첫인상을 포함해서) 판단의 오류를 유발할 수 있으며, 우리는 이를 경계해야 한다. 연구자들은 이에 대하여 두 가지 반응을 보인다. 첫째로, 이러한 패턴이 모든 경우에 있어서 옳지는 않으며, 느리고 숙고하는 프로세스에 기반한 판단이 더 에러를 일으키는 경우도 있다. 둘째로, 연구자들은 에러를 일으키는 요인들이 가지는 가치가 있는지 의문을 표하며, 실험실을 벗어나면 시스템1 프로세스를 사용한 판단이 오히려 더 정확할 수 있음을 지적한다. 또한 대부분의 판단에 있어서 정확성은 - 첫인상을 포함하여 - 상대적인 개념이며 절대적인 옳고 그름의 황금률은 없다. 이러한 상대성을 고려한다면, 어떤 프로세스가 에러를 유발하는가를 보는가에만 초점을 두기보다는 판단의 정확성이 높아지거나 낮아질 수 있는 조건을 연구하는 것이 나을 것이다.

In summary, a common perspective from dual code theorists is that reliance on System 1 processes, including first impressions, can lead to errors in judgment and that we need to be wary of relying on these processes when making judgments. Researchers have had two responses to this perspective. First, the pattern is not necessarily true in all cases and it has been shown that judgments based on slow deliberative processes can be more error-prone than those made on first impressions. Second, some researchers have questioned whether studies of factors that produce errors are of value, and point out that often errors made in the laboratory using System 1 processes are actually correct judgments when studied outside the laboratory. In addition, accuracy of most judgments, including first impres- sions, is relative because there is often no gold standard that determines right from wrong. Given this relativity, it may be more fruitful to study conditions that cause accuracy to increase or decrease rather than focus solely on whether one process leads to errors.

첫인상의 정확도에 영향을 주는 요인은 무엇인가?

What factors modify the accuracy of a first impression?

첫인상의 정확도에 영향을 주는 요인을 찾는 것이 중요하다. Gingerich 등은 평가자의 기분, 평가자가 알던 다른 사람과의 유사성, 사전에 접한 정보 등을 지적했다. 예컨대, 피평가자의 관찰가능한 성격 (외향성 등)은 덜 관찰가능한 성격 (신경증, 개방성) 등에 비해서 더 정확하게 평가가능하다.

An examination of other factors that could modify the accuracy of a first impression would be of value. Gingerich et al. (2011) has reviewed some of these factors within the larger impression formation literature and they include: mood of the rater, similarity to other people the rater knows, and seeing information in advance. For example, with regard to the people being rated, observable personality traits like extroversion are typically judged more accurately than less obser- vable traits like neuroticism or openness (Ambady et al. 1999; Borkenau and Liebler 1992; Lippa and Dietz 2000).

평가자-기반 요인에 있어서..지능intelligence, 젠더, 평가자의 기분mood (슬픈 평가자가 덜 정확하다)

With regard to rater-based factors,

• intelligence has been identified as a factor that could influence the accuracy of first impressions judgments.
• Gender has also been identified as a potential factor that can influence the accuracy of judgments based on first impressions (Ambady et al. 1995; Chan et al. 2011; c.f. Lippa and Dietz 2000).
• Another rater-based factor that can influence the accuracy of a first impression is the mood of the rater, with sad raters having less accurate first impressions than happy raters (Ambady and Gray 2002).

인상 관리impression management와 안정성stability에 대한 연구. 인상 관리란 직업 면접에서 흔히 연구되며, 피면담자가 면담자와의 상호작용을 컨트롤하여 영향을 주고자 하는 것.

Another issue related to factors that could influence the accuracy of a judgment based on a first impression is related to impression management and the stability of the first impression. Impression management is most commonly studied in the job interview lit- erature and refers to situations in which interviewees attempt to influence an interviewer by controlling the interaction between themselves and interviewer. Barrick et al. (2009) found evidence that

• 외모 appearance (i.e. physical and professional),
• 인상관리 impression management (i.e., self-promotion, ingratiation to the interviewer, emphasizing positives, focusing attention on the interviewer), and
• 언어/비언어적 특성 verbal (voice) and non-verbal (smiling, eye contact) characteristics

 

can all have an influence on the impression a rater may form.

 

 

요약하면, 첫인상에 영향을 줄 수 있는 요인은 다양하다. 이들 중 일부는 판단의 대상이 되는 사람과 관련되어있다. '평가자와 얼마나 유사한가'와 같은 비의도적인 요인들 뿐 아니라 '피평가자가 평가자가 받는 인상을 관리하려는 노력'과 같은 의도적 요인들도 있다. 젠더/지능/기분 등이 관련된다. 외향성은 더 판단하기 쉬운 특성이다.

In summary, a number of factors were identified that can influence the accuracy of a first impression. Some of these factors are related to the person being judged: either uninten- tional factors like similarity to the rater or intention factors like those deliberately used by ratees to manage impressions raters may create. Other factors that influence accuracy like gender, intelligence, or mood are related to the rater. Finally, some traits like extraversion appear to be easier to judge than other traits.

 

평가에 있어서 첫인상의 영향력은?

What is the impact of first impressions for assessment?

첫인상 연구의 대부분은 다음 등이다

The majority of studies of first impressions have focused on

• the ability of raters to make a personality judgment of some kind,
• rate the abilities of a teacher, or
• predict the success of a job interview.

'자기충족적 예언' 혹은 '예언효과'와 관련된 것이다. 이는 첫인상이 이후의 평가자와 피평가자의 관계에 영향을 준다는 것이다.

The first area deals with a phenomenon called self-fulfilling prophecies or an expectancy effect. This phenomenon occurs when an initial impression influences subsequent interactions between the rater and the person being rated (Dipboye 1982; Harris and Garris 2008; Rosenthal 1994).

Snyder 등은 남성 참가자가 부정적 기대를 가지고 있으면, 여성 참가자를 덜 친절하게 대하고 여성으로부터 부정적 반응을 얻는다.

Snyder et al. concluded that if male participants had negative expectations, they treated the female participants in a less friendly manner, getting a negative reaction from the females.

유사하게, 직무 면접에서 Dougherty 등은 긍정적 첫인상이 면접관의 긍정적 커뮤니케이션 스타일과 연결되며, 합격 가능성이 높아지고, 더 긍정적인 보컬 스타일과 연결된다.

Similarly, in a study using job interviews, Dougherty et al. (1994) found that positive first impressions were related to more positive communication styles by the interviewer, increased likelihood to extend an offer, and more positive vocal style.

두 번째로, 첫인상과 후광효과에 대한 것이다. 후광효과는 평가자가 피평가자를 판단할 때 '독립적인 특성들 간' 분별에 실패할 때 발생한다. 후광효과는 모든 평가 영역간 상관관계가 다 높게 나타나는 식으로 드러나거나, 혹은 평균 SD가 작은 방식으로 드러난다. 이는 다양한 측면dimensions에 걸쳐서 한 가지 요인이 모든 variability를 설명할 수 있는 경우이며, 또는 유의미한 평가자-피평가자 상호작용 rater-ratee interaction이 발견되는 경우이다.

The second area to which first impressions could impact on assessment is a type of rater bias called a halo effect. A halo effect is thought to occur when a rater fails to discriminate among independent aspects of behavior when making a judgment about a person. Halo is typically manifested as either high average correlations across all dimensions being assessed, low average standard deviations across all dimensions being assessed, when a single factor accounts for all of the variability in scores across multiple dimensions, or when a significant rater 9 ratee interaction is found (Balzer and Sulsky 1992; Cooper 1981).

후광의 원인에는 여러가지가 있다.

Several sources of halo have also been identified by researchers.

• 일반적 인상general impression이 이어지는 모든 판단에 영향을 주는 경우
  The first source of halo occurs when a rater makes a judgment about a person based on a general impression (e.g. a first impression) that they form. This impression then influences all subsequent ratings or judgments about the person. For example, if a rater forms a first impression of a learner that is either positive or negative in nature, then this impression will guide the ratings on all dimensions being rated.
• 한 영역에서 두드러지는 특징salient dimension이 다른 영역에도 줄줄이 영향을 미치는 것
  The second source of halo occurs when a salient dimension or trait drives the ratings on other dimensions being judged. For example, a high or low rating on communication skills could influence ratings on other dimensions, even those that may be unrelated, like technical skills or knowledge.
• 평가대상이 되는 영역 간의 분간에 실패한 것inadequate discrimination between dimension
  A third source of halo is an inadequate discrimination between dimensions being rated. This source of halo usually occurs when the dimensions being rated are ambiguous and raters end up grouping what are intended to be unrelated dimensions and providing similar ratings.

후광효과가 평가의 정확성/비정확성을 가져오는가? 후광의 존재는 시스템1 프로세스에 따른 것으로 이해되며, 따라서 첫인상의 정확도에 대한 것과 같이 논쟁의 여지가 있다.

Does the presence of a halo effect lead to accurate or inaccurate ratings? The presence of halo is considered to be due to a System 1 process (i.e., use of a general impression or memory of behaviors rather than independent ratings) and therefore, like the discussion around the accuracy of first impressions, the accuracy of judgments influenced by a halo is debatable.

Cook 등은 후광효과와 정확성간 차이가 거의 없다고 밝힘. 더 연구 필요

Cook et al. (2008) found similar results as the Bernadin and Pence study in that there was little difference in halo and accuracy between raters who were trained and those in a control group. It would appear, therefore, that the relationship between halo and accuracy is an area that warrants further research to understand the conditions that influ- ence this relationship.

요약하자면, 첫인상은 두 가지 방향으로 영향을 줄 수 있다. 자기충족적 예언, 그리고 후광효과
In summary, first impressions may influence the types of assessments used in medicine in two ways. It could contribute to a self-fulfilling prophecy in which negative or positive first impressions influence the way a rater thinks about or interacts with a target. It could also contribute to the presence of a type of rater bias called the halo effect because one of the causal mechanisms behind halo is the use of a general impression by a rater when making a judgment about a target.

결론과 함의

Conclusion, implications for assessment in medical education

Factors related to impression formation (Gingerich et al. 2011), cognitive load (Tavares and Eva 2013; van Merrie¨nboer and Sweller 2010; Wood 2013), familiarity with the examinee (Stroud et al. 2011), rater expertise (Berendonk et al. 2013) as well as rater-biases (Ira- maneerat and Yudkowsky 2007; Williams et al. 2003) and overly structured assessments within competency-based frameworks (Ginsburg et al. 2010) have all been identified as influencing the way assessors assign scores.

1) 첫인상이 평가에 얼마나 영향을 주는가?

1) To what degree will first impressions influence subsequent ratings within a particular assessment context or tool?

영향을 준다는 것은 확실해 보이(is related to subsequent scores)나, 다양한 맥락에서의 확인이 필요

The basic finding, that first impressions are related to subsequent scores, is compelling but requires demonstration in a variety of contexts.

2) 한 평가 상황내에서도 첫인상이 바뀌는가?
2) Do first impressions change within the context of a single assessment session and if so under what conditions?

OSCE에서 초반에 못하다가 점차 회복하는 학생들이 있다. 그러나 좀 더 rigorous한 연구가 필요
Anecdotally, many physician examiners can describe an examinee that started off an OSCE station or oral examination badly and then recovered brilliantly. These stories suggest that impressions can change, but such anecdotal evidence must be supported by rigorous research. The stability of a first impression is particularly important for examinations like OSCEs,

만약 판단이 첫 몇분간 끝난다면, 평가 시간이 길어지는 것이 평가의 퀄리티에 주는 영향이 없을 것이다.

If a judgment about the examinee’s ability is made within the first couple of minutes, and that judgment remains stable throughout the assessment despite a change in an examinee’s performance, then longer assessments may not be adding anything to the quality of the rating that one cannot get within a few minutes.

3) 시스템1과 시스템2 프로세스의 조화

3) How does the coordination of System 1 and System 2 processes influence the use of and the accuracy of a first impression?

어떤 경우에는 시스템1이 더 정확

Under some circumstances, System 1 processes, like first impressions, can lead to more accurate judgments than System 2 processes, but it is not clear under what conditions this may occur.

무슨 평가방법을 사용하느냐

One such factor is the scoring method used. There is a considerable amount of literature on the advantages and disadvantages of using checklists versus rating scales for assessments (Hawkins and Boulet 2008; Van der Vleuten and Swanson 1990).

• 체크리스트: 고도로 심사숙고하는deliberative 평가법. 시스템2를 활용함 A checklist is a highly deliberative scoring process so would likely reflect the use of System 2 processes.
• 평가스케일Rating scales: 시스템1의 역할이 더 커질 수 있음(평가자가 해석할 여지가 많고 덜 rigid함). Rating scales, on the other hand, have more room for rater interpretation and are less rigid, so could allow a larger role for System 1 processes like first impressions to influence scoring.

 

평가의 목적이 무엇이냐

The purpose of the assessment (i.e., formative or summative assessment), is also important in terms of whether System 1 or System 2 processes should be favored.

• 형성적 피드백을 위한 평가는 더 심사숙고해야하고 분석적 채점 프로세스를 위한 설계
  It is possible that an examination designed for formative feedback might favor a deliberative, analytical scoring process in order to provide feed- back,
• 총괄평가를 위한 평가는 더 global하고 덜 analytic함.
  whereas an examination designed solely for summative assessment may favor a more global, less analytical scoring process.

 

인지부하: 어떤 과제는 인지적 자원을 더 필요로 함. 예를 들면 응급실에서 피평가자의 병력청취, 의사소통기술, 프로페셔널리즘을 판단해야 하는 경우

Cognitive load is another factor that would likely influence the use of and accuracy of first impressions. Because some rating tasks require a higher degree of cognitive resources (i.e., attention) than other tasks, the resulting scores could start to mimic the results found under divided attention manipulations described earlier. For example, imagine a situation in which a rater must evaluate an examinee’s history taking, communication skills and professionalism while they interact with a live patient in a busy Emergency Department.

4) 자기충족적 예언이 얼마나 평가에 영향을 주는가?

4) To what degree does a self-fulfilling prophecy influence the ratings?

5) 후광효과와 첫인상의 관계

5) What is the relationship between first impressions and the halo effect?

 

상황에 따라 다름;

First impressions are thought to contribute to the presence of a halo effect.

• Under some circumstances, especially when one wants to identify specific strengths and weakness within a person, the presence of halo would make the assessment difficult.
• In other cir- cumstances, especially when trying to discriminate abilities between individuals, the presence of halo may actually be a benefit due to the high reliability.

What is unclear is what those circumstances are, and how manipulations that influence first impressions impact on the presence or absence of halo.

checklist vs rating scale

First impression ratings could be compared to a condition in which examiners score examinees using a checklist versus a condition in which they use a rating scale. If rating scales support the use of System 1 processes and checklists facilitate System 2 process, one might find a larger correlation with the former scoring system.

---

Adv Health Sci Educ Theory Pract. 2014 Aug;19(3):409-27. doi: 10.1007/s10459-013-9453-9. Epub 2013 Mar 26.

Exploring the role of first impressions in rater-based assessments.

Wood TJ1.

Author information

• 1Academy for Innovation in Medical Education (AIME), RGN2206, Faculty of Medicine, University of Ottawa, Ottawa, ON, K1H-8M5, Canada, twood@uottawa.ca.

Abstract

Medical education relies heavily on assessment formats that require raters to assess the competence and skills of learners. Unfortunately, there are often inconsistencies and variability in the scores raters assign. To ensure the scores from these assessment tools have validity, it is important to understand the underlying cognitive processes that raters use when judging the abilities of their learners. The goal of this paper, therefore, is to contribute to a better understanding of the cognitive processes used by raters. Representative findings from the social judgment and decision making, cognitive psychology, and educational measurement literature will be used to enlighten the underpinnings of these rater-based assessments. Of particular interest is the impact judgments referred to as first impressions (or thin slices) have on rater-based assessments. These are judgments about people made very quickly and based on very little information. A narrative review will provide a synthesis of research in these three literatures (social judgment and decision making, educational psychology, and cognitive psychology) and will focus on the underlying cognitive processes, the accuracy and the impact of first impressions on rater-based assessments. The application of these findings to the types of rater-based assessmentsused in medical education will then be reviewed. Gaps in understanding will be identified and suggested directions for future research studies will be discussed.

Comment in

• Are we willing to change our impression of first impressions? [Adv Health Sci Educ Theory Pract. 2014]

PMID:

 

23529821

 

[PubMed - in process]