지원자에게 '어떻게 할 것인가요?' 를 묻기 vs '무엇을 했나요?' 묻기: Situational Interview와 Behavior Employment Interview 비교의 메타분석 (J Occup Organ Psychol., 2002)

지원자에게 '어떻게 할 것인가요?' 를 묻기 vs '무엇을 했나요?' 묻기: Situational Interview와 Behavior Employment Interview 비교의 메타분석 (J Occup Organ Psychol., 2002)

Asking applicants what they would do versus what they did do: A meta-analytic comparison of situational and past behaviour employment interview questions

Paul J. Taylor1* and Bruce Small2

1Chinese University of Hong Kong and University of Waikato, Hamilton, New Zealand 2AgResearch, Hamilton, New Zealand

 

 

 

 

 

Situational question(SQ, 다음과 같은 상황에서 어떻게 할 것인가?)혹은 Past behavior question (PBQ, ~한 경험을 떠올려볼 수 있나요? 어떻게 했나요?)를 활용한 구조화된 면접의 준거-관련 타당도와 평가자간 신뢰도를 분석하였다. 신뢰도와 타당도는  descriptively-anchored rating scales 을 사용했을 때, 그리고 직무 복잡성에 따라 나누었을 때를 비교했다.

Criterion-related validities and inter-rater reliabilities for structured employment interview studies using situational questions (e.g. ‘‘Assume that you were faced with the following situation . . . what would you do?’’) were compared meta-analytically with studies using past behaviour questions (e.g. ‘‘Can you think of a time when . . . what did you do?’’). Validities and reliabilities were further analysed in terms of whether descriptively-anchored rating scales were used to judge interviewees’ answers, and validities for each question type were also assessed across three levels of job complexity.

 

SQ와 PBQ 모두 높은 타당도를 보여주나, PBQ를 활용한 연구는 DARS를 사용하면 SQ를 DARS로 했을 때보다 훨씬 더 높은 타당도를 보여주었다(.63 vs .47). rating scale을 보정하고도 질문의 종류(SQ vs PBQ)는 면접 타당도의 moderator인 것으로 밝혀졌다. SQ가 높은 복잡성을 가진 직부에 덜 타당하다는 가설에 근거는 없었다.

While both question formats yielded high validity estimates, studies using past behaviour questions, when used with discriptively anchored answer rating scales, yielded a substantially higher mean validity estimate than studies using the situational question format with descriptively-anchored answer rating scales (.63 versus .47). Question type (situational versus past behaviour) was found to moderate interview validity, after controlling for whether studies used answer rating scales. No support was found for the hypothesis that situational questions are less valid for predicting job performance in high-complexity jobs.

 

DARS를 사용한 경우 SQ와 PBQ의 Sample-weighted mean inter-rater reliabilities 는 비슷했으며, DARS를 사용하지 않은 PBQ는 조금 더 낮았다.

Sample-weighted mean inter-rater reliabilities were similar for both situational and past behaviour questions, provided that descriptively-anchored rating scales were used (.79 and .77, respectively), although they were slightly lower (.73) for past behaviour question studies lacking such rating scales.

 

 

---

 

 

직무 수행능력의 결정요인에 대한 모델은 PBQ가 SQ보다 우월할 것을 기대하는 토대이다. 두 개의 수행능력 결정요인 Can do와 Will do.

In contrast, models of the determinants of job performance provide a basis for expecting past behaviour questions to exhibit superior criterion-related validity over situational questions. While various theorists have specified somewhat different variables as performance determinants (see Blumberg & Pringle, 1982; Campbell, 1990; McCloy, Campbell, & Cudeck, 1994; Vroom, 1964), all have in common two fundamental groups of performance determinants: ‘can do’ and ‘will do’ variables.

 

• ‘Can do’ variables include job knowledge, skills and abilities, while
• ‘will do’ variables primarily concern workers’ motivation to perform.

 

Campbell 등의 수행능력 결정요인 모델에서는 수행능력은 세 가지 변인의 함수이다.

• (1) 서술적 지식 declarative knowledge,
• (2) 절차적 지식과 기술 procedural knowledge and skills, and
• (3) 동기 motivation

In Campbell and colleagues’ performance determinants model (Campbell, 1990; McCloy et al., 1994), for example, performance is viewed as a function of three variables: (1) declarative knowledge, (2) procedural knowledge and skills, and (3) motivation.

 

서술적 지식은 절차적 지식과 기술의 필요조건이나 충분조건은 아니다. 한편 'motivation'은 수행능력의 직접적 결정요인이다.

Declarative knowledge is seen as a necessary, though insufficient, condition for procedural knowledge and skills, while motivation is a direct determinant of performance.

 

 

최대 수행능력(maximal performance)의 척도는 서술적 지식과 절차적 지식의 함수라는 것으로 이론화되어있는 반면, 일상적 수행능력(typical performance, 즉 상관의 평가와 같은 것)은 위의 세 가지 결정요인이 모두 포함되는 함수이다. 따라서 최대 수행능력과 일상 수행능력의 차이는 -시험 상황에서는 모든 사람이 수행의 동기부여가 되어있기 때문에 - 최대 수행능력을 측정할 때에는 개개인이 지식과 기술을 일상적 직무 수행에 적용하고자 하는 동기를 측정하지 못한다는 것이다.

Measures of maximal performance, such as job knowledge tests and work sample tests, have been theorized to be a function of declarative knowledge and procedural knowledge/skills, while measures of typical performance, such as supervisory ratings of job performance, are believed to be a function of all three performance deter- minants (McCloy et al., 1994). Thus the critical difference between maximal and typical measures of performance is that maximal performance measures fail to assess differences in individuals’ motivation to apply knowledge and skills to day-to-day job performance, since all performers are motivated to perform well during the testing situation.

 

우리는 MP와 TP를 구분하는 것과 그리고 직무 수행능력의 결정요인으로서 그들의 관계가 어떻게 면접문항의 형태를 구성하는가에 관련이 된다고 본다.

We believe that the distinction between measures of maximal and typical performance, and their relationship to the determinants of job performance, are relevant to how structured interview questions are formatted.

 

SQ는 시뮬레이션과 마찬가지로, MP의 척도로서 서술적 지식과 절차적 지식을 평가할 수 있다. 그러나 지원자가 이미 최선의 답안을 할 준비가 되어있기 때문에, 일상적 수행에 대한 지원자의 동기를 평가하지는 못한다.

Situational questions, like simulations, work samples and situational judgment tests, are measures of maximal performance, and so they can assess declarative knowledge and procedural knowledge/skills; but since interviewees are all motivated to provide the best answer possible, answers do not necessarily reflect interviewees’ motivation to apply that knowledge/skill to day-to-day job performance. For example, an interviewee who is able to describe the appropriate response to a hypothetical situation certainly demon- strates the requisite knowledge/skill, but it remains uncertain whether the individual would actually apply that knowledge/skill in an actual job situation.

 

반면 PBQ는 TP를 평가할 가능성이 높은데, 왜냐하면 지원자가 겪은 일상적 상황에 초점을 맞추고 있기 때문이며, motivation을 포함한 세 가지 결정요인을 모두 평가할 수 있다. 과거 상황에 대해서 효과적으로 대답했다고 응답한 지원자는 지식과 기술, 그리고 충분히 동기부여가 되어있었다. 고용시에 보통 관심을 갖는 것은 TP이므로 PBQ는 미래의 수행능력에 더 정확한 지표가 될 수 있다.

Past behaviour questions, however, are more likely to assess typical performance since they focus on candidates’ responses to the day-to-day situations that candidates have faced, and so they can assess all three performance determinants (including motivation). Interviewees who report that they have responded effectively in a past situation demonstrate both the necessary knowledge and skills, and also that they were sufficiently motivated to apply their knowledge/skills in that situation. Since the cri- terion of interest in employment settings is usually typical performance, past behaviour questions could be expected to provide a more accurate indication of future job performance than situational questions.

 

 

 

 

 

 

 

---

 

 

Asking applicants what they would do versus what they did do: A meta-analytic comparison of situational and past behaviour employment interview questions

1. Paul J. Taylor^1,* and
2. Bruce Small²

Article first published online: 16 DEC 2010

DOI: 10.1348/096317902320369712

2002 The British Psychological Society

Issue

Journal of Occupational and Organizational Psychology

Volume 75, Issue 3, pages 277–294, September 2002