개인적인 특징(인종, 성별, 사회경제적 지위)이 면접 점수에 영향을 줄까?

면접의 평등 : 개인적인 특징(인종, 성별, 사회경제적 지위)이 면접 점수에 영향을 줄까?

Equity in interviews: do personal characteristics impact on admission interview scores?

Andrew B Lumb,1 Matthew Homer1 & Amy Miller2

배경 

일부 사회 계층이 의대 선발 시스템상 불이익을 받고 있다는 연구결과가 있다. 선발의 각 단계 중 어느 단계에서 이런 일이 벌어지는지는 알 수 없지만, 서로 면대 면으로 만나게 되는 면접 상황에서는 이러한 사회적 비뚤림(bias)가 생길 수 있다.

방법

우리는 영국 의과대학의 한 해의 입학에서 이뤄진 면접에 대한 자세한 조사를 하였다. 면접관과 응시자에 대한 개인적인 성향을 조사하여 이들 중 어떤 요인이 면접관-응시자 매칭에 따라서 면접 점수에 영향을 주는가를 보았다.

결과

총 320명의 면접관이 734명의 응시자를 평가하였고, 2007년의 면접관-응시자 상호작용에 대한 자료를 분석하였다. 일반화 이론(generalizability theory)에 따른 면접의 신뢰도는 0.82-0.87이었다. 면접관과 응시자 모두에서 성별, 인종, 사회경제적상태, 학적에 따라서 받은 점수와 준 점수의 차이는 없었다. 스텝면접관과 학생면접관의 점수간에도 유의미한 차이가 있지 않았다. 각각의 스텝면접관 그룹은 그 수가 너무 작아서 통계적 분석을 할 수 없었지만, 서로 다른 전공을 가진 면접관 또는 서로 다른 면접경험을 가진 면접관 사이에 유의미한 차이는 없었다.

결론

이 연구결과로부터 '면접' 단계가 일부 사회적 그룹이 의과대학 선발에서 불이익을 받는 단계는 아닌 것으로 나타났다. 이러한 결과는 또한 시니어 의과대학 학생을 면접에 참여시키는 것을 지지해준다. 면접 점수가 미래의 학업적, 임상적 성공을 예측하는데 유용하다는 근거는 부족하지만 대부분의 의과대학은 면접을 선발 과정에서 중요하게 활용하고 있다. 우리의 연구는 면접이 선발 과정에 사회적 bias를 증가시키는 것은 아님을 보여주었다. 

---

At interview, the candidates are assessed in five separate areas covering: 

insight into a career in medicine; 

responsibility; 

social and cultural awareness; 

non-academic achievements, 

interpersonal skills. 

These five areas are designed to cover the personal qualities of applicants that are regarded by the admissions committee as desirable characteristics in future doctors 

(see http://www.leeds.ac.uk/medicine/admissions/personal.html for further details). 

The constructs explored in these areas have been evolved over several years through continuous development and monitoring by the admissions committee of the medical school.

---

The reliability of the interviewing process was calculated using variance components MINQUE methods in SPSS Version 15 (SPSS, Inc., Chicago, IL, USA), treating interview scores as the dependent variable and both interviewee and interviewer as random effects in a mixed-effects linear model.7 This allows a generalisability coefficient to be calculated as the proportion of the variance in the interview scores that can be properly attributed to the interviewees, with all non-interviewee variance treated as error. 

Data analysis was carried out in three separate parts: 

(i) analysis of interviewee performance; 

(ii) analysis of interviewer performance, and 

(iii) analysis of interviewee–interviewer interactions. 

In all three parts, the potential effect of dependency in the data (interview scores are partially nested within candidates and interviewers) has been ignored in order to simplify the statistical analysis. It is therefore possible that any effects that appear to be statistically significant are (slightly) overstated in our findings. However, the substantive nature of the main findings is not affected.

---

Analysis of interviewee performance

Potential determinants of performance by interviewees were analysed using univariate general linear models 

with total interview score as the outcome variable, 

gender, ethnicity (White ⁄non-White), school type (independent ⁄ state selective ⁄ state nonselective [as the reference group]) as fixed effects, 

and socio-economic classification and date of birth as covariate dependent variables.

A main effects-only model indicated that no predictors were playing a significant role in determining the marks awarded to interviewees and explained < 1% of the variation in the data. 

The full factorial model (all main effects and their interactions) was still relatively poor, explaining only approximately 2% of the variation in the interview total mark. 

Thus, most of the variation in the marks was not accounted for by the available predictors. In this model, no main effects were statistically significant and the largest interaction effect was for school type (state selective versus ethnicity; F1,609 = 9.352, p = 0.002, effect size 2%), whereby non-White students from selective schools tended to be awarded slightly lower marks than their White counterparts. The difference lay in the opposite direction for applicants who did not come from such schools

---

Analysis of interviewer performance

The potential determinants of marks awarded by interviewers were also analysed using univariate general linear models, 

with mean interview score awarded by the interviewer as the outcome variable, 

gender, ethnicity (White ⁄non-White), staff or student and school type as fixed effects, 

and socio-economic classification as the covariate dependent variable.

A simple model including only main effects found no predictors playing a significant role and explained almost none of the variance in the data. 

The full factorial model, including predictors and all interactions and explaining 4.8% of the variation in mean marks, found a small but significant gender main effect, whereby male interviewers awarded slightly higher marks than females (estimated marginal means 11.1 and 10.7, respectively; F1,280 = 3.999, p = 0.047, effect size 1%). There was also evidence in this model of small interaction effects, including school type (independent) with ethnicity: those from independent schools tended to give slightly higher marks to non-White candidates, whereas those not from independent schools tended to give higher marks to White candidates (F1,280 = 9.569, p = 0.002, effect size 3%).

A separate analysis pertaining to staff interviewers only was carried out with interviewer experience included in the model as a covariate. However, this variable did not play a significant role in influencing interview scores.

---

Med Educ. 2010 Nov;44(11):1077-83. doi: 10.1111/j.1365-2923.2010.03771.x.

Equity in interviews: do personal characteristics impact on admission interview scores?

Lumb AB, Homer M, Miller A.

Source

Leeds Institute of Medical Education, School of Medicine, University of Leeds, Leeds, UK. a.lumb@leeds.ac.uk

Abstract

CONTEXT:

Research indicates that some social groups are disadvantaged by medical school selection systems. The stage(s) of a selection process at which this occurs is unknown, but at interview, when applicant and interviewer are face-to-face, there is potential for social bias to occur.

METHODS:

We performed a detailed audit of the interview process for a single-entry year to a large UK medical school. Our audit included investigating the personal characteristics of both interviewees and interviewers to find out whether any of these factors, including the degree of social matching between individual pairs of interviewees and interviewers, influenced the interview scores awarded.

RESULTS:

A total of 320 interviewers interviewed 734 applicants, providing complete data for 2007 interviewer-interviewee interactions. The reliability of the interview process was estimated using generalisability theory at 0.82-0.87. For both interviewers and interviewees, gender, ethnic background, socio-economic group and type of school attended had no influence on the interview scores awarded or achieved. Staff and student interviewer marks did not differ significantly. Although numbers in each group of staff interviewers were too small for formal statistical analysis, there were no obvious differences in marks awarded between different medical specialties or between interviewers with varying amounts of interviewing experience.

CONCLUSIONS:

Our data provide reassurance that the interview does not seem to be the stage of selection at which some social groups are disadvantaged. These results support the continued involvement of senior medical students in the interview process. Despite the lack of evidence that an interview is useful for predicting future academic or clinical success, most medical schools continue to use interviews as a fundamental component of their selection process. Our study has shown that at least this arguably misplaced reliance upon interviewing is not introducing further social bias into the selection system.

© Blackwell Publishing Ltd 2010.

Comment in

• Selection for medical schools: re-imaging as an international discourse. [Med Educ. 2010]

PMID:

 

20946478

 

[PubMed - indexed for MEDLINE]