MMI 기반 선발의 결과와 의과대학 지원자의 인종/민족/사회경제적지위의 관계(Acad Med, 2015)

MMI 기반 선발의 결과와 의과대학 지원자의 인종/민족/사회경제적지위의 관계(Acad Med, 2015)

How Medical School Applicant Race, Ethnicity, and Socioeconomic Status Relate to Multiple Mini-Interview–Based Admissions Outcomes: Findings From One Medical School

Anthony Jerant, MD, Tonya Fancher, MD, MPH, Joshua J. Fenton, MD, MPH, Kevin Fiscella, MD, MPH, Francis Sousa, MD, Peter Franks, MD, and Mark Henderson, MD

MMI 도입에 따라 underrepresented racial/ethnic minority (URM) 집단이나 낮은 SES의 지원자가 어떤 영향을 받았는가에 대한 연구가 적다. 미국 의과대학에 URM과 Low SES 학생의 비율이 불균형을 이루고 있음을 감안하면 중요한 사안이다.

Little studied is how underrepresented racial/ethnic minority (URM) and lower socioeconomic status (SES) applicants may be affected by adoption of the MMI. This is a key issue given that U.S. medical schools admit disproportionately few URM and lower SES individuals.6–8

전통적인 비구조화면접은 오랜 시간 면접관의 편견에 취약하다는 지적이 있었다. 무의식적 편견이 인종/민족 소수자들과 낮은 SES 지원자를 탈락시키는 방향으로 작용하는 것은 의사는 물론 미국에 흔한 현상이다. 면접에서 발생하는 비뚤림의 영향은 구조화를 높임으로서(모호성을 제거하고, 정형화된 구조에 따라 판단하게 하는) 줄일 수 있고, 다양한 평가자의 평가결과를 취함함으로써 개개인의 편견의 영향을 희석시킬 수 있다.

A long-recognized problem with traditional nonstructured interviews is vulnerability to interviewer biases triggered by various applicant characteristics.17–22 Implicit (i.e., unconscious) biases disfavoring racial/ ethnic minority and lower SES persons are common in U.S. society,23 including among physicians.24 The effects of bias during interviews can be reduced by increasing structure (removing ambiguity and, therefore, the tendency to rely on stereotype-driven judgments) and pooling evaluations from multiple raters (potentially diluting or offsetting individual biases).20,25–27

우리가 아는 한 MMI수행능력과 URM, SES의 관련성에 대한 연구는 세 개이다.

Only three studies to our knowledge have explored the associations of medical school applicants’ racial/ethnic minority status or SES with MMI performance.

MMI를 치른 이후 합격에 인종/민족이 영향을 주었는지에 대한 연구는 없다. 혹은 인종/민족, SES가 MMI invitation영향에 대한 연구도 없다.

To our knowledge, no studies have examined whether applicants’ race/ ethnicity influences acceptance following MMI participation, or whether race/ ethnicity or SES influences the likelihoodof being invited to an MMI.

방법

Method

지원, 스크리닝, MMI초청, 일정조정

Application, screening, and MMI invitation and scheduling

다음에 따라서 MMI invitation을 평가함

Faculty evaluated secondary applications for invitation to an MMI based on cumulative GPA and MCAT scores, personal statements, extracurricular activities, recommendation letters, and other characteristics that could contribute to fulfilling the educational and service missions of the school.

MMI절차와 점수

MMI process and scoring

2분-8분, 다음의 10개 주제

The MMI consisted of 10 individual 10-minute stations. At each station, applicants had 2 minutes to read a brief set of instructions, and 8 minutes to address the assigned tasks on entering the room. Nine stations assessed skills in the following domains: 

• integrity/ ethics, 
• professionalism, 
• interpersonal communication, 
• diversity/cultural awareness, 
• teamwork, 
• ability to handle stress, and 
• problem solving. 
• An additional station asked applicants to explain their choice to pursue a career in medicine. 

Most stations were adapted from content developed at McMaster University and marketed by ProFitHR.34

학생의 AMCAS 지원 정보를 모르는 한 명의 숙련된 평가자가 각 스테이션에 배정됨

A single trained rater, blinded to participants’ AMCAS application information, attended each station.

총 216명의 서로 다른 평가자

There were 216 different raters during the study period; 

• 평균 참가 스테이션 the mean number of MMI stations that each evaluated was 104 (standard deviation [SD] 61.9; range 8–276). 
• 여성 Women made up 61% of raters. 
• 평가자 Background Rater professional backgrounds were as follows: physicians, 31%; medical students, 15%; other clinicians (e.g., nurses), 11%; basic science faculty, 6%; patients, 2%; and various nonclinician leaders (e.g., deans), professionals (e.g., lawyers), and high- level administrative staff (e.g., curriculum manager), 35%. 

평가자의 배경이 다양한 것은 다양한 관점이 미래에 온갖 계층의 사람들과 효율적으로 일할 의사를 선발하는데 도움이 된다고 생각했기 때문. 의무적인 평가자 훈련은 입학절차에 대한 1시간의 리뷰, 평가자 역할과 의무, 계급문제를 지양할 필요성 등을 다뤘다.

The range of rater backgrounds reflected the conviction that diverse perspectives are helpful in selecting future physicians who will be able to work effectively with people from all walks of life. Mandatory rater training included a one-hour course reviewing the admissions process, rater roles and duties, and the need to avoid pursuing protected class issues (e.g., race/ethnicity, gender).36

각각 스테이션의 평가 (4점 척도)

At each station, raters scored overall applicant performance using an anchored four-point scale: 

• 0, < 25th percentile performance (relative to other applicants); 
• 1, 25th–50th percentile; 
• 2, 51st–75th percentile; or 
• 3, > 75th percentile. 

또한 지원자의 의사소통능력과 이해도를 고려하도록 함. 

Raters were instructed to consider both the applicant’s communication abilities and the content (e.g., comprehensiveness) of their statements in assigning ratings. The total MMI score was the mean of each applicant’s individual station scores. Scale internal consistency (Cronbach alpha = 0.67) was comparable to that observed in other MMI studies.2,18,37–41

입학 판정

Acceptance recommendation

Subsequently, the committee made one of the following recommendations: reject, low waitlist, high waitlist, or offer acceptance.

URM 상태

URM status

AMCAS 지원정보를 바탕으로 URM Status를 판단

We determined URM status (URM [black, Southeast Asian, Native American, or Pacific Islander race and/or Hispanic ethnicity] versus not [all other responses]) from self-reported race/ethnicity information in the AMCAS application.

SES 불이익

Socioeconomic disadvantage

AMCAS 지원정보를 바탕으로 SES 척도를 개발

We developed a composite measure of SES using self-reported information in the AMCAS application,

다음의 정보를 활용

The following predictors (yes/ no items except where indicated) were significant and maximized the area under the receiver operating characteristic curve (0.95):

• fee assistance received for medical school application (yes/no); 
• childhood spent in an underserved area; 
• family recipients of family assistance program; 
• income level category of applicant’s family (< $25,000; $25,000 to < $50,000; $50,000 to < $75,000; or > $75,000); 
• applicant contributed to family income; 
• any financial-need-based scholarship(s) in paying for postsecondary education; 
• percentage of postsecondary education costs contributed by the family; and 
• parents’ highest level of educational attainment (< high school, high school graduate, some college, or college graduate).

Applicant characteristics

MMI invitation

MMI score

Acceptance recommendation

Discussion

URM지원자는 non-URM지원자보다 MMI invitation을 받을 가능성이 더 낮지 않았고, MMI에서 유사한 정도의 점수를 받았으며, 입학할 가능성은 더 높았다.

Further, URM applicants were no less likely than non-URM applicants to receive an MMI invitation, performed similarly on the MMI, and were just as likely to be recommended for acceptance.

URM과 non-URM지원자 사이의 유사한 MMI점수는 구조화된 면접이 다양한 평가자의 관점을 포함하게 하면서 개개인이 은연중에 가지는 편견으로부터 덜 취약하게 해주는 효과가 있음을 보여준다. 비록 우리가 평가자의 implicit bias를 측정하지는 않았지만, 이렇한 정보는 미국사회에 널리 퍼져있음이 이미 여러 문헌에서 나타난 바 있으며, 의사나 다른 전문직도 예외는 아니고, 의료를 포함하여 다양한 고용면접의 결과에 영향을 준다. 따라서 implicit bias는 우리의 평가자들 사이에도 있었을 것이다. 그러나 이는 net로 보았을 때 유의한 영향은 없었고, URM과 non-URM사이에 차이가 있지도 않았다. 의료분야에 URM 비중이 낮은 것이 이미 많이 인정된 문제인만큼, URM에 대한 안좋은 편견은 (이러한 인종/민족의 문제를 해결하기 위하여 평가자들이 들이는 노력에 따른) URM지원자에 대한 우호적 편견으로 offset할 수 있다.

The similar MMI scores for URM and non-URM participants support the notion that structured interview processes that incorporate the perspectives of multiple evaluators like the MMI may be less vulnerable to the effects of individual evaluator implicit biases.20,25–27 Although we did not measure rater implicit biases regarding racial/ ethnic minorities, such biases have been documented to be pervasive in U.S. society, including among physicians and other professionals,23,24 and can affect the outcomes of employment interviews in various fields including medicine.17,19–22 Thus, it is likely that implicit biases were present among our raters; however, they did not exert a significant net influence, given that mean MMI scores did not differ between URM and non-URM applicants. Because lack of URMs in medicine is a widely acknowledged problem,6,7,13,33,42–44 it is possible that biases against URM applicants were offset by ratings biased in favor of URM applicants, made by raters seeking to address limited racial/ethnic diversity in the physician workforce.

반면, 낮은 SES는 더 낮은 MMI점수를 받았다.

In this context, our finding that lower SES applicants had worse adjusted MMI performance may be cause for concern. 

그럼에도 불구하고, 낮은 SES가 MMI점수에 미치는 영향은 작았다. SES를 0-1로 평가했을 때 그 감소 정도가 0.12정도였다. 또한 낮은 MMI점수는 더 높은 합격률로 offset되었다. 이러한 결과는 AAMC가 지향하는 바와 같이 순전히 metric-based의 지원자 검토보다 더 holistic process로 변하고 있음을 보여준다.

Nonetheless, the decrement in MMI performance with decreasing SES in our study was small: The MMI score (scale of 0–3 points) declined by a mean of 0.12 points across the 0–1 range of the SES score. Further, the lower MMI scores among lower SES applicants were more than offset by their greater likelihood of being invited to an MMI and recommended for acceptance. These findings may reflect the ongoing shift from a purely metric-based applicant review process toward the more holistic process advocated by the Association of American Medical Colleges.12,15

낮은 SES 지원자는 MMI에서 평가하는 생애 경험이 더 적을 수 있다. 더 낮은 MCAT점수를 받은 지원자에 대해서도 유사한 추론이 제기된 바 있다. 덜 부유한 지원자가 postsecondary education기간동안 임금노동을 더 많이 했을 수는 있지만, 그들이 일한 것이 MMI식의 선발절차를 거치진 않았을 것이다. MMI와 같은 유형의 선발절차 경험이 없는 것은 특정 면접 형식에 대한 과거 경험이 유사한 방식의 면접에서 더 높은 점수와 관계됨을 고려할 때 의과대학 MMI에서 약점으로 작용할 수 있다. 또한 낮은 수준의 일자리는 높은 수준의 의사소통, 비판적 사고, 문제해결 등 MMI에서 요구하는 능력 개발을 촉진시키지 않을 가능성이 높으며, 그러한 일자리에 투자하하는 시간이 이들 skill 개발에 장애가 될 것이다.

Lower SES applicants may have fewer life experiences bolstering skills assessed by the MMI. Similar reasoning has been suggested to explain the lower MCAT scores among such applicants.45 Although less affluent applicants are more likely to report paid employment during postsecondary education, their financial circumstances may require taking jobs that do not require MMI-type preemployment screening. Lack of prior experience with MMI-type screening may be a disadvantage in the medical school MMI because prior experience with a particular interview format is associated with better future performance with that format.46 Lower-level jobs also may not facilitate the higher-level communication, critical thinking, and problem-solving skills the MMI assesses, and the time required for such jobs may limit participation in pursuits that build such skills (e.g., scholarly presentations, volunteer clinic work).

기존의 연구를 보면 익숙하지 않은 언어(표현)를 사용하는 것이 낮은 평가로 비뚤리게 하는 요인이 된다고 한다. 지원자의 언어 기술은 면접관의 즉각적 인상을 결정하고, 그 결과 최종 평가에도 영향을 줄 수 있다. 의사인력의 SES 불균형은 인종/민족 불균형보다 관심을 덜 받아왔다. 따라서 면접관이 낮은 SES 지원자에게 우호적으로 bias하려고 의식적으로 신경을 썼을 가능성은 낮다. 

Prior work indicates that applicant factors such as use of language unfamiliar to the typical rater could trigger a biased low rating.20,21 Applicants’ verbal skills have been shown to determine immediate interviewer impressions and, in turn, final appraisals.49 The issue of SES-based physician workforce disparities has received less attention than race/ ethnicity-based disparities.6 Thus, it is less likely that raters consciously biased their evaluations in favor of lower SES applicants to address SES-based physician workforce disparities.

34 Advanced Psychometrics for Transitions Inc. Welcome to ProFitHR. http://www.profithr.com/. Accessed April 4, 2015.

---

Acad Med. 2015 Dec;90(12):1667-74. doi: 10.1097/ACM.0000000000000766.

How Medical School Applicant Race, Ethnicity, and Socioeconomic Status Relate to Multiple Mini-Interview-Based Admissions Outcomes: Findings From One Medical School.

Jerant A1, Fancher T, Fenton JJ, Fiscella K, Sousa F, Franks P, Henderson M.

Author information

• 1A. Jerant is professor, Department of Family and Community Medicine, Center for Healthcare Policy and Research, University of California, Davis,School of Medicine, Sacramento, California. T. Fancher is associate professor, Division of General Internal Medicine, Department of Internal Medicine, University of California, Davis, School of Medicine, Sacramento, California. J.J. Fenton is associate professor, Department of Family and Community Medicine, Center for Healthcare Policy and Research, University of California, Davis, School of Medicine, Sacramento, California. K. Fiscella is professor, Department of Family Medicine, University of Rochester School of Medicine and Dentistry, Rochester, New York. F. Sousa is assistant dean, Admissions and Student Development, and volunteer clinical professor, Department of Internal Medicine, University of California, Davis, School of Medicine, Sacramento, California. P. Franks is professor, Department of Family and Community Medicine, Center for Healthcare Policy and Research, University of California, Davis, School of Medicine, Sacramento, California. M. Henderson is associate dean, Admissions and Outreach, and professor, Division of General Medicine, Department of Internal Medicine, University of California, Davis, School of Medicine, Sacramento, California.

Abstract

PURPOSE:

To examine associations of medical school applicant underrepresented minority (URM) status and socioeconomic status (SES) withMultiple Mini-Interview (MMI) invitation and performance and acceptance recommendation.

METHOD:

The authors conducted a correlational study of applicants submitting secondary applications to the University of California, Davis, Schoolof Medicine, 2011-2013. URM applicants were black, Southeast Asian, Native American, Pacific Islander, and/or Hispanic. SES from eight application variables was modeled (0-1 score, higher score = lower SES). Regression analyses examined associations of URM status and SES with MMI invitation (yes/no), MMI score (mean of 10 station ratings, range 0-3), and admission committee recommendation (accept versus not), adjusting for age, sex, and academic performance.

RESULTS:

Of 7,964 secondary-application applicants, 19.7% were URM and 15.1% self-designated disadvantaged; 1,420 (17.8%) participated in the MMI and were evaluated for acceptance. URM status was not associated with MMI invitation (OR 1.14; 95% CI 0.98 to 1.33), MMI score (0.00-point difference, CI -0.08 to 0.08), or acceptance recommendation (OR 1.08; CI 0.69 to 1.68). Lower SES applicants were more likely to be invited to an MMI (OR 5.95; CI 4.76 to 7.44) and recommended for acceptance (OR 3.28; CI 1.79 to 6.00), but had lower MMI scores (-0.12 points, CI -0.23 to -0.01).

CONCLUSIONS:

MMI-based admissions did not disfavor URM applicants. Lower SES applicants had lower MMI scores but were more likely to be invited to an MMI and recommended for acceptance. Multischool collaborations should examine how MMI-based admissions affect URM and lower SES applicants.

PMID:

 

26017355

 

[PubMed - in process]