레지던트 선발에서 MMI의 활용가능성(Acceptability)

The Acceptability of the Multiple Mini Interview for Resident Selection

Marianna Hofmeister, PhD; Jocelyn Lockyer, PhD; Rod Crutcher, MD

캐나다 Alberta의 Alberta’s International Medical Graduate Program (AIMGP)에 지원한 학생을 대상으로 실시한 MMI에 대한 연구로서, 가정의학과 레지던트에 지원한 해외의과대학졸업생(IMG)의 면접에 대한 연구이다. 

Background and Objectives: This study describes and assesses the acceptability of the multiple mini interview (MMI) to both international medical graduate (IMG) applicants to family medicine residency training in Alberta, Canada, and also interviewers for Alberta’s International Medical Graduate Program (AIMGP), an Alberta Health and Wellness government initiative designed to help integrate IMGs into Canadian residency training. IMGs are physicians who completed undergraduate medical education outside of Canada and the United States. IMGs who live in the Canadian province of Alberta may obtain a limited number of government-funded positions for residency training by applying to AIMGP. 

12개 스테이션으로 이루어진 MMI를 설꼐하였고, 프로페셔널리즘에 대한 비인지적 특성을 보고자 했다. 가정의학과 교수와 의학교육자들이 문항을 개발하였으며, 시험이 종료된 후 설문을 하였다.

Methods: A literature review and faculty and medical community consultation informed the development of a 12-station MMI designed to identify non-cognitive characteristics associated with professionalism potential. Clinical scenarios were developed by family physicians and medical educators. Applicant and interviewer posttest acceptability was assessed using surveys. Quantitative data were analyzed using descriptive statistics, and qualitative data were analyzed using content analysis and thematic description. 

면접관들의 만족도가 높았으며, 캐나다 가정의학의 맥락에 잘 맞는 문항이라고 평가하였다. 지원자와 평가자 모두 8분이 충분한 시간이라고 했고, 지원자들은 이 면접이 성별 또는 문화 BIAS에 영향을 받지 않는다고 느꼈다고 응답했다. 평가자들은 MMI가 공정한 평가법이라는 것에 동의하였다.

Results: Our research demonstrates evidence for applicant and interviewer acceptability of the MMI. Interviewers reported high levels of satisfaction with the time-restricted process that addressed multiple situations pertinent to the Canadian family medicine context. Applicants and interviewers were each satisfied that 8 minutes was enough time at each station. Applicants reported that they felt the process was free from gender and cultural bias. Interviewers agreed that this MMI was a fair assessment of potential for family medicine. 

Conclusions: Standardized residency selection interviews can be adapted to measure professionalism potential characteristics important to family medicine in ways that are acceptable to IMG applicants and interviewers.

---

Background

MMI

MMI개발의 역사, Validity, Generalizability의 근거 확보. Acceptability 확보

The MMI is a multi-station interview with one interviewer rating candidates’ performance at each station. The MMI was developed at the Michael G. DeGroot School of Medicine at McMaster University in Hamilton, Ontario, Canada, and has been validated there,5,14,15 at the University of Calgary,6,16 in Australia,17 and in the UK.18 This interview instrument has demonstrated evidence for generalizability and validity in relation to future clinical and licensing examination performance as compared to traditional interview methods.5,7.9 Further, the MMI has established acceptability with members of applicant and interviewer stakeholder groups at the admissions level.5,16,17

MMI는 프로페셔널리즘 역량을 평가할 수 있는 flexibility가 있으며, 우리는 CFPC가 정의한 네 원칙에 따라 문항을 개발했다.

The flexibility of the MMI allows programs to select applicants whose behaviors best align with professionalism competency expectations. In this assessment, we developed an assessment in accordance with the College of Family Physicians of Canada’s four principles.6,19,20 The four family medicine principles are: 

• the family physician is a skilled clinician, 
• family medicine is a community-based discipline, 
• the family physician is a resource for a defined practice population, and 
• the patient-physician relationship is central to the role.20 

These principles provide a framework with which competencies, such as those outlined by the Accreditation Council for Graduate Medical Education (ACGME) (ie, professionalism, interpersonal and communication skills, and systems-based practice), can be addressed.

프로페셔널리즘 : 정의, Developmental, Context specific

“Professionalism potential” is derived from medical professionalism theory.21 The attributes associated with medical professionalism and professional behavior are of universal concern.22 Medical professionalism is conceptualized as developmental.5, 6 Professional behavior is context specific or situation dependent.23 This means that evidence of aspects of professional behavior in one challenging situation does not predict different aspects of professional behavior in another. It follows that professionalism potential for family medicine may be examined in multiple situations critical to best family practice using this new interview methodology.

연구의 목표

Objectives

The acceptability of the MMI in IMG groups and the acceptability of the MMI for professionalism potential measurement in IMG individuals for family medicine residency selection have not been previously investigated. The objective of this research was to investigate the acceptability to family medicine interviewers and to IMG applicants themselves of an MMI designed to measure professionalism potential of IMG applicants.

Methods

The process of preparing for the MMI and constructing stations began in March 2006 with the formation of the AIMG MMI Committee. The committee included a family physician chair who oversaw the process and made final decisions relating to what characteristics would be examined in which scenarios. Ethics approval for the study was provided by the University of Calgary Conjoint Health Research Ethics Board.

MMI Development

Station construction was guided by a table of specifications based on previously examined characteristics, characteristics important to family medicine,24 and those associated with medical professionalism.21 The characteristics examined in previous interviews included ...

relationship-building skills, 

team skills, 

recognition of professional limitations, 

integrity, 

decision-making skills, 

problem-solving skills, and 

communication in caring relationships. 

The medical education literature pertaining to professionalism and desirable personal traits in medical practitioners was reviewed.24-26 A comprehensive list of characteristics that might be assessed using the MMI was constructed and circulated to decision makers. A structured formal inquiry through e-mail correspondence, meetings, and discussion was used to gather input from the family medicine residency program directors at the University of Calgary and the University of Alberta and from other community-based and academic family physicians. This information was used to construct content-specific situations that would enable each characteristic to be assessed. The AIMG MMI Committee ultimately developed station content, question probes, and background information.

MMI Development

Ten stations, each designed to measure one characteristic, presented situations the applicant might face in a family medicine residency.27 The characteristics tested were ...

• teamwork, 
• honesty, 
• ability to accept feedback about one’s self, 
• ability to accept self-limitations, 
• caring and compassion, 
• responsibility taking, 
• time management, 
• the ability to accept professional limitations, 
• cultural sensitivity, 
• motivation for family medicine, and 
• goal setting. 

A sample station is shown in Table 1.

IMG Applicants

To qualify for the MMI, applicants were required to have completed the AIMGP’s entry requirements. These criteria include...

a passing score on the Medical Council of Canada Equivalency Examination, 

a passing score on the Medical Council of Canada Qualifying Examination Part 1, and 

proof of successful completion of undergraduate medical education in a medical school listed in the Foundation for Advancement of International Medical Education and Research (FAIMER) directory. 

In addition, applicants were required to pass all components of the AIMGP objective structured clinical examination (OSCE). 

Specifically they had to pass the minimum number of stations for the clinical skills component and exceed the benchmark scores on the communication, oral, and written English proficiency tests. 

Alberta International Medical Graduate applicants for family medicine residency training positions who exceeded the minimal pass level on the clinical skills OSCE were e-mailed an invitation to the family medicine MMI following notification of their success on the OSCE. After each MMI session, applicants were asked to complete the acceptability survey.

Interviewers

• 가정의학과 교수와 고년차 레지던트들이 면접관으로 들어감
• 면접관은 2주 전에 2시간의 의무적 트레이닝 세션에 참가해야 함
• 면접 48시간 전에 스테이션에 대한 정보 제공
• 모든 면접은 캘거리대학에서 진행됨.
• 면접관 특성에 따라 면접 점수가 달라질 수 있다는 연구 결과에 기반하여 면접관의 구성은 professional status와 gender에 따라서 하였음.

Interviewers were family medicine faculty and senior family medicine residents at the University of Alberta and the University of Calgary, community physicians from both urban centers, and stakeholders from other medical community-related groups (ie, medical education, language education, and human resources). All of the interviewers participated in a mandatory 2-hour training session 2 weeks before the MMI. Interviewers then received their station information 48 hours before the interviews took place. All of the interviews took place at the University of Calgary. Because previous research has shown that applicant scores may be related to interviewer characteristics, interviewers were organized in tracks and stations according to their professional status and gender to minimize these effects on applicant scores.14,28

Interview Procedure

• 한 스테이션당 1명의 면접관
• 두 세트를 활용하였고, 세 세션으로 나눠서 진행하였음.
• 2분간 문 앞에서 정보를 숙지하고 8분간 면접 진행

The MMI uses a multi-station format that is similar to an OSCE. Each applicant moves through the same set of stations and is evaluated by a single interviewer at each station. More than one set of stations can be run at the same time. We used two sets of stations per session and ran three sessions in a single day. At each station, the applicant read the information posted on the door for 2 minutes and discussed his/her response with the interviewer for 8 minutes. After each session, applicants were invited to complete the applicant survey and at the end of interview day, interviewers were invited to complete the interviewer acceptability survey.

---

Fam Med. 2008 Nov-Dec;40(10):734-40.

The acceptability of the multiple mini interview for resident selection.

Hofmeister M1, Lockyer J, Crutcher R.

Author information

Abstract

BACKGROUND AND OBJECTIVES:

This study describes and assesses the acceptability of the multiple mini interview (MMI) to both international medical graduate (IMG) applicants to family medicine residency training in Alberta, Canada, and also interviewers for Alberta's International Medical Graduate Program (AIMGP), an Alberta Health and Wellness government initiative designed to help integrate IMGs into Canadian residency training. IMGs are physicians who completed undergraduate medical education outside of Canada and the United States. IMGs who live in the Canadian province of Alberta may obtain a limited number of government-funded positions for residency training by applying to AIMGP.

METHODS:

A literature review and faculty and medical community consultation informed the development of a 12-station MMI designed to identify non-cognitive characteristics associated with professionalism potential. Clinical scenarios were developed by family physicians and medical educators. Applicant and interviewer posttest acceptability was assessed using surveys. Quantitative data were analyzed using descriptive statistics, and qualitative data were analyzed using content analysis and thematic description.

RESULTS:

Our research demonstrates evidence for applicant and interviewer acceptability of the MMI. Interviewers reported high levels of satisfaction with the time-restricted process that addressed multiple situations pertinent to the Canadian family medicine context. Applicants and interviewers were each satisfied that 8 minutes was enough time at each station. Applicants reported that they felt the process was free from gender and cultural bias. Interviewers agreed that this MMI was a fair assessment of potential for family medicine.

CONCLUSIONS:

Standardized residency selection interviews can be adapted to measure professionalism potential characteristics important to family medicine in ways that are acceptable to IMG applicants and interviewers.