입학전형과 학업능력: 세 가지 입학전형 비교(Medical Education, 2014)

입학전형과 학업능력: 세 가지 입학전형 비교(Medical Education, 2014)

Selection and study performance: comparing three admission processes within one medical school

Nienke R Schripsema,1,2 Anke M van Trigt,2 Jan C C Borleffs2 & Janke Cohen-Schotanus1,2

OBJECTIVES:

본 연구의 목적은 (1)고등학교 성적, 자발적 다면선발, 추첨이라는 세 가지 방법으로 입학한 학생들의 학업능력을 비교하고, (2)다면선발을 통해서 입학한 학생이 여기서 떨어진 학생보다 우수한지를 알아보고 (3)다면선발에 지원했다는 것 자체가 학업능력과 연관되는가를 보고자 한다.

This study was conducted to: (i) analyse whether students admitted to one medical school based on top pre-university grades, a voluntary multifaceted selection process, or lottery, respectively, differed in study performance; (ii) examine whether students who were accepted in the multifaceted selection process outperformed their rejected peers, and (iii) analyse whether participation in the multifaceted selection procedure was related to performance.

METHODS:

지식시험, 프로페셔널리즘 점수, 학습발달, 중퇴의 네 가지를 비교하였다. 추첨입학 학생군을 다면선발에 지원한 학생과 그렇지 않은 학생으로 나눴다. ANCOVA, 로지스틱 회귀분석, Bonferroni 검정을 사용하였다.

We examined knowledge test and professionalism scores, study progress and dropout in three cohorts of medical students admitted to the University of Groningen, the Netherlands in 2009, 2010 and 2011 (n = 1055). We divided the lottery-admitted group into, respectively, students who had not participated and students who had been rejected in the multifaceted selection process. We used ancova modelling, logistic regression and Bonferroni post hoc multiple-comparison tests and controlled for gender and cohort.

RESULTS:

고등학교에서 최고수준 성적을 받은 그룹이 지식검사에서 높은 점수를 받았으며, 1학년때 더 많은 학점을 이수하였다. 또한 이 그룹은 다면선발에 지원하지 않은 추첨입학 학생군에 비해서 최상위권 프로페셔널리즘 점수를 더 많이 받았다. 다면선발로 입학한 학생은 다면선발에 지원하지 않은 추첨선발 학생보다 지필고사 점수가 높았으며, 두 가지 추첨선발 학생군보다도 프로페셔널리즘 점수를 잘 받았다. 다면선발에 지원하지 않았던 추첨선발 학생군은 다른 모든 그룹보다 1학년과 2학년 이수학점이 적었고, 중퇴율은 학생군별로 달랐지만, 보정 후에는 유의하지 않았다.

The top pre-university grade group achieved higher knowledge test scores and more Year 1 course credits than all other groups (p < 0.05). This group received the highest possible professionalism score more often than the lottery-admitted group that had not participated in the multifaceted selection process (p < 0.05). The group of students accepted in the multifaceted selection process obtained higher written test scores than the lottery-admitted group that had not participated (p < 0.05) and achieved the highest possible professionalism score more often than both lottery-admitted groups. The lottery-admitted group that had not participated in the multifaceted selection process earned fewer Year 1 and 2 course credits than all other groups (p < 0.05). Dropout rates differed among the groups (p < 0.05), but correction for multiple comparisons rendered all pairwise differences non-significant.

CONCLUSIONS:

고등학교에서 최상위 GPA를 받은 것이 학업능력을 가장 잘 예측하였다. 비학업적 수행능력에 있어서는 다면선발이 가장 효율적인 방식인 것으로 보인다. 다면선발에 지원하여 참여하는 것 자체가 높은 수행능력의 예측인자였다. 

A top pre-university grade point average was the best predictor of performance. For so-called non-academic performance, the multifaceted selection process was efficient in identifying applicants with suitable skills. Participation in the multifaceted selection procedure seems to be predictive of higher performance. Further research is needed to assess whether our results are generalisable to other medical schools.

---

입학점수와 입학 후 수행능력의 관계에 대한 연구에서 두 가지 한계가 있다. 첫째는 합격자만을 대상으로 하기에 restriction of range가 생긴다는 것이고, 둘째는 단일한 전형으로 학생을 선발하는 많은 학교에서는 서로 다른 입학전형간 비교가 어렵다는 점이다.

Studies on the relationships between admission scores and later performance, however, are characterised by two limitations. Firstly, most study samples have a restriction of range because performance data are only available for applicants who were accepted in the selection process. It remains unclear how rejected applicants would have performed if they had been admitted. Secondly, in most medical schools, all students are admitted through the same admission process based on a single set of criteria, which limits opportunities to compare different admission processes.

합격자와 불합격자의 비교연구

In the literature, few studies investigating performance differences between accepted and rejected applicants have been reported.8,9

서로 다른 입학전형간 비교

A few studies have addressed the effects of different admission processes on student performance within one medical school.

네덜란드의 3단계 의과대학 학생선발 시스템

The Netherlands has a three-step admission system. In the first step, applicants with a top pre-university GPA are offered admission. Participation in the second step is voluntary. In this step, students participate in a multifaceted selection process that is organised by each medical school separately. The third step is a national weighted lottery, in which applicants who were rejected in the second step as well as applicants who have not participated in the second step can enrol.

METHODS

Context

네덜란드의 그로닝겐 의과대학. 3년의 BA와 3년의 MA. 교육과정 개괄

This study was performed at the University of Groningen in the Netherlands. The problem-based curriculum consists of a 3-year pre-clinical Bachelor’s and a 3-year clinical Master’s programme. The first year of the Bachelor’s programme includes four 10- week blocks, a year-long professionalism course, and the inter-university progress test.15,16 Each part provides students with a fixed number of course credits under the European Credit Transfer System. One course credit equals 28 hours of study. The maximum number of course credits per year is 60.

Admission processes in the Netherlands

네덜란드 학생선발 단계

The Netherlands has a national policy for medical school admissions according to which applicants can be admitted through one of three steps. 

• In the first step, students with a pre-university GPA of ≥ 8 (on a scale ranging from 1 = poor to 10 = excellent) gain admission to the medical school of their choice without further assessment. This grade average is calculated by averaging each applicant’s mean grade on pre-university school examinations and mean grade on national final examinations. As only approximately 4% of all pre-university graduates achieve a GPA of ≥ 8, this grade indicates excellent achievement. 
• In the second step, applicants can be accepted in a multifaceted selection process. Each medical school organises its own selection process in which participation is voluntary. Selection processes differ among medical schools, but they usually consist of two rounds of assessments in which various knowledge-based and behavioural variables are measured. 
• In the third step, applicants are admitted through the national weighted lottery. In the lottery, applicants are categorised based on their pre-university GPA. Four categories of GPA are distinguished: 7.5–7.9; 7.0–7.4; 6.5–6.9, and < 6.5. The ratio of applicants admitted by category is 9 : 6 : 4 : 3.11,17 Applicants who were rejected in the multifaceted selection process can be admitted through the lottery. Until 2011, approximately 50% of the places available at each medical school were assigned through the national weighted lottery.

네덜란드에서는 매년 8500명이 지원하여 2780개의 정원을 놓고 경쟁함. 410명이 그로닝겐 의과대학에 배정됨. 

Every year, around 8500 applicants apply to medical school in the Netherlands. There are 2780 places available, 410 of which are at the University of Groningen. Approximately 60 of these places are allocated to the International Bachelor of Medicine programme. The remaining 350 are allocated to the Dutch Bachelor of Medicine programme. In the current study, we included only students enrolled in the Dutch Bachelor of Medicine programme.

Participants

연구대상자

We included all 1055 students admitted to the Dutch Bachelor of Medicine programme at the University of Groningen in 2009, 2010 and 2011 (69% female; mean age at the start of Year 1: 18.6 years; mean pre-university GPA: 7.3). We defined four groups of students: 

• (i) students who were admitted based on a pre-university GPA of ≥ 8 out of 10 (n = 143; 71% female; mean age at the start of Year 1: 18.0 years; mean pre-university GPA: 8.2); 
• (ii) students who were accepted in the multifaceted selection process (n = 295; 74% female; mean age at the start of Year 1: 18.5 years; mean pre-university GPA: 7.1); 
• (iii) lottery-admitted students who had been rejected previously in the multifaceted selection process (n = 315; 69% female; mean age at the start of Year 1: 18.5 years; mean pre-university GPA: 7.1), and 
• (iv) lottery-admitted students who had not participated in the selection process (n = 302; 63% female; mean age at the start of Year 1: 19.1 years; mean pre-university GPA: 7.0). Descriptive statistics on percentages of women, mean age and pre-university GPA within the four groups are depicted in Table 1.

The privacy policy of the University of Groningen states that student records can be used for research purposes as long as reports cannot be traced back to individual students.18 In accordance with this privacy policy, anonymised data were derived fromthe university administration.

다면선발 절차 Multifaceted selection process

1단계와 2단계가 있음.

In 2009, 2010 and 2011, the University of Groningen selection process consisted of two rounds.

1단계

In the first round, applicants were asked to send in a pre-structured written portfolio, based on the procedure developed by Erasmus MC medical school,11 with an additional section on reflection. 

• The portfolio contained three sections which covered, respectively, pre-university education, extracurricular activities and reflection. 
• In the section on extracurricular activities, points were granted when applicants were able to show an ability to participate in multiple activities at the same time (i.e. to combine pre-university education with additional activities). Extracurricular activities yielded points if they met fixed criteria on total duration and the amount of time spent on the activity per week. Only activities that had been carried out in the preceding 1.5 years, for at least 5 months consecutively, for more than 3.5 hours per week, yielded points. 최근 1년6개월 내에 5개월 이상 연속적으로 매주 3.5시간 이상 한 활동만을 인정함
• For the section on reflection, applicants were asked to carry out a number of reflective assignments. For example, applicants were required to ask three people in their network to give two reasons why the medical profession might suit them and one reason why they might be better off not practising medicine. Applicants were asked to reflect on these statements in a short essay. In the evaluation process, only the reflection was assessed. Evaluations of the selection process indicated that applicants required 40–60 hours to complete the first-round portfolio. 한 가지 사례는 3명의 사람에게 지원자가 의과대학에 적절한 이유와 그렇지 않은 이유에 대해 받은 뒤, 이것에 대한 짧은 에세이 기술

2단계: 225명

The 225 applicants who scored highest in the first round were invited to participate in the second round of the multifaceted selection process, which lasted an entire day and took place at the University of Groningen Medical School. 

The day was divided into four blocks comprising, respectively, a writing assignment, a patient lecture with subsequent assignments, a scientific reasoning block, and a series of short interviews and role-plays. 

• For the writing assignment, applicants were asked to write an essay about a societal problem for which ethical decision making was key. For example, in 2009, applicants were asked to write an essay on China’s one-child policy. To help them prepare for this assignment, applicants were sent a package with information about the subject a week before the second round. 
• The lecture was focused on medical knowledge, ethical dilemmas and professional behaviour, and consisted of a presentation with integrated videos, similar to the format of a video-based situational judgement test.19,20 Applicants were required to answer questions about the content of the lecture, describe and analyse the presented ethical issues and recognise (un)professional behaviour. 
• In the scientific reasoning block, applicants were asked to read a scientific article about which they were then required to answer questions that tested analytic, creative and practical skills.21,22 
• The fourth block consisted of an MMI like 5,23 series of short assignments in which applicants were asked to reflect on the assignments in their first-round portfolio, carry out a role-play that focused on communication skills, and a three-phase role-play in which they collaborated with two fellow applicants. In this scenario, three applicants first had to prepare a problematic interaction with an actor. They then held the actual conversation, after which they were asked to reflect on the course of the conversation.

Scores on the four blocks were calculated and applicants were ranked based on their total scores. The fourth block was given a double weighting. 

통계분석 Statistical analysis

We performed analysis of covariance (ANCOVA) to examine group differences in written test grades and in the number of course credits students earned in their first, second and third years. To determine which groups differed, we performed Bonferroni post hoc multiple-comparison tests for the corrected means. We conducted logistic regression analyses with changing reference groups to examine group differences in the percentages of students who achieved the highest possible score in the professionalism course. Students who dropped out of medical school in the first half of each year were excluded from these analyses because they were unable to earn course credits in the entire period under analysis and their inclusion might have caused us to overestimate effects on study progress. To assess group differences in dropout rates, we conducted logistic regression analysis including the entire group (n = 1055). All analyses were performed using IBM SPSS Statistics for Windows Version 20.0 (IBM Corp., Armonk, NY, USA) and controlled for gender, age and cohort. After initial analyses, age was eliminated as a covariate as it was not a significant predictor in any of the models. We did not correct for pre-university GPA because the group of students who were admitted based on a top pre-university GPA was defined by this variable and the other groups did not differ significantly.

기존 연구에서 중퇴율, 성적, 임상실습성적 등에서 추첨선발 학생군(4군)의 학업능력이 낮게 보고된 것은 추첨선발 학생군 전체(3군+4군)이 아니라 4군에 의한 영향일 가능성이 높아 보인다. 우리의 연구 결과에 따르면 지식검사와 학습발전정도에 있어서는 다면선발이 좀 더 예측력을 갖는 것으로 보인다. 이러한 결과를 종합하면 의과대학은 지원절차에 시간과 노력을 쏟은 지원자만을 선발하는 것이 도움이 될지도 모른다. 이전에 어떤 전형에도 지원한 적이 없는 추첨입학학생군의 낮은 학업능력은 다면선발에 내재된 '자발성'에 의해 설명될 수 있다.이는 '자기선발'과정이기도 하다. 동기부여가 더 잘 된 학생은 의과대학 입학을 위한 모든 가능한 기회를 다 사용해볼 것이며, 동기부여가 덜 된 학생은 그냥 추첨선발까지 기다려볼 것이다.

The lower performance of non-participants might indicate that previously reported differences in dropout rates,11 test grades28 and clerkship grades12 between selected and lottery-admitted students in the Dutch system can be attributed to lottery- admitted applicants who have not participated in the selection process, rather than to the lottery-admitted group as a whole. Our results in fact indicate that for knowledge test scores and study progress, participation in the multifaceted selection process may be more predictive than acceptance in this process. As such, it might be profitable for medical schools to admit only applicants who have put time and effort into their applications. The lower performance of non-participants may be explained by the voluntary nature of the multifaceted selection process, which implies a mechanism of self-selection that is induced by the Dutch admissions system.29 Highly motivated students might use every opportunity to achieve admission into medical school, whereas less motivated students might choose to wait for the lottery.

이러한 동기부여의 차이에 대하여 가능한 설명은 선발절차에 참여(participation)하는 것에도 40~60시간의 노력이 필요하다는 것이다. 동기부여가 잘 된 학생은 이러한 것에 쉽게 노력과 시간을 투자할 수 있는 반면 그렇지 않은 학생은 이 자체가 힘들 수 있다.

A possible explanation for this difference in motivation is that participation in the selection process requires 40–60 hours of work. Highly motivated students might easily have invested this kind of time and effort, whereas less motivated students may have perceived this threshold as too high.

---

Med Educ. 2014 Dec;48(12):1201-10. doi: 10.1111/medu.12537.

Selection and study performance: comparing three admission processes within one medical school.

Schripsema NR1, van Trigt AM, Borleffs JC, Cohen-Schotanus J.

Author information

• 1Center for Research and Innovation in Medical Education, University of Groningen and University Medical Center Groningen, University of Groningen, Groningen, the Netherlands; Institute for Medical Education, University of Groningen and University Medical Center Groningen, Groningen, the Netherlands.

Abstract

OBJECTIVES:

This study was conducted to: (i) analyse whether students admitted to one medical school based on top pre-university grades, a voluntary multifaceted selection process, or lottery, respectively, differed in study performance; (ii) examine whether students who were accepted in the multifaceted selection process outperformed their rejected peers, and (iii) analyse whether participation in the multifaceted selection procedure was related to performance.

METHODS:

We examined knowledge test and professionalism scores, study progress and dropout in three cohorts of medical students admitted to the University of Groningen, the Netherlands in 2009, 2010 and 2011 (n = 1055). We divided the lottery-admitted group into, respectively, students who had not participated and students who had been rejected in the multifaceted selection process. We used ancova modelling, logistic regression and Bonferroni post hoc multiple-comparison tests and controlled for gender and cohort.

RESULTS:

The top pre-university grade group achieved higher knowledge test scores and more Year 1 course credits than all other groups (p < 0.05). This group received the highest possible professionalism score more often than the lottery-admitted group that had not participated in the multifaceted selection process (p < 0.05). The group of students accepted in the multifaceted selection process obtained higher written test scores than the lottery-admitted group that had not participated (p < 0.05) and achieved the highest possible professionalism score more often than both lottery-admitted groups. The lottery-admitted group that had not participated in the multifaceted selection process earned fewer Year 1 and 2 course credits than all other groups (p < 0.05). Dropout rates differed among the groups (p < 0.05), but correction for multiple comparisons rendered all pairwise differences non-significant.

CONCLUSIONS:

A top pre-university grade point average was the best predictor of performance. For so-called non-academic performance, the multifaceted selection process was efficient in identifying applicants with suitable skills. Participation in the multifaceted selection procedure seems to be predictive of higher performance. Further research is needed to assess whether our results are generalisable to other medical schools.

© 2014 John Wiley & Sons Ltd.

Comment in

• The emotional side of selecting for medicine. [Med Educ. 2014]

PMID:

 

25413913

 

[PubMed - in process]