입학 후 학업능력 예측: 학생의 배경환경과 과거 학업능력의 상대적 중요성 (Med Educ, 2015)

입학 후 학업능력 예측: 학생의 배경환경과 과거 학업능력의 상대적 중요성 (Med Educ, 2015)

Predicting performance: relative importance of students’ background and past performance

Karen M Stegers-Jager,1 Axel P N Themmen,1,2 Janke Cohen-Schotanus3 & Ewout W Steyerberg4

academic failure와 관계되어있다고 보고된 입학전 특성들

Pre-admission characteristics that have been reported to relate to academic failure are ethnic minority status,6,7 maturity,8,9 male gender,7,8,10 and lower levels of previous academic performance, in particular low Medical College Admission Test (MCAT) scores and low science grade point averages (GPAs).7–9

의과대학 입학 후 첫 달의 성적과의 관계를 밝힌 것도 있음.

Several studies have confirmed the relationship between student performance during the first months at university and subsequent performance. 15–17 A recent study by Winston et al. showed that results on an examination administered after the first 2 weeks of medical school represented a strong early predictor of success or failure.4

그러나 각 의학교육의 단계마다 낮은 학업능력의 risk factor는 서로 다름

In addition, it has been shown that risk factors for poor performance vary at different stages of the medical course.7,19,20

Participants and procedure

코호트 선택의 이유

Students from six consecutive cohorts (2002–2007) at Erasmus MC Medical School were included in this study (n = 2357). We selected these six cohorts for two reasons: (i) the curriculum did not change during this period, and (ii) data on the ethnicity of these cohorts were available from a national database of students in higher education in the Netherlands 1cijferHO. Data on academic performance were derived from the university student administration system and confidentiality was guaranteed. Because data were collected as part of regular academic activities, individual consent was not necessary.

모든 variable을 넣고 logistic regression.

As all variables are known to be associated with medical school performance, all were entered simultaneously in a multivariable logistic regression model. We also included cohort as a stratification variable. Statistical interaction terms were used to study the potentially differential effects of one predictor by values of another predictor.

1학년의 첫 4개월의 GPA의 영향은 남학생보다 여학생에서 두드러졌다. 같은 성적이라면 남학생이 1학년을 이수할 가능성이 더 높았다.

The effect of GPA at 4 months on the Year 1 completion rate was less prominent for males than for females: male students with a high GPA at 4 months were less likely to complete the Year 1 course on time than female students with a similar GPA, whereas male students with a low GPA at 4 months (< 4.5 on a scale of 1–10) were more likely to complete the Year 1 course on time than female students with a similar GPA.

본 연구는 미래 학업능력 예측에 가장 최근의 학업능력(입학 전이든 후든)의 중요성을 보여준다. 그러나 임상실습 단계에서는 학생의 background가 주요한 예측인자로 작용한다.

This study confirms the importance of the most recent past performance – either before or at medical school – as a predictor of future performance in pre-clinical training. However, it also reveals that in clinical training the student’s background becomes the main predictor of performance as students from all minority groups and first-generation university students had a higher risk of achieving lower clinical grades in a model that included pre-clinical performance.

입학전 GPA는 그 자체로는 강력한 예측인자였지만 입학 후 Performance가 포함된 모델에서는 유의성을 잃었다. 이 말은 의과대학 입학 직후 4개월간의 학업능력이 훨씬 더 중요하다는 것을 보여준다. 의과대학에서의 첫 시험에서의 성적을 바탕으로 위험학생을 선별해내는 것을 권고하는 것이며, 학생과 학업환경의 상호작용에 따른 결과가 입학시점에서의 자질 평가를 바탕으로 한 것보다 더 정확하다는 것을 보여준다. 

However, our study offers a more nuanced picture. Although pu-GPA was the most important predictor in the model that included only preadmission data, this factor was rendered insignificant by the addition of early performance at medical school. In other words, the factor pu-GPA was greatly outweighed by study performance data that became available during the first 4 months at medical school. This confirms the suggestion that the identification of at-risk students based on the first results of the interaction between a student and the academic environment is more accurate than identification using entry qualifications and is also in line with the findings of others.4,18

그러나 남학생의 경우는 1학년 학업능력을 첫 몇 개월의 성적으로 예측하는 것에서 예외였는데, 여학생에서보다 남학생에서 예측력이 더 낮았다. 이는 아마도 남학생의 self-efficacy가 더 높은 것이 원인일 수 있다. 남학생은 첫 4개월동안 성적이 높으면 스스로를 과대평가 할 수도 있으나, 낮은 성적을 받는 것 역시 여학생에 비해서 self-confidence에 덜 detrimental하다.

Apparently, the differences found in Year 1 performance for these subgroups can be explained to a large extent by performance during the first months at medical school. An exception to this is male gender, as being male remains a predictor of poorer Year 1 performance after the addition of early medical school performance. The interaction effect we found between gender and GPA at 4 months suggests that early performance is less predictive of later performance for males than it is for females. This may be explained by higher self-efficacy in males than in females, which has been reported previously in medical students.30,31 In male students, high grades during the first 4 months may lead to an over-estimation of their own ability, whereas achieving low early grades may be less detrimental to their self-confidence than it is in female students

인종과 사회적 배경은 임상 수행능력의 중요한 예측인자였다. 이는 임상수련은 전임상교육과는 서로 다른 메커니즘에 의해서 작동됨을 보여준다. 한 가지 가능한 이론은 문화적 자본(cultural capital)이다. Bourdieu에 따르면 이는 다음과 같이 정의된다. 비전통적인 소수인종 혹은 가계 내 최초대학입학자의 경우 이들의 cultural capital이 institutional habitus에 잘 맞지 않는다고 볼 수 있다. 임상실습에서는 보다 주관적인 평가가 많이 작동하기 때문에, 문화적 자본이 전임상교육 기간보다 더 강하게 작동할 수 있다. 

The finding that ethnicity and social background were important predictors of clinical performance, even after adjusting for pre-clinical performance, suggests that performance in clinical training is explained by mechanisms other than those referred to in pre-clinical training. A possible mechanism refers to the concept of cultural capital, which, according to Bourdieu, can be understood as ‘knowledge of the norms, styles, conventions and tastes that pervade specific social settings and allow individuals to navigate them in ways that increase their odds of success’ (see Massey et al.33). The cultural capital of non-traditional – ethnic minority and first-generation university – medical students is less likely than that of traditional medical students to be recognised and positively valued within medical school; that is, it does not fit the ‘institutional habitus’ (see Thomas34). As more subjective examination methods are used in clinical training than in pre-clinical training,35 it may be that the role of cultural capital is more prominent during clinical than pre-clinical training. Although further research is required to confirm these proposed effects of cultural capital, our assumption is supported by our finding that having a medical doctor as a parent is related to poorer performance in pre-clinical but better performance in clinical training.

임상실습과 관련하여 또 다른 가능성은, 문화적 자본과도 관련되어있지만, 문화적 편견이 평가자에게 작용했을 수 있다는 것이다. 사람들은 자기와 같은 그룹에 속한 사람을 보다 신뢰하는 경향(in-group bias)가 있고, 자신과 비슷하거나 그들이 좋아하는 사람과 비슷한 경우 더 신뢰하는 경향이 있다(similarity principle) 이러한 것을 평가자가 스스로 통제하려 하지만, 소수인종이나 first-generation university student는 "전통적" 학생그룹보다 낮은 점수를 받았을  수 있다.

Another possible mechanism in clinical training, which is related to cultural capital, is cultural bias on the part of the examiners. Inevitably, people will have more positive views of those they believe to be part of their group (referred to as ‘in-group bias’36) and people tend to trust those who are similar to themselves or who are similar to people they like (a phenomenon known as the ‘similarity principle’37). Unless traditional examiners are aware of and attempt to control these automatic reactions,38 it is likely that ethnic minority and first-generation university students will receive lower grades than their traditional counterparts. More detailed experimental studies may assist in elucidating the processes underlying judgement and decision making in clinical assessments.

본 연구의 첫 번째 한계점은 일부 factor에 대해서 제한된 숫자의 학생만 응답한 것인데 missing value에 대해서 multiple imputation 기법을 활용하였다. 

A first limitation of our study is that data on the pre-admission factors ‘first-generation university student’ and ‘medical doctor as parent’ were collected for a restricted number of participants. However, to deal with the missing values, we used the technique of multiple imputation, which is widely accepted as suitable.26 As they allow the use of data that are available for other predictors that would otherwise be lost, imputation methods, especially multiple imputations, are superior to complete case analysis. 26,41,42 The ORs calculated in the imputed dataset in our study were similar and, if different, generally more conservative than the ORs in the unimputed dataset (Table S1).

---

Med Educ. 2015 Sep;49(9):933-45. doi: 10.1111/medu.12779.

Predicting performance: relative importance of students' background and past performance.

Stegers-Jager KM1, Themmen AP1,2, Cohen-Schotanus J3, Steyerberg EW4.

Author information

• 1Institute of Medical Education Research Rotterdam, Erasmus MC University Medical Centre Rotterdam, Rotterdam, the Netherlands.
• 2Department of Internal Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, the Netherlands.
• 3Centre for Research and Innovation in Medical Education, University Medical Centre Groningen, University of Groningen, Groningen, the Netherlands.
• 4Centre for Medical Decision Making, Department of Public Health, Erasmus MC-University Medical Centre Rotterdam, Rotterdam, the Netherlands.

Abstract

CONTEXT:

Despite evidence for the predictive value of both pre-admission characteristics and past performance at medical school, their relativecontribution to predicting medical school performance has not been thoroughly investigated.

OBJECTIVES:

This study was designed to determine the relative importance of pre-admission characteristics and past performance in medical school in predicting student performance in pre-clinical and clinical training.

METHODS:

This longitudinal prospective study followed six cohorts of students admitted to a Dutch, 6-year, undergraduate medical course during 2002-2007 (n = 2357). Four prediction models were developed using multivariate logistic regression analysis. Main outcome measures were 'Year 1 course completion within 1 year' (models 1a, 1b), 'Pre-clinical course completion within 4 years' (model 2) and 'Achievement of at least three of five clerkship grades of ≥ 8.0' (model 3). Pre-admission characteristics (models 1a, 1b, 2, 3) and past performance at medical school (models 1b, 2, 3) were included as predictor variables.

RESULTS:

In model 1a - including pre-admission characteristics only - the strongest predictor for Year 1 course completion was pre-university grade point average (GPA). Success factors were 'selected by admission testing' and 'age > 21 years'; risk factors were 'Surinamese/Antillean background', 'foreign pre-university degree', 'doctor parent' and male gender. In model 1b, number of attempts and GPA at 4 months were the strongest predictors for Year 1 course completion, and male gender remained a risk factor. Year 1 GPA was the strongest predictor for pre-clinical course completion, whereas being male or aged 19-21 years were risk factors. Pre-clinical course GPA positively predicted clinical performance, whereas being non-Dutch or a first-generation university student were important risk factors for lower clinical grades. Nagelkerke's R(2) ranged from 0.16 to 0.62.

CONCLUSIONS:

This study not only confirms the importance of past performance as a predictor of future performance in pre-clinical training, but also reveals the importance of a student's background as a predictor in clinical training. These findings have important practical implications for selection and support during medical school.

© 2015 John Wiley & Sons Ltd.

PMID:

 

26296410

 

[PubMed - in process]