무시험 입학 학생과 시험을 보고 입학한 학생의 진전(progress) 비교

무시험 입학 학생과 시험을 보고 입학한 학생의 진전(progress) 비교

Progress of medical students after open admission or admission based on knowledge tests

Gilbert Reibnegger, Hans-Christian Caluba, Daniel Ithaler, Simone Manhal, Heide Maria Neges & Josef Smolle

배경

오스트리아에서 일반적으로 대학 입학은 고등학교를 마친 사람이라면 누구에게나 열려있으나, 일부 과(의학과 치의학 포함)에서는 몇 가지 추가적인 입학기준을 적용하는 것이 2005년 European Court의 결정에 따라 가능해졌다. 우리는 의과대학 학생들의 성취도 변화의 차이를 시간에 따라 보고자 하였다.

방법

2002/03부터 2007/08 년까지 Medical University of Graz의 Human medicine programme에 입학한 모든 2532명의 입학생을 대상으로 하였다.  Non-parametric, 그리고 semiparametric 생존분석 기술을 사용해서 admission test도입 전/후에 대해 첫 번째 두 학기를 마치는데까지 걸리는 시간을 비교하였다. 이 목표(첫 번째 두 학기를 마치는 것)를 도달하지 못하고 유급하게 되는 학생들의 시간적 패턴도 살펴보았다. 성별, 연령, 국적을 교란변수로서 고려하였다.

결과

학생들이 성공적으로 학업을 마치는 것에 대한 누적확률이 무시험으로 입학한(admitted openly) 학생보다 selected student에서 훨씬 더 높았다. 무시험으로 입학한 학생들 중 20.1~26.4%의 학생만이 첫 두 학기를 1년에 마친데 비해서, 입학시험을 치르고 들어온 학생에서는 75.6~91.9%의 학생이 그 목표에 도달했다.

결론

이 분석으로 의과대학에 있어 open admission에 비해서 performance-based 선발이 학습 성취를 크게 향상시킨다는 것을 확인할 수 있었다. 추가적으로, 유급의 비율이 크게 감소하였다. 입학시험은 학생의 시간이라는 측면과 공공의 자원이라는 측면에서 비용을 크게 절감시킨다고 할 수 있다.

---

Statistical methods

We decided to measure the effects of different modes of student admission on study progress by identifying the length of time that elapsed between a defined starting point (i.e. the start of term in the year of enrolment) and a defined endpoint (i.e. the time of successful completion of the first part of the study programme). 

When analysing such ‘waiting time’ data, the application of ordinary statistical methods is frequently hindered by the presence of so-called censored data: students differ substantially in their individual study progress; some of them succeed within the expected time, but others, for quite diverse reasons, need more time to achieve the same goal. 

In addition, a certain proportion of students ‘disappear’ from study: they change to another programme or university, or they drop out of study altogether. These individuals who do not reach the defined endpoint within the period of investigation contribute information to the study for a certain amount of time but not thereafter and are hence referred to as ‘censored’ observations. Commonly used multivariate techniques such as linear or logistic regression analysis cannot handle censored data and, thus, typically are not adequate for analysing waiting times properly. 

Consequently, for our investigation into data on study progress, we adopted statistical methods from the field of survival analysis.6 The non-parametric product limit technique by Kaplan and Meier7 was used to compute the cumulative probabilities for the study success of defined categories of students. 

In ‘normal’ survival studies, the results of Kaplan–Meier calculations are usually represented by depicting the cumulative probability of survival as a step function decreasing from 100% to smaller percentages, as observation time progresses. In such studies, the endpoint is usually a negative event, such as death. As the endpoint in our study is a desirable event, namely, the successful completion of the first part of the study programme, by contrast with conventional survival curves we decided to represent the results as ‘one minus survival’ curves (i.e. cumulative probabilities of success start at 0% and increase with observation time). 

Differences of these cumulative probabilities among different categories were tested by the generalised likelihood ratio method (Breslow chi-squared statistic). The semi-parametric proportional hazards model of Cox8 was then employed in order to study the effect of potential predictor variables in a multivariate manner and to identify the relative strength of each individual predictor variable in the context of all other variables.

The ‘hazard’ can be described as the instantaneous probability that an individual will experience an event at time t while this individual is at risk for an event.

In survival analysis, we generally distinguish between non-parametric, semi-parametric and parametric methods. 

For example, the Kaplan–Meier product limit method does not make any assumption about the underlying hazard function (‘baseline hazard’), but in a completely empirical way computes the cumulative probabilities of the terminating events merely from the data at hand: it is a non-parametric approach. 

Likewise, the Cox model does not make assumptions about the baseline hazard, but the effect of covariates is modelled in a parameterised fashion. The parameters are estimated from the data and allow quantification of the relative strength of the respective covariate (predictor variable). Therefore, the Cox technique is called a semi-parametric approach. 

A parametric model provides an explicit mathematical model for the baseline hazard assuming one of several possible distribution models (exponential distribution, Weibull distribution, Gompertz distribution and others) with adjustable parameters and, if appropriate, allows not only the estimation of the relative strengths of predictor variables, but also the prediction of the cumulative probabilities as a function of time by means of an analytic expression.

---

Med Educ. 2010 Feb;44(2):205-14. doi: 10.1111/j.1365-2923.2009.03576.x. Epub 2010 Jan 5.

Progress of medical students after open admission or admission based on knowledge tests.

Reibnegger G, Caluba HC, Ithaler D, Manhal S, Neges HM, Smolle J.

Source

Medical University of Graz, Graz, Austria. gilbert.reibnegger@medunigraz.at

Abstract

CONTEXT:

Although admission to university in Austria is generally open for applicants who have successfully completed secondary school, in some areas of study, including human medicine and dentistry, the selection of students by additional criteria has become legally possible as a result of a decision by the European Court in 2005. We studied the impact of this important change on the temporal pattern of medical students' progressthrough the study programme.

METHODS:

All 2532 regular students admitted to the diploma programme in human medicine at the Medical University of Graz during the academic years 2002/03-2007/08 were included in the analysis. Non-parametric and semi-parametric survival analysis techniques were employed to compare the time required to complete the first two study semesters (first part of the curriculum) before and after the implementation of admission tests. Temporal patterns of dropout before this goal was achieved were also investigated. Sex, age and nationality of students were assessed as potential confounding variables.

RESULTS:

The cumulative probability of study success was dramatically better in selected students versus those who were admitted openly (P < 0.0001). Whereas only 20.1-26.4% of openly admitted students completed the first two study semesters within the scheduled time of 1 year, this percentage rose to 75.6-91.9% for those selected by admission tests. Similarly, the cumulative probability for dropping out of study was also significantly lower in selected students (P < 0.0001). By univariate as well as multivariate techniques, student nationality, age and sex were also identified as partly significant, albeit weak, predictors.

DISCUSSION:

The analysis convincingly demonstrates that, by contrast with open admission, performance-based selection of medical studentssignificantly raises the probability of successful study progress. Additionally, the proportion of dropouts is significantly reduced. Thus, admissiontests save considerable costs, in terms of both student time and public resources.

PMID:

 

20059671

 

[PubMed - indexed for MEDLINE]