의과대학생 선발을 위한 포괄적 모델(Med Teach, 2009)
A comprehensive model for the selection of medical students
MILES BORE, DON MUNRO & DAVID POWIS
The University of Newcastle, Australia
의과대학에 입학절차가 필요한 두 가지 이유가 있다. 하나는 입학 가능한 정원보다 지원자가 많기 때문이며, 두 번째는 사회와 전문직 집단이 유능하고 윤리적인 의사가 될 사람만을 원하기 때문이다. 지금까지 선호되었던 방법은 학업성취만을 가지고 뽑는 것이었다. 그러나 많은 국가에서 활용가능한 학업성취만으로는 지원자들 사이에 충분히 의미있는 차이를 보여주지 못한다.
There are essentially two reasons why medical schools around the world need to have a selection procedure for medical students. Firstly, there are almost invariably more applicants than there are places available. Secondly, there is a social and professional desire to admit only those who will become competent and ethical practitioners. The erstwhile preferred method to achieve the first aim is to select on prior academic achievement alone. However, in many countries the available measures of academic performance do not provide sufficient variance to allow meaningful differentiation in performance (Rolfe & Powis 1997; McManus et al. 2005) on which to base selection decisions.
우리가 제시한 모델은 의과대학생선발, 인성 심리학, psychometrics로부터 만든 것이다.
The model we propose has evolved from our research and work in the fields of medical student selection, personality psychology and psychometrics (Powis 1998; Powis & Rolfe 1998; Bore et al. 2005; Lumsden et al. 2005; Munro et al. 2005).
이 모델의 개괄은 그림1에 있다.
The model consists of the following components and is shown schematically in Figure 1.
- Informed self-selection through the provision of timely vocational guidance.
- Academic achievement as indicated by performance at school and/or undergraduate studies.
- Cognitive ability as measured by psychometric testing.
- Personality as measured by psychometric testing.
- Interpersonal skills as measured by interview.
충분한 정보를 기반으로 한 자기선발
Informed self-selection
많은 지원자가 지원시에 17~18세에 불과하다는 것을 감안하면, 학교를 다니는 동안 자신이 정말 의학에 적합한가에 대한 생각을 할 수 있는 기회가 주어져야 한다.
Given that many applicants are just 17 or 18 years old at the time of application, such insight usually needs to have been gained in the school years.
한가지 방법은 의학교육과 의료에 대한 모든 것을 다 보여주는 웹사이트를 만드는 것이다. 다음의 내용이 담길 수 있다.
One possible approach would be the development of a website that presents a vocational ‘warts and all’ view of medical education and practice (Blundell et al. 2007).
- Descriptions of being a medical student supplied by current and past students.
- Descriptions of internship and specialty training.
- A typical day in the life of each specialty, with negatives emphasised as much as positives.
- Suggestions on where to get more information.
- Suggestions on how to find out if one is suited to medicine (e.g. by doing voluntary work in a hospital).
- Suggestions about other health professional careers.
학업성취도
Academic achievement
높은 학업성취도만으로는 유능하고 윤리적인 의료를 보장해주지 않는다. 그러나, 미래의 행동의 가장 좋은 예측인자는 과거의 행동이고, 과거의 학업성취는 미래의 학업성취와 유의미한 상관관계가 있다.
High academic achievement alone does not ensure the competent and ethical practice of medicine. However, the best predictor of future behaviour is past behaviour and past academic achievement is correlated significantly with future academic performance (Kuncel et al. 2001; McManus et al. 2005).
학업성취를 보여주는 척도가 지원자간에 차이를 충분히 보여주지 못할 수도 있다.
The metric used to indicate academic achievement might not provide sufficient discrimination between candidates,
게다가 만약 UAI와 같이 지원자간 차이를 크게 보여주는 척도라 하더라도 98.6점을 받은 지원자가 95점 96점 97점을 받은 지원자보다 더 나은 학생이나 의사가 될 것이라는 이유는 없다.
Even where the range of the metric allows greater discrimination, as with the Universities Admissions Index (UAI) in Australia, there is no reason to suppose that an applicant with a UAI of 98.6 (out of 100) will make a better student or doctor than a person with 95 or 96 or 97, for example.
또한 특정 과목이 선수과목으로 요구되어야 하는가도 중요하다.
There is also the issue of whether achievement in specific subjects should be used as prerequisites.
우리는 그러한 선수과목이 단순히 허들을 하나 더 추가하는 것이 되어서는 안되며, 그 과목의 의과대학 교육과정에서 필요할 때에만 사용해야 한다고 생각한다.
we believe that it should not be imposed simply as an extra hurdle, but only when required by the medical school curriculum.
선수과목과 같은 것들은 의과대학 프로그램에 대한 접근성 정도에 큰 차이를 만든다. 낮은 레벨의 더 적은 요구조건이 낮은 사회경제적 배경에 있는 사람들의 접근성을 크게 높여준다. 반면 높은 수준의 서로 다른 가중치가 부여되는 요구조건은 지원자 풀을 축소시키고, 다음 단계의 선발을 처리하기 쉽게 만든다.
These will influence the degree of access to the medical school program: lower levels and fewer prerequisites would provide greater access to people from lower social and economic backgrounds (Powis et al. 2007), while higher levels and differently weighted prerequisites will reduce the applicant pool so that the next stage of selection is manageable..
한 가지 확실한 것은, 학업성취도에 가중치를 덜 줄수록 다른 것에 가중치가 더 들어가야 한다는 것이다. 이것이 다른 선발변수를 사용하는 것을 정당화시켜준다.
One point is obvious: Where less weight is given to academic criteria, more weight has to be given to other selection variables. It is this very point that, in part, justifies using other selection variables.
인지적 능력
Cognitive ability
대부분의 의과대학 입학시험은 인지기술 검사가 포함된다.
Most medical school selection procedures in Australia and the UK include a test of cognitive skills.
지식검사에 관해서 두 가지 이슈가 있다.
- 하나는 그러한 지식검사는 불필요하다는 것인데, 왜냐하면 학업성취 지표에 의해서 이미 신뢰성있게 측정되었기 때문이다.
- 또 다른 이슈는 특정 영역에 대한 지식을 가졌다 하더라도 그 영역은 이미 의과대학 교육과정에 포함되어 있기 때문이 그 변수를 선발과정에 활용하는 것에 대한 논쟁이 있다.
There are two issues with knowledge tests. First, they appear to be redundant given that such knowledge has been measured more reliably by academic achievement indicators (also probably more validly than is possible with brief tests). The other issue is that gaining knowledge in these specific areas is part of the medical curriculum anyway, and so the justification for using such a variable in selection is debatable.
그러나 구체적인 지식을 검사하는 시험은 기간이 단축된 GEP에서는 적절할 수 있는데, 이 경우에는 생물/화학 등과 같이 교육과정에서 다루지 않는 지식을 점검할 수 있기 때문이다.
However, the use of specific knowledge tests may be appropriate for shortened graduate entry programs where it is necessary to check for some content knowledge of biology and chemistry not covered in the program’s curriculum.
또 다른 접근법은 구체적인 인지능력(지식이 아니라)을 검사하는 것으로, 이것은 의과대학과 의사로서의 성공과 연관되어있을 수 있다. 여기에 포함되는 것은 언어능력, 언어추론, 수리능력, 수리추론 이런 것들이다. 그러나 메타분석을 보면 일반인지능력(General Cognitive Ability, GCA)가 직업 내에서의 성취와 수행능력에 중간정도~강한 예측인자가 되지만, 세부적인 능력(specific ability)는 GCA의 전체적인 예측력을 높여주지는 못하는 것으로 나온다.
An alternative approach is to measure specific cognitive abilities (rather than knowledge) that may be related to success in medical school and medical practice, such as verbal ability/ reasoning, numerical ability/reasoning and so on. However, meta-analytic research has clearly shown that General Cognitive Ability (GCA) is a moderate to strong predictor of occupational attainment and performance within occupations, and that measures of specific abilities do not appreciably improve the overall predictive power of GCA (Schmidt & Hunter 2004; Brown et al. 2006).
An advantage of using an indicator of GCA rather than measuring narrower specific abilities hinges on the concept of ‘compensation’. A test of GCA might include questions that measure specific cognitive abilities, e.g. verbal, numerical, abstract, spatial and other reasoning abilities. If summed to produce a single score (indicating individual differences in GCA), then a lesser ability in one specific area can be compensated for by higher abilities in other areas. However, if people are selected on their performance on individual specific abilities, then high ability in one or two areas cannot compensate for lower ability in others. The outcome of using a general (compensatory) approach to ability testing in selection is higher variability within the selected pool of applicants. All will have high GCA, but specific abilities will vary within this pool and this we see as important, given the diversity of medical practice.
While some research has found academic achievement to be a better predictor of some occupational outcomes in medicine than GCA (McManus et al. 2003), the broader research reported in the GCA meta-analysis literature does demonstrate the significant relationship of GCA to educational performance and occupational attainment.
인성
Personality
의과대학생 선발에서 인성검사를 사용하는 것은 아마 가장 논쟁이 많은 영역일 것이다. 그러나 인성척도는 지금까지 경찰, 군, 공무원, 상업이나 기업 영역 등에서 선발에서 상당히 많이 사용되어 왔다.
The use of personality tests in the selection of medical students is possibly the most contentious area in the selection debate. However, personality measures have been (and are) used extensively for selection in the commercial and industrial sectors as well as the police, military and other government services in many countries.
FFM이 가장 많은 근거를 가지고 있다.
‘Five Factor Model’ (FFM) has emerged as the dominant empirically supported approach.
각각의 다섯 개 특징은 하위 영역이 있다.
Each of the five traits consists of a number of lower-order facets.
최근의 성격검사와 관한 메타분석에서 강조되어야 할 세 가지가 있다.
There are three relevant points made in a recent personality meta-analysis that may be emphasised.
첫 번째는, 성격특성이 예측타당도를 보여주었다는 것이다. 비록 0.10~0.45정도로 높지는 않지만, 수년간에 걸쳐서 이러한 중간 정도의 타당도만으로도 선발결정이 매우 중요하고, 선발되는 비율이 매우 낮은 의과대학생선발과 같은 과정에서는 충분히 가치가 있는 것으로 드러났다.
First, personality traits have demonstrated predictive validity: there are correlations between personality predictors and work-related criteria of 0.10 and 0.45. While such coefficients appear low, it has been recognised for many years that tests of even modest validity can make a valuable contribution to selection decisions where the proportion to be selected from the applicant pool is low and the importance of good selection decisions is high (as in medicine).
두 번째는, 비록 인지능력이 근무지에서 수행능력에 대해서 많은 부분을 설명해주지만, 성격이 그 예측타당도에 추가적으로 기여하는 바가 있다는 점이다. 즉, 성격특성이 추가되면 인지적능력 단독으로 예측한 것보다 더 많은 부분을 예측할 수 있다.
Second, while cognitive ability accounts for a greater proportion of variance in work-related performance criteria, personality has incremental predictive validity: that is, the proportion of variance accounted for in job-related criterion measures increases when personality traits are included alongside cognitive ability (Ones et al. 2007).
세 번째는, 사람들이 '좋은 척'하는 경향이 있지만, 이것으로 인한 예측타당도의 저하는 매우 낮다는 점이다. 이것이 고무적이기는 하지만 high-stake 검사에서 사람들이 고의로 인격척도를 속이는 것에 대응할 수 있는 두 가지 전략을 다루고자 한다.
- 하나는 lie scale을 포함하는 것이며,
- 다른 하나는 극단치를 제외하는 것이다.
The third point is that while there is a tendency for people to ‘fake good’ on personality tests when they are taken under high stakes conditions, it has been found that this reduces the predictive validity of the tests only minimally (Ones et al. 2007). While this is encouraging, we suggest that faking on personality measures in high stakes testing can be countered to some extent by two strategies: inclusion of a ‘lie scale’ and by exclusion of extreme scorers. The details of these strategies are elaborated below.
The inclusion of non-cognitive variables in selection careful consideration of two aspects. First is the question of what variables to include. Our view based on the literature is that there are four essential non-cognitive criteria for competent and ethical practitioners in the medical and allied health professions:
- ‘Involved with’ rather than ‘detached from’ or ‘manipulative of’ others (Agreeableness in terms of the Big 5).
- ‘Emotionally stable’ and ‘resilient’ rather than ‘overly emo- tionally reactive’ or ‘unpredictable’ (Big 5 ‘Emotional Stability’).
- ‘Self-controlled’ and ‘conscientious’ rather than ‘impulsive’ and ‘disorderly’ (Big 5 ‘Conscientiousness’).
- Neither too judgemental nor too permissive in one’s moral/ ethical values.
예측타당도에 대한 근거가 있는 다른 비인지적 변인들도 있다.
There are other non-cognitive variables that can be, and have been, used in selection, for example,
- integrity (considered to be a ‘compound’ personality construct related to conscien- tiousness, agreeableness and emotional stability; Ones & Viswesvaran 2001),
- stress tolerance (Hogan R & Hogan J 1995) and
- Moral Orientation (Bore et al. 2005) tests for these qualities can be valuable where there is an evidence of predictive validity for specific occupations.
주요 이슈는 어떻게 이 변인들을 사용할 것인가다. 단순합을 하는 것은 부적절함.
A major issue in the present context is how these variables should be used in the selection model.
인터뷰 대상자 선정을 위한 점수 합 구하기
Combining scores to create the interview pool
선발결정을 내릴 때 몇 가지 고려할 수 있는 모델이 있다. 가장 그럴듯한 것은 regression model로서, 선발변인들을 하나의 criterion이나 outcome에 대해서 regress하는 것이다. 그리고 regression weight를 구해서 실제 선발 점수에 적용한다. 이러한 접근법은 'outcome variable'이 쉽게 가능한 직종에서 널리 사용되어 왔다.
There are several models that can be considered when using scores to make selection decisions (Gatewood & Feild 2001). Perhaps the most appealing is the regression model where, experimentally, the selection variables are regressed against a criterion/outcome variable and the regression weights found are then applied to actual selection scores. This approach has been used extensively in selecting people for jobs in which outcome variables are readily available (e.g. sales perfor- mance, production rates, accident rates, absenteeism and so on).
그러나 의료에 있어서 쉽게 측정가능하고 타당도가 있으면서 윤리적으로 수집가능한 criterion variable의 집합은 아직 밝혀지지 않았다. 놀라운 일도 아니다.
A set of criterion variables (or compound of criteria) for the practice of medicine that can be reliably measured, that have evidence of validity and that can be obtained ethically has yet to be agreed and established. This is perhaps not surprising.
regression model의 대안은 점수를 multiple cut-off model로 활용하는 것이다. 여기서는 특정 범위에 있는 지원자가 선택된다.
An alternative to the regression model is to use the scores of each test in a multiple cut-off model. That is, applicants who score within a particular range for each variable measured are selected (in the model we are outlining here) into the interview pool.
선발 절차 - 예시
The selection procedure – A demonstration
어떤 선발절차와 마찬가지로, 이 모델은 가장 적절한 학생을 선발할 가능성을 최대화하기 위해서 만들어졌다. 첫 번째로 과거 학업성취도와 informed self-selection으로 지원자 풀이 만들어진다. 그러나 우리는 상위 10%의 학업성취도 집단으로 제한하기를 권고한다. 이는 학교에서의 수행능력의 기여수준을 효과적으로 낮춘다. 그리고 전체적인 능력과 관련된 더 많은 변인이 이후의 선발과정에서 다뤄진다.
Like any selection procedure, the model here is designed to maximise the probability of selecting the most appropriate students. In the first instance a pool of applicants is created based on past academic performance and informed self- selection. Each medical school sets its own academic perfor- mance criterion; however, we would suggest a cut point that allows the top 10% of academic achievers over this hurdle (Neame et al. 1992). This effectively lowers the contribution of school performance: more of the variance in overall ability is dealt with by the subsequent selection steps.
가상의 1000명 집단이 있을 때를 가정.
To demonstrate the multiple-cut off method, we have used actual scores from experimental testing conducted by us in a number of sample groups to create a hypothetical n¼1000 pool.
6개의 variable로 검사점수를 얻는다. z score로 변환하여 이 점수를 가지고 면접대상자를 선발한다.
In our sample of 1000 applicants, test scores were obtained for six variables as shown in Table 1. The raw scores were normed and transformed into z scores (mean of 0, SD of 1). We can now use the scores to create the interview pool in a two-step procedure.
In Step 1, excessively positive self-representation on the non-cognitive tests can be managed by removing extreme high scorers on a Lie scale
We also suggest that the extreme high and low scorers (z scores less than 2 or greater than +2) on the non-cognitive tests should not progress to the interview pool.
The reason for this is two-fold. First, ‘faking good’ would produce extremely high scores so this is another strategy to manage lying. Second, while being excessively reactive, impulsive or detached (narcissistic and aloof) is inappropriate to the practice of medicine, so too is being at the other extreme: overly controlled, resilient (lacking in emotion) or involved (overly empathic and confident). With regard to moral orientation, we suggest that being too liberal or too socially rule-bound is potentially problematic in terms of ethical practice and a balance between these two extremes is more appropriate (Bore et al. 2005).
Step 1에서 1000명에서 9.9%가 줄어 901명이 됨.
Applying Step 1 to our sample of 1000 applicants resulted in a reduction of 9.9% to 901.
Step 2에서 성격검사와 인지검사에 하한치를 적용하고 면접대상자를 정한다. 이 단계는 반복적으로 수행하여 하한치를 점차 높여가며 면접 대상자의 수 만큼만 남길 수 있다.
The light shaded area in Figure 2 results fromStep 2, which involves applying lowest score ‘cut points’ for the personality (Control, Resilience and Involvement) and cognitive test results to reach the number of applicants to be interviewed. This process can be done iteratively, raising and lowering the lowest score cut points, until only the number to be interviewed remain in the pool.
하한치를 변화하는 것은 각 변인간의 가중치를 바꾸는 것과 마찬가지이다. 가중치가 높은 변인은 더 높은 하한선을 가진다.
Changing the lowest score cut points on any of the the given variables changes weighting to the variable. Greater weight is given to a variable if its cut point is raised, and lesser weight is given if the cut point is lowered, relative to the remaining variables.
이러한 방법의 장점은 개별 의과대학마다 자신이 바라는 입학생의 혼합 구성비를 조정할 수 있다는 것이다.
This method has several advantages. It allows individual medical schools to determine the mix of qualities they want in their incoming students.
또 다른 장점은 하나의 시험에서 나타난 수행능력에만 의존하여 면접대상자를 선별하지 않는다는 점이다.
Another advantage is that proceeding to the interview pool does not ultimately rest on performance in one test (usually a cognitive skills test).
아마도 이 방법에서 가장 중요한 장점은 validity 근거가 더 확실한 검사에 더 가중치를 줄 수 있다는 점이다.
Perhaps, the most important advantage is that the method allows tests with a greater evidence of validity to have greater weighting in selection decision-making.
면접
Interviews
이 선발모델의 마지막 요소는 면접이다. 두 가지 이슈가 있다. 하나는 교수들의 부담스러워 하더라도 면접에 참여시킬 것이냐는 문제이고, 다른 하나는 어떤 것을 평가할 것인가에 대한 문제이다.
The final component of our selection model is the interview. There are two major issues in relation to medical school selection interviews. First, whether to interview at all given the costs and inconvenience to faculty staff; and second, what to assess in the interview.
첫 번째에 관한 우리의 관점은 어떤 선발모델도 지원자와 의과대학의 대표단(representative)사이의 접촉을 생략해서는 안된다는 것이다.
Our view regarding the first is that no selection model should omit an opportunity for personal contact between the applicant and a representative of the medical school.
면접이 갖는 문제는 '의사가 되고자 하는 동기'에 대해서 지원자들이 이미 충분한 연습이 되어있다는 점이다. 그들의 대답은 진실될수도 그렇지 않을 수도 있다. 따라서 대안적인 방법은 면접을 통해서 직접적으로 관찰가능한 것만 측정하는 것이다.
A problem with interviews is that they are sometimes used as a measure of ‘motivation to be a doctor’ with applicants usually having carefully rehearsed their answer to the question ‘Why do you want to be a doctor?’ Their answer might, or might not, reflect reality. An alternative, and preferable, approach is to use the interview to measure only what can be directly observed.
- interpersonal skills/communication,
- punctuality and presentation,
- decision-making (in response to presented scenarios) and
- behaviour under pressure (globally throughout the inter- view or in a specifically designed task),
An extension of the multiple mini interviews might be to have the applicants rated at all points of contact with school admission staff (including administrators as well as interviewers) over a series of different tasks.
각 의과대학은 자신만의 고유한 면접 절차를 개발할 기회가 있고 많은 의대가 그렇게 해왔다. 우리는 다양한 의과대학이 존재하여 학생들이 단일한 시스템만 대면하지 않도록 하는 것이 중요하다고 생각한다.
There is an opportunity for medical schools to develop their own interview procedures and many have done so. We see it as important that there is variety among medical schools so that candidates are not faced by a monolithic system.
또한 면접과 면접과제의 중요한 요소는 구조화되어야 하고, 객관적이어야 한다는 것이다. 이를 통해서 관찰가능한 행동만 평가해야 하며 모든 평가자와 면접관이 그 과정에 훈련되어 있어야 한다.
The important point is that the interviews and tasks need to be structured, objective in that only observable behaviour is rated and all interviewers/raters should have been trained in the procedure.
Brown KG, Le H, Schmidt FL. 2006. Specific aptitude theory revisited: Is there incremental validity for training performance? Int J Select Assess 14:87–100
A comprehensive model for the selection of medical students.
Author information
- 1School of Psychology, University of Newcastle, Callaghan, NSW, Australia. Miles.Bore@newcastle.edu.au
Abstract
BACKGROUND:
AIM:
METHODS:
RESULTS:
CONCLUSION:
- PMID:
- 19995169
- [PubMed - indexed for MEDLINE]
'Articles (Medical Education) > 입학, 선발(Admission and Selection)' 카테고리의 다른 글
의과대학생 선발의 신뢰성에 평가자와 상황이 미치는 영향(Med Educ, 2009) (0) | 2015.11.20 |
---|---|
배제의 과정에서 포용을 추구하는 법: 의과대학 학생선발(Med Educ, 2015) (0) | 2015.11.20 |
의과대학 입학에 있어서 인적특성 평가(Med Educ, 2005) (0) | 2015.11.02 |
의과대학 입학생의 성공에 필요한 핵심 인성역량 : 무엇이고, 어떻게 입학단계에서 평가할 수 있는가?(Acad Med, 2013) (0) | 2015.10.23 |
지난 25년간의 의과대학 입학과 관련된 연구와 실제(Teach Learn Med, 2013) ● (0) | 2015.10.22 |