유럽의사면허시험 또는 국가의사면허시험에 대한 다섯 가지 신화(Med Teach, 2009)

유럽의사면허시험 또는 국가의사면허시험에 대한 다섯 가지 신화(Med Teach, 2009)

Five myths and the case against a European or national licensing examination

RONALD M. HARDEN

University of Dundee, UK

유럽의사면허시험은 매혹적인 것이지만, 부작용은 충분히 논의되지 않았다.

The case for a European or national examination is a seductive one,

What have not been discussed sufficiently, however, are the associated disadvantages or side effects –

신화 1: 유럽의사시험 혹은 의사국가시험은 의료행위에 있어서 중요한 영역을 평가하게 해줄 것이다.

Myth 1: A European or national examination will ensure that the candidate is assessed in important areas of medical practice

아니다! 중앙화된 시험은 가장 중요한 학습성과 대신 국가 혹은 유럽 수준에서 쉽게 평가가능한 것을 강조할 것이다. 시험의 초점은 written exam에 맞춰질 것이고, MCQ가 주를 이룰 것이며, 임상술기나 태도보다는 지식을 평가하게 될 것이다. Schuwirth가 주장한 것처럼 '국가시험은 '낮은' 레벨의 스킬에 초점을 두게 되나, 최근의 추세는 프로페셔널리즘, 스칼라십, 과학적 사고 등의 '높은' 수준의 스킬을 강조한다'. 역량평가프로그램으로 이행함에 따라서 우리는 전통적인 형식과 새로운 형식의 평가를 모두 사용해야 하며, psychometric과 edumetric 모두에서 유도derive된 질평가준거를 활용해야 한다. 이러한 조합은 중앙화된 시험에서는 갖추기 어렵다.

No! A centralized examination will emphasize not necessarily the most important learning outcomes but instead those which can be readily assessed at a national or European level. The focus will be on written examinations, often with multiple choice questions, where knowledge rather than clinical skills or attitudes are assessed. As Schuwirth (2007, p. 1023) argued ‘National tests typically focus on ‘‘low’’ level skills, whereas currently there is growing interest in higher order skills and competences, such as ‘‘professionalism’’, ‘‘scholarship’’, the ability to think as a ‘‘scientist’’, etc.’. With the move to competence assessment programmes, we need a combination of different assessment methods including both traditional and new forms of assessment with the criteria used to evaluate their quality derived from both psychometrics and edumetrics (Baartman et al. 2007). This combination is unlikely to be found with a central examination.

문제는, 과거에는 우리가 측정에 대해 논쟁을 할 때 '신뢰도'를 강조했다. 우리는 이제는 'systemic validity'의 개념을 강조하는 데에 이르렀으며, 이것은 교육시스템, 교육과정, 교육변화instructional change에 대한 것이다.

A problem is that in the past we have emphasized in the measurement debate, the concept of reliability. We need now to focus more on Fredericksen and Collins’ (1989) concept of ‘systemic validity’ which they define in terms of inducing in the education system, curricular and instructional changes.

중앙화된 시험은 덜 중요한 학습성과를 평가하게 되면서, 교사의 교육과 학생의 학습 패턴에 영향을 주어 뒤틀리게 한다. soft skill을 강조하는 방식으로 변화하는 현재의 교육과정은 지식의 recall과 사실의 manipulation을 다루는 시험에서는 축소undercut될 것이다.

A centralized examination, assessing as it does less important learning outcomes, is likely to influence and distort a teacher’s teaching and the students’ study patterns. Current moves to increase the emphasis in the curriculum on ‘soft skills’ such as the ability to generate fresh and original ideas, to work in teams, and to be empathetic are undercut when assessment regimes largely test the recall and manipulation of facts (Cole 2007).

신화 2: 유럽의사시험 혹은 의사국가시험은 평가행위assessment practice의 향상을 가져올 것이다.

Myth 2: A European or national examination is likely to encourage change and lead to improvements in assessment practice

아니다! 현재의 근거를 보면, 중앙화된 시험이 있는 시스템에서는 평가의 혁신은 저해된다.

No! The evidence is that where there is a system of centralized examinations, innovation in assessment is hindered.

NBME는 아마도 중앙화된 평가전문가들이 모인 가장 좋은 예시일 것이며, 전 세계적으로 평가행위의 발전에 기여해왔고, 그 분야의 이해에 기여하였다. 그러나 그 조직의 우선순위상 필요한 것을 감안하면, 그러한 조직은 평가practice의 최신의 지식을 활용하거나 그 분야의 새로운 이니셔티브를 가져오는 것에는 적합하지 않다. Peter Scoles는 NBME가 미국에서 Gateway Examination의 지위를 갖게 되면서, USMLE를 변화시키기 위해서는 6~10년의 시간이 필요하다고 강조하였다. 의과대학의 졸업시험으로 OSCE가 처음 도입된 것은 30년 전이었지만, 다수의 의과대학에서 이미 그것을 도입한지 수 년이 지나서야 중앙화된 시험에 도입되기 시작하였다.

The NMBE is probably the best example of a central team of assessment experts and it has made significant contributions internation- ally to advances in the practice of assessment and to our understanding of the field. Because, by necessity its priorities have to be different, however, such a body is not well placed to work at the cutting edge of assessment practice and to lead new initiatives in the field. Peter Scoles highlighted, at a session on the NBME position on the Gateway Examination in the USA, that 6–10 years was required to change the process for the USMLE. It was 30 years after the objective structured examination (OSCE) was first introduced in the final examina- tion at the medical school in Dundee (Harden et al. 1975) and many years after it was introduced by a number of other schools that it was adopted as an approach to be used in a centrally organized examination.

중앙화된 시험에 대해서는 근본적인 우려가 있는데, 그 특성상 평가에 대한 낡은outmoded 관점을 가질 수 밖에 없고, 학생들로 하여금 평가란 교육과정이나 교육 프로그램에서 분리되어 어느 한 시점에서 가장 잘 이뤄질 수 있는 것이라는 오해를 가지게 만든다는 것이다. Schuwirth가 '역량의 관점에서, 이러한 시험(한 차례로 끝나는 시험)은 평가와 교육의 misalignment를 가져온다'

There is a fundamental concern about a central examina- tion and its impact on progress with assessment practice. A central examination by its very nature perpetuates an outmoded view of assessment and encourages the misconcep- tion that students are best evaluated at one point in time and that assessment is something separate from the curriculum and the teaching programme. As Schuwirth (2007, p. 1022) put it ‘in terms of competences, this (single-shot assessment) constitutes a misalignment between assessment and educa- tion’.

미래를 볼 때 Schuwirth and Van der Vleuten 는 '평가는 교육 프로세스의 결과에 대한 외부적 측정이 아니라, 교육 프로세스의 핵심적 부분으로 인식될 것이다'라고 하였다. 이들은 또한 '평가는 psychometric measurement problem으로만 인식되지 않을 것이며, 그보다는 교육설계의 문제educational design problem로 인식될 것이다'라고 했다. 이는 평가의 목적은 단순히 한 지원자가 기준을 충족시켰는지를 보는 것이 아니라, 한 지원자의 역량에 대한 정보를 교육과 교육과정을 개개인의 요구에 따라 tailor하는데 사용되게끔 하는 것으로 봐야 함을 시사한다. 그리고 이것은 평가가 교육과정에 통합될 때 (교육이 끝날 때 single-shot으로 하는 것이 아니라) 가능할 것이다.

Looking to the future, Schuwirth and Van der Vleuten (2006, p. 18) suggest that ‘Assessment will be less viewed as an external measurement of the results of the educational process but more as an integral part of the process’. They argue that ‘Assessment is no longer seen exclusively as a psychometric measurement problem, but more as an educational design problem. This implies that the purpose of assessment is not merely to determine whether a candidate is up to standard, but more how the information about the candidate’s competence can best be used to tailor the teaching or the courses to individual needs’. This is most likely to be achieved when assessment is integrated into the curriculum and not a single- shot end of course in national or international test.

신화 3: 스탠다드와 국제화의 시대에, 유럽의사시험 혹은 의사국가시험은 uniformity를 향상시킴으로서 우리의 요구를 충족시킬 것이다.

Myth 3: In an age of standards and globalization, a European or national examination will meet our needs by encouraging uniformity

아니다! 의심할 여지 없이, 모든 학교에 해당되는 핵심 역량이 있고, 공통의 평가가 적절한 부분이 있지만, 중요하고 유의한 차이 또한 존재한다.

No! Undoubtedly there are core competencies that are likely to be common to all schools and where a common assessment might be appropriate, but there are also important and significant differences. 

실제로 UK에서는 GMC가 'promoting equality and diversity'를 질관리프로그램의 한 목표로 지정하고 있다. 이 문제는, Darney가 정의한 것으로서, 국제적 donor 국가에 의해서 주도되는 시험의 활용이 인지적 성취를 글로벌하게 강조하는 결과를 낳으면서, 좋은 교육은 무엇인가에 대한 지역의 가치는 무시하는 결과를 낳는 것이다.

Indeed in the UK, the General Medical Council has been ‘promoting equality and diversity’ among the aims of its Quality Assurance Programme. The problem, as defined by Carney (2003), is that the use of tests promoted by international donor nations has resulted in a global emphasis on cognitive achievement without a con- sideration of what local values might define as good education.

신화 4: 의사국가시험은 시스템의 퍼포먼스를 추적할 수 있는 지표이다.

Myth 4: A national examination is of value as an indicator that helps to track the performance of the system

Smith는 ‘tunnel vision, sub-optimization, myopia, measure fixation, gaming, ossification, misinterpretation and misrepresentation’이라는 문제를 언급했다.

Smith (1995) identified a ‘huge number of instances of unintended behavioural consequences of the publication of performance data’ and described problems such as ‘tunnel vision, sub-optimization, myopia, measure fixation, gaming, ossification, misinterpretation and misrepresentation’.

Fitz-Gibbon and Tymms 는 지표가 복잡한 상황을 제대로 반영하지 못할 수 있으며, 각 기관이 지표에 초점을 맞추려는 노력에 대한 게이밍이 발생하여 그 시스템이 훼손될 가능성도 있다고 지적했다.

Fitz-Gibbon and Tymms (2002) argue that an indicator may not reflect the complexity of the situation and that the system will be undermined as gaming takes hold with the efforts of the institution focussed on the indicator (student performance in the national examination, for example) at the expense of other issues.

Hecker and Violato의 연구에 따르면, 의사국가시험의 퍼포먼스를 교육과정평가의 지표로 사용할 때 문제가 발생할 수 있다. USMLE 점수에 기반하여 이들은 '의학교육개혁의 변화는 학생의 성취에 별다른 영향을 주지 못한다'는 결론을 도출하였다. 그러나 이들은 자신들의 연구에서 종속변수가 면허시험 점수이며, 의사의 여러 중요한 특성인 공감/문제해결/임상추론 등을 평가하지 못한다는 한계를 인지하고 있었다. 

The study of Hecker and Violato (2007, p. 112) demon- strates the problems that can arise when performance in a national examination is used as an indicator for curriculum evaluation. Based on USMLE scores, they made the sweeping conclusion that ‘changing curricula in medical education reform is not likely to have much impact on improvement in student achievement’. They did recognize, however, that in their study the dependent variables are licensing exam scores, which may not fully access important physician characteristics such as empathy, problem solving, clinical reasoning and so on, that are learning objectives of some medical schools.

표준화시험과 고정된 교육과정의 문제는 가장 중요하게 다뤄져야 할 유형의 교육을 죽인다는 데 있다. 그러나 학생이 고득점으로 시험을 통과하게 도와주려는 선생의 노력을 누가 비난할 수 있겠는가? 학생들이 표준화 시험에서 잘 하도록 가르치려는 유혹은 그러한 시험이 전체 학교 시스템을 평가하는 자료로 사용될 때 거의 피할 수 없는 상황이다.

This problem with standardized tests was highlighted in ‘Engines for Education’ (Engines for Education 2008). ‘The problem with standardized tests and the fixed curricula they engender is their tendency to kill off the kind of education that matters most. But who can blame a teacher or school for orienting the lesson towards helping students pass those tests with high marks? The temptation to teach students to do well on standardized tests is almost unavoidable when performance on such tests is how entire school systems are evaluated.’

신화 5: 유럽의사시험 혹은 의사국가시험은 더 안전한 의료행위를 하게 해서 공공을 보호할 것이다.

Myth 5: A European or national examination will lead to safer medical practice and protect the public from substandard practitioners

아니다! 중앙화된 시험이 존재하지 않는다고 해서 수준미달의 의사를 양성한다는 근거는 없다. 또한 그러한 시험을 도입한다고 향상이 있다는 근거 또한 없다. 의사국가시험이 없는 UK의 의과대학 졸업생이 진료역량 측면에서 의사국가시험이 있는 미국의 의사보다 열등하다는 근거는 없다.

No! There is no evidence that the absence of a centralized examination leads to the graduation of substandard doctors (Noble 2008) nor that the introduction of such an examination will lead to an improvement in the practice of medicine. There is no evidence that the graduate of a UK school where there is no national examination is in some way inferior in terms of their competence to practice medicine than the graduate of an American school who has passed the relevant national examinations.

Papadakis 는 진료중인 의사에 대해 medical board가 시행하는 displinary action이 (의사면국가시험에서 측정된 퍼포먼스보다) 비전문직업적 행위와 더 강력한 연관이 있음을 보여준 바 있다. 

Papadakis et al. (2005) found that disciplinary action taken against practicing doctors by medical boards in the USA was more strongly associated with records of unprofessional behaviour in medical school, than with their performance as measured by a national examination.

결론

Conclusions

전체적인 이득에 대한 근거가 없음을 고려하면, 왜 상당한 양의 자원이 유럽의사시험 혹은 의사국가시험에 투입되어야 하는가를 물어야 할 것이다. 이는 연속체에 따라 구분해볼 수 있다.

Given the lack of evidence as to the overall benefits, one has to ask whether the significant resources required to implement a European or national examination might be used in other ways to improve the assessment practice. This can be considered as a continuum with stages in the continuum: 

(1) Assessment procedures in the medical school are completely independent with no external standards. 

(2) The school’s assessment procedures are subject to external inspection and approval in line with specified standards (such as those offered by the WFME).

 (3) Examinations in a school are monitored by external examiners including student pass/fail decisions (the current practice in the UK, with the process quality assured by the General Medical Council). 

(4) Assessment instruments and some questions are shared between schools and common questions are intro- duced in examinations. 

(5) Students are required to take and pass both an internal school examination and an external European or national examination.

(6) Only a single European or national examination is taken by the student.

Noble ISG. 2008. Are national qualifying examinations a fair way to rank medical students? No. Br Med J 337:1a279.

---

Med Teach. 2009 Mar;31(3):217-20.

Five myths and the case against a European or national licensing examination.

Harden RM1.

Author information

• 1University of Dundee, UK. r.m.harden@dundee.ac.uk

Abstract

The introduction of a European licensing examination or national examinations, where these do not already exist, offers significant advantages. These are more than offset, however, by the disadvantages and the collateral damage incurred. Five myths about centralizing examinations are explored. Myth 1: The claim that a central examination will ensure that candidates are assessed in important areas of medical practice is unfounded. What tends to be assessed are learning outcomes that can be easily assessed. These are often not the important outcomes related to the overall competence of a doctor. Myth 2: It is claimed that a central examination will lead to improvements in assessment practice. The evidence is that this is not the case and that, in fact, a central examination stifles change and inhibits innovation. Myth 3: A central examination, it is suggested, will meet a need for greater uniformity. There is also an important need to recognize diversity. Myth 4: Central examinations are seen as an indicator that will track the performance of the system. The limitations of the data, however, are usually not recognized and there maybe unfortunate and unintended consequences if the results are used in this way. Myth 5: Finally, a major argument proposed for a Europeanor national examination is that it will lead to safer medical practice and that this will protect the patient from substandard practitioners. There is, in fact, no evidence to support this argument. There is a need for further work and new initiatives on standards and quality improvement in assessment approaches. This can be achieved in a number of ways including monitoring the assessment process and sharing tools and assessment approaches between schools.

PMID:

 

19811119

[PubMed - indexed for MEDLINE]