보건의료전문직 선발시 평가방법 : 오타와 2010컨퍼런스의 컨센서스와 제언 (2)

보건의료전문직 선발시 평가방법 : 오타와 2010컨퍼런스에서의 컨센서스와 제언

Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010 Conference

DAVID PRIDEAUX1, CHRIS ROBERTS2, KEVIN EVA3, ANGEL CENTENO4, PETER MCCRORIE5,

CHRIS MCMANUS6, FIONA PATTERSON7, DAVID POWIS8, ARA TEKIAN9 & DAVID WILKINSON10

의학 및 보건의료 전문직에 있어 선발을 위한 평가는 선발 후 교육과정에서 이루어지는 평과와 동일한 수준의 질이 확보되어야 한다. 선발에 대한 논문들은 확고한 이론이나 개념에 의해 뒷받침되지 못하고 제한적이다. 

우선 필기시험에 대해서는, MCAT이 의과대학과 면허시험에서의 수행능력에 대해 예측타당도를 갖는다는 근거들이 있다.

GPA의 예측타당도에 대한 근거도 있는데, 특히 MCAT과 결합해서 사용했을 때 예측타당도가 높다는 것이 (북미 North America의) 의학전문대학원(graduate entry) 체제에서 많은 연구가 되어 있다. 반면 호주/영국 등에서의 졸업시학점(school leaver score)의 예측타당도에 대한 근거는 적다.

여러 연구에서 MMI가 좋은 예측타당도와 신뢰도를 갖는다는 결과가 보고되어 있다. 선발에 사용되는 여러 방법들 중 인성(personality)시험에 대해서만이 관심이 높아지고 있으며, 미래에도 많은 연구가 될 것으로 생각된다. 

의과대학과 보건의료직에 대한 문호를 넓히는(widening access) 문제는 보건의료전문직의 사회적 책무성 문제와 연관되어 점차 관심이 높아지고 있다. 전통적인 선발 방식은 많은 인구집단을 배제하게 되는 특징이 있는데, 그렇다고 해서 새로이 등장하는 비전통적인 방법이 여러 집단에게 '문호를 넓힌다'는 근거가 있는 것도 아니다. 사전준비(preperation)프로그램과 지역사회지원(outreach)프로그램 정도가 효과가 있을 것으로 기대된다.

요약하자면 선발시 평가에 대한 지금까지의 컨센서스 영역은 넓지 않다. 좋은 평가원칙을 적용하여 교과과정과 잘 맞는 방법을 적용하고, 다양한 방식을 활용한 프로그램적 접근법(programmatic approach)를 사용하며, 다학문적 관점에서 정교한 측정모델을 활용해야 한다. 사회적 책무성을 다하기 위해서는 다양한 계층을 포용하여 노동력(배치) 문제, 문호를 넓히는 문제 등이 선발 원칙에 적용되어야 할 것이다.

---

The current situation: Widening access

지방 출신 학생들이 더 지방에서 자리를 잡는다는 것은 잘 알려져 있다.

It is acknowledged that rural students are more likely to practise in rural locations after graduation. A common approach has been to institute quotas for such groups

Consensus and conclusion

Regehr가 제시한 programmatic research로 돌아가볼 필요가 있다. van der Wleuten과 Schuwirth는 'programmatic assessment'를 이야기한다.

It may be useful to take a step backwards from the pursuit of unifying theory to consider Regehr’s (2004) concept of programmatic research where ‘communities’ of researchers work together towards an eventual goal of consensus

van der Vleuten and Schuwirth (2005) have argued that thinking about assessment should be moved from a consideration of methods to programmes; another use of the term ‘programmatic’ this time in programmatic assessment. Programmatic assessment concentrates on the overall programme of assessment with a combination of methods, each with their differing psychometric properties, to make decisions about student performance.

Recommendations

(1) Admissions committees and all who have an interest in selection processes should adopt the principles of good assessment in defining the purpose of selection; 

blueprinting of assessable domains and attributes, 

selecting appropriate formats, 

employing transparent standard setting and decision making, 

and including an evaluation cycle in a programmatic manner.

(2) An integrative approach should apply the principles of good assessment and curriculum alignment along the education and training pathway including the progression hurdles between health professional degrees, prevocational practice and basic and advanced speciality training.

(3) There should be a focus on multi-method programmatic approaches in collecting, analysing, interpreting and reporting data from a range of selection instruments, which are fit for purpose

(4) There needs to be an emphasis on developing interdisciplinary theoretical frameworks that underpin development of both policy and the research agenda.

(5) There is an urgent need for the development of sophisticated measurement models from the family of regression methods which will require application to multi-site high-quality data sets, for increasing the sophistication of predictive validity studies using a range of attributes from selection blueprints, and for a focus on test–retest reliability.

(6) The social accountability of universities demands that social inclusion, workforce issues, consumer choice and widening of access to students of promise are embedded in the principles of good assessment for selection with recognition that there are political (and non-universal) issues that need to be considered in the definition of optimal decisions.

(7) Outreach, targeting strategies, preparation programmes and conditional selection should be considered as core strategies for medical and health professional schools to achieve their widening access missions.

---

Med Teach. 2011;33(3):215-23. doi: 10.3109/0142159X.2011.551560.

Assessment for selection for the health care professions and specialty training: consensus statement andrecommendations from the Ottawa 2010 Conference.

Prideaux D, Roberts C, Eva K, Centeno A, McCrorie P, McManus C, Patterson F, Powis D, Tekian A, Wilkinson D.

Source

Medical Education, Flinders University, GPO Box 2100, Adelaide, South Australia 5064, Australia. david.prideaux@flinders.edu.au

Abstract

Assessment for selection in medicine and the health professions should follow the same quality assurance processes as in-course assessment. The literature on selection is limited and is not strongly theoretical or conceptual. For written testing, there is evidence of the predictive validity of Medical College Admission Test (MCAT) for medical school and licensing examination performance. There is also evidence for the predictive validity of grade point average, particularly in combination with MCAT for graduate entry but little evidence about the predictive validity of school leaver scores. Interviews have not been shown to be robust selection measures. Studies of multiple mini-interviews have indicated good predictive validity and reliability. Of other measures used in selection, only the growing interest in personality testing appears to warrant future work. Widening access to medical and health professional programmes is an increasing priority and relates to the social accountability mandate of medical and healthprofessional schools. While traditional selection measures do discriminate against various population groups, there is little evidence on the effect of non-traditional measures in widening access. Preparation and outreach programmes show most promise. In summary, the areas of consensus forassessment for selection are small in number. Recommendations for future action focus on the adoption of principles of good assessment and curriculum alignment, use of multi-method programmatic approaches, development of interdisciplinary frameworks and utilisation of sophisticated measurement models. The social accountability mandate of medical and health professional schools demands that social inclusion, workforce issues and widening of access are embedded in the principles of good assessment for selection.