미래 의학교육을 위한 다섯 가지 제언

1연세대학교 의과대학 의학교육학과, 2가톨릭대학교 의과대학 예방의학교실

양은배1, 맹광호2

우리나라 의학교육의 변화 특징들

우리나라가 현대의학을 체계적으로 교육하기 시작한 것은 1886년 3월 29일 제중원 의학교가 개설되면서부터이다. 이로부터 2014년 현재까지 128년 동안 우리나라 의학교육은 많은 발전을 하였다. 의학교육이 현재의 모습으로 자리 잡기까지는 많은 제도적 변화와 교육적 실험들이 있었으며, 의학교육의 내실화와 수월성을 추구하기 위한 많은 노력들이 전개되었다. 이러한 과정에서 나타난 우리나라 의학교육의 특징은 다음과 같다.

1. 전문학교 교육으로서의 의학교육

20세기 초 우리나라 의학교육은 일제 식민지 교육정책과 밀접하게 연결된 ‘전문학교 교육’이었다. 조선총독부의 조선교육령은 한국에서의 교육을 보통교육, 실업교육 및 전문교육으로 한정했으며, 차별적인 학제를 도입하여 조선인과 일본인을 별도로 교육하였다. 학교의 명칭, 수업연한과 내용을 차별화 하였다. 전문학교는 실업학교 이상의 고급 기술을 교육하기 위해 근대 일본과 일제 강점기 당시 조선, 대만과 만주국에 있었던 학교 형태였다. 전문학교에서는 대학 수준의 고등교육을 제공하였으나 학술적 내용이 아닌 전문적 기예나 지식을 교육하는 것에 국한되었다. 당시 의학교육을 실시한 전문학교는 조선총독부 전문학교 인가 연도 기준으로 조선총독부의원부설강습소(1913, 이 강습소는 1916년 경성의학전문학교로 변경), 세브란스의학전문학교(1917), 대구의학전문학교(1933), 평양의학전문학교(1933), 경성여자의학전문학교(1938), 광주의학전문학교(1944) 등이다. 이러한 전문학교의 수업연한은 3년으로 제한되었으나 얼마 지나지 않아 4년제로 승격되었다. 1932년에 예과 2년, 본과 4년의 6년제 학제를 갖춘 경성제국대학 의학부가 설립되었는데, 이 대학은 일본인 중심의 의학교육 기관이었다.

2. 고등교육으로서의 의학교육

1945년 해방 이후 3년 동안은 미군정이 실시된 기간이다. 이 시기에는 교육 관련 자문 및 심의 역할을 담당한 미 군정청 학무국의 ‘조선교육심의회’가 우리나라 교육 전반에 대한 문제를 다루었다. 조선교육심의회는 6-3-3-4 (의과대학 6년)의 단선형 학제를 채택하였다. 이에 따라 의학교육은 의학 기능인 양성에서 학술적 연구를 포함한 고등교육의 일환으로 승격되었다. 경성대학교 의학부와 경성의학전문학교가 폐지되고, 국립서울대학교 의과대학이 설립되었고, 세브란스의학교(지금의 연세대학교 의과대학), 경성여자의학전문학교(지금의 고려대학교 의과대학), 대구의학전문학교(지금의 경북대학교 의과대학), 광주의학전문학교(지금의 전남대학교 의과대학) 등이 6년제 의과대학으로 개편되었으며, 이화여자대학교에 의과대학이 신설되면서 1950년대 초까지 6개의 의과대학이 있었다.

3. 의학교육 기관의 양적인 성장

1950년대 이후 의과대학이 급격하게 신설되기 시작하였다. 

• 의과대학은 1950년대 초반에 가톨릭의대와 부산의대가 설립되어 8개교로 늘어났고 
• 1960년대에 경희의대, 전북의대, 조선의대, 충남의대 및 한양의대의 5개교가 신설되어 의과대학의 수는 13개가 되었다. 
• 1970년대에도 1971년의 중앙의대를 시작으로 계명의대, 순천향의대, 연세원주의대, 영남의대 및 인제의대 등의 6개교가 신설되었다. 
• 1980년대에는 모두 12개의 의과대학(건국의대, 경상의대, 고신의대, 단국의대, 동국의대, 동아의대, 원광의대, 아주의대, 울산의대, 인하의대, 충북의대, 한림의대)이 신설되어 총 31개의 의과대학이 되었다. 
• 1990년대에 들어서도 의과대학의 신설은 계속되었는데 1995년까지 대구가톨릭의대, 강원의대, 건양의대, 관동의대, 서남의대 등 5개교가 신설되었고 
• 1990년대 후반에도 제주의대, 을지의대, 성균관의대, 포천중문의대, 가천의대 등 5개교가 신설되어 의과대학의 수는 모두 41개교가 되었다. 이들 대학의 연간 입학정원은 3,300명에 달하게 되었다. 
• 2000년 이후에도 의사인력 공급확대에 대한 요구와 지역사회의 의학교육기관 설립 의지가 결합되면서 지속적으로 의학교육 기관 신설에 대한 요구가 계속되고 있다.

4. 의학교육 학제의 이원화

2000년대 의학교육의 중요한 변화 중의 하나는 학사 후 의학교육제도라고 불리는 ‘의학전문대학원제도’의 도입이다. 1990년대 이후 정부는 우리나라 대학원 교육과 관련한 고등교육의 기본 방향을 일반대학원, 특수대학원 및 전문대학원으로 구분하고, 의학, 법학, 경영학 등의 분야를 미국과 같은 개념의 전문대학원 체제로 개편하는 정책을 추진하였다[1]. 의학전문대학원 제도는 2004년 5개 의과대학이 의학전문대학원 전환을 결정하면서 도입이 확정되었다. 2006년 1월에는 교육인적자원부가 의학전문대학원 전환추진 기본원칙을 발표함에 따라 27개교가 전면 혹은 부분적으로 의학전문대학원 전환을 결정함으로써 이 제도는 우리나라 의사 양성의 중요한 기구가 되었다. 의학교육 학제 변화와 관련하여 당시 한국의학교육학회, 한국의과대학장협의회 및 한국의학교육협의회 등에서는 의학교육 학제 이원화에 대해서 우려를 표명하였으며, 우리나라 현실에서 의학전문대학원제도가 적절하지 않음을 지속적으로 지적하기도 하였다. 의학전문대학원제도에 대한 의학교육계의 비판에 따라 정부는 2010년 7월 의학교육 학제를 대학 자율로 선택하도록 하는 ‘의학교육제도 개선 계획’을 발표하였다. 의사 양성 학제의 선택이 대학의 자율로 주어지게 되자 의과대학 체제를 선호하는 현상이 두드러지게 나타났으며, 2015학년도부터는 단계적으로 의학전문대학원제도가 폐지되어 5개 의학전문대학원만 남게 될 전망이다.

5. 의과대학 교육과정 선진화

세계 각국의 많은 의과대학은 의료 및 사회 환경의 변화에 따라 교육과정을 개선하기 위한 노력을 하고 있다. 교육과정은 무엇을 가르치고, 그것을 왜 가르치며, 그것을 어떻게 가르칠 것인지, 가르친 것들을 어떻게 평가할 것인가에 대한 교육의 청사진이다. 교육과정 개발자들과 의학교육자들은 끊임없이 사회의 요구를 재해석하고, 교수학습 이론에 근거하여 교육과정의 새로운 청사진을 만들고 있다. 우리나라도 1970년대 이후 의학 교육과정을 세계적인 교육 패러다임에 부응하는 방향으로 선진화하기 위한 노력을 지속적으로 전개해 왔다. 우리나라 의학교육과정 개발의 주요 특징을 살펴보면 다음과 같다. 

• 첫째, 전통적인 교과목중심 교육과정(discipline-based curriculum)을 탈피하고 통합교육과정(integrated curriculum) 편성이 확산되기 시작하였다. 통합교육과정은 교과목을 학문단위보다는 계통이나 장기중심으로 편성하거나 문제바탕학습(problem-based learning)의 형태로 나타났다. 이러한 교육과정 조직은 기초의학과 임상의학이라는 전통적인 학문단위 구분을 지양하고, 기초의학과 임상의학의 통합적 이해, 실제 환자 진료를 위해 요구되는 지식의 적용에 대한 교육을 강조하였다. 전국 41개 의과대학의 대부분 교육과정이 어떤 형태로든 통합교육을 지향하고 있다. 
• 둘째, 교육과정 운영 관점에서 교수중심의 지식전달에서 학생중심의 자기주도학습(self-directed learning)과 경험적 실천을 강조하는 방향으로 교육과정이 개편되었다. 지식의 일방적인 주입보다는 강의시간을 축소하고, 학생들의 적성과 선호에 따른 선택과목들을 개설하기 시작하였다. 이것은 학생들이 더 이상 지식의 단순한 수용자가 아니라 스스로 지식을 구성해가는 주체로 인식되고 있다는 의미이다. 
• 셋째, 2000년 이후 의학교육과정 개발의 새로운 방향은 성과 또는 역량중심 교육과정(outcome- or competency-based curriculum) 개발이다. 성과중심 교육과정 편성과 졸업생들이 갖추어야 하는 역량교육에 대한 체계적인 접근은 2012년 (재)한국의학교육평가원의 ‘Post 2주기 의과대학 인증평가 기준’이 발표되면서부터이다. 의학교육에서 역량기반교육은 성공적인 졸업생의 수행요목과 수준을 먼저 결정하고 이를 달성하기 위한 요소들을 학습경험으로 선정한다는 점에서 기존의 계통중심이나 과목중심의 교육적 접근과는 다르다. 역량기반교육은 지식을 습득하는 것에 머무르지 않고, 학습자가 습득한 지식을 활용하여 어떤 수행을 할 수 있는가에 관심을 갖는다. 즉, 역량기반 교육과정은 의사로서 필요한 지식뿐 아니라, 의사로서의 업무를 성공적으로 수행할 수 있는 전문적 능력 자체를 교육할 것을 강조한다. 이제 성과 또는 역량중심 교육이 의학교육의 내실화와 수월성을 지향하는 방향으로 올바르게 정착될 수 있도록 하기 위한 노력이 필요하다.

6. 인문사회의학 교육 강화

2000년 이후 의학교육의 주요한 변화중의 하나는 의과대학교육과정에 인문사회의학 관련 교육이 증가한 것이다. 이 같은 현상은 일차적으로 의과대학들이 자연과학적 의학지식과 기술교육만을 강조해 온 것을 반성하고 인성교육을 강화해야 한다는 인식의 전환을 의미한다. 의과대학 교육에 인문사회의학 교육을 강조하는 다양한 노력들이 전개되어 왔다. 예를 들어, 한국의학교육학회와 의과대학학장협의회가 번갈아 가며 주관해 온 의학교육학술대회에서 관련 교육의 중요성을 주제로 발표와 토론을 해온 일이라든지, 1997년에 발족해서 2000년부터 본격적인 활동을 시작한 한국의학교육평가원의 의과대학 인증평가 사업이나 1999년에 한국의과대학학장협의회가 발간한 ‘21세기 한국의학교육계획’ 등에서 의료윤리를 포함한 인문사회의학 교육의 필요성이 크게 강조된 것이다. 지금은 41개 의과대학 모두가 어떤 형태로든지 의료인문학 관련 교육을 실시하고 있는 상태다. 인문사회의학적 소양은 21세기 의사들에게 있어서 필수적인 덕목인 것이다. 그러나 인문사회의학 교육의 강화가 의사들이 인문학이나 사회학에 대학 지식을 갖고 있어야 한다는 의미는 아니다. 그것은 의사들이 자신을 스스로 성찰하고, 스스로 규칙을 세우고 지키며, 사회의 변화와 요구에 민감하게 하며, 사회의 모든 구성원과 함께 호흡하도록 만드는 것이다. 일부 대학에서는 인문사회의학 또는 의료인문학을 전담하는 부서의 신설이나 교원의 채용과 같은 변화가 일어나고 있다.

7. 교수방법의 효과성과 평가방법의 타당성 제고

지난 반세기 이상 동안 지속적으로 의학교육의 관심을 받아온 분야는 교수방법의 효과성과 평가방법의 타당성 확보였다. 어떻게 하면 학습효과가 높은 교수학습 방법을 개발하여 적용할 수 있을지, 학생들의 수행능력을 평가하는 타당한 방법에는 어떤 것들이 있는지는 많은 의학교육자들의 관심이었다. 

• 첫 번째 변화는 1990년대 후반 캐나다 McMaster대학에서 시작한 문제바탕학습이 의학교육의 교수방법으로 우리나라에 도입되기 시작하였다는 점이다. 2000년대 의과대학 인증평가 기준에 다양한 교수방법의 하나로 문제바탕학습이 소개되면서 이 교수방법은 41개 대학에 급속도로 전파되었다. 일부 대학의 경우에는 교육과정 전체를 문제바탕학습 형태로 운영하였다. 
• 둘째는 Oklahoma대학의 Michaelsen교수가 개발한 팀바탕학습(team-based learning)이 일부 의과대학에 소개되기 시작하였다. 팀바탕학습은 문제바탕학습과 달리 대형 집단을 대상으로 한다는 점에서 실용성을 가진 것으로 평가되고 있다. 그러나 팀티칭의 형태로 통합중심 교육과정을 운영하고 있는 교육과정에서는 팀바탕학습을 설계하기 어려운 점이 있다는 비판이 있다. 이에 따라 몇 몇 대학에서 팀바탕학습을 시범적으로 도입하고 있는 것이 현재의 실정이다. 
• 세 번째 변화는 객관구조화진료시험(objective structured clinical examination)과 진료수행평가(clinical performance examination)의 도입이다. 많은 대학들이 병원의 임상실습환경과는 별도로 임상술기교육센터와 같은 교육장을 별도로 설치하여 다양한 모형, 표준화 환자 등을 통한 교육과 평가를 하고 있다. 이러한 변화는 2009년 의사면허국가실기시험이 도입되면서 더욱 주목 받기 시작하였는데, 현재는 대표적인 교육 및 평가방법으로 정착하고 있다. 
• 마지막으로 의학교육에서 e-learning의 확산을 지적할 수 있다. 의학교육에서 테크놀로지를 활용하는 사례들이 많이 늘어나고 있지만, 강의를 촬영한 동영상을 제공하거나, 임상술기 동영상을 제공하는 수준에 머물고 있는 것이 현실이다. 일부 대학들을 중심으로 Edx, Coursera, Udacity 등 새로운 형태의 온라인 플랫폼 개발과 콘텐츠 공유 움직임이 일어나고 있다.

8. 의사국가시험을 통한 개인의 자격 검증 타당화

의사국가시험의 기능은 의과대학 교육을 마친 졸업생을 대상으로 기본 진료의사로서 적절한 능력을 가졌는지 여부를 평가하여 이 기준을 통과한 사람에게 의사 자격을 주고 의료업에 종사할 수 있는 면허를 부여하는 데 있다. 우리나라에 의사국가시험 제도가 처음 도입된 것은 1952년이다. 그 뒤 62년이 지난 지금에 이르기까지 의사국가시험에는 많은 변화가 있었다. 그 중에서도 몇 가지 큰 변화를 지적하자면[2], 

• 시험을 주관하는 기관이 정부 부서에서 공익 중심의 민간 전문 평가기관으로 옮겨진 것을 시작으로 
• 시험목표 설정, 
• 시험과목수의 감소, 
• 과목별 시험문항 수의 증가,
•  문항의 질 및 형태 개선, 
• 시험장소의 분산, 
• 시험기관의 조직 보강 및 업무 확장, 
• 시험업무의 전산화, 
• 국제기구와의 협력체계 구축, 
• 2009년부터 도입된 실기시험의 도입 등이다. 

이제 의사국가시험은 의학지식뿐만 아니라 의과대학 졸업생들의 수행능력을 종합적으로 평가하게 됨으로써 개인의 자격 검증에 있어서 상당한 타당화를 이룩하였다고 할 수 있다. 최근에는 의사국가시험에 윤리 관련 문항의 포함, 실제 임상표현 중심의 국가고시 출제, 컴퓨터기반 국가시험 개발 등 새로운 도약을 준비하고 있다. 의사국가시험의 변화는 대학의 의학교육 내용, 방법과 학생들의 시험 준비 태도에도 상당 부분 영향을 주고 있다.

9. 의과대학 인증평가를 통한 대학의 책무성 강화

의과대학 인증평가제도는 외부기관이 실시하는 평가를 통해 의학교육의 질적 향상을 추구하고, 의과대학의 교육 책무성을 향상시키는 수단이다. 우리나라는 의사 양성에 대한 국민적 염원과 고등교육의 대중화 정책에 따라 1980년대 이후 많은 의과대학이 설립되었다. 그러나 의학교육기관의 급격한 양적 성장에도 불구하고, 질적인 성장이 동반되지 못함에 따라 의학교육의 질 관리에 대한 체계적인 접근이 필요하다는 주장이 강하게 제기되었다. 이러한 배경에서 1998년 설립된 한국의과대학인정평가위원회(이후 2003년 재단법인 한국의학교육평가원으로 변경)는 2000년부터 2004년까지 전국 41개 의과대학을 대상으로 제1주기 의과대학 인증평가를 실시하였으며, 2007년부터는 제2주기 인증평가를, 2012년부터는 Post 2주기 평가 사업을 진행하고 있다. 의과대학 인증평가제도는 대학 스스로 자체평가를 수행하도록 함으로써 의과대학 스스로 강점과 미비점을 분석하도록 한다는 점, 의과대학의 교육여건과 교육과정의 수준에 대해 사회적 인정을 부여한다는 점, 의학교육과 관계된 이해관계자들에게 교육의 질적인 수준에 대한 정보를 제공한다는 측면에서 많은 장점을 갖고 있다. 지난 10여 년 동안의 의과대학 인증평가 사업은 우리나라 의학교육의 질적인 수준을 한 단계 높였으며, 의과대학들의 사회적 책무성 수행을 강화한 것으로 평가된다[3].

의과대학 인증평가 시스템은 세계적인 추세이다. 

• 세계의학교육연맹(World Federation for Medical Education)은 의학교육의 국제 표준을 발표하고, 각국의 의과대학 인증평가(accreditation) 기구를 인정(recognition)하는 사업을 추진하고 있다. 
• 미국은 2023년부터 인증을 받지 않는 대학 졸업자들은 국내외를 막론하고 연수와 취업을 제한할 예정이다. 
• 우리나라도 의과대학 인증평가를 받지 않은 의과대학 졸업생들은 의사면허국가시험 응시 자격을 부여하지 않기로 하는 법률이 2017년부터 시행될 예정이다.

10. 의학교육 전문 기구의 설립

환자진료 또는 의학연구보다는 학생과 전공의 교육을 전담하는 사람들이 많아지고 있다. 1990년대 중반만 하더라도 대표적인 의학교육 전문가 그룹인 ‘한국의학교육학회’가 개최되면 약 50여 명의 교수들이 모여서 의학교육의 방향에 대해 논의를 했다. 2013년 현재 의학교육학술대회는 500명 이상의 교수들이 모여 지혜를 나눈다. 이러한 변화는 의학교육에 대한 관심의 증가와 의학교육 전문가 그룹이 성장했다는 것을 의미한다. 의학교육 관련 기관들도 체제를 정비하거나 활발한 교육 사업을 전개하고 있다. 예를 들어, 2003년 설립된 (재)한국의학교육평가원은 의학교육 기관의 평가와 인증을 담당하는 기구로 현재 Post 2주기 의과대학 평가 사업을 진행하고 있다. (재)한국보건의료인국가시험원은 의사면허시험을 주관한다. 한국의학교육학회는 의학교육 전문가들의 학술단체이다. 한국의과대학·의학전문대학원협회는 전국 의과대학과 의학전문대학원의 협의체로 의학교육정책을 개발하고, 관련 자료를 생산, 공유하는 중요한 역할을 담당하고 있다. 그러나 무엇보다 의학교육 관련 전문 기구의 설립에서 큰 특징은 개별 대학들이 의학교육 관련 전담 부서를 설치하고 있다는 점이다. 1996년 연세대학교 의과대학이 의학교육학과를 처음 설치한 이후 2013년 12월 현재 전국 41개 대학 중 37개 대학이 명칭을 달리하지만 의학교육 전담 부서를 갖고 있으며, 전임교원과 연구원을 포함하여 약 90여 명이 재직하고 있다.

의학교육에 영향을 미치는 의료 환경의 변화

대학의 교육과정은 사회의 다양한 요소들과 밀접하게 관련되어 서로 영향을 주고받는다. 사회는 교육이 지향해야 하는 방향에 대해서 어떤 형태로든 영향을 미치는 세력으로 작용하고, 대학은 사회에 부합하는 인적 자원을 개발하기 위해서 끊임없이 사회의 변화를 모니터링하고, 교육과정을 변화시켜 나간다. 즉, 사회에서 요구하는 지식, 기술 및 태도와 같은 역량을 분석하여 교육과정에 반영하는 작업을 반복한다. 이런 맥락에서 의과대학이 사회 또는 의료 환경의 변화를 모니터링하고 그 결과를 반영하는 것은 기본적으로 수행해야 할 책무이다. 의학교육에 영향을 미치는 국내외 의료 환경의 변화는 무엇인가, 의과대학은 어떤 역량을 갖춘 졸업생을 배출해야 하는가, 이러한 역량을 갖춘 졸업생을 배출하기 위해서 어떤 교육시스템을 갖추어야 하는가는 모든 의학교육 관계자들이 고민해야 하는 중요한 주제이다.

1. 의학기술의 발달

의학기술의 발달은 미래 의료분야 인적자원개발에 영향을 미치는 중요한 요인이다. 1984년 이후 2029년까지의 과학기술 세계를 묘사한 과학자 Robert Prehoda의 ‘Your next fifty years’, 미국의 해부병리학자 Jeffrey Fisher가 ‘RX 2000: breakthroughs in health, medicine, and longevity by the year 2000 and beyond’에서 서기 2000년대 의료기술 전반에 걸친 발전 양상을 소개한 내용들은 놀라울 정도이다. 최근 발표된 Eric Topol의 ‘The creative destruction of medicine’은 빅 데이터와 의료분야의 연관성을 설명하고 있는데, 인간의 디지털화가 의사와 병원, 생명과학기업, 규제기관들 그리고 개인의 삶에 지대한 영향을 미칠 것이라는 점을 예견하고, 곧 닥칠 미래에는 의학이 개인에게 초점을 맞출 것이라고 말한다. 또한 Clayton Christensen의 ‘The innovator’s prescription: a disruptive solution for health care’는 병원 사업모델, 의사의 진료모델, 만성질환 관리모델 등 의료의 새로운 패러다임을 역설한다. (...) 이런 여러 가지 의학 기술의 발달은 결국 의료서비스체계에서 의사의 역할을 크게 바꾸어 놓을 것이 확실하다. 진단과 치료의 많은 부분을 기계에 의존하게 될 가능성 많아지고, 의사는 그만큼 환자들로부터 물리적으로 멀어지게 될지도 모른다. 우리는 의학기술의 발달이 가져올 의료 환경의 변화와 의사의 역할에 대한 새로운 담론이 필요한 시점에 왔다.

2. 인구 및 질병 구조의 변화

한 국가의 의료서비스는 인구학적 특성의 영향을 크게 받는다. 최근 통계청이 작성한 생명표에 의하면, 우리나라 국민의 평균수명은 2012년 출생 기준으로 81.44세(남자 77.95세, 여자 84.64세)로 2001년에 비하여 4.91세(남자 5.13세, 여자 4.60세)가 늘어났다. 평균수명 연장에 따른 인구의 노령화와 그에 수반되는 만성질병 중심의 질병 발생 양상 변화는 우리나라 보건의료 환경의 중요한 변화가 되고 있다. 또한 경제 수준 향상에 따른 식생활의 변화와 환경오염, 그리고 운동부족은 각종 만성 비전염성 질환의 발생률을 높이게 됨으로써 이들 질병에 대한 치료기술의 발달에도 불구하고 유병상태가 큰 보건문제로 등장하게 될 것이다. 질병 구조의 변화는 의료의 소비 수요 증가와 밀접하게 관련되어 있다. 과거 인류 역사에서 질병의 전형적인 형태였던 전염병이나 위생과 관련된 급성질환들은 점차 감소하고 있고, 인구의 노령화와 생활행태의 변화에 따른 만성퇴행성 질환이 주류를 이루어 가는 추세가 뚜렷하다. 건강개념의 변화에 따라 질병 발생 이후의 치료서비스 추구와 같은 소극적인 개념에서 탈피하여 조기 진단 조기 치료, 그리고 질병의 예방, 건강 증진 등과 같은 보다 적극적인 개념의 건강관리가 보편화되는 추세이다. 이에 따라 과거에 병원이 맡아 왔던 각종 의료서비스 이외의 보건의료서비스가 개발되어 제공되어야 할 필요성이 있다. 이러한 변화들은 환자의 진료를 주 기능으로 하는 병원의 기능과 역할의 급속한 변화를 요구한다.

3. 정보통신 발달과 소비자 중심 사회

의료 환경의 변화와 관련된 최근의 사회적 변화 중의 하나는 정보통신 매체의 발달과 의학 데이터베이스의 구축이다. 이러한 변화는 일반 대중들이 질병의 증상, 진단, 치료에 대한 정보를 손쉽게 접할 수 있도록 하였다. 의사들이 중심이 되어 네트워크를 기반으로 의학정보를 제공하기도 한다. 이제 환자들은 전문적인 의학지식을 더 이상 의사들에게만 의존하지 않는다. 그들은 의료의 합리적 소비를 위해서 자신의 질병과 질환에 대해 스스로 학습하는 주체로 바뀌어 가고 있다. 의학지식은 의사들의 전유물이 아닌 시대가 되어 가고 있다. 일반 대중은 의사-환자 관계에서 파트너로서 성장하고 있다. 치료의 대상이자 치료의 주체이고 동반자로 인식되어 가고 있는 것이다. 21세기 우리 사회의 또 다른 특징은 소비자 중심의 사회가 구현되고 있다는 점이다. 의료인은 자본주의 시장에서 의료라는 서비스를 고객에게 제공하고 환자는 의료인으로부터 진료서비스를 제공받는 대가로 진료비를 지불하는 풍토가 조성된 것이다. 민주주의 정착에 따른 시민의식의 제고와 시장경제 체제하에서의 소비자 우대 현상은 의사-환자 관계에 있어서도 더 이상 의사의 가부장적 위치를 인정하지 않게 될 것이 분명하다. 의사 입장에서 보면 21세기는 더 이상 의사직이 권위적인 직업이 아니며 환자와 협력하여 건강문제를 풀어가야 하는 동업자적인 위치가 되는 것이다. 이러한 사회변화와 함께 일반대중에게 다가온 새로운 개념은 건강권의 개념 확립이다. 건강권에 대한 개념의 변화는 현대 산업사회에서 의료가 과거와는 매우 다른 양상을 띠기 때문에 초래된 현상이다. 과거에는 질병의 형태가 전염성 질환 중심으로 발생하였으나, 물질적으로 풍요해진 현대사회에서는 만성질환, 유전성 질환, 정신질환 등으로 주요 질환의 형태가 바뀌었다. 뿐만 아니라 일반대중의 건강상태도 향상됨에 따라 의료서비스에 대한 추구행태도 변모할 수밖에 없고 의료에 대한 인식은 물론 의료인에 대한 인식도 달라졌다. 건강권의 확립추세는 의료기술의 혁신적인 발전과 맞물려서 의료의 내용이나 형태는 물론 의료의 주체가 의사로부터 진료를 받는 환자 쪽으로 전환되는 경향을 보이고 있으며, 의료보험제도 실시 이후 의료수요가 증가하면서 이러한 추세는 더욱 가속화되었다.

4. 이해관계자의 의료에 대한 개입 증가

의료는 전문영역이어서 전통적으로 의료인의 자율성이 존중되는 부문이었다. 특히 민간중심의 시장경제 원리를 중요시하는 국가들은 보건의료 영역에 대한 국가의 간섭을 최소화함으로써 의료인들이 직업전문성(professionalism)을 발달시켜 자유전문직으로 발전할 수 있었다. 그러나 20세기 중반 이후 의료 전문직의 사회적 책무성(social accountability)이 강조되고, 보건의료를 단지 개인이나 가족의 책임으로 두었던 데에서 국가의 책임을 강조하는 추세로 바뀌면서 의료전문직과 사회 간에 이루어졌던 묵시적 사회적 계약은 더 이상 유효하지 않게 되었다. 이러한 변화는 전통적으로 의료인의 전문영역으로 간주되었던 의료서비스 전반에 규제와 간섭의 모습으로 나타나기 시작하였다. 예를 들어, 우리나라는 전 국민 의료보험제도를 도입하면서 사회보장형 보건의료체계에 가까운 형태로 의료서비스 체제를 전환하였다. 이제 보건의료는 주택, 환경, 교통, 교육 등과 마찬가지로 사회 구성원들의 삶의 질을 향상시키기 위해 필수적인 공공재화로 인식되고 있어서 국가와 사회가 책임지고 관여하는 서비스 영역으로 되었다. 이에 따라 국가 또는 시민단체가 의료 영역에 대하여 개입하기 시작하였고, 점차 그 정도가 심화되는 추세에 있다. 의료계 내에서는 보험제도의 실시 이후 의료인의 직업지위에 대한 우려가 커지게 되었고, 의료보험제도의 도입 이후에 의원들의 경영상태가 대체적으로 악화되었고, 의사들이 전문직으로서 최대한의 자율성을 가지고 진료를 할 수 없게 되었다. 이것은 우리나라의 의료보장제도 확립 과정에서 의사들의 진료에도 외부적 영향력이 개입됨으로써 의료인상이 상당히 변모되었음을 시사한다. 대부분 자본주의 체제하에서의 자유방임적 의료제도는 소비자의 의료 및 의사 선택권이나 의료의 질 향상에 긍정적인 영향을 미쳐온 것이 사실이지만 지역 간, 계층 간 의료 서비스의 불균형과 의료비의 상승같은 부정적 현상을 심화시켜 온 것 또한 사실이다. 이런 제도를 채택해 온 미국 등 많은 나라가 점차 의료의 규격화 내지 사회화 방향으로 의료제도를 수정해 가고 있는 것도 바로 자유방임적 의료가 갖는 부정적인 면 때문이다. 우리나라도 이미 저수가 의료보험정책 등을 통해 의료비 상승을 억제해오고 있지만, 앞으로 포괄수가제 등을 도입하는 등 더욱더 의료의 규격화가 가속화되고, 건강보험심사평가원, 의료소비자 단체, 보험회사 등 제3자의 개입이 확대될 전망이다.

미래 의학교육을 위한 다섯 가지 제언

선진 외국은 1980년대 초부터 21세기 의료 환경 변화를 예측한 의학교육계획을 수립하고, 대학들이 경쟁적으로 의학교육의 내용과 방법을 개선해 왔다. 미국의 경우 1981년 미국의사협회와 미국의과대학협회는 특별 연구 패널을 만들고, 3년여에 걸친 대대적인 연구를 거쳐 ‘21세기 의사상’으로 알려진 GPEP리포트(Report of the Panel on the General Professional Education of the Physician and College Preparation for Medicine)라는 책자를 펴낸 바 있고, 영국의학협회는 1993년에 ‘미래의사(Tomorrow's doctors)’라는 지침서를 발간하였으며, 2003년, 2009년에는 개정판을 만들어 모든 의과대학들이 그 내용을 준수하도록 한 바 있다. 일본도 1985년 이후 여러 차례 의학교육 전문가회의를 거쳐 1998년에 ‘생명과 건강을 지키는 21세기 의료인의 육성을 지향하며’라는 새로운 의료인 양성 교육계획 보고서를 발표한 일이 있다. 이들 나라의 21세기 의학교육계획이 모두 똑같은 것은 아니지만 이들이 진단하는 21세기 사회의 모습이나 이런 사회에서 필요한 의료인의 모습, 그리고 이런 의료인 양성을 위한 의과대학에서의 교육과정 내용이나 방법이 어떠해야 하는지는 거의 대동소이한 것을 알 수가 있다.

1. 의사들이 갖추어야 하는 역량에 기초한 교육을 실시해야 한다.

의과대학 학생들은 과거와는 비교할 수 없을 정도로 증가된 의학지식, 술기 및 의사로서의 태도를 습득한다. 그럼에도 의료현장에서는 의과대학을 졸업하는 학생들이 활용 가능한 실제적인 능력을 갖추고 있지 못하다는 비판이 있다. 의과대학 기간 동안 무엇을 가르칠 것인가는 중요한 문제 중에 하나이다. 의료 환경의 급격한 변화와 의료분야에 종사하는 사람들에게 요구되는 역량에 대한 연구가 활발하게 이루어지고있다. 의료분야에서 역량에 대한 최초의 논의는 미국소아과학회에서 ‘소아과 의사의 역량평가를 위한 토대(Foundations for evaluating the competency of pediatricians)’라는 논문이 발표되면서부터이다. 이후 미국의과대학협회는 1988년 의사들이 갖추어야 하는 네 가지 역량(knowledgable, skillful, dutiful, altruism)을 정의한 바 있으며, 미국졸업후의학교육인증평가위원회는 1999년에 전공의들이 획득해야 할 여섯 가지 역량(medical knowledge, patient care, interpersonal & communication skills, system-based practice, practicebased learning, professionalism)을 발표하였다. 캐나다에서는 왕립의학회가 주축이 되어 의사가 수행해야 하는 여섯개 역할(medical expert, communicator, manager, collaborator, health advocate, scholar, professional)을 중심으로 CanMEDS 2005를 도출했다. 영국의학협회도 1993년 발간된 ‘미래의 의사상’을 개정한 보고서를 2009년에 발간하면서 의사들이 가져야 하는 역량을 구체화하였다. 유럽에서도 2002년 스코틀랜드의 5개 의과대학이 열 두 개의 성과를 도출해 Scottish Doctor 모델을 개발하였다. EU 차원에서도 2008년에 Tuning Project를 통해 의학교육의 성과를 규정한 바 있다.

우리나라에서는 의료 환경 변화에 따라 요구되는 의사의 역량을 국가적 차원이나 의학교육 관련 협회 차원에서 규정되어 있지는 않다. 다만 국내의 의학교육 관련 문헌들에서는 바람직한 의사상을 규명하려는 노력과 의료 분야에 종사하는 사람들이 갖추어야 하는 역량을 제시한 다양한 연구들이 있다[4,5,6,7,8]. 이러한 연구 결과들은 의료 환경 변화에 따라 미래 사회에서 요구되는 의사의 역량을 규명하기 위한 시도들을 하였는데, 맹광호[4]는 ‘21세기 한국 사회에서 바람직한의사상’이라는 논문에서 선행 연구 결과들을 종합하여 미래사회 의사들에게는 예방적 일차 진료능력, 질병에 대한 자세한 설명능력, 높은 윤리의식, 의료제도에 대한 이해와 협력, 지역사회 보건의료팀 지도자로서의 자질, 보건의료정보의 생산 및 처리, 국제화 및 세계화 관련 의료 활동 능력, 평생교육의 생활화 등에 관한 역량이 필요함을 지적하였다. 또한 안덕선 등[9]은 한국의 의사상 연구를 통해 다섯 가지 역량(patient care, communication & collaboration, social accountability, professionalism, education & research)을 개념화한 바 있다. 대한의사협회는 2013년 창립 105주년을 기념하여 의학교육 전문가들을 초청해서 ‘위기의 의사, 직업전문성 확보를 위한 제언’이라는 제목으로 특집 좌담회를 가진 바 있다. 이 자리에서 의학교육 전문가들은 인구 구조와 질병 양상, 그리고 건강에 대한 사회적 욕구의 변화 등으로 전세계는 이제 새로운 가치와 의무를 수반한 의사 양성교육이 시급하게 요구되고 있다는 점에 동의하면서 이런 세계적 추세에 부응하는 ‘새로운 의사의 역할(global role of doctor)’ 정립이 필요하다고 강조했다. 그러면서도 이들은 이런 상황에 대한 인식조차 제대로 되어있지 않은 우리의 현실을 ‘위기’로 진단한 바 있다. 새로운 시대, 새로운 의사 양성의 필요성과 그 구체적 실천을 위한 의료계 특히 의학교육계의 좀 더 구체적인 행동이 필요한 때이다.

2. 의과대학교육, 졸업 후 수련교육의 연계성이 확보되어야 한다.

의사 양성교육은 의과대학교육과 졸업 후 수련교육으로 구분된다. 의과대학을 졸업하는 시점에서 의사국가시험을 치르고 의사 자격을 취득하기는 하지만, 졸업생의 95% 이상이 인턴과 전공의 과정을 거치면서 실제 환자진료능력을 갖춘 의사로 성장한다. 이것은 의사 양성교육이 의과대학 교육, 졸업 후 수련교육이라는 연속선상에서 통합적으로 계획되어야 한다는 것을 의미한다. 단계별 의사 양성과정이 잘 연계되어 있지 못하다는 지적이 많다. 의과대학교육은 교육부에서 관장하는 학위과정으로 규정되어 있고, 졸업 후 수련교육은 보건복지부가 관장하는 수련과정으로 분류되는 것만 보아도 알 수 있다. 의과대학교육을 담당하는 의과대학·의학전문대학원협회, 졸업 후 수련 교육과 관계되는 대한의학회 및 대한병원협회가 의사 양성이라는 큰 틀에서 유기적 협의를 충분히 하지 못하고 있는 것이 현실이다. 이러한 현실은 우리나라 의학교육 계획의 부재라고도 설명된다. 의사 양성 단계별로 중요하다고 생각하는 것이 다른 단계에서는 반영되지 않는 어려움이 있다. 예를 들면 의과대학에서 새로운 교육 과정을 도입하고 새로운 평가 결과를 만든다 할지라도, 그것이 의사국가시험, 병원의 인턴, 전공의 선발에 반영되지 않는다면, 새로운 의학교육의 효과는 기대하기 어렵다. 인턴제 폐지 논의도 마찬가지이다. 의사들에게 요구되는 역량이 의과대학교육에서는 중요하게 다루어지고, 전공의 수련교육에서는 별로 중시되고 있지 않다면 그 문제는 매우 심각해진다. 이 두 단계의 교육이 얼마나 유기적으로 잘 연결되어 의사 양성을 하는가에 우리나라 의료의 미래가 달려 있다. 이제 우리나라에서도 의사 양성 전 과정의 교육을 효율적으로 관리, 운영하는 체계를 구축할 필요가 있다. 의과대학·의학전문대학원협회, 대한의학회 및 대한병원협회가 함께 참여하는 ‘의학교육위원회’부터 시작해도 좋은 방법이다.

3. 의학교육의 질 개선 활동이 강화되어야 한다.

대학의 양적인 성장과는 달리 의과대학의 기능을 제대로 평가하고 그 질적인 수준에 대한 논의가 종합적으로 검토되기 시작한 것은 1990년대 후반의 일이다. 의학교육의 질에 대한 체계적인 접근의 필요성은 의과대학의 사회적 책무성 수행에 대한 요구와 의과대학의 교육적 기능에 대한 비판적 의식이 고조되면서 활발히 논의되기 시작하였다. 의학교육의 질 관리는 교육의 질에 대해 관심을 갖는 주체에 따라 크게 국가, 대학 및 개인단위에서의 질 관리로 구분할 수 있다[10]. 

• 첫째, 국가적 단위에서 의학교육의 질을 관리한다는 것은 의료서비스를 제공하는 의사들에게 요구되는 최소한의 능력과 윤리적 태도를 확보하기 위한 것이다. 또한 이것은 보건의료정책의 개발, 인적 자원의 배분, 제한된 자원의 효율적인 사용 등과 관계된 국가적 수준에서 결정되는 질 관리를 의미한다. 국가적 수준에서의 질 관리는 행정 및 재정과 관련된 감사, 국가면허시험, 평가인증제도 등이 여기에 해당한다. 
• 둘째, 대학단위에서의 질 관리는 대학의 많은 사명 가운에 교육의 책무성 수행에 대한 요구라고 할 수 있다. 대학단위에서의 질 관리는 흔히 내적인 질 관리라고 할 수 있는데, 대학의 교육과정 중에 일어나는 각종 평가시험, 대학의 모니터링 시스템, 지속적 질 관리 활동이나 자체평가과정이 여기에 해당된다. 
• 마지막으로 개인수준에서의 질 관리는 학생 또는 졸업생 개개인의 능력과 관계된다. 우리나라에서는 아직도 개별 대학에 따라서는 교육과정과 교육여건의 차이가 많은 것은 사실이지만 한국보건의료인국가시험원이 주관하는 의사면허 필기시험 및 실기시험의 도입과 의과대학인증평가제도의 도입으로 국가수준에서의 질 관리는 어느 정도 자리를 잡았다. 이제 대학단위의 자율적인 질 관리와 학생 개인 수준에서의 학습의 질관리에 관심을 가져야 한다. 

한편, 그 동안의 질 관리가 의과대학교육 수준에 머물러 있었다고 한다면, 이제 졸업 후 수련교육과정에서의 질 관리로 관심을 집중해야 할 때이다. 그 동안 졸업 후 수련교육과 관련된 많은 기구들이 질 관리 노력을 하고 있으나, 아직은 근본적인 문제들을 해결하지 못하고 있다. 국가적 차원에서 필요로 되는 전공의 숫자 조정과 엄격한 수련교육의 질 관리를 위한 새로운 기구가 필요할지 모른다. 또한 의사 연수교육 단계에서의 질 관리도 필요하다. 의사들이 스스로를 향하여 이런 엄격성을 보이지 못한다면, 의사가 국가나 국민에게 설득할 힘은 없을 것이다.

4. 의사양성 시스템을 재정비해야 한다.

의학교육기관들이 우수한 의사를 양성해 왔다는 사실을 부정할 사람은 없을 것이다. 연구를 통하여 의학의 발전을 이끌었으며, 탁월한 의료서비스 제공을 통하여 질병 치료와 국민의 건강 증진에 기여하였다. 그러나 우리에게 성찰적 역량이 조금만 있다면 의사 양성의 핵심인 의과대학 교육이 얼마나 체계적이지 못한지 금방 알 수 있다. 2004년 의학전문대학원 제도가 도입되면서 우리나라 의학교육은 의과대학과 의학전문대학원 시스템이 양립하게 되었다. 정부의 정책에 따라 많은 의과대학들이 의학전문대학원으로 전환하거나 의과대학과 의학전문대학원을 병행하였고, 이제 다시 2015학년도부터는 5개 의학전문대학원을 제외하고는 의과대학 체제로 복귀한다. 이러한 변화과정에서 학생 선발, 의과대학교육, 수여 학위, 학술 학위 대학원 교육 등 많은 혼란을 겪었다. 아직도 고등학교 학생을 선발하는 의과대학, 학사학위 취득자를 선발하는 의학전문대학원, 그리고 의과대학 학사편입학 제도가 공존하는 상황에서 의사양성 시스템은 여전히 혼돈스럽다. 우수한 의사를 양성하는 데 있어서 의과대학 또는 의학전문대학원 시스템이 더 좋은가에 대한 논쟁은 시기적으로 바람직하지 않다. 의학전문대학원 제도를 통해 배출되는 의사인력에 대한 중장기적 평가 결과가 도출되기도 전에 의학교육은 대학자율로 결정하도록 되었고, 많은 대학이 의과대학체제로 돌아가기로 했기 때문이다. 이제 혼돈의 과정에서 나타난 문제를 지혜롭게 해결하는 것이 중요하다.

• 첫째, 의과대학이든 의학전문대학원 체제이든 각 대학이 선택한 제도 하에서 양질의 의사들이 양성될 수 있도록 제도적 정비를 해야 한다. 어떤 의사인력 양성 체제가 더 좋은 제도인가에 대한 논쟁보다는 의학교육의 질적인 수준 향상과 교육여건을 향상시키기 위한 노력이 더 중요하다. 정부는 법률적 정비와 재정적 지원을 강구하고, 각 대학은 자신이 선택한 체제의 장점을 최대화하는 교육 정책을 실천하는 것이 필요하다. 
• 둘째, 고등학교 학생을 선발하여 교육하는 의과대학 체제 하에서도 다양한 학문적 배경을 가진 학생들을 선발하여 교육할 수 있는 프로그램의 다양성은 유지되어야 한다. 예를 들어, BS-MD,MD-PhD 등 프로그램의 목적에 따라 국가 발전에 기여할 수 있는 다양한 인재를 선발하여 교육할 수 있어야 한다. 이러한 제도가 의학전문대학원은 가능하고 의과대학은 불가능하다는 논리는 적절하지 않다. 학사편입학제도를 포함해서 더 다양한 프로그램들이 개발되고 운영될 수 있어야 한다. 
• 셋째, 다양한 학문분야에 대한 이해와 인성 개발을 위한 최소한의 교육기간과 교육내용이 담보되어야 한다. 의과대학 교육이 의학지식 및 술기를 함양하는 직업교육에 머물러서는 안 된다. 고등학교 졸업생들이 의과대학에 진학해서 인문, 사회 등 다양한 분야의 교양과 인문학적 소양을 함양하도록 하고, 학생들의 잠재력과 창의력을 신장하는 교육 시스템이 되어야 한다. 
• 넷째, 동일한 교육과정을 이수한 학생들에게는 동일한 학위가 수여되는 것이 바람직하다. 의과대학과 의학전문대학원교육은 세계의학교육이 구분한 의학교육 기본과정(basic medical education)에 해당한다. 따라서 현재와 같이 의과대학은 학사학위, 의학전문대학원은 전문학위로서의 석사학위수여는 타당하지 않다. 학술학위이든 전문학위이든 의학교육시스템에 부합하는 새로운 학위개념이 정비되어야 한다.

5. ‘대학의학’의 개념이 정립되어야 한다.

미국 의료계에서 사용하는 용어에 ‘대학의학(academic medicine)’이란 말이 있다. 이 용어는 의사 양성 단계별 교육, 의학 발전을 위한 연구 및 보건의료가 대학의학에 의해 결정된다는 인식과 자부심에 근거하여 만들어진 말이다. 미국의과대학협회 연차 총회 개막연설에서 Kirch [11]는 대학의학의 의미를 다음과 같이 설명하고 있다. 그는 “긍정적이든 부정적이든 한 나라의 의학교육이나 의학연구, 그리고 의료제도와 국민들의 의료행태 변화에는 의료인들의 책임이 작지 않다. 따라서 의료인을 양성하는 의과대학과 수련병원들이 이런 변화를 확실히 인식하고 있어야 할 뿐 아니라 그 변화를 올바른 방향으로 유도해가야 할 책임이 있다.”라고 주장하였다. 그의 연설은 대학의학의 뚜렷한 목표와 발전 전략 없이는 결코 한 나라의 보건의료문제나 국민들의 건강문제에 효과적으로 대처할 수 없다는 점을 강조한 것이었다. 우리나라의 대학의학은 한마디로 뚜렷한 리더십이 없는 상황에서 관련 단체들 사이에 힘의 분산과 불필요한 경쟁만을 계속해 온 면이 없지 않다. 의과대학은 의과대학대로, 병원은 병원대로, 그리고 전문 학회는 그들 학회대로 제각기 자신들만의 활동을 해온 것이 우리의 현실이고, 이 일은 결국 의료계 전체의 발전은 물론 각 관련 단체들 간에도 이해관계만을 증폭시켜 불필요한 경쟁 상태로 서로의 힘을 약화시켜온 점이 없지 않다. 이제 우리나라도 어떤 형태로든지 강력한 ‘대학의학’을 창출하고 발전시켜 나가야 한다. 그러기 위해 대학의학 관련 여러 단체가 의료 인력의 교육과 훈련, 그리고 연구 등에 관해 힘을 모아야 한다. 그리고 이를 바탕으로 국민건강 증진을 위한 의료시스템 구축과 운영에도 주체적으로 참여해야 한다. 그렇지 않으면 의학교육도, 보건의료 시스템도 모두 현실적으로 정부의 계획에 따라 수시로 변화해야 하고 그로 인한 경제적, 사회적, 인적 비효율성의 피해가 고스란히 국민들에게 돌아가는 일이 계속될 수밖에 없다.

Korean J Med Educ > Volume 26(3); 2014 > Article

REVIEW

Korean J Med Educ 2014; 26(3): 167-178.

Published online: September 1, 2014

DOI: http://dx.doi.org/10.3946/kjme.2014.26.3.167

미래 의학교육을 위한 다섯 가지 제언

양은배¹, 맹광호²

¹연세대학교 의과대학 의학교육학과

²가톨릭대학교 의과대학 예방의학교실

Five suggestions for future medical education in Korea

Eunbae B. Yang¹, Kwang Ho Meng²

¹Department of Medical Education, Yonsei University College of Medicine, Seoul, Korea

²Department of Preventive Medicine, The Catholic University of Korea College of Medicine, Seoul, Korea

Corresponding Author: Eunbae B. Yang (http://orcid.org/0000-0002-6771-1929) Department of Medical Education, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 120-752, Korea Tel: +82.2.2228.2511 Fax: +82.2.364.5450 email: nara@yuhs.ac

Received May 17, 2014       Revised June 16, 2014       Accepted July 3, 2014

의학교육 변화의 리더십

Leadership Challenges in the Advancement of Medical Education

신좌섭

서울대학교 의과대학 의학교육학교실

Jwa-Seop Shin, M.D., Ph.D.

Department of Medical Education, Seoul National University

College of Medicine, Seoul, Korea

1. 의학교육의 변화에 대한 요구

의학교육의 변화에 대한 요구로는 의과대학 외부로부터의 압력과 내부로부터의 동력이 존재한다. 외부로부터의 압력으로는 ‘교육-연구-진료의 삼중고’로 상징되는 서로 상충하는 미션에 대한 사회적 요구, 의과대학 인증 및 의사국가시험의 압력, 재정적 유인 혹은 압박 등을 들 수 있다. 내부로부터의 동력은 의사집단, 졸업생, 교육환경의 현 상태와 이상적 상태의 격차로부터 변화의 필요성을 인식하고 비전을 설정한 리더 혹은 리더 집단으로부터 나온다[5].

1) 외부로부터의 압력

(1) 상충하는 미션에 대한 사회적 요구

의과대학 및 제휴 교육병원(affiliated training hospital)에 주어진 사명은 교육, 연구, 진료(봉사)이다. 이 3가지 사명은 서로 유기적으로 결합하여 의학과 의학교육의 발전에 기여해야 한다. 그러나 시대와 지역의 상황에 따라 3가지 사명은 서로 충돌하기도 한다[5]. 오늘날 우리나라의 교육병원들은 진료를 최우선으로 강조하고 의과대학들은 연구를 최우선으로 강조하고 있다. 이는 대학과 병원에 대한 사회적, 재정적 요구에 부응하기 위한 것으로서 연구와 진료는 각각 대학과 병원의 가장 중요한 생존 조건이 되고 있다.

이 같이 교육과 상충할 수 있는 미션에 대한 사회적 요구는 의학교육을 변화시키는 외부적 압력으로 작용한다. 진료를 효율적으로 수행하면서도 학생이나 전공의 교육을 병행하는 방법론들이 의학교육의 화두가 되고 있는 것은 교육과 상충하는 미션의 압력에 부응하기 위한 노력의 일환이라고 하겠다.

(2) 인증 및 자격시험의 압력

의과대학 인증평가도 의학교육 변화에 대한 외부적 압력의 하나로 작용한다. 아직까지 우리나라에서는 인증유예나 인증불가 판정을 받는다고 해서 특별한 불이익이 있는 것은 아니지만, 대학의 평판에는 적지 않은 영향을 미치는 것으로 보인다. 때문에 인증유예나 인증불가를 받지 않기 위한 혹은 우수기준을 충족시키기 위한 의학교육의 변화는 인증평가 매 주기마다 크든 작든 각 의과대학에서 반복되고 있다.

의사국가시험도 의학교육 변화에 대한 압력으로 작용한다. 최근 의사국시 실기시험 도입은 임상수기센터 설립, 반복적인 모의 실기시험의 시행, 임상실습과정의 강화 등 각 의과대학의 교육에 적지 않은 변화를 가져왔다. 인증 및 자격시험의 압력에 의한 의학교육의 변화는 근본적으로 의학교육의 발전을 지향하지만 ‘평가 주도적 변화’라는 측면에서 규정이 요구하는 형식 요건이나 성과 지표를 충족하는데 주안점이 두어져 지속가능성에 한계를 보이는 경우가 많다.

(3) 재정적 유인 혹은 압박

의과대학과 교육병원에 대한 재정적 유인이나 압박도 의학교육 변화의 동력으로 작용한다. 의학전문대학원 전환을 조건으로 한 재정 지원과 이에 따른 교육변화가 최근 우리나라 의과대학들이 겪은 대표적 사례라고 할 수 있을 것이다. 재정적 유인에 의한 변화는 비교적 추진하기가 용이하지만, 지원의 지속성에 따라 성공 여부가 크게 좌우된다는 문제를 안고 있다.

정부의 의료수가 정책도 압박으로 작용한다. 의료수가 관리정책은 진료 행태에 영향을 미칠 뿐만 아니라 의료진이 학생, 전공의 교육에 할애하는 절대적 시간에 영향을 미치고, 재원 일수의 감축은 학생들이 환자의 질병 경과를 추적할 기회를 박탈하는 효과를 가져온다. 재원일수 감축은 최근 미국에서 본격적으로 시도되고 있는 과목간 통합임상실습(longitudinal integrated curriculum) 도입의 배경이 되었다[6].

2) 내부로부터의 동력

(1) 의사집단, 졸업생 역량에 대한 인식

내부로부터의 동력은 의사집단, 졸업생의 역량과 관련하여 교육과정의 현 상태와 이상적 상태의 격차로부터 변화의 필요성을 인식하고 변화의 목표를 세운 리더 혹은 리더 집단으로부터 나온다. 2000년 의약분업 사태 이후 각 의과대학 교육과정에서 프로페셔널리즘과 커뮤니케이션 스킬 교육이 강조되고 있는 것은 졸업생이나 이미 배출되어 활동하고 있는 의사집단의 특정 역량 부족에 대한 인식이 교육 변화에 영향을 미친 대표적인 사례가 될 것이다. 수련기간 연장이나 인턴제도의 폐지 등도 의사집단의 필요 역량에 대한 인식변화에 따른 교육 변화의 사례라고 할 수 있다.

(2) 교육환경(의학지식의 팽창과 세분화, 환자의 권리의식 향상 등)의 변화

1950년대 미국 케이스웨스턴 리저브 대학이 통합교육을 도입한 것은 전문분야의 과도 세분화와 의학지식의 고도팽창에 대응하기 위해, 전통적인 플렉스너 스타일의 기초 2년, 임상 2년 교육의 한계를 통합교육이라는 교각(bridging science)으로 연결한 창의적 변화의 좋은 예라고 할 수 있다[7].

최근 세계적으로 널리 보급되고 있는 성과바탕의학교육(outcome-based medical education)이나 임상표현교육과정(clinical presentation curriculum)도 의학지식의 팽창과 세분화 속에서 필수 역량을 제대로 갖춘 졸업생을 배출하려는 의도를 갖고 있다. 한편 의료소비자로서 환자의 권리의식 향상은 환자안전교육, 시뮬레이션 교육이 급속히 보급되는 배경이 되고 있다.

(3) 새로운 교육이론이나 기법의 등장

1970년대 캐나다 맥마스터 대학이 문제바탕학습을 도입한 것은 임상적 맥락으로부터 동떨어진 무미건조한 이론학습에 대한 문제 인식에서 비롯된 것으로 어차피 의사가 될 학생들이 보다 즐겁게 흥미를 가지고 공부할 수 있도록 돕기 위한 것이었다[8]. 이렇게 도입된 문제바탕학습이 구성주의(constructivism) 교육철학의 이론적 성원을 받아 전세계로 보급된 것은 교육이론이 교육변화를 촉발한 사례라고 할 수 있다.

새로운 기법이나 이론을 벤치마킹하는 교육변화는 목표가 명확하고 가시적이어서 변화를 추구하기가 비교적 용이하다는 장점을 갖고 있으나, 새로운 기법을 적용하기 위해 새로운 행동방식을 채택해야 하는 교수와 학생의 특성, 교육환경의 특성 때문에 곤란을 겪는 경우가 많다. 1970년대 우리나라에 도입된 통합교육이 미국과는 달리 기초-임상 교실간 갈등으로 파행을 겪은 것이나[7], 북미에서 성공적이었던 문제바탕학습이 아시아권에서 정착에 어려움을 겪고 있는 것이 그 좋은 예이다.

2. 의학교육변화의 성공에 영향을 미치는 요인

3. 변화 추진 주체에 대하여

변화는 후원자(sponsor)와 변화촉진자(change agent), 동참자(target)의 역동적 관계 속에서 이루어진다. 으레 학장단 등 정책결정자로 간주할 수 있는 후원자의 역할은 변화의 범위와 깊이를 결정하고 변화의 비전을 조직구성원에게 설파하며 변화에 필요한 자원을 조달하고 스스로 변화된 행동을 솔선수범하는 것이다. 변화를 기획하고 이론적으로 지원하는 변화촉진자의 역할은 변화의 각 단계를 이끌어나갈 전략을 수립하고 변화에 대한 저항에 대처하며, 변화의 성과를 측정하고 이를 조직에 전파시키며, 변화활동이 각 현장에 잘 접목되도록 하는 것이다. 변화를 실제로 현장에서 실천해나가는 조직구성원이라고 할 수 있는 동참자의 역할은 변화활동의 기획에 함께 참여하고 현장의 목소리로 의견을 내며 변화에 필요한 새로운 스킬을 배우는 것이다.

우리나라 대부분의 의과대학 상황에 비추어 본다면 여기서 후원자는 학장단, 변화촉진자는 의학교육실이나 의학교육학 교실, 교육위원회 등이 될 것이고 동참자는 일반 교수와 학생들이 될 것이다. 즉, 학장단이 의학교육담당기구의 전문성을 활용하여 교수와 학생들을 설득하고 변화를 추진해나가는 것이 우리나라 의학교육에서 변화 추진의 가장 일반적인 구조이다.

그러나 의학교육 변화에 많은 시간이 걸리는 데 비해 후원자 역할을 해야 할 의과대학 리더십의 수명은 대부분의 대학에서 2년이고, 변화촉진자 역할을 해야 할 의학교육실의 전문성을 일반 중견 교수들은 잘 받아들이지 않는 것이 현실이다. 따라서 의학교육에 대한 이해가 깊으면서 대학구성원으로부터 신뢰를 받는 중견, 원로 교수들의 역할이 매우 중요하다[9]. 중견, 원로 교수 집단이 의학교육실의 부족한 전문성을 보완하고 학장단 교체 후에도 리더십의 연속성을 담보하는 구조를 창출해야 하는 것이다.

4. 변화의 단계

변화의 실패는 무엇보다도 교육을 개혁하겠다는 사람들이 갖고 있는 잘못된 신화들이 원인으로 작용하는 경우가 많다. 많은 교육개혁자들은 구성원들이 자기들의 생각에 동참하기만 하면 개혁이 이루어질 것이라는 순진한 생각을 갖고 있다. 이들은 개혁이 잘 이루어지지 않으면 ‘그들이 따라오지 않기 때문’이라거나 ‘그들의 완고한 사고방식 때문’이라고 탓한다. 따라서 변화를 성공시키려면 먼저 개혁자가 되고자 하는 사람들을 먼저 개혁해야 할 필요성이 생긴다[11].

변화를 성공시키기 위해서는 과학에 기반한 방법론, 즉 변화관리(change management) 이론에 근거하여 변화를 추진할 필요가 있다. 변화관리란 변화를 이끌어가는 변화촉진자가 변화를 실제로 수행해야 하는 조직구성원과 대화를 어떻게 해나갈 것인가, 변화가 일어날 수 있는 조직상황을 어떻게 창출할 것인가, 성공적인 변화를 위해 조직적 유대감을 어떻게 창출할 것인가에 관한 방법론이다. 변화관리 이론은 기업 조직에서 만들어진 것이지만 대학에서 변화를 추진하는 데에도 여전히 유효하다. 교육을 바꾸고자 하는 변화촉진자(change agent)들도 변화관리 이론을 현장에 적용할 수 있어야 한다.

변화이론의 대가로 널리 인정받고 있는 Kotter에 의하면 성공적인 변화를 위해서는 변화의 8단계를 따라야 한다[12]. Kotter가 말하는 변화의 8단계는 

조직 내 위기감 조성(increase urgency), 

변화추진팀 구축(build the guiding team), 

비전 개발(get the vision right), 

비전의 전파(communicate for buy-in), 

변화의 실천 독려(empower action),

단기적 성공사례 창출(create short-term wins), 

변화의 지속적 추진(don’t let up), 

변화의 고착(make change stick)이다.

8단계 각각은 변화촉진자가 수행해야 할 임무이며, 여덟 단계 각각의 가장 중요한 과제는 후원자(sponsor)와 동참자(target) 등 조직 구성원들의 행동을 변화시키는 것이다.

8단계 이론에 비추어 볼 때 우리나라 의과대학에서 변화실패의 잦은 원인으로는

변화이론에 무지하거나 인간행동의 변화를 지나치게 단순하게 바라보는 ‘변화촉진자 측의 미숙’, 

총론에는 찬성을 얻지만 각론에서 반대에 부딪치는 ‘비전 구체화의 실패’, 

설득과 동참에 걸리는 긴 시간을 버티지 못하고 변화를 중도에 포기하는 ‘변화 동력의 조기 무력화’, 

2년 주기로 학장단이 교체되고 새로운 학장단은 전임자들이 한 일을 무력화시키는 ‘후원자의 잦은 교체’ 등을 들 수 있겠다.

5. 의과대학의 조직 특성: 느슨하게 결합된 조직

기업에서의 변화보다도 의과대학이나 교육병원에서의 변화가 더 어려운 것은 의과대학과 교육병원의 독특한 조직특성 때문이다. 의과대학이나 교육병원은 조직이론가들이 말하는 ‘느슨하게 결합된 조직(loosely coupled organization)’의 전형적인 예이다[13]. 느슨하게 결합된 조직은 조직을 구성하고 있는 하위조직 단위들이 상대적으로 강한 자율성을 갖고 있어서 시스템 내 한 단위가 다른 단위에 별로 혹은 전혀 영향을 미치지 못하거나, 때로는 정반대로 예상 밖의 강력한 반응을 촉발하는 특성을 갖고 있다. 느슨하게 결합된 조직에서는 전문화, 세분화를 지향하는 힘에 비해 조직 통합의 힘(전체로서의 정체성, 전체로서의 미래에 대한 고민)이 훨씬 더 약하다.

때문에 권위가 상부로부터 권한을 위임받은 구성원으로부터 나오는 것이 아니라 조직구성원으로부터 나온다. 따라서 느슨하게 결합된 조직을 이끄는 데에는 치밀하게 결합된 조직(closely coupled organization)을 이끄는 것과는 다른 접근이 필요하다.

이런 조직 특성 때문에 의과대학에서의 변화는 상명하달의 위계적 조직의 변화와는 달리 일종의 캠페인적 성격을 가질 수 밖에 없다. Gilmore에 따르면 성공적인 캠페인은 아래 4가지 요건을 갖추어야 한다[14].

1) 조직에 귀를 기울여야 한다(Listen to the Institution).

미래가(아직 완전한 형태로 드러나지는 않지만) 이미 조직 내에 존재한다는 것을 믿고 조직 내에서 파일럿 사례를 찾아야 한다. 이미 잘 작동하고 있는 혁신의 사례를 발견하고 조직 내 혁신의 리듬에 귀를 기울여야 한다. 또한 갈등과 긴장에 귀를 기울여야 한다.

2) 널리 공명할 수 있는 테마를 찾아야 한다(Develop a theme that resonates and mobilizes).

사람들을 스스로 동요하게 하고 새로운 방향을 제시할 수 있는 테마를 찾아야 한다. 조직에 귀를 기울임으로써 테마를 발견할 수 있다. 좋은 테마는 창발적이고 개방형(open-ended)이며 사람들을 자발적으로 끌어들인다.

3) 사람들을 끌어들여야 한다(Sweep people in).

이미 변화를 실행하고 있는 사람들을 찾아야 한다. 만남을 새로 만들려 하지 말고 기존의 만남과 포럼을 활용한다. 사람들의 에너지를 포착하고 이것을 극대화하여 구성원들과 연합해야 한다.

4) 인프라를 구축해야 한다(Build the infrastructure).

전략회의실을 만들고 신속하게 대응해야 한다. 종자돈과 같은 재정적 인센티브를 제공하고 명료한 커뮤니케이션 전략을 개발해야 한다. 또한 기존의 조직구조를 효과적으로 활용해야 한다.

Leadership Challenges in the Advancement of Medical Education 의학교육 변화의 리더십

신좌섭 서울대학교 의과대학 의학교육학교실 Jwa-Seop Shin, M.D., Ph.D. Department of Medical Education, Seoul National University College of Medicine, Seoul, Korea

Abstract Constant change is inevitable in medical education. Medical education is continually influenced as medical schools adapt to new environments, as the society redefines the role of doctors, by ongoing advancements in medical practice, and as educational theory and practice continues to improve. In addition, the external stakeholders such as consumers, government, and accreditation bodies and internal stakeholders such as professors and students are seeking for changes in medical education. Developing an adequate response to the ongoing change in medical education is not easy. Making changes in a complex system like medical education has been compared to ‘moving a graveyard’. In order to facilitate successful adaptation to the evolving social and educational parameters involved in medical education, leadership would benefit greatly by the study and application of change management theory that has proven successful in corporate manage ment. A number of authors have suggested Leadership Challenges in the Advancement of Medical Education that ‘in loosely coupled organizations like medical schools, a campaign approach is more effective than a general change management approach’. To make the campaign approach successful, change leaders in medical education need to be facilitative leaders who can stimulate and guide constructive dialogue between faculty members and students, and who can promote a sense of ownership of the ongoing changes developing in the consultations between the internal stakeholders comprising the professors and students.

의학교육학회의 발자취

가천의과학대학교 의학전문대학원

백상호

1. 학회 설립 이전의 의학교육 상황

1970년대는 우리나라에 두 가지 개념, 즉 전통적으로 지켜오던 의학교육과 새 개념의 의학교육이 처음으로 마주치던 시기였다. 그 결과 의학교육을 담당하던 사람들 사이에 자연스럽게 그 흐름에 공감하는 그룹이 생겼는가 하면 그런 변화에 대하여 거부감을 가지는 보수적 그룹도 생겨나게 되었다. 생각의 차이는 학자들의 속성인 고집과 주장으로 명확히 나타나게 되었으며 새 트렌드가 현실화되어 교육 현장에 자리를 잡기에는 어려움이 많았다. 그 때나 지금이나 의학교육을 실행으로 옮기는 것은 교수 개인이고 그것을 총괄하여 방향을 설정하는 총책임자는 각 대학의 학장이다. 새 개념 도입의 결정 여부, 방향 전환의 칼자루는 학장이 쥐고 있었다. 대체로 학장 그룹은 보수적인 전통 지지자였다. 그러나 몇몇 학장은 그 때부터도 매우 앞서가는 생각을 가지고 있었다. 새로운 의학교육 흐름 속에는 커리큘럼 개선, 새 교수법의 연수, 의학교육 연구, 타당성 있는 학습 평가와 능력 검증, 교육 환경에 대한 기관 자체의 객관적인 평가 등 할 일이 산 같이 쌓여 있었다. 학회 설립 이전에는 학장모임이 유일한 의학교육 관련 기구였다. 따라서 교육개선을 위한 연구, 실행을 위한 정보교류, 발표도 학장 모임 외에는 달리 주관할 주체가 없었다. 그러나 그 때는 그런 분위기가 성숙되기에는 조금 이른 시기였기 때문에 주제로 다루어지지도 않았고 모르기도 하였지만 관심도 매우 낮았다. 힘의 차이, 생각의 차이, 설득력의 부족 등으로 새 개념의 도입 과정에는 시련과 정신적 갈등이 불가피하였다. 그런 가운데서도 한 때는 학장모임이 강한 리더십으로 새로운 교육 패턴으로의 변화를 유도하기도 하였던 시절이 있었고 리더십 중단으로 과거로 회귀하는 경우도 생겼다. 그런 과거를 가장 잘 보여주는 흔적이 학장 모임의 변화와 이에 따르는 학회의 역사 속에 남아있다.

2. 학장모임의 변천과 학회

우리나라 의과대학 학장모임(Deans meeting: 너무 이름이 다양하여 연대를 초월하여 이렇게 부르고자 한다)의 목표는 연대별로 달라졌고 따라서 모임의 이름도 달라졌다. 50년 사이에 모임의 이름이 여러 번 바뀐 것과 1800년대에 수립된 The Association of American Medical Colleges (AAMC)가 아직도 그 설립 당시의 이름으로 활동을 하고 있는 것과는 대비가 된다. 

학장 모임은...

• 1960년대에는 정보교류, 친교였고, 
• 1970년대에는 강한 리더십으로 한 때 새 개념 도입에 선봉으로 나섰다. 그러나 
• 1980년대에 들어와 초기 이념을 잃고 표류하다가 과거로 회귀하였다. 
• 1990년대에는 리더십 상실로 이견, 대치, 긴장의 경과를 보였고 
• 2000년대에 들어서서는 복잡한 환경 속에 학장모임이 교육 개선의 중심 축 역할을 해야 한다는 생각에서 새로운 견인차 역할의 시동을 걸었다. 

지나간 50년의 학장모임의 역사적 이벤트를 요약하여 한 마디로 표현한다면 awakening, rising, fluctuating, pioneering으로 나타낼 수 있다.

3. 의학교육 관련 기관의 설립과 학장모임

새로운 의학교육 관련 기구 탄생은 불가피한 시대적 소명이었고 학장 모임은 새 기관과는 긴장 상태를 가지면서도 탄생에 많은 힘을 실어주었다. 새로 생긴 기관은 10년 단위로 네 기관이 된다.

1. 1975년: 의학교육연수원(National Teacher Training Center, NTTC), 교수 연수
2. 1983년: 한국의학교육학회(The Korean Society of Medical Education, KSME), 연구, 개발
3. 1992년: 한국보건의료인국가시험원(National Health Personnel Licensing Examination Board, NHPLEB), 의사면허시험
4. 2003년: 한국의학교육평가원(Korean Institute of Medical Education and Evaluation, KIMEE), 평가, 인증

첫 기구 설립은 의학교육연수원이었다. 

• 1970년대에 학장모임이 강한 리더십을 발휘할 때 전체 결의에 의해 만들어진 것으로 새로운 개념의 교육 실행을 위해서는 지속적이고도 전문적인 연수 업무가 시급히 필요했으며 학장모임이 이것을 수행하기에는 벅찼고 이에 따라 일을 분업화함으로써 더욱 발전시키는 계기를 마련하고자 탄생시킨 것이다. 
• 그 후 1983년에는 의학교육학회가 자생적 설립을 하였고, 
• 1992년에는 졸업생의 자격검증 수준을 바로 잡기 위해 의사국시원이 생겼으며, 
• 2003년에는 의학교육평가원이 설립되었다. 

이 모든 기관은 의학교육에 관심을 가졌던 교수들의 규합에서 시작된 것이다. 자생적인 설립이었지만 기관 운영을 하는 이사회에는 반드시 학장모임의 대표를 참여하도록 하는 것을 잊지 않고 시행에 옮겼다. 네 의학교육 관련 기관은 학장 기구와는 어떤 의미에서는 매우 긴장되게 하는 성격을 띠었지만 의과대학의 교육을 발전시키는 데 간접적인 자극제 역할을 하여 의학교육을 오늘의 수준까지 끌어올리고 발전시킨 것은 엄연한 사실이다. 비록 각 기관의 목표는 달랐지만 (...) 궁극적으로는 한결같이 ‘좋은 의사 교육’을 하도록 돕는 것을 공통된 목표로 하고 있으며 입학에서 졸업까지의 여러 과정에서 의학교육(커리큘럼, 교수, 학생, 시설, 재정 등)의 개선을 위해 노력했다는 점에서 그 업적이나 활동을 인정받고 있다. 다만 의학교육 관련 기관마다 시기를 달리하고 별도로 성장하는 과정에서 다섯 기관 사이에는 목적은 달리 명시했으나 기능의 중복도 나타났고, 소통의 소홀로 인한 긴장 상태, 경쟁심도 생겨났다. 모든 기관은 올바른 교육 실행을 도우려는 초기 목표를 잊어서는 안 되고 그런 점에서는 학장모임과 긴밀한 접촉이 꾸준히 있어야 했다. 다만 학장모임이 지속성 면에서 취약하고 업무의 단절이 자주 생기는 현실은 학장모임이 다른 기관의 빠른 발전에 밀리는 현상을 빚게 했다. 이제는 다른 의학교육 관련 기구를 ruling 한다는 권위적 자세를 접고 다른 어느 업무보다 우선하여 학장모임이 선도적 역할을 할 수 있도록 스스로 개선할 과제이다. 어제의 학장이 내일은 기관에서 일을 하는 회원으로 될 수도 있고 그 회원이 다시 학장모임의 일원으로 참여할 수도 있다는 점에서 모두가 단일 공동체라는 의식의 변화가 앞서야 한다. 때로는 속도의 조절도 절실히 필요하고 언젠가는 궤도 수정을 하고 조율해야 할 중요 과제로 보이며 이를 위해서는 끊임없는 의사소통이 있어야 한다.

4. 의학교육학회 설립의 배경과 경과

학회가 태어나기 이전부터 오늘에 이르기까지의 배경과 경과를 기술하자면 대략 아래와 같이 요약할 수 있다.

• 1) 의학교육의 새 패러다임을 감지(1960년대)
• 2) 비슷한 경험을 한 동료들의 공통 화제에서 시작(1970년대)
• 3) 학장협의회 전문위원으로 활동하며 공감 유도(1970년대)
• 4) 18회에 걸친 의학교육 세미나, 보고서 발간 유지(1970 년대)
• 5) 의학교육 활동 기능을 분업화하도록 의견 규합(1980년대)
• 6) 학장회의와의 공감 형성에 부진 상태 생겨 타개책 강구(1980년대)
• 7) 학장회의 목표가 과거로 회귀(친목, 교류)(1983)
• 8) 학장협의회 주관 전국적 세미나, 발간 중단(1984)
• 9) 학회 탄생, 회장단과 임원진 구성(1983)
• 10) 설립 초기부터 활동 정지, 휴면 상태로(1983~1989)
• 11) 창설 6년 뒤 학회 재건, 활동시작(1989)
• 12) 많은 교수의 동조 공감으로 빠른 발전(1989~현재)

이처럼 학회는 복잡한 환경, 답답한 감정 대립 속에서 탄생 성장하였다. (...) 일단 학회가 정상 궤도를 달리게 되었을 때 학회 중요 활동인 학술대회는 역사의 변화에 따라 이 역시 변동을 보였다. 

• 처음에는 봄·가을 두 번에 걸쳐 학회 주최(1989~1994)로 시작하였으나 
• 냉담하던 학장모임이 마음을 바꾸어 봄·가을 학장협의회와의 합동으로 하자고 제안하여 공동주최(1994~2006)로 바뀌었고 
• 그 뒤 의학회가 합류하여 지금은 연 1회 봄에 합동학술대회가 열리게 되었다. 

주최를 어느 기관에서 하느냐의 문제는 예산, 관리 능력, 기획력, 지속성 등 여러 문제가 있었기 때문에 이것 역시 복잡한 경과를 밟게 된 것이다. (...) 회원들의 강한 의지, 섭섭함, 욕심, 열정이 뒤섞인 복잡한 감정 속에 무모하기까지 한 자세로 학장그룹과 대립하였던 점은 다시는 되풀이되지 말아야 할 잘못된 일로 여겨진다. 아무리 새 개념의 전파 보급이 빠르게 이루어졌다고 해도 대학에서의 실행 단계에 턱이 높아지면 헛일이 된다는 점에서 학장모임과의 관계 중요성을 1차적으로 고려했어야 했다는 아쉬움이 생긴다. 상대적인 입장의 학장모임도 역사를 돌이켜보며 새로운 방향으로 전환하지 않으면 더 어려운 처지가 닥치게 될지도 모른다. 많은 의학교육자들은 학장이 자율적으로 앞장서서 더 많이 공부하면서 교수들을 이끌어 가는 견인차 역할을 해 주었으면 하는 것이 간절히 바라는 바였다. 실제로 모든 관련 기관은 학장 모임과 불가분의 관계에 놓여 있고 모든 활동은 의대 교육을 개선하기 위한 것이므로 소통과 교류 그리고 협조가 절대적임을 뒤늦게나마 뉘우치게 되었다. 이제는 그것이 원활하게 돌아갈 수 있는 방법을 찾아야 할 때가 왔다.

5. 설립 이후의 활동 중단 사연

그토록 갈망하던 학회가 1983년 설립된 이후 제대로 활동 한번 못하면서 6년을 보냈다. 그 사연은 과연 무엇이었을까? 천신만고 끝에 발족한 학회에는 초대 회장과 부회장을 포함한 임원 7명, 평의원 25명이 있었다. 그러나 이들 초기 임원 중 많은 수의 멤버에게 같은 해 신상 변동이 있었다. 가장 직접적인 원인은 회장이 지방 대학 학장으로 부임하게 됨에 따라 서울에서 중심을 잡고 활동하던 때와는 달리 신설 대학의 ‘올인’해야 할 학장 업무에다 학회 활동의 시동이라는 두 가지 작업을 감당하기 어려웠던 것이 결정적이었을 것으로 본다. 불행하게도 초대 회장은 2년 만에 급서를 하게 되었고 리더를 잃은 나머지 멤버도 의욕을 잃게 되었으며 그런 가운데 초기 멤버 여러 사람이 비슷한 때에 학장, 병원장, 부원장, 의료원장 등으로 부임함으로써 차질은 점점 커지고 말았다. (...) 이 6년의 공백에서 얻은 교훈은 리더십의 중요성, 사람 사이의 감정 조정이 절실히 필요했으며 의욕은 맨 주먹으로도 가능했으나 현실은 최소한의 재정이 필요했다는 사실이다.

6. 맺는 말

학회의 발자취 성찰을 통해 얻은 교훈을 정리하자면 다음의 여섯 가지로 정리된다. 

1) 의학교육 관련 기관은 시간을 나타내는 시계의 톱니바퀴 관계이다, 

2) 각 기관 업무에는 전체적인 구심점이 필요하다, 

3) 힘겨루기는 톱니바퀴의 기능에 장애를 가져올 뿐이다, 

4) 어떤 기관이든 일은 권위, 힘보다는 능력과 포용력으로 추진해야 한다, 

5) 중복되는 업무는 경쟁심을 가져오지 않도록 정리가 시급하다, 

6) 학회 일을 배가시키기 위해서는 인력 인프라의 확대가 시급하다 등이다. 

다만 수준 낮추기의 하향평준화는 절대 있어서는 안 되며 속도 조절로 나란히 가도록 해야 한다.

Korean J Med Educ > Volume 24(1); 2012 > Article

교육과정 통합의 열한 단계 -교육과정 기획과 평가의 도구로서-

The integration ladder: a tool for curriculum planning and evaluation

Ronald M Harden

Eleven steps in the ladder

The integration ladder is shown in Fig. 1. It has 11 steps from subject-based to integrated teaching and learning. The ladder builds on previous descriptions or models of integrated curricula, notably the work of Jacobs,6 Fogarty7 and Drake.8 

• In the first four steps on the ladder, the emphasis is on the subjects or disciplines. 
• Moving up the ladder, the following six steps emphasize integration across several disciplines. 
• In the final step, the student takes more responsibility for the integration and is given the tools to do so.

Step 1 Isolation (고립)

(Synonym – fragmentation, anarchy)

The first step is ‘isolation’. Departments or subject specialists (represented by squares in the diagram) organize their teaching without consideration of other subjects or disciplines. Each discipline looks, from the perspective of their own discipline, at the curriculum content in terms of areas to be covered, depth of coverage, sequence and timing. No attention is paid to other, or related subjects which contribute to the curriculum.

The slots in the timetable are labelled with the name of the subject, which is taught by specialists in the discipline. Each subject is seen as an entity in itself. The objectives are seen as mastery of the subject and these are tested in a subject-based assessment of the student’s knowledge and understanding of the subject. The relationships between subjects are not explicitly covered and related topics from two disciplines are not intentionally correlated.

This ‘isolation’ approach may be found in the traditional medical curriculum with blocks of time allocated to the individual disciplines. Students attend a lecture on anatomy, and then move on to a lecture in physiology with neither lecturer being aware of what was covered in the other lecture.

Step 2 Awareness (인지)

The second step is ‘awareness’. As with ‘isolation’, the teaching is subject-based. Some mechanisms are in place, however, whereby the teacher in one subject is made aware of what is covered in other subjects in the curriculum. This can be achieved through appropriate documentation and communication about the aims and objectives of each course and the content and topics covered in lectures and other teaching sessions.

Lecture notes or handouts may be circulated to other course teachers as well as to students. Given this information, the teacher can take account of what colleagues cover in other parts of the course when planning his or her teaching, avoiding unnecessary duplication or redundancy and cross-referring, where appropriate, to other parts of the course. At this step, however, there is no explicit attempt to help the student to take an integrated view of the subject.

Step 3 Harmonization (조화)

(Synonym – connection, consultation)

In harmonization, teachers responsible for different courses, or different parts of the same course, consult each other and communicate about their courses. The consultation process takes place through informal discussions between teachers or through more formal curriculum planning committees and meetings. The consultation may involve individual teachers or groups of teachers. The process of consultation may be overseen by a member of staff who has some overall responsibility for the curriculum and who has, as his or her remit, the facilitation or organization of discussion between teachers from different subjects. This consultation process encourages teachers to adapt their programmes so that each course makes an appropriate contribution to the curriculum and the overall curriculum objectives are more likely to be achieved.

Fogarty7 has described this stage of integration as ‘connection’. The disciplines remain separate but the teacher may make explicit connections within the subject area to other subject areas – connecting topics in one session to later or earlier sessions. ‘The key to this model’ suggests Fogarty ‘is the effort to deliberately relate curricula within the discipline rather than assuming that students will understand the connections automatically.’

Step 4 Nesting (내포화)

(Synonym – infusion)

‘Nesting’ is the fourth step of integration. It has been used by Fogarty7 to describe an integrated approach where the teacher targets, within a subject-based course, skills relating to other subjects. Content drawn from different subjects in the curriculum may be used to enrich the teaching of one subject. The term ‘infusion’ has also been applied to this stage of integration where teachers ‘analyse the separate subject’s goals and identify ways in which these generic skills can be refined into existing subjects’.9

An example of nesting is a pathology course which introduces aspects of clinical medicine to demonstrate the application of pathological principles, and where students develop problem-solving skills. Another example is the integration of health promotion and disease prevention into a number of major courses in the curriculum.10 This created an environment in which students could experience their learning about disease prevention in the same manner that they should practice it – integrated throughout clinical medicine. It also avoided the necessity of addressing yet another isolated course to an already crowded curriculum.

In nesting, the individual subjects or disciplines recognize the broader curriculum outcomes and relate their teaching programme to these.11 They may do this by arranging content specific inputs from other parts of the course into the subject’s teaching programme, and by recognizing the generic competencies, such as communication and problem-solving skills, to be acquired in the programme. The teaching, however, remains subject-based and the course is the responsibility of and in the control of the subject or discipline.

Step 5 Temporal co-ordination (수업 시간 배치 조율)

(Synonym – parallel teaching, concurrent teaching)

In temporal co-ordination, each subject remains responsible for it’s own teaching programme. The timing of the teaching of topics within a subject, however, is done in consultation with other disciplines. The timetable is adjusted so that topics within the subjects or disciplines which are related, are scheduled at the same time. Similar topics are taught on the same day or week while remaining part of a subject-based teaching programme. Students study the concepts of the different subjects separately, and are left themselves to uncover the relationships. This approach has been described also as ‘parallel’ or ‘concurrent’ teaching.

In a basic medical science programme with temporal co-ordination, physiologists address the subject of the function of the heart at the same time as the anatomists look at the structure of the heart. Students are left to make the links between the two subjects but this is facilitated by the timetabling, with the heart being examined from the two perspectives in the same time-frame. In contrast, in a more traditional course, students might study the function of the heart in physiology, while looking at the same time at the structure of the head and neck in the anatomy course.

Programmes described as ‘integrated teaching programmes’ are often, in practice, programmes which are temporally co-ordinated. The implementation of a temporally co-ordinated programme introduces some of the advantages of integrated teaching and is a good stepping off point for a more integrated curriculum.

Step 6 Sharing (공동 운영)

(Synonym – joint teaching)

Two disciplines may agree to plan and jointly implement a teaching programme. The ‘shared planning and teaching takes place in two disciplines in which overlapping concepts or ideas emerge as organising elements’.7 The two disciplines which come together to offer such a programme are usually complementary subjects and the joint course produced emphasizes shared concepts, skills and attitudes. The focus of the course is usually in these shared elements.

An example of a shared programme is a course in community child health run jointly by a department of child health and a department of general practice. Another is a course on behavioural sciences run jointly by the department of psychiatry and the department of public health.

The impetus for shared programmes often comes from the subjects or departments themselves, through the identification of common areas of teaching or the need to include a new topic in the curriculum. The departments appreciate that together they can teach the subject better, more effectively and more efficiently, than either could alone.

Unlike temporal co-ordination which may be a step towards a more fully integrated overall programme, shared programmes are often seen as ends in themselves. They tend to be perceived as special cases which, even if they are successful, are not necessarily examples to be followed in other parts of the curriculum. Occasionally, however, this is not so and a shared programme may be a step towards more complete integration.

Step 7 Correlation (상호 연관짓기)

(Synonym – concomitant programme, democratic programme)

In the ‘correlation’ step of integration, the emphasis remains on disciplines or subjects with subject-based courses taking up most of the curriculum time. Within this framework, an integrated teaching session or course is introduced in addition to the subject-based teaching. This session brings together areas of interest common to each of the subjects.

An example of correlation is a basic medical science programme where students study topics, such as the gastrointestinal system first from the perspective of each of the subjects, and then meet on Friday afternoons for an integrated session. In this session, the discussion may focus round a patient who illustrates aspects of normal structure and function considered during the week. The contributions of the different subjects are used to illuminate the problem. Another example of correlation is a subject-based programme in which the project or assignment given to students, is designed to integrate the subjects. The students may be required to submit a written assignment or to present a report on the project at an integrated plenary session.

Step 8 Complementary programme (보조적 프로그램)

(Synonym – mixed programmes)

The ‘complementary’ approach has both subject-based and integrated teaching. The integrated sessions now represent a major feature of the curriculum. These sessions are recognized to be, in terms of time, allocated resources and assessment as important, if not more important, than the subject-based teaching.

The focus for the teaching may be a theme or topic to which the disciplines can contribute. This is discussed further in the following sections. Running alongside the integrated teaching are scheduled opportunities for subject-based teaching.

The implications of the approach for assessment are important. Examinations need to reflect the emphasis on both integration and subjects or disciplines.

Step 9 Multi-disciplinary (다학제)

(Synonym – webbed, contributory)

A multidisciplinary approach brings together a number of subject areas in a single course with themes, problems, topics or issues as the focus for the students’ learning. The themes selected as the focus in an integrated course may function in different ways.12

The themes can delineate an area in which practical decisions have to be made and which serve as a focal point of interdisciplinary thinking. Problems and the tasks to be undertaken by the professional may also be used as a focus for integrated teaching. The task may be the management of a patient with abdominal pain, screening for diabetes or the mounting of a coronary artery disease prevention programme. In task-based learning,13 the learning is concerned not only with mastery of the tasks but with learning related to the tasks, including an understanding of the relevant basic and applied medical sciences. A course for teachers of healthcare professionals, at the Institute of Public Health in Malaysia, covers educational theory including curriculum planning, assessment, learning theory and instructional design. This theory is taught in an integrated manner round the practical tasks which will confront the teacher on completion of the course. These include small group teaching, lecturing and teaching practical skills.

The theme in a multidisciplinary programme may be a structured body of knowledge that needs to be mastered but which transcends subject boundaries. The systems of the body are used frequently as an integrating theme.3 Courses are developed round the cardiovascular system, the respiratory system, the nervous system and so on. 

In the thyroid module of the endocrine system block, for example, 

physiology may contribute to thyroid hormone synthesis and its regulation, 

pathology to the underlying disease processes, 

pharmacology to the action of anti-thyroid drugs, 

surgery to the management of goitre, and 

medicine to the clinical manifestations and investigations of thyroid disease. 

The stages of the life cycle from conception through birth, childhood, adolescence, adulthood, the elderly to death may be used as an alternative to the body systems theme.

Finally, the theme may be a complex of information and skills which are relevant to medicine. Clinical methods, ethics and health promotion are examples.

The characteristic of multidisciplinary integration is that, whatever the nature of the theme, it is viewed through the lens of subjects or disciplines. The theme or problem is the focus for the student’s learning but the disciplines preserve their identity and each demonstrates how their subject contributes to the student’s understanding of the theme or problem. ‘A discipline’, suggested Drake,8‘is easily identifiable within the teaching strategy, and the discreteness of the procedures of the discipline can be kept intact by the teacher who will probably approach the task from her own area of specialisation’. In multidisciplinary teaching, the contributions of the individual disciplines to the theme are stated implicitly in the curriculum documents and the timetables. In the multidisciplinary step on the integration ladder, however, the subjects and disciplines give up a large measure of their own autonomy.

The term ‘webbed’, was used by Fogarty7 to describe this stage of integration. ‘A fertile theme is webbed to curriculum contents, and disciplines or subjects use the theme to sift out appropriate concepts, topics or ideas.’

Step 10 Inter-disciplinary (학제간)

(Synonym – monolithic)

In interdisciplinary integration there is a further shift of emphasis to themes as a focus for the learning of and to the commonalties across the disciplines or subjects as they relate to the theme. Jarvis14 defines interdisciplinary as ‘a study of a phenomenon that involves the use of two or more academic disciplines simultaneously’. In the taxonomy proposed in this paper, interdisciplinary teaching implies a higher level of integration, with the content of all or most subjects combined into a new course with a new menu.15 In the interdisciplinary course there may be no reference to individual disciplines or subjects, and subjects are not identified as such in the timetable.

Implicit in the move from a multidisciplinary to an interdisciplinary approach may be the loss of the disciplines’ perspectives.

Step 11 Trans-disciplinary 

(Synonym – fusion, immersion, authentic)

Alfred North Whitehead wrote in 1929 ‘The solution which I am arguing is to eradicate the fatal disconnection of subjects which kills the vitality of our modern curriculum. There is only one subject-matter for education, and that is Life in all its manifestations.’ In trans-disciplinary, as in interdisciplinary integration, the curriculum transcends the individual disciplines. The focus with trans-disciplinary integration for learning, however, is not a theme or topic selected for this purpose, but the field of knowledge as exemplified in the real world. The teacher provides a structure or framework of learning opportunities, but the integration is done in the mind of the student, based on hi-fidelity situations in the real world of clinical care.

Trans-disciplinary education is reflected in learning described by McCombs17 as ‘an individual process of constructing meaning from information and experience, filtered through each individual’s unique perceptions, thoughts and feelings.’

An example of trans-disciplinary integration is the final phase of the medical curriculum at Dundee. 

The curriculum in the first three years of the students’ studies is integrated round the body systems.18 In the last two years, students are attached for periods of time to a range of specialties in the hospital and in the community and experience the various contexts in which medicine is practised. 

• A set of 113 clinical problems or tasks provides the students with a framework for integrating their experiences.19 
• Students look at each of the tasks from the perspective of the different attachments. 
• Abdominal pain can be taken as an example. 
• Students have an acute surgical perspective in their surgical attachment, and different perspectives in the medical attachment, in the gynaecological attachment and in their community attachment in general practice. 
• A printed or electronic study guide20 is a key element in helping the student with the challenge of integrating these different experiences.

Another example of trans-disciplinary integration is the third year programme of integrated clerkships in ambulatory–care settings at South Dakota.21 

• The students spend the year attached to clinics staffed by physicians from various specialties. ‘The student is not “on” a given specialty for a block of time; rather the required specialties are simultaneous and integrated throughout the year.’ 
• Students have the entire year to achieve the course objectives at their own speed and by their own methods.

Thus, in a trans-disciplinary approach the disciplines become part of the learner’s real world experience and through these they filter the broader aims and goals of the integrated curriculum. In this environment, the learner is driven to find out as dictated by the prescribed tasks. This stage of integration has been termed ‘authentic’ integration, reflecting that the learning occurs in the real world. The term ‘fusion’ has also been applied to this stage. As the student learns, he or she integrates internally and intrinsically and completes the mastery of the competencies related to the task. Fogarty7 uses the term ‘immersion’ when ‘disciplines become part of the learner’s experience and through these filter the broader aims and goals’.

Discussion and conclusions

교육과정 통합은 의학교육에서 중요한 전략이지만 복잡한 개념이기도 하다. 

Curriculum integration is an important strategy in medical education but is a complex concept.12 This paper attempts to clarify the concept by presenting a taxonomy which defines 11 steps between the two extremes of subject-based and integrated teaching.

통합의 사다리를 한 단계 올라갈수록 개별 학문의 역할에 대한 강조가 줄어들고, 적절히 자원을 배분할 수 있는 중앙의 교육과정 조직 구조가 필요해지며 staff들의 참여가 더 많아져야 한다.

As one moves up the integration ladder there is less emphasis on the role of the disciplines in the curriculum, an increasing requirement for a central curriculum organizational structure with appropriate resources at its disposal, and a requirement for greater participation by staff in curriculum discussions and planning.

또한 사다리를 올라갈수록 의사소통이 더 중요하며, 다른 과목을 가르치는 교수자간의 협동적 계획이 중요하다. 교육 프로그램의 개요부터 가르치는 순서, 목적과 목표, 세부사항, 평가법 등에 대한 협의가 필요하다.

The higher up one goes on the integration ladder, the more important is the communication and joint planning between teachers from different subjects. Agreement between departments may be required concerning the outline of the teaching programme, the sequence of the teaching, the aims and objectives of the programme, the details relating to content and the method of student assessment.

시간표나 syllabus는 교육과정 통합이 어느 정도로 이뤄졌는지 보여주는 지표이며, 통합의 수준이 높을수록 개별 학문은 덜 강조된다.

The published timetable or syllabus will usually give an indication of the level of integration in the curriculum. The higher the level of integration, the less prominence will be given to disciplines.

통합의 사다리는 의학을 가르치는 교수들이나 의학교육자들에게 다양한 상황에서 도움이 될 것이다. 교수자들은 통합의 가치에 대해서는 동의하면서도 그 균형을 어디에 둘 것인가에 대해서는 의견이 다를 수 있다. 다양한 범위의 옵션을 제공하는 것은 논쟁이 과열되는 것을 막을 수 있다. 통합의 사다리는 선택의 다양한 메뉴를 제공함으로써 교수들로 하여금 교육과정에 가장 적합한 범위나 형태의 통합을 찾아낼 수 있도록 도와준다. 

The ‘integration ladder’ is a useful tool for the medical teacher or educator and can help in a number of situations. Teachers may agree about the value of integration, but may differ in their views as to the optimum balance between integrated and subject-based teaching. The demonstration of the range of options helps to avoid a polarization in the debate. The ‘integration ladder’, by setting out the menu of choices, encourages teachers to explore the integration options available and to discuss the extent or form of integration most appropriate in the curriculum. Such informed decision making is preferable to a debate, usually sterile, as to whether the curriculum should be integrated or not.

한 의과대학에 있어서 통합의 사다리에서 가장 적절한 단계가 어디인지는 다양한 요인에 영향을 받을 것이다. 전통적인 학문중심의 교육과정에서 통합 교육과정으로 옮겨가는 것은 상당한 변화이다. 그 대안으로서 correlated teaching approach를 사용하여 소수의 통합된 주제를 중심으로 작고 관리가 가능한 것부터 시작해 볼 수도 있다.

The most appropriate step on the integration ladder for a school will depend on many factors, including the existing curriculum, the experience and views of the teachers, the organizational structure of the medical school, and the overall aims of the curriculum. The move from a traditional subject-based to an integrated curriculum may involve major changes. Alternatively, one can start with something small and manageable such as a few integrated themes using a correlated teaching approach.22

교육과정 계획 외에도 통합의 사다리는 교육과정 평가와 통합의 정도를 평가하기 위하여 사용할 수도 있다.

In addition to its use in curriculum planning, the integration ladder may also be used as a tool to assist curriculum evaluation and to evaluate the level of integration in a curriculum.

The integration ladder: a tool for curriculum planning and evaluation.

프로그램 평가의 구조: 교과목, 임상실습, 전공의 또는 펠로우 수련 프로그램 평가

The Structure of Program Evaluation: An Approach for Evaluating a Course, Clerkship, or Components of a Residency or Fellowship Training Program

Steven J. Durning a , Paul Hemmer a & Louis N. Pangaro a 

a Department of Medicine , Uniformed Services University of the Health Sciences , Bethesda, Maryland, USA

교과목, 임상실습, 전공의수련 관리자들은 개별 수련자(trainee)들이 교육 목적을 달성했는지 뿐만 아니라 그 프로그램 자체의 질도 담보할 수 있어야 한다. "프로그램 평가"란 개별 수련자들의 정보를 모으는 것을 넘어서 관리자로 하여금 다양한 요인과 성과 지표를 역동적, 지속적으로 평가하도록 한다.

Directors of courses, clerkships, and residencies are responsible not only for determining whether individual trainees have met educational goals but also for ensuring the quality of the training program itself. This “program evaluation” is more than the aggregate of individual trainee data; it requires academic directors to employ a dynamic, longitudinal evaluation process that tracks multiple contributing factors and outcome measurements.

Academic director가 프로그램 평가에 대한 프레임워크를 가지고 있어야 한다는 점은 자명하다.

The desirability and necessity for academic directors to have a framework for program evaluation is evident ...

• ACGME
• LCME

논문의 목표

The purpose of this article is to discuss a framework for program evaluation that has sufficient rigor to satisfy accreditation expectations and still be flexible and responsive to the uniqueness of individual educational programs. (...) Our intent is to demonstrate how practical our framework can be for conducting program evaluation in medical education training programs. (...)

이 프레임워크는 baseline, process, product의 역할을 강조하고 있으며, 프로그램의 성공을 판단하는데 있어서 양적과 질적 정보를 모두 강조하고 있다. 또한 최근 ACGME에서 "outcome"을 강조하고 있으나, 우리는 'process'지표도 강조하고자 한다. 왜냐하면 많은 성과지표는 신롸도와 타당도에서 확실하지 않기 때문이며, 또한 성과라는 것은 궁극적으로는 절차(process)를 정교하게 하기 위해서 필요하기 때문이다. 마지막으로 baseline 측정을 강조하고자 하는데, 이는 성과가 수련자에 따라서가 아니라 프로그램에 의해서 얼마나 바뀌었는지를 확인하고자 함이다.

The proposed framework emphasizes the role of baseline, process, and product (outcome) information, both quantitative and qualitative, for describing program “success.” (...) Nevertheless, despite the recent emphasis of the ACGME on “outcomes,” we emphasize the importance of process measurements (such as the number and kinds of patients seen during training and the level of proficiency obtained), as many available outcome measurements have uncertain reliability and validity and because, eventually, the outcomes should be used to refine the curricular process. (...) Finally, we advocate for the inclusion of baseline measurements, which may allow us to determine how much of an eventual outcome depends on the curriculum, as opposed to the prior characteristics of trainees.

프로그램 평가의 과업 정의하기

Defining the Task of Program Evaluation

프로그램평가에서의 "성공"이란?

In this article, we define “success” for program evaluation (PEv) as achieving information that can relate “inputs” to “outputs” and therefore be used to help understand sources of success or failure. (...) In other words, our goal is to understand how the program is working rather than simple classification of graduates (e.g., competent or not competent).

프로그램평가는 특정 프로그램의 '성공'이 무엇인지를 정하고 시작해야 함. 최소 yes/no 형태로 답할 수 있어야하며, 가능하다면 성공의 정도를 묘사할 수 있어야 한다. 어떤 경우든 목표와 기대치가 분명하고 구체적이어야 하며 실질적이어야 한다. 또한 성공의 일부 지표는 외부(LCME, ACGME)에 의해서 정의되기도 한다.

PEv should begin with a definition of success for the specific program. It is not sufficient to say that “good patient care” or “patient safety” is the goal. The description of success must be construed with sufficiently precise words to allow, at a minimum, the determination of whether success has been met, in a dichotomous yes/no fashion. If possible, it is also desirable to describe degrees of success. In either case, these goals and expectations should be clear, specific, and tangible. (...) However, we all recognize that demonstrating success is often defined externally, by accrediting bodies such as the LCME or ACGME. (...)

목적을 열거한 후에는 목표를 설정해야 한다.

After listing goals, objectives should be constructed for determining success in achieving the goals. (...)

그러나 성공을 정의하는 것은 첫 단계일 뿐이며, 어떻게 그 질문에 답할 것인가를 이해하는 것은 더 어렵다. 교육 프로그램에 대한 총체적인 평가에 대한 문헌이 많지 않다.

However, defining success is only the first step; understanding how to approach the question is more difficult. We have found that relatively little practical guidance exists for systematic evaluation of educational programs; the limited guidance that does exist is restricted to graduate medical education arena. 8 , 9 , 10 Indeed, the research pertaining to program evaluation in medical education is less developed than other educational fields and is largely descriptive. 11 (...)

프로그램평가 프레임워크

PEv Framework Overview

세 단계의 평가를 해야 한다.

We advocate a three-phase framework for program evaluation. This framework allows for establishing relationships among baseline, process, and product measurements—Before, During, and After.(...) All of these measurements have often been based on what the graduate does under testing circumstances (such in vitro measurements would include licensure or certifying examinations), but we also want to include in vivo observations such as what trainees do in patient care and graduates do in their practice.

• Before (baseline) measurements are necessary to determine “how learners change,” and they are especially important to determine the effect of curriculum as opposed to selection of trainees. (...)
• During (process) measurements are those that monitor the activities of learners during the training program (Table 2). These measurements are often collected prospectively, that is, in real time. (...) “During” measurements, therefore, need to be prioritized for program evaluation purposes so that response to critical, potentially unexpected, information is not delayed or potentially overlooked.
• After (product) or outcome measurements, in the clinical research literature are analogous to primary and secondary end points. 
• Primary end points indicate the overall success of management. (...) 
• Secondary clinical end points indicate intermediate success or complications. As the majority of outcomes in medical education are as complex as in clinical studies, data gathering should optimally be done through multiple measurements (triangulation). (...)

절차 측정의 중요성을 강조하고자 한다.

It is evident that...

• we emphasize the importance of process measurements; 
• we believe that we must not ignore systematic process measurements, or baseline measurements, at the expense of focusing on “outcomes” for several reasons: 
• Our current outcomes in medical education are imprecise, 
• process measures are required by regulating bodies (LCME with ED2, ACGME with outcomes project), and 
• process measurements are indispensable to explaining the variance in the outcomes (products) of interest.

세 단계의 프레임워크에서 다양한 척도가 사용될 수 있다. 양적 질적 평가법이 모두 사용되어야 한다.

A variety of measurements can, and should, be used in this three-phase framework (see Table 2). Both qualitative (descriptive) as well as quantitative (numerical scores) assessments can, and in many cases, should be used for program evaluation 14 as the quantitative measurements alone may overlook important findings that are revealed through qualitative analysis. (...)

이 프레임워크는 기존의 프로그램과 새 프로그램 모두에 적용가능하다.

The basic three-phase structure that we propose for program evaluation readily applies to both existing programs and new programs, curricula, or interventions. (...)

각 단계에서 다양한 측정을 하기 위해서는 상당한 자원이 필요하다. 프로그램평가정보는 다양한 의사결정자에게 제공되어야 한다.

Collecting multiple measurements in each phase of our framework can require significant time and human resources. (...) Indeed, program evaluation information should inform multiple decision makers in the program; if this is not the case, then the utility is too limited. (...)

세단계 프레임워크는 의학교육 연속체의 모든 단계를 걸쳐서 협력을 장려할 수 있다.

In addition, our three-phase framework to program evaluation could foster collaboration across the medical education continuum, as the baseline measurements for a 3rd-year clerkship director may comprise the outcome measurements of a 2nd-year course director and the outcome measurements of a clerkship director can serve as baseline measurements for a residency training program director.

Distinctions and Definitions

평가의 이상적 특징들

Next, we define the desirable attributes for the assessment tools that are placed within the model (the micro level) and afterward the desirable attributes of the overall framework (model) for the macro level. Optimally, a tool for assessing the success of a program, is feasible, reliable, and valid. 

미시적 관점에서..

For our purposes at the micro level, 

• feasibility means the percentage of possible measurements that are actually obtained and the unit cost per measurement (i.e., cost of printing, mailing, and/or entering survey data); 
• reliability means the internal consistency of specific assessment tools, and 
• validity is the confidence that the inferences drawn from the data are true. Thus, in our model, validity will mean that process measurements have a significant, meaningful, predictive association or correlation with outcomes measurements and that outcomes measurements have a similar correspondence with patient outcomes.

거시적 관점에서..

At macro level, academic directors optimally should be able to collect the same set of data the same way for each trainee, at each site, each year to help ensure reliable and valid observations collected for program evaluation purposes. 

• The overall method for program evaluation must be feasible—measurements must be obtained effectively (method should allow at least 90% of possible observations about trainees, faculty, etc., to be captured each evaluation time), 
• consistently (observations and ratings are recorded, transferred, and stored without degradation),
• efficiently (no more than 10% of a course, clerkship, program, or fellowship director's time must be consumed, and no more than 10% of an administrator's time is needed), 
• economically (cost of program evaluation should be no more than 5% of the operating budget for the course, clerkship, or graduate medical education program), and 
• securely (trainees are protected from their data being shared, and any risk is minimized).

거시적 관점에서..

At the macro level, this means that each set of measurements (before, during, and after) adequately reflects the construct appropriate to the framework.(...)

프로그램 평가모델이 '타당'하려면, 절차가 성과의 원인이 된다는 논리적 추론이 가능해야 한다.

• For the program evaluation model to be valid, inferences that the process caused, or contributed to, the desired outcome must be reasonable. The usual standard for causality in clinical medicine is the prospective, randomized, blinded trial. This is difficult to achieve in the educational setting, except for subtotal modifications of the curriculum such as restructuring an individual clerkship or changing a 1-month rotation during residency. 
• Therefore, validation of our program evaluation model would mean that proposed explanations for how variance in outcomes (skill of our individual graduates, in the aggregate, in taking care of their diabetic patients) was related to specific curricular elements (whether they actually performed the practice-based learning and improvement review of their own care of diabetics) would require two levels of evidence: 
• initially statistical demonstration (e.g., through correlation or multiple regression models) and 
• subsequently improvement in diabetic care given by residents graduating after further modification in their curriculum. The latter has not been published in the literature.

자원과 관리

PEv Resources and Measurements

We think that it is helpful to define essential and desirable resources for program evaluation at the outset, as this can assist with requests for funding as well as facilitate best use of limited resources. (...)

프로그램평가의 실현가능성

PEv Practicalities

Timing—Identifying Problems Early

The appropriate timeline for data collection, analysis, and reporting can differ based on the academic program. (...)

We believe that a robust formative evaluation process, based on “during” or process measurements, is a critical component of successful program evaluation. For example, program information that might cause concern, such as having a sufficient number of patients for each intern, should be collected, analyzed, and reported more frequently than on an annual basis. These concerns might be categorized as red and yellow “flags” (Table 3). (...)

Program Evaluators

- Internal evaluators 

- External evaluators 

Visualization of Goals and Objectives

It is not sufficient to say that “good patient care” or “patient safety” is the goal. The description of success must be construed with sufficiently precise words or quantifiable measurements to allow, at least, the determination of whether success has been met, in a dichotomous yes/no fashion.

한계점

Limitations of the Approach

• In the three-phase approach, learners serve as their own control, which is not necessarily an optimal design (i.e., randomized approach) to study the benefits of a curricular innovation. 
• Complete data collection may not be feasible for directors of educational programs with limited resources. 
• Success with using this model requires close cooperation from others (registrars, other course and clerkship directors). 
• Data collection may be constrained by local Institutional Review Boards if you wish to use findings for more than quality control in your own program. 
• Further, the model illustrates correlation and not causation. 
• As all factors cannot be controlled, the opportunity exists for multiple confounders with data analysis. 
• Also, many measurements will have not been sufficiently studied to demonstrate reliability and validity in each institution. 
• Despite these limitations, such a framework can be useful to guide efforts to evaluate the program.

(...)However, the use of a similar conceptual model to what we propose in the quality assurance literature does enhance the validity of our approach. 3 , 4 , 5 We do believe that our model can be effective at monitoring educational programs to make them more effective. In our program, we monitored and subsequently minimized intersite inconsistencies. Although this did not necessarily lead to any specific changes, successful program evaluation informs the stakeholders and guides their decision making, whether or not the decisions lead to change. We also propose using red and yellow flags allowing a course, clerkship, residency, and/or fellowship director to identify and remediate potentially harmful curricular and/or teacher anomalies (a definition of quality).

PEv Recommendations

1. Begin with defining the goal: “I would be happy about my program if I knew that …”
2. It is essential to list outcomes measurements for key parameters of success. Use triangulation—collect at least two measurements for each domain. We advocate collecting at least three measurements for each phase.
3. Then list process measurements that you think will lead to successful outcomes.
4. It is desirable to specify baseline measurements that would attribute success to the learner rather than the program.
5. It is desirable to include qualitative information along with quantitative data measurements.
6. Define the needed resources—time, human resources, and money. This will assist with feasibility of program evaluation efforts.
7. Include red and yellow flags to prioritize if unexpected/undesirable process measurements or outcomes are observed
8. Define your unit of analysis. As a principle, we recommend using the unit of analysis that would be most likely to reveal problems.
9. The analysis of data should include measurements of both statistical and functional significance. Decide the functional significance that would constitute success. Decide the statistical significance that would constitute failure.

The structure of program evaluation: an approach for evaluating a course, clerkship, or components of a residency or fellowship training program.

졸업후의학교육(GME)에 대한 프로그램 평가의 개념적 모델

A Conceptual Model for Program Evaluation in Graduate Medical Education

David W. Musick, MA, PhD

GME는 새로운 시대에 접어들었다. 

Graduate medical education (GME) has entered into a new era, one that has been described as a “paradigm shift.”1,2 Accreditation standards adopted in recent years by the Accreditation Council for Graduate Medical Education (ACGME) have stressed the importance of program evaluation as part of an overall shift from a process-oriented to an outcomes-oriented system of education. (...) following statement taken from ACGME accreditation standards: “The program should use resident performance and outcome assessment in its evaluation of the educational effectiveness of the residency program.”3

outcome에 대한 강조는 GME가 어떠해야하는가에 대해서 여러가지 시사점이 있다. 그러나 educational effectiveness는 무엇이며, 그것을 어떻게 측정할 것인가?

This new emphasis on outcomes has many implications for how GME training programs function, and perhaps for how the process of program accreditation site reviews will function in the future. But what is meant by “educational effectiveness” and how does one evaluate it?(...)

(...). It is apparent that there is not a shared definition of the term program evaluation, particularly when it is associated with terminology referring to educational outcomes.4

Evaluation이란 단어가 갖는 의미의 혼동. 

Part of the confusion may be an issue of semantics. The term evaluation is often used interchangeably with the term assessment. And evaluation is used broadly within medical education, and can refer to several distinct processes: 

overall evaluation of training programs as a whole; 

of an individual resident’s performance; 

of faculty teaching; 

or of a given educational lecture, conference, rotation or other learning experience within a training program. (...)

본 논문의 목적

The purpose of this article is four-fold:

• * to briefly review the literature pertaining to program evaluation, both in general terms and in reference to medical education,
• * to present a task-oriented conceptual model of program evaluation,
• * to discuss outcomes evaluation as one type of program evaluation (distinguishing between relevant institutional and program standards), and
• * to provide a five-step process that will assist program directors and/or other medical educators in developing effective ways to use program evaluation data to improve GME training programs.

문헌 조사

Review of the Literature

Evaluation의 정의와 역사적 흐름

The term evaluation is best defined as a process of decision making about the object being evaluated and how it compares to some standard of acceptability.5 (...). Much of the evaluation literature is found within the realm of the social sciences. Evaluation as a distinct discipline gained impetus in the early 1970s as part of the Great Society movement. (...) An early evaluation model described program evaluation as a process where participants agreed in advance on the purpose and design of evaluation procedures, and on how the results would be used.6

교육적 상황에서 프로그램 평가

Program evaluation in the educational setting began to receive more emphasis during the late 1970s and 1980s as a result of increased governmental funding of reform initiatives at all levels of education. (...) Most often, evaluation in an educational setting was conceptualized as a process of making decisions about whether an educational program was meeting its goals and objectives. (...)

의학교육에서의 프로그램 평가

Regarding program evaluation in medical education, the literature pertaining to work done with medical schools and residency programs is less developed than for other fields of education and is largely descriptive.10 Evaluation models specific to GME programs have been nonexistent; there is simply no overarching theoretical base or consistent approach provided whereby GME program directors can determine what is expected in this regard. (...) 

프로그램 평가의 개념적 모델

Conceptual Model of Program Evaluation

List 1 provides a task-oriented conceptual model for evaluation within a GME training program. The central notion of the model is to identify the steps involved in planning and carrying out various types of evaluation, consistent with evaluation best practices and accreditation requirements. 

The evaluation need and focus represent the initial stages of determining why the evaluation is to be done, what or who is to be evaluated and what “rules” or standards will inform the evaluation. 

The evaluation methodology is the stage where procedures are established for how to collect and analyze evaluation data. 

Finally, the evaluation results stage represents the presentation of data to key stakeholders in an established forum (such as an annual program evaluation meeting). and also the written documentation of all steps taken in performing the evaluation as well as decisions made as a result. 

성과평가란 무엇인가?

What is Outcomes Evaluation?

ACGME의 정의

Outcomes evaluation refers to a particular type of program evaluation. It is defined by the ACGME as

"evidence showing the degree to which program purposes and objectives are or are not being attained, including achievement of appropriate skills and competencies by students.11"

다른 평가법과의 차이

The primary distinction between an outcomes-oriented approach and other approaches to evaluation is found in the word evidence. In GME, accreditation reviews have traditionally focused on the process of education.(...) This emphasis on process occurred because of the dominance of the Flexnerian model of medical education and because of the difficulty in defining competence in precise terms within a given discipline.12

최근 six general competencies와 함께 outcome에도 관심을 갖게 됨

In recent years, especially with a shift to an educational framework based on the “six general competencies” of GME, accreditation has now begun to emphasize not only the educational process but also its outcomes, (...)

ACGME는 성과평가에 대하여 아래와 같이 강조하고 있음.

The new emphasis on outcomes evaluation is illustrated by the following statement of the ACGME:

"Assessing the actual accomplishments of a program requires a different set of questions: (1). Do the residents achieve the learning objectives set by the program? (2). What evidence can the program provide that it does so? (3). How does the program demonstrate continuous improvement in its educational processes?13"

성과평가를 위한 두 가지의 교육적 요소가 있음

To illustrate this new emphasis on outcomes evaluation, in the following paragraphs, two common educational components of all GME programs will be considered, one simple and the other more complex.

첫 번째: didactic instruction자체의 process와 outcome을 같이 평가함.

As a simple illustration of documenting an educational program outcome, consider that all ACGME-accredited training programs are required to provide didactic (i.e., lecture-based) instruction in an organized fashion. (...)

Based on an outcomes-oriented approach to accreditation, a site visitor will review the training program’s didactic program focusing on two things: 

• the process of education (e.g., What lectures are provided and when? Lecture topics? Comprehensiveness? Organization?); and, 
• the programmatic outcome pertaining to the didactic schedule (e.g., Did these lectures actually occur? Who showed up for them? Who presented them?). 

The point of this simple illustration is this: under an outcomes approach to program evaluation, both types of information (i.e., process and outcome) will likely be necessary to satisfy accreditation requirements pertaining to resident didactics.

두 번째: didactic instruction이 실제로 individual resident와 program performance를 향상시켰는지 평가함

A second, more complex example will also illustrate the difference between process-oriented and outcomes-oriented accreditation procedures. (...) Is this fact sufficient for accreditation purposes? More than likely, it is not. After all, residents could be attending didactics but still not learning! (...) Answering these questions satisfactorily results in what the ACMGE labels “evidence of how educational outcomes data is used to improve individual resident and overall program performance.”13

성과바탕 접근법에 기반하여, ACGME는 개별 레지던트의 역량을 측정하기 위한 기대 수준을 상당히 올려두었다. apprenticeship model으로 가르칠 수는 있겠지만, faculty와 충분한 시간을 보낸다는 것만으로는 개별 전공의의 역량을 보여주는 것으로 충분치 않다. 

(...)With the advent of an outcomes-oriented approach, the ACGME has considerably raised the expectation level regarding teaching and measuring the competency of individual residents. Put simply, each training program must redesign its curriculum around the six general competencies and must put into place educational assessment procedures that will effectively document that residents’ learning has taken place, and that such learning has positively affected patient care. This will require programs to institute measures of residents’ knowledge, skills and attitudes in a more formal way than has previously been done. While the apprenticeship model will continue to be valuable from a teaching process standpoint, the presumption that residents have gained sufficient clinical competence by spending time with attending faculty over the course of the training program is no longer acceptable for the purpose of documenting an individual resident’s competency under an outcomes-oriented model of accreditation.

GME의 outcome measure

List 2 provides a list of categories of educational outcome measures that can be used by residency directors to document (i.e., provide evidence of) residents’ learning and/or program success.(...)

이러한 달라진 접근법은 GME 수련 프로그램의 '결과물', 의학교육 연속체의 연속성 증대, 그리고 궁극적으로 더 강력한 의료전문직을 강조하고 있다.

The ACGME has stated that this modified approach to accreditation review, which consists of examining both the process and the outcomes of the GME training program, will result in stronger residency training, increased accountability for the “product” of GME training programs (i.e., the competent physician), greater continuity between various levels of the medical education continuum and, ultimately, a stronger medical profession.14

성과 평가는 상당히 맥락의존적이다.

Outcomes evaluation is highly context-dependent, in that the expectations and needs of various constituents involved in the program being evaluated must be considered. (...) A key part of any outcomes evaluation system is to determine what outcomes are to be measured and who is to select those outcomes. In residency programs, measurable outcomes are gradually being added to disciplinary ACGME standards by the respective RRCs. National discipline-specific groups (e.g., specialty certification boards, residency program director associations) may also contribute to discussions about appropriate educational outcomes for GME training programs.

기관별 기준 또는 프로그램별 기준?

Institutional or Program Standards?

또한 성과에 대해서 기관 차원의 기준과 프로그램 차원의 기준이 다를 수 있음도 염두에 두어야 한다.

It must also be remembered that there is a difference between institutional and program standards related to evaluation outcomes. (...)

Sponsoring institution은 teaching hospital과 그 병원의 GME프로그램들이 institutional standard에 신경을 쓰도록 해야 한다.

There are certain institutional standards to which the sponsoring institution must ensure sufficient attention by the teaching hospital and all its GME programs. (...) Institutions are expected to work closely with individual programs to develop educational objectives and methods of measuring educational outcomes in each of the six competency categories (i.e., medical knowledge, patient care, practice-based learning/improvement, interpersonal and communication skills, professionalism and systems-based practice). This dual accountability for educational outcomes means that a more centralized approach to managing GME will be needed to achieve compliance with all applicable ACGME standards. (...)

Applying the Conceptual Model of Evaluation

(...)

Conclusion

In this article, I have discussed how the outcomes approach to education has influenced program evaluation procedures in GME 

• by reviewing the literature; 
• by offering a conceptual model of evaluation that emphasizes systematic, rigorous attention to evaluation need, focus, methods, and results; 
• by distinguishing between programmatic and institutional outcomes; and 
• by outlining a stepwise process of evaluation design and implementation. (...)

이런 새로운 이론적 접근을 활용하여 GME의 독특한 세팅에 어떻게 적용할 것인가에 대한 전문가가 필요하다.

With this new theoretical approach has come a need for additional expertise in how evaluation procedures are applied to the unique setting of GME. (...).

성과 중심으로 옮겨가는 이러한 과정에 비판이 없는 것은 아니다. 

The outcomes movement in education is not without its critics. (...) There is philosophical tension between the need to extensively document educational outcomes via the methods I have described here, and the need to encourage resident physicians to be self-motivated, independent learners. The previous example of documenting lecture attendance is germane here. How can we encourage self-motivated, self-directed adult learning among resident physicians while we take attendance at every lecture as if our learners are still in grade school? (...)

하지만 수년간은 이러한 접근법이 유지될 것으로 보인다. 이미 많은 부분이 성과중심으로 진행되고 있기 때문이다. 

It appears likely that the evidence-oriented approach represented by the educational outcomes movement will remain predominant for years to come. 

• Indeed, the outcomes approach (with particular emphasis on the use of the six general competencies) has also been adopted as a substantial part of continuing medical education 17 and physician certification and recertification 14 processes. 
• It is also prominent within the accreditation process for medical schools, as evidenced by language contained in standards promulgated by the Liaison Committee for Medical Education: “Educational objectives state what students are expected to learn, not what is to be taught … student achievement of these objectives must be documented by specific and measurable outcomes.”18

종합 정리, 추가적으로 필요한 것.

One GME leader has stated that “the ACGME is interested in the competency of the training program and whether the program has demonstrated a pattern of graduating individuals who are competent.”14 This is indeed an important goal and one that deserves support. However, achieving this goal will require two additional strategies. 

• One, institutions that sponsor GME must recognize that the outcomes framework (whereby training programs are expected to formally measure and document individual resident competency) represents a major educational paradigm shift within residency training. Achieving the goal of increased competency of graduating resident physicians will require additional resources devoted to faculty development, curriculum planning and competency measurement. 
• Two, as the ACGME continues to develop its final procedures for determining whether training programs are adequately measuring competency in its graduates, it must respond to criticism by many GME program directors that the expectations in this regard are open to subjective interpretation. As a DIO, I frequently saw instances where programs in certain disciplines were held to higher documentation standards than programs in other disciplines; or where various programs within a given discipline seemed to receive different accreditation results in spite of similar approaches taken to program evaluation issues. 

전공과목별로 어떤 성과를 측정할 것인가에 대한 Flexibility는 바람직하나, arbitrariness는 부적절하다.

Flexibility among various specialty disciplines in choosing which outcomes to measure (and methods used to measure them) is desirable; but arbitrariness in accreditation decision making based on a lack of consistent understanding of expectations is not. (...)

A conceptual model for program evaluation in graduate medical education.

학부의학교육의 설계와 도입으로부터 얻은 교훈(University of Dundee)

Planning and implementing an undergraduate medical curriculum: the lessons learned

MARGERY H. DAVIS1 & RONALD M. HARDEN2

1Centre for Medical Education, University of Dundee and 2The International Virtual Medical School (IVIMEDS), Dundee, Scotland, UK

배경

Introduction

교육과정의 여섯가지 특징

The medical school at the University of Dundee introduced a new curriculum in 1995. The curriculum combined idealism and pragmatism and six aspects were described by Harden et al. (1997): 

• the spiral curriculum; 
• a systems-based approach; 
• a core curriculum with options; 
• the educational strategies; 
• the student assessment approach; and 
• organization and management of the curriculum. 

The medical school has now had eight years’ experience of implementing the curriculum in practice. Since 1995 there have been significant changes both in the healthcare delivery system and in medical education. In this paper we look at how the curriculum has withstood the test of time and responded to change; which aspects of the curriculum are still in place; and what new approaches have been added. (...)

던디 의과대학의 학부 교육과정

The Dundee undergraduate medical curriculum

The 1995 curriculum was implemented as the result of proposals by a working group of the Dundee faculty of medicine for curriculum development (Davis, 1993). The focus for implementation was a sophisticated blend of educational strategies, which underpinned the curriculum. These included...

• a spiral curriculum with three interlocking phases; 
• a systems-based approach with themes running through the curriculum that provided a focus for the students’ learning; 
• a core curriculum with options; elements of problem-based learning (PBL); 
• community-based learning; 
• student-centred approaches to teaching and learning that encouraged students to take more responsibility for their own learning; and 
• an approach to assessment that emphasized the overall objectives of the course. 

An organizational and management structure and the allocation of resources were designed to support the educational philosophy. Since the programme was introduced, not unexpectedly, many details relating to the curriculum have changed. There have also been significant developments. In 1997, an outcome-based approach (Harden et al., 1999a, 1999b) was adopted for all five years of the curriculum, task-based learning (Harden et al., 2000) was introduced as the framework for student learning in phase 3 and the portfolio assessment process (Davis et al., 2001) was introduced as the medical students’ final examination.

교육과정 평가

Evaluation of the curriculum

The curriculum has been evaluated on evidence from a number of sources. These include both internal and external reviews and student examination data. The conclusions reached in this paper are based on this evaluation.

• (1) The internal reviews
• (2) External reviews, both formal and informal
• (3) Student examination data

교육과정

The curriculum

(1) The spiral curriculum

Lessons learned. The spiral design, with students revisiting topics in each phase, building on what they already know and adding further complexities is a robust and useful model for the undergraduate medical curriculum. One should not underestimate, however, the difficulty students may find in moving from one phase to the next, each with different approaches to teaching and learning. We found that an introduction to the overall curriculum and an interface between the phases was necessary.

(2) Outcome-based education

Lessons learned. There is a significant difference between outcome-based education and the production of a list of learning outcomes for an existing curriculum. ‘‘Outcome-based’’ suggested Spady (1993), ‘‘does not mean curriculum based with outcomes sprinkled on top. It is a transformational way of doing business in education.’’ The implementation in a curriculum of outcome-based education is not easy and requires the use of curriculum mapping. The effort is, however, worthwhile. The outcomes provide a valuable focus for student learning and direct the students’ attention towards learning outcomes that are easily ignored in the traditional curriculum. The outcomes also provide a sound basis for the student assessment process.

(3) Core curriculum with options

The core curriculum - Lessons learned. Identification of the core basic science components of the curriculum is not easy and is best done by basic scientists working in collaboration with clinicians. The specified learning outcomes play a key role in identification of what is core for all students

The options - Lessons learned. Options have proved popular with both staff and students, but providing large numbers of optional courses each attended by a few students is probably not cost effective. Staff and departments are more likely to accept a reduction of teaching time in the core if they can compensate with time in the optional part of the programme.

(4) The adaptive curriculum

Lessons learned. We learned that the adaptive curriculum approach could be employed in an undergraduate medical programme. Logistical difficulties were encountered with the implementation of the adaptive curriculum that led to changes in the number of attempts at an examination without, however, compromising the basic principles. What has been more challenging, however, is the creation of a new mindset which recognizes that curriculum time is finite and that how students may best use this time may vary from student to student.

(5) An integrated, systems-based approach

Lessons learned. System-based teaching is a key strategy in the early years of an undergraduate medical programme and is popular with both staff and students. Implementation may vary according to the preferred approach of individual teachers. Significant input from clinicians throughout the curriculum is needed to achieve both horizontal and vertical integration.

(6) Multiprofessional learning

Lessons learned. Multiprofessional education offers advantages and has attractions as a tool for enabling students to understand and respect the role of other healthcare professions. The true role of multiprofessional teaching and learning in medical education, however, is not clear. Unless there is a strong proponent or standard-bearer for the approach, significant change is unlikely. Multiprofessional education has to be institutionalized if it is to survive and contribute to the curriculum.

(7) Problem-based learning

Lessons learned. Implementation of an educational approach such as PBL may run into difficulties unless it is enthusiastically endorsed by the medical school. Task-based learning provides an option to PBL that is, in many ways, more attractive to staff, particularly in the clinical years. Task-based learning has been one of the curriculum’s successes. It is a valuable strategy for introducing an integrated and problem-based approach in the clinical years of an undergraduate medical curriculum. The tasks, matched with the appropriate learning outcomes, provide a framework or grid for identification of the core curriculum

Teaching and learning: student support

(1) Study guides

Lessons learned. The introduction of study guides clarifies what has to be taught and learned and has proved to be one of the most important innovations of the new curriculum. The costs of study guide production are substantial and need to be addressed. Both electronic and printed study guides have a role to play.

(2) Student-support scheme

Lessons learned. Student support is a taxing process that needs, to ensure its ongoing success, personal commitment from a large number of staff, secretarial resource to administer the system and a dedicated member of staff to provide the required leadership.

(3) Curriculum mapping

Lessons learned. Planned learning throughout a curriculum needs to be made explicit to both staff and students. Curriculum mapping aids this process. The complexities require an electronic learning environment.

(4) P-2-P learning

Lessons learned. P-2-P learning has an important part to play in supporting students

Teaching and learning: educational facilities

(1) Computer learning suite

(2) Educational resource area

(3) Integrated learning area

(4) Clinical skills centre

Lessons learned. The computer learning suite, the clinical skills centre and the integrated learning area are essential resources in a medical school. They support an integrated student-centred curriculum and help students to achieve the learning outcomes. Institutionalization of the educational facilities is essential for their successful and continued use. The clinical skills centre provides an important focus for clinical teaching, particularly in the early years of the curriculum. Unpaid volunteers have successfully provided a bank of simulated patients who are able to meet most needs of undergraduate medical student teaching in the centre. In the UK context, payment of simulated patients is not necessary and may make the extensive use of simulated patients unaffordable.

Assessment

(1) Twenty principles of assessment

Lessons learned. Principles of assessment are a useful tool to guide the development of an integrated assessment system. The principles need to be customized for the individual medical school context and kept up to date.

(2) Self-assessment

Lessons learned. One of the first principles of assessment is that the purpose of the assessment should be clear. Confusion between self-marking of summative assessment and self-assessment for formative reasons caused anxiety in some students. While self-marking of examination papers has potential for provision of rapid feedback to students, the purpose of the self-marking has to be clearly communicated to the students. The introduction of the process has to be carefully managed.

(3) Assessment to a standard

Lessons learned. The assessment-to-a-standard approach has been successful in supporting slower learners through the provision of supplemental instruction.

(4) Integrated assessment

Lessons learned. Integration of assessment is important to support curriculum integration. Integrated assessment may result in an academic staff ‘stand off’ from the assessment process related to lack of ownership.

(5) External examiners

Lessons learned. External examiners have an important contribution to make in ensuring that standards are set and maintained at an appropriate level.

(6) SSC(student selected components) assessment

Lessons learned. Relating SSC assessment to the learning outcomes facilitates standardization across SSCs. Standardizing student effort and marking criteria across disparate SSCs is challenging but achievable.

(7) Formative assessment

Lessons learned. The logistical implications of integrated assessment are considerable and had a deleterious effect on the provision of formative assessment opportunities for students.

(8) Progress test

Lessons learned. A progress test can provide useful feedback to staff and students. Student assessment needs to be adequately resourced in terms of academic staff, professional assessment advice, time and money.

(9) Portfolio assessment

Lessons learned. Portfolio assessment provided a framework within which student performance across a range of outcomes could be assessed. The portfolios identified student problems that the medical school did not have the processes to deal with; for example, a fitness to practice committee had to be set up for the undergraduates. The portfolio process is a major logistical exercise for the medical school, but it is considered to be worth the effort.

Organization and management: committee structures

(1) UMEC

(2) UMEC working group

(3) The three phase sub-committees of UMEC and the SSC committee

(4) The theme committee

(5) The faculty assessment committee

(6) The academic standards committee

(7) The computer committee

Lessons learned. The education committee structure and membership should reflect the needs of the curriculum. As a curriculum becomes established there is a tendency for the committee structure and membership to change, reflecting a more administrative and maintenance role rather than a forward-planning education role.

Organization and management: administrative support

Lessons learned. Administrative expertise residing at departmental level can be lost with the move to centralized administration of the curriculum. High-quality/senior administrative support is necessary for a successful shift to a centrally administered integrated curriculum. Separation of educational and administrative functions is needed as medical education staff can easily be regarded as administrators if academic staff are not aware of their areas of expertise. The assessment expertise residing within departments can be lost with the move to a centrally administered assessment system.

Conclusions

(1) The importance of initial planning

The 1995 curriculum was introduced as a result of a significant curriculum review. The review group was headed by the Dean and comprised 10 individuals with key roles in the curriculum and two students. They met regularly in the evenings for two to three hours at two weekly intervals over a six-month period. Attendance at the meetings was almost 100%. A careful analysis of the problems of the existing curriculum was carried out. Future needs were also considered. A key feature of the review was the wide consultation with a range of stakeholders, including recent graduates, general practitioners, current teachers and students. These stakeholders were invited to attend a meeting of, and discuss their views with, the review group. Other medical schools were surveyed for information regarding how they addressed a range of educational issues. The group issued a draft report for consultation and following this the final recommendations were produced and approved by UMEC and the faculty board. Communication of the finalized curriculum revision plans took place at a well-attended staff meeting and through circulation of the working group report.

(2) The need for a big picture

An overall structure and clearly enunciated educational principles provided a framework to inform ongoing discussions regarding curriculum implementation and guide change. The identification of the core clinical problems as the basis of task-based learning, integrated systems-based teaching and learning, outcome-based education with identification of 12 exit learning outcomes, the spiral curriculum, the core and options model and the 20 principles of assessment provided this structure and guidance.

(3) Facilitation of student learning

A range of approaches is needed to support student learning. Student study guides, introductory courses, educational facilities such as a clinical skills centre, computing suite and integrated learning area, P-2-P or collaborative learning, curriculum mapping, the student assessment system and a student support scheme are all important in facilitation of student learning.

(4) The student assessment system

The student assessment system needs to be integrated with the teaching and learning and be capable of supporting student learning. This is part of the paradigm shift from testing to assessment. If the curriculum is integrated in terms of disciplines and specialities, then the assessment system must also be integrated or the curriculum integration will be lost. The assessment-to-a-standard approach recognized the needs of different students and supported slower learners through the curriculum, even though the underlying concepts and shift in thinking involved in the approach had difficulty gaining general acceptance.

(5) Committee and administrative structure

The committee and administrative structure needs to support the curriculum. Changes in this structure may be necessary for different stages in the life of the curriculum: planning, implementation and maintenance. Implementation of an integrated curriculum where the responsibilities lie centrally in a school and where staff are located within departments does cause difficulties that need to be addressed. A matrix management system within a medical school may resolve the tensions between departmental and central control of the curriculum.

(6) Professionalism in medical education

There is a need for the commitment of all staff to the curriculum process. Different levels of educational expertise, however, are required. A critical mass of staff need to have an understanding of the underpinning educational principles and concepts and the educational vocabulary to discuss educational developments and to take part in the decision-making processes. Medical staff with educational expertise are needed for educational facilities such as the clinical skills centre. Professionalism in medical education is needed to support the curriculum, the assessment and the staff in their teaching activities. Research into medical education is necessary and professional medical educators can provide a focus for research activities. They can also engage students in the teaching and learning process and involve them in educational research.

(7) Leadership

Leadership is intimately associated with change and its sustainability. Leadership by the dean and other senior staff was needed for the curriculum revision and for the institutionalization of change. It is needed for the endorsement of change by curriculum committees. When leadership for the multiprofessional initiatives was lost, most failed.

(8) Flexibility

Built-in flexibility is important for sustainability. A curriculum is a living entity where ongoing change is almost certainly needed. When major changes such as outcome-based education, task-based learning and portfolio learning and assessment were introduced, the curriculum could adapt and cope.

Planning and implementing an undergraduate medical curriculum: the lessons learned.

학부 의학교육프로그램의 평가 강화

Enhancing Evaluation in an Undergraduate Medical Education Program

Kathryn A. Gibson, BMBCh, PhD, Patrick Boyle, MEd, Deborah A. Black, PhD, Margaret Cunningham, MSW, Michael C. Grimm, MBBS, PhD, and H. Patrick McNeil, MBBS, PhD

UNSW의 새로운 학부의학교육 과정 도입

The implementation of a new undergraduate medical education program at the University of New South Wales (UNSW) in Sydney, Australia, brought many challenges, not the least of which was determining whether the new program was effective. In this article, we describe the development and initial progress in implementing a comprehensive whole-program evaluation and improvement strategy for the new undergraduate medical education program at UNSW. Our literature review found relatively little practical guidance about establishing systems to define and maintain the quality of undergraduate medical education programs that encompass multiple aspects of quality. (...)

배경: 전체 프로그램에 대한 평가틀 개발

Background: Development of a Whole-Program Evaluation Framework

새 교육과정

A new Medicine program

교육과정 개요

In March 2004, the Faculty of Medicine at UNSW implemented an innovative six-year, three-phase undergraduate Medicine program.1 Compared with the previous content-based Flexnerian-style curriculum, the new program is explicitly outcome based,...

• requiring students to demonstrate specified levels of performance in a range of medicine-specific capabilities (biomedical science, social aspects of health, clinical performance, and ethics) and 
• generic capabilities (critical evaluation, reflection, communication, and teamwork) at defined levels as they progress through the program. All courses are interdisciplinary and highly integrated both horizontally and vertically. 

Important features include...

• early clinical experience, 
• small-group teaching, 
• flexibility in courses and assessments, and 
• a high degree of alignment between graduate outcomes, learning activities, and assessments. 

Each two-year phase uses a distinct learning process which aims to develop autonomous learning progressively during six years. 

The approaches emphasize important adult education themes: 

• student autonomy, 
• learning from experience, 
• collaborative learning, and 
• adult teacher–learner relationships.

새 교육과정에서의 평가시스템(포트폴리오)

The program’s assessment system is particularly important and incorporates many novel features, including...

• criterion referencing of results, 
• interdisciplinary examinations, 
• a balance between continuous and barrier assessments, 
• peer feedback, and 
• performance assessments of clinical competence. 

To examine the generic capabilities that are not easily measured by traditional assessments, and to ensure overall alignment between assessments, learning, and outcomes, a data-driven portfolio examination occurs in each phase.2

• The portfolio assessment is supported by an information technology system, eMed,3 which records student assessment grades, examiner feedback on assessment tasks, and peer comments on teamwork skills. 
• At the end of each two-year phase, students submit evidence of achievement for each capability using reflections on and reference to their performance recorded in eMed, compared with the expected level of achievement for that phase of the program. (...)
• Portfolio examiners look at the patterns of grades, teamwork comments, and other evidence cited by a student to see what lies behind the student data, and then they make a judgment from all the evidence. The portfolio assessment system is discussed extensively elsewhere.2,3

관리 주체(OME and Interdisciplinary groups)

In contrast to previous arrangements in which individual disciplines and departments managed content-specific components of the old curriculum, responsibility for the planning and implementation of the new program was given to a single unit—the Office of Medical Education under the direction of the faculty’s associate dean of education. However, as part of a planned change management process to maintain faculty ownership and encourage disciplinary integration, much of the work was devolved to interdisciplinary course design and implementation groups, each responsible for the delivery of one or more modular, integrated, eight-week courses, or a limited number of vertical strands, such as clinical and communication skills and ethics. The interdisciplinary groups were assisted by instructional designers and/or included faculty members with pedagogical training or experience.

효과적인 평가의 필요성

The need for effective evaluation

인증기구(AMC)의 요구조건

Such major pedagogical and organizational change required the development of systems and processes to 

(1) evaluate the effectiveness of the change, 

(2) monitor its implementation to enable continual improvement, and 

(3) use evaluation to recognize and report on excellence in teaching. 

An important driver was the accreditation authority, the Australian Medical Council (AMC), which requires highly specific standards with respect to ongoing monitoring, evaluation, feedback, and reporting of schools’ operations,4 similar to those specified by the Liaison Committee for Medical Education for North American medical schools.5 The AMC lists nine relevant standards that include...

• the need for “ongoing monitoring … [of] curriculum content, quality of teaching, assessment and student progress,”
• the need for “teacher and student feedback,” and “using the results [of evaluation] for course development.” 
• Schools are required to analyze “the performance of student cohorts … in relation to the curriculum and the outcomes of the medical course” and “evaluate the outcomes of the course in terms of postgraduate performance, career choice and career satisfaction.” 
• Finally, schools are required to report “outcome evaluation … to academic staff, students … [and] … the full range of groups with an interest in graduate outcomes.”4

이러한 통합 교육과정 기준을 맞추기 위해서 많은 혁신이 필요하다

Meeting these standards for an integrated program containing many innovative features required the development of new strategies and approaches to evaluation and improvement. 

• Initially, we conducted a review of previous evaluation processes at UNSW and found these to be largely content driven and not integrated into an overall program evaluation framework. 
• We then reviewed the medical and health education literature, which acknowledges the difficulty in conducting methodologically rigorous research or evaluation studies in education.6 
• We found many examples of evaluations of individual aspects of particular programs but few descriptions of evaluations of multiple aspects of a program and very little around the optimal strategy for systematically evaluating and then improving a medical program as a whole. Of those reviewed, two examples stand out for their breadth of evaluation approach adopted—the McGill dental curriculum 7 and the medical program at Dundee University.8 (...)

Dundee대학의 방법

The University of Dundee group, in describing its approach to evaluation of the medicine curriculum introduced in 1995,8 used evidence from a number of sources, which included internal reviews, external reviews, and student examination data. (...)

학부 교육과정과 관련한 기존 연구가 많지 않음.

Overall, we found little practical guidance in developing frameworks for systematic evaluation of undergraduate educational programs; the limited literature...

• that does exist is more applicable to GME.9–11 In the wider evaluation literature, 
• a topic of currency is the importance of program evaluation leading to explicit actions, particularly improvement for students and other key stakeholders,12–14 
• an issue commonly referred to as “closing the loop.”15

다요소모델 개발

Developing a Multicomponent Model to Evaluate Program Quality

프로그램평가 및 향상 그룹 설립

To develop a comprehensive evaluation process, a Program Evaluation and Improvement Group (PEIG) was established by the dean of medicine with administrative support from the Office of Medical Education and the associate dean of education. (...)

PEIG의 주 과업 

A primary task of the PEIG was to establish a framework or model that we considered would encompass all facets of the curriculum at UNSW. With the AMC standards 4 as an important driver, we formulated six strategic principles to guide development of a program evaluation and improvement strategy (List 1). (...)

위의 원칙에 따른 프로그램평가 모델 개발

With these issues and ideas in mind, we undertook the development of a progressive model for program evaluation and improvement. (...)

핵심 quality indicators 만들 때 염두에 둔 것들

In deriving the key quality indicators, we considered the following questions: 

• (1) Would the indicator facilitate action for improvement? 
• (2) Is valid information available for the indicator? 
• (3) Will the indicator convey clear meaning to stakeholders? 
• (4) Would the information content of the indicator be considered trustworthy and valuable? and 
• (5) Does the indicator represent a balance of perspectives for multiple stakeholders? 
• In addition, we were cognizant of the general principles of content validity and relative importance. 

(...)

Ownership과 이해관계자들의 참여의 중요성

We recognized that any such model can only be effective if there is ownership of it by key stakeholders and if these stakeholders are actively participating in the processes that are developed as an extension of the model. A key practice principle we adopted to implement the model is that, wherever possible, evaluation activities should be viewed as an integral part of day-to-day practice rather than as externally imposed additional tasks. Thus, an important philosophy is that teachers, course and phase convenors, and relevant administrators should undertake evaluation and improvement activities as an inherent part of effective and scholarly teaching. (...) The PEIG sees its principal role as supporting and facilitating this process, rather than being an external body undertaking such tasks. (...)

각 영역의 평가와 향상

Addressing Evaluation and Improvement of Each Quality Aspect

(...) Before establishing specific projects, each working party reviewed their respective indicators (Table 1) to identify relevant available data, areas where instrument development needed to occur, and the reporting, communication, and implementation implications for indicator development. To illustrate the evaluation model in practice, two areas of work to date are elaborated on below.

학생경험의 평가와 향상

Evaluation and improvement of the medical student experience

네 단계

In 2005, the student experience working party established a baseline project to research and evaluate the medical student experience at UNSW and to establish a sustainable process for its evaluation and improvement. Four stages were involved: 

(1) identifying the students’ understanding of the construct “student experience,” 

(2) developing a trial instrument and process for collecting evaluative data from students, 

(3) undertaking appropriate analyses and communication of findings to stakeholders, and 

(4) establishing an agreed and sustainable process within the faculty for evaluation and improvement of the medical student experience. 

우선 학생들이 'student experience'라는 단어를 어떻게 이해하는지 조사함. 다섯가지 영역 도출

Before developing any instrumentation for tapping student perceptions of their experience as medical students at UNSW, we decided to investigate what students understood by the term student experience. (...) Five main facets emerged and were adopted for the trial medical student experience questionnaire (MEDSEQ): 

• learning, teaching and assessment; 
• organization and student understanding of the program; 
• community interaction and value; 
• student support; and 
• resources. 

These encompassed most of the key quality indicators for the student experience aspect shown in Table 1.

The trial MEDSEQ 

The trial MEDSEQ used 32 fixed-response items arranged under the five facets referred to above, plus the option of open-ended comments to collect student participants’ responses. We used a five-point scale (ranging from only rarely to almost always) for students to assess the frequency with which they had experienced the circumstances described by each facet (Chart 1). (...)

In early May 2007, after review of the major MEDSEQ report, the Faculty of Medicine’s principal curriculum committee endorsed embedding the MEDSEQ evaluation and improvement process in the Faculty of Medicine’s continuing operations. Data gathering, analysis, and communication of findings will occur every two years, and a range of responses to the findings in the inaugural report will be initiated to ensure provision of more effective feedback on learning and increased availability of appropriate mentoring.

교육 및 교육업무량 관리의 평가와 향상

두 가지 측면에 대한 key quality indicator 평가

The working party on staff and teaching has been addressing two of the key quality indicator components for this program aspect: support for teaching and improving the quality of teaching (Table 1). 

• With respect to support for teaching, we developed a policy document and framework for implementation entitled “Managing Teaching Workloads in the Faculty of Medicine,” which was endorsed by the faculty’s major management committee and is being used by department heads to help manage their teaching and research missions. (...)
• With respect to improving the quality of teaching, guided by the principle of “closing the loop,” efforts are being made to enhance the extent to which student evaluative data are acted on to implement improvement, stimulate reflection, and foster more scholarly activities in relation to learning and teaching. Improving teaching on the basis of student feedback has proved challenging at UNSW in the past because feedback on individuals’ teaching is reported only to the teacher affected and is not available to curriculum leaders or coordinators. (...) 

Expected Outcomes and Initial Evidence of Effectiveness

기대 성과

The key outcomes we expect as a result of the processes and strategies we have implemented can be distilled to the following: 

• multiple aspects of the quality of the UNSW Medicine program are able to be evaluated and reported (e.g., student and staff experiences, student outcomes, content and resources); 
• the evaluation process is continual; 
• action to improve the program follows measurement; and 
• the processes we describe are an inherent part of the responsibility of teachers and course coordinators. 

(...)

평가 결과

Phase 1 (medical school years 1 and 2) of the program consists of four integrated eight-week courses per year, the majority of which have been formally evaluated using a standard UNSW course evaluation instrument. As shown in Table 2, initial evaluations in 2004 showed strong evidence that the new program encouraged active student participation and collaborative learning (92% and 97% agreement, respectively). However, students expressed significant dissatisfaction with the provision of feedback and with assessment tasks. (...)

Over subsequent years, there has been significant evidence of improved student experience of phase 1 with progressively greater levels of agreement that the courses provided adequate information about assessments and that the assessments were appropriate (both up to 81% agreement) (Table 2). (...)

Acad Med. 2008 Aug;83(8):787-93. doi: 10.1097/ACM.0b013e31817eb8ab.

Enhancing evaluation in an undergraduate medical education program.

임파워먼트 평가 : 의과대학 교육과정 변화와 평가를 위한 협력적 접근법

Empowerment Evaluation: A Collaborative Approach to Evaluating and Transforming a Medical School Curriculum

David M. Fetterman, PhD, Jennifer Deitz, MA, and Neil Gesundheit, MD, MPH

의과대학 교육과정 변화의 어려움

• Changing a medical school's curriculum is commonly an arduous process because faculty, students, and administrators may hold divergent opinions about both the goals for medical education and the best ways to achieve those goals.1,2 
• In addition, curriculum reformers may face stiff resistance if they challenge deeply entrenched interests, traditions, and institutional culture.3,4 
• Proposals for change may engender debate that can be rancorous and adversarial among competing faculty, such as basic scientists and clinicians.5 

그러나 철학적, 현실적 합의에 도달하는 것은 중요함.

Yet, reaching a philosophical and practical consensus for change is critical, because a unified vision is needed to achieve broad acceptance, coordinated implementation, and subsequent refinement of a new curriculum.6 

저자들의 두 가지 질문

(1) 교육과정의 이상적 내용에 대한 합의에 도달하는 가장 좋은 방법은?

(2) 지속가능한 평가 주기와 향상을 위한 교육과정 평가의 최선의 방법은?

We (i.e., the authors, who formed the evaluation team) ask two important questions: (1) What are the best methods for achieving consensus about the optimal content of a medical curriculum? and (2) What is the best way to evaluate a curriculum to ensure that it will undergo sustainable cycles of review and improvement?

지난 20년간, 기초과학 지식의 향상과 의학교육의 혁신에 의하여 많은 미국 의과대학 교육과정은 크게 변화하였음. 성공적으로 교육과정 변화를 이뤄낸 학교들에 대한 연구로부터 Bland 등은6개의 핵심 특징을 찾아내였음. 또한 Loeser등은 다섯 개의 교육과정 변화의 단계를 주장하였고 그 마지막 단계는 도입과 평가이다. 대부분의 교육과정은 그것이 처음 도입된 시점에서부터 완벽한 경우는 거의 없기 때문에 효과적인 평가는 교육과정 개혁의 핵심이라고 할 수 있다. 실제로 새로운 교육과정을 설계한 사람들조차 그들이 생각하는 이상적인 모습에 도달하기 위하여 비판과 교정의 과정을 바라고 있다.

During the past 20 years, stimulated by the growth of basic science knowledge and by innovations in medical education, most U.S. medical schools have implemented significant changes in their undergraduate curricula.7 In a review of schools that had undergone successful curricular change, Bland and colleagues 8 identified six predominant features: 

• leadership, 
• a cooperative climate, 
• participation by organization members, 
• favorable politics, 
• human resource development, and 
• effective evaluation. 

Similarly, Loeser and colleagues 6 described five phases of curriculum change that were used at the University of California, San Francisco; the last phase was the implementation and evaluation of the new curriculum. Thus, effective evaluation is recognized as a key component of curricular reform, in part because most curricula are far from perfect when first implemented. Indeed, those who devise a new curriculum depend on and anticipate cycles of critique and revision to approximate the ideal.9–12

스탠포드 의과대학의 교육과정 변화

The Stanford University School of Medicine underwent a major change in its undergraduate medical education when a new curriculum was introduced in academic year 2003–2004. The curriculum continued to be refined yearly from 2004 through 2008. We found the principles of “empowerment evaluation”13–15—an innovative evaluation method that involves stakeholders in an egalitarian process of review, critique, and improvement—to be useful in evaluating the curriculum and in guiding change. The empowerment evaluation method includes a process for ongoing scrutiny of the effectiveness of the curriculum along with the promotion of a cooperative atmosphere. In this paper, we describe the key characteristics of empowerment evaluation and show how using its tools has helped faculty, students, administrators, and evaluators (as a group, referred to here as “stakeholders”) to revise, refine, and improve our medical school curriculum.

Empowerment Evaluation: The Approach and Its Tools

The approach

Empowerment evaluation, broadly defined, is an approach to gathering, analyzing, and using data about a program and its outcomes that actively involves key stakeholders in all aspects of the evaluation process. The approach provides a strategy by which and a structure within which stakeholders can empower themselves to engage in system changes; this process is based on the assumption that the more closely stakeholders are engaged in interpreting, discussing, and reflecting on evaluation findings in a collaborative and collegial setting, the more likely they are to take ownership of the results and to use evaluation to guide curricular decision making and reform.

The tools

Five tools are integral to the empowerment evaluation approach.15 We discuss each one below.

"근거의 문화" 만들기

Developing a culture of evidence.

Emphasis is placed on rapidly disseminating formative feedback and data about the curriculum to help inform decision making. Reporting processes are established that will improve stakeholder access to evidence (evaluation data and findings) and that will provide regular opportunities for reviewing and discussing the evidence. Faculty are encouraged and supported in their efforts to use evaluation evidence as a tool to support or refute their positions in debates about curriculum. By making this practice a cultural norm within the institution, a culture of evidence is established.

"비판적 친구" 활용하기

Using a critical friend.

The empowerment evaluation method uses a professionally trained evaluator to help facilitate evaluation practices at an institution. However, the role the evaluator plays is not that of an outsider or an expert removed from curriculum development and implementation but, rather, that of a trusted colleague—a critical friend—who can coach stakeholders as they engage in self-evaluation, strategic planning, and curriculum implementation. (Critical friend의 역할)

• This critical friend can help set the tone for discussions about evaluation results and findings, by modeling and informally articulating the philosophy behind the empowerment evaluation approach.16–20 
• Specifically, the critical friend should help to establish a positive learning climate in which the views of all stakeholders are respected, input from all parties is solicited, and the conversation is guided so as to encourage comments that are constructive and improvement oriented. 
• Critical friends model a communication style that is inviting, unassuming, nonjudgmental, and supportive, so that stakeholders will feel comfortable in speaking openly about curricular issues and concerns. 
• Critical friends should also help to remind members of the group about what they have in common, including shared curricular goals and institutional values.

"성찰과 행동의 사이클 장려하기"

Encouraging a cycle of reflection and action.

Structured opportunities allow stakeholders to reflect on evaluation data, set priorities, and develop action plans for revising the courses and curriculum. Once revisions are adopted and implemented, those too are monitored and evaluated. The cycle of inquiry and evaluation is thus ongoing and continuous.

학습자의 커뮤니티를 활성화시키기

Cultivating a community of learners.

Under the empowerment evaluation model, the assumption is that each stakeholder, regardless of his or her status within the institution, has a unique and important perspective on curricular issues that should be shared and valued. As stakeholders engage in dialogue and discussion, they learn from one another and, in the process, are able to broaden and deepen their understanding of issues affecting individual courses and, more broadly, the curriculum. They also gain greater insight into the steps involved or hurdles encountered in making improvements and revisions. Emphasizing learning and discovery in a collaborative and egalitarian environment helps to make discussions about evaluation findings more empowering and less threatening. Reaching a consensus cooperatively also helps to reduce the likelihood that mandates for reform will be imposed on faculty or students in a top-down or hierarchical fashion.

자기성찰을 할 수 있도록 하기

Developing reflective practitioners.

Building in structured time, assignments, and formal expectations for all stakeholders helps to immerse stakeholders in the evaluation process. Faculty, students, and administrators develop the habit of continually reflecting on their practices and programs, both individually and as part of a larger group or community. By establishing a habit of regular self-assessment, stakeholders become reflective practitioners.

These tools of empowerment evaluation are mutually reinforcing, and they frequently operate together. However, it may be useful, at first implementation of this approach, for those involved to think of the progression as taking place in stages. The stages build on each other, generating a momentum, which culminates in the development of a more refined and constructive evaluative eye (Figure 1).

How Empowerment Evaluation Comes Together

The theoretical framework and the tools integral to the empowerment evaluation approach, as described, provide the basis for establishing and routinizing a system of evaluation that can be applied across the medical school curriculum. At Stanford, the stakeholders initially focused on developing this evaluation system at the preclinical level.

1. First, data were assembled from midquarter focus groups, classroom observations, and end-of-quarter student questionnaires. 
2. Second, faculty members received these data in advance of a group meeting; at the meeting, faculty members discussed the positive and negative data from the evaluations, their own reflections, and the proposed next steps. A trained evaluator, who served as a critical friend, mediated the meeting, providing data and support but also being willing to highlight weaknesses in the course(s) and approach. This facilitated process helped to build a culture of evidence, in which it was the norm to use data to support one's position and inform decision making. 
3. Third, the faculty members engaged in an initial cycle of reflection and action by discussing ways in which they could collaborate between courses to enhance student learning—for example, by improving the logical sequencing of topics across the curriculum and by removing unintended redundancies. The evaluation findings and faculty responses served to stimulate discussion at curriculum committee meetings and in the faculty senate, which helped to cultivate a community of learners across the school. 
4. Fourth, faculty and student task forces worked together to complete the design and implementation of the revised courses and/or curriculum. 
5. Fifth, the process came full circle: The courses and curriculum were evaluated by both students and faculty to determine the effectiveness of the new educational interventions and curricular modifications. 
6. This last phase is the full cycle of reflection and action that is most characteristic of empowerment evaluation. Through this process, faculty, students, and administrators enhanced their goal of becoming reflective practitioners, who routinely assess teaching and learning in the school.

Application of Empowerment Evaluation Principles and Tools in the Evaluation of a Medical School Curriculum

The challenge of transitioning students and faculty through a major restructuring of the curriculum at Stanford lent itself to a parallel reassessment of the existing evaluation methods. The stakeholders, led by the evaluation team, began to think more explicitly about the values that were implicit in our practices and to explore ways to address key weaknesses in our approach.

기존의 접근법 Prior approach to evaluation and the areas needing change

OME의 문제

In the past, the stakeholders and the Office of Medical Education (OME) did not explicitly articulate a set of values or goals to frame our approach to evaluation. Implicitly, we were guided by a tradition within the institution that each course was a discrete entity and that responsibility for that course lay primarily with the individual course director. (...)

이혜해관계자의 참여를 지원할 공식적 구조와 절차의 부족

Other weaknesses in our prior approach were a lack of formal structures and processes to encourage and support stakeholder engagement in the evaluation process. Within OME, we did not have dedicated staff members in charge of developing the evaluation program or managing and reporting evaluation data. Consequently, although the OME staff were vigilant about collecting evaluations from students at the end of each course, in some cases, data were “warehoused,” and evaluation results were not made available to course or clerkship directors for months or even years afterward.(...)

전국적으로 벌어지고 있는 의학교육의 변화를 잘 반영하는 평가법을 도입하고자 함.

"평가의 문화가 새로운 교수-학습 문화와 consonant하기 위함"

In identifying and instituting a new model of evaluation, we wanted an approach that would mirror the changes taking place in medical education nationwide.2,7,21,22 Our new medical school curriculum emphasized an integrated and applied approach to teaching and learning, and, in this new environment, faculty needed to work more collaboratively to develop the curriculum. Developing a single block of lectures in the integrated-organ-systems course on the heart or lung required input from and involvement of faculty in the departments of histology, pathology, microbiology, physiology, and pharmacology. Similarly, the stakeholders now expected more participatory involvement on the part of both teachers and learners as active learning exercises were developed in which preceptors would serve not as content experts but as facilitators, helping teams of students diagnose and develop treatment plans for patient problems.23 It was our hope that the new model of evaluation would reflect these changing institutional values and practices and, thus, would bring collaboration and participation to the forefront. The culture of evaluation would be consonant with the new culture of teaching and learning.

현재 접근법 Current approach to evaluation

구조화된, 일관성있는 프로그램 도입

The new Division of Evaluation staff worked to implement a structured, coherent program that would apply empowerment evaluation principles and tools in evaluating the effectiveness of the curriculum. 

• Empowerment evaluation tools served to address a wide range of issues affecting the educational program and curriculum at Stanford, including issues surrounding courses, clerkships, scholarly concentrations, advising, and faculty instruction. 
• The approach also helped us to address crosscutting curricular issues such as scheduling, standardization of syllabi, and clinical skills training. 
• Finally, the stakeholders used empowerment evaluation to identify broader programmatic issues, such as the need for changes in staffing and the resource allocation needs for integrated courses.

학생과 교수들이 교육과정에 대한 지속적인 대화에 정기적으로, 구조화(structured)되어 참여할 수 있는 기회제공.

Initially, however, the primary focus was on evaluating and monitoring the preclinical curriculum, because this aspect of our program was undergoing the most radical transformation. Shifting to an applied, integrated curriculum meant changes in the faculty's roles and responsibilities. Course content and hours were adjusted, and some faculty had great concern that eliminating content in some areas in order to expand content in others might jeopardize students' knowledge base and preparedness as researchers and clinicians. We therefore felt that it was important to establish regular, structured opportunities for students and faculty to engage in an ongoing dialogue about the curriculum, to reflect on strengths and weaknesses, and to collaboratively seek solutions when problems were identified.

임파워먼트 평가 모델을 적용하기

Applying the empowerment evaluation model to the preclinical curriculum

• We applied several familiar evaluation tools to our review of the preclinical curriculum, including focus groups and surveys. (...) 
• Applying the empowerment model at Stanford also required improvements in the data-collection and -reporting processes. (...) with empowerment evaluation, we distilled and analyzed qualitative and quantitative data, using both standard questions across the curriculum and questions rooted in specific concerns of course directors. (...) By emphasizing the positive steps that could be taken to improve a course, evaluators sought to minimize the disillusionment that can sometimes accompany poor student ratings and to provide encouragement and support for faculty as they worked to revise and refine courses, curriculum, and the quality of instruction. 
• In the past, scheduled meetings with course directors were rare or sporadic. (...). These meetings provided structured opportunities for what Bohm 25 called “dialogic engagement,” or exchanges among course directors, students, the curriculum committee, and the empowerment evaluators. 
• As access to evaluation data improved and the process of sharing evaluation data and results became less threatening, faculty became more comfortable engaging in dialogue about evaluation findings. 
• Working closely with many individual course directors, students, and administrators and adopting an inclusive and pluralistic perspective toward data collection, the evaluation team was in a unique position to identify patterns across the curriculum.(...) Thus, a continuous and systematic cycle of reflection and action across the curriculum became institutionalized.  
• Collaborative efforts at all levels contributed to providing a comprehensive self-assessment of the curriculum and placed stakeholders back in the driver's seat—providing them with forums and opportunities to become leading voices in setting priorities. Through continuous engagement in the evaluation process and daily immersion in evaluation discourse, stakeholders learned to mirror the process in their daily practice, thus becoming reflective practitioners.

Measuring the Effectiveness of Empowerment Evaluation

Internal metrics

Applying the empowerment evaluation model to curriculum evaluation at Stanford has contributed to improvements in course and clerkship ratings. In comparing evaluation results before and after stakeholders began using this approach, we found that the average student ratings for the required courses (18 courses) improved significantly (P = .04; Student's one-sample t test), as shown in Figure 2. Similarly, ratings for most of the required clerkships remained steady or improved. When dialogue around evaluation findings led to the development of targeted strategies for addressing key weaknesses in courses, clerkships, or the overall curriculum, student ratings showed a marked improvement. Three examples that stand out in this regard are the “Cells to Tissues” course, the obstetrics–gynecology and surgery clerkships, and our cross-curricular approach to clinical skills training.

Figure 2

External metrics

Finally, we explored the use of several external metrics to measure the effectiveness of the new Stanford curriculum as it was being refined by empowerment evaluation. 

• The median United States Medical Licensing Examination (USMLE) Step 1 score at Stanford for the three years before the introduction of the new curriculum and evaluation method was 230, whereas the median score rose to 237 by three years after the introduction. 
• The failure rates of students in USMLE Step 1 and USMLE Step 2 Clinical Knowledge, both before and after the curricular change, were each under 2% (national rates: 6% and 5%, respectively). 
• For clinical skills development, no data were available from 2000–2003 because USMLE Step 2 Clinical Skills was introduced for the first time in 2004–2005. However, in academic years 2005–2006, 2006–2007, and 2007–2008, the failure rate on this exam at Stanford was 1%, whereas the national failure rate was 3%. 
• We also examined student performance during the first postgraduate year of training. Approximately 80% of our graduating students match at competitive academic residencies. In their first year of residency, our graduates' performance generally met or exceeded expectations in skills and knowledge domains, as reported by residency program directors. 
• Although these metrics lack the precision and certainty of a controlled experiment, the aggregate measures indicate that student education was kept at a high level or was enhanced by the introduction of the new curriculum and the simultaneous use of empowerment evaluation to refine it.

Challenges and Barriers

Adopting an empowerment evaluation approach at a medical school is rewarding but also challenging. Faculty members...

• often have demanding schedules and little frame of reference for participating in collaborative feedback. 
• Time devoted to reviewing the curriculum and modifying courses may compete with time devoted to research and publication—activities that commonly bring greater institutional rewards. 
• Some faculty, accustomed to reviewing their evaluation results in private, initially had concerns about discussing course and curriculum strengths and weaknesses in a group setting. 
• For empowerment evaluation to work effectively, members of both the administration and the faculty needed to view their participation in teaching and a candid critique of the school's curriculum as priorities equal to research, publications, and grants.

(...)

Empowerment evaluation requires an investment of resources. 

• Institutions must be committed to supporting the evaluation staff with formal training to facilitate the process, and 
• faculty time must be protected to allow their participation in meetings about and in discussions of the curriculum. 
• In the absence of an evaluation unit and a regular meeting schedule, the collaborative process can be compromised. 
• If data are not collected, analyzed, disseminated, discussed, and reviewed in a timely manner, rigor may not be maintained.

토마스 에디슨은, "많은 사람들이 기회를 놓치곤 하는데, 왜냐하면 기회는 작업복을 걸치고 있어 일처럼 보이기 때문이다."

Thomas Edison once observed, “Opportunity is missed by most people, because it is dressed in overalls and looks like work.” (...)

Receptivity of Stakeholders to Empowerment Evaluation

A department chair said, 

We are grateful [to the empowerment evaluation approach] as we continue to improve our departmental teaching programs! In addition to the superb efforts [of our clerkship directors], our entire faculty and our residents and fellows are now much more committed to and involved with medical student education.

The director of clerkship education said,

Several clerkships have responded to this [evaluation] feedback by requesting individualized coaching [from the evaluation group, which has] made several site visits to assist clerkship directors in revising their orientations, syllabi, and student performance evaluations. End-of-clerkship evaluations demonstrate improved ratings as a result of feedback and coaching; narrative performance evaluations have improved in terms of completeness and level of detail.

The director of a medical student clinical clerkship said, 

“All of the Stanford sites want to help improve their teaching. The differences [that empowerment evaluation] made at our neighboring hospitals were visible and clearly effective.”

Empowerment evaluation: a collaborative approach to evaluating and transforming a medical school curriculum.

의과대학 교육과정에 대한 지속적 프로그램 평가 (Penn State University College of Medicine)

How we conduct ongoing programmatic evaluation of our medical education curriculum

KELLY KARPA & CATHERINE S. ABENDROTH Penn State University College of Medicine, USA

ABSTRACT

학부의학교육에 대한 평가는 인증기준에 도달하기 위해서 반드시 필요하나, 적절한 평가 단계를 도입하고 유지하는 것은 쉽지 않다. Penn State University College of Medicine에는 2000년 교육과정평가위원회(CEC)가 설립되어 기존의 의학교육부학장과 학부의학교육위원회의 활동을 보조하였다. 여기서는 CEC가 활용한 방법과 성과를 다루고자 한다. 우리가 활용한 단계의 강점은 지속적이고 정기적인 평가이며, 이를 통해서 매2년마다 교과목을 리뷰하고 변화가 필요할 때에는 교과목 담당자가 책임을 지도록 했다. 우리의 평가 단계는 효과적이었고, 지속가능하며, 교육과정 개선을 위한 추가적인 영역을 확인해주었다.

Evaluation of undergraduate medical education programs is necessary to meet accreditation standards; however, implementation and maintenance of an adequate evaluation process is challenging. A curriculum evaluation committee (CEC) was established at the Penn State University College of Medicine in 2000 to complement the already established activities of the Office of the Vice Dean for Medical Education and the Committee on Undergraduate Medical Education. Herein, we describe the methodology used by the CEC at our academic medical center and outcomes attributable to the curriculum evaluation process that was enacted. Strengths of our process include ongoing, regular assessments that guarantee a course is reviewed at least every two years and a feedback loop whereby course directors are held accountable for implementing changes when necessary. Our evaluative process has proven effective, sustainable, and has identified additional areas for curricular improvements.

Introduction

의학교육에서는 총괄적인 프로그램 평가 단계를 묘사한 것을 찾기 어렵다.

Medical education is continually evolving, necessitating ongoing programmatic evaluations to determine the worth of programmatic components and whether overall goals and objectives are being attained (Fitzpatrick et al. 2004). More than 35 types of evaluative theoretical frameworks (needs, cost/benefit, formative, summative, goal-based, outcomes-based, etc.) have been developed for virtually every type of social endeavor; however, within medical education it is unusual to find comprehensive programmatic evaluation processes described (Musick 2006; Durning et al. 2007; Gibson et al. 2008; Fetterman et al. 2010; Vassar et al. 2010).

의과대학은 LCME에서 설정한 인증기준을 만족시키기 위한 평가를 해야 한다. 인증의 요구조건을 맞추기 위해서는 다음과 같은 것을 해야 한다.

Medical schools must establish approaches to evaluation that satisfy the accreditation standards set by the Liaison Committee on Medical Education. To meet accreditation requirements, medical schools are mandated to 

consider student feedback; 

perform ongoing monitoring of curriculum content, teaching, and assessment of student progress; 

use the results of program evaluations for course development; and 

evaluate medical school coursework in terms of postgraduate performance. 

PSUCOM의 교육과정 평가 단계에 대해 설명하고자 함.

Despite these clear directives, medical schools have struggled with identifying procedures and personnel best suited to accomplish these tasks, as exemplified recently by the probation of one United States medical school (Mulder 2012). Herein, we describe the systematic curriculum evaluation process that has been used at the Penn State University College of Medicine (PSUCOM) since 2000. 

• Our programmatic evaluation process draws from both the Tylerian approach, which places substantial emphasis on objectives (Tyler 1942), 
• and the empowerment evaluation model, which depends upon a culture of evidence, a critical friend, a cycle of reflection and action, and reflective educational practitioners (Fitzpatrick et al. 2004).

How we started

CEC 출범과 그 과업

A decade ago, our institution formed a Curriculum Evaluation Committee (CEC), an independent evaluative body that made recommendations for consideration by the Committee on Undergraduate Medical Education (CUMED), the authoritative body. The CEC was charged with...

(a) evaluating the performance, outcomes, and impact of the medical school curriculum; 

(b) judging the value and worth of the instructional methods; and 

(c) working with other institutional committees to improve curriculum and course performance. 

To accomplish the directives, the appointed CEC members operate at a granular level to comprehensively review each pre-clinical course and clinical clerkship bi-annually.

교과목 평가시에 고려한 점

The four components the CEC considers when reviewing a course are: 

goals and objectives, 

content, 

examination/assessment, and 

student opinion. 

Two courses are discussed at each monthly (1 h) CEC meeting. In advance, a committee member is assigned to review one of the four components for a single course. Members are provided with course-specific documentation and on-line access to course materials via our electronic course management system. Overall, these bi-annual assessments allow good performance to be rewarded by departmental Chairs, problematic curricular areas to be identified and remedied, and facilitate a shared awareness of curricular issues at all levels from the Dean to individual instructors.

Goals and objectives

The CEC member evaluating course goals and objectives checks these parameters against institutional goals and objectives for alignment. Each educational session (e.g., lecture, laboratory, small-group session) is expected to have specific learning objectives addressing varying cognitive levels that are written in behavioral terms which describe knowledge, skills, and attitudes students should demonstrate as a result of the educational session.

Content

Course content is compared to course goals and the stated learning objectives, United States Medical Licensing Examination (USMLE) course content outlines, and/or curricular content published by national associations. Omissions, redundancy, organization and integration of content, pedagogy, quality of lecture slides, and ease of navigation of electronic course management tools are assessed. The ability of the CEC to assess horizontal and vertical integration of curricular content is facilitated by use of the Curricular Management and Information Tool developed by the American Association of Medical Colleges. Evaluations of clinical clerkships also assess parameters such as inpatient and outpatient experiences, opportunity to perform procedures, and comparability of student experiences across clerkship sites.

Examination/Assessment

Multiple choice question-based examinations are judged in terms of: test reliability, correlation between test questions and objectives and content, level of cognitive domain being tested, item difficulty, item discrimination value, grade distribution, and format and quality of questions. Rubrics and inter-observer validation are expected for subjective assessments, e.g., essays.

Student opinion

End-of-course evaluations by students are semi-quantitative (Likert scale) with opportunity for anonymous free-text comments. CEC members that review student evaluations look for themes pertaining to: quality of lectures; course organization; clarity of learning objectives; quality of problem-based learning sessions, on-line exercises and small group sessions; topics requiring more or less emphasis; clarity and content of examination questions; and level of knowledge necessary for examination in light of course and lecture objectives. Student focus group feedback is reviewed as well.

Putting it all together: Information flow

Results from CEC involvement

부족한 것으로 나타난 부분

To date, areas of curricular deficits identified during the CEC reviews have generally been addressed by adding didactic sessions. Since 2002, the total hours of didactic instruction in year 1 increased from 66.5% to 74.8% (p = 0.02). Even in year 2, where problem-based learning plays a larger role, didactic classroom instruction time has increased by 11% (p < 0.001). Although more complete, this outcome is not necessarily desirable given the current trend toward more student-directed learning.

향상된 것으로 나타난 부분

As a result of CEC feedback to course directors, the number of lectures that state learning objectives has been steadily increasing since 2002; furthermore, the number of lectures with appropriately written behavioral learning objectives has increased in both year 1 and year 2 courses (p < 0.001). Another facet of the curriculum that has experienced improvements as a direct result of CEC involvement pertains to examination questions written in National Board of Medical Examiners-approved format. Since the 2002–2003 academic year, significantly more questions are written appropriately (p < 0.001). These are areas that the CEC has heavily championed, and this committee is largely responsible for the improvements that are noted. In addition, the average percent of our students passing USMLE Step 1 between 2000 and 2005 was 90%. In comparison, within the last five years, the average pass rate has increased to 95% (without any change in student admission requirements). Although this may be multifactorial, we believe the curriculum evaluation process has played a significant role.

A subjective measure of the CEC's effectiveness is the current manner in which course/clerkship directors welcome and are engaged in the review and improvement process. The CEC is generally viewed as a “critical friend” that provides constructive criticism, rather than the “educational police.” This perception likely accounts for the cooperation we receive from faculty in response to our reviews. Some course directors have even requested that the CEC review their course annually (rather than the normal bi-annual cycle), illustrating the value that course directors place on CEC feedback.

How we conduct ongoing programmatic evaluation of our medical education curriculum.

하버드 의과대학생들에 대한 New Pathway Curriculum의 영향

The Influence of the New Pathway Curriculum on Harvard Medical Students

GORDON T. MOORE. MD. MPH. SUSAN D. BLOCK. MD. CAROLYN BRIGGS STYLE. PhD. and

RUDOLPH MITCHELL. EdD

배경 BACKGROUND:

급진적으로 재설계된 임상실습 전 학생(preclinical students)들의 교육과정의 효과를 평가하기 위한 연구

This study evaluated the effect of a radically redesigned curriculum at Harvard Medical School on preclinical students' knowledge, skills, personal characteristics, approaches to learning, and educational experiences.

방법 METHOD:

1989년과 1990년에 입학한 학생들 중 new curriculum에 배정되기를 지원한 121명의 학생들로부터 자료를 수집하였다. 학생들은 NPC와 TC 가운데 무작위로 배정되었다.

Multiple measures were used to collect data from 121 students from the entering classes of 1989 and 1990 who had been randomly assigned to the New Pathway or traditional curricula; all had applied to be in the new curriculum.

결과 RESULTS:

• The New Pathway students reported that they learned in a more reflective manner and memorized less than their control counterparts in the traditional curriculum during the preclinical years. 
• The New Pathway group preferred active learning and demonstrated greater psychosocial knowledge, better relational skills, and more humanistic attitudes. 
• They felt more challenged, had closer relationships with faculty, and were somewhat more anxious than those in the traditional program. 
• There was no difference in problem-solving skills or biomedical knowledge base.

Qualitative data from semistructured interviews at the end of the second year indicated that the NP students felt more responsible for their own educational experiences. They were also more anxious and frustrated than their control colleagues, particularly over intratutorial conflicts and what and how much to study. This finding was confirmed in questionnaires administered near graduation that asked students retrospectively to choose key words that described their preclinical curricula. “Stressful, engaging, and difficult" were descriptors cited statistically more frequently (using the chi-square statistic, p < .05, n :=: 52) by the NP students as opposed to “nonrelevant, passive, and boring" by the controls. The same instrument showed that the NP students were more than three times as likely as the controls to cite a close relationship with faculty members during their preclinical years.

결론 CONCLUSION:

Students in the new curriculum learned differently, acquired distinctive knowledge, skills, and attitudes, and underwent a more satisfying and challenging preclinical medical school experience without loss of biomedical competence. These findings should encourage other schools to consider such a curriculum.

NBME Part I score 점수비교

Psychosocial Skills 비교

학습유형과 학습환경에 대한 비교

The influence of the New Pathway curriculum on Harvard medical students.

교육 프로그램 평가(AMEE Education Guide no. 29)

AMEE Education Guide no. 29: Evaluating educational programmes

JOHN GOLDIE

Department of General Practice, University of Glasgow, UK

What is evaluation?

Evaluation is defined in the Collins English Dictionary (1994) as “the act of judgement of the worth of …”. As such it is an inherently value-laden activity. However, early evaluators paid little attention to values, perhaps because they naively believed their activities could, and should, be value free (Scriven, 1983). The purpose(s) of any scheme of evaluation often vary according to the aims, views and beliefs of the person or persons making the evaluation. Experience has shown it is impossible to make choices in the political world of social programming without values becoming important in choices regarding evaluative criteria, performance standards, or criteria weightings (Shadish et al., 1991). The values of the evaluator are often reflected in some of the definitions of evaluation which have emerged, definitions that have also been influenced by the context in which the evaluator operated. Gronlund (1976), influenced by Tyler's goal-based conception of evaluation, described it as “the systematic process of determining the extent to which instructional objectives are achieved”. Cronbach (Cronbach et al., 1980), through reflection on the wider field of evaluation and influenced by his view of evaluators as educators, defined evaluation as “an examination conducted to assist in improving a programme and other programmes having the same general purpose”.

In education the term evaluation is often used interchangeably with assessment, particularly in North America. 

• While assessment is primarily concerned with the measurement of student performance, 
• evaluation is generally understood to refer to the process of obtaining information about a course or programme of teaching for subsequent judgement and decision-making (Newble & Cannon, 1994). 

Mehrens (1991) identified two of the purposes of assessment as:

• to evaluate the teaching methods used;
• to evaluate the effectiveness of the course.

Assessment can, therefore, be looked upon as a subset of evaluation, its results potentially being used as a source of information about the programme. Indeed student gain by testing is a widely used evaluation method, although it requires student testing both pre- and post-course.

History of evaluation

With the realization of the political nature of the decision-making process, educational evaluators began to embrace Cronbach's view of the evaluator as an educator, in that he/she should rarely attempt to focus his/her efforts on satisfying a single decision-maker, but should focus those efforts on “informing the relevant political community” (Cronbach, 1982b). They also realized that, while many of their attempts at evaluation did not work, some did and when they worked programme quality improved to varying degrees. Improvement, even when modest, was recognized to be valuable (Popham, 1988).

Effecting programme evaluation

There are a number of steps to be taken in planning and implementing programme evaluation.

Initiation/commissioning

The initial stage of evaluation is where the institutions or individuals responsible for a programme take the decision to evaluate it. They must decide on the purpose(s) of the evaluation, and who will be responsible for undertaking it. There are potentially numerous reasons for undertaking evaluation. Muraskin (1997) lists some of the common reasons for conducting evaluations and common areas of evaluation activity (Table 1).

Chelimsky & Shadish (1997) suggest that the purposes of evaluation, along with the questions evaluators seek to answer, fall into three general categories:

• evaluation for accountability;
• evaluation for knowledge;
• evaluation for development.

Table 2.  Three perspectives and their positions along five dimensions (Adapted from Chelimsky & Shadish, 1997).

The potential cost of the evaluation often plays a major role in determining the scope of the evaluation and identity of the evaluator(s), as the cost will have to be met from the programme budget, or by seeking additional funding. The question of whether the evaluator should be internal or external to the programme's development and delivery is often considered at this point. In order to produce an effective educational evaluation, Coles & Grant (1985) point out that skills from many disciplines, for example psychology, sociology, philosophy, statistics, politics and economics, may be required. They rightly question whether one individual would have the competence to perform all these tasks, and whether an institution would necessarily have these skills in-house.

Defining the evaluator's role

The evaluator(s), having been appointed, must reflect on his/her role in the evaluation. This is important to establish as it will influence the decision-making process on the goals of the evaluation, and on the methodology to be used. It is at this point that the evaluator decides where, and to whom, his/her responsibility lies, and on the values he/she requires to make explicit. The questions to be asked in the evaluation, and their source of origin, will be influenced by these decisions.

The ethics of evaluation

Evaluators face potential ethical problems, for example, they have the potential to exercise power over people, which can injure self-esteem, damage reputations and affect careers. They can be engaged in relationships where they are vulnerable to people awarding future work. In addition, evaluators often come from the same social class and educational background as those who sponsor the evaluations. The ethics of an evaluation, however, are not the sole responsibility of the evaluator(s). Evaluation sponsors, participants and audiences share ethical responsibilities. House (1995) lists five ethical fallacies of evaluation: (평가시에 빠지기 쉬운 윤리적 오류)

• Clientism—the fallacy that doing whatever the client requests or whatever will benefit the client is ethically correct.
• Contractualism—the fallacy that the evaluator is obliged to follow the written contract slavishly, even if doing so is detrimental to the public good.
• Methodologicalism—the belief that following acceptable inquiry methods assures that the evaluator's behaviour will be ethical, even when some methodologies may actually compound the evaluator's ethical dilemmas.
• Relativism—the fallacy that opinion data the evaluator collects from various participants must be given equal weight, as if there is no bias for appropriately giving the opinions of peripheral groups less priority than that given to more pivotal groups.
• Pluralism/Elitism—the fallacy of allowing powerful voices to be given higher priority, not because they merit such priority, but merely because they hold more prestige and potency than the powerless or voiceless.

To assist evaluators a number of organizations including the Joint Committee on Standards for Educational Evaluation (1994); the American Evaluation Association (1995); the Canadian Evaluation Society (1992); and the Australasian Evaluation Society (AMIE 1995) have issued guidance for evaluators undertaking evaluation. Other authors and professional organizations have also implicitly or explicitly listed ethical standards for evaluators, for example, the American Educational Research Association (1992), Honea (1992), and Stufflebeam (1991). Drawing on these, Worthen et al. (1997) have suggested the following standards could be applied: (평가자가 지켜야 할 기준들)

1. Service orientation—evaluators should serve not only the interests of the individuals or groups sponsoring the evaluation, but also the learning needs of the programme participants, community and wider society.
2. Formal agreements—these should go beyond producing technically adequate evaluation procedures to include such issues as following protocol, having access to data, clearly warning clients about the evaluation's limitations and not promising too much.
3. Rights of human subjects—these include obtaining informed consent, maintaining rights to privacy and assuring confidentiality. They also extend into respecting human dignity and worth in all interactions so that no participants are humiliated or harmed.
4. Complete and fair assessment—this aims at assuring that both the strengths and weaknesses of a programme are accurately portrayed.
5. Disclosure of findings—this reflects the evaluator's responsibility to serve not only his/her client or sponsor, but also the broader public(s) who supposedly benefit from both the programme and its accurate evaluation.
6. Conflict of interest—this cannot always be resolved. However, if the evaluator makes his/her values and biases explicit in an open and honest way clients can be aware of potential biases.
7. Fiscal responsibility—this includes not only the responsibility of the evaluator to ensure all expenditures are appropriate, prudent and well documented, but also the hidden costs for personnel involved in the evaluation.

Choosing the questions to be asked

The aims of the evaluation depend not only on the interests of individuals or groups asking them, and the purpose(s) of the evaluation, but also on the views of the evaluator as to his/her role. The work of Cronbach is perhaps the most far reaching in this area. He views the evaluator's role as educator rather than judge, philosopher-king or servant to a particular stakeholder group. In deciding which questions to ask, he advocates asking both all-purpose and case-specific questions. The all-purpose questions depend on the evaluator's assessment of the leverage associated with a particular issue, the degree of prior uncertainty about the answer and the degree of possible and desirable reduction in uncertainty in light of trade-offs among questions, methods and resources. This results in different types of issues prevailing in different programme contexts. His case-particular questions relate to the substantive theories underpinning programme design and investigate why a programme is, or is not, successful, knowledge that not all stakeholders are interested in as they may only desire outcome knowledge, or knowledge specific to their needs. His views place a heavy burden on the evaluator in terms of the methodology being complicated by the range of questions generated (Shadish et al., 1991).

Shadish et al. (1991) supply a useful set of questions for evaluators to ask when starting an evaluation. These cover the five components of evaluation theory and provide a sound practical basis for evaluation planning (boxes 1–5).

• Box 1: Questions to ask about Social programming
• Box 2: Questions to ask about use
• Box 3: Questions to ask about knowledge construction
• Box 4: Questions to ask about valuing
• Box 5: Questions to ask about evaluation practice

Designing the evaluation

Having decided what needs to be done the evaluator has to design an appropriate plan to obtain the data required for the purpose(s) of his/her evaluation.

Dimensions of evaluation

Stake (1976) suggested eight dimensions along which evaluation methods may vary: (평가의 여덟 차원)

• Formative–summative: This distinction was first made by Scriven (1967). Formative evaluation is undertaken during the course of a programme with a view to adjusting the materials or activities. Summative evaluation is carried out at the end of a programme. In the case of an innovative programme it may be difficult to determine when the end has been reached, and often the length of time allowed before evaluation takes place will depend on the nature of the change.
• Formal–informal: Informal evaluation is undertaken naturally and spontaneously and is often subjective. Formal evaluation is structured and more objective.
• Case particular–generalization: Case-particular evaluation studies only one programme and relates the results only to that programme. Generalization may study one or more programmes, but allow results to be related to other programmes of the same type. In practice results may lend themselves to generalization, and the attempt to formulate rules for case study recognizes that generalizing requires greater control, and more regard to setting and context (Holt, 1981).
• Product–process: This distinction mirrors that of the formative–summative dimension. In recent years evaluators have been increasingly seeking information in the additional area of programme impact.
• Process information: In this dimension information is sought on the effectiveness of the programme's materials and activities. Often the materials are examined during both programme development and implementation. Examination of the implementation of programme activities documents what actually happens, and how closely it resembles the programme's goals. This information can also be of use in studying programme outcomes.
• Outcome information: In this dimension information is sought on the short-term or direct effects of the programme on participants. In medical education the effects on participants’ learning can be categorized as instructional or nurturant. The method of obtaining information on the effects of learning will depend on which category of learning outcome one attempts to measure.
• Impact information: This dimension looks beyond the immediate results of programmes to identify longer-term programme effects.
• Descriptive–-judgmental: Descriptive studies are carried out purely to secure information. Judgmental studies test results against stated value systems to establish the programme's effectiveness.
• Pre-ordinate–responsive: This dimension distinguishes between the situation where evaluators know in advance what they are looking for, and one where the evaluator is prepared to look at unexpected events that might come to light as he/she goes along.
• Holistic–analytic: This dimension marks the boundary between evaluations, which looks at the totality of a programme, from one that looks only at a selection of key characteristics.
• Internal–external: This separates evaluations using an institution's own staff from those that are designed by, or which require to satisfy, outside agencies.

Choosing the appropriate design

A range of methods, from psychometric measurement at one end to interpretive styles at the other, has been developed. Table 3 provides a list of common quantitative and qualitative methods and instruments available to educational evaluators.

Table 3.  Common quantitative and qualitative methods and instruments for evaluation.

Shadish (1993), building on Cook's (1985) concept that triangulation should be applied not only to the measurement phase but to other stages of evaluation as well, advocates using critical multiplism to unify qualitative and quantitative approaches. He proposes seven technical guidelines for the evaluator in planning and conducting his/her evaluation:(평가자가 평가 계획 및 수행시 따라야 할 가이드라인)

1. Identify the tasks to be done.
2. Identify different options for doing each task.
3. Identify strengths, biases and assumptions associated with each option.
4. When it is not clear which of the several defensible options is least biased, select more than one to reflect different biases, avoid constant biases and overlook only the least plausible biases.
5. Note convergence of results over options with different biases.
6. Explain differences of results yielded by options with different biases.
7. Publicly defend any decision to leave a task homogenous.

Approaches to evaluation

With the explosion in the numbers of approaches in recent years, many of which overlap, a number of attempts have been made to categorize the different evaluation approaches. One of the most useful was developed by Worthen et al. (1997), influenced by the work of House (1976, 1983). They classify evaluation approaches into the following six categories: (평가의 여섯 가지 접근법)

• Objectives-oriented approaches—where the focus is on specifying goals and objectives and determining the extent to which they have been attained.
• Management-oriented approaches—where the central concern is on identifying and meeting the informational needs of managerial decision-makers.
• Consumer-oriented approaches—where the central issue is developing evaluative information on ‘products’, broadly defined, for use by consumers in choosing among competing products, services etc.
• Expertise-oriented approaches—these depend primarily on the direct application of professional expertise to judge the quality of whatever endeavour is evaluated.
• Adversary-oriented approaches—where planned opposition in points of view of different evaluators (for and against) is the central focus of the evaluation.
• Participant-oriented approaches—where involvement of participants (stakeholders in the evaluation) is central in determining the values, criteria, needs and data for the evaluation.

These categories can be placed along House's (1983) dimension of utilitarian to intuitionist-pluralist evaluation (Figure 1). 

Utilitarian approaches determine value by assessing the overall impact of a programme on those affected, 

whereas intuitionist-pluralist approaches are based on the idea that value depends on the impact of the programme on each individual involved in the programme.

Figure 1. Distribution of the six evaluation approaches on the utilitarian to intuitionist–pluralist evaluation dimension. (직관론자-다원론자 차원에 배치)

Similarly, to provide a complete description of each example would be beyond the scope of this guide. To assist readers in choosing which of the approaches might be most helpful for their needs, the characteristics, strengths and limitations of the six approaches are summarized in Table 5. These are considered under the following headings after Worthen et al. (1997):

• Proponents—individuals who have written about the approach.
• Purpose of evaluation—the intended use(s) of evaluation proposed by writers advocating each particular approach or the purposes that may be inferred from their writings.
• Distinguishing characteristics—key descriptors associated with each approach.
• Past uses—ways in which each approach has been used in evaluating prior programmes.
• Contribution to the conceptualization of an evaluation—distinctions, new terms or concepts, logical relationships and other aids suggested by proponents of each approach that appear to be major or unique contributions.
• Criteria for judging evaluations—explicitly or implicitly defined expectations that may be used to judge the quality of evaluations that follow each approach.
• Benefits—strengths that may be attributed to each approach and reasons why one might want to use this approach.
• Limitations—risks associated with use of each approach.

Table 5.  Comparative analysis of the characteristics, strengths and limitations of the six categories (after Worthen et al., 1997).

Interpreting the findings

Having collected the relevant data the next stage in evaluation involves its interpretation. Coles & Grant (1985) view this process as involving two separate, though closely related activities: analysis and explanation.

Such meta-evaluations assert that all evaluations can be evaluated according to publicly justifiable criteria of merit and standards of performance, and that the data can help determine how good an evaluation is. The need for meta-evaluation implies recognition of the limitations of all social science, including evaluation (Hawkridge, 1979). Scriven (1980) developed the Key Evaluation checklist, a list of dimensions and questions to guide evaluators in this task (Table 6). (평가 체크리스트)

Table 6.  Key evaluation checklist.

Having analysed the data, the evaluator needs to account for the findings. In education the researcher accounts for the findings by recourse to the mechanisms embodied in the contributing disciplines of education (Coles & Grant, 1985). As few individuals have expert knowledge of all the fields possibly required, specialist help may be required at this point. This again has resource implications for the evaluation. Shadish et al.'s (1991) questions (boxes 2–5) also offer evaluators salient points to consider when interpreting the results of their evaluation.

Dissemination of the findings

Again Shadish et al.'s questions on evaluation use (box 2) are of value in considering how, and to whom, the evaluation findings are to be reported. Reporting will be in some verbal form, written or spoken, and may be for internal or external consumption. It is important for the evaluator to recognize for which stakeholder group(s) the particular report is being prepared. Coles & Grant (1985) list the following considerations: (보고서 작성시 유의해야 할 사항)

1. Different audiences require different styles of report writing.
2. The concerns of the audience should be reviewed and taken into account (even if not directly dealt with).
3. Wide audiences might require restricted discussion or omission of certain points.
4. The language, vocabulary and conceptual framework of a report should be selected or clarified to achieve effective communication.

Evaluators require to present results in an acceptable and comprehensible way. It is their responsibility to persuade the target audience of the validity and reliability of their results. Hawkridge (1979) identified three possible barriers to successful dissemination of educational research findings: (교육연구 결과물을 배포하는데 있어서 장애요인)

1. The problem of translating findings into frames of reference and language which the target audience can understand. However the danger in translating findings for a target audience is that the evaluator may as a result present the findings in a less than balanced manner.
2. If the findings are threatening to vested interests, they can often be politically manoeuvred out of the effective area.
3. The ‘scientific’, positivistic, approach to research still predominates in most academic institutions, which may view qualitative research methods and findings as ‘soft’, and be less persuaded by their findings. As qualitative methods receive greater acceptance this is becoming less of a problem.

A further problem concerns the ethics of reporting. As Coles & Grant (1985) suggested in their consideration of how—and to whom—to report, dissemination of information more widely may require to be censored, for example, information about a particular teacher would not usually be shared with anyone outside a select audience. The evaluator also has to be aware that the potential ramifications of a report may go wider than anticipated, for example into the mass media, where this may not be desired.

Influencing decision-making

As has been touched upon earlier, the initial enthusiasm of the 1970s educational evaluators became soured by the realization of the political nature of the educational decision-making process, and by the inconclusive results that were often obtained. Coles & Grant (1985) suggest the following ways in which evaluators can effect the educational decision-making process: (평가자들이 교육적 의사결정과정에 영향을 줄 수 있는 방법)

1. involving the people concerned with the educational event at all stages of the evaluation;
2. helping those who are likely to be associated with the change event to see more clearly for themselves the issues and problems together with putative solutions;
3. educating people to accept the findings of the evaluation, possibly by extending their knowledge and understanding of the disciplines contributing towards an explanation of the findings;
4. establishing appropriate communication channels linking the various groups of people involved with the educational event;
5. providing experimental protection for any development, allocating sufficient resources, ensuring it has a realistic life expectancy before judgements are made upon it, monitoring its progress;
6. appointing a coordinator for development, a so-called change agent;
7. reinforcing natural change. Evaluation might seek out such innovations, strengthen them and publicize them further.

AMEE Education Guide no. 29: evaluating educational programmes.

전 세계의 의과대학 개관

Overview of the world’s medical schools: an update

Robbert J Duvivier, John R Boulet, Amy Opalek, Marta van Zanten & John Norcini

Introduction

의료인력의 분포는 모성사망률과 같은 건강지표와 연관되어 있다. WHO는 전 세계적으로 57개국이 절대적인 의사, 간호사, 조산사 부족 상태에 있다고 하였으며 예방접종과 같은 이는 필수적인 의료 서비스를 제공할 보건의료인력이 부족함을 의미한다. 이러한 인력자원 문제를 해결하기 위한 여러가지 전략이 제시되었다.

The distribution of health workers is associated with health outcomes such as maternal and infant mortality.[1] The World Health Organization (WHO) estimates that 57 countries worldwide have an absolute shortage of 2.3 million physicians, nurses and midwives,[2] meaning they have insufficient numbers of health professionals to deliver essential medical care, such as skilled attendance at birth and immunisation programmes.[3] Because of this deficit, several strategies have been proposed to address the human resource crisis, including the increased production of community health workers and non-physician clinicians,[4] and task shifting[5, 6] to improve the effectiveness and efficiency of use of available workers.

어떤 보건의료시스템도 임상과 공공보건 영역의 잘 훈련된 의사 없이 작동할 수 없다. 더 많은 의사를 교육하기 위해서는 더 많은 의학교육 프로그램이 생겨야 한다. 현재 1년간 양성되는 의사의 추정되는 숫자는 386200명부터 1000000명 사이이다. 전 세계적인 의학교육의 팽창은 양적, 질적으로 모두 필요하며 이렇게 의학교육에 투자하기 위한 전략은 많더라도 현재 상태에 대한 충분한 정보는 부족하다. 

No health care system can function well without adequately trained doctors to participate in clinical and public health work. Educating greater numbers of doctors will demand significant growth in the number and capacity of medical education programmes. Estimations of the current annual global output of medical graduates range between 386 200[7] and 1 000 000.[8] A global scale-up of medical education should include increases in both the quantity and quality of doctors of the future.[9-11] Unfortunately, although interest in strategic investment in medical education has been growing,[12] there is insufficient information about the current status, capacity and content of medical programmes let alone about ongoing trends within medical education, across the world. Without this information and an understanding of health worker migration patterns,[13, 14] it will be difficult to develop sound workforce policies.

보건의료인력에 대한 자료 부족은 여러 정부들에 있어서 주요한 문제 중 하나이다. 의료 및 교육 정책을 더 효과적으로 하기 위해서는 양질의 최신의 정보가 필요하다. 

This paucity of health care worker data is a major challenge for governments, United Nations (UN) agencies, donor organisations and non-governmental organisations (NGOs) seeking to address shortages in physician supply,[15, 16] which can be quite daunting in some regions of the world. To ensure greater efficiencies in health and education policy planning, quality up-to-date information is urgently required. This paper aims to address this knowledge gap by describing the information base that can be consulted to examine the status of medical schools around the world. It uses an innovative approach to combine resources to provide the best available data on medical schools.

Knowing more about the number and characteristics of medical schools will help to promote meaningful health workforce policies.

Background

Historically, the WHO kept records of the number of medical schools in the world through its World Directory of Medical Schools.[17] 

The World Directory of Medical Schools lists training institutions by country and provides some descriptions of national medical education systems. The first edition was published in 1953 and the seventh in 2000. The number of medical schools listed increased over this 47-year span from 566 to 1642, a growth of 190%. Since 2000, the WHO has published additional data on its website, but no new schools have been added to the directory since 2004.

In the 21st century, several other organisations have tried to fill the void left by the discontinuation by the WHO of the World Directory of Medical Schools. A register of medical schools produced by the International Institute for Medical Education (IIME), established by the China Medical Board of New York, showed 1849 medical schools in 166 countries of the world in 2006.[18] 

This database has not been updated since. The information on medical schools was gathered through a survey conducted in 2000 that sought data on admission requirements, enrolment numbers, assessment methods and curriculum content. Regrettably, this additional information is currently not available in the public domain.

Various organisations provide partial listings of the world's medical schools, or more comprehensive data at a national level. For example, agencies such as the Medical Council of India[19] and the Medical and Dental Council of Nigeria[20] provide lists of the medical schools recognised or accredited by these bodies in their respective countries. In other countries, data about medical programmes can be found in national public databases of higher education institutions (e.g. Brazil[21] and Italy[22]). For medical schools worldwide, the International Federation of Medical Students’ Associations created a curriculum database in 2005. It included country-based data on numbers of medical schools, as well as additional information pertaining to training periods and numbers of graduates. Unfortunately, the website is no longer available.

FAIMER와 IMED. 의과대학에 대한 최신의 정보를 갖출 수 있는 데이터베이스

The WHO's 2000 listing of medical schools was updated in 2007 by the World Federation for Medical Education (WFME) and Copenhagen University, and established as a new database, the Avicenna Directories.[23] Currently, the number of medical schools included is 2147. In 2002, the Foundation for Advancement of International Medical Education and Research (FAIMER) established the International Medical Education Directory (IMED).[24] 

It contains information on medical schools that are recognised by the appropriate government agencies in the countries in which the schools are located. A medical school is listed in the IMED only after FAIMER verifies that the school is recognised by a ministry of health, ministry of education, accreditation body or other appropriate agency. 

International medical graduates (IMGs) who seek educational opportunities in the USA (i.e. residency training) must be certified by the Educational Commission for Foreign Medical Graduates (ECFMG). One of the eligibility requirements for ECFMG certification is that the candidate must have attended a medical school listed in the IMED. Other countries, such as Canada, use the IMED for establishing eligibility for registration. As a result, the IMED is continuously updated and data on closed schools are retained for historical reference. In 2012, the WFME and FAIMER agreed to combine the Avicenna and IMED directories to form a more comprehensive single directory of medical schools. This new consolidated directory, developed in collaboration with the WHO and the University of Copenhagen, is currently in development and will be known, like the historical WHO database, as the World Directory of Medical Schools.[25]

2002년과 2007년 자료의 간략한 리뷰

In the scientific literature, relatively little attention has been paid to the number or characteristics of medical schools worldwide. A comprehensive search yielded two papers that provide a general overview of the world's medical schools. 

• In 2002, Eckhert[7] tracked the number and distribution of medical schools using publicly available information and the WHO directory. The resulting paper described 1642 medical schools in 157 countries.[7] Fifty-five countries listed one medical school and 37 countries had none. At that time, three countries each had more than 100 medical schools: China (n = 150), India (n = 144) and the USA (n = 144; 125 allopathic and 19 osteopathic). 
• In 2007, Boulet et al.[26] described the IMED database and then-current listings of operating medical schools. They reported a total number of 1935 medical schools located in 169 countries.[26] Over one-third were located in five countries (India, the USA, China, Brazil, Japan), and nearly half were located in 10 nations (the aforementioned plus Mexico, Russia, South Korea, Iran and France). India had, at that time, the most medical schools (n = 219), followed by the USA (n = 147) and China (n = 130).

Given the difficulty of obtaining current information, and somewhat fragmented data resources, it seems timely to aggregate available records and provide an update of the numbers and locations of medical schools around the world. Although physicians do not provide the majority of patient care in many parts of the world, and physician production varies from one medical school to the next, gaining a better understanding of how medical education resources are currently distributed is a necessary step for health workforce planning and development. Moreover, through the process of combining data resources, areas for which information is lacking or inconsistent can be identified. This knowledge is essential for constructing and maintaining accurate medical school databases that can be used to support workforce and education policy initiatives.

Table 1. 가장 많은 의과대학을 보유한 국가들

Table 2. WHO 분류 지역별 의과대학의 수 (+ 한 의과대학당 인구)

Table 3. needs가 가장 높은 국가들의 의과대학 숫자

Table 4. 100만명당 의과대학 수 (상위 20개국, 하위 20개국)

Conclusion
전체적으로 보면, 졸업생의 분포와 그들이 궁극적으로 어느 장소에서 의료행위를 하는가가 의과대학이 어디에 있는지보다 보건의료시스템에 주는 영향이 더 크다. 그러나 지금까지 문헌을 살펴보면 해당 지역에서 졸업한 의사가 그 지역에 머물 가능성이 더 높다. 가장 요구도가 높은 15개 국가가 사하라 이남 아프리카에 있고, 각각에 단 하나의 의과대학밖에 없다는 것을 볼 때, 이 지역에 의과대학 설립이 필요한 것은 자명해보인다.

Overall, the distribution of their graduates, and where they ultimately settle to practise medicine, are more likely to impact the health care system than the locations of medical schools themselves.[31] The existing literature on physician migration shows, however, that doctors who are trained in regions of need are more likely to stay and practise in those areas after graduation.[32-34] Given that 15 of the countries with the highest need (i.e. with the lowest physician density per 10 000 population) are located in sub-Saharan Africa and have only one medical school each, the rationale for increasing their training capacity seems obvious.[12] 

이렇게 의과대학의 숫자와 졸업생에 강조를 하는 것은 이 논의를 벗어나는 보건의료인력의 더 넓은 부분들을 간과할 수 있지만, 다른 말로 풀어보면 이러하다. 조기 은퇴, 이민, 해고와 같은 예방가능한 의료인력의 감소의 효과를 볼 때, 많은 나라에서 양성되는 의사의 수보다 일하고 있는 의사의 수가 적음을 의미한다. 관리가 잘 되지 않으면 배치가 잘 되지 않고, 자리를 비우고(absenteeism), 유령 인력을 양성하게 된다. 그럼에도 불구하고 의사가 양성되는 이러한 '파이프라인'에 대한 정확한 정보 없이 각 정부는 타당한 정책을 수립하지 못할 것이다.

This emphasis on numbers of schools and numbers of graduates foregoes wider aspects of human resources for health that fall outside the scope of this discussion, but are worth paraphrasing. The effect of the preventable exit of professionals from the workforce, such as exits for early retirement, emigration and retrenchment, means the number of working physicians is lower than the number of physicians produced in many countries.[35, 36] Poor management results in suboptimal deployment, absenteeism and ‘ghost’ workers (i.e. physicians who appear on payrolls while being registered as off-post).[37] Nonetheless, without accurate information on the ‘production pipeline’ of physicians, national governments will not be able to develop sound policy.

Overview of the world's medical schools: an update.

Investigating the use of sampling for maximising the efficiency of student-generated faculty teaching evaluations

Clarence D Kreiter1 & Venkatesh Lakshman2

Purpose  

학생을 대상으로 설문을 하는 것은 강의나 교수 평가를 위해서 많이 사용되는 방법이다. 학생의 의견을 정확히 반영하기 위한 정보 수집을 위해서는 학생들을 평가 과정에 참여시키고 평가에 대한 사려깊은 응답을 제공해야 한다. 학생의 헌신을 유지하기 위해서 학생들이 '대충 설계된 평가(poorly planned evaluations)'를 하느라 부담을 느끼지 않도록 해야한다. 

Surveys of medical students are widely used to evaluate course content and faculty teaching within the medical school. Gathering information that accurately reflects student perceptions requires that students buy into the evaluation process and be willing to provide thoughtful responses to the teaching evaluation. To maintain student commitment, it is important that medical students are not overburdened with poorly planned evaluations. Sampling might decrease the number of evaluations required of students and might also reduce the proportion of non-responses and other forms of inattentive response biases.

Methods  

A sampling technique employed within a large medical lecture is described and evaluated. A generalisability study of the teacher evaluations is conducted.

Results  

A high response rate and high levels of reliability were obtained by sampling a small proportion of the total class. The largest source of error was related to rater and utilising sufficient numbers of student-raters is critical to achieving reliable results.

Conclusion  

Sampling can reduce evaluation demands placed on students, and preserve reliability and increase the validity of mean evaluation scores. With computer presentation, efficient sampling techniques become practical and should be part of software packages used to present teacher evaluations.

Investigating the use of sampling for maximising the efficiency of student-generated faculty teaching evaluations.

Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical school

Chris Roberts,1 Merrilyn Walton,1 Imogene Rothnie,1 Jim Crossley,2 Patricia Lyon,1 Koshila Kumar1

& David Tiller3

Introduction

전세계적으로 의과대학은 최고의 학생을 뽑아서 좋은 의사로 만들어내고자 한다. 그러나 '낮은 탈락률'이 의미하는 바는 한 번 입학하면 대부분의 학생이 그들의 인성이나 전문직업적 특성과 관련없이 의사가 된다는 것을 의미한다. 선발 과정은 의과대학에 있는 모든 평가 중에서도 분명히 가장 high-stakes, highly stressful, resource intensive한 평가이다. 일반적으로 선발과정에는 학업적 능력이 일부 포함되며, 지원자의 인성을 평가하기 위한 면접이나 추천서 등이 포함된다. 학생의 '성적'이란 이전 교육과정에서 학생이 얼만큼의 수행능력을 보였는가를 보여주는 것이고, 여러 연구에서 지속적으로 의과대학에서의 미래 수행능력을 예측하는 가장 우수한 예측인자로 보고되고 있다. 

Worldwide, medical schools aim to select the best students into their programmes and consequently expect to produce good doctors. Low attrition rates mean, however, that, once admitted, most students graduate as doctors regardless of their personal and professional characteristics.1 Selection procedures are arguably the most high-stakes, highly stressful and resource-intensive of all medical school assessments. They generally include some measure of academic ability (the ‘marks’) and some measure of a candidate’s personability as assessed in an interview or letter.2 Student ‘marks’ reflect past performance over a number of years of previous education and are consistently the best predictor of future performance, whether at medical school, or, for example, in North American licensing examinations.2

좋은 의사를 만드는 것은 좋은 성적 뿐만이 아니며, 대부분의 의과대학이 면접 등과 같은 방식으로 지원자의 가치관/헌신/비인지적 특성을 평가하고자 노력하고 있다. 그러나 면접은 미래 수행능력(학생이든, 의사든)을 예측하는데 뚜렷한 가치를 보여주지 못해서 입학과정의 중요한 요소로서의 공정성이 훼손되고 있다는 지적을 받았다. 면접에 대한 Psychometric studies를 보면 매우 다양한 결과가 나타나는데, 이는 신뢰도의 정의가 서로 다르고, 서로 연구방법론이 다르며, 소수의 연구만이 generalisability approach를 사용했기 때문이다.

It takes more than good marks to make a good doctor and most schools attempt to assess values and commitment and other important non-cognitive characteristics of candidates in some form of interview. However, the interview is of limited value in predicting anything about future performance, either as a student or as a doctor,2,3 which undermines its fairness as an important part of admissions procedures.3 Psychometric studies of interviews have produced highly variable results, largely because of differing definitions of reliability and differing research methodologies, and only a few studies have used a generalisability approach.3–6

다면인적성면접은 면접의 신뢰도에 대한 우려에서부터 시작되어 비교적 새롭게 등장한 평가방법이다. 이는 OSCE형식을 가져와서 긴 면접이 갖는 문제를 피해가고자 했다. 즉, 지원자에 대한 점수가 제한된 면접 주제 및 면접관에서 오는 편향에 영향을 받는다는 것이다. 이러한 면접 방식은 ‘stable qualities within candidates that have a high probability of occurring in an infinite range of contexts’이라는 인식에 도전하는데, MMI가 내용과 독립된 평가자라는 두 가지 측면에서 한 평가자가 갖는 단점을 극복하여 지원자의 행동에 대해서 더 신뢰도높게 generalisation이 가능하다.

The multiple mini-interview (MMI)5 is a relatively new assessment tool which addresses concerns about interview reliability. It uses the objective structured clinical examination (OSCE) format, and so avoids the issues of the long interview (cf. the long case in clinical competence), where much of the observed mark of the candidate relates to biases from the limited interview content and the interviewer panel.3 It challenges the notion that the interview can test ‘stable qualities within candidates that have a high probability of occurring in an infinite range of contexts’1 by confirming the issue of context specificity.5,6 Because the MMI tests a larger sample of both content and independent interviewers than a single interview can, more reliable generalisations about a candidate’s behaviour can be made.

원래 MMI가 개발된 센터 바깥에서 진행된 Pilots를 보면 입학생들에 대한 그들 나름의 비인지적 특성에 대한 framework를 개발하였다. 이번 연구에서 우리는 의학교육 연속체를 가로지르는 professionalism의 하나로서 pre-professionalism을 평가하고자 하였다. McMaster의과대학의 MMI역시 미래의 의과대학생, 그리고 미래의 의사로서 수행능력을 예측하는 상관관계가 우수하다는 예측타당도를 주장한 바 있다.

Pilots of MMIs conducted outside the original centre have developed their own frameworks to establish the preferred non-cognitive characteristics of entry-level students.7 In our study we assumed that we were measuring the behaviours of candidates that have been variously linked to frameworks of professionalism which cut across the medical education continuum,8 and have been called pre-professionalism, to reflect the potential of entry-level students for professionalism. The McMasters University (Hamilton, ON, Canada) MMI also claims predictive validity in that it makes good correlations with future performance as a medical student9 and as a doctor.10

우리 연구의 목적은 의학전문대학원 프로그램 선발에 있어서 면접관이 타당하고 신뢰도 높은 판정을 내릴 수 있는가를 pre-professionalism framework를 사용하여 연구하고자 했다. 또한 MMI의 어떤 특성이 가장 유용한지도 알아보고자 했다.

The aim of our study was to establish whether interviewers can make reliable and valid decisions about applicants when selecting candidates for entry to a graduate-entry medical programme, using a pre-professionalism framework and the MMI format. Secondly, we wanted to know which features of the MMI were most useful in guiding admissions committees to focus their resources in making robust decisions about candidates.

Methods  

Data came from a high-stakes admissions procedure. Content validity was assured by using a framework based on international criteria for sampling the behaviours expected of entry-level students. A variance components analysis was used to estimate the reliability and sources of measurement error. Further modelling was used to estimate the optimal configurations for future MMI iterations.

Results  

This study refers to 485 candidates, 155 interviewers and 21 questions taken from a pre- prepared bank. For a single MMI question and 1 assessor, 22% of the variance between scores reflected candidate-to-candidate variation. The reliability for an 8-question MMI was 0.7; to achieve 0.8 would require 14 questions. Typical inter-question correlations ranged from 0.08 to 0.38. A disattenuated correlation with the Graduate Australian Medical School Admissions Test (GAMSAT) subsection ‘Reasoning in Humanities and Social Sciences’ was 0.26.

Conclusions

In a high-stakes admissions procedure performed outside the original centre, on a large sample, using generalisability theory, 

• we confirmed that the MMI is a moderately reliable method of assessment. 
• We established the construct validity of the MMI by showing a small positive correlation with GAMSAT section scores for ‘Reasoning in Humanities and Social Sciences’ and ‘Written Communication’. 
• The largest source of identifiable measurement error relates to aspects of interviewer subjectivity, suggesting that further training of interviewers would be beneficial. 
• Applicant performance on one question did not correlate strongly with performance on another question, demonstrating the importance of context specificity when testing professional behaviours. 
• Multiple mini-interviews must have a sufficient number of questions for precise comparison for ranking purposes because of the size of the measurement error. 
• We demonstrated that a significant proportion of students with high GPAs and GAMSAT scores can fail an MMI. 

Further research is required into the construct and predictive validity of the MMI in order to justify its long-term use, and to establish the impact of training on measurement error through careful experimental design.

Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical school.

Anxiety in medical students: is preparation for full-time clinical attachments more dependent upon differences in maturity or on educational programmes for undergraduate and graduate entry students?

K Hayes, A Feather, A Hall, P Sedgwick, G Wannan, A Wessier-Smith, T Green & P McCrorie

Introduction

의과대학생에게 있어서 불안과 스트레스는 언제나 있어왔으며, 여기에는 임상실습으로 이행하는 과정의 불안감도 포함된다. 최근의 질적 연구 결과를 보면, 이러한 이행 시기가 특히 더 스트레스를 유발하는 것으로 나타난다.

Anxiety and stress have long been identified in medical students,1,2 including anxieties specific to the change to full-time clinical studies.3 A recent qualitative study has again highlighted these transition periods as being particularly stressful for students.4

대부분 교육과정에서 첫 2년(1학년, 2학년)에도 기초의학과목과 임상경험을 통합한 형태의 교육이 이뤄지고 있으나, 전일제(full-time)으로 임상현장에 있는 것은 매우 흥미진진하면서도 걱정이 되게 만드는 상황이다. '학생 중심'에서 '환자 중심'으로 교육의 초점이 이동하게 되고, 그리고 지금까지의 익숙한/구조화된/안전한 교육환경은 덜 조직화되고 가끔은 intimidating한 임상 현장으로 변화하게 된다. 추가적으로 학생들은 임상술기를 익히고 보일(demonstrate)줄 알아야 한다.

Although there is some integration of basic medical sciences and clinical contact in the first 2 years of most courses, the transition to full-time clinical attachments can be both an exciting and a worrying time for students. The focus of the educational process changes from being student- to patient-centred and the previously familiar, structured and safe learning environment changes to that of the less organised and often intimidating clinical arena. In addition, students are challenged by the need to acquire and regularly demonstrate clinical skills, including history taking, examination, giving information and performing practical procedures.

임상술기에 대한 introductory 과정이 스트레스와 불안을 줄여주는 것으로 나타나지만, 이 시기에 학생이 스스로 얼마나 준비되었다고 느끼는지에 영향을 주는 요인에 대한 연구는 적다.

Introductory clinical skills courses have been shown to reduce stress and anxiety5 but there is little in the literature about what influences how prepared students feel during this time.

의학전문대학원 프로그램은 영국에서 빠르게 확산되기 시작하였으며, 전통적인 의과대학 과정과는 다르다. GEP는 PBL과 같은 다른 교수법을 사용하는 경향이 많고, 더 기간이 짧으며, 더 임상적으로 관련된 교육과정을 다르고, 임상과 의사소통 기술을 완전히 통합한 프로그램을 운영한다. 이러한 변화는 학생들이 '임상적으로 준비될 수(clinically prepared)' 있도록 하는 목적이 있는데, 특히 GEP학생들이 3년의 임상실습 경험을 쌓는 의과대학 학생들과 달리 2년간밖에 시간이 없기 때문이기도 하다. 이러한 변화와 동시에 GEP에 참여하는 의과대학에는 인력과 재정에도 큰 영향을 주었다.

Graduate entry programmes (GEPs) are expanding rapidly in the UK and differ considerably to long-established traditional undergraduate courses; GEPs tend to involve different methods of delivery (e.g. problem-based learning), shorter and more clinically relevant curricula and fully integrated clinical and communication skills programmes. These changes are chiefly designed to make students more clinically prepared at all stages of their training, particularly as GEP students only have 2 full clinical years (versus 3 for undergraduates). Alongside the changing educational culture there have also been considerable staffing and funding implications for participating medical schools.

This paper examines student perspectives of the positive and negative influences leading up to this transition and compares the levels of anxiety and specific anxieties experienced by 2 sets of students at a London medical school.

Are we simply achieving the same outcome in different ways or do GEP students really feel better prepared for full-time clinical studies than their undergraduate (UG) colleagues? And if so, what are the most important factors?

Background

St George's Hospital Medical School (SGHMS) has been running parallel UG and GEP MBBS courses since 2000. The 2 courses have markedly different entry criteria, resulting in different student characteristics. 

• The UG course is dominated by students entering after their A-levels (or equivalent examinations), but includes a few university graduates (5%), all of whom have completed a science degree or higher qualification. Students on the UG course are expected to graduate in 5 or 6 years. At the time of the study their median entry age was 19 years.
• The graduate entry programme (among the first in the UK) reflects the UK government's determination to increase the number of UK graduating doctors in keeping with the European Working Time Directive.6 The students are unique in that they may hold any university degree (2 : 2 and above), but are selected primarily on their performance in the GAMSAT (Graduate Australian Medical Schools Admission Test) assessment7 and a structured interview. They are expected to graduate in 4 years. At the time of the study, their median age on entry was 25.5 years. The 2 courses run in parallel and are based on different educational models.

• The UG course is a relatively traditional systems-based course delivered in the early years predominantly by lectures and illustrative clinical case scenarios. 
• The GEP course is delivered using problem-based learning (PBL) cases (New Mexico PBL model) and uses contextualised learning8 wherever possible, integrating clinical skills throughout the first 2 years. Central to the learning process is revisitation of material based on Harden et al.'s spiral curriculum.9 
• The UG course, whilst introducing some early clinical skills, focuses on a 3-week Introduction to Clinical Practice (ICP) programme at the beginning of Year 3. Despite these differences, both courses will share the same finals assessment with a view to fitness to practise.

Statistical analyses

Statistical analyses were performed using spss Version 10. As distributional assumptions could not be made, students on the 2 courses were compared in age using the Mann–Whitney U-test (test statistic approximated to the normal distribution and denoted by z). Distribution of gender was compared between courses using the chi-squared test (test statistic denoted by ÷2). The 2 courses were compared on the 13 statements of anxiety using the Mann–Whitney U-test (test statistic approximated to the normal distribution and denoted by z). All reported P-values are 2-sided and given to 3 significant decimal places. The critical significance level was 0.05. A large number of significance tests were conducted overall but because of type I errors the results of such tests are difficult to interpret. No doubt these multiple statistical tests are correlated and the Bonferroni correction factor is inappropriate as it is conservative and may miss real differences.10,11 Therefore, a subjective approach was applied. More weight was given to comparisons yielding very small P-values than those close to the 5% significance level.

A total anxiety score was derived for each student by summing the student's scores across the 13 anxiety statements, with higher scores reflecting less anxiety. For questions 3, 10, 11 and 12, the scores were reversed because the questions were phrased negatively. An analysis of covariance was undertaken in which total anxiety was regressed on gender, course and age. The type III sum of squares was reported for each main effect and the associated significance test was therefore adjusted for all other main effects. A histogram of the residuals verified the assumption of normality.

Two open questions allowed students to discuss both negative and positive aspects that may have influenced them. Two authors (KH and AF) individually reviewed all free text comments and grouped them qualitatively into common recurring themes. We then compared our groupings and agreed on the main common themes. We selected certain individual comments that best reflected student feelings.

Results

• Demographic data

• Anxiety statements

• Analysis of covariance

Discussion

불안과 스트레스에 대처하는 방법을 익히는 것은 매우 중요하며, 긍정적 측면도 있을 수 있다. 그러나 과도하면 문제가 될 수 있고 여기에 영향을 주는 요인을 이해할 필요가 있다.

Learning to deal with anxiety and stress is an everyday part of a doctor's working life. 

• Indeed, anxiety and stress can have a positive influence, particularly with assessment, as an overly relaxed attitude may lead to complacency and a lack of work.13 
• However, excessive stress and anxiety may lead to poor assessment performance and clinical skills acquisition.14 

Clearly, factors that may ultimately lead to deficiencies in clinical performance and possibly in patient care need to be prevented wherever possible.

핵심 질문은 불안점수 총합이 나이 때문인가, 아니면 교육과정 때문인가 하는 것이다.

The key question is whether the significant differences in total anxiety scores were simply due to increased GEP maturity (on average 6.5 years older), or to the differences in curricula design (PBL versus non-PBL) and the markedly different clinical skills teaching programmes.

성인학습이론에 따르면 더 나이가 많을 수록 학습에 대한 접근 방법이 달라진다고 본다. 의학전문대학원으로 들어온 학생들이 더 동기부여가 잘 되어있고, 학습 기술이 더 성숙하며, 자기주도성이 강하고, 더 많은 삶의 경험을 가지고 있다. 그러나 부정적인 요소도 있는데 고착화된 학습법을 사용할 수 있고, 경제적 문제에 더 신경을 쓸 수도 있고, 과학 배경지식이 약할 수 있다.

Adult learning theory assumes that older students will be mature in their approaches to learning.15 Graduate entry students have potentially higher motivation (career change and self-funding issues), more mature learning skills, increased self-direction and more ‘life experience’.15 However, some negative factors include fixed learning approaches, greater financial concerns16 and a limited scientific background at enrolment.8

나이를 보정하고도 불안 정도에는 차이가 있었다. 따라서 성숙도가 두 그룹 사이에 차이를 주는 요인은 아니라고 할 수 있다.

A significant difference between courses in student anxiety levels persisted after adjusting for age. It would appear, therefore, that maturity is not the main factor contributing to differences between the 2 groups.

이렇게 연관성이 없다는 것이 영국에서 의학전문대학원 체제가 확장되고 있는 것을 지지하는 근거도 반대하는 근거도 아니다. 그러나 최근 Norman이 지지한 성인학습이론에 의문을 던질 수는 있는데, Norman은 주로 차이를 만드는 것은 학습환경이지 학습자의 내재적 차이가 아니라고 하였다.

This lack of association neither supports nor detracts from the growth of graduate entry in the UK. It does, however, cast further doubt on the support for adult learning theory recently championed by Norman,17 who considers it to be the learning environment that makes the main difference rather than any intrinsic differences between learners.

성별 분포는 양쪽에서 비슷하였기 때문에 차이를 설명하지 못한다. 그러나 남성이 여성보다 불안감을 덜 느끼는 것으로 나타났으며 이전 연구와도 부합하는 결과이다.

Gender distribution was similar on both courses, so this does not explain the differences either. Interestingly, however, men expressed significantly less anxiety compared to women, in keeping with reports in previous literature comparing male and female medical students.18

GEP의 integrated practical and communication skills 

The GEP course provides fully integrated practical and communication skills throughout the first 2 years, compared to mainly the ICP programme on the UG course. The previously well supported acquisition, practice and continuous assessment − arguably the most important factor − of skills may have left the GEP students feeling significantly more confident in this area, as was borne out in their free text responses. The support and enthusiasm of teachers has previously been identified as a major influence on student acquisition of clinical skills.19 Interestingly, staff support was more important for GEP than UG students and the much smaller GEP cohort probably made it easier for students to form closer relationships with teaching staff.

Conversely, peer support was more often identified by UG students as helping to relieve anxiety, in keeping with reports of other undergraduate studies.20

These results suggest that early integration of clinical skills, placed in context, well supported by clinical and communication skills teachers, with appropriate assessment, are essential elements of a modern medical school curriculum.

영국 의대생을 대상으로 한 조사에서 거의 40%가까운 학생이 건강상이나 개인상 이유로 학교를 떠난다고 했는데, 현재 이러한 문제에 대한 지원은 UG모델을 기반으로 하면서, GEP에서도 동일하게 적용될 것이라고 가정하는 경우가 많다. 그러나 본 연구결과 두 그룹은 상당한 차이가 있는 것으로 보인다.

One UK study of medical student wastage reported nearly 40% leaving due to poor health (particularly psychological) and personal problems (many related to course stressors).21 A recent review of the literature concerning stress management in medical education concluded that there was proven benefit for students who used available methods of support.22 Much of the current provision is based on UG models and assumes that graduate entrants need the same resources. As there are major differences between the 2 groups, it may be that a more tailored approach to personal support is required for each of them.

Hill et al.,23 examining whether junior doctors felt prepared for hospital practice, found that non-traditionally (PBL) taught medical students expressed greater confidence around communication and general clinical aspects of care than those who had been taught traditionally. They attributed these differences to different student selection processes, learning processes and curriculum content. We intend to follow these students to see how they feel on starting work as house officers in order to establish whether the differences persist.

Conclusion

Running 2 parallel MBBS courses in 1 institution has allowed us to directly compare the factors affecting the development of differently prepared future doctors.

Clearly, universally agreed learning outcomes include the development of clinical skills and a professional approach to real patients.24 In order for students to master and maintain these core attributes they need exposure, practice and assessment, introduced as early as possible, and continued until graduation25 and beyond, fostered by a curriculum that ingrains a desire for lifelong learning. It would appear likely that this is more important than any individual student characteristics.

Anxiety in medical students: is preparation for full-time clinical attachments more dependent upon differences inmaturity or on educational programmes for undergraduate and graduate entry students?

Residents’ Response to Duty-Hour Regulations — A Follow-up National Survey

Brian C. Drolet, M.D., Derrick A. Christopher, M.D., M.B.A., and Staci A. Fischer, M.D.

레지던트의 역할은 지난 한 세기동안 크게 변화했다. 미국 GME의 초기 시스템을 만든 William Steward Halsted의 레지던트들은 일년에 362일을 근무해야 했다. 그러나 지금 1년차 레지던트(인턴)들은 16시간 이상 연속으로 병원에 있을 수 없다.

The role of the resident physician has evolved substantially over the past century. William Stewart Halsted, who is credited with developing the early system of graduate medical education in the United States, required 362 days per year of service from his residents. However, unlike Halsted's trainees, who lived in the hospitals in which they worked, today's first-year residents (interns) must adhere to various work restrictions, including spending no longer than 16 consecutive hours in the hospital.

레지던트의 근무시간에 대한 국가적 규제는 연장근무시간으로 인한 피로가 실수를 유발하여 환자 돌봄(patient care)에 안좋은 영향을 미치며, 교육적으로도 안 좋은 영향을 미친다는 인식에 따른 것이다. 궁극적으로 환자 진료와 교육 사이의 세밀한 균형을 맞추는 것이 논쟁의 핵심이라 할 수 있다.

National regulation of resident duty hours has occurred in response to the recognition that fatigue from extended work hours may result in errors and compromise patient care1 and may also lead to diminishing educational returns. Ultimately, the sensitive balance between patient care and education — given that residents are trainees — remains at the forefront of this discussion.

2003년 ACGME는 첫 번째 근무시간 규제를 도입하여 1주당 80시간의 근무제한을 두었다. 2008년 IOM은 추가적인 제한을 둘 것을 권고하였고, 레지던트를 직접 감독하여 환자 안전을 향상시키도록 권고하였다. 또한 ACGME가 이러한 변화를 도입하지 않는다면 OSHA에서 규제를 시도할 것이라고 하였다. 미국 대중은 이러한 추가적 규제를 지지하는 입장이다.

In 2003, the Accreditation Council for Graduate Medical Education (ACGME) implemented the first national regulation of work hours, establishing the 80-hour workweek. In 2008, the Institute of Medicine recommended additional limitations on work hours and an increase in direct supervision of residents to improve patient safety. It also suggested that if the ACGME and sponsoring institutions did not make changes, the Joint Commission or the Occupational Safety and Health Administration (OSHA) should perhaps step in to regulate residents' hours. U.S. public opinion supported further regulation as well.2

이러한 압력에 따라 ACGME Duty Hours Task Force 는 근무시간과 근무감독에 대한 최신의 요건을 2011년 7월 도입하였다. 이 두 가지를 도입하기에 앞서서 두 개의 대규모 전국 조사가 시행되었는데 여기서 프로그램 관리자와 레지던트는 이같은 변화가 도입되는 것에 대해서 다른 생각을 하는 것으로 나타났다. 이들은 잦은 근무교대가 연속성을 떨어뜨려 환자 진료에 안좋은 영향을 줄 것을 우려했다. 또한 근무시간을 줄이는 것이, 특히 인턴의 근무시간이 줄어드는 것이 교육의 질을 떨어뜨리고, 수련의들이 관리감독을 해야 하는 시니어로 제대로 준비되지 못하게끔 한다는 우려가 있었다. 이러한 우려가 실제로 도입된 이후에 현실로 나타났는지 알아보기 위해서 후속 전국 조사를 실시하였다.

In response to these pressures, the ACGME Duty Hours Task Force implemented the latest Common Program Requirements for Resident Duty Hours and Supervision in July 2011. Before the implementation of these requirements, two large national studies had shown that program directors and residents had mixed feelings about the potential impact of the proposed changes — fearing specifically that increased frequency of handoffs and loss of continuity might have a negative effect on patient care. Furthermore, there was concern that shortened duty hours, particularly for interns, would impair education and leave trainees less prepared for more senior, supervisory roles.3,4 To understand whether these concerns have become a reality during the year after the changes were adopted, we conducted a follow-up national survey of residents.

(중략)

설문에서 나타난 '불만족'의 이유에 대한 가능한 설명은 몇 가지가 있다.

The survey results suggest several possible explanations for this dissatisfaction. 

레지던트들은 작년과 비교했을 때 휴식시간의 변화가 전혀 없는, 같은 근무시간 스케쥴을 받아들었고 전반적인 QOL이 떨어졌다.

First, residents are working the same number of hours with no change in the amount of rest they receive and with worse schedules than last year, which diminishes their overall quality of life. 

레지던트들이 생각하는 수련의 최우선 목표는 인턴에서 시니어로, 궁극적으로는 attending physician으로 되는 것인데, 이것이 지연된다고 느낀다.

Second, residents believe that a chief goal of training — preparedness for transitioning from intern to senior resident, then ultimately to attending physician — is being delayed. 

관리감독이 눈에 띄게 향상되었다거나 환자안전이나 교육이 좋아졌다거나 하는 것이 없다.

Third, there has been no ostensible increase in available supervision or in the benefits for safety and education that would accompany this increased attendance. 

근무교대가 늘어났고, 연속성이 떨어져서 환자-의사 관계에 있어서 교육적/정서적 경험에 부정적 영향을 주었다.

Finally, the frequency of handoffs has increased, reducing continuity of care and thereby negatively affecting the educational and emotional experience associated with a strong doctor–patient relationship.

대부분의 레지던트들은 근무시간 규제를 지지할 것이고, Halsted의 시절로 돌아가고 싶어하지는 않을 것이다. 그러나 one-size-fits-all 방식의 접근법은 모든 프로그램에 적합하지 않을 거싱며, 궁극적으로 2011 ACGME 근무시간 규제가 의도한 효과가 나타나지 않을 수 있다. 그럼에도 불구하고 더 연구가 필요하다.

We firmly believe that most residents support some form of duty-hour regulation and would not choose to revert to the Halstedian model. However, a one-size-fits-all approach may not be adequate or appropriate for all trainees and training programs. Ultimately, the intended and actual effects of the 2011 ACGME duty-hours requirements may not be aligned. Nevertheless, more study will be needed to quantify how safety and quality of care, as well as resident education, are being affected.

Residents' response to duty-hour regulations--a follow-up national survey.

PMID:

 

22646511

 

[PubMed - indexed for MEDLINE] 

Free full text

Introducing Medical Students to Careers in Medical Education: The Student Track at an Annual Medical Education Conference

Benjamin Blatt, MD, Margaret Plack, PT, EdD, Mari Suzuki, MD, Sruthi Arepalli, Scott Schroth, MD, MPH, and Alex Stagnaro-Green, MD, MHPE

Junior faculty를 모집하고 유지하는 것은 쉽지 않은 과정이며, 특히 academic medicine이 마주하고 있는 쉽지 않은 도전들을 감안할때 더욱 그러하다. 

The recruitment and retention of junior faculty is a daunting process, especially given the challenges presently confronting academic medicine.1,2 

• New faculty are entering a revenue-driven environment focused on clinical productivity and grant procurement at a time when health care reform legislation promises to transform the practice of clinical medicine. 
• In addition, they are fulfilling the educational core mission of an academic health center—training the next generation of clinicians.3,4 
• Faculty physicians must negotiate this complex landscape while sometimes receiving lower income than their colleagues in clinical practice.

academic medicine으로의 진로에 흥미를 갖게 하려면 일찍 노출시키고, 잘못된 인식을 바로잡아주고, 확실한 진로 방향을 잡아줘야 한다. 임상의 또는 연구자로서의 진로는 제법 명확하게 규정되어 있다. 그러나 의학교육에 관심이 있는 학생들이 교실 바깥에서 의학교육자들과 활발히 교류하기 위한 방법은 많지 않다. 이렇게 불분명한 진로 pathway는 현재와 같이 전 국가적으로 교육과정이 개편되고, competency based education을 하고, 혁신적인 교육 기법들이 활용되는 시기에 더욱 문제가 된다.

To generate interest in careers in academic medicine, experts recommend early exposure, clarification of misperceptions, and delineation of clear career pathways.5–7 Medical students—who are in the process of considering, testing, and exploring career options—are an obvious group to target. 

• Interestingly, the pathway to a career as a clinician or funded researcher is relatively well delineated, as medical schools and national organizations offer students opportunities to participate in student interest groups and research experiences. 
• Few avenues exist, however, for students interested in medical education to become actively involved with medical educators beyond the classroom. The lack of a clear pathway to a career as a clinician–educator is particularly troubling given the current national-level focus on curriculum reform, competency-based education, and innovative pedagogical techniques—all of which require trained professionals for successful design and implementation.

아마 학생들이 의학교육 진로에 노출될 수 있는 가장 용이한 방법은 교육과정 위원회에 대표로서 참여하는 것일 것이다. 그러나 이는 영역이 제한적이고, 참여할 수 있는 학생도 제한적이다. 또 다른 접근법은 여른방학 시기를 이용한 의학교육 리더십 프로그램이 있고, 또한 scholarly concentration 또는 track system도 가능하다. 

Perhaps the most common way in which students are exposed to medical education careers is through involvement as representatives on curriculum committees. This exposure is limited in scope, though, and typically includes few students in a given class. Other approaches include summer medical education leadership programs 8 and medical schools’ structured scholarly concentrations (SCs) or track systems. SCs, some of which span all four years of medical school, enable students to elect longitudinal experiences in education, research, global health, and other areas, depending on their interests.9 Initially, most SC programs focused on basic or clinical research, but a number of schools have recently implemented SCs in other domains, including medical education.10–13

의과대학생들이 의학교육에 노출될 수 있는 또 다른 기회는 의학교육학술대회로서 학문적인 것 뿐만 아니라, 네트워크, 협동, 정보공유의 장이 될 수 있다. 이러한 학회에 참여함으로서 진로 개발에 필요한 학습을 하고, 기술을 개발하고, 롤모델을 찾고, 긍정적 학습환경을 경험하고, 네트워크를 쌓을 수 있다. 2011년 NEGEA의 연례 컨퍼런스 행사.

A venue with rich potential for exposing medical students to careers in medical education is the annual medical education conference, which is a principal source of educational scholarship, networking, collaboration, and information sharing. Attendance at such a conference offers students opportunities for career-specific learning, skill development, role models, positive and encouraging learning environments, and networking, all of which can have a positive effect on career choice.6 To harness the March 2011 Northeast Group on Educational Affairs (NEGEA) annual retreat’s career-building potential for medical students, we incorporated a student track, with an array of events in which they could become actively involved.

의학교육 SC가 있는 대학에 다니는 학생에게 있어서 의학교육학회에 학생으로서 참여하는 것은 아주 좋은 경험이 될 수 있다. 대부분의 SC에서 학술 업적을 이룰 것을 요구하기 때문에, 학생은 여기에 참여함으로서 자신의 연구결과를 발표하고, 동료로부터 의견을 받을 수도 있다. 그 외에도 여러 장점이 있음.

For students who attend a school with a medical education SC, participating in a student track of a medical education conference fits nicely with their concentration. 

• Because most SCs require scholarly products, a conference student track complements the SC by providing a means for students to present their work, thus fulfilling the peer-review and dissemination elements of scholarship. 
• In addition, a student track enables students to acquire experience, knowledge, and skills through participation in activities specially designed for and focused on them. 

의학교육 SC가 없는 대학 학생이라면 의학교육 관련 진로에 대한 안내를 받는 경험이 될 수 있다.

For students who attend schools without medical education SCs (the vast majority), a student track may serve as the only introduction to careers in medical education.

NEGA컨퍼런스의 사례를 보여주고자 함. SCCT는 의학 분야에서 만들어진 이로느 아니나, 의학 분야에서 career-building effort에 유용한 도구로 사용되어왔다.

In this article, we describe how we instituted the NEGEA conference’s first student track and provide guides for building future student tracks based on our experience and the constructs of social cognitive career theory (SCCT). This theory, which was developed in nonmedical fields, has emerged in the medical literature as a useful means for anchoring career-building efforts in medicine.7 To the best of our knowledge, no other student career-building program nested within a medical education conference has been described in the literature.

Guides for Planning Student Tracks

In considering how we might enhance future student tracks, we reflected on our experience and drew from SCCT. Below, we offer guides for recruiting and designing student tracks and share lessons learned. Although our focus is on medical education, we believe the suggestions they present will be useful for educators planning conference-based student tracks in any field.

참여자 모집 가이드

A guide to recruiting participants

The recommendations in the following guide are generalized from our experience and participant input regarding recruiting students:

• 초기에 운영위원회에 요청
  Solicit student participation early in planning committee activities. Enlist students in important roles in designing and implementing the student track. 

• 비용은 낮게, 숙박과 식사 제공.
  Design the conference to be low cost for students and, if possible, provide free student housing and meals. 

• 교수 및 다른 학교 학생과의 교류 기회 강조
  Emphasize opportunities to engage with faculty and students from other institutions in a nonthreatening, learner-centered environment 

• 타겟 학생 확인
  Identify and target students with a special interest in the conference topics. (Most of our attendees had prior interest in medical education.) 

• 다른 학생들 대상 의학교육학회의 전반적 가치를 강조
  Target other students by emphasizing the general value of a medical education conference. 
  (We emphasized the usefulness of teaching, leadership, and medical education research skills to any academic career.) 

• 교수의 직접 접촉
  Make personal and direct contact with faculty at other institutions to tell them about the student track and ask them to identify and invite interested students. (This was our most successful means of recruitment.)

학생 트렉 설계 가이드

A guide to designing student tracks

As we considered ways to maximize the potential long-term impact of a student track, we looked to SCCT. SCCT has long history in nonmedical fields and has built an evidence base to support its effectiveness in predicting career choice and success.5 Because of its predictive value, SCCT can be used to guide the development and estimate the long-term career influence of a student track such as ours.

SCCT contends that the interaction of personal characteristics with the environment results in experiences that can affect future career choices and work performance. These experiences exert their effect through influencing self-efficacy (the belief in one’s ability to succeed) and outcome expectations (the belief that a career choice will lead to valued outcomes).5,14,15 

자기효능감을 강화하고 기대 성과 강화하기 위한 SCCT에서 제시하는 네 가지 요인

SCCT proposes that four factors can strengthen self-efficacy and outcomes expectations: 

• personal success experience, 
• exposure to successful role models, 
• social and verbal persuasive communications, and 
• positive emotional reactions (as would be elicited by a supportive learning climate).5,7 

To promote career building through a medical education conference, it then follows that planners should aim to fully incorporate each of these factors into the student track experience. In Table 1, we present a guide for planners. In it we populate each of the four factors with features that, from our experience, we believe will help planners realize this goal.

효과적인 학생 트렉 설계를 위한 기타 고려사항

Other considerations for designing effective student tracks

In reflecting further on our experience with the NEGEA student track, we identified concerns that should be addressed when creating a new program. 

네 개의 SCCT factor가 잘 달성되었는지 불확실함.

First, although our track addressed the four SCCT factors, it is unclear whether it addressed them optimally. Many students’ comments indicated that they would have liked the conference to have provided them with more opportunities to develop self-efficacy and interact with faculty role models. Planners of future student tracks should consider ways to maximize these opportunities and to evaluate their quantity and quality.

학생들에게 barrier가 될 수 있는 것을 고려해야 함.

Second is a concern about barriers, which, as noted by Lent and colleagues,16 prevent students from pursuing educational exposures and may influence career choices. Although we recognized the need to provide a low-cost conference with housing and food included, we did not address financial and other barriers to career choice and progression beyond the conference. Student track planners should consider including programming that addresses these barriers. Also, studies are needed to clarify the barriers and supports that may most influence the decision to choose a career in medical education.

지속적 경험이 되기 위한 강화(reinforcement)가 필요하다. 

Third, as our student track was a one-time event without built-in follow-up, it could not offer the reinforcing benefits of a continuity experience. Without reinforcement, the inspiration that students glean from attending a conference may dissipate when they return to their home institutions. SCCT theorists emphasize the importance of continuity for effective career building.5,7 Continuity provides repeated exposure to opportunities that SCCT deems critical to optimizing student career interest.6 From such exposure comes the chance for in-depth understanding of the scope of a medical education career, clarification of potential misperceptions, and bonding with role models.6,17 Continuity experiences can also promote ongoing goal setting, which is an important predictor of career choice.6,17

Considering the value of continuity through mentoring,18 we recommend that future student tracks be designed as springboards for participants’ further development in medical education at their home institutions. Continuity may be arranged through faculty advisor relationships with medical educators, perhaps the very educators that sponsored their attendance at the conference. It may also be promoted by participation in formal students-as-teachers programs, which as of 2010 were offered by 43% of medical schools.19 Better still is the continuity provided when a student track is integrated with a medical school SC in medical education. In doing so, the annual conference student track can become part of a comprehensive model to support students in their journey toward a medical education career. Ideally, SC student participation should be yearly and progressive, with increasing involvement in medical education conferences as students advance through their concentration. For example, first-year medical students could focus on gleaning information and skills from conference workshops and from others’ scholarship, while more advanced medical students might focus on their own scholarship through submitting and presenting posters, workshops, and papers.