교육효과성을 높이기 위한 FD initiative 의 systemic review (BEME Guide No. 8) (Med Teach, 2006)
A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8
YVONNE STEINERT1, KAREN MANN2, ANGEL CENTENO3, DIANA DOLMANS4, JOHN SPENCER5, MARK GELULA6 & DAVID PRIDEAUX7
1McGill University, Montreal, Canada; 2Dalhousie University, Halifax, Canada; 3Austral University, Buenos Aires, Argentina; 4University of Maastricht, Maastricht, The Netherlands; 5University of Newcastle upon Tyne, Newcastle, UK; 6University of Illinois at Chicago, Chicago, USA; 7Flinders University, Adelaide, Australia
결론: 대부분의 인터벤션은 진료를 하는 의사를 대상으로 하였다. 모든 연구는 교육의 향상을 목적으로 하여 워크숍/세미나 시리즈/단기코스/장기프로그램 등으로 분류되었다.
Results: The majority of the interventions targeted practicing clinicians. All of the reports focused on teaching improvement and the interventions included workshops, seminar series, short courses, longitudinal programs and ‘other interventions’. The study designs included 6 randomized controlled trials and 47quasi-experimental studies, of which 31 used a pre-test–post-test design.
방법론적 한계를 감안하더라도 다음의 결과를 지지한다.
Key points: Despite methodological limitations, the faculty development literature tends to support the following outcomes:
- FDP의 전반적인 만족도는 높다. 참가자들은 지속적으로 프로그램이 수용가능하고/유용하고/자신들의 목표와 관련된다고 응답하였다.
Overall satisfaction with faculty development programs was high. Participants consistently found programs acceptable,useful and relevant to their objectives. - 응답자들은 FD와 교육에 관하여 긍정적인 태도 변화를 보고하였다.
Participants reported positive changes in attitudes toward faculty development and teaching. - 참가자들은 교육 원칙에 대한 지식이 향상되었으며, 교육 기술이 향상되었다고 응답하였다. 지식 향상 점검을 위해 시험을 보았을 때에는 유의미한 향상이 있었다.
Participants reported increased knowledge of educational principles and gains in teaching skills. Where formal tests of knowledge were used, significant gains were shown. - 교육 행동의 변화는 지속적으로 참가자들에 의해 보고되었으며, 학생들도 그렇게 보고하였다.
Changes in teaching behavior were consistently reported by participants and were also detected by students. - 조직 차원의 변화와 학생들의 배움의 향상은 자주 연구되는 것은 아니었다. 그러나 연구 결과를 보면 교육에 대한 참여가 높아지고, 동료간의 네트워크가 확립되었다.
Changes in organizational practice and student learning were not frequently investigated. However, reported changes included greater educational involvement and establishment of collegiate networks. - 경험학습, 긍정적 피드백의 제공, 효과적인 동료관계, 교수학습 원칙을 따른 잘 설계된 인터벤션, 하나의 인터벤션 내에서 다양한 교수법 활용 등이 효과적인 FD에 기여한다.
Key features of effective faculty development contributing to effectiveness included the use of experiential learning, provision of feedback, effective peer and colleague relationships, well-designed interventions following principles of teaching and learning, and the use of a diversity of educational methods within single interventions.
학계의 생명력은 교수의 흥미와 전문성에 달려있다. FD는 학문적 수월성과 혁신을 촉진하는데 핵심적 역할을 한다.
Academic vitality is dependent upon faculty mem-bers’ interest and expertise; faculty development has a critical role to play in promoting academic excellence and innovation. (Wilkerson & Irby,1998)
교수들이 다양한 역할을 할 수 있게 돕기 위해서 다양한 FDP가 설계되고 도입되었다. 워크숍/세미나...등등을 포함한다. 이들 활동의 많은 부분이 의학교육연속체의 교수 효과성을 높이기 위해 설계되었다. 또한 local, regional, national level의 보건의료전문직에게 제공되어왔다.
To help faculty members fulfill their multiple roles, a varietyof faculty development programs and activities have been designed and implemented. These activities include work-shops and seminars, short courses and site visits, fellowships and other longitudinal programs. Many of these activitieshave been designed to improve teacher effectiveness acrossthe medical education continuum (e.g. undergraduate andpostgraduate education), and they have been offered tohealthcare professionals at local, regional and national levels(Clark al., 2004; Skeff al., 1997).
교수개발
Faculty development
FD는 다양하게 정의되어왔다.
Faculty development has been defined as
- 교수의 역할을 지원하거나 새롭게 하기 위해서 기관 차원에서 활용하는 활동
that broad range of activities that institutions use to renew or assist faculty in their roles (Centra, 1978), and - 교수들의 교육/연구/행정에 있어서의 수행역량을 향상시키기 위한 것
includes initiatives designed to improve the performance of faculty members in teaching,research and administration (Sheets & Schwenk, 1990). - 기관과 교수들을 학문적 역할(교육, 연구, 행정, 저술, 경력 관리)을 준비시키는 계획된 프로그램
In many ways, faculty development is a planned program to prepare institutions and faculty members for their academic roles, including teaching, research, administration, writingand career management (Bland et al., 1990). - 개인 강점과 능력, 조직적 역량과 문화를 향상시킴으로서 변화의 실천과 관리를 개선시키는 것
Faculty development is also meant to improve practice and manage change (Bligh, 2005), by enhancing individual strengths and abilities as well as organizational capacities and culture.
FDP는 다양한 방식으로 분류되었다.
Faculty development programs have been classified indifferent ways.- 조직차원의 전략, 펠로우십, 포괄적 지역 프로그램, 워크숍과 세미나, 개별활동 으로 분류
Ullian & Stritter (1997) describe a typology that includes organizational strategies, fellowships, compre-hensive local programs, workshops and seminars, and individual activities. - 신임교수들의 전문직으로서 방향 설정, 교수역량 개발, 리더십역량 개발, 조직 개발.
Wilkerson & Irby (1998) offer a different classification, ranging from professional orientation for new faculty members to instructional development, leadership development and organizational development. These authors also suggest that all four elements comprise a comprehensive approach to faculty development that is fundamental to academic vitality. - FDP는 기관이 그들의 직원에 대해서 가지고 있는 내적 신념을 외부로 보여주는 신호이며, 성공적인 FD는 교육의 향상과 학생 혹은 의사의 더 나은 학습성과로 나타난다.
Bligh (2005) has made a similar suggestion,stating that faculty development programs are outward signs of the inner faith that institutions have in their workforce, and that successful faculty development performance is expected to result in improved teaching and better learning outcomes for students or doctors.
지금까지 여러 문헌에서 FD활동의 효과성을 리뷰하였다.
To date, a number of publications have reviewed the effectiveness of faculty development activities.
- FDP는 많지만, 평가는 잘 되지 않고 있으며, 주로 만족도를 평가하는 짧은 설문에 그치고 있다.
In 1984, Sheets & Henry observed that despite the growth in faculty development programs, evaluation of these initiatives was a rare occurrence, usually consisting of short questionnaires tapping participants’ satisfaction. - 가정의학전공 교육자들에 관한 FD에 관한 연구로부터 비슷한 결론을 도출하고, 관찰한 행동 변화에 근거한 더 철저한 평가를 요구했다.
In 1990, Sheets & Schwenk reviewed the literature on faculty development activities for family medicine educators and made a similar observation, calling for more rigorous evaluations based on observed changes in participant behavior. - 이전의 연구를 요약하면서, FD의 개념이 진화/확장하고 있다고 하였음. 특히, 교육 기술이 FD의 두드러지는 측면이며, 펠로우십이 새로운 교수를 모집하고 훈련시키는데 효과적이며, FD의 효과는 더 연구가 필요하다
In 1992, Hitchcock et al. summarized earlier reviews of the faculty development literature (e.g. Stritter, 1983; Bland & Schmitz, 1986; Sheets & Schwenk, 1990) and concluded that the concept of faculty development was evolving and expanding. In particular, they observed that teaching skills were a prominent aspect of faculty development, that fellowships were being used effectively to recruit and train new faculty, and that the efficacy of faculty development needed better research documentation. - 24개의 문헌을 리뷰하여, 비록 일부 긍정적인 성과가 보고되지만, 방법론적 약점이 단정적인 결론을 내는 것에 장애가 된다
In 1997, Reid et al. reviewed 24 papers (published between 1980 and 1996) and concluded that despite some positive outcomes for fellowships, workshops and seminars, methodological weaknesses precluded definitive conclusions regarding faculty development outcomes. - FD가 의학교육과 의료의 변화에 대응해야 하며, 교수들의 진화하는 역할에 지속적으로 적응해야 한다. 또한 더 철저한 프로그램평가가 필요하다. FDP는 초점을 더 확장시켜서 다양한 훈련 방법과 형식을 고려하고, 새로운 파트너십과 협력을 모색해야 한다.
In 2000, Steinert highlighted the need for faculty development to respond to changes in medical education and healthcare delivery, to continue to adapt to the evolving roles of faculty members, and to conduct more rigorous program evaluations. She also commented that faculty development programs need to broaden their focus, consider diverse training methods and formats, and foster new partnerships and collaborations.
목적
Objectives
교수들의 교육역량 강화에 초점을 둔 연구에만 한정함.
The goal of this review is to determine the effect of faculty development activities on faculty members’ teaching abilities and to assess the impact of these activities on the institutions in which these individuals work. We focused specifically on programs designed to improve faculty members’ teaching abilities because the majority of faculty development programs have targeted this particular role (Hitchcock et al., 1992; Irby 1996); instructional effectiveness is central to the mission of medical education; and we wanted to limit the scope of our search to a feasible task. We did not examine faculty development programs designed to improve research or writing skills, administrative or management skills, or professional academic skills (career development). We also chose to limit the review to faculty development programs designed for teachers in medicine, and did not examine those programs specifically designed for residents or other healthcare professionals (e.g. nurses; dentists). All types of faculty development interventions (e.g. workshops, short courses and seminars, and fellowships) were included in the review.
검토 질문
Review question
FD를 효과가 있게 하는 특징은 무엇인가?
What are the features of faculty development that make it effective?
FD가 차이를 만드는가?
Does faculty development make a difference?
- What makes for effective faculty development?
- Does participation in faculty development improve facultymembers’ teaching, research and administrative skills?
- Does faculty development have an impact on the institu-tional climate and organization?
FD인터벤션이 교사의 지식,, 태도 술기에 미치는 효과는 무엇이며, 그 교수가 속한 기관에 미치는 영향은?
What are the effects of faculty development interventions on the knowledge, attitudes and skillsof teachers in medical education, and on the institu-tions in which they work?
In addition, we also explored the following questions:
- What characterizes the faculty development activities tha thave been described?
- What are the methodological strengths and weaknesses ofthe reported studies?
- What are the implications of this review for facultydevelopment practices and ongoing research in this area?
리뷰 방법
Review methodology
그룹 형성
Group formation
An international Topic Review Group (TRG) of individualsrepresenting six countries was constituted. Three criteriaparticipation:were used to invite individuals for TRG international diversity; practical experience in faculty devel-opment and medical education; and expertise in educationalresearch methodology.
파일럿 단계
The pilot process
A two-step pilot process was undertaken to prepare for the formal, systematic review.
개념틀 개발
Development of a conceptual framework
그림 1
The pilot phase led to the development of a conceptual framework that guided this review (see Figure 1).
커크패트릭 모델 사용
To classify and analyze outcomes, we used Kirkpatrick’s model of educational outcomes (Kirkpatrick, 1994), which offers a useful evaluation framework for this purpose (see Figure 2).
커크패트릭은 이 성과가 위계적이지 않으며, 모델은 정책과 프로그램 개발에 더 전체적이고 포괄적인 평가를 의도한 것이라 하였음.
In his original work, Kirkpatrick (1967) asserted that these outcomes were not hierarchical and that the model is intended to provide a more holistic and comprehensive evaluation that can inform policy and program development. The model has also been used by other BEME groups (e.g. Issenberg et al., 2005) as well as other review groups (e.g. Freeth et al., 2003), and with some modifications, was well suited to our review.
포함/배제 기준
Inclusion/exclusion criteria
Based on the pilot studies, the following criteria guided theselection of articles for review:
탐색 전략, 논문 출처
Search strategy and sources of papers
A literature search was conducted on Medline and ERICusing the following key words: staff development; in-service training; medical faculty; faculty training/development; andcontinuing medical education. (A copy of the search strategyis included in Appendix I, which is available on the BEMEwebsite: http://www.bemecollaboration.org)
선택 방법과 방법론적 질 점검
Selection methods and judgment of methodological quality
The literature search resulted in a total of 2777 abstracts. A two-stage process was employed in the selection of studies eligible for review (Freeth et al., 2003) and is outlined in Figure 3.
데이터 관리 기술
Data management techniques
Data extraction, analysis and synthesis
어떤 자료를 추출하여 분석하고 종합하였는가?
결과
Review findings
리뷰에 포함된 연구의 개괄
Overview of studies included in review
(a) Description of the interventions and expected outcomes—which will be further divided into: setting, professional discipline, focus of the intervention, program type,instructional methods, duration, and level of outcome assessed.
(b) studies—which will be Methodological quality of the further divided into: study goal and theoretical frame-work, study design, data-collection methods, data sources, and study quality and strength of findings.
(a) Description of the interventions and expected outcomes
세팅(국가, 기관)
Setting: Of the 53 papers reviewed, 38 studies (72%) took place in the US, the remainder being in Canada, Egypt, Israel, Malta, Nigeria, the UK, Switzerland and South Africa. Most activities were delivered in a university, hospital or community setting, with several initiatives offered by profes- sional associations.
참여 대상의 전공(내과, 가정의학과, 40%에서는 참가자의 전공 종류가 2개 이상, 기초과학자 대상 등)
참가자 수(6~399명, 평균 60명)
Professional discipline: The majority of faculty development interventions targeted practicing clinicians, with a prepon- derance of activities in family medicine and internal medicine. Interestingly, 21 of the faculty development initiatives (40%) welcomed more than one clinical discipline. Five interventions (10%) were designed for both clinicians and basic scientists; an additional two (4%) targeted basic scientists only. The number of participants in the interven- tions (which does not equal respondents for the evaluative component) ranged from six to 399, with a mean attendance of 60.
Table 2. Summary of faculty development outcomes by Kirkpatrick level.*
인터벤션의 초점: 교육 개선, 임상교육, 피드백과 평가, 소그룹 교수법, 강의기술, 학습자-중심 교육, 특정 내용 교육에 대한 것, 일반적인 교육 향상, 개인적/진로 개발, 조직변화, 행정과 리더십, 연구기술
Focus of the intervention: As a result of the selection criteria, all of the reports focused on teaching improvement. The majority aimed to improve clinical teaching, with a secondary emphasis on feedback and evaluation, small-group teaching and lecturing skills. Several studies highlighted ‘learner centeredness’ as an outcome, and several others focused on the teaching of specific content areas in addition to general teaching improvement (e.g. communication skills and medical interviewing; principles of family medicine and preventive medicine). Although the primary focus of these reports was instructional improvement, many also addressed personal/career development, organizational change, administration and educational leadership, and research skills.
프로그램 유형: 워크숍, 세미나 시리즈, 단기 코스, 장기 프로그램, 개인별 피드백, 증강(augmented) 피드백, 현장 방문. 용어의 비일관적인 그리고 다양한 사용이 분류를 어렵게 하는 측면이 있음.
Program type: The majority of activities were workshops (n¼23; 43%), of varying duration. Ten (19%) of the interventions were described as a seminar series and six (11%) as a short course. Five (10%) were described as a longitudinal program (e.g. fellowship) and nine (17%) fell under ‘other’, which included a seminar method, individual or augmented feedback, or site visits. An inconsistent and variable use of terms (e.g. workshops and seminars; seminars and short courses), complicated this classification; however, whenever possible, the authors’ terminology was used.
교수법: 강의, 소그룹토론, 상호작용 연습, 역할극, 시뮬레이션, 비디오-녹화 리뷰. 강의로만 진행되는 프로그램은 없으며, 대부분의 프로그램은 피드백이 동반된(microteaching과 같은) 실습(experiential) 부분을 포함하고 있음. 일부 프로그램은 현장훈련을 제공하며, 배운 것을 바로 적응할 수 있게 한다. 비록 교육 프로젝트와 in vivo practice가 일부 인터벤션(대부분 세미나와 단기 코스)의 일부였지만, 교사의 지속적 교육활동과 연관되어 있는 것을 묘사한 연구는 극히 적었다. 요구도 조사가 이루어진 경우는 적었다.
Instructional methods: All reports described a wide range of instructional methods that included lectures, small-group discussions, interactive exercises, role plays and simulations, films and videotape reviews of performance. No programs were completely lecture-based, and the majority included an experiential component with opportunities for guided practice with feedback (i.e. micro-teaching). Some programs offered on-site training opportunities where teachers could readily apply what they learned. Few described a direct link to teachers’ ongoing educational activities, although educational projects and in vivo practice were part of several interventions (most notably seminars and short courses). Needs assessments were used sparingly.
길이: FDP 인터벤션은 1시간에서 1년까지 다양했다. 워크숍(대체로 one-time 인터벤션)의 경우 3시간에서 1주까지 분포하고 있었으며, 중간값은 2일이었다. 세미나 시리즈(장기간에 걸쳐 진행되는 것)은 12시간에서 1달까지 분포하고 있었으며, 중간값은 14시간이었다. 단기코스는 1주에서 1달에 분포하였다. 펠로우십은 full-time와 part-time이 모두 있는데, 한 인터벤션은 18개월에 걸쳐 50시간에 달하였다.
Duration: The faculty development interventions ranged in duration from one hour to one year. Workshops, which were generally one-time interventions, ranged in duration from three hours to one week, with a median duration of two days. The seminar series, which occurred over time, ranged in duration from 12 hours to one month (with a median duration of 14 hours), and the short courses ranged from one week to one month. Fellowships were both full time and part time in nature, and one intervention, entitled a ‘longitudinal program’, was 50 hours in length over 18 months.
평가 항목: 74%의 연구가 reaction을 평가하였으며(만족도, 유용성 인식, 수용가능성, 학습활동의 가치), 77%에서 learning을 평가하였다(태도, 지식, 술기의 변화), 72%에서 '행동'을 평가하였으며, 'Result'평가는 13%에서 조직의 변화를, 6%에서 학생/레지던트 학습의 변화를 평가하였다.
Level of outcome assessed: Table 2 shows that 39 studies (74%) assessed reaction, which included participant satisfaction, perception of program usefulness and acceptability, and value of the activity. Forty-one studies (77%) assessed learning, which included changes in attitudes, knowledge or skills. Thirty-eight (72%) assessed change in behavior. At the results level, seven studies (13%) reported change in organizational practice and three (6%) assessed change in student or resident learning.
(b) 연구의 방법론적 품질
(b) Methodological quality of the studies
연구 목표와 이론틀: 모든 연구에서 목표를 기술하였으며, 일부는 더 구체적으로 목표를 기술하였음.(FDP의 교육행동 또는 교육태도에 대한 효과성 평가). 7개를 제외한 모든 연구에서 관련 문헌을 인용하였으며, 57%에서 개념틀 혹은 이론틀(주로 성인학습, Instructional design, experiential learning, reflective practice)과 연결지었다.
Study goal and theoretical framework: All 53 reports stated their objective, sometimes quite broadly (e.g. to describe, implement and evaluate a faculty development initiative). Some reports described more specific objectives, outlining a particular study question such as assessing the effectiveness of a faculty development program on teaching behaviors (Hewson, 2000) or attitudes (Schmidt et al., 1989). One study examined the effect of experience on workshop gains (Baroffio et al., 1999), and several others assessed different methods of assessment (Nasmith et al., 1997; Hewson et al., 2001) and program evaluation (Sheets, 1985). All but seven cited the relevant literature, though often in a very limited fashion. Thirty reports (57%) placed their work within a conceptual or theoretical framework, primarily drawing upon principles of adult learning, instructional design, experiential learning and reflective practice.
연구 설계: 11% RCT, 89% quasi-experimental. 45개의 단일그룹설계 중, 69%는 pre-post test, 26%는 post-test만. 질적 접근만 활용한 경우는 없었으며, 21%는 질적연구방법 사용
Study design: Of the 53 papers reviewed, there were six (11%) randomized controlled trials. The majority of studies (n¼47; 89%) were quasi-experimental in design, with two including a comparison group in the main part of the study. Of the 45 single-group designs, 31 (69%) employed a pretest– post-test design. Fourteen studies (26%) used a post-test only. None of the reports used a qualitative approach only, though 11 (21%) incorporated a qualitative method (or analysis) in their design.
데이터 수집: 워크숍 후 설문, pre and post 측정, 학생/레지던트/자기 평가, 교육행동 관찰 등. 설문이 가장 흔히 사용되는 방법이었음. 55%에서는 설문만 사용. 38%에서는 설문+alpha. 대부분의 설문은 특정 연구를 위해 개발되었고, 매우 소수 연구에서는 psychometric properties도 보고함. 30%는 (비디오 녹화 등) 직접 관찰 평가를 수행.
Data collection methods: Methods to evaluate faculty development programs included end-of-workshop questionnaires, pre- and post-test measures to assess attitudinal or cognitive change, student, resident and self-assessment of post-training performance, and direct observations of teaching behavior. Questionnaires were the most popular method of data collection. All but four of the interventions used a survey or questionnaire. Twenty-nine (55%) of the interventions used a questionnaire only; 20 (38%) used a questionnaire and another method (e.g. observation; expert opinion). Most questionnaires were designed for a particular study, and few reports described psychometric properties. Sixteen studies (30%) included direct observation (of live or videotaped teaching sessions) as part of their assessment methodology.
데이터 출처: 대부분 교육에 대한 자기보고에 의존하고 있으며, 수행-기반 변화측정은 매우 제한적으로 사용되었음. 28%는 학생/레지더트의 평가를 도입하였음. 전문가 견해를 사용하기도 함. 학생의 시험점수, 레지던트 행동에 관한 환자의 평가 등도 있음. 응답률은 low 또는 unspecified.
Data sources: The majority of programs relied on selfreported ratings of teaching, with a limited use of performance-based measures of change. Fifteen studies (28%) employed student or resident ratings to assess changes in teaching behaviors. An additional two used expert opinions to assess outcomes. One study assessed student exam scores; another included patient ratings of resident behaviors. In many studies, the response rates for outcome measures were low or unspecified; statistical methods or differences were often not described.
연구의 품질과 연구결과의 견고성: 5점척도로 평가했을 때(원래는 subscale도 넣었지만 reliable하지 않았음) study quality는 평균적으로 3.14점이었으며 1점에서 5점까지 분포(1 낮음, 5 높음). strength of finding은 2.88이 평균이었고 1점에서 4점까지 분포 (1점: no clear conclusions can be drawn; 3점: conclusions can probably be based on results; 5점 results are unequivocal)
Study quality and strength of findings: Study quality was rated on a five-point scale (1¼low; 5¼high), and reviewers were asked to indicate study strengths and weaknesses. We had originally included subscales to rate the evaluation methods (e.g. appropriateness of and implementation of study design; appropriateness of data analysis), but this did not yield reliable results. We therefore chose to use an overall rating for this variable. Strength of findings was rated on a five-point scale with specific anchors (1¼no clear conclusions can be drawn; 3¼conclusions can probably be based on results; 5¼results are unequivocal). The mean rating for study quality was 3.14, with a range from 1 to 5. The mean rating for strength of findings was 2.88 (with a range of 1–4).
인터벤션 유형에 따른 결과 요약
Summary of findings by intervention type
(a) 워크숍
(a) Workshops
23개의 인터벤션이 워크숍이었으며, 대부분은 duration이 다양한 single intervention이었음.
Twenty-three of the interventions reported were described as workshops, most commonly a single intervention of varying duration.
23개 중 7개 만이 개념 또는 이론틀을 기술함
Only seven of the 23 stated a theoretical or conceptual framework.
(b) 단기 코스
(b) Short courses
54개의 인터벤션 중 6개가 단기코스 형태였고 1주에서 1달까지 분포하였다. 모두 목표를 기술하였고, 6개 중 5개에서 이론틀을 제공하였다.
Six of the 54 interventions (Sheets & Henry, 1984, 1988;Gordon & Levinson, 1990; Skeff et al., 1992b; DaRosa et al.,1996; Pololi et al., 2001) were in the form of a short course,ranging in duration from one week to one month. All hada stated objective and all but one provided a theoretical framework.
(c) 세미나 시리즈
(c) Seminar series
10개의 연구에서 세미나시리즈를 하였고, 이것의 특징은 각 세션이 시간 간격을 두고 진행되는 것이다.
Ten studies described a seminar series characterized by the fact that the sessions were spaced over time
(d) 장기 프로그램과 펠로우십
(d) Longitudinal programs and fellowships
한 연구에서 장기프로그램을 보고하였으며, 모두 목표를 기술했다.
One report described a longitudinal program.
All had stated objectives and all but one incorporated a theoretical framework.
우수 연구 자세히 들여다보기
The focused picture
8개의 연구가 study quality와 strenght of findings에서 4점 혹은 그 이상의 점수를 받았다. 이 연구만 따로 보면 다음과 같다.
Eight articles scored 4 (or higher) for both study quality andstrength of findings, and we chose to examine these separately in order to provide a more focused picture offaculty development.
8개 중 4개의 연구에서 effect size를 계산 할 수 있었다. 평균점수와 SD로 effect size를 계산함. Table 3에 그 결과가 나와있다.effect size는 다양했지만 moderate to high 한 정도의 효과크기가 네 가지 모두에서 나타났다. 즉 인터벤션의 효과가 있었으며 특히 교육의 측면에서 그리고 인터벤션에서 도움을 받은 교수들의 측면에서 있었다.
Four of the eight studies included in our focused review provided data that allowed for the calculation of effect size(Baroffio et al., 1999; Skeff, 1983; Skeff et al., 1986; Mahler& Benor, 1984). Mean scores and standard deviations weredrawn from the data and were converted into effect sizes (d)using Cohen d’s calculation (Cohen, 1988). These effects areshown in Table 3, where these studies are summarized. Whileeffect sizes varied, moderate to high effect sizes were found inall four studies, highlighting the effects of the interventions,particular aspects of teaching that were affected, and groupsof teachers who might benefit from the intervention.
(a) 인터벤션과 기대 효과
(a) Description of the interventions and expected outcomes
45분짜리 피드백 세션에서부터 1달짜리 세미나 시리즈까지 다양했다.
The interventions described in these eight reports rangedfrom a 45-minute feedback session for clinical teachers(Marvel, 1991) to a month-long seminar series designed tofacilitate dissemination of workshop concepts (Stratos et al.,1997). One study described two workshops aimed at improving tutor behavior, each consisting of several phases(Baroffio et al., 1999). Another study provided augmented feedback, consisting of norm-referenced graphic summaries of teachers’ clinical teaching performance ratings, together with individually written clinical teaching effectiveness guide-lines, to attending staff and residents (Litzelman et al., 1998).Two studies assessed the benefits of a four-day workshop designed to improve teachers’ cognitive styles (Mahler & Benor, 1984; Mahler & Neumann, 1987), and two studies assessed the impact of an intensive feedback and seminar method on clinicians’ teaching behaviors (Skeff, 1983; Skeff et al., 1986).
모든 연구에서 behavior change를 평가했으며, 이는 3단계와 4단계에 해당한다. 4개의 연구에서 참가자의 만족도를 조사하였고, 3개의 연구에서 학습의 변화를, 7개의 연구에서 교육행동의 변화를 3개에서 학생과 시스템의 변화를 평가하였다.
All of the studies assessed behavioral change, targeting level 3 or 4 of Kirkpatrick’s model. Four studies included participant satisfaction. Three studies examined changes in learning (i.e. knowledge, attitudes or skills); seven studies assessed change in teacher behavior and three assessed change at the level of the student or system. One study assessed outcome at all four levels (Skeff et al., 1986).
(b) 방법론적 품질
(b) Methodological quality of the studies
3개는 RCT. 5개는 single-group design. 1개 연구에서 non-equivalent control group을 포함함. 8개 연구에서 pre- post- test 디자인을 활용. 3개의 연구에서는 delayed post-test 활용
Three of the eight studies (38%) were randomized controlled trials; the remaining five (62%) were single-group designs, with one study including a non-equivalent control group for one part of the intervention. All eight studies employed a pre- test–post-test design, with the addition of a delayed post-test in three.
8개 중 6개에서 설문을 활용했으며(이론적 구인에 따라 reliability를 점검함). 이 6개 중 3개는 수행능력을 객관적으로 측정함. 2개는 수행능력에 대한 관측측정만 함.
Six of the eight studies (75%) used questionnaires (the majority of which were tested for reliability and based on a theoretical construct). Three of these same six studies also incorporated objective measures of performance. The two remaining studies used observed measures of performance only.
8개 모두 참가자의 자기-보고 외에 다른 자료 출처를 사용함. 5개의 연구는 교육 행동에 대한 학생, 레지던트의 평가 활용. 5개는 숙련된 관찰자의 평가를 활용
All of the eight studies used data sources other than participants’ self-report. Five of the studies incorporated student and resident ratings of teacher behavior; five utilized trained observer ratings.
Discussion
결과 요약
Summary of outcomes
FDP에 대한 높은 만족도: 참가자가 자발적으로 참여했다는 사실 외에도, 일관되게 FDP가 수용가능했고, 유용했으며, 개인 목표에 부합한다고 응답함. 실습과 스킬-기반 방법이 높은 평가를 받았음.
High satisfaction with faculty development programs: Overall satisfaction with faculty development programs was high. Notwithstanding the fact that the participants were volunteers, they consistently found the programs acceptable, useful and relevant to their personal objectives. The methods used, especially those with a practical and skills-based focus, were also valued by program participants.
교육과 교수개발에 대한 태도 변화: 두 가지 모두에서 긍정적으로 변화했다. 개인의 강점과 약점을 더 잘 인식하게 되었고, 동기부여가 더 되었으며, 교육에 대한 열의가 생겼고, professional development에 대해 긍정적으로 평가하게 되었다. 이러한 효과는 개방형 설문과 pre-post 측정에서 모두 나타났다.
Changes in attitudes towards teaching and faculty development: Participants reported a positive change in attitudes towards faculty development and towards teaching as a result of their involvement in a faculty development activity. They cited a greater awareness of personal strengths and limitations, increased motivation and enthusiasm for teaching, and a notable appreciation of the benefits of professional development. This impact was observed both in answers to open-ended questions and in pre–post measures of attitudinal change.
지식과 스킬의 습득: 교육의 다양한 측면(구체적인 교육전략, 보다 학습자-중심적 접근) 외에도 교육적 개념에 대한 지식 향상을 언급하였다. 스킬의 습득(학습자 요구사정, 성찰 촉진법, 피드백 제공법)도 이뤄졌다. 지식의 평가를 위한 시험은 흔히 사용되진 않더라도 긍정적 변화를 보여준다.
Gains in knowledge and skills: Participants often reported increased knowledge of educational concepts and principles as well as various aspects of teaching (e.g. specific teaching strategies; a more learner-centered approach). They also described gains in skills (e.g. assessing learners’ needs, promoting reflection and providing feedback). Formal tests of knowledge, though infrequently used, also demonstrated positive changes.
교육 행동의 변화: 스스로 인식한 교육행동의 변화는 지속적으로 보고되었다. 학생들의 평가에서 FDP 참가자들이 인식하는 변화를 항상 반영하는 것은 아니지만, 교육 행동에 변화가 있는 것은 명확해보인다. 예컨대 교육 행동의 변화는 23개의 워크숍 중 15개, 10개의 세미나 시리즈 중 7개에서 드러난다.
Changes in teaching behavior: Self-perceived changes in teaching behavior were consistently reported. While student evaluations did not always reflect the changes that participants perceived, there was evidence that change in teaching performance was detectable. For example, changes in teaching behavior were reported for 15 (of 23) workshops and seven (of 10) seminar series. New educational initiatives, designed and implemented during the intervention, were also described.
조직과 학생 학습의 변화: 흔히 평가되는 것은 아니지만, 이것을 평가한 소수 연구를 보면 새로운 교육활동에 더 적극적으로 참여하고, 동료들과 새로운/개선된 네트워크를 형성한다.
Changes in organizational practice and student learning: Changes in student (or resident) behavior as well as organizational practice were not frequently investigated. However, in those few studies that examined organizational practice, participants reported a greater involvement in new educational activities and the establishment of new and improved networks of colleagues. The latter outcome was most frequently noted for the seminar series and longitudinal programs.
핵심 특징 요약
Summary of ‘key features’
경험학습의 역할: 배운 내용을 적용하는 것, 스킬을 연습하는 것, 스킬에 대한 피드백을 받는 것의 중요성이 여러 연구에서 강조되었다. 모든 연구자들은 교수들이 배운 것을 연습하고, 즉각적 관련성과 실용성이 핵심이라고 하였다.
The role of experiential learning: The importance of applying what has been learned (during the intervention and afterwards), practicing skills, and receiving feedback on skills learned was highlighted by several authors (Irby et al., 1982; Coles & Tomlinson, 1994; Hewson, 2000), all of whom suggest that faculty members need to practice what they learn, and that immediate relevance and practicality is key (e.g. Sheets & Henry, 1984, 1988).
피드백의 가치: 변화를 만드는데 피드백의 역할을 여러 보고된 인터벤션에서 명백하다. 추가로 여러 연구에서 인터벤션 전략으로서 피드백 활용을 조사하였으며, systematic한 건설적 피드백이 교육 행동의 개선을 가져올 수 있음을 보여주었다. 그러나 한 연구에서 augmented feedback은 일부 부정적 효과를 보였다.
The value of feedback: The role of feedback in promoting change was evident in many of the reported interventions. In addition, several studies (Skeff, 1983; Litzelman et al., 1998) specifically examined the use of feedback as an intervention strategy and found that systematic and constructive feedback can result in improved teaching performance. However, in one study (Litzelman et al., 1998), augmented feedback was shown to have some negative effects; this potential effect should be considered and investigated further.
동료의 중요성: 많은 연구에서 동료 관계의 이점에 대해서 언급했다. 특히 동료를 롤모델로 삼는 것, 정보와 아이디어를 서로 교환하는 것, 변화를 촉진하고 유지하는데 동료의 지지의 중요성 등을 언급했다.
The importance of peers: A number of reports (DeWitt et al., 1993; Elliot et al., 1999) commented on the benefits of peer and collegial relationships. In particular, they highlighted the value of using peers as role models, the mutual exchange of information and ideas, and the importance of collegial support to promote and maintain change.
교수-학습의 원칙을 고수하는 것: 많은 FDP가 이론/개념적 프레임워크에 기반하고 있지 않지만, 많은 연구에서 성인학습을 인용하였으며 경험학습을 인용하였다. 실제로 이러한 원칙에 입각하여 진행하는 것이 더 효과적인 교수-학습을 가져온다는 컨센서스가 나타나고 있다. instructional design 원칙 역시 자주 언급된다.
Adherence to principles of teaching and learning: Although many of the programs were not grounded in a theoretical or conceptual framework, many cited principles of adult learning (e.g. Knowles, 1988) and experiential learning (e.g. Kolb, 1984) as an organizing structure. In fact, there appears to be a developing consensus that adherence to these principles promotes more effective learning and teaching. Principles of instructional design were also frequently cited.
목표 달성을 위한 다양한 교수법 활용: 앞서 언급된 바와 같이 모든 인터벤션은 여러 교수법을 활용하며(소그룹 토의, 상호작용 연습, 롤플레이, 시뮬레이션) 강의만 하는 것은 없다. 명백히 모든 프로그램은 다양한 학습스타일에 맞춰야 할 필요성과 더불어 다양한 목표를 달성하기 위해서는 다양한 방법이 필요함을 인식하고 있다.
The use of multiple instructional methods to achieve objectives: As mentioned earlier, all of the interventions included a wide range of instructional methods (e.g. smallgroup discussions; interactive exercises; role plays and simulations) and none relied on lectures alone. Apparently, each program was aware of the need to accommodate different learning styles as well as the fact that different methods are required to meet diverse objectives.
FD인터벤션과 관련한 관찰결과
Observations re faculty development interventions
맥락의 역할: 대부분의 연구는 교수 중 특정 맥락에 있는 특정 그룹의 니즈에 맞는 프로그램을 개발하였다. 이 프로그램 개발과 '맞춤형' 프로그램이 종종 성공을 이루곤 했지만, 놀라운 것은 아니다. 이러한 관찰 결과에서 배워야 할 점은 '맥락'이 핵심이라는 것이며, 연구의 결과가 일반화가능하지 않을 수 있지만, FDP의 개발의 원칙은 일반화가능할 수 있다는 점이다.
The role of context: The majority of reports describe programs that were developed to meet the needs of a particular group of faculty members, in a particular context. To the extent that this development and ‘match’ were often successful, it is not surprising that there were many reports of changes in the desired direction. One lesson to be learned from this observation is that context is key, and that although the results of these studies may not be generalizable, the principles of faculty development might be.
맥락은 또 다른 의미에서 중요한데, Kirkpatrick에 따르면 변화가 일어나려면 네 가지 조건이 맞아야 한다. (1)변하고자 하는 욕망이 있어야 하며, (2)무엇을 어떻게 할지에 대한 지식이 있어야 하고 (3)지지적 환경이 필요하며 (4)변화의 보상이 필요하다. 흥미롭게도 처음 두 개의 요소는 FDP를 통해 달성가능하나 나머지 두 개는 그렇지 않다. 그러나 우리가 바라는 변화는 이 지점에 있다.
Context is important in another way as well. According to Kirkpatrick (1994), four conditions are necessary for change to occur: the person must have the desire to change, knowledge of what to do and how to do it, a supportive work environment, and rewards for changing. Interestingly, the first two elements of change can potentially be achieved through faculty development activities; the last two cannot, and yet it is at this level that we expect change to occur.
참여의 특성: FDP에 참여하고자 하는 동기는 아직 해결되지 않은 의문이다. 왜 참여하는가? 왜 어떤 사람이 특정 프로그램에 특정 시점에 참여하고자 하는가? 지금까지 대부분의 참여자는 자발적 참여자였다. 아마 이제는 이 '자발성'을 넘어서야 할 때인지도 모른다. 개인적 차원의 것을 넘어서 FDP참여를 촉진하거나 방해하는 요인을 알아봐야 한다. '교육'이란 것은 '사회적 활동'이기 때문에, 참여의 사회적 결정요인에 대해 살펴볼 필요가 있을 수도 있다. 경험을 통해서 얻는 것과 워크숍을 통해서 얻는 것의 차이를 볼 필요도 있다.
The nature of participation: Motivation to attend faculty development activities remains an unanswered question. What motivates participation? What determines whether someone will take advantage of specific offerings at a particular time? To date, the majority of participants are volunteers. Perhaps it is time for us to move beyond ‘volunteerism’ as we strive to enhance teaching and learning. It would also be worth exploring factors beyond the individual that encourage or impede attendance. As teaching is a ‘social activity’ (D’Eon et al., 2000), the social determinants of participation merit further inquiry. It would also be worthwhile to conduct further studies to determine what is learned through workshops vs. experience.
FD의 facilitator로 참여하는 것의 효과도 연구할 가치가 있을 것이다. "가르치는 것은 두 번 배우는 것과 같다" 라는 말이 있다. 흥미롭게도 지금까지 어떤 연구도 FD facilitator의 참여의 영향을 연구하지 않았다. FD intervention에 faciitator로 참여하기 위해서는 독특한 스킬과 자질이 필요할 것이라는 것이 우리의 생각이다.
The impact of participation on faculty development facilitators would also be worthy of investigation. It has been said that ‘‘to teach is to learn twice’’. Interestingly, no studies to date have examined the impact of participation on faculty development facilitators. It is our impression that facilitating a faculty development intervention requires a unique blend of skills and aptitudes that should be examined in greater depth.
확장 프로그램의 가치: 더 장기간 이뤄지는 프로그램(세미나 시리즈)이 단기성 프로그램보다 더 성과를 낼 가능성이 높아 보인다. 예컨대 세미나 시리즈는 네트워크를 형성시키고, 협력적 관계를 만들어준다. 이러한 인터벤션은(펠로우십 포함) FDP이후 교육활동에 더 많이 참여하게 하며, 이는 지속가능성을 시사한다. 단기 프로그램과 장기 프로그램의 더 철저한 비교가 필요하다.
The value of extended programs: Our review of findings by intervention type suggests that longer programs, extended over time (e.g. the seminar series), tend to produce outcomes not apparent in one-time interventions (e.g. short courses or workshops). For example, in several instances the seminar series resulted in the creation of networks and cooperative interactions among colleagues that are possible when a group meets over time (e.g. Rayner et al., 1997). These interventions, as well as fellowships, also reported more involvement in educational activities following the faculty development activity, implying sustainability over time. A more rigorous comparison of ‘short’ and ‘long’ interventions would be beneficial to test out the hypothesis that extended programs yield more long-term changes.
FDP의 다양한 대안 고려: 이번 연구에서 전통적인 면-대-면 방식의 FDP에 지나치게 의존하고 있음이 드러났다. 이러한 인터벤션이 일정관리에 장점이 있고, 관심이 있는 교육자들의 커뮤니티를 형성해주며, 동기를 더 부여해주는 것으로 나타나지만, 다른 방법(온라인 교육, 자기주도 학습, 피어-코칭)도 고려해봐야 하며 멘토링도 고려해봐야 한다. 여기에 속하는 일부 연구들이 'strength of findings'에서 높은 점수를 받았다.
The use of ‘alternative’ practices: The current literature demonstrates an over-reliance on traditional face-to-face methods such as workshops and seminars. Whereas these interventions seem to have the stated advantage of ease of scheduling, building a community of interested educators and increasing motivation, we should consider other methods that include online and self-directed learning, peer coaching (Flynn et al., 1994) and mentorship (Morzinski et al., 1996). It is interesting to note that some of the studies that scored highly on ‘strength of findings’ used alternative methods (e.g. individual feedback session).
방법론적 이슈와 관련한 관찰결과
Observations re methodological issues
더 철저한 연구설계의 필요성: 1992년 Hitchcock 등은 FDP를 더 철저한 질적/양적 설계로 평가해야 한다고 언급했다. 그 때부터 상황은 크게 달라진 것 같지 않다. 본 리뷰에서도 더 철저한 연구를 통해서 흔히 마주치는 연구설정상의 문제를 극복할 필요를 제시한다. 만약 가능하다면, RCT를 고려하거나 최소한 대조군을 포함해서 FDP가 정말 차이를 만드는지 더 일반화가능한 결론을 내야 할 것이다.
The need for more rigorous designs: In 1992, Hitchcock et al. commented on the need to better evaluate faculty development programs and use sound qualitative and quantitative designs to document outcomes. The situation does not seem to have changed significantly since then. The results of this review suggest the need to conduct more rigorous research studies and overcome commonly encountered design problems. If possible, we should consider the use of randomized controlled trials, or at least comparison groups, so that we can make more generalizable statements about whether faculty development does, indeed, make a difference.
문헌을 검토한 바, 엄격한 질적연구방법이 잘 활용되지 않음을 발견했다. 동시에, 많은 저자들이 FD활동 이후에 교수들의 열정/갱신(renewal)/변화에 대한 직관적인 인상을 기술했다. 그러나 현재까지의 연구방법은 이러한 직관이나 관찰 일화를 잘 잡아내지 못하고 있다. 더 나아가 비록 FD활동이 교육활동에 대한 흥미에 불을 지핀다는 일반적 동의가 있지만, 이것이 어떻게 도달되는 것인지, 이 열망이 어떤 것인지 등이 더 면밀히 조사될 필요가 있다. 많은 경우 질적연구를더 많이 활용할 경우 얻을 수 있는 이점이 많다.
In reviewing the literature, we perceived an underutilization of rigorous qualitative methodologies. At the same time, many authors described an intuitive impression of enthusiasm, renewal and change following a particular faculty development activity. Current methods do not adequately capture these intuitions or anecdotal observations. Moreover, although there is general agreement that faculty development activities kindle interest in educational activities, how this is achieved, and what this inspires, needs to be examined more carefully. In many ways, a greater use of qualitative methods (e.g. Freeman et al., 1992) would yield considerable benefits.
FDP는 복잡한 세팅에서 이뤄지는 복잡한 인터벤션이다. 우리의 개념틀에서 지적한 바와 같이, 많은 매개변수(개인 특성, 교사의 직위와 책임) 등이 통제불가능한 외적 요인으로 작용한다. 이것이 평가가 어려운 이유이며(변화가 있다고 해서 프로그램의 기여가 아닐 수 있다), 새로운 연구방법론이 필요한 이유이다. Blumberg와 Deveau는 교육 혁신/인터벤션의 학문적 전파/교육제품(product)개발/도입을 평가하기 위한 모델을 개발하였다. 이것이 우리가 고려해야 할 것이며, 조직에 대한 영향과 더불어 기대 성과와 '기대하지 않은' 성과의 가치를 고려해야 한다.
Faculty development activities represent complex interventions in complex settings (Drescher et al., 2004). As noted in our conceptual framework, many intervening, mediating variables (e.g. personal attributes; teacher’s status and responsibilities) interact with uncontrollable, extraneous factors. This is one of the many reasons that evaluation of effectiveness is difficult (for even if changes are noted, they may not definitively be attributed to the program) and that new research methodologies are required (e.g. Campbell et al., 2000). Blumberg & Deveau (1995) have developed a model by which to evaluate an educational innovation/ intervention that looks at academic dissemination, product development and implementation. This is something that we should consider in faculty development. We should also consider the value of examining anticipated and ‘unanticipated’ outcomes (e.g. Blumberg & Deveau, 1995), including impact on the organization.
참가자 만족도에 보다 관심을 기울이기: 참가자 만족도 자료의 가치를 다시 돌아보아야 할 때이다. 비록 FDP에 대한 반응은 초보 단계의 평가이지만, 변화의 토대가 된다. 참가자들의 만족이 중요한 이유는 높은 만족도가 더 학습하고자 하는 동기를 부여해주고, 전문성-개발 활동에 참여하게 해주기 때문이다. 또한 프로그램 개발자에게 가치있는 피드백이 되기도 한다. Belfield 등이 말한 바와 같이 참가자들의 만족도는 교육의 잠재적 효과에 대한 대강의 대리지표이다. 그러나 특정 프로그램에 대한 만족도는 그러한 정보의 목적과 활용이 명확하기만 한다면중요한 정보가 되기도 한다. 우리의 견해로는 만족도를 완전히 무시하기보다는 가치를 만들어나갈 수 있어야 한다. 참가자의 경험과 스토리에 대한 질적 연구방법(네러티브 분석, 결정적 사건 분석)은 또 다른 접근법이 된다.
Attention to participant satisfaction: It is time to re-affirm the value of participant satisfaction data. Although reaction to the program is an elementary level of evaluation, it is fundamental for change to occur. Participant satisfaction is important if faculty members are to be motivated to learn and to attend professional development activities. It also gives valuable feedback to program planners. As Belfield et al. (2001) have said, participant satisfaction is a crude proxy for the substantive effects of education. However, information on the reactions of participants to a specific program provides valuable information, as long as the purpose and use of such information is made explicit. In our opinion, we must build on the value of participant satisfaction rather than discredit it completely. Applying qualitative methodologies to participants’ experiences and stories (e.g. analysis of narratives; critical incident technique) is another approach worth
pursuing as we try to understand participants’ reactions to faculty development offerings.
성과 측정: 지금까지의 연구결과를 보면 변화 측정에 있어 자기평가와 설문에 지나치게 의존한다. 더 나아가기 위해서 새로운 평가방법을 고려해야 한다. 예컨대 Simpson 등은 교수의 교육능력을 개발하기 위한 표준화된 교육 상황을 개발하였으며, Zabar 등은 objective structured teaching examinations 를 통해서 효과를 평가했다.
Outcome measures: The literature to date suggests an overreliance on self-assessments and survey questionnaires to assess change. To move forward, we should consider the use of novel assessment methods. For example, Simpson et al. (1992) have developed standardized teaching situations to develop faculty teaching skills; Zabar et al. (2004) have utilized objective structured teaching examinations to evaluate impact.
적절한 측정은 reliable하고 valid 해야 한다. 대부분의 연구는 psychometric property를 보고하지 않았다. FD개발자들과 연구자들은 validity와 reliability가 확립된 설문지 활용을 고려할 필요가 있다. 혹은 그러한 척도를 개발하고자 노력해야 한다. 예컨대 많은 교수 효과에 대한 여러 scale이 개발되어 있으며, 가능하다면 이러한 평가도구를 사용하고 리소스를 더 공유해야 한다.
Accurately measuring change requires reliable and valid measures. The majority of studies in this review used questionnaires for which psychometric properties were not reported. Faculty developers and researchers interested in assessing change should consider using questionnaires that have already been tested for validity and reliability, or work to establish these measures. For example, a number of scales and measures of teacher effectiveness have been developed in education (e.g. Gibbs & Coffey, 2004). Whenever possible, we should try to make use of these assessment tools and collaborate in order to share resources more consistently.
우리는 다양한 수행능력의 척도(자기평가, 비디오테입평가, 학생평가) 간 상관관계를 살펴봄으로써 모든 척도를 모든 연구에 사용하지 않아도 되게 해야 한다. 예컨대 일부 연구에서는 비디오테입 평가와 지식 검사의 강한 상관관계를 보고했다. 이들 연구결과는 입증되기만 한다면 항상 직접 관찰(비용과 시간이 많이 드는)을 사용하지 않아도 된다는 것을 시사한다. 비슷한 결과에 따르면 학생이나 레지던트의 교수 수행능력에 대한 평가를 (지식검사와 함께 사용하여) 비디오테입 녹화 대신 사용할 수 있다. 그러나 삼각측량의 가치는 축소될 필요가 없다. 대부분의 높이 평가된 연구들을 보면 성과 측정을 위해서 다양한 방법을 사용하였다.
We should also try to correlate different measures of performance (e.g. self-assessment questionnaires and videotape recordings; student assessments and faculty self-ratings) so that we do not need to include all measures of change in every study. For example, several studies (e.g. (Mahler & Benor, 1984; Sheets & Henry, 1984) found a strong correlation between videotape ratings (albeit sometimes based on single observations) and knowledge tests. These findings, if corroborated, suggest the possibility of conducting reliable evaluations without always using direct observation (which can be costly and time-consuming). Based on similar results, we might be able to use student or resident evaluations of teachers’ performance (together with knowledge tests) instead of videotaped observations. However, the value of triangulation to validate results cannot be understated. Some of the most highly rated studies (Skeff, 1983; Skeff et al., 1986) used multiple measures to assess outcome (e.g. self-ratings, videotaped observations and student ratings).
FDP의 중요한 성과는 학생들의 수행능력 향상이 되어야 한다. 우리는 따라서 교수들의 교육행동과 학생들의 성과와의 관계를 봐야 한다. 즉, 더 철저하게 학생과 레지던트의 자료를 수집해야 하며, 학생들의 교육 역량에 대한 평가는 매우 유용하다. 이러한 방식이 더 활용되어야 하나, 학생과 레지던트의 지식/태도/술기에 대한 평가가 더 면멸히 이뤄져야 한다.
An important outcome of faculty development is improved student performance. We must therefore work to seek evidence of a relationship between changes in faculty members’ teaching behaviors and learner outcomes. That is, we need to collect student and resident data (including indices of learner behaviour) more rigorously. Student evaluations of teaching competencies are invaluable; they need to be augmented, however, by a careful assessment of changes in students’ and residents’ own knowledge, attitudes and skills.
응답 편향에 관심가지기: 편향된 응답에도 관심을 기울여야 한다. Skeff등이 언급한 바와 같이 FDP 이후의 자기평가는 종종 기대보다 낮거나 더 떨어지기도 하는데, 이는 개개인이 시작시에는 스스로를 과대평가하다가 과정이 끝나면 스스로를 더 정확하게 평가하기 때문일 수 있다. Skeff 등이 말한 바와 같이 더 조직적으로 후향적 사전-, 사후- 검사를 평가하여 이러한 편향을 극복해야 한다. 한 흥미로운 연구에서 후향적 사전-검사의 결과는 (일반적인 사전검사보다) 학생의 교수에 대한 워크숍-이전 평가가 더 정확함을 보여준다. 이에 더하여 후향적 사전- 사후- 검사는 태도 측면에서 유의미한 결과를 보여주는데, 이는 전통적인 사전- 사후- 에서는 잘 드러나지 않는다.
Attention to response shift bias: The notion of ‘response shift bias’ warrants more careful attention. As noted by Skeff et al. (1992a), post-course self-ratings are often lower than expected, and occasionally decrease, when increases are expected. This may occur because individuals overrate themselves at the beginning of a course, and then after the course (when they have a better idea of what is meant by different aspects of teaching and learning), they rate themselves more accurately (Nayer, 1995). As Skeff et al. have argued, we should more systematically consider the value of retrospective pre–post testing to overcome this possible response shift bias. In an interesting study (Skeff et al., 1992a), retrospective pre-tests correlated better with students’ pre-workshop evaluations of their teachers’ performance than did the regular pre-test. In addition, the retrospective pre- and post-tests showed significant differences in attitudes towards teaching that were not apparent in more traditional pre- and post-tests.
시간 변화에 따른 변화 평가: 소수의 연구에서 시간 변화에 따른 FDP성과의 유지를 보았다. 많은 경우 1년까지 그 변화가 유지됨을 보여주었다.
Assessment of change over time: A few studies assessed the maintenance of change over time. Most of them (Mahler & Benor, 1984; Skeff et al., 1986; Steinert et al., 2001)
FD 전략간 비교: 비록 우리가 효과적인 FDP의 '핵심 특징'을 따로 떼어 놓았지만, FDP의 어떤 요소가 가장 유용한지에 대한 비교 연구는 거의 없으며, 한 방법이 다른 방법보다 우월한지에 대한 연구도 없다. 예컨대 비록 워크숍이 가장 흔한 방식이지만, 많은 연구자들이 지속적 변화를 가져오기에는 너무 짧다고 지적한다. 그러나 워크숍은 여전히 가장 많이 쓰이는 방법이다. 우리의 연구에 따르면 더 긴 인터벤션이 더 오래 지속되는 성과를 가져온다.
Comparison of faculty development strategies: Although we have attempted to tease apart key ‘features’ of effective faculty development, there is little comparative research on which components of faculty development interventions are most useful (e.g. micro-teaching; role plays) and whether one method (e.g. seminar series) is more effective than another (e.g. short courses). For example, although workshops are one of the most common methods, many have suggested that they are too short to bring about lasting change. At the same time, they persist as a method of choice. Our findings suggest that longer interventions may have more durable outcomes. This, too, requires further investigation.
이론과 실천에 기반한 FD: 리뷰 결과에 따르면 단 하나의 '완벽한 인터벤션'을 찾으려는 노력을 경계해야 한다. 실제로 다양한 접근법이 존재하며, 적절한 활용이란 환경에 따라 다 다르다. 그러나 FDP는 이론과 실제적 근거에 기반해야 한다. 아직 교육이론이 어떻게 학습이 일어나는지에 대한 통합된 이해를 제공해주지는 못하나, 학습에 있어 상당한 지지를 받는 모델이나 원칙이 있고, 이것을 기획/성과측정/효과분석에 활용해야 한다. 여기에는 다음과 같은 것들이 있다.
Grounding faculty development in theory and practice: Based on the findings of our review, we should caution ourselves against searching for the single ‘perfect intervention’. In fact, an array of approaches exists and their appropriate use may differ from activity to activity and across settings. However, the work of faculty development should be grounded in both theory and empirical evidence. While educational theory has not yet provided us with a unified understanding of how learning occurs, there are well-supported models and principles of learning that can inform us in planning interventions, measuring outcomes and analysing effects (Mann, 2002). These include principles that draw
- 인지과학 on the science of cognition (e.g. how individuals make meaning of information and store it in memory) (Regehr & Norman, 1996);
- 사회적 학습 on understandings of social learning (e.g. how learning occurs from and with others;
- 환경 the influence of the learning environment) (Bandura, 1986);
- 경험 learning through experience (Kolb, 1984);
- 성찰 and making meaning of learning and experience through reflection (Scho¨n, 1987; Moon, 1999).
- 실천 커뮤니티에의 참여 More recently, the idea of learning through participation in communities of practice has also been explored (Lave & Wenger, 1991; Boud & Middleton, 2003), and this notion will have important implications for faculty development.
프로그램과 다양한 전공간 협력: 자원을 공유하고 프로그램간 협력을 해야 한다. 교육 영역에서 배울 것이 많은데, 우리의 결과를 보면 다른 대학 교수들의 트레이닝의 리뷰에서의 결과와 유사하다. 많은 경우 이들 연구로부터 배우고 우리에게 적용해야 한다.
Collaborating across programs and disciplines: The value of sharing resources and collaborating across programs has been highlighted earlier in this review. There is also much for us to learn from colleagues in the field of education. For example, many of our findings resemble what has been found in reviews of research on training of university teachers (Gibbs & Coffey, 2004); in many ways, it would be wise to learn from these studies and incorporate their methodologies (and findings) into our work.
FDP 실천에 대한 함의
Implications for practice:
We need to:
- 우리의 성공을 토대로 하자. 성공적 프로그램에 들어있는 식별가능한, 복제가능한 요소들을 활용하자
Build on our successes. The literature describes successful programs, with recognizable, replicable elements. It is now important to tease apart the elements that work. - 이론과 교육 원칙을 설계와 개발에 더 잘 활용하자. 더 나아가 이론을 실천에 연결시켜야 한다. 교수들의 실제 교육행위를 더 잘 이해하고, 실제로 맞닥뜨리는 문제를 이해하여 이 정보를 이론에 관련지어 더 향상된 인터벤션을 개발과 효과성 평가로 이끌어야 한다.
Make more deliberate use of theory (particularly theories of learning) and educational principles in the design and development of our faculty development programs. Further, we need to link theory with practice, in an iterative cycle of asking questions in practice, studying these questions and testing our answers. We also need to better understand teachers’ educational practices and the real problems that teachers encounter so that we can use this knowledge to inform theory, which can help us in developing improved interventions and evaluating effectiveness. - 맥락의 중요성을 인정하자. 조직문화, 교육과정, 교사와 학생이 모두 '맥락'에 기여한다.
Acknowledge the importance of context. The organizational culture, the curriculum, teachers and students all contribute to a context that is critical to the effectiveness of educational change. - 장기간에 걸쳐 진행되는 프로그램을 개발하자. 학습, 실천, 성장을 축적하자
Develop more programs that extend over time, to allow for cumulative learning, practice and growth. - 학습자 간 성찰과 배움을 촉진하는 프로그램을 개발하자. 그들이 스스로를 교사로 인식하게 하자. 이것이 교수자-지도 인터벤션이 아니라 지속적인 자기주도발전의 토대가 될 것이다.
Develop programs that stimulate reflection and learning among participants, raising their awareness of themselves as teachers. This would form the basis for ongoing self-directed development rather than the need to primarily have ‘teacher-directed’ interventions. - 자발적 참여의 문제를 다시 생각하자. 많은 경우 효과적인 교육을 위해 필요한 전제조건은 참여가 없으면 달성되지 않는다. 더 나아가 FD의 자발성이란 특징을 생각할 때 조직문화와 그 조직이 교수-학습에 있어 어디에 가치를 두는지를 고려해야 한다.
Re-examine the question of voluntary participation. In many contexts, the requirement to prepare for teaching effectiveness may not be met unless participation is expected and required. Moreover, the voluntary nature of faculty development raises questions about the institutional culture and the values (both explicit and implicit) that it places on teaching and learning.
연구에 대한 함의
Implications for future research:
We need to:
- 더 철저한 연구를 수행하자. 통제그룹과 비교집단, 질적연구를 활용하자. 성과를 더 잘 정의하고, 프로그램의 시작시부터 평가를 계획하고 연구를 함께 하는 동료들과 협력하라
Conduct more rigorous research studies, using control or comparison groups and qualitative methodologies. This requires careful definitions of outcomes, planning for evaluation at the inception of any program, and closer collaboration with research colleagues. We must also find a way to corroborate anecdotal observations and capture faculty members’ stories. - 결과 중심의 연구가 아닌 과정 중심의 연구를 하자. 즉, 어떻게 변화가 일어나는 것인지 더 잘 이해할 필요가 있다. (어떻게 교수의 신념이 변하는가, 인터벤션이 교수의 성찰기술을 향상시켰는가) 질적연구방법이 더 적합할 것이다.
Carry out process-oriented studies in addition to outcomeoriented ones. That is, we need to better understand how change occurs, both as a result of the intervention and within the individual (e.g. how did teachers’ beliefs change; did the intervention result in improving teachers’ reflective skills). In fact, qualitative methods may be more appropriate here. - 수행능력-기반 변화를 측정하고 이를 위한 척도를 개발하자.
Continue to develop and utilize performance-based measures of change. The use of these methods, which do exist, is an essential and natural next step. - 데이터 수집에 다양한 방법을 사용하자
Use multiple methods and data sources to allow for triangulation of data. - 평가도구의 타당도와 신뢰도를 평가하자. 적절한 도구가 있다면 새로운 도구 개발에 앞어서 먼저 고려되어야 한다. 표준화된/비교가능한 도구를 사용하자.
Assess and report the validity and reliability of instruments used. Further, where appropriate instruments exist, these should be considered in preference to developing new instruments. Using standardized or comparable measures across studies will help to understand the field and improve the quality of research in this area. - 다양한 변수가 예측불가능하게 돌아가는 복잡한 환경에서 이뤄지는 인터벤션에 대한 연구를 장려하자. 여러 요인 간 상호작용이 있는 연구를 더 해야 한다.
Promote studies in which an intervention is recognized as occurring in a complex environment in which many unforeseen and unpredictable variables play a role. We need to conduct more studies in which the interaction between different factors is investigated, highlighting under what conditions and why an intervention might be successful or not. - 서로 다른 FD 방법 간 비교하자
Compare different faculty development methods to enable an analysis of which features of faculty development contribute to changes in teacher performance. - 시간에 따른 변화를 평가하자.
Assess change over time. This is important both in determining any enduring effects, and in understanding which interventions or factors may be associated with more sustained change. Longitudinal follow-ups may also help us to understand the development of faculty members throughout their careers. - 기관이나 조직에 대한 FD의 효과를 더 철저하게 평가할 수 있는 수단 개발
Develop means of assessing the impact of faculty development on the institution/organization in a more rigorous and systematic fashion. - 이론/개념틀 안에서 연구 진행. 결과 해석에 이론 활용
Embed our research studies in a theoretical or conceptual framework, and utilize theory in the interpretation of our results. - 의학 외 분야와 협력
Collaborate with colleagues within and outside medicine.
HITCHCOCK, M.A., STRITTER, F.T. & BLAND, C.J. (1992) Faculty development in the health professions: conclusions and recommenda-tions, Medical Teacher, 14(4), pp. 295–309.
A systematic review of faculty development initiatives designed to improve teaching effectiveness in medicaleducation: BEME Guide No. 8.
Author information
- 1Faculty of Medicine, McGill University, Montreal, Quebec, Canada. yvonne.steinert@mcgill.ca
Abstract
BACKGROUND:
OBJECTIVE:
METHODS:
RESULTS:
KEY POINTS:
CONCLUSIONS:
- PMID:
- 17074699
- [PubMed - indexed for MEDLINE]
'Articles (Medical Education) > 교수개발(Faculty Development)' 카테고리의 다른 글
| 교수개발: 꿈의 영역? (Med Educ, 2009) (0) | 2016.01.15 |
|---|---|
| 새천년의 교수개발: 도전과 미래 방향(Med Teach, 2000) (0) | 2016.01.15 |
| 평가에 관한 교수개발: 역량중심교육과정의 잃어버린 고리(Acad Med, 2011) (0) | 2015.09.08 |
| 교수개발프로그램의 전이(transfer)를 높이기 위한 12가지 팁(Medical Teacher, 2014) (0) | 2015.04.17 |
| ● 의사를 더 나은 교육자(educator), 변화의 주체(change agent)가 되게 하기 위한 Harvard Macy Institute (0) | 2014.06.19 |