국제 보건전문직 교수개발 펠로우십의 효과 측정 : FAIMER Institute

Measuring the effects of an international health professions faculty development fellowship: the FAIMER Institute

W. P. BURDICK1, D. DISERENS1, S. R. FRIEDMAN1, P. S. MORAHAN1, S. KALISHMAN2, M. A. EKLUND2,

S. MENNIN3 & J. J. NORCINI1

1FAIMER, Philadelphia, PA 19104, USA, 2Undergraduate Medical Education and PEAR, University of New Mexico SOM,

Albuquerque, USA, 3Mennin Consulting & Associates Inc., Sao Paulo, Brazil

교수개발 프로그램의 평가에 대해서 많은 예들이 있지만, 초국가적인(transnational) 프로그램에 대한 평가는 제한적이다. 우리는 여기서 국제보건전문직 교육 프로그램인 FAIMER Institute의 효과성 평가를 다루고자 한다. 이 프로그램은 교육과 관련된 것 뿐만 아니라 리더십과 경영에 대한 내용도 다루고 있으며, 교육자들의 지속가능한 커뮤니티를 개발하고자 했다. 

While there are many examples of evaluations of faculty development programs in resource rich countries, evaluation of transnational programs for faculty from developing countries is limited. We describe evaluation of the effects of the FAIMER Institute, an international health professions education fellowship that incorporates not only education content, but also leadership and management topics and, in addition, strives to develop a sustained community of educators. 

데이터는 후향적 사전/사후 설문조사와 인터뷰를 활용하여 수집하였다. 결과를 살펴보면 이 프로그램에 참여함으로서 교육리더십의 지식과 기술, 방법론을 향상시킬 수 있었으며, 그 지식을 본국의 기관에 적용할 수 있었다고 응답했다. 참가자들은 스스로 모든 교육과정 주제에 대한 역량이 향상되었다고 평가하였으며 인터뷰를 통해서 최소 하나 이상의 리더십 기술을 습득했음을 확인할 수 있었다. 

Data were obtained via retrospective pre/post surveys, as well as interviews. Results indicate that participating health professions faculty from developing countries are augmenting their knowledge and skills in education leadership, management, and methodology, and applying that knowledge at their home institutions. Fellows’ perceptions of importance of, and their own competence in, all curriculum theme areas increased. Interviews confirmed a nearly universal gain of at least one leadership skill. 

이러한 결과는 FAIMER 모델이 제공한 고관여(high engagement)경험이 변화를 이루는데 필요한 교육과 리더십/경영을 통합했음을 보여주며, 여러 문화권과 국가적 맥락을 넘어서는 지식과 기술을 제공하였고, 지지적/세계적/전문가의 네트워크 구축을 이루었음을 보여준다.

Findings suggest that the high engagement experience of the FAIMER model offering integration of education and leadership/management tools necessary to implement change, provides knowledge and skills which are useful across cultural and national contexts and results in the development of a supportive, global, professional network.

---

Background

보건전문직 교육과 관련한 교수개발에 있어서 리더십과 경영에 대한 것은 중요한 요소 중 하나로 지적되어왔다.

Leadership and management development has been recognized as a critical component of faculty development in health professions education, along with teaching skills (Gruppen et al. 2003; Steinert et al. 2003; Steinert et al. 2003; Souba 2004; Dwyer et al. 2006; Robins et al. 2006; Mouradian & Huebner 2007).

교수개발과 교육리더십 프로그램의 방법, 내용, 평가는 주로 고소득국가(resource rich)에서 이루어져왔고, 저소득국가(resource limited)에서는 연구가 제한적이었다. 우리는 국제 보건전문직 교수개발 프로그램에 대해 평가해보고자 한다.

The methods, content and evaluation of faculty development and education leadership programs have been widely addressed in resource rich countries (Rubeck & Witzke 1998; Hewson et al. 2001; Armstrong et al. 2003; Searle et al. 2006). However, they have been less well studied in resource limited regions (Svab et al. 1999; Yolsal et al. 2003; Dwyer et al. 2006; Wong & Agisheva 2007), where studies have focused on changes in management systems and processes, or improvements in work climate (Perry et al. 2005; LeMay & Ellis 2008) with little attention to the usefulness and relevance of curriculum content in diverse contexts. To address these questions we evaluated the effects of an international faculty development program in health professions education methods and leadership.

문헌에서 주로 다뤄지고 있는 교육리더십과 경영의 개념/방법은 마찬가지로 고소득국가에 집중되어 있고, 다른 국가들에는 적용가능하지 않을 수도 있다.

The methods and concepts of educational leadership and management commonly espoused in the literature are oriented to institutions in relatively resource rich regions and may not apply in other parts of the world (Sapre 2000; House et al. 2004; Mendelek et al. 2006; Morahan et al. 2006).

이러한 개념을 가르칠 때는 맥락(context)를 인지하고 민감하게 반응할 수 있어야 한다. 리더십에 중요하게 작용하는 위계/협력/상호작용/혁신 등의 요소들이 문화적 가치와 사회적 규범에 들어가 있기 때문이다.

Teaching these concepts, however, requires awareness and sensitivity to context, with particular attention to authentic local application since hierarchy, collaboration, interaction, innovation and other factors important to leadership are embedded in cultural values and social norms (Sapre 2000; House et al. 2004; Mendelek et al. 2006; Morahan et al. 2006).

교육자들의 국제적 네트워크를 개발하는 것은 리더십개발에 있어서 중요한 요소로 지적되어왔다. 그러나 이러한 개념은 저소득국가의 환경에서는 잘 작동하지 않으며, 국소적/지역적인 교수개발 프로그램이 보건전문직 교육자들을 집중시킬 수 있을지는 몰라도 교육자들간의 전문직 네트워크를 이루는 데에는 관심이 부족하다.

Development of international networks of educators has been cited as an important component of leadership development for over a quarter of a century (Culbertson 1981; Bolam et al. 2005; Cross et al. 2006). However, this concept rarely has been operationalized for educators from resource limited environments. Local and regional faculty development programs may result in concentrations of health professions educators, but there appears to be little intentional encouragement of professional networks for educators (Gruppen et al. 2006; MacLeod et al. 2007; Moses et al. in press).

2001년 FAIMER institute는 라틴아메리카, 아프리카, 아시아 등에서 개발도상국의 중견급 교수들을 대상으로하여 펠로우십 프로그램을 시작하였다. FAIMER institute의 목적은 교육리더십/경영/방법론적 기술을 강화하여 보건전문직 교육자들의 커뮤니티를 형성하는 것이었다.

In 2001, the Foundation for Advancement of International Medical Education and Research (FAIMER) initiated the FAIMER Institute, a fellowship program for mid-career faculty members from health professions schools in developing countries in Latin America, Africa, and Asia (Norcini et al. 2005; Burdick et al. 2006). The goals of the FAIMER Institute are to strengthen education leadership, management, and methodology skills, and develop a community of health professions educators

FAIMER 프로그램은 초국가적 접근법을 취했다는 점, 저소득국가의 보건전문직에 대한 강조를 했다는 점, 리더십 스킬과 전세계적 커뮤니티 형성을 목적으로 했다는 특징이 있다. 이러한 강조는 지금 이뤄지고 있는 국제적 보건자원 투자를 보완하는 역할을 한다고 할 수 있다. 또한 점차 많은 사람들이 보건시스템의 인력양성과 Capacity building에 집중할 것을 권고하고 있기도 하다.

The FAIMER program is unusual in its transnational approach and its emphasis on a mix of health professions faculty from resource limited countries, with an explicit goal of improving leadership skills and creating a global community of health professions educators. This emphasis is complementary to the approach of most current international health resource investments, which are devoted to priority disease prevention and treatment programs, and consistent with an increasing number of authors recommending funding focused specifically on capacity building of human resources in the health system (Drager et al. 2006; McCourt & Awases 2007; Awofeso et al. 2008; Global Health Workforce Alliance 2008).

FAIMER Institute model은 자신의 기관에서 교육혁신프로젝트를 집중적으로 진행하는 2년의 파트타임 fellowship으로 되어 있다.

The FAIMER Institute model consists of a two-year parttime fellowship that focuses on an education innovation project in the Fellow’s own institution, supported by the institution’s leadership.

Fellow들은 첫 해에 3주간 본 기관에 상주하다가, 1년간의 원거리 학습 세션을 가진 후, 두 번째 상주 시간을 갖고, 마지막으로 남은 두 번째 해에 원거리 학습을 하게 된다.

Fellows are in residence at the beginning of the first year for three weeks (Session 1), followed by a year-long distance learning session (Session 2), culminating in a second residential session lasting two weeks (Session 3), with a final distance learning curriculum for the remainder of the second year (Session 4).

이 모델은 다양한 고관여(high-engagement methods) 방법을 사용하여 의도적으로 fellow간 유대를 만들고 강화시킨다. 여기에는 다음과 같은 방법이 포함된다.

The Institute model intentionally creates and reinforces the bonds between Fellows by a variety of high-engagement methods. These include...

• team building and group dynamics exercises, 
• intensive interaction and dialogue during the residential sessions, evening ‘‘learning circles’’ during which personal stories are shared (Baldwin 1998; Danzig 1999; Wheatley 2002), 
• continuous emphasis on development of a ‘‘safe’’ learning environment, 
• creation of on-line discussion leader teams, 
• telephone contact every several weeks during the non-residential sessions, 
• and encouragement of ‘‘social presence’’ on the listserv (Kreijns et al. 2004).

목적

Aims

The purpose of this paper is to present participants’ perceptions of the utility and personal impact of the FAIMER Institute fellowship

방법

Method

University of New Mexico School of Medicine Office of Program Evaluation, Education and Research 에서 평가팀이 와서 프로그램 평가, 자료 수집, 데이터 분석을 수행하였다.

An external evaluation team from the University of New Mexico School of Medicine Office of Program Evaluation, Education and Research has conducted the program evaluation, collected data, and performed initial data analysis (Mennin et al. 2003; Kalishman et al. 2004; Kalishman et al. 2005; Mennin et al. 2006; Mennin et al. 2007; Office of Program Evaluation, Education, and Research 2008).

연구 대상 Study population

척도 Measures

자기평가지 사용

We used self-ratings of knowledge and importance of curriculum topics and open-ended responses to a structured interview to study the effects of the Institute fellowship on Fellows’ knowledge, skills, networks, and leadership.

후향적 설문조사 Retrospective Pre/Post Survey

Fellows completed a retrospective-pre/post questionnaire (Lamb & Tschillard 2005), at the end of each of the two face-to-face residential sessions (Sessions 1 and 3). The first part of the survey asked respondents to ‘‘rate the importance to you’’ of a series of FAIMER curriculum topics on a scale from 1 (none) to 7 (very high), with separate rating scales for ‘‘before FAIMER’’ (retrospective pre-test) and ‘‘today’’ (posttest). The second portion of the survey asked respondents to ‘‘rate your skills, knowledge or competence to address’’ each of the same topics on a scale from 1 (none or no skill) to 7 (expert, teach others), also for ‘‘before FAIMER’’ (retrospective pre-test) and ‘‘today’’ ( post-test). The issues and topics for the retrospective pre/post survey were chosen to be reflective of the learning emphases and goals of the FAIMER Institute.

구조화된 면접 Structured interview

An individual structured 1–2 hour interview was conducted with each Fellow by two members of the external evaluation team (SK and MAE) at the end of their second residential session (Session 3).

분석 Analyses

Mean topic sub-scale scores were created for each Fellow by adding the ratings across items within each sub-scale and dividing by the number of items. Means, standard deviations and effect size were then calculated for each pre/post topic subscale for both importance and knowledge/skills. Effect size, the standard mean difference between paired postparticipation and pre-participation ratings, was calculated by using the mean paired difference between post-participation and pre-participation ratings as the numerator and the standard deviation of the paired differences as the denominator (Cohen 1988).

Internal consistency of responses for both pre and post responses within each topic sub-scale of the retrospective pre/post survey, based on five years of data, was calculated via Cronbach’s alpha for topics containing more than two items and via Pearson correlation coefficients for topics with two items. Cronbach’s alpha was calculated separately for years with different numbers of items. 

The internal consistency coefficients for Session 1 ratings of Importance varied from .61 to .95 with a mean of .83, and for Session 3 varied from .48 to .85 with a mean of .71. The internal consistency coefficients for rating of Knowledge for Session 1 varied from .64 to .96 with a mean of .86, and for Session 3 varied from .54 to .89 with a mean of .74.

Interviews were recorded and transcribed, and responses sorted into general categories of Useful Knowledge, Skills and Tools, Network, and Leadership by the external evaluation team, (SK, MAE). A qualitative content analysis approach (Hsieh & Shannon 2005) was used by two of the authors (WPB, DD) to independently code the interview response data and identify sub-themes within each major category. The two coders refined coding subcategories by consensus. The number of responses related to each category was tabulated independently, followed by review and reconciliation of discrepant counts.

결과 Results

fellow들의 코멘트는 크게 세 카테고리로 분류되었다.

Fellows’ comments about the social network among their FAIMER colleagues fell into three general categories: 

• the network’s qualities, 
• the network’s utility or value to them, 
• and the nature of the work done with FAIMER colleagues.

결론

Conclusions

구조화된 후향적 설문조사에서 fellow들은 지식과 awareness가 유의미하게 향상되었다고 응답하였다. 

In a structured retrospective pre/post survey, Fellows reported significant increases in their knowledge and awareness of concepts in education methods and leadership/management. In addition, nearly all Fellows mentioned using at least one leadership skill or tool, suggesting that these tools can be successfully adapted to non-Western and resource-limited cultural, institutional and national contexts (Sapre 2000; House et al. 2004; Mendelek et al. 2006; Morahan et al. 2006). 

특히 appreciative inquiry와 리더십이 유용하다고 대답하였다. 자기가 지역에서 담당하고 있는 프로젝트에서 적용하도록 강조한 것이 transferability에 도움이 되었다고 말했다. 또한 스스로의 리더십 스타일이 변화했으며, 자신감, 자기효용감에도 영향을 주었다고 했다.

Fellows particularly identified appreciative inquiry and leadership as useful to them (Cameron et al. 2003; Stavros & Torres 2005). The emphasis on application of concepts to an authentic local project may be partly responsible for the transferability of these concepts to the Fellows’ home systems (Vicere & Fulmer 1998; Nchinda 2002). In addition, a majority of Fellows reported a change in their own leadership style. This change included increased self-confidence, a fundamental component of self-efficacy (Sloma-Williams et al. 2009, in press).

저개발국가에서 이루어지는 대부분의 교수개발은 교육스킬의 향상에만 집중하고, 리더십이나 관리기술에는 덜 집중하는 편이다. 또한 교육자들간의 국제적 네트워크를 형성하는 것에 목적을 두는 프로그램은 거의 없다.

Most of faculty development in health professions education in resource-limited areas, when it exists, focuses on improvement of teaching skills with less, if any, emphasis on development of leadership and management skills (Yolsal et al. 2003; Beier-Holgersen et al. 2004; Anonymous 2007; Bansal & Supe 2007). Few programs explicitly aim to create an international social network of educators (Mouradian & Huebner 2007).

5년간의 FAIMER fellow들에 대한 조사와 면접으로부터 대부분의 fellow가 리더십이나 경영에 대한 교육이 도움이 되었다고 응답하였으며, 국제적 네트워크가 상당한 가치가 있다고 말했다.

Survey and interview data from five years of FAIMER Fellows provide convergent evidence that the majority of Fellows find the concepts and methods of education leadership and management useful and applicable in their professional settings and that the international network of colleagues created is both valuable and supportive.

대부분의 fellow들이 교육방법이나 개념에 비해서 리더십/경영 방법과 개념에 대해서 많이 언급했다.

In the present study, data from the retrospective pre/post surveys about education, leadership and management topics illuminates data from qualitative analysis of interviews. A higher percentage of Fellows mentioned leadership/management methods and concepts compared to education methods and concepts when interviewed about what they had applied in their work that they had learned at the FAIMER Institute. 

한 가지 가능한 설명은 리더십/경영에 대한 주제가 그들에게 비교적 새로운 것이라는 점이다. 후향적 사전조사가 이것을 설명해준다. 사전 지식 수준은 변화관리나 경영 등에 대해서 거의 제일 낮았으나 이들 영역이 효과크기의 증가가 가장 컸다.

One possible explanation is that leadership/management topics were newer to most of them. Data from the retrospective pre survey supports this explanation; ‘‘pre’’ levels of importance and knowledge were lowest for change theory and management, personal professional development, program evaluation, project management, advanced leadership, and whole systems models to sustain change and these areas had the largest increase in effect size. 

우리는 교수개발 프로그램은 리더십이나 경영 기술까지 다루어야 한다고 주장해왔는데, 이는 모든 교육과정개혁은 반드시 '관리되어야 하는 조직의 변화'를 수반하기 때문이다. 

Our thesis has been that faculty development programs need to be expanded to include leadership or management skills in addition to education methods, because any curriculum innovation inevitably involves organizational change that needs to be led and managed. The evaluation data presented here, both quantitative and qualitative, suggest that FAIMER Fellows appreciated, learned and used these skills. 

교육방법, 학생평가, 국제적 의학교육 등은 작은 효과크기를 보였으며, 이는 fellow들이 이미 그 전에 지식과 개념을 많이 가지고 있음을 말한다. 그럼에도 불구하고 질적/양적 연구결과 모두 fellow들이 교육영역의 지식과 기술 역시 많이 사용하고 있음을 확인해주었다.

The topics education methods, student assessment, and international medical education showed slightly smaller effect sizes, which may be reflective of Fellows’ higher pre-intervention appreciation and knowledge of education methods compared to leadership and management concepts. Nevertheless, the significant increases reported in the quantitative assessment are also consistent with the qualitative data showing that a majority of Fellows reported using knowledge or skills gained in the education domain.

교수개발 프로그램의 평가는 언제나 쉬운 일은 아니지만, 여기에서는 데이터의 의미있는 해석에 양적/질적 데이터를 병합하는 것의 가치를 보여주고 있다.

Evaluation of faculty development programs in health professions education continues to present a challenge (Jayawickramarajah 1992; Irby & Hekelman 1997; Amin et al. 2004); the data here illustrate the value of qualitative and quantitative data in converging on a meaningful interpretation of the data (Greene & Caracelli 1997; Reinelt & Russon 2003; Grove et al. 2006; Westhues et al. 2008).

연구의 한계는 자기보고식이며, 개개 참가자들의 reflection에 기반하였다는 것이다. 여러 상황에 의해서 응답이 bias되어있을 수 있다. 우리는 외부 평가팀에게 의뢰하여 이러한 것을 최소화하려고 했다. 후향적 방법이 bias를 줄이기 위하여, 그리고 robust한 response rate를 위하여 사용되었지만 이것도 한계가 없는 것은 아니다. 향후에 institutional case study나 survey를 해야 할 필요가 있다.

A limitation of this study is that the source of all the effects data is self-report and reflection by individual participants. Responses may be biased by respondents’ appreciation of sponsorship and hesitancy to criticize a source of funding, or social desirability of an affirmative response. We sought to minimize this by having an external evaluation team conduct the interviews. (Mennin & Kalishman 2002). In addition, while, the retrospective pre/post-test methodology was chosen to ensure a robust response rate as well as to avoid the response bias of a traditional pre-test as reported by some researchers (Lamb & Tschillard 2005), this methodology is not without limitations (Hill & Betz 2005). In future research, institutional case studies and surveys of school leaders may be a means to assess further the impact of FAIMER Fellows as agents of change in their schools (Dannels et al. 2009).

저소득 국가에서의 교수개발 프로그램은 고소득 국가에서보다 더 제한적인데, 이는 이 분야가 덜 성숙되어있을 뿐만 아니라, 분야가 성장할 기회가 적기 때문일 수 있다. 시간과 돈이 부족한 것도 주요한 제한점이다.

Faculty development programs in health professions education in resource poor countries are more limited than those in resource rich countries, probably due to limited maturation of the field of health professions education and few opportunities for advancement in that domain (Nchinda 2002; Morahan & Fleetwood 2008). Lack of time and money are also major impediments to faculty development (Amin et al. 2005; Supe & Burdick 2006).

FAIMER institute의 효과에 대한 평가는 개발도상국의 교수들도 교육리더십, 경영, 방법론 등을 학습하고 이것을 적용할 수 있음을 보여준다. 지식을 축적하고 적절한 도구가 있고, 지지적 전문직 네트워크가 형성된다면 국제적 교수개발 프로그램으로서 효과적인 모델이 될 수 있음을 보여준다.

Evidence about the impact of the FAIMER Institute indicates that health professions faculty from developing countries are augmenting their knowledge and skills in education leadership, management, and methodology, and applying that knowledge at their home institutions. Integration of knowledge acquisition with the tools necessary to implement change, coupled with a supportive professional network, appears to be an effective model for an international faculty development program for participants from diverse developing regions.

---

Med Teach. 2010;32(5):414-21. doi: 10.3109/01421590903394587.

Measuring the effects of an international health professions faculty development fellowship: the FAIMER Institute.

Burdick WP1, Diserens D, Friedman SR, Morahan PS, Kalishman S, Eklund MA, Mennin S, Norcini JJ.

Author information

Abstract

While there are many examples of evaluations of faculty development programs in resource rich countries, evaluation of transnational programs forfaculty from developing countries is limited. We describe evaluation of the effects of the FAIMER Institute, an international health professionseducation fellowship that incorporates not only education content, but also leadership and management topics and, in addition, strives to develop a sustained community of educators. Data were obtained via retrospective pre/post surveys, as well as interviews. Results indicate that participatinghealth professions faculty from developing countries are augmenting their knowledge and skills in education leadership, management, and methodology, and applying that knowledge at their home institutions. Fellows' perceptions of importance of, and their own competence in, all curriculum theme areas increased. Interviews confirmed a nearly universal gain of at least one leadership skill. Findings suggest that the high-engagement experience of the FAIMER model offering integration of education and leadership/management tools necessary to implement change, provides knowledge and skills which are useful across cultural and national contexts and results in the development of a supportive, global, professional network.

Comment in

• FAIMER Regional institutes: A positive direction in medical education. [Med Teach. 2010]