프로페셔널리즘 평가 : 문헌 리뷰

프로페셔널리즘 평가 : 문헌 리뷰

Assessing professionalism: a review of the literature

DEIRDRE C. LYNCH1, PATRICIA M. SURDYK1 & ARNOLD R. EISER2

1Accreditation Council for Graduate Medical Education, Chicago, IL, USA;

2Drexel University College of Medicine and Consultant to the American Board of Medical Specialties, Philadelphia, USA

Introduction

사회는 의사들이 프로페셔널하게 행동할 것을 기대한다. 의사들의 프로페셔널하지 못한 행동에 대한 최근의 비판에 대하여 이것은 교육과 평가를 통해서만 바꿀 수 있다고 주장하는 사람도 있다. 프로페셔널리즘을 평가하는 것이 쉽지는 않지만, '측정' 없이 프로페셔널리즘을 측정하고, 그것의 성장을 확인하는 것은 불가능하다. 의학교육에서 프로페셔널리즘을 평가하는 것에 대한 리뷰문헌을 보면, Arnold는 "좋은(solid) 평가도구 없이 여러 접근법에 대해서 각각이 프로페셔널한 행동에 대해 가르치는 것에 얼마나 효과적인지에 대한 질문에 효율적으로 대답할 수 없을 것"이라고 했다.

Society expects physicians to act professionally. In response to recent criticism regarding unprofessional behavior in medicine, some argue that improving medical professionalism can only occur through changes in teaching and assessing it (Cruess & Cruess, 1997; Relman, 1998). Although assessing professionalism poses many challenges, gauging and ascertaining growth in professionalism is impossible without measurement. Concluding her review of approaches to assess professionalism in medical education, Arnold stated that ‘‘[w]ithout solid assessment tools, questions about the efficacy of approaches to educating learners about professional behavior will not be effectively answered’’ (Arnold, 2002).

의학적 프로페셔널리즘을 "ability to meet the relationship-centered expectations required to practice medicine competently"로 정의하고자 한다. 그리고 이는 의학교육의 어느 시기인지에 따라서 달라질 수 있다.

The present review defines medical professionalism as the ability to meet the relationship-centered expectations required to practice medicine competently. It varies along a continuum from positive to negative engagement and encompasses constructs such as: respect for others, integrity, altruism, accountability, duty, composure and sensitivity to diversity (ACGME, 1999; ABIMF, ACP-ASIM, & EFIM, 2002; Kuczewski et al., 2003; Surdyk et al., 2003). 

Arnold의 리뷰와 Ginsburg의 리뷰가 프로페셔널리즘 평가에 대해서 다루고 있다. Arnold는 세 가지로 분류하였다.  

• those assessing professionalism as part of clinical performance; 
• those assessing it as a comprehensive entity; and 
• those assessing separate elements of professionalism, such as humanism and ethical decision-making. 

또한 Arnold는 평가를 향상시키기 위해서 프로페셔널리즘을 독립적인 개체(separate entity)로 보아야 한다고 하면서 다음의 것들을 이야기했다. 

• including rigorous qualitative approaches, 
• exploring the extent to which the environment supports assessment, and 
• determining the usefulness of developmental expectations in assessing professionalism.

Two reviews, that by Arnold (2002) cited above and the other by Ginsburg & colleagues (2000) provide the most recent analyses regarding conceptual issues pertinent to assessing professionalism. Arnold categorized instruments into three groups: those assessing professionalism as part of clinical performance; those assessing it as a comprehensive entity; and those assessing separate elements of professionalism, such as humanism and ethical decision-making. She suggested that assessments could be improved by focusing on professionalism as a separate entity, including rigorous qualitative approaches, exploring the extent to which the environment supports assessment, and determining the usefulness of developmental expectations in assessing professionalism.

Ginsburg등은 기존의 평가를 평가자의 타입에 따라 조사하였는데, 프로페셔널리즘 평가가 어려운 것에 대해서 다음과 같은 이유가 있다고 하였다. 

• the frequent use of abstract idealized definitions, 
• the context-specific nature of professionalism, and 
• evaluator reluctance to address relatively minor lapses.

또한 이들은 평가는 인지적 outcome과 행동적 outcome을 봐야 한다고 주장하였다.

In their review, Ginsburg and colleagues (2000) examined existing assessments in terms of types of raters, such as faculty supervisors and peers. They reasoned that difficulties involved in assessing professionalism are due to the frequent use of abstract idealized definitions, the context-specific nature of professionalism, and evaluator reluctance to address relatively minor lapses. The same authors proposed that professionalism assessments should address cognitive and behavioral outcomes.

본 연구의 목적

The purpose of the present review is twofold: (1) to extend and update the work of previous authors, and (2) different from the format of previous reviews, to provide a catalog of the many and varied assessments available in the literature. The first goal is achieved by including assessments that measure professionalism in the educational environment and assessments that address diversity. The second goal is achieved by: (a) organizing assessments according to content and to types of outcomes assessed, and (b) listing assessments in a table together with brief descriptive summaries for easy reference by educators searching for specific assessments. The concluding section presents general recommendations for designing and implementing a system to assess professionalism and specific suggestions for assessing medical students, residents and practicing physicians.

Results

Of approximately 6200 abstracts, 359 were selected for further review and manual reference list searches yielded 52 more references. Of these 411 articles, 220 were excluded because they did not meet selection criteria. The remaining 191 articles provided descriptions of 88 assessments (Table 1a) [1] that fit into one or more of the following 12 categories: 

• ethics–affective (15), 
• ethics–cognitive (25), 
• ethics–behavioral (9), 
• ethics–environmental (14), 

• personal characteristics–affective (15), 

• comprehensive professionalism– cognitive (2), 
• comprehensive professionalism–behavioral (21), 
• comprehensive professionalism–environmental (5), 

• diversity–affective (8), 
• diversity–cognitive (2), 
• diversity– behavioral (1), 
• diversity–environmental (1).

Discussion

General recommendations for assessing professionalism

의학교육에서는 총괄평가가 대부분이다. 그러나 평가를 학습을 향상시키기 위한 목적으로 사용해야 한다는 관점이 신빙성을 얻음에 따라 형성평가를 다루는 문헌이 점차 늘어나기 시작했다. 일부 문헌들은 teaching에 집중하는 것이 moral reasoning을 향상시킬 수 있다고 보고하였는데, 더 구체적으로는 평가로부터 제공되는 피드백이 프로페셔널한 행동을 향상시킨다는 것이다. 이러한 결과는 프로페셔널리즘이 형성평가적으로 평가되어야 함을 제안한다. 즉 평가는 일찍 시작되어야 하고, 자주 시행되어야 하며, 장기간에 걸쳐서 진해오디어야 하고, 학습자가 변화할 기회를 주어야 한다.

Summative assessment, often undertaken to classify learners, dominates medical education. Using assessment to improve learning is gaining credibility, however, with reports of formative assessments appearing more often in the literature. Some evidence suggests that focused teaching can improve moral reasoning (Self et al., 1992; Goldie et al., 2002); more specifically, that feedback derived from assessment may improve professional behaviors (Phelan et al., 1993; Papadakis et al., 2001). These findings suggest that professionalism should be formatively assessed. This means that assessment should begin early (Lowe et al., 2001) be conducted frequently, be implemented long term, and provide learners with opportunities to change (Van Luijk et al., 2000).

공식적 교육과정이든 비공식적 교육과정이든, 교육 환경은 학습자들의 태도와 행동에 영향을 준다. 윤리적 환경과 의과대학생들의 윤리적 행동이 관계가 있음을 밝힌 연구도 있으며, 또 다른 연구에서는 전공의가 롤모델을 관찰하는 것에서 가장 프로페셔널리즘을 많이 배운다고 보고하고 있다. 사업적, 문화적 환경이 진료중인 의사들의 프로페셔널리즘에 많은 영향을 준다고 보고되고 있다. 이러한 것에 따라서 환경에 의한 프로페셔널리즘을 평가하는 것이 개인의 프로페셔널리즘에 대한 insight를 제공할 수 있다.

The educational environment, whether through formal or informal curricula, appears to influence learner attitudes and behavior (Stern, 1996). One study discerned relationships between the ethical environment and medical students’ ethical behavior (Feudtner et al., 1994). In another study, residents reported learning most about professionalism from observing role models (Brownell & Cote, 2001). Research suggests that the business (Freeman et al., 1999) and cultural environment (Hoffmaster et al., 1991) influence professionalism among practicing physicians. Consequently, assessments that gauge professionalism in the environment may provide insight into the professionalism of individuals.

프로페셔널리즘에 대한 총체적인 평가는 다양한 평가자. 다양한 평가방법, 다양한 평가환경을 필요로 한다. 서로 다른 평가자는 서로 다른 관점을 통해서 평가의 폭을 넓혀주고, 신뢰도를 높여준다. 또한 하나 이상의 평가법을 활용하는 것이 다른 평가법의 약점을 보완할 수 있다. 단 한 번의 평가로 프로페셔널리즘을 모두 평가할 수 없다. 여러 평가를 종합하는 것이 적절할 것이다. 좀 더 일반화가 가능할 것이다.

Systematic assessment of professionalism should also include many different assessors, more than one assessment method and assessment in different settings (Littlefield et al., 1996; Van Luijk et al., 2000; Ginsburg et al., 2002). 

• Research indicates that different assessors offer 
• different perspectives, thus 
• enhancing the breadth of assessment (Wooliscroft et al., 1994) and 
• multiple assessors enhance reliability (Swanson, 1987). 
• Each assessment method has strengths and weaknesses. 
• Rating forms are considered relatively easy to use, but are plagued with the ‘halo’ or ‘horns’ effect (Gray, 1996). 
• The Ethics OSCE may mitigate the latter weakness but requires several cases, and hence testing time, to obtain stable estimates of learner performance (Singer et al., 1996). 
• Consequently, using more than one assessment method may help to compensate for the weaknesses associated with any single approach. 
• Because professionalism is a complex construct, it is unlikely that a single assessment will adequately measure it.
• Using a combination of assessments, however, such as a moral reasoning assessment together with a behavioral assessment, may be adequate. 
• Assessment in different settings can help to 
• determine the generalizability of learners’ professionalism and 
• identify context specific issues relevant to learners.

Suggestions for assessing medical students, residents, and practicing physicians 

Medical students. 

의대생에 대한 평가는 대개 임상실습시기까지 미뤄지곤 하지만, 의과대학 1학년때부터 평가하는 것이 바람직하고 또한 가능하다는 연구가 있다. 또한 Ethics OSCE등을 이용하여 임상상황에서 윤리적 문제를 다루는 연습을 해야 한다.

• Assessment of medical student professionalism is often delayed until clerkship rotations. The Physicianship Evaluation Form (Papadakis et al., 2001), however, indicates that it is both desirable and possible to begin assessing student professionalism during the first year of medical school. 
• Initial use of this assessment is formative; students are invited to present their perspective and data are used to provide feedback and guide remediation. 
• Persistent patterns of unprofessional behavior, despite remediation, may provide grounds for dismissal. This performance-based, longitudinal approach helps to set professionalism expectations early, both for learners and for faculty who must commit to addressing and attempting to improve students’ professional behavior. 

• Students should also be prepared to address ethical issues in clinical settings. 
• Behavioral simulations, such as the Moral Behavior Analysis (Sheehan et al., 1987) and the Ethics OSCE (Singer et al., 1996), in which ethical dilemmas are role-played, are ideal for this purpose, especially if they include post-encounter oral or written assessments that probe reasoning used during encounters. 
• To obtain reliable data, many cases would be required, thus, these simulations should be used for formative assessment only.

Residents. Residents typically rotate through many different settings and interact with a broad range of personnel. These characteristics of residency training may present challenges to coordinating comprehensive assessment and providing opportunities for the development of long-term relationships helpful to assessing professionalism. On the other hand, changing rotations provide the option of obtaining input from various observers (e.g. nurses, patients, supervising physicians and peers), and assessing the extent to which resident professionalism varies across settings. 

• Three-sixty assessments, which obtain data from key people within a learner’s sphere of influence, capitalize on the availability of a broad scope of potential assessors (Tornow & London, 1998). 
• The Nurse Evaluation of Medical Housestaff Form (Butterfield & Mazzaferri, 1991), which involves assessment by nursing staff, may be adapted for that purpose. 
• To assess residents’ knowledge of professionalism and to stimulate instructional discussion, the Barry’s Challenges to Professionalism Questionnaire may be used (Barry et al., 2000). This self-administered questionnaire consists of vignettes that address conflict of interest, gifts and physician impairment. Research results indicated that the questionnaire was able to detect a broad range of knowledge about professionalism and discriminate among respondents depending on years of experience.

Practicing physicians. 

• 환자에 의한 평가가 많이 사용되며 사용하기 용이하다. 
• Assessments of ethics knowledge by surveys and of comprehensive professionalism by patient questionnaires were the most frequently used, and are probably the most feasible, approaches to assessing professionalism in practicing physicians. 
• For instance, the Defining Issues Test, a written test in which ethical dilemmas are presented in vignettes, is capable of discriminating different levels of moral reasoning among practicing physicians (Baldwin & Bunch, 2000). The Barry’s Challenges to Professionalism Questionnaire would also be suitable for use with practicing physicians.

• 예시들
• Examples of patient questionnaires designed to assess practicing physician professionalism include the Humanism Scale (Hauck et al., 1990) and the Wake Forest Physician Trust Scale (Hall et al., 2002). Both have yielded valid and reliable data. Patient questionnaires have advantages: they obtain information about physicians’ behaviors in actual practice, they seem to be acceptable to physicians, and they may prompt changes in behaviors assessed. As part of the American Board of Internal Medicine Continuous Professional Development Program, patients assessed physicians’ professionalism and communication skills (Lipner et al., 2002). 

---

Med Teach. 2004 Jun;26(4):366-73.

Assessing professionalism: a review of the literature.

Lynch DC1, Surdyk PM, Eiser AR.

Author information

Abstract

Although assessing professionalism poses many challenges, gauging and detecting changes in professionalism is impossible without measurement. This paper is a review of techniques used to assess professionalism during the past 20 years. The authors searched five electronic databases and reference lists from 1982 to 2002. Eighty-eight assessments were retained and organized into content area addressed (i.e. ethics, personal characteristics, comprehensive professionalism, diversity) and type of outcome examined (i.e. affective, cognitive, behavioral, environmental). Instead of creating new professionalism assessments, existing assessments should be improved. Also, more studies on the predictive validity of assessments and their use as part of formative evaluation systems are recommended. Based on the review, suggestions are presented for assessing medical students, resident physicians and practicing physicians.

PMID:

 

15203852

 

[PubMed - indexed for MEDLINE]