의사소통역량 평가하기 : 기존 도구들에 대한 평가

의사소통역량 평가하기 : 기존 도구들에 대한 평가

Assessing Communication Competence: A Review of Current Tools

Julie M. Schirmer, LCSW; Larry Mauksch, MEd; Forrest Lang, MD; M. Kim Marvel, PhD;

Kathy Zoppi, PhD; Ronald M. Epstein, MD; Doug Brock, PhD; Michael Pryzbylski, PhD

효과적인 진료를 위해서 의사는 뛰어난 의사소통가가 되어야 하고, 의사소통능력은 ACGME가 요구하는 여섯 가지 역량 중 하나이다. 여섯 가지 역량 중 네 가지에서 좋은 의사소통의 요소들이 포함되어 있다. AAMC는 MSOP에서 의사소통에 대한 권고를 하였다. NBME는 OSCE를 통해서 인터뷰 및 의사소통 기술을 평가하고 있다. IOM은 2004년의 보고서(“Improving Medical Education: Enhancing the Behavioral and Social Science Content of Medical School Curricula,”)에서  의사소통 기술을 여섯 개의 핵심 교육과정영역 중 하나로 지칭하고 있다. 뛰어난 의사소통에 대한 중요성은 역량을 결정하기 위한 도구의 개발로 이어졌다.

Physicians must be competent communicators to effectively practice medicine, and communication is one of six required competencies identified by the Accreditation Council on Graduate Medical Education (ACGME).1 Elements of competent communication are featured in four of the six ACGME competencies. The Association of American Medical Colleges (AAMC) also published recommendations for communication in the Medical School Objective Project, Paper III.2 The National Board of Medical Examiners (NBME) is requiring Objective Standardized Clinical Examinations (OSCEs) to assess interviewing and communication skills. The Institute of Medicine, in its 2004 report, “Improving Medical Education: Enhancing the Behavioral and Social Science Content of Medical School Curricula,” names communication skills as one of six curricular domains.3 The strong agreement about the importance of competent communication in medical practice challenges medical educators to develop effective tools to determine competence. 

의사소통역량을 평가하는 것은 복잡하다. 수행능력을 필요로 하는 기술적인 측면에 대한 평가는 서술형 평가와 같은 방법으로는 평가가 어렵고 in-vivo demonstration을 통해서 가능하다. 또한 역량은 특정한 행동을 하느냐 마느냐가 아니고 효과적으로 언어적, 비언어적 행동을 적절한 시기에, 환자/환자가족과의 상호작용의 맥락 안에서 할 수 있어야 한다. 효과적인 의사소통은 기술을 적절히 적용하고, 상대방에 적절히 반응하고, 대화 중에 스스로의 행동을 인식하는 것이 모두 필요하다. 추가적으로 효과적인 의사소통은 관찰가능한 행동에만 의존하는 것이 아니라 환자의 행동, 환자의 인식에 따라서도 달라진다. 한 상황에서 효과적인 의사소통이라도 다른 산황에서는 그렇지 않을 수 있다. 환자간의 다양성과 효과적인 의사소통의 미묘한 지점들이 표준화된 평가를 어렵게 만드는 주요 요인이다.

Assessing communication competence is complex. Skills that require performance are difficult to assess through disembodied means (such as written tests) but require in-vivo demonstration.4 Further, competence is not defined solely by the presence or absence of specific behaviors but rather by the presence and timing of effective verbal and nonverbal behaviors within the context of individual interactions with patients or families. 5 Effective communication includes the ability to adapt, to be responsive, and to manage self-awareness during the process of talking and listening. Additionally, effective communication is not only dependent on the observable behaviors of the physician but also on the behaviors and perceptions of patients. What constitutes effective communication in one setting or with one patient may be ineffective in another.6 The variation among patients and the subtleties of effective communication make standardized evaluation difficult.

점차 의사소통은 피훈련자의 승진, 졸업, 면허를 위한 평가도구로서 사용되고 있다. 이러한 고위험상황(high-stakes)의 평가는 높은 수준의 신뢰도, 타당도, 커트라인을 필요로 하며, 이를 기준으로 미달한 사람은 통과하지 못하게 된다. 높은 수준의 신뢰도를 달성하기 위해서는 목표하는 기술이 분명히 규정되어야 하고, 관찰자들은 서로 일치도가 최대한이 될 수 있게 훈련받아야 한다. 환자에 의한 평가는 타당도를 높이는 방법이며 다른 관찰자에 의해 평가되는 차원과 다른 차원에서 중요하다. 그러나 psychometric properties를 강력하게 하기 위해서 관찰의 숫자를 너무 늘리거나 환자에게 평가를 너무 많이 부탁하는 것은 평가 자체에 대한 현실성을 저하시키며 특히 한정된 자원으로 진행되는 레지던트 프로그램에서 더욱 그러하다.

Increasingly, communication is evaluated to determine a trainee’s suitability for promotion, graduation, and licensure. These high-stakes evaluations require assessment instruments with a high degree of reliability, validity, and specified cut-off points, below which trainees do not pass.7 To achieve a high level of reliability, target skills must be clearly defined, and observers may need to be trained to maximize agreement. Ratings by patients increase validity and add an important dimension beyond the ratings of observers. However, the effort to attain strong psychometric properties by expanding the number of observations and soliciting patient ratings may make the assessment process impractical, especially for smaller residency programs with limited resources.

의사소통 기술을 평가하는 것의 복잡성에도 불구하고 신뢰도/타당도를 갖춘 평가도구가 많이 개발되어야 한다. 이상적인 평가도구에 대해서는 합의된 바가 적으나 ACGME에 의해서 제시되는 옵션이나 다른 것들이 일부 기준을 제시해주고 있다. administration, design, focus, psychometric properties, practicality, and ease of use에 대해서 여러가지가 모두 다르다.

Despite the complexities of assessing communication skills, medical educators must develop and implement reliable and valid assessment methods. There is little agreement on ideal assessment tools.7 Options offered by the ACGME and other resources offer few criteria on which an educator can make an informed choice. Checklists and scales vary in their administration, design, focus, psychometric properties, practicality, and ease of use.

본 연구의 목적은 의사의 의사소통의 필수요소를 측정하기 위한 기존의 의사소통 평가도구를 평가하는 것이다. 

Our goal was to conduct a study that evaluated the degree to which available communication assessment instruments measure the essential elements of physician communication. Results from this study will guide family medicine educators to select appropriate tools that assess physician communication competence. This study also may provide guidelines for others developing new instruments and for those refining existing ones.

Methods

Rating Tool

The six reviewers in this study developed a rating form with a 5-point Likert scale, along with a space for comments for each of the evaluation criteria (Table 1). The evaluation criteria included the seven essential elements of physician-patient communication identified in the Kalamazoo Consensus Statement (KCS)8 (Table 2). The KCS was developed by 21 experts from medical schools, residencies, and representatives from medical education organizations in North America.

In addition to the KCS criteria, we added two additional dimensions of interviewing: 

• dealing with more than one family member in the room and 
• interview efficiency. 

The addition of multiple member family interview skills was included because family and friends are frequently present during outpatient and inpatient encounters. Interview efficiency was added because trainees must be able to communicate effectively without losing control of time. Indeed, one result of effective communication may be to enhance time management. 9,10

The rating form also evaluated three instrument characteristics: psychometric properties, practicality/usability, and overall value. 

• Evaluations of psychometric properties reflect the presence and strength of psychometric data. 
• Practicality/usability is a gestalt evaluation reflecting the raters’ judgment about the ease of use when considering who would be using the form, the complexity of form design, and the form length. 
• Overall value was the final rating reflecting a summary or global impression of the entire instrument.

Instruments Rated

Instruments were identified through a review of the literature, personal contacts, and proceedings from national and international conferences attended by the authors. Fifteen instruments were included in the study (Table 3). To be included in the review, assessment instruments 

(1) directly measured observed encounters between physicians (or medical students) and patients (real, simulated, or standardized) and 

(2) were designed for use in educational settings and not just for research. 11-28 

The instruments were placed in three categories, reflecting the intended rater: observers (eg, faculty), standardized patient, or patients.

Rating Methods

Six of the authors rated each of the 15 instruments. Raters did not communicate with one another about their impressions during the rating process. Each of the raters had at least 10 years of experience teaching communication skills, including many hours observing medical students and primary care residents, and had published on the topic of physician-patient communication.

Discussion

이 파일럿연구의 결과 기존의 의사소통평가도구들은 내용/psychometric properties/실용성 등에 있어서 크게 다르다는 것을 확인할 수 있었다. 이 모든 카테고리에서 높은 평가를 받은 도구는 없었다. 

The results of this pilot study indicate that existing communication assessment instruments vary considerably in their content, psychometric properties, and usability. 

• No instrument received high ratings in all of those categories. 
• Instruments designed for faculty observers that received the highest ratings (Kalamazoo, Macy, and MISCE) varied in their ratings for practicality/ usability. 
• Few instruments had strong psychometric properties, assessed family interviewing, or had interview efficiency. 
• Only one of the instruments (Common Ground) that had strong psychometric properties had relatively high ratings on the KCS elements. 
• Few instruments assessed family issues and interview efficiency.

체크리스트를 사용한 경우가 rating scale을 사용한 경우보다 많았다. 경험이 적은 관찰자에게는 체크리스트가 좀 더 행동에 대한 정의가 명확하여 신뢰도가 높아질 수 있다. 그러나 전문가에 대해서는 체크리스트보다는 criteria를 활용한 준거가 더 좋을 수 있다. 따라서 체크리스트는 의사소통기술평가를 배우고 있는 교수들에게 적합할 수 있다. 일부 체크리스트는 경험이 적은 교수들에게 적합할 수 있다. Rating scale을 사용한 도구는 전문가그룹이 잘 형성된 경우에 사용하는 것이 좋을 수 있다.

Many instruments use checklists (the presence/absence of behaviors) rather than rating scales (assigned weight to an interaction). For less-experienced observers, checklists provide clearer behavioral definitions that may improve reliability.29 Conversely, experts do as well or better using ratings that use criteria rather than checklists. 29,30 Therefore, a checklist may be the preferred tool when faculty are learning to assess communication skills. Some checklists (Macy, Kalamazoo, and SEGUE) may be useful for faculty with less experience. Instruments that use rating scales (Common Ground, MISCE, MAAS, and Four Habits) might be used when the medical communication expertise of faculty is well developed.

환자에게 평가를 요구한 도구에서는 KCS에서 언급하고 있는 의사소통기술을 평가하고 있지 않았으며, 이는 교수평가자를 위한 도구를 같이 활용해야 함을 의미한다. 환자의 평가는 만족도, halo, ceiling effect에 의해서 영향을 받을 수 있다. Kalamazoo II report는 환자 대상 설문은 존중, 환자의 비언어적/언어적 큐에 대한 집중, personally present, caring intent, flexibility를 평가하는데 좋다고 제안하고 있다.

This study shows that patient surveys may not assess communication skills identified in the KCS, suggesting that training programs should include instruments for faculty raters. Patients provide global impressions that are strongly influenced by the degree to which their reason for seeking care is satisfied and by the “halo” or “ceiling effect” that inflates with relationship continuity.31-33 The Kalamazoo II Report suggests that patient surveys are best used to assess interpersonal skills such as respect, paying attention to patient’s nonverbal and verbal cues, being personally present, having a caring intent, and flexibility.34

---

Fam Med. 2005 Mar;37(3):184-92.

Assessing communication competence: a review of current tools.

Schirmer JM1, Mauksch L, Lang F, Marvel MK, Zoppi K, Epstein RM, Brock D, Pryzbylski M.

Author information

Abstract

BACKGROUND:

The assessment of communication competence has become a major priority of medical educational, policy, and licensing organizations in the United States and Canada. Multiple tools are available to assess communication competence, but there are few studies that compare the tools.

METHODS:

A consensus panel of six family medicine educators evaluated 15 instruments measuring the physician-patient interview. The primary evaluation criteria came from the Kalamazoo Consensus Statement (KCS), which derived from a multidisciplinary panel of experts that defined seven essential elements of physician-patient communication. We evaluated psychometric properties of the instruments and other assessment criteria felt to be important to family physicians (exploring family issues, interview efficiency, and usability/practicality).

RESULTS:

Instruments that received the highest ratings on KCS elements were designed for faculty raters and varied in their practicality/usability ratings and psychometric properties. Few instruments were rated high on psychometric properties or exploring family issues.

CONCLUSIONS:

The process successfully reviewed and provided a framework for assessing communication skills instruments. There is a need to expand the study, including use of a larger cohort of reviewers to provide more validity to the results and minimize potential biases.

PMID:

 

15739134

 

[PubMed - indexed for MEDLINE] 

Free full text