프로페셔널리즘 평가: 예비적 결과(Acad Med, 2006)
The Professionalism Mini-Evaluation Exercise: A Preliminary Investigation
Richard Cruess, Jodi Herold McIlroy, Sylvia Cruess, Shiphra Ginsburg, and Yvonne Steinert
의과 대학에서 관찰 된 전문적인 행동의 lapse는 이후 의사가 된 이후에 unprofessional conduct으로 이어집니다.
lapses in professional behavior observed in medical school are associated with subsequent unprofessional conduct in practice.9,10
Norcini와 동료 연구자들은 내과의 레지던트를 대상으로 미니 임상 시험 운동 (mini-CEX)을 사용했음을보고했으며, 결과가 다른 성과 기반 평가의 결과와 잘 일치 함을 지적했습니다 .14 형식이 너무 매력적이므로 , 우리는 의대생의 전문적인 행동을 평가하기 위해 mini-CEX를 기반으로 한 도구를 개발하기로 결정하고 P-MEX (Professionalism Mini-Evaluation Exercise)라고 불렀습니다.
Norcini and colleagues reported on the use of the mini-Clinical Examination Exercise (mini-CEX) for residents in internal medicine,11–13 noting that the results correlated well with those of other performance-based assessments.14 Because the format is so attractive, we decided to develop a tool based upon the mini-CEX to evaluate professional behaviors in medical students, and have called it the Professionalism Mini-Evaluation Exercise (P-MEX).
Method
Instrument development
As part of an ongoing faculty development programon teaching and evaluating professionalism,19 a workshop on evaluating the attributes of the physician was attended by 92 McGill faculty members and residents
The authors subsequently distilled them to 24 behaviors to evaluate as many attributes as possible with the smallest number of behaviors.
he selected behaviors were then inserted into the mini-CEX format using a 4point scale where
4 exceeded expectations,
3 met expectations,
2 below expectations, and
1 unacceptable.
There was also a fifth category entitled “not observed” or “not applicable.”
The P-MEX was designed to be used in any situation where a student’s behavior can be observed, including patient encounters, small group sessions, and sign-out rounds. The evaluation is to be based on interactions that are relatively short and that occur frequently as a part of training so that each student can be evaluated on several occasions by different faculty members. Each form has two copies, one of which is given to the student, the other being retained. The evaluator is expected to give timely feedback to the student, thus giving it the potential to be formative and summative.
Data analysis
Exploratory factor analysis was conducted to understand the internal structure of the scale. The analysis was conducted in SPSS using unweighted least squares extraction and varimax rotation. The factor analysis was conducted using form(n 211) as the unit of analysis. Each occasion of measurement (form) should “stand alone,” because other sources of variance (i.e., context/occasion/rater) are all confounded and likely contribute sufficient measurement that each measurement circumstance for a given student could be considered independent of the others.
Additionally, a generalizability analysis32 and decision study was performed to investigate the number of forms (occasions of measurement) required to obtain a dependable estimate of the calculated average score. This analysis was done using the approach described by Norcini and colleagues11,14 in that the persons by forms (occasions) design was used, and the calculated average score (i.e., mean over the 24 items) was the score of interest. Using subjects for whom at least two forms were completed, the variance components were estimated for student, occasion, and the student by occasion interaction (error) using urGENOVA.33 These variance components were then used in a series of decision studies using GENOVA to compute the reproducibility coefficients, and standard errors of measurement were computed for 1 to 14 encounters/ occasions. Confidence intervals were computed around the mean calculated average score to obtain a sense of the precision of measurement in each case.
Results
인구통계
Demographics
In all, 211 P-MEX forms were collected on 74 undergraduate medical students.
The number of forms per student in the 2-rotation study period ranged from 1 to 9, with a mean of 2.85 ( 1.9) and a median of 2.0.
When we eliminate those subjects for whom there was only one form, there are 189 forms on 52 subjects, with a median number of forms per subject of 3.5.
문항 분석
Item analysis
On average, 6 of the 24 items were not completed or were marked “not applicable.”
판단불가
Four items were marked not applicable in 40%or more of the 211 forms:
“accepted inconvenience to meet patient needs,”
“advocated on behalf of a patient and/or family member,”
“admitted errors/omissions,” and
“assisted a colleague as needed.”
Item mean scores ranged from 3.10 to 3.35 out of 4.
기대이하
4 items showed 3% or more of the ratings as “below expectations”:
“demonstrated awareness of limitations,”
“solicited feedback,”
“was on time,” and
“addressed gaps in own knowledge and skills.”
This may indicate that these items are more “sensitive” to breaches of professionalism than others.
불필요
Three items were seen to be redundant:
“showed respect for patient” (correlation with items 1 and 2, r 0.78 and 0.79),
“assisted a colleague as needed” (correlation with items 22, 23 and 24, r 0.87, 0.82, and 0.83, respectively), and
“respected rules and procedures of the system” (correlation with items 14, 21, 22 and 23, r 0.79, 0.83, 0.77, and 0.76, respectively).
척도 분석
Scale analysis
The factor loadings of the 24 items onto the four factors are shown in Table 1.
Using these two estimates, the reproducibility coefficients and standard errors of measurement were computed for 1 to 14 encounters/ occasions (Table 2). Between 10 and 12 encounters are required to obtain a reproducibility coefficient of 0.80, which is consistent with results published by Norcini and colleagues in 1995.11
10개 이하도 괜찮음
Using these confidence intervals as guidelines, educators may feel comfortable using the average of fewer than 10 forms, depending on the intended use of the scores. For example, the confidence interval obtained with an SEMof 0.11 may be sufficiently precise for many criterion-referenced measurement purposes, requiring only eight forms per student to be collected.
질적 피드백
Qualitative feedback
the P-MEX was useful in
자기성찰 promoting self-reflection,
일상의 프로페셔널리즘의 중요성 인식 awareness of the importance of professionalismin daily encounters,
프로페셔널리즘에 부합하는 행동 identifying behaviors consistent with professionalism, and
내용 교육 teaching about this subject matter.
가장 문제는 시간
The major limitations have been time – time to observe; time to record; and time to give feedback.
환자의사 관계
Doctor–patient relationship skills
1 환자에게 적극적으로 듣기
2 사람으로서의 환자에 대한 관심도 나타남.
3 환자에 대한 존중 표명
4 환자의 필요를 인식하고 충족
5 환자의 요구를 충족시키기 위해 불편을 감수
6 환자 치료의 연속성 확보
7 환자 및 / 또는 가족 구성원을 지지
12 환자 / 동료와 적절한 경계 유지
1. Listened actively to patient .493 .668 .372 .175
2. Showed interest in patient as a person .282 .812 .359 .219
3. Showed respect for patient .427 .752 .333 .184
4. Recognized and met patient needs .417 .708 .245 .260
5. Accepted inconvenience to meet patient needs .344 .677 .449 .227
6. Ensured continuity of patient care .285 .750 .290 .436
7. Advocated on behalf of a patient and/or family member .307 .631 .298 .440
12. Maintained appropriate boundaries with patients/colleagues .528 .562 .393 .189
성찰 기술
Reflective skills
8 자신의 한계 인식
9 오류 / 누락을 인정
10 피드백 요청
11 피드백 수용
13 어려운 상황에서 평정 유지
8. Demonstrated awareness of limitations .459 .344 .632 .019
9. Admitted errors/omissions .257 .249 .783 .094
10. Solicited feedback .248 .335 .783 .314
11. Accepted feedback .199 .247 .825 .245
13. Maintained composure in a difficult situation .444 .419 .598 .224
시간 관리
Time management
15 시간 지키기
16 신뢰할 수있는 방식으로 완료된 작업
18 환자나 동료에게 available함
15. Was on time .248 .228 .327 .804
16. Completed tasks in a reliable fashion .321 .447 .063 .632
18. Was available to patients or colleagues .459 .228 .183 .746
전문직간 관계 기술
Interprofessional relationship skills
12 환자 / 동료와 적절한 경계 유지
14 적절한 외관 유지
17 지식과 기술에있어서 자신의 gap을 극복
19 동료를위한 존경심을 나타냈다.
20 경멸적인 언어 사용을 피함
21 필요에 따라 동료 지원
22 환자 기밀 유지
23 적절한 자원의 활용
24 시스템의 규칙 및 절차를 존중
12. Maintained appropriate boundaries with patients/colleagues .528 .562 .393 .189
14. Maintained appropriate appearance .648 .270 .515 .304
17. Addressed own gaps in knowledge and skills .523 .377 .338 .317
19. Demonstrated respect for colleagues .726 .249 .186 .381
20. Avoided derogatory language .777 .357 .227 .248
21. Assisted a colleague as needed .722 .424 .261 .258
22. Maintained patient confidentiality .797 .314 .349 .257
23. Used health resources appropriately .772 .388 .305 .235
24. Respected rules and procedures of the system .709 .340 .365 .336
Discussion
계산 된 평균 점수의 재현성은 mini-CEX의 예비 연구에서보고 된 것과 비교되는 것으로 나타 났는데, 0.80의 신뢰도 계수를 얻기 위해서는 10-12 개의 완성 된 양식이 필요합니다. 그러나 측정의 다양한 목적에 따라서 4 ~ 6개 만으로도 CI가 충분히 작을 수 있습니다.
The reproducibility of the calculated average score was shown to be comparable to that reported in the preliminary study of the mini-CEX, in that between 10 and 12 completed forms are required to achieve a dependability coefficient of 0.80. However, confidence intervals may be sufficiently small at 4 to 6 forms for many measurement purposes.
한 가지 흥미로운 사실은 "기대 이하"로 표시된 네 가지 항목이 성찰적 기술과 밀접한 관련이 있다는 것입니다. 개별 항목이나 항목 그룹이 미래의 어려움을 예측하는 정도는 향후 조사를위한 많은 영역 중 하나입니다.
One interesting finding is that the four items marked “below expectations” closely relate to reflective skills. The extent to which individual items or groups of items are predictive of future difficulties is one of the many areas for future investigation.
Acad Med. 2006 Oct;81(10 Suppl):S74-8.
The Professionalism Mini-evaluation Exercise: a preliminary investigation.
Author information
- 1
- Centre for Medical Education, Lady Meredith House, McGill University, 1110 Pine Ave. W., Montreal, QC, H3A 1A3 Canada. richard.cruess@mcgill.ca
Abstract
BACKGROUND:
METHOD:
RESULTS:
CONCLUSION:
- PMID:
- 17001141
'Articles (Medical Education) > 전문직업성(Professionalism)' 카테고리의 다른 글
| 행동에서 귀인으로: 프로페셔널리즘 평가의 추가적 고려사항(Med Educ, 2009) (0) | 2017.09.27 |
|---|---|
| 의과대학 재학중 프로페셔널하지 못한 행동과 CPI의 관계(Acad Med, 2007) (0) | 2017.09.27 |
| 바람직한 행동목록 그 이상: 프로페셔널리즘의 근원적 이해(Acad Med, 2014) (0) | 2017.09.26 |
| 의과대학생의 명시적 담론에서 확인되는 프로페셔널리즘의 차이: 행동, 표상, 됨(Med Educ, 2011) (0) | 2017.09.26 |
| "나는 의사에요": PIF. 표준화와 다양화 담론 사이에서 협상하기 (Acad Med, 2013) (0) | 2017.09.26 |