TTT(TOT)코스의 단기 효과와 장기 효과(Med Educ, 2008)

TTT(TOT)코스의 단기 효과와 장기 효과(Med Educ, 2008)

A controlled study of the short- and long-term effects of a Train the Trainers course

Sune Rubak,1,2 Lene Mortensen,3 Charlotte Ringsted4 & Bente Malling5

도입

INTRODUCTION

강사 교육 과정 (TTC)은 의사를 clinical teacher로서 준비시키는 데 널리 사용됩니다. 이러한 코스는 커크 패트릭 (Kirkpatrick)이 정의한 것과 비슷한 반응 수준 측면에서 매우 만족스러운 것으로 평가되었습니다.

Training for Trainers  courses (TTCs) are widely used to prepare doctors for their role as clinical teachers. Such courses have been rated as highly satisfactory in terms of levels of reaction similar to those defined by Kirkpatrick.1

몇몇 연구는 대학원 의학 교육에서의 임상 교사들을위한 교수 개발 이니셔티브에 대한 지속적인 필요성을 제안했으며, supervisor 중 75 %는 임상 감독의 훈련에 대한 필요성을 인식한다고 보고되었다 .2,4 교수진 개발 평가 코스는 대개 반응의 수준에 국한되며 5,6,10,11,13이며 학습 수준에 대한 객관적인 평가는 거의 포함되지 않습니다 .14,15
Several studies have suggested a sustained need for faculty development initiatives for clinical teachers in postgraduate medical education,2–12 and as many as 75% of supervisors have been reported to perceive a need for training in clinical supervision.2,4 Evaluation of faculty development courses is usually confined to the level of reaction5,6,10,11,13 and rarely includes objective assessment of the level of learning.14,15

Kirkpatrick은 교수 활동의 효과가 4 단계로 측정 될 수 있다고 제안했다 : 반응; 배우기; 행동 및 조직 1  Postgraduate 임상 상황에서 학습 환경을 측정하는 것은 조직 수준에서 TTC의 영향을 나타내는 척도로 사용될 수 있습니다. 학습 환경climate는 기술하고 측정하기에는 insubstantial 하지만 연수생의 임상 학습 결과에 영향을 미치는 것으로 여겨진다 .21) 교육 환경을 조사 할 때 교육 환경 내 모든 참가자를 모니터링하는 것이 매우 중요하다 - 모든면을 드러내는 것. Genn은 임상학과에서 학습 목표를 성취하는 것과 대조되는 학습 환경의 다른 구성 요소로 구성된 climate index의 사용에 대해 논의했다 .21)그것은 학습 환경의 개선이 학습 목표의 달성에 향상을 가져 왔는지를 평가하는 데 사용되었다 .21

Kirkpatrick suggested that the effects of a teaching activity can be measured at four levels: reaction; learning; behaviour, and organisation.1 Measuring learning climate in the postgraduate clinical context could be used as an indicator of the effect of TTCs at organisational level. Although learning climate is an insubstantial phenomenon to describe and to mea- sure,20–23 it is assumed to influence the clinical learning outcome of trainees.21,24 When investigating educational climate, it is extremely important to monitor all participants within an educational envi- ronment in order to reveal all aspects. Genn dis- cussed the use of a climate index consisting of the different components of the learning climate viewed against the achievement of learning objectives in a clinical department.21 This was used to evaluate whether improvements in the learning climate led to higher achievement of learning objectives.21

METHODS

연구 설계

Study design

We performed an intervention study with pre-post and long-term (6 months) measurements using a matched control group (C-group). The intervention group (I-group) comprised 118 doctors from four departments of internal medicine (rheumatology, endocrinology, pulmonary diseases and gastroenter- ology) and the department of orthopaedic surgery at one university hospital.

인터벤션

The intervention

The intervention comprised a 3-day residential TTC. The course was mandatory for all doctors in the I-group. The participants were randomly distributed into four separate courses conducted within a 3-month period.

필기 시험

Written test

The written test consisted of six essay questions on knowledge of teaching skills developed according to previously published principles.26 The number of questions was restricted to six for feasibility reasons. The aim was to cover the main themes of the course curriculum (Table 2). Based on the course curricu- lum and literature on good clinical teaching skills, a scoring key was developed for each question. The test and scoring keys were pilot-tested using participants of similar courses.

설문지

Questionnaire

The questionnaire regarding teaching behaviour and learning climate was constructed on the basis of literature recommendations and the TTC curricu- lum.2,3,12 A questionnaire was designed in order to evaluate both the themes of the TTC curriculum and specific themes relating to previous standard ques- tionnaires:  

• structuring a clinical training session ;  

• feedback ;  

• supervision ;  

• career counselling , and  

• learning climate .

The questionnaire (in Danish) was pilot-tested by doctors attending a similar TTC. The validation process ensured that questions were understandable and covered the themes, that the questions did not overlap one another and that answering categories were sufficient to discriminate between different answers. The questionnaire contained questions with fixed answering categories, using a 9-point Likert scale where the maximum score was 9.

분석, 통계

Analysis and statistics

Two educators, with master degrees in health pro- fessional education, scored all written test results individually. Subsequently, scores were compared and, in cases of disagreements, the scoring reviewed by both educators and a consensus score decided. Test results for the various measurement time-points were scored randomly and the scorers were blinded to participants study groups.

The percentage of the maximum score was calculated for each question and a total mean score for all questions was calculated for each participant. The maximum score for the test was 58 points, corre- sponding to a 100%mean score. Within-group comparisons were performed using paired-sample t-tests. Between-group comparisons were performed at baseline and 6 months using one-way ANOVA. The relationships betweenanindividual s test performance at various measurement points was estimated using Pearson s correlation coefficient. The total score on the written test was considered the primary effect parameter. Effect size (ES) between the I-group and the C-group was estimated by calculating Cohen s D from the mean and standard deviation (SD) of differences between results at baseline and after 6 months. An ES = 0.5 was considered moderate and an ES > 1.0 was considered substantial.

결과

RESULTS

연구 대상 특성

Study sample characteristics

There was no statistically significant difference between the I- and C-groups regarding distribution in seniority and specialty (Table 1).

 

 

지식과 교육 스킬 향상

Gains in knowledge of teaching skills

Subscores on each item are shown in Table 2. Although scores in the I-group appeared higher thanthose in the C-group at baseline, the large variance inthe data lead to no statistical significance at baseline,except for item 6 ( appraisal meeting ). The item  giving feedback  received the lowest scores at all measurement points in both the I- and C-groups. 

 

 

교육행동에 대한 TTC의 영향

Effect of the TTC on teaching behaviour

Table 3 shows frequencies of receiving and giving feedback and supervision at baseline and at 6 months after the TTC. Data were stratified according to seniority (specialist or trainee).

교육 분위기에 대한 TTC의 영향

Effects of the TTC on learning climate

Table 4 shows changes in the learning climates of the clinical departments.

 

Table 5 presents data concerning only trainees  rat- ings of the learning climate.

 

 

DISCUSSION

주 결과

Main findings

At 6 months, the learning climates in the various departments had changed, as indicated by:

1 실제 실수 또는 거의 놓친 임상 상황으로부터의 학습 증가.

2 교육 배경에 대한 의사 간의 개인 지식 교환의 증가;

3 학습자의 학습요구를 충족시켜주는 데 사용되는 자원의 증가.

4 Relevance of teaching sessions이 C 그룹 구성원이 운영하는 동등한 세션과 비교하여 개선되었습니다.

1 an increase in learning from actual mistakes or near-miss clinical situations; 

2 an increase in personal knowledge exchange between the doctors on teaching backgrounds; 

3 an increase in resources used to fulfil trainees  learning needs in daily practice, and

4 improvements in the relevance of teaching sessions in the department compared with equivalent sessions run by C-group members.

This study is one of very few to study the long-term effect of TTCs on changes in teaching behaviour and learning climate.14

연구의 강점과 약점

Strengths and limitations of the study

The main strengths of the study are:

1 the large number of participants; 

2 the use of a matched control group; 

3 the fact that > 98.4% of the doctors in the I-group participated in the TTC, and 

4 the high response rates achieved at all measure- ment points in both study groups.

세부 결과

Detailed findings

우리는 TTC에서 얻은 지식이 6 개월 후에도 남아 있다는 것을 발견했습니다. 이것은 이전에보고되지 않았습니다. 이것은 specialist와  trainees를 포함한 모든 의사가 과정에 참석했기 때문에 설명 될 수 있습니다. 이는 임상 교육 기술에 대한 공통어를 육성함으로써 학습 강화에 기여했을 수 있습니다. 또한, 일상 업무에서 혁신을 구현하기 위한 critical mass을 달성하는 문제를 해결했습니다.

We found that knowledge gained fromthe TTC was retained 6 months later, which has not been reported before.6 This may be explained by the fact that all doctors, including both specialists and trainees, attended the course. This may have contributed to the reinforcement of learning through fostering a com- mon language about clinical teaching skills. Moreover, it resolved the issue of achieving a critical mass for the implementation of innovation in daily practice.29

Specialist와 Trainee의 결과를 비교하면 각 그룹별로 학습 곡선이 다르게 나타나는데, 기준 수준이 현저히 낮더라도 Trainee이 6 개월까지 급격한 개선을 보였고 전문가 (표 3 및 표 4)를 따라 잡습니다. 이러한 결과는 TTC가 교육 역량을 향상시키고 연공 서열에 관계없이 학과에서 교육 행동 및 학습 분위기에 영향을 미친다는 것을 의미합니다.

A comparison of results between specialists and trainees shows different learning curves for each group, where trainees, despite significantly lower baseline levels, showed precipitous improvement to 6 months, catching up with the specialists (Tables 3 and 4). These results imply that the TTC improves teaching competency and influences teaching behaviour and learning climate in a department regardless of seniority.

함의

Implications for future courses and research

이 연구에서 우리는 6 개월 후에도 유지 된 교수 기술에 대한 TTC의 실질적인 효과를 입증했습니다. 또한, 우리는 교습 행동이 30 % 향상되어 예상보다 많은 것으로 나타났습니다. 10,16,18,25
In this study we have demonstrated the substantial effects of a TTC on knowledge of teaching skills retained after 6 months. Furthermore, we found that teaching behaviour improved by 30%, which was more than expected.10,16,18,25

우리는 6 개월에 지속되는 효과는 모든 department의 의사가 참여하여 혁신에 대한 확산에 필요한 임계 질량critical mass 을 형성한 것과 관련이있을 수 있다고 믿습니다 .29
We believe that the sustained effect at 6 months may be related to the fact that all doctors in the department participated, thus representing the critical mass required for the diffusion on innovation.29

CONCLUSIONS

23 Boor K, Scheele F, van der Vleuten CP, Scherpbier AJ, Teunissen PW, Sijtsma K. Psychometric properties of an instrument to measure the clinical learning envi- ronment. Med Educ 2007;41 (1):92–9.

 

 

 

 

 

---

Med Educ. 2008 Jul;42(7):693-702. doi: 10.1111/j.1365-2923.2008.03044.x. Epub 2008 May 23.

A controlled study of the short- and long-term effects of a Train the Trainers course.

Rubak S1, Mortensen L, Ringsted C, Malling B.

Author information

• 1Department of Paediatrics, Skejby Hospital; Department of Paediatrics, Randers Regional Hospital, Aarhus, Denmark. sr@alm.au.dk

Abstract

Objectives This study aimed to establish the longterm effects of a 3-day 'Training for Trainers' course (TTC) on doctors' knowledge, teaching behaviour and clinical learning climate. Methods The study was designed as an intervention study with pre-, post- and long-term measurements. The intervention group (I-group) included 118 doctors from the departments of internal medicine and orthopaedic surgery at one university hospital. The control group (C-group) consisted of 125 doctors from the corresponding departments at another university hospital. Gains in knowledge about teaching skills were assessed by a written test. Teaching behaviour and learning climate were evaluated by questionnaires. Results In the I-group, 98.4% of doctors, both specialists and trainees, participated in a TTC. Response rates on the written test varied from 90% at baseline to 70% at 6 months after the intervention. Knowledge about teaching skills increased in the I-group by 25% after the TTC and was sustained at 6 months. Questionnaire response rates varied from 98.4% at baseline to 84.8% at 6 months. Post-course, the teaching behaviour of the I-group significantly changed and its learning climate improved compared with the C-group. Scores for use of feedback and supervision in the I-group increased from 4-5 to 6-7 (maximum score = 9). This was significantly higher than in the C-group. Conclusions A 3-day residential TTC has a significant impact in terms of gains of knowledge concerning teaching skills, teaching behaviour and learning climate after 6 months. The positive effects demonstrated in this study were rooted in both the specialists and trainees who attended the course.

PMID:

 

18507769

 

DOI:

 

10.1111/j.1365-2923.2008.03044.x

[PubMed - indexed for MEDLINE]