정형외과학의 PBL 교육에서의 Blended learning(e-learning)의 활용

Blended learning approach improves teaching in a problem-based learning environment in orthopedics - a pilot study

David A Back1,2,7*, Nicole Haberstroh3, Andrea Antolic4, Kai Sostmann5, Gerhard Schmidmaier6 and Eike Hoff3,7

Results

Participants (n = 53) rated their experiences very positively. An enhancement in knowledge was found directly after the course and at the final written test for both groups (p < 0.001). NESTOR users scored higher than non-users in the post-tests, while the OSCE revealed no differences between the groups.

Conclusions

This pilot study showed a positive effect of the blended learning approach on knowledge enhancement and satisfaction of participating students. However, it will be an aim for the future to further explore the chances of this approach and internet-based technologies for possibilities to improve also practical examination skills.

Background

최근 e-learning에 대한 관심과 연구 노력이 많아지고 있다.

In recent years, there has been a growing interest in research on the education and learning progress of students in medicine. Great effort has been put into knowledge transfer via internet-based electronic learning (e-learning) [1].

의학교육에서 e-learning은 중요한 부분이 되었으며, e-learning과 면대면 교육과정을 혼합한 blended learning의 장점에 대해서도 많은 보고가 있다. 

E-learning has also become an integral part of medical education [2-10]. Various authors have shown that greatest benefit and student satisfaction is achieved when combining e-learning with face-to-face courses, as blended learning[2-4]. Blended learning comprises the systematic integration of online and face-to-face engagement to support and enhance a meaningful interaction between students, teachers and resources [11,12]. 

지식의 전달을 촉진하기 위해서 교육은 competent, appealing, recipient oriented 되어야 한다. 이러한 맥락에서 e-learning은 여러 방법을 통해 이를 달성할 수 있다.

When attempting to successfully facilitate the transfer of knowledge, it is essential that teaching be competent, appealing, and recipient oriented[5]. In this context, e-learning can be achieved e.g. 

by providing videos [6], 
podcasts [7] or
interactive diagnostic tools [8], 

...leading to a considerable improvement of knowledge transfer capabilities in a mix with face-to-face lessons [9,10]. However, more studies are still needed to proof the impact of e-learning and blended learning on the enhancement of students’ knowledge and clinical skills in general and in the field of orthopedics and traumatology particularly.

e-learning을 도입한 이후에 많은 연구들이 학생의 만족도를 조사하는데 초점을 두고 있다. 이는 e-learning을 받아들이고 활용하는데 있어서 중요한 요인이다. 또한 지식 및 술기의 습득에 미치는 영향을 분석한 연구도 있다.

After the launch of e-learning courses in recent years, many studies have focused primarily on evaluating students’ satisfaction – an important factor in acceptance and use of e-learning [13-15]. Others have additionally analyzed e-learning’s influence on the acquisition of knowledge or skills[3,6,8,9,12,16].

그러나 이와 같은 밝혀진 효과에도 불구하고 외상학, 정형외과학 교육에서 이러한 연구는 매우 적다. 근골결계 질환이 지난 수년간 급격하게 증가하고 있고, 학생 교육에서 이 질환의 진단과 치료가 좀 더 강조되어야 한다. E-learning은 다양한 멀티미디어의 활용으로 이 분야를 더 매력적으로 만들고 지식 향상을 더 도울 수 있으나, 여전히 많은 데이터가 필요하다.  

In traumatology and orthopedics education such studies are comparatively still rare [13,17,18], despite the fact that e-learning might substantially improve quality and success of teaching in these highly clinically and practically oriented disciplines. While the number of musculoskeletal diseases and injuries has been steadily increasing over the last years [19], data indicate that medical student education concerning the diagnosis and treatment of musculoskeletal diseases should be enhanced [20]. Here, e-learning could add appeal [21] and promote better knowledge and clinical skills [18] by providing useful multimedia adjuncts (e.g. videos, podcasts, or radiological cases). However, more data is still needed to guide the design of blended learning curricula in these subjects, questioning especially to what extend the use of e-learning might be beneficial and to investigate the effect of different approaches or configurations of e-learning.

이러한 문제를 풀기 위해 외상학/정형외과학의 파일럿 연구를 시도하였고, 연구진이 선택한 교육과정은 학제간 지식에 초점을 둔, PBL을 주로 활용하는 교육과정을 연구 대상으로 하였다. 다음의 질문에 대해 연구하고자 한다.

To further address these issues, we performed a pilot study in the teaching of students in orthopedics and traumatology. The chosen curriculum focused on strengthening interdisciplinary knowledge and heavily utilized problem based learning (PBL) with a student-centered teaching approach, encouraging problem-oriented, self-directed and self-organized learning. To evaluate different aspects of incorporating a supplementary e-learning component, following questions were asked in this study:

1. Will students appreciate an additional e-learning offer in a blended learning context?

2. Will user of the e-learning offer show a superior improvement in theoretical knowledge compared to non-users?

3. Will users perform better in clinical examination skills compared to non-users?

The findings should allow a more informed discussion about the aspects that may have to be considered when integrating blended learning approaches into a PBL curriculum of orthopedics and traumatology.

Methods

Setting

All students taking part in the pilot study were in their fifth year of medical studies in winter semester 2009/ 2010. In their curriculum, for the first five of six years (two semesters per year) teaching was organized into block courses, covering different topics along a longitudinal learning helix. Following basic orthopedic concepts in the first semester, a two week block course, “Upper Extremities and Spine”, was specifically dedicated to increase knowledge of traumatologic and orthopedic diseases and furthering clinical skills in the ninth semester. Participation in the latter course was mandatory for all students and this course was chosen to incorporate a new e-learning module called NESTOR (network for students in traumatology and orthopedics), provided through the learning management system (LMS) Blackboard (Blackboard Inc., Washington D.C., USA) with multiple features:

–Orthopedic examination videos (covering inspection, palpation, motion and special tests) (Figure 1).

Figure 1. Examination video. Legend: Example of an examination situation of the shoulder (here: Special tests – Apprehension-Test) as shown in the videos on NESTOR.

–Interactive radiology cases with X-rays, MRI- or CT-images and a patient history. After being asked to generate and enter a hypothesis for the diagnosis, the correct answer was given along with explanations for the ensuing treatment.

–Audiovisual podcasts for common traumatologic or orthopedic diseases (with medical history, diagnostics, therapy, and prognosis).

–Multiple-choice questions were available all the time to enable the students to self-test their gain in knowledge.

Design of the study

Prior to the beginning of the course, students were informed where to find and how to access NESTOR on the LMS Blackboard and about its contents. Clinical tutors were provided with similar information. All students of the semester who were as well participants in the course were asked to take part in the study. The option of enrolling in NESTOR was voluntary. All participants had continuous access to NESTOR during the whole semester. The e-learning module contained no information not otherwise taught (e.g. in classes or regular study books). A tracking function to detect the individual accessed e-learning tools or the time students spend with them was not included in this pilot study.

All students were asked to complete a 20-item multiple-choice test before (pre-test), directly after the block course (post-test 1), and then three months later at the end of the semester (post-test 2). For every item one correct and four wrong answers were given. Tests were created by four independent clinical specialists without knowledge about the content of NESTOR. All tests were anonymized using code names. Students were asked to tick a box if they had used NESTOR for learning and preparation during the semester. Those who did were regarded as “user”, those who did not as “non-user”. At the end of semester students had to pass a mandatory objective structured clinical examination (OSCE) with taking patients history, performing a physical examination and diagnosing actor-patients. The results of the OSCE were taken as evaluation of practical examination skills. Additionally, students’ opinion about the course was evaluated anonymously. As in the written tests (see above) students were asked to tick a box, if they had used NESTOR during the semester and to continue with different questionnaires for NESTOR users and non-users (5-points Likert scale or self-response short answers):

1. Users were asked for (1) their use of NESTOR during the study and also their prior use of the LMS Blackboard to get an idea of the experiences with electronic media and e-learning in general, (2) efficiency of use and structure of NESTOR, (3) satisfaction with its contents and technique, (4) general information concerning the course, and (5) personal information (20 items – 17 Likert scale, 3 short answers).

2. Non-users were asked for (1) their use of the LMS Blackboard, (2) general information concerning the course, (3) reasons for not having used NESTOR and general acceptance of e-learning, (4) personal information (14 items – 12 Likert scale, 2 short answers).

Results of Likert-scaled questions were tabulated and free text answers were reviewed for recurring topics by two reviewers independently.

Written informed consent was obtained from all participants including the allowance to use test and evaluation results as anonymous data for the study (regarding students) and to use a picture of the video for publication (regarding the “actors” of the video, both medical doctors). Additionally, permission was obtained from the responsible educational Ethikkommission der Charité - Universitätsmedizin Berlin.

Statistical analysis

Written tests were validated by calculating Cronbach’s Alpha. Data of the written tests were analyzed for changes between pre- and post-tests as well as post-test 1 and 2 within the groups of users and non-users using unpaired student’s t-test. To detect differences in the evaluation between NESTOR users and non-users a chi-square test was performed for each question. A p-value less than 0.05 was considered to indicate a significant (< 0.01: highly significant) difference between the observations and the expectations based on the null-hypothesis. Statistical analysis was performed with SPSS® 17.0 statistics software (SPSS Inc., Chicago, IL, USA) and GraphPad Prism®5 (GraphPad Software Inc., San Diego, Ca, USA).

Discussion

정형외과학과 외상학 분야에서 blended learning의 효과를 본 첫 번째 연구이다.

The purpose of this pilot study was to give first impressions of the effect of a blended learning concept in orthopedics and traumatology called NESTOR both on students’ satisfaction and on its contribution to acquisition of knowledge and clinical skills in a problem-based learning curriculum, which already provides an intensely practice-oriented teaching environment. To the best of the author’s knowledge, this is the first study examining the influence of blended learning not only on satisfaction, but also on knowledge and practical clinical skills of students in traumatology and orthopedics, two highly practically oriented medical subjects.

학생들의 의견과 수용정도를 보는 것은 e-learning을 평가하는 첫 단계이다. 기존 연구들과 마찬가지로 high approval을 확인할 수 있었다.

Evaluations of students’ opinion and acceptance can be seen as first step when establishing a new e-learning program [4,14,21]. Consistent with the literature, this study revealed a high approval of the participating users for the additionally offered e-learning contents. While a broad acceptance is crucial for successful e-learning implementation [1], it is also important to evaluate its influence on students’ gain of knowledge and skills [9].

지식 향상에 대한 평가를 수행하였고, 필기시험을 활용하였다. e-learning이 지식 향상에 도움이 된다고 보고하는 연구도 있고, 그렇지 않은 연구도 있다. 이 연구에서 우리는 pre- 와 post- 간에 유의미한 차이를 확인하였다. NESTOR 사용자 그룹이 더 높은 점수를 받았을 뿐만 아니라 향상도도 더 높았다.  그러나 이러한 결과를 NESTOR 덕분이라고만 해석하는 것은 조심해야 한다. 최근 연구에서 Rpwe 등은 clinical competency의 향상에 있어 blended learning의 효과를 평가하는 것은 rudimentary하며 71개의 연구 중 64개가 방법론적으로 문제가 있었다.

Thus, as second step, not only the impact on users’ satisfaction, but also on their knowledge should be demonstrated [6,21,22]. For this pilot study we have chosen newly developed written tests to evaluate improvement in theoretical knowledge, which seem to be valid as indicated by the measured Cronbach’s Alpha values. While some studies showed benefits of e-learning on improvement of students’ knowledge [6,23] others did not – despite of positive evaluation[2,8,22]. In this pilot study, we found a significant improvement from pre- to post-tests for both groups. NESTOR users scored better in the written post-tests than non-users and showed further improvement between post-test 1 and 2. A possible interpretation for this success in the group of NESTOR users may be students’ very positive attitude towards e-learning and a high satisfaction with structure and contents. However, these results should be interpreted carefully, especially when referring this effect exclusively to the use of NESTOR. In a recent review, Rowe et al. [12] showed that the existing data to evaluate an improvement of clinical competencies by blended learning can still be regarded as rudimentary. It seemed to be a problem of the study design in a clinical environment to determine the effect of blended learning exclusively. Rowe et al. identified 71 studies dealing with the role of blended learning in the clinical education of healthcare students, but only 7 articles were enrolled for the review due to methodological flaws of the remaining 64.

지식 향상에 대한 평가의 다음 단계로는 clinical skill의 향상을 보는 것이 될 것이며, 일부 연구에서 수행한 바 있다. OSCE결과 사용 여부에 따른 차이는 없었다. 이는 기존의 다른 연구와 유사한 결과이다. 이에 대한 한 가지 설명은 이미 존재하는 highly clinically oriented curriculum이 추가적인 e-learning 노출에 대한 효과가 나타나기 어렵게 만들었다는 것이다.

As potential third step it might be anticipated that even practical clinical skills may be improved through blended learning in this context [9], which has been shown in some studies [9,16,24]. The results of the OSCE revealed no differences between users and non-users in this pilot study with high scores in both groups. These findings are consistent with other studies which failed to detect significant benefits on examination performance or other practical clinical skills with e-learning implementation [21,25,26]. A possible explanation could be that the pre-existing, highly clinically oriented curriculum made it difficult for any additional e-learning exposure to further improve skills.

이러한 맥락에서 e-learning을 통해 달성할 수 있는 것이 무엇인지에 대한 의문이 있을 수 있다. 임상skill은 개인적 경험과 훈련에 의한 것이지 e-learning을 사용한 여부에 달린 것이 아니라는 주장이 가능하다. 따라서 이러한 상황에서 blended learning 접근법을 적절히 조절하여 적용을 가능하도록 하는 방법이 무엇일지 연구해볼 필요가 있다. 그러나 적어도 지식의 향상 측면과 전반적으로 높은 approval을 보면 NESTOR를 계속 지속시킬 만한 주장의 근거는 된다고 할 수 있다. 또한 해부학과 같은 다른 과목의 e-learning과 연결시키는 것도 좋을 것이다.

In this context, the question may arise what e-learning potentially can achieve [1]. It can be argued that clinical examination skills will always be preferentially based on personal experiences and training rather than on the use of e-learning – unlike acquiring skills in other areas such as radiological diagnosis [16]. In the presented pilot study, e-learning enhanced competencies for gaining theoretical medical knowledge. Further research will be necessary to determine, if it is possible to adjust the components of a blended learning approach in this context to achieve also an improvement of practical skills compared to mere face-to-face teaching. However, as knowledge about diseases is an important basis for developing treatment and examination skills, this and the overall high approval provide good arguments for the continued use of NESTOR in the preparation for the tested subjects. Following suggestions of the non-users, acceptance of the program might be further increased by improving announcements about it. Additionally, it could be made even more appealing with links to e-learning programs of other subjects (e.g. anatomy) of the faculty.

e-learning에 대한 자발적 사용 의사와 관련해서 살펴보면, NESTOR의 사용 여부가 LMS Blackboard의 사용 여부와 매우 연관되어 있었기 때문에 LMS를 도입함으로서 e-learning의 자발적 활용 가능성을 높일 수 있다. 전체 교수들이 e-learning을 활용하면 학생들의 e-learning에 대한 친밀도를 높일 수 있을 것이며 자발적인 것 뿐만 아니라 심지어는 의무적으로 e-learning을 활용하게 하는 것 역시 가능할 것이다.

Concerning the willingness to use e-learning offerings voluntarily, an additional inference can be taken from this study en passent: As the use of NESTOR was significantly linked to the use of LMS Blackboard, the likelihood of voluntarily using an e-learning offering may be directly connected to the acceptance and use of the hosting LMS. This would require the need for the entire faculty to join in a combined effort to establish e-learning offerings broadly to increase students’ familiarity with such resources. Thereby, voluntary and perhaps even mandatory use of e-learning components could be increased.

---

BMC Med Educ. 2014 Jan 27;14:17. doi: 10.1186/1472-6920-14-17.

Blended learning approach improves teaching in a problem-based learning environment in orthopedics - a pilotstudy.

Back DA, Haberstroh N, Antolic A, Sostmann K, Schmidmaier G, Hoff E.

Abstract

BACKGROUND:

While e-learning is enjoying increasing popularity as adjunct in modern teaching, studies on this topic should shift from mere evaluation of students' satisfaction towards assessing its benefits on enhancement of knowledge and skills. This pilot study aimed to detect theteaching effects of a blended learning program on students of orthopedics and traumatology in the context of a problem-based learning environment.

METHODS:

The project NESTOR (network for students in traumatology and orthopedics) was offered to students in a problem-based learning course. Participants completed written tests before and directly after the course, followed by a final written test and an objective structured clinical examination (OSCE) as well as an evaluation questionnaire at the end of the semester. Results were compared within the group of NESTOR users and non-users and between these two groups.

RESULTS:

Participants (n = 53) rated their experiences very positively. An enhancement in knowledge was found directly after the course and at the final written test for both groups (p < 0.001). NESTOR users scored higher than non-users in the post-tests, while the OSCE revealed no differences between the groups.

CONCLUSIONS:

This pilot study showed a positive effect of the blended learning approach on knowledge enhancement and satisfaction of participating students. However, it will be an aim for the future to further explore the chances of this approach and internet-based technologies for possibilities to improve also practical examination skills.

PMID:

 

24690365

 

[PubMed - in process] 

PMCID:

 

PMC3905287

 

Free PMC Article