거꾸로교실의 학업 성과: 메타분석(Med Educ, 2018)

거꾸로교실의 학업 성과: 메타분석(Med Educ, 2018)

Academic outcomes of flipped classroom learning: a meta-analysis

Kuo-Su Chen,1,2,3 Lynn Monrouxe,2,3 Yi-Hsuan Lu,1,2,3 Chang-Chyi Jenq,2,3,4 Yeu-Jhy Chang,2,3,5

Yu-Che Chang2,3,6 & Pony Yee-Chee Chai7

도입

INTRODUCTION

플립된 교실(FC)은 온라인 학습과 직접 대면하는 교실 활동을 결합한 복합적인 접근방식이다. 이 교육 모델에서, 학생들은 수업 전에 콘텐츠 학습에 참여하여, 능동적인 학습을 위한 강의실 내 시간을 극대화한다. FC는 강의 전 학습에 기술(테크놀로지)을 활용하며, 직접 대면 강의실을 대화형 학습 활동으로 활용한다. 이 방법론은 (교사가 아닌) 학생들을 배움의 중심으로 이동시킴으로써 전통적인 강의 기반 접근 방식을 재구성하고 재정렬한다.2–4 그러한 능동적 학습은 학습자가 강의 전 학습에 대해 연습하고, 참여하고, 적용함에 따라 결과를 개선하게 된다.5 10년 전에 현대적 버전의 FC가 나타났으며, 교육 전반과 의료 교육에서 특히 인기가 있음에도 불구하고 우리는 그 효능에 대해 확고한 결론을 내리지 못하고 있다.

The flipped classroom (FC) is a hybrid approach, combining online learning and face-to-face classroom activities. In this pedagogical model, students engage in content learning before class, thereby maximising in-class time for active learning.1 The FC utilises technology for pre-class learning, with face-to-face classrooms becoming interactive learning activities. This methodology restructures and reorders traditional lecture-based (LB) approaches by moving students, rather than teachers, to the centre of learning.2–4 Such active learning should improve outcomes as learners practise with, engage with and apply their pre-class learning.5 Although modern versions of the FC appeared over 10 years ago,6 and despite its popularity in education generally and medical education specifically, we lack firm conclusions regarding its efficacy.7,8

FC 접근법은 능동적 학습 이론에 의해 뒷받침된다.9 능동적 학습은 '학습 과정에 학생들을 참여시키는 모든 교육적 방법'으로 정의되었다.10 능동적 학습 접근법은 학생들이 의미 있는 학습 활동을 수행하며, 인지적으로 그러한 활동에 참여하는engaging 것이다. 이전의 연구는 학생들의 이해와 성취도가 능동적 학습을 통해 향상된다는 것을 암시한다.10-13 메타 분석은 능동적 학습이 학생들의 성적을 높이고 대학생들의 실패율을 감소시킨다는 것을 확인했다.14 따라서, FC는 능동적 학습의 한 형태로서, 4,15는 더 나은 학습 성과, 특히 고차적 사고에서 더 나은 학습 성과와 관련이 있었다.17,18 실제로, 연구에 따르면 FC 접근법이 동기, 만족도, 성과 및 출석률을 증가시킨다고 한다.4,19,20 그러나 연구 결과는 동일한 성과 측정치를 고려한 경우에조차 일반적으로 일관되지 않는다.

The FC approach is underpinned by active learning theory.9 Active learning has been defined as ‘any instructional method that engages students in the learning process’.10 An active learning approach is one in which students undertake meaningful learning activities, cognitively engaging in those activities. Previous research suggests that students’ understanding and performance are improved via active learning.10–13 A meta-analysis confirmed that active learning increased student performance and decreased failure rate in undergraduates.14 Thus, as FC is a form of active learning,4,15 it has been associated with better learning outcomes,16 especially in higher-order thinking.17,18 Indeed, research suggests that an FC approach increases motivation, satisfaction, performance and class attendance rate.4,19,20 However, research results are generally inconsistent,2,19,21–29 even when the same outcome measure is considered.7,8

이 모순 문제를 강조하는 의료전문가의 교육 분야에서 많은 체계적인 검토가 수행되었다.

A number of systematic reviews have been undertaken in the area of health care professionals’ education that highlight this issue of inconsistency.

비록 FC 과정에 대한 학생들의 인식이 통제된 9개 연구 모두에서 전형적으로 긍정적이었지만, Chen 등,7명은 지식과 기술skill의 변화 측면에서 엇갈린 결과를 보고했다. 예를 들어, 다중 선택 질문(MCQ)을 사용한 결과와 관련하여, 일부 연구는 LB 접근법 30,31에 대해 FC에 대해 긍정적인 결과를 보고한 반면, 다른 연구들은 차이를 발견하지 못했다.22,32, Chen et al. 는 연구들 간에 효과 크기와 신뢰구간(CI)의 다양한 방향과 크기를 발견했다.7Hughes와 Lyons8의 검토에서는 MCQ 결과를 고려할 때 11개 연구에서 혼합된 결과를 발견했다.

Although students’ perceptions of the FC process were typically positive across all nine of the controlled studies, Chen et al.,7 reported mixed results in terms of changes in knowledge, and skills. For example, regarding outcomes using multiple-choice questions (MCQs), some studies reported positive findings for the FC over the LB approach,30,31 whereas others found no difference.22,32 Furthermore, Chen et al. found a varying direction and magnitude of the effect sizes and confidence intervals (CIs) across studies.7 A more recent systematic review by Hughes and Lyons8 also found mixed results across 11 studies when considering MCQ outcomes:

간호 교육 분야에서도 체계적인 검토가 실시되어 마찬가지로 애매한 결과를 보고하고 있다. 예를 들어, 베타바스 외 연구진.33은 체계적인 검토에 포함시키기 위한 5개의 연구만 식별하였다. 그들은 FC 조건이 이벤트 후 즉시 더 나은 결과를 도출하는 경향이 있었지만, 최종 시험 수행에 관한 한 어떠한 장점도 잃게 된다는 것을 발견했다.26  게다가, 모든 연구가 FC,2의 이점을 보고한 것은 아니며, 모든 과목 평가가 긍정적인 것은 아니었다.34

Systematic reviews have also been undertaken in the area of nurse education, reporting similar ambiguous results. For example, Betihavas et al.33 identified only five studies for inclusion in their systematic review. They found that although the FC condition tended to produce better outcomes immediately post-event26,34 alongside positive course evaluations,35,36 when it came to performance in the final examination, any advantage was lost.26 Furthermore, not all studies reported an advantage for the FC,2 and not all course evaluations were positive.34

게다가, 간호사 교육에서 FC 방법에 관한 13개 기사에 대한 서술적 리뷰에서, Presti20은 '간호 교육에서 FC에 대한 연구는 거의 없고, 오직 한 연구만이 시험 점수 향상에 대한 가치를 통계적으로 입증한다'고 결론지었다.

Furthermore, in a narrative review of 13 articles reporting on the FC method in nurse education, Presti20 concluded that ‘few studies exist on the flipped classroom in nursing education, and only one study statistically validates its value in improving examination scores’.

이러한 문학 비평에서 알 수 있듯이, 우리는 LB 학습 대비 FC의 효능에 관한 다소 애매한 그림을 남기고 있다. FC의 중요성을 고려할 때, FC 방법이 작동하는지 그리고 어떤 상황에서 작동하는지 더 잘 이해하기 위해 연구 방법을 사용하는 것이 타당해 보인다. 메타 분석37–39에 의한 데이터 통합은 충돌하는 연구의 논쟁을 해결할 수 있으며, 상대적 개입의 효과를 추정하기 위한 더 큰 힘과 정밀도를 가능하게 하며, 새로운 가설을 생성할 수 있다.

As we can see from these literature reviews, we are left with a rather muddy picture regarding the efficacy of FC over LB learning. Given the importance of the FC, it seems pertinent that we employ research methods to further understand whether the FC method works and in what circumstances. Data aggregation by meta-analysis37–39 can settle controversies from conflicting studies, allowing greater power and precision to estimate the effectiveness of relative interventions, and can generate new hypotheses.

메타 분석의 경우 부분군 분석과 메타 규제를 통해 연구 간 불일치의 원인을 탐색할 수 있다. 따라서 메타 분석은 개별 연구가 할 수 없는 질문에 대답하는 데 도움이 될 수 있다.

In a meta-analysis, the cause of inconsistency between studies can be explored through subgroup analyses and meta-regression. Thus, a meta-analysis may help to answer questions that individual studies cannot.

이 개념에 따라, 두 메타분석은 최근 중국의 연구자들에 의해 발표되었고, 간호학에서 FC 방법을 연구하는 연구에 초점을 맞추고 있다.41

Following this notion, two meta-analyses have very recently been published by researchers from China, drawing on studies examining the FC method in nursing.40,41

LB 접근법에 대한 FC의 상대적 효능을 검토하는 최근의 검토의 수에도 불구하고, 문헌의 gap은 여전히 남아 있다. 

• 모든 검토는 단일 분야로 제한되었다(예: 간호교육33,40,41 및 의료교육7,8). 

• 일반적으로 FC 접근법을 선호하는 소수의 논문만 있다(종종 5~11, 29개 포함 1개만 해당), 

• 메타분석은 두 개 뿐이고 주로 narrative synthesis이다.40,41 

이 두 메타분석가는 단일 국가(중국)의 데이터를 고려했고, 중국 간호학 학술지에 실린 RCT만 풀링했다. 이 매우 제한된 데이터 출처에서 나온 그러한 메타분석의 결과는 일반화하기가 어려울 수 있다.

Despite the number of recent reviews examining the relative efficacy of the FC over the LB approach, a gap in the literature remains: 

• all of the reviews were limited to a single discipline (e.g. nursing education33,40,41 and medical education7,8), 

• with a small number of articles generally favouring the FC approach (often between five and 11, with only one including 29), 

• mostly being narrative syntheses with only two meta-analyses.40,41 

These two meta-analyses considered data from a single country (China), only pooling RCTs from Chinese nursing education published in Chinese journals. The results of such meta-analyses from this very restricted data source may be difficult to generalise.

FC 교육학적 모델이 광범위한 분야에 걸쳐 채택된 경우, 여러 분야에 걸친 데이터 풀링은 통계적 파워를 높이고 서로 다른 영역의 영향 차이를 식별하는 데 도움이 될 수 있다.

Given that the FC pedagogical model has been adopted across a wide range of disciplines, data pooling across multiple disciplines may help to increase statistical power and identify differences in the effects across different domains.

• RQ1 FC 학습 접근법이 고등교육에서 LB 학습 접근법보다 학생들의 강화된 지식과 행동과 더 관련이 있는가?

• RQ2 이것이 맥락에 따라 다른가(예: 연구 방법, 출판 연도, 출판 포럼, 학생 그룹 및 학습 환경)?

• RQ1 Is the FC learning approach associated with students’ enhanced knowledge and behaviour more than the LB learning approach in higher education?

• RQ2 Does this differ by context (e.g. research method, publication year, publication forum, student group and learning environment)?

방법

METHODS

The meta-analysis was undertaken following the PRISMA (preferred reporting items for systematic reviews and meta-analyses) statement (see Box 1 for definitions of terms used in a meta-analysis).42

Box 1 Definition of terms in a meta-analysis

Meta-analysis

A statistical analysis that combines the results of multiple quantitative information from related studies and produces results that summarise a whole body of research.

Begg’s test

An indicator of publication bias. Begg et al. proposed testing the interdependence of variance and effect size using Kendall’s method. This bias indicator makes fewer assumptions than that of Egger et al. but it is insensitive to many types of bias to which Egger’s test is sensitive.

Cochrane Q

A classical measure of heterogeneity in meta-analysis. Q is distributed as a chi-squared statistic with n 1 degrees of freedom. For the Q-statistic test, a ‘Cochrane Q-value > degree of freedom’ or a ‘p-value of < 0.10’ suggests the presence of heterogeneity.

Egger’s test

Another indicator of publication bias. Egger et al. suggested a test to examine asymmetry in funnel plots. This is a test for the Y-intercept = 0 from a linear regression of the normalised effect estimate (i.e., the estimate divided by its standard error) against precision (the reciprocal of the standard error of that estimate).

Funnel plot

A graph that provides a visual check of the existence of publication bias. In the absence of publication bias, it assumes that studies with high precision will be plotted near the average, and studies with low precision will be spread evenly on both sides of the average, creating a roughly funnel-shaped distribution. Asymmetry in funnel plots may indicate the presence of publication bias in meta-analysis.

Heterogeneity

Heterogeneity in meta-analysis refers to the variation or inconsistency in the study outcomes between studies.

I-squared (I2)

Another measure of heterogeneity in meta-analysis. The I² statistic is the percentage of variation across studies attributable to heterogeneity. The formula of I² = 100% 9 (Q–df)/Q. An I2 of < 25% represents low heterogeneity, 25–50% moderate heterogeneity and > 50% high heterogeneity.

Meta-regression

A statistical method that can be used in meta-analysis to examine the impact of moderator variables on study effect size using regression-based techniques.

PRISMA statement

A guideline for the reporting of systematic reviews and meta-analyses. The PRISMA statement comprises 27 items and a four-phase flow diagram.

Publication bias

Occurs when the published results depend not just on the quality of the research but also on the hypothesis tested, and the significance and direction of effects detected. It suggests that results not supporting the hypotheses of researchers often go unpublished, leading to a bias in published research.

Random-effects model

• The process of meta-analysis is undertaken by either a fixed-effect or a random-effects statistical model. A fixed-effect meta-analysis assumes all studies are estimating the same (fixed) treatment effect, whereas a random-effects meta-analysis permits differences in the treatment effect across studies.

• In a random-effects model, the observed heterogeneity is attributed to two sources: (i) between-study heterogeneity in true effects, and (ii) within-study sampling error.

Eligibility criteria

Data sources

Search strategy

‘flipped classroom’, ‘flipped education’, ‘flipped learning’, ‘reverse classroom’, ‘backward classroom’, ‘inverted classroom’ and ‘inverse classroom’.

Study selection

Assessment of methodological quality

The Effective Public Health Practice Project (EPHPP) Quality Assessment Tool44 was used to appraise methodological quality as it is suitable for both interventional (RCT, quasi-experimental) and observational (cohort, historical control and case– control) studies.

Data synthesis

Data were independently extracted by two reviewers (Y-HL and K-SC) using a standardised form. The extracted information included publication type and year, design, demographics of participants, intervention and control, and type of outcomes measured. The cognitive levels of learning outcomes measured were categorised according to the Kirkpatrick model43 and Bloom’s taxonomy.45

결과

RESULTS

연구 검색 및 선택

Study search and selection

The process is summarised in a PRISMA42 flow diagram (Fig. 1).

포함된 연구들

Scope of the included studies

Of the 46 included studies, 36 were journal articles, nine were conference abstracts and one was a thesis (Table S1).

Outcomes measured

Multiple effect measures were reported. We classified the effect measures into three categories: 

• examination scores (n = 41); 

• course grades (n = 9); and 

• objective structured clinical examinations (OSCEs) (n = 2). 

The examination scores were 

• the results of MCQ tests (n = 20) and 

• written examinations or essays (n = 3), 

• a combination of different formats (n = 5) or 

• were unclear (n = 13).

Of the 41 studies reporting examination scores, seven were before–after studies and provided pre–post score changes. All examination score outcomes evaluated knowledge change according to Kirkpatrick’s level 2 measurement (learning).43

The level of cognitive learning evaluated in these tests comprised only lower cognitive levels in one study, ranging from low to high (levels 1–6 in Bloom’s taxonomy45)in 16 studies and were not described in 24 studies.

연구의 퀄리티

Quality assessment of the included studies

방법론적 품질 평가(36개 저널, 논문 1개)는 36개 연구가 높은 편향 위험을 나타냈으며, 한 연구는 편향의 위험이 낮다는 것을 보여주었다(자세한 내용은 표 S2 참조). 가장 약한 지표는 blinding이었다. 데이터 수집, 교란 요인 및 인출 도메인에서도 위험성이 높았다.

Methodological quality assessment (36 journal articles, one thesis) showed that 36 studies exhibited a high risk of bias and one study had a moderate risk of bias (see Table S2 for details). The weakest indicator was blinding. Risks were also high in the data collection, confounders and withdrawal domains.

낮은 보고 질은 높은 편향 위험을 야기하는 중요한 원인이었다. 많은 품질 지표가 명확하게 설명되지 않았거나 누락되었다. 자주 누락된 정보: 연구 설계, 적격 모집단, 참가자 및 중퇴자 모집단, 기준 그룹 간의 중요한 교란 요인, 결과 평가 도구의 신뢰성/유효성. 이것은 많은 연구를 편견의 높은 위험성으로 평가하게 했다.

Poor reporting quality was an important cause of a high risk of bias. Many quality indicators were not clearly described or were absent. Frequently missing information included: study design; eligible population; participant and dropout populations; important confounders between groups at baseline, and the reliability/validity of outcome assessment tools. This resulted in many studies being rated with a high risk of bias.

메인그룹, 서브그룹 메타분석

Main and subgroup meta-analyses

2개 연구로도 가능함

Following Valentine et al.,57 we note that it is possible to undertake a meta-analysis with as few as n = 2 studies.

성과 척도에 따라

Meta-analyses by outcome measures

Data pooling of all 46 studies was undertaken according to the type of outcome measure reported. Significantly higher examination scores, pre–post examination score improvements and course grades for the FC approach were identified (Table 2, Fig. 2). However, no significant effect was noted when the outcome measure was an OSCE score.

맥락에 따라 

Subgroup analyses by context

A subgroup analysis was undertaken for the outcome of examination scores.

메타 리그레션

Meta-regression for the examination score outcome measure

Meta-regression by year (Fig. S8) showed a significant correlation (p = 0.023) between publication year and effect size (by excluding the year 2012, which had only one single study), but nosignificant relationship between sample size and outcome (p = 0.856). 

Publication bias

고찰

DISCUSSION

체계적 검토에 대한 서술적 접근법의 주관성을 피하고, 다수의 영역에서 FC 방법의 상대적 효과를 확인하기 위해 데이터를 수집하기 위해 메타 분석 접근법을 채택했다.59

Our meta-analytical approach was adopted to avoid the subjectivity of the narrative approach to systematic reviews, and to pool the data from a large number of studies to ascertain the relative effectiveness of the FC method across a number of domains.59

우리는 (우리가 검색한 전자 데이터베이스에 연구가 제공된다면), 모든 분야와 국가에 대한 연구를 포함하였고, 다양한 교육 환경에서 시험 점수와 과목 성적(즉, 이론적 지식과 관련된)에 대한 중국 연구의 결과를 재확인했다. 그러나 OSCE 결과(기술과 관련된)의 경우, 분석 결과 FC 조건에서 이점이 없었다. 메타 분석은 최소 두 개의 연구를 사용하여 수행될 수 있다는 주장에도 불구하고, 57 이 후자의 결과는 우리 데이터베이스에 OSCE 결과를 보고하는 연구 결과가 소수라는 점과 그 연구마다 결과가 크게 다르다는 사실에 기인할 수 있다.

We included studies across all disciplines and countries, as long as the studies were available in the electronic databases we searched, verifying the findings from the Chinese studies for examination scores and course grades (i.e. related to theoretical knowledge) across a range of educational settings. However, for the OSCE outcome (related to skills) our analysis found no advantage for the FC condition. Despite the assertion that meta-analysis can be undertaken using as few as two studies,57 this latter finding may be a result of the small number of studies reporting OSCE outcomes in our database and the fact that their results were greatly divergent.

또한 여러 연구 설계 간에 FC 유효성 불일치가 나타났다. 

RCT에서는 장점이 없었지만, 코호트 또는 준 실험 연구는 LB 조건에 비해 FC의 유의한 장점을 보여주었다. 우리 연구에서 발견된 FC 조건의 이러한 유효성 부족은 RCT에서 FC 접근법의 상당한 이점을 보여준 이전의 두 메타 분석 7,8의 결과와 직접적으로 모순된다. 이러한 불일치는 데이터 집합의 소수의 RCT가 다시 한 번 이유일 수 있다.

Our study also revealed inconsistencies in FC efficacy between different study designs: cohort or quasi-experimental studies, but not RCTs, showed a significant advantage of the FC over the LB condition. This lack of efficacy for the FC condition found in our study directly contradicts the findings from the two earlier meta-analyses,7,8 which demonstrated a significant advantage of the FC approach in RCTs. This discrepancy may reflect, once again, the small number of RCTs in our dataset.

연구 중인 특정 학생 그룹에 따르면 우리의 결과에서 더 많은 차이가 발견되었다. 따라서 의료, 약국, 비건강 과학 교육 학생 그룹은 FC 경험으로부터 이익을 얻는 것처럼 보였지만, 간호학과 다른 건강 과학 학생들에게는 그렇지 않았다.

Further differences were found in our results according to the particular student group under study. Thus, medical, pharmacy and non-health science education student groups appeared to benefit from their FC experiences, but this was not the case for nursing and other health science students.

또한, 각 종목별 FC 방법론에서 상대적인 성숙도를 고려할 때, 우리의 누적 분석은 시간이 경과함에 따라 FC 조건의 결과가 점진적으로 개선되는 경향을 시사한다는 것을 유념할 필요가 있다. 따라서 FC 방법에 대한 점진적인 이해, 새로운 기술의 활용, FC 방법과 관련된 교수 기술의 발달 성숙도 및 FC 교육 형식의 누적된 경험이 모두 결과 개선에 기여할 수 있다.7

Furthermore, when considering the relative maturity of the FC methodology in each respective discipline, it is worth noting that our cumulative analyses suggest a tendency of progressive improvement in the outcome of the FC condition over time. Thus, progressive understanding of the FC method, the utilisation of newer technology, the developing maturity of teaching skills associated with the FC method and accumulated experience in the FC teaching format may all contribute to the improvement of outcomes.7

우리의 메타분석 결과에는 몇 가지 한계가 있다. 예를 들어, 연구는 통계적 이질성이 높았다. 통계적 이질성은 많은 수준의 방법론적 다양성의 결과일 가능성이 있다.

The results of our meta-analyses have several limitations. For example, the studies had a high degree of statistical heterogeneity. It is likely that the statistical heterogeneity is a result of the large degree of methodological diversity.

이와 같이, 모든 연구가 FC 스타일의 교육의 실행을 다루었지만, 그들은 교실에서의 flipping은 형식과 내용 둘 다 달랐다. 예를 들어, FC 활동을 설계하고 이끌었던 교사의 경험과 전문지식, 클래스 전 학습 자료의 유형과 특성 등은 다른 경우가 많다. 평가된 지식의 내용 또는 인지적 수준(즉, 결과)도 다양했다.

As such, although all studies addressed the implementation of an FC style of education, they varied in both the format and content of classroom flipping. For example, the experiences and expertise of teachers who designed and led the FC activity, and the types and qualities of pre-class learning material were likely to differ. The content or cognitive level of knowledge evaluated (i.e. outcomes) also varied.

FC 조건은 더 높은 인지 수준을 촉진하기 위해 설계된다(예: 애플리케이션). 따라서, 낮은 수준의 평가(예: 리콜)는 반드시 FC 접근법에 적절한 결과로 간주되지 않을 수 있다. 우리가 포함된 연구들 사이의 학업 성과는 주로 시험 점수, OSCE, 과정 성적의 세 가지 유형으로 구성되어 있다. 이들 대부분은 Kirkpatrick 수준 2(학습) 결과를 반영한다.43 또한 OSCE는 Kirkpatrick 수준 3(행동) 결과를 반영할 수 있으며, MCQ는 단순히 암기보다 높은 인지 수준의 질문을 포함할 수 있다. (표 S2에서 확인한 바와 같이) 과목 점수 평가 공식이 연구마다 다르기 때문에 평가된 인지 수준도 달랐다. 연구 설계, 결과 및 대상 모집단의 다양성은 효과 크기의 이질성에 기여한다. 이질성이 높다는 점을 감안할 때, 우리는 FC 방법의 효능에 대해 포괄적인 결론을 도출하는데 신중해야 한다.

The FC condition is designed to promote higher cognitive levels (e.g. application). As such, lower levels of assessment (e.g. recall) may not necessarily be considered as an appropriate outcome for the FC approach. The academic outcomes amongst our included studies mainly comprised three types: examination scores; OSCEs, and course grades. Most of these reflect a Kirkpatrick level 2 (learning) outcome.43 Additionally, the OSCE may also reflect a Kirkpatrick level 3 (behaviour) outcome and the MCQ may also include questions of a higher cognitive level than mere memorisation. As the formula for assessing course grades differed between studies (as we identified in Table S2), the cognitive levels evaluated also differed. Such diversity in study designs, outcomes and target populations contributes towards the heterogeneity in effect sizes. Given that heterogeneity was high, we must be cautious in drawing sweeping conclusions around the efficacy of the FC method.

출판 편향의 문제도 다루어질 필요가 있다. 우리는 중요한 정보와 출판 편향을 누락시키지 않도록 하기 위해 이 검토에서 컨퍼런스 연구와 논문을 통합했다. 그러나, 이용 가능한 정보가 제한되어 있기 때문에 회의 연구의 방법론적인 특성은 평가할 수 없었다.

The issue of publication bias also needs to be addressed. We incorporated conference studies and a thesis in this review to avoid missing important information and publication bias. However, the methodological qualities of conference studies could not be assessed because of the limited information available.

메타 분석의 또 다른 한 가지 제약은 2016년 6월에 문헌 검색을 수행했으며, 그러한 세부 프로세스를 수행하는 데 오랜 시간이 걸리는 특성 때문에

Another limitation of our meta-analysis is that the literature search was undertaken in June 2016 and because of the lengthy nature of undertaking such a detailed process

하지만, 이러한 제한에도 불구하고, 우리의 연구는 장점을 가지고 있다. 우리는 전 세계의 다양한 의료 및 비의료 교육 분야의 총 9026명의 참가자와 함께 46개 연구의 데이터를 구성하는 메타 분석을 수행했다. 이것은 우리가 알고 있는 종류의 가장 큰 연구일 뿐만 아니라 메타 분석이기 때문에 현재 FC 문헌을 지배하는 서술적 검토 방법보다 더 높은 수준의 증거 60,61을 제공한다.

However, despite these limitations, our study has strengths. We have undertaken a meta-analysis comprising data from 46 studies with a total of 9026 participants from different health care and non-health care educational disciplines across the world. This is not only the largest study of its kind that we know of, but, because it is a meta-analysis, it provides a higher level of evidence60,61 than do the descriptive review methods presently dominating the FC literature.7,27,33

우리의 연구는 주로 연구 방법, 연구 설계, 보고의 개선에 초점을 맞춘 미래 연구 제안들을 하도록 우리를 이끈다. 따라서, 잘 훈련되고 동기 부여가 된 촉진자를 포함한 FC의 표준형식은 미래 연구 연구를 위한 FC와 LB 방법을 비교하기 위한 기본 요건이 된다. 또한, FC 방법의 목적은 더 높은 인지 학습을 개선하고 행동 변화를 촉진하는 것이기 때문에, 향후 연구 설계가 FC의 효능을 기술할 것으로 예상되는 더 높은 인지 결과를 측정할 수 있을 만큼 충분히 민감하다는 것을 보장하기 위해 더 많은 주의를 기울여야 한다. 실제로, 새로운 교육학의 평가는 개념적 지식이 임상적 추리 방법과 같은 새로운 도구를 사용하여 평가되어야 한다는 것을 암시한다.62

Our study leads us to make future research suggestions, focusing mainly on improving study methods, research design and reporting. Thus, a standard format of classroom flipping, with well-trained and motivated facilitators, comprises the basic requirement for a comparison between FC and LB methods for future research studies. Further, given that the purpose of the FC method is to improve higher cognitive learning and promote behavioural change, greater attention should be given to ensuring that future research study designs are sensitive enough to measure the higher cognitive outcomes expected to delineate the efficacy of the FC. Indeed, the appraisal of a new pedagogy suggests that conceptual knowledge should be evaluated using novel tools, such as clinical reasoning measures.62

또한 미래 연구는 연구 보고를 다룰 필요가 있다. 즉, 교육 연구의 불충분한 보고의 질은 위험편향 평가 과정에 영향을 미치는 중요한 사안이다. 우리의 메타분석에 포함된 많은 연구에서, 많은 필요한 정보가 누락되었다. 예를 들어 적격 및 등록된 참가자의 수, 이들의 인구학적 특징, 중퇴율, 평가 기구의 유효성과 신뢰성에 대한 세부사항은 종종 누락되었다. 나이와 성별은 임상 연구에서 중요한 교란 요인이며, 교육 연구에서도 그렇다. 그러나, 우리의 검토의 많은 출판물들은 그러한 인구통계를 언급하지 않았다. 우리의 조사 결과와 유사하게, Horsley et al.63은 '... 건강 직업 교육 분야의 무작위 연구에 대한 보고서에는 CONSORT 성명에서 권고한 요소가 자주 누락되는 경우가 많다. 대부분의 보고서는 편견의 위험이 높거나 불명확한 것으로 평가되었다.' 따라서, 의료 교육 연구에서 연구 보고를 위한 구조화된 형식이 보장된다.

Future research also needs to address the reporting of research: the poor reporting quality of educational studies is an important issue influencing the process of risk bias assessment. In many of the studies included in our meta-analysis, a lot of necessary information was absent. For example, details of the number of eligible and enrolled participants, their demographic features, dropout rate, and the validity and reliability of assessment instruments were often missing. Age and gender are important confounders in clinical studies, and this is also the case in education research. However, many publications in our review failed to mention such demographics. Similar to our findings, Horsley et al.63 concluded that: ‘... reports of randomised studies in health professions education frequently omit elements recommended by the CONSORT statement. Most reports were assessed as having a high or unclear risk of bias.’ Thus, a structured format for reporting research in medical education studies is warranted.

---

Med Educ. 2018 Jun 25. doi: 10.1111/medu.13616. [Epub ahead of print]

Academic outcomes of flipped classroom learning: a meta-analysis.

Chen KS1,2,3, Monrouxe L2,3, Lu YH1,2,3, Jenq CC2,3,4, Chang YJ2,3,5, Chang YC2,3,6, Chai PY7.

Author information

1

Department of Nephrology, Chang Gung Memorial Hospital, Keelung branch, Keelung, Taiwan.

2

Chang Gung Medical Education Research Center (CG-MERC), Chang Gung Memorial Hospital, Linkou Branch, Taoyuan, Taiwan.

3

College of Medicine, Chang Gung University, Taoyuan, Taiwan.

4

Department of Nephrology, Chang Gung Memorial Hospital, Linkou branch, Taoyuan, Taiwan.

5

Department of Neurology, Chang Gung Memorial Hospital, Linkou branch, Taoyuan, Taiwan.

6

Department of Emergency Medicine, Chang Gung Memorial Hospital, Linkou branch, Taoyuan, Taiwan.

7

Department of Pharmacy, Chang Gung Memorial Hospital, Keelung branch, Keelung, Taiwan.

Abstract

CONTEXT:

The flipped classroom (FC), reversing lecture and homework elements of a course, is popular in medical education. The FC uses technology-enhanced pre-class learning to transmit knowledge, incorporating in-class interaction to enhance higher cognitive learning. However, the FC model is expensive and research on its effectiveness remains inconclusive. The aim of this study was to compare the efficacy of the FC model over traditional lecture-based (LB) learning by meta-analysis.

METHODS:

We systematically searched MEDLINE, PubMed, ERIC, CINAHL, EMBASE, reference lists and Association for Medical Education in Europe (AMEE) conference books. Controlled trials comparing academic outcomes between the FC and LB approaches in higher education were considered eligible. The main findings were pooled using a random-effects model when appropriate.

RESULTS:

Forty-six studies (9026 participants) were included, comprising four randomised controlled trials (RCTs), 19 quasi-experimental studies and 23 cohort studies. Study populations were health science (n = 32) and non health science (n = 14) students. The risk of bias was high (36/37 articles). Meta-analyses revealed that the FC had significantly better outcomes than the LB method in examination scores (post-intervention and pre-post change) and course grades, but not in objective structured clinical examination scores. Subgroup analyses showed the advantage of the FC was not observed in RCTs, non-USA countries, nursing and other health science disciplines and earlier publication years (2013 and 2014). Cumulative analysis and meta-regression suggested a tendency for progressively better outcomes by year. Outcome assessments rarely focused on behaviour change.

CONCLUSIONS:

The FC method is associated with greater academic achievement than the LB approach for higher-level learning outcomes, which has become more obvious in recent years. However, results should be interpreted with caution because of the high methodological diversity, statistical heterogeneity and risk of bias in the studies used. Future studies should have high methodological rigour, a standardised FC format and utilise assessment tools evaluating higher cognitive learning and behaviour change to further examine differences between FC and LB learning.

© 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd.

PMID:

 

29943399

 

PMCID:

 

PMC6120558

 

DOI:

 

10.1111/medu.13616