학생선발과정에서 얻은 네러티브 정보가 문제행동을 예측한다 (Med Teach, 2016)

학생선발과정에서 얻은 네러티브 정보가 문제행동을 예측한다 (Med Teach, 2016)

Narrative information obtained during student selection predicts problematic study behavior

MIRJAM G. A. OUDE EGBRINK & LAMBERT W. T. SCHUWIRTH

Maastricht University, The Netherlands

도입

Introduction

최근까지 초점은 cognitive academic performance 의 예측인자에 있었다. 그러나 이제 비인지적 quality도 미래 의과대학생과 의사로서 중요하다는 것이 명확하다.

Until recently, the focus has been primarily on predictors of cognitive academic perform- ance (Salvatori 2001; Siu & Reiter 2009). Nowadays, however, it is clear that, besides cognitive skills, non-cognitive qualities are important competencies of future medical students and doctors.

MMI가 사용되고 있음.

Recently, the so-called multiple mini-interview (MMI) show that multiple individual human judgments of non- cognitive skills when combined predict future performance in a sufficiently reliable way.

2007년 Maastricht University 의 P-CI 선발에 MMI를 사용하기 시작. 선발 과정에서 순위리스트가 나오는데, research master로서의 성공적인 수행 적합도를 예측에 대한 순위이다.

In 2007, the MMI method was introduced as part of the selection procedure for the four-year medical research master Physician-Clinical Investigator (P-CI) at Maastricht University (Guyaux et al. 2010). The selection procedure results in a ranking list, representing differences in predicted suitability to perform successfully in this research master.

대부분의 선발된 학생이 인지적 측면과 비인지적 측면 모두에서 성공적이지만, 일부는 문제행동을 보인다. 명확하게 이들 문제는 MMI점수에 의해서 예측되지 않으며 선발과정의 다른 부분에 의해서도 예측되지 못한다. 이론적으로 MMI 진행과정에서 면접관이 기록한 narrative information은 학생 파일에 저장되고, 이것이 미래 행동을 더 잘 예측해줄 수도 있다.

Although most selected students are successful in both cognitive and non-cognitive aspects of the study, some encounter professional lapses or problematic study behavior. Clearly, these problems were not predicted by the MMI scores or any other part of the selection procedure. Theoretically, the narrative information that is written down by the interviewers during the MMIs and stored in the student files could be a better predictor of such problems and could constitute a useful resource for the student mentors (called counselors in the P-CI master), but till now this information has been unused.

방법

Methods

 

맥락

Context

The four-year P-CI research master is a graduate-entry program that enables students to become medical doctor as well as clinical investigator. This combination makes it a challenging program for the students. Each year, a selection procedure decides which 30 students are allowed to enter this master.

• They must have finished a biomedical bachelor with good results; GPAs as well as a cognitive test are taken into account in the first part of the selection procedure.

• The second part consists of MMIs on different topics, such as motivation, past performance, empathy and communication skills. The applicants’ performances on each individual interview are graded independently by the interviewers as being ‘‘suffi- cient’’, ‘‘doubtful’’ or ‘‘insufficient’’, and the combination of all individual scores adds up to a ranking list. In each station, interviewers also make notes that are not used in the procedure itself; both notes and grading are completed in the time interval between individual interviews. The notes are stored for possible use in appeals, to underpin the inter- viewers’ judgments.

학생과 카운셀러(지도교수)

Students and counselors

In this study, we focused on students who enrolled into the n¼30) P-CI master in 2007 (cohort 2007; and 2008 (cohort 2008; n¼30). In this master, each student is assigned to a counselor at the start of the first year, who mentors the student on an individual basis throughout his/her study. Each counselor typically takes care of 3–8 students per cohort. Every year, student and counselor meet at least four times.

Seven counselors mentored the 60 students in cohorts 2007 and 2008 (five in cohort 2007 and six in cohort 2008; four of them were active in both cohorts). In the end, 54 out of 60 students have finished their study within four to five years, while one student is currently finishing the last part.

연구설계

Study design

This retrospective exploratory study was subdivided into three parts.

 

• First, the seven counselors were asked to name the three most prevalent non-cognitive problems they encountered in ‘their’ students, and grade them (3-2-1) to indicate the graduate-entry (3 ¼most From their frequency of occurrence frequent). program that enables students to become medical doctor as reactions the two most highly-graded problems were selected well as clinical investigator. This combination makes it a for further analysis.

• Second, two independent and blinded investigators (MoE and LS) analyzed the de-identified notes written down during the MMIs of 15 randomly chosen students out of the total of 55, and identified what they thought to be possible indicators for these two most frequent non-cognitive problems.

• Third, a case-control study design was used. The coun- selors were asked to identify the students who exhibited either one or both of these non-cognitive problems during their study (cases). The notes of their MMIs were de-identified and screened by the same two independent and blinded investi- gators (MoE and LS) to investigate whether the proposed indicators of these problems were indeed present. As a control, the MMI notes of a similar number of control students from the same cohorts (without the identified non-cognitive problems) were screened for the presence of these indicators as well.

Results

두 가지 가장흔한 비인지적 문제

Part 1: The two most prevalent non-cognitive problems

계획 문제

Planning difficulties related to problems with

• 시간 관리 time management,

• 학습량의 과소추정 under- estimation of study load, and

• 우선순위 배정 문제 problematic prioritizing of tasks.

 

자기성찰 문제

Self-reflection-related problems were addressed as

• 자신의 행동의 결과에 대한 인식 부족 insufficient awareness of (the consequences of) own functioning,

• 방어적 행동 indica- tions of defensive behavior, and

• 개선을 위한 불충분한/비효과적 행동 insufficient or non-effective actions to improve this.

 

 

 

MMI노트에서 나타난 지표들

Part 2: Indicators in MMI notes

The narrative information that was written down during MMIs with 15 randomly chosen students was analyzed to investigate whether indications for the two most prevalent non-cognitive problems were already present during the selection procedure preceding the master.

In the MMI notes of five students both investigators found no indicators at all for the two non-cognitive problems. In the MMI notes of the other 10 students one or more potential indicators were found. In four of them potential indicators for both planning-related and self-reflection-related problems were present.

As a result of this analysis, a limited number of potential indicators for planning-related and self-reflection-related problems were identified (Table 2).

 

 

 

사례-대조군 연구

Part 3: Case-control study

Based on the above-mentioned findings, a case-control study was performed to investigate how predictive these indicators were for planning-related and/or reflection-related problems during the research master P-CI.

The seven counselors identified 23 students who exhibited prob-lems during their study  planning-related and/or reflection-related had (cases).

• Thirteen students planning-related problems, while

• six had reflection-related problems; another

• four students showed problems in both domains.

Altogether, the data indicate a statistically-significant asso- ciation between the presence of indicators for planning-related problems in MMI notes and the actual occurrence of such problems during the subsequent study (Table 3A: odds ratio 9.33; 95% confidence interval 2.12–41.07; p ¼0.003). No such evidence was found for self-reflection-related problems (Table 3B: odds ratio 1.39; 95% confidence interval 0.29–6.68).

 

 

 

 

고찰

Discussion

보통 선발 단계는 누구를 선발하고 떨어뜨릴지 결정에만 사용된다. 이번 연구에서 선발단계에서 얻어진 정부를 미래의 문제행동을 예측하는데 사용하였다.

As a result, the selection proced- ure is merely used to decide on who is admitted and who is not. In the current study, we propose to use narrative information obtained during selection interviews to predict future problems

선발된 학생이 성공할 수 있도록 early and dedicated counseling and remediation을 가능하게 해줄 것이다. 선발은 단순히 assessment-of-learning이 아니라 assessment-for-learning의 역할을 할 것이다.

This may enable early and dedicated counseling and remediation to improve the selected students’ study success. This way, selection will not only serve as an assessment-of-learning measure but also as a first assessment-for-learning step (Shepard 2000; Schuwirth & Van der Vleuten 2011).

Counseling은 연구커리어의 초반부터 이뤄지는 것이 educational, therapeutic intervention을 가능하게 해줄 것이다. Unorganized한 학생은 사전에 정해진 시간표에 따라 학습이 이뤄지는 과정에서의 학업부담과 압박때문에 힘들어한다. 적성 외에도 시간관리와 우선순위 설정은 학업적 성취에 중요하다. Organized한 학습은 progress와 success 모두와 연결된다. 따라서 early and dedicated counseling은 계획-관련 학습문제를 예방하거나 없애줄 것이며, study success를 높여줄 것이다.

With the cur- rent knowledge, however, counseling can be more focused right from the beginning of a study career, enabling specific educational and even therapeutic interventions. Literature shows that unorganized students suffer most from workload and pressure of progressing in their studies according to a predetermined timetable (Ruohoniemi et al. 2010). More than aptitude, time management and prioritizing are important for academic achievement (West & Sadoski 2011). Organized studying appears to be related to both study progress and success (Rytkonen et al. 2012). Therefore, early and dedicated counseling will help to prevent or diminish planning-related study problems and, as a consequence, improve study success.

절절한 자기-성찰은 의료전문직에게 중요하다. 이것이 우리가 포트폴리오와 카운셀링 시스템에서 학생에게 자기-성찰의 중요성을 깨닫게 하고, 성찰 스킬 개발을 자극하는 것을 중요한 목표로 삼은 이유이다.

Adequate self-reflection is nowadays considered an essential attribute of competent healthcare professionals. This is why it is one of the important goals of our portfolio and counseling system to increase students’ awareness of the importance of self-reflection and to stimulate development of their reflective skills (Driessen et al. 2005).

선발에 들인 노력에도 불구하고 의과대학 기간에 낙제하거나 유급이 발생하는 것은 우려를 낳는다. personal distress로 힘들어 하는 학생도 걱정하고, 대학 역시 struggling student에 쏟는 시간과 에너지가 disproportionate하여 걱정하며, 사회도 이들 학생에게 들어가는 공적 자금의 부담 때문에 걱정한다.

Drop-out from or delay during medical school, in spite of selection efforts, is a cause for concern (Yates 2011; Stratton & Elam 2014). This is the case

• for the students involved who suffer from personal distress,

• for the university that is faced with a disproportionate amount of time and energy spent on struggling students, and

• for society that has to bear the financial in burden for drop-out and delayed students countries where they receive public funding.

실제로, 선발자료의 사용 용도가 많아지는 것은 재정적 관점에서도 매력적이다. 네덜란드같이 교육이 공적 자금으로 이뤄지는 국가에서, delay 나 drop-out을 막는 것은 상당한 비용을 보상한다.

Indeed, the additional use of selection data is attractive from a financial perspective. In countries like the Netherlands, where education is publicly funded, the gains of avoiding delay or drop-out will compensate largely for the costs of a selection procedure and counseling system.

Siu E, Reiter HI. 2009. Overview: What’s worked and what hasn’t as a guide towards predictive admissions tool development. Adv Health Sci Educ Theory Pract 14:759–775.

Stratton TD, Elam CL. 2014. A holistic review of the medical school admission process: examining correlates of academic underperform- ance. Med Educ Online 19:22919.

 

 

 

---

Med Teach. 2016 Aug;38(8):844-9. doi: 10.3109/0142159X.2015.1132410. Epub 2016 Jan 25.

Narrative information obtained during student selection predicts problematic study behavior.

Oude Egbrink MG1, Schuwirth LW1.

Author information

• 1a Maastricht University , The Netherlands.

Abstract

INTRODUCTION:

Up to now, student selection for medical schools is merely used to decide which applicants will be admitted. We investigated whether narrative information obtained during multiple mini-interviews (MMIs) can also be used to predict problematicstudy behavior.

METHODS:

A retrospective exploratory study was performed on students who were selected into a four-year research master's program Physician-Clinical Investigator in 2007 and 2008 (n = 60). First, counselors were asked for the most prevalent non-cognitive problems among their students. Second, MMI notes were analyzed to identify potential indicators for these problems. Third, a case-control study was performed to investigate the association between students exhibiting the non-cognitive problems and the presence of indicators for these problems in their MMI notes.

RESULTS:

The most prevalent non-cognitive problems concerned planning and self-reflection. Potential indicators for these problems were identified in randomly chosen MMI notes. The case-control analysis demonstrated a significant association between indicators in the notes and actual planning problems (odds ratio: 9.33, p = 0.003). No such evidence was found for self-reflection-related problems (odds ratio: 1.39, p = 0.68).

CONCLUSIONS:

Narrative information obtained during MMIs contains predictive indicators for planning-related problems during study. This information would be useful for early identification of students-at-risk, which would enable focused counseling and interventions to improve their academic achievement.

PMID:

 

26805655

 

DOI:

 

10.3109/0142159X.2015.1132410

[PubMed - in process]