근거기반피드백 프로그램에서 의미있는 피드백을 위한 조건(Med Educ, 2016)

근거기반피드백 프로그램에서 의미있는 피드백을 위한 조건(Med Educ, 2016)

Investigating conditions for meaningful feedback in the context of an evidence-based feedback programme

Stephane Voyer,1 Cary Cuncic,1 Deborah L Butler,2 Kimberley MacNeil,2 Christopher Watling3 &

Rose Hatala1

도입

INTRODUCTION

피드백은 '미래의 성과를 이끌어 내기 위해 주어진 활동에서 [연수생]의 수행을 기술하는 정보'로 정의되었다 .2 이것은 다음과 같은 모델로 번역 될 수있다 :

(i) 성능이 관찰된다.

(ii) 관찰 결과가 학습자에게 전달되고,

(iii) 학습자 (관찰자가 있거나 없음)는 성과를 향상시키기위한 계획을 수립합니다.

Feedback has been defined as ‘information describing [trainees’] performance in a given activity that is intended to guide their future performance’.2 This can be translated into a model wherein: 

(i) performance is observed; 

(ii) the observation is communicated to the learner, and 

(iii) the learner (with or without the observer) creates a plan to improve performance.

그러나 의학 교육 문헌을 살펴보면이 단순화 된 모델의 각 단계에서 장벽이 드러납니다. 첫째, 연수생들이 임상 의무를 수행하는 과정에서 거의 관찰되지 않는다는 증거가있다.

However, examination of the medical education literature reveals barriers at each step in this simplified model. Firstly, there is evidence that trainees are rarely observed in the course of carrying out their clinical duties.6

둘째, 임상 교사가 정직하고 비판적 피드백을 생성하고 전달할 수있는 능력을 제한하는 장벽이있을 수 있습니다. 예를 들어, 강사는 학습자의 감정을 상하게하고 부정적인 피드백을 제공하는 것에 따르는 개인적인 영향을 우려 할 수 있습니다.

Secondly, there may be barriers that limit the ability of clinical teachers to generate and communicate honest, critical feedback.4,5 For example, instructors may be reluctant to hurt learners’ feelings, concerned about personal repercussions following the provision of negative feedback.

동시에 수퍼바이저는 형성 피드백을 제공하고 총괄 평가를 수행하는 이중 역할을 수행하기 때문에 trainee는 자신의 임상적 신뢰성을 감소시키고 평가에 영향을 줄 것이라고 생각하면 약점을 드러내지 않을 수 있습니다.

Simultaneously, because supervisors frequently carry the dual roles of providing formative feedback and undertaking the summative evaluation, trainees may not expose weaknesses if they feel it will reduce their clinical credibility and affect their assessments.9

피드백 생성 및 전달에 대한 이러한 장벽은 학습자와 감독자 사이의 일시적인 관계 만 허용하는 로테이션 모델 때문에 더 어려워진다.

these barriers to generating and communicating feedback are compounded by a rotational model that allows only transient relationships between learners and supervisors.10

개선 계획을 세우기 위해 학습자를 지원하는 장애요인들은 다음과 같다. 한 가지 문제는 임상 강사들이 성과가 실제로 어떻게 개선 될 수 있는지에 대한 유용한 지침을 제공하지 못하는 경우가 종종 있다는 것입니다 .11,12 또한, 연수생은 자신이받은 피드백을 해석하고 이로부터 이익을 얻거나 자신의 의견을 분석 할 수있는 충분한 기술이 없을 수 있습니다 학습 계획을 세웁니다.

Barriers to supporting learners to create plans for improvement . One problem is that clinical teachers often fail to offer any useful guidance as to how performance might actually be improved.11,12 Moreover, trainees may not have sufficient skills either to interpret and benefit from the feedback they do receive, or to analyse their own performance and generate a learning plan.

우리는 다음 세 가지 핵심 원칙을 바탕으로 프로그램을 구성했습니다.

(i) 피드백은 임상 수행의 직접적인 관찰에 기초한다;

(ii) 레지던트와 교직원이 종단적 교육 관계에 있고,

(iii) 피드백의 유일한 목적은 성과 개선이며 총괄적 요소를 포함하지 않습니다.

we structured our programme around three key principles: 

(i) feedback is based on direct observation of clinical performance; 

(ii) the resident and faculty member are in a longitudinal educational relationship, and 

(iii) feedback is offered for the sole purpose of improving performance and does not have a summative component.

우리의 개념적 틀은 피드백 제공에 영향을 미치는 많은 한계들이 학습에 대한 사회 문화적 관점에 대한 관심이 부족하기 때문에 가능하다는 견해에 근거했다. 즉, 학습자는 그들이 legitimate participant로 속하게 되는 문화의 가치에 따라 배우고 형성됩니다 .13

Our conceptual framework was grounded in the perspective that many of the limitations affecting the provision of feedback are attributable to insufficient attention to the socio-cultural perspective on learning. That is to say, learners learn and are shaped by the values of the culture in which they wish to be legitimate participants.13

우리는 특히 SRL모델에 영향을 받았는데, 여기서 피드백은 [교사 교직원과 레지던트 간의 '대화']로 구성 될 때 더 큰 잠재력을 갖게 되며, 이를 통해 레지던트는 학습 목표에 대한 더 많은 통제권을 행사하게 되며, 진도를 모니터링하고 그에 맞춰서 목표와 practice를 정비할 수 있는 갖게 된다.

We were further influenced by a model of self-regulated learning which identifies the potential of feedback, particularly when structured as a ‘dialogue’ between faculty staff and residents, to create opportunities for residents to take more control over learning goals, and to monitor progress and refine goals and practice as needed.15

METHODS

Research methodology

We used a qualitative case study design to address these research questions. The case study design facilitates examination of ‘how’ and ‘why’ a programme is working, and is particularly well suited to the investigating of complex, multidimensional phenomena in a natural environment.16

Setting

The internal medicine residency programme at the University of British Columbia (UBC) is a large, mostly urban training programme.

Participants

In September 2012, we recruited residents in postgraduate year 1 (PGY-1) of the UBC internal medicine residency programme; all PGY-1 residents were eligible for participation. We chose PGY-1 participants as they are at the start of their residency training and are thus perhaps less entrenched in the learning culture and more open to other influences on learning. As novices in internal medicine, they are also poised on a steep part of the learning curve at which educational interventions may have the greatest influence on learning. For this small pilot, nine volunteers were purposefully selected to represent a range of ages, genders and prior educational experiences. Resident and faculty member demographics are presented in Table 1.

Faculty participating in the programme consisted of three clinical educator faculty members (CC, RH, SV), who also designed the feedback programme and were members of our research team. Their participation as researcher-participants allowed us to gather evidence about how the programme was working from the ‘inside’ and then to maintain a consistent and coherent vision of the faculty role in this programme. Prior to the start of the study, these three faculty members participated in a 2-hourfaculty training session that reviewed current approaches to effective feedback in conjunction with the UBC Faculty Development Office. As the programme unfolded, they also engaged in frequent discussions about their own behaviours as faculty members during the study, which served as an informal method of faculty development, as well as a way of addressing reflexivity. The Department of Medicine paid for their participation in the programme at a stipend equivalent to that of attending physicians on clinical wards. 

Feedback programme

Each faculty member was responsible for three of the PGY-1s from September 2012 to July 2013. Residents initially met with their assigned faculty members to develop a shared understanding of the goals of the programme, and to develop their initial learning objectives. Subsequently, the faculty member observed each resident’s clinical performance every 2–3 months in the setting of the resident’s current clinical rotation. Prior to each observation, the faculty member and resident reviewed the previous learning objectives and discussed any new developments since the last observation. Faculty–resident pairs and frequencies of observations are presented in Table 1.

Each faculty member observed the resident for up to half a day as the resident performed his or her usual clinical duties on typical internal medicine inpatient rotations, which included those in critical care, internal medicine teaching units, and subspecialty rotations (Table 1). Observations occurred during daytime working hours and in the evening on-call period. At each patient encounter, the resident obtained the patient’s consent for the observation. The faculty member had no supervisory or patient care responsibilities.

Feedback was shared between patient consultations and summarised during a post-observation debriefing. The feedback was typically co-constructed as the faculty member and resident considered what had and had not gone well, how the resident was thinking through the situation, and what might have been done differently. Together, the resident and faculty member refined the resident’s plan of action and updated learning objectives. The resident was free to contact the faculty member at any time between observations to discuss issues or request an observation.

Research team

The research team initially included the three internal medicine clinical educators (CC, RH, SV), who designed and implemented the feedback programme and who brought perspectives from cognitive psychology and socio-cultural theory. 

• They developed the initial project concept and then approached DLB, a PhD education researcher with expertise in case study methodology, feedback and self-regulated learning, who joined the team to offer an ‘outsider’ perspective on the programme design, study methodology and data interpretation. 

• Once a first round of data collection by that four-member project team was complete, a doctoral student from the Faculty of Education (KM) was brought in for professional data analysis expertise. 

• Her distance from the project allowed her to take a fresh look at the full set of data, test the initial ‘categories’ developed by the project team against the original evidence, and bring confirming and disconfirming evidence to the team for discussion. 

• Neither DLB nor KM were clinicians and neither were directly involved in the feedback programme. 

• As a final addition, the team expanded to include a clinical educator with expertise in feedback and credibility judgements (CW), who was able to provide another ‘lens’ through which to examine our interpretation of findings.

Data collection

Multiple forms of data were collected. 

• Each introductory meeting and each debriefing between a faculty member and resident was audiotaped, transcribed and anonymised. 

• During each observation, faculty members consistently recorded structured written field notes, which captured basic descriptions of the context (e.g. setting, patients) linked with observations for potential discussion with the resident. 

• These data provided evidence of the logistics of the programme, the context of the observations and the nature of the feedback discussed. 

• Emails and text messages between residents and faculty staff were collected as they could potentially provide evidence of the influence of the programme upon the resident (such as when a resident reached out to a faculty member for emotional support).

At the end of the study period, semi-structured interviews were conducted individually with each resident by his or her faculty member. The interviews included questions on: 

• (i) the programme and how it had worked for the resident (Had the programme fulfilled his or her goals? What helped? What benefits were obtained? What were the challenges?); 

• (ii) participants’ understanding of feedback (How had it affected learning and decision making? How did it compare with feedback in other educational settings?), and 

• (iii) the impact of having a continuous connection with a faculty member. 

Because we recognised that residents might hesitate to be critical of the programme with their faculty members, they were also asked to participate in a focus group interview led by one of the non-clinician co-investigators (DLB). The focus group questions covered the same domains as the individual interviews.

Data analysis

Throughout our process of analysis, we moved cyclically through inductive (emerging from data) and deductive (theory-informed) analysis. 

• We involved researchers from multiple perspectives and levels of involvement in the project so that we were well positioned to identify converging themes. 

• The individual interviews and focus group served as the foundation for the analyses and were cross-referenced with the observation debriefing audiotapes, field notes, and communications between residents and faculty staff. 

• We considered all of our data sources as one dataset and did not separately analyse by data source.

At the outset of analysis, four researchers (CC, RH, SV, DLB) individually engaged in the act of memoing in order to capture insights and meaning patterns related to the dataset.17 

• They then met to identify key themes related to the research questions overall. 

• They did so by discussing the similarities and differences among perspectives and through visual representation (e.g. clustering ideas into overarching conceptual categories). 

• Discrepancies were resolved through discussion and by re-reading the original transcripts.

Next, KM was brought in to analyse data from her outsider perspective. 

• She engaged in the same process (i.e. memoing, determining key themes) as had the first four researchers and then brought her thinking to the group. 

• With her added input, the same process of testing and refining categories played out across several team meetings. 

• Over time, KM created a coding manual, refined it by bringing ‘test’ samples to group members for discussion, andcoded the data using a constant comparative method.18 

• CW participated in team meetings to discuss data analysis. 

• Across numerous meetings, the team worked from their multiple positions and perspectives to test and refine emerging codes in order to ensure that findings were linked to evidence from the dataset. 

결과

RESULTS

참가자가 언급한 이점들

Benefits described by participants

We found that residents reported benefiting not only 

• in ways of working, but also 

• in how they viewed and understood their learning in residency (ways of learning), as well as 

• in their confidence and emotional well-being (ways of feeling).

일 하는 법

Ways of working

레지던트의 관점에서 볼 때, 임상 기술에 대한 피드백은 즉각적인 우려사항을 말해주거나, 특히 구체적이고 다르게해야 할 일을 이해하는 데 도움이되었을 때 특히 의미있었습니다. 몇몇 레지던트는 교수진 관찰자로부터 후속 수행능력에 지속적인 영향을 미치게 된 매우 구체적인 조언을 받았다.

From residents’ perspectives, feedback on clinical skills was particularly meaningful when it addressed their immediate concerns, was specific and helped them understand what to do differently. Several residents received very specific tips from their faculty observer that they described as having an enduring impact on further performance.

레지던트들은 또한 고등 기술이라고 할 수있는 것에 대한 이득을 설명했습니다. 관찰 기간 동안, 레지던트들은 종종 자발적으로 소리내어 생각하여 임상 추론에 대한 토론을 촉발 시켰습니다. 레지던트들은 이런 종류의 고등 기술에 대한 피드백을 특별히 높이 평가했습니다. 왜냐하면 그것이 기존 교육에서 누락 된 것으로 느껴졌 기 때문입니다.

Residents also described gains in what might be referred to as higher-order skills. During the observations, residents often spontaneously thought aloud, sparking discussion about clinical reasoning. Residents specifically appreciated feedback on this kind of higher-order skill because it was felt to be otherwise missing from their training.

생각하고 배우는 법

Ways of thinking and learning

참가자들은 학습에 대해 새로운 방식으로 생각할 기회를 얻었습니다. 예를 들어, 참가자들은 자신의 학습 방향을 결정하는 능력과 책임에 대한 인식이 커지고 있다고보고했습니다.

participants benefited from having opportunities to think about their learning in new ways. For example, participants reported a growing awareness of their ability and responsibility to direct their own learning:

구체적이고 개인화 된 목표를 확인하고 일하도록work 요청 받으면 자신의 학습에 대한 책임감이 커지게 된 것 같습니다.

It seemed that being asked to identify and work towards specific, personalised goals fostered an increased sense of accountability towards their own learning.

그녀는 내가 구체적으로 어떤 조언을 받기를 원하는지, 어떤 것을 관찰하기를 원하는지 물어봤다.그래서 저는 이런 종류의 일들을 미리 생각해야했습니다.

she would ask me what do you specifically want me to give you advice on, what do you want me to observe you on. So then I had to think about these kind of things ahead of time.

몇몇 참가자들은 교수들과의 성찰을 통해 자신의 학습 목표를 성과지향 목표에서 (의미있는 목표라고 생각하는) 학습지향 목표로 향하는 방향으로 극적으로 reorient하고 있다고보고했습니다.

Several participants reported having dramatically reoriented their learning goals through reflection with their faculty members, either towards what they felt were more meaningful goals, or away from performance goals towards learning goals:

참가자들은 학습 목표에 대한 지속적인 토론을 통해 연중 진행 상황을 파악할 수있었습니다. 이러한 논의 과정에서 그들은 종종 특정한 목표가 달성되었다는 것을 알게되었습니다.

participants were able to reflect on their progress throughout the year through ongoing discussion of their learning goals. In the course of these discussions, they would often appreciate that a specific goal had been achieved.

참가자들은 종종이 프로그램의 경험을 임상 로테이션에서 받은 피드백과 비교했습니다.

Participants often compared their experiences in this programme with the feedback they usually received on clinical rotations.

교수진은 형성적 목적으로만 존재했으며 레지던트 평가에서 아무런 역할을하지 않았기 때문에 피드백은 평가와 얽매이지 않고 발전을 위한 지도로 보다 쉽게 ​​받아 들여지는 것처럼 보였습니다. 일부 레지던트 들은 임상 로테이션 중에 의도적으로 피드백을 찾기 시작했습니다.

Because the faculty member was present for formative purposes only and played no role in the resident’s assessment, the feedback appeared to be more readily accepted as developmental guidance, rather than being intertwined with assessment. Some residents began to purposely seek out feedback during their clinical rotations:

느끼는 법, 존재하는 법

Ways of feeling, ways of being

참가자들은 또한 여러 측면에서 정서적 웰빙의 중요한 이점을 설명했습니다. 몇몇 주민들은 교수 관찰자로부터 지지받는다는 느낌을 보였습니다.

Participants also described important benefits to many aspects of their emotional well-being. Several residents reported feeling supported by the faculty observer:

몇몇 참가자들은 교수진과의 연계가 중요한 이점이라고 보고했다. 일부 참가자의 경우, 이 프로그램 이외의 다른 참석 교직원과의 관계에 대해서도 새로운 관점을 갖게 되었습니다.

Several participants reported that establishing a connection with a faculty member is a significant benefit. For some participants, this translated into a new perspective on relationships with other attending faculty staff outside this programme.

교수진과의 상호 작용은 레지던트로 하여금 자신의 전반적인 수행능력에 대한 reassurance에 도움이되었습니다. 참여자들은 자신의 경험을 공유하고자하는 교수진과의 개방적이고 정직한 상호 작용이 자신의 업무 관련 불안정을 정상화시키는normalise 방법이라고 설명했습니다.

The interactions with faculty members helped to reassure participants about their overall performance as residents. Participants described how the open and honest interactions with faculty members who were willing to share their own experiences served to normalise their own work-related insecurities.

프로그램의 어떤 요소가 그러한 이점을 촉진했는가?

Which elements of the programme facilitated the benefits?

프로그램의 구조적 요소

Structured elements of the programme

직접 관찰은 이 프로그램의 중요한 요소 인 것으로 보였다. 대부분의 참가자들은 직접적인 관찰이 없었다면 발생하지 않았을 학습 기회를 제공하는 상황을 설명했습니다.

Direct observation appeared to be a critical element of this programme. Most participants described situations in which direct observation afforded a learning opportunity that would not otherwise have occurred.

참가자들은 일상의 work를 관찰받으며 얻은 가치에 대해 설명했습니다.

Participants described the value they placed on having their day-to-day work observed.

참가자들은 또한 구체적인 학습 목표에 맞게 피드백을 받는 것이 프로그램의 유용한 요소라고보고했습니다. 참가자들은 교직원에게 관심을 준 것에 감사해했다.

Participants also reported that receiving feedback personalised to their specific learning goals was a helpful element of the programme. Participants appreciated having the faculty member focus attention on them: 

레지던트들은 또한 피드백이 어떻게 [교수진과 자신 사이의 대화]로서 발전하게 된 것을 감사해했다.

Residents also appreciated how feedback developed as a conversation between the faculty member and themselves:

피드백 프로그램 내에서, 각 레지던트 - 교수진 쌍은 1 년 내내 일정하게 유지되었습니다. 참가자들은 이 종단적 관계를 두 가지 측면에서 유익하다고 설명했습니다. 

• (i) 동일한 교수진과의 만남은 개인화되고 구체적인 피드백을 촉진했으며, 

• (ii)이 피드백은 일반적인 임상 로테이션에서 제공되는 피드백보다 더 많은 가치와 신뢰성을 갖는 것처럼 보였습니다.

Within the feedback programme, each resident–faculty pair remained constant for the entire year. Participants described this longitudinal relationship as beneficial on two fronts: 

• (i) meeting with the same faculty member fostered feedback that was personalised and specific, and 

• (ii) this feedback seemed to have more value and credibility than the feedback delivered on usual clinical rotations.

주치의 한 명에게 종단적으로 노출되었기 때문에, 참가자들도 피드백에 반응하여 개선되었음을 보여줄 시간을 가질 수있었습니다.

The longitudinal exposure to one attending physician also afforded participants the time to respond to feedback and to demonstrate improvement.

많은 레지던트들은 또한 피드백이 형성적 목표만을 갖는다는 것을 알게 되었고, 이것이 프로그램의 유용한 요소임을 확인했습니다. 어떤 사람들은 총괄평가가 없었다고 평하면서, 그랬기 때문에 평상시처럼 편안하게 일할 수 있다고 설명했다.

Many residents also identified that feedback which occurred solely for formative purposes, in the absence of any assessment that could be forwarded to the postgraduate training programme, was a helpful element of the programme. Some described the lack of summative assessment as helping them to feel comfortable in doing what they would normally do:

대인관계적 요소

Interpersonal elements

의미있는 피드백 제공을 지원하는 것으로 보이는 다른 요소로는 피드백의 대인 관계 측면과 관련된 것들이 있었다. 몇몇 참가자는 교수 관찰자의 퀄리티 덕분에 경험으로부터 더 많은 것을 얻을 수 있었다고 느꼈다. 관찰자의 두 가지 특성, 즉 'relaxed'또는 'easygoing'의 특성이 특히 중요한 것으로 보였다.

Other elements that appeared to support the provision of meaningful feedback related to interpersonal aspects of feedback. Several participants felt that qualities in the faculty observers helped them get more out of the experience. Two such characteristics in observers, namely, those of being ‘relaxed’ or ‘easygoing’, seemed to be critical.

교수진의 자질에 덧붙여, 레지던트들은 의미있는 피드백을 만들어낸 관계의 특성을 설명했습니다. 대부분의 경우, 레지던트들은 다른 감독자와 경험하지 못한 방식으로 '보살핌을받는'느낌을 묘사했습니다. 레지던트들은 또한 이러한 관계가 다른 주치의들과 일반적으로 경험했던 것보다 더 평등 한 토대footing로 발전하여보다 정직한 대화를 할 수 있다고 생각했습니다.

In addition to qualities in the faculty member, residents described qualities of the relationship that promoted meaningful feedback. For the most part, residents described feeling ‘cared for’ in ways they did not experience with other supervisors. Residents also felt that these relationships developed on a more equal footing than those they typically experienced with attending physicians and hence allowed for more honest conversation.

많은 피드백 대화가 레지던트가 어떻게 개선 될 수 있는지에 초점을 맞추었지만 참여자들은 긍정적인 격려를 받는 것도 매우 도움이된다고 설명했습니다. 참가자들은 교수진 관찰자로부터 긍정적 강화를 받으며 더 많은 피드백을 얻기 위해 자신의 업무와 자신감에 자신감을 가질 수 있다고보고했습니다.

Although many of the feedback conversations focused on how the resident could improve, participants also described the provision of positive encouragement as very helpful. Participants reported receiving positive reinforcement from their faculty observers, allowing them to feel more confident in their work and in their drive to seek out more feedback.

Logistical considerations

특정 시기에 관측이 특히 의미있는 피드백을 내는 듯 했다. 예를 들어, 한 레지던트는 시니어가 되기 시작한 학년 말에 관찰을 설명했습니다.

There were also specific times in the year during which observations seemed to yield particularly meaningful feedback. For example, one resident described an observation that took place towards the end of the academic year as she was beginning to make the transition to becoming a senior resident:

이 프로그램의 시간-집약적 성격을 감안할 때, 우리는 레지던트 참여자들과의 관찰 빈도를 조사하는 것이 중요하다고 생각했습니다. 의견은 다양했지만 일반적으로는 연간 3 ~ 6 건의 관찰이 최적 일 것이라는 것이었다.

Given the time-intensive nature of this programme, we thought it important to explore the frequency of observations with resident participants. Although opinions varied, the general sense was that somewhere between three and six observations per year would be optimal.

프로그램의 한계와 개선점은?

What were the limitations and challenges of the programme?

로지스틱 문제. 예를 들어 참가자들은 바쁜 일정과 시간 부족으로 인해 교수진과 임상 관찰 시간을 정하기가 어렵다는 것을 발견했습니다.

logistical issues. For example, participants sometimes found it difficult to set up clinical observations with their faculty member as a result of busy schedules and lack of time.

다른 중요한 한계는 직접적인 관찰이 임상 맥락에서 생소하다고 여겨졌다는 것이다. 이 프로그램에 등록하기 전에 대부분의 참가자는 일상적인 임상 연구에서 정기적으로 observed 받지 않았습니다.

The other significant limitation was that direct observation was seen as unfamiliar in the clinical context. Prior to enrolment within this programme, most participants had not been observed on a regular basis in their day-to-day clinical work.

고찰

DISCUSSION

기존 연구에서는 

• 직접적인 관찰의 부족, 

• 의미있는 피드백을 유발할 수있는 교직원과 레지던트 간의 종단적 관계의 결핍 

• 교수-레지던트 관계에서 느껴지는 총괄평가적인 느낌

...등이 임상 학습 환경의 문화적 제한요소가 되어 레지던트들이 피드백이 덜 의미있게 느끼게 되었다고 제안했다.

prior research suggested that feedback was made less meaningful to residents by limitations in the culture of the clinical learning environment, including 

• a lack of direct observation, 

• a lack of longitudinal relationships between faculty members and residents that could foster meaningful feedback,

• and the sense of summative assessment that loomed over faculty–resident interactions.

• 레지던트들은 임상 기술, 의사 소통, 임상 적 추론, 조직화 기술과 같은 고등 기술에서 이점을 설명했습니다. 

• 레지던트는 학습자로서의 자기 인식의 변화를보고했다. 

• 그들은 학습 목표의 생성과 피드백 추구 행동에 대한 명백한 증거로서 자신의 학습에 대해 더 많은 책임을 져야한다는 증거를 보여주었습니다. 

• 레지던트들은 또한 정서적 웰빙과 보살핌과 연결성에 대한 경험을 보고했으며 

• 일부는 일반적으로 주치의 교수를 바라보는 방법을 바꾸었습니다.

residents described benefits in their clinical skills, and in higher-order skills such as communication, clinical reasoning and organisation skills. Residents reported changes in their self-perceptions as learners. They demonstrated evidence of becoming more accountable for their own learning as manifest in their generation of learning goals and their feedback-seeking behaviours . Residents also reported benefits to their emotional well-being and the experiencing of a sense of caring and connectedness that, for some, shifted the way in which they viewed attending faculty members in general.

우리의 분석은 [의미있는 피드백 생성에 중요하다고 잘 묘사 된] 간단한 직접 관찰보다 더 큰 범위에서, 실제의 일일 관찰은 연수생에게 특히 중요하며 의미있는 피드백 대화를 용이하게한다는 것을 보여줍니다.

Our analysis demonstrates that to a greater extent than simple direct observation, which is well described as important to the generation of meaningful feedback,6,19 the observation of authentic day-to-day work is especially valuable to trainees and facilitates meaningful feedback conversations.

우리의 분석은 프로세스의 총괄적 평가를 제거하는 것이 직접적인 관측과 그에 따른 피드백의 진정성에 기여했다는 것을 암시합니다. 레지던트의 일상 업무는 로테이션이 끝날 때 자신을 평가할 사람의 감독하에 수행됩니다 .24 이러한 환경에서 레지던트들은 자신에게 권력power를 갖는 사람들을 감동시키고자하는 경향이 있고, 이 때문에 학습에 '성과performance 지향'을 갖게 된다. 이것은 대개 학습 목표를 우선시하는 사고 방식 인 '학습 지향(learning orientation)'과는 확연한 차이가 있습니다 .25)

Our analysis suggests that removing summative assessment from the process contributed to the authenticity of the direct observations and the resulting feedback. A resident’s daily work is carried out under the supervision of the person who will be assessing him or her at the end of the rotation.24 In this environment, there is a tendency for residents to want to impress those who have power over them, creating a ‘performance orientation’ to learning. This is usually at odds with a ‘learning orientation’, a mindset in which learners prioritise learning goals, even at the cost of revealing performance deficits to a teacher.25

이 프로그램에서 교수진과 정기적 인 면담을 통해 목표를 검토하고 진행 상황을 지적하고 새로운 목표를 수립 할 수있는 공간을 마련했습니다. 피드백 대화는 레지던트의 

• 구체적인 학습 목표에 일상적으로 영향을 미쳤으며, 

• 자신의 구체적인 요구를 고려하게 해주었고

• 피드백에 대해 논의 할 수있는 기회를 만들었습니다. 

이러한 과정에 참여함으로써 많은 참가자들은 임상 실습을 통해 새로운 작업 방식과 학습 방식을 개발하는 것처럼 보였습니다.

In this programme, regular meetings with faculty staff created a space for reviewing goals, noting progress and creating new goals. The feedback conversations 

• routinely touched on the resident’s specific learning objectives, 

• took his or her specific needs into consideration, and 

• created an opportunity for the feedback to be discussed. 

By engaging in these processes, many participants seemed to develop new ways of working and learning through their clinical practice

우리 프로그램의 또 다른 주요 구성 요소는 그 해 동안 길러진 종단적 교수-레지던트 관계였습니다. 연구에서 확인할 수 있었던 이익은 관찰의 상대적 빈도가 높지 않다는 점(1 년에 2 ~ 5 회)를 감안할 때 특히 두드러졌습니다. 의견의 효과성에 대한 관계의 핵심은 LIC에서 확인된 많은 문헌과 일치합니다.

Another major structural component of our programme was the longitudinal faculty–resident relationship, nurtured over the course of an academic year. The reported benefits were particularly striking given the relative infrequency of observations (two to five over the course of the year). The centrality of the relationship to the effectiveness of the feedback is consistent with much of the literature emerging from longitudinal integrated clerkships.7,22,23

Limitations

감독하는 교수진이 연구 조사원 중 3 명을 대표한다는 점에 유의하는 것이 중요합니다. 이 교수진이 현장 노트를 기록하고 감독 한 거주자들과 개인 인터뷰를 실시하여 잠재적으로 우리의 관찰과 분석에 편향을 도입했습니다. 이러한 한계점을 극복하기 위해 우리는 분석 단계에서 지속적인 반사성을 발휘하여 각자의 관점과 편견을 드러내고 해결할 수있었습니다 .31 팀에 '내부자'와 '외부인'을 포함시키는 것이 더 큰 시각과 균형을 제공했습니다.

It is important to note that the supervising faculty members represented three of the study researchers. That these faculty members recorded field notes and conducted individual interviews with residents they supervised potentially introduces bias into our observations and analyses. To counter this limitation, we engaged in constant reflexivity during the analysis phase so that our respective perspectives and biases could be surfaced and addressed.31 Including ‘insiders’ and ‘outsiders’ on the team offered greater perspective and balance.

Implications for research

의미있는 피드백을 유발하는 데 필요한 최소한의 관찰 및 기간은 얼마입니까? 

짧은 기간 내에 교육 동맹을 창출 할 수있는 방법을 개발할 수 있습니까?

what are the minimum number and duration of observations required to foster meaningful feedback? 

Can methods be developed that create an educational alliance in a shorter timeframe?

Implications for educational practice

교직원이 학습자에 관심을 갖고, 그들의 학습목표에 초점을 둠으로써 감독자와 학습자 간의 교육 동맹을 창출할 수만 있다면, 최소한의 접촉 (즉, 여기서는 2 ~ 5 회의 관찰)로도 돌봄과 지지적 관계가 만들어질 수 있는 것으로 보인다.

Caring and supportive relationships can be established with what might be perceived as a minimal amount of contact (i.e. herein, just two to five observations per year) if the faculty member is able to take an interest in the learner and to focus on his or her learning goals, thus creating an educational alliance between supervisor and learner.

이 피드백 프로그램을 이상적인 조건의 파일럿 연구에서 대규모 구현에 이르기까지 해결해야하는 로지스틱 문제를 제시합니다. 관찰이 이루어질 수 있도록 로지스틱 장벽을 극복하고자하는 교수집단을 만들기 위해서는 교수개발이 필요합니다. 교직원과 레지던트 간의 일정 계획을 돕기 위해 행정적administrative 지원이 필요할 수 있습니다. 분산 된 훈련 프로그램은 몇 가지 훈련 사이트를 중심으로하는 것보다 더 많은 논리적 문제가있을 것이다. 교수진은 시간과 노력에 대한 보상을 요구합니다.

Scaling this feedback programme from a pilot study, with ideal conditions, to large-scale implementation presents logistical issues that require resolution. Faculty development will be necessary to develop a cadre of engaged faculty members willing to overcome logistical barriers to ensure observation occurs. Administrative support may be required to assist with scheduling between faculty members and residents; distributed training programmes will have more logistical issues than those centred in a few training sites. Faculty members will require remuneration for their time and effort.

15 Brydges R, Butler D. A reflective analysis of medical education research on self-regulation in learning and practice. Med Educ 2012;46 (1):71–9.

---

Med Educ. 2016 Sep;50(9):943-54. doi: 10.1111/medu.13067.

Investigating conditions for meaningful feedback in the context of an evidence-based feedbackprogramme.

Voyer S1, Cuncic C1, Butler DL2, MacNeil K2, Watling C3, Hatala R1.

Author information

1

Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada.

2

Department of Educational and Counselling Psychology, and Special Education, Faculty of Education, University of British Columbia, Vancouver, British Columbia, Canada.

3

Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, University of Western Ontario, London, Ontario, Canada.

Abstract

CONTEXT:

We developed, implemented and evaluated an evidence-based programme of feedback designed to address limitations identified in the current literature.

OBJECTIVES:

We sought to advance understanding about how and why feedback processes might be more effective in clinical education.

METHODS:

Three faculty members and nine first-year internal medicine residents participated in the pilot programme. To counter challenges identified in the literature, feedback was based on direct observation, grounded in longitudinal faculty-resident relationships, and devoid of summative assessment. We used a qualitative case study design to address three research questions: (i) What benefits did the participants describe? (ii) What elements of the programme facilitated these benefits? (iii) What were the limitations and challenges of the programme? Collected data included audiotapes of interactions between faculty members and residents, field notes written during observations, and semi-structured interviews and focus groups with resident participants. Data analysis moved cyclically and iteratively through inductive and deductive analysis.

RESULTS:

Residents described benefits relating to their ways of working (clinical skills), ways of learning (accountability for learning) and ways of feeling (emotional well-being). According to participants, specific elements of the programme that achieved these benefits included the direct observation of authentic clinical work, the longitudinal relationship with a faculty member and the emergence of feedback as a conversation between the faculty member and learner.

CONCLUSIONS:

We conclude that the conditions established within our pilot feedback programme influenced the learning culture for first-year internal medicine residents by grounding direct observation in authentic clinical work and setting the observations in the context of a longitudinal, non-assessment-based relationship between a faculty member and resident. These conditions appeared to influence residents' participation in the feedback process, their ways of approaching their daily clinical work, their emotional well-being and their engagement in their own learning.

© 2016 John Wiley & Sons Ltd and The Association for the Study of Medical Education.

PMID:

 

27562894

 

DOI:

 

10.1111/medu.13067

[Indexed for MEDLINE]