임상교육에서의 피드백(Do's and Don'ts) (Perspect Med Educ. 2015)

임상교육에서의 피드백(Do's and Don'ts) (Perspect Med Educ. 2015)

Guidelines: the do’s, don’ts and don’t knows of feedback for clinical education

Janet Lefroy · Chris Watling · Pim W. Teunissen ·

Paul Brand

Do’s—educational activity for which there is evidence of efficacy

Don’ts—educational activity for which there is evidence of no efficacy or of harms (negative effects)

Don’t Knows—educational activity for which there is no evidence of efficacy

Table 1 Summary of guidelines. For the individual clinical supervisor giving feedback

Introduction

요약 표

In the summary (Table 1) we indicate the strength of this evidence and therefore of our recommenda- tion using the criteria outlined in Table 2.

어떤 피드백이 유용할지를 알기는 쉽지 않다. (주는 것과 받는 것 사이의) 'feedback gap'이 있다. 이것이 supervisor에게 의미하는 것은 trainee의 요구와 수용도를 진단하지 않고 피드백을 주는 것은 노력낭비의 가능성이 있다. formative feedback의 효과는 trainee가 얼마나 스스로 향상되고자 하는지와 그 자신감의 강도에 달려있다.

It is not easy to know what feedback will be useful to a trainee. There is a recognized feedback gap (between feed- back given and what is received by the trainee [1]). What this means for supervisors is that delivering feedback with-out  first  diagnosing  our  trainee’s  need  and  receptiveness risks wasting effort. The impact of formative feedback will depend on the strength of the trainee’s desire to improve and their confidence in their ability to do so [2].

Foundation paper

Methods and ‘way of working’

결과

Results

 

Background evidence to guidelines for the individual clinical supervisor giving feedback

프로세스에 관한 Do's

Do’s for the process of feedback

1. 피드백이 단순히 한 사람이 다른사람에게 도움을 주기 위한 정보를 주는 것이라 생각하면 안된다. 피드백은 문화/가치/기대/개인적 역사/관계/권위에 의해 영향을 받는 사회적 상호작용이다. 피드백을 conversation이 아니라 commodity라고 보면 안된다.

Guideline 1. Do realize that feedback is not just one per- son providing information to another to help them improve. Feedback is part of a social interaction influenced by cul- ture, values, expectations, personal histories, relationships, and power. Do treat feedback as a conversation rather than as a commodity.

부정적인 피드백을 어떻게 Reconciling and assimilating 할 것인지는 사회적 맥락에 따라 달라진다. 피드백의 효과와 역할에 대한 사회적/문화적 가치를 강조한 문헌도 있다. 피드백을 단지 'trainee의 수행을 향상시키기 위한 목적으로 trainee의 관측된 수행능력을 표준적 기준에 대해서 비교한 정보를 제공하는 것'으로 바라볼 때, 피드백을 통해 드러나는 문화/가치/기대/역사/관계/권위를 무시하는 것이다.

Reconciling and assimilating negative feedback with views held by the individual was found to be influ- enced by social context. The differences between professions described in that study highlight the influence of social and cultural values on the role and impact of feedback [4]. Viewing feedback only as ‘specific information about the comparison between a trainee’s observed performance and a standard, given with the intent to improve the trainee’s per- formance’ [5] ignores the complex ways in which culture, values, expectations, personal histories, relationships, and power manifest themselves through feedback [6].

2. 피드백이 효과가 있으려면 Trainee는 반드시 피드백을 믿을 만credible하다고 믿어야 한다. Credible한 피드백은 직접적 관찰을 통해서 well-informed되어있어야 하며, 피드백이 신뢰할만한trustworthy 곳에서 와야 한다. Supervisor로서 신뢰할만한 롤모델이 되어야 한다.

Guideline 2. Do recognize that trainees must perceive feed- back as credible in order for it to be influential. Credible feedback is well-informed, typically by direct observation of the task or event, and it comes from a trustworthy source. Make sure that you as supervisor set a good example as a credible role model.

학습자는 credible한 사람의 피드백은 가치있게 여기지만, credibility가 낮다고 생각되는 피드백은 무시하는 성향이 있다. 피드백의 credibility는

• 그 출처의 credibility,

• 그 피드백이 informed and created하는 프로세스,

• 피드백의 내용과 특성 그 자체

등에 의해 영향을 받는다.

A number of qualitative studies have shown that learners value feedback that they deem to be credible, but may dis- miss feedback that they perceive to lack credibility [7–10]. The credibility of feedback is influenced by the credibility of the source, by the process by which the feedback was informed and created, and by the content and characteristics of the feedback itself [9].

3. Trainee의 역량수준과 과제의 복잡도에 따라 피드백의 타이밍을 정하라

Guideline 3. Decide the timing of feedback depending on the competence level of the trainee and on the complexity of the task.

적절한 시간에 제공해야 피드백의 수용도와 활용이 높아진다. 그러나 '적절한 시간'의 의미는 모호하다. 예컨대 술기의 시뮬레이션 교육에서는 terminal feedback이 concurrent feedback보다 낫다.

Studies of learners’ perceptions of effective feedback have highlighted the importance of timeliness to learners’ acceptance and use of feedback [7, 11], the concept of optimal timeliness appears to be a nuanced one. For exam-ple, for simulation training of procedural skills, terminal feedback (at the end of the task performance) may be supe-rior to concurrent feedback (during the task performance) for enhancing learning [12]. 

타이밍에 따라 서로 다른 레벨의 피드백이 필요하다

Hattie and Timperley [13] provide evidence that differentlevels of feedback deserve different timing.

4. Trainee가 피드백을 찾아나서게끔, 그리고 그것을 자신의 퍼포먼스 향상에 사용하게끔 장려하라.

Guideline 4. Do encourage trainees to look for feedback and use it to enhance their performance.

자존감에 영향을 받을까봐 피드백을 두려워한다. trainer에게 좋은 인상을 남기고 싶을 수도 있고, 어떻게 향상되어야 할지에 대한 피드백 정보를 원할 수도 있다. 피드백-탐색에는 복잡하고 방대한 무의식적 심리가 있다.

Our trainees may approach feedback with trepidation about the harm it might do to their self-esteem [15]; they may desire to make a good impression on their trainer among others; they may also desire the information which feedback gives them about how to improve [16]. These are the com- plex and largely unconscious psychological influences on feedback-seeking [17].

Teunissen 는 목표지향성이 뚜렷할수록 피드백 탐색의 빈도가 더 높아짐을 보여주었다.

Teunissen et al. showed that this relationship between goal orientation and increased fre- quency of feedback seeking also holds in a population of postgraduate medical trainees [20].

내용에 관한 피드백의 Do's

Do’s for the content of feedback

5. 개별 trainee에 맞춤형 피드백을 하라. Trainee는 다음의 것에 도움을 받을 것이다.

Guideline 5. Do tailor bespoke feedback to the individual trainee. The trainee might benefit from:

• 잘 된 부분을 강화 ‘reinforcement of key points done well’;

• 빼놓거나 개선되어야 할 부분을 지적 ‘identification of key points which might have been done better or omissions’;

• 퀄리티 향상을 위한 전략 찾기 ‘working out strategies for improving the quality of their work’;

• 자기-인식 향상 ‘an increased self-awareness’

 

피드백은 '만족스럽지 못한 요소'에 대해서 제공될 때, 구체적인 학습 목표와 연결될 때 가장 효과적이다. 따라서 피드백의 내용은 supervisor와 trainee간 진단적, 지지적 대화에서 arise해야 한다.

Feedback is most effective if directed at unsatisfactory ele- ments of performance and linked to specific learning aims [23]. The content of feedback should therefore arise from a diagnostic and supportive dialogue between supervisor and trainee [24].

6. 어떻게 과제가 수행되었고, 어떻게 수행되었어야하는지에 대한 구체적 피드백을 주라

Guideline 6. Do give specific feedback, focused on how the task was done and how that type of task should/might be done.

퍼포먼스와 관련이 없는 일반적 정보 (좋은 퍼포먼스는 무엇이고 나쁜 퍼포먼스는 무엇인지) 등은 구체적인 코멘트보다 효과가 떨어진다. 구체성이 부족한 피드백은 medical training에서 오래 전부터 문제로 지적되어왔다. 그러나 피드백의 구체성을 향상시키는 것이 더 효과적인 학습을 가져왔다는 근거를 찾는 것도 모호하다. Goodman 등은 피드백의 구체성을 향상시키는 것이 초기 퍼포먼스에는 도움이 되었지만, exploration을 discourage하여 independent performance에 필요한 더 심화된 학습은 훼손하는 것으로 나타났다.

General infor- mation unrelated to the performance, comments about a good or poor performance or compliments are less effec- tive than specific comments [27, 28]. Lack of specificity has repeatedly been identified as an all-too-common weak- ness of the feedback that is typically exchanged in medi- cal training [29]. When, however, one looks for evidence that increasing feedback specificity leads to more effective learning, the waters become murkier. Goodman et al. [30], for example, showed that increasing the specificity of feed- back benefits initial performance, but discourages explora- tion, potentially undermining the deeper learning required for independent performance.

Kluger and DeNisi의 피드백 이론은 task에서 멀어지고 individual에 대한 피드백은 덜 효과적이라고 지적한다. 줄이면, 자존감에 위협이 되는 피드백은 도움이 되지 않는다.

Kluger and DeNisi’s feedback intervention theory, also posits that feedback becomes less effective as attention shifts away from the task and toward the individual; in short, feedback that is threatening to self-esteem is unlikely to be effective [28].

요약하자면, 피드백은 task level에서 이뤄질 때 가장 효과적이고, 자기조절이나 프로세스에 대한 피드백은 'deep processing and mastery of tasks'에 도움이 될 수 있다. 그러나 'person level'에 맞춘 피드백은 하지 마라.

To sum up feedback is most effective when directed at the task level and may assist in ‘deep processing and mastery of tasks’ when it is about processing of tasks or self-regulation. A ‘Don’t’ is providing feedback that focuses on the person level.

 

7. 피드백이 '최소한의 역량수준 달성'을 위한 것인지, '최적의 수행능력과 비교한' possible variation에 대한 성찰인지를 명확히 하라

Guideline 7. Do make sure to indicate whether feedback is about necessary improvement for minimally acceptable per- formance or whether it is a reflection on possible vari ations to build upon adequate performance.

자기조절이론에서는 두 개의 기본적 자기조절시스템이 공존하면서 상충한다고 말한다. 두 개의 시스템은 promotion- 그리고 prevention- 접근법이다. supervisor는 자신의 trainee가 promotion focus인지 prevention focus인지 파악해야 한다.

Self-regulation theories suggest that within each of our trainees are two basic self-regulation systems which co- exist but may conflict [33]. These two systems—the pro- motion (doing things because you want to) and prevention (doing things because you have to in order to avoid harm) approaches—may both be active in response to feedback [34]. It is important that the supervisor recognizes that his/ her trainee is predominantly in promotion or prevention focus with respect to the focus of feedback, because

• positive feedback is more effective in motivating performance improvement for learners in promotion focus, while

• negative feedback is more useful in motivating performance improvement for learners in prevention focus [28].

Prevention system은 performance goal을 가지는 사람들에게 왕성하다. 이들은 '난 이미 충분한 실력을 갖췄음'을 증명하고 싶어하며, 비판을 받고싶지 않아한다. Trainee가 (performance goal을 가지고 있을 때보다) learning goal 을 가지고 있을 때 피드백이 더 잘 작동한다. 따라서 피드백 그 자체가 trainee를 performance goal을 가지게 압력을 주어서는 안된다.

The prevention system is active in individuals with performance goals—aiming to prove that one is already adequately competent and avoiding criticism. Feedback works best for learning when the trainee has learning goals rather than performance goals [17, 35] so it is important that the feedback itself should not push the trainee towards performance goals.

성적은 피드백이 promotion and prevention response 모두를 자극하는 방식이다. 이러한 심리상태에서 성적을 받는 것은 향상전략을 탐구하는 행동을 강화해주는 것으로 나타났으며, 특히 성적부여의 기준이 확실할 때 그러하다. 성적의 나쁜 효과도 나타나곤 하는데, 고등교육과 의학교육에서 formative feedback에서는 grade를 optional하게 만들 것을 제안한다.

Grades are a clear and non-nuanced form of feedback which can trigger both promotion and prevention responses in trainees [28]. Receiv- ing grades in this frame of mind was found to enhance the seeking of strategies for improvement, especially if criteria for allocation of grades are understood [26, 36]. Harmful effects of grades have also been noted in some participants in school, higher education and medical education [13, 26, 34], suggesting that making grades optional in formative feedback may be wise,

 

8. Actionable한 피드백을 줘서 Trainee가 개선을 위한 전략을 수립할 수 있게 하라. Trainee의 수행능력에 대해 토론한 후, 현재 역량을 넘어설 수 있게 하는 '가이드'나 'scaffolding'을 제공하라

Guideline 8. Do ensure that feedback is actionable, enabling the trainee to construct strategies for improve- ment. After discussing the trainee’s performance of a task, provide some guidance or ‘scaffolding’ to enable them to step beyond their current competence.

학습자의 피드백 경험에 대한 연구는 피드백은 actionable할 때 그 가치가 높아짐을 강조한다. Actionable한 피드백은 학습자 발달의 로드맵을 보여주며, 강점을 강화하고 약점을 보완하기 위한 명확한 suggestion을 준다.

Research into learners’ experiences of feedback has highlighted the value placed on feedback that is actionable [38]. Actionable feedback contains a roadmap for learner development; it provides explicit suggestions for building on strengths or addressing weaknesses in performance.

 

‘scaffolding’ 의 이론적 개념은 Wood의 '학습의 구성주의자 모델'에서 잘 개발되었다. 이들은 Vygotsky의 ‘zone of proximal development’ 에 근간을 두고 있다.

The theoretical concept of ‘scaffolding’ by tutors has been well developed by Wood et al. in their constructivist model of learning [39]. They based this on Vygotsky’s many studies in children of how the learner is helped to develop into their ‘zone of proximal development’ (beyond their current ability) by social interaction with tutors or peers [40].

성인학습의 사회적 상호작용에서도 scaffolding 개념이 도움이 된다.

In the social interactions of adult learning the scaffolding concept can also be helpful [27, 41–43].

Wood가 말한 scaffolding은..

The tasks of scaffolding as described by Wood et al. are:

● Orient the learner to the task

● Simplify into steps

● Motivate to maintain effort to achieve the goal

● Highlight critical features of the task

● Control frustration and the risk of failure

● Provide a model of the required actions

역량수준이 낮은 trainee에게 scaffolding은 directive feedback을 주거나 specific instruction을 주는 것이 될 수 있다. 높은 수준의 trainee에게는 less directive 할 수 있다. 즉 Suggestion, hints, tips 등이 그것이다.

For trainees with a low level of competence, scaffolding involves giving directive feedback or specific instructions; for trainees with a high level of competence scaffolding can be less directive i.e. suggestions, hints and tips for (further) improvement (facilitative feedback) [27]. 

9. 개선전략을 토의할 때 Trainee motivation에 관심을 가지라

Guideline 9. Do attend to trainee motivation when discussing strategies for improvement.

성인학습자에게 동기부여는 주로 '내부에서 형성'된다. 그리고 이는 피드백에 의해서 영향을 받는다. 충분히 동기부여가 되고 충분한 자기효능감을 위해서 supervisor은 피드백을 주면서 trainee의 반응을 보아야 한다.

In adult learners, motivation is more likely to be internally generated [44] but it is no less important to learning, and is influenced by feedback [28, 34, 45]. In aiming for sufficient motivation to learn to do the task and sufficient self-efficacy that their effort is likely to succeed, clinical supervisors should check trainee response to their feedback as they go along.

10. 피드백을 주는 구체적인 전략에 무관하게, trainee를 '성찰적 대화'에 참여하게 해서 '자기평가'와 'Supervisor의 관측&설명'을 같이 생각해볼 수 있게 하라

Guideline 10. Regardless of the specific approach to feedback that is used, do engage the trainee in a reflective conversation that marries their self-assessment with your observations and elaborations.

 

trainee를 토론에 참여시킴으로서, supervisor는 reflection-in-action and reflection-on-action을 활용하여 자신의 퍼포먼스에 대해 인식하게끔 할 수 있다. 그 이후의 코칭은 trainee의 자기평가에 대한 confirming 또는 challenging이 될 수 있다.

By involving trainees in a discussion, supervisors can raise their awareness of their performance relative to their goals of quality performance through reflection-in-action and reflection-on-action [22, 32]. Coaching then includes confirming or challenging the trainee’s self-assessment,

 

피드백의 효과에 영향을 주는 것은 많은데, 어떻게 피드백을 전달할 것인가는 task, recipient, relationship에 따라 달라질 수 있다. 피드백은 ‘A supported sequential process rather than a series of unrelated events’가 되어야 한다.

There are many factors which influence the effect of feedback, and the choice of how to deliver the feedback will depend on the task, the recipient and the feedback relationship [23]. Feedback should be ‘A supported sequential process rather than a series of unrelated events’ [32].

여러 피드백의 방법이 있으나, 뭐 하나가 가장 효과적인 것은 없다.

Several approaches to feedback have been described in the literature (sandwich, Pendleton, reflective feedback conversation, agenda-led outcome-based analysis, feedfor- ward), but no single approach has been established to be the most effective.

 

샌드위치

The feedback sandwich (in which the supervisor describes

• 잘 된것 what went well,

• 개선할 것 what can be improved,

• 잘 된 것 then re-emphasizes what went well) [46]

 

칭찬의 심리적 효과를 활용하여 비판이 더 잘 수용되게 한다. 이러한 접근법은 피드백 관계의 미묘한 초반 관계에서 효과적이지만, 일단 관계가 확고하면 불필요할 수 있다.

harnesses the psychological effect of praise to enable the reception of criticism. This approach is thought helpful especially in the delicate start of a feedback relationship, but unnecessary once the relationship is robust. Evidence of its effectiveness is lacking. 

펜들톤

Pendleton [47] outlined a method for giving feedback aiming to engage the learner in self-reflection and to bal- ance positive and critical feedback.

● Check the learner wants and is ready for feedback. 

● Let the learner give comments/background to the mate- rial that is being assessed. 

● The learner states what was done well. 

● The observer(s) state(s) what was done well. 

● The learner states what could be improved.

● The observer(s) state(s) how it could be improved. 

● An action plan for improvement is made.

이 규칙은 투박하다거나 정형화되었다고 비판받으나, 피드백을 주고받는 것에 도움이 된다.

The rules have been criticized as clunky and formulaic, but the framework can be helpful for learning to give and receive feedback.

 

성찰적 피드백 대화

The reflective feedback conversation unfolds like this: 

● The teacher asks the learner to share concerns about performance. 

● The learner describes concerns and what they would have liked to have done better. 

● The teacher provides views and offers support, then asks the learner what might improve the situation. 

● The learner responds, then the teacher elaborates on that response, correcting if needed, and checking understanding.

대화는 '프로세스'이며 'event'가 아니다. Revisiting과 F/U이 필요하다.

Importantly, the conversation should be viewed as a process rather than an event; revisit- ing and follow-up are often required.

 

Agenda-led outcome-based analysis

Agenda-led outcome-based analysis

• starts with the trainee’s agenda,

• looks at the outcomes they were aiming for,

• encourages self-assessment and problem-solving,

• pro- vides balanced feedback and suggests alternatives

 

학습자 중심적 방법

This method is described in Kurtz, Silverman and Draper’s Cal- gary Cambridge method for teaching communication skills [49] and is a learner-centred way of identifying the most helpful focus for a feedback discussion. 

feedforward interview

피드백은 아니다. 'best performance'에 초점을 두어 preferred standard 와 the actual state 의 차이를 회피하기 위한 전략.

By contrast, the feedforward interview [34] is not actu- ally a technique for feedback. It aims to avoid creating a dis- crepancy between a preferred standard and the actual state of affairs (seen as a key element of feedback, but also recog- nized as problematic for trainees who have low self-esteem) by focusing learners on their best performances.

• The trainee recalls peak moments in his/her performance and

• is asked to reflect on what conditions in themselves and their surround- ings made that possible,

• then considers strategies to ensure sustainable peak performance.

 

Trainee에게 peak experience를 정기적으로 feedforward하여 'promotion'접근법에 필요한 ground를 마련하라고 제안함.

Kluger and van Dijk recom- mend periodical feedforward interviews with trainees about their peak experiences, partly in order to prepare the ground for necessary feedback to be received with a ‘promotion’ approach.

하지 말 것

Don’t assume

11. 한 방법이 늘 작동할거라고는 생각하지 말라. Player와 Context가 변화하면 가장 유용한 접근법도 변화한다.

Guideline 11. Don’t assume that a single approach to feedback will be effective with all trainees or in all circum- stances. As the players and the contexts change, so too does the most useful approach to feedback.

 

 

● You know what a trainee wants to learn 

● You know why a trainee is struggling 

● You know if or why a trainee wants feedback 

● You know what information a trainee takes out of a situ- ation or feedback conversation

 

임상과제나 교육과제에 따라 사람마다 매우 다르다. 피드백에 대한 반응도 학습자마다 다르고, 수행능력이 비슷해도 다를 수 있다. Dijksterhuis 는 개인간 피드백의 수용도와 반응도 차이를 보여주었고 Kluger and van Dijk는 regulatory focus theory가 이러한 차이의 원인을 설명해줄지도 모른다고 했다. Watling등의 임상학습환경에서의 연구

Individuals vary in their orientation toward clinical and edu- cational tasks. Responses to feedback also differ between learners, even regarding similar performance on similar tasks. Dijksterhuis showed individual variability in the acceptance and responsiveness to feedback [50]. Kluger and van Dijk [34] proposed that regulatory focus theory might explain some of the observed variability in feedback responses, and Watling et al.’s naturalistic exploration of the usefulness of this theory showed it offered some insights into feedback responses in clinical learning situations [25].

피드백의 impact에 대한 차이는 개인간 차이를 넘어선다. 피드백에 대한 반응은 학습문화와 학습문화가 피드백에 대해서 만들어내는 규범 또는 기대에 따라서도 다르다. 그리고 Context는 피드백의 관계적 맥락을 포함하는데, 이 역시 큰 영향을 준다. Telio는 최근 교사-학습자간 'educational alliance'가 피드백에 미치는 맥락적 영향을 강조했다. 이러한 차이가 있기 때문에 교사는 학습자에 대한 이해를 바탕으로 다양한 접근법을 사용해야 한다. 피드백 exchange는 교사와 학습자의 목표가 서로 합치되었을 때 가장 효과적일 것이다.

Variability in the impact of feedback extends beyond the individual. Responses to feedback are also shaped by learning culture, and the norms and expectations it creates for feedback [38], And context, including the relational ele-ment of feedback, is increasingly recognized as influential;Telio [51] has recently highlighted the contextual influence of the ‘educational alliance’ that develops between teacher and learner on the feedback that is exchanged. In the face of such variability, teachers must develop versatile approaches to feedback that are grounded in an understanding of the learner. The feedback exchange is perhaps at its most effec-tive when teachers’ and learners’ goals are aligned [38]. 

12. F/U 없이 피드백을 제공하지 마라. 향상된 퍼포먼스를 보여줄 기회가 따라오지 않는 피드백은 효과를 미치지 않는다.

Guideline 12. Don’t provide feedback without follow-up. Trainees are unlikely to be influenced by feedback that is not followed by an opportunity for them to demonstrate improv- ing performance.

13. (어디서 듣고 온) poorly informed 정보로 피드백을 주지 마라. 그렇게 하는 것은 피드백에 대한 가치만 떨어뜨릴 뿐이다.

Guideline 13. Don’t provide feedback that is poorly informed (or is based on hearsay); doing so diminishes the value that trainees assign to feedback in general.

Trainee들은 피드백을 원칙적으로는 중요하다고 생각하며, 피드백을 주는 것을 임상교사가 갖추어야 할 바람직한 행동이라고 생각한다. 그러나 현실에서 medical training동안 받는 피드백의 퀄리티는 낮다고 보고되고 있으며, 직접관찰이 제한되는 것 등의 이유로 피드백은 poorly informed된다. 그 결과, trainee는 외부 피드백의 가치를 일반적으로 평가절하하게 되고, 자기평가에 의존한다.

Surveys have demonstrated that trainees value feedback in principle, and value the provision of feedback as a desired quality of clinical teachers [53]. In reality, however, the quality of the feedback received in medical training is often reported as low, and poorly informed due to factors includ- ing limited direct observation of performance. As a conse- quence, trainees may begin to devalue external feedback in general, relying instead on self-assessment [10, 11].

14. 피드백이 정서에 미치는 부정적 영향을 과소평가하지 말라. 정서적 스트레스는 피드백 수용의 장애가 된다.

Guideline 14. Don’t underestimate the emotional impact of feedback that is perceived as negative. Emotional distress may be a barrier to acceptance and use of feedback.

 

 

Feedback intervention theory 는 피드백이 trainee의 자존감에 위협을 가하면 효과성이 떨어진다고 강조한다. Sargeant는 '자기평가와 반대되는 부정적 피드백을 받는 사람은 스트레스를 받고, (그러한 피드백을) 수용하고 그에 따라 행동할 수 있는 능력을 제한시키는 지속적 감정을 유발한다. Eva는 피드백의 해석과 수용이 감정의 복잡한 interplay에 영향을 받는다고 하였는데, 여기에는 자신감/공포감 등이 있고, 학습자에게 피드백을 전할 때 self-concept를 유지할 수 있게 해주는 것이 중요함을 강조하였다.

Feedback intervention theory [28] posits that feedback which threatens self-esteem is much less likely to be effec- tive. Sargeant found that those who had received negative feedback that conflicted with their self-assessment experienced distressing and long-lasting emotions that limited their ability to accept and act upon the feedback [31]. Eva showed that the interpretation and acceptance of feedback was influenced by a complex inter- play of emotions, including confidence and fear, and high- lighted the importance of allowing the learner to maintain their self-concept when delivering feedback [2].

15. 성적을 매기는 기준을 설명하지 않은 채로, 그리고 개선 전략을 주지 않은 채로 성적만 매기지 마라

Guideline 15. Don’t give grades without explaining the criteria for allocation of grades and providing strategies for improvement.

너무 자주 comment box에는 아무것도 쓰여있지 않은 상태로 성적이 부여된다. 성적은 학생의 동기를 저하시키거나 노력을 덜 하게 만들기 때문에 formally assessing하지 않고 성적을 주는 것은 피해야 한다.

all too often grades are given with the comment boxes left blank [56]. Because of the potential for grades to demotivate or to reduce effort, it has been suggested that it might be wisest to avoid giving grades except when formally assessing the learner (in infre- quent ‘high stakes’ assessments) [58].

잘 모르는 것.

Don’t knows

16. 피드백의 credibility를 결정하는 것은 무엇인가?

Guideline 16. What determines the credibility of feedback?

17. 얼마나 많은 내용을 피드백해야 하는가?

Guideline 17. How much is the right amount of content when giving feedback?

18. '개방적, 안전한 상호작용'은 무엇이 결정하는가?

Guideline 18. What determines the ‘open and safe interac- tion’ in the feedback conversation?

19. Trainee의 반응에 영향을 주는 것은?

Guideline 19. What influences the trainee’s response? (constructive or destructive outcomes)

Workplace learning theorists는 어떻게 개인적 요인과 사회문화적 요인이 상호작용하는지 이해해야 한다고 강조한다. Billett는 affordances and agency를 강조하였다.

Workplace learning theorists (e.g. Eraut [61], Billett [62]) have highlighted the need to understand how individual and the sociocultural influences on learning interact. Billett emphasizes the notions of affordances and agency;

• 학습환경은 다양한 Affordance(학습의 기회)를 제공한다
  a learning environment offers a range of affor- dances, or opportunities to learn, but

• 개별 학습자는 Agency(주체성)을 발휘하여 이 Affordance에 engage해야한다.
  an individual learner must exercise agency to engage with those affordances.

 

피드백은 affordance (좋은 피드백이 이뤄지는가?)와 agency(학습자들이 피드백에 engage하는가?) 둘 모두와 관련된다.

Feedback challenges may lie with either affordance (is good feedback made available to learners?) or agency (do learn- ers choose to engage with feedback?), or both;

20. 노골적인 동료와의 비교가 도움이 되는가? 노골적인 'standard'와의 비교는 도움이 되는가?

Guideline 20. Is overt comparison with peers when made by the supervisor helpful to the trainee? Indeed, is overt comparison with required performance standards helpful?

동료와 비교하는 것은 피드백을 덜 효과적으로 만드는데, 왜냐하면 task에서 벗어나 self에 초점을 맞추게 하기 때문이다. Self-referenced 와 other-referenced 피드백은 직간접적으로 self에 초점을 맞추게 만든다. 다른 사람과의 비교에서 부정적인 평가를 받으면 자존감에 위협을 느낄 수 있고 performance goal orientation을 유도하며, 학습을 방해한다. 그러나 이러한 우려에도 불구하고 comparison의 가치를 강조하기도 한다.

What about comparison with peers? There is evidence that feedback becomes less effective as its focus moves away from the task and toward the self [13, 28]. Both self- referenced and other-referenced feedback (in)directly focus the attention to the self. Unfavourable comparisons with others may threaten self-esteem and promote a performance goal orientation, potentially hindering learning [17]. But despite these concerns, some research has suggested value in comparisons:

21. Written summary가 학습에 도움이 되는가?

Guideline 21. Does a written summary of the feedback dis- cussion enhance learning?

의대생들은 비공식적 verbal feedback을 WBA에서의 written feedback보다 좋아한다.

Medical students have been found to value informal ver-bal feedback more than formal workplace-based assessment(WBA) with written feedback [66, 67]. 

written summary 의 가치는....

The value of the written summary is therefore secondary but could include:

● Aiding reflection on the feedback at a later date 

● Aiding discussion between tutor and trainee at a later date 

● Enhancing tutor effort at the time of generating the feedback

학습문화를 위한 가이드라인

Guidelines for the learning culture: what elements of learning culture support the exchange of meaningful feed- back, and what elements constrain it?

Do's

Do’s

22. 피드백을 학습프로세스에 포함시키기 위한 시스템 접근법을 갖추라

Guideline 22. Do have a systems approach, building feed- back into the learning processes.

 

각 기관은 장기적 교사-학습자 관계가 번창할 기회를 줄 수 있다.

Institutions can create opportunities for longitudinal teacher-learner relationships to flourish, such as extended placements [68–70].

Trainee 에 대한 기대는 reflection-on-feedback 일 것이며, 개선전략 도입을 강제할 시스템이 있어야 한다. 신입 Trainee는 특정 academic community의 규칙을 안내받아야 한다.

Expectations of the trainee might be reflection-on-feedback with some system of reinforcing implementation of strategies for improvement [32]. New trainees will require induction into the rules of the particular academic community.

피드백의 문화를 확립하려면, trainee에 대한 피드백 뿐만 아니라 supervisor에 대한 정기적 피드백 시스템도 있어야 한다.

In order to ensure a climate of feedback, an institution should provide a system of regular feedback not only for trainees but also for supervisors [32, 72].

FD course 제공 외에도 supervisor와 그들의 사회적 네트워크에 대한 교육적 support를 제공함으로써 피드백을 사용하고 받아들이도록 촉진할 수 있다.

In addition to providing faculty development courses, educational support can be offered to supervisors and the supervisors’ social networks can be used and supported to facilitate acceptance and use of feedback [73].

23. 장기적인 신뢰할 수 있는 supervisor-trainee 관계 형성을 지원하라. 영향력이 큰 피드백은 신뢰관계의 맥락에서 번창한다.

Guideline 23. Do support the development of longitudinal, trusting supervisor-trainee relationships in medical train- ing; influential feedback thrives in the context of trusting relationships.

Trainee가 supervisor와 관계형성이 되어있을 때 피드백의 credibility를 신뢰할 수 있고, 교사의 목표와 자신의 목표의 합치를 이룰 수 있다. Bok 등이 말한 바와 같이 교사-학생의 durable한 관계는 즉각적으로 유도한다.

When trainees can build a relationship with their supervi- sors, it allows them to trust the credibility of the feedback they receive and the alignment of the teacher’s goals with their own. As Bok et al. showed, durable teacher-learner relationships also prompt readily [69]. 

Bates 등에 따르면 그러한 관계가 '비판적 피드백의 건설적 해석'을 가능하게 해준다. 학생은 challenging or corrective 피드백을 supportive하게 해석한다. 신뢰가 있고 지지적인 관계에서 피드백은 더 credible하다고 여겨질 가능성이 높다. 피드백 프로세스에서 관계의 centrality를 인지하여, educational alliance라는 개념이 제안되었고, 이는 교사-학습자 관계의 link를 이해하고 그 안에서 이뤄지는 피드백의 impact를 이해하기 위한 것이다.

Bates et al. [67] found that such relationships afforded ‘constructive interpretation of critical feedback’ (p. 366); students were able to interpret even challenging or corrective feedback as supportive. Within a trusting and supportive relationship, feedback is also more likely to be viewed as credible [50, 78]. Recog- nizing the centrality of relationship in the feedback process, the concept of the ‘educational alliance’ has been proposed as a framework for understanding the links between the teacher-learner relationship and the impact of the feedback generated within it [51, 79].

24. 비디오를 사용하라

Guideline 24. Do use video review with feedback as a com- ponent of training.

Supervisor의 피드백은 직접 관찰을 할 때와 비디오로 볼 때 다르지 않을 수 있으나, Trainee는 자신의 강점과 약점을 confirm할 수 있다.

The super- visor’s feedback may not differ whether following direct observation or following video observation but the trainee will be able to confirm the strengths and weaknesses in their own performance.

Supervisor은 같은 비디오 녹화를 보고도 상당히 다른 피드백을 준다. Systematic review에서 비디오녹화를 자기평가하면서 보았을 때는 효과적이지 않았지만, 전문가의 피드백과 연결되었을 때는 더 효과가 좋았다. 

Supervisors differ considerably in the feedback they give after reviewing the same videotaped consultation [80]. In a systematic review Hammoud et al. con- cluded that video review with self-assessment alone was not found to be generally effective for medical students, but when linked with expert feedback it was superior to tra- ditional feedback alone [81].

25. 피드백이 일상화/정기화/가치로운 실천공동체를 형성하라

Guideline 25. Do promote communities of practice in clin- ical workplaces in which feedback is routine, regular and valued.

피드백이 '일반적인 것'이라고 여겨지는 환경이 필요하다. 이는 trainee가 supervisor에게 피드백을 줄 수 있음을 의미한다. 비판적 피드백이 learning culture에 의해서 normalize 되었을 때 더 활발히 exchange 된다.

environment in which providing feedback is considered ‘normal’. This would mean, for example, that trainees are encouraged to give feedback to their supervi- sors [82, 83]. It has also been found in the training of ath- letes and musicians that critical feedback is exchanged more readily when it is normalized by a learning culture [38, 84].

 

 

26. 근무지 교육시스템에서 공식적 역할을 하는 사람이 그 역할을 인식하고, 학습자의 교육적 목표가 무엇이 되어야 하는지 이해하게 하라

Guideline 26. Make sure that those who have a formal role in a workplace’s educational system are aware of that role and understand what learners’ educational objectives should be.

Boor 는 좋은 관계 외에도 학습자는 교육시스템을 이해하고, 그 지식을 개별 학습자에게 적용할 수있는 supervisor를 좋아한다. VDV는 programmatic assessment에서 다음과 같이 말했으며, 이는 피드백에 대해서도 마찬가지다.

Boor et al. found that, next to the importance of a good relationship, learners value clinical supervisors who are aware of the educational system and expectations and who can apply that knowledge to the individual learner [85]. Van der Vleuten’s comments on programmatic assessment are useful here:

 

‘If a programme of assessment is to provide meaningful outcomes, all the players should understand what they are doing, why they are doing it, and why they are doing it this way. ’ [86]

If we substitute ‘feedback’ for ‘assessment’, the comment rings equally true.

27. 팀에서 정기적으로 피드백을 주고, 피드백을 주는 방식에 대해서 성찰하고, 피드백 제공 역량을 유지/향상시키기 위한 refresher course를 듣게 하라

Guideline 27. Make sure that the team give feedback regu- larly, reflect on the practice of giving feedback, and follow refresher courses to maintain and improve competency in providing feedback.

평가 프로세스에 대한 교수들의 insight가 없는 것도 문제이다. Tutor training이 필요한데, tutor는 academic and social needs를 정확히 진단해야 한다. 학습자의 관점을 이해하고 강조해야 하며, 적절한 scaffolding tool을 사용할 스킬이 있어야 한다.

Lack of faculty insight in the assessment process remains an issue [87]. The feedback landscape described by Evans [1] indicates the need for tutor training: the tutor must accu- rately diagnose academic and social needs; understand and empathize with the learner’s perspective, and have skills to employ appropriate scaffolding tools.

Don’ts

28. 피드백 효과성 향상을 위해 FD에만 의존하지 마라

Guideline 28. Don’t rely exclusively on faculty develop- ment to improve the effectiveness of feedback.

역사적으로 피드백에 관한 FD가 피드백의 퀄리티를 향상시키기 위한 primary approach였다. 그러나 '어떻게 피드백을 줘야하는가'에 집중한 나머지 '어떻게 학습자가 받아들일 것인가'를 놓쳐왔다. 또한 효과적인 피드백을 만드는 학습문화의 역할, 건설적 비판의 normalizing, 피드백을 학습에 가치있게 만들기 등등도 놓쳐왔다. FD는 필요하지만 충분하진 않다.

Historically, faculty development in feedback delivery has been the primary approach to improving the quality and effectiveness of feedback [29, 73]. This focus on how feedback is given ignores the important element of how it is received by learners [88]. The crucial role of

• learning culture in making effective feedback possible,

• normalizing constructive criticism, and

• establishing the value of feed- back for learning

...is also missed by an approach focused on individual teachers [38]. Faculty development is important but not sufficient;

29. mini-CEX같은 임상스킬의 formal assessment가 관측과 피드백 없이 종결되게  하지 마라

Guideline 29. Don’t allow formal assessments of clinical skills, such as the mini-CEX, to be completed without obser- vation and feedback.

 

모르는 것

Don’t knows

Guideline 30. What are the vital components that ensure a constructive system of workplace learning that caters to trainees, workers, and the educational system? How can the institution nourish a climate which encourages the provi- sion and seeking of feedback?

'피드백이 효과가 있었나' 가 아니라 '어떤 상황에서 효과가 있는가'이다.

They conclude that thereal question is not whether feedback works, but under what circumstances is works best.

팀워크 프로세스에 대한 피드백, 팀워크 프로세스가 작동하는 동안의 피드백이 팀이나 팀원의 퍼포먼스에 대한 피드백보다 효과가 있었다.

Feedback about and during the process of teamwork was more reliably effective than feedback about performance given to the team or to individuals within the team.

Guideline 31. Is it most effective to give feedback to indi- viduals alone or in a group setting?

Guideline 32. Does the use of formative assessment out- comes for summative purposes (such as having supervisors provide formative feedback that at the end of a rotation is also used for a summative assessment) corrupt a well-inten- tioned educational system?

형성평가를 최종평가의 data point로 사용하는데 어려움을 겪고 있으며, 이로 인해서 형성평가가 assessment-for-leraning의 역할을 하지 못하고 있다.

A qualitative study with clinical undergraduate veterinary students and their supervisors highlighted that both struggled with formative assessments that are used as ‘data points’ for a final summative judgment. As a result, the formative assessments did not play the powerful assess- ment-for-learning role they are meant to have in lum based on programmatic assessment [69].

의학교육은 총괄평가와 형성평가의 경계를 흐릿하게 할 뿐만 아니라 교사의 코치로서의 역할과 평가자로서의 역할 구분도 흐릿하게 한다. 교사가 이렇게 dual role을 하게 될 때 피드백의 퀄리티와 impact가 훼손될 수 있다.

Medical education not only blurs the line, at times, between summative and formative assessment, but also blurs the line for its teachers between the roles of coach and assessor. Recent literature has begun to challenge this approach, suggesting that the quality and impact of feedback may be compro- mised when the teacher is assigned this dual role [38, 91, 92].

피드백 프로세스는 개선전략을 만들어내지 못하면 불완전한 것이다. 가장 좋은 피드백 프로세스 루프는 어떤 결과가 있었는지에 대한 subsequent assessment이다.

The feedback process is incomplete if it does not result in the generation of strategies for improvement— either recommendations, or self-generated as a result of feedback. And the best feedback process loops back into a subsequent assessment with feedback about whether this has resulted in improved clinical performance.

 

 

Summary

34. Kluger A, Van Dijk A. Feedback, the various tasks of the doctor, and the feedforward alternative. Med Educ. 2010;44:1166.

 

 

---

Perspect Med Educ. 2015 Dec;4(6):284-99. doi: 10.1007/s40037-015-0231-7.

Guidelines: the do's, don'ts and don't knows of feedback for clinical education.

Lefroy J1, Watling C2, Teunissen PW3, Brand P4.

Author information

• 1Keele University School of Medicine, Clinical Education Centre RSUH, ST4 6QG, Staffordshire, UK. j.e.lefroy@keele.ac.uk.
• 2Schulich School of Medicine and Dentistry, Western University, Ontario, Canada.
• 3Maastricht University and VU University Medical Center, Amsterdam, The Netherlands.
• 4Isala Klinieken, Zwolle, The Netherlands.

Abstract

INTRODUCTION:

The guidelines offered in this paper aim to amalgamate the literature on formative feedback into practical Do's, Don'tsand Don't Knows for individual clinical supervisors and for the institutions that support clinical learning.

METHODS:

The authors built consensus by an iterative process. Do's and Don'ts were proposed based on authors' individual teaching experience and awareness of the literature, and the amalgamated set of guidelines were then refined by all authors and the evidence was summarized for each guideline. Don't Knows were identified as being important questions to this international group of educators which if answered would change practice. The criteria for inclusion of evidence for these guidelines were not those of a systematic review, so indicators of strength of these recommendations were developed which combine the evidence with the authors' consensus.

RESULTS:

A set of 32 Do and Don't guidelines with the important Don't Knows was compiled along with a summary of the evidence for each. These are divided into guidelines for the individual clinical supervisor giving feedback to their trainee (recommendations about both the process and the content of feedback) and guidelines for the learning culture (what elements of learning culture support the exchange of meaningful feedback, and what elements constrain it?)

CONCLUSION:

Feedback is not easy to get right, but it is essential to learning in medicine, and there is a wealth of evidence supporting the Do's and warning against the Don'ts. Further research into the critical Don't Knows of feedback is required. A new definition is offered: Helpful feedback is a supportive conversation that clarifies the trainee's awareness of their developing competencies, enhances their self-efficacy for making progress, challenges them to set objectives for improvement, and facilitates their development of strategies to enable that improvement to occur.

KEYWORDS:

Feedback; Feedback culture; Feedback relationship; Formative assessment; Workplace based assessment

PMID:

 

26621488

 

PMCID:

 

PMC4673072

 

DOI:

 

10.1007/s40037-015-0231-7

[PubMed] 

Free PMC Article