성찰에 대한 평가의 교란요인: 비판적 리뷰 (BMC Med Educ, 2011)

성찰에 대한 평가의 교란요인: 비판적 리뷰 (BMC Med Educ, 2011)

Factors confounding the assessment of reflection: a critical review

Sebastiaan Koole1*, Tim Dornan2, Leen Aper1, Albert Scherpbier3, Martin Valcke4, Janke Cohen-Schotanus5 and

Anselme Derese1

배경

Background

평생학습은 최신의 헬스케어 서비스 제공을 위해 필수적이다. 평생학습이 단순히 컨퍼런스 참석을 의미하는 것이 아니며, 오늘날 평생학습이란 지속적 프로세스로서, 일상의 전문직 행동에 embed된 것이다. 평생학습의 핵심은 자신의 행동에 대해서 성찰하는 능력, 치료의 과정과 성과를 검토하고, 새로운 학습목표를 세우고, 수월성을 추구하기 위한 미래의 행동을 계획하는 것이다.

Lifelong learning is, consequently, crucial to the provision of up-to-date healthcare services [1].Rather than just attending conferences, lifelong learning today is seen as a continuous process, embedded in everyday professional practice. At its core lies practi-tioners’ ability to reflect upon their own actions, con-tinuously reviewing the processes and outcomes of treatments, defining new personal learning objectives, and planning future actions in pursuit of excellence [2-5].

많은 교육기관에서 성찰능력을 직헙훈련 프로그램의 목표로 삼고 있으며, 성찰적 사고가 개발될 수 있다는 것이 전제이다.

As a result, many educational institutions incorporate the ability to reflect as an objective of their vocational pro- grams, premised on a belief that reflective thinking is something that can be developed rather than a stable personality trait [4,10,11].

그러나 성찰능력을 어떻게 가장 잘 개발하도록 도와줄 수 있는가는 불확실하다. 합의된 방법이 부족하다.

There is, however, uncertainty about how best to help people develop their ability to reflect [11]. Lack of an agreed way of assessing reflection is a particular obstacle

 

평가는 피드백의 원천으로서 motivation에 영향을 주며, 요구되는 수준의 역량이 달성되었는지를 언제 판단할지에 따라서도 영향을 준다. 어떻게 '성찰적 학습'을 조작화할 것인가는 더 심각한 문제이다. 서로 다른, 광범위하게 받아들여지는 서로 다른 방식의 성찰에 대한 정의가 평가에 있어서 성찰의 성과, 성찰의 차원, 성찰의 기준 등을 다양하게 한다. 그 결과 연구결과를 비교하기도 어렵다.

Assessment has also a motiva- tional influence as a source for feedback (formative assessment) and when to judge whether requisite levels of competence have been attained (summative assess-ment) [3,4,12]. The persisting lack of clarity about how to operationalise reflective learning is symptomatic of an even deeper problem. Different, widely accepted theories define reflection in different ways, consider different outcomes as important, define different dimensions along which reflection could be assessed and point towards different standards [11]. Consequently, research findings are hard to compare. This unsatisfactory state of affairs leaves curriculum leaders without practical guidelines,

논문의 목적

The purpose of this article is to review four factors,which confound the assessment of reflection: 

• 1. Non-uniformity in defining reflection and linking theory with practice. 
• 2. A lack of agreed standards to interpret the results of assessments. 
• 3. Threats to the validity of current methods of asses- sing reflection. 
• 4. The influence of internal and external contextual factors on the assessment of reflection.

고찰

Discussion

1. '성찰'의 정의

1. Defining reflection

 

다양한 성찰의 정의 

• Boe- nink 등은 '상황을 분석하는 서로 다른 관점의 숫자'로서 성찰을 묘사했다. 즉 하나의 관점에서부터 다수의 관점까지 다양할 수 있다.

• Aukes 등은 자기성찰/공감적 성찰/성찰적 의사소통의 조합으로서 개인의 성찰을 개념화하면서, 정서적, 의사소통적 요소를 강조했다.

• Sobral은 학습의 관점에서 reflection-in-learning을 강조했다.

Boe- nink et al [10] described reflection in terms of the num- ber of different perspectives a person used to analyse a situation. Reflection ranged from a single perspective to a balanced approach considering multiple relevant per- spectives. Aukes et al [13] emphasised emotional and communication components when they conceptualised personal reflection as a combination of self-reflection, empathic reflection, and reflective communication. Sobral’s [14] emphasis on reflection-in-learning approached reflection from a learning perspective.

이 세 가지 관점이 이 분야에서의 비일관성을 보여준다면 Dewey, Boud, Schön, Kolb, Moon, and Mezirow 의 연구는 공통점을 보여준다.

If those three perspectives exemplify inconsistency in the field, the work of Dewey, Boud, Schön, Kolb, Moon, and Mezirow exemplifies shared ground between reflec- tion theories and used terms.

 

• Dewey is usually regarded as the founder of the concept of reflection in an educa- tional context. He described reflective thought as “active, persistent, and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends” [15]. He saw reflective thinking in the education of individuals as a lever for the development of a wider democratic society.

• In line with his work, Boud et al emphasised reflection as a tool to learn from experience in experiential learn- ing [16]. They identified reflection as a process that looks back on experience to obtain new perspectives and inform future behaviour. A special feature of their description of reflection in three stages -

• 1. 경험으로 돌아가기 Returning to an experience;

• 2. 감정에 집중하기 attending to feelings; and

• 3. 경험을 재평가하기 re-evaluat- ing the experience - was the emphasis it placed on the role of emotions.

• Moon described reflection as an input-outcome pro- cess [17]. She identified reflection as a mental function transforming factual or theoretical, verbal or non-verbal knowledge, and emotional components generated in the past or present into the output of reflection (e.g. learn- ing, critical review or self-development). (사실적 또는 이론적/ 언어적 또는 비언어적/ 과거 혹은 현재의 감정적 요소를 성찰의 output으로 만드는 것)

• Schön’s concept of reflective practitioner the identi-fied reflection as a tool to deal with complex profes- sional situations [18,19]. Reflection in a situation (reflection-in-action) is linked to practitioners’ immedi- ate behaviour. Reflection after the event (reflection-on- action) provides insights that can improve future prac- tice. Those two types of reflection together form a con- tinuum for practice improvement.

• The term ’ reflective learning ’ describes reflection in the context of experiential learning. Kolb’s widely accepted experiential learning cycle describes four stages of learning:

• 1. 경험을 한다(구체적 경험) having an experience (concrete experi- ence),

• 2. 성찰적 관찰(경험을 성찰한다) reflective observation (reflecting on this experi- ence),

• 3. 추상적 개념화(경험에서 배운다) abstract conceptualisation (learning from the experience) and

• 4. 능동적 실험(배운 것을 시도해본다) active experimentation (trying out what you have learned) [20].

• 이 네 단계는 나선형이다. These four stages are con- ceptualised as a spiral, each of whose turns is a step for- ward in a person’s experiential learning.

• Lifelong learning is considered today as essential for maintaining a high standard of professional practice. Mezirow’s transformative learning theory described life- long learning in terms of learners’ transforming frames of reference, in which reflection is the driving force [21].

공통 요소의 '절충 모델'

Towards an ‘eclectic model’ of common elements

Atkins and Murphy 는 성찰을 다음과 같이 밝혔다.

Atkins and Murphy [22] identified reflec- tion as:

• 1. 불편한 감정 또는 생각의 인지 ‘awareness of uncomfortable feelings and thoughts’, resulting in

• 2. 감정과 지식의 분석 an ‘analysis of feelings and knowledge’, finally leading to

• 3. 새로운 관점 ‘new perspectives’.

Korthagen의 ALACT 모델(’Action, Looking back on action, Awareness of essential aspects, Creating alternative methods of action, and Trial’)은 '인식하게 됨' 시기의 첫 두 단계를 보여준다. 일반적인 회상적 행동, 더 해석적인 행동

Korthagen’s ALACT model (’Action, Looking back on action, Awareness of essential aspects, Creating alternative methods of action, and Trial’)[23]describes the first phase of ‘becoming aware’ in two steps: a general retrospective action and a more interpretive action.

이 두 가지 이론을 통합하면 첫 번째 phase가 나온다(경험 검토’reviewing an experience’). 두 가지 하부 구성요소

Integrating those two theories, resulted in a first phase (’reviewing an experience’)with two subcomponents:

• 1. 무슨 일이 일어났는지를 일반적으로 기술한다. generally describing what hap- pened and

• 2. 생각/사고/맥락적 요인을 고려하여 본질적 측면을 밝힌다. identifying essential aspects by considering both thoughts, feelings and contextual factors.

그러나 단순히 경험을 검토하는 것이 효과적 성찰로 이어지지는 않는다. Bourner에게 있어서, 경험에서 더 정보를 얻기 위해서interrogate 탐색 질문을  활용하는 것은 '성찰'과 '생각thinking'의 차이였고, 그는 '성찰적 탐구reflective inquiry'를 성찰의 중요한 요소로 보았다. 성찰에 대한 이러한 관점은 Mamede and Schmidt가 성찰적 행동을 '성찰에 대한 개방성openness to reflection'으로 본 것과 마찬가지이다. Bourner는 탐색질문searching question을 하는 것만 강조했고, 그것에 대한 답을 구하는 것을 강조하지는 않았다. Korthagen의 접근법은 Bourner의 접근법에다가 질문에 대답을 하는 과정으로서 '대안적 행동방법 만들기creating alternative methods of action'를 추가하여 이를 보완해준다. 이렇게 추가한 것은 Boud가 분석을 association, integration, validation and appropriation의 조합이라고 한 것과 잘 맞는다.

Just reviewing an experience, however, does not neces- sarily lead to effective reflection. For Bourner [24], using searching questions to interrogate an experience was the key difference between reflecting and thinking and he saw ‘reflective inquiry’ as a crucial component of reflec- tion. This aspect of reflection was also represented in Mamede and Schmidt’s proposed structure of reflective practice as ‘openness to reflection’ [25]. Bourner only emphasised posing searching questions, however, not answering them. Korthagen’s approach supplements Bourner’s by contributing ‘creating alternative methods of action’ as a process of answering questions. This addition is compatible with Boud’s characterization of analysis as a combination of association, integration, validation and appropriation.

 

개인의 Frame of reference 안에서 이뤄지는 내면의 대화internal dialogue를 하는 것은 분석의 방향을 제시해주며, "감각적 인상을 거르는 가정과 기대의 구조"를 보여준다. 이러한 개인의 관점은 인식/인지/감정/성향(intentions, expectations and purposes) 등으로 구성되어 있으며, 우리의 감각 경험에 의미를 부여하는 맥락을 제공한다. 성찰의 첫 번째 phase가 경험을 묘사하고, 감정/생각/다른 측면을 인식하는 것이라면, 두 번째 phase는 성찰적 탐구reflective inquiry를 가지고 경험을 분석하여 개인의 독특한 Frame of reference 안에서 분석 프로세스를 trigger하는 것이다.

The internal dialogue that results is conducted within a ‘personal frame of reference’ that, according to Mezirow, directs the analy- sis and represents “the structure of assumptions and expectations through which we filter sense impressions” [21]. This personal perspective, made up of our percep- tions, cognitions, feelings and dispositions (intentions, expectations and purposes), creates a context in which we give meaning to our sensory experiences. If the first phase of reflection, then, is identified as the description of an experience and the awareness of feelings, thoughts, and other essential aspects, our second phase of reflec- tion is analysing experiences by reflective inquiry, which triggers a process of analysis within a person’sunique frame of reference.

Moon의 투입-산출 모델은 성찰이 '목적성을 가짐purposeful'을 강조한다. Atkins and Murphy에 의해 밝혀진 이 목적, 즉 세 번째 phase는 '새로운 관점의 발견identification of new perspectives'이다. Korthagen and Boud는 둘 다 추가 단계를 하나 더 넣었는데, 이 새로운 관점을 행동으로 옮겨서 새로운 성찰적 사이클의 시작점으로 삼는 것이다. Stockhau- sen의 성찰적 실천의 임상 나선 모델에서 '재건축reconstruction phase'가 같은 기능을 한다. 이 phase에서 성찰적 통찰reflective insights은 미래의 행동을 위한 계획으로 전환된다. 이러한 행동이 미래의 성찰로 이끌어줄 수 있으므로, 경험에 대해서 성찰을 하는 것은 중요한 경험을 well-informed practical action으로 전환시켜주는 순환적 과정이다.

Moon’s input-outcome model emphasises that reflec- tion is purposeful [17]. This purpose is identified by Atkins and Murphy in the third phase of reflection as the ‘identification of new perspectives’ [22]. Both Korthagen and Boud, however, included an additional stage - the conversion of those new perspectives into actions that are the starting point for new reflective cycles [16,23]. The ‘reconstruction phase’ of Stockhau- sen’s clinical learning spiral model of reflective practice among undergraduate nursing students in clinical prac- tice settings had the same function [26]. During this phase, reflective insights were transformed into plans for future actions. Since those actions could lead to further reflections, reflecting on experiences was identified as a cyclic process that transformed significant experiences into deliberate, well informed practical actions.

 

이러한 insight를 가지고 '새로운 관점의 발견'을 성찰 프로세스의 성과로서 정의했으며, 이 새로운 관점은 '성찰을 통한 미래의 행동'으로 이끌어준다. 이 phase를 연구자에 따라서는 행동-전-성찰reflection-before-action이라고 부르며, 이번 절충 모델에서는 성찰을 순환적 과정으로 만듦으로서 포함되었다. 이 모델에서 성찰은 과거의 성찰에서부터 나온 학습목표로부터 정보를 받고, 발달 프로세스로서 성찰의 중요성을 강조한다. Korthagen and Stockhausen은 모두 성찰나선reflection spiral이라는 용어와 함께 이 프로세스를 강조하였으며, 이를 통해서 더 높은 차원의 이해/실천/학습으로 갈 수 있는 길이라고 보았다.

We incorporated those insights into the eclectic model by defining the outcome of a reflection process as the iden- tification of new perspectives, which leads to future actions informed by reflection. Stockhausen also described a preparatory phase to establish objectives for a new clinical experience. This phase, which other authors have labelled as reflection-before-action [27,28], is incorporated into the eclectic model by representing reflection as a cyclical process. It allows reflection to be informed by learning goals arising from past reflections and stresses the importance of reflection as a develop- mental process. Both Korthagen and Stockhausen have highlighted this process with the term reflection spiral with each winding leading to a higher order of under- standing, practice or learning [23,26].

요약

Reviewing the experience has two compo- nents:

• ‘description of the experience as a whole’,and

• ‘awareness of essential aspects based on the considera- tion of personal thoughts, feelings, and important con- textual factors’.

Critical analysis starts with

• ‘reflective inquiry’ - posing searching questions about an experi- ence - and progresses to

• ‘searching for answers’ while remaining aware of the ‘frame of reference’ within which the inquiry is being conducted.

Reflective out- come comprises the

• ‘new perspectives’ resulting from phase two, and the

• ‘translation of those perspectives into behaviour that has been informed by reflection’.

 

모델을 만드는 것부터 성찰 평가를 위한 지표 개발까지

From model building to developing indicators for assessment of reflection

 

성찰 프로세스의 적절성 지표

indicator of the adequacy of reflection processes (table 2).

 

 

 

 

 

2. 성찰 평가 해석를 위한 기준

2. Standards to interpret reflection assessment

Boud의 이론에는 여섯 가지 항목이 있다.

Boud’s theory, had six items:

• attending to feelings,

• association,

• integration,

• vali- dation,

• appropriation and

• outcome of reflection.

Mezirow는 학생을 다음과 같이 나눴다.

Mezirow, labelled students as:

• 비-성찰자 non-reflectors (no evidence of reflective thinking),

• 성찰자(경험을 학습 기회에 연관짓기) reflectors (evidence of relating experience to learning opportunities) and

• 비판적 성찰자(성찰의 결과를 전문직적 행동에 통합하기) critical reflectors (evidence of inte- grating reflective outcomes in professional behaviour).

연구자들은 Boud의 카테고리가 written material에 적용하기 어렵다는 것을 알았고, Mezirow의 것보다 신뢰도가 떨어졌다. 그러나 Mezirow의 카테고리는 세 개밖에 없어서 사람들 간 변별력이 떨어졌다. Kember 등은 이 문제를 finer-tuned을 통해서 해결하고자 했다. 이들이 제시한 7개 카테고리는

The researchers found Boud’s categories hard to apply to written materials, resulting in less reliable coding than using Mezirow’s scheme. With only three cate- gories, however, this latter scheme had a limited capa- city to discriminate between people. Kember et al [31] addressed this issue by using a finer-tuned coding scheme based on the work of Mezirow. Their seven categories ranged from

• 비성찰적 사고 unreflective thinking (

• 습관적 행동 habitual action,

• 자기반성 introspection and

• 사려깊은 행동 thoughtful action) to

• 성찰적 사고 reflective thinking (

• 내용 성찰 content reflection,

• 과정 성찰 process reflection,

• 내용과 과정 성찰 content and process reflection and

• 전제premise 성찰 premise reflection).

Boenink 는 성찰을 1~10까지 순위를 매겼고, 이는 학생이 쓴 성찰적 반응에서 드러난 관점의 숫자을 기준으로 매긴 것이였다. 그러나 이 척도는 성찰의 한 가지 측면밖에 보여주지 못한다는 한계가 있다.

Boenink et al [10] used an alternative approach, which ranked reflections from 1-10. Their scale was based on the number of perspectives students described in short written reflective reactions to a case vignette describing a challenging situation. The scale was limited, however, by measuring only one aspect of reflection (being aware of the frame of reference used).

 

Duke and Appleton 는 8개의 스킬을 평가했다. Grade를 줌으로써 이 연구자들은 성찰적 스킬에 대한 기준을 처음으로 설정했으나, 어떻게 level을 grade로 연결했는지를 밝히지는 않았다.

Duke and Appleton [29]developed a broader marking grid to score reflectivereports. It assessed eight skills that support reflection,identified by a literature review, on five-level scales,‘anchored’ and linked to a grade (A, B+, B, C and F).By providing grades, these authors were the first to set standards for reflective skills , however, the authors did not disclose how they linked the levels to grades.

Boyd 는 성찰적 판단을 King and Kitchener가 제안한 7개의 지적발달을 기준으로 코딩하였다.

Boyd [32] assessed reflective judgement using a coding scheme based on seven stages of intellectual development described by King and Kitchener:

• 전-성찰적 사고 Pre-reflective thinking (stages 1-3);

• 유사-성찰적 사고 quasi- reflective thinking (stages 4 and 5); and

• 성찰적 사고 reflective think- ing (stages 6 and 7).

Measurements made with the scale had an interrater reliability of 0.76 (Cronbach alpha).

접근법은 두 그룹으로 나눌 수 있다. 한 가지 접근법은 level에 따라 순위를 매기는 것이고, 다른 하나는 성찰 프로세스의 phase를 밝히는 것이다.

Based on the approach coding schemes can be divided into two groups. A first approach ranks reflections according to levels. The other approach is the iden- tification of phases in the reflection process considering items of reviewing an experience, analysis and reflective outcome based on the used model of reflection [29,30].

연구 결과에서 공통된 것은 학생들이 성찰에 숙달된 수준이 매우 낮고, 따라서 발전의 여지가 충분하다는 것이다. 

their results share a common feature. Within their own scale, all stu- dies demonstrate learners to have very limited mastery of reflection, indicating an apparent room for improve- ment.

충분한 성찰의 수준을 갖추고 있는 의사를 구분할 수 있는 기준이 나올 때 까지는, 이해관계자들에게 어떤 성찰스킬이 필요한지 명확하게 설명해주고, 학습자들에게 최대한으로 그것을 개발하게끔 해야 할 것이다. 

Until standards have been formulated that can identify practitioners whose level of reflection is adequate, it seems reasonable to clarify to stakeholders (curriculum developers, students, practitioners, assessors) what reflection skills are expected and urge learners to develop them as far as possible.

성찰적 학습을 촉진하기 위해서 성찰능력을 개발하는 것과, 성찰의 빈도를 늘리는 것 사이에 균형이 필요하다.

In promoting reflective learning, however, a balance has to be struck between developing an ability to reflect and increasing the frequency of reflection.

3. 평가를 어렵게 만드는 요인들

3. Factors that complicate assessment

 

성찰의 메타인지적 성격 때문에, 평가를 위해서는 '성찰'을 '글written words'로 바꿔야 한다 (인터뷰 기록, 포트폴리오 성찰일지 등)

The metacognitive nature of reflection is an important complicating factor of reflection assessment [4]. Sub-jects are most often asked to ‘translate’ their reflections into written words, which are assessed against coding schemes or scoring grids [29-31,38-40]. Other suggested methods to ‘visualise’ reflections include the verbalisa-tion in interviews [32,41,42], written responses to vign-ettes [10], or reflective writings in portfolios [34,43].

 

따라서 평가자는 성찰을 기록한 사람의 '선택적 묘사'에 대해서 그것이 과연 '적절한지'를 확인해보지도 못하고, 평가해야 한다. 이 때 (비)의도적 뒤늦은 깨달음, 자성적 능력의 부족 등으로 편향이 생길 수 있다. 기록된 것은 선택적으로 기록된 것이고, 불완전하다. 이런 측면에서 인터뷰가 장점이 있으나 여전히 주관적인 평가이며, 성찰활동에 대한 선택적 네러티브만 평가할 수 있다.

Assessors’ dependency on a person’s interpretative description is a serious threat to the validity of assess-ments of reflection because they have to judge selective descriptions without being able to verify their adequacy. Accordingly this approach fails to detect bias caused by a lack of (un)intentional hindsight and introspection ability [44,45], reflections being determined by the requirements of the assessment and selectivity and/or incompleteness of aspects they portray [44]. Interviews have the advantage that they can pose clarifying ques- tions and monitor a reflecting person’s reactions, but they still leave assessors to ground their judgements in potentially subjective and selective narrative accounts of reflective activity.

두 가지 문제가 있다. 

There are two related problems in that.

• 성찰을 기술하는 의미론적 스킬semantic skill이 효과적인 성찰에 중요한 부분이긴 하지만, 성찰을 글 또는 말로 바꿔야 하기에, 순수한 성찰스킬이 아닌 다른 것(글쓰기 기술, 말하기 기술)에 영향을 받을 수 있다.
  Although the semantic skill of describing reflec- tions is considered integral to effective reflection [46], skills other than pure reflective skills are needed to turn reflection into writing and/or speech, which has a self- evident effect on reflective narratives [44].

• 다른 문제는 평가를 위해서 written approach를 하는 것이 학습자가 선호하는 학습스타일과 맞지 않을 수 있다. 인터넷 세대의 학생들은 그룹-기반의 테크놀로지 멀티미디어 활동을 선호한다(블로그, SNS 등). 또한 창의적으로 멀티미디어를 사용하게 지지해주는 것이 성찰에 더 헌신하게끔 해주고, 더 효율적 성찰을 도와줄지도 모른다.
  The other problem lies in a decrease of motivation caused by the non-alignment between the written approach to assess- ment and a learners preferred learning style [12]. Find- ings of Sandars and Homer [47] suggest the discrepancy between ‘net generation’ students learning preference of group-based and technological multimedia activities (blogs, social networks, digital storytelling) and the text based approaches to reflective learning. Moreover, sup- porting learners to reflect with the creative use of multi- media, will likely increase their commitment to reflect and stimulate even more efficient reflection [48].

자기기입식 설문: 정확한 introspect가 요구된다. Eva and Regehr 는 자기-평가적 접근만 활용하는 것은 부정확한 결과를 가져오며, introspection에 대한 삼각측량이 필요함을 강조함.

Self-assessment questionnaires have the advantage of circumventing indirect observation [13,14,49,50], but their requirement to introspect accurately introduces another validity threat [22,51], because it is then unclear if it is reflection or the ability to introspect that is being tested. Eva and Regehr [45] concluded that it is best not to build solely on self-assessment approaches as they tend to be inaccurate and they recommended triangulat- ing introspection with other forms of feedback.

이러한 이유로 과연 성찰이 평가가능한 것인가라는 질문이 남는다. 두 가지 요소가 중요해보인다. 타당한 접근을 위해서 Bourner는 내용Content과 프로세스Process에 대한 평가가 서로 구분되어야 한다고 제안했다. 주관성 때문에 내용이 평가에 있어서 장애요인이 된다면, 프로세스는 더 일반적인 특징이 될 수 있다. 유사하게, Bourner는 관찰가능한 항목 (학습목표 기술 등)이 성찰능력을 보여주는 것으로 사용되어야 한다고 주장했다.

Since there are such serious validity threats, the ques-tion remains whether it is possible to assess reflection at all. Two elements appear to be important. In search fora valid approach, Bourner [24] suggested the content and the process of reflection should be viewed as two separate entities. While the content is a barrier to assessment because of its subjective nature, the process has a more general character. Similarly Bourner proposed that observa- ble items, like the ability to formulate learning goals, should be used to demonstrate a person’s capacity for reflecting.

4. 성찰의 평가에 영향을 주는 내적, 외적 맥락요인

4. Internal and external contextual factors affecting reflection assessment

성찰에 대한 평가는 성찰능력 뿐 아니라 맥락적 요인에 의해서도 영향을 받는다. Motivation은 학습과 성취의 중요한 매개인자이다. 기대-가치 모델Expectation-value model은 과제에 대한 개인의 가치와 그 과제를 성공적으로 수행했을 때의 기대치가 과업 수행의 주요 예측인자라고 하였다. 이것을 성찰에 적용시키면, practice에 있어 성찰을 얼마나 중요하게 생각하느냐가 성찰에 얼마나 많은 시간과 노력을 쏟는지를 결정할 것이다. 성찰이 주는 보상에 대해 긍정적인 기대를 하지 않는 사람은 깊이있고 비판적인 성찰을 하지 않을 것이다. 이 motivational model은 성찰적 학습에 대한 개인의 과거 경험과 성찰 과정에 대한 개인의 이해가 motivation에 영향을 미치고, 궁극적으로 행동에도 영향을 준다고 설명한다. 따라서 성찰의 가치를 frame하고 의도한 결과를 얻기 위해서는 introductory session이 중요하다.

The results of assessments of reflection are influenced by contextual factors as well as people’s ability to reflect. Motivation is considered to be an important mediator of learning and achievement in medical education [55,56]. The expectancy-value model proposed by Wigfield and Eccles identifies the subjective value of a task to a per- son and their expectation of performing it successfully as main predictors of task performance [57]. Applied to reflection, it predicts that the perceived importance of reflection for (professional) practice will determine the time and effort a person is willing to invest in it; those who do not expect a positive return are unlikely to reflect profoundly and critically [4]. This motivational model also explains how personal factors like prior experience of reflective learning and a person’sunder- standing of the reflection process will influence motiva- tion and consequently reflective behaviour. Hence introductory sessions are important to frame the value and intended outcomes of reflection [4].

과거에는 성찰을 지극히 개인적인 프로세스라고 보았다. 그러나 점차 사회적 상호작용에 의해서 촉진되는 프로세스라고 개념화하는 쪽으로 생각이 바뀌고 있다. supervision과 동료들이 학습자에게 정기적으로 피드백을 주고, 생각을 자극하는 질문을 함으로써 성찰을 향상시킬 수 있다. 퍼실리테이터는 비-판단적 질문을 통해서 (학습자가) 그 상황을 더 탐구하고, 대안적 관점과 solution을 찾고, 당연하게 여겼던 가정이 무엇이었는지를 알게 해줄 수 있다. 더 나아가서 situations 이 강력한 감정과 부정적인 생각을 불러일으켜서 효율적인 성찰을 방해할 수 있다. 퍼실리테이터는 이러항 강력한 감정들을 동화assimilate시키고 성찰 프로세스에 초점을 맞추게끔 도와줄 수 있다. 성찰적 사고, 감정, 정서 등을 완전히 탐구하기 위해서는 성찰을 하는 사람과 퍼실리테이터 사이에 안전한 환경이 마련되는 것이 중요하다. 다른 사람을 돕는다는 의미 외에도, 퍼실리테이터가 된다는 것은 자기자신의 성찰도 더 효과적으로 할 수 있게 됨을 뜻한다. 그러나 Schon은 학습자와 코치 사이의 관계가 균형잡히지 않았을 경우에, 그리고 맥락적 요인에 과도하게 영향을 받았을 경우에 성찰적 실천이 방해받을 수 있음을 경고하였으며, 방어적 태도defensiveness를 보일 수 있다고 하였다. 맥락적 요인의 강조와 더불어 Schaub 등은 성찰적 학습을 장려하는 교사의 능력을 평가하는 척도를 개발하였다. 여기서는 학습자에게 교사가 self-insight를 지지하는지, 안전한 환경을 조성하는지, 자기-조절을 장려하는지 등을 물어본다.

Whereas reflection was traditionally conceived of as a strictly individual process, ideas are shifting towards conceptualising it as a process facilitated by social inter- action [4,45]. A stimulating environment in which supervisors and peers give learners regular feedback and ask thought-provoking questions can, from that point of view, be expected to improve reflection. With non-jud- gemental questions, facilitators can encourage to fully explore the situation, to consider alternative perspectives and solutions, and to uncover taken-for-granted assumptions [3]. Furthermore, situations can provoke strong emotions and negative thoughts which could potentially form a barrier obstructing efficient reflection and reflection upon . A facilitator can help to assimilate these strong emotions and refocus on the reflection process [12,16]. To fully explore reflective thoughts, feelings and possible emotions, it is crucial to create a safe environment established between the reflecting person and the facilitator(s) [3]. Next to sup- porting others, being a facilitator is also reported as even more effective for a person’s own reflections[58]. Schön, however, warned that an unbalanced relationship between learner and coach and an undue influence of contextual factors could hinder reflective practice, as it could lead to defensiveness [18]. In line with this emphasis on contextual factors, Schaub et al developed a scale to assess teachers’ competence in encouraging reflective learning [59]. It asks learners to identify whether teachers support self-insights, create a safe environment, and encourage self-regulation.

 

요약

Summary

성찰은 메타인지적 과정이므로, 성찰기록, 포트폴리오, 면접 등의 간접적으로만 평가될 수 있다. 이 방법에서 평가자들은 보고받은 성찰이 진실인지 확인하는 것verify이 어렵다. 자기평가식 설문지가 널리 사용되고 있는데 이 역시 마찬가지의 타당도 문제를 가지고 있고, 근본적으로 자기-평가의 문제도 있다. 이러한 타당도 문제를 해결하기 위하여, 평가는 주관적으로 미화된 성찰의 내용이 아니라 성찰의 프로세스에 초점을 맞춰야 한다는 주장이 제기되고 있다. 추가적으로, 성찰이 그 성찰을 자극triggering 상황적 맥락과 엮여 있기 때문에, 이러한 triggering situation에 대한 객관적 정보를 고려하는 것이 평가자로 하여금 묘사된 성찰을 verify할 수 있게 해준다. 성찰 프로세스는 내적(동기, 기대, 과거경험)과 외적(평가의 성격, 퍼실리테이터의 존재, 평가에 대한 introduction) 요인에 영향을 받는다. 이러한 요인들에 대해서 인식하는 것이 효과적인 교육 전략을 개발하고, 평가결과를 해석하고, 성찰 프로세스에 대하여 이해를 높이는데 중요할 것이다.

Because reflection is a metacognitive process, it can only be assessed indirectly; through written reflections in vignettes or portfolios, or spoken expressions in inter- views. These methods do not allow assessors to verify information related to the reflections reported, which is a serious limitation. The widespread use of self- assessment questionnaires shares both that validity pro- blem and the inherent limitations of self-assessment. To counter these validity threats, it has been proposed that assessment should focus on the process rather than the subjectively coloured content of reflection. In addition, as reflections are intimately entangled with their trigger- ing situational context, we suggest where possible to consider objective information about this triggering situation allowing assessors to verify described reflec- tions. The reflection process is influenced by internal (eg. motivation, expectancy and prior experiences with reflection) and external factors (formative or summative character of assessment, presence of facilitators and introduction to the assessment). Awareness of these fac- tors are important to develop effective educational stra- tegies, interpreting assessment results and finally the increase in understanding about the reflection process.

실용 가이드라인

practical guidelines

• 1. Clearly define the concept of reflection and verify that all stakeholders (curriculum developers, students, assessors and supervisors) adopt the same definition and intended outcomes. 

• 2. Be specific about what level of reflection skills is expected, identifying good and inadequate reflection and communicate this to all stakeholders. 

• 3. Be aware of possible bias in self-assessment meth- ods, caused by inadequate ability to introspect. 

• 4. Provide assessors with a perspective on the situation triggering the reflection to create the ability to verify the described reflections in an objective frame of additional information. 

• 5. Consider and report contextual factors when asses- sing reflection and/or when engaging in reflective educa- tion in support to interpret the outcomes.

 

 

 

---

BMC Med Educ. 2011 Dec 28;11:104. doi: 10.1186/1472-6920-11-104.

Factors confounding the assessment of reflection: a critical review.

Koole S1, Dornan T, Aper L, Scherpbier A, Valcke M, Cohen-Schotanus J, Derese A.

Author information

• 1Centre for Educational Development, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium. sebastiaan.koole@ugent.be

Abstract

BACKGROUND:

Reflection on experience is an increasingly critical part of professional development and lifelong learning. There is, however, continuing uncertainty about how best to put principle into practice, particularly as regards assessment. This article explores those uncertainties in order to find practical ways of assessing reflection.

DISCUSSION:

We critically review four problems: 1. Inconsistent definitions of reflection; 2. Lack of standards to determine (in)adequate reflection; 3. Factors that complicate assessment; 4. Internal and external contextual factors affecting the assessment of reflection.

SUMMARY:

To address the problem of inconsistency, we identified processes that were common to a number of widely quoted theories and synthesised a model, which yielded six indicators that could be used in assessment instruments. We arrived at the conclusion that, until further progress has been made in defining standards, assessment must depend on developing and communicating local consensus between stakeholders (students, practitioners, teachers, supervisors, curriculum developers) about what is expected in exercises and formal tests. Major factors that complicate assessment are the subjective nature of reflection's content and the dependency on descriptions by persons being assessed about theirreflection process, without any objective means of verification. To counter these validity threats, we suggest that assessment should focus on generic process skills rather than the subjective content of reflection and where possible to consider objective information about the triggering situation to verify described reflections. Finally, internal and external contextual factors such as motivation, instruction, character of assessment(formative or summative) and the ability of individual learning environments to stimulate reflection should be considered.

PMID:

 

22204704

 

[PubMed - indexed for MEDLINE] 

PMCID:

 

PMC3268719

 

Free PMC Article