TOP 10 HINTS FOR UNDERSTANDING YOUR ONTOLOGY, EPISTEMOLOGY AND METHODOLOGY

1. Words에 대해 걱정하지 마십시오.

"-올로지"의 단어들은 심지어 그들이 파생된 언어인 그리스어에서도 흔하게 사용되지 않는다.

1. DON’T WORRY ABOUT THE WORDS

The “ology” words are not commonly used even in Greek, the language from which they are derived.

당신이 연구를 위해 글을 쓰기 시작할 때, 당신은 도전적인 학문적 언어를 이해할 필요가 있을 것이다. 특히 다음과 같은 세 가지 매우 중요한 개념 위에 올라설 필요가 있다. 종양학, 종양학, 방법론. 뚜렷한 이유 없이, 연구 철학은 연구생들을 가벼운 공황 상태에 빠뜨리는 경향이 있다. 당신이 성취하고자 하는 모든 것이 당신의 연구에서 당신이 주장하는 지식의 상태에 대한 약간의 명확성이 있을 때 지식의 철학과 관련된 다양한 새로운 용어에 의해 야기되는 당혹감은 불필요하다. 사회과학의 광범위한 맥락 안에서, 당신이 착수할 계획인 특정한 형태의 연구를 명시하기 위해 요구되는 표준적인 철학적 입장이 있다. 

When you begin writing for research you’ll need to get to grips with some challenging academic language. In particular, you need to get on top of three very important concepts: Ontology, Epistemology and Methodology. For no apparent reason, research philosophy tends to send research students into a mild panic. The befuddlement caused by a range of new terminology relating to the philosophy of knowledge is unnecessary when all that you are trying to achieve is some clarity over the status of any knowledge claims you make in your study. Within the broader context of the social sciences, there are standard philosophical positions required to specify the particular form of research you plan to undertake. 

종합적으로, 이러한 자세들은 연구 설계라고 불리는 것을 정의하게 될 것이다. 연구 설계를 포괄적으로 명시하기 위해 연구자인 당신이 연구 실행 계획을 지정할 때 해야 할 다섯 가지 연동 선택이 있다: 1. 온톨로지랑 2번. 인지학(이것들이 함께 당신의 연구 패러다임을 형성한다)은 3이다. 방법론 4. 기술(데이터 수집) 및 5. 데이터 분석 접근.  연구에 착수하는 '올바른' 방법은 단 한 가지도 없지만, 각기 고유한 전통이 있는데, 그 전통들은 각자 자기 자신의, 내적으로 일관된, 선택의 집합을 가지고 운용하는 경향이 있다.

Collectively, these positions will define what is sometimes referred to as a research design. To comprehensively specify your research design there are five interlocking choices that you, the researcher, should make when specifying how you plan to execute your research: 1. Ontology  and 2. Epistemology (which together form your research paradigm) then 3. Methodology 4. Techniques (your data gathering) and 5. Data Analysis Approaches.  There is no single ‘right’ way to undertake research, but there are distinct traditions, each of which tends to operate with its own, internally consistent, set of choices.

2. OnTOLOGY 선택

온톨로지는 현실의 본질이라는 사소한 문제를 다루는 철학의 한 분야다.

2. CHOOSE YOUR ONTOLOGY

Ontology is the branch of philosophy that deals with the trivial issue of the nature of reality.

온톨로지적인 위치를 선택할 때, 당신은 세계의 본성과 그 안에서 당신의 위치를 정하고 있다.  간단하지만 근본적인 것.  온톨로지는 학원을 넘어서는 거의 사용되지 않기 때문에 그것을 어떻게 자신 있게 사용하는지 아는 것은 어려울 수 있다. '생물학'이라는 단어는 생명에 대한 연구를 의미한다. 같은 논리를 사용한다면, 'on'to'는 'being' 또는 'reality'로 번역되므로, 온톨로지는 현실의 본질에 관한 것이다.  공상과학소설이나 판타지소설의 영역을 넘어서, 우리는 오직 하나의 현실만이 존재한다는 관점을 가지고 일상을 돌아다니는 경향이 있다. 그러나 매트릭스, 나니아 그리고 많은 다른 소설들은 이것이 불필요하게 제한된 세계관이라는 생각에서 영감을 받았다. 

In choosing an ontological position, you are setting out the nature of the world and your place within it.  Simple yet fundamental stuff.  Ontology is rarely used beyond academic institutions and it can therefore be difficult to know how to use it confidently. The word ‘biology’ means the study of life (since ‘bios’ means life). Using the same logic, ‘onto’ translates as ‘being’ or ‘reality’ hence ontology concerns the nature of reality.  Beyond the realms of science fiction or fantasy novels, we tend to go about our daily lives with a view that there is only one reality. Yet the Matrix, Narnia and many other fictions are inspired by the idea that this is an unnecessarily limited view of the world. 

아마도 이것들 중 가장 잘 알려진 것은 과학자들이 우리가 현실이라고 부르는 것에 대한 현실적인 참여의식을 모방할 정도로 정밀하게 신체 부위가 없는 뇌를 자극하는 브레인-인-바트 시나리오일 것이다. 뇌는 현실을 경험하는가, 아니면 과학자의 경험이 어떻게든 더 현실인가?

Perhaps, the most well-known of these is the brain-in-a-vat scenario, whereby scientists stimulate a disembodied brain with such precision that it emulates a realistic sense of participation in what we call reality. Does the brain experience reality, or is the experience of the scientist somehow more real?

3. 당신의 EPISTEMOLOGY를 알고 있다.

인식론은 지식의 본질을 다루는 철학의 한 분야로, 따라서 새로운 지식을 기여하기 위한 모든 연구 주장의 핵심이다.

3. KNOW YOUR EPISTEMOLOGY

Epistemology is the branch of philosophy that deals with the nature of knowledge and is therefore central to any research claims to contribute new knowledge.

인식론은 우리가 올바른 지식을 얻는 방법에 관한 것이다. 예를 들어, 만약 여러분이 시간을 달라는 요청을 받고 시계 없이 정확하게 추측한다면, 이것이 믿을 만한 지식인가? 아니면 이 추측이 어떻게든 검증되어야 하는가? 관련 시간을 알리는 익숙한 삐 소리가 정확한 시간에 대한 확증을 나타낸다.  아니면, AM, FM, 디지털 형태의 라디오 전송이 특정 장치를 통해 재생될 때 다양한 지연을 발생시킬 수 있다는 사실을 알고 싶지 않으십니까? 그 시간의 검증된 정확성에 대한 중요성은 당신이 알아야 할 상황에 달려있다.  연결 항공편을 이용하려는 경우 허용되는 변동 수준은 몇 분으로 연장될 수 있다. 만약 당신이 올림픽 개막식을 안무하려고 한다면, 아마 그렇지 않을 것이다.  

Epistemology concerns the way(s) in which we set about obtaining valid knowledge. For instance, if you are asked for the time, and guess it correctly without a watch, is this reliable knowledge? Or should this guess be verified somehow? Would hearing the familiar beeping that announces the time having struck the relevant hour represent definitive confirmation of the precise time.  Or, would you be unsettled to know that transmissions in AM, FM and digital forms of radio can generate varying delays when replayed through particular devices? The importance placed on the verified accuracy of the time would depend upon the context in which you need to know.  If you’re trying to catch a connecting flight the acceptable level of variation may extend to a few minutes. If you are trying to choreograph an Olympic opening ceremony it probably doesn’t.  

인식론이라는 용어는 또한 해체될 수 있다; 'episteme'은 지식을 의미하며 문자 그대로, 인식론은 지식의 학문이다.  우리가 유효한 지식을 얻을 수 있는 방법에 대해 명확히 함으로서, 우리가 할 수 있는 모든 지식의 본질에 대해 명확하게 함.  행복한 근로자들이 더 생산적이 되는 경향이 있다는 관찰은 그러한 지식의 주장 중 하나이다.  연구자로서, 여러분은 행복, 생산성 또는 둘 사이의 관계에 영향을 미칠 수 있는 다른 요소들을 언급하면서 그러한 주장에 대한 타당성에 대해 토론하기를 원할 수도 있다.  따라서, 우리는 종종 형식적이고 다소 잘못 표시된 방법 이외의 다른 방법에서는 명시적으로 그렇게 하지 않는 경향이 있더라도, 현실에 대해 우리가 가지고 있는 가정(온톨로지)과 유효한 지식(에피스테몰로지)을 개발할 수 있는 방법 사이에 연결고리를 그려야 한다.

The term epistemology can be also be deconstructed; ‘episteme’ means knowledge and in literal terms, epistemology is the study of knowledge.  By being clear about the way(s) in which we might obtain valid knowledge we are in turn being clear about the nature of any knowledge claim that we might make.  The observation that happier workers tend to be more productive is one such knowledge claim.  As a researcher, you may wish to debate the validity of such a claim, citing other factors that might influence happiness, productivity, or the relationship between the two.  Hence, we are required to draw connections between the assumptions we hold about reality (ontology) and the ways in which we might develop valid knowledge (epistemology), even if we often tend not to do so explicitly in anything other than the formal, and somewhat erroneously labelled, setting of a methodology chapter.

4. 방법론 설명

양적 및 질적 비교를 기본값으로 하지 말고 보다 정교한 용어로 방법론을 정의하십시오.

4. ESTABLISH YOUR METHODOLOGY

Don’t default to contrasting quantitative and qualitative, define your methodology in more sophisticated terms.

방법론은 난해한 단어 중에서 가장 많이 사용된다. 그것은 당신의 인식론, 온톨로지, 방법론이 상호 연결되는 방법의 shorthand로 사용되는 경향이 있다. 이러한 암묵적 경로가 고정되어 있지 않다는 것을 알아두는 것이 중요하지만, 특정한 연구 패러다임으로부터 특정한 데이터 수집과 분석 방법을 따르는 경향이 있다. 진정으로 중요한 것은 자신의 연구 설계를 나타내는 연동 선택을 인식하고 정당화할 수 있는 능력이다. 그것은 근본적으로 박사 시험관이나 저널 편집자가 당신의 방법론 챕터/섹션을 읽을 때 찾고 있는 것이다. 

Methodology is the most commonly used of the ology words. It tends to be used as a shorthand for the ways in which your epistemology, ontology and methodology interconnect. Certain methods of data gathering and analysis tend to follow from certain research paradigms, although it is important to notice that these implied pathways are not fixed. What is truly important is your ability to recognise and justify the interlocking choices which represent your own research design. That is essentially what any PhD examiner or journal editor is looking for when reading your methodology chapter/section.  

실증주의적 인식론적 접근방식으로 객관적 온톨로지를 표현하는 사람은 흔히 관습적이고 과학적인 전통으로 보여지는 것에 자연스럽게 맞춰지는 두 가지 선택을 할 것이다.  그러한 전통 안에서 행복한 노동자들이 더 생산적인지 이해하려고 노력하는 것은 통계적 기법, 통제 그룹 그리고 행복과 생산성 사이의 신뢰할 수 있는 관계를 규정하는 일반화 가능한 법률의 생성을 포함할 가능성이 높다.  동일한 연구 주제는 보다 해석주의적인 접근법을 생성하는 주관적인 온톨로지로부터 똑같이 접근될 수 있지만, 연구 자체와 청구권의 성격 모두 근본적으로 다를 것이다. 당신이 질적(또는 그 반대)보다 양적( chose的)을 선택했다고 독자들에게 말하는 것은 단순히 그것을 줄이지는 않는다.

Someone expressing an objective ontology with a positivist epistemological approach would be making two choices that are naturally aligned in what would often be seen as the conventional and scientific tradition.  Trying to understand whether happy workers are more productive from within such a tradition would likely involve statistical techniques, control groups and the generation of generalizable laws setting out reliable relationships between happiness and productivity.  The same research topic could equally be approached from a subjective ontology generating a more interpretivist approach but both the research itself and the nature of the claims made would be fundamentally different. Telling the reader that you chose quantitative over qualitative (or vice versa) simply doesn’t cut it.

5. 그렇다면 온토로그, EPISTEMOLOGY, METHODOLOGY의 차이점은 무엇인가?

그들은 각각 당신이 당신의 연구를 통해 주장하는 지식의 측면들을 제시한다.

5. SO WHAT IS THE DIFFERENCE BETWEEN ONTOLOGY, EPISTEMOLOGY AND METHODOLOGY?

They each set out aspects of the knowledge claim you are making from your research

간단히 말해서, 온톨로지(Ontology)는 현실의 본질에 대해 우리가 하는 가정과 관련되며, 인식론은 어떻게 그 현실에 대한 지식을 발견하게 될 것인가에 대한 믿음을 설정하며, 방법론은 우리가 연구를 수행할 때 사용하는 도구와 기법을 명시한다.  비판적으로 이 세 단어는 서로 관계를 형성한다. 당신의 온톨로지 및 인식론적 입장은 당신의 방법론과 어느 정도 관련이 있어야 하며, 이것은 당신이 채택할 데이터 수집과 분석 기법을 제시한다(물론 당신의 온톨로지 및 인식론이 데이터나 분석의 바로 그 아이디어에 도전하지 않는다고 가정한다). 사회과학에서 이러한 개별적 개념과 그 상호간의 관계는 당신이 출판하기 위해 명확하게 글을 쓸 수 있고, (2) 당신의 viva / 논문 / 논문에서 사회적 실수를 피하기를 원한다면 필수적이다.

Simply put, ontology relates to the assumptions we make about the nature of reality, epistemology sets out beliefs about how one might discover knowledge about that reality and methodology specifies the tools and techniques that we use in the conduct of our research.  Critically, these three words form relationships to each other. You ontological and epistemological positions should have some bearing on your methodology, which in turn sets out the data collection and analysis techniques that you will employ (assuming of course that your ontology and epistemology don’t challenge the very idea of either data or analyses). In the social sciences getting on top of these individual concepts and their relationship(s) to each other is vital if you want to (a) be able to write articulately for publication and (b) want to avoid social gaffes in your viva / thesis / dissertation.

6. IDEALLY, 가장 마지막에 번에 테크닉을 선택하라

-올로지 에 대한 위치를 잡을 때까지 데이터를 수집하기 시작하지 마십시오.  기술은 -올로지에서 흘러 나오고 그 반대는 아니다.

6. IDEALLY, CHOOSE YOUR TECHNIQUES LAST

Don’t start gathering data until you have taken a position on the ologies.  Techniques flow from ologies and not the other way around.

얼마나 많은 인터뷰가 충분할지에 대한 질문은 비판적으로 당신이 왜 인터뷰들을 하느냐에 달려있다. 당신은 면접을 할 수 있다: 사람들이 A가 B라고 말할 때, 그들은 B라고 말할 수 있다.  또는, 같은 면접관과 면접관은 B가 하지 않았을 때 어떻게 사람들이 A가 일어나는 것을 이해하는지 이해하기 시작할 수 있는 의미를 탐구하려고 할 수 있다.  그러므로 여러분의 연구 측면에서 좋은 관행이 될 수 있는 것은 여러분이 만들고 싶어하는 지식의 본질에 달려 있다. 당신은 -올로지과 관련하여 당신의 입장을 정함으로써 방어할 수 있는 입장을 분명히 말할 수 있을 것이다.  그렇기 때문에 박사학위는 철학 박사학위고, thesis를 '방어'해야 하는 것이다.

Asking how many interviews will be enough depends critically on why you are doing them. You could be doing interviews ‘as counting’: how many times when people say A do they also say B.  Alternatively, the same interviewer and interviewees could be trying to explore meaning such that you begin to understand how people make sense of A happening when B has not.  What would constitute good practice in terms of your research is therefore contingent on the nature of the knowledge claim that you hope to make. You will only be able to articulate a defensible position by setting out your position in relation to the ologies.  This is why a PhD is a doctorate in philosophy and why you have to “defend” your thesis.

7. 테크닉을 혼합하되 -올로지를 혼합하지 말라

혼합 데이터 수집 기법은 디 리그루르(de rigueur)이지만 혼합된 -올로지는 학문적 허실을 나타낸다.

7. MIX YOUR TECHNIQUES NOT YOUR OLOGIES

Mixed data collection techniques are de rigueur, however mixed ologies represent an academic faux pas

베건은 스테이크를 주문하는 일이 거의 없고, 민주주의자들은 공화주의자들을 투표하는 일이 거의 없다.  두 가지 옵션 모두, 가설적으로는 가능하지만, 신뢰할 수 없는 것으로 읽히는 경향이 있는 일관성의 결여를 나타낸다.  자신의 깊이를 벗어나거나 완전히 기만하는 것처럼 보이지 않도록 난색을 선택함에 있어 명확하고 일관적인 태도를 바꾸어야 한다. 개별 연구자들은 그들의 난을 혼합할 수 있지만 같은 연구 프로젝트에서는 그렇지 않다.  이 세 가지 핵심 개념은 철학에서 나오지만 용어를 이해하기 위해 철학을 공부할 필요는 없다.  본질적으로, 당신은 당신의 연구가 그들의 세계에 맞는 다른 연구원들에게 신호를 보내기 위해 당신의 연구 철학을 세울 필요가 있다. 만약 당신이 (박사 학위 또는 아마도 편집자나 검토자에 의해) 검사를 받는다면, 당신은 당신이 내부적으로 일관되는 의식적인 선택들에 관여했음을 보여줄 필요가 있다. 역사적으로, 어떤 연구 철학은 특정한 주제와 방법에 사용되었을지도 모르지만, 아이디어와 혼합 방법을 결합하는데 있어서 발견되는 혁신의 가능성을 무시하는 것은 무모할 것이다.

Vegan rarely order steak, democrats rarely vote republican.  Both options, whilst hypothetically possible, represent a lack of consistency that tends to be read as untrustworthy.  Be clear and consistent in your choice of ologies in order to avoid being seen as flaky, out of your depth or downright deceptive. Individual researchers can mix their ologies but not within the same research project.  These three key concepts emanate from philosophy but it isn’t necessary to have studied philosophy in order to make sense of the terminology.  In essence, you need to set out your research philosophy in order to signal to other researchers where your research fits in their world. If you are being examined (for a PhD or perhaps by an editor or reviewer), you need to show that you have engaged in a conscious set of choices that are internally consistent. Historically, certain research philosophies may have been used for certain topics and methods, yet it would be foolhardy to dismiss the potential for innovation to be found in combining ideas and mixing methods.

8. 당신의 영웅을 분류하라

당신의 분야의 신학파 작가들은 아마도 그들의 작문 작품에서 그들의 오리고기를 선택한 것을 명시적으로 표현하지 않을 것이다. 하지만, 당신은 그들의 작품을 분류할 수 있어야 한다.

8. CLASSIFY YOUR HEROES

The seminal authors on your field will probably don’t state their choice of ologies explicitly in their written work. However, you should be able to classify their works

당신 분야의 신학 작가들은 많은 사람들이 읽었을 것이다. 이것이 그들에게 영웅으로서의 지위를 부여하는 것이며, 종종 국제 회의와 공항 라운지의 커피 휴식 시간에 그들에게 확실한 기사로 지명될 권리를 갖게 한다. 그들의 광범위한 독자성과 중량감 있는 H 지수에도 불구하고, 그들은 아마도 그들의 글로 된 출력물에서 모호한 단어를 사용하지 않을 것이다. 실제로, 질적 방법론을 채택하면서, 주관적 온톨로지 내에서 연구가 진행되었고 인식론에서 해석론자였다는 논문을 찾는 일은 비교적 드물다.  여기에는 여러 가지 이유가 있는데, 적어도 지금 여러분의 마음속에 떠오르는 것은 그렇다! 그러나, 이러한 용어에 대한 당신의 이해를 확인하는 수단으로서, 당신은 당신의 분야에서 신학계의 인물들의 경험적인 작품을 분류할 수도 있고 또 분류하려고 시도해야 한다.

The seminal authors in your field will have been read by many. This is what confers on them their status as a hero, often earning them the right to be named as the definite article in the coffee breaks of international conferences and airport lounges … “that’s THE [insert name]”.  Despite their extensive readership and weighty H index, they probably don’t use the ology words in their written output. Indeed, it is relatively rare to find a paper that states that the research was conducted from within a subjective ontology and was interpretivist in its epistemology, whilst adopting a qualitative methodology.  There are many reasons for this, not least the one that is springing into your mind just now! However, as a means of checking your understanding of these terms, you could and should attempt to classify the empirical works of the seminal figures in your field.

9. 단순한 사례를 생각해라.

무엇을 공부하든 간에 온도[그래픽 – 온도계]와 같은 간단한 예를 사용하여 이러한 모호한 용어에 대한 이해를 확인하는 것이 도움이 된다.

9. THINK OF SIMPLE EXAMPLE.

Regardless of what you are studying it is helpful to check your understanding of these obscure terms using a simple example like temperature [Graphic – Thermometer]

경험적으로 실증주의자 관점에서 바깥 온도는 현재 +10.5이다.°C. 이것은 일반적으로 온도계를 사용하는 개별 관찰자에 의해 독립적으로 검증될 수 있는 모호하지 않은 사실로 제시될 수 있다. 대부분 누가 온도계를 잡고 있든, 혹은 누가 판독을 하고 있든 상관없다, 그것은 여전히 +10.5를 읽어야 한다. 

From an empirically positivist point of view the temperature outside is currently +10.5°C. This could be presented as an unambiguous fact, verifiable independently by individual observers normally using a thermometer. Largely it shouldn’t matter who is holding the thermometer or taking the reading, it should still read +10.5°C.  

반면에, 기온에 대한 구성주의적인 관점은 사회적 규범, 양육 그리고 믿음의 영향을 받을 것이다. 그것은 문맥과 개인마다 다르기 때문에 누가 온도계를 잡고 있느냐가 매우 중요할 것이다. 어린 시절을 적도 근처에서 보낸 사람은 +10.5를 찾을 수 있을 것이다.°C는 확실히 쌀쌀한 반면, 북극권에서 어린 시절을 보낸 누군가는 북극이 긍정적으로 상쾌하다고 느낄지도 모른다.  더 많은 뉘앙스는 따뜻한 옷이 발랄함의 표시인지 아니면 어떤 면에서는 의지가 약하다는 표시인지를 고려함으로써 드러날 것이다. 가족 휴일에 툰드라/사구(적절한 경우 삭제)에서 보내는 유년시절의 기억은 온도에 대한 인식에 더 많은 색을 더할 것이다. 무엇보다도, 여러분이 처음 오글리를 선택했을 때, 그 연구는 여러분의 연구를 위한 적절한 방법으로 온도계나 일기 연구를 선택해야 한다는 것을 기억하라.

In contrast, a constructionist view of temperature would be influenced by social norms, upbringing and beliefs. It would vary between contexts and individual such that it would matter very much who was holding the thermometer. Someone whose childhood was spent near the equator would find +10.5°C decidedly chilly whereas someone whose childhood was spent in the Arctic Circle might find it positively balmy.  Further nuances would be revealed by considering whether warm clothing was seen as a sign of opulence or an indication that you were in some way weak-willed. Fond childhood memories of family holidays spend on the tundra / sand dunes (delete as appropriate) would likely add further colour to one’s perception of the temperature. Remember above all that you, the research should choose a thermometer or a diary study as the appropriate methods for your study once you have made your initial choice of ology.

10. 슈퍼바이저를 확인

당신의 영웅들을 분류한 후, 당신의 상사가 당신의 분류에 동의하는지 여부를 확인한다.

10. CHECK IN WITH YOUR SUPERVISOR

Having classified some of your heroes check whether your supervisor agrees with your classification.

첫째, 당신의 상사를 영웅으로 분류하도록 하라.  편리한 망토로 무장한 타이트를 입은 그들의 생각이 여러분의 잠재의식에 불편하게 뿌리내리게 되더라도, 그것은 감독 과정이 잘 진행되도록 도와줄 것이다(졸업날에도 불구하고, 어떤 실제적인 분장 사례도 보고하고 싶을지도 모른다).  둘째, 당신의 실제 영웅들 중 몇몇은 당신의 감독관들에게도 영웅이 될 가능성이 있다.  이것은 그들의 경험적 작품들 중 일부가 잘 알려질 것이고 당신과 당신의 감독관들에게 공통된 참조점을 나타내야 한다는 것을 의미해야 한다. 인터뷰, 설문지, 포커스 그룹 등과 같은 다른 방법을 찾아보고 당신의 영웅이 같은 연구나 다른 연구에 이런 것들을 배치하는지, 그리고 연구 프로젝트를 통해 당신의 영웅들이 한 세트의 오로리에서 다른 세트로 이행하는지 자문해 보십시오.   마지막으로, 동일한 연구 주제를 다른 일련의 오로지로부터 접근하면 어떻게 보일지 생각해 보라.

First, be sure to classify your supervisors as a heroes.  Even if the thought of them in tights armed with a handy cape become uncomfortably rooted in your subconscious, it will help the supervision process go well (though you may wish to report any actual instances of dressing up, even on graduation day).  Second, some of your actual heroes are likely to be heroes to your supervisors too.  This should mean that some of their empirical works will be well known and should represent shared points of reference for you and your supervisors. Look for different method e.g. interviews, questionnaires, focus groups etc. and ask yourself if your heroes deploy these in the same or different studies and whether, across research projects, your heroes transition from one set of ologies to another.   Finally, reflect on what the same research topic would look like approached from a different set of ologies.

http://www.itsnotyouitsyourdata.com/the-early-years/top-10-hints-for-understanding-your-ontology-epistemology-and-methodology/

선택의 목적: 대선에서 적폐를 청산하고 미래를 이끌어갈 가장 적합한 사람이 당선된다.

[1]#내용 타당도: 공약에 위 목적과 관련된 내용이 포함되어 있는가?
⭕타당한 선택: A후보는 공약에서 '적폐청산'과 '미래'를 강조하고 있다.
❌타당하지 못한 선택: B후보는 공약에서 '적폐청산'과 '미래'를 전혀 강조하고 있지 않다.

웹 설문조사를 위한 Scalar 질문 설계(Sociological Methods & Research, 2009)

Designing Scalar Questions for Web Surveys

Leah Melani Christian

Pew Research Center for the People & the Press, Washington, DC

Nicholas L. Parsons

Eastern Connecticut State University, Willimantic, CT

Don A. Dillman

Washington State University, Pullman, WA

고찰 및 결론

Discussion and Conclusion

지난 연구와 우리 실험의 결과  응답자들은 문항의 워딩이나 보기에 붙은 label 그 이상의 것에 영향을 받는다. 특히, 응답자는 척도의 전반적인 레이아웃에서 정보를 얻는데, 여기에는 다른 보기와의 물리적 위치관계, 숫자 라벨의 활용 여부 등이 있다. 시각적 설계 효과visual design effect에 관한 다른 연구 결과와 유사하게, 우리 연구에서도 약간의 visual manipulation이 크고 유의한 차이를 냈음을 보여준다.

Results from past research and our experiments indicate that respon- dents often rely on more than just the question wording and verbal cate- gory labels when answering scalar questions. Specifically, respondents gain information from the overall layout of the scale, each category’s phy- sical position in relation to other responses on the scale, and any numeric labels that may be used. Similar to other research on visual design effects, our results indicate that some visual manipulations of the response scale produce large and significant differences and others more minor differ- ences (see Table 7 for a summary of our results).

일관된 순서(긍정적->부정적 순서 와 부정적->긍정적 순서 를 섞거나 안 섞거나)로 보기를 배치한 것은 응답 성향에는 영향이 없었으나 일관된 순서로 한게 더 대답을 빠르게 하는 결과가 있었다. 보기의 순서가 일관되었을 때, 즉 "높은 번호가 긍정적인 것up means good" 휴리스틱이 발동했다고 볼 수 있으며, 척도를 인지하고 이해하는데 시간이 덜 걸려서 더 빠르게 응답했을 것이다. 추가로, 보기를 한 칼럼으로 배치한 것은 응답자가 척도를 처리하는 속도를 빠르게 하여 선택을 내리기 더 쉽게 만든다. 반면, 응답을 여러 칼럼으로 배치하는 것은 답을 선택하는데 더 오래 걸리게 한다. 중간점midpoint는 중요한 anchor가 된다. '모르겠다' 응답으로서 midpoint를 분리하는 것(시각적으로, 개념적으로 midpoint가 align되어 있기만 하면) 은 응답에 영향을 주지 않는다.

Overall, we find that consistently presenting the positive end of the scale first for all questions did not affect people’s responses, but it did result in respondents providing a response more quickly. When the options are pre- sented consistently with the ‘‘up means good’’ heuristic (Tourangeau et al. 2004), it seems to take less time for respondents to perceive and comprehend the scale, so they can provide a response faster. In addition, presenting the options linearly in one column facilitates respondents’ processing of the scale and encourages them to process the categories in the same order, making it easier to provide a response. In contrast, presenting the categories in multiple columns different responses based on whether results in respondents process the options horizontally or vertically and in respondents taking longer to pro- vide an answer. Our findings confirm that the midpoint is an important anchor for respondents when answering scalar questions. Separating the midpoint, don’t know option, or endpoints through the use of space does not seem to affect responses as long as the visual and conceptual midpoints of the scale are aligned.

추가로, graphic layout을 완전히 없애고, scale에서 각 척도가 몇 점인지만 설명해주고 직접 점수를 쓰게 했을 때는 응답에 크게 영향을 주었고, 응답 시간도 늘어났다. 그러나 이전 연구 결과를 보면, 점수가 응답자의 expectation과 일관되게 배정되어 있으면 (높은 점수가 긍정적 응답) graphic display를 없애는 것이 응답에 영향을 주지 않았다. 유사하게, 과거 연구에서 음수negative number를 사용하는 것이 양수positive number만 사용하는 것에 비해서 응답에 크게 영향을 주었다. 마지막으로 우리 연구결과에서 응답자들은 numerical label의 양 끝에만 설명이 붙어있을 경우 응답에 더 오랜 시간이 걸렸다(응답 결과에 차이는 없었다)

Additionally, we find that removing the graphic layout of the scale entirely and providing only a verbal description of the scale in the question stem significantly influence respondents’ answers and increase response time when lower numbers indicate more positive categories. However, pre- vious research has shown that when the numbers are assigned consistently with respondents’ expectations, with higher number assigned to positive categories, removing the graphic display of the scale does not significantly affect responses (Christian et al. 2007a). Similarly, past research has shown that the use of negative numbers dramatically affects responses compared to the use of only positive numbers (Schwarz et al. 1991; Tourangeau et al. 2007). Finally, our results demonstrate that respondents take longer to respond to polar point scales with numeric labels because of the additional information they must cognitively process; however, there are no statisti- cally significant differences in responses.

전통적으로 설문 방법론자들은 워딩에만 초점을 두었으며, 시각적으로 어떻게 보기를 배치하는가에는 관심이 별로 없었다. 이는 종종 설문을 하는 사람들이 (비교를 목적으로) 다른 설문에서 워딩을 그대로 배끼면서 (공간 문제로) 시각적 레이아웃은 바꾸는 결과를 가져왔다.

Traditionally, survey methodologists have focused on wording alone when constructing response scales, and the visual presentation of response scales has received little attention (Krosnick and Fabrigar 1997). This has sometimes led to the unfortunate practice of surveyors replicating the wording of questions from one study to another with the intent of compar- ing responses across studies, but changing the visual layout to fit with the style of a particular questionnaire design or because of space limitations.

이번 연구와 예전 연구에서, 모든 시각적 제시 방법의 차이가 응답에 영향을 가져오는 것은 아님을 보여주었다. 추가로, verbal label이나 numeric label이 (개개의 질문에 대한 답에 초점을 둔다면) visual cue에 비해서 더 영향력이 클 수 있다. 그러나 어떻게 visual/numeric/symbolic 정보가 verbal 정보보다 더 강력할 수 있는지도 보여진 바 있다. 특히 응답자가 정보를 처음 처리할 때에는 언어 정보보다 시각 정보가 더 우선권을 갖는다.

As shown here and in previous research, not all differences in the visual presentation of scalar questions significantly influence respondents’ answers. In addition, verbal and numeric labels may take precedence over subtle visual cues, particularly when respondents are focused on respond- ing to individual questions (Tourangeau et al. 2007). However, it has also been demonstrated how visual, numeric, and symbolic information can be more powerful than verbal information (Christian, Dillman, and Smyth 2007b; Redline et al. 2003). In particular, visual information often takes precedence over verbal information when respondents first process the information presented (Ware 2004). The development of an overall theory of when and why visual layout makes a difference remains in its early stages (see Dillman et al. 2009).

함의점. 종이 또는 웹 설문을 개발할 때, 얼마나 보기를 많이 제시할지 뿐만 아니라, 어떤 언어적 라벨을 붙일지, 그리고 시각적으로 어떻게 제시할지도 고민해야 한다. 또한 과거 설문을 활용하여 결과를 비교하려고 할 때, 워딩 뿐만 아니라 visual quality 에도 신경을 써야 한다. 또 다른 함의는 mixed-model 설문이 점차 많이 사용되고 있는데, 설문을 받는 사람들은 여러 설문 방식간 유사성을 유지하는데 신경써야 한다.

Our overall finding that the visual design of response scales makes a difference has significant practical implications. When designing response scales for paper and Web questionnaires, surveyors must not only consider how many categories to offer and what verbal labels to provide but also how the scale will be presented visually to respondents. It also seems important that when surveyors use questions from earlier surveys with the intent of comparing results, they attempt to maintain the visual qualities of previous questions as well as the question wording. Another implication is that at a time when mixed-mode surveys are being conducted with greater frequency, surveyors must recognize that attention needs to be given to maintaining similarity across survey modes.

마지막으로 응답자의 부담burden에 대한 함의. 전통적으로 응답자의 부담은 문항의 워딩이나 내용에 따른 것이라고 생각했다. 여기에서의 실험 결과는 응답 결과에 영향을 주지 않더라도, 어떤 시각적 레이아웃은 정보를 더 빠르게 처리하는데 도움이 된다는 것을 보여준다. (숫자 없이 polar point scale만 제시하는 것). 또 어떤 경우에서는 시각적 디자인에 따라 결과가 달라질 뿐만 아니라 응답 시간에도 영향을 준다(polar point vs 숫자 기입하기).  응답시간은 응답자가 정보처리가 어려울 경우 더 길어진다. 따라서 paradata를 통해서 어떤 문항이 특히 burdensome했고, 혹은 응답자가 잘 못 이해했고, 어떤 문항을 먼저 응답했는지 등을 명확하게 보여준다.

Finally, the results presented here provide insight into respondent bur- den. Traditionally, respondent burden has been thought of as an attribute of a question that was dependent on words and substance. The experimental data reported here show that even when people’s responses are not affected, some visual layouts are more quickly processed than others (e.g., polar point scales without numbers). In other instances, not only does the visual design influence responses, but also it can affect the time taken to provide a response (e.g., polar point vs. number box when the more positive answers are assigned lower numbers). Response time is longer for formats that are difficult for respondents to process. It is increasingly clear that paradata have multiple uses for survey methodologists, by helping them to under- stand when questions are especially burdensome or misunderstood by respondents as well as the order in which respondents complete questions and select individual answers.

Designing Scalar Questions for Web Surveys

역량이란 무엇인가? (Human Resource Development International, 2005)

What Is Competence?

Françoise Delamare Le Deist & Jonathan Winterton

competence 또는 competency 의 개념은 (competence는 일반적으로 functional area를 말하며, competency는 behavioral area를 말하나, 그 활용에 일관성은 별로 없다.) 경영전략 논문에서 1990년를 군림했다. 이 분야에서는 'core competence'를 조직의 핵심 리소스로 강조했으며, 경쟁력을 갖추기 위해서 개발해야 할 것이었다. Hamel과 Prahalad는 CC를 ‘조직의 집단 수준의 학습으로서, 특히 어떻게 다양한 생산기술을 조화롭게 하고 다양한 기술의 흐름을 통합하느냐에 대한 것' 이라 정의했다.

The concept of competence or competency (‘competence’ generally refers to functional areas and ‘competency’ to behavioural areas but usage is inconsistent, as shown below) dominated the management strategy literature of the 1990s, which emphasized ‘core competence’ as a key organizational resource that could be exploited to gain competitive advantage (e.g. Campbell and Sommers Luchs, 1997; Mitrani et al., 1992; Nadler and Tushman, 1999). Hamel and Prahalad (1994) defined core competence as ‘the collective learning in the organisation, especially how to co-ordinate diverse production skills and integrate multiple streams of technologies’ (Prahalad and Hamel, 1990: 82).

경영 전략가들이 competences를 강조하면서 '유니크'하고 '기업 특이적인' 것을 강조했는데 HRD 문헌에서는 대부분의 직업이나 특정 직군에서 보다 transferable한 generic competence를 개발하는 것에 관심을 둔 것은 역설적이다. 경영전략에서의 접근과 HRd에서의 접근 사이에는 기본적으로 긴장관계가 있다. 만약 '개별 기관에 독특하고 특이한' 역량에 집중하면 generic 역량에 대한 관점이 협소해진다. Thompson 등이 언급한 바와 같이 작은 회사에서 generic list에 너무 집착하면 현재의 성공을 이루게 한 것을 훼손시킬 수도 있다고 하였다.

It is paradoxical that, while management strategists were emphasizing compe- tences that are unique and firm-specific, the HRD literature was more concerned with developing highly transferable generic competences that are required for most jobs or particular occupations or job roles (Le´vy-Leboyer, 1996; Stasz, 1997). There is an inherent tension between the strategy and HRD approaches. If concentrating on core competences that are ‘distinctive and specific to each individual organisation’ is what gives competitive advantage (Bergenhenegouwen et al., 1996), the scope for generic competence frameworks is limited; as Thompson et al. (1996) note, rigid adherence to a generic list for managers of a small firm may undermine the very things that have led to its current success.

generic 역량 접근법에 있어서 역량을 적절히 분류하는 것은 교육과 훈련을 통합하고, 노동시장의 요구를 맞추면서 개개인의 진로 발전을 촉진시키는 것에 중요하다. 역량이 중요하다는 것은 그것의 의미도 중요하다는 것이다. 공동의 이해 없이는 통합/정렬/기동성 모두가 불가능 하기 때문이다. 그러나 역량의 중심적 역할에도 불구하고, 이 용어를 둘러싼 혼란이 엄청난데 VET와 HRD 의 시스템, 문화나 구조에 있어서 그 용례가 일관되지 않고 독특한 개념들이 융합하는 것을 보여준다.

In relation to this generic competence approach, the development of an appropriate typology of competence is important for integrating education and training, aligning both with the needs of the labour market and promoting mobility for individuals (vertical as in career progression, lateral as in movement between sectors and spatial as in geographically), especially for workers faced with job insecurity (van der Klink and Boon, 2002). If competence is important, it follows that its meaning is also important, since without a common understanding there is little chance of integration, alignment or mobility in practice. However, despite the central role of competence, there is considerable confusion surrounding the term, which reflects conflation of distinct concepts and inconsistent usage as much as differences in systems, structures and cultures of HRDand VET.

'역량'에 대한 개념 혼란

Competence as a ‘Fuzzy Concept’

'역량'이란 단어를 사용하는 대한 무수한 방식을 조화시킬수 있는 일관된 이론이나 정의란 사실상 불가능하다. Norris는 '역량에 대한 암묵적 이해가 개념을 명확히 정의하고 조작화할 필요를 앞질러버렸다. 현실이 이론적 혼란을 덮고 있으며, 겉으로는 단순해 보이는 것이 엄청나게 복잡해졌다'라고 했다. Boon과 van der Klink는 역량을 '혼란스러운 개념'이라고 묘사하긴 했으나 교육과 직무요건의 갭을 채우는 데 유용하다'라고 했다. 

There is such confusion and debate concerning the concept of ‘competence’ that it is impossible to identify or impute a coherent theory or to arrive at a definition capable of accommodating and reconciling all the different ways that the term is used. As Norris argued, ‘as tacit understandings of the word [competence] have been overtaken by the need to define precisely and [to] operationalize concepts, the practical has become shrouded in theoretical confusion and the apparently simple has become profoundly complicated’ (1991: 332). Describing competence as a ‘fuzzy concept’, Boon and van der Klink nonetheless acknowledge it as a ‘useful term, bridging the gap between education and job requirements’ (2002: 6).

다양한 문화적 맥락이 역량에 대한 이해에 영향을 주고, 이것은 특히 인종/성별/연령/계층 등을 포함하는 문화적 문헌에 의해서 정의될 때 특히 중요하다. 역량을 사회-문화적 실천의 용어로 정의하고자 하는 몇몇 시도도 있었다.

Different cultural contexts influence the understanding of competence (Cseh, 2003) and this is especially important in relation to the extent to which competence is defined by cultural literacy involving group identities such as race, gender, age and class (ascription), as opposed to demonstrable behaviour (achievement). As Jeris and Johnson note, the distinction is confounded by the role of ascription in providing access to education and career opportunities that enable achievement: ‘As much as the behavioral and skill-based performance assessments portend to be ‘‘neutral and objective,’’ the ascriptive elements remain present and troubling for today’s increasingly diverse workplaces’ (2004: 1104). There have been few attempts (notably Boon & van der Klink, 2002 in the USA; Eraut, 1994 in the UK) to situate competence in terms of socio-cultural practices, which as Jeris and Johnson note:

조직문화와 직업현장의 맥락을 무시하는 것에 대해서 같은 주장을 할 수 있다. 왜냐하면 generic competence는 서로 다른 지식 영역간 전이가능하지 않을 수 있기 때문이다. 

The same argument can be made in relation to the neglect of organizational culture and workplace context, since generic competences may not be transferable across different knowledge domains (Burgoyne, 1989; Canning, 1990; Kilcourse, 1994). 

• 찬: 매니저에게 요구되는 스킬은 여러 직종에 걸쳐서 일반화가능하다
  The Job Competences Survey developed by Dulewicz and Herbert (1992) demonstrated that the skill needs of managers are sufficiently generic to permit generalizations across the occupation. Despite differences in the managerial function in different contexts, 
• 찬: 직장 특이적 역량은 전체 필요한 역량의 30%밖에 되지 않는다.
  Dulewicz (1989) found that firm-specific competencies represented only 30 per cent of the total competencies basket, while the remaining 70 per cent were common to a wide range of organizations. 
• 반: 여러 조직에서 같은 용어를 사용한다고 해서, 어디서나 통용되는 경영 역량을 정의할 수 있는 것은 아니다.
  However, Antonacopoulou and FitzGerald warn that the ‘fact that many organisations use the same terminology to describe a set of managerial characteristics is not a strong argument for claiming that it is possible to identify a set of universal management competencies’ (1996: 31). 
• 반: 합리주의적 접근이 역량을 추상적이고, 협소하고, 과도하게 단순화시켜서 복잡한 역량을 적절히 반영하지 못한다.
  Such critics claim that rationalist approaches create abstract, narrow and over- simplified descriptions of competence that fail adequately to reflect the complexity of competence in work performance (Attewell, 1990; Norris, 1991; Sandberg, 1994).

역량은 개개인이 지니는 것이라는 점에서 사회나 직무 특이적 맥락과 독립적이라고 생각할 수도 있지만, 여전히 '기술 수준'이란 것은 한 개인이 아니라 그 맥락이 가지는 특징이다. 사람들은 맥락과 동떨어져서 역량을 가질 수 없다. 현상학적 관점에서의 구성주의적 그리고 해석적 접근은 역량을 그것이 적용되는 맥락의 함수로 이해한다. ‘worker and work form one entity through lived experience of work’ 

Since competences are centred on the individual, they are viewed as independent of the social and task-specific context in which performance occurs, yet ‘skill level is a characteristic not only of a person but also of a context. People do not have competences independent of context’ (Fischer et al., 1993: 113). Constructivist andinterpretative approaches derived from phenomenology view competence as afunction of the context in which it is applied, where ‘worker and work form oneentity through lived experience of work’ (Sandberg, 2000: 50). Competence is constituted by the meaning that the work has for the worker in their experience(Stoof et al., 2002; Velde, 1999).

미국식 접근법: 행동주의적 접근

The Behavioural Approach: The US Tradition

많은 미국의 문헌들은 직무와 관련된 역량을 다루고 있으며, 행동적 역량과의 관계속에 있다. 예컨대 리더십 역량 모델에서는 조직/절차/개인 수준에서의 수행능력과 관련된 여섯개의 역량 영역을 정의하였다. 이 영역들은 더 세부적으로 나눠진다.

Much of the recent US literature focuses on job-related (functional) competences (Aragon and Johnson, 2002; Boon and van der Klink, 2002), often with associated underpinning behavioural competencies. For example, in the influential leadership competency model developed by Holton and Lynham (2000), six ‘competency domains’ are identified relating to performance at the organization, process and individual levels. These domains are broken down into ‘competency groups’ and then further divided into ‘sub-competencies’.

행동적 역량 접근법이 David McClelland and Hay-McBer 에 의해서 많이 발전했지만, 직무와 관련된 기능적 기술과 지식을 강조하는 역량에 대한 보다 넓은 개념이 힘을 얻고 있다.

While the behavioural competency approach promoted most notably by David McClelland and Hay-McBer is still much in evidence in the US, a broader conception of competence, which emphasizes also job-related functional skills and underpinning knowledge, is clearly gaining ground.

영국식 접근법: 기능적 접근

The Functional Approach: The UK Tradition

영국 정부는 1980년대부터 VET에 전국적으로 역량바탕접근을 도입했다. 

Recognizing endemic deficiencies of skill formation in the UK, governments during the 1980s introduced a competence-based approach to VET in order to establish a nation-wide unified system of work-based qualifications. This VET reform was driven by the adoption of a competence-based qualifications framework, which subsequently influenced similar developments in other countries in the Common- wealth and the European Union.

Hodkinson and Issitt 는 보다 전인적 접근법을 주장했다. 지식과 이해와 가치와 술기를 통합하였다. 유사하게 Cheetham and Chivers 는 다섯 개의 상호 연결된 역량을 통해서 종합적 모델을 만들었다.

Hodkinson and Issitt (1995: 149) argued for a more holistic approach to competence in the caring professions, integrating knowledge, understanding, values and skills that ‘reside within the person who is the practitioner.’ Similarly, Cheetham and Chivers (1996, 1998) claimed to develop a holistic model of professional competence, comprising five sets of inter-connected competences and competencies. Their competence framework comprises five dimensions:

• 인지적 역량 Cognitive competence, including underpinning theory and concepts, as well as informal tacit knowledge gained experientially. Knowledge (know-that), under- pinned by understanding (know-why), is distinguished from competence. 
• 기능적 역량 Functional competences (skills or know-how), those things that ‘a person who works in a given occupational area should be able to do. . .[and] able to demonstrate’. 
• 개인적 역량 Personal competency (behavioural competencies, ‘know how to behave’), defined as a ‘relatively enduring characteristic of a person causally related to effective or superior performance in a job’. 
• 윤리적 역량 Ethical competencies, defined as ‘the possession of appropriate personal and professional values and the ability to make sound judgements based upon these in work-related situations’. 
• 메타 역량 Meta-competencies, concerned with the ability to cope with uncertainty, as well as with learning and reflection.

프랑스, 독일, 오스트리아: 다차원적 총체적 접근

A Multi-dimensional and Holistic Approach: France, Germany and Austria

In mainland Europe, two other approaches are evident, exemplified by France and Germany, which each adopted competence in their approaches to HRD more recently. Most other European countries have followed the UK, French or German approaches (Winterton et al., 2005), so confining the discussion to these nonetheless permits a wider generalization of the issues.

역량의 유형 분류

Towards a Typology of Competence

이 리뷰에서는 단일 차원적 역량 프레임워크는 부적절하다고 주장한다.

This review has argued that one-dimensional frameworks of competence are inadequate and are giving way to multi-dimensional frameworks.

기능적, 인지적 역량은 점차 미국의 행동적 역량에 덧붙여지고 있으며, 영국의 인지적, 행동적 역량은 기능적 역량 모델에 덧붙여지고 있다. 프랑스, 독일, 오스트리아는 보다 총체적인 프레임워크로부터 시작하여 지식/술기/행동을 역량의 영역으로 보았다.

Functional and cognitive competences are increasing being added to the behavioural competencies in the USA, while in the UK cognitive and behavioural competences are being added to the occupational functional competence model. France, Germany and Austria,entering the arena more recently, appear to be adopting from the outset a moreholistic framework, considering knowledge, skills and behaviours as dimensions of competence, as have most of the other countries that have adopted competence-based vocational training. 

지식/술기/사회적 역량을 아우르는 전체론적 유형화가 유용하다. 

From this analysis, we argue that a holistic typology is useful in understanding the combination of knowledge, skills and social competences that are necessary for particular occupations.

한 직업에서 요구하는 역량은 conceptual하면서 operational 하다. 한 개인과 연관된 역량도 역시 conceptual하면서 operational하다. 

The competences required of an occupation include both 

• conceptual (cognitive, knowledge and understanding) and 
• operational (functional,psycho-motor and applied skill) competences. 

The competences more associated with individual effectiveness are also both 

• conceptual (meta-competence, including learning to learn) and 
• operational (social competence, including behaviours and attitudes)

처음의 세 가지 영역은 상당히 universal하며 명백히 전통적 KSA 혹은 프랑스적 접근과 비슷하다. 

The first three dimensions, cognitive, functional and social competences, are fairly universal and are clearly consistent with the French approach (savoir, savoir faire,savoir eˆtre) as well as the longstanding KSA (knowledge, skills and attitudes) of the training profession. Thus, 

• knowledge (and understanding) is captured by cognitive competence, 
• skills are captured by functional competence and 
• ‘competencies’(behavioural and attitudinal) are captured by social competence

메타-역량은 처음의 셋과 다른데, 다른 역량 획득을 촉진하는 것과 관련되어있기 때문이다.

Meta-competence is rather different from the first three dimensions since it is concerned with facilitating the acquisition of the other substantive competences.

전체론적 역량모델은 정사면체로 묘사될 수 있다.

The holistic competence model is perhaps better represented as a tetrahedron,reflecting the unity of competence and the difficulty of separating cognitive,functional and social dimensions in practice. In Figure 2, the holistic competence model is represented as a tetrahedron in plan view. Meta-competence is presented as an over-arching input that facilitates the acquisition of output competences at the base of the tetrahedron

Conclusion

The challenge is to develop a consistent and coherent typology of competence in a context where even within countries there is apparent diversity in the approaches.

Each of the four dominant approaches has particular strengths. 

• The traditional American approach has demonstrated the importance of individual characteristics and the use of behavioural competence as a means of developing superior performance. 
• The mainstream UK approach has shown the value of occupationally defined standards of functional competence and their applicability to the workplace.
• The approach adopted in France and Germany demonstrates the potential of a multi-dimensional and more analytical concept of competence. 

역량에 대한 국가적 접근법 있어서 통합의 움직임이 있다.

Moreover, there are signs of convergence in national approaches to competence, not only within Europe but also between the European and American models, suggesting that there is value

in a multi-dimensional approach for developing a more global understanding of the term.

Reviews

What Is Competence?

Full text HTML

PDF

Full access

DOI:

10.1080/1367886042000338227

Françoise Delamare Le Deist & Jonathan Winterton

pages 27-46

Publishing models and article dates explained

• Published online: 17 Feb 2007

Alert me

Big-five 성격의 안정성(Economic letters, 2012)

The stability of big-five personality traits

Deborah A. Cobb-Clark a,b, Stefanie Schurer c,∗

성인이 되어 겪는 고용, 건강, 가정사와 관련된 사건들과 성격의 어떤 부분이 안정적이고 독립적인지에 대한 연구가 거의 없다. 국가를 대표하는 패널 조사자료를 활용하여 다음의 세 가지 질문에 답해보았다.

• (1) 나이에 따라서 성격특성에 전반적인 변화가 있는가?
• (2) 생애 사건에 따라 성격이 변하는가?
• (3) 성인의 성격 변화가 경제적으로 의미가 있는가?

Surprisingly little evidence exists on the extent to which adult personality traits are stable and independent of the employment-, health-, and family-related events that people experience. In this note, we use data from a nationally representative panel survey that includes measures of individuals’ Big-Five personality traits both 2005 and 2009 to answer the following questions: (1) Does the overall change in personality traits depend on age? (2) Is adult personality change related to adverse life events? (3) Are changes in adult personality economically meaningful?

결과를 살펴보면, 경제활동을 하는 연령대의 성인(working-age)에서는 성격특성은 비교적 시간이 지나도 안정적으로 변화가 적었음. Big-five 성격의 평균 수준의 변화는 적었고, 그룹간 크게 차이가 나지 않았다. 또한 고용, 건강, 가정의 안 좋은 일이 있을 때 경제적으로 의미있는, 내적 성격 변화가 있다는 근거도 거의 없었다. 마지막으로, 이 결과에서 Big-Five 성격은 앞선 연구의 locus of control의 결과를 보여주는데, 즉 비인지적 스킬이 경제적 의사결정에서 비교적 안정적인 input임을 보여준다.

Making continued progress in our effort to assess the role of personality in economic behavior relies heavily on understanding the way that our standard measures of personality evolve over time as people age and their lives unfold. Our results indicate that – while not literally fixed – personality traits do appear to be stable among working-age adults. The mean-level changes in Big-Five personality traits are small and do not vary substantially across age groups. Moreover, there is little evidence that economically meaningful, intra-individual personality change can be linked to the adverse employment, health, or family events that individuals experience. Finally, these results for the Big-Five traits mirror previous results for locus of control (see Cobb-Clark and Schurer, 2011), suggesting that non-cognitive skills more generally may be seen as stable inputs into many economic decisions.

마음 혹은 덕: 학습에 관한 서양과 중국의 믿음 (Current Directions in Psychological Science, 2005)

Mind or Virtue: Western and Chinese Beliefs About Learning

Jin Li

소크라테스가 노예 Meno에게 피타고라스 정리를 가르칠 때, 소크라테스는 학생들이 스스로 지식을 발견할 수 있는 능력이 있다고 믿게 놔두었다. 이러한 소크라테스 식의 아이디어는 서양이 가르침과 배움을 바라보는 믿음에 깔려 있다. 최고의 학습자는 그의 마음(mind)를 발전시키고 사용하여 세상에 대한 질문을 잘 던지는 사람이다.

When Socrates tutored the slave boy Meno about the Pythagorean theorem, he led the pupil to believe that he possessed a mind capable of discovering knowledge. The Socratic ideal still underlies Western cultural beliefs about teaching and learning. The best learner is one who develops and uses his or her mind well to inquire into the world.

중국 문화도 학습에 대한 오랜 전통이 있는데, 하나는 맹자가 그의 제자를 가르친 예에서 드러난다. 그리스의 소크라테스와 달리, 맹자의 제자들은 힘없는 노예가 아니라 왕과 기사들이었다. 맹자가 그들에 대해서 동정심과 선에 대한 인간의 능력을 가르칠 때, 맹자는 그들에게 스스로를 도덕적으로 완벽하게 다듬지 않으면 힘이 없는 것(정치적 힘은 있더라도)이라고 믿게끔 하였다. 이러한 기본적인 방향 (도덕적으로 더 나은 사람이 되어야 한다 - 는 여전이 중국의 문화에서 가장 중요한 지점이다.

Chinese culture also carries a long learning tradition, one that is exemplified by the great tutor Mencius and his students. Unlike those of his Greek counterpart, Mencius's pupils were not powerless slaves, but rather kings and dukes. When Mencius tutored them about compassion and human capacity for goodness, they, like Meno, were led to believe that they were powerless (despite, in this case, having political power) unless they sought to perfect themselves morally. This basic orientation—to become a better (more virtuous) person—is still believed, in Chinese culture, to be the most essential quality for any learner.

학습에 관한 신념은 인간의 학습에 필수적이다.

LEARNING BELIEFS ARE ESSENTIAL TO HUMAN LEARNING

사람들이 학습에 대해서 이해하는 것, Belief about learning 이라고 하는 것은, 인지/정동/행동적 과정에 대한 신념을 말한다. 그러한 신념은 다음과 같은 것에 대한 것이다.

People's own understanding of learning, which I refer to as beliefs about learning (Li, 2003), include their beliefs regarding cognition, affect, and behavioral processes (Dweck, 1999). Such beliefs concern 

• 목적 purposes (e.g., what people think they gain from learning), 
• 과정 processes (e.g., what it takes to learn something), 
• 개인적 중요성 personal regard (e.g., whether or not and why they regard learning as important), 
• 정동 affects (e.g., whether they experience joy or dread from learning), and 
• 사회적 인식 social perceptions (e.g., perception of those who learn well vs. those who do not and their perception of teachers). 

이러한 신념은 사람들의 학습동기를 이루며, 어린이들이 실제로 학습하고 성취하는데 영향을 준다. 이러한 신념은 타고나는 것이 아니라 사화회과정을 거치면서 스스로 능동적으로 구성해나가며 발달해나가는 것이다. 신념은 문화적 가치 시스템에 의해서 크게 영향을 받는다. 소크라테스와 맹자가 가르친 예시와 같이, 문화권마다의 지적 전통은 개개인의 신념을 서로 다른 형태로 만들어간다. 최근의 연구를 살펴보면, 문화적 영향이 중요함을 보여준다. 문화권 사이의 학습모델에 대한 비교 연구는 많지 않으나, 상당한 진전이 이뤄진 바 있다.

These beliefs underlie people's motivation for learning and influence children's actual learning and achievement.

Because learning beliefs are not innate but develop through the process of socialization together with individuals' own active construction, the beliefs are profoundly influenced by cultural value systems. As the Socratic and Mencian tutoring examples illustrate, intellectual traditions across cultures may shape individuals' beliefs differently. Recent research on thinking styles in Asians and Westerners (Nisbett, 2003) underscores the importance of cultural influence. Although less research on learning models across these cultures exists, significant advancements have been made.

'학습에 관한 신념'에 대한 연구

RESEARCH ON BELIEFS ABOUT LEARNING

다음과 같은 것이 있다.

Recent research yields important findings in three areas: cultural beliefs about learning, their influence on individuals' learning orientations, and development of learning beliefs.

문화적 신념

Cultural Beliefs

European American(EA)이 학습에 대해 갖는 문화적 신념은 mind orientation이다. 이는 미세하게 분화하는 정신적 기능으로서, 이를 통해서 세계를 이해하고, 개인적 기술을 단련하고, 개인의 목표를 실현하는 것이다. 학습자는 능동적으로 다양한 범위의 활동과 경험을 쌓는다.

EA beliefs indicated what I (Li, 2002) termed a mind orientation, which elaborates on finely differentiated mental functions to understand the world, develop personal skills, and realize personal goals. The learner is actively engaged in a broad range of activities and experiences. 

반면 중국인들은 value orientation을 갖는다. 이는 한 사람 전체에 대한 도덕적인 것을 일컫는다. 중국인들은 또한 세상을 이해하려고자 하는 노력에 가치를 두나, 인지적인 이해(understanding achieved by articulation, analysis, and reasoning rather than by experience, practice, or meditation)는 그것만으로 학습 신념의 중심이 되지는 못한다. 

Chinese beliefs revealed what I (Li, 2002) termed a virtue orientation, which has an overarching moral tone regarding the whole person. Chinese also value efforts to achieve understanding of the world, but mentally oriented understanding (understanding achieved by articulation, analysis, and reasoning rather than by experience, practice, or meditation) alone is not central to their learning beliefs. 

'지식'이 무엇인가를 물었을 때 중국 대학생의 79%는(EA학생의 15%는) 그것을 "스스로 완벽해지고자 하는 욕구" 그리고 "영적 풍요로움/영적 힘"이라고 했다. 반면 32%의 학생은(EA학생의 96%는) 그것을 팩트, 정보, 기술, 세상에 대한 이해 라고 답하였다.

When asked to define what knowledge is, 79% of Chinese college students (but only 15% of EA students) defined it as “a need to self-perfect” and “spiritual wealth/power”; 32% of Chinese students (as opposed to 96% of EA students) defined it in terms of facts, information, skill, and understanding of the world. 

다른 연구에서는 학습 신념의 기본적 차이를 지지하는데, 중국에서는 잘 사는 사람이든 가난한 사람이든 그들의 자녀를 학교에 보내고, 이 때 목적은 언어나 수학을 배우는 것이 아니라 세상에 대하여 알고, 사회적 관계에서 기능할 수 있고, 무엇보다 도덕적 함양을 위해서이다.

Other research supports these basic differences in learning beliefs. Cheng (1996) found that parents in China, whether well off or destitute, sent their children to school primarily not to learn literacy and numeracy skills, but to become knowledgeable of the world, able to function well in social relations, and, most important of all, morally cultivated.

이러한 결과는 두 문화 사이에 학습에 대한 개념이 완전히 다름을 보여준다. 

These findings show that, despite the comparable complexity of the two cultures' conceptual maps, the actual meanings that people construct about learning differ markedly

개개인의 학습 지향점

Individuals' Learning Orientations

근거들을 찾아보면 서양과 중국 모두 문화적 신념이 개개인이 실제로 학습하는 것에 영향을 미친다. Hess와 Azuma는 미국의 유치원생들이 호기심, 내적 동기, 정신적 독립성, 과제 효율성, 창의성 등에 기반하여 사회화되기 때문에 인내와 끈기가 부족하고 자세한 내용에 대한 집중이 일본 어린이들보다 낮았다. 각각의 문화권에서 이런 문화적 기반의 학습 지향은(일본의 성실/끈기와 미국의 독립/독창성) 학생들이 5학년 ,6학년때 어느 정도 성취를 이루는가를 예측한다. 높은 독립/독창성을 보인 미국 어린이들이 나중에 높은 학업성취를 보였으며, 성실/끈기가 높은 일본 어린이들이 마찬가지로 높은 학업성취를 보였다.

Available evidence indicates that both Western and Chinese beliefs influence how individuals in the respective cultures actually learn. Hess and Azuma (1991) documented that because U.S. preschool children were socialized to rely on their curiosity, intrinsic motivation, mental independence, task efficiency, and creativity, they showed less patience, persistence, and attention to detail in actual learning than did Japanese children (who are culturally similar to Chinese children). In each culture, these culturally based learning orientations (diligence/persistence for Japanese children and independence/originality for U.S. children) predicted children's school achievement in 5th and 6th grade: Those U.S. students who showed higher levels of independence/originality than their own peers in kindergarten also showed higher achievement in later grades; the Japanese students who showed higher levels of diligence/persistence likewise showed higher achievement than their peers.

Dahlin과 Watkins는 중국 학생들이 암기(고된 학습과 헌신)와 반복(성실함)을 영국 학생들보다 더 사용하고, 이러한 학습활도을 다른 목적으로 활용했다. 영국 학생들은 '이해'를 어느 순간 통찰력이 생기는 과정(mind orientation)으로 보았으며, 반복은 기억을 확인하기 위해서 사용했다. 반대로 중국 학생들은 '이해'를 상당한 개인적 노력과 암기를 필요로 하는 긴 과정으로 보았으며, 반복은 노력을 보여주는 확실한 방법이엇다.

Dahlin and Watkins (2000) also found that Chinese students used memorization (hard work and commitment) and repetition (i.e., diligence) more than their British peers did, and they used these learning activities for different purposes. British students viewed understanding as a process of sudden insight (i.e., mind orientation), and they used repetition to check their memory. In contrast, Chinese students believed understanding to be a long process that requires extensive personal effort, and memorization (aiming at deeper understanding through persistence) and repetition are two such concrete ways of showing effort.

Pratt, Kelly, Wong은 홍콩의 서양 교사들이 중국 학생들을 조용하고, 수용적이고, 권위에 도전적이지 못하다라고 평가함을 밝혔다. 그러나 중국 학생들은 학습이 점진적 과정으로서 상당한 헌신과 꼼꼼한 단계를 밟아야 한다고 믿고 있었다. 앞서 보여준 것처럼 전형적인 중국 학습자는 새로운 자료를 암기하고, 그 후에 이해하며, 그 다음 지식을 실제 상황에 적용시킨다. 질문하고 원본 자료를 변형하는 단계는 최종 단계이다. 이 마지막 단계가 구두 상호작용과 다른 사람과의 토론이나, 첫 번째 단계는 보다 홀로 진행하는 과정이며, 사색과 숙고의 단계이다. 명백하게 이러한 학습에 대한 방향은 서양처럼 즉각적인 구두 의사교환을 강조하지 않으며, 매우 오랜 기간 지속될 수도 있다. 서양 교사들이 중국 학생들에게 즉각적인 질문과 분석을 요구할 때 더 당황하고 좌절하는 것으로 나타났다.

Pratt, Kelly, and Wong (1999) found that Western teachers in Hong Kong often characterized Chinese students as quiet and receptive, lacking a challenging attitude toward authority. However, Chinese students believed that learning is a gradual process that requires tremendous dedication and methodical steps. As shown previously, typical Chinese learners initially commit the new material to memory, then seek to understand it, and next try to apply the knowledge to real life situations; questioning and modifying the original material is the final step. Whereas the last step calls for verbal interaction and discussion with others, the first steps call for more solitary learning and contemplation (an important aspect of Chinese intellectual tradition). Clearly, this learning orientation does not promote the immediate verbal exchange as emphasized by Western learning beliefs, and can extend over a period of time (e.g., publishing a paper to challenge a teacher with whom one disagreed several years earlier). Chinese students were found to feel frustrated and bewildered when Western teachers expected them to engage immediately in questioning and analysis.

아시아 학습자들이 거의 말을 하지 않는다는 것은 널리 알려져 있는데, 이것을 설명할 때 Kim은 말하는것(말하지 않는 것)에 대한 신념의 차이에 대한 근거와 그것이 사고와 학습에 미치는 영향을 제시했다. EA학생들은 생각하고 배운 것에 대해서 말하는 것에 대해서 더 인과관계가 있다고 믿었다. 더 나아가 말하는 것이 EA학생들의 수행능력에는 영향을 주지 않았으나, Asian American 학생들의 수행능력은 떨어뜨리는 것으로 나타났다. 이러한 결과는 서양의 학습모델에서 구두 의사표현이 중요함을 보여주는 것과 일치한다. 또한 아시아 학생들이 말하는 것을 도덕적 차원에서 불신하는 것과도 연관되어 있다. 유교에 따르면 말하는 것은 스스로의 주장에 대해서 스스로 책임을 지는 것이다. 따라서 만약 스스로의 말에 대해서 행동으로 책임질 수 없다면 말하지 말아야 한다.

In an attempt to explain the widely noted quietness among Asian learners, Kim (2002) provided further evidence for differences in beliefs surrounding speaking (and not speaking) and their effects on thinking and learning among EA and Asian American college students. It was found that EA students believed more in the causal effect of speaking on thinking and learning than Asian-Americans did. Moreover, speaking did not interfere with EA students' performance, but did interfere with the performance of Asian American students. These findings correspond to my (Li, 2003) documentation of the importance of verbal expression in the Western learning model. The Asian belief that speaking interferes with learning may well reflect the essential learning virtue of concentration. It is also likely related to Asians' distrust in speaking on moral grounds: Speaking is viewed by Confucians as an act of committing oneself to one's claim; if one is unable to back one's claim with action, one should remain silent

발전

Development

세 가지 종류의 이야기를 활용한 연구

The first set of stories elicited children's perceptions of the purposes of school learning; the second set tapped children's perceptions of their achieving peers; and the third set focused on their construals of what the learning process involves.

문화적 모델과 일관된 모습을 보여주는데, 

• 4살 수준의 EA 어린이들은 smartness , literacy, friendship, play 를 학교에 가는 목적으로 꼽았다. 
• 중국 어린이들은 need to self-improve morally, mastery of knowledge, social contribution, social respect/economic reward 를 꼽았다.

Results showed that, consistent with their cultural models, EA children as young as 4 talked more about smartness (i.e., emphasizing the mental), literacy, friendship, and play as their purposes for going to school. Chinese children mentioned more the need to self-improve morally, mastery of knowledge (defined with a moral component), social contribution, and social respect/economic reward as their purposes.

우수한 동료에 대해서 인식을 살펴보면 EA학생들은 우수한 친구들의 지적 발전과 긍정적인 측면에 대해서 말했다. 그러나 동시에 중국어린이들보다 사회적 고립과 같은 부정적 측면도 더 많이 언급했다.

Regarding their perceptions of the achieving peer in the second of the three stories, EA children mentioned the achiever's intellectual development and positive affects (e.g., happiness, pride) among the achiever and his or her parents and teachers. But they expressed significantly more concerns about negative consequences of achieving well in school (e.g., rejection and social isolation from peers) than Chinese children did.

학습 과정을 이해하는데 있어서 EA학생들은 능력과 관련된 정신적 과정을 주로 언급했으며, 학습을 문제에 대한 해결, 전략을 세우는 것으로 표현했다. 중국 어린이들은 그보다는 근면함, 인내, 집중을 언급했다. 더 나아가서 이러한 추세가 아이들의 연령이 높아짐에 따라서 일관되게 나타났다. 이러한 결과는 학습에 대한 문화적 모델이 개개인이 학습에 대한 신념과 방향을 정하는데 영향을 준다는 것을 보여준다.

In their construals of the learning process, EA children referred more to ability and related mental processes, to learning as a task to be tackled, and to creative strategies. Chinese children mentioned more the virtues of diligence, persistence, and concentration. Moreover, the observed trends in all three sets of narratives became more consistent as children's age increased. These developmental data suggest that cultural models of learning are likely to shape the beliefs and learning orientations of individuals in the respective cultures.

‘창의력’을 교육과정 목표로 앞세우는 나라 드물어

A4면2단| 기사입력 2015-09-01 20:35 

 14  

12

출처: http://news.naver.com/main/read.nhn?mode=LSD&mid=sec&sid1=102&oid=028&aid=0002287694

역량바탕교육 도입의 절차 워크북 (Clinical and Translational Science Institute, University of Pittsburgh)

Implementing Competency-Based Education

A Process Workbook

2009-2010

Process

Step 1: Conducted a Literature Review

Step 2: Developed a Theoretical Competency Framework

Step 3: Developed a Program-Specific Model

Step 4: Developed Curricular Frameworks

Step 5: Combined University of Pittsburgh and National Center for Research Resources Competency Documents

Step 6: Reviewed Curricular Materials and Aligned Curricular Frameworks with Core Courses

Step 7: Reviewed Core Course Alignments with Course Faculty

Step 8: Identified Curricular Gaps and Made Recommendations

Step 9: Reviewed Curricular Materials and Aligned Track-specific Frameworks with Core and Track Courses

Step 1 0: Established Standards of Comparison

Step 11: Compiled Teaching Methods

Step 12: Compiled Assessment Methods

Step 13: Created Project Web Site

Implementation

Step 1: Piloted Curricular Structure

Step 2: Evaluated Effects of Implementing a Competency-Based Educational Curriculum in Piloted Courses

Step 3: Created a Database to Track Core Course Changes

Dissemination

• Aligned NCRR National Core Curriculum to Virtual University Course

• Presented at Association for Clinical Research Training (ACRT) Conference in April 2010

• Organized a Faculty Seminar on Teaching Methods

• Preparing CBE Process Paper for Publication in CTS Journal

• Published Paper on Developing a PhD Program in CTS Journal

• Preparing Paper on Assessment in CBE for Publication in CTS Journal

Barriers to Implementation

Recommendations

• Process Recommendations

• Implementation Recommendations

• Dissemination Recommendations

http://www.academic.pitt.edu/assessment/pdf/CompetencyBasedEducation.pdf

논문의 홍수에서 살아남기

HOW TO TAME THE FLOOD OF LITERATURE

BY ELIZABETH GIBNEY

Bergman에게 쏟아지는 컨텐츠는 '그럭저럭 감당할 만' 했지만, "점차 더 부담스러워진다."라고 말했다.

Casey Bergman’s daily research routine used to include checking all his e-mails and web alerts to pick out fresh papers in his field. But he grew dissatisfied with table-of-contents alerts from journals, RSS (Rich Site Summary) feeds and automated e-mails from the PubMed database. The flow of content was manageable, but if he left it for more than a day, “it became a burden”, he says.

그래서 작년에 그는 초파리라는 단어가 포함된 논문을 찾아서 그 팔로워에게 보내주는 FlyPapers라는 이름의 트위터 계정(봇)을 만들었다. 이후 다른 분야에서도 55개정도의 유사한 봇들이 생겼다. 

So last year Bergman, a computational geneticist studying fruit flies (Drosophila) at the University of Manchester, UK, turned to a fresh approach: an automated Twitter account (or ‘twitterbot’) that he named FlyPapers. The bot trawls PubMed and the arXiv preprint server to find papers containing the word Drosophila, and spits them out into its followers’ feeds. Bergman finds it much easier to catch up with FlyPapers popping up in his Twitter feed — and his idea has spawned around 55 twitterbots in other disciplines.

매일 6000개의 논문이 쏟아지는 상황에서조차 중요한 논문을 놓치는 것은 "사망선고(mortifying)"와 다를 바 없다. Sally Burn은 Scizzle이라는 서비스를 활용해서 PubMed연구 중 저장한 내용을 그녀에게 보낸다. 

It is no surprise that academics are coming up with their own ways to keep on top of the flood of literature. “It’s a common struggle,” says Bergman. A staggering 6,000 papers are published every day — and although no one wants to be overloaded with recommendations, missing key papers is “mortifying”, says Sally Burn, a developmental geneticist at Columbia University in New York City. She uses a service called Scizzle, which regularly sends her the results of saved PubMed searches. “Unless you have all day, and ten people working for you trawling the literature, I think it’s the best situation you’re going to get,” she says.

그러나 키워드를 중심으로 논문을 찾아 보는 것은 기술적으로 가능한 것의 극히 일부분일 뿐이다. 최근 등장한 논문추천엔진은 쏟아지는 논문들 중 필요한 것만 걸러줄 뿐만 아니라 사용자의 관심사를 바탕으로 다른 내용을 추천해주기도 한다. "기본적으로 넷플릭스나 아마존이 하는 것과 같다고 할 수 있죠" Matthew Davis는 말한다. 

But a stream of papers based on keywords only scratches the surface of what is technologically possible. Emerging literature-recommendation engines promise not only to filter the flood of papers to a trickle, but also to learn from their users’ interests to add personalized suggestions (see ‘A guide to reading’). “In spirit, it’s similar to what Netflix or Amazon do,” says Matthew Davis, a computational biologist at the University of Texas at Austin who wrote the algorithm for one such service, PubChase — now owned by ZappyLab, a firm in Berkeley, California, that makes web- and phone-based tools for scientists.

이걸 좋아하면, 요것도 좋아할걸? If you like that, you’ll like this

초창기 서비스면서 아직까지도 최고의 자리를 지치고 있는 것으로 구글스칼라가 있다. 연구자의 논문이나 인용을 바탕으로 통계적인 모델을 적용해서 논문을 추천해준다. 그러나 대학원생은 이렇게 추천을 받을만큼 논문을 쓰지 않았기에 어려움이 있다고 말한다.

One of the first, and still best-known, services comes from Google Scholar. Its Updates tool suggests articles by applying a statistical model to a record of a researcher’s authored papers and citations. “The recommendations are almost scarily good,” says Roger Schonfeld, programme director at Ithaka S+R, a non-profit consultancy based in New York City that advises academia on digital technology. But graduate students may not have a sufficient body of work for the site to help, notes Patrick Mineault, a computational neuroscientist at the University of California, Los Angeles.

PubChase는 사용자의 출판기록을 토대로 PubMed에서 논문을 추천해준다. 그러나 이 역시 사용자가 읽거나 저장한 내용을 기반으로 한다. 여기에 기계-학습 기술을 더했는데 다른 사람의 저장소와 비교해주는 것이다. 

PubChase suggests articles from PubMed on the basis of a user’s publishing record, but it also learns from the articles that the user has read and stored in his or her online library. And it adds another machine-learning technique: comparing this library with other people’s collections, with the logic that people with common research interests might benefit from each others’ preferences. “I’ve been really impressed: nearly every article it has recommended has been relevant to my research,” says Kelsey Wood, a geneticist at the University of California, Davis, who uses the service along with reference-manager tool Mendeley, owned by Amsterdam-based publisher Elsevier.

Ross Mounce는 Pubchase는 PubMed의 바깥에 있는 연구자들에게는 그다지 유용하지 않다고 말한다. 그는 Sparrho를 선호하는데, 여기서는 키워드피드를 바탕으로 추천을 해주며 사용자에게 관련도를 평가하게 하여 추천목록을 검토한다. 또한 논문, 연구비, 특허, 포스터 등등도 포함한다. PubChase와 마찬가지로 비슷한 사용자들사이의 관계를 바탕으로 추천을 해준다.

Ross Mounce, an evolutionary biologist at the University of Bath, UK, says that PubChase is not useful for those whose interests fall outside the boundaries of PubMed. He prefers Sparrho, a fledgling London-based venture that generates recommendations with a keyword-based feed, and asks users to train the tool by flagging suggestions as relevant or irrelevant. It includes articles, grants, patents, posters and conference proceedings from all the sciences. “The breadth is a real strength,” says Mounce. As with PubChase, recommendations are based on connections between similar users. “We’re allowing intelligent curators, humans, to join the scattered dots,” says chief executive Vivian Chan, who co-founded Sparrho after she struggled to keep up with the literature while studying for a biochemistry PhD at the University of Cambridge, UK.

PubChase와 Sparrho는 투자를 유치하고자 하는 스타트업들이기에 많은 사용자가 쓰는 것이 중요하다. 이 서비스의 사용자수는 적긴 하나, 둘 다 빠르게 성장하는 중이다.

As start-ups seeking investment, PubChase and Sparrho are guarded about how many users they have. It is clear that numbers are small. (A Nature survey of more than 3,000 scientists found that only 8% had heard of PubChase, and fewer than 1% visited it regularly; see Nature 512, 126–129 (2014).) But both say that their user base is growing quickly.

기본으로 돌아가기 Back to basics

Bergman은 알고리즘 기반 검색에 대해 우려하는 입장이다. 추천 목록을 스스로 학습하고 관리해주는 기계야말로 "당신의 지적 시력을 떨어뜨릴 것입니다"라고 한다. 또한 그는 그가 수행한 '학제간연구'적 특성이 구글스칼라를 오히려 혼동스럽게 만든다고 지적한다. 그러나 Davis는 이렇게 분야를 좁혀주는 것과 달리 비슷한 관심사를 가진 다른 사람들의 특징을 기반으로 새로운 문을 열어주는 경우도 있다고 말한다.

Bergman is wary of algorithm-based searches. A machine that learns and tailors recommendations can become like “blinders on your intellectual scope”, he says. And he has found that the interdisciplinary nature of his work, which melds genomics and text-mining, confused Google Scholar — the tool threw up irrelevant papers and missed important ones. But Davis says that this narrowing is counteracted by the new doors opened by recommendations based on the profiles of people with similar interests.

많은 연구자들은 알고리즘ㅇ르 회피하고, 자신의 사회적 네트워크에 속한 다른 동료들을 따라 읽기도 한다. "트위터는 조용한 영웅이죠" 라고 Cassie Ettinger는 말한다. 비슷한 것으로 1000 Prime과 Mendeley가 있다.

Many researchers eschew algorithms altogether, and simply follow colleagues on social networks to find out what is worth reading. “Twitter is the unsung hero of the paper-recommendation world,” says Cassie Ettinger, a geneticist in the same research group as Wood. Other scientists check which papers rise to the top in online communities or among users of reference-management services such as Faculty of 1000 Prime and Mendeley.

그러나 추천목록을 공유하고 새로운 논문을 찾기 위해서 라이브러리를 업로드하는 것이 매우 일반적인 것은 아니다. RSS를 선호하는 Derek Lowe는 그녀는 추천엔진을 훈련시킬 시간이 없다고 말한다. Mineault는 자동화 검색엔진은 과학자가 찾고자 하는 논문은 절대 찾아주지 못할 것이라고 하면서, 대신 그는 이들 검색엔진이 향후에 발전할 가능성은 높다고 생각한다고 말했다. 

But the desire to share recommendations or upload libraries to find new papers is hardly universal. Derek Lowe, a chemist at Vertex Pharmaceuticals in Boston, Massachusetts, who writes the blog In the Pipeline, remains a fan of RSS feeds from journal websites. And Burn says that she does not have the time to train a recommendation engine. Mineault acknowledges that automated learning devices will never find all the papers a scientist wants, but he thinks that they will improve. Techniques for gleaning meaning from content will become more sophisticated, he says, and will eventually have a significant role in guiding scientists’ reading choices.

Bergman에게 이런 것들은 대체로 취향의 문제다. 그의 트위터봇은 온라인 초파리 커뮤니티를 만들기에 이르렀고, 이 봇이 추천한 것은 심지어 과학자가 아닌 다른 분야 연구자들에 의해서도 리트윗된다. Bergman은 다른 기술들을 안 쓰려는 생각은 없지만 아직까지는 FlyPapers만을 사용한다. "아직은 다른 것이 필요한지를 잘 모르겠어요. 저한테는 이정도면 충분하고 그게 중요한거죠."

For Bergman, a lot of this is a matter of taste. His twitterbot has convened an online fruit-fly community; its suggestions have been retweeted by researchers in other disciplines, and even by non-scientists. Bergman has not ruled out trying further technologies, but he is sticking to FlyPapers for now. “I haven’t felt the need to try any others. It’s working for me, and that’s all that matters,” he says.

How to tame the flood of literature.

HEXACO에 따른 정직성 및 겸손 관련 준거와 성격의 5요인 모델

The prediction of Honesty–Humility-related criteria by the HEXACO and Five-Factor Models of personality

Michael C. Ashton a,*, Kibeom Lee b,*

a Department of Psychology, Brock University, 500 Glenridge Avenue, St. Catharines, ON, Canada L2S 3A1

b Department of Psychology, University of Calgary, Calgary, AB, Canada T2N 1N4

HEXACO와 FFM에 대한 두 가지 질문을 연구하였다. 첫째로 HEXACP모델이 FFM을 outpredict하는가? 만약 그렇다면 H-H scale과 근접한 것을 더하면 FFM이 HEXACO model의 predictive validity를 HEXACO수준으로 높여줄까? 자기평가와 관찰자평가 결과를 보면 HEXACO모델이 FFM보다 predictive validity가 더 높았다. FFM에 HH척도가 더해지면 일부 criteria에 대해서는 HEXACO모델만큼 predictive validity가 올라갔지만, 다른 것들에 대해서는 여전히 낮았다.

We examined two questions involving the relative validity of the HEXACO and Five-Factor Models of personality structure. First, would the HEXACO model outpredict the Five-Factor Model (FFM) with regard to several diverse criteria that are conceptually relevant to the Honesty–Humility dimension of personality? If so, would the addition of a proxy Honesty–Humility scale—as computed from relevant facets of the FFM Agreeableness domain—allow the FFM to achieve predictive validities matching those of the HEXACO model? Results from self- and observer ratings in three samples (each N > 200) indicated that the HEXACO model showed considerable predictive validity advantages over the FFM. When a measure of Honesty–Humility derived from the FFM was added to the original five domains of that model, the predictive validity reached that of the HEXACO model for some criteria, but remained substantially below for others.

1.4. Purpose of the present research

In the present research, therefore, we aimed to address two main questions. 

Our first question was whether or not the predictive advantage previously observed for the six HEXACO dimensions in comparisons with the Big Five factors would also be observed in comparisons with the dimensions of the FFM. Given the incorporation of some elements of Honesty–Humility within the FFM variant of Agreeableness, one might expect that the latter factor would be a strong predictor of criteria that are conceptually related to Honesty–Humility. In this case, the advantage of the HEXACO model over the Big Five would not be generalizable; that is, there would be little or no advantage for the HEXACO framework over the FFM. On the other hand, however, the results of Ashton and Lee (2005) suggest that at least some Honesty–Humility-related criteria would show only modest associations with FFM Agreeableness. In this case, there would remain a substantial advantage for the HEXACO framework over the FFM.

Pending the answer to our first question described above, our second question was whether or not a proxy Honesty–Humility factor derived from the relevant aspects of FFM Agreeableness—that is, the NEO-PI-R Straightforwardness and Modesty facets—could match the predictive validity of the HEXACO-PI Honesty–Humility factor in predicting a wide variety of criterion variables. The results reported by Ashton and Lee (2005) indicate that such a result should be expected for at least some criteria. But as was also noted by Ashton and Lee, other criteria might be associated with aspects of (low) Honesty–Humility that have weaker links with the Straightforwardness and Modesty facets of NEO-PI-R Agreeableness. For example, the constructs of greed or status-seeking have a limited conceptual overlap with those facets, as do the “outlaw” tendencies that are also relevant to low Honesty–Humility. Thus, a proxy FFM-derived measure of Honesty–Humility might not predict those criteria as strongly as would a broader measure of Honesty–Humility, one that is likely to contain both (a) a larger proportion of common factor variance than does a measure consisting of only two facets and (b) specific variance associated with a wider array of Honesty–Humility traits.

In answering the above questions, we decided to assess several important constructs that appear to be poorly accommodated by the Big Five space, but that also show some conceptual overlap with the HEXACO space, especially the Honesty–Humility dimension. For example, the construct of materialism (e.g., Richins & Dawson, 1992) shows relatively weak associations with the Big Five ( Shafer, 2003 and Sharpe and Ramanaiah, 1999), but resembles traits found at the low pole of Honesty–Humility. Several constructs involving exploitive and/or criminal activity—including unethical business decisions, sexual harassment proclivity, and general delinquency—are relevant to some aspects of the Big Five (specifically, the low poles of Big Five Agreeableness and Conscientiousness) but have an even stronger conceptual overlap with low Honesty–Humility. Some sexuality-related constructs, such as seductiveness ( Paunonen, 2002) and unrestricted sociosexual orientation ( Simpson & Gangestad, 1991)—are conceptually linked to some extent with Big Five (and HEXACO) Extraversion, but also have an element of manipulation or exploitation that suggests low Honesty–Humility. Finally, as noted earlier, another construct involving a form of manipulation is Jackson’s (1970)social adroitness, which is weakly related to the Big Five but is associated with low Honesty–Humility. Thus, all of the constructs listed above are of much interest in the context of predictive validity comparisons between the HEXACO framework and the FFM.

In assessing the personality constructs of the FFM and the HEXACO structure, we used both self-reports and observer reports as our methods of measurement. The validity of self-report measures of personality in predicting objective criteria—such as directly observed behaviors or recorded life outcomes—is well established (e.g., Kolar et al., 1996 and Ozer and Benet-Martinez, 2006). However, there is also some evidence that observer reports of personality, as provided by persons who are well acquainted with the target individual, may provide even more valid assessments than do self-reports (Kolar et al., 1996). In some predictive contexts, moreover, observer report may be the preferred method of personality assessment (for example, in selection settings the target individuals might be motivated to provide highly socially desirable self-reports, and in some clinical settings the target individuals might have limited ability to provide accurate self-reports). The use of observer report measures of personality is particularly useful in the context of the present investigation, because the criterion constructs described above are generally measured by self-report scales. Therefore, by including observer reports as well as self-reports of personality, we would be able to examine the cross-source validity of the FFM and HEXACO dimensions, and to determine whether any predictive advantage of the latter framework would be generalizable across the two sources of personality data.

2.2. Materials

2.2.1. Personality inventories assessing FFM and HEXACO dimensions

We assessed the personality dimensions of the FFM and the HEXACO framework in each of the three samples. 

In the first sample, we obtained self-reports only, using the NEO-PI-R (Costa & McCrae, 1992) and the (full-length) HEXACO-PI (Lee and Ashton, 2004 and Lee and Ashton, 2006). 

In the second and third samples, we obtained both self- and observer reports on shorter versions of these instruments. 

By arrangement with the test publisher, we administered to both samples a subset of the NEO-PI-R items that included the items of the five NEO-FFI scales as well as the remaining items of the Straightforwardness and Modesty facet scales, with items permuted randomly. The second sample completed the half-length form of the HEXACO-PI, and the third sample completed the half-length form of the HEXACO-PI-R.2

2.2.2. Criterion variables

We also assessed the following criterion scales in the three participant samples; except as noted otherwise, all items used 1-to-5 response scales.

In the first participant sample, we administered the seven-item Materialism Centrality scale (Richins & Dawson, 1992), along with an eight-item version of Jackson’s (1970) Social Adroitness scale (as used by Ashton, 1998) and a 21-item Delinquency scale. (The Delinquency scale was based on that of Ashton (1998), but was expanded to include a wide array of delinquent behaviors, including vandalism, impaired driving, workplace safety violations, cheating on exams, speeding, stealing from employers, and vandalism. Each item used a 1-to-8 response scale, with options arranged in increasing order of the amount of delinquent activity, expressed as a frequency or a dollar value. This instrument is available from the authors.)

In the second participant sample, we administered the 15-item SPI Seductiveness scale (Paunonen, 2002), and the five attitude-based items of the Sociosexual Orientation Inventory (Simpson & Gangestad, 1991). For a subsample of 180 participants from Sample 2, we also administered a six-item scale measuring unethical business decisions, the items of which consist of detailed hypothetical scenarios rather than simple statements (adapted from Lee, Ashton, Morrison, Cordery, & Dunlop, 2008; see Appendix A).3 The items of the Unethical Business Decisions scale used a 1-to-4 response scale.

In the third participant sample, we again administered the Materialism Centrality scale and the Unethical Business Decisions scale, as well as a four-item scale assessing willingness to participate in sexual “quid pro quos”. Like the Unethical Business Decisions scale, the Sexual Quid Pro Quos scale is also a scenario-based measure (see Appendix B). The latter scale includes two items that describe a willingness to provide workplace-related favors in exchange for sex, and two items that describe a willingness to provide sex in exchange for workplace-related favors; all of these items use a 1-to-4 response scale.

3.1. FFM versus HEXACO variables: Self-reports

We begin by comparing the validity of self-reports on the FFM and HEXACO variables in predicting the criterion variables. Table 2 shows the zero-order and multiple correlations of the self-report personality scales with the criteria, as obtained in each of the three samples.

As seen in the table, the FFM predictors obtained moderately high multiple correlations with the various criteria, with values ranging from .31 to .53. However, the HEXACO predictors generally achieved considerably higher multiple correlations with the same criteria, with values ranging from .39 to .71. The predictive advantage of the HEXACO variables was chiefly due to the Honesty–Humility scale, which in several cases showed zero-order correlations that exceeded the multiple correlations produced by the FFM measures.

With regard to the specific criteria, the greatest differences in predictive validity between the FFM and HEXACO instruments were observed for the Materialism scale (Samples 1 and 3) and the Social Adroitness scale (Sample 1). For both of these criteria, the advantage for the HEXACO model over the FFM was rather large, with differences of about .20 in the observed multiple correlations. A moderately large difference between the predictive validities of the two frameworks was also observed with regard to the Delinquency criterion (Sample 1), for which the multiple correlation obtained by the HEXACO variables was .11 units higher than was that obtained by the FFM scales. A similar pattern was also observed for the Unethical Business Decisions scale, for which the difference in multiple correlations yielded by the HEXACO and FFM variable sets was .15 in Sample 2 and .08 in Sample 3. The HEXACO predictors also showed modest advantages over those of the FFM in predicting the remaining criteria of Samples 2 or 3, all of which involved some element of sexuality—Seductiveness, (Unrestricted) Sociosexuality, and Sexual Quid Pro Quos—with differences in multiple correlations ranging from .05 to .08.

3.2. FFM versus HEXACO variables: Observer reports

Next we compare the validity of observer reports on the FFM and HEXACO variables in predicting the (self-report) criterion variables. Table 3 displays the zero-order and multiple correlations of the observer report personality scales with the criteria, as obtained in both of the samples in which observer reports of personality were collected (i.e., Samples 2 and 3).

The correlations reported in Table 3 are somewhat lower than those of Table 2, but generally are still substantial in size. The multiple correlations obtained by the FFM scales in predicting the criterion variables ranged from .21 to .40; for the HEXACO scales, the multiple correlations with those criteria ranged from .29 to .50. As was observed in the self-report data, the Honesty–Humility measure of the HEXACO framework yielded zero-order correlations with some criteria that exceeded the multiple correlations achieved by the FFM measures.

For the Materialism criterion, the predictive advantage of the HEXACO variable set over that of the FFM was rather large, with a difference of .18 in the observed multiple correlation. A similarly large difference in the multiple correlations was observed in Sample 2 for the Unethical Business Decisions criterion (.18), but in Sample 3 the difference was only moderately large (.10). The predictive advantage of the HEXACO predictors over those of the FFM was small for the Seductiveness and Sociosexuality criteria of Sample 2 (about .05 correlation units), but rather large for the criterion of Sexual Quid Pro Quos of Sample 3, with a difference in multiple correlations of .13.

In general, the predictive advantages of the HEXACO framework over the FFM as observed in self-report data were also recovered in observer report data; in fact, the differences between the multiple correlations obtained by the two models tended to be at least as large in observer reports as in self-reports.

3.3. FFM plus NEO-PI-R Honesty–Humility composite: Self-reports

We next examined the ability of an augmented FFM—one that incorporates a separate variable assessing Honesty–Humility—to predict the various criterion variables. Recall that we computed an ad hoc NEO-PI-R Honesty–Humility composite scale as the mean of the Straightforwardness and Modesty facets of that inventory. For the analyses below, we examine the validity of that variable in combination with the remaining FFM variables, but with the FFM Agreeableness variable being recomputed to remove the Straightforwardness and Modesty content (see Section 2.3).

Within the self-report personality data, the inclusion of the NEO-PI-R Honesty–Humility composite generally added substantially to the validity of the FFM scales in predicting several criterion variables (see Table 2). The increment in observed multiple correlations was especially large (.18) for the criterion of Social Adroitness (Sample 1), but was also moderately large (about .10) for the Seductiveness and Sociosexuality criteria (Sample 2) and for Materialism (in Sample 3). Somewhat smaller increments were achieved in predicting Materialism (in Sample 1), Unethical Business Decisions (Sample 3), and Sexual Quid Pro Quos (Sample 3). The criterion variables for which the addition of the NEO-PI-R Honesty–Humility scale did not produce any appreciable improvement in predictive validity were the Delinquency criterion (Sample 1) and the Unethical Business Decisions criterion (Sample 2).

The predictive validity of this expanded FFM variable set—that is, with the addition of the NEO-PI-R Honesty–Humility composite—can be compared with that of the HEXACO framework. For several criteria—Social Adroitness, Seductiveness, Sociosexuality, and Sexual Quid Pro Quos—the predictive validity of the augmented “FFM-plus-H” framework was about equal to that of the HEXACO framework, as the multiple correlations achieved by the former framework either closely approached or slightly exceeded those achieved by the latter. This was also observed for the Unethical Business Decisions criterion within Sample 3, but the HEXACO model maintained a considerable predictive advantage in Sample 2, with a difference in multiple correlations of about .15. For several other criterion variables, the expansion of the FFM to include the NEO-PI-R Honesty–Humility variable did not produce predictive validities matching those of the HEXACO model; instead, the latter framework produced multiple correlations about .10 units higher in predicting the criteria of Delinquency and Materialism.

3.4. FFM plus NEO-PI-R Honesty–Humility composite: Observer reports

Within the observer report data, the addition of the ad hoc NEO-PI-R Honesty–Humility scale to the FFM variable set again produced some improvements in predictive validity with regard to several criteria (see Table 3). Across the criterion variables of Samples 2 and 3, the augmented “FFM-plus-H” achieved multiple correlations roughly .05 units higher than those of the original FFM. For the criteria of Seductiveness and Sociosexuality, this resulted in predictive validities that matched those yielded by the HEXACO dimensions. However, for the remaining criterion variables of Samples 2 and 3, the advantage of the HEXACO model over the FFM was preserved in spite of the addition of the NEO-PI-R Honesty–Humility composite to the latter framework. In particular, the HEXACO variable set produced multiple correlations with the criteria of Materialism, Unethical Business Decisions, and Sexual Quid Pro Quos that were .06 to .18 units higher than those generated by the “FFM-plus-H” variable set.

Thus, the increment in predictive validity that was afforded by the addition of NEO-PI-R Honesty–Humility to the FFM variable set was found to be generalizable across self- and observer reports of personality. However, the observed differences between the predictive validities of the expanded FFM and the HEXACO framework were somewhat larger in observer reports than in self-reports, with the advantage of the HEXACO framework being maintained across more of the criterion variables within observer report data.

4.1. Predictive validity of the HEXACO and FFM

As shown in Table 2 and Table 3, the six HEXACO dimensions consistently showed substantially higher multiple correlations than did the FFM dimensions in predicting the various criteria. Across such diverse variables as Materialism, Delinquency, Unethical Business Decisions, Sexual Quid Pro Quos, Seductiveness, and Sociosexuality, the multiple correlations yielded by the HEXACO factors were generally about .05 to .20 units higher than were those yielded by the FFM dimensions. These differences generalized across self- and observer reports of the personality predictor variables, and in fact were at least as strong for observer reports as for self-reports.

It should be noted, of course, that the criterion variables examined in this investigation were selected specifically because of their relevance to those aspects of the HEXACO factor space that are largely outside the space of the FFM. In particular, these criteria generally show some clear conceptual link to traits associated with Honesty–Humility. Naturally, the difference in predictive validity reported here would not be observed for criterion variables that are conceptually relevant to the much larger region of the personality space that is spanned by both the HEXACO framework and the FFM. But the fact remains that the various criteria considered in this investigation represent a diverse array of variables that are of obvious importance in human affairs. For example, the serious physical and financial harms caused to individuals and to societies by “common” criminality (as assessed by the delinquency criterion) and by “white-collar” exploitation (as assessed by the Unethical Business Decisions criterion) are self-evident. The prediction of variables such as these is an important aim for personality assessment, and hence the ability of the HEXACO model to predict these variables successfully is an important indication of its practical utility.

Related to this point, we should note that the advantage of the HEXACO model over the FFM in predicting the criterion variables of this investigation was mainly due to the Honesty–Humility factor of the former model. For example, when we added HEXACO-PI(-R) Honesty–Humility to the regression equations involving the NEO-PI-R or NEO-FFI scales, the multiple correlations were in most cases quite similar to those yielded by the six HEXACO-PI(-R) scales.4 In contrast, however, other investigations have examined criteria that show a predictive advantage for the HEXACO model over the FFM, but for which this advantage is not due to Honesty–Humility. Perhaps the best such example is that of the criterion of “phobic tendency” (see Ashton et al., 2008), which was predicted better by the HEXACO-PI Emotionality scale than by the five NEO-FFI scales in combination.5

5. Conclusion

The Agreeableness domain of the FFM, unlike its counterpart in the classic Big Five framework, incorporates some traits associated with the Honesty–Humility factor of the HEXACO model of personality structure. Nevertheless, several important criterion variables that have conceptual links with Honesty–Humility are better predicted by the HEXACO model of personality structure than by the FFM. When a separate Honesty–Humility variable is computed from measures of the FFM dimensions, some of these criteria can be predicted as effectively as by the HEXACO factors. However, several criteria—particularly those associated with materialism, ethical violations, and criminality—are not well captured by those aspects of Honesty–Humility that are represented in the FFM, and are better accommodated by the dimensions of the HEXACO framework.

창의성의 정의와 측정: 창의성 검사는 쓸만한가?

Defining and measuring creativity: Are creativity tests worth using?

Arthur J. Cropley

창의성 검사는 발산적 사고, 연관짓기, 다양한 분야를 구성하고 결합하기 등등의 인지적 과정을 평가한다. 또한 창의성 검사는 동기부여, 유연성, 비의존상태에 대한 용인, 다름에 대한 긍정적 태도 등과 같은 비인지적 측면을 측정하기도 한다. 평가자들은 다양한 종류의 검사에 대하여 높은 수준으로 점수를 줄 수 있고, 수용가능한 수준에서 내적으로 안정적이다. 창의성검사는 선생님의 평가와 같은 다른 준거들과 어느 정도의 연관성을 보이며, 성인이 되었을 때의 행동을 예측하는데 유용하기도 하다. 즉, 창의성검사는 연구나 교육 목적 모두에 있어서 유용하다. 그러나 이 검사들은 창의성의 잠재력을 평가하는 것에 불과하며, 그 이유는 창의적인 성취는 창의성 검사로는 측정할 수 없는 기술, 지식, 정신건강, 심지어는 '운'에 의해서도 좌우되기 때문이다. 그러나 창의성 개념에 대한 다양한 측면을 보면, 평가는 하나의 검사점수가 아닌 다수의 점수에 기반해서 이뤄져야 함을 알 수 있다.

Creativity tests measure specific cognitive processes such as thinking divergently, making associations, constructing and combining broad categories, or working on many ideas simultaneously. They also measure noncognitive aspects of creativity such as motivation (e.g., impulse expression, desire for novelty, risk-taking), and facilitatory personal properties like flexibility, tolerance for independence, or positive attitudes to differentness. Raters can score the various kinds of test with substantial levels of agreement, while scores are internally stable to an acceptable degree. The tests also correlate to a reasonable degree with various criteria of creativity such as teacher ratings, and are useful predictors of adult behavior. Thus, they are useful in both research and education. However, they are best thought of as measures of creative potential because creative achievement depends on additional factors not measured by creativity tests, such as technical skill, knowledge of a field, mental health, or even opportunity. However, the multidimensional creativity concept they define indicates that assessments should be based on several tests, rather than relying on a single score.

Creativity tests measure specific cognitive processes such as thinking divergently, making associations, constructing and combining broad categories, or working on many ideas simultaneously. They also measure noncognitive aspects of creativity such as motivation (e.g., impulse expression, desire for novelty, risk‐taking), and facilitatory personal properties like flexibility, tolerance for independence, or positive attitudes to differentness. Raters can score the various kinds of test with substantial levels of agreement, while scores are internally stable to an acceptable degree. The tests also correlate to a reasonable degree with various criteria of creativity such as teacher ratings, and are useful predictors of adult behavior. Thus, they are useful in both research and education. However, they are best thought of as measures of creative potential because creative achievement depends on additional factors not measured by creativity tests, such as technical skill, knowledge of a field, mental health, or even opportunity. However, the multidimensional creativity concept they define indicates that assessments should be based on several tests, rather than relying on a single score.

창의성에 대한 편견: 왜 사람들은 창의적인 생각을 갈망하면서도 거부하는가?

The Bias Against Creativity: Why People Desire but Reject Creative Ideas

Jennifer S. Mueller1, Shimul Melwani2, and Jack A. Goncalo3

1Management Department, The Wharton School, University of Pennsylvania; 2Organizational Behavior Department, University of North Carolina, Chapel Hill; and 3School of Industrial and Labor Relations, Cornell University

사람들은 창의적인 생각을 갈망할까? 대부분의 학자들은 당연히 '그렇다'라고 대답할 것이다. 창의성이야말로 과학적 발전과 긍정적 변화의 원동력이기 때문이다. 또한 창의력은 지능, 지혜, 도덕적 선함과도 관계가 있는 것처럼 보여지기도 한다. 학자들은 지금까지 사람들이 창의성의 긍정적인 측면을 인정하면서도, 조직과 과학기관, 의사결정자들이 창의적인 생각을 거부하는 것을 의아하게 생각해왔다. 이런 현상은 심지어 창의성을 중요한 목표라 생각하는 사람들에게서도 나타났다. 유사하게, 연구자들은 선생님들이 호기심이 많고 창의적인 생각을 많이 하는 학생들을 싫어한다고 보고해왔으며, 이는 교사들이 창의성을 중요한 교육의 목표로 생각하는 경우에도 마찬가지였다.

Do people desire creative ideas? Most scholars would answer this question with an obvious “yes,” asserting that creativity is the engine of scientific discovery and the fundamental driving force of positive change (Hennessey & Amabile, 2010). Furthermore, creativity is seen as being associated with intelligence, wisdom, and moral goodness (Niu & Sternberg, 2006; Sternberg, 1985). However, although people strongly endorse this positive view of creativity, scholars have long been puzzled by the finding that organizations, scientific institutions, and decision makers routinely reject creative ideas, even when espousing creativity as an important goal (Staw, 1995). Similarly, research has documented that teachers dislike students who exhibit curiosity and creative thinking, even though teachers acknowledge creativity as an important educational goal (Westby & Dawson, 1995).

우리는 이러한 현상에 대한 새로운 관점을 제시하고자 한다. 사람들이 나이, 인종, 성별에 대한 뿌리깊은 편견을 가지고 있으면서도 밖으로는 드러내지 않듯이, 사람들은 공공연하게 인정하지는 않아도 창의성에 대한 뿌리깊은 부정적 인식을 하고 있을수도 있다. 창의성에 대한 이러한 편견의 존재와 특징을 드러내는 것은 왜 사람들이 그러지 말아야한다고 생각하면서도 창의적인 생각을 거부하고 과학적 진보를 억누르는지에 대한 설명을 줄지도 모른다.

We offer a new perspective to explain this puzzle. Just as people have deeply rooted biases against people of a certain age, race, or gender that are not necessarily overt (Greenwald & Banaji, 1995), so too can people hold deeply rooted negative views of creativity that are not openly acknowledged. Revealing the existence and nature of a bias against creativity can help explain why people might reject creative ideas and stifle scientific advancement, even in the face of strong intentions to the contrary.

창의적인 생각이란 새로우면서도 유용해야 한다. 그리고 '새로움'은 창의성의 중요한 특징으로서 인정되고 있다. 그러나 이 '새로움'이라는 특징이 창의적인 생각을 평가하는 사람들의 마음에 과연 이 아이디어를 따라도 될 것인가에 대한 걱정(tension)을 유발한다. 실제로, 평가자들은 '새로움'과 '유용성'이 반드시 서로 동반되는 것이라고 보기보다는 부의 상관관계(inversely related)를 갖는다고 보는 경우도 많다. 여기에는 몇 가지 이육 ㅏ있다. 유용한 아이디어들은 대체로 그 가치를 인정받는다. 그러나 새로운 생각일수록 그 아이디어가 과연 실용적이고, 유용하고, 실수로부터 자유롭고, 신뢰할 수 있는 결과를 반복적으로 보여줄지에 대한 불확실성이 커진다. 따라서 새로운 아이디어를 수용할 때, 사람들은 실패를 경험할 수도 있고, 사회적인 거절을 당할 수도 있고, 과연 그 아이디어가 성공적으로 마무리될지에 대한 불확실성도 느끼게 된다. 불확실성은 불쾌한(aversive)상태로서, 사람들이 그 상황을 종결시키거나 피하고 싶게 만든다. 따라서 사람들은 '새로움'에 대해 부정적인 연관을 짓게 되는 것이다.

Creative ideas are both novel and useful (Hennessey & Amabile, 2010), and novelty is the key distinguishing feature of creativity beyond ideas that are merely well conceived. Yet the requirement that creative ideas contain novelty can also promote a tension in evaluators’ minds when they judge whether to pursue an idea. Indeed, evaluators have a hard time viewing novelty and practicality as attributes that go hand in hand, often viewing them as inversely related (Rietzschel, Nijstad, & Stroebe, 2009). There are several reasons why. Practical ideas are generally valued (Sanchez-Burks, 2005). However, the more novel an idea, the more uncertainty can exist about whether the idea is practical, useful, error free, and reliably reproducible (Amabile, 1996). When endorsing a novel idea, people can experience failure (Simonton, 1984), social rejection (Nemeth, 1986), and uncertainty about when their idea will reach completion (Metcalfe, 1986). Uncertainty is an aversive state (Heider, 1958) that people feel a strong motivation to diminish and avoid (Whitson & Galinsky, 2008). Hence, people can also have negative associations with novelty—an attribute at the heart of what makes ideas creative in the first place.

학자와 practitioner들에게는 창의성이라는 것이 대개 긍정적인 것과 연결되나, 창의적인 생각 평가해보면 부정적인 것과 연관되기도 한다. 예를 들어서 사고의 연상(associative thinking)에 대한 연구를 보면 강한 불확실성은 창의성의 부정적인 측면과 연관된다.

Although positive associations with creativity are typically the focus of attention among both scholars and practitioners, negative associations may also be activated when people evaluate a creative idea. For example, research on associative thinking suggests that strong uncertainty feelings may make the negative attributes of creativity, particularly those related to uncertainty, more salient (Bower, 1981).

이 평가 절차는 창의성에 대해서 드러내놓고 평가하고 있지 않기 때문에, 창의성에 대한 편견을 내밀한 것(insidioius)으로 만들 수 있다. 사실 창의적인 생각을 지지해야 한다는 사회적 규범으로서의 압력도 있고, 창의성을 부정적으로 보는 생각을 겉으로 표현하는 것은 바람직하지 않다는 사회적 편견도 있다. 그 결과, 인종 편견에 대한 연구 결과와 유사한 결론을 얻게 된다. 겉으로 드러내는 창의성에 대한 우호적 태도와 속으로 감추는 창의성에 대한 부정적 인식은 서로 갈등이 존재한다. 즉, 창의적인 생각에 대한 편견을 드러내는 것은 직접적으로 '편견을 가지고 있느냐'라고 물어보는 것보다 더 정교한 방법이 필요하다. 따라서 우리는 내면적 태도와 더불어 겉으로 드러나는 태도를 평가하기 위한 방법을 활용하였다. 두 실험으로부터 우리는 서로 다른 두 가지 방법을 통해서 조작되고 측정된 불확실성이 창의성에 대한 더 심한 편견을 유발하는지를 살펴보았다. 

This evaluative process is not necessarily overt, which makes the bias against creativity potentially insidious. In fact, there is often strong normative pressure to endorse creative ideas (Flynn & Chatman, 2001) and a strong social desirability bias against expressing any view of creativity as negative (Runco, 2010). This resulting state is similar to that identified in research on racial bias: There is a conflict between an explicit preference toward creativity and unacknowledged negative associations with creativity (much as there can be a conflict between explicit and implicit attitudes toward a specific social group; Gaertner & Dovidio, 1986). In other words, uncovering a bias against creative ideas requires a method more subtle than simply asking directly. Therefore, we decided to employ a measure that assesses explicit attitudes in addition to implicit attitudes, which are less susceptible to self-presentation biases and normative pressures (Greenwald, Poehlman, Uhlmann, & Banaji, 2009). In two studies, we tested whether uncertainty measured and manipulated in two different ways promotes a greater bias against creativity relative to practicality. In the second study, we investigated whether this bias deters peoples’ ability to recognize creative ideas.

General Discussion

현대 로켓 추진체의 아버지라 불리는 Robert Goddard는 그 당시 동료들로부터 그의 아이디어는 터무니없고 불가능하다며 조롱과 조소의 대상이 되었다. 이러한 사례는 적지 않으나 창의성에 대한 이론을 다루는 사람들을 혼란스럽게 한다. 기존의 연구에서 더 창의적인 사람들이 덜 창의적인 사람들보다 창의성을 더 정확히 인식하고 평가했기 때문이다. 우리의 연구 결과에서 사람들은 얼마나 개방적인지와 무관하게 불확실성을 줄여야겠다는 느낌이 들었을 때 창의성을 더 부정적으로 연관시켰고 그 결과 창의적인 생각에 더 낮은 점수를 줬다. 여기에는 상당한 아이러니가 있다. 기존의 연구에서는 불확실성이 창의적인 생각을 더 추구하도록 한다고 했으나, 우리의 연구에서는 상황이 불확실해질수록 사람들은 심지어 창의성이 가장 필요한 때에조차 창의성을 인지하지 못했다. 

Robert Goddard, the father of modern rocket propulsion, endured ridicule and derision from his contemporary scientific peers, who stated that his ideas were ludicrous and impossible. This example is not unique, but it would puzzle creativity theorists, as research shows that expert raters who are themselves creative are even more likely to accurately recognize and assess creativity than expert raters who are less creative (Hennessey, Amabile, & Mueller, 2011). Our results show that regardless of the degree to which people are open minded, when they feel motivated to reduce uncertainty (either because they have an immediate goal of reducing uncertainty or they feel uncertain generally), they may experience more negative associations with creativity, which results in lower evaluations of a creative idea. Our findings imply a deep irony. Prior research shows that uncertainty spurs the search for and generation of creative ideas (Audia & Goncalo, 2007), yet our findings reveal that uncertainty also makes people less able to recognize creativity, perhaps when they need it most.

단순히 현재 상태나 익숙한 아이디어를 선호하는 것을 넘어서서 사람들은 창의성에 대한 이중적인 생각을 가지고 있다. 우선 Exp1의 baseline이나 Exp2의 불확실성에 대한 고용인(high tolerance)그룹에서는 창의성과 유용성을 연관시키는데 좀 더 긍정적이었다. 또한 불확실성 조건에 있는 참여자 중 95%와 불확실성에 대한 저용인 상태에 있는 사람들도 긍정적이라는 단어와 연관지어서 창의성에 대한 겉으로 드러내는 태도에서 7점 중 4점 이상을 주었고, 두 그룹은 통계적으로 동등하였다.

Beyond merely having a preference for the status quo or for familiar ideas (Eidelman, Crandall, & Pattershall, 2009; Zajonc, 2001), people appear to have ambivalent feelings toward creativity. On the one hand, participants in the baseline condition of Experiment 1 and the high-tolerance-for- uncertainty condition of Experiment 2 demonstrated positive implicit associations with creativity relative to practicality. Additionally, 95% of participants in the uncertainty condition of Experiment 1 and the low-tolerance-for-uncertainty condition of Experiment 2 rated their explicit attitudes toward creativity-related words as positive—higher than 4, the midpoint of a 7-point scale; these ratings were statistically equivalent to the ratings of attitudes toward practicality-related words.

그러나 다른 한 편에서 내포적 척도(implicit measure)는 고불확실성 조건에서 '구토', '독', '고뇌' 등과 연관되었다. 사회적으로 창의성을 지지해야 한다는 강력한 규범이 있고, 사람들이 또한 실제로도 창의성에 긍정적인 느낌을 가지기 때문에 사람들은 그들이 창의적인 생각을 원하지 않는다는 것을 인정하기를 꺼려한다. 그 결과 창의성에 대한 편견은 진단해내기 힘들어진다. 

On the other hand, the implicit measure showed that participants in each high-uncertainty condition (i.e., the uncertainty condition of Experiment 1 and the low-tolerance condition of Experiment 2) associated words such as “vomit,” “poison,” and “agony” with creativity more than with practicality. Because there is such a strong social norm to endorse creativity, and people also feel authentic positive attitudes toward creativity, people may be reluctant to admit that they do not want creativity; hence, the bias against creativity may be particularly slippery to diagnose. The implicit measures may have picked up negative associations with creativity under conditions of uncertainty because the methodology is more resistant to social desirability bias (Greenwald et al., 2009).

사람들의 창의성에 대한 편견이 겉으로 드러나는 것이 아니라면, 우리는 조직이나 기관이나 심지어 과학적인 투자(endeavor)에 있어서도 공식적으로 창의성을 원한다고 언급한다고 해서 진정으로 창의성을 원할지에 대해서 확신할 수 없다. 이는 많은 저널이 창의적인 연구를 극찬하고, 대학에서 창의적인 해결책을 추구하는 과학자를 훈련시키고, 연구와 개발 회사들이 새로운 상품을 개발하고자 하고, 제약회사들이 창의적 돌파구를 극찬한다고 해도 단 하나의 최선의, 또는 가장 정확한 아이디어를 찾아내기 위한 수문장을 두는 방식으로서 불확실성을 장려하며, 이는 창의성에 대한 무의식적(unacknowledged) 혐오를 유발할 수 있다. 또한 우리의 결과가 제안하는 것은 사람들이 창의적 아이디어를 수용하는데 어려움을 겪는다면, 초점은 어떻게 더 창의적인 생각을 만들어낼 것인가를 찾아내는 것이 아니라 창의적인 기관들이 어떻게 인정받게 만들 것이며 어떻게 창의성을 받아들이게 할 것인가가 되어야 한다는 것이다. 후속 연구로 창의성에 대한 편견을 낮추거나 되돌릴 수 있는 요소를 찾아볼 필요가 있다.

If people hold an implicit bias against creativity, then we cannot assume that organizations, institutions, or even scientific endeavors will desire and recognize creative ideas even when they explicitly state that they want them. This is because when journals extol creative research, universities train scientists to promote creative solutions, research and development companies commend the development of new products, and pharmaceutical companies praise creative medical breakthroughs, they may do so in ways that promote uncertainty by requiring gatekeepers to identify the single “best” and most “accurate” idea, thereby creating an unacknowledged aversion to creativity. In addition, our results suggest that if people have difficulty gaining acceptance for creative ideas, especially when more practical and unoriginal options are readily available, the field of creativity may need to shift its current focus from identifying how to generate more creative ideas to identifying how to help innovative institutions recognize and accept creativity. Future research should identify factors that mitigate or reverse the bias against creativity.

The bias against creativity: why people desire but reject creative ideas.

임파워먼트 평가(Empowerment Evaluation)에 대한 고찰과 정부업무평가에의 적용 가능성 탐색: 자체평가를 중심으로

1)이석민(서울대학교) newmind6@snu.ac.kr

임파워먼트 평가는 지역사회심리학, 행동인류학, 현장연구 등에 뿌리를 두고, 참여와 협동을 통한 자기평가와 개선, 자기결정 능력 향상 등 개인역량을 배양함으로써 주체적인 삶의 복원을 그 특징으로 한다. 또한 개인의 변화를 넘어 조직 그리고 공동체의 사회문제 해결능력을 향상시킴으로써 제도개선과 사회변화를 추구한다. 이 평가는 10가지의 평가원리, 3단계의 평가과정을 통해 5단계의 평가의 주요 국면들을 거치게 된다.

한편 한국의 정부업무평가 중 자체평가는 임파워먼트 평가의 자기평가와 동일한 취지와 의미로 사용되고 있으나 실제 운용은 그렇지 않아 지속적인 문제점을 노정하고 있다. 그래서 자체평가의 좀 더 실효적인 성과를 위해서는 임파워먼트 평가모형의 개념과 방법을 적용할 필요성이 있고 적용 가능성도 있음을 본문에서 살펴보았다. 결론적으로 이 글은 자체평가제도의 새로운 대안으로서의 제시보다는 자체평가제도의 숨겨진 고유한 특성을 드러내고 더욱 발전시키는 관점에서 임파워먼트 평가모형의 적용을 제안하고자 한다.

Ⅱ. 임파워먼트 평가

1. 평가의 배경과 이론

1) 배경

Fetterman은 그의 저서 “Speaking the language of power: Communication, collaboration, and advocacy(1993)”에서 그리고 1993년 미국 평가학회(American Evaluation Association) 회장 연설에서 임파워먼트 평가의 아이디어를 언급하고 필요성을 주장하였다. 당시 사회적인 문제에 관심이 많았던 학자들과 정책결정자들은 갈등해결, 낙오(dropout)문제, 건강과 안전, 주거, 교육개혁, 에이즈, 인디언 문제 등의 문제를 해결하고자 하였으며, 이들의 목적은 성공적인 전략을 개발하고 경험을 공유하며 시민과 정책결정기관들과 소통하는 능력을 향상시키는 것이었다. 이때 협력(collaboration), 참여(participation), 그리고 임파워먼트가 중요 요소로 등장하면서 임파워먼트 평가의 개념을 형성하였다.

임파워먼트 평가는 지역사회심리학(community psychology), 행동인류학(action anthropology), 현장연구(action research)에 뿌리를 두고 있다. 

• 지역사회심리학에서는 그들의 일상사에서 주체가 되려는 사람들, 조직, 공동체에 초점을 맞추어 시민참여와 지역사회발전에 대한 광범위한 연구가 이루어져 오고 있다(Rappaport, 1987). 
• 행동인류학자들은 예를 들어 아메리카 원주민과 같은 그룹들이 외부로부터 간섭받지 않고 스스로 목표를 설정하고 성과를 얻기 위해 노력하도록 돕는 것에 더 많은 관심을 기울인다(Tax, 1958; Gearing et al., 1960). 
• 한편 현장연구와 임파워먼트 평가는 둘 다 프로그램 개선을 위해 구체적이고, 시의적절하고, 목표지향적이고, 실용지향적이란 점에서 공통점을 가지고 있으며, 성찰(reflection)과 실행(action)의 순환 그리고 가장 적합한 단순한 데이터 수집방법에 집중한다는 점에서 서로 영향을 주고 받았다(Soffer, 1995).

2) 이론

일반적으로 정치, 사회, 교육, 환경 등의 분야에서 임파워먼트3)는 정치사회적으로 소외된 사람이나 계층에게 힘을 부여하는 의미로 사용이 되고 있다. 그리고 경영학에서는 조직의 팀이나 구성원에게 의사결정권한과 자율성을 부여하는 의미로 사용이 되고 있다. 그러나 어떤 분야에서든 임파워먼트는 단순히 힘을 주거나 권한을 위임하는 것으로 한정되지는 않는다. 경영학에서는 권한부여를 단순한 권한의 위임이나 분배가 아니라 권한을 창조하고 확산하는 것을 의미한다. 이는 개인적 차원에서 자신의 사고변화와 역량을 증대시키고, 아울러 집단 내 구성원들 간의 상호작용을 통해서 더 많은 힘을 발생시켜서 이를 조직이나 제도의 변화 즉 집단과 조직의 임파워먼트로 연결시키고 있다(박원우, 1997). 한편 사회복지나 교육 분야에서의 임파워먼트는 “개인, 집단 또는 지역사회가 자신의 상황을 통제하고 목적을 달성하며, 이로써 자신과 타인의 삶의 질을 극대화 할 수 있도록 돕는 수단”으로 정의하는 등(Adams, 2003: 8) 좀 더 사회변화(social change)의 관점에서 바라보고 있다.

임파워먼트 평가는 다양한 분야(고등교육, 정부, 저소득층 공교육, 비영리단체, 재단 등)에서 사회복지, 범죄예방, 에이즈 예방, 약물남용 방지, 장애인 재활, 농촌발전 등의 프로그램 그리고 정책영역에서 활용되고 있는 혁신적이고 진화하는 접근방법이다. 그리고 개선(improvement)과 자기 결정(self-determination)을 촉진하기 위한 개념이자 기술로서 양적, 질적방법론 모두 수용한다. 또한 이 평가는 개인, 조직, 공동체(community) 그리고 사회와 문화에도 적용이 가능하지만 주로 프로그램에 초점을 맞추고 있다(Fetterman, 1996: 3-4).

특히 이 평가에서 역량부여 과정(empowering process)과 역량이 체화된 결과(empowered outcome)의 의미는 중요하다. 

• 먼저 역량부여 과정에서 우선 개인적 차원에서 사람들은 다른 사람들과 같이 일하고 결정능력을 습득하고, 자원을 관리하는 기회를 얻게 된다. 조직차원에서는 책임성과 리더십이 공유되고 결정에 참여하는 기회가 부여되어야 한다. 공동체 차원에서의 역량부여 과정은 정부, 미디어 그리고 다른 공동체에 대한 접근이 포함되어야 한다. 
• 한편 역량체화 결과에서 먼저 역량이 체화된 개인은 무엇이 변해야 하고 통제되어야 하는지 분석하고 자신감있게 행동한다. 역량이 체화된 그룹은 자원획득을 위해 효율적으로 경쟁하고 정책에 영향을 미치며 사회적 네트워크로 연결되어 있다(Schulz et al.,1995; Zimmerman, 2000: 46).

임파워먼트 평가는 사람들이 스스로를 돕고, 자기평가(self-evaluation)와 성찰(reflection)을 통해 그들의 프로그램을 개선시키도록 디자인되어야 한다. 

• 프로그램 참여자들은 그들 자신의 평가를 수행하며 또한 촉진자로서의 역할을 한다. 
• 한편 평가자의 역할은 협력자 내지 촉진자로의 역할을 수행한다. 협력자로서 평가자는 프로그램 참여자들의 문화, 세계관, 삶의 방식을 통하여 참여자에 관해 배우고 그들을 이끌기보다는 같이 일하고, 평가자들의 기술, 관심, 계획은 공동체에 부여되기보다는 공동체의 자원으로서 역할을 수행하게 된다(Zimmerman, 2000: 44-45).

임파워먼트 평가는 개인적 활동보다는 협력적인 집단활동 과정에서 사람들은 스스로(때로 도움을 받아) 자신을 역량부여할 수 있으며, 이는 참여적이며 민주적인 과정으로 이루어진다. 한편 프로그램의 가치에 대한 사정(assesment)은 평가의 마지막 목적이 아니고 프로그램 개선과정의 한 부분이 된다. 프로그램 참가자들은 스스로 결정한 목표를 향한 과정을 사정하면서 이 결과에 따라 계획과 전략을 수정한다. 대상집단, 목표, 프로그램 실행과 가치에 관한 지식, 그리고 외부조건들이 변화하기 때문에 자기평가와 성찰의 순환 과정을 내부화, 제도화함으로 가치사정과 프로그램 개선이 이루어진다. 그 과정에서 자기결정 능력이 길러지고 계발(illumination)이 일어나고 해방(liberation)이 실현된다(Fetterman, 1996: 5-6).

2. 평가의 원리

Fetterman(2005a: 42-72, 92-122)과 Wandersman(2005: 27-41)은 임파워먼트 평가의 실무를 지도하는 원리를 10가지(개선, 공동체 주인의식, 포섭, 민주적 참여, 사회정의, 공동체의 지식, 증거에 기초한 전략, 능력배양, 조직학습, 책임성 등)로 나누고 각 원리가 평가실무에서 어떻게 적용되는지를 설명하고 있다.

• 첫 번째 원리인 개선(Improvement)에서, 임파워먼트 평가 또는 평가자들은 평가 참여자들이 계획하고 실행하고 평가하도록 평가개념과 평가기법을 활용하는 방법을 습득하게 하고, 평가의 목적을 프로그램 개선에 두게 한다. 
• 두 번째, 공동체는 평가의 설계와 전반적인 방향을 결정하고, 공동체 구성원들은 자신들의 목적을 달성하기 위한 목표와 전략을 구체화하면서 공동체 주인의식(Community ownership)을 형성하게 된다. 
• 세 번째로 포섭(inclusion)이란 가능한 많은 이해관계자들을 참여시키고 독려하는 협동적인 노력을 말한다. 특히 다양성이 높은 조직일수록 다양한 이해관계자들을 목표와 전략에 대한 토론에 참여시킴으로써 장래의 공동체적 합의도출을 용이하게 한다. 
• 네 번째로 민주적 참여(democratic participation)는 포섭된 집단들이 어떻게 상호작용하고 의사결정을 하는가에 대한 것이다. 민주적 참여는 공평성과 공정성을 확보하는 수단이 되는 한편, 증거에 입각한 토론과 신중한 사고 및 행동을 통해 사회현상을 분석하는 능력을 향상시키는 측면도 있다. 
• 다섯 번째로 사회정의(social justice)는 임파워먼트 평가의 근본적 지도원리이다. 평가자료에 따라 효과성이 없는 사업이라 해도 사회정의 아젠다가 부정적 평가에 우선하는 경우에는 효과성을 보완할 수 있는 다른 방안을 강구할 것이다.
• 여섯 번째 원리인 공동체의 지식(community knowledge)에서, 지역공동체는 자체적으로 공동체의 지식을 개발하고 공유하는 상향식 접근을 취하는데, 이러한 지식이 확산된다면 조직변화를 효과적으로 촉진할 수 있을 것이다. 
• 일곱 번째로 증거에 기초한 전략(evidence-based strategies)은 특정 전문가의 판단에 기초하기 보다는 다른 유사한 공동체나 집단에서 사용되고 검증된 성과에 대한 정보와 신뢰성 있는 대안들을 제시해준다. 
• 여덟 번째로 역량을 배양(capacity building)하는 것은 임파워먼트 평가의 가장 두드러지는 특징으로서, 프로그램 참여자들은 평가논리, 논리모형, 평가설계, 평가윤리의 습득뿐만 아니라 판단과 해석, 자료의 활용, 프로그램에 대한 형성적(formative)･총괄적(summative) 평가를 수행하는 분야에서도 역량을 키우게 된다.
• 아홉 번째로 임파워먼트 평가는 조직학습(organizational learning)을 통해 학습공동체를 창출할 수 있게 한다. 피드백을 통해 환경변화나 새로운 도전에 대한 조직의 반응성을 높이고 새로 도입된 전략의 수용성을 높인다. 
• 마지막 원리로서 책임성(accountability)은 먼저 조직이나 프로그램에 구조적으로 피드백 메커니즘이 설계되어 제도적으로 정착되면, 개인이나 조직 모두 내부적 압력에 책임을 지게 된다. 이와 같은 내적 책임성은 외부적 요구 및 수요에 대응하는 외적 책임성도 달성할 수 있다.

<그림 1>은 공동체, 자금제공자, 평가자 등 필수요소로 구성된 사회구조(social container)에서 참여자 간 상호작용과 평가의 원리에 의해 임파워먼트와 자기결정의 수준이 높아지고 있음을 보여주고 있다.

How much of the relation between population mortality and unequal distribution of income is a statistical artefact?

Hugh Gravelle

Summary points

절대소득가설, 즉 개개인의 소득이 많을수록 사망의 위험이 감소한다는 가설을 뒷받침하는 여러 근거가 있다.

The absolute income hypothesis, which states that the higher an individual's income, the lower his or her risk of mortality, is supported by a considerable body of evidence

그러나 상대소득가설, 즉 한 사회 내에서의 소득 분배가 개개인의 사망위험에 영향을 준다는 가설은 점점 더 많은 empirical work 에서 사용되고 있다.

However, the relative income hypothesis—that the distribution of income in a society affects the individual's risk of mortality—is being used increasingly in empirical work 

소득의 불균등한 분배와 인구집단의 건강 사이의 연관성은 통계적 artifact일 수도 있으며, 이는 개개인의 자료가 아닌 인구집단 수준의 자료를 사용한 것에서 기인한다. 즉 "생태학적 오류"의 사례인 것이다.

Associations between unequal income distribution and population health may be a statistical artefact resulting from the use of aggregate rather than individual data—an example of the “ecological fallacy” 

인구집단의 데이터를 사용한 경우 절대소득가설과 상대소득가설을 구분할 수 없는데, 소득 재분배 정책의 효과를 판단하기 위해서는 개개인의 자료를 분석해야 한다.

Because studies using population data cannot distinguish between the absolute and relative income hypotheses, the effects of income redistribution policies on population health can only be judged from individual data, interpreted by models of behaviours that affect health 

만약 [개개인의 사망위험에 개개인의 소득이 미치는 영향]이 [소득이 높을 때 더 작다]면, 이러한 인구집단 수준의 artifactual correlation은 항상 발생하기 마련이다. 개개인 수준에서 소득 분배 정도와 사망 위험 사이에 아무런 상관관계가 없을 때도 이런 일이 생긴다. 

The spurious or artefactual correlation at population level between population mortality and income dispersion will always occur if the effect of individual income on the individual risk of mortality is smaller at higher incomes than at lower incomes. This will be so even if there is no underlying relation between the distribution of income and the risk of mortality at the level of the individual.

국가B가 소득 격차가 더 크지만 국가A와 국가B는 서로 같은 평균소득을 가진다. 그러나 국가B의 인구집단 사망이 국가A의 인구집단 사망보다 더 크다. 

Individual mortality and individual risk

The greater population mortality in the country that has a less equal income distribution (country B), results entirely from the shape of the relation between individual income and the individual risk of mortality. 

The higher income of rich people in country B compared with rich people in country A reduces their risk of mortality by m2A - m2B compared with rich people in country A. 

However, the lower income of poor people in country B compared with poor people in country A increases their risk of mortality by m1B - m1A compared with poor people in country A. 

고소득 구간에서는 소득이 사망에 미치는 영향이 적다. 이렇게 소득이 올라갈수록 단위소득 증가당 사망감소가 점차 줄어드는 상황에서는 저소득층의 소득을 떼어다가 고소득층에게 주는 것은 고소득층의 사망을 줄이는 것보다 저소득층의 사망을 늘리는 효과가 더 클 것이다. 

Because the impact of income on mortality is smaller at higher incomes, the reduced mortality of the rich is more than offset by the increased mortality of the poor and popu­lation mortality is therefore higher in country B. If mortality declines with income, but at a decreas­ing rate, transferring income from the poor to the rich will increase the mortality risk of the poor more than it reduces the mortality risk of the rich. 

모든 개인의 사망위험이 다른 사람의 소득수준이 아닌 그 자신의 소득에 따라서만 변하더라도, 인구집단의 사망은 불균형이 커질수록 높아진다. 

Overall population mortality increases when inequality increases, even though every individual's risk of mortality depends only on their own income level and not on the income level of anyone else.

How much of the relation between population mortality and unequal distribution of income is a statistical artefact?

PMID:

 

9487182

 

[PubMed - indexed for MEDLINE] 

PMCID:

 

PMC2665558

 

Free PMC Article

A Meta-Analysis of the Five-Factor Model of Personality and Academic Performance

Arthur E. Poropat

Griffith University

This article reports a meta-analysis of personality–academic performance relationships, based on the 5-factor model, in which cumulative sample sizes ranged to over 70,000. Most analyzed studies came from the tertiary level of education, but there were similar aggregate samples from secondary and tertiary education. There was a comparatively smaller sample derived from studies at the primary level. 

다섯 가지 특징 

외향성(Extroversion vs. Introversion), 

개방성(Openness vs. Non-openness), 

동조성(Agreeableness vs. Antagonism), 

성실성(Conscientiousness vs. Undirectedness), 

신경성(Neuroticism vs. Emotional Stability)

다섯 가지 중 학업수행능력과 유의미하게 관련이 있는 것은 동조성, 성실성, 개방성이었다.

Academic performance was found to correlate significantly with Agreeableness, Conscientiousness, and Openness. Where tested, correlations between Conscientiousness and academic performance were largely independent of intelligence. 

When secondary academic performance was controlled for, Conscientiousness added as much to the prediction of tertiary academic performance as did intelligence. Strong evidence was found for moderators of correlations. 

Academic level (primary, secondary, or tertiary), average age of participant, and the interaction between academic level and age significantly moderated correlations with academic performance. Possible explanations for these moderator effects are discussed, and recommendations for future research are provided.

Conclusion

In their review, De Raad and Schouwenburg (1996) concluded that “personality usually comes at the bottom of the list of theorizing” (p. 328) about learning and education. The results of this meta-analysis indicate that personality should take a more prominent place in future theories of academic performance and not merely as an adjunct to intelligence. This research has demonstrated that the optimism of earlier researchers on academic performance–personality relationships was justified; personality is definitely associated with academic performance. At the same time, the results of this research have provided further evidence of the validity of the lexical hypothesis and have established a firm basis for viewing personality as an important component of students’ willingness to perform. And, just as with work performance, Conscientiousness has the strongest association with academic performance of all the FFM dimensions; its association with academic performance rivaled that of intelligence except in primary education. Yet the complications highlighted by the moderator analyses indicate that the relationship between personality and academic performance must be understood as a complex phenomenon in its own right.

Future considerations of individual differences with respect to academic performance will need to consider not only the g factor of intelligence but also the w factor of Conscientiousness. However, the strength of the various moderators examined in this meta-analysis shows that, although it can be stated that personality is related to academic performance, any such statement must be subject to qualifications relating to academic level, age, and the interaction between these variables, and most likely also to range restriction. The degree of heterogeneity identified within the samples indicates that there are likely to be further substantial moderators to be identified. Although the role of personality in academic performance may be both statistically and practically significant, it is also subtle and complex. As such, it is will require much further exploration.

A meta-analysis of the five-factor model of personality and academic performance.

Admission to Medical Curricula in the Netherlands

rules and regulations

Albert Scherpbier, Dear Faculty, Health, Medicine and Life, SciencesFHML, Maastricht University

Halle, May 2013

네덜란드의 의학교육 교육과정 : 아래의 두 가지에서 다루는 요건을 만족해야 함

Medical curricula in the Netherlands

Programmes have to fulfil the Requirements as described in the 2009 Framework for Undergraduate Medical Education in the Netherlands and the European Directive 2005/36/EC of the European Parliament

네덜란드에는 8개의 의과대학이 있음

The Netherlands → 8 medical schools

매년 7천~8천명이 2850개의 자리를 놓고 경쟁함. 즉 의과대학은 '정원이 정해진' 프로그램

Every year: 7000-8000 applicants for 2850 places

Therefore, Medicine is a so-called fixed quota degree programme

의과대학 총 정원 결정

Decision on quota for medical schools

정부가 결정함

Decision about quota → national government

의료수요의 기대수요를 예측하는 기관의 조언에 따라서

Based on advices from a national institute, that predicts the expected need for medical healthcare, 

다음의 파라미터들을 고려함

taking into account parameters like:

-number and age of medical doctors and percentage of medical doctors that work full-time/parttime

-demographic information concerning the Dutch population

-percentage of students that finish medical school succesfully

-percentage of these students that enter a postgraduate specialist trainee programme in the Netherlands

학교별 정원의 결정

Decision on quota per medical school

8개 의과대학에 총 정원을 나눔

The total quota is divided over the eight medical schools →

- roughly 1/8 per school (315-410 per school) (대략 1/8씩)

- variations based on regional availability of clerkship-places (임상실습이 가능한 수준을 고려하여)

- all in joint agreement (more or less) (joint agreement에 따라서)

의과대학에 입학하기 위해서 지원자는...

Demands for admission to Medical School

Applicants should have:

네덜란드 학생은 수학, 물리학, 화학, 생물학에 대해서 충분한 수준의 교육을 받아야 함.

-successfully finished a recognised form of (Dutch) education,including mathematics, physics, chemistry and biology at sufficient level

외국 학생은 네덜란드 교육에서 요구하는 수준에 상당하는 교육을 이수해야 함

-in case of a foreign diploma: finished a form of education that has to be equivalent to the required Dutch education

non-Dutch의 경우 네덜란드어 능력에 대한 증명 필요

-in case of a non-Dutch diploma: proof of sufficient mastery of the Dutch language (NT2 diploma)

어떻게 선발하는가?

4. How are the study places assigned?

세 가지 방법

Three ways:

GPA가 8을 넘으면 일순위 합격

중앙 선발 -> 가중 추첨(GPA)

분권 선발 -> 각 의과대학이 선발

-applicants with GPA>8 are admitted to the school of first choice (8+)

-central procedure → weighted lottery (GPA)

-decentralised procedures → selection by individual medical schools

Up to 2011:

2011년까지는 50%까지 중앙 선발을 하고, 50%까지 분권 선발을 했음.

-at least 50% of admissions per medical school by central procedure

-maximum of 50% of admissions per medical school by decentralised selection (incl. 8+ admissions)

From 2012 onward:

2012년 이후에는, 각 학교가 100%를 다 뽑을 수 있고(0~80%까지 다양함), 나머지는 중앙 선발.

-schools can choose to select up to 100% → variation of 0-80%

-the rest is admitted through the central procedure

-8+ applicants are still admitted to the school of first choice

추첨?

A lottery?

중앙에서(국가가) 선발하는 방식이며, 가중치를 준 추첨 방식이다.

학점에 따라서 다섯 카테고리로 분류한다.

The central (=national) selection procedure = a weighted lottery

5 selection categories based on GPA’s →

A. GPA > 8 automatic admission to the school of first choice

B. GPA 7.5-8.0

C. GPA 7.0-7.5

D. GPA 6.5-7.0

E. GPA 6.0-6.5

Changes → B : C : D : E = 9 : 6 : 4 : 3

추첨?

A lottery?

합 또는 불합

Yes and no

사회적으로 추첨방식은 별로 지지를 받지 못하고 있고, 분권(학교별) 선발방식이 더 지지를 받고 있다.

In society lottery has not much support, decentralised selection procedures have more support!

중요한 질문은, 의과대학학생들이 내신에서 8점 이상을 받은 학생들이기를 원하는지, 아니면 다른 역량을 가진 사람을 찾을 것인지의 문제인 것이다.

The question is also do you want many medical students that have a mark higher then 8 or are you looking also for other competencies?

2011년부터 분권 선발방식과 관련한 여러가지 실험을 하고 2014년에는 100% 분권 선발을 도입할 것이다.

We started in 2011 with experiments around decentral selection And will go for 100% in 2014

Maastricht 의과대학의 선발방식

Decentralised Selection Procedure in Maastricht

1단계 : 포트폴리오(온라인) : 개인정보, 고등학교 내신, 독창적 자질, 지원 동기, PBL에 대한 의견 => 250명 선발

Part 1. Portfolio (online)

- personal information

- secondary school and grades

- distinctive qualities

- why Medicine in Maastricht?

- opinion on problem based learning (PBL)

Top 250 applicants →

2단계 : Maastricht에서의 토요일. CASPer를 활용한 몇 가지의 과제 수행

Part 2. Saturday in Maastricht

- several assignments, among which a computerized test with video vignets (CASPer)

CASPer = Computer-based Assessment for Sampling PERsonal Characteristics (Eva et al, McMaster University)

CASPer의 목적

Aim of CASPer

다음에 대한 정보를 얻기 위함임

The assignments aim to gain information about the following core competencies, such as:

- biomedical knowledge

- empathy

- reflection on own functioning and behaviour

- societal and medical awareness

- handling of ethical dilemmas

- logical reasoning

- cooperation

- organisation

2011학년도, 2012학년도의 분권선발

Decentralised selection 2011-2012  → numbers in Maastricht

미래

Future

이제 막 시작했으며, 이것은 첫 결과일 뿐

We just started, only first results from UM

다른 학교들은 더 오랜 경험을 가지고 있다.

Other schools have longer experience

7개 학교에서 서로 다른 방법을 쓴다.

We use different procedures in 7 schools

2011년 8개의 의과대학은 두 명의 PhD학생이 서로 다른 방법들의 효과에 대해서 연구하도록 지원하였음.

In 2011 the eight medical schools decided to fund two Phd students together to study the different procedures and the effects

(출처 : http://www.mft-online.de/files/9.40_fr_omft2013_scherpbier.pdf)

STRATEGICALLY DESIRABLE BRAND NAME CHARACTERISTICS

Kim Robertson

Inherent brand name characteristics leading to memorable names which support the desired product image are presented for the manager. This exposition of strategically desirable brand name characteristics is based upon a review of relevant psychological, linguistic, and marketing literature.

1. 브랜드네임은 간단해야 한다.

1. The brand name should be a simple word.

Often consumers themselves will simplify a more complex brand name. For example, a Chevrolet becomes a Chevy, a Corvette becomes a Vette, Coca-Cola becomes Coke and Pan American Airlines becomes Pan Am.

Examples of simple brand names are Aim, Raid, Edge, Bold, Sure, Off, Jif, Ban, Bic, and Tide,

2. 브랜드네임은 독특한 단어여야 한다.

2. The brand name should be a distinctive word.

Examples of distinctive brand names often cited are Apple, Mustang, Xerox, Kodak, and Exxon.

3. 브랜드네임은 의미있는 단어여야 한다.

3. The brand name should be a meaningful word.

Thus, brand names such as Slender (versus Metrecal) Die-Hard (versus Delco), Budget (versus Avis), and Sprint (versus MCI) should demonstrate an innate memory advantage.

4. 브랜드네임은 제품과 언어적, 음성적으로 연관되어야 한다.

4. The brand name should be a verbal or sound associate of the product class.

Examples of brand names that demonstrate this desirable characteristic are L'eggs (pantyhose), Craftsman (tools), Check-Up (toothpaste), Alley Cat (cat food), and Coin (financial services).

5. 브랜드네임은 '심상(mental image)'를 유도해야 한다.

5. The brand name should elicit a mental image.

Concrete nouns, with tangible, visual referents (e.g., "dog") more easily elicit these mental images than abstract nouns (e.g., "justice"). Therefore, concrete brand names such as Dove, Mustang, Rabbit, and Apple should inherently be more easily learned and/or retrieved from memory than abstract names such as Pledge, Tempo, Ban or Bold.

6. 브랜드네임은 정서적인 단어여야 한다.

6. The brand name should be an emotional word.

Practical marketing literature seems to recognize the desirability of emotional names by referencing "good" name characteristics such as the name having strong positive associations or connotations, engendering good effective feeling, arousing pleasant emotions, being pleasant sounding, and having strong symbolism.5, 15, 21, 26 Examples of such emotional names are Joy, Caress, Kiss, Love, My Sin,, and Opium.

7. 브랜드네임은 반복적인 소리를 활용해야 한다.

7. The brand name should make use of the repetitive sounds generated by alliteration, assonance, consonance, rhyme, and rhythm.

There are numerous brand names incorporating some type of repetitive sound. Some examples of such names include: Cascade, Solo, Crispy Critters, Rolls-Royce, Pampers, Deep Dish Danish, Max-Pax, Lean Cuisine, Gloss 'n Toss, and Shake 'n Bake.

8. 브랜드네임은 형태소(morphemes)를 활용해야 한다.

8. The brand name should make use of morphemes.

Such a morpheme-based approach to generating brand names has indeed been used4, 5, 33. For example, the brand names Compaq, Sentra, and Lucite were developed using English morpheme combinations. However, anytime a morpheme combination approach is used to build a brand name, it is critical to establish the target market's perceived meaning of the new name. While linguistic scholars may uniformly agree that the Greek root Nike, for example, means "victory," there is no guarantee that a consumer exposed to the running shoe brand name Nike is going to incorporate the concept of "victory" into his or her meaning perceptions.

9. 브랜드네임은 음소(phonemes)를 활용해야 한다.

9. The brand name should make use of phonemes.

For example, when asked to place "mal" and "mil" (differing only in the vowel phoneme /a/ or /i/) along the semantic dimension of size, about 80 percent of subjects agree that "mal" represents a large object and "mil" a smaller object.32 For example, the slogan "we sell the biggest burgers in town" encourages evaluation along the semantic dimension of size. If the desired brand image is one of big burgers, a brand name such as Mal Burgers is more likely to support that image than the name Mil Burgers.

음소 활용의 예

(1) 크기 Size: 

high, front vowel sounds (i,e) connote a small size while low, back vowel sounds (a,o,u) indicate a large size (e.g., Zee versus Koss);

(2) 움직임 Movement: 

high, front vowel sounds indicate dynamic movement while low, back vowel sounds are associated with slow or "heavy"movement, and the consonant combination sl indicates a gliding or slipping movement;

(3) 모양 Shape: 

the acute sounds associated with the high, front vowel sounds indicate sharp, angular shapes while the flat sounds associated with low, back vowel sounds indicate roundish shapes;

(4) 광택 Luminosity: 

high, front vowel sounds and the consonants k, s, and I indicate light while the low, back vowel sounds and the consonants d, m, gr, and br connote darkness;

(5) 젊음 Youth: 

the consonants j, g, ch, and the semi-vowels y and w connote youthfulness and joy (as in Joya and Jovan);

(6) 성 Gender: 

masculinity is associated with plosive and guttural sounds (as in "tiger" or "cougar") and with the low, back vowel sounds, while femininity is associated with the soft sibilants s and c, weak f sounds and the high, front vowel sounds (e.g., Silk-Ease, Zepher, and Cerissa.

The Authors

Kim Robertson, Assistant Professor in the Department of Business Administration at Trinity University in San Antonio, Texas

Acknowledgements

Note: This paper was subsequently published in Journal of Product & Brand Management, Volume 1 Number 3, 1992.

Abstract

Considers psychological, linguistic and marketing aspects of brand name characteristics. Presents the inherent brand name characteristics which lead to memorable names that support the desired product image. Concludes that brand names should be simple, distinctive, meaningful, emotional, make use of morphemes, phonemes, alliteration, consonance, and should make a sound associate of product class, as well as being legally protectable; a well-planned brand name will require less marketing money to achieve recall and image targets.

Article Type:

Keyword(s):

Brand names; Language; Marketing strategy; Psychology.

Journal:

Journal of Consumer Marketing

Volume:

6

Number:

4

Year:

1989

pp:

61-71

Copyright ©

MCB UP Ltd

ISSN:

0736-3761

International Track in Medicine 

• Maastricht 의과대학에서는 네덜란드의 의사가 점차 다양한 문화적 배경의, 다양한 언어를 구사하는 환자들을 만나게 되고, 네덜란드 이외의 국가에서 발생하는 질병까지도 다루게 되는 환경의 변화(세계화, 국제화)에 대비하여 ITM이라는 프로그램을 시작함.
• 이 프로그램은 2011년부터 시작되었으며 매년 60~80명 정도(전체정원 350명)를 선발하는 소규모 Bachelor program으로 영어로 진행함.
  
• 아래는 이 ITM 과목 학생들에 대한 교육 및 시험에 대한 규칙이다.

Education and Examination Rules,

International Track in Medicine (ITM) for numerus fixus students, Curriculum 2011, academic year 2013-2014

CONTENTS:

PARAGRAPH 1- GENERAL...................................................................................... 4

Article 1.1 – Scope of the rules.................................................................................. 4

Article 1.2 – Definitions............................................................................................ 4

Article 1.3 – Objectives and learning outcomes of the ITM...................................... 5

Article 1.4 – Format ITM: Full time, language and study counselling...................... 5

Article 1.5 – Study load............................................................................................. 5

Article 1.6 – Study advice......................................................................................... 5

Article 1.7 – Awarding credits.................................................................................. 5

    Article 1.8 – Fraud…………….……………………………………...……………..5

PARAGRAPH 2 – GENERAL PROVISIONS CONCERNING EXAMINATIONS AND RESITS          6

Article 2.1 – Block examination................................................................................ 6

Article 2.2 –  OSCE (Objective Structured Clinical Examination)/Skills examination       6

Article 2.3 – Progress examination............................................................................ 6

Article 2.4 - Professional behaviour........................................................................... 6

Article 2.5 – Portfolio examination............................................................................ 6

Article 2.6– Consultation skills and Reflection programme(CORE)…...……....…..8 Article 2.7 – Non-block related assignments.....................................................…....8
Article 2.8 –Examination format……………………………………………….......8

Article 2.9 –  Examination results............................................................................. 8

Article 2.10 – Grading and publication of examination results................................. 9

Article 2.11  - Right of inspection............................................................................ 9

Article 2.12 – Validity of the examinations and components of examinations......... 9

Article 2.13 – Exemption from examinations............................................................ 9

Article 2.14 - Resits....................................................................................................9                      

PARAGRAPH 3 – COURSE YEAR 1 ITM……………..

Article 3 – Study programme and examination programme of course year 1 ITM... 9

PARAGRAPH 4 – COURSE YEAR 2 ITM.............................................................. 10

Article 4.1 - Admission............................................................................................ 10

Article 4.2 –Study  and examination programme course year 2 ITM ................... 10

PARAGRAPH 5 – COURSE YEAR 3 ITM.............................................................. 10

Article 5.1 – Admission .......................................................................................... 10

Article 5.2 – Study and examination programme of course year 3 ITM................. 11

PARAGRAPH 6 – Reserved for later provisions....................................................... 11

PARAGRAPH 7 – EXAMINATIONS………………...............……………....….....11

Article 7 – Examinations in the ITM………………………………………………………………………………..11

Article 7.1 - Final examination…………………………………………………....11

Article 7.2 - Flexible bachelor's programme and examination for the flexible   bachelor's programme................................................................................................................11

Article 7.3 - Degree..................................................................................................12

PARAGRAAF 8 – PREVIOUS EDUCATION........................................................ 12

    Article 8.1 – Further previous education requirements (profiles)

    Article 8.2 – Colloquium Doctum (Entrance examination)

    Article 8.3 – Previous education for students with non-Dutch diplomas

    Article 8.4 – vervallen

PARAGRAPH 9 – SERIOUS PROBLEMS REGARDING PROFESSIONAL BEHAVIOUR AND UNSUITABILITY (IUDICIUM ABEUNDI)………………………...........................................................……………13

Article 9.1 – Professional behaviour and performance in the programme including patient care...............................................................................................................13
Article 9.2 – Unsuitability (Iudicium abeundi)........................................................13

PARAGRAPH 10 – LEGAL PROTECTION…………………...……………….......13

Article 10 – Right of appeal………………..……………………………..…...….13

PARAGRAPH 11 – FINAL PROVISIONS.............................................................. 13

Article 11.1 - Amendments...................................................................................... 13

Article 11.2 – Publication........................................................................................ 13

Article 11.3 – Unforeseen circumstances................................................................. 14

Article 11.4 – Hardship clause................................................................................. 14

Article 11.5 – Official title……………………………………....………...……....14

Article 11.6 – Date of commencement…………………………....……………….14

PARAGRAPH 1- GENERAL

적용범위 Article 1.1 – Scope of the rules

1. Students may choose for the International Track in medicine for numerus fixus students, curriculum 2011. These rules apply to all students registered for the academic year 2013-2014 of the International Track in Medicine for numerus fixus students, curriculum 2011 (hereinafter referred to as ITM).

2. The ITM is offered within the Faculty of Health, Medicine and Life Sciences (hereinafter referred to as FHML) of Maastricht University (hereinafter referred to as UM).

각 용어에 대한 정의 Article 1.2 – Definitions

In these rules the following definitions apply:

- Academic year: the time period starting on 1 September and ending on 31 August of the subsequent calendar year.

- Block: theme-based study unit in the first or second course year.

- Block examination: the aggregate of partial assessments throughout the block programme, including the end test.

- Combination table: a table that is determined for each academic year by the Examination Board of Medicine and that shows the rules of combination which will be used to combine the marks a student receives in the progress tests throughout the course year to establish the result of the progress examination for the pertinent course year.

- Competency domains: the various roles present in the professional performance of a doctor as mentioned in the Dutch Blueprint (Raamplan).

- Counsellor: tutor at the Student Counselling Service responsible for general and specific tasks with regards to study advice for all students.

- Course year: Year 1, 2 or 3 of the ITM, offering a further detailed programme.

- Credit: a unit equal to one ECTS credit with a study load of 28 hours.

- ECTS: European Credit Transfer System.

- EleUM: Electronic learning environment of Maastricht University.

- End test: the test that, as part of the block examination, is taken at the end of the respective study unit.

- Examination: an examination is a component of the total examination. An examination may consist of several components/tests: these are weighted in a certain way to lead to one final result.

- Examination Board of Medicine (EC – Examencommissie Geneeskunde): the committee as established to administer the examinations and to execute the organisation and coordination of the examinations.

- Examiner: person appointed by the Examination Board of Medicine to conduct examinations

- Medicine Management Team: the central consultation body medicine within the FHML Institute for Education and tailor-made programmes.

- Mentor: tutor in charge of tasks related to study supervision and the portfolios of individual students.

- OSCE: objective structured clinical examination, being an examination, is a series of simulated clinical situations during which the medical skills and knowledge of students are tested in an integral fashion.

- Portfolio: the portfolio consists of documentation, administrated by the student, of the student’s professional and academic development.

- Portfolio review committee: committee as referred to in the relevant article of the Rules and Regulations

- Practical exercise: the exercise, as follows participation in a practical educational activity, geared towards attaining certain skills, knowledge and insight.

- Practical Medical Training (PMT): programme aimed at teaching knowledge and skills (skills training courses) which are important to exercise the medical profession and including periods in which the student does a practical internship in the health care service.

-Propedeutic phase: the first period of the ITM that corresponds with course year 1. The propedeutic phase of the ITM has no propedeutic examination.

- Raamplan: the Dutch Blueprint for the national curriculum for medical schools as decreed by the Dean’s Council of Medical Sciences, containing the learning outcomes for the education of medical doctors in the Netherlands.

- Rules and Regulations: the regulations to be specified by the Examination Board of Medicine regarding a smooth course of events during examinations and concerning any measures to be taken in that respect, and the guidelines and instructions to examiners regarding the assessment of the persons taking the examination and about determining the results of the examination.

- Skills examination: the skills examination consists of a variety of assessments throughout the year. Each assessment will result in feedback and/or a result that has to be included in the student’s portfolio. The final result of the skills examination will be based on aggregation / combination of all relevant assessment results – as specified in the assessment plan.
- Study portfolio: the portfolio managed by the chair of the Examination Board of Medicine, concerning the study results of the student, as detailed in articles 4.1 to 4.4 of the Rules and Regulations.

- Table of norms: a table presenting the norms put to use in the progress tests. The table of norms is calculated after the test has been taken, and added to EleUM.

- Test: a part of an examination. 

- Tutor: person who supervises students in the study groups of year 1 and 2.
- WHW: Wet op het Hoger onderwijs en Wetenschappelijk onderzoek (Higher Education and Scientific Research Act).

ITM의 목표/학습성과 Article 1.3 – Objectives and learning outcomes of the ITM

1. The objective of the ITM is to impart knowledge, insight and skills in the field of Medicine so they can meet the requirements of the Dutch Blueprint.

3. For the students of the International Track in Medicine (for numerus fixus students) - who wants to continue their education in Medicine- there is a master programme in Medicine at the FHML/UM: the “Reguliere, aansluitende masteropleiding Geneeskunde”.

ITM의 형식(시간, 언어, 지도) Article 1.4 – Format ITM: Full time, language and study counselling

1. The ITM is offered on a full-time basis.
2. The ITM is offered in the English language.
3. The Faculty ensures that the students registered for the programme receive adequate study counselling and guidance.

학습량(학점) Article 1.5 – Study load

The ITM has a study load of 180 credits, divided over three course years of 60 credits each.

학습 조언 Article 1.6 – Study advice : ITM을 계속 할지 말지에 대한 결정에 대한 조언.

1. The Examination Board of Medicine, acting on behalf of the board of the FHML, issues each student, no later than the end of the first year of the student’s registration for the first course year of the ITM, a study advice regarding the continuation of his or her study.

2. Without prejudice to the stipulation in the first section, the Examination Board of Medicine, on behalf of the FHML board, can issue the study advice to the student at any time he or she has not met the requirements to pass the first course year (with a combined study load of 60 credits).

학점 수여 Article 1.7 – Awarding credits

1. Credits are awarded as soon as the student has successfully completed the entire examination concerned.

2. For study taken abroad credits are only awarded if the programme was approved beforehand by the Examination Board of Medicine of the FHML.

사기(기만) Article 1.8 – Fraud : 표절을 포함한 사기(기만)행위는 그것을 '시도하는 것'까지도 포함함. 영구제명 가능.

1. Fraud, including plagiarism, is understood as a student’s act or failure to act that makes it partially or fully impossible to correctly assess his/her knowledge, insight and skills.

2. Plagiarism is understood as the presentation of one’s own or other people’s ideas or words without adequate reference to the source.
3. By fraud is also understood attempted fraud.

4. If the Examination Board of Medicine establishes that a student has committed fraud in an exam or exam component, it may impose suitable measures.

5. In serious cases of fraud, the Examination Board of Medicine can propose to the UM Executive Board that the student(s) concerned be permanently deregistered from the programme.

6. The General FHML-Regulation for Fraud, as drawn up by the Examination Boards, further details what is understood as fraud and what measures can be imposed by the Examination Board of Medicine.

PARAGRAPH 2 – GENERAL PROVISIONS CONCERNING EXAMINATIONS AND RESITS

블록과 클러스터 시험 Article 2.1 – Block and cluster examination

1. Each block and cluster has an examination. The examination can consist of several parts.

2. The content of each block or cluster is prescribed in a description that is posted on Eleum.

OSCE와 술기 시험Article 2.2 – OSCE (objective structural clinical examination)/SKILLS EXAMINATION

1. In course year 1 a skills examination is conducted. In course year 2 and 3 an OSCE is conducted.

진단 평가 Article 2.3 – Progress examination

1. In course year 1, 2 and 3 of the ITM a progress examination is administered to all students. It consists of four tests (test moments).

2. The Interuniversity Progress Test Review Committee determines the rules with regard to the format and content of the progress examination.

3. The progress examination can only be taken one time per academic year. If a student fails to pass the progress test in the first instance, the result of the progress test of the following year of study or the result of a number of progress tests specified by the Board of Examiners  is also counted as the resit test. The student still passes the progress test he failed initially if the student meets the examination requirements of the progress test(s) in question.
3.1 The results obtained in the progress tests are also an obligatory (mandatory) part of the portfolio described in article 2.5 of these rules. The student must include the results in the portfolio in their entirety and without alteration. The progress of the results will be discussed in the progress meetings with the mentor as referred to in article 2.5 of these rules.

전문직 다운 행동Article 2.4 - Professional behaviour
1. Course year 1, 2 and 3 of the ITM is concluded with an examination of professional behaviour, consisting of an assessment on the basis of (a.) the assessments of professional behaviour that are part of regular assessments as described in the Rules and Regulations article 2.4 and (b.) possible reports as meant in article 9(1) of these rules.


포트폴리오 평가 Article 2.5 – Portfolio examination

1               General

The portfolio is 학생이 작성kept up to date by the student and is used as the basis of all formal progress meetings between mentor and student.

2              The portfolio consists of 세 부분three parts: a dossier part, a reflection part and a progress part.(3~5는 각각의 파트에 대한 설명)

2.1          The dossier part contains all required information concerning the development of competencies prescribed and the results of all assessments.

2.2          The reflection part presents a strength-weakness analysis of the competency development.

2.3          The progress part includes a written report of the discussions between mentor and student during the progress meetings, approved by the mentor.

3              The student is responsible for the dossier part to contain sufficient information concerning all formal progress meetings as referred to in paragraph 5.1 of this article.

The dossier part contains at least everything that is available and up to date:

a. Results and assessments as mentioned in the examination regulations (toetsplannen) c.q. portfolio regulations;

b. Other results/assessments as agreed upon in meetings referred to in section 5.1 sub c; and

c. Any other evidence to demonstrate competency development as contributed by the student.

4            The reflection part contains strength-weakness analysis of the student.

The strength-weakness analyses comprise all four roles of the education and examination programme. These roles (competencies) are: role as medical expert, role as scientist, role as a healthcare worker, and role as a person. The strength-weakness analyses need to be updated after each progress meeting.

4.1           The reflection part is always supported by concrete evidence from the file section.

4.2          The reflection part contains concrete learning goals for the time period until the next progress meeting.

5            The progress part contains reports of all formal progress meetings between mentor and student

5.1            The meetings referred to in section 5 have at least the following three points on the           agenda:

a. Retrospection: looking back on earlier agreements reviewing if and how they have been realised.

b. Evaluation and analysis: discussing how everything is going and why.

c. Planning further actions, meetings and agreements: determining for example the planning of the next meeting, the attendance of any extra or remediation education parts/modules, additional examinations or information gathering.

6               Progress meetings : Student and mentor will have several formal progress meetings in year 1, 2 and 3.

6.1          The student will take minutes (notes) of the content of each meeting and present them to the mentor for approval, after which they are included in the progress section of the portfolio.

7               멘토의 조언자적 역할 Advising role of the mentors

7.1          At the end of each study year, the mentor will issue a formal advice regarding the student’s academic progress and competency development on the basis of the portfolio kept by the student, in relation to the learning outcomes of the ITM. This advice is presented to the group of mentors of the relevant year group.

7.2          The advice referred to in section 1 is issued in the last planned meeting between mentor and student, is discussed and is put in writing.

7.3          The advice as referred to in section 1 is in line with the assessments as recorded during the meetings, if these were held in conformity with the regulations that apply as listed in aforementioned sections of this article 5.

8             일년 단위 그룹에 대한 포트폴리오 평가  The portfolio assessment procedure per year group

8.1          The group of mentors of the relevant year group issues a final advice on the basis of the advice of the mentor and the portfolio, in which the mentor concerned has no say.

8.2          An independent assessment committee consisting of 2 members (not mentors) makes a recommendation – on the basis of the portfolio of the student and the final advice issued by the mentor group - to either award or not award the study credits associated with the portfolio of the relevant year group to the Examination Board of Medicine.

8.3          If the recommendation as referred to in section 2 is negative, and is accepted by the Examination Board of Medicine, the student will have to follow a remediation programme.

8.4          If the remediation programme as mentioned in section 3 has not led to the required competency development, the student will not be admitted to the next course year, unless decided otherwise by the Examination Board of Medicine.

9               학생/멘토를 바꿀 권리 Right to choose another student/mentor

If there is an irreconcilable difference of character between mentor and student they are both at liberty to request a different combination. The request will be reviewed by the Examination Board of Medicine.

10            second opinion을 제시할 권리  Right to a second opinion

In the case of a difference of opinion, the student and the mentor both have a right to a second opinion by an expert third party, to be appointed by both.

11            포트폴리오의 진실성  Authenticity of portfolio material

By including materials in the portfolio the student is implying by default that this is authentic own material.

상담/성찰 Article 2.6 – Consultation skills and Reflection programme (CORE)

1. Course year 1, 2 and 3 provides a CORE programme.

블록 비관련 과제 Article 2.7 – Non-block-related assignments

1. Course year 1, 2 and 3 has some non-block-related assignments, which extend in time over multiple blocks. These interdisciplinary assignments are included in the provisions in these rules for course year 1, 2 and 3.

시험의 형식 Article 2.8 –Examination format

1. The Examination Board of Medicine determines if examinations are conducted in written, oral or other form.

2. Examinations are carried out in the English language.

장애가 있는 학생에 대한 조항

3. For students with disabilities, arrangements can be made, once a request to this effect has been submitted, to take examinations in a manner adapted to accommodate their individual disability, in so far as reasonably possible in view of the objective of the study programme and the educational and organisational provisions.

Where necessary the Examination Board of Medicine will seek expert advice before making their decision.

4. The oral examination is public, unless the Examination Board of Medicine decrees otherwise in a special case or the student objects to this.

5. The Examination Board of Medicine specifies, at the suggestion of the examiners as mentioned in article 2.1 – 2.7 written examination regulations and resit regulations. These regulations describe the format and content of the tests and resit tests, the timetable of the testing, the appropriate pass/fail scores and the method for determining the results of the test. These regulations are included on Eleum.

시험 결과 Article 2.9 – Examination results

1. The result of the examination will be qualified as decreed for the concerned examination in these rules or in the Rules and Regulations.

2. The examination is completed successfully if the result has been qualified as at least a ‘pass’.

시험 결과의 고지 및 성적부여 Article 2.10 – Grading and publication of examination results

일반적으로 20일 이내, 구술시험은 즉시 또는 여러 학생이 연속해서 볼 경우 5일 이내

1. Unless other rules have been specified in these rules or the Rules and Regulations, the examiner will grade an examination or a part of an examination within 20 weekdays (weekdays being Mo-Fr) of the day the examination was taken. The examiner provides the organisation of education/the secretariat of the Examination Board of Medicine with the relevant information in behalf of the publication of the examination results to the student. The publication of the examination results to the students will also take place within 20 weekdays (weekdays being Mo-Fr) of the day the examination or a part of the examination was taken.

2. In contravention to section 1, the examiner will grade an oral examination (or part thereof) immediately after taking said examination and provide the concerned student and the organisation of education/ the secretariat of the Examination Board of Medicine with evidence of the grade. If multiple students take the same examination soon after one another, this term of notification may be extended by 5 weekdays.

성적 검토 권한 Article 2.11 - Right of inspection

1. Within 10 weekdays (weekdays being Mo-Fr) (at most) of the announcement of the results of a written (including computer-based) exam, the student will have the right to request inspection of the assessed work.

2. Within the period mentioned in paragraph 1, the student will have the right to request inspection of the exam questions and assignments, and the standards used to assess them.

3. The announcement of a written (including computer-based) exam’s results will specify how the right of inspection can be obtained.

시험과 구성요소의 유효기간 Article 2.12 – Validity of the examinations and components of examinations

통과한 시험의 유효기간은 60개월

1. The period of validity for passed examinations is 60 months.

2. Passed examination components of a not yet passed examination remain also valid for 60 months unless determined otherwise in the relevant rules, including the examination regulations (toetsplannen).

3. The Examination Board of Medicine can extend in exceptional cases the period of validity of passed examinations or components of examinations by a period to be decided by the Examination Board of Medicine itself. The Examination Board of Medicine may impose additional or alternative requirements on the student in the process.

시험의 면제 Article 2.13 – Exemption from examinations

1. The Examination Board of Medicine can at the request of the student grant exemption from certain examinations or parts thereof, if the student can prove that he or she has passed a module comparable in scope and content in another study programme, or if the student can demonstrate to the satisfaction of the Examination Board of Medicine that he/she has acquired competences elsewhere that are comparable to the module for which exemption is being required.

2. Granting exemption does not create any obligation on the part of the Faculty to offer an alternative study programme module.

사기(기만)행위로 인한 처벌기간동안 치른 시험은 고려대상 아님

3. The Examination Board of Medicine will not grant any exemption based on exams passed by a student outside the programme during the period in which the student was barred from taking exams for the programme by the Examination Board of Medicine because of fraud.

재시험 Article 2.14 – Resits

For each failed exam a resit exam will be offered once within the academic year. If you failed an exam that consists of multiple components, you can only resit the components you have failed. Regarding the Professional Behaviour examination year 3, the EC will provide an appropriate resit arrangement/opportunity.

PARAGRAPH 3 – COURSE YEAR 1 ITM

1학년 Article 3 – Study programme and examination programme of course year 1 ITM

1. Course year 1 of the ITM contains the following examination components:

a. Block 1.1 Growth and Development I (7 credits);

b. Block 1.2 Breathing and Circulation I (7 credits);

c. Block 1.3 Regulation and Integration (4 credits);

d. Block 1.4 Thinking and Doing I (7 credits);

e. Block 1.5 Digestion and Defence I (7 credits);

f. Block 1.6 Diabetes, Obesity and Lifestyle (4 credits);

g. Skills examination  year 1 (6 credits);

h. Progress test (5 credits) and the following examinations and assignments:

i. Professional behaviour examination year 1 (4 credits);

j. Portfolio examination year 1 (5 credits).

k. Non-block related assignment: Imaging techniques (1 credit)

l. Non-block related assignment: Personal Formulary (1 credit)

m. Consulting skills and Reflection Education (CORE) year 1 (2 credit)

2. An outline of the content and aims of the examination components listed in section 1 can be found via EleUM.

PARAGRAPH 4 – COURSE YEAR 2 ITM

진입 Article 4.1 - Admission

1. The student is awarded unconditional admission to the study programme and examination programme of course year 2 if the 60 credits of course year 1 have been obtained.

2. The Examination Board of Medicine can conditionally admit the student to the study programme and examination programme of course year 2 before said student has obtained the necessary 60 credits of the first year, on the condition that at least 40 credits have been obtained, of which at least the 25 credits from the block examinations of course year 1, unless the Examination Board of Medicine decides otherwise.

3. The Examination Board of Medicine determines the period of conditional admission. This period is maximum 12 months after the conditional commencement of the study and examination programme of course year 2.

4. Admission to the study programme and examination programme of course year 2 becomes unconditional if at the latest within the period as determined by the Examination Board of Medicine (see section 3 above) the requirements for gaining the 60 credits of course year 1 have been met.
5. The credits obtained from the examinations of course year 2, that the student takes during his or her conditional period of admission, are awarded unconditionally.

2학년 과목 Article 4.2 –  Study programme and examination programme course year 2 ITM

a. Block 2.1  Breathing and circulation II  (7 credits);

b. Block 2.2 Growth and development II  (7 credits);

c. Block 2.3  Elective (4 credits)

d. Block 2.4 Digestion and defense II  (7 credits);

e. Block 2.5 Thinking and doing II (7 credits);

f. Block 2.6 Elective (4 credits)

g. OSCE tests year 2 (6 credits);

and the following examinations and assignments:

h. Progress test (5 credits)

i. Professional behaviour examination year 2 (4 credits);

j. Portfolio examination year 2 (5 credits).

k. Non-block related assignment: Personal Formulary (1 credit)

l. Consulting skills and Reflection Education (CORE) year 2 (3 credits)

2. An outline of the content and aims of the examination components  listed in section 1 can be found via eleUM.

PARAGRAPH 5 – COURSE YEAR 3 ITM

진입 Article 5.1 – Admission

1. The student is awarded unconditional admission to the study programme and examination programme of course year 3 if the 120 credits of course year 1 and 2 have been obtained.

2. The Examination Board of Medicine can conditionally admit the student to the study programme and examination programme of course year 3 before said student has obtained the necessary 60 credits of the second year, on the condition that at least 100 credits have been obtained  of which 60 credits out of year 1, of which at least the 21 credits from the block examinations of course year 2 unless the Examination Board of Medicine decides otherwise.

3. The Examination Board of Medicine determines the period of conditional admission. This period is maximum 12 months after the conditional commencement of the study and examination programme of course year 3.

4. Admission to the study programme and examination programme of course year 3 becomes unconditional if at the latest within the period as determined by the Examination Board of Medicine (see section 3 above) the requirements for gaining the 60 credits of course year 2 have been met.
5. The credits obtained from the examinations of course year 3, that the student takes during his or her conditional period of admission, are awarded unconditionally.

3학년 과목 Article 5.2 Study programme and examination programme course year 3 ITM

1. Course year 3 has as its theme ‘Chronic disorders’. Course year 3 contains the following examination components:

a. Cluster Abdomen (10 credits)

b. Cluster Circulation and Lungs (10 credits)

c. Cluster Locomotor Apparatus (10 credits)

d. Cluster Psychomedical Problems (10 credits)

e. OSCE tests year 3 (2 credits) and the following examinations:

f. Progress test (4 credits);

g. Professional behaviour examination year 3 (4 credits)

h. Portfolio examination year 3 (4 credits)

i. Non-cluster-related programme: Chronicity (1 credit)

j. Non-cluster related programme: health law and health ethics (1 credit)

k. Consultation skills and Reflection Education (CORE) year 3 (2 credits)
l. Academic skills (2 credits)

2. A broad outline of the aims and content of the examination components as listed in section 1 can be found via eleUM.

PARAGRAPH 6 – RESERVED FOR LATER PROVISIONS

시험 PARAGRAPH 7 – EXAMINATIONS

ITM에서의 시험 Article 7 – Examinations in the ITM

The following examinations referred to in Chapter 7 of the WHW must be taken as part of the ITM:

a. The final bachelor’s examination (Article 7.10 of the WHW);

b. The examination for the flexible bachelor’s degree programme (Article 7.3d of the WHW).

최종 시험 Article 7.1 – Final examination

A student will have passed the final bachelor’s examination if he or she has attained all 180 credits for course years 1, 2 and 3 of the ITM.

유연성 Article 7.2 – Flexible bachelor’s programme and examination for the flexible bachelor’s programme

1. A student who is enrolled for the ITM may, with the Examination Board of Medicine’s permission, design his/her own programme from study units given in English by an institution of academic education, with this programme including an examination.

2. The flexible programme must entail a study workload of 180 credits.

3. The Examination Board of Medicine decides whether to grant permission within four weeks after receiving the student’s proposal.

4. The examination for this flexible bachelor’s programme does not entitle the student to admission to the education and examination programme for the follow-up master’s degree.

학위 Article 7.3 – Degree

1.1 Students passing the bachelor’s examination of the ITM for numerus fixus students are awarded the degree of ‘Bachelor of Science’. The degree also indicates the subject area and professional field to which the degree relates.

1.2 Students passing the examination for the flexible bachelor’s programme are awarded a bachelor’s degree. Pursuant to Article 7.10a of the WHW “of science” is added to the degree granted.

2. As proof that the examination was passed, a testimony (getuigschrift) is issued by the Examination Board. The testimony for the examination passed indicates in any event the following:

a. the name of the programme;

b. the components of the examination;

c. (where appropriate) the right to engage in a profession specified in the testimony;

d. the degree awarded;

e. the date on which the programme was most recently accredited or reviewed.

3. Students entitled to receive a testimony may, stating reasons and in accordance with UM rules, ask the Examination Board to postpone issue of the testimony.

4. The testimony is signed by the chair of the Examination Board of Medicine and the dean.

5. The testimony is issued in public, unless the Examination Board of Medicine decides otherwise in exceptional cases.

6. A list of the examination components and a diploma supplement are issued with the testimony.

7. The Examination Board of Medicine can add the distinction ‘Cum Laude’ in accordance with the provisions in the Rules and Regulations.

PARAGRAPH 8 – PREVIOUS EDUCATION

Article 8.1 – Further previous education requirements (profiles)

1. Students who have a diploma referred to in Article 7.24 or 7.28 of the WHW which does not meet the further previous education requirements (profiles) designated under Article 7.25 of the WHW cannot be admitted to the programme until, in the assessment of the Colloquium Doctum and Special Admissions Board, requirements substantively similar to the further previous education requirements (profiles) have been met.

Article 8.2 – Colloquium Doctum (Entrance examination)

1. The admissions test referred to in Article 7.29 of the WHW is performed by the Colloquium Doctum and Special Admissions Board for Medicine.

2. The admissions test consists of the components English language, physics, chemistry, mathematics and biology.

3. The Colloquium Doctum and Special Admissions Board for Medicine can grant an exemption for components of the test if, in the Board’s judgment, the candidate has demonstrated that he or she meets substantively similar requirements.

4. The rules and procedures regarding the admissions test will be included in regulations to be adopted by the Examination Board of Medicine concerning admission requirements to the Maastricht Medicine programme.

Article 8.3 – Previous education for students with non-Dutch diplomas

1. Subject to the third paragraph of this article, students who have a diploma which, pursuant to Article 7.28(2) of the WHW, has been designated by ministerial regulation as at least equivalent to the diploma for pre-university education are exempted from the previous education requirements.

2. Subject to the third paragraph of this article, students who have a diploma, whether issued in the Netherlands or not, which, in the judgment of the Colloquium Doctum and Special Admissions Board for Medicine, is at least equivalent to the diploma for pre-university education may be exempted by the Executive Board from the previous education requirements.

3. Students who have received an exemption from the previous education requirements pursuant to the first and second paragraph cannot be admitted to the programme until, in the assessment of the Colloquium Doctum and Special Admissions Board for Medicine, requirements substantively similar to the further previous education requirements (profiles) have been met.

4. The rules and procedures regarding the assessment referred to in the third paragraph will be included in regulations to be adopted by the Examination Board of Medicine concerning admission requirements to the Maastricht Medicine programme.

Article 8.4 –Vervallen

PARAGRAPH 9 –SERIOUS PROBLEMS REGARDING PROFESSIONAL BEHAVIOUR AND UNSUITABILITY (IUDICIUM ABEUNDI)

부적절한(unprofessional) 행동 Article 9.1 – Professional behaviour and performance in the programme including patient care

1. If at any moment during the programme the professional behaviour of the student is judged, by a tutor, a staff member or a person not directly involved in the programme (in teaching), to be below standard for performing in the programme including patient care, the tutor, the staff member or the person not directly involved in the programme will report this in writing and with reasons stated to the (review) committee of professional behaviour (this applies to all three course years of the ITM). 

2. On the basis of the report, the Examination Board of Medicine, if necessary in consultation with the (review) committee of professional behaviour, will instigate an investigation into the professional behaviour of the student. The Examination Board of Medicine will send written notice of the instigation of the investigation on the basis of this article to the student and to the FHML Board. The investigation will be completed within a period of six weeks.

3. During the investigation, the student and the involved tutor, staff member or person not directly involved in the programme will be heard. The findings of the investigation will be recorded in written form.

4. The results of the investigation will be attached to the notice and included in the study portfolio.

5.The Board of Examiners will seek an individual solution.

등록 취소 Article 9.2 - Unsuitability (Iudicium Abeundi)

1. In exceptional circumstances and after carefully weighing the interests at stake, the Examination Board may, stating reasons, ask the Dean to request that the Executive Board terminate or deny a student’s registration if, through his/her conduct or statements, the student shows that he/she is unsuitable to practice one or more professions for which the ITM is training him/her or is unsuitable for the practical preparation for the profession.

2. If the Dean of the faculty is asked by the Executive Board for a recommendation on a proposed termination or denial of registration based on the reasons stated in paragraph 1, the Dean will in turn ask for a recommendation from the Examination Board. The recommendation to the Dean will be supported by reasons.

PARAGRAPH 10 – LEGAL PROTECTION

항고 권리 Article 10.1 - Right of appeal

When a decision by the examiner and the Examination Board is announced to a student, the student will also be notified of the right to file an appeal with the Office for Student Legal Protection within six weeks after the decision is announced.

PARAGRAPH 11 – FINAL PROVISIONS

Article 11.1 - Amendments

1. Amendments of these rules are determined by the FHML Faculty Board.

2. No amendments will be made that apply to the current academic year, unless it is reasonable to assume that no student interests will be prejudiced.

Article11.2 – Publication

1. The FHML Faculty Board shall ensure suitable publication of these rules, of the Rules and Regulations, and of any amendments to these papers.

2. Any interested party can obtain a copy of the papers as referred to in section 1 from the secretarial office of the FHML Institute for Education.

Article 11.3 – Unforeseen circumstances

In circumstances not provided for by these rules, the Examination Board of Medicine will make a decision on behalf of the Faculty Board. This decision will be in writing and supported by arguments.

Article 11.4 – Hardship clause

The Examination Board of Medicine is authorised to deviate from these rules in individual cases, if the unaltered application will in their opinion, due to exceptional circumstances, lead to serious injustices.

Article 11.5 – Official title

These rules will be referred to as the Education and Examination Rules, International Track in Medicine (ITM) for numerus fixus students, curriculum 2011, academic year 2013-2014.

Article 11.6 – Date of commencement

These rules take effect on 1 September 2013and apply to the academic year 2013-2014.

As established by the dean of the FHML on behalf of the Board of the UMC⁺   on 14 May 2013.

(출처 : https://www.google.co.kr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CEAQFjAB&url=http%3A%2F%2Fwww.maastrichtuniversity.nl%2Fweb%2Ffile%3Fuuid%3D47550d1f-5317-452e-a9fd-dd05a2891c77%26owner%3D10767728-c015-44db-968c-eb022e3b3932&ei=6-Q0U5b_EMfdigfiyoC4BQ&usg=AFQjCNEI-qNRNc31cfb1YXSMfGe3PIGhMw&sig2=8QT3Hk3-0QPPuNYH6v3W7A)

신뢰도 : 한 척도가 측정하고자 하는 것을 일관되게 반영하고 있는가

신뢰도에 대해서 생각해볼 수 있는 한 가지 방법은, 한 사람이 서로 다른 두 시점에 시험을 보았을 때 동일한 점수를 받아야 한다는 것이다. 

다른 방법으로 split half reliability가 있는데, 임의로 반으로 나눠서 두 개의 절반을 비교하면 그 둘이 같거나 거의 같아야 한다는 것이다. 그러나 이 방법의 문제점은 어떻게 반을 나누느냐에 따라서 결과가 달라질 수 있다는 것이다. 이 문제를 해결하기 위해서 Cronbach는 loosely equivalent to splitting data in two in every possible way and computing the correlation coefficient for each split을 사용했다. 이 값의 평균은 Cronbach's alpha와 같은데, 흔히 신뢰도의 척도로 사용되는 것이다.

두 가지 버전이 있는데 한 가지는 normal이고 다른 하나는 standardized version이다. normal은 한 척도에 속하는 아이템의 총합이 그 척도에 대한 하나의 대표값을 이루는 것이다. 이 경우 standardized alpha는 적절하지 않다. Standardized alpha는 합해지기 전에 한 척도에 속하는 아이템들이 standardized 될 때 사용한다.

Cronbach's alpha의 해석.

흔히 0.7~0.8이 수용가능한 Cronbach's alpha라는 말을 듣곤 할 것이다. 이것보다 값이 크게 낮다면 신뢰할 수 없는 척도라고 할 수 있다. Kline은 비록 일반적으로 0.8이 수용할 수 있는 수준이긴 하나, 지능검사 같은 인지적 검사에서는 0.7을 쓸 수도 있다고 말했다. 또한 psychological construct에 대해서는 심지어 0.7보다 작을 수도 있다고 주장했다.

또한 Cortina는 alpha의 값이 그 척도에 몇 개의 아이템이 있느냐에 따라 달라지기 때문에 이러한 일반적인 가이드라인은 유의해서 사용해야 한다고 지적했다.

alpha는 또한 '역으로 채점되는 아이템'에 의해서도 영향을 받는다.

신뢰도 분석에서 심지어 이런 'reverse scored item'은 Cronbach's alpha를 음수로 만들 수도 있다.

따라서 그러한 문항이 있다면 그 문항이 계산되는 방식을 바꿔서 다시 계산해야 한다.

Factor Analysis 편에서 orthogonal rotation을 사용하여 4개의 subscale을 만든 바 있다.

Scale if item deleted를 체크하면, 그 아이템이 지워졌을 경우 alpha가 어떻게 변하는지 보여준다. 만약 설문이 reliable하다면 하나의 아이템이 지워졌다고 해서 전체의 reliability에 크게 영향을 주면 안된다.

Corrected Item-Total Correlation은 아이템과 총점 사이의 correlation울 보여준다. 신뢰성이 있는 척도라면 모든 아이템은 총합 점수와 correlate해야한다. 만약 0.3보다 작은 것이 있다면 그 특정 아이템이 잘 correlate하지 않는 다는 것을 말하고, 그 아이템은 제거되어야 한다.

마지막으로, 가장 중요한 것은 아래 나와있는 alpha는 Cronbach's alpha로서 overall reliability of the scale을 뜻한다. 

만약 어떤 아이템을 제거해야 할 필요가 생기면 factor analysis를 다시 수행하는 것이 좋다.

Cronbach's alpha를 보고하는 법은 간단하다.

~은 good internal consistency를 가지는 것으로 나타났으며, alpha=.82이다.

(부가적) 모든 아이템은 포함시킬 가치가 있으며(worthy of retention), 10번 item을 삭제시 alpha가 가장 많이 증가하였지만, 0.005수주이었다. 모든 아이템은 총점과 일정정도의 상관관계를 가졌다. (r 최저값 = .40)

(출처 : http://www.statisticshell.com/docs/reliability.pdf)

(출처 : http://www.statisticshell.com/docs/onewayanova.pdf)

Step 6: Post-hoc tests

Once you have determined that differences exist among the group means, post hoc pairwise and multiple comparisons can determine which means differ. SPSS presents several choices, but different post hoc tests vary in their level by which they control Type I error. Furthermore, some tests are more appropriate than other based on the organization of one's data. The following information focuses on choosing an appropriate test by comparing the tests.

A summary leading up to using a Post Hoc (multiple comparisons):

Step 1. Test homogeneity of variance using the Levene statistic in SPSS.

a. If the test statistic's significance is greater than 0.05, one may assume equal variances.

b. Otherwise, one may not assume equal variances.

Step 2. If you can assume equal variances, the F statistic is used to test the hypothesis. 

If the test statistic's significance is below the desired alpha (typically, alpha = 0.05), then at least one group is significantly different from another group.

Step 3. Once you have determined that differences exist among the means, post hoc pairwise and multiple comparisons can be used to determine which means differ. Pairwise multiple comparisons test the difference between each pair of means, and yield a matrix where asterisks indicate significantly different group means at an alpha level of 0.05.

Step 4. Choose an appropriate post hoc test:

a. Unequal Group Sizes: Whenever you violate the equal n assumption for groups, select any of the following post hoc procedures in SPSS: LSD, Games-Howell, Dunnett's T3, Scheffé, and Dunnett's C.

b. Unequal Variances: Whenever you violate the equal variance assumption for groups (i.e., the homogeneity of variance assumption), check any of the following post hoc procedures in SPSS: Tamhane’s T2, Games-Howell, Dunnett's T3, and Dunnett's C.

c. Selecting from some of the more popular post hoc tests:.

▷Fisher's LSD (Least Significant Different): This test is the most liberal of all Post Hoc tests and its critical t for significance is not affected by the number of groups. This test is appropriate when you have 3 means to compare. It is not appropriate for additional means. (비교할 그룹이 3개일때)

▷Bonferroni (AKA, Dunn’s Bonferroni): This test does not require the overall ANOVA to be significant. It is appropriate when the number of comparisons (c = number of comparisons = k(k-1))/2) exceeds the number of degrees of freedom (df) between groups (df = k-1). This test is very conservative and its power quickly declines as the c increases. A good rule of thumb is that the number of comparisons (c) be no larger than the degrees of freedom (df).  (비교의 개수가 df보다 크지 않을 때)

▷Newman-Keuls: If there is more than one true null hypothesis in a set of means, this test will overestimate they familywise error rate. It is appropriate to use this test when the number of comparisons exceeds the number of degrees of freedom (df) between groups (df = k-1) and one does not wish to be as conservative as the Bonferroni. (비교의 개수가 df보다 크면서 Bonferroni만큼 보수적이지 않아도 될 때)

▷Tukey's HSD (Honestly Significant Difference): This test is perhaps the most popular post hoc. It reduces Type I error at the expense of Power. It is appropriate to use this test when one desires all the possible comparisons between a large set of means (6 or more means). (가장 유명한 것. 비교의 개수가 많을 때(6개 혹은 그 이상의 평균들의 비교))

▷Tukey's b (AKA, Tukey’s WSD (Wholly Significant Difference)): This test strikes a balance between the Newman-Keuls and Tukey's more conservative HSD regarding Type I error and Power. Tukey's b is appropriate to use when one is making more than k-1 comparisons, yet fewer than (k(k-1))/2 comparisons, and needs more control of Type I error than Newman-Kuels. (k-1개 이상의 비교)

▷Scheffé: This test is the most conservative of all post hoc tests. Compared to Tukey's HSD, Scheffé has less Power when making pairwise (simple) comparisons, but more Power when making complex comparisons. It is appropriate to use Scheffé's test only when making many post hoc complex comparisons (e.g. more than k-1). (많은 post hoc complex comparison을 하고자 할 때)

(출처: https://umdrive.memphis.edu/yxu/public/SPSS%20ANOVA.pdf)

• Post hoc tests에 대해서는 p-value와 effect size를 보고한다.
• 헐크와 슈퍼맨, 헐크와 스파이더맨을 제외하고는 General homogeneity가 이뤄지지 않아서, 이 경우 Games-Howell post hoc test를 사용하였다. local homogeneity의 경우 Gabriel's test를 사용하였다. 이들 test는 슈퍼맨 그룹과 헐크 그룹의 경우 p=0.008, d=1.62였으며, 슈퍼맨 그룹과 닌자거북이 그룹에서는 p=0.016, d=2.60이었고 ... 

Effect Sizes : Cohen's d

• 유의성 검증에서 끝나지 않고 effect를 계산하는 것이 유용하다. 
• d에 hat을 씌운 것은, 이것이 'estimate of'라는 의미이다. 집단의 effect size를 알고 싶지만 이것을 직접적으로 측정할 수 없기 때문에 샘플로부터 추정하는 것이며 d는 signal-to-noise ratio라고 할 수 있다.

• 그러나 평균이 두 개라면 어떤 SD를 사용하느냐의 문제가 있다. 세 가지 방법을 고려할 수 있다.
• (1) 한 그룹이 통제집단이라면 그 그룹의 SD를 이용하여 d를 계산하는 것이 합리적이다. 이 경우 통제집단의 SD는 natural variation의 'purer' measure이다.
• (2) 그룹 variance가 동일하다고 가정하는 경우가 있는데(HoV), 이 때라면 어떤 그룹의 SD를 선택해도 무관하다.
• (3) 'pooled estimate'라는 것을 사용할 수 있다. 계산하는 공식은 위와 같다.

• Cohen은 effect size에 대해서 0.2를 small, 0.5를 medium, 0.8을 large라고 제안했다. 
• 그룹 사이에 SD가 서로 다를 때 pooled estimate가 유용할 수 있다. 그러나 이 경우 d의 의미가 바뀌는데, 그 이유는 모든 background noise에 대해서 그 차이를 비교했기 때문이다. 이것은 normal circumstance에서 나타날 수 있는 noise에 대해서 계산된 것이 아니다.

(출처 : http://www.statisticshell.com/docs/onewayanova.pdf)

1.판단을 하지 말고, 상황을 묘사하라

1.Make it descriptive, not judgmental

2.대략적으로, 광범위한 것보다는 구체적인 것에 대하여 하라

2.Be specific, not general

3.개인에 대한 것보다는 변화될 수 있는 행동에 초점을 두라

3.Focus on changeable behaviors rather than individual

4.행동에 따른 결과에 초점을 두라

4.Emphasize consequences

5.적절한 시점에 하라

5.Be timely

6.직접 보고 들은 정보를 바탕으로 하라

6.Base on first hand information

(출처 : 9th Jan. 2014. : The Art of Giving Effective Feedback – SCIME – YH & SK.)

처음의 네 단계는 경화된 현재 상태를 녹이는 역할을 하며, 5단계부터 7단계까지는 새로운 실천방식을 도입하는 과정이다. 8단계는 조직문화의 변화를 정착시키는 과정이다. 리더들은 종종 5,6,7단계만 시도하고는 하나 1단계부터 4단계까지의 '녹이는'과정을 무시한다면 굳건한 기반을 마련하지 못할 것이다. 그리고 8단계가 없이는 변화가 유지되지 못할 것이다.

The first four steps help unfreeze a hardened status quo. Steps five to seven introduce new practices. Step eight grounds the changes in the organizational culture. Leaders often attempt only steps five, six, and seven. However, neglecting any of the unfreezing activities (steps one through four), precludes the establishment of a solid base. Without step eight, the changes won't stick.

1. 변화가 시급하다는 분위기 조성하기 ↔ 변화가 시급하다는 분위기 조성에 실패

1. Establishing a Sense of Urgency
Examining the circumstances and identifying and discussing crises, potential problems, or major opportunities.

2. 리더십 그룹 형성하기↔ 강력한 지도부 부재

2. Creating the Leadership Group
Putting together a group with enough power to lead the change and getting the group to work together as a team.

3. 비전과 전략 개발하기↔ 비전의 부재

3. Developing a Vision and Strategy
Creating a vision to help direct the change effort and developing strategies for achieving that vision.

4. 의사소통을 통해 바뀐 비전을 공유하기↔ 비전의 소통과 공유의 부족

4. Communicating the Change Vision
Using every means possible to communicate the new vision and strategies, and having the leadership group model behavior expected of others in the organization.

5. 광범위한 행동을 지원(empowering)하고 이에 맞게 조직을 조정(align)하기↔ 새 비전을 위한 장애물 제거 안함

5. Empowering Broad-based Action or Aligning the Organization
Getting rid of obstacles, changing structures or systems that undermine vision, and encouraging risk taking nontraditional ideas, activities, and actions.

6. 단기 성과를 이룩하기↔ 체계적으로 계획하지 않고, 단기 성과를 이룩하지 않음

6. Generating Short-term Wins
Planning for visible improvements in performance, creating those wins, and visibly recognizing and rewarding people who make the wins possible.

7. 이점/이득을 축적하고 더 많은 변화를 만들어나가기↔ 너무 일찍 성공을 선언함

7. Consolidating Gains and Producing More Change
Using newly developed and increased credibility to change all systems structures and policies that don't fit the vision; developing the capacity of people who can implement the vision; and "reimaginating" the process with new projects, themes, and change agents.

8. 새로운 접근법을 조직문화로 만들기↔ 변화된 것을 조직의 문화로 들여오지 않음

8. Institutionalizing New Approaches in the Culture
Creating better performance through customer-oriented focus, better leadership, and more effective management; articulating the connections between new behaviors and organizational successes; and creating means to ensure leadership development and succession.

학습 조직

The Learning Organization

학습 조직이란 사람들이 '알기'보다 '배우기'를 추구하는 곳이다. 그들은 그들이 가진 세계관이 완벽하지 못함을 인정하기에, 전체 시스템을 더 잘 이해하기 위해서 서로로부터 배운다. 이것은 서로의 관점을 존중하며 알아가는(respectful inquiry)과정을 통해 이루어진다.

A learning organization is a place in which people seek to learn rather than know. They admit that their pictures of the world are incomplete and therefore they need to learn from each other in order to fully understand the whole system. They do this through a process of respectful inquiry into each other's world views.

학습 조직에서는 탐구(inquiry)와 성찰(reflection)이 일이 진행되는 것과 함께 자연스럽게 같이 이루어지며, 이것이 장기적으로 조직을 효율적으로 만든다.

A learning organization is structured to include mechanisms which ensure inquiry and reflection as a natural part of ongoing work, which leads to long term organizational effectiveness

리더십

- Hierarchical                 :            위계적

- Symbolic (스티브잡스)    :            상징적

- Bureaucratic                 :           관료적

- Facilitative                   :            촉진적

The Four Developmental Stages of the Organization

1단계 - 위계 조직

Phase 1 - The Hierarchical Organization

Leadership in the hierarchical organization operates in the style of benevolent paternalism. Orders and incentives come from the top down. Management may believe in spending time listening to what subordinates say, but this feedback is commonly ignored in the real process of management. The worker is imaged as a child who is cared for by a fatherly leader, and who, by following the rules and working hard, can win favour in the organization. The top-rated skills involve problem-solving, administrative effectiveness and “keeping calm.” Basically, this style is reactive. The main agenda is responding to problems and crises as they occur. A major preoccupation of management is keeping labour submissive, which may involve discouraging unions.

Harrison Owen in his book, Spirit, has a telling description of the reactive organization: “It doesn’t seem to make much difference what you do, just do something: react. Things get done but what things and to what purpose is not always clear.” Overly zealous accountability means the workforce sees keeping and looking busy as a prime value.

2단계 - 제도 조직

Phase 2 - The Institutional Organization

While the hierarchical organization at its worst is characterized by the “big boss” style and the crisis mode, the institutional organization is a miracle of organization. There are the board of directors, the shareholders, the CEO, the vice presidents of this, that and the other, the managers, the supervisors, the workers.

This is the large, efficient bureaucracy. Its style is that of responsiveness. As Harrison Owen remarks, this kind of organization “is truly a pleasure to work with, for they seem to recognize what the business is, and are prepared to go all the way to ensure that you, the customer, are fully met, even if they do not understand all the details of the operation.”

3단계 - 협력 조직

Phase 3 - The Collaborative Organization

If the first two phases were primarily concerned with the patterns of power relations, profit, efficient production and customer service, the preoccupations of the last two phases are with maximizing vision, creativity, interaction, communication and collaboration.

If the first two phases are all about structure, the last two are more about process, though structure remains important.

Interaction is the core characteristic of the collaborative phase. These organizations aim for real teamwork between all members and departments. Their missional goal is to make a quality impact on society. Structurally, this is a lattice organization. All the parts are integral to the whole, and no part may be replaced without altering the whole. They are concerned about reducing rigidity, and increasing the flow of creativity.

4단계 - 학습 조직

Phase 4 - The Learning Organization

The learning organization is blessed with a high degree of interactive learning, an emphasis on human resource development and concern with “making a difference.” To some extent, the organization itself becomes a message to the world, offering its own vision of human relations for the future. This phase involves a new take on “quality” and “learning”. The learning organization is necessarily a network, made up of self-directed teams.

Using the model of servant leadership, the leaders quietly enable others to maximize their performance through a system of layered mentoring. The quality of communication is empathetic throughout. The worker is a microcosm of the organization, and is encouraged to assume responsibility for the whole, beyond his or her job description. Outside involvement in the community and personal growth are encouraged as relevant to the organization’s vision.

218vjjm5vbb6m03n9i0us72tn401OrgJourneyArticle.pdf

(http://www.inc.com/lou-adler/best-interview-question-ever.html)

The Only Interview Question That Matters BY Lou Adler

This single question provides the depth and breadth of insight needed to reach a fast, unanimous Hire/Don't Hire decision.

지금까지 당신의 경력(커리어)에서 가장 의미있는(중요한, Significant) 성취를 한 가지 고른다면 무엇을 꼽겠습니까?

What single project or task would you consider your most significant accomplishment in your career to date?

이 질문이 왜 그렇게 강력한지를 알아보려면, 당신이 면접을 보고 있고, 방금 그 질문을 받았다고 생각해보라. 어떤 성취를 선택하겠는가?

To see why this simple question is so powerful, imagine you're the candidate and I've just asked you this question. What accomplishment would you select?

그리고는 다음 15분~20분동안 다음의 질문을 추가로 던질 것이다. 어떻게 대답하겠는가?

Then imagine that over the course of the next 15-20 minutes I asked you the following follow-up questions. How would you respond?

•좀 더 구체적으로 묘사해줄 수 있나요?

Can you give me a detailed overview of the accomplishment?
•당신의 동료, 직함, 위치, 역할, 관여된 팀에 대해서 말해주세요.

Tell me about the company, your title, your position, your role, and the team involved.
•실제로 성취한 결과는 무엇인가요?

What were the actual results achieved?
•언제 있었던 일이고, 얼마나 오래동안 일어난 일이었나요?

When did it take place and how long did the project take?
•당신이 선택된 이유는?

Why were you chosen?
•그 문제를 해결하는 동안 부딪친 3~4개의 가장 큰 과제는 무엇이었나요?

What were the 3-4 biggest challenges you faced and how did you deal with them?
•특별히 애를 쓰거나 솔선해서 했던 것은 무엇인가요?

Where did you go the extra mile or take the initiative?
•그 계획에 대해서 설명해주세요. 어떻게 Manage했는지, 어떻게 성공 여부를 측정했는지

Walk me through the plan, how you managed it, and its measured success.
•그 일을 둘러싼 환경과 자원에 대해서 말해보세요.

Describe the environment and resources.
•상관(manager)의 스타일과 당신은 그 스타일을 어떻게 생각했는지 말해주세요

Explain your manager's style and whether you liked it.
•목표를 달성하기 위해서 필요로 했던 기술들은 무엇이며, 그 기술들은 어떻게 사용되었나요?

What were the technical skills needed to accomplish the objective and how were they used?
•가장 큰 실수는 무엇인가요?

What were some of the biggest mistakes you made?
•그 프로젝트에서 진정으로 즐겼던 면은 어떤 점인가요?

What aspects of the project did you truly enjoy?
•가장 관심을 두었던 측면과 그것을 어떻게 다루었는지(handle) 설명해주세요.

What aspects did you not especially care about and how did you handle them?
•다른 사람들을 어떻게 관리했고 어떻게 영향을 주었는지 예를 들어보세요.

Give examples of how you managed and influenced others.
•한 인간으로서 어떻게 변화했으며 어떻게 성장했나요?

How did you change and grow as a person?
•다시 한다면 어떤 면을 다르게 할 것 같은가요?

What you would do differently if you could do it again?
•그 일로 인해 공식적으로 인정받은 것이 무엇인가요?

What type of formal recognition did your receive?

충분히 대단한 성취이고, 그 세세한 내용이 20분은 충분히 채울 수 있다면 면접관은 이 한 가지의 질문만으로도 응시자에 대해 알아야 할 모든 것을 알아낼 수 있다. 중요한 것은 이 질문 자체가 아니고, 이것은 그저 장치일 뿐이다. 가장 중요한 것은 그 성취에 관한 디테일이며, 그 디테일을 파고드는 것이 바로 면접에서 진짜로 다루고자 하는 것이다.

With an accomplishment big enough, and answers detailed enough to fill 20 minutes, this one line of questioning can tell an interviewer everything he or she needs to know about a candidate. The insight gained is remarkable. But the real secret ingredient is not the question; that's just a setup. The most important elements are the details underlying the accomplishment. This is what real interviewing is about -- delving into the details.

똑똑한 면접 질문을 던지기 위해서 시간을 많이 쓰지 말고, 대신 이 한 가지 질문에 시간을 투자하라. 질문을 다시 던지고 점을 연결하라.

Don't spend time asking clever interview questions; instead, spend time learning to get the answer to just this one question. Then ask it again and begin to connect the dots. After you hire a few people this way, you'll also call it the most important interview question of all time.

(http://en.wikipedia.org/wiki/Instructional_design)

Instructional Design (also called Instructional Systems Design (ISD)) is the practice of creating "instructional experiences which make the acquisition of knowledge and skill more efficient, effective, and appealing."^[1] The process consists broadly of determining the current state and needs of the learner, defining the end goal of instruction, and creating some "intervention" to assist in the transition.

Ideally the process is informed by pedagogically (process of teaching) and andragogically (adult learning) tested theories of learning and may take place in student-only, teacher-led or community-based settings. The outcome of this instruction may be directly observable and scientifically measured or completely hidden and assumed. There are many instructional design models but many are based on the ADDIE model with the five phases: analysis, design, development, implementation, and evaluation. As a field, instructional design is historically and traditionally rooted in cognitive and behavioral psychology, though recently Constructivism (learning theory) has influenced thinking in the field.^[2][3][4]

Understanding, using and calculating effect size

 “Effect size … allows us to move beyond the simplistic, “Does it work or not?” to the far more sophisticated, “How well does it work in a range of contexts?” (Coe, 2002)