설문지 선택, 설계, 개발 (BMJ, 2004)

설문지 선택, 설계, 개발 (BMJ, 2004)

Selecting, designing, and developing your questionnaire

Petra M Boynton, Trisha Greenhalgh

설문이 이렇게 유명해 진 것은 "응급조치"로 쓸수 있기 때문이다. 이렇게 남용된 방법론은 없었다.

The great popularity with questionnaires is they provide a “quick fix” for research methodology. No single method has been so abused.1

무슨 정모를 모으고자 하는가?

What information are you trying to collect?

간략히 말하자면, 연구분야에 대해서 충분히 잘 알지 못하거나, 가능한 응답의 범위를 예측하기 위한 특정 인구집단에 친숙하지 않고, 특히 그러한 자료가 문헌에 없다면 우선적으로 질적접근(포커스 그룹) 등을 통해서 영역territory를 파악하고, 핵심 영역을 파악해야 한다.

As a rule of thumb, if you are not familiar enough with the research area or with a particular population subgroup to predict the range of possible responses, and especially if such details are not available in the lit- erature, you should first use a qualitative approach (such as focus groups) to explore the territory and map key areas for further study.6

설문이 적합한 방법인가?

Is a questionnaire appropriate?

설문의 부적절한 활용 Table A

Table A on bmj.com gives some real examples where questionnaires were used inappropriately.1

기존 도구를 사용할 수 있는가?

Could you use an existing instrument?

시간과 자원을 절약할 수 있고, 다른 연구와 비교할 수 있다. 논문을 쓸 때도 개요만 적어주면 되고 논문 내기도 쉽다.

Using a previously validated and published question- naire will save you time and resources; you will be able to compare your own findings with those from other studies, you need only give outline details of the instru- ment when you write up your work, and you may find it easier to get published (box 1).

설문지가 타당성 신뢰성을 갖추었는가?

Is the questionnaire valid and reliable?

질문을 어떻게 보여줄 것인가?

How should you present your questions?

Open or Closed. 두 접근의 장단점(Table B)

Questionnaire items may be open or closed ended and be presented in various formats (figure). Table B on bmj.com examines the pros and cons of the two approaches.

• 자료를 빠르게 정리하기는 좋으나, 잠재적 응답의 풍요로움이 낮다.
  Closed ended designs enable researchers to produce aggregated data quickly, but the range of pos- sible answers is set by the researchers not respondents, and the richness of potential responses is lower. Closed ended items often cause frustration, usually because researchers have not considered all potential responses (box 2).18 

• 개방형 질문을 쓸 때는 사전에 어떤 식으로 분석할지 계획이 있어야 한다. 시간이나 전문성이 없다면 애초에 하지 마라
  If you plan to use open ended questions or invite free text comments, you must plan in advance how you will analyse these data (drawing on the skills of a quali- tative researcher if necessary).19 You must also build into the study design adequate time, skills, and resources for this analysis; otherwise you will waste participants’ and researchers’ time. If you do not have the time or expertise to analyse free text responses, do not invite any.

질문 외에 더 넣을 것은?

Apart from questions, what else should you include?

Table C 체크리스트. 도입부 혹은 설문 정보

A common error by people designing questionnaires for the first time is simply to hand out a list of the ques- tions they want answered. Table C on bmj.com gives a checklist of other things to consider. It is particularly important to provide an introductory letter or information sheet for participants to take away after completing the questionnaire.

 

 

 

어떤 식으로 보여야 하는가? (레이아웃)

What should the questionnaire look like?

연구자들인 물리적 레이아웃에 신경을 잘 안쓴다. 그러나 연구 결과를 보면, 낮은 응답률은 질문을 읽거나 따라잡기 어려워서 생기는 경우가 많다. 일반적으로 질문은 짧고, 요점을 바로 향해야 하며(12단어 이내), 민감하거나 내인적인 질문이 있어서는 안된다. 짧은 질문이 너무 abrupt하거나 threatening하다면, 더 긴 문장을 써도 된다.

Researchers rarely spend sufficient time on the physical layout of their questionnaire, believing that the science lies in the content of the questions and not in such details as the font size or colour. Yet empirical studies have repeatedly shown that low response rates are often due to participants being unable to read or follow the questionnaire (box 3).3w6In general, questions should be short and to the point (around 12 words or less), but for issues of a sensitive and personal nature, short questions can be perceived as abrupt and threatening, and longer sentences are preferred.w6

표본 선정

How should you select your sample?

Table D

Different sampling techniques will affect the questions you ask and how you administer your questionnaire (see table D on bmj.com).

 

 

무슨 승인을 받아야 하는가?

What approvals do you need before you start?

과학적으로 타당하지 않거나 의도하지 않은 offence나 트라우마를 유발하거나, 비밀유지를 깨거나, 사람의 시간이나 돈을 낭비하게 한다면 비윤리적인 것이다.

A study is unethical if it is scientifically unsound, causes undue offence or trauma, breaches confidential- ity, or wastes people’s time or money.

결론

Conclusion

Table E 체크리스트

Table E on bmj.com gives a critical appraisal checklist for evaluating questionnaire studies.

Boynton PM, Wood GW, Greenhalgh T. Hands-on guide to questionnaire 7 research: reaching beyond the white middle classes. BMJ (in press).

Table E Critical appraisal checklist for a questionnaire study

Research question and study design

• What information did the researchers seek to obtain?

• Was a questionnaire the most appropriate method and if not, what design might have been more appropriate?

• Were there any existing measures (questionnaires) that the researchers could have used? If so, why was a new one developed and was this justified?

• Were the views of consumers sought about the design, distribution, and administration of the questionnaire?

Validity and reliability

• What claims for validity have been made, and are they justified?
  (In other words, what evidence is there that the instrument measures what it sets out to measure?)

• What claims for reliability have been made, and are they justified?
  (In other words, what evidence is there that the instrument provides stable responses over time and between researchers?)

Format

• Was the title of the questionnaire appropriate and if not, what were its limitations?

• What format did the questionnaire take, and were open and closed questions used appropriately?

• Were easy, nonthreatening questions placed at the beginning of the measure and sensitive ones near the end?

• Was the questionnaire kept as brief as the study allowed?

• Did the questions make sense, and could the participants in the sample understand them? Were any questions ambiguous or overly complicated?

Instructions

• Did the questionnaire contain adequate instructions for completion
  —eg example answers, or an explanation of whether a ticked or written response was required?

• Were participants told how to return the questionnaire once completed?

• Did the questionnaire contain an explanation of the research, a summary of what would happen to the data, and a thank you message?

Piloting

• Was the questionnaire adequately piloted in terms of the method and means of administration, on people who were representative of the study population?

• How was the piloting exercise undertaken—what details are given?

• In what ways was the definitive instrument changed as a result of piloting?

Sampling

• What was the sampling frame for the definitive study and was it sufficiently large and representative?

• Was the instrument suitable for all participants and potential participants? In particular, did it take account of the likely range of physical/mental/cognitive abilities, language/literacy, understanding of numbers/scaling, and perceived threat of questions or questioner?

Distribution, administration and response

• How was the questionnaire distributed?

• How was the questionnaire administered?

• Were the response rates reported fully, including details of participants who were unsuitable for the research or refused to take part?

• Have any potential response biases been discussed?

Coding and analysis

• What sort of analysis was carried out and was this appropriate?

• (eg correct statistical tests for quantitative answers, qualitative analysis for open ended questions)

• What measures were in place to maintain the accuracy of the data, and were these adequate?

• Is there any evidence of data dredging—that is, analyses that were not hypothesis driven?

Results

• What were the results and were all relevant data reported?

• Are quantitative results definitive (significant), and are relevant nonsignificant results also reported?

• Have qualitative results been adequately interpreted (e.g. using an explicit theoretical framework), and have any quotes been properly justified and contextualised?

Conclusions and discussion

• What do the results mean and have the researchers drawn an appropriate link between the data and their conclusions?

• Have the findings been placed within the wider body of knowledge in the field (eg via a comprehensive literature review), and are any recommendations justified?

 

---

BMJ. 2004 May 29;328(7451):1312-5.

Selecting, designing, and developing your questionnaire.

Boynton PM1, Greenhalgh T.

Author information

• 1Department of Primary Care and Population Sciences, University College London, Archway Campus, London N19 5LW. p.boynton@pcps.ucl.ac.uk

PMID:

 

15166072

 

[PubMed - indexed for MEDLINE] 

PMCID:

 

PMC420179

 

Free PMC Article