학술활동을 위한 역량, 마일스톤, EPA의 감독 수준 (Acad Med, 2018)

학술활동을 위한 역량, 마일스톤, EPA의 감독 수준 (Acad Med, 2018)

Competencies, Milestones, and a Level of Supervision Scale for Entrustable Professional Activities for Scholarship
Richard B. Mink, MD, MACM, Angela L. Myers, MD, MPH, David A. Turner, MD,
and Carol L. Carraccio, MD, MA

문제
Problem

학술활동은 학부 및 대학원 의학 교육(GME)의 중요한 요소이며, 소아과 커뮤니티가 역량 기반 평가 모델을 거주지 및 동료에 완벽하게 통합하기 위해 노력해왔기 때문에, 우리는 교육생들의 [학술활동 수행능력 수준]을 평가하기 위한 도구가 부족하다는 것을 발견했다. 모든 전문 분야의 전공의에게 학술 활동에 참여할 것을 강력히 권장하지만, 소아 세부전공에서는 필수 사항이다. 3년 과정인 소아 세부전공 각각에 대한 ACME(Acreditation Council for Graduate Medical Education) 요건에 명시된 바와 같이, "각 펠로우십 책임자 및 지정된 멘토의 지침에 따라 학술 프로젝트를 설계 및 수행해야 합니다."1  

Scholarship is an important element of both undergraduate and graduate medical education (GME), and as the pediatric community has worked to fully integrate a competency-based assessment model into its residencies and fellowships, we discovered a lack of tools for evaluating trainees’ level of performance in scholarship. Participation in scholarly activity is highly encouraged for residents in all specialties but is a requirement for pediatric subspecialty training. As noted in the Accreditation Council for Graduate Medical Education (ACGME) requirements for each of the three-year pediatric subspecialties, “Each fellow must design and conduct a scholarly project … with the guidance of the fellowship director and a designated mentor.”1 

미국 소아과 위원회(ABP)-
American Board of Pediatrics (ABP)-

EPA는 [역량 및 마일스톤 평가에 대한 맥락을 정의하고, 전체 의대 교육 연속체에 걸쳐 평가에 대한 접근방식을 통합하기 때문에] 평가에 대한 성과-기반 접근방식의 핵심 요소이다. 
EPAs are a key element of an outcomes-based approach to assessment, as they define the context for competency and milestone assessment and unify the approach to assessment across the full continuum of medical education. 

저자들이 학술활동 평가를 위한 도구 부족 문제에 접근하기 시작하면서 다른 EPA에 사용된 것과 다른 프레임워크를 사용하면 전문 활동으로서 연구research를 과대평가하거나 과소평가할 위험이 있다고 생각했다. 따라서 우리는 소아과가 EPA, 역량 및 이정표의 기존 프레임워크에 통합함으로써 학술활동 평가가 최선의 구현 전략이라고 강하게 느꼈다. 
As the authors began approaching the problem of a lack of tools to assess scholarship, we felt that using a framework different from that used for the other EPAs would run the risk of either over- or undervaluing research as a professional activity. Hence, we felt strongly that assessment of scholarship, through incorporation into pediatrics’ existing framework of EPAs, competencies, and milestones, was the best implementation strategy. 

접근법
Approach

처음에 2015년 5월에 우리는 학술활동 EPA가 개념화될 수 있는 세 가지 잠재적 패러다임 6-8을 밝히는 문헌 검색을 수행했다. 

• 그 중 첫 번째는 Glassick의 것으로, 6은 Boyer5가 제안한 모든 형태의 학술활동에 적용할 수 있는 6가지 기준을 제안했다. 그러나 이 프레임워크에는 성과 수준이 포함되지 않았다. 
• 의사-학자 특유의 역량을 포함하는 CANMED는 두 번째 잠재적 프레임워크를 제공했지만 교육 연속체에 걸친 포괄적인 일련의 이정표를 제공하지는 않았다.7 
• 세 번째 프레임워크인 Vitae Researcher Development Framework(RDF) 8은 이러한 노력의 템플릿 역할을 가장 잘 수행했습니다.

Initially, in May 2015, we performed a literature search that revealed three potential paradigms6–8 in which the scholarship EPA could be conceptualized. 

• The first of these was from Glassick,6 who proposed six standards for scholarship applicable to all of the forms of scholarship proposed by Boyer5; however, this framework did not include performance levels. 
• CanMEDS, which includes competencies specific to the physician–scholar, provided a second potential framework but did not provide a comprehensive set of milestones across the educational continuum.7  
• The third framework, the Vitae Researcher Development Framework (RDF),8 was the one that best served as a template for this effort.

RDF는 영국에서 조사관 교육을 지원하기 위해 설립되었다.8 연구자 인터뷰를 토대로 구성되었으며, 아래 4개 분야를 포함한다.

• (1) 지식과 지적 능력, 
• (2) 개인의 효과, 
• (3) 연구 거버넌스 및 조직, 
• (4) 참여, 영향력, 영향 

The RDF was created to support the training of investigators in the United Kingdom.8 It was constructed based on interviews with researchers and includes four domains: 

• (1) knowledge and intellectual abilities; 
• (2) personal effectiveness; 
• (3) research governance and organization; and 
• (4) engagement, influence, and impact.
  
  

평가 도구(표 1 및 2)를 개발하면서 2015년 6월 ACGME에서 요구하는 기술을 포함하여 장학 EPA의 핵심 역량을 확인하고 ABP의 학술활동 커리큘럼 표준에 기술하였다.

• 문헌 검토 수행 및 연구 설계의 이해, 
• 연구비 지원 방법,
• 생물 통계학
• 연구에 관련된 윤리적 원칙과 같은 것이다.

In developing the assessment tools (Tables 1 and 2), we started in June 2015 by identifying key competences for the scholarship EPA, including those skills required by the ACGME1 and described in the ABP’s scholarship curriculum standards,2 such as 

• performing a literature review and understanding study design, 
• how to apply for funding, 
• biostatistics, and 
• the ethical principles involved in research.

 

결과
Outcomes

평가서 최종본에서는 학술활동 [관련 위탁 결정 시 중요한 역량 8가지]를 확인하였습니다(표 1). 이러한 역량은 대부분 Glassick,6가 제안하는 내용과 유사하며, 이는 이 툴의 콘텐츠 유효성에 대한 추가 증거를 제공합니다. 각 역량에 대해 5가지 마일스톤 레벨을 개발했으며, 다른 소아과 이정표와 일관성을 유지하기 위해 [초보자에서 전문가에 이르는 단계]에 걸쳐져 있다. 
In the final version of the assessment instruments, we identified eight competencies important in making entrustment decisions related to scholarship (Table 1). These competencies largely mirror those suggested by Glassick,6 providing additional evidence for the content validity of this tool. For each competency, we created five milestone levels and, to be consistent with the other pediatric milestones, these levels span the continuum from novice to expert. 


"연구research"라는 단어는 더 전통적인 가설 중심 연구 노력과만 관련되는 경향이 있기 때문에 의도적으로 포함하지 않았습니다. 그래서 가능한 한 포괄적으로 표현하기 위해 "연구" 대신 "장학scholarship"을, "연구 과제"보다는 "학술 프로젝트scholarship project"를, "가설" 대신 "질문 또는 개입question or intervention"과 같은 용어를 사용했습니다. 우리는 또한 온라인 포털, 회의 진행, 스마트폰 애플리케이션 개발과 같은 다른 수단을 통한 지식 전파뿐만 아니라 기존의 연구 기사뿐만 아니라 대중에게 지식을 전파하는 것을 포함하는 광범위한 의미에서 "출판publication"이라는 용어를 사용했다. 
We purposely did not include the word “research” because this term tends to be associated only with more traditional hypothesis-driven research efforts. Thus, to be as inclusive as possible, we used terms such as “scholarship” in place of “research,” “scholarly project” rather than “research project,” and “question or intervention” as opposed to “hypothesis.” We also used the term “publication” in a broad sense that includes not only traditional research articles but also knowledge dissemination to the public through other means, such as online portals, conference proceedings, and the development of smartphone applications. 


8개의 역량 중 6개는 ACME와 ABP가 규정한 요건에 명시되어 있다. 명시되지 않은 두 가지는 협업과 멘토링이며, 이는 학술 활동에서 생산성을 내기 위해 이러한 역량을 통합해야 한다는 중요한 필요성을 강조한 소아 전문가들의 의견을 토대로 추가되었다.
Six of the eight competencies are explicitly addressed in requirements stipulated by the ACGME and ABP.1,2 The two that are not, collaboration and mentoring in scholarship, were added on the basis of input from the pediatric experts that emphasized the critical need to incorporate these competencies for one to be productive in scholarly activities. 


Subspecialty Pediatrics Investigator Network의 운영위원회가 개발한 학술활동 EPA의 감독 규모에는 5가지 점진적 위탁 수준이 있다(표 2). 

The level of supervision scale for the scholarship EPA that the Steering Committee of the Subspecialty Pediatrics Investigator Network developed has five progressive levels of entrustment (Table 2). 

다음 단계
Next Steps

이 EPA의 다음 단계에는 이 척도의 타당성을 연구하고 이정표와 감독 수준 간의 상관관계를 결정하기 위한 연구가 포함된다. 우리는 최근에 [모든 소아과 펠로우십 프로그램 책임자]를 대상으로 [학술활동 EPA 및 해당 세부전공과 관련된 모든 EPA]에 대해서, entrustment for unsupervised practice에 해당하는 대상 감독 수준을 선택하도록 설문 조사를 실시했다. 
Next steps for this EPA will involve a study of this scale’s validity and to determine the correlation between milestone and supervision levels. We recently administered a survey to all pediatric fellowship program directors asking them to choose a targeted supervision level expected for graduation from fellowship and a targeted level that equates with entrustment for unsupervised practice for the scholarship EPA, as well as for all EPAs relevant to that subspecialty. 

학술활동 EPA를 포함한 모든 세부전공 EPA의 커리큘럼 구성요소가 완료되었으며, ABP 웹 사이트에서 프로그램 책임자와 학습자가 교육 중에 가이드로 사용할 수 있다.3 이 보고서의 EPA 학술활동의 기반을 다음에 두고 있다.

• (1) 일부 전공의와 동료에게 요구되는 학술적 기술에 대한 평가 전략의 부족  
• (2) 역량 및 이정표가 수련생에게 제공할 수 있는 학습 로드맵과 피드백의 기질substrate

Curricular components for all the subspecialty EPAs, including the scholarship EPA, have been completed and are available on the ABP website for program directors and learners to use as a guide during training.3 We focus on the scholarship EPA in this report on the basis of

• (1) the scarcity of assessment strategies for scholarly skills, which are required in some residencies and fellowships; and
• (2) the road map for learning and the substrate for feedback that competencies and milestones can provide to trainees. 

특히 중요한 것은 MD/PhD 학생들을 포함한 의사-과학자 평가에서 이러한 기구가 할 수 있는 잠재적인 역할로서, 이들은 의대에서 박사과정을 시작하고 GME 전반에 걸쳐 그리고 실무적으로 전문적인 연구 개발을 계속한다.

Especially important is the potential role that these instruments could play in the assessment of physician–scientists, including MD/PhD students, who begin their PhD work in medical school and continue their professional research development throughout GME and into practice.

---

Acad Med. 2018 Nov;93(11):1668-1672.

 doi: 10.1097/ACM.0000000000002353.

Competencies, Milestones, and a Level of Supervision Scale for Entrustable Professional Activities for Scholarship

Richard B Mink 1, Angela L Myers, David A Turner, Carol L Carraccio

Affiliations collapse

Affiliation

• 1R.B. Mink is professor of pediatrics, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, California, and director, Pediatric Critical Care Medicine Fellowship, Harbor-UCLA Medical Center, Torrance, California; ORCID: http://orcid.org/0000-0002-7998-4713. A.L. Myers is associate professor and director, Infectious Diseases Fellowship Program, Children's Mercy Hospital and University of Missouri-Kansas City School of Medicine, Kansas City, Missouri; ORCID: http://orcid.org/0000-0002-7686-0854. D.A. Turner is associate professor of pediatrics, Duke University School of Medicine, and associate director of graduate medical education, Duke University Medical Center, Durham, North Carolina. C.L. Carraccio is vice president of competency-based assessment, American Board of Pediatrics, Chapel Hill, North Carolina.

• PMID: 29995669
• DOI: 10.1097/ACM.0000000000002353Abstract
• Problem: Scholarship is an important element of both undergraduate and graduate medical education, and scholarly activity is required for all pediatric fellows. However, despite the creation of entrustable professional activities (EPAs) for scholarship, the specific progressive levels of performance and the appropriate level of supervision for a given performance level have not been defined. The authors developed competencies and milestones for the scholarship EPA to provide a framework for assessment across the continuum; a level of supervision scale was also developed.Outcomes: Eight competencies were identified as important in making entrustment decisions related to scholarship. For each competency, five milestone levels that span the continuum from novice to expert were created. A supervision scale with five progressive levels of entrustment was also created.
• Next steps: Next steps include a study to obtain validity evidence for the supervision scale and determine the correlation between milestone and supervision levels. These competencies, milestones, and supervision levels can potentially serve as a road map for trainees and junior faculty and also play a role in the assessment of physician-scientists.
• Approach: The Vitae Researcher Development Framework served as a template to create the competencies and milestones for the scholarship EPA. Beginning in September 2015 and using a modified Delphi approach, three drafts were circulated to individuals with expertise in various types of scholarship until broad agreement was achieved. Then, in October 2016, the Steering Committee of the Subspecialty Pediatrics Investigator Network created a level of supervision scale, modeled after one it had previously developed.