This is an update on an affiliated project. The project is a Delphi study implemented by Jasmine Rana, BA, a Harvard Medical School student, and K. Meredith Atkins, M.D., an obstetrician/gynecologist at Beth Israel Deaconess Medical Center, Boston, MA.
Progress Update: Student-as-Teacher (SaT) Delphi Study
Pictorial of Delphi methodology
The SaT Delphi study was inspired by our efforts at our institution to create a SaT program for senior medical students. We wanted to survey an expert panel to answer the question: what are core topics to include in a SaT curriculum for medical students? The Delphi method is a popular consensus methodology used for years in medicine to create clinical diagnostic and treatment guidelines and also curricula in UME and GME. It is useful when there is insufficient empirical evidence to guide an answer to a question and expert opinion is the guiding source of knowledge in a field.
The “Delphi” method is named after the Oracle at Delphi who was credited with predicting the future in ancient Greece. It is a nod to reason the Delphi method was created: in the 1950s, the RAND organization is widely credited for developing the Delphi to predict the impact of technology on warfare where theoretical and/or modelling approaches were insufficient. The original Delphi method has since been modified for use in many disciplines for predictive and non-predictive purposes. Although the term “modified” Delphi method is popular to distinguish it from the RAND Delphi method (though, the term “modified” more often than not carries no clearly understood designation), one of the limitations of the Delphi methodology is the lack of standardized definitions and rigorous criteria employed by researchers to define consensus. Nonetheless, the core elements of a Delphi method [1,2], as in our study, are:
A pictorial of the Delphi methodology used in our study is shown below. The expert panel was identified as members of Academies Collaborative. 28 members self-identified from this group (*includes 3 snow- ball participants nominated by Academy Directors at their institution) to participate and we are pleased to report 100% response rate for Rounds 1-3, held April – June 2016. Five topics have reached consensus as “essential” for inclusion in a SaT curriculum (i.e. small group-teaching, feedback, bedside teaching and precepting in clinical contexts, case-based teaching, and professionalism as a medical educator).
In the final round of the SaT study, planned for July 2016, our plan is to feedback these essential topics back to the expert panel to brainstorm a list of knowledge and skills that can guide appropriate learning objectives for medical students in a SaT curriculum. We look forward to sharing our complete results in the near future and we hope it is a useful resource for medical educators developing SaT curricula.
Special thank you to Academies Collaborative participants for their commitment to this study! We would also like to thank our co-investigators, Amy Sullivan, EdD, Amy Weinstein, MD, and Molly Brett, MD.
Jasmine Rana, BA, and K. Meredith Atkins, MD
Harvard Medical School, Boston, MA, USA
References:
1. Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research: Acad Med. January 2016:1. doi:10.1097/ACM.0000000000001092.
2. Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401-409. doi:10.1016/j.jclinepi.2013.12.002.
The “Delphi” method is named after the Oracle at Delphi who was credited with predicting the future in ancient Greece. It is a nod to reason the Delphi method was created: in the 1950s, the RAND organization is widely credited for developing the Delphi to predict the impact of technology on warfare where theoretical and/or modelling approaches were insufficient. The original Delphi method has since been modified for use in many disciplines for predictive and non-predictive purposes. Although the term “modified” Delphi method is popular to distinguish it from the RAND Delphi method (though, the term “modified” more often than not carries no clearly understood designation), one of the limitations of the Delphi methodology is the lack of standardized definitions and rigorous criteria employed by researchers to define consensus. Nonetheless, the core elements of a Delphi method [1,2], as in our study, are:
- Identification of an expert panel (generally n = 6-11, but may be larger). Notably, unlike in-person consensus processes, the Delphi is administered remotely (i.e. via mail-in or online surveys) AND anonymously (i.e. identity of panel members is kept anonymous until study conclusion).
- Iterative survey rounds. There are typically two to four sequential survey rounds in a Delphi study. Summary statistics (qualitative and quantitative) are provided to panel members after each round to help inform their answers to the next round with the goal being to reach group consensus (often defined using percent threshold and/or Likert scale cut-offs). Surveys in each round are modified based on comments/results in previous rounds.
A pictorial of the Delphi methodology used in our study is shown below. The expert panel was identified as members of Academies Collaborative. 28 members self-identified from this group (*includes 3 snow- ball participants nominated by Academy Directors at their institution) to participate and we are pleased to report 100% response rate for Rounds 1-3, held April – June 2016. Five topics have reached consensus as “essential” for inclusion in a SaT curriculum (i.e. small group-teaching, feedback, bedside teaching and precepting in clinical contexts, case-based teaching, and professionalism as a medical educator).
In the final round of the SaT study, planned for July 2016, our plan is to feedback these essential topics back to the expert panel to brainstorm a list of knowledge and skills that can guide appropriate learning objectives for medical students in a SaT curriculum. We look forward to sharing our complete results in the near future and we hope it is a useful resource for medical educators developing SaT curricula.
Special thank you to Academies Collaborative participants for their commitment to this study! We would also like to thank our co-investigators, Amy Sullivan, EdD, Amy Weinstein, MD, and Molly Brett, MD.
Jasmine Rana, BA, and K. Meredith Atkins, MD
Harvard Medical School, Boston, MA, USA
References:
1. Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research: Acad Med. January 2016:1. doi:10.1097/ACM.0000000000001092.
2. Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401-409. doi:10.1016/j.jclinepi.2013.12.002.
Jasmine Rana, BA
Authors:
Jasmine Rana, BA
K. Meredith Atkins, MD
Jasmine Rana, BA
K. Meredith Atkins, MD
