Medical Science Educator

, Volume 26, Issue 3, pp 297–305 | Cite as

Measuring Faculty Skill and Effectiveness in Conducting Interactive Learning Sessions—a Proposed Instrument

  • Patrick John GannonEmail author
  • David Elkowitz
  • Alice Fornari
  • Gina Sorrentino
  • Joel N. H. Stern
  • Nancy Kheck
  • Terri Shapiro
  • Naomi L. Lacy
  • David L. Steele
  • Latha Chandran
Original Research


Medical educators have created approaches such as the “flipped classroom” to cultivate robust knowledge transfer with high-order thinking skills and retention. Hybrid approaches, with independent self-directed learning and large group interactive “application” activities, may also better foster learning. No standard instrument exists to rate the effectiveness of interactive teaching in large groups. This study developed an objective criterion-based instrument to measure the effectiveness of large group teaching by educators who engage with an audience that is pre-prepared by self-directed learning. We call such an educator a “Conductor of Interactive Learning” (COIL) and the interactive skillset, COIL skills. Based on needs assessment surveys of local and national end users, we designed a measurable standard-based instrument to evaluate effectiveness of a COIL. The instrument was designed to delineate measurable objective standards that characterize COIL skills and serve as a peer/self-assessment tool to identify individual COIL skills. After beta testing across institutional medical educators (n = 28), improved versions were presented to national experts (n = 12) and potential medical educator end users (three conference workshops) for constructive feedback on the quality and usefulness of the standards and the rating instrument. We developed standards that captured both interactive skills (n = 7) and non-interactive educational fundamentals (n = 9) for large group teaching. Using an iterative consensus building national process, we finalized a hybrid rating instrument consisting of these standards with anchors adjacent to merged 4-point Likert and visual analog scales. The standards and rating instrument for COIL skills could serve as an individualized guide for educator development.


Self-directed learning Interactive learning Higher-order learning 



Thanks to the consultants at the Hofstra North Shore-LIJ School of Medicine for the support and advice in the development of the COIL program: David Battinelli MD; Samara Ginzburg MD; and Lawrence Smith MD.

The following medical education research experts acted as consultants in the formulation of the COIL program for which we thank them very much: Elizabeth G Armstrong PhD, Director of Harvard Macy Institute; Lisa Coplit MD, Associate Dean for Assessment & Faculty, Frank H. Netter MD School of Medicine at Quinnipiac University; Susan Farrell MD EdM, Faculty of Harvard Macy Institute, Professor Harvard Medical School; Robert Milner PhD, University of Massachusetts Medical School; Lori R Newman EdM, Harvard Medical School; Klara K Papp PhD, Director of Assessment and Evaluations, Office of Curricular Affairs, Case Western Reserve University School of Medicine; David Roberts MD, Harvard Medical School; Luanne E Thorndyke MD, Vice Provost of Faculty Affairs, University Massachusetts Med School. A particular thank you to Maryellen E Gusic MD who served as a major academic contributor to the COIL program since its conception at the 2012 Harvard Macy Institute.


  1. 1.
    Prober CG, Heath C. Lecture halls without lectures—a proposal for medical education. N Engl J Med. 2012;366:1657–9.CrossRefGoogle Scholar
  2. 2.
    Prober CG, Khan S. Medical education reimagined: a call to action. Acad Med. 2013;88:1407–10.CrossRefGoogle Scholar
  3. 3.
    McLaughlin JE, Roth MT, Glatt DM, Gharkholonarehe N, Davidson CA, Griffin LM, Esserman DA, Mumper RJ. The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Acad Med. 2014;89:236–43.CrossRefGoogle Scholar
  4. 4.
    Chandran L, Gusic M, Baldwin C, Turner T, Zenni E, Lane JL, Balmer D, Bar-On M, Rauch DA, Indyk D, Gruppen LD. Evaluating the performance of medical educators: a novel analysis tool to demonstrate the quality and impact of educational activities. Acad Med. 2009;84:58–66.CrossRefGoogle Scholar
  5. 5.
    Srinivasan M, Li ST, Meyers FJ, Pratt DD, Collins JB, Braddock C, Skeff KM, West DC, Henderson M, Hales RE, Hilty DM. “Teaching as a competency”: competencies for medical educators. Acad Med. 2011;86:1211–20.CrossRefGoogle Scholar
  6. 6.
    Milner RJ, Gusic ME, Thorndyke LE. Perspective: toward a competency framework for faculty. Acad Med. 2011;86:1204–10.CrossRefGoogle Scholar
  7. 7.
    Berman AC. Good teaching is good teaching: a narrative review for effective medical educators. Anat Sci Educ. 2015;8:386–94.CrossRefGoogle Scholar
  8. 8.
    Newman LR, Lown BA, Jones RN, Johansson A, Schwartzstein RM. Developing a peer assessment of lecturing instrument: lessons learned. Acad Med. 2009;84:1104–10.CrossRefGoogle Scholar
  9. 9.
    Newman LR, Brodsky DD, Roberts DH, Pelletier SR, Johansson A, Vollmer Jr CM, Atkins KM, Schwartzstein RM. Developing expert-derived rating standards for the peer assessment of lectures. Acad Med. 2012;87:356–6.Google Scholar
  10. 10.
    Kost A, Chen F. Socrates was not a pimp: changing the paradigm of questioning in medical education. Acad Med. 2015;90:20–4.CrossRefGoogle Scholar
  11. 11.
    Artino Jr AR, Durning SJ, Creel AH. AM last page: reliability and validity in educational measurement. Acad Med. 2010;85:1545.CrossRefGoogle Scholar
  12. 12.
    Lee H, Paek I. In search of the optimal number of response categories in a rating scale. J Psychoeduc Assess. 2014;32:663–73.CrossRefGoogle Scholar
  13. 13.
    Englbrecht M1, Tarner IH, van der Heijde DM, Manger B, Bombardier C, Müller-Ladner U. Measuring pain and efficacy of pain treatment in inflammatory arthritis: a systematic literature review. J Rheumatol Suppl. 2012; 90:3–10.Google Scholar
  14. 14.
    Dreyfus SE, Dreyfus HL. A five stage model of the mental activities involved in direct skill acquisition. Washington, DC: Storming Media; 1990.Google Scholar
  15. 15.
    Kogan JR, Conforti LN, Iobst WF, Holmboe ES. Reconceptualizing variable rater assessments as both an educational and clinical care problem. Acad Med. 2014;89:721–7.CrossRefGoogle Scholar
  16. 16.
    Bloom BS [ed.], Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of educational objectives: the classification of educational goals. Handbook 1: cognitive Domain. New York, David McKay. 1956.Google Scholar
  17. 17.
    Anderson, L. W. and Krathwohl, D. R., et al (Eds.). A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives. Allyn & Bacon. Boston, MA (Pearson Education Group). 2001.Google Scholar

Copyright information

© International Association of Medical Science Educators 2016

Authors and Affiliations

  • Patrick John Gannon
    • 1
    • 2
    • 3
    Email author
  • David Elkowitz
    • 3
    • 4
    • 5
  • Alice Fornari
    • 3
    • 6
  • Gina Sorrentino
    • 3
  • Joel N. H. Stern
    • 3
    • 7
    • 8
  • Nancy Kheck
    • 2
  • Terri Shapiro
    • 9
  • Naomi L. Lacy
    • 10
  • David L. Steele
    • 10
  • Latha Chandran
    • 11
  1. 1.International American UniversityVieux FortSaint Lucia
  2. 2.International American University College of MedicineVieux FortSaint Lucia
  3. 3.Department of Science EducationHofstra Northwell School of MedicineHempsteadUSA
  4. 4.Academy of Medical EducatorsHofstra Northwell School of MedicineHempsteadUSA
  5. 5.Department of Pathology and Laboratory MedicineHofstra Northwell School of MedicineHempsteadUSA
  6. 6.Educational Skills Development DivisionHofstra Northwell School of MedicineHempsteadUSA
  7. 7.Department of NeurologyHofstra Northwell School of MedicineHempsteadUSA
  8. 8.Department of NeurologyLenox Hill Hospital, Northwell HealthNew YorkUSA
  9. 9.Provosts Office, Hofstra UniversityHempsteadUSA
  10. 10.Paul L Foster School of MedicineTexas Tech University Health Sciences CenterEl PasoUSA
  11. 11.Stony Brook University School of MedicineStony BrookUSA

Personalised recommendations