Abstract
Longitudinal academic advising (AA) and coaching programs are increasingly implemented in competency based medical education (CBME) to help residents reflect and act on the voluminous assessment data they receive. Documents created by residents for purposes of reflection are often used for a second, summative purpose—to help competence committees make decisions—which may be problematic. Using inductive, thematic analysis we analyzed written comments generated by 21 resident-AA dyads in one large internal medicine program who met over a 2 year period to determine what residents write when asked to reflect, how this aligns with what the AAs report, and what changes occur over time (total 109 resident self-reflections and 105 AA reports). Residents commented more on their developing autonomy, progress and improvement than AAs, who commented far more on performance measures. Over time, residents’ writing shifted away from intrinsic roles, patient care and improvement towards what AAs focused on, including getting EPAs (entrustable professional activities), studying and exams. For EPAs, the emphasis was on getting sufficient numbers rather than reflecting on what residents were learning. Our findings challenge the practice of dual-purposing documents, by questioning the blurring of formative and summative intent, the structure of forms and their multiple conflicting purposes, and assumptions about the advising relationship over time. Our study suggests a need to re-evaluate how reflective documents are used in CBME programs. Further research should explore whether and how documentation can best be used to support resident growth and development.
Similar content being viewed by others
References
Atkinson, A., Watling, C. J., & Brand, P. L. P. (2021). Feedback and coaching. European Journal of Pediatrics. https://doi.org/10.1007/s00431-021-04118-8
Baenziger, K., Chan, M., & Colman, S. (2023). Coaching in postgraduate competency-based medical education: A qualitative exploration of three models. Academic Psychiatry, 47(1), 10–17. https://doi.org/10.1007/s40596-022-01628-x
Balmer, D. F., Varpio, L., Bennett, D., & Teunissen, P. W. (2021). Longitudinal qualitative research in medical education: Time to conceptualise time. Medical Education, 55(11), 1253–1260. https://doi.org/10.1111/medu.14542
Bates, J., Konkin, J., Suddards, C., Dobson, S., & Pratt, D. (2013). Student perceptions of assessment and feedback in longitudinal integrated clerkships. Medical Education, 47(4), 362–374. https://doi.org/10.1111/medu.12087
Brand, P. L. P., Jaarsma, A. D. C., & van der Vleuten, C. P. M. (2021). Driving lesson or driving test? Perspectives on Medical Education, 10(1), 50–56. https://doi.org/10.1007/s40037-020-00617-w
Branfield Day, L., Butler, D., Shah, R., Kuper, A., Tavares, W., Ginsburg, S., Stroud, L., & Brydges, R. (2022). Co-Constructors of Meaning? On How Residents and Academic Advisors (Co-) Regulate Learning Over Time.ICRE Top Research Abstracts Journal of Graduate Medical Education,
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Cuncic, C., Regehr, G., Frost, H., & Bates, J. (2018). It’s all about relationships. Perspectives on Medical Education, 7(2), 100–109. https://doi.org/10.1007/s40037-018-0416-y
de la Croix, A., & Veen, M. (2018). The reflective zombie: Problematizing the conceptual framework of reflection in medical education [journal article]. Perspectives on Medical Education. https://doi.org/10.1007/s40037-018-0479-9
Deiorio, N. M., Foster, K. W., & Santen, S. A. (2021). Coaching a learner in medical education. Academic Medicine, 96(12), 1758. https://doi.org/10.1097/acm.0000000000004168
Eva, K. W., & Hodges, B. D. (2012). Scylla or Charybdis? Can we navigate between objectification and judgement in assessment? Medical Education, 46(9), 914–919. https://doi.org/10.1111/j.1365-2923.2012.04310.x
Ginsburg, S., Watling, C. J., Schumacher, D. J., Gingerich, A., & Hatala, R. (2021). Numbers encapsulate, words elaborate: Toward the best use of comments for assessment and feedback on entrustment ratings. Academic Medicine, 96(7S), S81–S86. https://doi.org/10.1097/ACM.0000000000004089
Gonzalo, J. D., Wolpaw, D. R., Krok, K. L., Pfeiffer, M. P., & McCall-Hosenfeld, J. S. (2019). A developmental approach to internal medicine residency education: Lessons learned from the design and implementation of a novel longitudinal coaching program. Medical Education Online, 24(1), 1591256. https://doi.org/10.1080/10872981.2019.1591256
Hatala, R., Ginsburg, S., Gauthier, S., Melvin, L., Taylor, D., & Gingerich, A. (2022). Supervising the senior medical resident: Entrusting the role, supporting the tasks. Medical Education, 56(12), 1194–1202. https://doi.org/10.1111/medu.14883
Henderson, S., & Segal, E. H. (2013). Visualizing qualitative data in evaluation research. New Directions for Evaluation, 2013(139), 53–71. https://doi.org/10.1002/ev.20067
Huffman, B. M., Hafferty, F. W., Bhagra, A., Leasure, E. L., Santivasi, W. L., & Sawatsky, A. P. (2021). Resident impression management within feedback conversations: A qualitative study. Medical Education, 55(2), 266–274. https://doi.org/10.1111/medu.14360
Kiger, M. E., & Varpio, L. (2020). Thematic analysis of qualitative data: AMEE Guide No. 131. Medical Teacher, 42(8), 846–854. https://doi.org/10.1080/0142159X.2020.1755030
Landreville, J., Cheung, W., Frank, J., & Richardson, D. (2019). A definition for coaching in medical education. Can Med Educ J, 10(4), e109–e110.
Lovell, B. (2018). What do we know about coaching in medical education? A Literature Review. Med Educ, 52(4), 376–390. https://doi.org/10.1111/medu.13482
MacNeil, K., Cuncic, C., Voyer, S., Butler, D., & Hatala, R. (2020). Necessary but not sufficient: Identifying conditions for effective feedback during internal medicine residents’ clinical education. Advances in Health Sciences Education, 25(3), 641–654. https://doi.org/10.1007/s10459-019-09948-8
Miles, A., Ginsburg, S., Sibbald, M., Tavares, W., Watling, C., & Stroud, L. (2021). Feedback from health professionals in postgraduate medical education: Influence of interprofessional relationship, identity and power. Medical Education, 55(4), 518–529. https://doi.org/10.1111/medu.14426
Ng, S. L., Baker, L., Cristancho, S., Kennedy, T. J., & Lingard, L. (2018). Qualitative research in medical education: methodologies and methods. In T. Swanwick, K. Forrest, & B. O'Brien (Eds.), Understanding medical education: Evidence, theory, and practice (3rd ed., pp. 427–441). John Wiley & Sons.
Ng, S. L., Kinsella, E. A., Friesen, F., & Hodges, B. (2015). Reclaiming a theoretical orientation to reflection in medical education research: A critical narrative review. Medical Education, 49(5), 461–475. https://doi.org/10.1111/medu.12680
Nothnagle, M., Goldman, R., Quirk, M., & Reis, S. (2010). Promoting self-directed learning skills in residency: A case study in program development. Academic Medicine, 85(12), 1874–1879. https://doi.org/10.1097/ACM.0b013e3181fa02a4
Parsons, A. S., Kon, R. H., Plews-Ogan, M., & Gusic, M. E. (2021). You can have both: Coaching to promote clinical competency and professional identity formation. Perspectives on Medical Education, 10(1), 57–63. https://doi.org/10.1007/s40037-020-00612-1
Ramani, S., Könings, K. D., Ginsburg, S., & van der Vleuten, C. P. M. (2020). Relationships as the backbone of feedback: exploring preceptor and resident perceptions of their behaviors during feedback conversations. Academic Medicine, 95(7), 1073–1081. https://doi.org/10.1097/acm.0000000000002971
Ramani, S., Könings, K. D., Mann, K. V., Pisarski, E. E., & van der Vleuten, C. P. M. (2018). About politeness, face, and feedback: Exploring resident and faculty perceptions of how institutional feedback culture influences feedback practices. Academic Medicine, 93(9), 1348–1358. https://doi.org/10.1097/acm.0000000000002193
Rich, J. V., Luhanga, U., Fostaty Young, S., Wagner, N., Dagnone, J. D., Chamberlain, S., & McEwen, L. A. (2022). Operationalizing programmatic assessment: The CBME programmatic assessment practice guidelines. Academic Medicine, 97(5), 674–678. https://doi.org/10.1097/acm.0000000000004574
Royal College :: Competence by Design (CBD). In.
Sandelowski, M. (2001). Real qualitative researchers do not count: The use of numbers in qualitative research. Research in Nursing & Health, 24(3), 230–240.
Sargeant, J., Lockyer, J., Mann, K., Holmboe, E., Silver, I., Armson, H., Driessen, E., MacLeod, T., Yen, W., Ross, K., & Power, M. (2015). Facilitated reflective performance feedback: Developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Academic Medicine, 90(12), 1698–1706. https://doi.org/10.1097/ACM.0000000000000809
Schut, S., Driessen, E., Tartwijk, J., Vleuten, C., & Heeneman, S. (2018). Stakes in the eye of the beholder: An international study of learners’ perceptions within programmatic assessment. Medical Education, 52(6), 654–663. https://doi.org/10.1111/medu.13532
Schut, S., Heeneman, S., Bierer, B., Driessen, E., van Tartwijk, J., & van der Vleuten, C. (2020). Between trust and control: Teachers’ assessment conceptualisations within programmatic assessment. Medical Education, 54(6), 528–537. https://doi.org/10.1111/medu.14075
Schut, S., Maggio, L. A., Heeneman, S., van Tartwijk, J., van der Vleuten, C., & Driessen, E. (2021). Where the rubber meets the road—An integrative review of programmatic assessment in health care professions education. Perspect Med Educ, 10(1), 6–13. https://doi.org/10.1007/s40037-020-00625-w
Sepulveda, A. (2021, 2021/03/31). Coaching is Not Academic Advising. Evolllution: A Modern Campus Illumination. https://evolllution.com/programming/personal-development/coaching-is-not-academic-advising/
Soleas, E., Dagnone, D., Stockley, D., Garton, K., & van Wylick, R. (2020). Developing Academic Advisors and Competence Committees members: A community approach to developing CBME faculty leaders. Can Med Educ J, 11(1), e46-e56. https://doi.org/10.36834/cmej.68181
Stroud, L., Feilchenfeld, Z., Miles, A., Piquette, D., Watling, C., Brydges, R., & Ginsburg, S. (2022, May). Value added but not intended: A 2-year longitudinal study of the evolution of academic advisor and resident relationships. [Canadian Conference on Medical Education 2022 abstracts]. Can Med Educ J,
Tavares, W., Young, M., Gauthier, G., & St-Onge, C. (2020). The Effect of Foregrounding Intended Use on Observers’ Ratings and Comments in the Assessment of Clinical Competence. Academic Medicine, 95(5), 777–785. https://www.ingentaconnect.com/content/wk/acm/2020/00000095/00000005/art00034;jsessionid=rhaafbimbc3c.x-ic-live-02
van der Vleuten, C. P. M., Schuwirth, L. W., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34(3), 205–214. https://doi.org/10.3109/0142159X.2012.652239
Watling, C. J., & Ginsburg, S. (2019). Assessment, feedback and the alchemy of learning. Medical Education, 53(1), 76–85. https://doi.org/10.1111/medu.13645
Wolcott, M. D., McLaughlin, J. E., Hann, A., Miklavec, A., Beck Dallaghan, G. L., Rhoney, D. H., & Zomorodi, M. (2021). A review to characterise and map the growth mindset theory in health professions education. Medical Education, 55(4), 430–440. https://doi.org/10.1111/medu.14381
Acknowledgements
This study was funded by a Royal College of Physicians and Surgeons of Canada, Medical Education Research Grant. The authors gratefully acknowledge earlier contributions of Dr. James Rassos and Dr. Ryan Luther who were both residents at the time the study started.
Author information
Authors and Affiliations
Contributions
ALL authors conceived of the study, conducted the study including collecting and analyzing data, and helped write and critically revise the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ginsburg, S., Stroud, L., Brydges, R. et al. Dual purposes by design: exploring alignment between residents’ and academic advisors’ documents in a longitudinal program. Adv in Health Sci Educ (2024). https://doi.org/10.1007/s10459-024-10318-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10459-024-10318-2