Abstract
Physicians function as clinicians, teachers, and role models within the clinical environment. Negative learning environments have been shown to be due to many factors, including the presence of unprofessional behaviors among clinical teachers. Reliable and valid assessments of clinical teacher performance, including professional behaviors, may provide a foundation for evidence-based feedback to clinical teachers, enable targeted remediation or recognition, and help to improve the learning environment. However, few tools exist for the evaluation of clinical teachers that include a focus on both professional and clinical teaching behaviors. The Professionalism Assessment of Clinical Teachers (PACT) was developed and implemented at one Canadian institution and was assessed for evidence of reliability and validity. Following each clerkship rotation, students in the 2009–2010 third-year undergraduate clerkship cohort (n = 178) anonymously evaluated a minimum of two clinical teachers using the PACT. 4,715 forms on 567 faculty members were completed. Reliability, validity, and free text comments (present in 45 % of the forms) were examined. An average of 8.6 PACT forms were completed per faculty (range 1–60), with a reliability of 0.31 for 2.9 forms (harmonic mean); 12 forms were necessary for a reliability of 0.65. Global evaluations of teachers aligned with ratings of free-text comments (r = 0.77, p < 0.001). Comment length related negatively with overall rating (r = −0.19, p < 0.001). Mean performance related negatively with variability of performance (r = −0.72, p < 0.001), although this may be related to a ceiling effect. Most faculty members were rated highly; however ‘provided constructive feedback’ was the least well-rated item. Respectful interactions with students appeared to be the most influential item in the global rating of faculty performance. The PACT is a moderately reliable tool for the assessment of professional behaviors of clinical teachers, with evidence supporting its validity.
Similar content being viewed by others
References
Blackall, G. F., Melnick, S. A., Shoop, G. H., George, J., Lerner, S. M., Wilson, P. K., et al. (2007). Professionalism in medical education: The development and validation of a survey instrument to assess attitudes toward professionalism. Medical Teacher, 29, E58–E62.
Boudreau, J. D., Cassell, E. J., & Fuks, A. (2007). A healing curriculum. Medical Education, 41, 1193–1201.
Brainard, A. H., & Bilsen, H. C. (2007). Learning professionalism: A view from the trenches. Academic Medicine, 82, 1010–1014.
Brennan, R. L. (2001). Manual for urGENOVA (version 2.1). Iowa City, IA: The University of Iowa.
Campbell, J., Richards, S. H., Dickens, A., Greco, M., Narayanan, A., & Brearley, S. (2008). Assessing the professional performance of UK doctors: An evaluation of the utility of the General Medical Council patient and colleague questionnaires. Quality and Safety in Health Care, 17, 187–193.
Cohen, J. J. (2006). Professionalism in medical education, an American perspective: From evidence to accountability. Medical Education, 40, 607–617.
Cruess, R. L., & Cruess, S. R. (2009). Principles for designing a program for the teaching and learning of professionalism at the undergraduate level. In R. L. Cruess, S. R. Cruess, & Y. Steinert (Eds.), Teaching medical professionalism (pp. 73–93). Cambridge: Cambridge University Press.
Cruess, R. L., Herold-McIlroy, J., Cruess, S. R., Ginsberg, S., & Steinert, Y. (2006). The P-MEX (professionalism mini evaluation exercise): A preliminary investigation. Academic Medicine, 81, S74–S79.
Evans, S. M., Berry, J. G., Smith, B. J., Esterman, A., Selim, P., O’Shaughnessy, J., et al. (2006). Attitudes and barriers to incident reporting: A collaborative hospital study. Quality & Safety in Health Care, 15(1), 39–43.
Feudtner, C., Christakis, D. A., & Christakis, N. A. (1994). Do clinical clerks suffer ethical erosion? Student perceptions of their ethical environment and personal development. Academic Medicine, 69, 670–679.
Fluit, C. R. M., Bolhuis, S., Grol, R., Loan, R., & Wensing, M. (2010). Assessing the quality of clinical teachers: A systematic review of content and quality of questionnaires for assessing clinical teachers. Journal of General Internal Medicine, 25, 1337–1345.
Garra, G., Wackett, A., & Thode, H. (2011). Feasibility and reliability of a multisource feedback tool for emergency medicine residents. Journal of Graduate Medical Education, 3, 356–360.
Ginsburg, S., Regehr, G., Hatala, R., McNaughton, N., Frohna, A., Hodges, B., et al. (2000). Context, conflict, and resolutions: A new conceptual framework for evaluating professionalism. Academic Medicine, 75, 96–511.
Ginsburg, S., Regehr, G., & Lingard, L. (2004). Basing the evaluation of professionalism on observable behaviors: A precautionary tale. Academic Medicine, 79, S1–S4.
Hickson, G. B., Pichert, J. W., Webb, L. E., & Gabbe, S. G. (2007). A complementary approach to promoting professionalism: Identifying, measuring, and addressing unprofessional behaviors. Academic Medicine, 82, 1040–1048.
Hodges, B. D., Ginsburg, S., Cruess, R., Cruess, S., Delport, R., Hafferty, F., et al. (2011). Assessment of professionalism: Recommendations from the Ottawa 2010 conference. Medical Teacher, 33, 354–363.
Lockyer, J. (2003). Multisource feedback in the assessment of physician competencies. Journal of Continued Education in Health Professions, 23(1), 4–12.
Lockyer, J. M., Violato, C., & Fidler, H. (2006). A multisource feedback program for anesthesiologists. Canadian Journal of Anaesthesia, 53(1), 33–39.
Petrusa, E. R. (2002). Clinical performance assessments. In G. R. Norman, C. P. M. van der Vleuten, & D. I. Newble (Eds.), International handbook of research in medical education (pp. 673–709). Dordrecht: Kluwer.
Prislin, M. D., Lie, D., Shapiro, J., Boker, J., & Radecki, S. (2001). Using standardized patients to assess medical students’ professionalism. Academic Medicine, 76(10 suppl), 90–92.
Quaintance, J. L., Arnold, L., & Thompson, G. S. (2008). Development of an instrument to measure the climate of professionalism in a clinical teaching environment. Academic Medicine, 83, S5–S8.
Razack, S., Faremo, S., Drolet, F., Snell, L., Wiseman, J., & Pickering, J. (2009). Multiple mini-interviews versus traditional interviews: Stakeholder acceptability comparison. Medical Education, 43, 993–1000.
Shannon, H. S., & Norman, G. R. (2009). Deriving the factor structure of safety climate scales. Safety Science, 27, 327–329.
Snell, L. (2009). Teaching professionalism and fostering professional values during residency: The McGill experience. In R. L. Cruess, S. R. Cruess, & Y. Steinert (Eds.), Teaching medical professionalism (pp. 246–263). Cambridge: Cambridge University Press.
Srinivasan, M., Li, S. T., Meyers, F. J., Pratt, D., Collins, J. B., Braddock, C., et al. (2011). “Teaching as a competency”: Competencies for medical educators. Academic Medicine, 86, 1211–1220.
Stern, D. T. (Ed.). (2005). Measuring medical professionalism. New York: Oxford University Press.
Streiner, D., & Norman, G. (2008). Health measurement scales: A practical guide to their development and use. USA: Oxford University Press.
Sutkin, G., Littlefield, J. H., & Laube, D. W. (2009). Nursing staff assessment of residents’ professionalism and communication skills. Medical Education, 43(11), 1104.
Sutkin, G., Wagner, E., Harris, I., & Schiffer, R. (2008). What makes a good clinical teacher in medicine? A review of the literature. Academic Medicine, 83, 452–466.
Todhunter, S., Cruess, S. R., Cruess, R. L., Young, M., & Steinert, Y. (2011). Developing and piloting a form for student assessment of faculty professionalism. Advances in Health Sciences Education, 16, 223–238.
Tromp, F., Vernooij-Dassen, M., Kramer, A., Grol, R., & Bottema, K. (2010). Behavioral elements of professionalism: Assessment of a fundamental concept in medical care. Medical Teacher, 32, E161–E169.
Wilkinson, T. J., Wade, W. B., & Knock, L. D. (2009). A blueprint to assess professionalism: Results of a systematic review. Academic Medicine, 84(5), 551–558.
Wood, T. (2009). Assessment not only drives learning, it may also help learning. Medical Education, 43, 5–6.
Wood, J., Collins, J., Burnside, E. S., Albanese, M. A., Propeck, P. A., Kelcz, F., et al. (2004). Patient, faculty and self-assessment of radiology resident performance: A 360-degree method of measuring professionalism and interpersonal/communication skills. Academic Radiology, 11, 931–939.
Wright, S. M., Kern, D. E., Kolodner, K., Howard, D. M., & Brancati, F. L. (1998). Attributes of excellent attending-physician role models. New England Journal of Medicine, 339, 1986–1993.
Acknowledgments
The authors would like to thank Dr. Joyce Pickering, previous Associate Dean of Undergraduate Medical Education at McGill University for her assistance in integrating the PACT into the assessment and evaluation program at McGill. We would also like to thank Samantha Eidinger and Katharine Fischer for their work in anonymizing and coordinating the databases used in this study. Finally, we would like to thank the Office of the Associate Dean, Undergraduate Medical Education at McGill University for the financial support needed to implement the PACT and for continued support of this project.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Young, M.E., Cruess, S.R., Cruess, R.L. et al. The Professionalism Assessment of Clinical Teachers (PACT): the reliability and validity of a novel tool to evaluate professional and clinical teaching behaviors. Adv in Health Sci Educ 19, 99–113 (2014). https://doi.org/10.1007/s10459-013-9466-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-013-9466-4