Skip to main content

Advertisement

Log in

The Professionalism Assessment of Clinical Teachers (PACT): the reliability and validity of a novel tool to evaluate professional and clinical teaching behaviors

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Physicians function as clinicians, teachers, and role models within the clinical environment. Negative learning environments have been shown to be due to many factors, including the presence of unprofessional behaviors among clinical teachers. Reliable and valid assessments of clinical teacher performance, including professional behaviors, may provide a foundation for evidence-based feedback to clinical teachers, enable targeted remediation or recognition, and help to improve the learning environment. However, few tools exist for the evaluation of clinical teachers that include a focus on both professional and clinical teaching behaviors. The Professionalism Assessment of Clinical Teachers (PACT) was developed and implemented at one Canadian institution and was assessed for evidence of reliability and validity. Following each clerkship rotation, students in the 2009–2010 third-year undergraduate clerkship cohort (n = 178) anonymously evaluated a minimum of two clinical teachers using the PACT. 4,715 forms on 567 faculty members were completed. Reliability, validity, and free text comments (present in 45 % of the forms) were examined. An average of 8.6 PACT forms were completed per faculty (range 1–60), with a reliability of 0.31 for 2.9 forms (harmonic mean); 12 forms were necessary for a reliability of 0.65. Global evaluations of teachers aligned with ratings of free-text comments (r = 0.77, p < 0.001). Comment length related negatively with overall rating (r = −0.19, p < 0.001). Mean performance related negatively with variability of performance (r = −0.72, p < 0.001), although this may be related to a ceiling effect. Most faculty members were rated highly; however ‘provided constructive feedback’ was the least well-rated item. Respectful interactions with students appeared to be the most influential item in the global rating of faculty performance. The PACT is a moderately reliable tool for the assessment of professional behaviors of clinical teachers, with evidence supporting its validity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Blackall, G. F., Melnick, S. A., Shoop, G. H., George, J., Lerner, S. M., Wilson, P. K., et al. (2007). Professionalism in medical education: The development and validation of a survey instrument to assess attitudes toward professionalism. Medical Teacher, 29, E58–E62.

    Article  Google Scholar 

  • Boudreau, J. D., Cassell, E. J., & Fuks, A. (2007). A healing curriculum. Medical Education, 41, 1193–1201.

    Google Scholar 

  • Brainard, A. H., & Bilsen, H. C. (2007). Learning professionalism: A view from the trenches. Academic Medicine, 82, 1010–1014.

    Article  Google Scholar 

  • Brennan, R. L. (2001). Manual for urGENOVA (version 2.1). Iowa City, IA: The University of Iowa.

    Google Scholar 

  • Campbell, J., Richards, S. H., Dickens, A., Greco, M., Narayanan, A., & Brearley, S. (2008). Assessing the professional performance of UK doctors: An evaluation of the utility of the General Medical Council patient and colleague questionnaires. Quality and Safety in Health Care, 17, 187–193.

    Article  Google Scholar 

  • Cohen, J. J. (2006). Professionalism in medical education, an American perspective: From evidence to accountability. Medical Education, 40, 607–617.

    Article  Google Scholar 

  • Cruess, R. L., & Cruess, S. R. (2009). Principles for designing a program for the teaching and learning of professionalism at the undergraduate level. In R. L. Cruess, S. R. Cruess, & Y. Steinert (Eds.), Teaching medical professionalism (pp. 73–93). Cambridge: Cambridge University Press.

    Google Scholar 

  • Cruess, R. L., Herold-McIlroy, J., Cruess, S. R., Ginsberg, S., & Steinert, Y. (2006). The P-MEX (professionalism mini evaluation exercise): A preliminary investigation. Academic Medicine, 81, S74–S79.

    Article  Google Scholar 

  • Evans, S. M., Berry, J. G., Smith, B. J., Esterman, A., Selim, P., O’Shaughnessy, J., et al. (2006). Attitudes and barriers to incident reporting: A collaborative hospital study. Quality & Safety in Health Care, 15(1), 39–43.

    Article  Google Scholar 

  • Feudtner, C., Christakis, D. A., & Christakis, N. A. (1994). Do clinical clerks suffer ethical erosion? Student perceptions of their ethical environment and personal development. Academic Medicine, 69, 670–679.

    Article  Google Scholar 

  • Fluit, C. R. M., Bolhuis, S., Grol, R., Loan, R., & Wensing, M. (2010). Assessing the quality of clinical teachers: A systematic review of content and quality of questionnaires for assessing clinical teachers. Journal of General Internal Medicine, 25, 1337–1345.

    Article  Google Scholar 

  • Garra, G., Wackett, A., & Thode, H. (2011). Feasibility and reliability of a multisource feedback tool for emergency medicine residents. Journal of Graduate Medical Education, 3, 356–360.

    Article  Google Scholar 

  • Ginsburg, S., Regehr, G., Hatala, R., McNaughton, N., Frohna, A., Hodges, B., et al. (2000). Context, conflict, and resolutions: A new conceptual framework for evaluating professionalism. Academic Medicine, 75, 96–511.

    Article  Google Scholar 

  • Ginsburg, S., Regehr, G., & Lingard, L. (2004). Basing the evaluation of professionalism on observable behaviors: A precautionary tale. Academic Medicine, 79, S1–S4.

    Article  Google Scholar 

  • Hickson, G. B., Pichert, J. W., Webb, L. E., & Gabbe, S. G. (2007). A complementary approach to promoting professionalism: Identifying, measuring, and addressing unprofessional behaviors. Academic Medicine, 82, 1040–1048.

    Article  Google Scholar 

  • Hodges, B. D., Ginsburg, S., Cruess, R., Cruess, S., Delport, R., Hafferty, F., et al. (2011). Assessment of professionalism: Recommendations from the Ottawa 2010 conference. Medical Teacher, 33, 354–363.

    Article  Google Scholar 

  • Lockyer, J. (2003). Multisource feedback in the assessment of physician competencies. Journal of Continued Education in Health Professions, 23(1), 4–12.

    Article  Google Scholar 

  • Lockyer, J. M., Violato, C., & Fidler, H. (2006). A multisource feedback program for anesthesiologists. Canadian Journal of Anaesthesia, 53(1), 33–39.

    Article  Google Scholar 

  • Petrusa, E. R. (2002). Clinical performance assessments. In G. R. Norman, C. P. M. van der Vleuten, & D. I. Newble (Eds.), International handbook of research in medical education (pp. 673–709). Dordrecht: Kluwer.

    Chapter  Google Scholar 

  • Prislin, M. D., Lie, D., Shapiro, J., Boker, J., & Radecki, S. (2001). Using standardized patients to assess medical students’ professionalism. Academic Medicine, 76(10 suppl), 90–92.

    Article  Google Scholar 

  • Quaintance, J. L., Arnold, L., & Thompson, G. S. (2008). Development of an instrument to measure the climate of professionalism in a clinical teaching environment. Academic Medicine, 83, S5–S8.

    Article  Google Scholar 

  • Razack, S., Faremo, S., Drolet, F., Snell, L., Wiseman, J., & Pickering, J. (2009). Multiple mini-interviews versus traditional interviews: Stakeholder acceptability comparison. Medical Education, 43, 993–1000.

    Article  Google Scholar 

  • Shannon, H. S., & Norman, G. R. (2009). Deriving the factor structure of safety climate scales. Safety Science, 27, 327–329.

    Article  Google Scholar 

  • Snell, L. (2009). Teaching professionalism and fostering professional values during residency: The McGill experience. In R. L. Cruess, S. R. Cruess, & Y. Steinert (Eds.), Teaching medical professionalism (pp. 246–263). Cambridge: Cambridge University Press.

    Google Scholar 

  • Srinivasan, M., Li, S. T., Meyers, F. J., Pratt, D., Collins, J. B., Braddock, C., et al. (2011). “Teaching as a competency”: Competencies for medical educators. Academic Medicine, 86, 1211–1220.

    Article  Google Scholar 

  • Stern, D. T. (Ed.). (2005). Measuring medical professionalism. New York: Oxford University Press.

    Google Scholar 

  • Streiner, D., & Norman, G. (2008). Health measurement scales: A practical guide to their development and use. USA: Oxford University Press.

    Book  Google Scholar 

  • Sutkin, G., Littlefield, J. H., & Laube, D. W. (2009). Nursing staff assessment of residents’ professionalism and communication skills. Medical Education, 43(11), 1104.

    Article  Google Scholar 

  • Sutkin, G., Wagner, E., Harris, I., & Schiffer, R. (2008). What makes a good clinical teacher in medicine? A review of the literature. Academic Medicine, 83, 452–466.

    Article  Google Scholar 

  • Todhunter, S., Cruess, S. R., Cruess, R. L., Young, M., & Steinert, Y. (2011). Developing and piloting a form for student assessment of faculty professionalism. Advances in Health Sciences Education, 16, 223–238.

    Article  Google Scholar 

  • Tromp, F., Vernooij-Dassen, M., Kramer, A., Grol, R., & Bottema, K. (2010). Behavioral elements of professionalism: Assessment of a fundamental concept in medical care. Medical Teacher, 32, E161–E169.

    Article  Google Scholar 

  • Wilkinson, T. J., Wade, W. B., & Knock, L. D. (2009). A blueprint to assess professionalism: Results of a systematic review. Academic Medicine, 84(5), 551–558.

    Article  Google Scholar 

  • Wood, T. (2009). Assessment not only drives learning, it may also help learning. Medical Education, 43, 5–6.

    Article  Google Scholar 

  • Wood, J., Collins, J., Burnside, E. S., Albanese, M. A., Propeck, P. A., Kelcz, F., et al. (2004). Patient, faculty and self-assessment of radiology resident performance: A 360-degree method of measuring professionalism and interpersonal/communication skills. Academic Radiology, 11, 931–939.

    Google Scholar 

  • Wright, S. M., Kern, D. E., Kolodner, K., Howard, D. M., & Brancati, F. L. (1998). Attributes of excellent attending-physician role models. New England Journal of Medicine, 339, 1986–1993.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Dr. Joyce Pickering, previous Associate Dean of Undergraduate Medical Education at McGill University for her assistance in integrating the PACT into the assessment and evaluation program at McGill. We would also like to thank Samantha Eidinger and Katharine Fischer for their work in anonymizing and coordinating the databases used in this study. Finally, we would like to thank the Office of the Associate Dean, Undergraduate Medical Education at McGill University for the financial support needed to implement the PACT and for continued support of this project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Meredith E. Young.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Young, M.E., Cruess, S.R., Cruess, R.L. et al. The Professionalism Assessment of Clinical Teachers (PACT): the reliability and validity of a novel tool to evaluate professional and clinical teaching behaviors. Adv in Health Sci Educ 19, 99–113 (2014). https://doi.org/10.1007/s10459-013-9466-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-013-9466-4

Keywords

Navigation