Advertisement

Can physician examiners overcome their first impression when examinee performance changes?

  • Timothy J. Wood
  • Debra Pugh
  • Claire Touchie
  • James Chan
  • Susan Humphrey-Murto
Article

Abstract

There is an increasing focus on factors that influence the variability of rater-based judgments. First impressions are one such factor. First impressions are judgments about people that are made quickly and are based on little information. Under some circumstances, these judgments can be predictive of subsequent decisions. A concern for both examinees and test administrators is whether the relationship remains stable when the performance of the examinee changes. That is, once a first impression is formed, to what degree will an examiner be willing to modify it? The purpose of this study is to determine the degree that first impressions influence final ratings when the performance of examinees changes within the context of an objective structured clinical examination (OSCE). Physician examiners (n = 29) viewed seven videos of examinees (i.e., actors) performing a physical exam on a single OSCE station. They rated the examinees’ clinical abilities on a six-point global rating scale after 60 s (first impression or FIGR). They then observed the examinee for the remainder of the station and provided a final global rating (GRS). For three of the videos, the examinees’ performance remained consistent throughout the videos. For two videos, examinee performance changed from initially strong to weak and for two videos, performance changed from initially weak to strong. The mean FIGR rating for the Consistent condition (M = 4.80) and the Strong to Weak condition (M = 4.87) were higher compared to their respective GRS ratings (M = 3.93, M = 2.73) with a greater decline for the Strong to Weak condition. The mean FIGR rating for the Weak to Strong condition was lower (3.60) than the corresponding mean GRS (4.81). This pattern of findings suggests that raters were willing to change their judgments based on examinee performance. Future work should explore the impact of making a first impression judgment explicit versus implicit and the role of context on the relationship between a first impression and a subsequent judgment.

Keywords

Rater cognition First impressions Rater-based assessment 

Notes

Acknowledgements

This research was supported in part by a research grant from the Medical Council of Canada and from the Department of Medicine, University of Ottawa. The authors would like to acknowledge Lesley Ananny, Meredith Mackay, and Katherine Scowcroft for their help on this study as well as the Department of Innovation in Medical Education.

References

  1. Ambady, N., Bernieri, F., & Richeson, J. (2000). Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. In Advances in experimental social psychology (pp. 201–271).Google Scholar
  2. Bown, M. H., Regehr, G., & Reznick, R. (1996). the effect of early performance on examiners’ marking patterns during an oral examination. Academic Medicine, 71(1), s73–s75.CrossRefGoogle Scholar
  3. Carney, D., Colvin, C., & Hall, J. (2007). A thin slice perspective on the accuracy of first impressions. Journal of Research in Personality, 41(5), 1054–1072.CrossRefGoogle Scholar
  4. Crossley, J., Johnson, G., Booth, J., & Wade, W. (2011). Good questions, good answers: Construct alignment improves the performance of workplace-based assessment scales. Medical Education, 45, 560–569.CrossRefGoogle Scholar
  5. Gauthier, G., St-Onge, C., & Tavares, W. (2016). Rater cognition: Review and integration of research findings. Medical Education, 50, 511–522.CrossRefGoogle Scholar
  6. Gingerich, A., Kogan, J. R., Yeates, P., Govaerts, M. J. B., & Holmboe, E. S. (2014). Seeing the “Black Box” differently: Assesssor cognition from three research perspectives. Medical Education, 48, 1055–1068.CrossRefGoogle Scholar
  7. Gingerich, A., Regehr, G., & Eva, K. W. (2011). Rater-based assessments as social judgments: Rethinking the etiology of rater errors. Academic Medicine, 86(10 Suppl), S1–S7.CrossRefGoogle Scholar
  8. Govaerts, M. J. B., Schuwirth, L. W. T., Van der Vleuten, C. P. M., & Muijtjens, A. M. M. (2011). Workplace-based assessment: Effects of rater expertise. Advances in Health Sciences Education, 16(2), 151–165.CrossRefGoogle Scholar
  9. Govaerts, M. J. B., Van de Wiel, M. W. J., Schuwirth, L. W. T., Van der Vleuten, C. P. M., & Muijtjens, a M. M. (2013). Workplace-based assessment: Raters’ performance theories and constructs. Advances in Health Sciences Education, 18(3), 375–396.Google Scholar
  10. Harris, M., & Garris, C. (2008). You never get a second chance to make a first impression. In N. Ambady & J. Skowronski (Eds.), First impressions (pp. 147–168). New York: Guilford Press.Google Scholar
  11. Hilton, J. L., Klein, J. G., & von Hippel, W. (1991). Attention allocation and impression formation. Personality and Social Psychology Bulletin, 17(5), 548–559.CrossRefGoogle Scholar
  12. Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32, 676–682.CrossRefGoogle Scholar
  13. Iramaneerat, C., & Yudkowsky, R. (2007). Rater errors in a clinical skills assessment of medical students. Evaluation and the Health Professions, 30(3), 266–283.CrossRefGoogle Scholar
  14. Kogan, J., Conforti, L., Bernabeo, E., Iobst, W., & Holmboe, E. (2011). Opening the black box of clinical skills assessment via observation: A conceptual model. Medical Education, 45(10), 1048–1060.CrossRefGoogle Scholar
  15. Macan, T. H., & Dipboye, R. L. (1990). The relationship of interviewers’ preinterview impressions to selection and recruitment outcomes. Personnel Psychology, 43(4), 745–768.CrossRefGoogle Scholar
  16. Mann, T. C., & Ferguson, M. J. (2015). Can we undo our first impressions? The role of reinterpretation in reversing implicit evaluations. Journal of Personality and Social Psychology, 108(6), 823–849.CrossRefGoogle Scholar
  17. Murphy, K. R., Balzer, W. K., Lockhart, M. C., & Eisenman, E. J. (1985). Effects of previous performance on evaluations of present performance. Journal of Applied Psychology, 70(1), 72–84.CrossRefGoogle Scholar
  18. Skowronski, J. J., & Carlston, D. E. (1989). Negativity and extremity biases in impression formation: A review of explanations. Psychological Bulletin, 105(1), 131–142.CrossRefGoogle Scholar
  19. Sumer, H. C., & Knight, P. A. (1996). Assimilation and contrast effects in performance ratings: Effects of rating the previous performance on rating subsequent performance. The Journal of Psychology, 81(4), 436–442.Google Scholar
  20. Tavares, W., & Eva, K. W. (2013). Exploring the impact of mental workload on rater-based assessments. Advances in Health Sciences Education, 18(2), 291–303.Google Scholar
  21. Williams, R. G., Klamen, D. A., & McGaghie, W. C. (2003). Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Medicine, 15(4), 270–292.CrossRefGoogle Scholar
  22. Wood, T. J. (2014). Exploring the role of first impressions in rater-based assessments. Advances in Health Sciences Education: Theory and Practice, 19(3), 409–427.CrossRefGoogle Scholar
  23. Wood, T. J., Chan, J., Humphrey-Murto, S., Pugh, D., & Touchie, C. (2017). The influence of first impressions on subsequent ratings within an OSCE station. Advances in Health Sciences Education, 22, 969–983.CrossRefGoogle Scholar
  24. Wyer, N. A. (2010). You never get a second chance to make a first (implicit) impression: The role of elaboration in the formation and revision of implicit impressions. Social Cognition, 28(1), 1–19.CrossRefGoogle Scholar
  25. Yaphe, J., & Street, S. (2003). How do examiners decide? A qualitative study of the process of decision making in the oral examination component of the MRCGP examination. Medical Education, 37(9), 764–771.CrossRefGoogle Scholar
  26. Yeates, P., O’Neill, P., Mann, K., & Eva, K. (2013). Seeing the same thing differently: Mechanisms that contribute to assessor differences in directly-observed performance assessments. Advances in Health Sciences Education: Theory and Practice, 18(3), 325–341.CrossRefGoogle Scholar
  27. Yeates, P., Moreau, M., & Eva, K. (2015). Are examiner’s judgments in OSCE-style assessments influenced by contrast effects? Academic Medicine, 90(7), 975–980.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Innovation in Medical Education, Faculty of MedicineUniversity of OttawaOttawaCanada
  2. 2.Department of Medicine, Faculty of MedicineUniversity of OttawaOttawaCanada

Personalised recommendations