Skip to main content
SpringerLink
Log in

The influence of first impressions on subsequent ratings within an OSCE station

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Competency-based assessment is placing increasing emphasis on the direct observation of learners. For this process to produce valid results, it is important that raters provide quality judgments that are accurate. Unfortunately, the quality of these judgments is variable and the roles of factors that influence the accuracy of those judgments are not clearly understood. One such factor is first impressions: that is, judgments about people we do not know, made quickly and based on very little information. This study explores the influence of first impressions in an OSCE. Specifically, the purpose is to begin to examine the accuracy of a first impression and its influence on subsequent ratings. We created six videotapes of history-taking performance. Each video was scripted from a real performance by six examinee residents within a single OSCE station. Each performance was re-enacted with six different actors playing the role of the examinees and one actor playing the role of the patient and videotaped. A total of 23 raters (i.e., physician examiners) reviewed each video and were asked to make a global judgment of the examinee’s clinical abilities after 60 s (First Impression GR) by providing a rating on a six-point global rating scale and then to rate their confidence in the accuracy of that judgment by providing a rating on a five-point rating scale (Confidence GR). After making these ratings, raters then watched the remainder of the examinee’s performance and made another global rating of performance (Final GR) before moving on to the next video. First impression ratings of ability varied across examinees and were moderately correlated to expert ratings (r = .59, 95% CI [−.13, .90]). There were significant differences in mean ratings for three examinees. Correlations ranged from .05 to .56 but were only significant for three examinees. Rater confidence in their first impression was not related to the likelihood of a rater changing their rating between the first impression and a subsequent rating. The findings suggest that first impressions could play a role in explaining variability in judgments, but their importance was determined by the videotaped performance of the examinees. More work is needed to clarify conditions that support or discourage the use of first impressions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The term first impression is being used as a general term to refer to not only first impressions, but also thin slice judgements and zero acquaintance judgments.

References

  • AERA, APA, & NCME. (2014). Standards for educational and psychological testing (pp. 11–31). Washington, DC: American Educational Research Association.

    Google Scholar 

  • Ambady, N. (2010). The perils of pondering: Intuition and thin slice judgments. Psychological Inquiry, 21(4), 271–278.

    Article  Google Scholar 

  • Ambady, N., Bernieri, F., & Richeson, J. (2000). Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. Advances in Experimental Social Psychology, 32, 201–271.

    Article  Google Scholar 

  • Ames, D. R., Kammrath, L. K., Suppes, A., & Bolger, N. (2010). Not so fast: The (not-quite-complete) dissociation between accuracy and confidence in thin-slice impressions. Personality and Social Psychology Bulletin, 36(2), 264–277.

    Article  Google Scholar 

  • Barrick, M. R., Swider, B. W., & Stewart, G. L. (2010). Initial evaluations in the interview: Relationships with subsequent interviewer evaluations and employment offers. The Journal of Applied Psychology, 95(6), 1163–1172.

    Article  Google Scholar 

  • Borkenau, P., & Liebler, A. (1992). Trait inferences: Sources of validity at zero acquaintance. Journal of Personality and Social Psychology, 62(4), 645–657.

    Article  Google Scholar 

  • Bown, M. H., Regehr, G., & Reznick, R. (1996). The effect of early performance on examiners’ marking patterns during an oral examination. Academic Medicine, 71(1), s73–s75.

    Article  Google Scholar 

  • Burdick, M. P., & Schoffstall, J. (1995). Observation of emergency medicine residents at the bedside: How often does it happen. Academic Emergency Medicine, 2(10), 909–913.

    Article  Google Scholar 

  • Carney, D., Colvin, C., & Hall, J. (2007). A thin slice perspective on the accuracy of first impressions. Journal of Research in Personality, 41(5), 1054–1072.

    Article  Google Scholar 

  • Cooper, W. H. (1981). Ubiquitous halo. Psychological Bulletin, 90(2), 218–244.

    Article  Google Scholar 

  • Croskerry, P. (2009). Clinical cognition and diagnostic error: Applications of a dual process model of reasoning. Advances in Health Sciences Education, 14, 27–35.

    Article  Google Scholar 

  • Dougherty, T. W., Turban, D. B., & Callender, J. C. (1994). Confirming first impressions in the employment interview: A field study of interview behaviour. Journal of Applied Psychology, 5(5), 659–665.

    Article  Google Scholar 

  • Evans, J. S. B. T. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59, 255–278.

    Article  Google Scholar 

  • Funder, D. C. (1987). Errors and mistakes: Evaluating the accuracy of social judgment. Psychological Bulletin, 101(1), 75–90.

    Article  Google Scholar 

  • Govaerts, M. J. B., Schuwirth, L. W. T., Van der Vleuten, C. P. M., & Muijtjens, A. M. M. (2011). Workplace-based assessment: Effects of rater expertise. Advances in Health Sciences Education, 16(2), 151–165.

    Article  Google Scholar 

  • Harris, M., & Garris, C. (2008). You never get a second chance to make a first impression. In N. Ambady & J. Skowronski (Eds.), First impressions (pp. 147–168). New York: Guilford Press.

    Google Scholar 

  • Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32(8), 676–682.

    Article  Google Scholar 

  • Kahneman, D. (2011). Thinking, fast and slow. Canada: Doubleday.

    Google Scholar 

  • Lopes, L. L. (1991). The rhetoric of irrationality. Theory and Psychology, 1(1), 65–82.

    Article  Google Scholar 

  • Macan, T. H., & Dipboye, R. L. (1990). The relationship of interviewer’s preinterview impressions to selection and recruitment outcomes. Personnel Psychology, 43(4), 745–768.

    Article  Google Scholar 

  • Martin, J. A., Reznick, R., Rothman, A., Tamblyn, R., & Regehr, G. (1996). Who should rate candidates in an objective structured clinical examination? Academic Emergency Medicine, 71(2), 170–175.

    Google Scholar 

  • Murphy, K. R., Jako, R. A., & Anhalt, R. L. (1993). Nature and consequences of halo error: A critical analysis. Journal of Applied Psychology, 78(2), 218–225.

    Article  Google Scholar 

  • Norman, G. R. (2009). Dual processing and diagnostic errors. Advances in Health Sciences Education, 14(1), 37–49.

    Article  Google Scholar 

  • Pelgrim, E. A., Kramer, A. W., Mokkink, H. G., & Van der Vleuten, C. P. M. (2012). The process of feedback in workplace-based assessment: Organisation, delivery, continuity. Medical Education, 46(6), 604–612.

    Article  Google Scholar 

  • Rannelli, L., Coderre, S., Paget, M., Woloschuk, W., Wright, B., & Mclaughlin, K. (2014). How do medical students form impressions of the effectiveness of classroom teachers? Medical Education, 48(8), 831–837.

    Article  Google Scholar 

  • Smith, H. J., Archer, D., & Costanzo, M. (1991). Just a hunch?: Accuracy and awareness in person perception. Journal of Nonverbal Behavior, 15(1), 3–18.

    Article  Google Scholar 

  • Touchie, C., Humphrey-Murto, S., Ainslie, M., Myers, K., & Wood, T. J. (2010). Two models of raters in a structured oral examination: Does it make a differnece? Advances in Health Sciences Education, 15(1), 97–108.

    Article  Google Scholar 

  • Williams, R. G., Dunnington, G. L., Mellinger, J. D., & Klamen, D. L. (2015). Placing constraints on the use of the ACGME milestones: A commentary on the limitations of global performance ratings. Academic Medicine, 90(4), 404–407.

    Article  Google Scholar 

  • Wood, T. J. (2014). Exploring the role of first impressions in rater-based assessments. Advances in Health Sciences Education, 19(3), 409–427.

    Article  Google Scholar 

  • Yaphe, J., & Street, S. (2003). How do examiners decide?: A qualitative study of the process of decision making in the oral examination component of the MRCGP examination. Medical Education, 37(9), 764–771.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported in part by a research grant from the Department of Medicine, University of Ottawa. The authors would like to acknowledge Lesley Ananny, Sarah Lynch, and Meredith Mackay for their help on this study as well as the Department of Innovation in Medical Education. In addition, the authors would like to thank the anonymous reviewers for their kind suggestions.

Author information

Authors and Affiliations

  1. Department of Innovation in Medical Education, Faculty of Medicine, University of Ottawa, RGN 2206, 451 Smyth Road, Ottawa, ON, K1H 8M5, Canada

    Timothy J. Wood

  2. Department of Medicine, University of Ottawa, 501 Smyth Road, Ottawa, ON, K1H 8L6, Canada

    James Chan, Susan Humphrey-Murto, Debra Pugh & Claire Touchie

Authors
  1. Timothy J. Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susan Humphrey-Murto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Debra Pugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Claire Touchie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy J. Wood.

Appendix: Rating scales used in the study

Appendix: Rating scales used in the study

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wood, T.J., Chan, J., Humphrey-Murto, S. et al. The influence of first impressions on subsequent ratings within an OSCE station. Adv in Health Sci Educ 22, 969–983 (2017). https://doi.org/10.1007/s10459-016-9736-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-016-9736-z

Keywords

Search

Navigation