Skip to main content
SpringerLink
Log in

Are raters influenced by prior information about a learner? A review of assimilation and contrast effects in assessment

  • Review
  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Understanding which factors can impact rater judgments in assessments is important to ensure quality ratings. One such factor is whether prior performance information (PPI) about learners influences subsequent decision making. The information can be acquired directly, when the rater sees the same learner, or different learners over multiple performances, or indirectly, when the rater is provided with external information about the same learner prior to rating a performance (i.e., learner handover). The purpose of this narrative review was to summarize and highlight key concepts from multiple disciplines regarding the influence of PPI on subsequent ratings, discuss implications for assessment and provide a common conceptualization to inform research. Key findings include (a) assimilation (rater judgments are biased towards the PPI) occurs with indirect PPI and contrast (rater judgments are biased away from the PPI) with direct PPI; (b) negative PPI appears to have a greater effect than positive PPI; (c) when viewing multiple performances, context effects of indirect PPI appear to diminish over time; and (d) context effects may occur with any level of target performance. Furthermore, some raters are not susceptible to context effects, but it is unclear what factors are predictive. Rater expertise and training do not consistently reduce effects. Making raters more accountable, providing specific standards and reducing rater cognitive load may reduce context effects. Theoretical explanations for these findings will be discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baltes, B. B., & Parker, C. P. (2000). Reducing the effects of performance expectations on behavioral ratings. Organizational Behavior and Human Decision Processes, 82(2), 237–267.

    Article  Google Scholar 

  • Becker, G. A., & Miller, C. E. (2002). Examining contrast effects in performance appraisals: Using appropriate controls and assessing accuracy. The Journal of psychology, 136(6), 667–683.

    Article  Google Scholar 

  • Becker, G. A., & Villanova, P. (1995). Effects of rating procedure and temporal delay on the magnitude of contrast effects in performance ratings. The Journal of Psychology, 129(2), 157–166.

    Article  Google Scholar 

  • Bravo, I. M., & Kravitz, D. A. (1996). Context effects in performance appraisals - influence of target value, context polarity, and individual differences. Journal of Applied Social Psychology, 26(19), 1681–1701.

    Article  Google Scholar 

  • Buckley, M. R., Villanova, P., & Benson, P. G. (1989). Contrast effects in performance ratings: Another look across time. Applied Psychology, 38(2), 131–143.

    Article  Google Scholar 

  • Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. The Journal of Personality and Social Psychology, 42(1), 116–131. https://doi.org/10.1037/0022-3514.42.1.116.

    Article  Google Scholar 

  • Cox, S. M. (2008). “Forward feeding” about students’ progress: Information on struggling medical students should not be shared among clerkship directors or with students’ current teachers. Academic Medicine, 83(9), 801.

    Article  Google Scholar 

  • Damisch, L., & Mussweiler, T. (2006). Olympic medals as fruits of comparison? Assimilation and contrast in sequential performance judgments. Journal of Experimental Psychology: Applied, 12(3), 166–178.

    Google Scholar 

  • Dory, V., Danoff, D., Plotnick, L. H., Cummings, B. A., Gomez-Garibello, C., Pal, N. E., et al. (2020). Does educational handover influence subsequent assessment? Academic Medicine. https://doi.org/10.1097/ACM.0000000000003528.

    Article  Google Scholar 

  • Duerksen, G. L. (1972). Some effects of expectation on evaluation of recorded musical performance. Journal of Research in Music Education, 20(2), 268–272.

    Article  Google Scholar 

  • Eva, K. W. (2018). Cognitive influences on complex performance assessment: Lessons from the interplay between medicine and psychology. Journal of Applied Research in Memory and Cognition, 7(2), 177–188.

    Article  Google Scholar 

  • Evans, J. S. B. T. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 66(2), 127–148.

    Google Scholar 

  • Fenner, D. B., Lerch, F. J., & Kulik, C. T. (1993). The impact of computerized performance monitoring and prior performance knowledge on performance evaluation. Journal of Applied Social Psychology, 23(7), 573–601.

    Article  Google Scholar 

  • Findlay, L. C., & Ste-Marie, D. M. (2004). A reputation bias in figure skating judging. Journal of Sport and Exercise Psychology, 26, 154–166.

    Article  Google Scholar 

  • Foster, G., Algozzine, B., & Ysseldyke, J. (1980). Classroom teacher and teacher-in-training susceptibility to stereotypical bias. The Personnel and Guidance Journal, 59(1), 27–30.

    Article  Google Scholar 

  • Foti, R. J., & Hauenstein, N. M. A. (1993). Processing demands and the effects of prior impressions on subsequent judgments: Clarifying the assimilation/contrast debate. Organizational Behavior and Human Decision Processes, 56, 167–189.

    Article  Google Scholar 

  • Frellsen, S. L., Baker, E. A., Papp, K. K., & Durning, S. J. (2008). Medical school policies regarding struggling medical students during the internal medicine clerkships: Results of a national survey. Academic Medicine, 83(9), 876–881.

    Article  Google Scholar 

  • Gauthier, G., St-Onge, C., & Tavares, W. (2016). Rater cognition: Review and integration of research findings. Medical Education, 50(5), 511–522.

    Article  Google Scholar 

  • Gingerich, A., Regehr, G., & Eva, K. W. (2011). Rater-based assessments as social judgments: Rethinking the etiology of rater errors. Academic Medicine, 86(10 Suppl), 1–7. https://doi.org/10.1097/acm.0b013e31822a6cf8.

    Article  Google Scholar 

  • Gingerich, A., Schokking, E., & Yeates, P. (2018). Comparatively salient: Examining the influence of preceding performances on assessors ’ focus and interpretations in written assessment comments. Advances in Health Sciences Education, 23(5), 937–959.

    Article  Google Scholar 

  • Hanges, P. J., Braverman, E. P., & Rentsch, J. R. (1991). Changes in raters’ perceptions of subordinates: A catastrophe model. Journal of Applied Psychology, 76(6), 878–888.

    Article  Google Scholar 

  • Heslin, P. A., Latham, G. P., & VandeWalle, D. (2005). The effect of implicit person theory on performance appraisals. Journal of Applied Psychology, 90, 842–856.

    Article  Google Scholar 

  • Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32(8), 676–682.

    Article  Google Scholar 

  • Huber, V. L. (1989). Comparison of the effects of specific and general performance standards on performance appraisal decisions. Decision Sciences, 20(3), 545–557.

    Article  Google Scholar 

  • Humphrey-Murto, S., Leblanc, A., Touchie, C., Pugh, D., Wood, T. J., Cowley, L., & Shaw, T. (2019). The influence of prior performance information on ratings of present performance: Implications for learner handover: A scoping review. Academic Medicine, 94(7), 1050–1057.

    Article  Google Scholar 

  • Iramaneerat, C., & Yudkowsky, R. (2007). Rater errors in a clinical skills assessment of medical students. Evaluation and the Health Profession, 30(3), 266–283.

    Article  Google Scholar 

  • Kaplan, S. (1995). An examination of auditors reporting intentions upon discovery of procedures prematurely signed-off. Auditing, 14, 90–104.

    Google Scholar 

  • Kassam, A., Ruetalo, M., Topps, M., Mountjoy, M., Walton, M., Edwards, S., & Nickell, L. (2019). Key stakeholder opinions for a national learner education handover. BMC Medical Education, 19(1), 150. https://doi.org/10.1186/s12909-019-1598-7.

    Article  Google Scholar 

  • Kravitz, D. A., Balzer, W. K., Martin, B., Mcgrath, J., & Murphy, K. (1992). Context effects in performance appraisal: A methodological critique and empirical study. Journal of Applied Psychology, 77(1), 24–31.

    Article  Google Scholar 

  • Larson, J. R., Lingle, J. H., & Scerbo, M. M. (1984). The impact of performance cues on leader-behavior ratings: The role of selective information availability and probabilistic response bias. Organizational Behavior and Human Performance, 33, 323–349.

    Article  Google Scholar 

  • Maurer, T. J., & Alexander, R. A. (1991). Contrast effects in behavioral measurement: An investigation of alternative process explanations. Journal of Applied Psychology, 76(1), 3–10.

    Article  Google Scholar 

  • Maurer, T. J., Palmer, J. K., & Ashe, D. K. (1993). Diaries, checklists, evaluations, and contrast effects in measurement of behavior. Journal of Applied Psychology, 78, 226–231.

    Article  Google Scholar 

  • Maurer, T. J., Palmer, J. K., & Lisnov, S. S. (1995). Distinguishing context effects from context errors in judgments of behavior. Journal of Applied Social Psychology, 25(18), 1637–1651.

    Article  Google Scholar 

  • McGaghie, W. C. (2015). Varieties of integrative ccholarship: Why rules of evidence, criteria, and standards matter. Academic Medicine, 90(3), 294–302.

    Article  Google Scholar 

  • Miles, W. S., Shaw, V., & Risucci, D. (2001). The role of blinded interviews in the assessment of surgical residency candidates. American Journal of Surgery, 182(2), 143–146.

    Article  Google Scholar 

  • Mims, L. D., DeCastro, A. O., & Kelly, A. G. (2017). Perspectives of family medicine clerkship directors regarding forward feeding: A CERA Study. Family Medicine, 49(9), 699–705.

    Google Scholar 

  • Murphy, K. R., Balzer, W. K., Lockhart, M. C., & Eisenman, E. J. (1985). Effects of previous performance on evaluations of present performance. Journal of Applied Psychology, 70, 72–84.

    Article  Google Scholar 

  • Murphy, M. R., & Jones, A. P. (1993). The influences of performance cues and observational focus on performance rating accuracy. Journal of Applied Psychology, 23(18), 1523–1545.

    Google Scholar 

  • Mussweiler, T. (2003). Comparison processes in social judgment: Mechanisms and consequences. Psychological Review, 220, 472–489.

    Article  Google Scholar 

  • Nieminen, L. R. G., Rudolph, C. W., Baltes, B. B., Casper, C. M., Wynne, K. T., & Kirby, L. C. (2013). The combined effect of ratee’s bodyweight and past performance information on performance judgments. Journal of Applied Social Psychology, 43, 527–543.

    Article  Google Scholar 

  • Palmer, J. K., & Feldman, J. M. (2005). Accountability and need for cognition effects on contrast, halo, and accuracy in performance ratings. The Journal of Psychology, 139(2), 119–137.

    Article  Google Scholar 

  • Palmer, J. K., & Gore, J. S. (2014). A theory of contrast effects in performance appraisal and social cognitive judgments. Psychological Studies, 59(4), 323–336.

    Article  Google Scholar 

  • Palmer, J. K., Maurer, T. J., & Feldman, J. M. (2002). Context and prior impression effects on attention, judgment standards, and ratings: Contrast effects revisited. Journal of Applied Social Psychology, 32(12), 2575–2597.

    Article  Google Scholar 

  • Perry, R. P., Abrami, P. C., Leventhal, L., & Check, J. (1979). Instructor reputation: An expectancy relationship involving student ratings and achievement. Journal of Educational Psychology, 71(6), 776–787.

    Article  Google Scholar 

  • Perry, R. P., Niemi, R. R., & Jones, K. (1974). Effect of prior teaching evaluations and lecture presentation on ratings of teaching performance. Journal of Educational Psychology, 66(6), 851–856.

    Article  Google Scholar 

  • Radocy, R. E. (1976). Effects of authority figure Biases on changing judgments of musical events. Journal of Research in Music Education, 24(1), 119–128.

    Article  Google Scholar 

  • Raymark, P. H., Balzer, W. K., & DeLaTorre, F. (1999). A preliminary investigation of the sources of information used by raters when appraising performance. Journal of Business and Psychology, 14(2), 319–339.

    Article  Google Scholar 

  • Reilly, S. P., Bank, C. M., Smither, J. W., Warech, M. A., & Reilly, R. R. (1998). The influence of indirect knowledge of previous performance on ratings of present performance: The effects of job familiarity and rater training. Journal of Business and Psychology, 12(4), 421–435.

    Article  Google Scholar 

  • Salvemini, N. J., Reilly, R. R., & Smither, J. W. (1993). The influence of rater motivation on assimilation effects and accuracy in performance ratings. Organizational Behavior and Human Decision Processes, 55, 41–60.

    Article  Google Scholar 

  • Shaw, T., Wood, T. J., Touchie, C., Pugh, D., & Humphrey-Murto, S. (2020). How biased are you? The effect of prior performance information on attending physician ratings and implications for learner handover. Advances in Health Sciences Education. https://doi.org/10.1007/s10459-020-09979-6.

    Article  Google Scholar 

  • Siegall, M. (1992). The effect of rater expectations on the evaluation of a hypothetical subordinate. The Journal of Psychology, 126(5), 453–463.

    Article  Google Scholar 

  • Sisco, H., & Leventhal, G. (2007). Effect of prior performance on subsequent performance evaluation by field independent-dependent raters. Perceptual and Motor Skills, 105, 852–861.

    Article  Google Scholar 

  • Skowronski, J. J., & Carlston, D. E. (1989). Negativity and extremity biases in impression formation: A review of explanations. Psychological Bulletin, 105, 131–142.

    Article  Google Scholar 

  • Smither, J. W., Reilly, R. R., & Buda, R. (1988). Effect of prior performance information on ratings of present performance: Contrast versus assimilation revisited. Journal of Applied Psychology, 73(3), 487–496.

    Article  Google Scholar 

  • Steiner, D. D., & Rain, J. S. (1989). Immediate and delayed primacy and recency effects in performance evaluation. Journal of Applied Psychology, 74(1), 236–242.

    Article  Google Scholar 

  • St-Onge, C., Chamberland, M., Lévesque, A., & Varpio, L. (2016). Expectations, observations, and the cognitive processes that find them: Expert assessment of examinee performance. Advances in Health Sciences Education, 21, 627–642.

    Article  Google Scholar 

  • Stroud, L., Herold, J., Tomlinson, G., & Cavalcanti, R. B. (2011). Who you know or what you know? Effect of examiner familiarity with residents on OSCE Scores. Academic Medicine, 86(10), S8–S11.

    Article  Google Scholar 

  • Sulsky, L. M., & Day, D. V. (1994). Effects of frame-of-reference training on rater accuracy under alternative time delays. Journal of Applied Psychology, 79, 535–543.

    Article  Google Scholar 

  • Sumer, H. C., & Knight, P. A. (1996). Assimilation and contrast effects in performance ratings: Effects of rating the previous performance on rating subsequent performance. Journal of Applied Psychology, 81(4), 436–442.

    Article  Google Scholar 

  • Tavares, W., & Eva, K. W. (2013). Exploring the impact of mental workload on rater-based assessments. Advances in Health Sciences Education, 18(2), 291–303.

    Article  Google Scholar 

  • Taylor, S. (1991). Asymmetrical effects of positive and negative events: The mobilization- minimization hypothesis. Psychological Bulletin, 110, 67–85.

    Article  Google Scholar 

  • ten Cate, O., & Regehr, G. (2019). The Power of subjectivity in the assessment of medical trainees. Academic Medicine, 94(3), 333–337.

    Article  Google Scholar 

  • Thelwell, R. C., Page, J. L., Lush, A., Greenlees, I. A., & Manley, A. J. (2013). Can reputation biases influence the outcome and process of making competence judgments of a coach? Scandinavian Journal of Medicine and Science in Sports, 23(1), 65–73.

    Article  Google Scholar 

  • Towler, A., & Dipboye, R. L. (2006). Effects of trainer reputation and trainees’ need for cognition on training outcomes. Journal of Psychology, 140(6), 549–564.

    Article  Google Scholar 

  • Uggerslev, K. L., & Sulsky, L. M. (2002). Presentation modality and indirect performance information: Effects on ratings, reactions, and memory. Journal of Applied Psychology, 87, 940–950.

    Article  Google Scholar 

  • van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & Van Tartwijk, J. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34(3), 205–214.

    Article  Google Scholar 

  • Van Merriënboer, J. J. G., & Sweller, J. (2010). Cognitive load theory in health professional education: Design principles and strategies. Medical Education, 44(1), 85–93.

    Article  Google Scholar 

  • Williams, R. G., Klamen, D. A., & Mcgaghie, W. C. (2003). Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Medicine, 15(4), 270–292.

    Article  Google Scholar 

  • Woehr, D. J., & Roch, S. G. (1996). Context effects in performance evaluation: The impact of ratee sex and performance level on performance ratings and behavioral recall. Organizational Behavior and Human Decision Processes, 66(1), 31–41.

    Article  Google Scholar 

  • Wood, T. J. (2013). Mental workload as a tool for understanding dual processes in rater-based assessments. Advances in Health Sciences Education, 18(3), 523–525. https://doi.org/10.1007/s10459-012-9396-6.

    Article  Google Scholar 

  • Wood, T. J. (2014). Exploring the role of first impressions in rater-based assessments. Advances in Health Sciences Education, 19(3), 409–427.

    Article  Google Scholar 

  • Yeates, P., Cardell, J., Byrne, G., & Eva, K. W. (2015a). Relatively speaking: Contrast effects influence assessors’ scores and narrative feedback. Medical Education, 49, 909–919.

    Article  Google Scholar 

  • Yeates, P., Moreau, M., & Eva, K. (2015b). Are examiners’ judgments in OSCE-style assessments influenced by contrast effects? Academic Medicine, 90(7), 975–980.

    Article  Google Scholar 

  • Yeates, P., O’Neill, P., Mann, K., & Eva, K. W. (2012). Effect of exposure to good versus poor medical trainee performance on attending physician ratings of subsequent performances. JAMA, 308(21), 2226–2232.

    Article  Google Scholar 

  • Yeates, P., O’Neill, P., Mann, K., & Eva, W. K. (2013). “You’re certainly relatively competent”: Assessor bias due to recent experiences. Medical Education, 47, 910–922.

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to acknowledge the contributions of the librarian Becky Skidmore who helped design and conduct the searches.

Funding

A portion of this study was funded by a grant from the Department of Innovation in Medical Education at the University of Ottawa. Dr. Humphrey-Murto receives salary support from the Department of Medicine, University of Ottawa.

Author information

Authors and Affiliations

  1. Department of Medicine, Faculty of Medicine, The Ottawa Hospital-Riverside Campus, University of Ottawa, 1967 Riverside Drive, Box 67, Ottawa, ON, Canada

    Susan Humphrey-Murto, Claire Touchie & Debra Pugh

  2. Department of Medicine, Faculty of Medicine, The Ottawa Hospital-General Campus, Ottawa, ON, Canada

    Tammy Shaw

  3. Department of Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada

    Susan Humphrey-Murto, Lindsay Cowley & Timothy J. Wood

  4. Medical Council of Canada, Ottawa, ON, Canada

    Claire Touchie & Debra Pugh

Authors
  1. Susan Humphrey-Murto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tammy Shaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claire Touchie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Debra Pugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lindsay Cowley
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Timothy J. Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by all authors. The first draft of the manuscript was written by SH-M and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Susan Humphrey-Murto.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Humphrey-Murto, S., Shaw, T., Touchie, C. et al. Are raters influenced by prior information about a learner? A review of assimilation and contrast effects in assessment. Adv in Health Sci Educ 26, 1133–1156 (2021). https://doi.org/10.1007/s10459-021-10032-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-021-10032-3

Keywords

Search

Navigation