Skip to main content
SpringerLink
Log in

Cognitive categories of raters and rating accuracy

  • Full Articles
  • Published:
Journal of Business and Psychology Aims and scope Submit manuscript

Abstract

This research investigated cognitive processes affecting the accuracy of performance evaluation. One hundred and twenty-five nurses from three hospitals completed questionnaires measuring their cognitive categorization processes and then rated a videotaped nurse's performance. Results focusing on the match between raters' cognitive categories and rating scales indicated that: a) rating accuracy was related to the match between raters' cognitive category dimensions and rating scale dimensions, but b) not related to the match between rating scales and raters' general tendency to describe ratees in terms of behavioral or trait characteristics. Raters' ability to differentiate clearly between rating dimensions was related to accuracy and to halo in ratings. Finally, work experience was related to raters' cognitive systems while prior rating experience was important for accuracy. Limitations of the study and implications for future research are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adams-Webber, J. R. (1979).Personal Construct Theory. New York: John Wiley & Sons.

    Google Scholar 

  • Banks, C. G., & Murphy, K. R. (1985). Toward narrowing the research-practice gap in performance appraisal.Personnel Psychology, 38, 335–345.

    Google Scholar 

  • Bernardin, H. J., Cardy, R. L., & Carlyle, J. J. (1982). Cognitive complexity and appraisal effectiveness: Back to the drawing board?Journal of Applied Psychology, 67, 151–160.

    Google Scholar 

  • Bernardin, H. J., & Pence, E. C. (1980). Effects of rater training: Creating new response sets and decreasing accuracy.Journal of Applied Psychology, 65, 60–66.

    Google Scholar 

  • Bernardin, H. J., & Villanova, P. (1986). Performance appraisal. In E. Locke (Ed.),Generalizing from laboratory studies to field settings. Lexington, MA: Lexington Books.

    Google Scholar 

  • Bonarius, J. C. (1965). Research in the personal construct theory of George A. Kelly. In B. A. Maher (Ed.),Progress in experimental personality research Vol. 2. New York: Academic Press.

    Google Scholar 

  • Borman, W. C. (1975). Effects of instructions to avoid halo error on reliability and validity of performance evaluation ratings.Journal of Applied Psychology, 60, 556–560.

    Google Scholar 

  • Borman, W. C. (1977). Consistency of rating accuracy and rating errors in the judgment of human performance.Organizational Behavior and Human Performance, 20, 233–252.

    Google Scholar 

  • Borman, W. C. (1978). Exploring upper limits of reliability and validity in job performance ratings.Journal of Applied Psychology, 63, 135–144.

    Google Scholar 

  • Borman, W. C. (1979). Individual differences correlates of accuracy in evaluating others' performance effectiveness.Applied Psychological Measurement, 3, 103–115.

    Google Scholar 

  • Borman, W. C. (1983). Implication of personality theory and research for the rating of work performance in organizations. In F. Landy, S. Zedeck, & J. Cleveland (Eds.),Performance Measurement and Theory. Hillsdale, NJ: Earlbaum. pp. 127–165.

    Google Scholar 

  • Borman, W. C. (1987). Personal constructs, performance schema, and “folk theories” of subordinate effectiveness: Explorations in an army officer sample.Organizational Behavior and Human Decision Processes, 40, 307–322.

    Google Scholar 

  • Bruner, J. S., & Tagiuri, R. (1954). The perception of people. In G. Lindzey (Ed.),The handbook of social psychology Vol. 2. Reading, MA: Addison-Wesley.

    Google Scholar 

  • Cardy, R. L., Bernardin, H. J., Abbot, J. G., Senderak, M. P. & Taylor, K. (1987). The effects of individual performance schemata and dimension familiarization on rating accuracy.Journal of Occupational Psychology, 60, 197–205.

    Google Scholar 

  • Cooper, W. H. (1981). Ubiquitous halo.Psychological Bulletin, 90, 218–244.

    Google Scholar 

  • Cronbach, L. J. (1955). Processes affecting scores on “understanding of others” and “assumed similarity.”Psychological Bulletin, 52, 177–193.

    Google Scholar 

  • DeNisi, A. S., Cafferty, T. P., & Meglino, B. M. (1984). A cognitive view of the performance appraisal process: A model and research propositions.Organizational Behavior and Human Performance, 33, 360–396.

    Google Scholar 

  • Feldman, J. (1981). Beyond attribution theory: Cognitive processes in performance appraisal.Journal of Applied Psychology, 66, 127–148.

    Google Scholar 

  • Hakel, M. (1969).Significance of implicit personality theories in personality research and theory. Proceedings of the 77th convention of the American Psychological Association, pp. 403–404.

  • Ilgen, D. R., & Feldman, J. M. (1983). Performance appraisal: A process approach. In B. M. Staw (Ed.),Research in organization behavior Vol. 2. Greenwich, CT: JAI Press.

    Google Scholar 

  • Isaacson, G. I. (1966).A comparative study of meaningfulness of personal and common constructs. Unpublished dissertation, University of Missouri.

  • Kane, J. S., & Lawler, E. E., III. (1979). Performance appraisal effectiveness: Its assessments and determinants. In B. M. Staw (Ed.),Research in organizational behavior. Greenwich, CT: JAI Press.

    Google Scholar 

  • Kelly, G. A. (1955).The psychology of personal constructs. New York: Norton.

    Google Scholar 

  • Kozlowski, S. W. J., & Kirsh, M. (1987). The systematic distortion hypothesis, halo and accuracy: An individual-level analysis.Journal of Applied Psychology, 72, 252–264.

    Google Scholar 

  • Kryzystofiak, F., Cardy, R., & Newman, J. (1988). Implicit personality and performance appraisal: The influence of trait inferences on evaluations of behavior.Journal of Applied Psychology, 73, 515–521.

    Google Scholar 

  • Lahey, M. A., & Saal, F. E. (1981). Evidence incompatible with a cognitive compatibility theory of rating behavior.Journal of Applied Psychology, 6, 706–715.

    Google Scholar 

  • Landy, F. J., & Fair J. (1980). Performance rating.Psychological Bulletin, 87, 72–107.

    Google Scholar 

  • Lord, R. G. (1985). Accuracy in development measurement: An alternative definition based on raters' cognitive schema and signal detection theory.Journal of Applied Psychology, 70, 66–71.

    Google Scholar 

  • McIntyre, R. M., Smith, D. E., & Hassett, C. E. (1984). Accuracy of performance ratings as affected by rater training and perceived purpose of rating.Journal of Applied Psychology, 69, 147–156.

    Google Scholar 

  • Murphy, K. R., & Cleveland, J. N. (1991).Performance Appraisal. Boston: Allyn & Bacon.

    Google Scholar 

  • Murphy, K. R., & Constans, J. (1987). Behavioral anchors as a source of bias in ratings.Journal of Applied Psychology, 72, 523–579.

    Google Scholar 

  • Murphy, K. R., & Jako, R. (1989). Under what conditions are observed intercorrelations greater or smaller than true intercorrelations.Journal of Applied Psychology, 74, 827–830.

    Google Scholar 

  • Murphy, K. R., Martin, C., & Garcia, M. (1982). Do behavioral observation scales measure observation?Journal of Applied Psychology, 67, 562–567.

    Google Scholar 

  • Nathan, B. R., & Alexander, R. A. (1985). The role of inferential accuracy in performance rating.Academy of Management Review, 10, 109–115.

    Google Scholar 

  • Nathan, B. R., & Lord, R. G. (1983). Cognitive categorization and dimensional schemata: A process approach to the study of halo in performance ratings.Journal of Applied Psychology, 68, 102–114.

    Google Scholar 

  • Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes.Psychological Review, 84, 231–259.

    Google Scholar 

  • Norman, W. T., & Goldberg, L. R. (1966). Raters, ratees, and randomness in personality structure.Journal of Personality and Social Psychology, 4, 681–691.

    Google Scholar 

  • Passini, F. T., & Norman, W. T. (1966). A universal conception of personality structure?Journal of Personality and Social Psychology, 4, 44–49.

    Google Scholar 

  • Pulakos, E. D. (1984). A comparison of rater training programs: Error training versus accuracy training.Journal of Applied Psychology, 69, 581–588.

    Google Scholar 

  • Pulakos, E. D. (1986). The development of training programs to increase accuracy with different rating tasks.Organizational Behavior and Human Decision Processes, 38, 76–91.

    Google Scholar 

  • Rosch, E., Mervis, C. G., Gray, W. D., Johnson, D. M., & Boyers-Braem, P. (1976). Basic objects in natural categories.Cognitive Psychology, 8, 382–439.

    Google Scholar 

  • Rosenberg, S. (1977). New approaches to the analysis of personal constructs in person perception. InNebraska Symposium on Motivation 1976. Lincoln: University of Nebraska Press.

    Google Scholar 

  • Sauser, W. I., & Pond, S. B. (1981). Effects of rater training and participation on cognitive complexity: An exploration of Schneier's cognitive reinterpretation.Personnel Psychology, 34, 563–577.

    Google Scholar 

  • Schneider, D. J. (1973). Implicit personality theory: A review.Psychological Bulletin, 79, 294–309.

    Google Scholar 

  • Schneier, E. C. (1977). Operational utility and psychometric characteristics of behavioral expectation scales: A cognitive reinterpretation.Journal of Applied Psychology, 62, 541–548.

    Google Scholar 

  • Scrull, T. K., & Wyer, R. S. (1980). Category accessibility and social perception: Some implications for the study of person memory and interpersonal judgment.Journal of Personality and Social Psychology, 38, 841–856.

    Google Scholar 

  • Sechrest, L. B. (1968). Personal constructs and personal characteristics.Journal of Individual Psychology, 24, 162–166.

    Google Scholar 

  • Smith, P. D., & Kendall, L. M. (1963). Retranslation of expectations: An approach to construction of unambiguous anchors for rating scales.Journal of Applied Psychology, 47, 149–155.

    Google Scholar 

  • Shweder, R. A., & D'Andrade, R. G. (1980). The systematic distortion hypothesis. In R. A. Shweder (Ed.), Fallible judgment in behavioral research Vol. 4. San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Wexley, K. N., & Klimoski, R. (1984). Performance appraisal: An update. In K. M. Rowland & G. R. Ferris (Eds.),Research in personnel and human resources management, 2, 35–80.

Download references

Author information

Authors and Affiliations

  1. Industrial Relations Center, University of Minnesota, 537 Management and Economics Bldg., 271 19th Avenue South, 55455, Minneapolis, MN

    Cheri Ostroff

  2. Michigan State University, USA

    Daniel R. Ilgen

Authors
  1. Cheri Ostroff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel R. Ilgen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ostroff, C., Ilgen, D.R. Cognitive categories of raters and rating accuracy. J Bus Psychol 7, 3–26 (1992). https://doi.org/10.1007/BF01014340

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01014340

Keywords

Search

Navigation