Skip to main content
SpringerLink
Log in

The effect of implicit theories on raters' inference in performance judgment: Consequences for the validity of student ratings of instruction

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

Performance judgment is a situation of incomplete information where raters' inference would play an important role. Consequently, the schematic nature of human cognition may introduce implicit personality theory bias in performance judgment. To demonstrate this, a causal model of performance rating judgment was framed from the theories of person perception and social cognition. The model yielded a good fit to the data obtained from a performance rating task where the availability of performance information was manipulated. The results supported the hypotheses that student raters' inferences are partly contaminated by their implicit theories of a good instructor. Student raters inferred traits and behaviors and provided ratings for corresponding items even when the instructor behavior was limited to a subset of performance data only. The findings imply that one aspect of invalidity in student ratings of instructors is the bias in human inference due to the implicit theories of effective instructional behavior.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abrami, P. C., d'Apollonia, S., and Cohen, P. A. (1990). Validity of student ratings of instruction: What we know and what we do not.Journal of Educational Psychology 82(2): 219–231.

    Google Scholar 

  • Anderson, J. C., and Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach.Psychological Bulletin 103(3): 411–423.

    Google Scholar 

  • Anderson, N. H. (1981).Foundations of Information Integration Theory. New York: Academic Press.

    Google Scholar 

  • Bollen, K. A. (1989).Structural Equations with Latent Variables. New York: Wiley.

    Google Scholar 

  • Breckler, S. J. (1990). Applications of covariance structure modeling in psychology: Cause for concern.Psychological Bulletin 107(2): 260–273.

    Google Scholar 

  • Cadwell, J., and Jenkins, J. (1985). The effects of semantic similarity of items on student ratings of instructors.Journal of Educational Psychology 77(4): 383–393.

    Google Scholar 

  • Cantor, N., and Mischel, W. (1979). Prototypes in person perception. In L. Berkowitz (ed.),Advances in Experimental Social Psychology, vol. 12 (pp. 3–52). New York: Academic Press.

    Google Scholar 

  • Carlson, M., and Muliak, S. (1993). Trait ratings from descriptions of behavior as mediated by components of meaning.Multivariate Behavioral Research 28(1): 111–159.

    Google Scholar 

  • Cliff, N. (1983). Some cautions concerning the application of causal modeling methods.Multivariate Behavioral Research 18: 115–126.

    Google Scholar 

  • Cook, S. S. (1989). Improving the quality of student ratings of instruction: A look at two strategies.Research in Higher Education 30(1): 31–45.

    Google Scholar 

  • Cooper, W. H. (1981). Ubiquitous halo.Psychological Bulletin 90: 218–244.

    Google Scholar 

  • DiNisi, A. S., Cafferty, T. P., and Meglino, B. M. (1984). A cognitive view of the performance appraisal processes.Organizational Behavior and Human Performance 33: 360–396.

    Google Scholar 

  • Doyle, K. O., Jr. (1981). Validity and perplexity: An incomplete list of disturbing issues in instructional evaluation research.Instructional Evaluation 6(1): 23–25.

    Google Scholar 

  • Feldman, K. A. (1989). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers.Research in Higher Education 30(2): 137–194.

    Google Scholar 

  • Gaski, J. F. (1987). On construct validity of measures of college teaching effectiveness.Journal of Educational Psychology 79(3): 326–330.

    Google Scholar 

  • Hastie, R. (1981). Schematic principles in human memory. In E. Higgins, C. Herman, and M. Zanna (eds.),Social Cognition: The Ontario Symposium, vol. 1, (pp. 39–88). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Hayduk, L. A. (1987).Structural Equational Modelling with LISREL: Essentials and Advances. Balitmore: The John Hopkins University Press.

    Google Scholar 

  • Holland, J. H., Holyoak, K. J., Nisbett, R. E., and Thagard, P. R. (1986).Induction: Processes of Inference Learning and Discovery. Cambridge, MA: MIT Press.

    Google Scholar 

  • Howard, G. S., and Maxwell, S. E. (1982). Do grades contaminate student evaluations of instruction?Research in Higher Education 6: 175–188.

    Google Scholar 

  • Ilgen, D. R., and Feldman, J. M. (1983). Performance appraisal: A process focus. In B. M. Staw and L. Cummings (eds.),Research in Organizational Behavior, vol. 5 (pp. 141–197). Greenwich, CT: JAI Press.

    Google Scholar 

  • Jenkins, J. (1987). Implicit theories and semantic similarities: Reply to Marsh and Groves (1987).Journal of Educational Psychology 79(4): 490–493.

    Google Scholar 

  • Joreskog, K. G., and Sorbom, D. (1989).LISREL 7 User's Reference Guide. Moresville, IN: Scientific Software Inc.

    Google Scholar 

  • Kishor, N. (1990). The effect of cognitive complexity on halo in performance judgment.Journal of Personnel Evaluation in Education 3: 377–386.

    Google Scholar 

  • Kishor, N. (1992). Compensatory and noncompensatory information integration and halo error in performance rating judgments.Journal of Personnel Evaluation in Education 5: 257–268.

    Google Scholar 

  • Krzystofiak, F., Cardy, R., and Newman, J. (1988). Implicit personality and performance appraisal: The influence of trait inferences on evaluations of behavior.Journal of Applied Psychology 73(3): 515–521.

    Google Scholar 

  • Landy, F. J., and Farr, J. L. (1980). Performance rating.Psychological Bulletin 87: 72–107.

    Google Scholar 

  • Larson, J. R., Jr. (1979). The limited utility of factor analytic techniques for the study of implicit theories in student ratings of teacher behavior.American Educational Research Journal 16(2): 201–211.

    Google Scholar 

  • L'Hommedieu, R., Menges, R. J., and Brinko, K. T. (1990). Methodological explanations for the modest effects of feedback from student ratings.Journal of Educational Psychology 82(2): 232–241.

    Google Scholar 

  • Marsh, H. W. (1984). Students' evaluation of university teaching: Dimensionality, reliability, validity, potential biases, and utility.Journal of Educational Psychology 76(5): 707–754.

    Google Scholar 

  • Marsh, H. W. (1987). Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research.International Journal of Educational Research 11(3): 253–388.

    Google Scholar 

  • Marsh, H. W., and Groves, M. A. (1987). Students' evaluation of teaching effectiveness and implicit theories: A critique of Cadwell and Jenkins (1985).Journal of Educational Psychology 79(4): 483–489.

    Google Scholar 

  • McCallum, L. W. (1984). A meta-analysis of course evaluation data and its use in the tenure decision.Research in Higher Education 21: 150–158.

    Google Scholar 

  • Medley, D. (1979). The effectiveness of teachers. In P. Paterson and H. Walberg (eds.),Research on Teaching: Concepts, Findings and Implications. Berkeley, CA: McCutchan.

    Google Scholar 

  • Mook, D. J. (1983). In defense of external validity.American Psychologist 38: 379–387.

    Google Scholar 

  • Nisbett, R., and Ross, L. (1980).Human Inference: Strategies and Shortcomings of Social Judgment. Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Rotem, A., and Glasman, N. S. (1979). On the effectiveness on students' evaluative feedback to university instructors.Review of Educational Research 49: 497–511.

    Google Scholar 

  • Simon, H. (1978). Rationality as process and as product of thought.American Economic Review: Papers and Proceedings 68: 1–16.

    Google Scholar 

  • Sobel, M. E., and Bohmstedt, G. W. (1985). Use of null models in evaluating the fit of covariance structure models. In N. B. Tuma (ed.),Sociological Methodology (pp. 152–178). San Francisco: Jossey-Bass.

    Google Scholar 

  • Srull, T. K., and Wyer, R. S. (1989). Person memory and judgment.Psychological Review 96: 58–83.

    Google Scholar 

  • Sulman, L. S. (1986). Paradigms and research programs in the study of teaching: A contemporary perspective. In M. C. Wittrock (ed.).Handbook of Research on Teaching, 3rd ed. (pp. 3–37). New York: Macmillan.

    Google Scholar 

  • Taylor, S. E., and Crocker, J. (1981). Schematic bases of social information processing. In E. Higgins, C. Herman, and M. Zanna (eds.),Social Cognition: The Ontario Symposium, vol. 1 (pp. 89–134). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Trzebinski, J. (1985). Action-oriented representations of implicit personality theories.Journal of Personality and Social Psychology 48(5): 1266–1278.

    Google Scholar 

  • Trzebinski, J., McGlynn, R. P., Gray, G., and Tubbs, D. (1985). The role of categories of an actor's goals in organizing inferences about a person.Journal of Personality and Social Psychology 48(6): 1387–1397.

    Google Scholar 

  • Whitely, S. E., and Doyle, K. O. (1976). Implicit theories in student ratings.American Journal of Educational Research 13(4): 241–253.

    Google Scholar 

  • Wyer, R. S., and Martin, L. L. (1986). Person memory: The role of traits, group stereotypes and specific behaviors in the cognitive representation of persons.Journal of Personality and Social Psychology 50: 661–675.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Educational Psychology & Special Education, The University of British Columbia, 2125 Main Mall, V6T 1Z4, Vancouver

    Nand Kishor Ph.D.

Authors
  1. Nand Kishor Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kishor, N. The effect of implicit theories on raters' inference in performance judgment: Consequences for the validity of student ratings of instruction. Res High Educ 36, 177–195 (1995). https://doi.org/10.1007/BF02207787

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02207787

Keywords

Search

Navigation