Publication prejudices: An experimental study of confirmatory bias in the peer review system

Abstract

Confirmatory bias is the tendency to emphasize and believe experiences which support one's views and to ignore or discredit those which do not. The effects of this tendency have been repeatedly documented in clinical research. However, its ramifications for the behavior of scientists have yet to be adequately explored. For example, although publication is a critical element in determining the contribution and impact of scientific findings, little research attention has been devoted to the variables operative in journal review policies. In the present study, 75 journal reviewers were asked to referee manuscripts which described identical experimental procedures but which reported positive, negative, mixed, or no results. In addition to showing poor interrater agreement, reviewers were strongly biased against manuscripts which reported results contrary to their theoretical perspective. The implications of these findings for epistemology and the peer review system are briefly addressed.

This is a preview of subscription content, log in to check access.

References

  1. Adams, J. A.Human memory. New York: McGraw-Hill, 1967.

    Google Scholar 

  2. American Psychological Association.Standards for educational and psychological tests and manuals. Washington, D.C.: American Psychological Association, 1966.

    Google Scholar 

  3. Bacon, F.Novum organum. New York: Bobbs-Merrill, 1960. (Originally published, 1621.)

    Google Scholar 

  4. Beck, A. T.Cognitive therapy and the emotional disorders. New York: International Universities Press, 1976.

    Google Scholar 

  5. Caplow, T., & McGee, R. J.The academic marketplace. New York: Basic Books, 1958.

    Google Scholar 

  6. Cole, J. R., & Cole, S.Social stratification in science. Chicago: University of Chicago Press, 1973.

    Google Scholar 

  7. Deci, E. L. Effects of externally mediated rewards on intrinsic motivation.Journal of Personality and Social Psychology 1971,18 105–115.

    Google Scholar 

  8. Deci, E. L. Intrinsic motivation, extrinsic reinforcement, and inequity.Journal of Personality and Social Psychology 1972,22 113–130.

    Google Scholar 

  9. Dixon, B.What is science for? New York: Harper & Row, 1973.

    Google Scholar 

  10. Hagstrom, W. O.The scientific community. New York: Basic Books, 1965.

    Google Scholar 

  11. Kintsch, W.Learning, memory, and conceptual processes. New York: Wiley, 1970.

    Google Scholar 

  12. Lakatos, I., & Musgrave, A. (Eds.),Criticism and the growth of knowledge. London: Cambridge University Press, 1970.

    Google Scholar 

  13. Lepper, M. R., Greene, D., & Nisbett, R. E. Undermining children's intrinsic interest with extrinsic reward: A test of the “overjustification” hypothesis.Journal of Personality and Social Psychology 1973,28 129–137.

    Google Scholar 

  14. Levine, F. M., & Fasnacht, G. Token rewards may lead to token learning.American Psychologist 1974,29 814–820.

    Google Scholar 

  15. Mahoney, M. J.Cognition and behavior modification. Cambridge, Massachusetts: Ballinger, 1974.

    Google Scholar 

  16. Mahoney, M. J.Scientist as subject: The psychological imperative. Cambridge, Massachusetts: Ballinger, 1976.

    Google Scholar 

  17. Mahoney, M. J., & DeMonbreun, B. G. Confirmatory bias in scientists and non-scientists.Cognitive Therapy and Research, 1977 (in press).

  18. Mahoney, M. J., Kazdin, A. E., & Kenigsberg, M.Getting published: The effects of self-citation and institutional affiliation. Unpublished manuscript, Pennsylvania State University, 1975.

  19. Mahoney, M. J., & Kimper, T. P. From ethics to logic: A survey of scientists. In M. J. Mahoney,Scientist as subject. Cambridge, Massachusetts: Ballinger, 1976. Pp. 187–193.

    Google Scholar 

  20. Merton, R. K.Social theory and social structure. New York: Free Press, 1968.

    Google Scholar 

  21. Neisser, V.Cognitive psychology. New York: Appleton-Century-Crofts, 1967.

    Google Scholar 

  22. Norman, D. A.Memory and attention. New York: Wiley, 1969.

    Google Scholar 

  23. Popper, K. R.Objective knowledge: An evaluative approach. London: Oxford University Press, 1972.

    Google Scholar 

  24. Raimy, V.Misunderstandings of the self. San Francisco: Jossey-Bass, 1975.

    Google Scholar 

  25. Weimer, W. B.Psychology and the conceptual foundations of science. Hillsdale, New Jersey; Lawrence Erlbaum Associates, 1977.

    Google Scholar 

  26. Ziman, J.Public knowledge: The social dimension of science. London: Cambridge University Press, 1968.

    Google Scholar 

  27. Zuckerman, H. Stratification in American science.Sociological Inquiry 1970,40 235–257.

    Google Scholar 

  28. Zuckerman, H., & Merton, R. K. Patterns of evaluation in science: Institutionalization, structure and functions of the referee system.Minerva 1971,9 66–100.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mahoney, M.J. Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cogn Ther Res 1, 161–175 (1977). https://doi.org/10.1007/BF01173636

Download citation

Keywords

  • Experimental Study
  • Review System
  • Cognitive Psychology
  • Clinical Research
  • Theoretical Perspective