Current Psychology

, Volume 29, Issue 3, pp 188–209 | Cite as

Biased Evaluation of Abstracts Depending on Topic and Conclusion: Further Evidence of a Confirmation Bias Within Scientific Psychology

  • Andreas Hergovich
  • Reinhard Schott
  • Christoph Burger
Article
First Online:

Abstract

The present paper investigated whether academic psychologists show a tendency to rate the quality and appropriateness of scientific studies more favorably when results and conclusions are consistent with their own prior beliefs (i.e., confirmation bias). In an online experiment, 711 psychologists completed a questionnaire (e.g., about their belief in astrology) and evaluated research that was presented in form of a short abstract in which 40 different behaviors (e.g., alcohol consumption, willingness to share money) have been tried to be predicted. The research to be evaluated varied on three dimensions which were all manipulated between subjects: (1) the predictors of the 40 behaviors (either Big Five or astrological factors), (2) the methodological quality of the study (low, medium, high), and (3) the results and subsequent conclusion of the study (confirmation or disconfirmation of the hypotheses). Factor-analyzed scores of participants’ ratings on 8 scales, resulting in 2 factors termed quality and appropriateness, served as dependent measures. The main result of the study is a two-way interaction: Psychologists tended to evaluate results qualitatively higher when they conformed to their own prior expectations, as in this case, when astrological hypotheses were disconfirmed.

Keywords

Confirmation bias Innovation Science Psychologists Astrology 

References

  1. Armstrong, J. S. (1996). Publication of research on controversial topics: the early acceptance procedure. International Journal of Forecasting, 12, 299–302.CrossRefGoogle Scholar
  2. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale: Erlbaum.Google Scholar
  3. Edwards, W. (1968). Conservatism in human information processing. In B. Kleinmuntz (Ed.), Formal representations of human judgment (pp. 17–52). New York: Wiley.Google Scholar
  4. Epstein, W. M. (2004). Confirmational response bias and the quality of the editorial processes among American social work journals. Research on Social Work Practice, 14, 450–458.CrossRefGoogle Scholar
  5. Goodstein, L. D., & Brazis, K. L. (1970). Psychology of scientist: XXX. Credibility of psychologists: an empirical study. Psychological Reports, 27, 835–838.Google Scholar
  6. Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress? Psychological Review, 93, 216–229.CrossRefPubMedGoogle Scholar
  7. Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L. (2009). Feeling validated versus being correct: a meta-analysis of selective exposure to information. Psychological Bulletin, 135, 555–588.CrossRefPubMedGoogle Scholar
  8. Horrobin, D. F. (1990). The philosophical basis of peer review and the suppression of innovation. Journal of the American Medical Association, 263, 1438–1441.CrossRefPubMedGoogle Scholar
  9. Koehler, J. J. (1993). The influence of prior beliefs on scientific judgments of evidence quality. Organizational Behavior and Human Decision Processes, 56, 28–55.CrossRefGoogle Scholar
  10. Krems, J. F., & Zierer, C. (1994). Sind Experten gegen kognitive Täuschungen gefeit? Zur Abhängigkeit des confirmation bias von Fachwissen. [Are experts immune to cognitive bias? The dependence of confirmation bias on specialist knowledge]. Zeitschrift für Experimentelle und Angewandte Psychologie, 41, 98–115.PubMedGoogle Scholar
  11. Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37, 2098–2109.CrossRefGoogle Scholar
  12. Maccoun, R. J. (1998). Biases in the interpretation and use of research results. Annual Review of Psychology, 49, 259–87.CrossRefPubMedGoogle Scholar
  13. Mahoney, M. J. (1977). Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1, 161–175.CrossRefGoogle Scholar
  14. Moss, S., & Butler, D. C. (1978). The scientific credibility of ESP. Perceptual and Motor Skills, 46, 1063–1079.Google Scholar
  15. Nickerson, R. S. (1998). Confirmation bias: a ubiquitous phenomenon in many guises. Review of General Psychology, 2, 175–220.CrossRefGoogle Scholar
  16. Oswald, M. E., & Grosjean, S. (2004). Confirmation bias. In R. Pohl (Ed.), Cognitive illusions. A handbook on fallacies and biases in thinking, judgement and memory (pp. 79–96). Hove: Psychology Press.Google Scholar
  17. Paunonen, S. V., & Ashton, M. C. (2001). Big Five factors and facets and the prediction of behavior. Journal of Personality and Social Psychology, 81, 524–539.CrossRefPubMedGoogle Scholar
  18. Ross, L., Lepper, M. R., & Hubbard, M. (1975). Perseverance in self-perception and social perception: biased attributional processes in the debriefing paradigm. Journal of Personality and Social Psychology, 32, 880–892.CrossRefPubMedGoogle Scholar
  19. Van Ophuysen, S. (2006). Comparison of diagnostic decisions between novices and experts: the example of school career recommendation. Zeitschrift für Entwicklungspsychologie und pädagogische Psychologie, 38, 154–161.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Andreas Hergovich
    • 1
  • Reinhard Schott
    • 1
  • Christoph Burger
    • 1
  1. 1.Faculty of PsychologyUniversity of ViennaViennaAustria

Personalised recommendations