Skip to main content

The Epistemic Contract: Fostering an Appropriate Level of Public Trust in Experts

  • Chapter

Part of the Nebraska Symposium on Motivation book series (NSM,volume 62)

Abstract

Citizens should have an appropriate level of trust in experts—neither too low nor too high. Experts cannot guarantee accuracy in a complex world. But they can attempt to be well calibrated, meaning that they clearly communicate their confidence in their knowledge, and that confidence is lower in domains where their accuracy is likely to be lower. I review research on expert calibration, on the effects of confidence and calibration on perceived credibility, and on the role that “naive realism” plays in biasing our assessments of experts who say what we want to hear.

Keywords

  • Experts
  • Credibility
  • Accuracy
  • Confidence
  • Calibration
  • Biased assimilation

Prepared for the Nebraska Symposium on Motivation. I thank Brian Bornstein, John Campbell, Dan Kahan, and Saul Perlmutter for helpful conversations.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-16151-8_9
  • Chapter length: 24 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   99.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-16151-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   129.00
Price excludes VAT (USA)
Hardcover Book
USD   139.99
Price excludes VAT (USA)
Fig. 9.1
Fig. 9.2
Fig. 9.3
Fig. 9.4

Notes

  1. 1.

    Routledge Dictionary of Latin Quotations, 2004, p. 107.

  2. 2.

    http://www.imdb.com/title/tt0087332/quotes

  3. 3.

    This is quite similar to the definition given by Mayer et al. (1995, p. 712): “…the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party.” But the latter phrase seems unnecessarily restrictive; the phrase “trust but verify” suggests that we often seek to monitor those we have entrusted with a task (see Williamson, 1993).

  4. 4.

    http://www.merriam-webster.com/dictionary/expert

  5. 5.

    Ioannidis (2005) famously argues that the percentage of reported findings in the literature that are true might be quite low (anywhere from 85 % down to .15 % in his simulations) depending on various assumptions about typical statistical power, the prior probability of our hypotheses, and the nature and direction of biases in our research methods.

  6. 6.

    The adversarial motive is nonepistemic, but experts can have nonepistemic motives (e.g., to make money) without caring whether they win. In such cases, it is often their sponsor who wants to win.

  7. 7.

    A technicality: At the bottom of the confidence scale, one can never be overconfident, and at the top, one can never be underconfident—but from the consumer’s standpoint, the source is still overconfident. And overconfidence can be observed in datasets that are not vulnerable to this problem (e.g., Brenner, Koehler, Liberman, & Tversky, 1996).

  8. 8.

    When my colleagues and I published a study demonstrating why the effects of marijuana legalization on use and revenues were extremely uncertain (Kilmer, Caulkins, Pacula, MacCoun, & Reuter, 2010), we were denounced on various websites for being either useless or cowardly.

  9. 9.

    This also implies that an overconfident expert is unfairly restricting the decision maker’s zone of discretion.

  10. 10.

    Given the complexity and stochastic nature of many causal systems, I think it is probably theoretically possible for two experts, each well calibrated in the past, to be fairly confident in opposing predictions, but only under rare circumstances. In the three-dimensional space of confidence, calibration, and disagreement, that corner mostly will be empty.

  11. 11.

    Another relevant literature looks at expert testimony at trial (Cutler & Kovera, 2011).

  12. 12.

    The IEM can be found at https://tippie.uiowa.edu/iem/; the 2014 Senate trading is summarized at https://tippie.uiowa.edu/iem/media/story.cfm?ID=3389

References

  • Allum, N., Sturgis, P., Tabourazi, D., & Brunton-Smith, I. (2008). Science knowledge and attitudes across cultures: a meta-analysis. Public Understanding of Science, 17, 35–54.

    CrossRef  Google Scholar 

  • Alogna, V. K., Attaya, M. K., Aucoin, P., Bahnik, S., Birch, S., Birt, A. R., et al. (2014). Registered replication report: Schooler & Engstler-Schooler (1990). Perspectives on Psychological Science, 9, 556–578.

    CrossRef  Google Scholar 

  • Angrist, J. D., & Pischke, J. S. (2010). The credibility revolution in empirical economics: how better research design is taking the con out of econometrics. Journal of Economic Perspectives, 24, 3–30.

    CrossRef  Google Scholar 

  • Arrow, K. J., et al. (2008). The promise of prediction markets. Science, 320, 877–878.

    CrossRef  PubMed  Google Scholar 

  • Berezow, A. B., & Campbell, H. (2012). Science left behind: Feel-good fallacies and the rise of the anti-scientific left. New York, NY: PublicAffairs.

    Google Scholar 

  • Bornmann, L., & Mungra, P. (2011). Improving peer review in scholarly journals. European Science Editing, 37, 41–43.

    Google Scholar 

  • Boykoff, M. T., & Boykoff, J. M. (2004). Balance as bias: global warming and the US prestige press. Global Environmental Change, 14, 125–136.

    CrossRef  Google Scholar 

  • Bradfield, A. L., & Wells, G. L. (2000). The perceived validity of eyewitness identification testimony: A test of the five Biggers criteria. Law and Human Behavior, 24, 581–594.

    CrossRef  PubMed  Google Scholar 

  • Braun, P. A., & Yaniv, I. (1992). A case study of expert judgment: Economists’ probabilities versus base-rate model forecasts. Journal of Behavioral Decision Making, 5, 217–231.

    CrossRef  Google Scholar 

  • Braver, S. L., Thoemmes, F. J., & Rosenthal, R. (2014). Continuously cumulating meta-analysis and replicability. Perspectives on Psychological Science, 9, 333–342.

    CrossRef  Google Scholar 

  • Brenner, L. A., Koehler, D. J., Liberman, V., & Tversky, A. (1996). Overconfidence in probability and frequency judgments: A critical examination. Organizational Behavior and Human Decision Processes, 65, 212–219.

    CrossRef  Google Scholar 

  • Carlisle, J. E., Feezell, J. T., Michaud, K. E., Smith, E. R., & Smith, L. (2010). The public’s trust in scientific claims regarding offshore oil drilling. Public Understanding of Science, 19, 514–527.

    CrossRef  PubMed  Google Scholar 

  • Commons, M. L., Miller, P. M., & Gutheil, T. G. (2004). Expert witness perceptions of bias in experts. Journal of the American Academy of Psychiatry and the Law, 32, 70–75.

    PubMed  Google Scholar 

  • Cosmides, L., & Tooby, J. (1992). Cognitive adaptations for social exchange. In J. Barkow, L. Cosmides, & J. Tooby (Eds.), The adapted mind: Evolutionary psychology and the generation of culture. New York, NY: Oxford University Press.

    Google Scholar 

  • Critchley, C. R. (2008). Public opinion and trust in scientists: The role of the research context, and the perceived motivation of stem cell researchers. Public Understanding of Science, 17, 309–327.

    CrossRef  PubMed  Google Scholar 

  • Cronin, E. B., & Sugimoto, C. R. (2014). Beyond bibliometrics: Multidimensional indicators of scholarly impact. Cambridge, MA: MIT Press.

    Google Scholar 

  • Cutler, B. L., & Kovera, M. B. (2011). Expert psychological testimony. Current Directions in Psychological Science, 20, 53–57.

    CrossRef  Google Scholar 

  • Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1674.

    CrossRef  PubMed  Google Scholar 

  • Eagly, A. H., Wood, W., & Chaiken, S. (1978). Causal inferences about communicators and their effect on opinion change. Journal of Personality and Social Psychology, 36, 424–435.

    CrossRef  Google Scholar 

  • Fanelli, D., & Ioannidis, J. P. A. (2013). US studies may overestimate effect sizes in softer research. Proceedings of the National Academy of Science, 110, 15031–15036.

    CrossRef  Google Scholar 

  • Fink, W., Lipatov, V., & Konitzer, M. (2009). Diagnoses by general practitioners: Accuracy and reliability. International Journal of Forecasting, 25, 784–793.

    CrossRef  Google Scholar 

  • French, J. P. R., Jr., & Raven, B. (1960). The bases of social power. In D. Cartwright & A. Zander (Eds.), Group dynamics (pp. 607–623). New York, NY: Harper and Row.

    Google Scholar 

  • Gauchat, G. (2012). Politicization of science in the public sphere: A study of public trust in the United States, 1974 to 2010. American Sociological Review, 77, 167–187.

    CrossRef  Google Scholar 

  • Giles, J. (2002). Scientific wagers: Wanna bet? Nature, 420, 354–355.

    CrossRef  PubMed  Google Scholar 

  • Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge: Cambridge University Press.

    CrossRef  Google Scholar 

  • Goodman-Delahunty, J., Granhag, P. A., Hartwig, M., & Loftus, E. F. (2010). Insightful or wishful: Lawyers’ ability to predict case outcomes. Psychology, Public Policy, and Law, 16, 133–157.

    CrossRef  Google Scholar 

  • Grice, P. (1989). Studies in the ways of words. Cambridge MA: Harvard University Press.

    Google Scholar 

  • Henrion, M., & Fischhoff, B. (1986). Assessing uncertainty in physical constants. American Journal of Physics, 54, 791–798.

    CrossRef  Google Scholar 

  • Hmielowski, J. D., Feldman, L., Myers, T. A., Leiserowitz, A., & Maibach, E. (2014) An attack on science? Media use, trust in scientists, and perceptions of global warming. Public Understanding of Science, 23, 866–883.

    Google Scholar 

  • Hovland, C. I., Janis, I. L., & Kelley, J. J. (1953). Communication and persuasion. New Haven, CT: Yale University Press.

    Google Scholar 

  • Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, 696–701.

    Google Scholar 

  • John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524–532.

    CrossRef  PubMed  Google Scholar 

  • Jones, E. E., & Harris, V. A. (1967). The attribution of attitudes. Journal of Experimental Psychology, 3, 1–24.

    Google Scholar 

  • Kahan, D. M. (2013). Ideology, motivated reasoning, and cognitive reflection. Judgment and Decision Making, 8, 407–424.

    Google Scholar 

  • Kahan, D. M., Jenkins-Smith, H., & Braman, D. (2011). Cultural cognition of scientific consensus. Journal of Risk Research, 14, 147–174.

    CrossRef  Google Scholar 

  • Kahan, D. M., Braman, D., Slovic, P., Gastil, J., & Cohen, G. (2012). Cultural cognition of the risks and benefits of nanotechnology. Nature Nanotechnology, 4, 87–90.

    Google Scholar 

  • Kahan, D. M., Peters, E., Dawson, E. C., & Slovic, P. (2013). Motivated numeracy and enlightened self-government. New Haven, CT: Yale Law School. Public Law Working Paper No. 307. Retrieved from http://ssrn.com/abstract=2319992.

    Google Scholar 

  • Kahan, D. M., Peters, E., Wittlin, M., Slovic, P., Larrimore Oullette, L., Braman, D., et al. (2012). The polarizing impact of science literacy and numeracy on perceived climate change. Nature Climate Change, 2, 732–735.

    CrossRef  Google Scholar 

  • Kilmer, B., Caulkins, J. P., Pacula, R. L., MacCoun, R. J., & Reuter, P. H. (2010). Altered state? Assessing how marijuana legalization in California could influence marijuana consumption and public budgets. Santa Monica, CA: RAND.

    Google Scholar 

  • Kramer, R. M. (1999). Trust and distrust in organizations: Emerging perspectives, enduring questions. Annual Review of Psychology, 50, 69–98.

    CrossRef  Google Scholar 

  • Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108, 480–498.

    CrossRef  PubMed  Google Scholar 

  • Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of probabilities: the state of the art to 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.

    Google Scholar 

  • Lillienfeld, S. O. (2012). Public skepticism of psychology: Why many people perceive the study of human behavior as unscientific. American Psychologist, 67, 111–129.

    CrossRef  Google Scholar 

  • Lin, S. W., & Bier, V. M. (2008). A study of expert overconfidence. Reliability Engineering & System Safety, 93, 711–721.

    CrossRef  Google Scholar 

  • Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37, 2098–2109.

    CrossRef  Google Scholar 

  • MacCoun, R. (1998). Biases in the interpretation and use of research results. Annual Review of Psychology, 49, 259–287.

    CrossRef  PubMed  Google Scholar 

  • MacCoun, R. J., & Paletz, S. (2009). Citizens’ perceptions of ideological bias in research on public policy controversies. Political Psychology, 30, 43–65.

    CrossRef  Google Scholar 

  • MacCoun, R. J., & Perlmutter, S. (in press). Blind analysis as a correction for confirmatory bias in physics and in psychology. In Lilienfeld, S. O., & Waldman, I. (Eds.), Psychological science under scrutiny: Recent challenges and proposed solutions. John Wiley and Sons.

    Google Scholar 

  • Makridakis, S., Hogarth, R. M., & Gaba, A. (2009). Forecasting and uncertainty in the economic arena. International Journal of Forecasting, 25, 794–812.

    CrossRef  Google Scholar 

  • Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20, 709–734.

    Google Scholar 

  • McKenzie, C. R. M., Liersch, M. J., & Yaniv, I. (2008). Overconfidence in interval estimates: What does expertise buy you? Organizational Behavior and Human Decision Processes, 107, 179–191.

    CrossRef  Google Scholar 

  • Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., et al. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science, 25, 1106–1115.

    CrossRef  PubMed  Google Scholar 

  • Merton, R. K. (1938). Science and the social order. Philosophy of Science, 5, 321–337.

    CrossRef  Google Scholar 

  • Michaels, D., & Monforton, C. (2005). Manufacturing uncertainty: Contested science and the protection of the public’s health and environment. American Journal of Public Health, 95, S39–S48.

    CrossRef  PubMed  Google Scholar 

  • Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., et al. (2014). Promoting transparency in social science research. Science, 343, 30–31.

    CrossRef  PubMed Central  PubMed  Google Scholar 

  • Mooney, C. (2006). The Republican war on science. New York, NY: Basic Books.

    Google Scholar 

  • Mooney, C. (2012). The Republican brain: The science of why they deny science and reality. New York, NY: Wiley.

    Google Scholar 

  • Moore, D. A., Cain, D. M., Loewenstein, G. and Bazerman, M. (eds.). (2005). Conflicts of interest: Problems and solutions from law, medicine and organizational settings. London: Cambridge University Press.

    Google Scholar 

  • Murphy, A. H., & Winkler, R. L. (1977). Reliability of subjective probability forecasts of precipitation and temperature. Journal of the Royal Statistical Society: Applied Statistics, 26, 41–47.

    Google Scholar 

  • Murrie, D. C., Boccaccini, M. T., Guarnera, L. A., & Rufino, K. A. (2013). Are forensic experts biased by the side that retained them? Psychological Science , 24, 1889–1897.

    Google Scholar 

  • NSF. (2014). Science and engineering indicators 2014. Washington, DC: National Science Foundation.

    Google Scholar 

  • Oreskes, N., & Conway, E. M. (2010). Merchants of doubt: How a handful of scientists obscured the truth on issues from tobacco smoke to global warming. New York, NY: Bloomsbury Press.

    Google Scholar 

  • Pasher, H., & Wagenmakers, E. J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7, 528–530.

    CrossRef  Google Scholar 

  • Petty, R. E., & Cacioppo, J. T. (1986). Communication and persuasion: Central and peripheral routes to attitude change. New York, NY: Springer-Verlag.

    CrossRef  Google Scholar 

  • Pew Research. (2009). Public praises science; scientists fault public, media. Washington, DC: Pew Research Center for the People and the Press.

    Google Scholar 

  • Price, P. C., & Stone, E. R. (2004). I ntuitive evaluation of likelihood judgment producers: Evidence for a confidence heuristic. Journal of Behavioral Decision Making, 17, 39–57.

    CrossRef  Google Scholar 

  • Pronin, E., Gilovich, T., & Ross, L. (2004). Objectivity in the eye of the beholder: Divergent perceptions of bias in self vs. others. Psychological Review, 111, 781–799.

    Google Scholar 

  • Retzbach, A., & Maier, M. (2014). Communicating scientific uncertainty: Media effects on public engagement with science. Communication Research. [print version not out yet]

    Google Scholar 

  • Rotter, J. B., & Stein, D. K. (1971). Public attitudes toward the trustworthiness, competence, and altruism of twenty selected occupations. Journal of Applied Social Psychology, 1, 334–343.

    CrossRef  Google Scholar 

  • Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. (1998). Not so different after all: A cross-discipline view of trust. Academy of Management Review, 23, 393–404.

    CrossRef  Google Scholar 

  • Sah, S., Moore, D., & MacCoun, R. (2013). Cheap talk and credibility: The consequences of confidence and accuracy on advisor credibility and persuasiveness. Organizational Behavior and Human Decision Processes, 121, 246–255.

    CrossRef  Google Scholar 

  • Satterfield, T., Kandlikar, M., Beaudrie, C. E. H., Conti, J., & Harthorn, B. H. (2009). Anticipating the perceived risk of nanotechnologies. Nature Nanotechnology, 4, 752–758.

    CrossRef  PubMed  Google Scholar 

  • Silver, N. (2012). The signal and the noise. New York, NY: Penguin.

    Google Scholar 

  • Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366.

    CrossRef  PubMed  Google Scholar 

  • Simonsohn, U. (2013). Just post it: The lesson from two cases of fabricated data detected by statistics alone. Psychological Science, 24, 1875–1888.

    CrossRef  PubMed  Google Scholar 

  • Tenney, E. R., MacCoun, R. J., Spellman, B. A., & Hastie, R. (2007). Calibration trumps confidence as a basis for witness credibility. Psychological Science, 18, 46–50.

    CrossRef  PubMed  Google Scholar 

  • Tenney, E. R., Spellman, B. A., & MacCoun, R. J. (2008). The benefits of knowing what you know (and what you don’t): Fact-finders rely on others who are well calibrated. Journal of Experimental Social Psychology, 44, 1368–1375.

    CrossRef  Google Scholar 

  • Tetlock, P. E. (2005). Expert political judgment: How good is it? How can we know? Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Tetlock, P. E., Mellers, B. A., Rohrbaugh, N., & Chen, E. (2014). Forecasting tournaments: Tools for increasing transparency and improving the quality of debate. Current Directions in Psychological Science, 23, 290–295.

    CrossRef  Google Scholar 

  • Thibaut, J., & Walker, L. (1978). A theory of procedure. California Law Review, 66, 541–566.

    CrossRef  Google Scholar 

  • Twenge, J. M., Campbell, W. K., & Carter, N. T. (2014). Declines in trust in others and confidence in institutions among American adults and late adolescents, 1972−2012. Psychological Science, 25, 1914–1923.

    CrossRef  PubMed  Google Scholar 

  • Wakeman, N. (2011, February 11). IBM’s ‘Jeopardy!’ match more than game playing: Big Blue sees Watson driving strides in analytics. Washington Technology.

    Google Scholar 

  • Walster, E., Aronson, E., & Abrahams, D. (1966). On increasing the persuasiveness of a low prestige communicator. Journal of Experimental Social Psychology, 2, 325–342.

    CrossRef  Google Scholar 

  • Williamson, O. E. (1993). Calculativeness, trust, and economic organization. Journal of Law and Economics, 36, 453–486.

    CrossRef  Google Scholar 

  • Wood, W., Jones, M., & Benjamin, L. T. (1986). Surveying psychology’s public image. American Psychologist, 41, 947–953.

    CrossRef  Google Scholar 

  • Yong, E. (2012). Bad copy. Nature, 485, 298–300.

    CrossRef  PubMed  Google Scholar 

  • Zahavi, A., & Zahavi, A. (1997). The handicap principle. Oxford: Oxford University Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Robert J. MacCoun .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

MacCoun, R.J. (2015). The Epistemic Contract: Fostering an Appropriate Level of Public Trust in Experts. In: Bornstein, B., Tomkins, A. (eds) Motivating Cooperation and Compliance with Authority. Nebraska Symposium on Motivation, vol 62. Springer, Cham. https://doi.org/10.1007/978-3-319-16151-8_9

Download citation