Skip to main content
Springer Nature Link
Log in

Arguments from Expert Opinion and Persistent Bias

  • Original Research
  • Published:
Argumentation Aims and scope Submit manuscript

Abstract

Accounts of arguments from expert opinion take it for granted that expert judgments count as (defeasible) evidence for propositions, and so an argument that proceeds from premises about what an expert judges to a conclusion that the expert is probably right is a strong argument. In Mizrahi (Informal Log 33:57–79, 2013), I consider a potential justification for this assumption, namely, that expert judgments are significantly more likely to be true than novice judgments, and find it wanting because of empirical evidence suggesting that expert judgments under uncertainty are not significantly more likely to be true than novice judgments or even chance. In this paper, I consider another potential justification for this assumption, namely, that expert judgments are not influenced by the cognitive biases novice judgments are influenced by, and find it wanting, too, because of empirical evidence suggesting that experts are vulnerable to pretty much the same cognitive biases that novices are vulnerable to. If this is correct, then the basic assumption at the core of accounts of arguments from expert opinion, namely, that expert judgments count as (defeasible) evidence for propositions, remains unjustified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Notes

  1. See also Walton (2016), pp. 117–144.

  2. Cf. Jackson (2015), pp. 227–243.

  3. See also Linker (2014) and Gelfert (2011) who take the “authority” of expert opinion for granted.

  4. It is important to note that the kind of authority in question here is what Walton (1992, p. 48) calls “cognitive authority,” i.e., the kind of authority that “looks for the auditor to take what is stated as true and believe it” (Goodwin 1998, pp. 268–269), as opposed to what Walton (1992, p. 48) calls “administrative authority,” i.e., the kind of authority that “looks for the auditor to take what is commanded and make it true” (Goodwin 1998, pp. 268–269). See also Mizrahi (2010). This distinction will become important later on in Sect. 3.

  5. See, e.g., this article in The Guardian on how “Experts get it wrong again by failing to predict Trump victory” (Blanchflower 2016): https://www.theguardian.com/business/2016/nov/09/experts-trump-victory-economic-political-forecasters-recession.

  6. For more on cognitive biases in clinical decision-making, see Groopman (2007).

  7. Assistant referees are not better at making offside decisions than amateur soccer players are, but the former do take more time to make their decisions than the latter do. The researchers think that this result shows that assistant referees make offside decisions on a cognitive level, as opposed to a merely perceptual level (Put et al 2013). In general, however, studies show that “[i]ntelligence and experience do not make a person immune to cognitive biases” (Cooper and Frain 2017, p. 26). Accordingly, even if the fact that assistant referees take more time to make offside decisions than amateur soccer players do suggests that they think about it more carefully, it doesn’t necessarily follow that they are immune to cognitive illusions, as opposed to perceptual illusions, which is what this study focused on.

  8. An alternative hypothesis is that it is not expertise per se that makes (medical) experts less susceptible to confirmation bias than novices but rather awareness of the problem of cognitive bias itself. In a feature article in Nature, Nuzzo (2015) quotes Robert MacCoun who says that “When crises like this issue of reproducibility come along, it’s a good opportunity to advance our scientific tools.” For example, “when scientists in the mid-twentieth century realized that experimenters and subjects often unconsciously changed their behaviour to match expectations” […] the double-blind standard was born” (p. 183). If this is correct, then it’s not being an expert in a particular field of study that makes one less susceptible to confirmation bias, but rather it’s knowing about confirmation biases themselves and how they can affect one’s judgments.

  9. See also Anderson (1983) for a distinction between “declarative knowledge” and “procedural knowledge,” and Ryle (1946) for a distinction between “knowing that” and “knowing how”.

  10. Cf. Wagemans (2011, p. 337): “E is able to provide further evidence for O”.

  11. For more on Locke on testimony, see Shieber (2009).

  12. For more on neural networks outperforming human players in games, see Mnih et al (2013).

  13. For more on scientific instruments as “enlarging the empire of the senses,” see Shaping and Schaffer (1985), Ch. 2.

  14. See Kitcher (2001) on how Galileo argued for the reliability of the telescope as an instrument of celestial observation.

References

  • Anderson, J.R. 1983. The architecture of cognition. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Armenakis, A., K.W. Mossholder, and S.G. Harris. 1990. Diagnostic bias in organizational consultation. Omega: International Journal of Management Science 18: 563–572.

    Article  Google Scholar 

  • Bacchieri, A., and G. Della Cioppa. 2007. Fundamentals of clinical research: Bridging medicine, statistics and operations. Milano: Springer.

    Book  Google Scholar 

  • Blanchflower, D. 2016. Experts get it wrong again by failing to predict Trump victory. The Guardian, 9 Novemb 2016. https://www.theguardian.com/business/2016/nov/09/experts-trump-victory-economic-political-forecasters-recession.

  • Bond, G.D. 2008. Deception detection expertise. Law and Human Behavior 32: 339–351.

    Article  Google Scholar 

  • Bostrom, N. 2014. Superintelligence: Paths, dangers, strategies. New York: Oxford University Press.

    Google Scholar 

  • Brailey, K., J.J. Vasterling, and J.J. Franks. 2001. Memory of psychodiagnostic information: Biases and effects of expertise. American Journal of Psychology 114: 55–92.

    Article  Google Scholar 

  • Calikli, G., and A. Bener. 2015. Empirical analysis of factors affecting confirmation bias levels of software engineers. Software Quality Journal 23: 695–722.

    Article  Google Scholar 

  • Canal-Bruland, R., and M. Schmidt. 2009. Response bias in judging deceptive movements. Acta Psychologica 130: 235–240.

    Article  Google Scholar 

  • Chen, Z., and S. Kemp. 2012. Lie hard: The effect of self-assessments on academic promotion decisions. Journal of Economic Psychology 33: 578–589.

    Article  Google Scholar 

  • Chung, J.S., A. Senior, O. Vinyals, and A. Zisserman. 2016. Lip reading sentences in the wild. arXiv:1611.05358 [cs.CV].

  • Cohen, E.D. 2009. Critical thinking unleashed. Lanham, MD: Rowman & Littlefield.

    Google Scholar 

  • Cooper, N., and J. Frain. 2017. ABC of clinical reasoning. Hoboken, NJ: Wiley Blackwell.

    Google Scholar 

  • Craig, J.C., G.J. Williams, M. Jones, M. Codarini, P. Macaskill, A. Hayen, et al. 2010. The accuracy of clinical symptoms and signs for the diagnosis of serious bacterial infection in young febrile children: Prospective cohort study of 15 781 febrile illnesses. BMJ 340: c1594.

    Article  Google Scholar 

  • Daston, L., and P. Galison. 2007. Objectivity. New York: Zone Books.

    Google Scholar 

  • DeCarlo, T., T. Roy, and M. Barone. 2015. How sales manager experience and historical data trends affect decision making. European Journal of Marketing 49: 1484–1504.

    Article  Google Scholar 

  • Devitt, M. 2015. Relying on intuitions: Where cappelen and deutsch go wrong. Inquiry 58(7–8): 669–699.

    Article  Google Scholar 

  • Douglas, H. 2009. Science, policy, and the value-free ideal. Pittsburgh, PA: University of Pittsburgh Press.

    Book  Google Scholar 

  • Eisenstein, E.M., and L. Lodish. 2002. Marketing decision support and intelligent systems: Precisely worthwhile or vaguely worthless? In Handbook of marketing, ed. B.A. Weitz, and R. Wensley, 436–455. London: SAGE Publications.

    Chapter  Google Scholar 

  • Englich, B., T. Mussweiler, and F. Strack. 2005. The last word in court—A hidden disadvantage for the defense. Law and Human Behavior 29: 705–722.

    Article  Google Scholar 

  • Fahsing, I., and K. Ask. 2016. The making of an expert detective: the role of experience in English and Norwegian police officers’ investigative decision-making. Psychology, Crime and Law 22: 203–223.

    Article  Google Scholar 

  • Ganzach, Y. 1997. Theory and configurality in clinical judgments of expert and novice psychologists. Journal of Applied Psychology 82: 954–960.

    Article  Google Scholar 

  • Gegenfurtner, A., E. Lehtinen, and R. Säljö. 2011. Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review 23: 523–552.

    Article  Google Scholar 

  • Gelfert, A. 2011. Expertise, argumentation, and the end of inquiry. Argumentation 25: 297–312.

    Article  Google Scholar 

  • Goodwin, C.J. 2010. Research in psychology: Methods and design, 6th ed. Hoboken, NJ: Wiley.

    Google Scholar 

  • Goodwin, J. 1998. Forms of authority and the real ad verecundiam. Argumentation 12: 267–280.

    Article  Google Scholar 

  • Goodwin, J. 2001. Cicero’s authority. Philosophy and Rhetoric 34: 38–60.

    Article  Google Scholar 

  • Goodwin, J. 2011. Accounting for the appeal to the authority of experts. Argumentation 25: 285–296.

    Article  Google Scholar 

  • Griffin, T.D., B.D. Jee, and J. Wiley. 2009. The effects of domain knowledge on metacomprehension accuracy. Memory and Cognition 37: 1001–1013.

    Article  Google Scholar 

  • Griffin, D., and A. Tversky. 1992. The weighing of evidence and the determinants of confidence. Cognitive Psychology 24: 411–435.

    Article  Google Scholar 

  • Groopman, J. 2007. How doctors think. New York: Houghton Mifflin Co.

    Google Scholar 

  • Guthrie, C., J. Rachlinski, and A. Wistrich. 2001. Inside the judicial mind. Cornell Law Review 86: 777–830.

    Google Scholar 

  • Guthrie, C., J. Rachlinski, and A. Wistrich. 2007. Blinking on the bench: how judges decide cases. Cornell Law Review 93: 1–43.

    Google Scholar 

  • Guthrie, C., J. Rachlinski, and A. Wistrich. 2011. Probable cause, probability, and hindsight. Journal of Empirical Legal Studies 8: 72–98.

    Article  Google Scholar 

  • Herppich, S., J. Wittwer, M. Nuckles, and A. Renkl. 2013. Does it make a difference? investigating the assessment accuracy of teacher tutors and student tutors. Journal of Experimental Education 81: 242–260.

    Article  Google Scholar 

  • Hinds, P.J. 1999. The curse of expertise: The effects of expertise and debiasing methods on predictions of novice performance. Journal of Experimental Psychology: Applied 5: 205–221.

    Google Scholar 

  • Hinton, M. 2015. Mizrahi and seidel: Experts in confusion. Informal Logic 35: 539–554.

    Article  Google Scholar 

  • Holden, M.P., N.S. Newcombe, I. Resnick, and T.F. Shipley. 2016. Seeing like a geologist: Bayesian use of expert categories in location memory. Cognitive Science 40: 440–454.

    Article  Google Scholar 

  • Hooke, R. 1664. Micrographia. London. https://ceb.nlm.nih.gov/proj/ttp/flash/hooke/hooke.html.

  • Huemer, M. 2002. Testimony. In Epistemology: Contemporary readings, ed. M. Huemer, 217–218. London: Routledge.

    Google Scholar 

  • IBM. 2013. Memorial Sloan-Kettering Cancer Center: IBM Watson helps fight cancer with evidence-based diagnosis and treatment suggestions. IBM Corporation, Jan 2013, http://www-935.ibm.com/services/multimedia/MSK_Case_Study_IMC14794.pdf.

  • Jackson, S. 2015. Deference, distrust, and delegation: Three design hypotheses. In Reflections on theoretical issues in argumentation theory, ed. F.H. van Eemeren, and B. Garssen, 227–243. Cham: Springer.

    Google Scholar 

  • Kahneman, D. 1991. Judgment and decision making: A personal view. Psychological Science 2: 142–145.

    Article  Google Scholar 

  • Kaufmann, E., U. Reips, and W. Wittmann. 2013. A critical meta-analysis of lens model studies in human judgment and decision-making. PLoS ONE 8: e83528.

    Article  Google Scholar 

  • Kidd, J.B. 1970. The utilization of subjective probabilities in production planning. Acta Psychologica 34: 338–347.

    Article  Google Scholar 

  • Kitcher, P. 2001. Real realism: The Galilean strategy. Philosophical Review 110: 151–197.

    Article  Google Scholar 

  • Kononenko, I. 2001. Machine learning for medical diagnosis: History, state of the art and perspective. Artificial Intelligence in Medicine 23: 89–109.

    Article  Google Scholar 

  • Krems, J.F., and C. Zierer. 1994. Are experts immune to cognitive bias? The dependence of confirmation bias on specialist knowledge. Zeitschrift fur experimentelle und angewandte Psychologie 41: 98–115.

    Google Scholar 

  • Kuhn, T.S. 2000. Afterwords. In The road since structure: Philosophical essays, 1970–1993, with an autobiographical interview, ed. J. Conant, and J. Haugeland, 224–252. Chicago: The University of Chicago Press.

    Google Scholar 

  • Lai, M. 2015. Giraffe: Using deep reinforcement learning to play chess. arXiv:1509.01549 [cs.AI].

  • Langenburg, G., C. Champod, and P. Wertheim. 2009. Testing for potential contextual bias effects during the verification stage of the ACE-V methodology when conducting fingerprint comparisons. Journal of Forensic Sciences 54: 571–582.

    Article  Google Scholar 

  • Lanzilotti, R., C. Ardito, M.F. Costabile, and A. De Angeli. 2011. Do patterns help novice evaluators? A comparative study. International Journal of Human–Computer Studies 69: 52–69.

    Article  Google Scholar 

  • Lash, T., M. Fox, and A. Fink. 2009. Applying quantitative bias analysis to epidemiologic data. Dordrecht: Springer.

    Book  Google Scholar 

  • Latour, B., and S. Wooglar. 1986. Laboratory life: The construction of scientific facts, 2nd ed. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Leventhal, L., B. Teasley, and D. Rohlman. 1994. Analyses of factors related to positive test bias in software testing. International Journal of Human–Computer Studies 41: 717–749.

    Article  Google Scholar 

  • Linker, M. 2014. Epistemic privilege and expertise in the context of meta-debate. Argumentation 28: 67–84.

    Article  Google Scholar 

  • Locke, J. 1975. In An essay concerning human understanding, ed. P.H. Nidditch. Oxford: Clarendon Press.

  • Lusted, L. B. 1977. A study of the efficacy of diagnostic radiologic procedures: Final report on diagnostic efficacy. Committee on Efficacy Studies, American College of Radiology.

  • MacLean, C.L., and I.E. Dror. 2016. A primer on the psychology of cognitive bias. In Blinding as a solution to bias, ed. C.T. Robertson, and A.S. Kesselheim, 13–24. London: Elsevier.

    Google Scholar 

  • Makary, M.A. 2016. Medical error—The third leading cause of death in the US. BMJ 353: i2139.

    Article  Google Scholar 

  • McGlone, M.S., D. Kobrynowicz, and R.B. Alexander. 2005. A certain Je ne sais quoi—Verbalization bias in evaluation. Human Communication Research 31: 241–267.

    Google Scholar 

  • McNeil, B.J., S.J. Pauker, H.C. Sox, and A. Tversky. 1982. On the elicitation of preferences for alternative therapies. New England Journal of Medicine 306: 1259–1262.

    Article  Google Scholar 

  • Mendel, R., E. Traut-Mattausch, E. Jonas, S. Leucht, J.M. Kane, K. Maino, W. Kissling, and J. Hamann. 2011. Confirmation bias: Why psychiatrists stick to wrong preliminary diagnoses. Psychological Medicine 41: 2651–2659.

    Article  Google Scholar 

  • Mizrahi, M. 2010. Take my advice—I am not following it: Ad Hominem arguments as legitimate rebuttals to appeals to authority. Informal Logic 30: 435–456.

    Article  Google Scholar 

  • Mizrahi, M. 2013. Why arguments from expert opinion are weak arguments. Informal Logic 33: 57–79.

    Article  Google Scholar 

  • Mizrahi, M. 2016. Why arguments from expert opinion are still weak: A reply to Seidel. Informal Logic 36: 238–252.

    Article  Google Scholar 

  • Mnih, V., K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wiestra, and M. Riedmiller. 2013. Playing Atari with deep reinforcement learning. arXiv:1312.5602 [cs.LG].

  • Neale, M.A., and M.H. Bazerman. 1991. Cognition and rationality in negotiation. New York: The Free Press.

    Google Scholar 

  • Northcraft, G.B., and M.A. Neale. 1987. Experts, amateurs, and real estate: An anchoring-and-adjustment perspective on property pricing decisions. Organizational Behavior and Human Decision Processes 39: 84–97.

    Article  Google Scholar 

  • Nuzzo, R. 2015. Fooling ourselves. Nature 526: 182–195.

    Article  Google Scholar 

  • Oskamp, S. 1965. Overconfidence in case-study judgments. The Journal of Consulting Psychology 29: 261–265.

    Article  Google Scholar 

  • Perez, O. 2015. Can experts be trusted and what can be done about it? Insights from the biases and heuristics literature. In Nudge and the law: A European perspective, ed. A. Alemanno, and A. Sibony, 115–138. Oxford: Hart Publishing.

    Google Scholar 

  • Phillips, J.K., G. Klein, and W.R. Sieck. 2008. Expertise in judgment and decision making: a case for training intuitive decision skills. In Blackwell handbook of judgment and decision making, ed. D.J. Koehler, and N. Harvey, 297–315. Malden, MA: Blackwell Publishing.

    Google Scholar 

  • Posavac, S., F.R. Kardes, D.M. Sanbonmatsu, and G.J. Fitzsimons. 2005. Blissful insularity: When brands are judged in isolation from competitors. Marketing Letters 16: 87–97.

    Article  Google Scholar 

  • Put, K., M. Baldo, A.M. Cravo, J. Wagemans, and W.F. Helsen. 2013. Experts in offside decision making learn to compensate for their illusory perceptions. Journal of Sport and Exercise Psychology 35: 576–584.

    Article  Google Scholar 

  • Ramos, J.G., B. Perondi, R.D. Dias, L.C. Miranda, C. Cohen, C.R. Carvalho, I.T. Velasco, and D.N. Forte. 2016. Development of an algorithm to aid triage decisions for intensive care unit admission: A clinical vignette and retrospective cohort study. Critical Care 20: 81.

    Article  Google Scholar 

  • Ryle, G. 1946. Knowing how and knowing that. Proceedings of the Aristotelian Society 46: 1–16.

    Article  Google Scholar 

  • Schwitzgebel, E., and F. Cushman. 2012. Expertise in moral reasoning? Order effects on moral judgment in professional philosophers and non-philosophers. Mind and Language 27: 135–153.

    Article  Google Scholar 

  • Schwitzgebel, E., and F. Cushman. 2015. Philosophers’ biased judgments persist despite training, expertise, and reflection. Cognition 141: 127–137.

    Article  Google Scholar 

  • Seidel, M. 2014. Throwing the baby out with the water: From reasonably scrutinizing authorities to rampant scepticism about expertise. Informal Logic 34: 192–218.

    Article  Google Scholar 

  • Shaping, S., and S. Schaffer. 1985. Leviathan and the air-pump: Hobbes, Boyle, and the experimental life. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Shieber, J. 2009. Locke on testimony: A reexamination. History of Philosophy Quarterly 26 (1): 21–41.

    Google Scholar 

  • Siegel, R. 2016. 20 years later, humans still no match for computers on the chessboard. NPR, 24 Oct 2016. http://www.npr.org/sections/alltechconsidered/2016/10/24/499162905/20-years-later-humans-still-no-match-for-computers-on-the-chessboard.

  • Staël von Holstein, C.A.S. 1972. Probabilistic forecasting: An experiment related to the stock market. Organizational Behavior and Human Performance 8: 139–158.

    Article  Google Scholar 

  • Ste-Marie, D.M., and T.D. Lee. 1991. Prior processing effects on gymnastic judging. Journal of Experimental Psychology. Learning, Memory, and Cognition 17: 126–136.

    Article  Google Scholar 

  • Tversky, A., and D. Kahneman. 1974. Judgment under uncertainty: Heuristics and biases. Science 185: 1124–1131.

    Article  Google Scholar 

  • Teichman, D., and E. Zamir. 2014. Judicial decision-making: A behavioral perspective. In The Oxford handbook of behavioral economics and the law, ed. E. Zamir, and D. Teichman, 664–702. New York: Oxford University Press.

    Google Scholar 

  • Wagemans, J.H.M. 2011. The assessment of argumentation from expert opinion. Argumentation 25: 329–339.

    Article  Google Scholar 

  • Wagenaar, W.A., and G. Keren. 1986. Does the expert know? The reliability of predictions and confidence ratings of experts. In Intelligent decision support in process environments, ed. E. Hollnagel, G. Maneini, and D. Woods, 87–107. Berlin: Springer.

    Chapter  Google Scholar 

  • Walton, D. 1992. The place of emotion in argument. University Park, PA: Pennsylvania State University.

    Google Scholar 

  • Walton, D. 2006. Examination dialogue: A framework for critically questioning an expert opinion. Journal of Pragmatics 38: 745–777.

    Article  Google Scholar 

  • Walton, D., C. Reed, and F. Macagno. 2008. Argumentation schemes. New York: Cambridge University Press.

    Book  Google Scholar 

  • Walton, D. 2014. On a razor’s edge: Evaluating arguments from expert opinion. Argument and Computation 5: 139–159.

    Article  Google Scholar 

  • Walton, D. N. 2016. Argument evaluation and evidence. Cham: Springer.

    Book  Google Scholar 

  • Waylen, A.E., M.S. Horswill, J.L. Alexander, and F.P. McKenna. 2004. Do expert drivers have a reduced illusion of superiority? Transportation Research Part F—Traffic Psychology and Behavior 7: 323–331.

    Article  Google Scholar 

  • Winters-Minder, L.A., P.S. Bolding, J.M. Hilbe, M. Goldstein, T. Hill, R. Nisbet, N. Walton, and G.D. Miner. 2015. Practical predictive analytics and decisioning systems for medicine: Informatics accuracy and cost-effectiveness for healthcare administration and delivery including medical research. San Diego, CA: Elsevier.

    Google Scholar 

  • Wöllner, C., and F.J.A. Deconinck. 2013. Gender recognition depends on type of movement and motor skill. Analyzing and perceiving biological motion in musical and nonmusical tasks. Acta Psychologica 143: 79–87.

    Article  Google Scholar 

Download references

Acknowledgements

I am grateful to two anonymous reviewers of Argumentation for helpful comments on an earlier draft of this paper.

Author information

Authors and Affiliations

  1. Florida Institute of Technology, 150 W. University Blvd., Melbourne, FL, 32901, USA

    Moti Mizrahi

Authors
  1. Moti Mizrahi
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Moti Mizrahi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mizrahi, M. Arguments from Expert Opinion and Persistent Bias. Argumentation 32, 175–195 (2018). https://doi.org/10.1007/s10503-017-9434-x

Download citation

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1007/s10503-017-9434-x

Keywords

Profiles

  1. Moti Mizrahi View author profile