Advertisement

Argumentation

, Volume 32, Issue 2, pp 175–195 | Cite as

Arguments from Expert Opinion and Persistent Bias

  • Moti MizrahiEmail author
Original Research

Abstract

Accounts of arguments from expert opinion take it for granted that expert judgments count as (defeasible) evidence for propositions, and so an argument that proceeds from premises about what an expert judges to a conclusion that the expert is probably right is a strong argument. In Mizrahi (Informal Log 33:57–79, 2013), I consider a potential justification for this assumption, namely, that expert judgments are significantly more likely to be true than novice judgments, and find it wanting because of empirical evidence suggesting that expert judgments under uncertainty are not significantly more likely to be true than novice judgments or even chance. In this paper, I consider another potential justification for this assumption, namely, that expert judgments are not influenced by the cognitive biases novice judgments are influenced by, and find it wanting, too, because of empirical evidence suggesting that experts are vulnerable to pretty much the same cognitive biases that novices are vulnerable to. If this is correct, then the basic assumption at the core of accounts of arguments from expert opinion, namely, that expert judgments count as (defeasible) evidence for propositions, remains unjustified.

Keywords

Arguments from expert opinion Cognitive bias Decision heuristics Expert performance Persistent bias 

Notes

Acknowledgements

I am grateful to two anonymous reviewers of Argumentation for helpful comments on an earlier draft of this paper.

References

  1. Anderson, J.R. 1983. The architecture of cognition. Cambridge, MA: Harvard University Press.Google Scholar
  2. Armenakis, A., K.W. Mossholder, and S.G. Harris. 1990. Diagnostic bias in organizational consultation. Omega: International Journal of Management Science 18: 563–572.CrossRefGoogle Scholar
  3. Bacchieri, A., and G. Della Cioppa. 2007. Fundamentals of clinical research: Bridging medicine, statistics and operations. Milano: Springer.CrossRefGoogle Scholar
  4. Blanchflower, D. 2016. Experts get it wrong again by failing to predict Trump victory. The Guardian, 9 Novemb 2016. https://www.theguardian.com/business/2016/nov/09/experts-trump-victory-economic-political-forecasters-recession.
  5. Bond, G.D. 2008. Deception detection expertise. Law and Human Behavior 32: 339–351.CrossRefGoogle Scholar
  6. Bostrom, N. 2014. Superintelligence: Paths, dangers, strategies. New York: Oxford University Press.Google Scholar
  7. Brailey, K., J.J. Vasterling, and J.J. Franks. 2001. Memory of psychodiagnostic information: Biases and effects of expertise. American Journal of Psychology 114: 55–92.CrossRefGoogle Scholar
  8. Calikli, G., and A. Bener. 2015. Empirical analysis of factors affecting confirmation bias levels of software engineers. Software Quality Journal 23: 695–722.CrossRefGoogle Scholar
  9. Canal-Bruland, R., and M. Schmidt. 2009. Response bias in judging deceptive movements. Acta Psychologica 130: 235–240.CrossRefGoogle Scholar
  10. Chen, Z., and S. Kemp. 2012. Lie hard: The effect of self-assessments on academic promotion decisions. Journal of Economic Psychology 33: 578–589.CrossRefGoogle Scholar
  11. Chung, J.S., A. Senior, O. Vinyals, and A. Zisserman. 2016. Lip reading sentences in the wild. arXiv:1611.05358 [cs.CV].
  12. Cohen, E.D. 2009. Critical thinking unleashed. Lanham, MD: Rowman & Littlefield.Google Scholar
  13. Cooper, N., and J. Frain. 2017. ABC of clinical reasoning. Hoboken, NJ: Wiley Blackwell.Google Scholar
  14. Craig, J.C., G.J. Williams, M. Jones, M. Codarini, P. Macaskill, A. Hayen, et al. 2010. The accuracy of clinical symptoms and signs for the diagnosis of serious bacterial infection in young febrile children: Prospective cohort study of 15 781 febrile illnesses. BMJ 340: c1594.CrossRefGoogle Scholar
  15. Daston, L., and P. Galison. 2007. Objectivity. New York: Zone Books.Google Scholar
  16. DeCarlo, T., T. Roy, and M. Barone. 2015. How sales manager experience and historical data trends affect decision making. European Journal of Marketing 49: 1484–1504.CrossRefGoogle Scholar
  17. Devitt, M. 2015. Relying on intuitions: Where cappelen and deutsch go wrong. Inquiry 58(7–8): 669–699.CrossRefGoogle Scholar
  18. Douglas, H. 2009. Science, policy, and the value-free ideal. Pittsburgh, PA: University of Pittsburgh Press.CrossRefGoogle Scholar
  19. Eisenstein, E.M., and L. Lodish. 2002. Marketing decision support and intelligent systems: Precisely worthwhile or vaguely worthless? In Handbook of marketing, ed. B.A. Weitz, and R. Wensley, 436–455. London: SAGE Publications.CrossRefGoogle Scholar
  20. Englich, B., T. Mussweiler, and F. Strack. 2005. The last word in court—A hidden disadvantage for the defense. Law and Human Behavior 29: 705–722.CrossRefGoogle Scholar
  21. Fahsing, I., and K. Ask. 2016. The making of an expert detective: the role of experience in English and Norwegian police officers’ investigative decision-making. Psychology, Crime and Law 22: 203–223.CrossRefGoogle Scholar
  22. Ganzach, Y. 1997. Theory and configurality in clinical judgments of expert and novice psychologists. Journal of Applied Psychology 82: 954–960.CrossRefGoogle Scholar
  23. Gegenfurtner, A., E. Lehtinen, and R. Säljö. 2011. Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review 23: 523–552.CrossRefGoogle Scholar
  24. Gelfert, A. 2011. Expertise, argumentation, and the end of inquiry. Argumentation 25: 297–312.CrossRefGoogle Scholar
  25. Goodwin, C.J. 2010. Research in psychology: Methods and design, 6th ed. Hoboken, NJ: Wiley.Google Scholar
  26. Goodwin, J. 1998. Forms of authority and the real ad verecundiam. Argumentation 12: 267–280.CrossRefGoogle Scholar
  27. Goodwin, J. 2001. Cicero’s authority. Philosophy and Rhetoric 34: 38–60.CrossRefGoogle Scholar
  28. Goodwin, J. 2011. Accounting for the appeal to the authority of experts. Argumentation 25: 285–296.CrossRefGoogle Scholar
  29. Griffin, T.D., B.D. Jee, and J. Wiley. 2009. The effects of domain knowledge on metacomprehension accuracy. Memory and Cognition 37: 1001–1013.CrossRefGoogle Scholar
  30. Griffin, D., and A. Tversky. 1992. The weighing of evidence and the determinants of confidence. Cognitive Psychology 24: 411–435.CrossRefGoogle Scholar
  31. Groopman, J. 2007. How doctors think. New York: Houghton Mifflin Co.Google Scholar
  32. Guthrie, C., J. Rachlinski, and A. Wistrich. 2001. Inside the judicial mind. Cornell Law Review 86: 777–830.Google Scholar
  33. Guthrie, C., J. Rachlinski, and A. Wistrich. 2007. Blinking on the bench: how judges decide cases. Cornell Law Review 93: 1–43.Google Scholar
  34. Guthrie, C., J. Rachlinski, and A. Wistrich. 2011. Probable cause, probability, and hindsight. Journal of Empirical Legal Studies 8: 72–98.CrossRefGoogle Scholar
  35. Herppich, S., J. Wittwer, M. Nuckles, and A. Renkl. 2013. Does it make a difference? investigating the assessment accuracy of teacher tutors and student tutors. Journal of Experimental Education 81: 242–260.CrossRefGoogle Scholar
  36. Hinds, P.J. 1999. The curse of expertise: The effects of expertise and debiasing methods on predictions of novice performance. Journal of Experimental Psychology: Applied 5: 205–221.Google Scholar
  37. Hinton, M. 2015. Mizrahi and seidel: Experts in confusion. Informal Logic 35: 539–554.CrossRefGoogle Scholar
  38. Holden, M.P., N.S. Newcombe, I. Resnick, and T.F. Shipley. 2016. Seeing like a geologist: Bayesian use of expert categories in location memory. Cognitive Science 40: 440–454.CrossRefGoogle Scholar
  39. Hooke, R. 1664. Micrographia. London. https://ceb.nlm.nih.gov/proj/ttp/flash/hooke/hooke.html.
  40. Huemer, M. 2002. Testimony. In Epistemology: Contemporary readings, ed. M. Huemer, 217–218. London: Routledge.Google Scholar
  41. IBM. 2013. Memorial Sloan-Kettering Cancer Center: IBM Watson helps fight cancer with evidence-based diagnosis and treatment suggestions. IBM Corporation, Jan 2013, http://www-935.ibm.com/services/multimedia/MSK_Case_Study_IMC14794.pdf.
  42. Jackson, S. 2015. Deference, distrust, and delegation: Three design hypotheses. In Reflections on theoretical issues in argumentation theory, ed. F.H. van Eemeren, and B. Garssen, 227–243. Cham: Springer.Google Scholar
  43. Kahneman, D. 1991. Judgment and decision making: A personal view. Psychological Science 2: 142–145.CrossRefGoogle Scholar
  44. Kaufmann, E., U. Reips, and W. Wittmann. 2013. A critical meta-analysis of lens model studies in human judgment and decision-making. PLoS ONE 8: e83528.CrossRefGoogle Scholar
  45. Kidd, J.B. 1970. The utilization of subjective probabilities in production planning. Acta Psychologica 34: 338–347.CrossRefGoogle Scholar
  46. Kitcher, P. 2001. Real realism: The Galilean strategy. Philosophical Review 110: 151–197.CrossRefGoogle Scholar
  47. Kononenko, I. 2001. Machine learning for medical diagnosis: History, state of the art and perspective. Artificial Intelligence in Medicine 23: 89–109.CrossRefGoogle Scholar
  48. Krems, J.F., and C. Zierer. 1994. Are experts immune to cognitive bias? The dependence of confirmation bias on specialist knowledge. Zeitschrift fur experimentelle und angewandte Psychologie 41: 98–115.Google Scholar
  49. Kuhn, T.S. 2000. Afterwords. In The road since structure: Philosophical essays, 1970–1993, with an autobiographical interview, ed. J. Conant, and J. Haugeland, 224–252. Chicago: The University of Chicago Press.Google Scholar
  50. Lai, M. 2015. Giraffe: Using deep reinforcement learning to play chess. arXiv:1509.01549 [cs.AI].
  51. Langenburg, G., C. Champod, and P. Wertheim. 2009. Testing for potential contextual bias effects during the verification stage of the ACE-V methodology when conducting fingerprint comparisons. Journal of Forensic Sciences 54: 571–582.CrossRefGoogle Scholar
  52. Lanzilotti, R., C. Ardito, M.F. Costabile, and A. De Angeli. 2011. Do patterns help novice evaluators? A comparative study. International Journal of Human–Computer Studies 69: 52–69.CrossRefGoogle Scholar
  53. Lash, T., M. Fox, and A. Fink. 2009. Applying quantitative bias analysis to epidemiologic data. Dordrecht: Springer.CrossRefGoogle Scholar
  54. Latour, B., and S. Wooglar. 1986. Laboratory life: The construction of scientific facts, 2nd ed. Princeton, NJ: Princeton University Press.Google Scholar
  55. Leventhal, L., B. Teasley, and D. Rohlman. 1994. Analyses of factors related to positive test bias in software testing. International Journal of Human–Computer Studies 41: 717–749.CrossRefGoogle Scholar
  56. Linker, M. 2014. Epistemic privilege and expertise in the context of meta-debate. Argumentation 28: 67–84.CrossRefGoogle Scholar
  57. Locke, J. 1975. In An essay concerning human understanding, ed. P.H. Nidditch. Oxford: Clarendon Press.Google Scholar
  58. Lusted, L. B. 1977. A study of the efficacy of diagnostic radiologic procedures: Final report on diagnostic efficacy. Committee on Efficacy Studies, American College of Radiology.Google Scholar
  59. MacLean, C.L., and I.E. Dror. 2016. A primer on the psychology of cognitive bias. In Blinding as a solution to bias, ed. C.T. Robertson, and A.S. Kesselheim, 13–24. London: Elsevier.Google Scholar
  60. Makary, M.A. 2016. Medical error—The third leading cause of death in the US. BMJ 353: i2139.CrossRefGoogle Scholar
  61. McGlone, M.S., D. Kobrynowicz, and R.B. Alexander. 2005. A certain Je ne sais quoi—Verbalization bias in evaluation. Human Communication Research 31: 241–267.Google Scholar
  62. McNeil, B.J., S.J. Pauker, H.C. Sox, and A. Tversky. 1982. On the elicitation of preferences for alternative therapies. New England Journal of Medicine 306: 1259–1262.CrossRefGoogle Scholar
  63. Mendel, R., E. Traut-Mattausch, E. Jonas, S. Leucht, J.M. Kane, K. Maino, W. Kissling, and J. Hamann. 2011. Confirmation bias: Why psychiatrists stick to wrong preliminary diagnoses. Psychological Medicine 41: 2651–2659.CrossRefGoogle Scholar
  64. Mizrahi, M. 2010. Take my advice—I am not following it: Ad Hominem arguments as legitimate rebuttals to appeals to authority. Informal Logic 30: 435–456.CrossRefGoogle Scholar
  65. Mizrahi, M. 2013. Why arguments from expert opinion are weak arguments. Informal Logic 33: 57–79.CrossRefGoogle Scholar
  66. Mizrahi, M. 2016. Why arguments from expert opinion are still weak: A reply to Seidel. Informal Logic 36: 238–252.CrossRefGoogle Scholar
  67. Mnih, V., K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wiestra, and M. Riedmiller. 2013. Playing Atari with deep reinforcement learning. arXiv:1312.5602 [cs.LG].
  68. Neale, M.A., and M.H. Bazerman. 1991. Cognition and rationality in negotiation. New York: The Free Press.Google Scholar
  69. Northcraft, G.B., and M.A. Neale. 1987. Experts, amateurs, and real estate: An anchoring-and-adjustment perspective on property pricing decisions. Organizational Behavior and Human Decision Processes 39: 84–97.CrossRefGoogle Scholar
  70. Nuzzo, R. 2015. Fooling ourselves. Nature 526: 182–195.CrossRefGoogle Scholar
  71. Oskamp, S. 1965. Overconfidence in case-study judgments. The Journal of Consulting Psychology 29: 261–265.CrossRefGoogle Scholar
  72. Perez, O. 2015. Can experts be trusted and what can be done about it? Insights from the biases and heuristics literature. In Nudge and the law: A European perspective, ed. A. Alemanno, and A. Sibony, 115–138. Oxford: Hart Publishing.Google Scholar
  73. Phillips, J.K., G. Klein, and W.R. Sieck. 2008. Expertise in judgment and decision making: a case for training intuitive decision skills. In Blackwell handbook of judgment and decision making, ed. D.J. Koehler, and N. Harvey, 297–315. Malden, MA: Blackwell Publishing.Google Scholar
  74. Posavac, S., F.R. Kardes, D.M. Sanbonmatsu, and G.J. Fitzsimons. 2005. Blissful insularity: When brands are judged in isolation from competitors. Marketing Letters 16: 87–97.CrossRefGoogle Scholar
  75. Put, K., M. Baldo, A.M. Cravo, J. Wagemans, and W.F. Helsen. 2013. Experts in offside decision making learn to compensate for their illusory perceptions. Journal of Sport and Exercise Psychology 35: 576–584.CrossRefGoogle Scholar
  76. Ramos, J.G., B. Perondi, R.D. Dias, L.C. Miranda, C. Cohen, C.R. Carvalho, I.T. Velasco, and D.N. Forte. 2016. Development of an algorithm to aid triage decisions for intensive care unit admission: A clinical vignette and retrospective cohort study. Critical Care 20: 81.CrossRefGoogle Scholar
  77. Ryle, G. 1946. Knowing how and knowing that. Proceedings of the Aristotelian Society 46: 1–16.CrossRefGoogle Scholar
  78. Schwitzgebel, E., and F. Cushman. 2012. Expertise in moral reasoning? Order effects on moral judgment in professional philosophers and non-philosophers. Mind and Language 27: 135–153.CrossRefGoogle Scholar
  79. Schwitzgebel, E., and F. Cushman. 2015. Philosophers’ biased judgments persist despite training, expertise, and reflection. Cognition 141: 127–137.CrossRefGoogle Scholar
  80. Seidel, M. 2014. Throwing the baby out with the water: From reasonably scrutinizing authorities to rampant scepticism about expertise. Informal Logic 34: 192–218.CrossRefGoogle Scholar
  81. Shaping, S., and S. Schaffer. 1985. Leviathan and the air-pump: Hobbes, Boyle, and the experimental life. Princeton, NJ: Princeton University Press.Google Scholar
  82. Shieber, J. 2009. Locke on testimony: A reexamination. History of Philosophy Quarterly 26 (1): 21–41.Google Scholar
  83. Siegel, R. 2016. 20 years later, humans still no match for computers on the chessboard. NPR, 24 Oct 2016. http://www.npr.org/sections/alltechconsidered/2016/10/24/499162905/20-years-later-humans-still-no-match-for-computers-on-the-chessboard.
  84. Staël von Holstein, C.A.S. 1972. Probabilistic forecasting: An experiment related to the stock market. Organizational Behavior and Human Performance 8: 139–158.CrossRefGoogle Scholar
  85. Ste-Marie, D.M., and T.D. Lee. 1991. Prior processing effects on gymnastic judging. Journal of Experimental Psychology. Learning, Memory, and Cognition 17: 126–136.CrossRefGoogle Scholar
  86. Tversky, A., and D. Kahneman. 1974. Judgment under uncertainty: Heuristics and biases. Science 185: 1124–1131.CrossRefGoogle Scholar
  87. Teichman, D., and E. Zamir. 2014. Judicial decision-making: A behavioral perspective. In The Oxford handbook of behavioral economics and the law, ed. E. Zamir, and D. Teichman, 664–702. New York: Oxford University Press.Google Scholar
  88. Wagemans, J.H.M. 2011. The assessment of argumentation from expert opinion. Argumentation 25: 329–339.CrossRefGoogle Scholar
  89. Wagenaar, W.A., and G. Keren. 1986. Does the expert know? The reliability of predictions and confidence ratings of experts. In Intelligent decision support in process environments, ed. E. Hollnagel, G. Maneini, and D. Woods, 87–107. Berlin: Springer.CrossRefGoogle Scholar
  90. Walton, D. 1992. The place of emotion in argument. University Park, PA: Pennsylvania State University.Google Scholar
  91. Walton, D. 2006. Examination dialogue: A framework for critically questioning an expert opinion. Journal of Pragmatics 38: 745–777.CrossRefGoogle Scholar
  92. Walton, D., C. Reed, and F. Macagno. 2008. Argumentation schemes. New York: Cambridge University Press.CrossRefGoogle Scholar
  93. Walton, D. 2014. On a razor’s edge: Evaluating arguments from expert opinion. Argument and Computation 5: 139–159.CrossRefGoogle Scholar
  94. Walton, D. N. 2016. Argument evaluation and evidence. Cham: Springer.CrossRefGoogle Scholar
  95. Waylen, A.E., M.S. Horswill, J.L. Alexander, and F.P. McKenna. 2004. Do expert drivers have a reduced illusion of superiority? Transportation Research Part F—Traffic Psychology and Behavior 7: 323–331.CrossRefGoogle Scholar
  96. Winters-Minder, L.A., P.S. Bolding, J.M. Hilbe, M. Goldstein, T. Hill, R. Nisbet, N. Walton, and G.D. Miner. 2015. Practical predictive analytics and decisioning systems for medicine: Informatics accuracy and cost-effectiveness for healthcare administration and delivery including medical research. San Diego, CA: Elsevier.Google Scholar
  97. Wöllner, C., and F.J.A. Deconinck. 2013. Gender recognition depends on type of movement and motor skill. Analyzing and perceiving biological motion in musical and nonmusical tasks. Acta Psychologica 143: 79–87.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  1. 1.Florida Institute of TechnologyMelbourneUSA

Personalised recommendations