Abstract
This chapter introduces the concept of empirical epistemology—the study of how humans should think based on empirical scientific studies about what actually improves accuracy in real-world contexts. It describes the origins of some pioneering research in empirical epistemology, origins that lie in work funded by the US intelligence community. It defends the presupposition that there are domain general methods of improving accuracy and that insights in a domain like geopolitics can tell us something about improving accuracy in other domains like medicine. It then outlines variables that correlate with improved accuracy. These include the following: situational variables about the environment in which one makes inferences, motivational variables about one’s motivation, cognitive variables about how one seeks out information and draws inferences from it, and metacognitive variables which concern how one assesses their own cognition. It also discusses negative lessons from empirical epistemology: that is, insights about what does not conduce to accuracy. The result is a wealth of insight about variables that can predict and improve judgmental accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
About IARPA. (n.d.). Retrieved July 2, 2020, from https://www.iarpa.gov/index.php/about-iarpa
Aggregative Contingent Estimation. (n.d.). Office of the Director of National Intelligence. Retrieved March 28, 2018, from https://www.iarpa.gov/index.php/research-programs/ace
Atanasov, P., Rescober, P., Stone, E., Swift, S. A., Servan-Schreiber, E., Tetlock, P., Ungar, L., & Mellers, B. (2017). Distilling the wisdom of crowds: Prediction markets vs. Prediction Polls. Management Science, 63(3), 691–706. https://doi.org/10.1287/mnsc.2015.2374
Atanasov, P., Witkowski, J., Ungar, L., Mellers, B., & Tetlock, P. (2020). Small steps to accuracy: Incremental belief updaters are better forecasters. Organizational Behavior and Human Decision Processes, 160, 19–35. https://doi.org/10.1016/j.obhdp.2020.02.001
Baron, J. (1993). Why teach thinking?-an essay. Applied Psychology: An International Review, 42(3), 191–237.
Baron, J., Scott, S., Fincher, K., & Metz, S. E. (2015). Why does the cognitive reflection test (sometimes) predict utilitarian moral judgment (and other things)? Journal of Applied Research in Memory and Cognition, 4(3), 265–284.
Broome, J. (2013). Rationality through reasoning. Wiley Blackwell.
Callender, A. A., Franco-Watkins, A. M., & Roberts, A. S. (2016). Improving metacognition in the classroom through instruction, training, and feedback. Metacognition and Learning, 11(2), 215–235. https://doi.org/10.1007/s11409-015-9142-6
Carnap, R. (1951). Logical foundations of probability (T. S. Gendler & J. Hawthorne, Eds.; Vol. 3). Routledge and K. Paul.
Chang, W., Atanasov, P., Patil, S., Mellers, B. A., & Tetlock, P. E. (2017). Accountability and adaptive performance under uncertainty: A long-term view. Judgment and Decision Making, 12, Issue 6.
Chater, N., & Oaksford, M. (2012). Normative systems: Logic, probability and rational choice. In The Oxford handbook of thinking and reasoning (pp. 11–21). Oxford University Press.
Dorling, J. (1979). Bayesian personalism, the methodology of scientific research programmes, and Duhem’s problem. Studies in History and Philosophy of Science Part A, 10(3), 177–187. https://doi.org/10.1016/0039-3681(79)90006-2
Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25–42.
Gilovich, T., Griffin, D. W., & Kahneman, D. (2002). Heuristics and biases: The psychology of intuitive judgment (p. 857). Cambridge University Press. https://searchworks.stanford.edu/view/4815978.
Gross, S. R. (2017). What we think, what we know and what we think we know about false convictions. Ohio State Journal of Criminal Law, 14(2), 753–786.
Haran, U., Ritov, I., & Mellers, B. (2013). The role of actively open-minded thinking in information acquisition, accuracy, and calibration. Judgment and Decision Making, 188.
Jaspan, O., Wysocka, A., Sanchez, C., & Schweitzer, A. D. (2022). Improving the relationship between confidence and competence: Implications for diagnostic radiology training from the psychology and medical literature. Academic Radiology, 29(3), 428–438. https://doi.org/10.1016/j.acra.2020.12.006
Karvetski, C. W., Meinel, C., Maxwell, D. T., Lu, Y., Mellers, B. A., & Tetlock, P. E. (2022). What do forecasting rationales reveal about thinking patterns of top geopolitical forecasters? International Journal of Forecasting, 38(2), 688–704. https://doi.org/10.1016/j.ijforecast.2021.09.003
Kyburg, H., & Teng, C. M. (2001). Uncertain inference. Cambridge University Press.
Mandel, D. R., & Irwin, D. (2021). Tracking accuracy of strategic intelligence forecasts: Findings from a long‐term Canadian study. Futures & Foresight Science, 3(3-4), e98.
Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E., & Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science, 25(5), 1106–1115. https://doi.org/10.1177/0956797614524255
Mellers, B., Stone, E., Atanasov, P., Rohrbaugh, N., Metz, S. E., Ungar, L., Bishop, M. M., Horowitz, M., Merkle, E., & Tetlock, P. (2015a). The psychology of intelligence analysis: Drivers of prediction accuracy in world politics. Journal of Experimental Psychology: Applied, 21(1), 1–14. https://doi.org/10.1037/xap0000040
Mellers, B., Stone, E., Murray, T., Minster, A., Rohrbaugh, N., Bishop, M., Chen, E., Baker, J., Hou, Y., Horowitz, M., Ungar, L., & Tetlock, P. (2015b). Identifying and cultivating Superforecasters as a method of improving probabilistic predictions. Perspectives on Psychological Science, 10(3), 267–281. https://doi.org/10.1177/1745691615577794
Mellers, B. A., Tetlock, P. E., Baker, J. D., Friedman, J. A., & Zeckhauser, R. (2019). Improving the accuracy of geopolitical risk assessments. In H. Kunreuther, R. J. Meyer, & E. O. Michel-Kerjan (Eds.), The future of risk management (pp. 209–226). University of Pennsylvania Press.
Moore, D. A., Swift, S. A., Minster, A., Mellers, B., Ungar, L., Tetlock, P., Yang, H. H. J., & Tenney, E. R. (2017). Confidence calibration in a multiyear geopolitical forecasting competition. Management Science, 63(11), 3552. https://doi.org/10.1287/mnsc.2016.2525
National Academies of Sciences, E. (2015). Improving diagnosis in health care. https://doi.org/10.17226/21794.
Pollock, J. (1990). Nomic probability and the foundations of induction. Oxford University Press.
Reichenbach, H. (1949). The theory of probability: An inquiry into the logical and mathematical foundations of the calculus of probability (2nd ed.). University of California Press.
Saenz, G. D., Geraci, L., & Tirso, R. (2019). Improving metacognition: A comparison of interventions. Applied Cognitive Psychology, 33(5), 918–929. https://doi.org/10.1002/acp.3556
Schkade, D. A., & Payne, J. W. (1994). How people respond to contingent valuation questions: A verbal protocol analysis of willingness to pay for an environmental regulation. Journal of Environmental Economics and Management, 26(1), 88–109. https://doi.org/10.1006/jeem.1994.1006
Stern, S. D. C., Cifu, A. S., & Altkorn, D. (2020). Symptom to diagnosis: An evidence-based guide (4th ed.). McGraw-Hill Medical.
Swinburne, R. (2004). The existence of god (2nd ed.). Clarendon Press.
Tetlock, P. (2005). Expert political judgment: How good is it? How can we know? Princeton University Press.
Tetlock, P., & Gardner, D. (2015). Superforecasting: The art and science of prediction. Broadway Books. https://doi.org/10.1201/b15410-25
Zwaan, L., & Hautz, W. E. (2019). Bridging the gap between uncertainty, confidence and diagnostic accuracy: Calibration is key. BMJ Quality & Safety, 28(5), 352–355. https://doi.org/10.1136/bmjqs-2018-009078
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Wilcox, J. (2022). What Correlates with Accuracy: The Empirical Epistemology of Optimal Cognition. In: Human Judgment. SpringerBriefs in Psychology. Springer, Cham. https://doi.org/10.1007/978-3-031-19205-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-19205-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19204-3
Online ISBN: 978-3-031-19205-0
eBook Packages: Behavioral Science and PsychologyBehavioral Science and Psychology (R0)