Abstract
Certain characteristics make machine learning (ML) a powerful tool for processing large amounts of data, and also particularly unsuitable for explanatory purposes. There are worries that its increasing use in science may sideline the explanatory goals of research. We analyze the key characteristics of ML that might have implications for the future directions in scientific research: epistemic opacity and the ‘theory-agnostic’ modeling. These characteristics are further analyzed in a comparison of ML with the traditional statistical methods, in order to demonstrate what it is specifically that makes ML methodology substantially unsuitable for reaching explanations. The analysis is given broader philosophical context by connecting it with the views on the role of prediction and explanation in science, their relationship, and the value of explanation. We proceed to show, first, that ML disrupts the relationship between prediction and explanation commonly understood as a functional relationship. Then we show that the value of explanation is not exhausted in purely predictive functions, but rather has a ubiquitously recognized value for both science and everyday life. We then invoke two hypothetical scenarios with different degrees of automatization of science, which help test our intuitions on the role of explanation in science. The main question we address is whether ML will reorient or otherwise impact our standard explanatory practice. We conclude with a prognosis that ML would diversify science into purely predictively oriented research based on ML-like techniques and, on the other hand, remaining faithful to anthropocentric research focused on the search for explanation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Not applicable.
Code Availability
Not applicable.
Notes
When discussing ML, we will take ANNs as the paradigmatic, and most popular type of model which implements machine learning, although our conclusions can, if needed, be (more or less successfully) transferred to other similar methods.
A similar attitude is reflected even in the everyday context. There is empirical evidence suggesting that there are consistent expectations of citing causes in everyday explanations (Miller, 2017).
The xAI literature is not always clear on whether the explanandum is the whole process or only its outcome. Since not understanding the process leads to not understanding the outcome, i.e., the decision, they are tightly connected and thus often addressed jointly. However, depending on the context, the ML process can be considered either as an explanandum, or it could be cited in an explanans of a particular outcome. Such considerations are irrelevant for our present purposes. Here we will refer only to the ML process as an explanandum which is lacking an explanans.
Of course, experts in the fields who participate in the design or debugging of ML models understand the general mechanism through which these models work, and in that sense, to them these models are more appropriately described as ‘grey’ rather than black boxes. However, there are aspects of the functioning of ML models that even the experts in the field do not have complete access to, and these aspects concern the paths through which the model proceeds from input to output. For example, even simple devices like calculators can be non-transparent to the end-users in the sense that they do not know how the device is made nor can they mentally follow a mathematical operation because it is too complicated. However, when we get a certain output from an input using a calculator, every step leading from that particular input to that particular output is known and can be explicated. In the case of ML, the path leading from the input to the output is not available. This is based on the fact that the internal logic of the model is altered as it ‘learns’ from the data, and it is what separates ML from other technological tools (Burrell, 2016).
By complexity we refer to a general notion of complexity of a method (which is based on a great number of layers, operations, and data involved in the ML processes), not the technical concept of computational complexity which could be measured in various ways (e.g. Kolmogorov complexity).
It is intractable to test all paths in software systems, which contain more than about 108 paths (which is about 300 lines of code), and it should be noted that most software systems used in science are much larger than that (Symons and Horner, 2014).
The development of various xAI tools should, in principle, alleviate the model’s opacity to a degree, since it would hopefully lead to more transparent processes, and full or partial detection of correlations (Zednik, 2021). This, of course, would be helpful only providing that the calculations of the xAI techniques are based on the same parameters as the ML models, which they are meant to explain, and not only on the input–output trends as pointed out by some more pessimistic views (Rudin, 2019). Since the development of xAI tools is a fast-growing area because of the numerous ethical considerations (see Sect. 2), such explanatory tools could be a thing of the not-too-distant future. The potential solution which the xAI techniques could provide is, unfortunately, somewhat limited, since it addresses only the obstacles posed by the lack of epistemic access. However, it could be the first step in extracting knowledge on causal relations from the ML model. To address other obstacles such as the mismatch between the explanatory and predictive modeling, other paths and techniques would need to be devised.
The massive datasets ML systems learn from are often referred to by the term ‘big data’, meaning “rapidly collected, complex data in such unprecedented quantities that terabytes (1012 bytes), petabytes (1015 bytes) or even zettabytes (1021 bytes) of storage may be required” (Wyber et al., 2015; Leonelli, 2020).
On the other hand, Trout argues that the close connection between the explanations and understanding should not be taken for granted. He argues that since a sense of understanding could be the result of various cognitive biases, we sometimes become overconfident in our explanations as a result of the biases that lead to understanding (Trout, 2002). By such a view, we should not consider that explanation and understanding are so closely tied. We understand Trout's suggestion as a warning that understanding should not be the only measure of the value of explanation because our sense of understanding can be misleading. Thus, if we relied only on understanding as the source of the value of an explanation, many misinterpretations would seem more plausible than some other correct but, say, more complex explanations. However, even if a sense of understanding should not be the only measure of the value of an explanation, as suggested by Trout, understanding that an explanation brings, nevertheless, makes a significant contribution to its value.
Discussing intrinsic value needs more caution than seems present in the literature. Some of the views assign intrinsic value to explanations and proceed to describe it in terms of some other value such as understanding (e.g. Hempel 1962; Strevens 2008; Lombrozo 2011). We do not consider these as proper intrinsic-value views. For explanation to qualify as intrinsically valuable, the value needs to be assigned to explanation per se, regardless of any relations it bears to other related concepts. Other views, which take explanations as bona fide intrinsically valuable, consider them as rewarding in themselves, and “may be sought out as such, even by the youngest of children, with no further agenda” (Keil, 2006, p. 234; See also Gopnik, 1998). If such views are correct, then it seems unlikely that the value of explanation could be impacted by some non-explanatory tool such as ML, since its value is not dependent on its relations to anything else. However, the already existing discussion on the impact of predictive methods such as ML on the explanatory practice in science, as well as the major historical discussions on the value of explanation and its relationship with prediction seem to be based on the belief that such views are not quite convincing, and we will proceed the inquiry in the directions implicated by these discussions.
We thank the anonymous reviewer for pointing out this possibility to us.
Admittedly, we needed to tweak the specifications of the Laplace’s demon to make a proper analogy with ML: the fact that at least some parts of his internal processes are available to us, and that he does not conclude by knowing the laws of nature (as in the original), but by capturing correlations among the data. We chose to do this in contrasting the Laplacean demon to a clairvoyant in order to focus primarily on what we found important for the epistemological analysis of the justification of ML predictions. Namely, the exact ways through which the predictions are reached—the fact that this method is designed by scientists according to a sound methodology, rather than through some kind of ‘magic’ or clairvoyance—emphasizes an aspect of ML which is important for justifying the method itself.
References
Berber, A., & Sreckovic, S. (2021). Inherent ethical problems of machine learning. Unpublished manuscript, Faculty of Philosophy, Belgrade University, Belgrade, Serbia.
Bien-Willner, G. A. (2015). Biomarkers and cancer: Correlation is not causation. Clinical OMICs, 2(1), 14–15. https://doi.org/10.1089/clinomi.02.01.06
Bird, A. (2011). Philosophy of Science and Epistemology. In S. French & J. Saatsi (Eds.), Continuum Companion to the Philosophy of Science (pp. 15–32). London: Continuum.
Bizzarri, M., Brash, D. E., Briscoe, J., Grieneisen, V. A., Stern, C. D., & Levin, M. (2019). A call for a better understanding of causation in cell biology. Nature Reviews Molecular Cell Biology, 20, 261–262. https://doi.org/10.1038/s41580-019-0127-1
Boge, F. J., & Grünke, P. (2019). Computer simulations, machine learning and the laplacean demon: Opacity in the case of high energy physics. In Kaminski, Resch, & Gehring (Eds.), The Science and Art of Simulation II, Springer.
Boge, F. J., & Poznic, M. (2021). Machine learning and the future of scientific explanation. Journal for General Philosophy of Science/Zeitschrift Für Allgemeine Wissenschaftstheorie, 52(1), 171–176. https://doi.org/10.1007/s10838-020-09537-z
BonJour, L. (1980). Externalist theories of empirical knowledge. Midwest Studies in Philosophy, 5, 53–73. https://doi.org/10.1111/j.1475-4975.1980.tb00396.x
Boon, M. (2020). How Scientists Are Brought Back into Science—The Error of Empiricism. In A Critical Reflection on Automated Science, Marta Bertolaso and Fabio Sterpetti (eds.), (Human Perspectives in Health Sciences and Technology 1) (pp. 43–65). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-25001-0_4
Breiman, L. (2001). Statistical modeling: The two cultures. Statistical Science, 16(3), 199–231.
Burrell, J. (2016). How the Machine ‘Thinks’: Understanding Opacity in Machine Learning Algorithms. Big Data & Society, 3(1), 1–12.
Bzdok, D. (2017). Classical statistics and statistical learning in imaging neuroscience. Frontiers in Neuroscience, 11, 543. https://doi.org/10.3389/fnins.2017.00543
Bzdok, D., Altman, N., & Krzywinski, M. (2018). Statistics versus machine learning. Nature Methods, 15, 233–234. https://doi.org/10.1038/nmeth.4642
Bzdok, D., & Yeo, T. B. T. (2017). Inference in the age of big data: Future perspectives on neuroscience. NeuroImage, 155, 549–564. https://doi.org/10.1016/j.neuroimage.2017.04.061
Calude, C. S., & Longo, G. (2017). The deluge of spurious correlations in big data. Foundations of Science, 22(3), 595–612.
Chouinard, M. M., Harris, P. L., & Maratsos, M. P. (2007). Children’s questions: A mechanism for cognitive development. Monographs of the Society for Research in Child Development, 72(1), 1–129.
Coeckelbergh, M. (2020). Artificial intelligence, responsibility attribution, and a relational justification of explainability. Science and Engineering Ethics, 26, 2051–2068. https://doi.org/10.1007/s11948-019-00146-8
Colombo, M., & Weinberger, N. (2018). Discovering brain mechanisms using network analysis and causal modeling. Minds & Machines, 28, 265–286. https://doi.org/10.1007/s11023-017-9447-0
De Regt, H. W., & Dieks, D. (2005). A Contextual approach to scientific understanding. Synthese, 144, 137–170.
Douglas, H. (2009). Reintroducing prediction to explanation. Philosophy of Science, 76(4), 444–463. https://doi.org/10.1086/648111
Floridi, L., Cowls, J., Beltrametti, M., et al. (2018). AI4People—An ethical framework for a good ai society: opportunities, risks, principles, and recommendations. Minds & Machines, 28, 689–707. https://doi.org/10.1007/s11023-018-9482-5
Frazier, B. N., Gelman, S. A., & Wellman, H. M. (2009). Preschoolers’ search for explanatory information within adult-child conversation. Child Development, 80(6), 1592–1611. https://doi.org/10.1111/j.1467-8624.2009.01356.x
Frazier, B. N., Gelman, S. A., & Wellman, H. M. (2016). Young children prefer and remember satisfying explanations. Official Journal of the Cognitive Development Society, 17(5), 718–736. https://doi.org/10.1080/15248372.2015.1098649
Geer, D. E., Jr. (2011). Correlation is not causation. IEEE Security & Privacy Magazine, 9(2), 93–94. https://doi.org/10.1109/msp.2011.26
Gilpin, L. H., Bau, D., Yuan, B. Z., et al. (2018). Explaining explanations: An overview of interpretability of machine learning. In Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) (pp.80–89). Turin, New York: IEEE.
Goldman, A., & Beddor, B. (2021). Reliabilist Epistemology. In Edward N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/archives/sum2021/entries/reliabilism/
Goodman B., & Flaxman S. (2017). EU Regulations on Algorithmic Decision-Making and a ‘Right to Explanation’. AI Magazine, 38(3). https://arxiv.org/abs/1606.08813
Gopnik, A. (1998). Explanation as orgasm. Minds and Machines, 8, 101–118. https://doi.org/10.1023/A:1008290415597
Gopnik, A., & Meltzoff, A. N. (1996). Words, thoughts and theories. Bradford, MIT Press.
Gorgoni, G. (2020). Stay Human. The quest for responsibility in the algorithmic society. Journal of Ethics and Legal Technologies, 2(1). https://doi.org/10.14658/pupj-jelt-2020-1-2.
Hanson, N. R. (1959). On the symmetry between explanation and prediction. Philosophical Review, 68, 349–358.
Heider, F. (1958). The psychology of interpersonal relations. Wiley.
Hempel, C. (1962). Explanation in science and in history. In R. G. Colodny (Ed.), Frontiers of Science and Philosophy (pp. 7–33). Pittsburgh, PA: University of Pittsburgh Press, 7–33.
Hempel, C. G., & Oppenheim, P. (1948). Studies in the logic of explanation. Philosophy of Science, 15, 135–175.
Hickling, A. K., & Wellman, H. M. (2001). The emergence of children’s causal explanations and theories: evidence from everyday conversation. Developmental Psychology, 37(5), 668.
Hoffman, R., Mueller, S., Klein, G., & Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. https://arxiv.org/abs/1812.04608.
Hofstadter, A. (1951). Explanation and necessity. Philosophy and Phenomenological Research, 11, 339–347.
Humphreys, P. (2009). The philosophical novelty of computer simulation methods. Synthese, 169, 615–626.
Imai, K., Tingley, D., & Yamamoto, T. (2013). Experimental designs for identifying causal mechanisms. Journal of the Royal Statistical Society. Series A (Statistics in Society), 176(1), 5–32. http://www.jstor.org/stable/23355175
Ivanova, M. (2013). Steven french and juha saatsi: The continuum companion to the philosophy of science. Science & Education, 22, 2363–2367. https://doi.org/10.1007/s11191-013-9591-8
Keenan, S. E., & Shvartsman, S. Y. (2017). Mechanisms and causality in molecular diseases. History and Philosophy of the Life Sciences, 39(4), 35. https://doi.org/10.1007/s40656-017-0162-1
Keil, F. C. (2006). Explanation and understanding. Annual Review of Psychology, 57, 227–254. https://doi.org/10.1146/annurev.psych.57.102904.190100
Laplace, P. S. (1814). Philosophical Essay of Probabilities, translated by Andrew Dale. (1999) New York: Springer.
Legare, C. H. (2012). Exploring explanation: explaining inconsistent evidence informs exploratory, hypothesis-testing behavior in young children. Child Development, 83(1), 173–185. https://doi.org/10.1111/j.1467-8624.2011.01691.x
Leonelli, S. (2020). Scientific research and big data. In Edward N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/archives/sum2020/entries/science-big-data/
Lipton, Z. C. (2016). The Mythos of Model Interpretability. In: 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016). New York. https://arxiv.org/abs/1606.03490
Lipton, P. (2009). Understanding without explanation. In H. W. de Regt, S. Leonelli, & K. Eigner (Eds.), Scientific understanding: Philosophical perspectives (pp. 43–63). University of Pittsburgh Press.
Liquin, E., & Lombrozo, T. (2018). Determinants and Consequences of the Need for Explanation. In T. T. Rogers, M. Rau, X. Zhu, & C. W. Kalish (Ed.), Proceedings of the 40th Annual Conference of the Cognitive Science Society (pp. 696–701). Austin, TX: Cognitive Science Society.
Liquin, E. G., Metz, S. E., & Lombrozo, T. (2020). Science Demands Explanation, Religion Tolerates Mystery. Cognition, 204, 104398. https://doi.org/10.1016/j.cognition.2020.104398
Lombrozo, T. (2006). The structure and function of explanations. Trends in Cognitive Sciences, 10(10), 464–470. https://doi.org/10.1016/j.tics.2006.08.004
Lombrozo, T. (2011). The instrumental value of explanations. Philosophy Compass, 6(8), 539551.
Lombrozo, T. (2016). Explanatory preferences shape learning and inference. Trends in Cognitive Sciences, 20(10), 748–759. https://doi.org/10.1016/j.tics.2016.08.001
Lombrozo, T., & Carey, S. (2006). Functional explanation and the function of explanation. Cognition, 99, 167–204. https://doi.org/10.1016/j.cognition.2004.12.009
Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, 6(3), 175–183.
Miller, T. (2017). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267, 1–38.
Mittelstadt, B., Russell, C., & Wachter, S. (2019). Explaining Explanations in AI. In FAT 2019: Conference on Fairness, Accountability, and Transparency. Atlanta, GA.
Money, J. (1991). Correlation is not causation. Behavioral and Brain Sciences, 14(2), 275–275. https://doi.org/10.1017/S0140525X00066632
Nagel, E. (1961). The structure of science: Problems in the logic of scientific explanation. Harcourt, Brace & World.
Napoletani, D., Panza, M., & Struppa, D. C. (2014). Is big data enough? A reflection on the changing role of mathematics in applications. Notices of the American Mathematical Society, 61(5), 485–490. https://doi.org/10.1090/noti1102
Páez, A. (2019). The pragmatic turn in explainable artificial intelligence (XAI). Minds & Machines, 29, 441–459.
Pott, L. M. (2008). Correlation is not causation. Academic Medicine: Journal of the Association of American Medical Colleges, 83(12), 1123. https://doi.org/10.1097/ACM.0b013e31818c749a
Quine, W. V. O., & Ullian, J. S. (1978). The web of belief. McGraw-Hill.
Raita, Y., Camargo, C. A., Jr., Liang, L., & Hasegawa, K. (2021). Leveraging “Big Data” in respiratory medicine – Data science, causal inference, and precision medicine. Expert Review of Respiratory Medicine. https://doi.org/10.1080/17476348.2021.1913061
Reichenbach, H. (1938). Experience and Prediction. University of Chicago Press.
Resch, M., & Kaminski, A. (2019). The epistemic importance of technology in computer simulation and machine learning. Minds and Machines, 29(1), 9–17.
Reutlinger, A., Colyvan, M., & Krzyżanowska, K. (2020). The prospects for a monist theory of non-causal explanation in science and mathematics. Erkenntnis. https://doi.org/10.1007/s10670-020-00273-w
Ribera, M., & Lapedriza, A. (2019). Can we do better explanations? A proposal of user-centered explainable AI. Presented at Explainable Smart Systems Conference 2019, Los Angeles.
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1, 206–215. https://doi.org/10.1038/s42256-019-0048-x.
Russo F. (2015). Causation and Correlation in Medical Science: Theoretical problems. In T. Schramme, S. Edwards (eds.), Handbook of the Philosophy of Medicine. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-8706-2_46-1.
Salmon, W. (1978). Why ask, ‘Why?’? An inquiry concerning scientific explanation. Proceedings and Addresses of the American Philosophical Association, 51, 683–705.
Salmon, W. (1999). The spirit of logical empiricism: Carl g. Hempel’s role in twentieth-century philosophy of science. Philosophy of Science, 66, 333–350.
Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., & Müller, K.-R. (eds.) (2019). Explainable AI: Interpreting, explaining and visualizing deep learning. LNCS, vol. 11700. Springer, Cham.
Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3), 289–310. https://doi.org/10.1214/10-STS330
Stone, L. (2015). PDE5i and Melanoma – Correlation is not causation. Nature Reviews Urology, 12(8), 418–418. https://doi.org/10.1038/nrurol.2015.170
Strevens, M. (2008). Depth: An account of scientific explanation. Harvard University Press.
Symons, J., & Horner, J. (2014). Software intensive science. Philosophy and Technology, 27(3), 461–477. https://doi.org/10.1007/s13347-014-0163-x
Trout, J. D. (2002). Scientific explanation and the sense of understanding. Philosophy of Science, 69(2), 212–233. https://doi.org/10.1086/341050
UNI Global Union. (2018). 10 Principles for Ethical AI. UNI Global Union, www.thefutureworldofwork.org/opinions/10-principles-for-ethical-ai/. Accessed 10 April 2019.
Van Camp, W. (2014). Explaining understanding (or understanding explanation). European Journal for Philosophy of Science, 4, 95–114. https://doi.org/10.1007/s13194-013-0077-y
Wachter, S., Mittelstadt, B., & Russell, C. (2017). Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology, 31(2), http://arxiv.org/abs/1711.00399
Wachter, S., Mittelstadt, B., & Floridi, L. (2016). Why a right to explanation of automated decision-making does not exist in the general data protection regulation. International Data Privacy Law, 7(2), 76–99. https://doi.org/10.1093/idpl/ipx005
Wellman, H. M., & Lagattuta, K. H. (2004). Theory of mind for learning and teaching: The nature and role of explanation. Cognitive Development, 19(4), 479–497. https://doi.org/10.1016/j.cogdev.2004.09.003
Willard, A. K., Busch, J., Cullum, K. A., Letourneau, S. M., Sobel, D. M., Callanan, M., & Legare, C. H. (2019). Explain this, explore that: A study of parent-child interaction in a children’s museum. Child Development, 90(5), 598–617. https://doi.org/10.1111/cdev.13232
Wilson, R. A., & Keil, F. (1998). The shadows and shallows of explanation. Minds and Machines, 8(1), 137–159.
Woodward, J. (2021). Scientific explanation. In Edward N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/archives/spr2021/entries/scientific-explanation.
Wyber, R., Vaillancourt, S., Perry, W., Mannava, P., Folaranmi, T., & Celi, L. A. (2015). Big data in global health: improving health in low- and middle-income countries. Bulletin of the World Health Organization, 93, 203–208.
Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: lessons from machine learning. Perspectives on Psychological Science, 12(6), 11001–11122. https://doi.org/10.1177/1745691617693393
Zednik, C. (2021). Explainable AI as a tool for scientific exploration. Presented online on 21.04.2021. Online Seminars on the Foundations and Ethics of AI in Lugano.
Zednik, C. (2019). Solving the black box problem: A normative framework for explainable artificial intelligence. Philosophy & Technology, 34, 265–288. https://doi.org/10.1007/s13347-019-00382-7
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
Not applicable.
Corresponding author
Ethics declarations
Conflicts of interest
The author declares that they have no conflict of interest.
Ethical Approval
Not applicable.
Consent to Participate
Not applicable.
Consent for publication (include appropriate statements)
Not applicable.
Humans and/or animals consent
Not applicable
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Srećković, S., Berber, A. & Filipović, N. The Automated Laplacean Demon: How ML Challenges Our Views on Prediction and Explanation. Minds & Machines 32, 159–183 (2022). https://doi.org/10.1007/s11023-021-09575-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11023-021-09575-6