Predictive Learning, Knowledge Discovery and Philosophy of Science
Various disciplines, such as machine learning, statistics, data mining and artificial neural networks, are concerned with estimation of data-analytic models. A common theme among all these methodologies is estimation of predictive models from data. In our digital age, an abundance of data and cheap computing power offers hope of knowledge discovery via application of statistical and machine learning algorithms to empirical data. This data-analytic knowledge has similarities and differences with classical scientific knowledge. For example, any scientific theory can be viewed as an inductive theory because it generalizes over a finite number of observations (or experiments). The philosophical aspects of induction and knowledge discovery have been thoroughly explored in Western philosophy of science. This philosophical analysis dates back to Kant and Hume. Any knowledge involves a combination of hypotheses/ideas and empirical data. In the modern digital age, the balance between ideas (mental constructs) and observed data (facts) has completely shifted. Classical scientific knowledge was produced mainly by a stroke of genius (e.g., Newton, Maxwell, and Einstein). In contrast, much of modern knowledge in life sciences and social sciences is derived via data-analytic modeling. We argue that such data-driven knowledge can be properly described following the methodology of predictive learning originally developed in VC-theory. This paper presents a brief survey of the philosophical concepts related to inductive inference, and then extends these ideas to predictive data-analytic knowledge discovery. We contrast the differences between classical first-principle knowledge, data-analytic knowledge and beliefs. Several application examples are used to illustrate the differences between classical statistical and predictive learning approaches to data-analytic modeling. Finally, we discuss interpretation of data-analytic models under predictive learning framework.
KeywordsSupport Vector Machine Knowledge Discovery Mutual Fund Support Vector Machine Model Empirical Knowledge
Unable to display preview. Download preview PDF.
- 1.The End of Science. Wired Magazine 16 (2007), http://www.wired.com/wired/issue/16-07
- 2.Vapnik, V.N.: Estimation of Dependencies Based on Empirical Data. In: Empirical Inference Science: Afterword of 2006. Springer, New York (2006)Google Scholar
- 4.Popper, K.: Objective Knowledge. An Evolutionary Approach. Oxford University Press (1979)Google Scholar
- 5.Popper, K.: Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge Press, London (2000)Google Scholar
- 6.Einstein, A.: Ideas and Opinions. Bonanza Books, New York (1988)Google Scholar
- 11.Cherkassky, V.: Introduction to Predictive Learning (to appear, 2012) Google Scholar
- 16.Diederich, J.: Rule Extraction from Support Vector Machines. Springer (2008)Google Scholar
- 17.Dhar, S., Cherkassky, V.: Understanding Black Box Data-Analytic Models. Neural Networks (2011) (submitted) Google Scholar
- 19.Cherkassky, V., Dhar, S.: Simple Method for Interpretation of High-Dimensional Nonlin-ear SVM Classification Models. In: The 6th International Conference on Data Mining (July 2010)Google Scholar
- 23.Frankel, T., Cunningham, L.A.: The mysterious ways of mutual funds: market timing. Annual Review of Banking and Financial Law 25(1) (2006)Google Scholar
- 24.Cherkassky, V., Dhar, S.: Market Timing of International Mutual Funds: A Decade after the Scandal. In: Proc. CIFEr (2012)Google Scholar