Abstract
The notion of meaningful information seems to be associated with the sweet spot between order and chaos. This form of meaningfulness of information, which is primarily what science is interested in, is not captured by both Shannon information and Kolmogorov complexity. In this paper I develop a theoretical framework that can be seen as a first approximation to a study of meaningful information. In this context I introduce the notion of facticity of a data set. I discuss the relation between thermodynamics and algorithmic complexity theory in the context of this problem. I prove that, under adequate measurement conditions, the free energy of a system in the world is associated with the randomness deficiency of a data set with observations about this system. These insights suggest an explanation of the efficiency of human intelligence in terms of helpful distributions. Finally I give a critical discussion of Schmidhuber’s views specifically his notion of low complexity art, I defend the view that artists optimize facticity instead.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Adriaans, P.W.: Using MDL for grammar induction. In: Sakakibara, Y., Kobayashi, S., Sato, K., Tomita, T.N.E. (eds.) Grammatical Inference: Algorithms and Applications, 8th International Colloquium, ICGI 2006, Tokyo, Japan. Springer, Berlin (2006)
Adriaans, P.W.: The philosophy of learning. In: Adriaans, P.W., van Benthem, J. (eds.) Handbook of the Philosophy of Information. Handbook of the Philosophy of Science, series edited by D.M. Gabbay, P. Thagard and J. Woods (2008)
Adriaans, P.W., Vitányi, P.M.B.: The power and perils of MDL. In: Proc. IEEE International Symposium on Information Theory (ISIT), Nice, France, 24–29 June, pp. 2216–2220 (2007)
Bais, F.A., Farmer, J.D.: The physics of information. In: Adriaans, P.W., van Benthem, J. (eds.) Handbook of the Philosophy of Information. Handbook of the Philosophy of Science, series edited by D.M. Gabbay, P. Thagard and J. Woods (2008)
Barron, A., Rissanen, J., Yu, B.: The minimum description length principle in coding and modeling. IEEE Trans. Inf. Theory 44(6), 2743–2760 (1998)
Chaitin, G.J.: Algorithmic Information Theory. Cambridge University Press, Cambridge (1987)
Chater, N., Vitányi, P.: Simplicity: a unifying principle in cognitive science? Trends Cogn. Sci. 7(1), 19–22 (2003)
Cilibrasi, R., Vitányi, P.: Clustering by compression. IEEE Trans. Inf. Theory 51(4), 1523–1545 (2005)
Cilibrasi, R., Vitányi, P.M.B.: Automatic meaning discovery using Google. http://www.citebase.org/abstract?id=oai:arXiv.org:cs/0412098 (2004)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (2006)
Crutchfield, J.P., Young, K.: Inferring statistical complexity. Phys. Rev. Lett. 63(2), 105–108 (1989)
Curnéjols, A., Miclet, L.: Apprentissage Artificiel, Concepts et Algorithmes. Eyrolles, Paris (2003)
Dalkilic, M.M., Clark, W.T., Costello, J.C., Radiovojac, P.: Using compression to identify classes of inauthenic texts. In: Proceedings of the 2006 SIAM Conference on Data Mining (2007). http://www.siam.org/meetings/sdm06/proceedings.htm
Domingos, P.: The role of Occam’s Razor in knowledge discovery. Data Min. Knowl. Discov. 3(4), 409–425 (1999)
Geusebroek, J.M., Smeulders, A.W.M.: A six-stimulus theory for stochastic texture. Int. J. Comput. Vis. 62(1/2), 7–16 (2005)
Grünwald, P.D.: The Minimum Description Length Principle. MIT Press, Cambridge (2007)
Grünwald, P.D., Langford, J.: Suboptimal behavior of Bayes and MDL in classification under misspecification. Mach. Learn. 66(2–3), 119–149 (2007). doi:10.1007/s10994-007-0716-7
Hume, D.: An Enquiry Concerning Human Understanding, The Harvard Classics, vol. XXXVII, Part 3. PF Collier, Toronto (1909–1914)
Hutter, M.: Universal algorithmic intelligence: a mathematical top→down approach. In: Goertzel, B., Pennachin, C. (eds.) Artificial General Intelligence. Cognitive Technologies, pp. 227–290. Springer, Berlin (2007)
Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications, 2nd edn. Springer, New York (1997)
Lloyd, S.: Ultimate physical limits to computation. Nature 406, 1047–1054 (2000)
Locke, J.: An Essay Concerning Human Understanding. Dent/Dutton, London/New York (1961)
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Ramachandran, V.S., Hirstein, W.: The science of art, a neurological theory of aesthetic experience. J. Conscious. Stud. 6(6–7), 15–51 (1999)
Schmidhuber, J.: Low-complexity art, Leonardo. J. Int. Soc. Arts Sci. Technol. 30(2), 97–103 (1997)
Schmidhuber, J.: Completely self-referential optimal reinforcement learners. In: Duch, W., (eds.) Proceedings of the International Conference on Artificial Neural Networks ICANN’05. LNCS, vol. 3697, pp. 223–233. Springer, Berlin (2005)
Schmidhuber, J.: Simple algorithmic principles of discovery, subjective beauty, selective attention, curiosity and creativity. In: V. Corruble, M. Takeda, E. Suzuki (eds.) DS 2007. LNAI, vol. 4755, pp. 26–38 (2007)
Solomonoff, R.J.: The discovery of algorithmic probability. J. Comput. Syst. Sci. 55(1), 73–88 (1997)
Vereshchagin, N.K., Vitányi, P.M.B.: Kolmogorov’s structure functions and model selection. IEEE Trans. Inf. Theory 50(12), 3265–3290 (2004)
Wolff, J.G.: Unifying Computing and Cognition, the SP Theory and Its Applications. CognitionResearch.org.uk, Menai Bridge (2006)
Wolfram, S.: A New Kind of Science. Wolfram Media, Champaign (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
This project is supported by a BSIK grant from the Dutch Ministry of Education, Culture and Science (OC&W) and is part of the ICT innovation program of the Ministry of Economic Affairs (EZ).
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Adriaans, P. Between Order and Chaos: The Quest for Meaningful Information. Theory Comput Syst 45, 650–674 (2009). https://doi.org/10.1007/s00224-009-9173-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00224-009-9173-y