Kolmogorov Complexity in Perspective Part II: Classification, Information Processing and Duality
- 513 Downloads
We survey diverse approaches to the notion of information: from Shannon entropy to Kolmogorov complexity. Two of the main applications of Kolmogorov complexity are presented: randomness and classification. The survey is divided in two parts in the same volume.
Part II is dedicated to the relation between logic and information system, within the scope of Kolmogorov algorithmic information theory. We present a recent application of Kolmogorov complexity: classification using compression, an idea with provocative implementation by authors such as Bennett, Vitányi and Cilibrasi among others. This stresses how Kolmogorov complexity, besides being a foundation to randomness, is also related to classification. Another approach to classification is also considered: the so-called “Google classification”. It uses another original and attractive idea which is connected to the classification using compression and to Kolmogorov complexity from a conceptual point of view. We present and unify these different approaches to classification in terms of Bottom-Up versus Top-Down operational modes, of which we point the fundamental principles and the underlying duality. We look at the way these two dual modes are used in different approaches to information system, particularly the relational model for database introduced by Codd in the 1970s. These operational modes are also reinterpreted in the context of the comprehension schema of axiomatic set theory ZF. This leads us to develop how Kolmogorov’s complexity is linked to intensionality, abstraction, classification and information system.
KeywordsRelational Database Relational Schema Comprehension Schema Sequent Calculus Kolmogorov Complexity
For Francine Ptakhine, who gave me liberty of thinking and writing.
Thanks to Serge Grigorieff and Chloé Ferbus for listening, fruitful communication and for the careful proofreading and thanks to Maurice Nivat who welcomed me at the LITP in 1983.
- Bennett, C. (1988). Logical depth and physical complexity. In R. Herken (Ed.), In the universal turing machine: A half-century survey (pp. 227–257). Oxford University Press: New York.Google Scholar
- Cilibrasi, R., & Vitányi, P. (2005). Google teaches computers the meaning of words. ERCIM News, 61.Google Scholar
- Codd, E. W. (1990). The relational model for database management. Version 2. Reading: Addison-Wesley.Google Scholar
- Danchin, A. (1998). The delphic boat: What genomes tell us. Paris: Odile Jacob [Cambridge/ London: Harvard University Press, 2003.]Google Scholar
- Delahaye, J. P. (1999). Information, complexité, hasard (2nd ed.). Hermès.Google Scholar
- Delahaye, J. P. (2004). Classer musiques, langues, images, textes et génomes. Pour La Science, 316, 98–103.Google Scholar
- Delahaye, J. P. (2006). Complexités: Aux limites des mathématiques et de l’informatique. Belin-Pour la Science.Google Scholar
- Dijkstra, E. W. (1972). The humble programmer. In ACM Turing Lecture. Available on the Web from http://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html
- Dijkstra, E. W. (1982). Selected writings on computing: A personal perspective. New York: Springer.Google Scholar
- Durand, B., & Zvonkin, A. (2004–2007). Kolmogorov complexity, In E. Charpentier, A. Lesne, & N. Nikolski (Eds.), Kolmogorov’s heritage in mathematics (pp. 269–287). Berlin: Springer. (pp. 281–300, 2007).Google Scholar
- Evangelista, A., & Kjos-Hanssen, B. (2006). Google distance between words. Frontiers in Undergraduate Research, University of Connecticut.Google Scholar
- Feller, W. (1968). Introduction to probability theory and its applications (3rd ed., Vol. 1). New York: Wiley.Google Scholar
- Ferbus-Zanda, M. (1986). La méthode de résolution et le langage Prolog [The resolution method and the language Prolog], Rapport LITP, No-8676.Google Scholar
- Ferbus-Zanda, M. (In preparation-a). Logic and information system: Relational and conceptual databases.Google Scholar
- Ferbus-Zanda, M. (In preparation-b). Kolmogorov complexity and abstract states machines: The relational point of view.Google Scholar
- Ferbus-Zanda, M. (In preparation-c). Duality: Logic, computer science and Boolean algebras.Google Scholar
- Ferbus-Zanda, M. (In preparation-d). Logic and information system: Cybernetics, cognition theory and psychoanalysis.Google Scholar
- Ferbus-Zanda, M., & Grigorieff, S. (2010). ASMs and operational algorithmic completeness of lambda calculus. In N. Dershowitz & W. Reisig (Eds.), Fields of logic and computation (Lecture notes in computer science, Vol. 6300, pp. 301–327). Berlin/Heidelberg: Springer.Google Scholar
- Howard, W. (1980). The formulas-as-types notion of construction. In J. P. Seldin & J. R. Hindley (Eds.), Essays on combinatory logic, lambda calculus and formalism (pp. 479–490). London: Academic.Google Scholar
- Irving, J. (1978). The world according to garp. Modern Library. Ballantine.Google Scholar
- Kolmogorov, A. N. (1956). Grundbegriffe der Wahscheinlichkeitsrechnung [Foundations of the theory of probability]. New York: Chelsea Publishing. (Springer, 1933).Google Scholar
- Kolmogorov, A. N. (1965). Three approaches to the quantitative definition of information. Problems of Information Transmission, 1(1), 1–7.Google Scholar
- Krivine, J. L. (2003). Dependent choice, ‘quote’ and the clock. Theoretical Computer Science, 308, 259–276. See also http://www.pps.univ-paris-diderot.fr/~krivine/
- Li, M., Chen, X., Li, X., Mav, B., & Vitányi, P. (2003). The similarity metrics. In 14th ACM-SIAM 1357 symposium on discrete algorithms, Baltimore. Philadelphia: SIAMGoogle Scholar
- Martin-Löf, P. (1979, August 22–29). Constructive mathematics and computer programming. Paper read at the 6-th International Congress for Logic, Methodology and Philosophy of Science, Hannover.Google Scholar
- Shoenfield, J. (2001). Recursion theory, (Lectures notes in logic 1, New ed.). Natick: A.K. PetersGoogle Scholar
- Solomonoff, R. (1964). A formal theory of inductive inference, Part I & II. Information and control, 7, 1–22; 224–254.Google Scholar
- Véronis, J. (2005). Web: Google perd la boole. (Web: Googlean logic) Blog. January 19, 2005 from http://aixtal.blogspot.com/2005/01/web-google-perd-la-boole.html, http://aixtal.blogspot.com/2005/02/web-le-mystre-des-pages-manquantes-de.html, http://aixtal.blogspot.com/2005/03/google-5-milliards-de-sont-partis-en.html.
- Wiener, N. (1948). Cybernetics or control and communication in the animal and the machine (2nd ed.). Paris/Hermann: The Technology Press. (Cambridge: MITs, 1965).Google Scholar