Kolmogorov Complexity in Perspective Part II: Classification, Information Processing and Duality

  • Marie Ferbus-ZandaEmail author
Part of the Logic, Epistemology, and the Unity of Science book series (LEUS, volume 34)


We survey diverse approaches to the notion of information: from Shannon entropy to Kolmogorov complexity. Two of the main applications of Kolmogorov complexity are presented: randomness and classification. The survey is divided in two parts in the same volume.

Part II is dedicated to the relation between logic and information system, within the scope of Kolmogorov algorithmic information theory. We present a recent application of Kolmogorov complexity: classification using compression, an idea with provocative implementation by authors such as Bennett, Vitányi and Cilibrasi among others. This stresses how Kolmogorov complexity, besides being a foundation to randomness, is also related to classification. Another approach to classification is also considered: the so-called “Google classification”. It uses another original and attractive idea which is connected to the classification using compression and to Kolmogorov complexity from a conceptual point of view. We present and unify these different approaches to classification in terms of Bottom-Up versus Top-Down operational modes, of which we point the fundamental principles and the underlying duality. We look at the way these two dual modes are used in different approaches to information system, particularly the relational model for database introduced by Codd in the 1970s. These operational modes are also reinterpreted in the context of the comprehension schema of axiomatic set theory ZF. This leads us to develop how Kolmogorov’s complexity is linked to intensionality, abstraction, classification and information system.


Relational Database Relational Schema Comprehension Schema Sequent Calculus Kolmogorov Complexity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



For Francine Ptakhine, who gave me liberty of thinking and writing.

Thanks to Serge Grigorieff and Chloé Ferbus for listening, fruitful communication and for the careful proofreading and thanks to Maurice Nivat who welcomed me at the LITP in 1983.


  1. Bennett, C. (1988). Logical depth and physical complexity. In R. Herken (Ed.), In the universal turing machine: A half-century survey (pp. 227–257). Oxford University Press: New York.Google Scholar
  2. Bennett, C., Gács, P., Li, M., Vitányi, P., & Zurek, W. (1998). Information distance. IEEE Transactions on Information Theory, 44(4), 1407–1423.CrossRefGoogle Scholar
  3. Chaitin, G. (1966). On the length of programs for computing finite binary sequences. Journal of the ACM, 13, 547–569.CrossRefGoogle Scholar
  4. Chaitin, G. (1969). On the length of programs for computing finite binary sequences: Statistical considerations. Journal of the ACM, 16, 145–159.CrossRefGoogle Scholar
  5. Chaitin, G. (1975). A theory of program size formally identical to information theory. Journal of the ACM, 22, 329–340.CrossRefGoogle Scholar
  6. Chen, P. S. (1976). The entity-relationship model: Toward a unified view of data. ACM Transactions on Database Systems, 1(1), 9–36.CrossRefGoogle Scholar
  7. Cilibrasi, R. (2003). Clustering by compression. IEEE Transactions on Information Theory, 51(4), 1523–1545.CrossRefGoogle Scholar
  8. Cilibrasi, R., & Vitányi, P. (2005). Google teaches computers the meaning of words. ERCIM News, 61.Google Scholar
  9. Cilibrasi, R., & Vitányi, P. (2007). The Google similarity distance. IEEE Transactions on Knowledge and Data Engineering, 19(3), 370–383.CrossRefGoogle Scholar
  10. Codd, E. W. (1970). A relational model of data for large shared databanks. Communications of the ACM, 13(6), 377–387.CrossRefGoogle Scholar
  11. Codd, E. W. (1990). The relational model for database management. Version 2. Reading: Addison-Wesley.Google Scholar
  12. Danchin, A. (1998). The delphic boat: What genomes tell us. Paris: Odile Jacob [Cambridge/ London: Harvard University Press, 2003.]Google Scholar
  13. Delahaye, J. P. (1999). Information, complexité, hasard (2nd ed.). Hermès.Google Scholar
  14. Delahaye, J. P. (2004). Classer musiques, langues, images, textes et génomes. Pour La Science, 316, 98–103.Google Scholar
  15. Delahaye, J. P. (2006). Complexités: Aux limites des mathématiques et de l’informatique. Belin-Pour la Science.Google Scholar
  16. Dershowitz, N., & Gurevich, Y. (2008). A natural axiomatization of computability and proof of Church’s thesis. The Bulletin of Symbolic Logic, 14(3), 299–350.CrossRefGoogle Scholar
  17. Dijkstra, E. W. (1972). The humble programmer. In ACM Turing Lecture. Available on the Web from
  18. Dijkstra, E. W. (1982). Selected writings on computing: A personal perspective. New York: Springer.Google Scholar
  19. Durand, B., & Zvonkin, A. (2004–2007). Kolmogorov complexity, In E. Charpentier, A. Lesne, & N. Nikolski (Eds.), Kolmogorov’s heritage in mathematics (pp. 269–287). Berlin: Springer. (pp. 281–300, 2007).Google Scholar
  20. Evangelista, A., & Kjos-Hanssen, B. (2006). Google distance between words. Frontiers in Undergraduate Research, University of Connecticut.Google Scholar
  21. Feller, W. (1968). Introduction to probability theory and its applications (3rd ed., Vol. 1). New York: Wiley.Google Scholar
  22. Ferbus-Zanda, M. (1986). La méthode de résolution et le langage Prolog [The resolution method and the language Prolog], Rapport LITP, No-8676.Google Scholar
  23. Ferbus-Zanda, M. (In preparation-a). Logic and information system: Relational and conceptual databases.Google Scholar
  24. Ferbus-Zanda, M. (In preparation-b). Kolmogorov complexity and abstract states machines: The relational point of view.Google Scholar
  25. Ferbus-Zanda, M. (In preparation-c). Duality: Logic, computer science and Boolean algebras.Google Scholar
  26. Ferbus-Zanda, M. (In preparation-d). Logic and information system: Cybernetics, cognition theory and psychoanalysis.Google Scholar
  27. Ferbus-Zanda, M., & Grigorieff, S. (2004). Is randomness native to computer science? In G. Paun, G. Rozenberg, & A. Salomaa (Eds.), Current trends in theoretical computer science (pp. 141–179). River Edge: World Scientific.CrossRefGoogle Scholar
  28. Ferbus-Zanda, M., & Grigorieff, S. (2006). Kolmogorov complexity and set theoretical representations of integers. Mathematical Logic Quarterly, 52(4), 381–409.CrossRefGoogle Scholar
  29. Ferbus-Zanda, M., & Grigorieff, S. (2010). ASMs and operational algorithmic completeness of lambda calculus. In N. Dershowitz & W. Reisig (Eds.), Fields of logic and computation (Lecture notes in computer science, Vol. 6300, pp. 301–327). Berlin/Heidelberg: Springer.Google Scholar
  30. Howard, W. (1980). The formulas-as-types notion of construction. In J. P. Seldin & J. R. Hindley (Eds.), Essays on combinatory logic, lambda calculus and formalism (pp. 479–490). London: Academic.Google Scholar
  31. Irving, J. (1978). The world according to garp. Modern Library. Ballantine.Google Scholar
  32. Kolmogorov, A. N. (1956). Grundbegriffe der Wahscheinlichkeitsrechnung [Foundations of the theory of probability]. New York: Chelsea Publishing. (Springer, 1933).Google Scholar
  33. Kolmogorov, A. N. (1965). Three approaches to the quantitative definition of information. Problems of Information Transmission, 1(1), 1–7.Google Scholar
  34. Kolmogorov, A. N. (1983). Combinatorial foundation of information theory and the calculus of probability. Russian Mathematical Surveys, 38(4), 29–40.CrossRefGoogle Scholar
  35. Krivine, J. L. (2003). Dependent choice, ‘quote’ and the clock. Theoretical Computer Science, 308, 259–276. See also
  36. Li, M., & Vitányi, P. (1997). An introduction to Kolmogorov complexity and its applications (2nd ed.). New York: Springer.CrossRefGoogle Scholar
  37. Li, M., Chen, X., Li, X., Mav, B., & Vitányi, P. (2003). The similarity metrics. In 14th ACM-SIAM 1357 symposium on discrete algorithms, Baltimore. Philadelphia: SIAMGoogle Scholar
  38. Martin-Löf, P. (1979, August 22–29). Constructive mathematics and computer programming. Paper read at the 6-th International Congress for Logic, Methodology and Philosophy of Science, Hannover.Google Scholar
  39. Shannon, C. E. (1948). The mathematical theory of communication. Bell System Technical Journal, 27, 379–423.CrossRefGoogle Scholar
  40. Shoenfield, J. (2001). Recursion theory, (Lectures notes in logic 1, New ed.). Natick: A.K. PetersGoogle Scholar
  41. Solomonoff, R. (1964). A formal theory of inductive inference, Part I & II. Information and control, 7, 1–22; 224–254.Google Scholar
  42. Wiener, N. (1948). Cybernetics or control and communication in the animal and the machine (2nd ed.). Paris/Hermann: The Technology Press. (Cambridge: MITs, 1965).Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  1. 1.LIAFA, CNRS & Université Paris Diderot – Paris 7Paris Cedex 13France

Personalised recommendations