On Hierarchical Compression and Power Laws in Nature

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10414)

Abstract

Since compressing data incrementally by a non-branching hierarchy has resulted in substantial efficiency gains for performing induction in previous work, we now explore branching hierarchical compression as a means for solving induction problems for generally intelligent systems. Even though assuming the compositionality of data generation and the locality of information may result in a loss of the universality of induction, it has still the potential to be general in the sense of reflecting the inherent structure of real world data imposed by the laws of physics. We derive a proof that branching compression hierarchies (BCHs) create power law functions of mutual algorithmic information between two strings as a function of their distance – a ubiquitous characteristic of natural data, which opens the possibility of efficient natural data compression by BCHs. Further, we show that such hierarchies guarantee the existence of short features in the data which in turn increases the efficiency of induction even more.

Keywords

Hierarchical compression Incremental compression Algorithmic complexity Universal induction Power laws Scale free structure 

References

  1. 1.
    Solomonoff, R.J.: A formal theory of inductive inference. Part I. Inf. Control 7(1), 1–22 (1964)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Solomonoff, R.J.: A formal theory of inductive inference. Part II. Inf. Control 7(2), 224–254 (1964)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Lin, H.W., Tegmark, M.: Why does deep and cheap learning work so well? arXiv preprint arXiv:1608.08225 (2016)
  4. 4.
    Franz, A.: Some theorems on incremental compression. In: Steunebrink, B., Wang, P., Goertzel, B. (eds.) AGI -2016. LNCS, vol. 9782, pp. 74–83. Springer, Cham (2016). doi:10.1007/978-3-319-41649-6_8 Google Scholar
  5. 5.
    Bak, P.: How Nature Works: The Science of Self-organized Criticality. Copernicus, New York (1996)CrossRefMATHGoogle Scholar
  6. 6.
    Saremi, S., Sejnowski, T.J.: Hierarchical model of natural images and the origin of scale invariance. Proc. Natl. Acad. Sci. 110(8), 3071–3076 (2013)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Lin, H.W., Tegmark, M.: Critical behavior from deep dynamics: a hidden dimension in natural language. arXiv preprint arXiv:1606.06737 (2016)
  8. 8.
    Li, M., Vitányi, P.: An Introduction to Kolmogorov Complexity and Its Applications. Springer, New York (2009)MATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Independent ResearcherOdessaUkraine

Personalised recommendations