# On Hierarchical Compression and Power Laws in Nature

## Abstract

Since compressing data incrementally by a non-branching hierarchy has resulted in substantial efficiency gains for performing induction in previous work, we now explore branching hierarchical compression as a means for solving induction problems for generally intelligent systems. Even though assuming the compositionality of data generation and the locality of information may result in a loss of the universality of induction, it has still the potential to be general in the sense of reflecting the inherent structure of real world data imposed by the laws of physics. We derive a proof that branching compression hierarchies (BCHs) create power law functions of mutual algorithmic information between two strings as a function of their distance – a ubiquitous characteristic of natural data, which opens the possibility of efficient natural data compression by BCHs. Further, we show that such hierarchies guarantee the existence of short features in the data which in turn increases the efficiency of induction even more.

## Keywords

Hierarchical compression Incremental compression Algorithmic complexity Universal induction Power laws Scale free structure## References

- 1.Solomonoff, R.J.: A formal theory of inductive inference. Part I. Inf. Control
**7**(1), 1–22 (1964)MathSciNetCrossRefzbMATHGoogle Scholar - 2.Solomonoff, R.J.: A formal theory of inductive inference. Part II. Inf. Control
**7**(2), 224–254 (1964)MathSciNetCrossRefzbMATHGoogle Scholar - 3.Lin, H.W., Tegmark, M.: Why does deep and cheap learning work so well? arXiv preprint arXiv:1608.08225 (2016)
- 4.Franz, A.: Some theorems on incremental compression. In: Steunebrink, B., Wang, P., Goertzel, B. (eds.) AGI -2016. LNCS, vol. 9782, pp. 74–83. Springer, Cham (2016). doi: 10.1007/978-3-319-41649-6_8 Google Scholar
- 5.Bak, P.: How Nature Works: The Science of Self-organized Criticality. Copernicus, New York (1996)CrossRefzbMATHGoogle Scholar
- 6.Saremi, S., Sejnowski, T.J.: Hierarchical model of natural images and the origin of scale invariance. Proc. Natl. Acad. Sci.
**110**(8), 3071–3076 (2013)MathSciNetCrossRefzbMATHGoogle Scholar - 7.Lin, H.W., Tegmark, M.: Critical behavior from deep dynamics: a hidden dimension in natural language. arXiv preprint arXiv:1606.06737 (2016)
- 8.Li, M., Vitányi, P.: An Introduction to Kolmogorov Complexity and Its Applications. Springer, New York (2009)zbMATHGoogle Scholar