Sophistication vs Logical Depth

Abstract

Sophistication and logical depth are two measures that express how complicated the structure in a string is. Sophistication is defined as the minimal complexity of a computable function that defines a two-part description for the string that is shortest within some precision; the second can be defined as the minimal computation time of a program that is shortest within some precision. We show that the Busy Beaver function of the sophistication of a string exceeds its logical depth with logarithmically bigger precision, and that logical depth exceeds the Busy Beaver function of sophistication with logarithmically bigger precision. We also show that sophistication is unstable in its precision: constant variations can change its value by a linear term in the length of the string.

Appendices

Appendix A: Machine Invariance of Logical Depth

Lemma 7

For all universal Turing machines U and V, there exist a constant $c^{\prime }$ such that for all c and x: depth _c,U (x)≥|x| [no Busy Beaver here!] implies

$$\text{depth}_{c+c^{\prime},V}^{bb}(x) \le \text{depth}_{c,U}^{bb}(x) + c^{\prime}. $$

Note that for some universal machines there exist a string w such that U(w x) = x for all x and the computation requires at most O(1) steps. For such machines U we have depth_{|w x|}(x) ≤ O(1) and hence $\text {depth}_{|wx|}^{bb}(x) \le O(1)$. Other universal machines always scan the input, and on such machines we have $\text {depth}_{|wx|}^{bb}(x) \ge bb(|x|) - O(1)$ for all x. Hence, the assumption in the lemma is necessary.

Proof

Let w _V be the prefix such that V(w _V p) simulates U(p) for all p. Our result would follow easily if we assume that for any halting programs p, q on U such that time(p) ≤ time(q) we have time(w _V p) ≤ time(w _V q) on V; i.e. simulating U on V preserves the order of computation time. Indeed, any pair (p, q) usable in the definition of depth on U defines a pair (w _V p, w _V q) that can be used in the definition of depth on V. The program w _V p is minimal on V within c+|w _V |+|w _U | error (where w _U is the string that allows to simulate U on V). Hence, the pair (w _V p, w _V q) witnesses an increase of sophistication by at most |w _V | for an increase of the significance of at most c+|w _V |+|w _U |.

In the case where the assumption is not true, we need to find $c^{\prime }$ and a program of length at most $|q|+c^{\prime }$ on V that computes longer than time(w _V p) (where $c^{\prime }$ does not depend on p, q, c). Consider the following algorithm on input q: determine all programs p that have running time at most time(q) on U, determine for all these p’s the maximal running time T of a program w _V p on V (assume for now that for finite time(q) there are finitely many such p), and finally print a string of length T. For (p, q) usable in the definition of depth _{c, U} (x), the algorithm with input q produces an output longer than time(w _V p), and by universality there is a program of length $|q|+c^{\prime }$ on V that prints this string and hence computes longer than T.

Above, we have assumed that only finitely many programs on U have a halting time at most time(q) for halting q. This assumption is not true in general, but by the additional assumption of the lemma: depth _{c, U} (x) ≥ |x|, it suffices to consider only a finite subset of candidates: we only need the pairs (p, q) on U such that |p|≤|x| + O(1) and |x|≤time(q), which implies |p|≤time(q) + O(1). The proof finishes by modifying the above algorithm such that it only considers programs p for which |p|≤time(q) + O(1). □

Recall that Bennett’s definition of logical depth is the minimal computation time of a program on a prefix-free machine W (of some type) that is c-incompressible. We show that when scaled by the inverse Busy Beaver function, both notions of depth are $O(\log |x|)$-close. On a prefix-free machine W, both (unscaled) depths are closely related: Bennett’s logical depth of x at significance c is at most depth _{c + O(1), W} (x), because any c-shortest program p for x is c + O(1)-incompressible on W. On the other hand, by Lemma 5.3 of [7] (attributed to Bennett [4]), depth _{c + O(1)}(x) is bounded by a computable function of Bennett’s logical depth of x with significance c. Hence, after rescaling with the inverse Busy Beaver function, both notions are O(1)-close. Exchanging prefix-free machine by a plain machine, both depth notions are $O(\log |x|)$-close; indeed this follows by the same argument as Lemma 7 for W = V and replacing |w _V | by $O(\log |x|)$-terms in the proof (since $|K_{W}(x)-C_{U}(x)| \le O(\log |x|)$).

Appendix B: Alternative Proof of Theorem 1

An alternative proof for the second inequality in Theorem 1 is given: there exists e such that for all c and x with |x|≥e we have

$$\text{soph}_{c + e\log |x|}(x) \le \text{depth}_{c}^{bb}(x) + e\log |x|\,. $$

A prefix stable machine V is a plain machine such that for all strings p and extensions q of p: if p∈DomV then q∈DomV and V(p) = V(q). For (infinite) sequences ω let V(ω) be V(p) if a prefix p of ω exists such that V(p) is defined, and undefined otherwise. For any string or sequence ω, let 0.ω be the real ${\sum }_{i} \omega _{i} 2^{-i}$. A prefix stable machine is left computable [17] if for p such that V(p) is defined and for all q such that 0.q ≤ 0.p, also V(q) is defined. There are universal prefix stable machines that are left computable (just rearrange the programs on a universal machine). Let $\Omega = \sup \{0.p: V(p) \textnormal {is defined} \}$.

In order to prove the result aforementioned, it is sufficient to show that $\text {soph}_{c + 2\log |x|,U}(x) \le \text {depth}_{c,W}^{bb}(x) + 2\log |x|$ for large x, where W is a universal left computable machine. Indeed, there exists a universal plain machine U such that

$$\text{depth}_{c + 2\log |x|,W}^{bb}(x) \le \text{depth}_{c,U}^{bb}(x) + O(\log |x|). $$

(translating plain programs to self-delimiting ones can happen by affecting program sizes by at most $O(\log |p|)$ and computation time by a computable function of |p| and the halting time).

Let p be a program satisfying the conditions in the definition of $\text {depth}_{c}^{bb}(x)$. We show that the initial segment where p and Ω are equal defines a computable function that satisfies the conditions in the definition of sophistication. More precisely, let i be the length of the common initial segment, then $F(d) = V(\Omega _{1}{\dots } \Omega _{i}0d)$ satisfies the conditions. Note that, $p_{1}{\dots } p_{i} = \Omega _{1}{\dots } \Omega _{i-1}0$ and Ω _i = 1 by construction of i. Thus $F(p_{i+2}{\dots } p_{|p|}) = x$. For any d we have $0.\Omega _{1}{\dots } \Omega _{i}0d < \Omega$ and by left computability this is in DomV, thus F is computable. It remains to show that $C(F) \le \text {depth}_{c,W}(x)+O(\log \text {depth}_{c,W}(x))$. We show that $C(\Omega _{1}{\dots } \Omega _{i-1}) \le \text {depth}_{c,W}(x) + O(\log i)$. In fact, given i and a t that exceeds the computation time of p, we can search for the maximal value 0.w for a program w that halts in t computation steps. We know that 0.p ≤ 0.w ≤ Ω, hence we can compute the first i−1 bits of Ω which completes the proof.