Algorithmic information and simplicity in statistical physics

Abstract

Applications of algorithmic information theory to statistical physics rely (a) on the fact that average conditional algorithmic information can be approximated by Shannon information and (b) on the existence ofsimple states described by short programs. More precisely, given a list ofN states with probabilities 0<p ₁ ≤ ... ≤ p _N , the average conditional algorithmic informationĪ to specify one of these states obeys the inequalityH≤ Ī<H+O(1), whereH=−Σp _j log₂ p _j andO(1) is a computer-dependent constant. We show how any universal computer can be slightly modified in such a way that (a) the inequality becomesH≤ Ī<H+1 and (b) states that are simple with respect to the original computer remain simple with respect to the modified computer, thereby eliminating the computer-dependent constant from statistical physics.