Non-sufficient Memories That Are Sufficient for Prediction
The causal states of computational mechanics define the minimal sufficient (prescient) memory for a given stationary stochastic process. They induce the ε-machine which is a hidden Markov model (HMM) generating the process. The ε-machine is, however, not the minimal generative HMM and minimal internal state entropy of a generative HMM is a tighter upper bound for excess entropy than provided by statistical complexity. We propose a notion of prediction that does not require sufficiency. The corresponding models can be substantially smaller than the ε-machine and are closely related to generative HMMs.
Keywordshidden Markov models HMM computational mechanics causal states ε-machine prediction
Unable to display preview. Download preview PDF.
- 3.Löhr, W., Ay, N.: On the generative nature of prediction. Accepted for publication in Advances in Complex Systems (preprint, 2008), http://www.mis.mpg.de/publications/preprints/2008/prepr2008-8.html
- 4.Still, S., Crutchfield, J.P.: Optimal causal inference. Informal publication (2007), http://arxiv.org/abs/0708.1580