On the Entropy Production of Time Series with Unidirectional Linearity
- 338 Downloads
There are non-Gaussian time series that admit a causal linear autoregressive moving average (ARMA) model when regressing the future on the past, but not when regressing the past on the future. The reason is that, in the latter case, the regression residuals are not statistically independent of the regressor. In previous work, we have experimentally verified that many empirical time series indeed show such a time inversion asymmetry.
For various physical systems, it is known that time-inversion asymmetries are linked to the thermodynamic entropy production in non-equilibrium states. Here we argue that unidirectional linearity is also accompanied by entropy generation.
To this end, we study the dynamical evolution of a physical toy system with linear coupling to an infinite environment and show that the linearity of the dynamics is inherited by the forward-time conditional probabilities, but not by the backward-time conditionals. The reason is that the environment permanently provides particles that are in a product state before they interact with the system, but show statistical dependence afterwards. From a coarse-grained perspective, the interaction thus generates entropy. We quantitatively relate the strength of the non-linearity of the backward process to the minimal amount of entropy generation.
The paper thus shows that unidirectional linearity is an indirect implication of the thermodynamic arrow of time, given that the joint dynamics of the system and its environment is linear.
- 1.Reichenbach, H.: The Direction of Time. University of California Press, Berkley (1956). Dover, New York (1999) Google Scholar
- 8.Peters, J., Janzing, D., Gretton, A., Schölkopf, B.: Detecting the direction of causal time series. In: Proceedings of the International Conference on Machine Learning, Montreal. ACM International Conference Proceeding Series, vol. 382, pp. 801–808. ACM, New York (2009). http://www.cs.mcgill.ca/~icml2009/papers/503.pdf and http://portal.acm.org/citation.cfm?doid=1553374.1553477 Google Scholar
- 9.Peters, J., Janzing, D., Gretton, A., Schölkopf, B.: Kernel methods for detecting the direction of time series. In: Proceedings of the 32nd Annual Conference of the German Classification Society (GfCKI 2008), pp. 1–10. Springer, Berlin (2009) Google Scholar
- 17.Kano, Y., Shimizu, S.: Causal inference using nonnormality. In: Proceedings of the International Symposium on Science of Modeling, the 30th Anniversary of the Information Criterion, Tokyo, Japan, pp. 261–270 (2003) Google Scholar
- 18.Shimizu, S., Hyvärinen, A., Kano, Y., Hoyer, P.O.: Discovery of non-Gaussian linear causal models using ICA. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, Edinburgh, UK, pp. 526–533 (2005) Google Scholar
- 20.Cover, T., Thomas, J.: Elements of Information Theory. Wileys Series in Telecommunications. Wiley, New York (1991) Google Scholar
- 21.Tolman, R.: The Principles of Statistical Mechanics. Oxford University Press, Oxford (1938) Google Scholar
- 22.Lemeire, J., Dirkx, E.: Causal models as minimal descriptions of multivariate systems. http://parallel.vub.ac.be/~jan/ (2006)
- 23.Janzing, D., Schölkopf, B.: Causal inference using the algorithmic Markov condition. http://arxiv.org/abs/0804.3678 (2008)
- 25.Hoyer, P., Janzing, D., Mooij, J., Peters, J., Schölkopf, B.: Nonlinear causal discovery with additive noise models. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Proceedings of the Conference on Neural Information Processing Systems (NIPS), Vancouver, Canada, 2008. MIT Press, Cambridge (2009). http://books.nips.cc/papers/files/nips21/NIPS2008_0266.pdf Google Scholar
- 26.Mooij, J., Janzing, D., Peters, J., Schölkopf, B.: Regression by dependence minimization and its application to causal inference. In: Proceedings of the International Conference on Machine Learning, Montreal. ACM International Conference Proceeding Series, vol. 382, pp. 745–752. ACM, New York (2009) http://www.cs.mcgill.ca/~icml2009/papers/279.pdf and http://portal.acm.org/citation.cfm?id=1553374.1553470 Google Scholar
- 27.Zhang, K., Hyvärinen, A.: On the identifiability of the post-nonlinear causal model. In: 25th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada (2009) Google Scholar
- 28.Janzing, D.: On causally asymmetric versions of Occam’s Razor and their relation to thermodynamics. http://arxiv.org/abs/0708.3411v2 (2008)
- 29.Allahverdyan, A., Janzing, D.: Relating the thermodynamic arrow of time to the causal arrow. J. Stat. Mech. P04001 (2008) Google Scholar