Abstract
There are non-Gaussian time series that admit a causal linear autoregressive moving average (ARMA) model when regressing the future on the past, but not when regressing the past on the future. The reason is that, in the latter case, the regression residuals are not statistically independent of the regressor. In previous work, we have experimentally verified that many empirical time series indeed show such a time inversion asymmetry.
For various physical systems, it is known that time-inversion asymmetries are linked to the thermodynamic entropy production in non-equilibrium states. Here we argue that unidirectional linearity is also accompanied by entropy generation.
To this end, we study the dynamical evolution of a physical toy system with linear coupling to an infinite environment and show that the linearity of the dynamics is inherited by the forward-time conditional probabilities, but not by the backward-time conditionals. The reason is that the environment permanently provides particles that are in a product state before they interact with the system, but show statistical dependence afterwards. From a coarse-grained perspective, the interaction thus generates entropy. We quantitatively relate the strength of the non-linearity of the backward process to the minimal amount of entropy generation.
The paper thus shows that unidirectional linearity is an indirect implication of the thermodynamic arrow of time, given that the joint dynamics of the system and its environment is linear.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Reichenbach, H.: The Direction of Time. University of California Press, Berkley (1956). Dover, New York (1999)
Penrose, O., Percival, I.: The direction of time. Proc. Phys. Soc. 79, 605–616 (1962)
Balian, R.: From Microphysics to Macrophysics. Springer, Berlin (1992)
Gibbs, J.W.: Elementary Principles in Statistical Mechanics. Ox Bow Press, Woodbridge (1902)
Jaynes, E.T.: Gibbs vs. Boltzmann entropies. Am. J. Phys. 33, 391 (1965)
Lebowitz, J.: Macroscopic dynamics, time’s arrow and Boltzmann entropy. Physica A 194, 1–27 (1993)
Wallace, C.: Statistical and Inductive Inference by Minimum Message Length. Springer, Berlin (2005)
Peters, J., Janzing, D., Gretton, A., Schölkopf, B.: Detecting the direction of causal time series. In: Proceedings of the International Conference on Machine Learning, Montreal. ACM International Conference Proceeding Series, vol. 382, pp. 801–808. ACM, New York (2009). http://www.cs.mcgill.ca/~icml2009/papers/503.pdf and http://portal.acm.org/citation.cfm?doid=1553374.1553477
Peters, J., Janzing, D., Gretton, A., Schölkopf, B.: Kernel methods for detecting the direction of time series. In: Proceedings of the 32nd Annual Conference of the German Classification Society (GfCKI 2008), pp. 1–10. Springer, Berlin (2009)
Maes, C., Redig, F., Van Moffaert, A.: On the definition of entropy production via examples. J. Math. Phys. 41, 1528–1554 (2000)
Gallavotti, G., Cohen, E.: Dynamical ensembles and nonequilibrium statistical mechanics. Phys. Rev. Lett. 74, 2694–2697 (1995)
Horowitz, E., Sahni, S.: Fundamentals of Data Structures. Computer Science Press, New York (1976)
Maes, C., Netocný, K.: Time reversal and entropy. J. Stat. Phys. 110(1–2), 269–309 (2003)
Chazottes, J.-R., Redig, F.: Testing the irreversibility of a Gibbsian process via hitting and return times. Nonlinearity 18(18), 2477–2489 (2005)
Darmois, G.: Analyse générale des liaisons stochastiques. Rev. Inst. Int. Stat. 21, 2–8 (1953)
Skitovic, V.: Linear combinations of independent random variables and the normal distribution law. Sel. Transl. Math. Stat. Probab. 2, 211–228 (1962)
Kano, Y., Shimizu, S.: Causal inference using nonnormality. In: Proceedings of the International Symposium on Science of Modeling, the 30th Anniversary of the Information Criterion, Tokyo, Japan, pp. 261–270 (2003)
Shimizu, S., Hyvärinen, A., Kano, Y., Hoyer, P.O.: Discovery of non-Gaussian linear causal models using ICA. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, Edinburgh, UK, pp. 526–533 (2005)
Brockwell, P., Davis, R.: Time Series: Theory and Methods. Springer, Berlin (1991)
Cover, T., Thomas, J.: Elements of Information Theory. Wileys Series in Telecommunications. Wiley, New York (1991)
Tolman, R.: The Principles of Statistical Mechanics. Oxford University Press, Oxford (1938)
Lemeire, J., Dirkx, E.: Causal models as minimal descriptions of multivariate systems. http://parallel.vub.ac.be/~jan/ (2006)
Janzing, D., Schölkopf, B.: Causal inference using the algorithmic Markov condition. http://arxiv.org/abs/0804.3678 (2008)
Zurek, W.: Algorithmic randomness and physical entropy. Phys Rev A 40(8), 4731–4751 (1989)
Hoyer, P., Janzing, D., Mooij, J., Peters, J., Schölkopf, B.: Nonlinear causal discovery with additive noise models. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Proceedings of the Conference on Neural Information Processing Systems (NIPS), Vancouver, Canada, 2008. MIT Press, Cambridge (2009). http://books.nips.cc/papers/files/nips21/NIPS2008_0266.pdf
Mooij, J., Janzing, D., Peters, J., Schölkopf, B.: Regression by dependence minimization and its application to causal inference. In: Proceedings of the International Conference on Machine Learning, Montreal. ACM International Conference Proceeding Series, vol. 382, pp. 745–752. ACM, New York (2009) http://www.cs.mcgill.ca/~icml2009/papers/279.pdf and http://portal.acm.org/citation.cfm?id=1553374.1553470
Zhang, K., Hyvärinen, A.: On the identifiability of the post-nonlinear causal model. In: 25th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada (2009)
Janzing, D.: On causally asymmetric versions of Occam’s Razor and their relation to thermodynamics. http://arxiv.org/abs/0708.3411v2 (2008)
Allahverdyan, A., Janzing, D.: Relating the thermodynamic arrow of time to the causal arrow. J. Stat. Mech. P04001 (2008)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Janzing, D. On the Entropy Production of Time Series with Unidirectional Linearity. J Stat Phys 138, 767–779 (2010). https://doi.org/10.1007/s10955-009-9897-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10955-009-9897-8