Skip to main content
Download PDF

On the Entropy Production of Time Series with Unidirectional Linearity

Journal of Statistical Physics volume 138, pages 767–779 (2010)Cite this article

Abstract

There are non-Gaussian time series that admit a causal linear autoregressive moving average (ARMA) model when regressing the future on the past, but not when regressing the past on the future. The reason is that, in the latter case, the regression residuals are not statistically independent of the regressor. In previous work, we have experimentally verified that many empirical time series indeed show such a time inversion asymmetry.

For various physical systems, it is known that time-inversion asymmetries are linked to the thermodynamic entropy production in non-equilibrium states. Here we argue that unidirectional linearity is also accompanied by entropy generation.

To this end, we study the dynamical evolution of a physical toy system with linear coupling to an infinite environment and show that the linearity of the dynamics is inherited by the forward-time conditional probabilities, but not by the backward-time conditionals. The reason is that the environment permanently provides particles that are in a product state before they interact with the system, but show statistical dependence afterwards. From a coarse-grained perspective, the interaction thus generates entropy. We quantitatively relate the strength of the non-linearity of the backward process to the minimal amount of entropy generation.

The paper thus shows that unidirectional linearity is an indirect implication of the thermodynamic arrow of time, given that the joint dynamics of the system and its environment is linear.

References

  1. Reichenbach, H.: The Direction of Time. University of California Press, Berkley (1956). Dover, New York (1999)

    Google Scholar 

  2. Penrose, O., Percival, I.: The direction of time. Proc. Phys. Soc. 79, 605–616 (1962)

    MATH  Article  MathSciNet  Google Scholar 

  3. Balian, R.: From Microphysics to Macrophysics. Springer, Berlin (1992)

    MATH  Google Scholar 

  4. Gibbs, J.W.: Elementary Principles in Statistical Mechanics. Ox Bow Press, Woodbridge (1902)

    MATH  Google Scholar 

  5. Jaynes, E.T.: Gibbs vs. Boltzmann entropies. Am. J. Phys. 33, 391 (1965)

    MATH  Article  ADS  Google Scholar 

  6. Lebowitz, J.: Macroscopic dynamics, time’s arrow and Boltzmann entropy. Physica A 194, 1–27 (1993)

    Article  MathSciNet  ADS  Google Scholar 

  7. Wallace, C.: Statistical and Inductive Inference by Minimum Message Length. Springer, Berlin (2005)

    MATH  Google Scholar 

  8. Peters, J., Janzing, D., Gretton, A., Schölkopf, B.: Detecting the direction of causal time series. In: Proceedings of the International Conference on Machine Learning, Montreal. ACM International Conference Proceeding Series, vol. 382, pp. 801–808. ACM, New York (2009). http://www.cs.mcgill.ca/~icml2009/papers/503.pdf and http://portal.acm.org/citation.cfm?doid=1553374.1553477

    Google Scholar 

  9. Peters, J., Janzing, D., Gretton, A., Schölkopf, B.: Kernel methods for detecting the direction of time series. In: Proceedings of the 32nd Annual Conference of the German Classification Society (GfCKI 2008), pp. 1–10. Springer, Berlin (2009)

    Google Scholar 

  10. Maes, C., Redig, F., Van Moffaert, A.: On the definition of entropy production via examples. J. Math. Phys. 41, 1528–1554 (2000)

    MATH  Article  MathSciNet  ADS  Google Scholar 

  11. Gallavotti, G., Cohen, E.: Dynamical ensembles and nonequilibrium statistical mechanics. Phys. Rev. Lett. 74, 2694–2697 (1995)

    Article  ADS  Google Scholar 

  12. Horowitz, E., Sahni, S.: Fundamentals of Data Structures. Computer Science Press, New York (1976)

    MATH  Google Scholar 

  13. Maes, C., Netocný, K.: Time reversal and entropy. J. Stat. Phys. 110(1–2), 269–309 (2003)

    MATH  Article  Google Scholar 

  14. Chazottes, J.-R., Redig, F.: Testing the irreversibility of a Gibbsian process via hitting and return times. Nonlinearity 18(18), 2477–2489 (2005)

    MATH  Article  MathSciNet  ADS  Google Scholar 

  15. Darmois, G.: Analyse générale des liaisons stochastiques. Rev. Inst. Int. Stat. 21, 2–8 (1953)

    Article  MathSciNet  Google Scholar 

  16. Skitovic, V.: Linear combinations of independent random variables and the normal distribution law. Sel. Transl. Math. Stat. Probab. 2, 211–228 (1962)

    MATH  MathSciNet  Google Scholar 

  17. Kano, Y., Shimizu, S.: Causal inference using nonnormality. In: Proceedings of the International Symposium on Science of Modeling, the 30th Anniversary of the Information Criterion, Tokyo, Japan, pp. 261–270 (2003)

  18. Shimizu, S., Hyvärinen, A., Kano, Y., Hoyer, P.O.: Discovery of non-Gaussian linear causal models using ICA. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, Edinburgh, UK, pp. 526–533 (2005)

  19. Brockwell, P., Davis, R.: Time Series: Theory and Methods. Springer, Berlin (1991)

    Book  Google Scholar 

  20. Cover, T., Thomas, J.: Elements of Information Theory. Wileys Series in Telecommunications. Wiley, New York (1991)

    Google Scholar 

  21. Tolman, R.: The Principles of Statistical Mechanics. Oxford University Press, Oxford (1938)

    Google Scholar 

  22. Lemeire, J., Dirkx, E.: Causal models as minimal descriptions of multivariate systems. http://parallel.vub.ac.be/~jan/ (2006)

  23. Janzing, D., Schölkopf, B.: Causal inference using the algorithmic Markov condition. http://arxiv.org/abs/0804.3678 (2008)

  24. Zurek, W.: Algorithmic randomness and physical entropy. Phys Rev A 40(8), 4731–4751 (1989)

    Article  MathSciNet  ADS  Google Scholar 

  25. Hoyer, P., Janzing, D., Mooij, J., Peters, J., Schölkopf, B.: Nonlinear causal discovery with additive noise models. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Proceedings of the Conference on Neural Information Processing Systems (NIPS), Vancouver, Canada, 2008. MIT Press, Cambridge (2009). http://books.nips.cc/papers/files/nips21/NIPS2008_0266.pdf

    Google Scholar 

  26. Mooij, J., Janzing, D., Peters, J., Schölkopf, B.: Regression by dependence minimization and its application to causal inference. In: Proceedings of the International Conference on Machine Learning, Montreal. ACM International Conference Proceeding Series, vol. 382, pp. 745–752. ACM, New York (2009) http://www.cs.mcgill.ca/~icml2009/papers/279.pdf and http://portal.acm.org/citation.cfm?id=1553374.1553470

    Google Scholar 

  27. Zhang, K., Hyvärinen, A.: On the identifiability of the post-nonlinear causal model. In: 25th Conference on Uncertainty in Artificial Intelligence, Montreal, Canada (2009)

  28. Janzing, D.: On causally asymmetric versions of Occam’s Razor and their relation to thermodynamics. http://arxiv.org/abs/0708.3411v2 (2008)

  29. Allahverdyan, A., Janzing, D.: Relating the thermodynamic arrow of time to the causal arrow. J. Stat. Mech. P04001 (2008)

Download references

Author information

Authors and Affiliations

  1. Max Planck Institute for Biological Cybernetics, Spemannstr. 38, 72076, Tübingen, Germany

    Dominik Janzing

Authors
  1. Dominik Janzing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominik Janzing.

Rights and permissions

Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Reprints and Permissions

About this article

Cite this article

Janzing, D. On the Entropy Production of Time Series with Unidirectional Linearity. J Stat Phys 138, 767–779 (2010). https://doi.org/10.1007/s10955-009-9897-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10955-009-9897-8

  • Arrow of time
  • Entropy production
  • Irreversible processes
  • Time series
  • ARMA models