Three perspectives on complexity: entropy, compression, subsymmetry

Abstract

There is no single universally accepted definition of ‘Complexity’. There are several perspectives on complexity and what constitutes complex behaviour or complex systems, as opposed to regular, predictable behaviour and simple systems. In this paper, we explore the following perspectives on complexity: effort-to-describe (Shannon entropy H, Lempel-Ziv complexity LZ), effort-to-compress (ETC complexity) and degree-of-order (Subsymmetry or SubSym). While Shannon entropy and LZ are very popular and widely used, ETC is relatively a new complexity measure. In this paper, we also propose a novel normalized complexity measure SubSym based on the existing idea of counting the number of subsymmetries or palindromes within a sequence. We compare the performance of these complexity measures on the following tasks: (A) characterizing complexity of short binary sequences of lengths 4 to 16, (B) distinguishing periodic and chaotic time series from 1D logistic map and 2D Hénon map, (C) analyzing the complexity of stochastic time series generated from 2-state Markov chains, and (D) distinguishing between tonic and irregular spiking patterns generated from the ‘Adaptive exponential integrate-and-fire’ neuron model. Our study reveals that each perspective has its own advantages and uniqueness while also having an overlap with each other.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    L.A. Lipsitz, A.L. Goldberger, JAMA 267, 1806 (1992)

    Article  Google Scholar 

  2. 2.

    P. Faure, H. Korn, C. R. Acad. Sci. III: Sci. 324, 773 (2001)

    Article  Google Scholar 

  3. 3.

    H. Korn, P. Faure, C. R. Biol. 326, 787 (2003)

    Article  Google Scholar 

  4. 4.

    N. Nagaraj, K.R. Sahasranand, Neural signal multiplexing via compressed sensing, in IEEE Int. Conf. on Signal Processing Communications (IEEE SPCOM) 2016, IISc, Bengaluru (2016), doi:10.1109/SPCOM.2016.7746641

  5. 5.

    S.P. Vadhan, Found. Trends Network 7, 1 (2012)

    Article  Google Scholar 

  6. 6.

    N. Gauvrit, H. Zenil, J.-P. Delahaye, F. Soler-Toscano, Behav. Res. Methods 46, 732 (2014)

    Article  Google Scholar 

  7. 7.

    G.T. Toussaint, N.S. Onea, Q.H. Vuong, Measuring the complexity of two-dimensional binary patterns – sub-symmetries versus papentin complexity, in 2015 14th IAPR International Conference on Machine Vision Applications (MVA) (2015), pp. 480–483

  8. 8.

    S. Lloyd, IEEE Control Syst. Mag. 21, 7 (2001)

    Article  Google Scholar 

  9. 9.

    C. Alexander, S. Carey, Percept. Psychophys. 4, 73 (1968)

    Article  Google Scholar 

  10. 10.

    C.E. Shannon, Bell Syst. Tech. J. 27, 379 (1948)

    Article  Google Scholar 

  11. 11.

    T.M. Cover, J.A. Thomas, Elements of Information Theory (John Wiley & Sons, 2012)

  12. 12.

    N. Nagaraj, K. Balasubramanian, in Handbook of Research on Applied Cybernetics and Systems Science (IGI Global, 2017), pp. 301–334

  13. 13.

    M. Li, P. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications (Springer Science & Business Media, 2009)

  14. 14.

    A. Lempel, J. Ziv, IEEE Trans. Inf. Theor. 22, 75 (1976)

    Article  Google Scholar 

  15. 15.

    J. Ziv, A. Lempel, IEEE Trans. Inf. Theor. 23, 337 (1977)

    Article  Google Scholar 

  16. 16.

    M. Aboy, R. Hornero, D. Abásolo, D. Álvarez, IEEE Trans. Biomed. Eng. 53, 2282 (2006)

    Article  Google Scholar 

  17. 17.

    N. Nagaraj, K. Balasubramanian, S. Dey, Eur. Phys. J. Special Topics 222, 847 (2013)

    ADS  Article  Google Scholar 

  18. 18.

    W. Ebeling, M.A. Jiménez-Montaño, Math. Biosci. 52, 53 (1980)

    Article  Google Scholar 

  19. 19.

    N. Nagaraj, K. Balasubramanian, Eur. Phys. J. Special Topics 226, 2191 (2017)

    ADS  Article  Google Scholar 

  20. 20.

    K. Balasubramanian, N. Nagaraj, PeerJ 4, e2755 (2016)

    Article  Google Scholar 

  21. 21.

    M. Virmani, N. Nagaraj, A compression-complexity measure of integrated information, arXiv:1608.08450v2 (2016)

  22. 22.

    J.M. Amigó, J. Szczepański, E. Wajnryb, M.V. Sanchez-Vives, Neural Comput. 16, 717 (2004)

    Article  Google Scholar 

  23. 23.

    J. Hu, J. Gao, J.C. Principe, IEEE Trans. Biomed. Eng. 53, 2606 (2006)

    Article  Google Scholar 

  24. 24.

    A.L. Goldberger, Physiology 6, 87 (1991)

    Article  Google Scholar 

  25. 25.

    L.A. Lipsitz, Chaos: Interdiscip. J. Nonlinear Sci. 5, 102 (1995)

    Article  Google Scholar 

  26. 26.

    K.T. Alligood, T.D. Sauer, J.A. Yorke, Chaos (Springer, 1997)

  27. 27.

    W. Gersch, D.M. Eddy, E. Dong Jr., Comput. Biomed. Res. 3, 385 (1970)

    Article  Google Scholar 

  28. 28.

    D. Coast, R.M. Stern, G.G. Cano, S. Briller, et al., IEEE Trans. Biomed. Eng. 37, 826 (1990)

    Article  Google Scholar 

  29. 29.

    W. Gersch, P. Lilly, E. Dong, Comput. Biomed. Res. 8, 370 (1975)

    Article  Google Scholar 

  30. 30.

    S.-T. Pan, Y.-H. Wu, Y.-L. Kung, H.-C. Chen, Heartbeat recognition from ECG signals using hidden Markov model with adaptive features, in 14th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD) (2013), pp. 586–591

  31. 31.

    M.S. Waterman, Mathematical methods for DNA sequences (CRC Press Inc., 1989)

  32. 32.

    T.-J. Wu, Y.-C. Hsieh, L.-A. Li, Biometrics 57, 441 (2001)

    MathSciNet  Article  Google Scholar 

  33. 33.

    I. Sergienko, A. Gupal, A. Ostrovsky, Cybernet. Syst. Anal. 48, 369 (2012)

    Article  Google Scholar 

  34. 34.

    L. Narlikar, N. Mehta, S. Galande, M. Arjunwadkar, Nucl. Acids Res. 41, 1416 (2013)

    Article  Google Scholar 

  35. 35.

    A. Varga, R. Moore, Hidden Markov model decomposition of speech and noise, in International Conference on Acoustics, Speech and Signal Processing (ICASSP) (1990), pp. 845–848

  36. 36.

    B.H. Juang, L.R. Rabiner, Technometrics 33, 251 (1991)

    MathSciNet  Article  Google Scholar 

  37. 37.

    H. Veisi, H. Sameti, Speech Commun. 55, 205 (2013)

    Article  Google Scholar 

  38. 38.

    R.P. Rao, N. Yadav, M.N. Vahia, H. Joglekar, R. Adhikari, I. Mahadevan, Proc. Natl. Acad. Sci. U. S. A. 106, 13685 (2009)

    ADS  Article  Google Scholar 

  39. 39.

    R.P. Rao, IEEE Comput. 43, 76 (2010)

    Article  Google Scholar 

  40. 40.

    G.A. Fink, Markov models for pattern recognition: from theory to applications (Springer Science & Business Media, 2014)

  41. 41.

    G.V. Cormack, R. Horspool, Comput. J. 30, 541 (1987)

    Article  Google Scholar 

  42. 42.

    H.S. Wang, N. Moayeri, IEEE Trans. Veh. Technol. 44, 163 (1995)

    Article  Google Scholar 

  43. 43.

    H. Zhou, J. Bruck, IEEE Trans. Inf. Theor. 58, 2490 (2012)

    Article  Google Scholar 

  44. 44.

    M. Svoboda, L. Lukas, Application of Markov chain analysis to trend prediction of stock indices, in Proceedings of 30th International Conference Mathematical Methodsin Economics (Silesian University, School of Business Administration, Karviná, 2012), pp. 848–853

  45. 45.

    F.O. Mettle, E.N.B. Quaye, R.A. Laryea, SpringerPlus 3, 1 (2014)

    Article  Google Scholar 

  46. 46.

    R. Gütig, Curr. Opin. Neurobiol. 25, 134 (2014)

    Article  Google Scholar 

  47. 47.

    R. Brette, W. Gerstner, J. Neurophysiol. 94, 3637 (2005)

    Article  Google Scholar 

  48. 48.

    R. Naud, N. Marcille, C. Clopath, W. Gerstner, Biol. Cybernet. 99, 335 (2008)

    MathSciNet  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding authors

Correspondence to Nithin Nagaraj or Karthi Balasubramanian.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nagaraj, N., Balasubramanian, K. Three perspectives on complexity: entropy, compression, subsymmetry. Eur. Phys. J. Spec. Top. 226, 3251–3272 (2017). https://doi.org/10.1140/epjst/e2016-60347-2

Download citation