Advertisement

The European Physical Journal Special Topics

, Volume 226, Issue 15, pp 3251–3272 | Cite as

Three perspectives on complexity: entropy, compression, subsymmetry

  • Nithin Nagaraj
  • Karthi Balasubramanian
Regular Article
  • 102 Downloads
Part of the following topical collections:
  1. Challenges in the Analysis of Complex Systems: Prediction, Causality and Communication

Abstract

There is no single universally accepted definition of ‘Complexity’. There are several perspectives on complexity and what constitutes complex behaviour or complex systems, as opposed to regular, predictable behaviour and simple systems. In this paper, we explore the following perspectives on complexity: effort-to-describe (Shannon entropy H, Lempel-Ziv complexity LZ), effort-to-compress (ETC complexity) and degree-of-order (Subsymmetry or SubSym). While Shannon entropy and LZ are very popular and widely used, ETC is relatively a new complexity measure. In this paper, we also propose a novel normalized complexity measure SubSym based on the existing idea of counting the number of subsymmetries or palindromes within a sequence. We compare the performance of these complexity measures on the following tasks: (A) characterizing complexity of short binary sequences of lengths 4 to 16, (B) distinguishing periodic and chaotic time series from 1D logistic map and 2D Hénon map, (C) analyzing the complexity of stochastic time series generated from 2-state Markov chains, and (D) distinguishing between tonic and irregular spiking patterns generated from the ‘Adaptive exponential integrate-and-fire’ neuron model. Our study reveals that each perspective has its own advantages and uniqueness while also having an overlap with each other.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    L.A. Lipsitz, A.L. Goldberger, JAMA 267, 1806 (1992) CrossRefGoogle Scholar
  2. 2.
    P. Faure, H. Korn, C. R. Acad. Sci. III: Sci. 324, 773 (2001) CrossRefGoogle Scholar
  3. 3.
    H. Korn, P. Faure, C. R. Biol. 326, 787 (2003) CrossRefGoogle Scholar
  4. 4.
    N. Nagaraj, K.R. Sahasranand, Neural signal multiplexing via compressed sensing, in IEEE Int. Conf. on Signal Processing Communications (IEEE SPCOM) 2016, IISc, Bengaluru (2016), doi: 10.1109/SPCOM.2016.7746641
  5. 5.
    S.P. Vadhan, Found. Trends Network 7, 1 (2012) CrossRefGoogle Scholar
  6. 6.
    N. Gauvrit, H. Zenil, J.-P. Delahaye, F. Soler-Toscano, Behav. Res. Methods 46, 732 (2014) CrossRefGoogle Scholar
  7. 7.
    G.T. Toussaint, N.S. Onea, Q.H. Vuong, Measuring the complexity of two-dimensional binary patterns – sub-symmetries versus papentin complexity, in 2015 14th IAPR International Conference on Machine Vision Applications (MVA) (2015), pp. 480–483 Google Scholar
  8. 8.
    S. Lloyd, IEEE Control Syst. Mag. 21, 7 (2001) CrossRefGoogle Scholar
  9. 9.
    C. Alexander, S. Carey, Percept. Psychophys. 4, 73 (1968) CrossRefGoogle Scholar
  10. 10.
    C.E. Shannon, Bell Syst. Tech. J. 27, 379 (1948) CrossRefGoogle Scholar
  11. 11.
    T.M. Cover, J.A. Thomas, Elements of Information Theory (John Wiley & Sons, 2012) Google Scholar
  12. 12.
    N. Nagaraj, K. Balasubramanian, in Handbook of Research on Applied Cybernetics and Systems Science (IGI Global, 2017), pp. 301–334 Google Scholar
  13. 13.
    M. Li, P. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications (Springer Science & Business Media, 2009) Google Scholar
  14. 14.
    A. Lempel, J. Ziv, IEEE Trans. Inf. Theor. 22, 75 (1976) CrossRefGoogle Scholar
  15. 15.
    J. Ziv, A. Lempel, IEEE Trans. Inf. Theor. 23, 337 (1977) CrossRefGoogle Scholar
  16. 16.
    M. Aboy, R. Hornero, D. Abásolo, D. Álvarez, IEEE Trans. Biomed. Eng. 53, 2282 (2006) CrossRefGoogle Scholar
  17. 17.
    N. Nagaraj, K. Balasubramanian, S. Dey, Eur. Phys. J. Special Topics 222, 847 (2013) ADSCrossRefGoogle Scholar
  18. 18.
    W. Ebeling, M.A. Jiménez-Montaño, Math. Biosci. 52, 53 (1980) CrossRefGoogle Scholar
  19. 19.
    N. Nagaraj, K. Balasubramanian, Eur. Phys. J. Special Topics 226, 2191 (2017) ADSCrossRefGoogle Scholar
  20. 20.
    K. Balasubramanian, N. Nagaraj, PeerJ 4, e2755 (2016) CrossRefGoogle Scholar
  21. 21.
    M. Virmani, N. Nagaraj, A compression-complexity measure of integrated information, arXiv:1608.08450v2 (2016)
  22. 22.
    J.M. Amigó, J. Szczepański, E. Wajnryb, M.V. Sanchez-Vives, Neural Comput. 16, 717 (2004) CrossRefGoogle Scholar
  23. 23.
    J. Hu, J. Gao, J.C. Principe, IEEE Trans. Biomed. Eng. 53, 2606 (2006) CrossRefGoogle Scholar
  24. 24.
    A.L. Goldberger, Physiology 6, 87 (1991) CrossRefGoogle Scholar
  25. 25.
    L.A. Lipsitz, Chaos: Interdiscip. J. Nonlinear Sci. 5, 102 (1995) CrossRefGoogle Scholar
  26. 26.
    K.T. Alligood, T.D. Sauer, J.A. Yorke, Chaos (Springer, 1997) Google Scholar
  27. 27.
    W. Gersch, D.M. Eddy, E. Dong Jr., Comput. Biomed. Res. 3, 385 (1970) CrossRefGoogle Scholar
  28. 28.
    D. Coast, R.M. Stern, G.G. Cano, S. Briller, et al., IEEE Trans. Biomed. Eng. 37, 826 (1990) CrossRefGoogle Scholar
  29. 29.
    W. Gersch, P. Lilly, E. Dong, Comput. Biomed. Res. 8, 370 (1975) CrossRefGoogle Scholar
  30. 30.
    S.-T. Pan, Y.-H. Wu, Y.-L. Kung, H.-C. Chen, Heartbeat recognition from ECG signals using hidden Markov model with adaptive features, in 14th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD) (2013), pp. 586–591 Google Scholar
  31. 31.
    M.S. Waterman, Mathematical methods for DNA sequences (CRC Press Inc., 1989) Google Scholar
  32. 32.
    T.-J. Wu, Y.-C. Hsieh, L.-A. Li, Biometrics 57, 441 (2001) MathSciNetCrossRefGoogle Scholar
  33. 33.
    I. Sergienko, A. Gupal, A. Ostrovsky, Cybernet. Syst. Anal. 48, 369 (2012) CrossRefGoogle Scholar
  34. 34.
    L. Narlikar, N. Mehta, S. Galande, M. Arjunwadkar, Nucl. Acids Res. 41, 1416 (2013) CrossRefGoogle Scholar
  35. 35.
    A. Varga, R. Moore, Hidden Markov model decomposition of speech and noise, in International Conference on Acoustics, Speech and Signal Processing (ICASSP) (1990), pp. 845–848 Google Scholar
  36. 36.
    B.H. Juang, L.R. Rabiner, Technometrics 33, 251 (1991) MathSciNetCrossRefGoogle Scholar
  37. 37.
    H. Veisi, H. Sameti, Speech Commun. 55, 205 (2013) CrossRefGoogle Scholar
  38. 38.
    R.P. Rao, N. Yadav, M.N. Vahia, H. Joglekar, R. Adhikari, I. Mahadevan, Proc. Natl. Acad. Sci. U. S. A. 106, 13685 (2009) ADSCrossRefGoogle Scholar
  39. 39.
    R.P. Rao, IEEE Comput. 43, 76 (2010) CrossRefGoogle Scholar
  40. 40.
    G.A. Fink, Markov models for pattern recognition: from theory to applications (Springer Science & Business Media, 2014) Google Scholar
  41. 41.
    G.V. Cormack, R. Horspool, Comput. J. 30, 541 (1987) CrossRefGoogle Scholar
  42. 42.
    H.S. Wang, N. Moayeri, IEEE Trans. Veh. Technol. 44, 163 (1995) CrossRefGoogle Scholar
  43. 43.
    H. Zhou, J. Bruck, IEEE Trans. Inf. Theor. 58, 2490 (2012) CrossRefGoogle Scholar
  44. 44.
    M. Svoboda, L. Lukas, Application of Markov chain analysis to trend prediction of stock indices, in Proceedings of 30th International Conference Mathematical Methodsin Economics (Silesian University, School of Business Administration, Karviná, 2012), pp. 848–853 Google Scholar
  45. 45.
    F.O. Mettle, E.N.B. Quaye, R.A. Laryea, SpringerPlus 3, 1 (2014) CrossRefGoogle Scholar
  46. 46.
    R. Gütig, Curr. Opin. Neurobiol. 25, 134 (2014) CrossRefGoogle Scholar
  47. 47.
    R. Brette, W. Gerstner, J. Neurophysiol. 94, 3637 (2005) CrossRefGoogle Scholar
  48. 48.
    R. Naud, N. Marcille, C. Clopath, W. Gerstner, Biol. Cybernet. 99, 335 (2008) MathSciNetCrossRefGoogle Scholar

Copyright information

© EDP Sciences and Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Consciousness Studies Programme, National Institute of Advanced Studies, Indian Institute of Science CampusBengaluruIndia
  2. 2.Department of Electronics and Communication EngineeringAmrita UniversityCoimbatoreIndia

Personalised recommendations