Skip to main content
SpringerLink
Log in

Entropy measures for biological signal analyses

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

Entropies are among the most popular and promising complexity measures for biological signal analyses. Various types of entropy measures exist, including Shannon entropy, Kolmogorov entropy, approximate entropy (ApEn), sample entropy (SampEn), multiscale entropy (MSE), and so on. A fundamental question is which entropy should be chosen for a specific biological application. To solve this issue, we focus on scaling laws of different entropy measures and introduce an ensemble forecasting framework to find the connections among them. One critical component of the ensemble forecasting framework is the scale-dependent Lyapunov exponent (SDLE), whose scaling behavior is found to be the richest among all the entropy measures. In fact, SDLE contains all the essential information of other entropy measures, and can act as a unifying multiscale complexity measure. Furthermore, SDLE has a unique scale separation property to aptly deal with nonstationarity and characterize high-dimensional and intermittent chaos. Therefore, SDLE can often be the first choice for exploratory studies in biology. The effectiveness of SDLE and the ensemble forecasting framework is illustrated by considering epileptic seizure detection from EEG.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Pincus, S.M.: Approximate entropy as a measure of system complexity. Proc. Natl. Acad. Sci. USA 88, 2297–2301 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  2. Cavanaugh, J.T., Guskiewicz, K.M., Giuliani, C., Marshal, S., Mercer, V., Stergiou, N.: Detecting altered postural-control after cerebral concussion in athletes with normal postural stability. Br. J. Sports Med. 39, 805–811 (2005)

    Article  Google Scholar 

  3. Sosnoff, J.J., Broglio, S.P., Shin, S.H., Ferrara, M.S.: Previous mild traumatic brain injury and postural-control dynamics. J. Athl. Train. 46, 85–91 (2011)

    Article  Google Scholar 

  4. National Center for Injury Prevention and Control: Report to Congress on Mild Traumatic Brain Injury in the United States: Steps to Prevent a Serious Public Health Problem. Centers for Disease Control and Prevention, Atlanta (2003)

    Google Scholar 

  5. Finkelstein, E.A., Corso, P.S., Miller, T.R.: The Incidence and Economic Burden of Injuries in the United States. Oxford University Press, New York (2006)

    Book  Google Scholar 

  6. Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K.M., Kelly, J., Lovell, M., McCrory, P., Meeuwiasse, W.H., Schamasch, P., Concussion in Sport (CIS) Group: Summary and agreement statement of the 1st international symposium on concussion in sport. Clin. J. Sport Med. 12, 6–11 (2002)

    Article  Google Scholar 

  7. Richman, J.S., Moorman, J.R.: Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol., Heart Circ. Physiol. 278, H2039–H2049 (2000)

    Google Scholar 

  8. McCrory, P., Meeuwisse, W., Johnston, K., Dvorak, J., Aubry, M., Molloy, M., Cantu, R.: Consensus statement on concussion in sport: The 3rd International Conference on Concussion in Sport held in Zurich, November 2008. J. Athl. Train. 44, 434–448 (2009)

    Article  Google Scholar 

  9. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991)

    Book  MATH  Google Scholar 

  10. Grassberger, P., Procaccia, I.: Estimation of the Kolmogorov entropy from a chaotic signal. Phys. Rev. A 28, 2591–2593 (1983)

    Article  Google Scholar 

  11. Costa, M., Goldberger, A.L., Peng, C.K.: Multiscale entropy analysis of biological signals. Phys. Rev. E 71, 021906 (2005)

    Article  MathSciNet  Google Scholar 

  12. Cao, Y.H., Tung, W.W., Gao, J.B., Protopopescu, V.A., Hively, L.M.: Detecting dynamical changes in time series using the permutation entropy. Phys. Rev. E 70, 046217 (2004)

    Article  MathSciNet  Google Scholar 

  13. Lempel, A., Ziv, J.: On the complexity of finite sequences. IEEE Trans. Inf. Theory 22, 75–81 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hu, J., Gao, J.B., Principe, J.C.: Analysis of biomedical signals by the Lempel-Ziv complexity: The effect of finite data size. IEEE Trans. Biomed. Eng. 53, 2606–2609 (2006)

    Article  Google Scholar 

  15. Gao, J.B., Cao, Y.H., Tung, W.W., Hu, J.: Multiscale Analysis of Complex Time Series—Integration of Chaos and Random Fractal Theory, and Beyond. Wiley, New York (2007)

    MATH  Google Scholar 

  16. Gao, J.B., Hu, J., Tung, W.-W., Cao, Y.H.: Distinguishing chaos from noise by scale-dependent Lyapunov exponent. Phys. Rev. E 74, 066204 (2006)

    Article  MathSciNet  Google Scholar 

  17. Andrzejak, R.G., Lehnertz, K., Rieke, C., Mormann, F., David, P., Elger, C.E.: Indications of nonlinear deterministic and finite dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Phys. Rev. E 64, 061907 (2001)

    Article  Google Scholar 

  18. Adeli, H., Ghosh-Dastidar, S., Dadmehr, N.: A wavelet-chaos methodology for analysis of EEGs and EEG subbands to detect seizure and epilepsy. IEEE Trans. Biomed. Eng. 54, 205–211 (2007)

    Article  Google Scholar 

  19. Gao, J.B., Hu, J., Tung, W.W.: Complexity measures of brain wave dynamics. Cogn. Neurodyn. 5, 171–182 (2011)

    Article  Google Scholar 

  20. Gao, J.B., Hu, J., Tung, W.W.: Facilitating joint chaos and fractal analysis of biosignals through nonlinear adaptive filtering. PLoS ONE 6(9), e24331 (2011). doi:10.1371/journal.pone.0024331

    Article  Google Scholar 

  21. Hu, J., Gao, J.B., Wang, X.S.: Multifractal analysis of sunspot time series: The effects of the 11-year cycle and Fourier truncation. J. Stat. Mech. 2009/02/P02066

  22. Gao, J.B., Sultan, H., Hu, J., Tung, W.W.: Denoising nonlinear time series by adaptive filtering and wavelet shrinkage: A comparison. IEEE Signal Process. Lett. 17, 237–240 (2010)

    Article  Google Scholar 

  23. Tung, W.W., Gao, J.B., Hu, J., Yang, L.: Recovering chaotic signals in heavy noise environments. Phys. Rev. E 83, 046210 (2011)

    Article  Google Scholar 

  24. Gao, J.B., Hu, J., Buckley, T., White, K., Hass, C.: Shannon and Renyi entropies to classify effects of mild traumatic brain injury on postural sway. PLoS ONE 6(9), e24446 (2011). doi:10.1371/journal.pone.0024446

    Article  Google Scholar 

  25. Packard, N.H., Crutchfield, J.P., Farmer, J.D., Shaw, R.S.: Geometry from a time series. Phys. Rev. Lett. 45, 712–716 (1980)

    Article  Google Scholar 

  26. Takens, F.: Detecting strange attractors in turbulence. In: Rand, D.A., Young, L.S. (eds.) Dynamical Systems and Turbulence, Lecture Notes in Mathematics, vol. 898, pp. 366–381. Springer, Berlin (1981)

    Google Scholar 

  27. Sauer, T., Yorke, J.A., Casdagli, M.: Embedology. J. Stat. Phys. 65, 579–616 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  28. Liebert, W., Pawelzik, K., Schuster, H.G.: Optimal embedding of chaotic attractors from topological considerations. Europhys. Lett. 14, 521–526 (1991)

    Article  MathSciNet  Google Scholar 

  29. Kennel, M.B., Brown, R., Abarbanel, H.D.I.: Determining embedding dimension for phase-space reconstruction using a geometrical construction. Phys. Rev. A 45, 3403–3411 (1992)

    Article  Google Scholar 

  30. Gao, J.B., Zheng, Z.M.: Local exponential divergence plot and optimal embedding of a chaotic time series. Phys. Lett. A 181, 153–158 (1993)

    Article  Google Scholar 

  31. Gao, J.B., Zheng, Z.M.: Direct dynamical test for deterministic chaos and optimal embedding of a chaotic time series. Phys. Rev. E 49, 3807–3814 (1994)

    Article  Google Scholar 

  32. Atmanspacher, H., Scheingraber, H.: A fundamental link between system theory and statistical mechanics. Found. Phys. 17, 939–963 (1987)

    Article  MathSciNet  Google Scholar 

  33. Gaspard, P., Wang, X.J.: Noise, chaos, and (ε,τ)-entropy per unit time. Phys. Rep. 235, 291–343 (1993)

    Article  MathSciNet  Google Scholar 

  34. Cohen, A., Procaccia, I.: Computing the Kolmogorov entropy from time series of dissipative and conservative dynamical systems. Phys. Rev. A 31, 1872–1882 (1985)

    Article  Google Scholar 

  35. Eckmann, J.P., Ruelle, D.: Ergodic theory of chaos and strange attractors. Rev. Mod. Phys. 57, 617–656 (1985)

    Article  MathSciNet  Google Scholar 

  36. Gao, J.B., Hu, J., Tung, W.W.: Multiscale entropy analysis of biological signals: A fundamental bi-scaling law. Europhys. Lett. (submitted)

  37. Lorenz, E.Z.: Deterministic nonperiodic flow. J. Atmos. Sci. 20, 130–141 (1963)

    Article  Google Scholar 

  38. Gao, J.B., Tung, W.W., Hu, J.: Quantifying dynamical predictability: The pseudo-ensemble approach (in honor of Professor Andrew Majda’s 60th birthday). Chin. Ann. Math., Ser. B 30, 569–588 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  39. Kleeman, R.: Measuring dynamical prediction utility using relative entropy. J. Atmos. Sci. 59, 2057–2072 (2002)

    Article  Google Scholar 

  40. Abramov, R., Majda, A., Kleeman, R.: Information theory and predictability for low frequency variability. J. Atmos. Sci. 62, 65–87 (2005)

    Article  MathSciNet  Google Scholar 

  41. Haven, K., Majda, A., Abramov, R.: Quantifying predictability through information theory: Small sample estimation in a non-Gaussian framework. J. Comput. Phys. 206, 334–362 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  42. Gao, J.B., Hwang, S.K., Liu, J.M.: When can noise induce chaos? Phys. Rev. Lett. 82, 1132–1135 (1999)

    Article  Google Scholar 

  43. Gao, J.B., Chen, C.C., Hwang, S.K., Liu, J.M.: Noise-induced chaos. Int. J. Mod. Phys. B 13, 3283–3305 (1999)

    Article  Google Scholar 

  44. Hwang, K., Gao, J.B., Liu, J.M.: Noise-induced chaos in an optically injected semiconductor laser. Phys. Rev. E 61, 5162–5170 (2000)

    Article  Google Scholar 

  45. Gao, J.B., Hu, J., Tung, W.W., Cao, Y.H., Sarshar, N., Roychowdhury, V.P.: Assessment of long range correlation in time series: How to avoid pitfalls. Phys. Rev. E 73, 016117 (2006)

    Article  Google Scholar 

  46. Gao, J.B., Hu, J., Tung, W.W., Zheng, Y.: Multiscale analysis of economic time series by scale-dependent Lyapunov exponent. Quant. Finance (2011). doi:10.1080/14697688.2011.580774

    Google Scholar 

  47. Gao, J.B., Hu, J., Mao, X., Tung, W.W.: Detecting low-dimensional chaos by the “noise titration” technique: Possible problems and remedies. Chaos Solitons Fractals (in press)

  48. Hu, J., Gao, J.B., Tung, W.W.: Characterizing heart rate variability by scale-dependent Lyapunov exponent. Chaos 19, 028506 (2009)

    Article  MathSciNet  Google Scholar 

  49. Hu, J., Gao, J.B., Tung, W.W., Cao, Y.H.: Multiscale analysis of heart rate variability: A comparison of different complexity measures. Ann. Biomed. Eng. 38, 854–864 (2010)

    Article  Google Scholar 

  50. Cellucci, C.J., Albano, A.M., Rapp, P.E., Pittenger, R.A., Josiassen, R.C.: Detecting noise in a time series. Chaos 7, 414–422 (1997)

    Article  MATH  Google Scholar 

  51. Hu, J., Gao, J.B., White, K.D.: Estimating measurement noise in a time series by exploiting nonstationarity. Chaos Solitons Fractals 22, 807–819 (2004)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PMB Intelligence LLC, PO Box 2077, West Lafayette, IN, 47996, USA

    Jianbo Gao

  2. Mechanical and Materials Engineering, Wright State University, Dayton, OH, 45435, USA

    Jianbo Gao

  3. Affymetrix, Inc., 3380 Central Expressway, Santa Clara, CA, 95051, USA

    Jing Hu

  4. Department of Earth & Atmospheric Sciences, Purdue University, West Lafayette, IN, 47907, USA

    Wen-wen Tung

Authors
  1. Jianbo Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wen-wen Tung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianbo Gao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gao, J., Hu, J. & Tung, Ww. Entropy measures for biological signal analyses. Nonlinear Dyn 68, 431–444 (2012). https://doi.org/10.1007/s11071-011-0281-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-011-0281-2

Keywords

Search

Navigation