Skip to main content
SpringerLink
Log in

Conditional Entropy-Based Evaluation of Information Dynamics in Physiological Systems

  • Chapter
Directed Information Measures in Neuroscience

Part of the book series: Understanding Complex Systems ((UCS))

Abstract

We present a framework for quantifying the dynamics of information in coupled physiological systems based on the notion of conditional entropy (CondEn). First, we revisit some basic concepts of information dynamics, providing definitions of self entropy (SE), cross entropy (CE) and transfer entropy (TE) as measures of information storage and transfer in bivariate systems. We discuss also the generalization to multivariate systems, showing the importance of SE, CE and TE as relevant factors in the decomposition of the system predictive information. Then, we show how all these measures can be expressed in terms of CondEn, and devise accordingly a framework for their data-efficient estimation. The framework builds on a CondEn estimator that follows a sequential conditioning procedure whereby the conditioning vectors are formed progressively according to a criterion for CondEn minimization, and performs a compensation for the bias occurring for conditioning vectors of increasing dimension. The framework is illustrated on numerical examples showing its capability to deal with the curse of dimensionality in the multivariate computation of CondEn, and to reliably estimate SE, CE and TE in the challenging conditions of biomedical time series analysis featuring noise and small sample size. Finally, we illustrate the practical application of the presented framework to cardiovascular and neural time series, reporting some applicative examples in which SE, CE and TE are estimated to quantify the information dynamics of the underlying physiological systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bullmore, E., Sporns, O.: Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci. 10, 186 (2009)

    Article  Google Scholar 

  2. Bashan, A., Bartsch, R.P., Kantelhardt, J.W., Havlin, S., Ivanov, P.C.: Network physiology reveals relations between network topology and physiological function. Nat. Communicat. 3 (2012)

    Google Scholar 

  3. Lizier, J.T.: The local information dynamics of distributed computation in complex systems. Springer, Heidelberg (2013)

    Book  MATH  Google Scholar 

  4. Faes, L., Nollo, G.: Multivariate frequency domain analysis of causal interactions in physiological time series. In: Laskovski, A.N. (ed.) Biomedical Engineering, Trends in Electronics, Communications and Software. InTech, Rijeka (2011)

    Google Scholar 

  5. Pincus, S.M.: Approximate Entropy As A Measure of System-Complexity. Proc. Natl. Acad. Sci. USA 88, 2297–2301 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  6. Richman, J.S., Moorman, J.R.: Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol. Heart Circ. Physiol. 278, H2039–H2049 (2000)

    Google Scholar 

  7. Porta, A., Baselli, G., Liberati, D., Montano, N., Cogliati, C., Gnecchi-Ruscone, T., Malliani, A., Cerutti, S.: Measuring regularity by means of a corrected conditional entropy in sympathetic outflow. Biol. Cybern. 78, 71–78 (1998)

    Article  MATH  Google Scholar 

  8. Paluš, M., Komárek, V., Hrnčíř, Z., Štěrbová, K.: Synchronization as adjustment of information rates: detection from bivariate time series. Phys. Rev. E 63, 046211 (2001)

    Google Scholar 

  9. Porta, A., Baselli, G., Lombardi, F., Montano, N., Malliani, A., Cerutti, S.: Conditional entropy approach for the evaluation of the coupling strength. Biol. Cybern. 81, 119–129 (1999)

    Article  MATH  Google Scholar 

  10. Schreiber, T.: Measuring information transfer. Phys. Rev. Lett. 85, 461–464 (2000)

    Article  Google Scholar 

  11. Lizier, J.T., Pritam, S., Prokopenko, M.: Information Dynamics in Small-World Boolean Networks. Artificial Life 17, 293–314 (2011)

    Article  Google Scholar 

  12. Chicharro, D., Ledberg, A.: Framework to study dynamic dependencies in networks of interacting processes. Phys. Rev. E 86, 041901 (2012)

    Google Scholar 

  13. Porta, A., Guzzetti, S., Montano, N., Pagani, M., Somers, V., Malliani, A., Baselli, G., Cerutti, S.: Information domain analysis of cardiovascular variability signals: evaluation of regularity, synchronisation and co-ordination. Med. Biol. Eng. Comput. 38, 180–188 (2000)

    Article  Google Scholar 

  14. Nollo, G., Faes, L., Porta, A., Pellegrini, B., Ravelli, F., Del Greco, M., Disertori, M., Antolini, R.: Evidence of unbalanced regulatory mechanism of heart rate and systolic pressure after acute myocardial infarction. Am. J. Physiol. Heart Circ. Physiol. 283, H1200–H1207 (2002)

    Google Scholar 

  15. Porta, A., Faes, L., Mase, M., D’Addio, G., Pinna, G.D., Maestri, R., Montano, N., Furlan, R., Guzzetti, S., Nollo, G., Malliani, A.: An integrated approach based on uniform quantization for the evaluation of complexity of short-term heart period variability: Application to 24 h Holter recordings in healthy and heart failure humans. Chaos 17, 015117 (2007)

    Google Scholar 

  16. Porta, A., Gnecchi-Ruscone, T., Tobaldini, E., Guzzetti, S., Furlan, R., Montano, N.: Progressive decrease of heart period variability entropy-based complexity during graded head-up tilt. J. Appl. Physiol. 103, 1143–1149 (2007)

    Article  Google Scholar 

  17. Faes, L., Nollo, G., Porta, A.: Information domain approach to the investigation of cardio-vascular, cardio-pulmonary, and vasculo-pulmonary causal couplings. Front. Physiol. 2, 1–13 (2011)

    Article  Google Scholar 

  18. Faes, L., Nollo, G., Porta, A.: Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique. Phys. Rev. E 83, 051112 (2011)

    Google Scholar 

  19. Faes, L., Nollo, G., Porta, A.: Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series. Comput. Biol. Med. 42, 290–297 (2012)

    Article  Google Scholar 

  20. Faes, L., Nollo, G., Porta, A.: Compensated transfer entropy as a tool for reliably estimating information transfer in physiological time series. Entropy 15, 198–219 (2013)

    Article  MathSciNet  Google Scholar 

  21. Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley, New York (2006)

    MATH  Google Scholar 

  22. Kaiser, A., Schreiber, T.: Information transfer in continuous processes. Physica D 166, 43–62 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  23. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local measures of information storage in complex distributed computation. Information Sciences 208, 39–54 (2012)

    Article  Google Scholar 

  24. Porta, A., Catai, A.M., Takahashi, A.C.M., Magagnin, V., Bassani, T., Tobaldini, E., Montano, N.: Information Transfer through the Spontaneous Baroreflex in Healthy Humans. Meth. Inf. Med. 49, 506–510 (2010)

    Article  Google Scholar 

  25. Porta, A., Catai, A.M., Takahashi, A.C., Magagnin, V., Bassani, T., Tobaldini, E., de van, B.P., Montano, N.: Causal relationships between heart period and systolic arterial pressure during graded head-up tilt. Am. J. Physiol Regul. Integr. Comp. Physiol. 300, R378–R386 (2011)

    Google Scholar 

  26. Vicente, R., Wibral, M., Lindner, M., Pipa, G.: Transfer entropy-a model-free measure of effective connectivity for the neurosciences. Journal of Computational Neuroscience 30, 45–67 (2011)

    Article  MathSciNet  Google Scholar 

  27. Barnett, L., Barrett, A.B., Seth, A.K.: Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett. 103, 238701 (2009)

    Article  Google Scholar 

  28. Amblard, P.O., Michel, O.J.: The relation between Granger causality and directed information theory: a review. Entropy 15, 113–143 (2013)

    Article  MathSciNet  Google Scholar 

  29. Vakorin, V.A., Krakovska, O.A., McIntosh, A.R.: Confounding effects of indirect connections on causality estimation. J. Neurosci. Methods 184, 152–160 (2009)

    Article  Google Scholar 

  30. Williams, P.L.: Nonnegative decomposition of multivariate information. ArXiv, 1004.2515 (2010)

    Google Scholar 

  31. Schreiber, T.: Interdisciplinary application of nonlinear time series methods. Phys. Rep. 308, 1–64 (1999)

    Article  MathSciNet  Google Scholar 

  32. Takens, F.: Detecting strange attractors in fluid turbulence. In: Rand, D., Young, S.L. (eds.) Dynamical Systems and Turbulence. Springer, Berlin (1981)

    Google Scholar 

  33. Vlachos, I., Kugiumtzis, D.: Nonuniform state-space reconstruction and coupling detection. Phys. Rev. E 82, 016207 (2010)

    Google Scholar 

  34. Small, M.: Applied nonlinear time series analysis: applications in physics, physiology and finance. World Scientific (2005)

    Google Scholar 

  35. Runge, J., Heitzig, J., Petoukhov, V., Kurths, J.: Escaping the Curse of Dimensionality in Estimating Multivariate Transfer Entropy. Phys. Rev. Lett. 108, 258701 (2012)

    Article  Google Scholar 

  36. Pincus, S.M.: Approximated entropy (ApEn) as a complexity measure. Chaos, 110–117 (1995)

    Google Scholar 

  37. Hlaváčková-Schindler, K., Paluš, M., Vejmelka, M., Bhattacharya, J.: Causality detection based on information-theoretic approaches in time series analysis. Phys. Rep. 441, 1–46 (2007)

    Article  Google Scholar 

  38. Kugiumtzis, D., Tsimpiris, A.: Measures of Analysis of Time Series (MATS): A MATLAB Toolkit for Computation of Multiple Measures on Time Series Data Bases. J. Stat. Software 33, 1–30 (2010)

    Google Scholar 

  39. Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., Kaiser, J.: Transfer entropy in magnetoencephalographic data: Quantifying information flow in cortical and cerebellar networks. Progr. Biophys. Mol. Biol. 105, 80–97 (2011)

    Article  Google Scholar 

  40. Lungarella, M., Pegors, T., Bulwinkle, D., Sporns, O.: Methods for quantifying the informational structure of sensory and motor data. Neuroinformatics 3, 243–262 (2005)

    Article  Google Scholar 

  41. Porta, A., Castiglioni, P., Bari, V., Bassani, T., Marchi, A., Cividjian, A., Quintin, L., Di Rienzo, M.: K-nearest-neighbor conditional entropy approach for the assessment of the short-term complexity of cardiovascular control. Phys. Meas. 34, 17–33 (2013)

    Article  Google Scholar 

  42. Faes, L., Nollo, G.: Decomposing the transfer entropy to quantify lag-specific Granger causality in cardiovascular variability. In: Proc. of the 35th Annual Int. Conf. IEEE-EMBS, pp. 5049–5052 (2013)

    Google Scholar 

  43. Kugiumtzis, D.: Direct-coupling information measure from nonuniform embedding. Phys. Rev. E 87, 062918 (2013)

    Google Scholar 

  44. Porta, A., Guzzetti, S., Montano, N., Furlan, R., Pagani, M., Malliani, A., Cerutti, S.: Entropy, entropy rate, and pattern classification as tools to typify complexity in short heart period variability series. IEEE Trans. Biomed. Eng. 48, 1282–1291 (2001)

    Article  Google Scholar 

  45. Viola, A.U., Tobaldini, E., Chellappa, S.L., Casali, K.R., Porta, A., Montano, N.: Short-Term Complexity of Cardiac Autonomic Control during Sleep: REM as a Potential Risk Factor for Cardiovascular System in Aging. PLoS One 6 (2011)

    Google Scholar 

  46. Porta, A., Castiglioni, P., Di Rienzo, M., Bari, V., Bassani, T., Marchi, A., Wu, M.A., Cividjian, A., Quintin, L.: Information domain analysis of the spontaneous baroreflex during pharmacological challenges. Auton. Neurosci. 178(1-2), 67–75 (2013)

    Article  Google Scholar 

  47. Faes, L., Porta, A., Rossato, G., Adami, A., Tonon, D., Corica, A., Nollo, G.: Investigating the mechanisms of cardiovascular and cerebrovascular regulation in orthostatic syncope through an information decomposition strategy. Auton. Neurosci. 178(1-2), 76–82 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department Physics and BIOtech Center, University of Trento, Trento, Italy

    Luca Faes

  2. Department of Biomedical Sciences for Health, Galeazzi Orthopaedic Institute, University of Milan, Milan, Italy

    Alberto Porta

Authors
  1. Luca Faes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto Porta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca Faes .

Editor information

Editors and Affiliations

  1. Brain Imaging Center, Frankfurt am Main, Germany

    Michael Wibral

  2. Max-Planck Institute for Brain Research, Frankfurt am Main, Germany

    Raul Vicente

  3. CSIRO Computational Informatics, Marsfield, Sydney, Australia

    Joseph T. Lizier

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Faes, L., Porta, A. (2014). Conditional Entropy-Based Evaluation of Information Dynamics in Physiological Systems. In: Wibral, M., Vicente, R., Lizier, J. (eds) Directed Information Measures in Neuroscience. Understanding Complex Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-54474-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-54474-3_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-54473-6

  • Online ISBN: 978-3-642-54474-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation