Skip to main content

Entropy Analysis in Health Informatics

Part of the Intelligent Systems Reference Library book series (ISRL,volume 192)

Abstract

This chapter aims at presenting entropy measures that are now widely used in health informatics. Entropy measures have been introduced in 1990s and are derived from the theory of chaos. The traditional entropy-based analysis methods evaluate the degree of regularity of signals. We therefore present, first, the very well-known entropy measures used in the biomedical field: their theoretical background is detailed and some medical applications are presented. Then, more and very recent entropy measures are exposed and some of their used in health informatics are listed. This is performed from two points of view: for time series (uni-dimensional data) and for images (bi-dimensional data).

Keywords

  • Entropy
  • Irregularity
  • Biomedical application
  • Time series
  • Image
  • Texture

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-54932-9_5
  • Chapter length: 21 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-54932-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)

References

  1. Costa, M., Goldberger, A.L., Peng, C.K.: Multiscale entropy analysis of biological signals. Phys. Rev. E 71(2), Article id. 021906 (2005)

    Google Scholar 

  2. Costa, M., Goldberger, A.L., Peng, C.K.: Multiscale entropy analysis of complex physiologic time series. Phys. Rev. Lett. 89(6), Article id. 068102 (2002)

    Google Scholar 

  3. Pincus, S.M.: Approximate entropy as a measure of system complexity. Proc. Natl. Acad. Sci. 88(6), 2297–2301 (1991)

    MathSciNet  CrossRef  Google Scholar 

  4. Pincus, S.M., Goldberger, A.L.: Physiological time-series analysis: what does regularity quantify? Am. J. Physiol. Heart Circ. Physiol. 266(4), H1643–H1656 (1994)

    CrossRef  Google Scholar 

  5. Richman, J.S., Moorman, J.R.: Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol. Heart Circ. Physiol. 278(6), H2039–H2049 (2000)

    CrossRef  Google Scholar 

  6. Porta, A., Gnecchi-Ruscone, T., Tobaldini, E., Guzzetti, S., Furlan, R., Montano, N.: Progressive decrease of heart period variability entropy-based complexity during graded head-up tilt. J. Appl. Physiol. 103(4), 1143–1149 (2007)

    CrossRef  Google Scholar 

  7. Bansal, T., Haji, G.S., Rossiter, H.B., Polkey, M.I., Hull, J.H.: Exercise ventilatory irregularity can be quantified by approximate entropy to detect breathing pattern disorder. Respir. Physiol. Neurobiol. 255, 1–6 (2018)

    CrossRef  Google Scholar 

  8. Udhayakumar, R.K., Karmakar, C., Palaniswami, M.: Approximate entropy profile: a novel approach to comprehend irregularity of short-term HRV signal. Nonlinear Dyn. 88(2), 823–837 (2017)

    CrossRef  Google Scholar 

  9. Li, X., Jiang, Y., Hong, J., Dong, Y., Yao, L.: Estimation of cognitive workload by approximate entropy of EEG. J. Mech. Med. Biol. 16(06), 1650077 (2016)

    CrossRef  Google Scholar 

  10. Martinez-Cagigal, V., Santamaria-Vazquez, E., Hornero, R.: Asynchronous control of P300-based brain-computer interfaces using sample entropy. Entropy 21(3), Article id. 230 (2019)

    Google Scholar 

  11. Jia, Y., Gu, H.: Identifying nonlinear dynamics of brain functional networks of patients with schizophrenia by sample entropy. Nonlinear Dyn. 96, 2327–2340 (2019)

    Google Scholar 

  12. Phukan, N., Kakoty, N.M.: Sample entropy based selection of wavelet decomposition level for finger movement recognition using EMG. In: Progress in Advanced Computing and Intelligent Engineering (pp. 61–73). Springer, Singapore (2019)

    Google Scholar 

  13. Chen, W., Wang, Z., Xie, H., Yu, W.: Characterization of surface EMG signal based on fuzzy entropy. IEEE Trans. Neural Syst. Rehabil. Eng. 15(2), 266–272 (2007)

    CrossRef  Google Scholar 

  14. Xie, H.B., Chen, W.T., He, W.X., Liu, H.: Complexity analysis of the biomedical signal using fuzzy entropy measurement. Appl. Soft Comput. 11(2), 2871–2879 (2011)

    CrossRef  Google Scholar 

  15. Liu, C., Li, K., Zhao, L., Liu, F., Zheng, D., Liu, C., Liu, S.: Analysis of heart rate variability using fuzzy measure entropy. Comput. Biol. Med. 43(2), 100–108 (2013)

    CrossRef  Google Scholar 

  16. Acharya, U.R., Hagiwara, Y., Koh, J.E.W., Oh, S.L., Tan, J.H., Adam, M., San Tan, R.: Entropies for automated detection of coronary artery disease using ECG signals: A review. Biocybern. Biomed. Eng. 38(2), 373–384 (2018)

    CrossRef  Google Scholar 

  17. Tibdewal, M.N., Dey, H.R., Mahadevappa, M., Ray, A., Malokar, M.: Multiple entropies performance measure for detection and localization of multi-channel epileptic EEG. Biomed. Signal Process. Control 38, 158–167 (2017)

    CrossRef  Google Scholar 

  18. Xiang, J., Li, C., Li, H., Cao, R., Wang, B., Han, X., Chen, J.: The detection of epileptic seizure signals based on fuzzy entropy. J. Neurosci. Methods 243, 18–25 (2015)

    CrossRef  Google Scholar 

  19. Bandt, C., Pompe, B.: Permutation entropy: a natural complexity measure for time series. Phys. Rev. Lett. 88(17), 174102 (2002)

    CrossRef  Google Scholar 

  20. Cuesta-Frau, D., Miro-Martinez, P., Oltra-Crespo, S., Jordan-Nunez, J., Vargas, B., Vigil, L.: Classification of glucose records from patients at diabetes risk using a combined permutation entropy algorithm. Comput. Methods Programs Biomed. 165, 197–204 (2018)

    CrossRef  Google Scholar 

  21. Yang, Y., Zhou, M., Niu, Y., Li, C., Cao, R., Wang, B., Yan, P., Ma, Y., Xiang, J.: Epileptic seizure prediction based on permutation entropy. Front. Computat. Neurosci. 12, Article id. 55 (2018)

    Google Scholar 

  22. Zanin, M., Gomez-Andres, D., Pulido-Valdeolivas, I., Martin-Gonzalo, J., Lopez-Lopez, J., Pascual-Pascual, S., Rausell, E.: Characterizing normal and pathological gait through permutation entropy. Entropy 20(1), Article id. 77 (2018)

    Google Scholar 

  23. Humeau-Heurtier, A.: The multiscale entropy algorithm and its variants: A review. Entropy 17(5), 3110–3123 (2015)

    MathSciNet  CrossRef  Google Scholar 

  24. Liao, F., Jan, Y.K.: Using modified sample entropy to characterize aging-associated microvascular dysfunction. Front. Physiol. 7, Article 126 (2016)

    Google Scholar 

  25. Porta, A., Bari, V., De Maria, B., Cairo, B., Vaini, E., Malacarne, M., Pagani, M., Lucini, D.: On the relevance of computing a local version of sample entropy in cardiovascular control analysis. IEEE Trans. Biomed. Eng. 66(3), 623–631 (2019)

    CrossRef  Google Scholar 

  26. Omidvarnia, A., Mesbah, M., Pedersen, M., Jackson, G.: Range entropy: A bridge between signal complexity and self-similarity. Entropy 20(12), 962 (2018)

    CrossRef  Google Scholar 

  27. Liu, C., Oster, J., Reinertsen, E., Li, Q., Zhao, L., Nemati, S., Clifford, G.D.: A comparison of entropy approaches for AF discrimination. Physiol. Meas. 39(7), Article 074002 (2018)

    Google Scholar 

  28. Girault, J.M., Humeau-Heurtier, A.: Centered and averaged fuzzy entropy to improve fuzzy entropy precision. Entropy 20(4), Article id. 287 (2018)

    Google Scholar 

  29. Fadlallah, B., Chen, B., Keil, A., Principe, J.: Weighted-permutation entropy: a complexity measure for time series incorporating amplitude information. Phys. Rev. E 87(2), Article id. 022911 (2013)

    Google Scholar 

  30. Bian, Z., Ouyang, G., Li, Z., Li, Q., Wang, L., Li, X.: Weighted-permutation entropy analysis of resting state EEG from diabetics with amnestic mild cognitive impairment. Entropy 18(8), Article id. 307 (2016)

    Google Scholar 

  31. Azami, H., Escudero, J.: Amplitude-aware permutation entropy: illustration in spike detection and signal segmentation. Comput. Methods Programs Biomed. 128, 40–51 (2016)

    CrossRef  Google Scholar 

  32. Bandt, C.: A new kind of permutation entropy used to classify sleep stages from invisible EEG microstructure. Entropy 19(5), Article id. 197 (2017)

    Google Scholar 

  33. Tao, M., Poskuviene, K., Alkayem, N., Cao, M., Ragulskis, M.: Permutation entropy based on non-uniform embedding. Entropy 20(8), Article id. 612 (2018)

    Google Scholar 

  34. Li, C., Shang, P.: Multiscale Tsallis permutation entropy analysis for complex physiological time series. Physica A 523, 10–20 (2019)

    CrossRef  Google Scholar 

  35. Chen, Z., Li, Y., Liang, H., Yu, J.: Improved permutation entropy for measuring complexity of time series under noisy condition. Complexity 2019, Article id. 1403829 (2019)

    Google Scholar 

  36. Watt, S.J., Politi, A.: Permutation entropy revisited. Chaos, Soliton Fract 120, 95–99 (2019)

    MathSciNet  CrossRef  Google Scholar 

  37. Wang, X., Si, S., Wei, Y., Li, Y.: The optimized multi-scale permutation entropy and its application in compound fault diagnosis of rotating machinery. Entropy 21(2), Article id. 170 (2019)

    Google Scholar 

  38. Chen, S., Shang, P., Wu, Y.: Weighted multiscale Rényi permutation entropy of nonlinear time series. Physica A 496, 548–570 (2018)

    CrossRef  Google Scholar 

  39. Yin, Y., Shang, P.: Weighted permutation entropy based on different symbolic approaches for financial time series. Physica A 443, 137–148 (2016)

    CrossRef  Google Scholar 

  40. Li, P., Liu, C., Li, K., Zheng, D., Liu, C., Hou, Y.: Assessing the complexity of short-term heart beat interval series by distribution entropy. Med. Biol. Eng. Compu. 53(1), 77–87 (2015)

    CrossRef  Google Scholar 

  41. Zhang, T., Chen, W., Li, M.: Fuzzy distribution entropy and its application in automated seizure detection technique. Biomed. Signal Process. Control 39, 360–377 (2018)

    CrossRef  Google Scholar 

  42. Rostaghi, M., Azami, H.: Dispersion entropy: a measure for time-series analysis. IEEE Signal Process. Lett. 23(5), 610–614 (2016)

    CrossRef  Google Scholar 

  43. Liu, X., Jiang, A., Xu, N., Xue, J.: Increment entropy as a measure of complexity for time series. Entropy 18(1), 22 (2016)

    CrossRef  Google Scholar 

  44. Chanwimalueang, T., Mandic, D.: Cosine similarity entropy: self-correlation-based complexity analysis of dynamical systems. Entropy 19(12), Article 652 (2017)

    Google Scholar 

  45. Manis, G., Aktaruzzaman, M.D., Sassi, R.: Bubble entropy: an entropy almost free of parameters. IEEE Trans. Biomed. Eng. 64(11), 2711–2718 (2017)

    CrossRef  Google Scholar 

  46. Hsu, C., Wei, S.Y., Huang, H.P., Hsu, L., Chi, S., Peng, C.K.: Entropy of entropy: measurement of dynamical complexity for biological systems. Entropy 19(10), 550 (2017)

    CrossRef  Google Scholar 

  47. Humeau-Heurtier, A.: Texture feature extraction methods: a survey. IEEE Access 7, 8975–9000 (2019)

    CrossRef  Google Scholar 

  48. Moore, C.J.: A threshold structure metric for medical image interrogation: the 2D extension of approximate entropy. In: 2016 20th International Conference Information Visualisation (IV) (pp. 336–341). IEEE (July, 2016)

    Google Scholar 

  49. Yeh, J.R., Lin, C.W., Shieh, J.S.: An approach of multiscale complexity in texture analysis of lymphomas. IEEE Signal Process. Lett. 18(4), 239–242 (2011)

    CrossRef  Google Scholar 

  50. Silva, L.E.V., Senra Filho, A.C.S., Fazan, V.P.S., Felipe, J.C., Junior, L.M.: Two dimensional sample entropy: assessing image texture through irregularity. Biomed. Phys. Eng. Express 2(4), 045002 (2016)

    CrossRef  Google Scholar 

  51. Silva, L.E., Duque, J.J., Felipe, J.C., Murta Jr., L.O., Humeau-Heurtier, A.: Two-dimensional multiscale entropy analysis: applications to image texture evaluation. Sig. Process. 147, 224–232 (2018)

    CrossRef  Google Scholar 

  52. Humeau-Heurtier, A., Omoto, A.C.M., Silva, L.E.: Bi-dimensional multiscale entropy: relation with discrete Fourier transform and biomedical application. Comput. Biol. Med. 100, 36–40 (2018)

    CrossRef  Google Scholar 

  53. dos Santos, L.F.S., Neves, L.A., Rozendo, G.B., Ribeiro, M.G., do Nascimento, M.Z., Tosta, T.A.A.: Multidimensional and fuzzy sample entropy (SampEnMF) for quantifying H&E histological images of colorectal cancer. Comput. Biol. Med. 103, 148–160 (2018)

    Google Scholar 

  54. Hilal, M., Berthin, C., Martin, L., Azami, H., Humeau-Heurtier, A.: Bidimensional multiscale fuzzy entropy and its application to pseudoxanthoma elasticum. IEEE Trans. Biomed. Eng. (in press) (2019)

    Google Scholar 

  55. Azami, H., Escudero, J., Humeau-Heurtier, A.: Bidimensional distribution entropy to analyze the irregularity of small-sized textures. IEEE Signal Process. Lett. 24(9), 1338–1342 (2017)

    CrossRef  Google Scholar 

  56. Azami, H., da Silva, L.E.V., Omoto, A.C.M., Humeau-Heurtier, A.: Two-dimensional dispersion entropy: an information-theoretic method for irregularity analysis of images. Sig. Process. Image Commun. 75, 178–187 (2019)

    CrossRef  Google Scholar 

  57. Sigaki, H.Y., Perc, M., Ribeiro, H.V.: History of art paintings through the lens of entropy and complexity. Proc. Natl. Acad. Sci. 115(37), E8585–E8594 (2018)

    CrossRef  Google Scholar 

  58. Zunino, L., Ribeiro, H.V.: Discriminating image textures with the multiscale two-dimensional complexity-entropy causality plane. Chaos, Soliton. Fract. 91, 679–688 (2016)

    MathSciNet  CrossRef  Google Scholar 

  59. Ribeiro, H.V., Zunino, L., Lenzi, E.K., Santoro, P.A., Mendes, R.S.: Complexity-entropy causality plane as a complexity measure for two-dimensional patterns. PLoS ONE 7(8), e40689 (2012)

    CrossRef  Google Scholar 

  60. Moore, C., Marchant, T.: The approximate entropy concept extended to three dimensions for calibrated, single parameter structural complexity interrogation of volumetric images. Phys. Med. Biol. 62(15), 6092 (2017)

    CrossRef  Google Scholar 

  61. Marchant, T., Murphy, M., Madden, G., Moore, C.: Quantifying structure regularity in fluorescence microscopy cell images using a novel multi-dimensional approximate entropy metric. In: 2011, 18th IEEE International Conference on Image Processing (pp. 3085–3088). IEEE (September, 2011)

    Google Scholar 

  62. Gaudêncio, A.S.F., Vaz, P.G., Hilal, M., Cardoso, J.M., Mahé, G., Lederlin, M., Humeau-Heurtier, A.: Three-dimensional multiscale fuzzy entropy: validation and application to idiopathic pulmonary fibrosis. IEEE J. Biomed. Health Inf. (in press) (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anne Humeau-Heurtier .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Humeau-Heurtier, A. (2021). Entropy Analysis in Health Informatics. In: Ahad, M.A.R., Ahmed, M.U. (eds) Signal Processing Techniques for Computational Health Informatics. Intelligent Systems Reference Library, vol 192. Springer, Cham. https://doi.org/10.1007/978-3-030-54932-9_5

Download citation