Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Estimation of Mutual Information: A Survey

  • Conference paper
Rough Sets and Knowledge Technology (RSKT 2009)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5589))

Included in the following conference series:

Abstract

A common problem found in statistics, signal processing, data analysis and image processing research is the estimation of mutual information, which tends to be difficult. The aim of this survey is threefold: an introduction for those new to the field, an overview for those working in the field and a reference for those searching for literature on different estimation methods. In this paper comparison studies on mutual information estimation is considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on the performance of different estimation methods and some future challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Francois, D., Wertz, V., Verleysen, M.: Proceedings of the European Symposium on Artificial Neural Networks (ESANN), pp. 239–244 (2006)

    Google Scholar 

  2. Daub, C.O., Steuer, R., Selbig, J., Kloska, S.: BMC Bioinformatics 5 (2004)

    Google Scholar 

  3. Trappenberg, T., Ouyang, J., Back, A.: Journal of Latex Class Files 1, 8 (2002)

    Google Scholar 

  4. Fransens, R., Strecha, C., Van Gool, L.: Proceedings of Theory and Applications of Knowledge-Driven Image Information Mining with Focus on Earth Observation (ESA-EUSC) (2004)

    Google Scholar 

  5. Hlaváčková-Schindler, K.: Information Theory and Statistical Learning. Springer Science & Business Media LLC (2009)

    Google Scholar 

  6. Rossi, F., Francois, D., Wertz, V., Meurens, M., Verleysen, M.: Chemometrics and Intelligent Laboratory Systems 86 (2007)

    Google Scholar 

  7. Aumônier, S.: Generalized Correlation Power Analysis in the workshop ECRYPT, Tools for Cryptanalysis (2007)

    Google Scholar 

  8. Vamucci, M.: Technical Reports. Duke University (1998)

    Google Scholar 

  9. Kraskov, A., Stögbauer, H., Andrzejak, R.G., Grassberger, P, http://arxiv.org/abs/q-bio/0311039

  10. Papana, A., Kugiumtzis, D.: Nonlinear Phenomena in Complex Systems 11, 2 (2008)

    Google Scholar 

  11. Darbellay, G.A., Vajda, I.: IEEE Transaction on Information Theory 45, 4 (1999)

    Google Scholar 

  12. Suzuki, T., Sugiyama, M, Sese, J., Kanamori, T.: Proceedings of JMLR: Workshop and Conference 4 (2008)

    Google Scholar 

  13. Endres, D., Földiák, P.: IEEE Transactions on Information Theory 51, 11 (2005)

    Google Scholar 

  14. Suzuki, T., Sugiyama, M., Sese, J., Kanamori, T.: Proceedings of the Seventh Asia-Pacific Bioinformatics Conference (APBC) (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing and Information Technology, University of Technology, Jamaica, Kingston 6, Jamaica

    Janett Walters-Williams

  2. Department of Mathematics and Computing, Centre for Systems Biology, University of Southern Queensland, QLD, 4350, Australia

    Janett Walters-Williams & Yan Li

Authors
  1. Janett Walters-Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Engineering and Surveying and Engineering, University of Southern Queensland, QLD 4350, Australia

    Peng Wen

  2. School of Information Technology, Queensland University of Technology, QLD 4001, Brisbane, Australia

    Yuefeng Li

  3. Institute of Mathematics, Warsaw University of Technology, Koszykowa 86, 02008, Warsaw, Poland

    Lech Polkowski

  4. Department of Computer Science, University of Regina, S4S 0A2, Regina, Saskatchewan, Canada

    Yiyu Yao

  5. Faculty of Medicine, Department of Medical Informatics, Shimane University, 89-1 Enya-cho, Izumo, 693-8501, Shimane, Japan

    Shusaku Tsumoto

  6. Institute of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China

    Guoyin Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Walters-Williams, J., Li, Y. (2009). Estimation of Mutual Information: A Survey. In: Wen, P., Li, Y., Polkowski, L., Yao, Y., Tsumoto, S., Wang, G. (eds) Rough Sets and Knowledge Technology. RSKT 2009. Lecture Notes in Computer Science(), vol 5589. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02962-2_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-02962-2_49

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-02961-5

  • Online ISBN: 978-3-642-02962-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation