Skip to main content
Book cover

Ibero-American Conference on Artificial Intelligence

IBERAMIA 2002: Advances in Artificial Intelligence — IBERAMIA 2002 pp 1–10Cite as

Improving Naive Bayes Using Class-Conditional ICA

  • Conference paper

    • We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

  • First Online:

Part of the Lecture Notes in Computer Science book series (LNAI,volume 2527)

Abstract

In the past years, Naive Bayes has experienced a renaissance in machine learning, particularly in the area of information retrieval. This classifier is based on the not always realistic assumption that class-conditional distributions can be factorized in the product of their marginal densities. On the other side, one of the most common ways of estimating the Independent Component Analysis (ICA) representation for a given random vector consists in minimizing the Kullback-Leibler distance between the joint density and the product of the marginal densities (mutual information). From this that ICA provides a representation where the independence assumption can be held on stronger grounds. In this paper we propose class-conditional ICA as a method that provides an adequate representation where Naive Bayes is the classifier of choice. Experiments on two public databases are performed in order to confirm this hypothesis.

Keywords

  • Feature Selection
  • Mutual Information
  • Independent Component Analysis
  • Feature Subset
  • Independent Component Analysis

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/3-540-36131-6_1
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   139.00
Price excludes VAT (USA)
  • ISBN: 978-3-540-36131-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   179.00
Price excludes VAT (USA)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Yang, Y., Slattery, S., Ghani, R.: A study of approaches to hypertext categorization. Journal of Intelligent Information Systems. Kluwer Academic Press (2002)

    Google Scholar 

  2. Lewis, D.: Naive bayes at forty: The independence assumption in information retrieval. In N’edellec, C., Rouveirol, C., eds.: Proceedings of ECML-98, 10th European Conference on Machine Learning. Volume 1398.25., Springer Verlag, Heidelberg, DE (1998) 4–15

    Google Scholar 

  3. Domingos, P., Pazzani, M.J.: On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning 29 (1997) 103–130

    MATH  CrossRef  Google Scholar 

  4. Turtle, H., Croft, W.: Evaluation of an inference network-based retrieval model. ACM Transactions on Information Systems 9 (1991) 187–222

    CrossRef  Google Scholar 

  5. Rish, I., Hellerstein, J., Thathachar, J.: An analysis of data characteristics that affect naive bayes performance. In N’edellec, C., Rouveirol, C., eds.: Proceedings of the Eighteenth Conference on Machine Learning-ICML2001, Morgan Kaufmann (2001)

    Google Scholar 

  6. M. Bressan, D. Guillamet, J. Vitria: Using an ica representation of high dimensional data for object recognition and classification. In: IEEE CSC in Computer Vision and Pattern Recognition (CVPR 2001). Volume 1. (2001) 1004–1009

    Google Scholar 

  7. Bell, A., Sejnowski, T.: An information-maximization approach for blind signal separation. Neural Computation 7 (1995) 1129–1159

    CrossRef  Google Scholar 

  8. Field, D.: What is the goal of sensory coding? Neural Computation 6 (1994) 559–601

    CrossRef  Google Scholar 

  9. Hyvärinen, A.: Sparse code shrinkage: Denoising of nongaussian data by maximum likelihood estimation. Neural Computation 11 (1999) 1739–1768

    CrossRef  Google Scholar 

  10. Vigario, R., Jousmäki, V., Hämäläinen, M., Hari, R., Oja, E.: Independent component analysis for identification of artifacts in magnetoencephalographic recordings. Advances in Neural Information Processing Systems 10 (1998) 229–235

    Google Scholar 

  11. Blake, C., Merz, C.: Uci repository of machine learning databases (1998)

    Google Scholar 

  12. LeCun, Y., Labs-Research, A.: The MNIST DataBase of Handwritten digits. http://www.research.att.com/ yann/ocr/mnist/index.html (1998)

  13. Scott, D.W.: Multivariate Density Estimation. John Wiley and sons, New York, NY (1992)

    MATH  Google Scholar 

  14. Duda, R., Hart, P., Stork, D.: Pattern Classication. John Wiley and Sons, Inc., New York, 2nd edition (2001)

    Google Scholar 

  15. Simpson, E.: The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, Ser. B 13 (1951) 238–241

    MATH  MathSciNet  Google Scholar 

  16. Bell, A., Sejnowski, T.: The ‘independent components’ of natural scenes are edge filters. Neural Computation 11 (1999) 1739–1768

    CrossRef  Google Scholar 

  17. Lee, T., Lewicki, M., Seynowski, T.: A mixture models for unsupervised classification of non-gaussian sources and automatic context switching in blind signal separation. IEEE Transactions on PAMI 22 (2000) 1–12

    Google Scholar 

  18. Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley and Sons (2001)

    Google Scholar 

  19. Comon, P.: Independent component analysis-a new concept? Signal Processing 36 (1994) 287–314

    MATH  CrossRef  Google Scholar 

  20. Hyvärinen, A.: New approximatins of differential entropy for independent component analysis and projection pursuit. Advances in Neural Processing Systems 10 (1998) 273–279

    Google Scholar 

  21. Marill, T., Green, D.: On the effectiveness of receptors in recognition systems. IEEE Trans. on Information Theory 9 (1963) 1–17

    CrossRef  Google Scholar 

  22. Kailath, T.: The divergence and bhattacharyya distance measures in signal selection. IEEE Trans. on Communication Technology COM-15 1 (1967) 52–60

    CrossRef  Google Scholar 

  23. Swain, P., King, R.: Two effective feature selection criteria for multispectral remote sensing. In: Proceedings of the 1st International Joint Conference on Pattern Recognition, IEEE 73 CHO821-9. (1973) 536–540

    Google Scholar 

  24. Swain, P., Davis, S.: Remote sensing: the quantitative approach. McGraw-Hill (1978)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre de Visiò per Computador, Dept. Informàtica, Universitat Autònoma de Barcelona, 08193, Bellaterra, Barcelona, Spain

    Marco Bressan & Jordi Vitrià

Authors
  1. Marco Bressan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jordi Vitrià
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Telefónica Investigación y Desarrollo, Emilio Vargas 6, 28043, Madrid, Spain

    Francisco J. Garijo

  2. Dpto. Lenguajes y Sistemas Informáticos, Universidad de Sevilla, ETS Ingeniería Informática, 41012, Seville, Spain

    José C. Riquelme & Miguel Toro & 

Rights and permissions

Reprints and Permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bressan, M., Vitrià, J. (2002). Improving Naive Bayes Using Class-Conditional ICA. In: Garijo, F.J., Riquelme, J.C., Toro, M. (eds) Advances in Artificial Intelligence — IBERAMIA 2002. IBERAMIA 2002. Lecture Notes in Computer Science(), vol 2527. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36131-6_1

Download citation

  • DOI: https://doi.org/10.1007/3-540-36131-6_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00131-7

  • Online ISBN: 978-3-540-36131-2

  • eBook Packages: Springer Book Archive

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.