A Comparative Study of PCA, ICA and Class-Conditional ICA for Naïve Bayes Classifier

  • Liwei Fan
  • Kim Leng Poh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4507)


The performance of the Naïve Bayes classifier can be improved by appropriate preprocessing procedures. This paper presents a comparative study of three preprocessing procedures, namely Principle Component Analysis (PCA), Independent Component Analysis (ICA) and class-conditional ICA, for Naïve Bayes classifier. It is found that all the three procedures keep improving the performance of the Naïve Bayes classifier with the increase of the number of attributes. Although class-conditional ICA has been found to be superior to PCA and ICA in most cases, it may not be suitable for the case where the sample size for each class is not large enough.


Classification Bayesian Network Naïve Bayes Classifier Independent Component Analysis Principle Component Analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Langley, P., Iba, W., Thompson, K.: An Analysis of Bayesian Classifiers. In: Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 223–228. AAAI Press, San Jose (1992)Google Scholar
  2. 2.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29, 131–163 (1997)CrossRefzbMATHGoogle Scholar
  3. 3.
    Sandberg, R., Winberg, G., Bränden, C., Kaske, A., Ernberg, I., Cöster, J.: Capturing Whole-Genome Characteristics in Short Sequences Using a Naïve Bayesian Classifier. Genome Research 11, 1404–1409 (2001)CrossRefGoogle Scholar
  4. 4.
    Rennie, J.D.M., Shih, L., Teevan, J., Karger, D.R.: Tackling the Poor Assumptions of Naïve Bayes Text Classifiers. In: Fawcett, T., Mishra, N. (eds.) Proceedings of the Twentieth International Conference on Machine Learning, pp. 616–623. AAAI Press, Menlo Park (2003)Google Scholar
  5. 5.
    Li, Y., Anderson-Sprecher, R.: Facies Identification from Well Logs: A Comparison of Discriminant Analysis and Naïve Bayes Classifier. Journal of Petroleum Science and Engineering 53, 149–157 (2006)CrossRefGoogle Scholar
  6. 6.
    Cheng, J., Greiner, R.: Comparing Bayesian Network Classifiers. In: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI’99), pp. 101–107. Morgan Kaufmann Publishers, Washington (1999)Google Scholar
  7. 7.
    Gupta, G.K.: Principal Component Analysis and Bayesian Classifier Based on Character Recognition. In: Gary, E., Zhai, Y. (eds.) Bayesian Inference and Maximum Entropy Methods in Science and Engineering. AIP Conference Proceedings, vol. 707, pp. 465–479. Springer, Berlin (2004)Google Scholar
  8. 8.
    Prasad, M.N., Sowmya, A., Koch, I.: Feature Subset Selection using ICA for Classifying Emphysema in HRCT Images. In: Kittler, J., Petrou, M., Nixon, M.S. (eds.) Proceedings of the 17th International Conference on Pattern Recognition, pp. 515–518. IEEE-CS Press, Washington (2004)CrossRefGoogle Scholar
  9. 9.
    Bressan, M., Vitria, J.: Improving Naïve Bayes Using Class-conditional ICA. In: Garijo, F.J., Riquelme, J.-C., Toro, M. (eds.) IBERAMIA 2002. LNCS (LNAI), vol. 2527, pp. 1–10. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  10. 10.
    Vitria, J., Bressan, M., Radeva, P.: Bayesian Classification of Cork Stoppers Using Class-conditional Independent Component Analysis. IEEE Transactions on Systems, Man and Cybernetics C 37, 32–38 (2007)CrossRefGoogle Scholar
  11. 11.
    Haykin, S.: Neural Netwoks: A Comprehensive Foundation. Prentice Hall, Englewood Cliffs (1999)zbMATHGoogle Scholar
  12. 12.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley and Sons, New York (2001)Google Scholar
  13. 13.
    Hyvärinen, A., Oja, E.: Independent Component Analysis: Algorithms and Applications. Neural Networks 13, 411–430 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Liwei Fan
    • 1
  • Kim Leng Poh
    • 1
  1. 1.Department of Industrial and Systems Engineering, National University of Singapore, 10 Kent Ridge Crescent 119260Singapore

Personalised recommendations