Selection of the Number of Components Using a Genetic Algorithm for Mixture Model Classifiers

  • Hiroshi Tenmoto
  • Mineichi Kudo
  • Masaru Shimbo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)

Abstract

A genetic algorithm is employed in order to select the appropriate number of components for mixture model classifiers. In this classifier, each class-conditional probability density function can be approximated well using the mixture model of Gaussian distributions. Therefore, the classification performance of this classifier depends on the number of components by nature. In this method, the appropriate number of components is selected on the basis of class separability, while a conventional method is based on likelihood. The combination of mixture models is evaluated by a classification oriented MDL (minimum description length) criterion, and its optimization is carried out using a genetic algorithm. The effectiveness of this method is shown through the experimental results on some artificial and real datasets.

Keywords

mixture model classifier class-conditional probability density function class separability minimum description length criterion genetic algorithm 

References

  1. 1.
    Rissanen, J.: A Universal Prior for Integers and Estimation by Minimum Description Length. Annals of Statistics. 11 (1983) 416–431MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Tenmoto, H., Kudo, M., Shimbo, M.: MDL-Based Selection of the Number of Components in Mixture Models for Pattern Recognition. Lecture Notes in Computer Science 1451, Advances in Pattern Recognition. Springer (1998) 831–836CrossRefGoogle Scholar
  3. 3.
    Tenmoto, H., Kudo, M., Shimbo, M.: Determination of the Number of Components Based on Class Separability in Mixture-Based Classifiers. Proceedings of the Third International Conference on Knowledge-Based Intelligent Information Engineering Systems. (1999) 439–442Google Scholar
  4. 4.
    Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press (1975)Google Scholar
  5. 5.
    Dempster, A. P., Laird, N. M., Rubin, D. B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B. 39 (1977) 1–38MATHMathSciNetGoogle Scholar
  6. 6.
    Bezdek, J.C.: Cluster Validity with Fuzzy Sets. Journal of Cybernetics. 33(1974) 58–73CrossRefMathSciNetGoogle Scholar
  7. 7.
    Kudo, M., Shimbo, M.: Selection of Classifiers Based on the MDL Principle Using the VC Dimension. Proceedings of the 11th International Conference on Pattern Recognition. (1996) 886–890.Google Scholar
  8. 8.
    Park, Y., Sklansky, J.: Automated Design of Multiple-Class Piecewise Linear Classifiers. Journal of Classification. 6 (1989) 195–222MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Murphy, P. M., Aha, D.W.: UCI Repository of Machine Learning Databases [Machine-Readable Data Repository]. University of California Irvine, Department of Information and Computation Science. (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Hiroshi Tenmoto
    • 1
  • Mineichi Kudo
    • 2
  • Masaru Shimbo
    • 2
  1. 1.Department of Information EngineeringKushiro National College of TechnologyHokkaidoJapan
  2. 2.Division of Systems and Information Engineering, Graduate School of EngineeringHokkaido UniversitySapporoJapan

Personalised recommendations