The Optimum Classifier and the Performance Evaluation by Bayesian Approach

  • Xuexian Han
  • Tetsushi Wakabayashi
  • Fumitaka Kimura
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)


This paper deals with the optimum classifier and the performance evaluation by the Bayesian approach. Gaussian population with unknown parameters is assumed. The conditional density given a limited sample of the population has a relationship to the multivariate t- distribution. The mean error rate of the optimum classifier is theoretically evaluated by the quadrature of the conditional density. To verify the optimality of the classifier and the correctness of the mean error calculation, the results of Monte Carlo simulation employing a new sampling procedure are shown. It is also shown that the Bayesian formulas of the mean error rate have the following characteristics. 1) The unknown population parameters are not required in its calculation. 2) The expression is simple and clearly shows the limited sample effect on the mean error rate. 3) The relationship between the prior parameters and the mean error rate is explicitly expressed.


Error Rate Monte Carlo Simulation Prior Distribution Bayesian Approach Mahalanobis Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973, p. 76, p.68.zbMATHGoogle Scholar
  2. 2.
    D. G. Keehn, “A Note on Learning for Gaussian Properties,” IEEE Trans. Inform. Theory, vol. IT-11, no. 1, pp. 126–132, Jan 1965.CrossRefMathSciNetGoogle Scholar
  3. 3.
    S. Raudys and V. Pikelis, “On Dimensionality, Sample Size, Classification Error, and Complexity of Classification Algorithm in Pattern Recognition,” IEEE Trans. PAMI, vol. PAMI-2, no. 3, pp. 242–252, May 1980.Google Scholar
  4. 4.
    K. Fukunaga and R. R. Hayes, ”Effects of Sample Size in Classifier Design,” IEEE Trans. PAMI, vol. 11, no. 8, pp. 873–885, Aug 1989.Google Scholar
  5. 5.
    K. Fukunaga, Introduction to Statistical Pattern Recognition, Second Edition, New York: Academic Press, 1990, pp. 392–393, pp.91–92.zbMATHGoogle Scholar
  6. 6.
    R. J. Muirhead, Aspects of Multivariate Statistical Theory. New York: Wiley, 1982, p. 32–49.zbMATHGoogle Scholar
  7. 7.
    S. J. Raudys and A.K. Jain, ”Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners,” IEEE Trans. PAMI, vol. 13, no. 3, pp. 252–264, Mar 1991.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Xuexian Han
    • 1
  • Tetsushi Wakabayashi
    • 1
  • Fumitaka Kimura
    • 1
  1. 1.Faculty of EngineeringMie UniversityTsuJapan

Personalised recommendations