A New Approximation Method of the Quadratic Discriminant Function

  • Shin’ichiro Omachi
  • Fang Sun
  • Hirotomo Aso
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)

Abstract

For many statistical pattern recognition methods, distributions of sample vectors are assumed to be normal, and the quadratic discriminant function derived from the probability density function of multivariate normal distribution is used for classification. However, the computational cost is O(n 2) for n-dimensional vectors. Moreover, if there are not enough training sample patterns, covariance matrix can not be estimated accurately. In the case that the dimensionality is large, these disadvantages markedly reduce classification performance. In order to avoid these problems, in this paper, a new approximation method of the quadratic discriminant function is proposed. This approximation is done by replacing the values of small eigenvalues by a constant which is estimated by the maximum likelihood estimation. This approximation not only reduces the computational cost but also improves the classification accuracy.

Keywords

Probability Density Function Training Sample Small Eigenvalue Multivariate Normal Distribution Minimum Description Length 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Wiley & Sons, New York London Sydney Toronto (1973)MATHGoogle Scholar
  2. 2.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition. Second Edition, Academic Press, San Diego (1990)MATHGoogle Scholar
  3. 3.
    Takeshita, T., Kimura, F., Miyake, Y.: On the Estimation Error of Mahalanobis Distance. Trans. IEICE J70-D (1987) 567–573Google Scholar
  4. 4.
    Fix, E., Hodges, J.L.: Discriminatory analysis, nonparametric discrimination, consistency properties. Report No.4, School of Aviation Medicine, Randolph Field, Texas (1951)Google Scholar
  5. 5.
    Cover, T.M., Hart, P.E.: Nearest Neighbor Pattern Classification. IEEE Trans. Inform. Theory IT-13 (1967) 21–27CrossRefGoogle Scholar
  6. 6.
    Rosenblatt, M.: Remarks on some nonparametric estimates of a density function. Ann. Math. Stat. 27 (1956) 832–837MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Parzen, E.: On the estimation of a probability density function and the mode. Ann. Math. Stat. 33 (1962) 1065–1076MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Fukunaga, K., Hummels, D.M.: Bayes Error Estimation Using Parzen and k-NN Procedures. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9 (1987) 634–643CrossRefGoogle Scholar
  9. 9.
    Buturović, L.J.: Toward Bayes-Optimal Linear Dimension Reduction. IEEE Trans. Pattern Anal. Mach. Intell. 16 (1994) 420–424CrossRefGoogle Scholar
  10. 10.
    Kurita, M., Tsuruoka, S., Yokoi, S., Miyake, Y.: Handprinted “Kanji” and “Hira-gana” character recognition using weighting direction index histograms and quasi-Mahalanobis distance. IEICE Technical Report PRL82-79 (1983) 105–112Google Scholar
  11. 11.
    Kato, N., Abe, M., Nemoto, Y.: A Handwritten Character Recognition System Using Modified Mahalanobis Distance. Trans. IEICE J79-D-II (1996) 45–52Google Scholar
  12. 12.
    Kimura, F., Shridhar, M.: Handwritten numerical recognition based on multiple algorithms. Pattern Recognition 24 (1991) 969–983CrossRefGoogle Scholar
  13. 13.
    Akaike, H.: A New Look at the Statistical Model Identification. IEEE Trans. Automatic Control 19 (1974) 716–723MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Rissanen, J.: Stochastic Complexity and Modeling. The Annals of Statistics 14 (1986) 1080–1100MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Grother, P.J.: NIST Special Database 19 Handprinted Forms and Characters Database. National Institute of Standards and Technology. (1995)Google Scholar
  16. 16.
    Kato, N., Suzuki, M., Omachi, S., Aso, H., Nemoto, Y.: A Handwritten Character Recognition System Using Directional Element Feature and Asymmetric Mahalanobis Distance. IEEE Trans. Pattern Anal. Mach. Intell. 21 (1999) 258–262CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Shin’ichiro Omachi
    • 1
  • Fang Sun
    • 2
  • Hirotomo Aso
    • 1
  1. 1.Graduate School of EngineeringTohoku UniversitySendai-shiJapan
  2. 2.Faculty of Science and TechnologyTohoku Bunka Gakuen UniversitySendai-shiJapan

Personalised recommendations