Error Rate Estimation in Discriminant Analysis: Recent Advances

  • G. J. McLachlan
Chapter
Part of the Theory and Decision Library book series (TDLB, volume 5)

Abstract

An important problem in discriminant analysis is the estimation of the error rates associated with a given discriminant rule for allocating an object of unknown origin to one of a finite number, say g, of distinct classes or populations. The rule is based on the observed value of a random vector X of p measurements on the object. Over the years there have been many investigations on this problem; see, for example, Hills (1966), Lachenbruch and Mickey (1968), and McLachlan (1974a, b, c), and the references therein. Toussaint (1974) has compiled an extensive bibliography, which has been updated recently by Hand (1986b). An overview of error rate estimation has been given by McLachlan (1986), while recent work on robust error rate estimation has been summarized by Knoke (1986).

Keywords

Error Rate Mean Square Error Posterior Probability Allocation Rule Classification Error Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anderson, T.W. (1951). ‘Classification by multivariate analysis’. Psychometrika 16, 31–52.MathSciNetCrossRefGoogle Scholar
  2. Anderson, T.W. (1984). ‘An Introduction to Multivariate Statistical Analysis’. New York: Wiley.MATHGoogle Scholar
  3. Basford, K.E. and McLachlan, G.J. (1985). ‘Estimation of allocation rates in a cluster analysis context’. J. Amer. Statist. Assoc. 80, 286–293.MathSciNetCrossRefGoogle Scholar
  4. Chatterjee, S. and Chatterjee, S. (1983). ‘Estimation of misclassification probabilities by bootstrap methods’. Commun. Statist.-Simula. Computa. 12, 645–656.CrossRefGoogle Scholar
  5. Chernick, M.R., Murthy, V.K. and Nealy, C.D. (1985). ‘Application of bootstrap and other resampling techniques: evaluation of classifier performance’. Pattern Recognition Letters 3, 167–178.CrossRefGoogle Scholar
  6. Chernick, M.R., Murthy, V.K., and Nealy, C.D. (1986a). ‘Correction note to Application of bootstrap and other resampling techniques: evaluation of classifier performance’. Pattern Recognition Letters 4, 133–142.CrossRefGoogle Scholar
  7. Chernick, M.R., Murthy, V.K., and Nealy, C.D. (1986b). ‘Estimation of error rate for linear discriminant functions by resampling: Non-Gaussian populations’. Unpublished manuscript.Google Scholar
  8. Efron, B. (1979). ‘Bootstrap methods: another look at the jackknife’. Ann. Statist. 7, 1–26.MathSciNetMATHCrossRefGoogle Scholar
  9. Efron, B. (1982). ‘The Jackknife, the Bootstrap, and Other Resampling Plans’. Philadelphia: SIAM.Google Scholar
  10. Efron, B. (1983). ‘Estimating the error rate of a prediction rule: improvement on cross-validation’. J. Amer. Statist. Assoc. 78, 316–331.MathSciNetMATHCrossRefGoogle Scholar
  11. Efron, B. (1986). ‘How biased is the apparent error rate of a logistic regression?’ J. Amer. Statist. Assoc. 81, 461–470.Google Scholar
  12. Fukunaga, K. and Kessell, D.L. (1973). ‘Nonparametric Bayes error estimation using unclassified samples’. IEEE Trans. Inf. Theory IT-19, 434–440.Google Scholar
  13. Ganesalingam, S. and McLachlan, G.J. (1980). ‘Error rate estimation on the basis of posterior probabilities’. Pattern Recognition 12, 405–413.MATHCrossRefGoogle Scholar
  14. Glick, N. (1978). ‘Additive estimators for probabilities of correct classification’. Pattern Recognition 10, 211–222.MATHCrossRefGoogle Scholar
  15. Gong, G. (1986). ‘Cross-validation, the jackknife, and the bootstrap: excess error estimation in forward logistic regression’. J. Amer. Statist. Assoc. 81, 108–113.CrossRefGoogle Scholar
  16. Hall, P. (1986). ‘Cross-validation in nonparametric density estimation’. Proc. XIIIth Int. Biometric Conference, 15pp. Seattle: Biometric Society.Google Scholar
  17. Hand, D.J. (1982). ‘Kernel Discriminant Analysis’. Chichester: Wiley.MATHGoogle Scholar
  18. Hand, D.J. (1986a). ‘Cross-validation in error rate estimation’. Proc. XIIIth Int.Biometric Conference, 15pp. Seattle: Biometric Society.Google Scholar
  19. Hand, D.J. (1986b). ‘Recent advances in error rate estimation’. Pattern Recognition Letters (to appear).Google Scholar
  20. Hills, M. (1966). ‘Allocation rules and their error rates’. J. R. Statist. Soc. B 28, 1–31.MathSciNetMATHGoogle Scholar
  21. Hora, S.C. and Wilcox, J.B. (1982). ‘Estimation of error rates in several-population discriminant analysis’. J. Marketing Res. 19, 57–61.CrossRefGoogle Scholar
  22. Knoke, J.D. (1986). ‘The robust estimation of classification error rates’. Comp. Math, with Appls. 12A, 253–260.MATHCrossRefGoogle Scholar
  23. Krzanowski, W.J. (1975). ‘Discrimination and classification using both binary and continuous variables’. J. Amer. Statist. Assoc. 70, 782–792.MATHCrossRefGoogle Scholar
  24. Lachenbruch, P.A. and Mickey, M.R. (1968). ‘Estimation of error rates in discriminant analysis’. Technometrics 10, 1–11.MathSciNetCrossRefGoogle Scholar
  25. Matloff, N. and Pruitt, R. (1984). ‘The asymptotic distribution of an estimator of the Bayes error rate’. Pattern Recognition Letters 2, 271–274.MATHCrossRefGoogle Scholar
  26. McLachlan, G.J. (1973). ‘An asymptotic expansion of the expectation of the estimated error rate indiscriminant analysis’. Austral. J. Statist. 15, 210–214.MathSciNetMATHCrossRefGoogle Scholar
  27. McLachlan, G.J. (1974a). ‘Estimation of the errors of misclassification on the criterion of asymptotic mean square error’. Technometrics 16, 255–260.MathSciNetMATHCrossRefGoogle Scholar
  28. McLachlan, G.J. (1974b). ‘The relationship in terms of asymptotic mean square error between the separate problems of estimating each of the three types of error rate of the linear discriminant function’. Technometrics 16, 569–575.MathSciNetMATHCrossRefGoogle Scholar
  29. McLachlan, G.J. (1974c). ‘An asymptotic unbiased technique for estimating the error rates in discriminant analysis’. Biometrics 30, 239–249.MathSciNetMATHCrossRefGoogle Scholar
  30. McLachlan, G.J. (1976). ‘The bias of the apparent error rate in discriminant analysis’. Biometrika 63, 239–244.MathSciNetMATHCrossRefGoogle Scholar
  31. McLachlan, G.J. (1977). ‘A note on the choice of a weighting function to give an efficient method for estimating the probability of misclassification’. Pattern Recognition 9, 147–149.MATHCrossRefGoogle Scholar
  32. McLachlan, G.J. (1980). ‘The efficiency of Efron’s bootstrap approach applied to error rate estimation in discriminant analysis’. J. Statist. Comput. Simul. 11, 273–279.MATHCrossRefGoogle Scholar
  33. McLachlan, G.J. (1986). ‘Assessing the performance of an allocation rule’. Comp. Maths. with Appls. 12A, 261–272.MATHCrossRefGoogle Scholar
  34. Page, J.T. (1985). ‘Error-rate estimation in discriminant analysis’. Technometrics 27, 189–198.MathSciNetCrossRefGoogle Scholar
  35. Rao, P.S.R.S. and Dorvlo, A.S. (1985). The jackknife procedure for the probabilities of misclassification’. Commun. Statist.-Simula. Computa. 14, 779–790.CrossRefGoogle Scholar
  36. Schwemer, G.T. and Dunn, 0.J. (1980). ‘Posterior probability estimators in classification simulations’. Commun. Statist.-Simula. Computa. B9, 133–140.MathSciNetCrossRefGoogle Scholar
  37. Snapinn,S.M. and Knoke, J.D. (1984). ‘Classification error rate estimators evaluated by unconditional mean squared error’. Technometrics 26, 371–378.CrossRefGoogle Scholar
  38. Snapinn,S.M. and Knoke, J.D. (1985). ‘An evaluation of smoothed classification error-rate estimators’. Technometrics 27, 199–206.MathSciNetGoogle Scholar
  39. Toussaint, G.T. (1974). ‘Bibliography on estimation of misclassification’. IEEE Trans. Inf. Theory IT-20, 472–479.Google Scholar
  40. Toussaint, G.T. and Sharpe, P.M. (1975), ‘An efficient method for estimating the probability of misclassification applied to a problem in medical diagnosis’. Comput. Biol. Med. 4, 269–278.CrossRefGoogle Scholar
  41. Tutz, G.E. (1985). ‘Smoothed additive estimators for non-error rates in multiple discriminant analysis’. Pattern Recognition 18, 151–159.MATHCrossRefGoogle Scholar
  42. Vlachonikolis, I.G. (1986). ‘Estimation of the expected probability of misclassification’. Comp Maths. with Appls. 12A, 187–195.MATHCrossRefGoogle Scholar
  43. Wang, M-C. (1986). ‘Re-sampling procedures for reducing bias of error rate estimation in multinomial classification’. Comput. Statist. Data Analysis 4, 15–39.MATHCrossRefGoogle Scholar
  44. Wernecke, K-D and Kalb, G. (1983). ‘Further results in estimating the classification error in discriminance analysis’. Biom. J. 25, 247–258.Google Scholar
  45. Wernecke, K-D, Kalb, G., and Stürzebecher, E. (1980). ‘Comparison of various procedures for estimation of the classification error in discriminance analysis’. Biom. J. 22, 639–649.MATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1987

Authors and Affiliations

  • G. J. McLachlan
    • 1
  1. 1.Department of MathematicsUniversity of QueenslandSt. LuciaAustralia

Personalised recommendations