Abstract
In multivariate discrimination of several normal populations, the optimal classification procedure is based on quadratic discriminant functions. We compare expected error rates of the quadratic classification procedure if the covariance matrices are estimated under the following four models: (i) arbitrary covariance matrices, (ii) common principal components, (iii) proportional covariance matrices, and (iv) identical covariance matrices. Using Monte Carlo simulation to estimate expected error rates, we study the performance of the four discrimination procedures for five different parameter setups corresponding to “standard” situations that have been used in the literature. The procedures are examined for sample sizes ranging from 10 to 60, and for two to four groups. Our results quantify the extent to which a parsimonious method reduces error rates, and demonstrate that choosing a simple method of discrimination is often beneficial even if the underlying model assumptions are wrong.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
ANDERSON, T. W. (1984),An Introduction to Multivariate Statistical Analysis, Wiley, New York.
CHATTERJEE, S., and NARAYANAN, A. (1992), “A New Method of Discrimination and Classification Using a Hausdorff Type Distance,”Australian Journal of Statistics, 34, 391–406.
DAVIES, R. B. (1973) “Numerical Inversion of a Characteristic Function,”Biometrika 60, 415–417.
DAVIES, R. B. (1980) “The Distribution of a Linear Combination of Chi-square Random Variables”, Algorithm AS 155,Applied Statistics, 29, 323–333.
EFRON, B. (1975), “The Efficiency of Logistic Regression Compared to Normal Discriminant Analysis”Journal of the American Statistical Association, 70, 892–898.
FLURY, B. (1986), “Proportionality ofk Covariance Matrices”Statistics and Probability Letters, 4, 29–33.
FLURY, B. (1988),Common Principal Components and Related Multivariate Models, Wiley, New York.
FLURY, B., and CONSTANTINE, G. (1985), “TheF-G Diagonalization Algorithm,” Algorithm AS 211,Applied Statistics, 34, 177–183.
FLURY, B. and SCHMID, M. (1992), “Quadratic Discriminant Functions with Constraints on the Covariance Matrices: Some Asymptotic Results,”Journal of Multivariate Analysis, 40, 244–261.
FRIEDMAN, J. H. (1989), “Regularized Discriminant Analysis,”Journal of the American Statistical Association, 84, 165–175.
GREENE, T., and RAYENS, W. S. (1989) “Partially Pooled Covariance Matrix Estimation in Discriminant Analysis,”Communications in Statistics-Theory and Methods, 18, 3679–3702.
IMHOF, J. P. (1961), “Computing the Distribution of Quadratic Forms in Normal Variables,”Biometrika, 48, 419–426.
KIRBY, S. P. J., THEOBALD, C. M., PIPER, J., and CAROTHEIS, A. D. (1991), “Some Methods of Combining Class Information in Multivariate Normal Discrimination for the Classification of Human Chromosomes,”Statistics in Medicine, 10, 141–149.
KNUTH, D. E. (1969)The Art of Computer Programming, Vol. 2, Addison-Wesley, Reading, Mass.
LACHENBRUCH, P. A. (1975),Discriminant Analysis, Hafner Press, New York.
MANLY, B. F. J., and RAYNER, J. C. W. (1987) “The Comparison of Sample Covariance Matrices Using Likelihood Ratio Tests,”Biometrika, 74, 841–847.
MARKS, S., and DUNN, O. J. (1974) “Discriminant Functions When the Covariance Matrices are Unequal,”Journal of the American Statistical Association, 69, 555–559.
MARCO, V. R., YOUNG, D. M. and TURNER, D. W. (1987) “The Euclidean Distance Classifier: An Alternative to the Linear Discriminant Function,”Communications in Statistics-B, Simulation and Computation, 16, 485–505.
McLACHILAN, G. J. (1992),Discriminant Analysis and Statistical Pattern Recognition, Wiley, New York.
O'NEILL, T. J. (1984), “A Theoretical Method of Comparing Classification Rules Under Non-optimal Conditions With Application to the Estimates of Fisher's Linear and Quadratic discriminant Rules Under Unequal Covariances Matrices” Technical report No. 217, Stanford University, Department of Statistics.
O'NEILL, T. J. (1992a), “The Bias of Fisher's Linear Discriminant Function when the Variances are not Equal,”Statistics and Probability Letters, 14, 205–210.
O'NEILL, T. J. (1992b), “Error Rates of Non-Bayes Classification Rules and Robustness of Fisher's Linear Discriminant Function,”Biometrika, 79, 177–184.
Panel on Discriminant Analysis, Classification, and Clustering (1989), “Discriminant Analysis and Clustering,”Statistical Science, 4, 34–69.
RAYENS, W.S. (1990), “A Rule for Covariance Stabilization in the Construction of the Classical 159–169
SCHMID, M. J. (1987), “Anwendungen der Theorie proportionaler Kovzarianzmatrizen und gemeinsamer Hauptkomponenten auf die quadratische Diskriminanzanalyse,” Unpublished Ph.D. thesis, University of Berne (Switzerland), Department of Statistics.
SEBER, G. A. F. (1984)Multivariate Observations, Wiley, New York.
SMITH, W. B. and HOCKING, R. R. (1972), “Wishart Variate Generator,” Algorithm AS 53,Applied Statistics, 21, 341–345.
WAHL, P. W., and KRONMAL, R. A. (1977) “Discriminant Functions when Covariances are Unequal and Sample Sizes are Moderate,”Biometrics, 33, 479–484.
WAKAKI, H. (1990), “Comparison of Linear and Quadratic Discriminant Functions,”Biometrika, 77, 227–229.
Author information
Authors and Affiliations
Additional information
The authors wish to thank the editor and three referees for their helpful comments on the first draft of this article. M. J. Schmid supported by grants no. 2.724-0.85 and 2.038-0.86 of the Swiss National Science Foundation.
Rights and permissions
About this article
Cite this article
Flury, B.W., Schmid, M.J. & Narayanan, A. Error rates in quadratic discrimination with constraints on the covariance matrices. Journal of Classification 11, 101–120 (1994). https://doi.org/10.1007/BF01201025
Issue Date:
DOI: https://doi.org/10.1007/BF01201025