A Comparative Study of Two Methods of Discrete Regularized Discriminant Analysis

Abstract

We discuss the problem of classifying an individual based on p-vector of binary variables. If p is large, it is difficult to estimate 2P cell probabilities when the sizes of the training samples are small or moderate. Celeux & Mkhadri (1991) and Kokolakis & Johnson (1989) recently proposed two alternative methods for addressing the problem of discrete discriminant analysis in the small sample setting. Celeux & Mkhadri method is an intermediate method between three classical discrete discriminat approachs, while Kokolakis & Johnson method is based on the Bayesian analysis. The common main aim of these methods is to regularize the Full Multinomial Model. This article details a critical comparison of these two methods, finding important strengths and weakness in both.