Generalized Gini linear and quadratic discriminant analyses

Abstract

In this paper, a linear discriminant analysis (LDA) is performed in the Gini sense (GDA). Maximizing the generalized Gini gross between-group matrix allows the data to be projected onto discriminant axes. Different methods are investigated, the geometrical approach—based on a particular distance—and the probabilistic approach, which consists in employing the generalized Gini within-group matrix in order to compute the conditional probability of ranking observations in specific groups. Tests based on U-statistics are proposed in order to test for discriminant variables instead of using the well-known Student test that requires homoskedasticity. Monte Carlo simulations show the robustness and the superiority of the GDA on contaminated data compared with various linear classifiers such as logit, SVM and LDA.

Appendix: Proofs

Proof of Proposition 2.1

The Gini variability decomposition is:

$$\begin{aligned} GC_{\nu }({\textit{\textbf{X}}})&= -\frac{2\nu }{n(n-1)}{\textit{\textbf{Z}}}_\nu ^{\intercal }\textit{\textbf{R}}_{\mathbf {z},\nu }^c \\&= -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}(\mathbf {z}_{i.}^g - {\bar{\mathbf {z}}})^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}^{\nu -1}] \\&= -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}(\mathbf {z}_{i.}^g + {\bar{\mathbf {z}}}^g - {\bar{\mathbf {z}}}^g - {\bar{\mathbf {z}}})^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}^{\nu -1}] \\&= -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}(\mathbf {z}_{i.}^g - {\bar{\mathbf {z}}}^g)^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}^{\nu -1}] \\&\ \ \ \ -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}({\bar{\mathbf {z}}}^g - {\bar{\mathbf {z}}})^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}^{\nu -1}]. \end{aligned}$$

Since,

$$\begin{aligned} \sum ^{n_g}_{i=1}\sum ^{G}_{g=1}(\mathbf {z}_{i.}^g - {\bar{\mathbf {z}}}^g)^{\intercal }[{\bar{\mathbf{r }}}^{\nu -1}] = \sum ^{n_g}_{i=1}\sum ^{G}_{g=1}(\mathbf {z}_{i.}^g - {\bar{\mathbf {z}}}^g)^{\intercal }[{\bar{\mathbf{r }}}_{g.}^{\nu -1}]=\mathbf {0} \end{aligned}$$

(9)

then,

$$\begin{aligned} GC_{\nu }({\textit{\textbf{X}}})&= -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}(\mathbf {z}_{i.}^g - {\bar{\mathbf {z}}}^g)^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}_{g.}^{\nu -1}] \\&\ \ \ \ -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}({\bar{\mathbf {z}}}^g - {\bar{\mathbf {z}}})^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}^{\nu -1}] \\&= -\frac{2\nu }{n(n-1)}{\textit{\textbf{Z}}}_\nu ^{{\intercal }_w}\textit{\textbf{R}}^c_{\mathbf {z},w} -\frac{2\nu }{n(n-1)} {\textit{\textbf{Z}}}_\nu ^{{\intercal }_b}\textit{\textbf{R}}^c_{\mathbf {z},b} \\&= GC_{\nu ,w}({\textit{\textbf{X}}}) + GC_{\nu ,w}({\textit{\textbf{X}}}). \end{aligned}$$

Let $\bar{{\textit{\textbf{Z}}}_\nu }^g$ is the $n\times K$ matrix in which each line i corresponds to the mean ${\bar{\mathbf {z}}}^g$ of the ith observation of ${\textit{\textbf{Z}}}_\nu$ being in ${\mathscr {S}}_g$, then by Eq. (9):

$$\begin{aligned} -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}(\mathbf {z}_{i.}^g - {\bar{\mathbf {z}}}^g)^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}_{g.}^{\nu -1}]&= ({\textit{\textbf{Z}}}_\nu -\bar{{\textit{\textbf{Z}}}_\nu }^g)^{\intercal }\textit{\textbf{R}}_{\mathbf {z},\nu }^c \\&=GC_{\nu ,w}({\textit{\textbf{X}}}). \end{aligned}$$

By Eq. (9) again:

$$\begin{aligned} -\frac{2\nu }{n(n-1)}\sum ^{n_g}_{i=1}\sum ^{G}_{g=1}({\bar{\mathbf {z}}}^g - {\bar{\mathbf {z}}})^{\intercal }[\mathbf{r }_{ig.}^{\nu -1}-{\bar{\mathbf{r }}}^{\nu -1}]&= \bar{{\textit{\textbf{Z}}}_\nu }^{g \intercal }\textit{\textbf{R}}_{\mathbf {z},\nu }^c\\&=GC_{\nu ,b}({\textit{\textbf{X}}}), \end{aligned}$$

and this ends the proof. $\square$

Proof of Proposition 2.2

• (i) Since $GC_{\nu ,w}({\textit{\textbf{X}}})$ and $GC_{\nu ,b}({\textit{\textbf{X}}})$ are not symmetric matrices, it comes that maximizing

  $$\begin{aligned} \lambda :=\frac{\mathbf {u}^{\intercal } GC_{\nu ,b}({\textit{\textbf{X}}}) \mathbf {u}}{\mathbf {u}^{\intercal } GC_\nu ({\textit{\textbf{X}}}) \mathbf {u}} \end{aligned}$$

  yields,

  $$\begin{aligned} \frac{ [GC_{\nu ,b}({\textit{\textbf{X}}})+GC_{\nu ,b}({\textit{\textbf{X}}})^{\intercal }] \mathbf {u} \left[ \mathbf {u}^{\intercal } GC_\nu ({\textit{\textbf{X}}}) \mathbf {u} \right] - \left[ \mathbf {u}^{\intercal } GC_{\nu ,b}({\textit{\textbf{X}}}) \mathbf {u} \right] [GC_\nu ({\textit{\textbf{X}}})+GC_\nu ({\textit{\textbf{X}}})^{\intercal }] \mathbf {u}}{[\mathbf {u}^{\intercal } GC_\nu ({\textit{\textbf{X}}}) \mathbf {u}]^2} = 0. \end{aligned}$$

  This entails the following eigenvalue equation:

  $$\begin{aligned} \left[ GC_\nu ({\textit{\textbf{X}}}) + GC_\nu ({\textit{\textbf{X}}})^{\intercal } \right] ^{-1} \left[ GC_{\nu ,b}({\textit{\textbf{X}}}) + GC({\textit{\textbf{X}}})_{\nu ,b}^{\intercal } \right] \mathbf {u} = \lambda \mathbf {u}. \end{aligned}$$

• (ii) The other option is to maximize the gross between-group variability as a share of the within-group one, i.e.

  $$\begin{aligned} \mu := \frac{\mathbf {u}^{\intercal } GC_{\nu ,b}({\textit{\textbf{X}}}) \mathbf {u}}{\mathbf {u}^{\intercal } GC_{\nu ,w}({\textit{\textbf{X}}}) \mathbf {u}}, \end{aligned}$$

  then we get:

  $$\begin{aligned} \frac{ [GC_{\nu ,b}({\textit{\textbf{X}}})+GC_{\nu ,b}({\textit{\textbf{X}}})^{\intercal }] \mathbf {u} \left[ \mathbf {u}^{\intercal } GC_{\nu ,w}({\textit{\textbf{X}}}) \mathbf {u} \right] - \left[ \mathbf {u}^{\intercal } GC_{\nu ,b}({\textit{\textbf{X}}}) \mathbf {u} \right] [GC_{\nu ,w}({\textit{\textbf{X}}}) + GC_{\nu ,w}({\textit{\textbf{X}}})^{\intercal }] \mathbf {u}}{[\mathbf {u}^{\intercal } GC_{\nu ,w}({\textit{\textbf{X}}}) \mathbf {u}]^2} = 0. \end{aligned}$$

  Therefore, a Mahalanobis metric in the Gini sense is obtained thanks to the following eigenvalue equation,

  $$\begin{aligned} \left[ GC_{\nu ,w}({\textit{\textbf{X}}}) + GC_{\nu ,w}({\textit{\textbf{X}}})^{\intercal } \right] ^{-1} \left[ GC_{\nu ,b}({\textit{\textbf{X}}}) + GC({\textit{\textbf{X}}})_{\nu ,b}^{\intercal } \right] \mathbf {u} = \mu \mathbf {u}. \end{aligned}$$

$\square$

Proof of Proposition 3.2

From (8), since ${\bar{\mathbf {a}}}_{.\ell } = {\bar{\mathbf {f}}}_{.\ell } = 0$:

$$\begin{aligned} GMD_\nu (\mathbf {a}_{.\ell },\mathbf {z}_{.k})&= -\frac{2\nu }{n(n-1)}\sum _{i=1}^n (a_{i\ell } -{\bar{\mathbf {a}}}_{.\ell }) (r_{ik}^{\nu -1}-{\bar{\mathbf{r }}}_{.k}^{\nu -1}) \\&= -\frac{2\nu }{n(n-1)}\frac{\sum _{i=1}^n (f_{i\ell } -{\bar{\mathbf {f}}}_{.\ell }) (r_{ik}^{\nu -1}-{\bar{\mathbf{r }}}_{.k}^{\nu -1})}{GMD_\nu (\mathbf {f}_{.\ell })} \\&= \frac{GMD_\nu (\mathbf {f}_{.\ell },\mathbf {z}_{.k})}{GMD_\nu (\mathbf {f}_{.\ell })} \\&= GC_\nu (\mathbf {f}_{.\ell },\mathbf {z}_{.k}). \end{aligned}$$

Therefore, the analysis of the variability between $\mathbf {a}_{.\ell }$ and $\mathbf {z}_{.k}$ is simply the study of the correlation between $\mathbf {f}_{.\ell }$ and $\mathbf {z}_{.k}$:

$$\begin{aligned} \mathbf {V}= [GMD_\nu (\mathbf {a}_{.\ell },\mathbf {z}_{.k})] = [GC_\nu (\mathbf {f}_{.\ell },\mathbf {z}_{.k})]. \end{aligned}$$

$\square$