A Quadratic Classifier for High-Dimension, Low-Sample-Size Data Under the Strongly Spiked Eigenvalue Model

Aki Ishii; Kazuyoshi Yata; Makoto Aoshima

doi:10.1007/978-3-030-28665-1_10

Workshop on Stochastic Models, Statistics and their Application

SMSA 2019: Stochastic Models, Statistics and Their Applications pp 131-142 | Cite as

A Quadratic Classifier for High-Dimension, Low-Sample-Size Data Under the Strongly Spiked Eigenvalue Model

Authors
Authors and affiliations

Aki Ishii
Kazuyoshi Yata
Makoto Aoshima

Conference paper

First Online: 16 October 2019

655 Downloads

Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 294)

Abstract

We consider a classifier for high-dimensional data under the strongly spiked eigenvalue (SSE) model. We create a new classification procedure on the basis of the high-dimensional eigenstructure. We propose a quadratic classification procedure by using a data transformation. We also prove that our proposed classification procedure has a consistency property for misclassification rates. We discuss performances of our classification procedure in simulations and real data analyses using microarray data sets.

Keywords

Classification Eigenstructure Geometrical quadratic discriminant analysis HDLSS Noise reduction methodology SSE model

This is a preview of subscription content, to check access.

Notes

Acknowledgements

We would like to thank an anonymous referee for his/her kind comments. Research of the first author was partially supported by Grant-in-Aid for Young Scientists, Japan Society for the Promotion of Science (JSPS), under Contract Number 18K18015. Research of the second author was partially supported by Grant-in-Aid for Scientific Research (C), JSPS, under Contract Number 18K03409. Research of the third author was partially supported by Grants-in-Aid for Scientific Research (A) and Challenging Research (Exploratory), JSPS, under Contract Numbers 15H01678 and 17K19956.

Appendix A: Estimation of Eigenstructures

Since $\lambda _{1(i)}$’s and $\varvec{h}_{1(i)}$’s are unknown, we estimate them by using the NR method. It is well known that the sample eigenvalues and eigenvectors include too much noise to have accuracy for high-dimensional data. See Jung and Marron [16], Ishii et al. [14] and Shen et al. [17] for the details. First, we consider estimating $\lambda _{1(i)}$’s by using the NR method. We denote the dual matrix of $\varvec{S}_{in_i}$ by $\varvec{S}_{iD}$ and define its eigen-decomposition as follows:

$$\begin{aligned} \varvec{S}_{iD}&=(n_i-1)^{-1}(\varvec{X}_i-\overline{\varvec{X}}_i)^{T}(\varvec{X}_i-\overline{\varvec{X}}_i) =\sum _{s=1}^{n_i-1}\hat{\lambda }_{s(i)}\hat{\varvec{u}}_{s(i)}\hat{\varvec{u}}_{s(i)}^{T}, \end{aligned}$$

(10)

where $\overline{\varvec{X}}_i=[\bar{\varvec{x}}_{in_i}, \ldots , \bar{\varvec{x}}_{in_i}]$ for $i=1,2$. If one uses the NR method, $\lambda _{j(i)}$’s are estimated by

$$\begin{aligned} \tilde{\lambda }_{j(i)}=\hat{\lambda }_{j(i)}-\frac{\text{ tr }(\varvec{S}_{iD})-\sum _{s=1}^{j}\hat{\lambda }_{s(i)} }{n_i-1-j}\quad (j=1,\ldots ,n_i-2). \nonumber \end{aligned}$$

Note that $\tilde{\lambda }_{j(i)}\ge 0$ w.p.1 for $j=1,\ldots ,n_i-2$. Yata and Aoshima [20, 21] showed that $\tilde{\lambda }_{j(i)}$ has consistency properties when $p\rightarrow \infty $ and $n_i\rightarrow \infty $. On the other hand, Ishii et al. [14] gave asymptotic distribution of $\tilde{\lambda }_{1(i)}$ when $p\rightarrow \infty $ while $n_i$ is fixed. By using the NR method, we estimate the first eigenvector as follows:

$$\begin{aligned} \tilde{\varvec{h}}_{1(i)}=\{(n_i-1)\tilde{\lambda }_{1(i)}\}^{-1/2}(\varvec{X}_i-\overline{\varvec{X}}_i)\hat{\varvec{u}}_{1(i)} \end{aligned}$$

for $i=1,2$, where $\hat{\varvec{u}}_{1(i)}$ is given in (10). Then, Ishii [13] gave the following results.

Lemma 2

([13]) Under (A-i) and (A-iii), it holds that as $p\rightarrow \infty $ for $i=1,2$

$$\begin{aligned} \frac{\tilde{\lambda }_{1(i)}}{\lambda _{1(i)}}=\frac{||\varvec{z}_{o1(i)}||^2}{n_i-1} +O_P\left( \frac{\delta _{i}^{1/2}}{ \lambda _{1(i)}}\right) \ \text{ and } \ \tilde{\varvec{h}}_{1(i)}^T{\varvec{h}}_{1(i)}=1+O_P\left( \frac{\delta _{i}^{1/2}}{ \lambda _{1(i)}}\right) . \end{aligned}$$

Remark 2

Yata and Aoshima [20, 21] and Ishii et al. [14] gave not only theoretical results but also many simulation results of the NR estimation. If readers are interested in the performances of the NR estimation numerically, see the above references for the details.

Appendix B: Proofs of Lemma 1 and Theorem 2

We assume (A-i)–(A-v). We first consider the case when $\varvec{x}_0 \in \pi _1$. Let $\phi =\min \{\Delta _\star /\lambda _{1(\max )}^{1/2},\Delta _{\star }^{1/2} \}$. From the proof of Theorem 4.1 in Ishii [13], we have that as $p\rightarrow \infty $

$$\begin{aligned} \tilde{\varvec{h}}_{1(i)}^T(\varvec{\mu }_1-\varvec{\mu }_2)=o_P(\phi ),\quad \tilde{\varvec{h}}_{1(i)}^T(\varvec{x}_0-\varvec{\mu }_{1})-{\varvec{h}}_{1}^T(\varvec{x}_0-\varvec{\mu }_{1})=o_P(\phi ) \nonumber \ \text{ for } i=1,2 \end{aligned}$$

and

$$\begin{aligned} \tilde{\varvec{h}}_{1(i)}^T(\bar{\varvec{x}}_{i'n_{i'}}-\varvec{\mu }_{i'})-{\varvec{h}}_{1}^T(\bar{\varvec{x}}_{i'n_{i'}}-\varvec{\mu }_{i'})=o_P(\phi ) \nonumber \ \text{ for } i,i'=1,2. \end{aligned}$$

Then, by noting that ${\varvec{h}}_{1}^T(\bar{\varvec{x}}_{i'n_{i'}}-\varvec{\mu }_{i'})=O_P(\lambda _{1(\max )}^{1/2})$ and ${\varvec{h}}_{1}^T(\varvec{x}_0-\varvec{\mu }_{1})=O_P(\lambda _{1(\max )}^{1/2})$, we have that for $i\ne i'$

$$ \{(\varvec{x}_{0}-\bar{\varvec{x}}_{in_i})^{T}\tilde{\varvec{h}}_{1(i')}\}^2- \{(\varvec{x}_{0}-\bar{\varvec{x}}_{in_i})^{T}{\varvec{h}}_{1}\}^2=o_P(\Delta _\star ). $$

Thus from Lemma 2, we have that

$$\begin{aligned} \widetilde{G}_{{ \text{ DT }}}(\varvec{x}_{0})/p=&\frac{||\varvec{A}(\varvec{x}_{0}-\bar{\varvec{x}}_{1n_1})||^{2}}{ \text{ tr }(\varvec{A}\varvec{S}_{1n_1})} -\frac{||\varvec{A}(\varvec{x}_{0}-\bar{\varvec{x}}_{2n_2})||^{2}}{\text{ tr }(\varvec{A}\varvec{S}_{2n_2})}\\&-\frac{1}{n_1}+\frac{1}{n_2} -\log \left\{ \frac{\text{ tr }(\varvec{A}\varvec{S}_{2n_2})}{\text{ tr }(\varvec{A}\varvec{S}_{1n_1})}\right\} +o_P(\Delta _\star /p) \end{aligned}$$

from the fact that $\text{ tr }(\varvec{A}\varvec{S}_{in_i})=\text{ tr }(\varvec{S}_{in_i})-\lambda _{1(i)}||\varvec{z}_{o1(i)}||^2/(n_i-1)$. We can conclude the result of Lemma 1 when $\varvec{x}_0 \in \pi _1$. For the case when $\varvec{x}_0\in \pi _2$, we can have the same arguments. It concludes the result of Lemma 1. From Lemma 1, similarly to the proof of Theorem 2 in Aoshima and Yata [2], we can conclude the result of Theorem 2.

References

1.
Aoshima, M., Yata, K.: Two-stage procedures for high-dimensional data. Seq. Anal. (Editor’s special invited paper) 30, 356–399 (2011)MathSciNetCrossRefGoogle Scholar
2.
Aoshima, M., Yata, K.: A distance-based, misclassification rate adjusted classifier for multiclass, high-dimensional data. Ann. Inst. Stat. Math. 66, 983–1010 (2014)MathSciNetCrossRefGoogle Scholar
3.
Aoshima, M., Yata, K.: Geometric classifier for multiclass, high-dimensional data. Seq. Anal. (Special Issue: Celebrating Seventy Years of Charles Stein’s 1945 Seminal Paper on Two-Stage Sampling) 34, 279–294 (2015)MathSciNetCrossRefGoogle Scholar
4.
Aoshima, M., Yata, K.: Two-sample tests for high-dimension, strongly spiked eigenvalue models. Stat. Sin. 28, 43–62 (2018)Google Scholar
5.
Aoshima, M., Yata, K.: High-dimensional quadratic classifiers in non-sparse settings. Methodol. Comput. Appl. Probab. (2018). https://doi.org/10.1007/s11009-018-9646-z
6.
Aoshima, M., Yata, K.: Distance-based classifier by data transformation for high-dimension, strongly spiked eigenvalue models. Ann. Inst. Stat. Math. 71, 473–503 (2019). https://doi.org/10.1007/s10463-018-0655-z
7.
Bai, Z., Saranadasa, H.: Effect of high dimension: by an example of a two sample problem. Stat. Sin. 6, 311–329 (1996)MathSciNetzbMATHGoogle Scholar
8.
Bickel, P.J., Levina, E.: Some theory for Fisher’s linear discriminant function, ‘naive Bayes’, and some alternatives when there are many more variables than observations. Bernoulli 10, 989–1010 (2004)MathSciNetCrossRefGoogle Scholar
9.
Borovecki, F., Lovrecic, L., Zhou, J., Jeong, H., Then, F., Rosas, H.D., Hersch, S.M., Hogarth, P., Bouzou, B., Jensen, R.V., Krainc, D.: Genome-wide expression profiling of human blood reveals biomarkers for Huntington’s disease. Proc. Natl. Acad. Sci. U.S.A. 102, 11023–11028 (2005)CrossRefGoogle Scholar
10.
Chan, Y.-B., Hall, P.: Scale adjustments for classifiers in high-dimensional, low sample size settings. Biometrika 96, 469–478 (2009)MathSciNetCrossRefGoogle Scholar
11.
Chen, S.X., Qin, Y.-L.: A two-sample test for high-dimensional data with applications to gene-set testing. Ann. Stat. 38, 808–835 (2010)MathSciNetCrossRefGoogle Scholar
12.
Dudoit, S., Fridlyand, J., Speed, T.P.: Comparison of discrimination methods for the classification of tumors using gene expression data. J. Am. Stat. Assoc. 97, 77–87 (2002)MathSciNetCrossRefGoogle Scholar
13.
Ishii, A.: A classifier under the strongly spiked eigenvalue model in high-dimension, low-sample-size context. Commun. Stat. Theory Methods (2019)Google Scholar
14.
Ishii, A., Yata, K., Aoshima, M.: Asymptotic properties of the first principal component and equality tests of covariance matrices in high-dimension, low-sample-size context. J. Stat. Plan. Inference 170, 186–199 (2016)MathSciNetCrossRefGoogle Scholar
15.
Ishii, A., Yata, K., Aoshima, M.: Equality tests of high-dimensional covariance matrices under the strongly spiked eigenvalue model. J. Stat. Plan. Inference 202, 99–111 (2019). https://doi.org/10.1016/j.jspi.2019.02.002
16.
Jung, S., Marron, J.S.: PCA consistency in high dimension, low sample size context. Ann. Stat. 37, 4104–4130 (2009)MathSciNetCrossRefGoogle Scholar
17.
Shen, D., Shen, H., Zhu, H., Marron, J.S.: The statistics and mathematics of high dimension low sample size asymptotics. Stat. Sin. 26, 1747–1770 (2016)MathSciNetzbMATHGoogle Scholar
18.
Singh, D., Febbo, P.G., Ross, K., Jackson, D.G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A.A., D’Amico, A.V., Richie, J.P., Lander, E.S., Loda, M., Kantoff, P.W., Golub, T.R., Sellers, W.R.: Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 1, 203–209 (2002)CrossRefGoogle Scholar
19.
Srivastava, M.S.: Minimum distance classification rules for high dimensional data. J. Multivar. Anal. 97, 2057–2070 (2006)MathSciNetCrossRefGoogle Scholar
20.
Yata, K., Aoshima, M.: Effective PCA for high-dimension, low-sample-size data with noise reduction via geometric representations. J. Multivar. Anal. 105, 193–215 (2012)MathSciNetCrossRefGoogle Scholar
21.
Yata, K., Aoshima, M.: PCA consistency for the power spiked model in high-dimensional settings. J. Multivar. Anal. 122, 334–354 (2013)MathSciNetCrossRefGoogle Scholar

Copyright information

Authors and Affiliations

Aki Ishii
- 1
Kazuyoshi Yata
- 2
Email author
Makoto Aoshima
- 2

1.Department of Information SciencesTokyo University of ScienceChibaJapan
2.Institute of MathematicsUniversity of TsukubaIbarakiJapan

A Quadratic Classifier for High-Dimension, Low-Sample-Size Data Under the Strongly Spiked Eigenvalue Model

Abstract

Keywords

Notes

Acknowledgements

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite paper