Principal component selection via adaptive regularization method and generalized information criterion

Abstract

The principal component analysis has been widely used in various fields of research (e.g., bioinformatics, medical statistics, etc.), especially high dimensional data analysis. Although crucial components selection is a vital matter in principal components analysis, relatively little attention was paid to this issue. The existing studies for principal component analysis were based on ad-hoc methods (e.g., method with cumulative percent variance or average eigenvalue). We propose a novel method for selecting principal component based on L$_{1}$-type regularized regression modeling. In order to effectively perform for principal component regression, we consider adaptive L$_{1}$-type penalty based on singular values of components, and propose adaptive penalized principal component regression. The proposed method can perform feature selection incorporating explanation power of components for not only high-dimensional predictor variables but also response variable. In sparse regression modeling, choosing the regularization parameter is a crucial issue, since feature selection and estimation heavily depend on the selected regularization parameter. We derive a model selection criterion for choosing the regularization parameter of the proposed adaptive L$_{1}$-type regularization method in line with a generalized information criterion. Monte Carlo simulations and real data analysis demonstrate that the proposed modeling strategies outperform for principal component regression modeling.