A theoretical investigation of feature partitioning principal component analysis methods

Abstract

Principal Component Analysis (PCA) is a well-known linear dimensionality reduction technique in the literature. It extracts global principal components (PCs) and lacks in capturing local variations in its global PCs. To overcome the issues of PCA, Feature Partitioning based PCA (FP-PCA) methods were proposed; they extract local PCs from subpatterns and they are not sensitive to global variations across the subpatterns. Subsequently, SubXPCA was proposed as a novel FP-PCA approach which extracts PCs by utilizing both global and local information; it was proved to be efficient in terms of computational time and classification. It is observed that there is no detailed theoretical investigation done on the properties of FP-PCA methods. Such theoretical analysis is essential to provide generalized and formal validation of the properties of the FP-PCA methods. In this paper, our focus is to show SubXPCA as an alternative to PCA and other FP-PCA methods by proving analytically the various properties of SubXPCA related to summarization of variance, feature orders, and subpattern sizes. We prove analytically that (i) SubXPCA approaches PCA in terms of summarizing variance with increase in number of local principal components of subpatterns; (ii) SubXPCA is robust against feature orders (permutations) of patterns and variety of partitions (subpattern sizes); (iii) SubXPCA shows higher dimensionality reduction as compared to other FP-PCA methods. These properties of SubXPCA are demonstrated empirically upon UCI Waveform and ORL face data sets.