Principal Component Analysis

Introduction

Large or massive data sets are increasingly common and often include measurements on many variables. It is frequently possible to reduce the number of variables considerably while still retaining much of the information in the original data set. Principal component analysis (PCA) is probably the best known and most widely used dimension-reducing technique for doing this. Suppose we have n measurements on a vector x of p random variables, and we wish to reduce the dimension from p to q, where q is typically much smaller than p. PCA does this by finding linear combinations, a ₁ ′ x, a ₂ ′ x, …, a _q ′ x, called principal components, that successively have maximum variance for the data, subject to being uncorrelated with previous { a} _k ′ { x}s. Solving this maximization problem, we find that the vectors a ₁, a ₂, …, a _q are the eigenvectors of the covariance matrix, S, of the data, corresponding to the q largest eigenvalues (see Eigenvalue, Eigenvector and Eigenspace). The...