# Principal Component Analysis

• Jianzhong Wang

## Abstract

Among linear DR methods, principal component analysis (PCA) perhaps is the most important one. In linear DR, the dissimilarity of two points in a data set is defined by the Euclidean distance between them, and correspondingly, the similarity is described by their inner product. Linear DR methods adopt the global neighborhood system: the neighbors of a point in the data set consist of all of other points. Let the original data set be H= {x 1 ··· x n ⊂ ℝ D and the DR data set of H be a d-dimensional set Y. Under the Euclidean measure, PCA finds a linear projection T: ℝ D → ℝ d so that the DR data Y = T(H) maximize the data energy. PCA is widely used in many applications. The present chapter is organized as follows. In Section 5.1, we discuss the description of PCA. In Section 5.2, we present the PCA algorithms. Some real-world applications of PCA are introduced in Section 5.3.

## Keywords

Principal Component Analysis Face Recognition Singular Value Decomposition Face Image Principal Direction
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

1. [1]
Pearson, K.: On lines and planes of closest fit to systems of points in space. Philosophical Magazine 2(6), 559–572 (1901).Google Scholar
2. [2]
Jolliffe, I.T.: Principal Component Analysis. Springer Series in Statistics. Springer-Verlag, Berlin (1986).Google Scholar
3. [3]
Rao, C., Rao, M.: Matrix Algebra and Its Applications to Statistics and Econometric. World Scientific, Singapore (1998).Google Scholar
4. [4]
Lehoucq, R., Sorensen, D.: Deflation techniques for an implicitly re-started arnoldi iteration. SIAM J. Matrix Analysis and Applications 17, 789–821 (1996).
5. [5]
Barrett, R., Berry, M.W., Chan, T.F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., van der Vorst, H.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. SIAM Publisher (1993).Google Scholar
6. [6]
Arnoldi, W.E.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quarterly of AppliedMathematics 9, 17–29 (1951).
7. [7]
Lehoucq, R., Sorensen, D., Yang, C.: ARPACK Users’ Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods. SIAM Publications, Philadelphia (1998).Google Scholar
8. [8]
Sorensen, D.: Implicit application of polynomial filters in a k-step arnoldi method. SIAM J. Matrix Analysis and Applications 13, 357–385 (1992).
9. [9]
Roweis, S.: EM algorithms for PCA and SPCA. In: anips, vol. 10, pp. 626–632 (1998).Google Scholar
10. [10]
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B 39(1), 1–38 (1977).
11. [11]
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. TPAMI 7(19), 711–720 (1997).
12. [12]
Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: Illumination cone models for face recognition under variable lighting and pose. TPAMI 6(23), 643–660 (2001).
13. [13]
Wang, W., Song, J., Yang, Z., Chi, Z.: Wavelet-based illumination compensation for face recognition using eigenface method. In: Proceedings of Intelligent Control and Automation. Dalian, China (2006).Google Scholar
14. [14]
Chen, T., Zhou, X.S., Comaniciu, D., Huang, T.: Total variation models for variable lighting face recognition. TPAMI 9(28), 1519–1524 (2006).
15. [15]
Wang, H.T., Li, S.Z., Wang, Y.S.: Face recognition under varying lighting conditions using self quotient image. In: Automatic Face and Gesture Recognition, International Conference on FGR, pp. 819–824 (2004).Google Scholar
16. [16]
Basri, R., Jacobs, D.: Photometric stereo with general, unknown lighting. In: IEEE Conference on CVPR, 374–381 (2001).Google Scholar
17. [17]
Horn, B.: The Psychology of Computer Vision, chap. 4. Obtaining Shape from Shading Information and chap 6. Shape from Shading, pp. 115–155. McGraw-Hill, New York (1975).Google Scholar
18. [18]
Xie, X.D., Lam, K.: Face recognition under varying illumination based on a 2D face shape model. Journal of Pattern Recognition 2(38), 221–230 (2005).Google Scholar
19. [19]
Hu, Y.K., Wang, Z.: A low-dimensional illumination space representation of human faces for arbitrary lighting conditions. In: Proceedings of ICPR, pp. 1147–1150. Hong Kong (2006).Google Scholar
20. [20]
Ramamoorthi, R.: Analytic PCA construction for theoretical analysis of lighting variability in images of a lambertian object. TPAMI 10(24), 1322–1333 (2002).
21. [21]
Wang, H.T., Li, S.Z., Wang, Y.: Generalized quotient image. In: Proceeding of CVPR (2004).Google Scholar
22. [22]
Xie, X., Zheng, W.S., Lai, J., Yuen, P.C.: Face illumination normalization on large and small scale features. In: IEEE Coference on CVPR (2008).Google Scholar
23. [23]
Xie, X., Lai, J., Zheng, W.S.: Extraction of illumination invariant facial features from a single image using nonsubsampled contourlet transform. Pattern Recognition 43(12), 4177–4189 (2010).
24. [24]
Xie, X., Zheng, W.S., Lai, J., Yuen, P.C., Suen, C.Y.: Normalization of face illumination based on large-and small-scale features. IEEE Trans. on Image Processing (2010). Accepted.Google Scholar
25. [25]
Zheng, W.S., Lai, J., Yuen, P.C.: Penalized pre-image learning in kernel principal component analysis. IEEE Trans. on Neural Networks 21(4), 551–570 (2010).
26. [26]
Turk, M., Pentland, A.: Eigenfaces for recognition. The Journal of Cognitive Neuroscience 3(1), 71–86 (1991).