Advertisement

Euler Principal Component Analysis

Abstract

Principal Component Analysis (PCA) is perhaps the most prominent learning tool for dimensionality reduction in pattern recognition and computer vision. However, the 2-norm employed by standard PCA is not robust to outliers. In this paper, we propose a kernel PCA method for fast and robust PCA, which we call Euler-PCA (e-PCA). In particular, our algorithm utilizes a robust dissimilarity measure based on the Euler representation of complex numbers. We show that Euler-PCA retains PCA’s desirable properties while suppressing outliers. Moreover, we formulate Euler-PCA in an incremental learning framework which allows for efficient computation. In our experiments we apply Euler-PCA to three different computer vision applications for which our method performs comparably with other state-of-the-art approaches.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Algorithm 4
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Notes

  1. 1.

    Without loss of generality we assume zero mean.

  2. 2.

    We set α=1.9, as will be discussed later, in Sect. 4.

  3. 3.

    The fingerspelling alphabet is a subset of sign language which is utilized for spelling names. Examples can be found at http://asl.ms/.

  4. 4.

    The Matlab implementation is publicly available at http://www.cs.toronto.edu/~dross/ivt/.

  5. 5.

    The Matlab implementation of the IKPCA was kindly provided by the authors of the paper.

  6. 6.

    The implementation is publicly available at http://www.ist.temple.edu/~hbling/code_data.htm.

  7. 7.

    The implementation (only for translation motion model) is publicly available at http://vision.ucsd.edu/~bbabenko/project_miltrack.shtml, we carefully modified it in order to support an affine motion model in a particle filter framework.

  8. 8.

    Videos V 4 and V 5 are available at http://vision.ucsd.edu/~bbabenko/project_miltrack.shtml and the remaining videos are published at http://www.cs.toronto.edu/~dross/ivt/.

  9. 9.

    MATLAB implementations on a desktop computer with Intel Core i7 870 at 2.93 GHz and 8 GB RAM.

References

  1. Avidan, S. (2004). Support vector tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1064–1072. doi:10.1109/TPAMI.2004.53.

  2. Babenko, B., Yang, M., & Belongie, S. (2009). Visual tracking with online multiple instance learning. In CVPR’09 (pp. 983–990).

  3. Babenko, B., Yang, M., & Belongie, S. (2011). Robust object tracking with online multiple instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2010.226.

  4. Candés, E., Li, X., Ma, Y., & Wright, J. (2009). Robust principal component analysis? Available at: http://arxiv.org/abs/0912.3599v1.

  5. Chin, T. J., & Suter, D. (2007). Incremental kernel principal component analysis. IEEE Transactions on Image Processing, 1662–1674. doi:10.1109/TIP.2007.896668.

  6. Chin, T., Schindler, K., & Suter, D. (2006). Incremental kernel SVD for face recognition with image sets. In FG’06 (pp. 461–466).

  7. Cohn, J., Zlochower, A., Lien, J., & Kanade, T. (1999). Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. Psychophysiology, 35–43. doi:10.1017/S0048577299971184.

  8. Collins, R., Lipton, J., Fujiyoshi, H., & Kanade, T. (2001). Algorithms for cooperative multisensor surveillance. In The IEEE (p. 89). doi:10.1109/5.959341.

  9. Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 564–577. doi:10.1109/TPAMI.2003.1195991.

  10. de la Torre, F., & Black, M. (2003). A framework for robust subspace learning. International Journal of Computer Vision, 117–142. doi:10.1023/A:1023709501986.

  11. Ding, D., Zhou, D., He, X., & Zha, H. (2006). R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In ACM (pp. 281–288). doi:10.1145/1143844.1143880.

  12. Fitch, A., Kadyrov, A., Christmas, W., & Kittler, J. (2005). Fast robust correlation. IEEE Transactions on Image Processing, 1063–1073. doi:10.1109/TIP.2005.849767.

  13. Fraundorfer, F., Engels, C., & Nistér, D. (2007). Topological mapping, localization and navigation using image collections. In Intell. robots and systems (pp. 3872–3877).

  14. Gunawan, H., Neswan, O., & Budhi, W. (2005). A formula for angles between subspaces of inner product spaces. Contributions to Algebra and Geometry, 46(2), 311–320.

  15. Gunes, H., & Pantic, M. (2010). Automatic, dimensional and continuous emotion recognition. International Journal of Synthetic Emotion, 68–99. doi:10.4018/jse.2010101605.

  16. Handmann, U., Kalinke, T., & Tzomakas, C. (1998). Computer vision for driver assistance systems. Proc. SPIE, 136–147. doi:10.1117/12.317463.

  17. Haritaoglu, I., Harwood, D., & Davis, L. (2000). W4: real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 809–830. doi:10.1109/34.868683.

  18. Hashima, M., Hasegawa, F., Kanda, S., Maruyama, T., & Uchiyama, T. (1997). Localization and obstacle detection for a robot for carrying food trays. In Intell. robots and systems (pp. 345–351).

  19. He, R., Hu, B., Zheng, W., & Kong, X. (2011). Robust principal component analysis based on maximum correntropy criterion. IEEE Transactions on Image Processing, 1485–1494. doi:10.1109/TIP.2010.2103949.

  20. Honeine, P., & Richard, C. (2011). Preimage problem in kernel-based machine learning. IEEE Signal Processing Magazine, 28(2), 77–88.

  21. Hsieh, J., Yu, S., Chen, Y., & Hu, W. (2006). Automatic traffic surveillance system for vehicle tracking and classification. IEEE Transactions on Intelligent Transportation Systems, 175–187. doi:10.1109/TITS.2006.874722.

  22. Jolliffe, T. (2002). Principal component analysis (2nd edn.). Berlin: Springer.

  23. Kamijo, S., Matsushita, Y., Ikeuchi, K., & Sakauchi, M. (2000). Traffic monitoring and accident detection at intersections. IEEE Transactions on Intelligent Transportation Systems, 108–118. doi:10.1109/6979.880968.

  24. Ke, Q., & Kanade, T. (2003). Robust subspace computation using L1 norm (Tech. Rep. CMU-CS-03-172). Computer Science Department, Carnegie Mellon University.

  25. Ke, Q., & Kanade, T. (2005). Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In CVPR’05 (pp. 739–746).

  26. Krzanowski, W. (1979). Between-groups comparison of principal components. Journal of the American Statistical Association, 703–707. doi:10.1080/01621459.1979.10481674.

  27. Kwak, N. (2008). Principal component analysis based on L1-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1672–1680. doi:10.1109/TPAMI.2008.114.

  28. Kwok, J., & Tsang, I. (2004). The pre-image problem in kernel methods. IEEE Transactions on Neural Networks, 15(6), 1517–1525.

  29. Levy, A., & Lindenbaum, M. (2000). Sequential Karhunen-Loeve basis extraction and its application to images. IEEE Transactions on Image Processing, 1371–1374. doi:10.1109/83.855432.

  30. Li, Y. (2004). On incremental and robust subspace learning. Pattern Recognition, 1509–1518. doi:10.1016/j.patcog.2003.11.010.

  31. Li, L., Huang, W., Gu, I., & Tian, Q. (2004). Statistical modeling of complex backgrounds for foreground object detection. IEEE Transactions on Image Processing, 13(11), 1459–1472.

  32. Liu, W., Pokharel, P., & Principe, J. (2007). Correntropy: properties and applications in non-Gaussian signal processing. IEEE Transactions on Signal Processing, 5286–5298. doi:10.1109/TSP.2007.896065.

  33. Liwicki, S., & Everingham, M. (2009). Automatic recognition of fingerspelled words in British sign language. In CVPR4HB’09, in conj. with CVPR’09 (pp. 50–57).

  34. Liwicki, S., et al. (2012). doi:10.1109/TNNLS.2012.2208654.

  35. Luo, Y., Wu, T., & Hwang, J. (2003). Object-based analysis and interpretation of human motion in sports video sequences by dynamic Bayesian networks. Computer Vision and Image Understanding, 196–216. doi:10.1016/j.cviu.2003.08.001.

  36. Maddalena, L., & Petrosino, A. (2008). A self-organizing approach to background subtraction for visual surveillance applications. IEEE Transactions on Image Processing, 1168–1177. doi:10.1109/TIP.2008.924285.

  37. Martinez, A., & Benavente, R. (1998). The AR face database (Tech. Rep. #24). The Ohio State University.

  38. Mei, X., & Ling, H. (2009). Robust visual tracking using L1 minimization. In ICCV’09.

  39. Mika, S., Schölkopf, B., Smola, A., Müller, K., Scholz, M., & Rätsch, G. (1999). Kernel pca and de-noising in feature spaces. Advances in Neural Information Processing Systems, 11(1), 536–542.

  40. Nguyen, M., & de la Torre, F. (2009). Robust kernel principal component analysis. In Advances in NIPS (pp. 1185–1192).

  41. Oliver, N., Rosario, B., & Pentland, A. (2000). A Bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 831–843. doi:10.1109/34.868684.

  42. Paulsen, V. (2009). An introduction to the theory of reproducing kernel Hilbert spaces. Available at: http://www.math.uh.edu/~vern/rkhs.pdf.

  43. Ross, D., Lim, J., Lin, R., & Yang, M. (2008). Incremental learning for robust visual tracking. International Journal of Computer Vision, 125–141. doi:10.1007/s11263-007-0075-7.

  44. Tzimiropoulos, G. (2010). IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2010.107.

  45. Tzimiropoulos, G. (2012). IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2012.40.

  46. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 71–86. doi:10.1162/jocn.1991.3.1.71.

  47. Wren, C., Azarbayejani, A., Darrel, T., & Pentland, A. (1997). Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence, 780–785. doi:10.1109/34.598236.

Download references

Acknowledgements

The research presented in this paper is supported in part by the European Research Council (ERC) under the ERC Starting Grant Agreement ERC-2007- StG-203143 (MAHNOB). The work of S. Liwicki is supported by the Engineering and Physical Science Research Council DTA Studentship. The work of G. Tzimiropoulos is currently supported in part by the European Community’s 7th Framework Programme FP7/2007-2013 under Grant Agreement 288235 (FROG).

Author information

Correspondence to Stephan Liwicki.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(WMV 3.7 MB)

(WMV 3.6 MB)

(WMV 3.6 MB)

(WMV 4.3 MB)

(WMV 1.3 MB)

(WMV 2.8 MB)

(WMV 3.4 MB)

(WMV 1.5 MB)

(WMV 2.8 MB)

(WMV 2.6 MB)

(WMV 2.3 MB)

(WMV 4.0 MB)

(WMV 2.5 MB)

(WMV 2.0 MB)

(WMV 3.4 MB)

(WMV 2.4 MB)

(WMV 4.3 MB)

(WMV 3.7 MB)

(WMV 3.6 MB)

(WMV 3.6 MB)

(WMV 4.3 MB)

(WMV 1.3 MB)

(WMV 2.8 MB)

(WMV 3.4 MB)

(WMV 1.5 MB)

(WMV 2.8 MB)

(WMV 2.6 MB)

(WMV 2.3 MB)

(WMV 4.0 MB)

(WMV 2.5 MB)

(WMV 2.0 MB)

(WMV 3.4 MB)

(WMV 2.4 MB)

(WMV 4.3 MB)

Appendices

Appendix A: Proof of Theorem 1

Proof

Given A=ΦΦ H and B=Φ H Φ their eigenspaces is provided by and . Furthermore, \(\mathbf{U}_{A}^{H}\mathbf{U}_{A} = \mathbf{U}_{B}^{H}\mathbf{U}_{B} = \mathbf{I}\). Let us define matrix . We get

(27)

Therefore, Λ A =Λ B and U A =M for non-zero eigenvalues. □

Appendix B: Proof that \(\Vert \frac{1}{\sqrt{2}} e^{i\angle\mathbf{b}}- \mathbf{b}\Vert _{F}^{2} = \Vert \frac{1}{\sqrt{2}} - \mathtt{R}(\mathbf {b}) \Vert _{F}^{2}\)

(28)

where \(\mathtt{R}(\mathbf{b})= [\sqrt{\mathtt{Re} (\mathbf {b}(c) )^{2} + \mathtt{Im} (\mathbf{b}(c) )^{2}} ]\) is a vector with the magnitude of the elements of b and 1 is a vector of ones. □

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Liwicki, S., Tzimiropoulos, G., Zafeiriou, S. et al. Euler Principal Component Analysis. Int J Comput Vis 101, 498–518 (2013) doi:10.1007/s11263-012-0558-z

Download citation

Keywords

  • Euler PCA
  • Robust subspace
  • Online learning
  • Tracking
  • Background modeling