Skip to main content
SpringerLink
Log in

Euler Principal Component Analysis

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Principal Component Analysis (PCA) is perhaps the most prominent learning tool for dimensionality reduction in pattern recognition and computer vision. However, the ℓ 2-norm employed by standard PCA is not robust to outliers. In this paper, we propose a kernel PCA method for fast and robust PCA, which we call Euler-PCA (e-PCA). In particular, our algorithm utilizes a robust dissimilarity measure based on the Euler representation of complex numbers. We show that Euler-PCA retains PCA’s desirable properties while suppressing outliers. Moreover, we formulate Euler-PCA in an incremental learning framework which allows for efficient computation. In our experiments we apply Euler-PCA to three different computer vision applications for which our method performs comparably with other state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Algorithm 4
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. Without loss of generality we assume zero mean.

  2. We set α=1.9, as will be discussed later, in Sect. 4.

  3. The fingerspelling alphabet is a subset of sign language which is utilized for spelling names. Examples can be found at http://asl.ms/.

  4. The Matlab implementation is publicly available at http://www.cs.toronto.edu/~dross/ivt/.

  5. The Matlab implementation of the IKPCA was kindly provided by the authors of the paper.

  6. The implementation is publicly available at http://www.ist.temple.edu/~hbling/code_data.htm.

  7. The implementation (only for translation motion model) is publicly available at http://vision.ucsd.edu/~bbabenko/project_miltrack.shtml, we carefully modified it in order to support an affine motion model in a particle filter framework.

  8. Videos V 4 and V 5 are available at http://vision.ucsd.edu/~bbabenko/project_miltrack.shtml and the remaining videos are published at http://www.cs.toronto.edu/~dross/ivt/.

  9. MATLAB implementations on a desktop computer with Intel Core i7 870 at 2.93 GHz and 8 GB RAM.

References

  • Avidan, S. (2004). Support vector tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1064–1072. doi:10.1109/TPAMI.2004.53.

  • Babenko, B., Yang, M., & Belongie, S. (2009). Visual tracking with online multiple instance learning. In CVPR’09 (pp. 983–990).

    Google Scholar 

  • Babenko, B., Yang, M., & Belongie, S. (2011). Robust object tracking with online multiple instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2010.226.

  • Candés, E., Li, X., Ma, Y., & Wright, J. (2009). Robust principal component analysis? Available at: http://arxiv.org/abs/0912.3599v1.

  • Chin, T. J., & Suter, D. (2007). Incremental kernel principal component analysis. IEEE Transactions on Image Processing, 1662–1674. doi:10.1109/TIP.2007.896668.

  • Chin, T., Schindler, K., & Suter, D. (2006). Incremental kernel SVD for face recognition with image sets. In FG’06 (pp. 461–466).

    Google Scholar 

  • Cohn, J., Zlochower, A., Lien, J., & Kanade, T. (1999). Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. Psychophysiology, 35–43. doi:10.1017/S0048577299971184.

  • Collins, R., Lipton, J., Fujiyoshi, H., & Kanade, T. (2001). Algorithms for cooperative multisensor surveillance. In The IEEE (p. 89). doi:10.1109/5.959341.

    Google Scholar 

  • Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 564–577. doi:10.1109/TPAMI.2003.1195991.

  • de la Torre, F., & Black, M. (2003). A framework for robust subspace learning. International Journal of Computer Vision, 117–142. doi:10.1023/A:1023709501986.

  • Ding, D., Zhou, D., He, X., & Zha, H. (2006). R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In ACM (pp. 281–288). doi:10.1145/1143844.1143880.

    Google Scholar 

  • Fitch, A., Kadyrov, A., Christmas, W., & Kittler, J. (2005). Fast robust correlation. IEEE Transactions on Image Processing, 1063–1073. doi:10.1109/TIP.2005.849767.

  • Fraundorfer, F., Engels, C., & Nistér, D. (2007). Topological mapping, localization and navigation using image collections. In Intell. robots and systems (pp. 3872–3877).

    Google Scholar 

  • Gunawan, H., Neswan, O., & Budhi, W. (2005). A formula for angles between subspaces of inner product spaces. Contributions to Algebra and Geometry, 46(2), 311–320.

    MathSciNet  MATH  Google Scholar 

  • Gunes, H., & Pantic, M. (2010). Automatic, dimensional and continuous emotion recognition. International Journal of Synthetic Emotion, 68–99. doi:10.4018/jse.2010101605.

  • Handmann, U., Kalinke, T., & Tzomakas, C. (1998). Computer vision for driver assistance systems. Proc. SPIE, 136–147. doi:10.1117/12.317463.

  • Haritaoglu, I., Harwood, D., & Davis, L. (2000). W4: real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 809–830. doi:10.1109/34.868683.

  • Hashima, M., Hasegawa, F., Kanda, S., Maruyama, T., & Uchiyama, T. (1997). Localization and obstacle detection for a robot for carrying food trays. In Intell. robots and systems (pp. 345–351).

    Google Scholar 

  • He, R., Hu, B., Zheng, W., & Kong, X. (2011). Robust principal component analysis based on maximum correntropy criterion. IEEE Transactions on Image Processing, 1485–1494. doi:10.1109/TIP.2010.2103949.

  • Honeine, P., & Richard, C. (2011). Preimage problem in kernel-based machine learning. IEEE Signal Processing Magazine, 28(2), 77–88.

    Article  Google Scholar 

  • Hsieh, J., Yu, S., Chen, Y., & Hu, W. (2006). Automatic traffic surveillance system for vehicle tracking and classification. IEEE Transactions on Intelligent Transportation Systems, 175–187. doi:10.1109/TITS.2006.874722.

  • Jolliffe, T. (2002). Principal component analysis (2nd edn.). Berlin: Springer.

    MATH  Google Scholar 

  • Kamijo, S., Matsushita, Y., Ikeuchi, K., & Sakauchi, M. (2000). Traffic monitoring and accident detection at intersections. IEEE Transactions on Intelligent Transportation Systems, 108–118. doi:10.1109/6979.880968.

  • Ke, Q., & Kanade, T. (2003). Robust subspace computation using L1 norm (Tech. Rep. CMU-CS-03-172). Computer Science Department, Carnegie Mellon University.

  • Ke, Q., & Kanade, T. (2005). Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In CVPR’05 (pp. 739–746).

    Google Scholar 

  • Krzanowski, W. (1979). Between-groups comparison of principal components. Journal of the American Statistical Association, 703–707. doi:10.1080/01621459.1979.10481674.

  • Kwak, N. (2008). Principal component analysis based on L1-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1672–1680. doi:10.1109/TPAMI.2008.114.

  • Kwok, J., & Tsang, I. (2004). The pre-image problem in kernel methods. IEEE Transactions on Neural Networks, 15(6), 1517–1525.

    Article  Google Scholar 

  • Levy, A., & Lindenbaum, M. (2000). Sequential Karhunen-Loeve basis extraction and its application to images. IEEE Transactions on Image Processing, 1371–1374. doi:10.1109/83.855432.

  • Li, Y. (2004). On incremental and robust subspace learning. Pattern Recognition, 1509–1518. doi:10.1016/j.patcog.2003.11.010.

  • Li, L., Huang, W., Gu, I., & Tian, Q. (2004). Statistical modeling of complex backgrounds for foreground object detection. IEEE Transactions on Image Processing, 13(11), 1459–1472.

    Article  Google Scholar 

  • Liu, W., Pokharel, P., & Principe, J. (2007). Correntropy: properties and applications in non-Gaussian signal processing. IEEE Transactions on Signal Processing, 5286–5298. doi:10.1109/TSP.2007.896065.

  • Liwicki, S., & Everingham, M. (2009). Automatic recognition of fingerspelled words in British sign language. In CVPR4HB’09, in conj. with CVPR’09 (pp. 50–57).

    Google Scholar 

  • Liwicki, S., et al. (2012). doi:10.1109/TNNLS.2012.2208654.

  • Luo, Y., Wu, T., & Hwang, J. (2003). Object-based analysis and interpretation of human motion in sports video sequences by dynamic Bayesian networks. Computer Vision and Image Understanding, 196–216. doi:10.1016/j.cviu.2003.08.001.

  • Maddalena, L., & Petrosino, A. (2008). A self-organizing approach to background subtraction for visual surveillance applications. IEEE Transactions on Image Processing, 1168–1177. doi:10.1109/TIP.2008.924285.

  • Martinez, A., & Benavente, R. (1998). The AR face database (Tech. Rep. #24). The Ohio State University.

  • Mei, X., & Ling, H. (2009). Robust visual tracking using L1 minimization. In ICCV’09.

    Google Scholar 

  • Mika, S., Schölkopf, B., Smola, A., Müller, K., Scholz, M., & Rätsch, G. (1999). Kernel pca and de-noising in feature spaces. Advances in Neural Information Processing Systems, 11(1), 536–542.

    Google Scholar 

  • Nguyen, M., & de la Torre, F. (2009). Robust kernel principal component analysis. In Advances in NIPS (pp. 1185–1192).

    Google Scholar 

  • Oliver, N., Rosario, B., & Pentland, A. (2000). A Bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 831–843. doi:10.1109/34.868684.

  • Paulsen, V. (2009). An introduction to the theory of reproducing kernel Hilbert spaces. Available at: http://www.math.uh.edu/~vern/rkhs.pdf.

  • Ross, D., Lim, J., Lin, R., & Yang, M. (2008). Incremental learning for robust visual tracking. International Journal of Computer Vision, 125–141. doi:10.1007/s11263-007-0075-7.

  • Tzimiropoulos, G. (2010). IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2010.107.

  • Tzimiropoulos, G. (2012). IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2012.40.

  • Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 71–86. doi:10.1162/jocn.1991.3.1.71.

  • Wren, C., Azarbayejani, A., Darrel, T., & Pentland, A. (1997). Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence, 780–785. doi:10.1109/34.598236.

Download references

Acknowledgements

The research presented in this paper is supported in part by the European Research Council (ERC) under the ERC Starting Grant Agreement ERC-2007- StG-203143 (MAHNOB). The work of S. Liwicki is supported by the Engineering and Physical Science Research Council DTA Studentship. The work of G. Tzimiropoulos is currently supported in part by the European Community’s 7th Framework Programme FP7/2007-2013 under Grant Agreement 288235 (FROG).

Author information

Authors and Affiliations

  1. Department of Computing, Imperial College London, 180 Queen’s Gate, London, SW7 2AZ, UK

    Stephan Liwicki, Georgios Tzimiropoulos, Stefanos Zafeiriou & Maja Pantic

  2. School of Computer Science, University of Lincoln, Brayford Pool, Lincoln, LN6 7TS, UK

    Georgios Tzimiropoulos

  3. Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, The Netherlands

    Maja Pantic

Authors
  1. Stephan Liwicki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georgios Tzimiropoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefanos Zafeiriou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maja Pantic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephan Liwicki.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(WMV 3.7 MB)

(WMV 3.6 MB)

(WMV 3.6 MB)

(WMV 4.3 MB)

(WMV 1.3 MB)

(WMV 2.8 MB)

(WMV 3.4 MB)

(WMV 1.5 MB)

(WMV 2.8 MB)

(WMV 2.6 MB)

(WMV 2.3 MB)

(WMV 4.0 MB)

(WMV 2.5 MB)

(WMV 2.0 MB)

(WMV 3.4 MB)

(WMV 2.4 MB)

(WMV 4.3 MB)

Appendices

Appendix A: Proof of Theorem 1

Proof

Given A=ΦΦ H and B=Φ H Φ their eigenspaces is provided by and . Furthermore, \(\mathbf{U}_{A}^{H}\mathbf{U}_{A} = \mathbf{U}_{B}^{H}\mathbf{U}_{B} = \mathbf{I}\). Let us define matrix . We get

(27)

Therefore, Λ A =Λ B and U A =M for non-zero eigenvalues. □

Appendix B: Proof that \(\Vert \frac{1}{\sqrt{2}} e^{i\angle\mathbf{b}}- \mathbf{b}\Vert _{F}^{2} = \Vert \frac{1}{\sqrt{2}} - \mathtt{R}(\mathbf {b}) \Vert _{F}^{2}\)

(28)

where \(\mathtt{R}(\mathbf{b})= [\sqrt{\mathtt{Re} (\mathbf {b}(c) )^{2} + \mathtt{Im} (\mathbf{b}(c) )^{2}} ]\) is a vector with the magnitude of the elements of b and 1 is a vector of ones. □

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liwicki, S., Tzimiropoulos, G., Zafeiriou, S. et al. Euler Principal Component Analysis. Int J Comput Vis 101, 498–518 (2013). https://doi.org/10.1007/s11263-012-0558-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-012-0558-z

Keywords

Search

Navigation