Skip to main content
SpringerLink
Log in

An Analysis of Linear Subspace Approaches for Computer Vision and Pattern Recognition

  • Regular Papers
  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Linear subspace analysis (LSA) has become rather ubiquitous in a wide range of problems arising in pattern recognition and computer vision. The essence of these approaches is that certain structures are intrinsically (or approximately) low dimensional: for example, the factorization approach to the problem of structure from motion (SFM) and principal component analysis (PCA) based approach to face recognition. In LSA, the singular value decomposition (SVD) is usually the basic mathematical tool. However, analysis of the performance, in the presence of noise, has been lacking. We present such an analysis here. First, the “denoising capacity” of the SVD is analysed. Specifically, given a rank-r matrix, corrupted by noise—how much noise remains in the rank-r projected version of that corrupted matrix? Second, we study the “learning capacity” of the LSA-based recognition system in a noise-corrupted environment. Specifically, LSA systems that attempt to capture a data class as belonging to a rank-r column space will be affected by noise in both the training samples (measurement noise will mean the learning samples will not produce the “true subspace”) and the test sample (which will also have measurement noise on top of the ideal clean sample belonging to the “true subspace”). These results should help one to predict aspects of performance and to design more optimal systems in computer vision, particularly in tasks, such as SFM and face recognition. Our analysis agrees with certain observed phenomenon, and these observations, together with our simulations, verify the correctness of our theory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Basri, R. and Jacobs, D.W. 1999. Lambertian reflectance and linear subspaces. Proc. Int'l Conf. Computer Vision.

  • Basri, R. and Jacobs, D.W. 2003. Lambertian reflectance and linear subspaces. IEEE Trans Pattern Analysis and Machine Intelligence, 25 (2):218–233.

    Article  Google Scholar 

  • Belhumeur, P.N., Hespanha, J. et al. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Analysis and Machine Intelligence, 19(7):711–720.

    Article  Google Scholar 

  • Belhumeur, P.N. and Kriegman, D. 1998. What is the set of images of an object under all possible illumination conditions? Int'l J. Computer vision, 28(3):245–260.

    Article  Google Scholar 

  • Chen, P. and Suter, D. 2004a. Recovering the missing components in a large noisy low-rank matrix: Application to SFM. IEEE Trans Pattern Analysis and Machine Intelligence, 26(8):1051–1063.

    Article  Google Scholar 

  • Chen, P. and Suter, D. 2004b. Subspace-based face recognition: Outlier detection and a new distance criterion. A sian Conf. Computer Vision.

  • Chen, P. and Suter, D. 2005. Subspace-based face recognition: outlier detection and a new distance criterion. International Journal of Pattern Recognition and Artificial Intelligence 19(4):479–493.

    Article  Google Scholar 

  • Eipstein, R., Hallinan, P. et al. 1995. 5 plus-or-minus 2 eigenimages suffices: An empirical investigation of low-dimensional lighting models. Proc. IEEE Workshop Physics-Based Vision.

  • Faugeras, O. and Luong, Q.T. 2001. The Geometry of Multiple Images : The Laws That Govern The Formation of Multiple Images of a Scene and Some of Their Applications. Cambridge, Mass., MIT Press.

    MATH  Google Scholar 

  • Georghiades, A., Belhumeur, P.N. et al. 2001. From few to many: Generative models of object recognition. IEEE Trans Pattern Analysis and Machine Intelligence, 23(6):643–660.

    Article  Google Scholar 

  • Georghiades, A., Kriegman, D. et al. 1998. Illumination cones for recognition under variable lighting: Faces. Proc. Conf. Computer Vision and Pattern Recognition.

  • Golub, G.H. and Loan, C.F.V. 1996. Matrix Computations. Baltimore, Johns Hopkins University Press.

  • Hallinan, P. 1994. A low-dimensional representation of human faces for arbitrary lighting conditions. Proc. Conf. Computer Vision and Pattern Recognition.

  • Hartley, R. and Zisserman, A. 2000. Multiple View Geometry in Computer Vision. Cambridge University Press.

  • Irani, M. 1999. Multi-frame optical flow estimation using subspace constraints.Proc. Int'l Conf. Computer Vision.

  • Irani, M. 2002. Multi-frame correspondence estimation using subspace constraints. Int'l J. Computer Vision, 48(3):173–194.

    Article  MATH  MathSciNet  Google Scholar 

  • Kahl, F. and Heyden, A. 1999. “Affine structure and motion from points, lines and conics.” Int'l J. Computer Vision, 33(3):163–180.

    Article  Google Scholar 

  • Kanatani, K. 2001. Motion segmentation by subspace separation and model selection. Proc. Int'l Conf. Computer Vision.

  • Leedan, Y. and Meer, P. 1999. Estimation with bilinear constraints in computer vision, Proc. Int'l Conf. Computer Vision.

  • Leedan, Y. and Meer, P. 2000. Heteroscedastic regression in computer vision: Problems with Bilinear Constraint. Int'l J. Computer Vision, 37(2):127–150.

    Article  MATH  Google Scholar 

  • Ma, Y., Huang, K. et al. 2004. Rank conditions on the multiple-view matrix. INT'L J. Computer Vision, 59(2):115–137.

    Article  Google Scholar 

  • Mathai, A.M. 1997. Jacobians of Matrix Transformations and Functions of Matrix Argument. World Scientific Publishers.

  • Morita, T. and Kanade, T. 1997. A sequential factorization method for recovering shape and motion from image streams. IEEE Trans Pattern Analysis and Machine Intelligence, 19(8):858–867.

    Article  Google Scholar 

  • Moses, Y., Adini, Y. et al. 1994. Face recognition: The problem of compensating for changes in illumination direction. Proc. European Conf. Computer Vision.

  • Murase, H. and Nayar, S.K. 1994. Illumination planning for object recognition using parametric eigenspaces, IEEE Trans Pattern Analysis and Machine Intelligence, 16(12):1219–1227.

    Article  Google Scholar 

  • Murase, H. and Nayar, S.K. 1995. Visual learning and recognition of 3-D objects from appearance, Int'l J. Computer Vision, 14:5–24.

    Article  Google Scholar 

  • Nene, S.A., Nayar, S.K. et al., 1994. Software library for appearance matching (SLAM). ARPA Image Understanding Workshop.

  • Papadopoulo, T. and Lourakis, M.I.A. 2000. Estimating the Jacobian of the Singular Value Decomposition: Theory and applications. Proc. European Conf. Computer Vision.

  • Poelman, C. and Kanade, T. 1997. A paraperspective factorization method for shape and motion recovery. IEEE Trans Pattern Analysis and Machine Intelligence 19(3):206–219.

    Article  Google Scholar 

  • Press, W.H., Teukolsky, S. A. et al., 1992. Numerical Recipes in C. Cambridge University Press.

  • Ramamoorthi, R. 2002. Analytic PCA construction for theoretical analysis of lighting variability in images of a Lambertian Object. IEEE Trans Pattern Analysis and Machine Intelligence, 24(10):1322–1333.

    Article  Google Scholar 

  • Ramamoorthi, R. and Hanrahan, P. 2001. On the relationship between radiance and irradiance: Determining the illumination from images of a convex Lambertian object. Journal of the Optical Society of America (JOSA A), 18(10):2448–2459.

    MathSciNet  Google Scholar 

  • Shashua, A. and Avidan, S. 1996. The Rank 4 Constraint in Multiple (>2) View Geometry. Proc. European Conf. Computer Vision.

  • Stewart, G.W. and Sun, J.G. 1990. Matrix Perturbation Theory, Academic press.

  • Thomas, J.I. and Oliensis, J. 1999. Dealing with noise in multiframe structure from motion, Computer Vision and Image Understanding, 76(2):109–124.

    Article  Google Scholar 

  • Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method, Int'l J. Computer Vision, 9(2):137–154.

    Article  Google Scholar 

  • Turk, M. and Pentland, A. 1991. Eigenfaces for recognition, J. Cognitive Neuroscience, 3(1):71–96.

    Article  Google Scholar 

  • Wilkinson, J.H. 1965. The Algebraic Eigenvalue Problem, Oxford, Clarendon Press.

    MATH  Google Scholar 

  • Yuille, A.L., Snow, D. et al., 1999. Determining generative models of objects under varying illumination: Shape and albedo from multiple images using SVD and integrability, Int'l J. Computer Vision, 35(3):203–222.

    Article  Google Scholar 

  • Zelnik-Manor, L. and Irani, M. 1999. Multi-View Subspace Constraints on Homographies. Proc. Int'l Conf. Computer Vision.

  • Zelnik-Manor, L. and Irani, M. 2002. Multi-View Subspace Constraints on Homographies. IEEE Trans Pattern Analysis and Machine Intelligence, 24(2):214-223.

    Article  Google Scholar 

  • Zhao, L. and Y.H. Yang (1999). Theoretical analysis of illumination in PCA-based vision systems. Pattern Recognition, 32:547–564.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ARC Centre for Perceptive and Intelligent Machines in Complex Environments, Department of Electrical and Computer Systems Engineering, Monash University, Australia, 3800

    Pei Chen & David Suter

Authors
  1. Pei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Suter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pei Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, P., Suter, D. An Analysis of Linear Subspace Approaches for Computer Vision and Pattern Recognition. Int J Comput Vision 68, 83–106 (2006). https://doi.org/10.1007/s11263-006-6659-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-006-6659-9

Keywords

Search

Navigation