Efficient Point-to-Subspace Query in ℓ1 with Application to Robust Face Recognition

  • Ju Sun
  • Yuqian Zhang
  • John Wright
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7575)


Motivated by vision tasks such as robust face and object recognition, we consider the following general problem: given a collection of low-dimensional linear subspaces in a high-dimensional ambient (image) space, and a query point (image), efficiently determine the nearest subspace to the query in ℓ1 distance. We show in theory this problem can be solved with a simple two-stage algorithm: (1) random Cauchy projection of query and subspaces into low-dimensional spaces followed by efficient distance evaluation (ℓ1 regression); (2) getting back to the high-dimensional space with very few candidates and performing exhaustive search. We present preliminary experiments on robust face recognition to corroborate our theory.


1 point-to-subspace distance nearest subspace search Cauchy projection face recognition subspace modeling 


  1. 1.
    Turk, M., Pentland, A.: Eigenfaces for recognition. In: CVPR (1991)Google Scholar
  2. 2.
    Murase, H., Nayar, S.: Visual learning and recognition of 3D objects from appearance. IJCV 14(1), 5–24 (1995)CrossRefGoogle Scholar
  3. 3.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. PAMI 23(6), 681–685 (2001)CrossRefGoogle Scholar
  4. 4.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. PAMI 25(9), 1063–1074 (2003)CrossRefGoogle Scholar
  5. 5.
    Wright, J., Yang, A., Ganesh, A., Sastry, S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. PAMI 31(2), 210–227 (2009)CrossRefGoogle Scholar
  6. 6.
    Wagner, A., Wright, J., Ganesh, A., Zhou, Z., Mobahi, H., Ma, Y.: Towards a practical automatic face recognition system: Robust alignment and illumination by sparse representation. IEEE Trans. PAMI 34(2), 372–386 (2012)CrossRefGoogle Scholar
  7. 7.
    Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. PAMI 23(6), 643–660 (2001)CrossRefGoogle Scholar
  8. 8.
    Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. IEEE Trans. PAMI 25(2), 218–233 (2003)CrossRefGoogle Scholar
  9. 9.
    Donoho, D., Grimes, C.: Image manifolds which are isometric to Euclidean space. J. of Math. Imag. and Vis. 23(1), 5–24 (2005)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Simard, P.Y., LeCun, Y.A., Denker, J.S., Victorri, B.: Transformation Invariance in Pattern Recognition - Tangent Distance and Tangent Propagation. In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 1524, pp. 239–274. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  11. 11.
    Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  12. 12.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)Google Scholar
  13. 13.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Annals of Statistics 32, 407–499 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. on Imag. Sci. 2(1), 183–202 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for ℓ1-minimization with applications to compressed sensing. SIAM J. Imag. Sci 1(1), 143–168Google Scholar
  16. 16.
    Yang, A., Ganesh, A., Ma, Y., Sastry, S.: Fast ℓ1-minimization algorithms and an application in robust face recognition: A review. In: ICIP (2010)Google Scholar
  17. 17.
    Donoho, D., Tsaig, Y.: Fast solution of ℓ1-norm minimization problems when the solution may be sparse. IEEE Trans. IT 54(11), 4789–4812 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Agarwal, A., Negahban, S., Wainwright, M.: Fast global convergence of gradient methods for high-dimensional statistical recovery. In: NIPS (2011)Google Scholar
  19. 19.
    Mattingley, J., Boyd, S.: CVXGEN: A code generator for embedded convex optimization. Optimization and Engineering 13(1), 1–27 (2012)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Datar, M., Indyk, P.: Locality-sensitive hashing scheme based on p-stable distributions. In: SCG, pp. 253–262. ACM Press (2004)Google Scholar
  21. 21.
    Andoni, A., Indyk, P., Krauthgamer, R., Nguyen, H.: Approximate line nearest neighbor in high dimensions. In: SODA (2009)Google Scholar
  22. 22.
    Basri, R., Hassner, T., Zelnik-Manor, L.: Approximate nearest subspace search. IEEE Trans. PAMI 33(2), 266–278 (2011)CrossRefGoogle Scholar
  23. 23.
    Jain, P., Vijayanarasimhan, S., Grauman, K.: Hashing hyperplane queries to near points with applications to large-scale active learning. In: NIPS (2010)Google Scholar
  24. 24.
    Williams, R.: A new algorithm for optimal 2-constraint satisfaction and its implications. Theo. Comp. Sci. 348, 357–365 (2005)zbMATHCrossRefGoogle Scholar
  25. 25.
    Magen, A., Zouzias, A.: Near Optimal Dimensionality Reductions That Preserve Volumes. In: Goel, A., Jansen, K., Rolim, J.D.P., Rubinfeld, R. (eds.) APPROX and RANDOM 2008. LNCS, vol. 5171, pp. 523–534. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  26. 26.
    Li, P., Hastie, T., Church, K.: Nonlinear estimators and tail bounds for dimension reduction in ℓ1 using cauchy random projections. JMLR 8, 2497–2532 (2007)MathSciNetGoogle Scholar
  27. 27.
    Sohler, C., Woodruff, D.: Subspace embeddings for the ℓ1-norm with applications. In: STOC (2011)Google Scholar
  28. 28.
    Brinkman, B., Charikar, M.: On the impossibility of dimension reduction in ℓ1. J. ACM 52, 766–788 (2005)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Candés, E., Tao, T.: Decoding by linear programming. IEEE Trans. IT 51(12), 4203–4215 (2005)CrossRefGoogle Scholar
  30. 30.
    Wright, J., Ma, Y.: Dense error correction via ℓ1-minimization. IEEE Trans. IT 56(7), 3540–3560 (2010)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Ledoux, M.: The Concentration of Measure Phenomenon. AMS (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ju Sun
    • 1
  • Yuqian Zhang
    • 1
  • John Wright
    • 1
  1. 1.Department of Electrical EngineeringColumbia UniversityUSA

Personalised recommendations