The success of many machine learning algorithms (e.g. the nearest neighborhood classification and k-means clustering) depends on the representation of the data as elements in a metric space. Learning an appropriate distance metric from data is usually superior to the default Euclidean distance. In this paper, we revisit the original model proposed by Xing et al. [25] and propose a general formulation of learning a Mahalanobis distance from data. We prove that this novel formulation is equivalent to a convex optimization problem over the spectrahedron. Then, a gradient-based optimization algorithm is proposed to obtain the optimal solution which only needs the computation of the largest eigenvalue of a matrix per iteration. Finally, experiments on various UCI datasets and a benchmark face verification dataset called Labeled Faces in the Wild (LFW) demonstrate that the proposed method compares competitively to those state-of-the-art methods.


Metric learning convex optimization Frank-Wolfe algorithm face verification 


  1. 1.
    Bar-Hillel, A., Hertz, T., Shental, N., Weinshall, D.: Learning a mahalanobis metric from equivalence constraints. Journal of Machine Learning Research 6, 937–965 (2005)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively with application to face verification. In: CVPR (2005)Google Scholar
  3. 3.
    Cox, T., Cox, M.: Multidimensional scaling. Chapman and Hall, London (1994)zbMATHGoogle Scholar
  4. 4.
    Davis, J., Kulis, B., Jain, P., Sra, S., Dhillon, I.: Information-theoretic metric learning. In: ICML (2007)Google Scholar
  5. 5.
    Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Research Logistics Quaterly 3, 149–154 (1956)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood component analysis. In: NIPS (2004)Google Scholar
  7. 7.
    Guillaumin, M., Verbeek, J., Schmid, C.: Is that you? Metric learning approaches for face identification. In: ICCV (2009)Google Scholar
  8. 8.
    Hazan, E.: Sparse Approximate Solutions to Semidefinite Programs. In: Laber, E.S., Bornstein, C., Nogueira, L.T., Faria, L. (eds.) LATIN 2008. LNCS, vol. 4957, pp. 306–316. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press (1991)Google Scholar
  10. 10.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments. University of Massachusetts, Amherst, Technical Report 07-49 (2007)Google Scholar
  11. 11.
    Jain, P., Kulis, B., Dhillon, I.S.: Inductive regularized learning of kernel functions. In: NIPS (2010)Google Scholar
  12. 12.
    Jin, R., Wang, S., Zhou, Y.: Regularized distance metric learning: theory and algorithm. In: NIPS (2009)Google Scholar
  13. 13.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)CrossRefGoogle Scholar
  14. 14.
    Taigman, Y., Wolf, L., Hassner, T.: Multiple one-shots for utilizing class label information. In: The British Machine Vision Conference (2009)Google Scholar
  15. 15.
    Tsang, I.W., Kwok, J.T.: Distance Metric Learning with Kernels. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714. Springer, Heidelberg (2003)Google Scholar
  16. 16.
    Tenenbaum, J., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)CrossRefGoogle Scholar
  17. 17.
    Roweis, S.T., Lawrance, K.S.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  18. 18.
    Pinto, N., Cox, D.: Beyond simple features: a large-scale feature search approach to unconstrained face recognition. In: International Conference on Automatic Face and Gesture Recognition (2011)Google Scholar
  19. 19.
    Shen, C., Kim, J., Wang, L., Hengel, A.: Positive semidefinite metric learning with boosting. In: NIPS (2009)Google Scholar
  20. 20.
    Torresani, L., Lee, K.: Large margin component analysis. In: NIPS (2007)Google Scholar
  21. 21.
    Vandenbergheand, L., Boyd, S.: Semidefinite programming. SIAM Review 38(1), 49–95 (1996)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Weinberger, K.Q., Blitzer, J., Saul, L.: Distance metric learning for large margin nearest neighbour classification. In: NIPS (2006)Google Scholar
  23. 23.
    Weinberger, K.Q., Saul, L.K.: Fast solvers and efficient implementations for distance metric learning. In: ICML (2008)Google Scholar
  24. 24.
    Wolf, L., Hassner, T., Taigman, Y.: Descriptor based methods in the wild. In: Workshop on Faces Real-Life Images at ECCV (2008)Google Scholar
  25. 25.
    Xing, E., Ng, A., Jordan, M., Russell, S.: Distance metric learning with application to clustering with side information. In: NIPS (2002)Google Scholar
  26. 26.
    Yang, L., Jin, R.: Distance metric learning: A comprehensive survey. Technical report, Department of Computer Science and Engineering, Michigan State University (2007)Google Scholar
  27. 27.
    Ying, Y., Huang, K., Campbell, C.: Sparse metric learning via smooth optimization. In: NIPS (2009)Google Scholar
  28. 28.
    Ying, Y., Li, P.: Distance metric learning with eigenvalue optimization. Journal of Machine Learning Research 13, 1–26 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Qiong Cao
    • 1
  • Yiming Ying
    • 1
  • Peng Li
    • 2
  1. 1.College of Engineering, Mathematics and Physical SciencesUniversity of ExeterExeterUK
  2. 2.Department of Engineering MathematicsUniversity of BristolBristolUK

Personalised recommendations