Local Metric Learning on Manifolds with Application to Query–Based Operations

  • Karim Abou-Moustafa
  • Frank Ferrie
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5342)

Abstract

We first investigate the combined effect of data complexity, curse of dimensionality and the definition of the Euclidean distance on the distance measure between points. Then, based on the concepts underlying manifold learning algorithms and the minimum volume ellipsoid metric, we design an algorithm that learns a local metric on the lower dimensional manifold on which the data is lying. Experiments in the context of classification on standard benchmark data sets showed very promising results when compared to state of the art algorithms, and consistent improvements over the Euclidean distance in the context of query–based learning.

Keywords

Mahalanobis Distance Query Point Error Difference Locally Linear Embedding Real Life Data 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Abou-Moustafa, K., Ferrie, F.: The minimum volume ellipsoid metric. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 335–344. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Bar-Hillel, A., Hertz, T., Shental, N., Weinshall, D.: Learning a mahalanobis metric from equivalence constraints. JMLR 6, 937–965 (2005)MathSciNetMATHGoogle Scholar
  3. 3.
    Bottou, L., Vapnik, V.: Local learning algorithms. Neural Computation 4(6), 888–900 (1992)CrossRefGoogle Scholar
  4. 4.
    Boyd, S., Vandenberghe, L. (eds.): Convex Optimization. Cambridge Univ. Press, Cambridge (2004)MATHGoogle Scholar
  5. 5.
    Ding, C., Li, T.: Adaptive dimension reduction using discriminant analysis and k–means clustering. In: Proceedings of the 24th ICML, Corvallis, OR (2007)Google Scholar
  6. 6.
    Dolia, A., De Bie, T., Harris, C., Shawe-Taylor, J., Titterington, D.: The minimum volume covering ellipsoid estimation in kernel-defined feature spaces. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS, vol. 4212. Springer, Heidelberg (2006)Google Scholar
  7. 7.
    Friedman, J.: Regularized discriminant analysis. J. of the American Statistical Assoc. 84(405), 165–175 (1989)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Fukunaga, K., Olsen, R.: An algorithm for finding intrinsic dimensionality of data. IEEE Trans. on Computers 20(2), 176–183 (1971)CrossRefMATHGoogle Scholar
  9. 9.
    Globerson, A., Roweis, S., Hinton, G., Salakhutdinov, R.: Metric learning by collapsing classes. In: NIPS, vol. 18, pp. 451–458. MIT Press, Cambridge (2006)Google Scholar
  10. 10.
    Goldberg, J., Roweis, S.: Neighbourhood component analysis. In: NIPS, vol. 17, pp. 513–520. MIT Press, Cambridge (2005)Google Scholar
  11. 11.
    Grant, M., Boyd, S., Yinyu, Y.: Matlab software for disciplined convex programming (2005), http://www.stanford.edu/~boyd/cvx
  12. 12.
    Hastie, T., Tibshirani, R.: Discriminant adaptive nearest neighbour classification. IEEE Trans. PAMI 18(6), 607–615 (1996)CrossRefGoogle Scholar
  13. 13.
    Lebanon, G.: Metric learning for text documents. IEEE Trans. PAMI 28(4), 497–508 (2006)CrossRefGoogle Scholar
  14. 14.
    Lin, T., Zha, H.: Riemannian manifold learning. IEEE. Trans. PAMI 30(5), 796–809 (2008)CrossRefGoogle Scholar
  15. 15.
    Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases (1998)Google Scholar
  16. 16.
    Rousseeuw, P.: Multivariate estimation with high breakdown point. In: Proc. of the 4th Pannonian Symp. on Mathematical Statistics, vol. 3, pp. 283–297 (1983)Google Scholar
  17. 17.
    Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding (LLE). Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  18. 18.
    Schultz, M., Joachims, T.: Learning a distance metric from relative comparisons. In: NIPS, vol. 16. MIT Press, Cambridge (2004)Google Scholar
  19. 19.
    Short, R., Fukunaga, K.: The optimal distance measure for nearest neighbour classification. IEEE Trans. on Information Theory 27(5), 622–627 (1981)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Tenenbaum, J., de Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)CrossRefGoogle Scholar
  21. 21.
    Titterington, D.: Estimation of correleation coefficients by ellipsoidal trimming. J. of Royal Statistical Society 27(3), 227–234 (1978)MATHGoogle Scholar
  22. 22.
    Weinberger, K., Blitzer, J., Saul, L.: Distance metric learning for large margin nearest neighbor classification. In: NIPS, vol. 18, pp. 1473–1480. MIT Press, Cambridge (2006)Google Scholar
  23. 23.
    Xing, E., Ng, A., Jordan, M., Russell, S.: Distance metric learning with application to clustering with side-information. In: NIPS, vol. 15, pp. 505–512 (2003)Google Scholar
  24. 24.
    Yang, L.: Distance metric learning: A comprehensive review. Technical report, Dept. of Computer Science and Engineering, Michigan State University (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Karim Abou-Moustafa
    • 1
  • Frank Ferrie
    • 1
  1. 1.The Artificial Perception Laboratory Centre for Intelligent MachinesMcGill UniversityMontrealCanada

Personalised recommendations