Advertisement

Kernel-Based Metric Adaptation with Pairwise Constraints

  • Hong Chang
  • Dit-Yan Yeung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3930)

Abstract

Many supervised and unsupervised learning algorithms depend on the choice of an appropriate distance metric. While metric learning for supervised learning tasks has a long history, extending it to learning tasks with weaker supervisory information has only been studied very recently. In particular, several methods have been proposed for semi-supervised metric learning based on pairwise (dis)similarity information. In this paper, we propose a kernel-based approach for nonlinear metric learning, which performs locally linear translation in the kernel-induced feature space. We formulate the metric learning problem as a kernel learning problem and solve it efficiently by kernel matrix adaptation. Experimental results based on synthetic and real-world data sets show that our approach is promising for semi-supervised metric learning.

Keywords

Radial Basis Function Network Kernel Matrix Neural Information Processing System Rand Index Pairwise Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Fukunaga, K., Hostetler, L.: Optimization of k-nearest neighbor density estimates. IEEE Transactions on Information Theory 19(3), 320–326 (1973)MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Friedman, J.H.: Flexible metric nearest neighbor classification. Technical report, Department of Statistics, Stanford University, Stanford, CA, USA (November 1994)Google Scholar
  3. 3.
    Hastie, T., Tibshirani, R.: Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(6), 607–616 (1996)CrossRefGoogle Scholar
  4. 4.
    Lowe, D.G.: Similarity metric learning for a variable-kernel classifier. Neural Computation 7(1), 72–85 (1995)CrossRefGoogle Scholar
  5. 5.
    Peng, J., Heisterkamp, D.R., Dai, H.K.: Adaptive kernel metric nearest neighbor classification. In: Proceedings of the Sixteenth International Conference on Pattern Recognition, Québec City, Québec, Canada, August 11-15, vol. 3, pp. 33–36 (2002)Google Scholar
  6. 6.
    Poggio, T., Girosi, F.: Networks for approximation and learning. Proceedings of the IEEE 78(9), 1481–1497 (1990)CrossRefGoogle Scholar
  7. 7.
    Basu, S., Banerjee, A., Mooney, R.: Semi-supervised clustering by seeding. In: Proceedings of the Nineteenth International Conference on Machine Learning, Sydney, Australia, July 8-12, pp. 19–26 (2002)Google Scholar
  8. 8.
    Sinkkonen, J., Kaski, S.: Clustering based on conditional distributions in an auxiliary space. Neural Computation 14(1), 217–239 (2002)MATHCrossRefGoogle Scholar
  9. 9.
    Zhang, Z., Kwok, J.T., Yeung, D.Y.: Parametric distance metric learning with label information. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, August 9-15, pp. 1450–1452 (2003)Google Scholar
  10. 10.
    Wagstaff, K., Cardie, C.: Clustering with instance-level constraints. In: Proceedings of the Seventeenth International Conference on Machine Learning, Standord, CA, USA, pp. 1103–1110 (2000)Google Scholar
  11. 11.
    Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained k-means clustering with backgroud knowledge. In: Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, USA, pp. 577–584 (2001)Google Scholar
  12. 12.
    Lu, Z., Leen, T.: Penalized probabilistic clustering. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17. MIT Press, Cambridge (2005)Google Scholar
  13. 13.
    Shental, N., Bar-Hillel, A., Hertz, T., Weinshall, D.: Computing Gaussian mixture models with EM using equivalence constraints. In: Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)Google Scholar
  14. 14.
    Bar-Hillel, A., Hertz, T., Shental, N., Weinshall, D.: Learning distance functions using equivalence relations. In: Proceedings of the Twentieh International Conference on Machine Learning, Washington, DC, USA, August 21-24, pp. 11–18 (2003)Google Scholar
  15. 15.
    Hertz, T., Bar-Hillel, A., Weinshall, D.: Boosting margin based distance functions for clustering. In: Proceedings of the Twenty-First International Conference on Machine Learning, Banff, Alberta, Canada, August 4-8, pp. 393–400 (2004)Google Scholar
  16. 16.
    Schultz, M., Joachims, T.: Learning a distance metric from relative comparisons. In: Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)Google Scholar
  17. 17.
    Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance metric learning, with application to clustering with side-information. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 505–512. MIT Press, Cambridge (2003)Google Scholar
  18. 18.
    Chang, H., Yeung, D.Y.: Locally linear metric adaptation for semi-supervised clustering. In: Proceedings of the Twenty-First International Conference on Machine Learning, Banff, Alberta, Canada, August 4-8, pp. 153–160 (2004)Google Scholar
  19. 19.
    Rand, W.M.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66, 846–850 (1971)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hong Chang
    • 1
  • Dit-Yan Yeung
    • 1
  1. 1.Department of Computer ScienceHong Kong University of Science and TechnologyKowloon, Hong Kong

Personalised recommendations