Discriminative Experimental Design

  • Yu Zhang
  • Dit-Yan Yeung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6913)

Abstract

Since labeling data is often both laborious and costly, the labeled data available in many applications is rather limited. Active learning is a learning approach which actively selects unlabeled data points to label as a way to alleviate the labeled data deficiency problem. In this paper, we extend a previous active learning method called transductive experimental design (TED) by proposing a new unlabeled data selection criterion. Our method, called discriminative experimental design (DED), incorporates both margin-based discriminative information and data distribution information and hence it can be seen as a discriminative extension of TED. We report experiments conducted on some benchmark data sets to demonstrate the effectiveness of DED.

Keywords

Support Vector Machine Active Learning Label Data Unlabeled Data Discriminative Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Chapelle, O., Zien, A., Schölkopf, B. (eds.): Semi-Supervised Learning. MIT Press, Boston (2006)Google Scholar
  2. 2.
    Cohn, D., Atlas, L., Ladner, R.: Improving generalization with active learning. Machine Learning 15, 201–221 (1994)Google Scholar
  3. 3.
    Cohn, D.A., Ghahramani, Z., Jordan, M.I.: Active learning with statistical models. Journal of Artificial Intelligence Research 4, 129–145 (1996)MATHGoogle Scholar
  4. 4.
    Campbell, C., Cristianini, N., Smola, A.J.: Query learning with large margin classifiers. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 111–118. Stanford University, Standord (2000)Google Scholar
  5. 5.
    Schohn, G., Cohn, D.: Less is more: Active learning with support vector machines. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 839–846. Stanford University, Standord (2000)Google Scholar
  6. 6.
    Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 999–1006. Stanford University, Standord (2000)Google Scholar
  7. 7.
    Hoi, S.C.H., Jin, R., Zhu, J., Lyu, M.R.: Batch mode active learning and its application to medical image classification. In: Proceedings of the Twenty-Third International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA, pp. 417–424 (2006)Google Scholar
  8. 8.
    Guo, Y., Schuurmans, D.: Discriminative batch mode active learning. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems, vol. 20, pp. 593–600 (2007)Google Scholar
  9. 9.
    Yu, K., Bi, J., Tresp, V.: Active learning via transductive experimental design. In: Proceedings of the Twenty-Third International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA, pp. 1081–1088 (2006)Google Scholar
  10. 10.
    Sindhwani, V., Melville, P., Lawrence, R.D.: Uncertainty sampling and transductive experimental design for active dual supervision. In: Proceedings of the 26th International Conference on Machine Learning, Montreal, Quebec, Canada, pp. 953–960 (2009)Google Scholar
  11. 11.
    Lewis, D.D., Catlett, J.: Heterogeneous uncertainty sampling for supervised learning. In: Proceedings of the 11th International Conference on Machine Learning, pp. 148–156. Morgan Kaufmann, San Francisco (1994)Google Scholar
  12. 12.
    Seung, H.S., Opper, M., Sompolinsky, H.: Query by committee. In: Proceedings of the 5th Annual Workshop on Computational Learning Theory, New York, NY, USA, 287–294 (1992)Google Scholar
  13. 13.
    Nguyen, H.T., Smeulders, A.: Active learning using preclustering. In: Proceedings of the 21st International Conference on Machine learning, Banff, Alberta, Canada (2004)Google Scholar
  14. 14.
    Roy, N., McCallum, A.: Toward optimal active learning through sampling estimation of error reduction. In: Proceedings of the 18th International Conference on Machine Learning, San Francisco, CA, USA, pp. 441–448 (2001)Google Scholar
  15. 15.
    Atkinson, A.C., Donev, A.N. (eds.): Optimum Experiment Designs. Oxford University Press, Boston (1992)Google Scholar
  16. 16.
    Gestel, T.V., Suykens, J.A.K., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., Moor, B.D., Vandewalle, J.: Benchmarking least squares support vector machine classifiers. Machine Learning 54(1), 5–32 (2004)CrossRefMATHGoogle Scholar
  17. 17.
    Hsieh, C.J., Chang, K.W., Lin, C.J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear SVM. In: Proceedings of the Twenty-Fifth International Conference on Machine Learning, Helsinki, Finland, pp. 408–415 (2008)Google Scholar
  18. 18.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, New York (1991)MATHGoogle Scholar
  19. 19.
    Hoi, S.C.H., Jin, R., Zhu, J., Lyu, M.R.: Semi-supervised svm batch mode active learning for image retrieval. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, USA (2008)Google Scholar
  20. 20.
    Yu, K., Zhu, S., Xu, W., Gong, Y.: Non-greedy active learning for text categorization using convex ansductive experimental design. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Singapore, pp. 635–642 (2008)Google Scholar
  21. 21.
    Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research 5, 361–397 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Yu Zhang
    • 1
  • Dit-Yan Yeung
    • 1
  1. 1.Hong Kong University of Science and TechnologyHong Kong

Personalised recommendations