Robust Active Learning for Linear Regression via Density Power Divergence

  • Yasuhiro Sogawa
  • Tsuyoshi Ueno
  • Yoshinobu Kawahara
  • Takashi Washio
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7665)


The performance of active learning (AL) is crucially influenced by the existence of outliers in input samples. In this paper, we propose a robust pool-based AL measure based on the density power divergence. It is known that the density power divergence can be accurately estimated even under the existence of outliers within data. We further derive an AL scheme based on an asymptotic statistical analysis on the M-estimator. The performance of the proposed framework is investigated empirically using artificial and real-world data.


Active Learning Density Power Divergence Regression 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Campbell, C., Cristianini, N., Smola, A.: Query learning with large margin classifiers. In: Proceedings of the 17th Int. Conf. on Machine Learning, pp. 111–118 (2000)Google Scholar
  2. 2.
    Hakkani-Tur, D., Riccardi, G., Gorin, A.: Active learning for automatic speech recognition. In: IEEE Int. Conf. on Acoustics, Speech and Signal Processing, vol. 4, pp. 3904–3907 (2002)Google Scholar
  3. 3.
    McCallum, A., Nigam, K.: Employing EM in pool-based active learning for text classification. In: Proceedings of the 15th Int. Conf. on Machine Learning, pp. 350–358 (1998)Google Scholar
  4. 4.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  5. 5.
    Settles, B., Craven, M.: An analysis of active learning strategies for sequence labeling tasks. In: Proceedings of the Conf. on Empirical Methods in Natural Language Processing, pp. 1070–1079. Association for Computational Linguistics (2008)Google Scholar
  6. 6.
    Settles, B.: Active learning literature survey. Technical Report Computer Science Technical Report 1648, University of Wisconsin-Madison (2010)Google Scholar
  7. 7.
    Zhang, T., Oles, F.: The value of unlabeled data for classification problems. In: Proceedings of the 17th Int. Conf. on Machine Learning, pp. 1191–1198 (2000)Google Scholar
  8. 8.
    Van der Vaart, A.: Asymptotic statistics. Cambridge Univ. Pr. (2000)Google Scholar
  9. 9.
    Basu, A., Harris, I., Hjort, N., Jones, M.: Robust and efficient estimation by minimising a density power divergence. Biometrika 85(3), 549–559 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Fujisawa, H., Eguchi, S.: Robust parameter estimation with a small bias against heavy contamination. Journal of Multivariate Analysis 99(9), 2053–2081 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Hoi, S., Jin, R., Zhu, J., Lyu, M.: Batch mode active learning and its application to medical image classification. In: Proceedings of the 23rd Int. Conf. on Machine Learning, pp. 417–424 (2006)Google Scholar
  12. 12.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007)Google Scholar
  13. 13.

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Yasuhiro Sogawa
    • 1
  • Tsuyoshi Ueno
    • 2
  • Yoshinobu Kawahara
    • 1
  • Takashi Washio
    • 1
  1. 1.ISIROsaka UniversityIbarakiJapan
  2. 2.Japan Science and Technology AgencyKita-kuJapan

Personalised recommendations