Active Learning for Regression Based on Query by Committee

  • Robert Burbidge
  • Jem J. Rowland
  • Ross D. King
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4881)


We investigate a committee-based approach for active learning of real-valued functions. This is a variance-only strategy for selection of informative training data. As such it is shown to suffer when the model class is misspecified since the learner’s bias is high. Conversely, the strategy outperforms passive selection when the model class is very expressive since active minimization of the variance avoids overfitting.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Seung, H., Opper, M., Sompolinsky, H.: Query by committee. In: Proceedings of the Fifth Workshop on Computational Learning Theory, pp. 287–294. Morgan Kaufmann, San Mateo, CA (1992)CrossRefGoogle Scholar
  2. 2.
    Freund, Y., Seung, H., Shamir, E., Tishby, N.: Selective sampling using the query by committee algorithm. Machine Learning 28(2–3), 133–168 (1997)zbMATHCrossRefGoogle Scholar
  3. 3.
    Abe, N., Mamitsuka, H.: Query learning strategies using boosting and bagging, pp. 1–9. Morgan Kaufmann, San Francisco, CA (1998)Google Scholar
  4. 4.
    Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neural Computation 4, 1–58 (1992)CrossRefGoogle Scholar
  5. 5.
    Cohn, D.A., Ghahramani, Z., Jordan, M.I.: Active learning with statistical models. Journal of Artificial Intelligence Research 4, 129–145 (1996)zbMATHGoogle Scholar
  6. 6.
    Cohn, D.: Neural network exploration using optimal experiment design. Morgan Kaufmann, Denver, CO (1996)Google Scholar
  7. 7.
    Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems 7, pp. 231–238. MIT Press, Denver, CO (1995)Google Scholar
  8. 8.
    RayChaudhuri, T., Hamey, L.G.: Minimisation of data collection by active learning. In: Proceedings of the 1995 IEEE International Conference on Neural Networks, Perth, Australia, vol. 3, pp. 1338–1341 (1995)Google Scholar
  9. 9.
    RayChaudhuri, T., Hamey, L.G.: Accurate modelling with minimised data collection — an active learning algorithm. In: Proceedings of the Seventh Australian Conference on Neural Networks, pp. 11–15. Australian National University, Canberra, Australia (1996)Google Scholar
  10. 10.
    RayChaudhuri, T., Hamey, L.G.: Active learning for nonlinear system identification and control. In: Gertler, J., Cruz, J.J.B., Peshkin, M. (eds.) Proceedings of 13th International Federation of Automatic Control World Congress, vol. F, pp. 193–197. Pergamon, San Francisco, CA (1996)Google Scholar
  11. 11.
    RayChaudhuri, T., Hamey, L.G.: Cost-effective querying leading to dual control. Technical Report 96-07, School of MPCE, Macquarie University, New South Wales 2109, Australia (June 1996)Google Scholar
  12. 12.
    Quenouille, M.: Approximate tests of correlation in time series. Journal of the Royal Statistical Society B 11, 68–84 (1949)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Zhang, C., Chen, T.: An active learning framework for content-based information retrieval. IEEE Transactions on multimedia 4(2), 260–268 (2002)CrossRefGoogle Scholar
  14. 14.
    Lindenbaum, M., Markovitch, S., Rusakov, D.: Selective sampling for nearest neighbor classifiers. Machine Learning 54(2), 125–152 (2004)zbMATHCrossRefGoogle Scholar
  15. 15.
    Sugiyama, M.: Active learning in approximately linear regression based on conditional expectation of generalization error. Journal of Machine Learning Research 7, 141–166 (2006)MathSciNetGoogle Scholar
  16. 16.
    Weins, D.: Robust weights and designs for biased regression models: least squares and generalized M-estimation. Journal of Statistical Planning and Inference 83(2), 395–412 (2000)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Bishop, C.: Neural Networks for Pattern Recognition. Clarendon Press, Oxford (1995)Google Scholar
  18. 18.
    Harrison, D., Rubinfeld, D.: Hedonic prices and the demand for clean air. J. Environ. Economics and Management 5, 81–102 (1978)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Robert Burbidge
    • 1
  • Jem J. Rowland
    • 1
  • Ross D. King
    • 1
  1. 1.Department of Computer Science, University of Wales, Aberystwyth, Penglais, Aberystwyth, SY23 3DBUK

Personalised recommendations