Neural Processing Letters

, Volume 15, Issue 3, pp 247–260 | Cite as

Supervised Training Using an Unsupervised Approach to Active Learning

  • A. P. Engelbrecht
  • R. Brits


Active learning algorithms allow neural networks to dynamically take part in the selection of the most informative training patterns. This paper introduces a new approach to active learning, which combines an unsupervised clustering of training data with a pattern selection approach based on sensitivity analysis. Training data is clustered into groups of similar patterns based on Euclidean distance, and the most informative pattern from each cluster is selected for training using the sensitivity analysis incremental learning algorithm in (Engelbrecht and Cloete, 1999). Experimental results show that the clustering approach improves on standard active learning as presented in (Engelbrecht and Cloete, 1999).

active learning clustering incremental learning pattern informativeness sensitivity analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Cohn, D. A.: Neural network exploration using optimal experiment design. AI Memo No 1491, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1994.Google Scholar
  2. Cohn, D. A., Atlas, L. and Ladner, R.: Improving Generalization with Active Learning. Machine Learning, 15 (1994), 201–221.Google Scholar
  3. Cohn, D. A., Ghahramani, Z. and Jordan, M. I.: Active learning with statistical models. Journal of Artificial Intelligence Research, 4 (1996), 129–145.Google Scholar
  4. Engelbrecht, A. P. and Cloete, I.: Selective learning using sensitivity analysis. In: Proceedings of IEEE World Congress on Computational Intelligence, Anchorage, Alaska, (1988), pp. 1150–1155.Google Scholar
  5. Engelbrecht, A. P. and Cloete, I.: Incremental learning using sensitivity analysis. In IEEE International Joint Conference on Neural Networks, Washington DC, USA, paper 380, 1999.Google Scholar
  6. Fukumizu, K.: Active learning in multilayer perceptrons. In: D. S. Touretzky, M. C. Mozer and M. E. Hasselmo, (eds.), Advances in Neural Information Processing Systems, 8 (1996), 295–301.Google Scholar
  7. Hampshire, J. B. and Waibel, A. H.: A Novel objective function for improved phoneme recognition using time-delay neural networks. IEEE Transactions on Neural Networks, 1(2) (1990), 216–228.Google Scholar
  8. Hunt, S. D. and Deller, Jr. J. R.: Selective training of feedforward artificial neural networks using matrix perturbation theory. Neural Networks, 8(6) (1995), 931–944.Google Scholar
  9. Hwang, J-N., Choi, J. J., Oh, S. and Marks II, R. J.: Query-based learning applied to partially trained multilayer perceptrons. IEEE Transactions on Neural Networks, 2(1) (1991), 131–136.Google Scholar
  10. MacKay, D. J. C.: Bayesian Methods for Adaptive Models. PhD Thesis, California Institute of Technology, 1992a.Google Scholar
  11. MacKay, D. J. C.: Information-based objective functions for active data selection. Neural Computation, 4 (1992b), 590–604.Google Scholar
  12. Plutowski, H. and White, H. Selecting concise training sets from clean data. IEEE Transactions on Neural Networks, 4(2) (1993), 305–318.Google Scholar
  13. Röbel, A.: Dynamic pattern selection: effectively training backpropagation neural networks. International Conference on Artificial Neural Networks, 1 (1994a), 643–646.Google Scholar
  14. Röbel, A.: Dynamic pattern selection for faster learning and controlled generalization of neural networks. European Symposium on Artificial Neural Networks, 1994b.Google Scholar
  15. Seung, H. S., Opper, M. and Sompolinsky, H.: Query by Committee. In: Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory (1992), pp. 287–299.Google Scholar
  16. Sung, K. K. and Niyogi, P.: A Formulation for active learning with applications to object detection. AI Memo No 1438, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1996.Google Scholar
  17. Zhang, B.-T.: Accelerated learning by active example Selection. International Journal of Neural Systems, 5(1) (1994), 67–75.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • A. P. Engelbrecht
    • 1
  • R. Brits
    • 1
  1. 1.Department of Computer ScienceUniversity of PretoriaPretoriaSouth Africa

Personalised recommendations