Active Learning with the Probabilistic RBF Classifier

  • Constantinos Constantinopoulos
  • Aristidis Likas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4131)


In this work we present an active learning methodology for training the probabilistic RBF (PRBF) network. It is a special case of the RBF network, and constitutes a generalization of the Gaussian mixture model. We propose an incremental method for semi-supervised learning based on the Expectation-Maximization (EM) algorithm. Then we present an active learning method that iteratively applies the semi-supervised method for learning the labeled and unlabeled observations concurrently, and then employs a suitable criterion to select an unlabeled observation and query its label. The proposed criterion selects points near the decision boundary, and facilitates the incremental semi-supervised learning that also exploits the decision boundary. The performance of the algorithm in experiments using well-known data sets is promising.


Active Learning Gaussian Mixture Model Decision Boundary Unlabeled Data Active Learning Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Castelli, V., Cover, T.: On the exponential value of labeled samples. Pattern Recognition Letters 16, 105–111 (1995)CrossRefGoogle Scholar
  2. 2.
    Cohn, D., Ghahramani, Z., Jordan, M.: Active learning with statistical models. Journal of Artificial Intelligence Research 4, 129–145 (1996)MATHGoogle Scholar
  3. 3.
    McLachlan, G., Peel, D.: Finite Mixture Models. John Wiley & Sons, Chichester (2000)MATHCrossRefGoogle Scholar
  4. 4.
    Freund, Y., Seung, H.S., Shamir, E., Tishby, N.: Selective sampling using the query by committee algorithm. Machine Learning 28, 133–168 (1997)MATHCrossRefGoogle Scholar
  5. 5.
    McCallum, A.K., Nigam, K.: Employing EM in pool-based active learning for text classification. In: Shavlik, J.W. (ed.) Proc. 15th International Conference on Machine Learning. Morgan Kaufmann, San Francisco (1998)Google Scholar
  6. 6.
    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)MATHMathSciNetGoogle Scholar
  7. 7.
    Zhu, X., Lafferty, J., Ghahramani, Z.: Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions. In: Proc. 20th International Conference on Machine Learning (2003)Google Scholar
  8. 8.
    Ghahramani, Z., Jordan, M.: Supervised learning from incomplete data via an EM approach. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems 6. Morgan Kaufmann, San Francisco (1994)Google Scholar
  9. 9.
    Tadjudin, S., Landgrebe, A.: Robust parameter estimation for mixture model. IEEE Trans. Geoscience and Remote Sensing 38, 439–445 (2000)CrossRefGoogle Scholar
  10. 10.
    Miller, D., Uyar, H.: Combined learning and use for a mixture model equivalent to the RBF classifier. Neural Computation 10, 281–293 (1998)CrossRefGoogle Scholar
  11. 11.
    Titsias, M.K., Likas, A.: Shared kernel models for class conditional density estimation. IEEE Trans. Neural Networks 12(5), 987–997 (2001)CrossRefGoogle Scholar
  12. 12.
    Titsias, M.K., Likas, A.: Class conditional density estimation using mixtures with constrained component sharing. IEEE Trans. Pattern Anal. and Machine Intell. 25(7), 924–928 (2003)CrossRefGoogle Scholar
  13. 13.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
  14. 14.
    Constantinopoulos, C., Likas, A.: An incremental training method for the probabilistic RBF network. IEEE Trans. Neural Networks (2006) (to appear)Google Scholar
  15. 15.
    Bentley, J.L.: Multidimensional binary search trees used for associative searching. Communications of the ACM 18(9), 509–517 (1975)MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Titsias, M.K., Likas, A.: Mixture of experts classification using a hierarchical mixture model. Neural Computation 14(9), 2221–2244 (2002)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Constantinos Constantinopoulos
    • 1
  • Aristidis Likas
    • 1
  1. 1.Department of Computer ScienceUniversity of IoanninaIoanninaGreece

Personalised recommendations