Improved Margin Sampling for Active Learning

  • Jin Zhou
  • Shiliang Sun
Part of the Communications in Computer and Information Science book series (CCIS, volume 483)


Active learning is a learning mechanism which can actively query the user for labels. The goal of an active learning algorithm is to build an effective training set by selecting those most informative samples and improve the efficiency of the model within the limited time and resource. In this paper, we mainly focus on a state-of-the-art active learning method, the SVM-based margin sampling. However, margin sampling does not consider the distribution and the structural space connectivity among the unlabeled data when several examples are chosen simultaneously, which may lead to oversampling on dense regions. To overcome this shortcoming, we propose an improved margin sampling method by applying the manifold-preserving graph reduction algorithm to the original margin sampling method. Experimental results on multiple data sets demonstrate that our method obtains better classification performance compared with the original margin sampling.


Active learning Margin sampling Support vector machine Manifold-preserving graph reduction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boser, B.E., Guyou, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: 5th Workshop on Computational Learning Theory, Pittsburgh, pp. 144–152 (1992)Google Scholar
  2. 2.
    Campbell, C., Cristianini, N., Smola, A.: Query learning with large margin classifiers. In: 17th International Conference on Machine Learning, Stanford, pp. 111–118 (2000)Google Scholar
  3. 3.
    Cohn, D., Atlas, L., Ladner, R.: Improving generalization with active learning. Machine Learning 15, 201–221 (1994)Google Scholar
  4. 4.
    Ferecatu, M., Boujemaa, N.: Interactive remote-sensing image retrieval image retrieval. IEEE Transactions on Geoscience Remote Sensing 45, 818–826 (2007)CrossRefGoogle Scholar
  5. 5.
    Freund, Y., Seung, H.S., Shamir, E., Tishby, N.: Selective sampling using the query by committee algorithm. Machine Learning 28, 133–168 (1997)CrossRefzbMATHGoogle Scholar
  6. 6.
    Hern\(\acute{a}\)ndez, E.P., Ambroladze, A., Taylor, J.S., Sun, S.: PAC-Bayes bounds with data dependent priors. The Journal of Machine Learning Research 13, 3507–3531 (2012)Google Scholar
  7. 7.
    Huang, S., Jin, R., Zhou, Z.: Active learning by querying informative and representative examples. In: 24th Annual Conference on Neural Information Processing Systems, Vancouver, pp. 892–900 (2010)Google Scholar
  8. 8.
    Kapoor, A., Grauman, K., Urtasun, R., Darrell, T.: Active learning with Gaussian processed for object categorization. In: 11th International Conference on Computer Vision, Rio de Janeiro, pp. 1–8 (2007)Google Scholar
  9. 9.
    Mackay, D.J.C.: Information-based objective functions for active data selection. Neural Computation 4, 590–604 (1992)CrossRefGoogle Scholar
  10. 10.
    Nguyen, H.T., Smeulders, A.: Active learning using pre-clustering. In: 21st International Conference on Machine Learning, Banff, Canada, pp. 623–630 (2004)Google Scholar
  11. 11.
    Oskoei, M.A., Hu, H.: Support vector machine-based classification scheme for myoelectric control applied to upper limb. IEEE Transactions on Biomedical Engineering 55, 1956–1965 (2008)CrossRefGoogle Scholar
  12. 12.
    Sch\(\ddot{o}\)lkopf, B., Smola, A.J.: Learning with Kernels. MIT press, Cambridge (2002)Google Scholar
  13. 13.
    Schohn, G., Cohn, D.: Less is more: Active learning with support vectors machines. In: 17th International Conference on Machine Learning, Stanford, pp. 839–846 (2000)Google Scholar
  14. 14.
    Silva, C., Ribeiro, B.: Margin-based active learning and background knowledge in text mining. In: 4th International Conference on Hybird Intelligent Systems, Washington, pp. 8–13 (2004)Google Scholar
  15. 15.
    Sun, S., Hussain, Z., Taylor, J.S.: Manifold-preserving graph reduction for sparse semi-supervised learning. Neurocomputing 124, 13–21 (2013)CrossRefGoogle Scholar
  16. 16.
    Sun, S., Hardoon, D.: Active learning with extremely sparse labeled examples. Neurocomputing 73, 2980–2988 (2010)CrossRefGoogle Scholar
  17. 17.
    Tuia, D., Ratle, F., Pacifici, F., Kanevski, M.F., Emery, W.J.: Active learning methods for remote sensing image classification. IEEE Transactions on Geoscience Remote Sensing 47, 2218–2232 (2009)CrossRefGoogle Scholar
  18. 18.
    Zhang, Q., Sun, S.: Multiple-view multiple-learner active learning. Pattern Recognition 43, 3113–3119 (2010)CrossRefzbMATHGoogle Scholar
  19. 19.
    Zhou, J., Sun, S.: Active learning of Gaussian processes with manifold-preserving graph reduction. Neural Computing & Applications (2014), doi:10.1007/s00521-014-1643-8Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Jin Zhou
    • 1
  • Shiliang Sun
    • 1
  1. 1.Department of Computer Science and TechnologyEast China Normal UniversityShanghaiChina

Personalised recommendations