Neural Computing and Applications

, Volume 20, Issue 7, pp 1061–1074 | Cite as

An incremental online semi-supervised active learning algorithm based on self-organizing incremental neural network

  • Furao Shen
  • Hui Yu
  • Keisuke Sakurai
  • Osamu Hasegawa
Original Article

Abstract

An incremental online semi-supervised active learning algorithm, which is based on a self-organizing incremental neural network (SOINN), is proposed. This paper describes improvement of the two-layer SOINN to a single-layer SOINN to represent the topological structure of input data and to separate the generated nodes into different groups and subclusters. We then actively label some teacher nodes and use such teacher nodes to label all unlabeled nodes. The proposed method can learn from both labeled and unlabeled samples. It can query the labels of some important samples rather than selecting the labeled samples randomly. It requires neither prior knowledge, such as the number of nodes, nor the number of classes. It can automatically learn the number of nodes and teacher vectors required for a current task. Moreover, it can realize online incremental learning. Experiments using artificial data and real-world data show that the proposed method performs effectively and efficiently.

Keywords

Semi-supervised learning Active learning Online incremental learning Self-organizing incremental neural network 

Notes

Acknowledgements

This work was supported in part by the Fund of the National Natural Science Foundation of China (Grant No. 60975047, 60723003, 60721002), 973 Program (2010CB327903), and Jiangsu NSF grant (\#BK2009080). This study was also supported in part by the New Energy and Industrial Technology Development Organization (NEDO) of Japan.

References

  1. 1.
    Cohen I, Cozman FG, Sebe N, Cirelo MC, Huang TS (2004) Semisupervised learning of classifiers: theory, algorithms, and their application to human-computer interaction. IEEE Trans Pattern Anal Mach Intell 26(12):1553–1566CrossRefGoogle Scholar
  2. 2.
    Baraldi A, Bruzzone L, Blonda P (2006) A multiscale expectation-maximization semisupervised classifier suitable for badly posed image classification. IEEE Trans Image Process 15(8):2208–2225CrossRefGoogle Scholar
  3. 3.
    Yeung D-Y, Chang H (2007) A kernel approach for semisupervised metric learning. IEEE Trans Neural Netw 18(1):141–149CrossRefGoogle Scholar
  4. 4.
    Zaki SM, Yin H (2008) A semi-supersized learning algorithm for growing neural gas in face recognition. J Math Model Algorithms 7:425–435CrossRefMATHGoogle Scholar
  5. 5.
    Freund Y, Seung H, Shamir E, Tishby N (1997) Selective sampling using the query by committee algorithm. Mach Learn 28:133–168CrossRefMATHGoogle Scholar
  6. 6.
    Muslea I, Minston S, Knoblock C (2000) Selective sampling with redundant views. In: Proceedings of the national conference on artificial intelligence, pp 621–626Google Scholar
  7. 7.
    Kothari R, Jain V (2003) Learning from labeled and unlabeled data using a minimal number of queries. IEEE Trans Neural Netw 14(6):1496–1505CrossRefGoogle Scholar
  8. 8.
    Mingkun L, Sethi IK (2006) Confidence-based active learning. IEEE Trans Pattern Anal Mach Intell 28(8):1251–1261CrossRefGoogle Scholar
  9. 9.
    Muslea I, Minston S, Knoblock C (2002) Active + semi-supervised learning = robust multi-view learning. In: Proceedings of ICML-02, 19th international conference on machine learning, pp 435–442Google Scholar
  10. 10.
    Nigam K, McCallum AK, Thrum S, Mitchell T (2000) Text classification from labeled and unlabeled documents using em. Mach Learn 39:103–134CrossRefMATHGoogle Scholar
  11. 11.
    Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the conference on computational learning theory, pp 92–100Google Scholar
  12. 12.
    Bennett K, Demiriz A (1999) Semi-supervised support vector machines. Adv Neural Inf Process Syst 11:368–374Google Scholar
  13. 13.
    Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: ICML-03, 20th international conference on machine learning, pp 912–919Google Scholar
  14. 14.
    Zhu X, Lafferty J, Ghahramani Z (2003) Combining active learning and semi-supervised learning using gaussian fields and harmonic functions. In: ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, pp 58–65Google Scholar
  15. 15.
    Wang Z, Song Y, Zhang C (2009) Efficient active learning with boosting. In: SDM 2009, pp 1230–1241Google Scholar
  16. 16.
    Huang R, Lam W (2009) An active learning framework for semi-supervised document clustering with language modeling. Data Knowl Eng 68(1):49–67CrossRefGoogle Scholar
  17. 17.
    Hoi SCH, Jin R, Zhu J, Lyu MR (2009) Semi-supervised SVM batch mode active learning with applications to image retrieval. ACM Trans Inf Syst 27(3):16:1–16:29CrossRefGoogle Scholar
  18. 18.
    Carpenter GA, Grossberg S (1988) The art of adaptive pattern recognition by a self-organizing neural network. IEEE Comput 21:77–88Google Scholar
  19. 19.
    Shen F, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19:90–106CrossRefMATHGoogle Scholar
  20. 20.
    Kamiya Y, Shen F, Hasegawa O (2007) An incremental neural network for online supervised learning and topology learning. J Adv Comput Intell Intell Inform 11(1):87–95Google Scholar
  21. 21.
    Shen F, Hasegawa O (2008) A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Netw 21:1537–1547CrossRefGoogle Scholar
  22. 22.
    Sudo A, Sato A, Hasegawa O (2007) Associative memory for online incremental learning ina noisy environment. In: The 2007 international joint conference on neural networksGoogle Scholar
  23. 23.
    Shen F, Sudo A, Hasegawa O (2010) An online incremental learning pattern-based reasoning system. Neural Netw 23(1):135–143CrossRefGoogle Scholar
  24. 24.
    Shen F, Ogura T, Hasegawa O (2007) An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw 20:893–903CrossRefMATHGoogle Scholar
  25. 25.
    He X, Kojima R, Hasegawa O (2007) Developmental word grounding through a growing neural network with a humanoid robot. IEEE Trans Syst Man Cybern B 37(2):451–462CrossRefGoogle Scholar
  26. 26.
    He X, Ogura T, Satou A, Hasegawa O (2007) Developmental word acquisition and grammar learning by humanoid robots through a self-organizing incremental neural network. IEEE Trans Syst Man Cybern B 37(5):1357–1372CrossRefGoogle Scholar
  27. 27.
    Merz C, Murphy M (1996) Uci repository of machine learning database. University of California Department of Information, IrvineGoogle Scholar
  28. 28.
    Chang C-C, Lin C-J (2001) IBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm
  29. 29.
    Fritzke B (1995) A growing neural gas network learns topology. In: Advances in neural information processing systems 7. MIT, CambridgeGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2010

Authors and Affiliations

  • Furao Shen
    • 1
    • 2
  • Hui Yu
    • 1
    • 2
  • Keisuke Sakurai
    • 3
  • Osamu Hasegawa
    • 3
  1. 1.The State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingPeople’s Republic of China
  2. 2.Jiangyin Information Technology Research InstituteNanjing UniversityNanjingPeople’s Republic of China
  3. 3.Imaging Science and Engineering LaboratoryTokyo Institute of TechnologyTokyoJapan

Personalised recommendations