Skip to main content
SpringerLink
Log in

Development of the algorithm of adaptive construction of hierarchical neural network classifiers

  • Published:
Optical Memory and Neural Networks Aims and scope Submit manuscript

Abstract

This paper presents the development of the algorithm for adaptive construction of hierarchical neural network classifiers based on automatic modification of the desired response of a perceptron with a small number of neurons in a single hidden layer. Improved versions of the algorithm are tested on standard benchmark problems Vowels and MNIST. A discussion of the results, strengths and weaknesses of the algorithm, directions of further work on its testing and improvement, is provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Dolenko, S.A., Orlov, Yu.V., Persiantsev, I.G., Shugai J.S., and Eremin, E.K., Adaptive method of construction of a perceptron-based hierarchical structure of classifiers, The 4th Open Russian-German Workshop on Pattern Recognition and Image Analysis (Valdai, Russia, 1996), pp. 45–49.

    Google Scholar 

  2. Dolenko, S.A., Orlov, Yu.V., Persiantsev, I.G., Shugai, Ju.S., and Eremin, E.K., The perceptron-based hierarchical structure of classifiers constructed by the adaptive method, Pattern Recognit. Image Anal., 1997, vol. 7, no. 1, pp. 24–28.

    Google Scholar 

  3. Dolenko, S.A., Orlov, Yu.V., Persiantsev, I.G., Shugai, Ju.S., and Eremin, E.K., Self-organizing construction of hierarchical structure of multi-layer perceptrons, Proceedings Fifth Int. Conf. on Artificial Neural Networks (UK, University of Cambridge, Churchill College, 1997), IEE Conference Publication, 1997, vol. 440, pp. 285–290.

    Article  Google Scholar 

  4. Dolenko, S.A., Orlov, Yu.V., Persiantsev, I.G., and Shugai, Yu.S., Use of self-organization principles in construction of hierarchical neural network classifiers, Pattern Recognit. Image Anal., 1998, vol. 8, no. 2, pp. 122–124.

    Google Scholar 

  5. Orlov, Yu.V., Development and study of the algorithms of construction of neural network complexes for classification of data of physical measurements, Candidate Thesis in Phys. and Math., SINP MSU, 2003 (in Russian).

    Google Scholar 

  6. Persiantsev, I.G., Adaptive construction of hierarchical neural network systems for classification and for segmentation of time series, XII All-Russian Scientific and Technical Conference “Neuroinformatics-2010”: Lectures in Neuroinformatics, Moscow, NIJaU MIFI, 2010, pp. 212–242 (in Russian).

    Google Scholar 

  7. Svetlov, V.A., Persiantsev, I.G., Shugay, J.S., and Dolenko, S.A., A New implementation of the algorithm of adaptive construction of hierarchical neural network classifiers, Opt. Mem. Neural Networks (Inform. Opt.), 2015, vol. 24, no. 4, pp. 288–294.

    Article  Google Scholar 

  8. Utgoff, P.E., Perceptron trees: A case study in hybrid concept representations, Connection Sci., 1989, vol. 1, no. 4, pp. 377–391.

    Article  Google Scholar 

  9. Utgoff, P.E. and Carla E., An incremental method for finding multivariate splits for decision trees, Proceedings of the Seventh International Conference on Machine Learning, 1990, pp. 58–65.

    Google Scholar 

  10. Guo, Heng and Gelfand, S.B., Classification trees with neural network feature extraction, IEEE Trans. Neural Networks, 1992, vol. 3, no. 6, pp. 923–933.

    Article  Google Scholar 

  11. Sakar, A. and Mammone, R.J., Growing and pruning neural tree networks, IEEE Trans. Computers, 1993, vol. 42, no. 3, pp. 291–299.

    Article  Google Scholar 

  12. Song, Hee-Heon and Lee, Seong-Whan, A self-organizing neural tree for large-set pattern classification, IEEE Trans. Neural Networks, 1998, vol. 9, no. 3, pp. 369–380.

    Article  Google Scholar 

  13. Huang, De-Shuang, Guo, Lin, et al., A novel clustering-neural tree for pattern classification, 2004 IEEE International Joint Conference on Neural Networks, Proceedings, 2004, vol. 2, pp. 1303–1308.

    Google Scholar 

  14. Foresti, Gian Luca and Dolso, T., An adaptive high-order neural tree for pattern recognition, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2004, vol. 34, no. 2, pp. 988–996.

    Article  Google Scholar 

  15. Foresti, Gian Luca and Micheloni, Christian, Generalized neural trees for pattern classification, IEEE Trans. Neural Networks, 2002, vol. 13, no. 6, pp. 1540–1547.

    Article  Google Scholar 

  16. Micheloni, Christian, Rani, Asha, Kumar, Sanjeev, and Foresti, Gian Luca, A balanced neural tree for pattern classification, Neural Networks, 2012, vol. 27, pp. 81–90.

    Article  Google Scholar 

  17. Lichman, M., UCI Machine Learning Repository. Irvine, CA: University of California, School of Information and Computer Science, 2013. http://archive.ics.uci.edu/ml. Connectionist Bench (Vowel Recognition–Deterding Data) Data Set.https://archive.ics.uci.edu/ml/datasets/Connectionist+Bench+(Vowel+Recognition+-+ Deterding+Data). Cited November 21, 2016.

    Google Scholar 

  18. LeCun, Y., Cortes, C., and Burges, C.J.C., The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/. Cited November 21, 2016.

    Google Scholar 

  19. Sutskever, I. et al., On the importance of initialization and momentum in deep learning, ICML, 2013, vol. 28, no. 3, pp. 1139–1147.

    Google Scholar 

  20. Softmax Classification. http://knet.readthedocs.io/en/latest/softmax.html. Cited November 21, 2016.

Download references

Author information

Authors and Affiliations

  1. Skobeltsyn Institute of Nuclear Physics, Lomonosov Moscow State University, Moscow, Russia

    V. A. Svetlov & S. A. Dolenko

  2. Physical Department, Lomonosov Moscow State University, Moscow, Russia

    V. A. Svetlov

Authors
  1. V. A. Svetlov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. A. Dolenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. A. Svetlov.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Svetlov, V.A., Dolenko, S.A. Development of the algorithm of adaptive construction of hierarchical neural network classifiers. Opt. Mem. Neural Networks 26, 40–46 (2017). https://doi.org/10.3103/S1060992X17010076

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S1060992X17010076

Keywords

Search

Navigation