Skip to main content

Advertisement

SpringerLink
Log in

Tree-structured multilayer neural network for classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In traditional neural trees (NTs), each internal node is designed as a neural network (NN), such as single- or two-layer neural networks, to determine which branch should be followed for an input sample. Because each NN contained in the internal nodes is designed separately, the produced NT does not consider overall effectiveness. Thus, the designed NT is usually not an optimal NT. In this study, the tree-structured multilayer neural network (TSMLNN) is proposed for classification. The TSMLNN is similar to an NT, which is the result of dividing a deep multilayer NN into many small sub-networks. The TSMLNN has the advantages of both a multilayer NN and an NT. In addition, the split method is proposed to determine how to split the network in the TSMLNN. The genetic algorithm is proposed to automatically search for the weights, activation threshold of each node and the proper number of nodes at each layer according to both the computing complexity and classification error rate in the TSMLNN, and the proposed TSMLNN tends to be optimal. A heuristic method is also proposed to help users to decide which TSMLNN is the best within the classification error rate range. Finally, the performance of the proposed TSMLNN is compared with that of state-of-the-art neural networks in experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Reza T, Mohammadreza K (2018) A method for handwritten word spotting based on particle swarm optimization and multi-layer perceptron. IET Softw 12(2):152–159

    Article  MathSciNet  Google Scholar 

  2. Sudholt S, Fink GA (2016) A deep convolutional neural network for word spotting in handwritten documents. In: 15th international conference frontiers in handwriting recognition (ICFHR), Shenzhen, China, pp 277–282

  3. Khayyat M, Lam L, Suen CY (2014) Learning-based word spotting system for Arabic handwritten documents. Pattern Recogn 47(3):1021–1030

    Article  Google Scholar 

  4. Xu Y et al (2015) A regression approach to speech enhancement based on deep neural networks. IEEE Trans Audio Speech Lang Process 23(1):7–19

    Article  Google Scholar 

  5. Zagoris K, Pratikakis I, Gatos B (2017) Unsupervised word spotting in historical handwritten document images using document-oriented local features. IEEE Trans Image Process 26(8):4032–4041

    Article  MathSciNet  MATH  Google Scholar 

  6. Toselli AH, Puigcerver J, Vidal E (2016) Two methods to improve confidence scores for Lexicon-free word spotting in handwritten text. In: 15th international conference frontiers in handwriting recognition (ICFHR), Shenzhen, China, pp 349–354

  7. Siniscalchi SM et al (2014) An artificial neural network approach to automatic speech processing. Neurocomputing 140(22):326–338

    Article  Google Scholar 

  8. Luca B, Henriques JF, Valmadre J, Torr P, Vedaldi A (2016) Learning feed-forward one-shot learners. In Advances in NIPS, pp 523–531

  9. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  10. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  11. Zang F, Du B, Zhang L, Xu M (2016) Weakly supervised learning based on coupled convolutional neural networks for aircraft detection. IEEE Trans Geosci Remote Sens 54(9):53–63

    Google Scholar 

  12. Larochelle H (2009) Exploring strategies for training deep neural networks. Mach Learn 1:1–40

    MATH  Google Scholar 

  13. Hinton G et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  14. Lu X et al (2013) Speech enhancement based on deep denoising autoencoder. In: Proceedings of Interspeech, pp 436–440

  15. Simard PY et al (2003) Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings of international conference on document analysis and recognition, pp 958–963

  16. Ciresan D et al (2012) Multi-column deep neural networks for image classification. In: Proceedings of annual conference on computer vision and pattern recognition, pp 3642–3649

  17. Chen TE, Yang SI, Ho LT, Tsai KH, Chen YH, Chang YF, Lai YH, Wang SS, Tsao Y, Wu CC (2017) S1 and S2 heart sound recognition using deep neural networks. IEEE Trans Biomed Eng 64(2):372–380

    Article  Google Scholar 

  18. Zhang H, Chow TWS, Jonathan Wu QM (2016) Organizing books and authors by multilayer SOM. IEEE Trans Neural Netw Learn Syst 27(12):2537–2550

    Article  Google Scholar 

  19. Zhang H, Wang S, Xu X, Chow TWS, Jonathan Wu QM (2018) Tree2Vector: learning a vectorial representation for tree-structured data. IEEE Trans Neural Netw Learn Syst 29(11):5304–5318

    Article  MathSciNet  Google Scholar 

  20. Zhang H, Wang S, Zhao M, Xu X, Ye Y (2018) Locality reconstruction models for book representation. IEEE Trans Knowl Data Eng 30(10):1873–1886

    Article  Google Scholar 

  21. Guo H, Gelfand SB (1992) Classification trees with neural networks feature extraction. IEEE Trans Neural Netw 3(6):923–933

    Article  Google Scholar 

  22. Foresti GL, Micheloni C (2002) Generalized neural trees for pattern classification. IEEE Trans Neural Netw 13(6):1540–1547

    Article  Google Scholar 

  23. Foresti GL (2004) An adaptive high-order neural tree for pattern recognition. IEEE Trans Syst Man Cybern Part B Cybern 34(2):988–996

    Article  Google Scholar 

  24. Maji P (2008) Efficient design of neural network tree using a single splitting criterion. Nerocomputing 71:787–800

    Article  Google Scholar 

  25. Micheloni C, Rani A, Kumarb S, Foresti GL (2012) A balanced neural tree for pattern classification. Neural Netw 27:81–90

    Article  Google Scholar 

  26. Balestriero R (2017) Neural decision trees, 1–11. arXiv:1702.07360v2

  27. Yang SB (2018) Constrained-storage variable-branch neural tree for classification. Neural Comput Appl. https://doi.org/10.1007/s00521-017-3315-y

    Article  Google Scholar 

  28. Mahmoudabadi H, Izadi M, Menhaj MB (2009) A hybrid method for grade estimation using genetic algorithm and neural networks. Comput Geosci 13:91–101

    Article  MATH  Google Scholar 

  29. Samanta B, Bandopadhyay S, Ganguli R (2004) Data segmentation and genetic algorithms for sparse data division in Nome placer gold grade estimation using neural network and geostatistics. Min Explor Geol 11(1):69–76

    Google Scholar 

  30. Chatterjee S, Bandopadhyay S, Machuca D (2010) Ore grade prediction using a genetic algorithm and clustering based ensemble neural network model. Math Geosci 42(3):309–326

    Article  MATH  Google Scholar 

  31. Tahmasebi P, Hezarkhani A (2009) Application of optimized neural network by genetic algorithm. IAMG09. Stanford University; California, 2(2):15–23

  32. David OE, Greental I (2014) Genetic algorithms for evolving deep neural networks. GECCO’14, pp 12–16

  33. Le Borgne H, Guérin-Dugué A, O’Connor NE (2007) Learning midlevel image features for natural scene and texture classification. IEEE Trans Circuits Syst Video Technol 17(3):286–297

    Article  Google Scholar 

  34. Gonzalez RC, Woods RE (1992) Digital image processing. Addison-Wesley, Boston

    Google Scholar 

  35. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  36. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. arXiv:1409.4842

  37. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385

  38. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv:1502.01852

  39. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  40. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2014) Imagenet large scale visual recognition challenge. arXiv:1409.0575

Download references

Author information

Authors and Affiliations

  1. Department of Digital Content of Application and Management, Wenzao Ursuline University of Languages, 900 Mintsu 1st Road, Kaohsiung, 807, Taiwan

    Shiueng-Bien Yang

Authors
  1. Shiueng-Bien Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiueng-Bien Yang.

Ethics declarations

Conflict of interest

The author declares that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, SB. Tree-structured multilayer neural network for classification. Neural Comput & Applic 32, 5859–5873 (2020). https://doi.org/10.1007/s00521-019-04058-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04058-3

Keywords

Search

Navigation