Skip to main content

A Hierarchical Neural Network Architecture for Classification

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7367)

Abstract

In this paper, a hierarchical neural network with cascading architecture is proposed and its application to classification is analyzed. This cascading architecture consists of multiple levels of neural network structure, in which the outputs of the hidden neurons in the higher hierarchical level are treated as an equivalent input data to the input neurons at the lower hierarchical level. The final predictive result is obtained through a modified weighted majority vote scheme. In this way, it is hoped that new patterns could be learned from hidden layers at each level and thus the combination result could significantly improve the learning performance of the whole system. In simulation, a comparison experiment is carried out among our approach and two popular ensemble learning approaches, bagging and AdaBoost. Various simulation results based on synthetic data and real data demonstrate this approach can improve the classification performance.

Keywords

  • hierarchical neural network
  • ensemble learning
  • classification
  • bagging
  • AdaBoost

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-31346-2_5
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-31346-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Madabhushi, A., Feldman, M.D., Metaxas, D.N., Tomaszeweski, J., Chute, D.: Automated detection of prostatic adenocarcinoma from high-resolution ex vivo MRI. IEEE Transactions on Medical Imaging 24, 1611–1625 (2005)

    CrossRef  Google Scholar 

  2. Oh, S., Lee, M.S., Zhang, B.T.: Ensemble Learning with Active Example Selection for Imbalanced Biomedical Data Classification. IEEE/ACM Transactions on Computational Biology and Bioinformatics 8, 316–325 (2011)

    CrossRef  Google Scholar 

  3. Zhang, K.H., Li, A.G., Song, B.W.: Fraud Detection in Tax Declaration Using Ensemble ISGNN. In: WRI World Congress on Computer Science and Information Engineering, vol. 4, pp. 237–240 (2009)

    Google Scholar 

  4. Lin, W.Y., Hu, Y.H., Tsai, C.F.: Machine Learning in Financial Crisis Prediction: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews (in press, 2012)

    Google Scholar 

  5. Chen, C.J., Chen, T.C., Ou, J.C.: Power System Stabilizer Using a New Recurrent Neural Network for Multi-Machine. In: Power and Energy Conference, pp. 68–72 (2006)

    Google Scholar 

  6. He, H., Cao, Y., Cao, Y., Wen, J.: Ensemble Learning for Wind Profile Prediction with Missing Values. Neural Computing & Applications 1–6 (2011)

    Google Scholar 

  7. He, H., Chen, S., Li, K., Xu, X.: Incremental Learning from Stream Data. IEEE Trans. Neural Networks 22(12), 1901–1914 (2011)

    CrossRef  Google Scholar 

  8. He, H., Chen, S.: IMORL: Incremental Multiple Objects Recognition and Localization. IEEE Trans. Neural Networks 19(10), 1727–1738 (2008)

    CrossRef  Google Scholar 

  9. Polikar, R.: Ensemble Based Systems in Decision Making. IEEE Circuits and Systems Magazine 6, 21–45 (2006)

    CrossRef  Google Scholar 

  10. Polikar, R.: Bootstrap Inspired Techniques in Computational Intelligence. IEEE Signal Processing Magazine 24, 56–72 (2007)

    CrossRef  Google Scholar 

  11. Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)

    MathSciNet  MATH  Google Scholar 

  12. Breiman, L.: Arching classifiers. Annals of Statistics 26, 801–849 (1998)

    MathSciNet  MATH  CrossRef  Google Scholar 

  13. Freund, Y., Schapire, R.E.: Experiments with new boosting algorithm. In: Proceedings of the 13th the International Conference on Machine Learning, pp. 148–156 (1996)

    Google Scholar 

  14. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and application to boosting. Journal of Computer and System Sciences 55, 119–139 (1996)

    MathSciNet  CrossRef  Google Scholar 

  15. LeCun, Y.A., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient BackProp. In: Orr, G.B., Müller, K.-R. (eds.) NIPS-WS 1996. LNCS, vol. 1524, pp. 9–50. Springer, Heidelberg (1998)

    CrossRef  Google Scholar 

  16. UCI Machine Learning Rerpository, http://archive.ics.uci.edu/ml/datasets.html

  17. He, H., Garcia, E.A.: Learning from Imbalanced Data. IEEE Trans. Knowledge and Data Engineering 21(9), 1263–1284 (2009)

    CrossRef  Google Scholar 

  18. Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)

    MathSciNet  MATH  CrossRef  Google Scholar 

  19. Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks. Communications of ACM 54, 95–103 (2009)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Wang, J., He, H., Cao, Y., Xu, J., Zhao, D. (2012). A Hierarchical Neural Network Architecture for Classification. In: Wang, J., Yen, G.G., Polycarpou, M.M. (eds) Advances in Neural Networks – ISNN 2012. ISNN 2012. Lecture Notes in Computer Science, vol 7367. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31346-2_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-31346-2_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-31345-5

  • Online ISBN: 978-3-642-31346-2

  • eBook Packages: Computer ScienceComputer Science (R0)