Advertisement

A New Constructive Algorithm for Designing and Training Artificial Neural Networks

  • Md. Abdus Sattar
  • Md. Monirul Islam
  • Kazuyuki Murase
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4984)

Abstract

This paper presents a new constructive algorithm, called problem dependent constructive algorithm (PDCA), for designing and training artificial neural networks (ANNs). Unlike most previous studies, PDCA puts emphasis on architectural adaptation as well as function level adaptation. The architectural adaptation is done by determining automatically the number of hidden layers in an ANN and of neurons in hidden layers. The function level adaptation, is done by training each hidden neuron with a different training set. PDCA uses a constructive approach to achieve both the architectural as well as function level adaptation. It has been tested on a number of benchmark classification problems in machine learning and ANNs. The experimental results show that PDCA can produce ANNs with good generalization ability in comparison with other algorithms.

Keywords

Artificial neural networks (ANNs) architectural adaptation function level adaptation constructive approach and generalization ability 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing System, vol. 2, pp. 524–532. Morgan Kaufmann, San Mateo, CA (1990)Google Scholar
  2. 2.
    Fahlman, S.E.: An empirical study of learning speed in back-propagation networks. Tech Report CMU-CS-88-162, Carnegie Mellon University (1988)Google Scholar
  3. 3.
    Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)CrossRefGoogle Scholar
  4. 4.
    Kwok, T.Y., Yeung, D.Y.: Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Transactions on Neural Networks 8, 630–645 (1997)CrossRefGoogle Scholar
  5. 5.
    Lehtokangas, M.: Modified cascade-correlation learning for classification. IEEE Transactions on Neural Networks 11, 795–798 (2000)CrossRefGoogle Scholar
  6. 6.
    Lehtokangas, M.: Fast initialization for cascade-correlation learning. IEEE Transactions on Neural Networks 10, 410–413 (1999)CrossRefGoogle Scholar
  7. 7.
    Lehtokangas, M.: Modeling with constructive backpropagation. Neural Networks 12, 707–716 (1999)CrossRefGoogle Scholar
  8. 8.
    Ma, L., Khorasani, K.: Constructive feedforward neural networks using hermite polynomial activation functions. IEEE Transactions on Neural Networks 16, 821–833 (2005)CrossRefGoogle Scholar
  9. 9.
    Monirul Islam, Md., Murase, K.: A new algorithm to design compact two-hidden-layer artificial neural networks. Neural Networks 14, 1265–1278 (2001)Google Scholar
  10. 10.
    Phatak, D.S., Koren, I.: Connectivity and performance tradeoffs in the cascade correlation learning architecture. IEEE Transactions on Neural Networks 5, 930–935 (1994)CrossRefGoogle Scholar
  11. 11.
    Prechelt, L.: Automatic early stopping using cross validation: quantifying the criteria. Neural Networks 11, 761–767 (1998)CrossRefGoogle Scholar
  12. 12.
    Reed, R.: Pruning algorithms - a survey. IEEE Transactions on Neural Networks 4, 740–747 (1993)CrossRefGoogle Scholar
  13. 13.
    Prechelt, L.: PROBEN1- A set of neural network benchmark problems and benchmarking rules. Technical Report 21/94, Faculty of Informatics, University of Karlsruhe, Germany (1994)Google Scholar
  14. 14.
    Riedmiller, M., Braun, H.: RPROP - A Fast Adaptive Learning Algorithm. In: Proc. of the 1992 International Symposium on Computer and Information Sciences, Turkey, November 1992, pp. 279–285 (1992)Google Scholar
  15. 15.
    Schapire, R.E.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)Google Scholar
  16. 16.
    Schwenk, H., Bengio, Y.: Boosting Neural Networks. Neural Computation 12, 1869–1887 (2000)CrossRefGoogle Scholar
  17. 17.
    Setiono, R., Hui, L.C.K.: Use of quasi-Newton method in a feed forward neural network construction algorithm. IEEE Trans. Neural Networks 6, 273–277 (1995)CrossRefGoogle Scholar
  18. 18.
    Tamura, S., Tateishi, M.: Capabilities of a four-layered feedforward neural network: four layers versus three. IEEE Transactions on Neural Networks 8, 251–255 (1997)CrossRefGoogle Scholar
  19. 19.
    Treadgold, N.K., Gedeon, T.D.: Exploring constructive cascade networks. IEEE Transactions on Neural Networks 10, 1335–1350 (1999)CrossRefGoogle Scholar
  20. 20.
    Yao, X.: A review of evolutionary artificial neural networks. International Journal of Intelligent Systems 8, 529–567 (1993)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Md. Abdus Sattar
    • 1
  • Md. Monirul Islam
    • 1
    • 2
  • Kazuyuki Murase
    • 2
    • 3
  1. 1.Department of Computer Science and EngineeringBangladesh University of Engineering and TechnologyDhakaBangladesh
  2. 2.Department of Human and Artificial Intelligence Systems, Graduate School of EngineeringUniversity of FukuiFukuiJapan
  3. 3.Research and Education Program for Life ScienceUniversity of FukuiFukuiJapan

Personalised recommendations