Advertisement

Using Parity-N Problems as a Way to Compare Abilities of Shallow, Very Shallow and Very Deep Architectures

  • Paweł RóżyckiEmail author
  • Janusz Kolbusz
  • Tomasz Bartczak
  • Bogdan M. Wilamowski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9119)

Abstract

This paper presents a new concept of a dual neural network which is hybrid of linear and nonlinear network. This approach allows for solving the problem of Parity-3 with only one sigmoid neuron or Parity-7 with 2 sigmoid neurons that is shown in the analytical and experimental manner. The paper describes the architecture of ANN, presents an analytical way of choosing the weights and the number of neurons, and provides the results of network training for different ANN architectures solving the Parity-N problem.

Keywords

Parity-N problem ANN architecture Summator in output layer 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Mandal, S., Choudhury, J., Bhadra, S., De, D.: Growth Estimation with Artificial Neural Network Considering Weather Parameters Using Factor and Principal Component Analysis. In: Proceedings of the 10th International Conference on Information Technology, Rourkela (2007)Google Scholar
  2. 2.
    Quan, Y.: Research on weather forecast based on neural networks. In: Proceedings of the 3rd World Congress on Intelligent Control and Automation, Beijing (2000)Google Scholar
  3. 3.
    Ghate, V.N., Dudu, S.V.: Cascade neural-network-based fault classifier for three-phase induction motor. IEEE Transactions on Industrial Electronics 58(5), 1555–1563 (2011)CrossRefGoogle Scholar
  4. 4.
    Orlowska-Kowalska, T., Dybkowski, M., Szabat, K.: Adaptive sliding-mode neuro-fuzzy control of the two-mass induction motor drive without mechanical sensors. IEEE Transactions on Industrial Electronics 57(2), 553–564 (2010)CrossRefGoogle Scholar
  5. 5.
    Pucci, M., Cirrincione, M.: Neural MPPT control of wind generators with induction machines without speed sensors. IEEE Transactions on Industrial Electronics 58(1), 37–47 (2011)CrossRefGoogle Scholar
  6. 6.
    El-Sousy, F.F.M.: Hybrid H based wavelet-neural-network tracking control for permanent-magnet synchronous motor servo drives. IEEE Transactions on Industrial Electronics 57(9), 3157–3166 (2010)CrossRefGoogle Scholar
  7. 7.
    Xia, C., Guo, C., Shi, T.: A neural-network-identifier and fuzzy-controller-based algorithm for dynamic decoupling control of permanent-magnet spherical motor. IEEE Transactions on Inductrial Electronics 57(8), 2868–2878 (2010)CrossRefGoogle Scholar
  8. 8.
    Le, Q.N., Jeon, J.W.: Neural-network-based low-speed-damping controller for stepper motor with an FPGA. IEEE Transactions on Industrial Electronics 57(9), 3167–3180 (2010)CrossRefGoogle Scholar
  9. 9.
    Juang, C.-F., Chang, Y.-C., Hsiao, C.-M.: Evolving gaits of a hexapod robot by recurrent neural networks with symbiotic species-based particle swarm optimization. IEEE Transactions on Industrial Electronics 58(7), 3110–3119 (2011)CrossRefGoogle Scholar
  10. 10.
    Tsa, C.-C.I., Huang, H.-C., Lin, S.-C.: Adaptive neural network control of a self-balancing two-wheeled scooter. IEEE Transactions on Industrial Electronics 57(4), 1420–1428 (2010)CrossRefGoogle Scholar
  11. 11.
    Charkhgard, M., Farrokhi, M.: State-of-charge estimation for lithium-ion batteries using neural networks and EKF. IEEE Transactions on Industrial Electronics 57(12), 4178–4187 (2010)CrossRefGoogle Scholar
  12. 12.
    Yahyaou, A., Fnaiech, N., Fnaiech, F.: A Suitable Initialization Procedure for Speeding a Neural Network Job-Shop Scheduling. IEEE Transactions on Industrial Electronics 58(3), 1052–1060 (2011)CrossRefGoogle Scholar
  13. 13.
    Machado, V., Neto, A., de Melo, J.D.: A neural network multiagent architecture applied to industrial networks for dynamic allocation of control strategies using standard function blocks. IEEE Transactions on Industrial Electronics 57(5), 1823–1834 (2010)CrossRefGoogle Scholar
  14. 14.
    Lu, C.-H.: Wavelet fuzzy neural networks for identification and predictive control of dynamic systems. IEEE Transactions on Industrial Electronics 58(7), 3046–3058 (2011)CrossRefGoogle Scholar
  15. 15.
    Wilamowski, B.M., Kaynak, O.: Oil well diagnosis by sensing terminal characteristics of the induction motor. IEEE Transactions on Industrial Electronics 47(5), 1100–1107 (2000)CrossRefGoogle Scholar
  16. 16.
    Wilamowski, B.M., Yu, H.: NNT - Neural Networks Trainer (2015), http://www.eng.auburn.edu/~wilambm/nnt/index.htm (accessed February 1, 2015)
  17. 17.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)CrossRefGoogle Scholar
  18. 18.
    Werbos, P.J.: Back-propagation: Past and future. In: Proceedings of International Conference on Neural Networks, San Diego (1988)Google Scholar
  19. 19.
    Levenberg, K.: A method for the solution of certain problems in least squares. Quarterly of Applied Mathematics 2, 164–168 (1944)MathSciNetGoogle Scholar
  20. 20.
    Hagan, M.T., Menha, M.B.J.: Training feedforward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks 5(6), 989–993 (1994)CrossRefGoogle Scholar
  21. 21.
    Wilamowski, B.M., Cotton, N.J., Kaynak, O., Dundar, G.: Computing gradient vector and jacobian matrix in arbitrarily connected neural networks. IEEE Transactions on Industrial Electronics 55(10), 3784–3790 (2008)CrossRefGoogle Scholar
  22. 22.
    Wilamowski, B.M., Yu, H.: Improved Computation for Levenberg Marquardt Training. IEEE Transactions on Neural Networks 21(6), 930–937 (2010)CrossRefGoogle Scholar
  23. 23.
    Wilamowski, B.M., Yu, H.: Neural Network Learning Without Backpropagation. IEEE Transactions on Neural Networks 21(11), 1793–1803 (2010)CrossRefGoogle Scholar
  24. 24.
    Phansalkar, V.V., Sastry, P.S.: Analysis of the back-propagation algorithm with momentum. IEEE Transactions on Neural Networks 5(3), 505–506 (1994)CrossRefGoogle Scholar
  25. 25.
    Wilamowski, B.M., Torvik, L.: Modification of gradient computation in the back-propagation algorithm. In: ANNIE 1993 - Artificial Neural Networks in Engineering, St. Louis (1993)Google Scholar
  26. 26.
    Hunter, D.S., Yu, H., Pukish, M.S., Kolbusz, J., Wilamowski, B.M.: Selection of Proper Neural Network Sizes and Architectures – A Comparative Study. IEEE Transactions on Industrial Informatics 8(2), 228–240 (2012)CrossRefGoogle Scholar
  27. 27.
    Hunter, D.S., Wilamowski, B.M.: Parallel Multi-Layer Neural Network Architecture with Improved Efficiency. In: Proceedings of the 4th International Conference on Human System Interaction, Yokohama (2011)Google Scholar
  28. 28.
    Trenn, S.: Mulit-layer Perceptrons: Approximation Order and Necessary Number of Hidden Units. IEEE Transactions on Neural Networks 19(5), 836–844 (2008)CrossRefGoogle Scholar
  29. 29.
    Wilamowski, B.M.: Neural Network Architectures and Learning Algorithms. IEEE Industrial Electronics Magazine, 56–63 (December 2009)Google Scholar
  30. 30.
    Hochreiter, S.: The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions. Int. J. Unc. Fuzz. Knowl. Based Syst. 06, 107 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Paweł Różycki
    • 1
    Email author
  • Janusz Kolbusz
    • 1
  • Tomasz Bartczak
    • 2
  • Bogdan M. Wilamowski
    • 2
  1. 1.University of Information Technology and Management in RzeszowRzeszowPoland
  2. 2.Auburn UniversityAuburnUSA

Personalised recommendations