Particle Swarm Optimization Based Higher Order Neural Network for Classification

  • Janmenjoy Nayak
  • Bighnaraj Naik
  • H. S. Behera
  • Ajith Abraham
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 31)


The maturity in the use of both the feed forward neural network and Multilayer perception brought the limitations of neural network like linear threshold unit and multi-layering in various applications. Hence, a higher order network can be useful to perform nonlinear mapping using the single layer of input units for overcoming the drawbacks of the above-mentioned neural networks. In this paper, a higher order neural network called Pi-Sigma neural network with standard back propagation Gradient descent learning and Particle Swarm Optimization algorithms has been coupled to develop an efficient robust hybrid training algorithm with the local and global searching capabilities for classification task. To demonstrate the capacity of the proposed PSO-PSNN model, the performance has been tested with various benchmark datasets from UCI machine learning repository and compared with the resulting performance of PSNN, GA-PSNN. Comparison result shows that the proposed model obtains a promising performance for classification problems.


Higher order neural network Classification PSO Pi-sigma neural network GA 



This work is supported by Department of Science and Technology (DST), Ministry of Science and Technology, New Delhi, Govt. of India, under grants No. DST/INSPIRE Fellowship/2013/585.


  1. 1.
    Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer, Berlin (2003)Google Scholar
  2. 2.
    Engelbrecht, A.P.: Computational Intelligence An Introduction, 2nd edn. Wiley, London (2007)CrossRefGoogle Scholar
  3. 3.
    Ugolottia, R., Nasheda, Y.S.G., Mesejoa, P., Ivekovi, S., Mussia, L., Cagnonia, S.: Particle swarm optimization and differential evolution for model-based object detection. Appl. Soft. Comput. 13, 3092–3105 (2013)CrossRefGoogle Scholar
  4. 4.
    Sun, C., Zeng, J., Pan, J., Xue, S., Jin, Y.: A new fitness estimation strategy for particle swarm optimization. Inf. Sci. 221, 355–370 (2013)CrossRefMathSciNetMATHGoogle Scholar
  5. 5.
    Imran, M., Hashima, R., Khalid, N.E.A.: An overview of particle swarm optimization variants. Procedia Eng. 53, 491–496 (2013)CrossRefGoogle Scholar
  6. 6.
    Pan, F., Li, X.T., Zhou, Q., Li, W.X., Gao, Q.: Analysis of standard particle swarm optimization algorithm based on Markov chain. Acta Automatica Sinica 39(4), 381–389 (2013)CrossRefMathSciNetMATHGoogle Scholar
  7. 7.
    Babaei, M.: A general approach to approximate solutions of nonlinear differential equations using particle swarm optimization. Appl. Soft Comput. 13, 3354–3365 (2013)CrossRefGoogle Scholar
  8. 8.
    Khare, A., Rangnekar, S.: A review of particle swarm optimization and its applications in solar photovoltaic system. Appl. Soft Comput. 13, 2997–3006 (2013)CrossRefGoogle Scholar
  9. 9.
    Wang, H., Sun, H., Li, C., Rahnamayan, S., Pan, J.S.: Diversity enhanced particle swarm optimization with neighborhood search. Inf. Sci. 223, 119–135 (2013)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Neri, F., Mininno, E., Iacca, G.: Compact particle swarm optimization. Inf. Sci. 239, 96–121 (2013)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Valdez, F., Melin, P., Castillo, O.: Modular neural networks architecture optimization with a new nature inspired method using a fuzzy combination of particle swarm optimization and genetic algorithms. Inf. Sci. 270, 143–153 (2014)CrossRefGoogle Scholar
  12. 12.
    Shin, Y., Ghosh, J.: The pi-sigma networks : an efficient higher order neural network for pattern classification and function approximation. In: Proceedings of International Joint Conference on Neural Networks, vol. 1, pp. 13–18. Seattle, Washington, (July 1991)Google Scholar
  13. 13.
    Liu,Y., Zhang, H., Yang, J., Wei, W.: Convergence of online gradient methods for pi-sigma neural network with a penalty term. In: IEEE International Conference on Anthology, pp. 1–4, (2013). doi: 10.1109/ANTHOLOGY.2013.6784769
  14. 14.
    Yu, X., Tanga, L., Chena, Q., Xub, C.: Monotonicity and convergence of asynchronous update gradient method for ridge polynomial neural network. Neurocomputing. 129, 437–444 (2014)Google Scholar
  15. 15.
    Yu, X., Deng, F.: Convergence of gradient method for training ridge polynomial neural network. Neural Comput. Appl. 22(1), 333–339 (2013)CrossRefGoogle Scholar
  16. 16.
    Deng, Y.Q., Song, G.: A verifiable visual cryptography scheme using neural networks. Adv. Mater. Res. 756–759, 1361–1365 (2013)CrossRefGoogle Scholar
  17. 17.
    Morissette, L., Chartier, S.: FEBAMSOM-BAM: Neural network model of human categorization of the N-bits parity problem. The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–5 (2013)Google Scholar
  18. 18.
    Lin, Qing, Cai, ZhiHao, Wang, Ying, Xun,Yang, JinPeng,Chen, LiFang: Adaptive Flight Control Design for Quadrotor UAV Based on Dynamic Inversion and Neural Networks. Third IEEE International Conference on Instrumentation, Measurement, Computer, Communication and Control (IMCCC), pp. 1461–1466 (2013)Google Scholar
  19. 19.
    Lee, B.Y., Lee, H.I., Tahk, M.J.: Analysis of adaptive control using on-line neural networks for a quadrotor UAV. 13th IEEE International Conference on Control. Automation and Systems (ICCAS), pp. 1840–1844 (2013)Google Scholar
  20. 20.
    Huang, D.S., Ip, H.H.S., Law, K.C.K.: Chi: Zeroing polynomials using modified constrained neural network approach. IEEE Trans. Neural Netw. 16(3), 721–732 (2005)CrossRefGoogle Scholar
  21. 21.
    Perantonis, S., Ampazis, N., Varoufakis, S., Antoniou, G.: Constrained learning in neural networks: application to stable factorization of 2nd polynomials. Neural Process. Lett. 7(1), 5–14 (1998)CrossRefGoogle Scholar
  22. 22.
    Holland, J.H.: Adaption in Natural and Artificial Systems. MIT Press, Cambridge (1975)MATHGoogle Scholar
  23. 23.
    Goldberg, D.E.: Genetic Algorithms In Search. Optimization and machine learning. Kluwer Academic Publishers, Boston (1989)MATHGoogle Scholar
  24. 24.
    Man, K.F., Tang, K.S., Kwong, S.: Genetic Algorithms: Concepts and Designs. Springer, Berlin (1999)CrossRefMATHGoogle Scholar
  25. 25.
    Chen, C.H., Khoo, L.P., Chong, Y.T., Yin, X.F.: Knowledge discovery using genetic algorithm for maritime situational awareness. Expert Syst. Appl. 41, 2742–2753 (2014)CrossRefGoogle Scholar
  26. 26.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Networks. vol. 4, 1942–1948 (1995)Google Scholar
  27. 27.
    Wei, J., Guangbin, L., Dong, L.: Elite particle swarm optimizaion with mutation. IEEE Asia Simulation Conference—7th Intl Computing Conference on System. Simulation and Scientific, pp. 800–803 (2008)Google Scholar
  28. 28.
    Khare, A., Rangnekar, S.: A review of particle swarm optimization and its applications in solar photovoltaic system. Appl. Soft Comput. 13, 2997–3006 (2013)CrossRefGoogle Scholar
  29. 29.
    Babaei, M.: A general approach to approximate solutions of nonlinear differential equations using particle swarm optimization. Appl. Soft Comput. 13, 3354–3365 (2013)CrossRefGoogle Scholar
  30. 30.
    Sivanandam, S.N., Deepa, S.N.: Introduction to Genetic Algorithms. Springer, Heidelberg (2008)MATHGoogle Scholar
  31. 31.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(9), 533–536 (1986)CrossRefGoogle Scholar
  32. 32.
    Bache, K., Lichman, M.: UCI machine learning repository. Irvine, CA: University of California, School of Information and Computer Science (2013)
  33. 33.
    Alcalá-Fdez, J., Fernandez, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Multiple-Valued Logic Soft Computing. 17(2–3), 255–287 (2011)Google Scholar
  34. 34.
    Fisher, R.A.: Statistical methods and scientific inference, 2nd edn. Hafner Publishing Co., New York (1959)Google Scholar
  35. 35.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Maching Learn. Res. 7, 1–30 (2006)MathSciNetMATHGoogle Scholar

Copyright information

© Springer India 2015

Authors and Affiliations

  • Janmenjoy Nayak
    • 1
  • Bighnaraj Naik
    • 1
  • H. S. Behera
    • 1
  • Ajith Abraham
    • 2
    • 3
  1. 1.Department of Computer Science Engineering and Information TechnologyVeer Surendra Sai University of TechnologySambalpurIndia
  2. 2.Machine Intelligence Research Labs (MIR Labs)WashingtonUSA
  3. 3.IT4 Innovations—Center of ExcellenceVSB—Technical University of OstravaOstravaCzech Republic

Personalised recommendations