Abstract
Neural network (NN) classifiers are very popular tools for solving classification tasks. Mostly known NN classifier is a multilayer perceptron (MLP). Although MLP has a good correct classification ratio, its structure could be very complex and network training may work for a long time. Pi-sigma NN (PSNN) is higher-order NN (HONN), which used higher-order correlations among the input components to establish a HONN, and the PSNN utilizes the product of neurons as the output units. By contrast with MLP and PSNN, single multiplicative neuron (SMN) is simple concerning its structure and mathematical model. The absence of the hidden layer(s) could be an advantage for easy implementation, and the mathematical model can be easily interpreted. In this paper, we propose a new hybrid NN classifier based on simple adaptive neurons and SMN which form the SMN as a whole, where input units are constituted by adaptive neurons. In contrast with conventional NN, our proposed classifier can use fewer learning parameters. To train this network, we use a modified particle swarm optimization (MPSO) algorithm. For the investigation of the generalization capability of the proposed classifier, we compare this method to other NN classifiers: MLP and PSNN together with other classification procedure classifiers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akoglu H (2018) User’s guide to correlation coefficients. Turk J Emerg Med 18:91–93. https://doi.org/10.1016/j.tjem.2018.08.001
Bishop CM (2006) Pattern recognition and machine learning. Academic Press, New York
Blake CL, Merz CJ (1998) UCI Repository of machine learning databases. University of California. http://archive.ics.uci.edu/ml/
Chuang C (2011) Huang S A hybrid neural network approach for credit scoring. Exp Syst 28:185–196. https://doi.org/10.1111/j.1468-0394.2010.00565.x
Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the MHS’95 sixth ınternational symposium micro machine human science, pp 39–43. doi:https://doi.org/10.1109/MHS.1995.494215.
Engelbrecht AP (2001) Cooperative learning in neural networks using particle swarm optimizers. Annu Res Conf South Afr Inst Comput Sci Inf Technol, pp 84–90
Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell. https://doi.org/10.1007/s10489-016-0767-1
Ghazali R (2009) Application of pi-sigma neural networks and ridge polynomial neural networks to financial time series prediction, 271–273.
Ghosh J, Shin Y (1995) Ecient higher-order neural networks for classification and function approximation. Int J Neural Syst 03:323–350
Hagan MT, Menhaj MB (1994) Training feedforward networks with the marquardt algorithm. IEEE Trans Neural Netw 5:989–993. https://doi.org/10.1109/72.329697
Hakan C, Erol A (2013) Robust multilayer neural network based on median neuron model. Neural Comput Appl. https://doi.org/10.1007/s00521-012-1315-5
Hatamlou A (2017) A hybrid bio-inspired algorithm and its application. Appl Intel. https://doi.org/10.1007/s10489-017-0951-y
Husaını NA, Ghazalı R, Nawı NM, Ismaıl LH (2012) The effect of network parameters on pi-sigma neural network for temperature forecasting. Int J Mod Phys Conf Ser 09:440–447. https://doi.org/10.1142/S2010194512005521
Hussain AJ, Liatsis P (2003) Recurrent pi-sigma networks for DPCM image coding. Neurocomputing 55:363–382. https://doi.org/10.1016/S0925-2312(02)00629-X
Ilonen J, Kamarainen J-K, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17:93–105. https://doi.org/10.1023/A:1022995128597
Jia W, Zhao D, Shen T, Ding S, Zhao Y, Hu C (2015) An optimized classification algorithm by BP neural network based on PLS and HCA. Appl Intel. https://doi.org/10.1007/s10489-014-0618-x
Juang C-F (2004) A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. IEEE Trans Syst Man Cybern B Cybern 34:997–1006. https://doi.org/10.1109/TSMCB.2003.818557
Karaboga D, Akay B, Ozturk C (2007) Artificial bee colony (ABC) optimization algorithm for training feed-forward neural. Int Conf Model Decis Artif Intel 4617:318–329
Khan K, Sahai A (2012) A comparison of BA, GA, PSO, BP and LM for training feed forward neural networks in e-learning context. Int J Intell Syst Appl 4(2012):23–29. https://doi.org/10.5815/ijisa.2012.07.03
Kolay E, Tunç T, Eğrioğlu E (2016) Classification with some artificial neural network classifiers trained a modified particle swarm optimization. Am J Intel 6:59–65. https://doi.org/10.5923/j.ajis.20160603.01
Li CK (2003) A sigma-pi-sigma neural network (SPSNN). Neural Process Lett 17:1–19. https://doi.org/10.1023/A:1022967523886
Li H (2009) Particle swarm optimization algorithm with exponent decreasing inertia weight and stochastic mutation, pp 66–69. doi:https://doi.org/10.1109/ICIC.2009.24.
Lim C, Leong J, Kuan M (2005) A hybrid neural network system for pattern classification tasks with missing features. IEEE Trans Pattern Anal Mach Intel 27:648–653
Marinakis Y, Marinaki M, Doumpos M, Zopounidis C (2009) Ant colony and particle swarm optimization for financial classification problems. Exp Syst Appl 36:10604–10611. https://doi.org/10.1016/j.eswa.2009.02.055
Mendes R, Cortez P, Rocha M, Neves J (2002) Particle swarms for feedforward neural network training. In: Proceedings of the 2002 International Joint Conference Neural Networks IJCNN02 Cat No02CH37290 6: 1895–1899. doi:https://doi.org/10.1109/IJCNN.2002.1007808.
Mirjalili S (2015) How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl Intell 43:150–161. https://doi.org/10.1007/s10489-014-0645-7
Montana DJ, Davis L (1989) Training feedforward neural networks using genetic algorithms. 762–767.
Nayak J, Naik B, Behera HS (2014) A hybrid PSO-GA based Pi sigma neural network (PSNN) with standard back propagation gradient descent learning for classification. In: Proceedings of the 2014 International Conference on Controlled Instrumentation, Communication Computer Technology ICCICCT 2014. pp 878–885. doi:https://doi.org/10.1109/ICCICCT.2014.6993082.
Nie Y (2008) A hybrid genetic learning algorithm for Pi-sigma neural network and the analysis of its convergence. pp 19–23. doi:https://doi.org/10.1109/ICNC.2008.896.
Nienhuis B (1992) Improving the convergence of the back-propagation algorithm. Neural Netw 5:465–471
Panigrahi S, Bhoi AK, Karali Y (2013) A modified differential evolution algorithm trained pi-sigma neural network for pattern classification. Int J Soft Comput Eng 3:133–136
Ratnaweera A, Halgamuge SK, Watson HC (2004) Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evol Comput 8:240–255. https://doi.org/10.1109/TEVC.2004.826071
Riedmiller M (1993) A direct adaptive method for faster backpropagation learning: the RPROP algorithm. IEEE Int Conf Neural Netw
Rumelhart DE, Hinton GE, Williams RJ (1988) Learning ınternal representations by error propagation. Read Cogn Sci. https://doi.org/10.1016/B978-1-4832-1446-7.50035-2
Shekara S, Reddy S, Wang L, Devabhaktuni V, Nelapati P (2011) A hybrid neural network model and encoding technique for enhanced classification of energy consumption data. pp 1–8.
Shin Y, Ghosh J (1991) The pi-sigma network: an efficient higher-order neural network for\npattern classification and function approximation. IJCNN-91-Seattle Int Jt Conf Neural Netw. doi:https://doi.org/10.1109/IJCNN.1991.155142.
Tunc T (2012) A new hybrid method logistic regression and feedforward neural network for lung cancer data. Math Problem Eng. https://doi.org/10.1155/2012/241690
Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural Comput 8:1341–1390. https://doi.org/10.1162/neco.1996.8.7.1341
Yadav RN (2003) Classification using single neuron. doi:https://doi.org/10.1109/INDIN.2003.1300258
Yadav RN, Kalra PK, John J (2007) Time series prediction with single multiplicative neuron model. Appl Soft Comput 7:1157–1163. https://doi.org/10.1016/j.asoc.2006.01.003
Yaghini M, Khoshraftar MM, Fallahi M (2012) Engineering applications of artificial ıntelligence a hybrid algorithm for artificial neural network training. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2012.01.023
Yolcu U, Egrioglu E, Aladag CH (2013) A new linear and nonlinear artificial neural network model for time series forecasting. Decis Support Syst 54:1340–1347. https://doi.org/10.1016/j.dss.2012.12.006
Zhang J, Zhang J, Lok T, Lyu MR (2007) A hybrid particle swarm optimization: back-propagation algorithm for feedforward neural network training. Appl Math Comput 185:1026–1037. https://doi.org/10.1016/j.amc.2006.07.025
Zhao L, Yang Y (2009) PSO-based single multiplicative neuron model for time series prediction. Exp Syst Appl 36:2805–2812. https://doi.org/10.1016/j.eswa.2008.01.061
Author information
Authors and Affiliations
Contributions
Taner Tunç conceived of the presented idea. Erdinç Kolay performed the computation. Both Taner Tunç ve Erdinç Kolay authors contributed to the final version of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Communicated by Mu-Yen Chen.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kolay, E., Tunç, T. A new hybrid neural network classifier based on adaptive neuron and multiplicative neuron. Soft Comput 27, 1797–1808 (2023). https://doi.org/10.1007/s00500-021-06093-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-021-06093-6