A multi-agent optimization algorithm and its application to training multilayer perceptron models

Abstract

The optimal parameter values in a feed-forward neural network model play an important role in determining the efficiency and significance of the trained model. In this paper, we propose an upgraded artificial electric field algorithm (AEFA) for training feed-forward neural network models. This paper also throws some light on the effective use of multi-agent meta-heuristic techniques for the training of neural network models and their future prospects. Seven real-life data sets are used to train neural network models, the results of these trained models show that the proposed scheme performs well in comparison to other training algorithms in terms of high classification accuracy and minimum test error values including gradient-based algorithms and differential evolution variants. Some fundamental modifications in AEFA are also proposed to make it more suitable for training neural networks. All the experimental findings show that the search capabilities and convergence rate of the proposed scheme are better than those of other capable schemes, including gradient-based schemes.

Appendix

Function 1

$$\begin{aligned} aK=&[.1957, .1947, .1735, .16, .0844, .0627, .0456, .0342, .0323, .0235, .0246]&\\ bK=&[.25, .5, 1, 2, 4, 6, 8, 10, 12, 14, 16]&\\bK1=&1./bK&\\ F1=&sum((aK-((\zeta _{1}(bK1^2+\zeta _{2}bK1))/(bK1^2+\zeta _{3}bK1+\zeta _{4})))^2); \end{aligned}$$

The range of this function is $[-5, 5]$ with dimension 4.

Function 2

$$\begin{aligned} F2=4\zeta _{1}^2-2.1\zeta _{1}^4+\zeta _{1}^6/3+\zeta _{1}\zeta _{2}-4\zeta _{2}^2+4\zeta _{2}^4 \end{aligned}$$

The range of this function is $[-5, 5]$ with dimension 2.

Function 3

$$\begin{aligned} F3=& \left(1+(\zeta _{1}+\zeta _{2}+1)^2(19-14\zeta _{1}+3\zeta _{1}^2-14\zeta _{2}+6\zeta _{1}\zeta _{2}+3\zeta _{2}^2)\right)\\ {}&\left(30+(2\zeta _{1}-3\zeta _{2})^2(18-32\zeta _{1}+12\zeta _{1}^2+48\zeta _{2}-36\zeta _{1}\zeta _{2}+27\zeta _{2}^2)\right) \end{aligned}$$

The range of this function is $[-2, 2]$ with dimension 2.

Function 4

$$\begin{aligned} aH=&[3, 10, 30;.1, 10, 35;3, 10, 30;.1, 10, 35];cH=[1, 1.2, 3, 3.2]\\ pH=&[.3689, .117, .2673;.4699, .4387, .747;.1091, .8732, .5547;.03815, .5743, .8828]\\ F4=&F4-cH(i)exp(-(sum(aH(i,:)((\zeta -pH(i,:))^2)))), i\in \{1,2,3,4\} \end{aligned}$$

The range of this function is [0, 1] with dimension 3.

Function 5

$$\begin{aligned} aH=&[10, 3, 17, 3.5, 1.7, 8;.05, 10, 17, .1, 8, 14;3, 3.5, 1.7, 10, 17, 8;17, 8, .05, 10, .1, 14]\\ cH=&[1, 1.2, 3, 3.2]\\ pH=&[.1312, .1696, .5569, .0124, .8283, .5886;.2329, .4135, .8307, .3736, .1004, .9991;.2348, .1415, .3522, .2883,\\ {}&.3047, .6650;.4047, .8828, .8732, .5743, .1091, .0381]\\ F5=&F5-cH(i)*exp(-(sum(aH(i,:).*((\zeta -pH(i,:))^2)))), \quad i\in \{1,2,3,4\} \end{aligned}$$

The range of this function is [0, 1] with dimension 6.

$$\begin{aligned}{} & {} aSH=[4, 4, 4, 4;1, 1, 1, 1;8, 8, 8, 8;6, 6, 6, 6;3, 7, 3, 7;2, 9, 2, 9;5, 5, 3, 3;8, 1, 8, 1;6, 2, 6, 2;7, 3.6, 7, 3.6]\\{} & {} cSH=[.1,.2,.2,.4,.4,.6,.3,.7,.5,.5]\\ \end{aligned}$$

Function 6

$$\begin{aligned} F6=F6-((\zeta -aSH(i,:))(\zeta -aSH(i,:))'+cSH(i))^{-1},\quad i\in \{1,2,3,4,5\} \end{aligned}$$

The range of this function is [0, 1] with dimension 4.

Function 7

$$\begin{aligned} F7=F7-((\zeta -aSH(i,:))(\zeta -aSH(i,:))'+cSH(i))^{-1}, \quad i\in \{1,2,3,4,5,6,7\} \end{aligned}$$

The range of this function is [0, 1] with dimension 4.

Function 8

$$\begin{aligned} F8=F8-((\zeta -aSH(i,:))(\zeta -aSH(i,:))'+cSH(i))^{-1},\quad i\in \{1,2,3,4,5,6,7,8,9,10\} \end{aligned}$$

The range of this function is [0, 1] with dimension 4.