Implementation of Metaheuristics with Extreme Learning Machines

Abstract

This chapter approaches the topic of Extreme Learning Machines (ELM); this technique is implemented in a single layer feedforward neural network and provides to the neural network with a fast performance since it is not necessary to adjust weights and hidden layer biases. ELM is implemented and combined with three metaheuristic algorithms. The selected algorithms belong to the family of Swarm intelligence such as Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC) and Grey Wolf Optimization (GWO); these algorithms are applied to optimize the hidden biases and input weights and are implemented for improving the performance along with techniques of Extreme learning machines in data classification. Also, the performance and behavior of each algorithm can be observed. The data employed in this work consist of information about Australian credit, diabetes detection and hearth disease. The investigation result show that GWO obtained better results in the first dataset and PSO are better in the second and third datasets.

Appendix

Below is show an example of the implementation of Extreme Learning machines algorithm for a single layer feedforward neural network in MATLAB, focused in the training and testing parts. In the first part of the code, the testing part will be seen, first of all the input weights and the biases of the hidden neurons are randomly generated, those values are stored in two variables, then the input weights are multiplied with the training data, furthermore the biases are added. The output of the hidden neuron is calculated selecting one activation function, then the output weights are estimated using the Moore-Penrose generalize inverse. In the case of H does not have inverse. Finally, is calculated the current output from the training data multiplying the pseudoinverse of H and the output weights, also the error and the accuracy of the training phase are obtained if it is a regression process, the accuracy is calculated implementing the mean square error function of MATLAB.

Code 1

MATLAB code of the training process of Extreme Learning Machines

• 1 InputWeight=rand(NumberofHiddenNeurons,NumberofInputNeurons)*2−1;

• 2 BiasofHiddenNeurons=rand(NumberofHiddenNeurons,1);

• 3 tempH=InputWeight*P;

• 4 clear P;

• 5 ind=ones(1,NumberofTrainingData);

• 6 BiasMatrix=BiasofHiddenNeurons(:,ind);

• 7 tempH=tempH+BiasMatrix;

• 8 switch lower(ActivationFunction)

• 9 case {‘sig’,’sigmoid’}

• 10 H = 1 ./ (1 + exp(-tempH));

• 11 case {‘sin’,’sine’}

• 12 H = sin(tempH);

• 13 case {‘hardlim’}

• 14 H = double(hardlim(tempH));

• 15 case {‘tribas’}

• 16 H = tribas(tempH);

• 17 case {‘radbas’}

• 18 H = radbas(tempH);

• 19 end

• 20 clear tempH;

• 21 OutputWeight=pinv(H’) * T’;

• 22 Y=(H’ * OutputWeight)’;

• 23 error = T−Y;

• 24 if Elm_Type == REGRESSION

• 25 TrainingAccuracy=sqrt(mse(T − Y));

• 26 End

• 27 clear H;

The testing part is almost the same as the training part. In this phase, the output is verified, in the top of the code the input weights and biases of the hidden layer are randomly initialized, the biases are added to the product between the input weights and the testing input data, one of the available activation functions is chosen to obtain the final output, if is a regression problem the accuracy is obtained through RMSE, in the contrary case the code where the training and testing data is corroborated by the output given by the neural network. The training and testing accuracy are calculated by 1 minus the percentage of the error in order to normalize the accuracy.