Preferred design of recurrent neural network architecture using a multiobjective evolutionary algorithm with un-supervised information recruitment: a paradigm for modeling shape memory alloy actuators

Abstract

Shape memory alloys (SMAs) are able to compensate any undergoing plastic deformations and return to their memorized shape. Such a behavior persuades industrialists to use them for different engineering applications, as smart actuators and sensors. Because of their vast applications, it is crucial to engineers to develop effective identification tools capable of simulating the behavior of SMAs. However, SMA actuators have complex and hysteric behavior that in turn obstructs the modeling process. The motivation behind the current study emanates in the pursuit of developing efficient prediction tools for effective modeling of SMA actuators. Actually, after several experiments and software simulations, the authors develop a hybrid intelligent tool which takes advantage of the self-organizing Pareto based evolutionary algorithm (SOPEA) and simultaneous recurrent neural network (SRNN), as a black-box model, to automatically identify the behavior of SMA. SOPEA is a multiobjective evolutionary algorithm which is based on the concepts of survival of the fittest, non-dominated sorting and information recruitment. The information recruitment is guaranteed by applying an un-supervised neuro computing technique, i.e. adaptive self organizing map (ASOM) with conscience mechanism. ASOM is an un-supervised network that assists SOPEA to recognize the non-dominated patterns and produce further non-dominated solutions. Together with the structure of SOPEA, the authors follow a comprehensive preference-based strategy to exploit the desired regions in the Pareto front. This occurs through introducing deliberate reference points. The outcome method is applied to the design of SRNN for modeling the SMA actuator. It is demonstrated that the designed optimization tool can show acceptable performance for the present case study within the imposed computational budget. Besides, through a rigorous experimental procedure, it is indicated that by applying an efficient artificial system, the behavior of SMA can be identified without any specific knowledge of the physical conditions and governing equations.

Appendix: Self organizing Pareto based evolutionary algorithm

In this section, the authors provide a succinct description of the main operators of the SOPEA algorithm. These operators can be divided into evolutionary operators (i.e. recombination, mutation, selection and the interaction of artificial bees) [20], an SOM operator, an external archive and sharing factor.

1.1 Adaptive SOM operator

An adaptive SOM with a gradient learning rule and concise mechanism is used to learn the characteristics of non-dominated solutions. Considering the non-dominated solutions as the input data, the SOM topology can emulate the non-dominated characteristics and augment the intensification of the Pareto front. The adaptive SOM proposed in [49] is an unsupervised neural network that automatically adapts the learning rate and neighborhood function of the neuron weights. One of the major applications of SOM is to minimize all of the distances between any input vector (non-dominated solutions in our case study) and the synaptic neuron weight vector. This is done by providing a suitable topological ordering for the input distribution. Figure 23 shows a schematic illustration of a typical SOM topology.

Fig. 23

A typical SOM with 3 × 3 hexagonal topology

The SOM network with concise mechanism uses the following updating rule:

$$ w_{j}^{n + 1} (t) = w_{j}^{n} (t) + y_{j} (t).h_{j} (n).\left( {R_{i}^{n} (t) - w_{j}^{n} (t)} \right);\quad t = 1,2, \ldots ,T $$

(26)

where t is the sub-generation of SOM network and n represents the generation number of SOPEA. y _j (t)is a controlling parameter that leads the weight vectors toward the non-dominated solutions. In other words, if the fitness of input value f _R , which is a non-dominated solution, is lower than $f_{{W_{j} (n)}}$ then y _j (t) = 1, and consequently, the neuron center moves toward the non-dominated solution (the input of network); otherwise y _j (t) = 0 and the neuron center do not approach the solution. The mathematical expression of the abovementioned descriptions can be given as:

$$ y_{j} (t) = \left\{ \begin{aligned} &1\quad if\quad R(t)\;do\hbox{min} ates\;w_{j} (t) \\ & 0\quad otherwise \\ \end{aligned} \right. $$

(27)

where w ⁿ⁺¹ _j (t) refers to the updated weight vector and w _j (t) is the old weight vector. ‖R ⁿ _i  − w ⁿ _j ‖ represents the distance between the input vectors where R ⁿ _i is the i _th non-dominate solution in the n _th generation. The learning rate which is a descending function is defined as follows:

$$ h{}_{j}(t + 1) = h_{j} (t) + \alpha \left( {f\left( {\frac{1}{{s_{f} .sl(t)}}\left\| {R_{i}^{n} - w_{j}^{n} } \right\|} \right)} \right) $$

(28)

The learning rate parameter h _j (0) should be initialized with a value close to unity. α can take any arbitrary value between 0 and 1. s _f is a descending constant and should be set based on the condition of problem. In this paper, s _f is set to be 1,000. The function f(.) should be designed in a fashion that the following criterion is satisfied appropriately:

$$ \left\{ \begin{array}{l} f(0) = 0 \\ \frac{df(z)}{dz} \ge 0;\quad if\quad z > 0 \\ 0 \le f(z) \le 1 \\ \end{array} \right. $$

(29)

f(z) can be set as:

$$ f(z) = 1 - \frac{1}{1 + z} $$

(30)

The scaling value sl for an n dimensional input is adjusted using the following equations:

$$ sl(t + 1) = \sqrt {\left( {\sum\limits_{i = 1}^{n} {E_{k}^{i} (t + 1)^{n - i} ( - 1)^{i + 1} } } \right)^{ + } } ,\quad k = 1 $$

(31)

$$ E_{k}^{i} (t + 1) = E_{k}^{i} (t) + \mu_{i} \left( {R_{k}^{i} (t) - E_{k}^{i} (t)} \right) $$

(32)

where i represents the number of variable in each solution. $E_{k}^{i} \left( 0 \right)$ is initialized with some small random values. Figure 24a depicts the concept of neurons updating mechanism in the SOM grid.

Fig. 24

a A gradual movement of the neurons in a defined topological neighborhood of data center (the non-dominated solution), b A schematic of concise mechanism

Concise mechanism is applied to revive the dead units (weights) in SOM neuron center. A schematic illustration of the procedure is visualized in Figure 24b. The dead unit is a term that refers to the weights with trivial chance of learning and adaption during the progress. Reviving of these units is often called concise mechanism. Here, a simple mechanism is used for tuning of the bias of nodes (neuron):

$$ b_{i} (t + 1) = \left\{ \begin{gathered} 0.8b_{i} (t) \hfill \\ b_{i} (t) - 0.3 \hfill \\ \end{gathered} \right. $$

(33)

1.2 External archive

An external archive is used to collect all non-dominated solutions. The main provocation of devising an external archive is to save a historical record of the non-dominated solutions detected over an optimization procedure. The archive interacts with the solutions of SOPEA in a real-time fashion to produce additional non-dominated solutions and eliminate the inferior solutions archived previously. A solution can be saved in the external archive if it complies any of the following criteria:

• 1. The candidate solution dominates any existing solution in the archive,

• 2. There is no stored solution in the archive,

• 3. The archive is full, but the candidate solution is non-dominated and enhances the diversity of the Pareto front significantly.

The prominent asset of using an external archive lies in a reduction of the computational complexity, especially, when the algorithm captures a higher number of the non-dominated solutions. This is because its task is just retaining the obtained non-dominated solutions. Furthermore, using an external archive provides some elitism by preserving non-dominated solutions. The feedback of applying such a policy illustrates that the speed of algorithm does not decline significantly when working with a higher number of the non-dominated solutions.

1.3 Sharing factor

Sharing factor (ξ) is a threshold value that determines the amount of solutions that should be inserted into each of the optimizing operators (i.e. bee and evolutionary operators). In this regard, SOPEA can be considered as ensemble of two co-evolutionary operators in which the sharing factor is ‘trigger of ensemble’. The value of sharing factor (ξ) is confined within the range of unity [0, 1]. Due to the characteristics of an optimization problem, it can be either constant or self-adaptive. The mathematical formulation of sharing process is implemented as:

$$ Solution{\text{'}}s\;Sharing: \left\{ {\begin{array}{*{20}l} {N_{{P_{evolutionary} }} = \left[ { \zeta \times P_{s } } \right]} \\ {N_{{P_{bee - inspired} }} = P_{s} - N_{{P_{evolutionary} }} } \\ \end{array} } \right. $$

(34)

where $N_{{P_{evolutionary} }}$ represents the matrix of solutions allocated to the evolutionary phase,$N_{{P_{bee - inspired} }}$ represents the number of solutions in the bee-inspired phase, $P_{s }$ is the number SOPEA solutions and ξ is the sharing factor. As it can be inferred, the proposed formulation is a strategy for shuffling all of the solutions between two evolutionary phases.