Enhanced Tunicate Swarm Algorithm for Solving Large-Scale Nonlinear Optimization Problems

Nowadays optimization problems become difficult and complex, traditional methods become inefficient to reach global optimal solutions. Meanwhile, a huge number of meta-heuristic algorithms have been suggested to overcome the shortcomings of traditional methods. Tunicate Swarm Algorithm (TSA) is a new biologically inspired meta-heuristic optimization algorithm which mimics jet propulsion and swarm intelligence during the searching for a food source. In this paper, we suggested an enhancement to TSA, named Enhanced Tunicate Swarm Algorithm (ETSA), based on a novel searching strategy to improve the exploration and exploitation abilities. The proposed ETSA is applied to 20 unimodal, multimodal and fixed dimensional benchmark test functions and compared with other algorithms. The statistical measures, error analysis and the Wilcoxon test have affirmed the robustness and effectiveness of the ETSA. Furthermore, the scalability of the ETSA is confirmed using high dimensions and results exhibited that the ETSA is least affected by increasing the dimensions. Additionally, the CPU time of the proposed algorithms are obtained, the ETSA provides less CPU time than the others for most functions. Finally, the proposed algorithm is applied at one of the important electrical applications, Economic Dispatch Problem, and the results affirmed its applicability to deal with practical optimization tasks.


Introduction
For almost all human problems there is a desire to reach the most with the least, for example, maximizing the strength and efficiency of industrial equipment with minimum cost. The optimization has a great role in human problems which achieve the optimum (maximum or minimum). Optimization is considered the process of obtaining the optimum results under available conditions. Optimization problems become complex with a huge number of decision variables and constraints. There is no definite method available for all different problems so different methods have been suggested to solve these problems. The classical methods of optimization (direct search methods) such as Newton methods, the steepest method and nonlinear programming are efficient in finding the optimum solution of continuous and differentiable functions only depending on reference starting point, gradient information and failed to deliver the optimum without converging to some local optimum. These problems made the researchers develop new methods of optimization. Deterministic, stochastic and hybrid algorithms are three categories of optimization algorithms. Deterministic algorithms follow the input data and parameters values and produce the same result such as Hill-climbing and downhill simplex algorithms. Stochastic algorithms or meta-heuristics algorithms are introduced based on randomness rules while the hybrid algorithms represent a mixed system among the deterministic and/or stochastic algorithms. For instance, a hybrid PSO-GA algorithm was proposed to deal with the constrained optimization problems [1], where the PSO has been used to explore the search regions while GA has been employed to improve the search direction. A hybrid GA-GSA algorithm for optimizing the performance of an industrial system by utilizing uncertain data was introduced [2], where its ability is proved through maximizing the reliability and maintainability in solving a nonlinear optimization problem. A novel TVAC-PSO based mutation strategies algorithm for generation scheduling of pumped storage hydrothermal system incorporating solar units [3] is presented to analyze the impact of renewable energy sources on the distribution of optimal power generation. Meta-heuristic algorithms are considered stochastic which are used with different functions such as discontinuous, non-differentiable, non-smooth functions. The meta-heuristic algorithms have been suggested to avoid the demerits of the classical methods. They follow an iterative generation process to find a high efficient solution near to the optimum and they can reach the promising areas, escape from local optima and solving different and complex problems in a few times [4]. Unless the classical methods, they do not have one initial point and do not depend on the gradient information. According to these merits [5], huge number of meta-heuristics optimization algorithms are proposed over the last few decades [6]. Nature can be represented as the most important inspiration of metaheuristics. Four categories of biologically inspired metaheuristics are discussed as evolution-based, swarm-based, physics-based and human-based algorithms [7]. The biologically algorithms mimic nature evolution such as Biogeography Based Optimizer (BBO) [8] that is established based on biogeography concept for solving optimization problems, Evolutionary Programming (EP) [9] has been introduced to improve the finite-state automata to solve time series forecasting problems and improve learning machines to solve continuous optimization problems, Genetic Algorithm (GA) [10] has been proposed based on the biological improvement for solving constrained and unconstrained problems through some operators including crossover, recombination, mutation and the selection strategy, Evolution Strategies (ES) [11] belongs to the evolutionary algorithms that is developed based on the ideas of the improvement and it is applied to deal with some fields of optimization, and Differential Evolution (DE) [12] is a population-based algorithm for improving the solution using the evolutionary process. Swarm-based algorithms mimic the behavior of living organisms to find the optimum food source such as Particle Swarm Optimization (PSO) [13] that is inspired by the behavior of birds or fish. It is affirmed its ability in dealing with comprehensive optimization fields, however, it suffers from trapping in local optima and premature convergence while solving complex natures, Ant Colony Optimization (ACO) [14] was proposed based on foraging behavior of ant colonies to solve difficult discrete optimization problems, Firefly Algorithm (FFA) [15] was presented by Xin-She Yang based on flashing behavior of fireflies to attract prey or mates. Its efficacy was demonstrated through a large number of test problems, Whale Optimization Algorithm (WOA) [16] was inspired by the bubble-net hunting to imitate the behavior of humpback whales, where it is benchmarked on 29 well-known test functions and 6 design problems, Ant Lion Optimizer (ALO) [17] was proposed by Seyedali Mirjalili which imitates the hunting behavior of antlions and it is benchmarked on 19 mathematical functions, 3 engineering problems and two constrained problems, Gray Wolf Optimizer(GWO) [18] mimics the hunting mechanism of grey wolves in nature. It is proposed to deal with optimization tasks including 29 well-known functions and some engineering design problems to confirm its effective performance, Xin-She Yang and Suash Deb developed a nonlinear algorithm called as Cuckoo Search (CS) [19] which is introduced based on the obligate brood parasitism of cuckoo species, the attacking and migration behavior of a seagull in nature are the main inspiration of the Seagull Optimization (SO) [20] that was proposed for solving expensive problems. It was competitive when applied on 44 test functions and 7 large-scale constrained optimization problems, and Moth Flame Optimizer (MFO) [21] is a novel optimization paradigm. The navigation behavior of moths is the main inspiration of this technique which is called transverse direction. It was benchmarked on 29 test functions and 7 engineering problems. Physics-based algorithms are developed based on laws and theories of physics such as Gravitational Search (GSA) [22] was proposed by E. Rashedi et al. based on the gravity law and mass interactions. The performance of GSA is evaluated through 23 test functions. Seyedali Mirjalili proposed a new methodology based on the mathematical model of sine and cosine called Sin Cosine Algorithm (SCA) [23] and applied it on set of test functions and on the cross section of the wing of an aircraft. Human-based algorithms mimic the social behaviors of human such as Exchange Market Algorithm (EMA) [24] that was proposed by N. Ghorbani et al. based on trading the shares on stock market for solving nonlinear continuous problems. It has been applied to 12 test functions with different dimensions. Bus Transportation Algorithm (BTA) [25] was proposed based on the social behavior of humans in transportation and it is applied to solve NP-complete problem. However, the above-mentioned algorithms may suffer from balancing the exploitation and exploration phases that are very important characteristics to achieve a more accurate solution.
TSA is one of the newest swarm-based meta-heuristic algorithms which simulates the swarm behavior of tunicates during its navigation and foraging for food source, it was firstly proposed by (S. Kaur, L.K. Awasthi, G. Dhiman) [26] for non-linear constrained problems. Its performance is proved by applying it on 74 benchmark problems including different types of functions. TSA is compared with other competitive algorithms. Also, its efficiency is confirmed by solving some constrained and unconstrained engineering design problems such as pressure vessel, welded beam, speed reducer, 25-bar truss, tension/compression spring, displacement of loaded structure and rolling element bearing. However, like the other metaheuristic approaches, TSA may prone to stuck on local optima when dealing with highdimensional and complicated problems. Therefore, there is still room to enhance the performance of TSA to deal with large-scale and realistic practical problems such as economic dispatch problem.
The main purpose of this work is to introduce an enhanced version of TSA algorithm by adding a new searching equation to the tunicate position. We prove the performance of ETSA by applying it on 20 test functions and the obtained results are compared with traditional TSA, GWO and other algorithms from the literature. The flexibility of the suggested ETSA algorithm is proven by the investigation on the large-scale dimension of functions (i.e., 30-dimension, 50-dimension and 100-dimension). Additionally, the proposed algorithm is realized on the economic dispatch problem as one of the most important electrical problems. The statistical tests and Wilcoxon test have proved the robustness and the efficacy of the proposed algorithm. Furthermore, the computational time of the implemented algorithms has affirmed that the proposed ETSA saves time over the other algorithms for most functions.
We arranged the rest of this paper as preliminaries, motivation of this work, and traditional TSA are presented in Sect. 2. The proposed algorithm is provided in Sect. 3. Experiments and results are in Sect. 4. The investigation on the economic dispatch problem is performed in Sect. 5. Analysis and discussion are exhibited in Sect. 6. Finally, the conclusion and future work are provided in Sect. 7.

Preliminaries
This section introduces the mathematical model of the optimization problem, economic dispatch problem, the main motivation, and the traditional TSA.

Problem Formulation
Optimization problems can be categorized into unconstrained and constrained problems [27]. The mathematical model of them is as follows.

Unconstrained Problem
Min f (u) = f u 1 , u 2 , … u k Subject to

Subject to
We could expand the constrained problem in one equation as follows.
where g i (u) Inequality constraint, h i (u) Equality constraints, u Vector of decision variables, LB, UB Lower and upper bounds of search space, M Inequality constraints number, P Number of all constraints, k Number of decision variables, R k k-dimensional rectangular space, Φ Search space that shows the domains of variables, Ω The feasible part of search space, (u) The total expanded cost function, i The penalty factor for all constraints, σ Small tolerance.
Economic dispatch problem is represented as a mathematical optimization problem with non-linear constraints. the traditional optimization techniques such as lamda iteration and gradient methods fail to give accurate results due to nonlinearities in cost function and thus the researchers have been worked on meta-heuristic algorithms to achieve more accurate solutions.

Economic Dispatch: Mathematical Formulation
The mathematical model of the economic dispatch problem can be defined as: Subject to: where PD is power demand for load, P gn i is real power generation that represents the decision variables, and ng defines the number of generation buses. Equation (4) can be expressed in two ways: Without valve point loading effect (smooth quadratic form) and it is considered as follows.
where a i , b i , c i are coefficients of generation cost and P gn i the output power of the generating unit in MW for ith unit.
With valve point loading effect (non-smoothing quadratic form), and it is formulated as non-linear, nonsmooth and non-convex cost function.
where a i , b i , c i , f i are coefficients of generation cost with valve point effect for ith unit.

Motivation of this Work
The researchers are concerned with two concepts: improving the current algorithms or making hybridizing between different techniques to enhance the convergence rate and improve the diversity of solutions. Although these techniques succeed with some problems, they suffer from some demerits of premature convergence and low diversity, especially when dealing with nonlinear problems and high-dimensional problems. For example, the TSA proved its effective performance in comparison with other different meta-heuristic algorithms but it suffers from some problems meanwhile it fails to escape from local optima while optimizing some functions. Therefore, improving its performance and increasing its searching capability present the main motivation. In this context, a novel searching equation is developed to enrich the exploitation phase while dealing with highdimensional complicated optimization problems. By this improvement, it is planned to prevent the trapping in local optimal points, reach to the global optimum after a few iterations. The proposed algorithm has been applied to several well-known benchmark test functions and is one of the important applications. The results illustrate that the proposed algorithm improves the solution quality and give an effective performance.

Traditional Tunicate Swarm Algorithm (TSA)
TSA is one of the latest bio-inspired meta-heuristic algorithm. Tunicates depend on two important behaviors in navigation and foraging for the food source, namely, jet propulsion and swarm intelligence. Jet propulsion is the propulsion of something in one direction, produced by ejecting a jet of fluid in the obverse direction [28]. Swarm intelligence deals with natural and artificial systems. They depend on the collective behavior between individuals of the swarm.

Mathematical Model of Jet Propulsion
To keep the balance of social forces between tunicates and avoid the collision between them, a random vector � ⃗ T is represented as follows.
where c 1 , c 2 , c 3 represents random values within range of TS min , TS max are the minimum and maximum speeds of tunicates, respectively, that are responsible for social interaction, where their values are set to 1 and 4. During the navigation search of tunicates for the best one (food source), the movement towards within the search space can performed as the following equation: where ������ ⃗ P pop (t + 1) is the update position of tunicates, t denotes the iteration number, ���� ⃗ FP denotes the food source position, and ����� ⃗ DT is the distance between each tunicate and food source.

Mathematical Model of Tunicates Intelligence
The two initial best solutions are saved and the positions of other tunicates are updated according to the positions of saved ones. The updated positions of tunicates are represented as follows.

Page 5 of 24 189
The flowchart of traditional TSA is shown in Fig. 1.

The Enhanced Tunicate Swarm Algorithm (ETSA)
In this section, ETSA is introduced based on a novel searching equation to enhance the exploitation ability while dealing with large-scale problems and to avoid the avoid the trapping in the local optima. The main steps of the ETSA are stated as follows.
Step 1: Initialization. In this step, the algorithm' parameters, maximum number of generations (T) and population size. Also, a swarm of tunicates is initialized within the search space at random.
Step 2: Evaluation. In this step, each tunicate is evaluated according to the fitness function.
Step 3: Improvement phase. This step aims to improve the searching process of the TSA. In this sense, a dynamic perturbation is presented to improve the exploitation pattern and explore the neighborhood solutions within the search region. In the searching equation, each position is perturbed with a dynamic step, where this position is survival if it is better than the old one. In this context, the bounds of the search space are updated using a dynamic manner. The new position of ETSA can be expressed as follows.
where represents a dynamic step that is shrunk with the progress of the optimization process to emphasize the neighborhood searching and thus enables the exploitation ability. It is expressed as follows.
where is a random number follows the uniform distribution ranged from 0 to 1, 1 and 2 define the dynamic bounds that are calculated as follows.
Step 4: Updating phase. In this step, swarm of tunicates is updated as in classical TSA using Eq. (11). In this sense, the tunicates can explore different regions with search space to perform the exploration ability and afterwards the fitness function of each tunicate is computed to update the best solution so far.
Step 5: The steps 2-4 can be repeated until the termination criterion is reached. The proposed ETSA flowchart is shown in Fig. 2, while the practical procedure of the proposed ETSA is depicted in Fig. 3.
By the means of Equation (11), more areas of the search region can be explored to perform exploration ability. On the other hand, Eq. (12) can enable the algorithm to exploit the promising regions to achieve the exploitation capability can be carried out. By this way, the balance between exploitation and exploration can be reached and thus the trapping into local optima can be avoided (Fig. 4).
Initialize the first population, all parameters and max no of iterations

Evaluation:
Compute the objective function of each tunicate then the best one is explored

Experiments and Results
The enhanced algorithm is applied on 20 benchmark test functions, two types of functions are used, unimodal and multimodal. The proposed ETSA is compared with the original TSA and GWO. The main details of the functions are mentioned in "Appendix A". The different parameters for all algorithms are mentioned in Table 1. Additionally, we compared the enhanced algorithm with other methods from the literature taken from Ref. [29,30]. The proposed algorithms are coded with Matlab R2015b using 64 bit Core i7 processor with 2.60 GHz and 8 GB main memory.

ETSA Results on Unconstrained Benchmark Test Functions
In this subsection, the ETSA is compared with GWO, original TSA. In this sense, 30 independently runs are performed to exhibit the stability of the proposed algorithms under the inherent haphazardness nature. The results are reported by the means of best, worst, median, mean and standard deviation (sd) for each algorithm as in

Comparison Results with Literature Review
To satisfy the validity of the proposed ETSA, its results are compared with other algorithms from the literature. In this context, the ETSA is investigated on unimodal and multimodal functions, where the comparisons have affirmed its capability for achieving the balance between exploitation and exploration.

1) Evaluation on unimodal functions (P1-P5)
The results of unimodal functions are compared with PSO, SCA, FFA, LSCA, OPI-SCA [29] as in Table 5 which illustrates the effective performance of the proposed algorithm against other algorithms. For (P1-P4), ETSA is the best algorithm and is the second optimizer for P5. We can conclude that the proposed ETSA has the capability to exploit the promising regions as it reaches tthe optimal solutions for most functions of this category.
2) Evaluation of multimodal functions (P6-P20) Tables 6 and 7 present the results of ETSA on multimodal functions against PSO, FFA, SCA, WOA, ALO, MDWA [30], the obtained results demonstrated that the proposed algorithm gives the best results for most problems whereas

Computational Time Analysis
The CPU-time that is consumed by all implemented techniques is reported in Table 8. We presented these results in Fig. 6. It can be concluded that the ETSA consumed less CPU-time than the classical TSA and GWO for most functions. The main reason for this can be shown as follows.
The dynamic perturbation that is presented to improve the   performance made the ETSA reaches the optimum values after little iterations. On the other hand, the CPU-time of ETSA for some functions is slightly larger than the other techniques. The main explanation for this can be analyzed as follows. ETSA seeks for the promising regions using two searching strategies which cause some delay. This delay can enable ETSA to give concise and more precise results with high performance than the other optimizers. All these results show the superiority and the high quality of the ETSA.

Performance Assessment
After applying the proposed ETSA on 20 unconstrained test functions and the results showed that it outperforms all other competitive algorithms. The ETSA is further investigated with high-dimensional nature to affirm its scalability and confirm its stable performance. One of the main properties of any optimizer is to solve problems of high dimensions efficiently. In this sense, the ETSA is conducted different dimensions include 30, 50, 100. By applying the proposed algorithm, the obtained results illustrate that the proposed algorithm is running efficiently and is least affected by increasing the number dimensions, where the obtained results are reported in Table 9.

Error and Ranking Analyses
In this subsection, the proposed algorithms are furthered assessed in terms of deviation from the global optimal solution. In this context, the absolute error metric is presented to prove and analyze the performance of the proposed algorithms. The differences between the optimum value and the obtained best values from presented algorithms over the benchmark problems of unimodal and multimodal test functions are calculated and recorded in Tables 10 and 11, respectively. It is clear that the proposed algorithm is more precise than the other competitive algorithms and its results are very close to the optimum value. Furthermore, the ranking of the presented algorithms is performed in terms of the absolute error metric as shown in Tables 12 and 13. Moreover, the ranking level for each algorithm according to its absolute error over the test functions are recorded regarding the statistical frequency or number of appearances as in Tables 14 and 15 (i.e. see

Non-parametric Statistical Analysis
Due to the inherent stochastic nature of swarm optimizers, the non-parametric statistical test is vital for confirming confidential comparisons. In this sense, the Wilcoxon sign-rank is presented in this work. Wilcoxon sign-rank computes the difference between two or more algorithms based on hypothesizes. Therefore, the ability of the proposed ETSA against the other methods are assessed based on the Wilcoxon test and the results are reported in Table 16. From these results, it can be observed that ETSA shows a superior performance over the TSA, GWO, SCA, FFA, PSO, LSCA, and ALO with a level of significance of = 0.05 and its competitors with OPI-SCA, WOA and MDWA. Finally, the reality of the proposed algorithm to solve more complicated problems has been approved using the practical power economic dispatch problem.

Economic Dispatch Problem
The power economic dispatch problem (PED) is one of the most important problems in the power system. In this essence, it is modeled as a mathematical optimization problem along with some operational constraints. The main  objective function of the economic dispatch problem is to achieve the optimal generated powers that minimizes the generation fuel cost with satisfying power limits and load demand.
The cost function can be represented in two ways as quadratic polynomial function when neglecting the valve point effect. In large generating units, due to the steam valve effect, any sudden increasing in losses making ripples in the fuel cost function. Therefore, there is a sinusoidal function with the traditional quadratic cost function to simulate the valve point effect of the PED. In this context, the PED is represented as a non-convex and non-smooth optimization problem having equality and inequality constraints, which makes the challenge of finding the global optimum hard. In this work, we applied the proposed algorithm on three test systems to prove its efficiency in solving non-convex and non-smooth PED. Also, the comparisons with other algorithms are presented.

Test Systems
Four test systems include 3 units, 6 units, 20 units and 118 bus system are applied to prove the capability of the proposed ETSA in solving EDP. The main data of the first three systems are taken from Refs. [31,32] and the data of 118 bus system are taken from matpower.

Results of Economic Dispatch
• Case1: The proposed algorithms are conducted on 3 unit system with considering valve point effect, where the demand of load is 850 MW with no transmission losses. We found that the proposed algorithm gives the   Table 17 and the convergence curves are shown in Fig. 9. • Case 2: In this case, 6 unit system is adopted, where the valve point effect is considered, and 283.4 MW for the load demand is applied. Table 18 shows the results of this case and the convergence curves are shown in Fig. 10. The generation cost obtained by the proposed algorithm is 8234.0717 $/h which is superior than the other algorithms. • Case 3: 20 unit system without valve point effect is conducted, where the total demand power is 2500 MW. It is found that the proposed algorithm gives the minimum fuel cost and the results are shown in Table 19. The convergence curves are shown in Fig. 11. • Case 4: 118 bus system of 54 units is solved, where the load demand is considered as 4242 MW. Table 20 shows the results of this case and the convergence curves are shown in Fig. 12. The generation cost which is obtained by the proposed algorithm is 127205.7 $/h.

Comparison Results with Literature Review
To satisfy the robustness of the proposed algorithm in solving different applications, comparisons with other methods are reported. In this line, the proposed ETSA is compared with GWO, TSA, and parallel hurricane optimization algorithm (PHOA) [33] for 118-bus system. The results are reported in Table 20 which shows that the proposed algorithm gives the minimum fuel cost over the others.

Analysis and discussion
This paper introduces a new methodology by enhancing the classical TSA. The proposed Algorithm is applied on twenty function, unimodal functions (P1-P5) and multimodal functions (P6-P20). The results show the superiority of the proposed algorithm with zero percentage error for eight functions and a very low percentage for other functions. It can be concluded from the CPU time that the suggested technique consumed less CPU time than the others for most functions due to the dynamic perturbation and the capability of reaching to the optimum solution after few iterations. The results are implemented at dimensions of 30, 50, and 100, where the ETSA is running efficiently and it is least affected by increasing the dimensions. We have evaluated the rank for each algorithm and the ETSA has the first rank for sixteen functions. We applied the ETSA on one of the major electrical applications (i.e., Economic dispatch problem). We obtained the minimum fuel cost from the four systems (i.e., 3-units,

Conclusion and Future Work
This paper presents a modification of Tunicate Swarm Algorithm based on a new search equation to overcome the drawbacks of the original TSA. We applied the proposed algorithm on 20 unconstrained test functions of unimodal and multimodal natures. By comparisons with GWO and traditional TSA, it is found that the ETSA performs efficiently and reaches the global optimum after a few iterations for most functions. Also the obtained results by ETSA     TSA  6  3  0  1  0  3  1  0  1  GWO  7  1  4  0  2  0  0  1  0  PSO  2  1  2  1  1  3  3  1  1  FFA  2  1  1  2  1  2  3  2  1  SCA  3  2  1  1  3  0  1  3          Although the ETSA proved its robustness and efficacy in different applications, there may elapsed some times for some functions. So in future work, we concerted on developing new mutation mechanism to accelerate its performance. Also, we will focus on solving multi-objective problems, solving very large systems of economic dispatch problem and solving other important applications.