Abstract
In this paper a novel computing paradigm aimed at solving non linear systems of equations and finding feasible solutions and local optima to scalar and multi objective optimizations problems is conceptualized. The underlying principle is to formulate a generic programming problem by a proper set of ordinary differential equations, whose equilibrium points correspond to the problem solutions. Starting from the Lyapunov theory, we will demonstrate that this artificial dynamic system could be designed to be stable with an exponential asymptotic convergence to equilibrium points. This important feature allows the analyst to overcome some of the inherent limitations of the traditional iterative solution algorithms that can fail to converge due to the highly nonlinearities of the first-order conditions. Besides we will demonstrate as the proposed paradigm could be applied to solve non linear equations systems, scalar and multi-objective optimization problems. Extensive numerical studies aimed at assessing the effectiveness of the proposed computing paradigm are presented and discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A local minimum/maxima \(x^*\) is defined as a point satisfying the following condition: \(\exists \delta >0'\left\| {x-x^*} \right\| <\delta \Rightarrow \left\{ {{\begin{array}{ll} {f(x^*)\le f(x)\hbox { Local Minima}} \\ {f(x^*)\ge f(x)\hbox { Local Maxima}} \\ \end{array} }} \right. .\)
An example of barrier function is the so called logarithmic barrier function defined as:
$$\begin{aligned} B(x,\mu )=f(x)-\mu \sum \limits _{i=1}^m {\ln (c_i (x))} \end{aligned}$$where \(f(x)\) is the objective function, \(\mu \) is the slack variable and \(c_i (x)\) is the i-th the inequality constraint function.
It is worth noting as the variable “\(t\)” is an artificial parameter and we are only interested in the final equilibrium points reached by the dynamic system and the transient nature of the trajectories.
Since the system of differential equations (7) is nonlinear, the corresponding equilibrium point depends of the initial condition. This implies that the dynamic model could have different equilibrium points for different initial conditions. Further studies aiming at characterizing the dimension of the regions of attraction is currently under investigation by the authors.
During this evolution it results \(q(t)\approx q(0)\ne f(\mathrm{\mathbf{x}}^*)\). Successively, when \(q(t)\) reaches the equilibrium point it results \(q^*=f(\mathrm{\mathbf{x}}^*)\) and we can conclude that \(q(t)\) converges to the value assumed by \(f(\mathrm{\mathbf{x}})\) in its minimum.
Under these hypothesis when \(\mathrm{\mathbf{x}}(t)\) converges to \(\mathrm{\mathbf{x}}^*\) it results \(q_i (t)\approx q_i (0)=0\ne f_i (\mathrm{\mathbf{x}}^*)\).
Intensive research activities aiming at formally defining the connections between the proposed dynamic paradigm, the gradient descent algorithm and the Pareto theory is currently under investigation by the authors.
We expect that a more accurate selection of the initial states (i.e. by considering a feasible solution generated by a traditional algorithm) could sensibly improve the algorithm convergence especially in solving non convex optimization problems.
References
Agrawal S, Dashora Y, Tiwari MK, Young-Jun S (2008) Interactive particle swarm: a pareto-adaptive metaheuristic to multiobjective optimization. IEEE Trans Syst Man Cybern Part A Syst Hum 38(2):258–277
Astfalk G, Lustig I, Marsten R, Shanno D (1992) The interior-point method for linear programming. IEEE Softw 9(4):61–68
Boggs PT, Domich PD, Rogers JE (1996) An interior-point method for general large scale quadratic programming problems. Ann Oper Res 62(1):419–437
Broyden CG (1965) A class of methods for solving nonlinear simultaneous equations. Math Comput 19(92):577–593
Chinchuluun A, Pardalos PM (2007) A survey of recent developments in multiobjective optimization. Ann Oper Res 154:29–50
Coello CA, Aguirre AH, Zitzler E (2007) Evolutionary multi-objective optimization. Eur J Oper Res 181(3):1617–1619
Conn AR, Gould NIM, Toint PL (2000) Trust-Region Methods. SIAM, Philadelphia
Delbos F, Gilbert JC (2005) Global linear convergence of an augmented Lagrangian algorithm for solving convex quadratic optimization problems. J Convex Anal 12(1):45–69
Denis JE, Wolkowicz H (1993) Least change secant methods, sizing, and shifting. SIAM J Numer Anal 30:1291–1314
Fonseca CM, Fleming PJ (1998) Multiobjective optimization and multiple constraint handling with evolutionary algorithms. I. A unified formulation. IEEE Trans Syst Man Cybern Part A Syst Hum 28(1):26–37
Grosan C, Abraham A (2008) A new approach for solving nonlinear equations systems. IEEE Trans Syst Man Cybern Part A Syst Hum 38(3):698–714
Li Y-X, Gen M (1996) Nonlinear mixed integer programming problems using genetic algorithm and penalty function. IEEE Int Conf Syst Man Cybern 2677–2682(4):14–17
Malick J, Roupin F (2013) On the bridge between combinatorial optimization and nonlinear optimization: a family of semidefinite bounds for 0–1 quadratic problems leading to quasi-Newton methods. Math Progr 140(1):99–124
Marler RT, Arora JS (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim 26(6):369–395
Martinez JM (1994) Algorithms for solving nonlinear systems of equations. In: Spedicato E (ed) Continuous optimization: the state of the art. Kluwer, Norwell, pp 81–108
Michalewicz Z (1995) Genetic algorithms, numerical optimization, and constraints. In: Proceedings of the 6th international conference on genetic algorithms, pp 151–158
Narula SC, Kirilov L, Vassilev V (1994) Reference direction approach for solving multiple objective nonlinear programming problems. IEEE Trans Syst Man Cybern 24859:804–806
Ng KYK (1987) Goal Programming. Method of weighted residuals, and optimal control problems. IEEE Trans Syst Man Cybern 17(1):102–106
Okabe T, Jin Y, Sendhoff B (2003) A critical survey of performance indices for multi-objective optimisation. In: The 2003 Congress on Evolutionary Computation, CEC ’03, vol 2, pp 878–885
Ortega JM, Rheinboldt WC (1970) Iterative solution of nonlinear equations in several variables. Academic, New York
Park KS (2004) Mathematical programming models for characterizing dominance and potential optimality when multicriteria alternative values and weights are simultaneously incomplete. IEEE Trans Syst Man Cybern Part A Syst Hum 34(5):601–614
Rodriguez-Vazquez K, Fonseca CM, Fleming PJ (2004) Identifying the structure of nonlinear dynamic systems using multiobjective genetic programming. IEEE Trans Syst Man Cybern Part A Syst Hum 34(4):531–545
Tinney WF, Walker JW (1967) Direct solutions of sparse network equations by optimally ordered triangular factorization. Proc IEEE 55(11):1801–1809
Torelli F, Vaccaro A, Xie N (2013) A novel optimal power flow formulation based on the Lyapunov Theory. IEEE Trans Power Syst 28(4): 4405–4415
Xie N, Torelli F, Bompard E, Vaccaro A (2013) A dynamic computing paradigm for comprehensive power flow analysis. IET Gener Transm Distribn 7(8):832–842
Yang X-S (2011) Metaheuristic optimization: algorithm analysis and open problems. Experimental Algorithms. In: Lecture notes in computer science, vol 6630, pp 21–32
Zeshui X, Xiaoqiang C, Shousheng L (2011) Nonlinear programming model integrating different preference structures. IEEE Trans Syst Man Cybern Part A Syst Hum 41(1):169–177
Zitzler E, Thiele L (1998) Multiobjective optimization using evolutionary algorithms: a comparative case study. In: Parallel problem solving from nature, Amsterdam, The Netherlands. Springer, Berlin, pp 292, 301
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by E. Munoz.
Rights and permissions
About this article
Cite this article
Torelli, F., Vaccaro, A. A generalized computing paradigm based on artificial dynamic models for mathematical programming. Soft Comput 18, 1561–1573 (2014). https://doi.org/10.1007/s00500-013-1162-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-013-1162-z