Abstract
In this paper, we present constrained simulated annealing (CSA), an algorithm that extends conventional simulated annealing to look for constrained local minima of nonlinear constrained optimization problems. The algorithm is based on the theory of extended saddle points (ESPs) that shows the one-to-one correspondence between a constrained local minimum and an ESP of the corresponding penalty function. CSA finds ESPs by systematically controlling probabilistic descents in the problem-variable subspace of the penalty function and probabilistic ascents in the penalty subspace. Based on the decomposition of the necessary and sufficient ESP condition into multiple necessary conditions, we present constraint-partitioned simulated annealing (CPSA) that exploits the locality of constraints in nonlinear optimization problems. CPSA leads to much lower complexity as compared to that of CSA by partitioning the constraints of a problem into significantly simpler subproblems, solving each independently, and resolving those violated global constraints across the subproblems. We prove that both CSA and CPSA asymptotically converge to a constrained global minimum with probability one in discrete optimization problems. The result extends conventional simulated annealing (SA), which guarantees asymptotic convergence in discrete unconstrained optimization, to that in discrete constrained optimization. Moreover, it establishes the condition under which optimal solutions can be found in constraint-partitioned nonlinear optimization problems. Finally, we evaluate CSA and CPSA by applying them to solve some continuous constrained optimization benchmarks and compare their performance to that of other penalty methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarts E., Korst J. (1989) Simulated Annealing and Boltzmann Machines. Wiley, New York
Anily S., Federgruen A. (1987) Ergodicity in parametric nonstationary Markov chains: an application to simulated annealing methods. Operations Res. 35(6):867–874
Anily S., Federgruen A. (1987) Simulated annealing methods with general acceptance probabilities. J.Appl. Prob. 24:657–667
Auslender A., Cominetti R., Maddou M. (1997) Asymptotic analysis for penalty and barrier methods in convex and linear programming. Math. Operations Res. 22:43–62
Back T., Hoffmeister F., Schwefel H.-P. A survey of evolution strategies. In: Proceedings of the 4th Int’l Conference on Genetic Algorithms, pp 2–9. San Diego, CA, (1991)
Bean J.C., Hadj-Alouane, A.B.: A dual genetic algorithm for bounded integer programs. In Technical Report TR 92-53, Department of Industrial and Operations Engineering, The University of Michigan (1992)
Bertsekas D.P., Koksal A.E. (2000) Enhanced optimality conditions and exact penalty functions. Proceedings of Allerton Conference, Allerton, IL
Bongartz I.,Conn A.R.,Gould N., Toint P.L. (1995) CUTE: Constrained and unconstrained testing environment. ACM Trans. Math Softw, 21(1):123–160
Chen, Y.X.: Solving nonlinear constrained optimization problems through constraint partitioning. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2005)
Corana A., Marchesi M., Martini C., Ridella S. (1987) Minimizing multimodal functions of continuous variables with the simulated annealing algorithm. ACM Trans. Math. Softw. 13(3):262–280
Evans J.P., Gould F.J., Tolle J.W. (1973) Exact penalty functions in nonlinear programming. Math. Program. 4:72–97
Fletcher R. (1970) A class of methods for nonlinear programming with termination and convergence properties. In: Abadie J.(eds) Integer and Nonlinear Programming. North-Holland, Amsterdam
Fletcher R. An exact penalty function for nonlinear programming with inequalities. Technical Report 478, Atomic Energy Research Establishment, Harwell (1972)
Fourer R., Gay D.M., Kernighan B.W. AMPL: A Modeling Language for Mathematical Programming. Brooks Cole Publishing Company (2002)
Freidlin M.I., Wentzell A.D. (1984) Random Perturbations of Dynamical Systems. Springer, Berlin
Gill P.E., Murray W., Saunders M. (2002) SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12:979–1006
Homaifar A., Lai S.H-Y., Qi X. (1994) Constrained optimization via genetic algorithms. Simulation 62(4):242–254
Joines J., Houck C.: On the use of non-stationary penalty functions to solve nonlinear constrained optimization problems with gas. In: Proceedings of the First IEEE Int’l Conf. on Evolutionary Computation, pp. 579–584. Orlando, FL (1994)
Kirkpatrick S., Gelatt Jr., C.D., Vecchi M.P. (1983). Optimization by simulated annealing. Science. 220(4598): 671–680
Krishnan, S., Krishnamoorthy, S., Baumgartner, G., Lam, C.C., Ramanujam, J., Sadayappan, P., Choppella, V.: Efficient synthesis of out-of-core algorithms using a nonlinear optimization solver.Technical report, Department of Computer and Information Science, Ohio State University, Columbus, OH (2004)
Kuri, A.: A universal electric genetic algorithm for constrained optimization. In: Proceedings of the 6th European Congress on Intelligent Techniques and Soft Computing, pp. 518–522. Aachen, Germany (1998)
Luenberger D.G. (1984) Linear and Nonlinear Programming. Addison-Wesley, Reading, MA
Mitra D., Romeo F., Vincentelli A.S. (1986) Convergence and finite-time behavior of simulated annealing. Adv. Appl. Prob. 18:747–771
Rardin R.L. (1998) Optimization in Operations Research. Prentice Hall, New York
Trouve, A.: Rough large deviation estimates for the optimal convergence speed exponent of generalized simulated annealing algorithms. Technical report, LMENS-94-8, Ecole Normale , France (1994)
Trouve A. (1996) Cycle decomposition and simulated annealing. SIAM J. Control Optim. 34(3):966–986
Wah B., Chen Y.X. (2006) Constraint partitioning in penalty formulations for solving temporal planning problems. Artif Intel 170(3):187–231
Wah, B.W., Chen, Y.X.: Solving large-scale nonlinear programming problems by constraint partitioning. In: Proceedigs of the Principles and Practice of Constraint Programming, LCNS-3709, pp. 697–711. Springer-Verlag, New York (2005)
Wah, B.W., Wang, T.: Simulated annealing with asymptotic convergence for nonlinear constrained global optimization. In: Proceedings of the Principles and Practice of Constraint Programming, pp. 461–475. Springer-Verlag, New York (1999)
Wah, B.W., Wu, Z.: The theory of discrete Lagrange multipliers for nonlinear discrete optimization. In: Proceedings of the Principles and Practice of Constraint Programming, pp. 28–42. Springer-Verlag, New York (1999)
Wang, T.: Global Optimization for Constrained Nonlinear Programming. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2000)
Wu, Z.: The Theory and Applications of Nonlinear Constrained Optimization using Lagrange Multipliers. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2001)
Zangwill W.I. (1967) Nonlinear programming via penalty functions. Manag. Sci. 13:344–358
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wah, B.W., Chen, Y. & Wang, T. Simulated annealing with asymptotic convergence for nonlinear constrained optimization. J Glob Optim 39, 1–37 (2007). https://doi.org/10.1007/s10898-006-9107-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-006-9107-z