Journal of Global Optimization

, Volume 39, Issue 1, pp 1–37 | Cite as

Simulated annealing with asymptotic convergence for nonlinear constrained optimization

Original Paper


In this paper, we present constrained simulated annealing (CSA), an algorithm that extends conventional simulated annealing to look for constrained local minima of nonlinear constrained optimization problems. The algorithm is based on the theory of extended saddle points (ESPs) that shows the one-to-one correspondence between a constrained local minimum and an ESP of the corresponding penalty function. CSA finds ESPs by systematically controlling probabilistic descents in the problem-variable subspace of the penalty function and probabilistic ascents in the penalty subspace. Based on the decomposition of the necessary and sufficient ESP condition into multiple necessary conditions, we present constraint-partitioned simulated annealing (CPSA) that exploits the locality of constraints in nonlinear optimization problems. CPSA leads to much lower complexity as compared to that of CSA by partitioning the constraints of a problem into significantly simpler subproblems, solving each independently, and resolving those violated global constraints across the subproblems. We prove that both CSA and CPSA asymptotically converge to a constrained global minimum with probability one in discrete optimization problems. The result extends conventional simulated annealing (SA), which guarantees asymptotic convergence in discrete unconstrained optimization, to that in discrete constrained optimization. Moreover, it establishes the condition under which optimal solutions can be found in constraint-partitioned nonlinear optimization problems. Finally, we evaluate CSA and CPSA by applying them to solve some continuous constrained optimization benchmarks and compare their performance to that of other penalty methods.


Asymptotic convergence Constrained local minimum Constraint partitioning Simulated annealing Dynamic penalty methods Extended saddle points Nonlinear constrained optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aarts E., Korst J. (1989) Simulated Annealing and Boltzmann Machines. Wiley, New YorkGoogle Scholar
  2. 2.
    Anily S., Federgruen A. (1987) Ergodicity in parametric nonstationary Markov chains: an application to simulated annealing methods. Operations Res. 35(6):867–874Google Scholar
  3. 3.
    Anily S., Federgruen A. (1987) Simulated annealing methods with general acceptance probabilities. J.Appl. Prob. 24:657–667CrossRefGoogle Scholar
  4. 4.
    Auslender A., Cominetti R., Maddou M. (1997) Asymptotic analysis for penalty and barrier methods in convex and linear programming. Math. Operations Res. 22:43–62Google Scholar
  5. 5.
    Back T., Hoffmeister F., Schwefel H.-P. A survey of evolution strategies. In: Proceedings of the 4th Int’l Conference on Genetic Algorithms, pp 2–9. San Diego, CA, (1991)Google Scholar
  6. 6.
    Bean J.C., Hadj-Alouane, A.B.: A dual genetic algorithm for bounded integer programs. In Technical Report TR 92-53, Department of Industrial and Operations Engineering, The University of Michigan (1992)Google Scholar
  7. 7.
    Bertsekas D.P., Koksal A.E. (2000) Enhanced optimality conditions and exact penalty functions. Proceedings of Allerton Conference, Allerton, ILGoogle Scholar
  8. 8.
    Bongartz I.,Conn A.R.,Gould N., Toint P.L. (1995) CUTE: Constrained and unconstrained testing environment. ACM Trans. Math Softw, 21(1):123–160CrossRefGoogle Scholar
  9. 9.
    Chen, Y.X.: Solving nonlinear constrained optimization problems through constraint partitioning. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2005)Google Scholar
  10. 10.
    Corana A., Marchesi M., Martini C., Ridella S. (1987) Minimizing multimodal functions of continuous variables with the simulated annealing algorithm. ACM Trans. Math. Softw. 13(3):262–280CrossRefGoogle Scholar
  11. 11.
    Evans J.P., Gould F.J., Tolle J.W. (1973) Exact penalty functions in nonlinear programming. Math. Program. 4:72–97CrossRefGoogle Scholar
  12. 12.
    Fletcher R. (1970) A class of methods for nonlinear programming with termination and convergence properties. In: Abadie J.(eds) Integer and Nonlinear Programming. North-Holland, AmsterdamGoogle Scholar
  13. 13.
    Fletcher R. An exact penalty function for nonlinear programming with inequalities. Technical Report 478, Atomic Energy Research Establishment, Harwell (1972)Google Scholar
  14. 14.
    Fourer R., Gay D.M., Kernighan B.W. AMPL: A Modeling Language for Mathematical Programming. Brooks Cole Publishing Company (2002)Google Scholar
  15. 15.
    Freidlin M.I., Wentzell A.D. (1984) Random Perturbations of Dynamical Systems. Springer, BerlinGoogle Scholar
  16. 16.
    Gill P.E., Murray W., Saunders M. (2002) SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12:979–1006CrossRefGoogle Scholar
  17. 17.
    Homaifar A., Lai S.H-Y., Qi X. (1994) Constrained optimization via genetic algorithms. Simulation 62(4):242–254Google Scholar
  18. 18.
    Joines J., Houck C.: On the use of non-stationary penalty functions to solve nonlinear constrained optimization problems with gas. In: Proceedings of the First IEEE Int’l Conf. on Evolutionary Computation, pp. 579–584. Orlando, FL (1994)Google Scholar
  19. 19.
    Kirkpatrick S., Gelatt Jr., C.D., Vecchi M.P. (1983). Optimization by simulated annealing. Science. 220(4598): 671–680Google Scholar
  20. 20.
    Krishnan, S., Krishnamoorthy, S., Baumgartner, G., Lam, C.C., Ramanujam, J., Sadayappan, P., Choppella, V.: Efficient synthesis of out-of-core algorithms using a nonlinear optimization solver.Technical report, Department of Computer and Information Science, Ohio State University, Columbus, OH (2004)Google Scholar
  21. 21.
    Kuri, A.: A universal electric genetic algorithm for constrained optimization. In: Proceedings of the 6th European Congress on Intelligent Techniques and Soft Computing, pp. 518–522. Aachen, Germany (1998)Google Scholar
  22. 22.
    Luenberger D.G. (1984) Linear and Nonlinear Programming. Addison-Wesley, Reading, MAGoogle Scholar
  23. 23.
    Mitra D., Romeo F., Vincentelli A.S. (1986) Convergence and finite-time behavior of simulated annealing. Adv. Appl. Prob. 18:747–771CrossRefGoogle Scholar
  24. 24.
    Rardin R.L. (1998) Optimization in Operations Research. Prentice Hall, New YorkGoogle Scholar
  25. 25.
    Trouve, A.: Rough large deviation estimates for the optimal convergence speed exponent of generalized simulated annealing algorithms. Technical report, LMENS-94-8, Ecole Normale , France (1994)Google Scholar
  26. 26.
    Trouve A. (1996) Cycle decomposition and simulated annealing. SIAM J. Control Optim. 34(3):966–986CrossRefGoogle Scholar
  27. 27.
    Wah B., Chen Y.X. (2006) Constraint partitioning in penalty formulations for solving temporal planning problems. Artif Intel 170(3):187–231CrossRefGoogle Scholar
  28. 28.
    Wah, B.W., Chen, Y.X.: Solving large-scale nonlinear programming problems by constraint partitioning. In: Proceedigs of the Principles and Practice of Constraint Programming, LCNS-3709, pp. 697–711. Springer-Verlag, New York (2005)Google Scholar
  29. 29.
    Wah, B.W., Wang, T.: Simulated annealing with asymptotic convergence for nonlinear constrained global optimization. In: Proceedings of the Principles and Practice of Constraint Programming, pp. 461–475. Springer-Verlag, New York (1999)Google Scholar
  30. 30.
    Wah, B.W., Wu, Z.: The theory of discrete Lagrange multipliers for nonlinear discrete optimization. In: Proceedings of the Principles and Practice of Constraint Programming, pp. 28–42. Springer-Verlag, New York (1999)Google Scholar
  31. 31.
    Wang, T.: Global Optimization for Constrained Nonlinear Programming. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2000)Google Scholar
  32. 32.
    Wu, Z.: The Theory and Applications of Nonlinear Constrained Optimization using Lagrange Multipliers. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2001)Google Scholar
  33. 33.
    Zangwill W.I. (1967) Nonlinear programming via penalty functions. Manag. Sci. 13:344–358CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2006

Authors and Affiliations

  1. 1.Department of Electrical and Computer Engineering and the Coordinated Science LaboratoryUniversity of IllinoisUrbanaUSA
  2. 2.Department of Computer ScienceWashington UniversitySt. LouisUSA
  3. 3.Synopsys Inc.Mountain ViewUSA

Personalised recommendations