Simulated annealing with asymptotic convergence for nonlinear constrained optimization
 Benjamin W. Wah,
 Yixin Chen,
 Tao Wang
 … show all 3 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
In this paper, we present constrained simulated annealing (CSA), an algorithm that extends conventional simulated annealing to look for constrained local minima of nonlinear constrained optimization problems. The algorithm is based on the theory of extended saddle points (ESPs) that shows the onetoone correspondence between a constrained local minimum and an ESP of the corresponding penalty function. CSA finds ESPs by systematically controlling probabilistic descents in the problemvariable subspace of the penalty function and probabilistic ascents in the penalty subspace. Based on the decomposition of the necessary and sufficient ESP condition into multiple necessary conditions, we present constraintpartitioned simulated annealing (CPSA) that exploits the locality of constraints in nonlinear optimization problems. CPSA leads to much lower complexity as compared to that of CSA by partitioning the constraints of a problem into significantly simpler subproblems, solving each independently, and resolving those violated global constraints across the subproblems. We prove that both CSA and CPSA asymptotically converge to a constrained global minimum with probability one in discrete optimization problems. The result extends conventional simulated annealing (SA), which guarantees asymptotic convergence in discrete unconstrained optimization, to that in discrete constrained optimization. Moreover, it establishes the condition under which optimal solutions can be found in constraintpartitioned nonlinear optimization problems. Finally, we evaluate CSA and CPSA by applying them to solve some continuous constrained optimization benchmarks and compare their performance to that of other penalty methods.
 Aarts, E., Korst, J. (1989) Simulated Annealing and Boltzmann Machines. Wiley, New York
 Anily, S., Federgruen, A. (1987) Ergodicity in parametric nonstationary Markov chains: an application to simulated annealing methods. Operations Res. 35: pp. 867874
 Anily, S., Federgruen, A. (1987) Simulated annealing methods with general acceptance probabilities. J.Appl. Prob. 24: pp. 657667 CrossRef
 Auslender, A., Cominetti, R., Maddou, M. (1997) Asymptotic analysis for penalty and barrier methods in convex and linear programming. Math. Operations Res. 22: pp. 4362
 Back T., Hoffmeister F., Schwefel H.P. A survey of evolution strategies. In: Proceedings of the 4th Int’l Conference on Genetic Algorithms, pp 2–9. San Diego, CA, (1991)
 Bean J.C., HadjAlouane, A.B.: A dual genetic algorithm for bounded integer programs. In Technical Report TR 9253, Department of Industrial and Operations Engineering, The University of Michigan (1992)
 Bertsekas, D.P., Koksal, A.E. (2000) Enhanced optimality conditions and exact penalty functions. Proceedings of Allerton Conference, Allerton, IL
 Bongartz, I., Conn, A.R., Gould, N., Toint, P.L. (1995) CUTE: Constrained and unconstrained testing environment. ACM Trans. Math Softw, 21: pp. 123160 CrossRef
 Chen, Y.X.: Solving nonlinear constrained optimization problems through constraint partitioning. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2005)
 Corana, A., Marchesi, M., Martini, C., Ridella, S. (1987) Minimizing multimodal functions of continuous variables with the simulated annealing algorithm. ACM Trans. Math. Softw. 13: pp. 262280 CrossRef
 Evans, J.P., Gould, F.J., Tolle, J.W. (1973) Exact penalty functions in nonlinear programming. Math. Program. 4: pp. 7297 CrossRef
 Fletcher, R. A class of methods for nonlinear programming with termination and convergence properties. In: Abadie, J. eds. (1970) Integer and Nonlinear Programming.. NorthHolland, Amsterdam
 Fletcher R. An exact penalty function for nonlinear programming with inequalities. Technical Report 478, Atomic Energy Research Establishment, Harwell (1972)
 Fourer R., Gay D.M., Kernighan B.W. AMPL: A Modeling Language for Mathematical Programming. Brooks Cole Publishing Company (2002)
 Freidlin, M.I., Wentzell, A.D. (1984) Random Perturbations of Dynamical Systems. Springer, Berlin
 Gill, P.E., Murray, W., Saunders, M. (2002) SNOPT: an SQP algorithm for largescale constrained optimization. SIAM J. Optim. 12: pp. 9791006 CrossRef
 Homaifar, A., Lai, S.HY., Qi, X. (1994) Constrained optimization via genetic algorithms. Simulation 62: pp. 242254
 Joines J., Houck C.: On the use of nonstationary penalty functions to solve nonlinear constrained optimization problems with gas. In: Proceedings of the First IEEE Int’l Conf. on Evolutionary Computation, pp. 579–584. Orlando, FL (1994)
 Kirkpatrick S., Gelatt Jr., C.D., Vecchi M.P. (1983). Optimization by simulated annealing. Science. 220(4598): 671–680
 Krishnan, S., Krishnamoorthy, S., Baumgartner, G., Lam, C.C., Ramanujam, J., Sadayappan, P., Choppella, V.: Efficient synthesis of outofcore algorithms using a nonlinear optimization solver.Technical report, Department of Computer and Information Science, Ohio State University, Columbus, OH (2004)
 Kuri, A.: A universal electric genetic algorithm for constrained optimization. In: Proceedings of the 6th European Congress on Intelligent Techniques and Soft Computing, pp. 518–522. Aachen, Germany (1998)
 Luenberger, D.G. (1984) Linear and Nonlinear Programming. AddisonWesley, Reading, MA
 Mitra, D., Romeo, F., Vincentelli, A.S. (1986) Convergence and finitetime behavior of simulated annealing. Adv. Appl. Prob. 18: pp. 747771 CrossRef
 Rardin, R.L. (1998) Optimization in Operations Research. Prentice Hall, New York
 Trouve, A.: Rough large deviation estimates for the optimal convergence speed exponent of generalized simulated annealing algorithms. Technical report, LMENS948, Ecole Normale , France (1994)
 Trouve, A. (1996) Cycle decomposition and simulated annealing. SIAM J. Control Optim. 34: pp. 966986 CrossRef
 Wah, B., Chen, Y.X. (2006) Constraint partitioning in penalty formulations for solving temporal planning problems. Artif Intel 170: pp. 187231 CrossRef
 Wah, B.W., Chen, Y.X.: Solving largescale nonlinear programming problems by constraint partitioning. In: Proceedigs of the Principles and Practice of Constraint Programming, LCNS3709, pp. 697–711. SpringerVerlag, New York (2005)
 Wah, B.W., Wang, T.: Simulated annealing with asymptotic convergence for nonlinear constrained global optimization. In: Proceedings of the Principles and Practice of Constraint Programming, pp. 461–475. SpringerVerlag, New York (1999)
 Wah, B.W., Wu, Z.: The theory of discrete Lagrange multipliers for nonlinear discrete optimization. In: Proceedings of the Principles and Practice of Constraint Programming, pp. 28–42. SpringerVerlag, New York (1999)
 Wang, T.: Global Optimization for Constrained Nonlinear Programming. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2000)
 Wu, Z.: The Theory and Applications of Nonlinear Constrained Optimization using Lagrange Multipliers. Ph.D. thesis, Department of Computer Science, University of Illinois, Urbana, IL (2001)
 Zangwill, W.I. (1967) Nonlinear programming via penalty functions. Manag. Sci. 13: pp. 344358 CrossRef
 Title
 Simulated annealing with asymptotic convergence for nonlinear constrained optimization
 Journal

Journal of Global Optimization
Volume 39, Issue 1 , pp 137
 Cover Date
 20070901
 DOI
 10.1007/s108980069107z
 Print ISSN
 09255001
 Online ISSN
 15732916
 Publisher
 Kluwer Academic PublishersPlenum Publishers
 Additional Links
 Topics
 Keywords

 Asymptotic convergence
 Constrained local minimum
 Constraint partitioning
 Simulated annealing
 Dynamic penalty methods
 Extended saddle points
 Nonlinear constrained optimization
 Industry Sectors
 Authors

 Benjamin W. Wah ^{(1)}
 Yixin Chen ^{(2)}
 Tao Wang ^{(3)}
 Author Affiliations

 1. Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University of Illinois, UrbanaChampaign, Urbana, IL, 61801, USA
 2. Department of Computer Science, Washington University, St. Louis, MO, 63130, USA
 3. Synopsys Inc., 700 East Middlefield Road, Mountain View, CA, 94043, USA