Skip to main content
SpringerLink
Log in

Trace-Based Methods for Solving Nonlinear Global Optimization and Satisfiability Problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper we present a method called NOVEL (Nonlinear Optimization via External Lead) forsolving continuous and discrete global optimization problems. NOVEL addresses the balance between global search and local search, using a trace to aid in identifying promising regions before committing to local searches. We discuss NOVEL for solving continuous constrained optimization problems and show how it can be extended to solve constrained satisfaction and discrete satisfiability problems. We first transform the problem using Lagrange multipliers into an unconstrained version. Since a stable solution in a Lagrangian formulation only guarantees a local optimum satisfying the constraints, we propose a global search phase in which an aperiodic and bounded trace function is added to the search to first identify promising regions for local search. The trace generates an information-bearing trajectory from which good starting points are identified for further local searches. Taking only a small portion of the total search time, this elegant approach significantly reduces unnecessary local searches in regions leading to the same local optimum. We demonstrate the effectiveness of NOVEL on a collection of continuous optimization benchmark problems, finding the same or better solutions while satisfying the constraints. We extend NOVEL to discrete constraint satisfaction problems (CPSs) by showing an efficient transformation method for CSPs and the associated representation in finite-difference equations in NOVEL. We apply NOVEL to solve Boolean satisfiability instances in circuit fault detection and circuit synthesis applications, and show comparable performance when compared to the best existing method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. E. Aarts and J. Korst. Simulated Annealing and Boltzmann Machines. J. Wiley and Sons, 1989.

  2. E. H. L. Aarts and P. J. van Laarhoven. Simulated Annealing:Theory and Practice. Wiley, New York, 1987.

    Google Scholar 

  3. F. Aluffi-Pentini, V. Parisi, and F. Zirilli. Global optimization and stochastic differential equations. Journal of Optimization Theory and Applications, 47(1):1–16, September 1985.

    Google Scholar 

  4. N. Anderson and G.Walsh. A graphical method for a class of Branin trajectories. Journal of Optimization Theory and Applications, 49(3):367–374, June 1986.

    Google Scholar 

  5. D. H. Ballard, C. O, Jelinek, and R. Schinzinger. An algorithm for the solution of constrained polynomial programming problems. The Computer Journal, 17:261–266, 1974.

    Google Scholar 

  6. W. Baritompa. Accelerations for a variety of global optimization methods. Journal of Global Optimization, 4:37–45, 1994.

    Google Scholar 

  7. W. Baritompa and A. Cutler. Accelarations for global optimization covering methods using second derivatives. Journal of Global Optimization, 4:329–341, 1994.

    Google Scholar 

  8. B. Betrò and F. Schoen. Sequential stopping rules for the multistart algorithm in global optimization. Mathematical Programming, 38, 1987.

  9. B. Betrò and F. Schoen. Optimal and sub-optimal stopping rules for the multistart algorithm in global optimization. Mathematical Programming, 57:445–458, 1992.

    Google Scholar 

  10. C. G. E. Boender and A. H. G. Rinnooy Kan. Bayesian stopping rules for multi-start global optimization methods. Mathematical Programming. 36, 1987.

  11. C. G. E. Boender and A. H. G. Rinnooy Kan. On when to stop sampling for the maximum. Journal of Global Optimization, 1(4):331–340, 1991.

    Google Scholar 

  12. C. G. E. Boender, A. H. G. Rinnooy Kan, L. Stougie, and G. T. Timmer. A stochastic method for global optimization. Mathematical Programming, 22:125–140, 1982.

    Google Scholar 

  13. I. O. Bohachevsky, M. E. Johnson, and M. L. Stein. Generalized simulated annealing for function optimization. Technometrics, 28:209–217, 1986.

    Google Scholar 

  14. V. Cerny. Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Application, 45:41–51, 1985.

    Google Scholar 

  15. B. C. Cetin, J. Barben, and J. W. Burdick. Terminal repeller unconstrained subenergy tunneling (TRUST) for fast global optimization. Journal of Optimization Theory and Applications, 77, April 1993.

  16. A. Corana, M. Marchesi, C. Martini, and S. Ridella. Minimizing multimodal functions of continuous variables with the simulated annealing algorithm. ACM Trans. Math. Software, 13:2–280, 1987.

    Google Scholar 

  17. A. Davenport, E. Tsang, C. Wang, and K. Zhu. Genet: A connectionist architecture for solving constraint satisfaction problems by iterative improvement. In Proc. of the 12 th National Conf. on Artificial Intelligence, pages 325–330, Seattle, WA, 1994.

  18. M. Davis and H. Putnam. A computing procedure for quantification theory. J. Assoc. Comput. Mach., 7:201–215, 1960.

    Google Scholar 

  19. I. Diener and R. Schaback. An extended continuous Newton method. Journal of Optimization Theory and Applications, 67(1):57–77, October 1990.

    Google Scholar 

  20. L. C. W. Dixon and G. P. Szegö, editors, Towards global optimization 2. North-Holland Pub. Co., Amsterdam, The Netherlands., 1978.

    Google Scholar 

  21. Y. G. Evtushenko, M. A. Potapov, and V. V. Korotkich. Numerical methods for global optimization. In C. A. Floundas and P. M. Pardalos, editors, Recent Advances in Global Optimization, pages 274–297, Princeton University Press, 1992.

  22. C. Fleurent and J. A. Ferland. Object-oriented implementation of heuristic search methods for graph coloring, maximum clique, and satisfiability. In M. A. Trick and D. S. Johnson, editors, The second DIMACS Challenge special issue. 1994.

  23. C. A. Floudas and P. M. Pardalos. A Collection of Test Problems for Constrained Global Optimization Algorithms, volume 455 of Lecture Notes in Computer Science. Springer-Verlag, 1990.

  24. C. A. Floudas and P. M. Pardalos, editors. Recent Advances in GLobal Optimization. Princeton University Press, 1992.

  25. D. B. Fogel. An introduction to simulated evolutionary optimization. IEEE Trans. Neural Networks, 5(1):3–14, January 1994.

    Google Scholar 

  26. R. P. Ge and Y. F. Qin. A class of filled functions for finding global minimizers of a function of several variables. Journal of Optimization Theory and Applications, 54(2):241–252, 1987.

    Google Scholar 

  27. I. Gent and T. Walsh. Towards an understanding of hill-climbing procedures for SAT. In Proc. of the 11th National Conf. on Artificial Intelligence, pages 28–33, Washington, DC, 1993.

  28. A. O. Grienwank. Generalized descent for global optimization. Journal of Optimization Theory and Applications,34:11–39, 1981.

    Google Scholar 

  29. J. Gu., Parrallel Algorithms and Architectures for Very Fast AI Search. PhD thesis, Dept. of Computer Science, University of Utah, August 1989.

  30. J. Gu., How to solve very large-scale satisfiability (VLSS) problems. Technical Report UCECE-TR-90-002, Univ. of Calgary, Canada, October 1990.

    Google Scholar 

  31. J. Gu. Efficient local search for very large-scale satisfiability problems. SIGART Bulletin,3(1);8–12, January 1992.

    Google Scholar 

  32. J. Gu. On optimizing a search problem. In N. G. Bourbakis, editor, Artificial Intelligence Methods and Applications. World Scientific Publishers, 1992.

  33. J. Gu. The UniSAT problem models (appendix). IEEE Trans. on Pattern Analysis and Machine Intelligence 14(8):865, Aug 1992.

    Google Scholar 

  34. J. Gu. Local search for satisfiability (SAT) problems. IEEE Trans. on Syeteme, Man, and Cybernetics, 23(4):1108–1129, 1993.

    Google Scholar 

  35. J. Gu. Global optimization for satisfiability (SAT) problems. IEEE. Trans. on Knowledge and Data Engineering,6(3):361–381, Jun 1994.

    Google Scholar 

  36. J. Gu and Q.-P. Gu., Average time complexities of several local search algorithms for the satisfiability problem (sat). Technical Report UCECE-TR-91-004, Univ, of Calgary, Canada, 1991.

    Google Scholar 

  37. J. Gu and Q.-P. Gu. Average time complexity of the sat1,3 algorithm. Technical report, Tech. Rep., Univ. of Calgary, Canada, 1992.

    Google Scholar 

  38. J. Gu and W. Wang. A novel discrete relaxation architecture. IEEE Trans. on Pattern Analysis and Machine Intelligence,. 14(8):857–865, August 1992.

    Google Scholar 

  39. E. Hansen. Bounding the set of solutions of a perturbed global optimization problem. Journal of Global Optimization, 1(4):359–374, 1991.

    Google Scholar 

  40. E. R. Hansen. Global optimization using interval analysis. M. Dekker, New York, 1992.

    Google Scholar 

  41. L. He and E. Polak. Multistart method with estimation scheme for global satisfying problems. Journal of Global Optimization, 3:130–156, 1993.

    Google Scholar 

  42. A. C. Hindmarsh. ODEPACK, a systematized collection of ode solvers. In R. S. Stepleman et al., editor, scientific computing, pages 55–64. north-holland, amsterdam, 1983.

    Google Scholar 

  43. J. H. Holland. Adaption in Natural and Adaptive Systems. University of Michigan Press, Ann Arbor, 1975.

    Google Scholar 

  44. J. J. Hopfield and D. W. Tank. Neural computation of decisions in optimization problems. Biological Cybernetics, 52:141–152, 1985.

    Google Scholar 

  45. R. Horst and H. Tuv. Global optimization: Deterministic approaches. Springer-Verlag, Berlin, 1993.

    Google Scholar 

  46. Reiner Horst and P. M. Pardalos, editors. Handbook of Global Optimization. Kluwer Academic Publishers, 1994.

  47. L. Ingber. Adaptive Simulated Annealing (ASA).Lester Ingber Reseach, 1995

  48. A. E. W. Jones and G. W. Forbes. An adaptive simulated annealing algorithm for global optimization over continuous variables. Journal of Optimization Theory and Applications,6:1–37, 1995.

    Google Scholar 

  49. A. P. Kamath, N. K. Karmarkar. K. G. Ramakrishnan, and M. G. C. Resende. A continuous approach to inductive inference. {tiMathematical Programming}, 57:215–238, 1992.

  50. S. Kirkpatrick, Jr. C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220(4598):671–680, 1983.

    Google Scholar 

  51. H. J. Kushner. A new method of locating the maximum of an arbitrary multipeak curve in the presence of noise. In Proc. Joint Automatic Control Conf., 1963.

  52. R. W. Beckerand G. W. Lago. A global optimization algorithm. In Proc. of the 8th Allerton Conf. on Circuits and Systems Theory, pages 3–12, Monticello, Illinois, 1970.

  53. S. Lucidi and M. Piccioni. Random tunneling by means of acceptance-rejection sampling for global optimization. Journal of Optimization Theory and Application, 62:255–277, 1989.

    Google Scholar 

  54. D. G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley Publishing Company, 1984.

  55. Z. Michalewicz. Genetic Algorithms + Data Structure = Evolution Programs. Springer-Verlag, 1994.

  56. S. Minton, M. D. Johnson, A. B. Philips, and P. Laird. Minimizing conflicts; a heuristic repair method for constraint satisfaction and scheduling problems. Artificial Intelligence, 8:161–205, Dec 1992.

    Google Scholar 

  57. D. Mitchell, B. Selman, and H. Levesque. Hard and easy distributions of SAT problems. In Proc. of the 1Oth National Conf. on Artificial Optimization,pages 459–465, 1992.

  58. J. Mockus. Bayesian Approach to Global Optimization. Kluwer Academic Publishers, Dordrecht-London-Boston, 1989.

    Google Scholar 

  59. J. Mockus. Application of bayesian approach to numerical methods of global and stochastic optimization. Journal of Global Optimization, {vn4}:347–365, 1994.

    Google Scholar 

  60. R. Moore and E. Hansen and A. Leclere. Rigorous methods for global optimization. In C. A. Floudas and P. M. Pardalos, editors, Recent Advances in Global Optimization,pages 321–342. Princeton University Press, 1992.

  61. P. Morris. The breakout method for escaping from local minima. In Proc. of the 11 th National Conf. on Artificial Intelligence, pages 40–45, Washington, DC, 1993.

  62. P. M. Pardalos. Complexity in numerical optimization. World Scientific, Singapore and River Edge, N J, 1993.

    Google Scholar 

  63. P. M. Pardalos and J. B. Rosen. Constrained Global Optimization: Algorithms and Applications, volume 268 of Lecture Notes in Computer Science. Springer-Verlag, 1987.

  64. N. R. Patel, R. L. Smith, and Z. B. Zabinsky. Pure adaptive search in Monte Carlo optimization. Mathematical Programming, 43:317–328, 1998.

    Google Scholar 

  65. M. Piccioni. A combined multistart-annealing algorithms for continuous global optimization. Technical Report 87-45, Systems and Research Center, The University of Maryland, College Park MD, 1987.

    Google Scholar 

  66. D. Powell and M. M. Skolnick. Using genetic algorithms in engineering design optimization with nonlinear constraints. In S. Forrest, editor, Proc. of the. Fifth Int'l Conf. on Genetic Algorithms, pages 424–431. Morgan Kaufmann, 1993.

  67. H. E. Romeijn and R. L. Smith. Simulated annealing for constrained global optimization. Journal of Global Optimization, 5(2):101–126, September 1994.

    Google Scholar 

  68. R. Rotondi. A new method for global optimization based on the k-th nearest neighbor. Technical report, CNR-IAMI, Milano, 1987.

  69. M. S. Sarma. On the convergence of the Baba and Dorea random optimization methods. Journal of Optimization Theory and Applications, 66:337–343, 1990.

    Google Scholar 

  70. S. Schäffler and H. Warsitz. A trajectory-following method for unconstrained optimization. Journal of Optimization Theory and Applications, 67(1):133–140, October 1990.

    Google Scholar 

  71. I. P. Schagen. Internal modeling of objective functions for global optimization. Journal of Optimization Theory and Applications, 51(2), 1986.

  72. F. Schoen. Stochastic techniques for global optimization: A survey on recent advances. Journal of Global Optimization,1(3):207–228, 1991.

    Google Scholar 

  73. R. Sebastiani. Applying GSAT to non-clausal formulas. Journal of Artificial Intelligence Research, 1:309–314, 1994.

    Google Scholar 

  74. B. Selman, 1995. private communication.

  75. D. Selman and H. Kautz, Domain-independent extensions to GSAT: Solving large structured satisfiability problems. In Proc. of the 13th Int’l Joint Conf. on Artificial Intelligence, pages 290–295, 1993.

  76. B. Selman, H. Kautz, and B. Cohen. Local search strategies for satisfiability testing. In Proc. of the Second DIMACS Challenge Workshops on Cliques, Coloring, and Satisfiability, Rutgers University, pages 290–295, oct 1993.

  77. B. Selman, H. Kautz, and R. Cohen. Noise strategies for improving local search. In Proc. of the 12th National Conf. on Artificial Intelligence, pages 337–343. Seattle, WA, 1994.

  78. B. Selman and H. A. Kautz. An empirical study of greedy local search for satisfiability testing. In Proc. of the 11th National Conf. on Artificial Intelligence, pages 46–51, Washington, DC, 1993.

  79. B. Selman, H. J. Levesque, and D. G. Mitchell. A new method for solving hard satisfiability problems. In Proc. of AAAl-92, pages 440–446, San Jose, CA, 1992.

  80. Y. Shang and B. W. Wah. Global optimization for neural network training. IEEE Computer, 29:45–54, March 1996.

    Google Scholar 

  81. J. A. Snyman and L. P. Fatti. A multi-start global minimization algorithm with dynamic search trajectories. Journal of Optimization Theory and Applications, 54(1):121–141, July 1987.

    Google Scholar 

  82. R. Socic and J. Gu. Fast search algorithms for the N-queen problem. IEEE Trans. on Systems, Man, and Cybernetics, 21(6):1572–1576, November 1991.

    Google Scholar 

  83. R. Sosiš and J. Gu. A polynomial time algorithms for the n-queens problem. SIGART Bulletin, 1(3):7–11, October 1990.

    Google Scholar 

  84. R. Sosiš and J. Gu. 3,000,000 queens in less than one minute. SIGART Bulletin, 2(2):22–24, April 1991.

    Google Scholar 

  85. R. Sosiš and J. Gu. Efficient local search with conflict minimiziation: A case study of the n-queens problem. IEEE Trans. on Knowledge and Data Engineering, 6(5):661–668, 1994.

    Google Scholar 

  86. R. Spaans and R. Luus. Importance of search-domain reduction in random optimization. Journal of Optimization Theory and Applications, 75(3):635–638, December 1992.

    Google Scholar 

  87. H. S. Stone and J. M. Stone. Efficient search techniques-an empirical study of the n-queens problem. IBM J. Res. Dev., 31:464–474, 1987.

    Google Scholar 

  88. B. E. Stuckman. A global search method for optimizing nonlinear systems. IEEE Trans. on Systems, Man, and Cybernetics., 18(6):965–977, 1988.

    Google Scholar 

  89. E. G. Sturua and S. K. Zavriev. A trajectory algorithm based on the gradient method I. the search on the quasioptimal trajectories. Journal of Global Optimization, 1991(4):375–388, 1991.

    Google Scholar 

  90. G. T. Timmer. Global Optimization: A Stochastic Approach. PhD thesis, Erasmus University, Rotterdam, 1984.

  91. A. Törn. Global optimization as a combination of global and local search. Gothenburg Business Adm. Studies, 17:191–206, 1973.

    Google Scholar 

  92. A. Törn. Cluster analysis using seed points and density determined hyperspheres as an aid to global optimization. IEEE Trans. Syst. Men and Cybernetics, 7:610–616, 1977.

    Google Scholar 

  93. A. Törn and S. Viitanen. Topographical global optimization. In C. A. Floudas and P. M. Pardalos, editors, Recent Advances in Global Optimization, pages 385–398, Princeton University Press, 1992.

  94. A. Törn and A. Žilinskas. Global Optimization, Springer-Verlag, 1989.

  95. D. Vanderbilt and S. G. Louie. A Monte Carlo simulated annealing approach to optimization over continuous variables. Journal of Computational Physics, 56:259–271, 1984.

    Google Scholar 

  96. T. L. Vincent, B. S. Goh, and K. L. Teo. Trajectory-following algorithms for min-max optimization problems. Journal of Optimization Theory and Applications,75(3):501–519, December 1992.

    Google Scholar 

  97. A. Žilinskas. A review of statistical models for global optimization. Journal for Global Optimization, 2:145–153, 1992.

    Google Scholar 

  98. B. W. Wah and Y. Shang. A discrete lagrangian-based global-search method for solving satisfiability problems. In Ding-Zhu Du, Jun Gu, and Panos Pardalos, editors, Proc. DIMACS Workshop on Satisfiability Problems: Theory and Applications. American Mathematical Society, March 1996.

  99. B. W. Wah, Y. Shang, T. Wang, and T. Yu. Global optimization design of QMF filter banks. In Proc. IEEE Midwest Symposium on Circuits and Systems, Iowa City, Iowa, (accepted to appear) Aug. 1996. IEEE.

  100. Z. B. Zabinsky and R. L. Smith. Pure adaptive search in global optimization. Mathematical Programming, 53:323–338, 1992.

    Google Scholar 

  101. Z. B. Zabinsky, e.t al. Improving hit-and-run for global optimization. Jonrnal of Global Optimization. 3:171–192, 1993.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering and Coordinated science Laboratory, University of Illinois, Urbana-Cahmpaign, Urbana, IL, 61801, USA

    Benjamin W. Wah & Yao-Jen Chang

Authors
  1. Benjamin W. Wah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yao-Jen Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wah, B.W., Chang, YJ. Trace-Based Methods for Solving Nonlinear Global Optimization and Satisfiability Problems. Journal of Global Optimization 10, 107–141 (1997). https://doi.org/10.1023/A:1008294209959

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008294209959

Search

Navigation