Journal of Global Optimization

, Volume 39, Issue 2, pp 197–219 | Cite as

A particle swarm pattern search method for bound constrained global optimization

Original Paper

Abstract

In this paper we develop, analyze, and test a new algorithm for the global minimization of a function subject to simple bounds without the use of derivatives. The underlying algorithm is a pattern search method, more specifically a coordinate search method, which guarantees convergence to stationary points from arbitrary starting points. In the optional search phase of pattern search we apply a particle swarm scheme to globally explore the possible nonconvexity of the objective function. Our extensive numerical experiments showed that the resulting algorithm is highly competitive with other global optimization methods also based on function values.

Keywords

Direct search Pattern search Particle swarm Derivative free optimization Global optimization Bound constrained nonlinear optimization 

AMS Subject Classifications

90C26 90C30 90C56 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alberto P., Nogueira F., Rocha H., Vicente L.N. (2004) Pattern search methods for user-provided points: application to molecular geometry problems. SIAM J. Optim. 14, 1216–1236CrossRefGoogle Scholar
  2. 2.
    Ali M.M., Khompatraporn C., Zabinsky Z.B. (2005) A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J. Global Optim. 31, 635–672CrossRefGoogle Scholar
  3. 3.
    Audet C., Dennis J.E. (2003) Analysis of generalized pattern searches. SIAM J. Optim. 13, 889–903CrossRefGoogle Scholar
  4. 4.
    Audet C., Dennis J.E. (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17, 188–217CrossRefGoogle Scholar
  5. 5.
    Audet C., Orban D. (2006) Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17, 642–664CrossRefGoogle Scholar
  6. 6.
    van den Bergh, F.: An analysis of particle swarm optimizers. Ph.D thesis, Faculty of Natural and Agricultural Science, University of Pretoria (2001)Google Scholar
  7. 7.
    Van Den Berghand F., Engelbrecht A.P. (2006) A study of particle swarm optimization particle trajectories. Inf. Sci. 176, 937–971CrossRefGoogle Scholar
  8. 8.
    Binder A.K., Stauffer A.D. (1985) A simple introduction to Monte Carlo simulations and some specialized topics. In: Binder E.K. (ed) Applications of the Monte Carlo Method in Statistical Physics. Springer, Berlin Heidelberg New York, pp. 1–36Google Scholar
  9. 9.
    Custódio, A.L., Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. (2007, in press)Google Scholar
  10. 10.
    Davis C. (1954) Theory of positive linear dependence. Am. J. Math. 76, 733–746CrossRefGoogle Scholar
  11. 11.
    Dolan E.D., Moré J.J. (2002) Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213CrossRefGoogle Scholar
  12. 12.
    Eberhart, R., Kennedy, J.: New optimizers using particle swarm theory. In: Proceedings of the 1995 6th International Symposium on Micro Machine and Human Science, pp. 39–43, Nagoya, Japan. IEEE Service Center, Piscata way, NJ (1995)Google Scholar
  13. 13.
    Finkel, D.E.: DIRECT Optimization Algorithm User Guide. North Carolina State University (2003) http://www4.ncsu.edu/~definkel/research/index.htmlGoogle Scholar
  14. 14.
    Fourer R., Gay D.M., Kernighan B.W. (1990) A modeling language for mathematical programming. Manage. Sci. 36, 519–554CrossRefGoogle Scholar
  15. 15.
    Gay, D.M.: Hooking your solver to AMPL. Numerical Analysis Manuscript 93-10, AT&T Bell Laboratories (1993) http://www.ampl.comGoogle Scholar
  16. 16.
    Hart W.E. (2003) Locally-adaptive and memetic evolutionary pattern search algorithms. Evol. Comput. 11, 29–52CrossRefGoogle Scholar
  17. 17.
    Hedar A.-R., Fukushima M. (2004) Heuristic pattern search and its hybridization with simulated annealing for nonlinear global optimization. Optim. Methods Softw. 19, 291–308CrossRefGoogle Scholar
  18. 18.
    Hough P., Kolda T.G., Torczon V. (2001) Asynchronous parallel pattern search for nonlinear optimization. SIAM J. Sci. Comput. 23, 134–156CrossRefGoogle Scholar
  19. 19.
    Huyer, W., Neumaier, A.: Global optimization by multilevel coordinate search. J. Global Optim. 14, 331–355 (1999) http://solon.cma.univie.ac.at/~neum/software/mcsGoogle Scholar
  20. 20.
    Ingber, L.: Adaptative simulated annealing (ASA): lessons learned. Control Cybern. 25, 33–54 (1996) http://www.ingber.comGoogle Scholar
  21. 21.
    Ingber L., Rosen B. (1992) Genetic algorithms and very fast simulated reannealing: a comparison. Math. Comput. Model. 16, 87–100CrossRefGoogle Scholar
  22. 22.
    James F. (1990) A review of pseudorandom number generators. Comput. Phys. Commun. 60, 329–344CrossRefGoogle Scholar
  23. 23.
    Jones D.R., Perttunen C.D., Stuckman B.E. (1993) Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79, 157–181CrossRefGoogle Scholar
  24. 24.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Networks, pp. 1942–1948, Perth, Australia. IEEE Service Center, Piscataway, NJ (1995)Google Scholar
  25. 25.
    Kiseleva E., Stepanchuk T. (2003) On the efficiency of a global non-differentiable optimization algorithm based on the method of optimal set partitioning. J. Global Optim. 25, 209–235CrossRefGoogle Scholar
  26. 26.
    Kolda T.G., Lewis R.M., Torczon V. (2003) Optimization by direct search: new prespectives on some classical and modern methods. SIAM Rev. 45, 385–482CrossRefGoogle Scholar
  27. 27.
    Levine, D.: Users guide to the PGAPack parallel genetic algorithm library. Technical Report ANL-95/18, Argonne National Laboratory (1996) http://www.mcs.anl.gov/pgapack.htmlGoogle Scholar
  28. 28.
    Locatelli M. (2003) A note on the Griewank test function. J. Global Optim. 25, 169–174CrossRefGoogle Scholar
  29. 29.
    Locatelli M., Schoen F. (2002) Fast global optimization of difficult Lennard-Jones clusters. Comput. Optim. Appl. 21, 55–70CrossRefGoogle Scholar
  30. 30.
    Marsden, A.L.: Aerodynamic noise control by optimal shape design. Ph.D thesis, Stanford University (2004)Google Scholar
  31. 31.
    Meza J.C., Martinez M.L. (1994) On the use of direct search methods for the molecular conformation problem. J. Comput. Chem. 15, 627–632CrossRefGoogle Scholar
  32. 32.
    Mongeau M., Karsenty H., Rouzé V., Hiriart-Urruty J.-B. (2000) Comparison of public-domain software for black box global optimization. Optim. Methods Softw. 13, 203–226CrossRefGoogle Scholar
  33. 33.
    Park S.K., Miller K.W. (1988) Random number generators: good ones are hard to find. Commun. ACM 31, 1192–1201CrossRefGoogle Scholar
  34. 34.
    Parsopoulos, K.E., Plagianakos, V.P., Magoulas, G.D., Vrahatis, M.N.: Stretching technique for obtaining global minimizers through particle swarm optimization. In: Proceedings of the Particle Swarm Optimization Workshop, pp. 22–29, Indianapolis, USA (2001)Google Scholar
  35. 35.
    Schutte J.F., Groenwold A.A. (2003) A study of global optimization using particle swarms. J. Global Optim. 31(1): 93–108CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  1. 1.Departamento de Produção e Sistemas, Escola de EngenhariaUniversidade do MinhoBragaPortugal
  2. 2.Departamento de MatemáticaUniversidade de CoimbraCoimbraPortugal

Personalised recommendations