Soft Computing

, Volume 21, Issue 21, pp 6331–6350 | Cite as

Gradient subspace approximation: a direct search method for memetic computing

  • Oliver Schütze
  • Sergio Alvarado
  • Carlos Segura
  • Ricardo Landa
Methodologies and Application

Abstract

The hybridization of evolutionary algorithms and local search techniques as, e.g., mathematical programming techniques, also referred to as memetic algorithms, has caught the interest of many researchers in the recent past. Reasons for this include that the resulting algorithms are typically robust and reliable since they take the best of both worlds. However, one crucial drawback of such hybrids is the relatively high cost of the local search techniques since many of them require the gradient or even the Hessian at each candidate solution. Here, we propose an alternative way to compute search directions by exploiting the neighborhood information. That is, for a given point within a population \(\mathcal{P}\), the neighboring solutions in \(\mathcal{P}\) are used to compute the most greedy search direction out of the given data. The method is hence particularly interesting for the usage within population-based search strategies since the search directions come ideally for free in terms of additional function evaluations. In this study, we analyze the novel method first as a stand-alone algorithm and show further on its benefit as a local searcher within differential evolution.

Keywords

Scalar optimization Direct search Memetic computing Evolutionary algorithms Differential evolution 

References

  1. Auger A, Hansen N, Zerpa Perez J M, Ros R, Schoenauer M (2009) Experimental comparisons of derivative free optimization algorithms. In: Experimental algorithms, Springer, pp 3–15Google Scholar
  2. Bäck T, Schwefel HP (1993) An overview of evolutionary algorithms for parameter optimization. Evol Comput 1(1):1–23CrossRefGoogle Scholar
  3. Bao Y, Hu Z, Xiong T (2013) A PSO and pattern search based memetic algorithm for SVMs parameters optimization. Neurocomputing 117:98–106Google Scholar
  4. Beyer HG, Finck S (2012) Happycat a simple function class where well-known direct search algorithms do fail. In: Coello Coello CA, et al., (ed) Parallel problem solving from nature—PPSN XII, vol 7491 of Lecture Notes in Computer Science, Springer, Berlin, pp 367–376Google Scholar
  5. Beyer HG, Schwefel HP (2002) Evol strategies: a comprehensive introduction Nat Comput 1(1):3–52Google Scholar
  6. Brent RP (1973) Algorithms for minimization without derivatives, 1st edn. Prentice-Hall, Upper Saddle River, NJMATHGoogle Scholar
  7. Brown M, Smith RE (2005) Directed multi-objective optimisation. Int J Comput Syst Signals 6(1):3–17Google Scholar
  8. Caraffini F, Neri F, Iacca G (2013) Parallel memetic structures. Inf Sci 227:60–82MathSciNetCrossRefGoogle Scholar
  9. Caraffini F, Neri F, Picinali L (2014) An analysis on separability for memetic computing automatic design. Inf Sci 225:1–22MathSciNetCrossRefGoogle Scholar
  10. Chao G, Detong Z (2011) A secant algorithm with line search filter method for nonlinear optimization. Appl Math Model 35(2):879–894MathSciNetCrossRefMATHGoogle Scholar
  11. Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–31CrossRefGoogle Scholar
  12. Dennis JE, Schnabel RB (1983) Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall, Upper Saddle RiverMATHGoogle Scholar
  13. Domínguez IS, Aguirre AH, Valdez SI (2014) A new EDA by a gradient-driven density. In: Parallel problem solving from nature—PPSN XIII—13th international conference, pp 352–361Google Scholar
  14. Durillo JJ, Nebro AJ, Coello Coello CA, Garcia-Nieto J, Luna F, Alba E (2010) A study of multiobjective metaheuristics when solving parameter scalable problems. IEEE Trans Evol Comput 14(4):618–635CrossRefMATHGoogle Scholar
  15. Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, NewYorkCrossRefMATHGoogle Scholar
  16. Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Natural computing series. Springer, NewYorkCrossRefMATHGoogle Scholar
  17. Gong W, Cai Z, Ling CX (2006) ODE: a fast and robust differential evolution based on orthogonal design. In: Sattar A, Kang BH (eds) AI 2006: advances in artificial intelligence, vol 4304. Lecture Notes in Computer Science, Springer, Berlin, pp 709–718Google Scholar
  18. Griewank A (2000) Evaluating derivatives: principles and techniques of algorithmic differentiation. Number 19 in Frontiers in Applied Mathematics SIAM, Philadelphia, PAGoogle Scholar
  19. Hazen M, Gupta MR (2006) A multiresolutional estimated gradient architecture for global optimization. In: IEEE international conference on evolutionary computation, CEC, pp 3013–3020Google Scholar
  20. Hooke R, Jeeves TA (1961) Direct search solution of numerical and statistical problems. J ACM 8(2):212–229CrossRefMATHGoogle Scholar
  21. Junhua Z, Yan X, Luo L, ZhaoYang D, Yaoyao P (2014) Power system fault diagnosis based on history driven differential evolution and stochastic time domain simulation. Inf Sci 275:13–29MathSciNetCrossRefGoogle Scholar
  22. Kleijnen J P C (2015) Response surface methodology. In: Fu MC (ed) Handbook of simulation optimization, vol 216 of international series in operations research & management science, Springer, New York, pp 81–104Google Scholar
  23. Kukkonen S, Lampinen J (2006) Constrained real-parameter optimization with generalized differential evolution. In: Evolutionary computation, 2006. CEC 2006. IEEE Congress on IEEE, pp 207–214Google Scholar
  24. Lara A, Sanchez G, Coello Coello CA, Schütze O (2010) HCS: a new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans Evol Comput 14(1):112–132CrossRefGoogle Scholar
  25. LaTorre A (2009) A framework for hybrid dynamic evolutionary algorithms: multiple offspring sampling (MOS). Ph.D. thesisGoogle Scholar
  26. LaTorre A, Muelas S, Pena JM (2011) A mos-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test. Soft Comput 15(11):2187–2199CrossRefGoogle Scholar
  27. Li X, Yao X (2012) Cooperatively coevolving particle swarms for large scale optimization. Evol Comput IEEE Trans 16(2):210–224MathSciNetCrossRefGoogle Scholar
  28. Liang JJ, Runarsson TP, Mezura-Montes E, Clerc M, Suganthan PN, Coello Coello CA, Deb K (2006) Problem definitions and evaluation criteria for the cec 2006 special session on constrained real-parameter optimization. Technical report, Nanyang Technological University, SingaporeGoogle Scholar
  29. Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms. Technical report C3P Report 826, California Institute of TechnologyGoogle Scholar
  30. Neri F, Cotta C, Moscato P (eds) (2012) Handbook of memetic algorithms, vol 379 of Studies in Computational Intelligence. SpringerGoogle Scholar
  31. Nocedal J, Wright S (2006) Numerical optimization. Springer series in operations research and financial engineering. Springer, NewYorkGoogle Scholar
  32. Omidvar MN, Mei Y, Li X (2014) Effective decomposition of large-scale separable continuous functions for cooperative co-evolutionary algorithms. In: Evolutionary computation (CEC), 2014 IEEE congress on IEEE, pp 1305–1312Google Scholar
  33. Osyczka A, Krenich S (2006) Evolutionary algorithms for global optimization. In: Pintér JD (ed) Global optimization, vol 85. Springer, NewYork, pp 267–300CrossRefGoogle Scholar
  34. Polak E, Mayne DQA Robust secant method for optimization problems with inequality constraints. J Optim Theory Appl 33(4):463–477Google Scholar
  35. Qin AK, Suganthan PN (2005) Self-adaptive differential evolution algorithm for numerical optimization. In IEEE congress on evolution computation 2005 (CEC’05), vol 2, pp 1785–1791Google Scholar
  36. Schütze O, Martín A, Lara A, Alvarado S, Salinas E, Coello Coello CA (2015) The directed search method for multi-objective memetic algorithms. Comput Optim Appl 63(2):1–28Google Scholar
  37. Schwefel HP (1993) Evolution and optimum seeking. Wiley, New York, NYMATHGoogle Scholar
  38. Shiwen Y, Anyong Q (2005) Design of high-power millimeter-wave TM\(_{01}\) - TE\(_{11}\) mode converters by the differential evolution algorithm. IEEE Trans Plasma Sci 33(4):1372–1376CrossRefGoogle Scholar
  39. Sivanandam SN, Deepa SN (2007) Introduction to genetic algorithms. Springer, NewYorkMATHGoogle Scholar
  40. Storn R, Price K (1995) Differential evolution: a simple and efficient adaptive scheme for global optimization over continuous spaces. Technical report, International Computer Science Institute, Berkeley, Technical report TR95012Google Scholar
  41. Talbi EG (2002) A taxonomy of hybrid metaheuristics. J. Heuristics 8(5):541–564CrossRefGoogle Scholar
  42. Tseng L-Y, Chen C (2008) Multiple trajectory search for large scale global optimization. In: Evolutionary computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence). IEEE Congress, pp 3052–3059Google Scholar
  43. Zapotecas Martínez S, Coello Coello CA, (2012) A direct local search mechanism for decomposition-based multi-objective evolutionary algorithms. (2012). In: IEEE congress on evolutionary computation (CEC’2012). IEEE Press, BrisbaneGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Oliver Schütze
    • 1
  • Sergio Alvarado
    • 1
  • Carlos Segura
    • 2
  • Ricardo Landa
    • 3
  1. 1.Computer Science DepartmentCinvestav-IPNMexico CityMexico
  2. 2.Centro de Investigación en MatemáticasGuanajuatoMexico
  3. 3.Information Technology LaboratoryCinvestav-IPN TamaulipasVictoriaMexico

Personalised recommendations