Evolving Smoothing Kernels for Global Optimization

  • Paul Manns
  • Kay Hamacher
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9598)


The Diffusion-Equation Method (DEM) – sometimes synonymously called the Continuation Method – is a well-known natural computation approach in optimization. The DEM continuously transforms the objective function by a (Gaussian) kernel technique to reduce barriers separating local and global minima. Now, the DEM can successfully solve problems of small sizes. Here, we present a generalization of the DEM to use convex combinations of smoothing kernels in Fourier space. We use a genetic algorithm to incrementally optimize the (meta-)combinatorial problem of finding better performing kernels for later optimization of an objective function. For two test applications we derive and show their transferability to larger problems. Most strikingly, the original DEM failed on a number of the test instances to find the global optimum while our transferable kernels – obtained via evolutionary computations – were able to find the global optimum.


Smoothing kernels Diffusion-equation method Fourier space 



KH gratefully acknowledges funding by the LOEWE project compuGene of the Hessen State Ministry of Higher Education, Research and the Arts.


  1. 1.
    Piela, L., Kostrowicki, J., Scheraga, H.A.: On the multiple-minima problem in the conformational analysis of molecules: deformation of the potential energy hypersurface by the diffusion equation method. J. Phys. Chem. 93(8), 3339–3346 (1989)CrossRefGoogle Scholar
  2. 2.
    Wawak, R.J., Pillardy, J., Liwo, A., Gibson, K.D., Scheraga, H.A.: Diffusion equation and distance scaling methods of global optimization. J. Phys. Chem. A 102(17), 2904 (1998)CrossRefGoogle Scholar
  3. 3.
    Mobahi, H., Fisher III, J.W.: On the link between Gaussian homotopy continuation and convex envelopes. In: Tai, X.-C., Bae, E., Chan, T.F., Lysaker, M. (eds.) EMMCVPR 2015. LNCS, vol. 8932, pp. 43–56. Springer, Heidelberg (2015)Google Scholar
  4. 4.
    Mobahi, H., Fisher III, J.W.: A theoretical analysis of optimization by Gaussian continuation. In: AAAI 2015: 29th Conference on Artificial Intelligence (2015)Google Scholar
  5. 5.
    Kostrowicki, J., Piela, L., Cherayil, B.J., Scheraga, H.A.: Performance of the diffusion equation method in searches for optimum structures of clusters of Lennard-Jones atoms. J. Phys. Chem. 95(10), 4113–4119 (1991)CrossRefGoogle Scholar
  6. 6.
    Wawak, R., Wimmer, M., Scheraga, H.: Application of the diffusion equation method of global optimization to water clusters. J. Phys. Chem. 96(12), 5138–5145 (1992)CrossRefGoogle Scholar
  7. 7.
    Hamacher, K.: Optimization of high-dimensional functions with respect to protein-structure-prediction. Dipl. thesis, Dortmund University (1998)Google Scholar
  8. 8.
    Galassi, M.: GNU Scientific Library Reference Manual. Network Theory Ltd. (2009)Google Scholar
  9. 9.
    Miklavcic, M.: Applied Functional Analysis and PDEs. World Scientific, Singapore (1998)zbMATHGoogle Scholar
  10. 10.
    Abels, H.: Pseudodifferential and Singular Integral Operators. Walter de Gruyter, Berlin (2011)CrossRefzbMATHGoogle Scholar
  11. 11.
    Marks, R.J.: Handbook of Fourier Analysis & Its Applications. Oxford University Press, Oxford (2009)zbMATHGoogle Scholar
  12. 12.
    Aluffi-Pentini, F., Parisi, V., Zirilli, F.: Global optimization and stochastic differential equations. J. Optim. Theor. Appl. 47(1), 1–16 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Blum, C., Roli, A.: Hybrid metaheuristics: an introduction. In: Blum, C., José Bleasa Aguilera, M., Roli, A., Sampels, M. (eds.) Hybrid Metaheuristics, pp. 1–30. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Kruse, R., Borgelt, C., Klawonn, F., Moewes, C., Steinbrecher, M., Held, P.: Computational Intelligence: Eine methodische Einführung in Künstliche Neuronale Netze, Evolutionäre Algorithmen, Fuzzy-Systeme und Bayes-Netze. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Mitchell, M.: An Introduction to Genetic Algorithms. Bradford, Denver (1998)zbMATHGoogle Scholar
  16. 16.
    Yu, X., Gen, M.: Decision Engineering. Springer, Heidelberg (2010)Google Scholar
  17. 17.
    Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations. Graduate Texts in Mathematics, vol. 194. Springer, Heidelberg (2000)zbMATHGoogle Scholar
  18. 18.
    Engel, K.J., Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations. Semi-group Forum, vol. 63. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  19. 19.
    Mayer, B.E., Hamacher, K.: Stochastic tunneling transformation during selection in genetic algorithm. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 801–806. ACM, New York (2014)Google Scholar
  20. 20.
    Nocedal, J., Wright, S.: Numerical Optimization. Operations Research and Financial Engineering. Springer, New York (2006)zbMATHGoogle Scholar
  21. 21.
    R Development Core Team.: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2008)Google Scholar
  22. 22.
    Dimitriadou, E., Hornik, K., Leisch, F., Meyer, D., Weingessel, A.: Misc functions of the Department of Statistics (e1071), TU Wien. R package 1–5 (2008)Google Scholar
  23. 23.
    Hopfinger, A.: Conformational Properties of Macromolecules. Academic Press, London (1973)Google Scholar
  24. 24.
    Wales, D.J., Doye, J.P.: Global optimization by basin-hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms. J. Phys. Chem. A 101(28), 5111–5116 (1997)CrossRefGoogle Scholar
  25. 25.
    Wagner, P.: A new constructive proof of the Malgrange-Ehrenpreis theorem. Am. Math. Month. 116(5), 457–462 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Diosan, L., Rogozan, A., Pecuchet, J.P.: Evolving kernel functions for svms by genetic programming. In: Sixth International Conference on Machine Learning and Applications, ICMLA 2007, 19–24 December 2007Google Scholar
  27. 27.
    Sullivan, K.M., Luke, S.: Evolving kernels for support vector machine classification. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1702–1707. ACM, New York (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceTU DarmstadtDarmstadtGermany
  2. 2.Department of BiologyTU DarmstadtDarmstadtGermany
  3. 3.Department of PhysicsTU DarmstadtDarmstadtGermany

Personalised recommendations