Dynamic Optimization Using Analytic and Evolutionary Approaches: A Comparative Review

  • Hendrik Richter
  • Shengxiang Yang
Part of the Intelligent Systems Reference Library book series (ISRL, volume 38)


Solving a dynamic optimization problem means that the obtained results depend explicitly on time as a parameter. There are two major branches in which dynamic optimization occurs: (i) in dynamic programming and optimal control, and (ii) in dynamic fitness landscapes and evolutionary computation. In both fields, solving such problems is established practice while at the same time special and advanced aspects are still subject of research. In this chapter, we intend to give a comparative study of the two branches of dynamic optimization. We review both problem settings, define them, and discuss approaches for and issues in solving them. The main focus here is to highlight the connections and parallels. In particular, we show that optimal control problems can be understood as dynamic fitness landscapes, where for linear systems this relationship can even be expressed analytically.


Search Space Optimal Control Problem Evolutionary Computation Evolutionary Approach Mass Extinction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ahmed–Ali, T., Mazenc, F., Lamnabhi–Lagarrigue, F.: Disturbance attenuation for discrete-time feedforward nonlinear systems. In: Aeyels, D., Lamnabhi–Lagarrigue, F., van der Schaft, A. (eds.) Stability and Stabilization of Nonlinear Systems, pp. 1–17. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  2. 2.
    Al–Tamimi, A., Lewis, F.L., Abu-Khalaf, M.: Discrete–time nonlinear HJB solution using approximate dynamic programming: Convergence proof. IEEE Trans Syst., Man, & Cybern. Part B: Cybern. 38, 943–949 (2008)CrossRefGoogle Scholar
  3. 3.
    Al–Tamimi, A., Abu-Khalaf, M., Lewis, F.L.: Heuristic dynamic programming nonlinear optimal controller. In: Mellouk, A., Chebira, A. (eds.) Machine Learning, pp. 361–380. InTech, Rijeka (2009)Google Scholar
  4. 4.
    Arnold, D.V., Beyer, H.G.: Optimum tracking with evolution strategies. Evol. Comput. 14, 291–308 (2006)CrossRefGoogle Scholar
  5. 5.
    Artstein, Z.: Stabilization with relaxed controls. Nonlinear Anal. 7, 1163–1173 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Bäck, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, New York (1996)zbMATHGoogle Scholar
  7. 7.
    Beard, R.W., Saridis, G.N.: Approximate solutions to the time–invariant Hamilton–Jacobi–Bellman equation. J. Optim. Theory Appl. 96, 589–626 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Bellman, R.E.: Dynamic Programming, p. 1957. Princeton University Press, Princeton (2010)zbMATHGoogle Scholar
  9. 9.
    Bendtsen, C.N., Krink, T.: Dynamic memory model for non–stationary optimization. In: Fogel, D.B., El–Sharkawi, M.A., Yao, X., Greenwood, G., Iba, H., Marrow, P.I., Shackleton, M. (eds.) Proc. Congress on Evolutionary Computation, IEEE CEC 2002, pp. 145–150. IEEE Press, Piscataway (2002)Google Scholar
  10. 10.
    Benton, M.J.: When Life Nearly Died–The Greatest Mass Extinction of All Time. Thames & Hudson, London (2003)Google Scholar
  11. 11.
    Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 1. Athena Scientific, Belmont (2005)zbMATHGoogle Scholar
  12. 12.
    Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 2. Athena Scientific, Belmont (2007)Google Scholar
  13. 13.
    Betts, J.T.: Practical Methods for Optimal Control using Nonlinear Programming. SIAM, Philadelphia (2001)zbMATHGoogle Scholar
  14. 14.
    Bobbin, J., Yao, X.: Solving optimal control problems with a cost on changing control by evolutionary algorithms. In: Bäck, T., Michalewicz, Z., Yao, X. (eds.) Proc. 1997 IEEE International Conference on Evolutionary Computation (ICEC 1997), pp. 331–336. IEEE Press, Piscataway (1997)CrossRefGoogle Scholar
  15. 15.
    Borenstein, E., Meilijson, I., Ruppin, E.: The effect of phenotypic plasticity on evolution in multipeaked fitness landscapes. Jour. Evolut. Biology 19, 1555–1570 (2006)CrossRefGoogle Scholar
  16. 16.
    Bosman, P.A.N.: Learning and anticipation in online dynamic optimization. In: Yang, S., Ong, Y.S., Jin, Y. (eds.) Evolutionary Computation in Dynamic and Uncertain Environments, pp. 129–152. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Boumaza, A.M.: Learning environment dynamics from self-adaptation. In: Yang, S., Branke, J. (eds.) GECCO Workshops 2005, pp. 48–54 (2005)Google Scholar
  18. 18.
    Branke, J.: Memory enhanced evolutionary algorithms for changing optimization problems. In: Angeline, P.J., Michalewicz, Z., Schoenauer, M., Yao, X., Zalzala, A. (eds.) Proc. Congress on Evolutionary Computation, IEEE CEC 1999, pp. 1875–1882. IEEE Press, Piscataway (1999)Google Scholar
  19. 19.
    Branke, J.: Evolutionary Optimization in Dynamic Environments. Kluwer Academic Publishers, Dordrecht (2001)Google Scholar
  20. 20.
    Branke, J., Kaußler, T., Schmidt, C., Schmeck, H.: A multi–population approach to dynamic optimization problems. In: Parmee, I.C. (ed.) Proc. of the 4th Int. Conf. on Adaptive Computing in Design and Manufacturing, pp. 299–308 (2000)Google Scholar
  21. 21.
    Bui, L.T., Branke, J., Abbass, H.A.: Diversity as a selection pressure in dynamic environments. In: Beyer, H.G., O’Reilly, U.M. (eds.) Proc. Genetic and Evolutionary Computation Conference (GECCO 2005), pp. 1557–1558. ACM Press, Seattle (2005)CrossRefGoogle Scholar
  22. 22.
    Chen, Z., Jagannathan, S.: Generalized Hamilton–Jacobi–Bellman formulation based neural network control of affine nonlinear discrete-time systems. IEEE Trans. Neural Networks 19, 90–106 (2008)CrossRefGoogle Scholar
  23. 23.
    Cobb, H.G.: An investigation into the use of hypermutation as an adaptive operator in genetic algorithms having continuouis, time–dependent nonstationary environments. Technical Report AIC-90-001, Naval Research Laboratory, Washington, USA (1990),
  24. 24.
    Defaweux, A., Lenaerts, T., van Hemert, J., Parent, J.: Complexity transitions in evolutionary algorithms: evaluating the impact of the initial population. In: Corne, D. (ed.) Proc. Congress on Evolutionary Computation, IEEE CEC 2005, pp. 2174–2181. IEEE Press, Piscataway (2005)CrossRefGoogle Scholar
  25. 25.
    The Darwin Correspondence Project, (retrieved July 08, 2011)
  26. 26.
    den Boer, P.J.: Natural selection or the non–survival of the non–fit. Acta Biotheoretica 47, 83–97 (1999)CrossRefGoogle Scholar
  27. 27.
    Drezewski, R., Siwik, L.: Agent–based multi–objective evolutionary algorithm with sexual selection. In: Wang, J., Liu, D., Feng, G., Michalewicz, Z. (eds.) Proc. 2008 IEEE Congress on Evolutionary Computation, pp. 3679–3684. IEEE Press, Piscataway (2008)Google Scholar
  28. 28.
    Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer, Heidelberg (2003)zbMATHGoogle Scholar
  29. 29.
    Fogel, D.B.: Applying evolutionary programming to selected control problems. Computers & Mathematics with Applications 27, 89–104 (1994)zbMATHCrossRefGoogle Scholar
  30. 30.
    Franks, S.J., Sim, S., Weis, A.E.: Rapid evolution of flowering time by an annual plant in response to a climate fluctuation. Proc. Natl. Acad. Sci. USA (PNAS) 104, 1278–1282 (2007)CrossRefGoogle Scholar
  31. 31.
    Freeman, R.A., Kokotovic, P.V.: Inverse optimality in robust stabilization. SIAM J. Control Optim. 34, 1365–1391 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Futuyma, D.J.: Evolution. Sinauer Associates, Sunderland (2005)Google Scholar
  33. 33.
    Grefenstette, J.J.: Genetic algorithms for changing environments. In: Männer, R., Manderick, B. (eds.) Parallel Problem Solving from Nature–PPSN II, pp. 137–144. North Holland, Amsterdam (1992)Google Scholar
  34. 34.
    Haddad, W.M., Chellaboina, V.: Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach. Princeton University Press, Princeton (2008)zbMATHGoogle Scholar
  35. 35.
    Haddad, W.M., Chellaboina, V.: Discrete–time nonlinear analysis and feedback control with nonquadratic performance criteria. J. Franklin Inst. 333B, 849–860 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Haddad, W.M., Chellaboina, V., Fausz, J.L., Abdallah, C.T.: Optimal discrete–time control for nonlinear cascade systems. J. Franklin Inst. 335B, 827–839 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  37. 37.
    Hoffmann, A.A., Will, Y.: Detecting genetic responses to environmental change. Nat. Rev. Genet. 9, 421–432 (2008)CrossRefGoogle Scholar
  38. 38.
    Hu, X., Eberhart, R.C.: Adaptive particle swarm optimization: detection and response to dynamic systems. In: Fogel, D.B., El–Sharkawi, M.A., Yao, X., Greenwood, G., Iba, H., Marrow, P.I., Shackleton, M. (eds.) Proc. 2002 IEEE Congress on Evolutionary Computation, pp. 1666–1670. IEEE Press, Piscataway (2002)Google Scholar
  39. 39.
    Jablonka, E., Oborny, B., Molnar, E., Kisdi, E., Hofbauer, J., Czaran, T.: The adaptive advantage of phenotypic memory. Philosophical Transactions of the Royal Society, London B350, 133–141 (1995)CrossRefGoogle Scholar
  40. 40.
    Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments – A survey. IEEE Trans. Evolut. Comput. 9, 303–317 (2005)CrossRefGoogle Scholar
  41. 41.
    Kashtan, N., Noor, E., Alon, U.: Varying environments can speed up evolution. Proc. Natl. Acad. Sci. USA (PNAS) 104, 13711–13716 (2007)CrossRefGoogle Scholar
  42. 42.
    Kirschner, M.W., Gerhart, J.C.: The Plausibility of Life: Resolving Darwin’s Dilemma. Yale Univ. Press, New Haven (2005)Google Scholar
  43. 43.
    Levins, R.: Evolution in Changing Environments. Princeton University Press, Princeton (1968)Google Scholar
  44. 44.
    Li, X., Yong, J.: Optimal Control Theory for Infinite Dimensional Systems. Birkhäuser, Boston (1995)CrossRefGoogle Scholar
  45. 45.
    Lewis, J., Hart, E., Ritchie, G.: A Comparison of Dominance Mechanisms and Simple Mutation on Non-stationary Problems. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H.-P. (eds.) PPSN 1998. LNCS, vol. 1498, pp. 139–148. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  46. 46.
    Lis, J., Eiben, A.E.: A multi–sexual genetic algorithm for multiobjective optimization. In: Fukuda, T., Furuhashi, T. (eds.) Proc. 3rd IEEE Conference on Evolutionary Computation, pp. 59–64. IEEE Press, Piscataway (1996)Google Scholar
  47. 47.
    Lopez Cruz, I.L., Van Willigenburg, L.G., Van Straten, G.: Efficient Differential Evolution algorithms for multimodal optimal control problems. Applied Soft Computing 3, 97–122 (2003)CrossRefGoogle Scholar
  48. 48.
    McElwain, J.C., Punyasena, S.W.: Mass extinction events and the plant fossil record. Trends in Ecology & Evolution 22, 548–557 (2007)CrossRefGoogle Scholar
  49. 49.
    Meyers, L.A., Bull, J.J.: Fighting change with change: adaptive variation in an uncertain world. Trends in Ecology & Evolution 17, 551–557 (2002)CrossRefGoogle Scholar
  50. 50.
    Michalewicz, Z., Janikow, C.Z., Krawczyk, J.B.: A modified genetic algorithm for optimal control problems. Computers and Mathematics with Applications 23, 83–94 (1992)zbMATHCrossRefGoogle Scholar
  51. 51.
    Morrison, R.W., De Jong, K.A.: Triggered hypermutation revisited. In: Zalzala, A., Fonseca, C., Kim, J.H., Smith, A., Yao, X. (eds.) Proc. Congress on Evolutionary Computation, IEEE CEC 2000, pp. 1025–1032. IEEE Press, Piscataway (2000)Google Scholar
  52. 52.
    Morrison, R.W.: Designing Evolutionary Algorithms for Dynamic Environments. Springer, Heidelberg (2004)zbMATHGoogle Scholar
  53. 53.
    Paenke, I., Branke, J., Jin, Y.: On the influence of phenotype plasticity on genotype diversity. In: Fogel, D.B., Yao, X., Mendel, J., Omori, T. (eds.) Proc. IEEE Symposium on Foundations of Computational Intelligence, FOCI 2007, pp. 33–40. IEEE Press, Piscataway (2007)CrossRefGoogle Scholar
  54. 54.
    Paenke, I., Branke, J., Jin, Y.: Balancing population– and individual–level adaptation in changing environments. Adaptive Behavior 17, 153–174 (2009)CrossRefGoogle Scholar
  55. 55.
    Parter, M., Kashtan, N., Alon, U.: Facilitated variation: How evolution learns from past environments to generalize to new environments. PLoS Comput. Biol. 4(11), e1000206 (2008), doi:10.1371/journal.pcbi.1000206CrossRefGoogle Scholar
  56. 56.
    Pigliucci, M., Kaplan, J.M.: Making Sense of Evolution: The Conceptual Foundations of Evolutionary Biology. University of Chicago Press, Chicago (2006)Google Scholar
  57. 57.
    Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.H.: A novel population initialization method for accelerating evolutionary algorithms. Computers & Mathematics with Applications 53, 1605–1614 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  58. 58.
    Richter, H.: Behavior of Evolutionary Algorithms in Chaotically Changing Fitness Landscapes. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 111–120. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  59. 59.
    Richter, H.: A study of dynamic severity in chaotic fitness landscapes. In: Corne, D. (ed.) Proc. 2005 IEEE Congress on Evolutionary Computation, pp. 2824–2831. IEEE Press, Piscataway (2005)CrossRefGoogle Scholar
  60. 60.
    Richter, H.: Evolutionary Optimization in Spatio–temporal Fitness Landscapes. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 1–10. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  61. 61.
    Richter, H.: Coupled map lattices as spatio–temporal fitness functions: Landscape measures and evolutionary optimization. Physica D237, 167–186 (2008)Google Scholar
  62. 62.
    Richter, H.: Detecting change in dynamic fitness landscapes. In: Tyrrell, A. (ed.) Proc. Congress on Evolutionary Computation, IEEE CEC 2009, pp. 1613–1620. IEEE Press, Piscataway (2009)CrossRefGoogle Scholar
  63. 63.
    Richter, H.: Change detection in dynamic fitness landscapes: An immunological approach. In: Abraham, A., Carvalho, A., Herrera, F., Pai, V. (eds.) World Congress on Nature and Biologically Inspired Computing (NaBIC 2009), pp. 719–724. IEEE Research Publishing Services, Singapore (2009)CrossRefGoogle Scholar
  64. 64.
    Richter, H.: Evolutionary optimization and dynamic fitness landscapes: From reaction-diffusion systems to chaotic CML. In: Zelinka, I., Celikovsky, S., Richter, H., Chen, G. (eds.) Evolutionary Algorithms and Chaotic Systems, pp. 409–446. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  65. 65.
    Richter, H., Yang, S.: Memory Based on Abstraction for Dynamic Fitness Functions. In: Giacobini, M., Brabazon, A., Cagnoni, S., Di Caro, G.A., Drechsler, R., Ekárt, A., Esparcia-Alcázar, A.I., Farooq, M., Fink, A., McCormack, J., O’Neill, M., Romero, J., Rothlauf, F., Squillero, G., Uyar, A.Ş., Yang, S., et al. (eds.) EvoWorkshops 2008. LNCS, vol. 4974, pp. 596–605. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  66. 66.
    Richter, H., Yang, S.: Learning behavior in abstract memory schemes for dynamic optimization problems. Soft Computing 13, 1163–1173 (2009)zbMATHCrossRefGoogle Scholar
  67. 67.
    Sahney, S., Benton, M.J.: Recovery from the most profound mass extinction of all time. Proc. of the Royal Society B275, 759–765 (2008)CrossRefGoogle Scholar
  68. 68.
    Seiffertt, J., Sanyal, S., Wunsch, D.C.: Hamilton–Jacobi–Bellman equations and approximate dynamic programming on time scales. IEEE Trans. Syst., Man, & Cybern. Part B: Cybern. 38, 918–923 (2008)CrossRefGoogle Scholar
  69. 69.
    Simões, A., Costa, E.: Variable-Size Memory Evolutionary Algorithm to Deal with Dynamic Environments. In: Giacobini, M., et al. (eds.) EvoWorkshops 2007. LNCS, vol. 4448, pp. 617–626. Springer, Heidelberg (2007)Google Scholar
  70. 70.
    Simões, A., Costa, E.: Evolutionary Algorithms for Dynamic Environments: Prediction Using Linear Regression and Markov Chains. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 306–315. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  71. 71.
    Sontag, E.D.: A “universal” construction of Artstein’s theorem on nonlinear stabilization. Systems & Control Letters 13, 117–123 (1989)MathSciNetzbMATHCrossRefGoogle Scholar
  72. 72.
    Stadler, B.M.R., Stadler, P.F., Wagner, G.P., Fontana, W.: The topology of the possible: Formal spaces underlying patterns of evolutionary change. J. Theor. Biol. 213, 241–274 (2001)MathSciNetCrossRefGoogle Scholar
  73. 73.
    Tinós, R., Yang, S.: A self–organizing random immigrants genetic algorithm for dynamic optimization problems. Genet. Program. Evol. Mach. 8, 255–286 (2007)CrossRefGoogle Scholar
  74. 74.
    Tsinias, J.: Sufficient Lyapunov–like conditions for stabilization. Math. Control Signals Systems 2, 343–347 (1989)MathSciNetzbMATHCrossRefGoogle Scholar
  75. 75.
    Uyar, A.Ş., Harmanci, A.E.: A new population based adaptive dominance change mechanism for diploid genetic algorithms in dynamic environments. Soft Computing 9, 803–815 (2005)zbMATHCrossRefGoogle Scholar
  76. 76.
    Wagner, A.: Robustness and Evolvability in Living Systems. Princeton University Press, Princeton (2007)Google Scholar
  77. 77.
    Wang, F.Y., Zhang, H., Liu, D.: Adaptive dynamic programming: An introduction. IEEE Computational Intelligence Magazine 4, 39–47 (2009)CrossRefGoogle Scholar
  78. 78.
    Werbos, P.J.: A menu of designs for reinforcement learning over time. In: Miller, W.T., Sutton, R.S., Werbos, P.J. (eds.) Neural Networks for Control, pp. 67–95. MIT Press, Cambridge (1991)Google Scholar
  79. 79.
    Werbos, P.J.: Approximate dynamic programming for real–time control and neural modeling. In: White, D.A., Sofge, D.A. (eds.) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–525. Van Nostrand Reinhold, New York (1992)Google Scholar
  80. 80.
    Yang, S.: Associative Memory Scheme for Genetic Algorithms in Dynamic Environments. In: Rothlauf, F., Branke, J., Cagnoni, S., Costa, E., Cotta, C., Drechsler, R., Lutton, E., Machado, P., Moore, J.H., Romero, J., Smith, G.D., Squillero, G., Takagi, H., et al. (eds.) EvoWorkshops 2006. LNCS, vol. 3907, pp. 788–799. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  81. 81.
    Zhang, X.S.: Evolution and maintenance of the environmental component of the phenotypic variance: Benefit of plastic traits under changing environments. The American Naturalist 166, 569–580 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Faculty of Electrical Engineering and Information Technology, Department of Measurement Technology and Control EngineeringHTWK Leipzig University of Applied SciencesLeipzigGermany
  2. 2.Department of Information Systems and ComputingBrunel UniversityUxbridgeUnited Kingdom

Personalised recommendations