Introduction to Extremum Seeking

  • Shu-Jun Liu
  • Miroslav KrsticEmail author
Part of the Communications and Control Engineering book series (CCE)


The motivation behind extremum seeking methodology is discussed and the advances in the field of extremum seeking of the last 15 years are reviewed. Then a basic introduction to stochastic extremum seeking is presented, including how it relates to standard deterministic extremum seeking with periodic perturbations and what ideas are behind the study of stability of the resulting stochastic nonlinear system.


Nash Equilibrium Periodic Perturbation Stochastic Perturbation Stochastic Average Stochastic Nonlinear System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Adetola V, Guay M (2006) Adaptive output feedback extremum seeking receding horizon control of linear systems. J Process Control 16:521–533 CrossRefGoogle Scholar
  2. 2.
    Adetola V, Guay M (2007) Guaranteed parameter convergence for extremum-seeking control of nonlinear systems. Automatica 43:105–110 MathSciNetCrossRefGoogle Scholar
  3. 6.
    Ariyur KB, Krstic M (2003) Real-time optimization by extremum seeking control. Wiley, Hoboken zbMATHCrossRefGoogle Scholar
  4. 7.
    Ariyur K, Krstic M (2004) Slope seeking: a generalization of extremum seeking. Int J Adapt Control Signal Process 18:1–22 zbMATHCrossRefGoogle Scholar
  5. 8.
    Aumann RJ (1964) Markets with a continuum of traders. Econometrica 32(1):39–50 MathSciNetzbMATHCrossRefGoogle Scholar
  6. 9.
    Banaszuk A, Ariyur KB, Krstic M, Jacobson CA (2004) An adaptive algorithm for control of combustion instability. Automatica 40:1965–1972 MathSciNetzbMATHCrossRefGoogle Scholar
  7. 14.
    Becker A, Kumar PR, Wei CZ (1985) Adaptive control with the stochastic approximation algorithm: geometry and convergence. IEEE Trans Autom Control 30:330–338 MathSciNetzbMATHCrossRefGoogle Scholar
  8. 15.
    Becker R, King R, Petz W, Nitsche W (2006) Adaptive closed-loop separation control on a high-lift configuration using extremum seeking. AIAA paper 2006-3493 Google Scholar
  9. 16.
    Becker R, King R, Petz W, Nitsche W (2007) Adaptive closed-loop separation control on a high-lift configuration using extremum seeking. AIAA J 45:1382–1392 CrossRefGoogle Scholar
  10. 20.
    Binetti P, Ariyur KB, Krstic M, Bernelli F (2003) Formation flight optimization using extremum seeking feedback. AIAA J Guid Control Dyn 26:132–142 CrossRefGoogle Scholar
  11. 24.
    Brunn A, Nitsche W, Henning L, King R Application of slope-seeking to a generic car model for active drag control. Preprint Google Scholar
  12. 25.
    Carnevale D, Astolfi A, Centioli C, Podda S, Vitale V, Zaccarian L (2009) A new extremum seeking technique and its application to maximize RF heating on FTU. Fusing Eng Des 84:554–558 CrossRefGoogle Scholar
  13. 26.
    Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, New York zbMATHCrossRefGoogle Scholar
  14. 27.
    Choi J-Y, Krstic M, Ariyur KB, Lee JS (2002) Extremum seeking control for discrete time systems. IEEE Trans Autom Control 47:318–323 MathSciNetCrossRefGoogle Scholar
  15. 28.
    Cochran J, Krstic M (2009) Nonholonomic source seeking with tuning of angular velocity. IEEE Trans Autom Control 54(4):717–731 MathSciNetCrossRefGoogle Scholar
  16. 30.
    Cochran J, Kanso E, Kelly SD, Xiong H, Krstic M (2009) Source seeking for two nonholonomic models of fish locomotion. IEEE Trans Robot 25:1166–1176 CrossRefGoogle Scholar
  17. 32.
    Creaby J, Li Y, Seem JE (2009) Maximizing wind turbine energy capture using multivariable extremum seeking control. Wind Eng 33:361–387 CrossRefGoogle Scholar
  18. 33.
    DeHaan D, Guay M (2005) Extremum-seeking control of state-constrained nonlinear systems. Automatica 41:1567–1574 MathSciNetzbMATHCrossRefGoogle Scholar
  19. 36.
    Favache A, Guay M, Perrier M, Dochain D (2008) Extremum seeking control of retention for a microparticulate system. Can J Chem Eng 86:815–827 CrossRefGoogle Scholar
  20. 37.
    Foster DP, Young HP (2003) Learning, hypothesis testing, and Nash equilibrium. Games Econ Behav 45:73–96 MathSciNetzbMATHCrossRefGoogle Scholar
  21. 38.
    Foster DP, Young HP (2006) Regret testing: learning to play Nash equilibrium without knowing you have an opponent. Theor Econ 1:341–367 Google Scholar
  22. 40.
    Frihauf P, Krstic M, Başar T (2011) Nash equilibrium seeking with infinitely-many players. In: Proceedings of 2011 American control conference, San Francisco, CA, USA, June 29–July 1, pp 3059–3064 Google Scholar
  23. 42.
    Fudenberg D, Levine DK (1998) The theory of learning in games. MIT Press, Cambridge zbMATHGoogle Scholar
  24. 43.
    Gelfand SB, Mitter SK (1991) Recursive stochastic algorithms for global optimization in ℝd. SIAM J Control Optim 29:999–1018 MathSciNetzbMATHCrossRefGoogle Scholar
  25. 44.
    Gelfand SB, Mitter SK (1993) Metropolis-type annealing algorithms for global optimization in ℝd. SIAM J Control Optim 31:111–131 MathSciNetzbMATHCrossRefGoogle Scholar
  26. 48.
    Green EJ (1984) Continuum and finite-player noncooperative models of competition. Econometrica 52(4):975–993 MathSciNetzbMATHCrossRefGoogle Scholar
  27. 49.
    Guay M, Perrier M, Dochain D (2005) Adaptive extremum seeking control of nonisothermal continuous stirred reactors. Chem Eng Sci 60:3671–3681 CrossRefGoogle Scholar
  28. 50.
    Guay M, Dochain D, Perrier M, Hudon N (2007) Flatness-based extremum-seeking control over periodic orbits. IEEE Trans Autom Control 52:2005–2012 MathSciNetCrossRefGoogle Scholar
  29. 53.
    Henning L, Becker R, Feuerbach G, Muminovic R, Brun A, Nitsche W, King R (2008) Extensions of adaptive slope-seeking for active flow control. Proc Inst Mech Eng, Part I, J Syst Control Eng 222:309–322 CrossRefGoogle Scholar
  30. 54.
    Jafari A, Greenwald A, Gondek D, Ercal G (2001) On no-regret learning, fictitious play, and Nash equilibrium. In: Proceedings of the 18th international conference on machine learning Google Scholar
  31. 61.
    Killingsworth NJ, Krstic M (2006) PID tuning using extremum seeking. IEEE Control Syst Mag 26:70–79 MathSciNetCrossRefGoogle Scholar
  32. 62.
    Killingsworth NJ, Krstic M, Flowers DL, Espinoza-Loza F, Ross T, Aceves SM (2009) HCCI engine combustion timing control: optimizing gains and fuel consumption via extremum seeking. IEEE Trans Control Syst Technol 17:1350–1361 CrossRefGoogle Scholar
  33. 63.
    Kim K, Kasnakoglu C, Serrani A, Samimy M (2009) Extremum-seeking control of subsonic cavity flow. AIAA J 47:195–205 CrossRefGoogle Scholar
  34. 64.
    King R, Becker R, Feuerbach G, Henning L, Petz R, Nitsche W, Lemke O, Neise W (2006) Adaptive flow control using slope seeking. In: Proceedings of the 14th IEEE Mediterranean conference on control and automation, June 28–30, pp 1–6 Google Scholar
  35. 69.
    Krieger JP, Krstic M (2011) Extremum seeking based on atmospheric turbulence for aircraft endurance. AIAA J Guid Control Dyn 34:1876–1885 CrossRefGoogle Scholar
  36. 70.
    Krstic M (2000) Performance improvement and limitations in extremum seeking control. Syst Control Lett 39:313–326 MathSciNetzbMATHCrossRefGoogle Scholar
  37. 71.
    Krstic M, Wang HH (2000) Stability of extremum seeking feedback for general nonlinear dynamic systems. Automatica 36:595–601 MathSciNetzbMATHCrossRefGoogle Scholar
  38. 75.
    Kumar PR, Varaiya P (1986) Stochastic systems: estimation, identification and adaptive control. Prentice Hall, Englewood Cliffs zbMATHGoogle Scholar
  39. 81.
    Lei P, Li Y, Chen Q, Seem JE (2010) Extremum seeking control based integration of MPPT and degradation detection for photovoltaic arrays. In: Proceedings of 2010 American control conference, Baltimore, MD, USA, June 30–July 2, pp 3536–3541 Google Scholar
  40. 82.
    Li P, Li Y, Seem JE (2009) Extremum seeking control for efficient and reliable operation of air-side economizers. In: Proceedings of 2009 American control conference, St. Louis, MO, USA, June 10–12, pp 20–25 CrossRefGoogle Scholar
  41. 83.
    Li S, Başar T (1987) Distributed algorithms for the computation of noncooperative equilibria. Automatica 23:523–533 zbMATHCrossRefGoogle Scholar
  42. 84.
    Li Y, Rotea MA, Chiu GT-C, Mongeau LG, Paek I-S (2005) Extremum seeking control of a tunable thermoacoustic cooler. IEEE Trans Control Syst Technol 13:527–536 CrossRefGoogle Scholar
  43. 93.
    Ljung L (1978) Strong convergence of a stochastic approximation algorithm. Ann Stat 6:680–696 MathSciNetzbMATHCrossRefGoogle Scholar
  44. 94.
    Ljung L (2001) Recursive least-squares and accelerated convergence in stochastic approximation schemes. Int J Adapt Control Signal Process 15:169–178 zbMATHCrossRefGoogle Scholar
  45. 95.
    Ljung L, Pflug G, Walk H (1992) Stochastic approximation and optimization of random systems. Birkhäuser, Basel zbMATHCrossRefGoogle Scholar
  46. 96.
    Luo L, Schuster E (2009) Mixing enhancement in 2D magnetohydrodynamic channel flow by extremum seeking boundary control. In: Proceedings of the 2009 American control conference, St. Louis, MO, USA, June 10–12, pp 1530–1535 CrossRefGoogle Scholar
  47. 98.
    Manzie C, Krstic M (2009) Extremum seeking with stochastic perturbations. IEEE Trans Autom Control 54:580–585 MathSciNetCrossRefGoogle Scholar
  48. 101.
    Moase WH, Manzie C, Brear MJ (2009) Newton-like extremum-seeking part I: theory. In: Proceedings of the joint 48th IEEE conference on decision and control and 28th Chinese control conference, Shanghai, China, December 16–18, pp 3839–3844 Google Scholar
  49. 102.
    Moase WH, Manzie C, Brear MJ (2009) Newton-like extremum-seeking part II: simulation and experiments. In: Proceedings of the joint 48th IEEE conference on decision and control and 28th Chinese control conference, Shanghai, China, December 16–18, pp 3845–3850 Google Scholar
  50. 104.
    Moeck JP, Bothien MR, Paschereit CO, Gelbert G, King R Two-parameter extremum seeking for control of thermoacoustic instabilities and characterization of linear growth. AIAA paper 2007-1416 Google Scholar
  51. 105.
    Murugappan S, Gutmark E, Acharya S, Krstic M (2000) Extremum seeking adaptive controller of swirl-stabilized spray combustion. Proc Combust Inst 28:731–737 CrossRefGoogle Scholar
  52. 108.
    Nešić D, Tan Y, Moase WH, Manzie C (2010) A unifying approach to extremum seeking: adaptive schemes based on estimation of derivatives. In: Proceedings of the 49th IEEE conference on decision and control, Atlanta, GA, USA, December 15–17, pp 4625–4630 Google Scholar
  53. 110.
    Ou Y, Xu C, Schuster E, Luce TC, Ferron JR, Walker ML, Humphreys DA (2008) Design and simulation of extremum-seeking open-loop optimal control of current profile in the DIII-D tokamak. Plasma Phys Control Fusion 50:115001 CrossRefGoogle Scholar
  54. 113.
    Peterson K, Stefanopoulou A (2004) Extremum seeking control for soft landing of an electromechanical valve actuator. Automatica 29:1063–1069 MathSciNetCrossRefGoogle Scholar
  55. 115.
    Ren B, Frihauf P, Krstic M, Rafac RJ (2012) Laser pulse shaping via extremum seeking. Control Eng Pract 20:678–683 CrossRefGoogle Scholar
  56. 119.
    Rotea MA (2000) Analysis of multivariable extremum seeking algorithms. In: Proceedings of the 2000 American control conference, Chicago, IL, USA, June 28–30, pp 433–437 Google Scholar
  57. 122.
    Schuster E, Torres N, Xu C (2006) Extremum seeking adaptive control of beam envelope in particle accelerators. In: Proceedings of the 2006 IEEE conference on control applications, Munich, Germany, October 4–6, pp 1837–1842 CrossRefGoogle Scholar
  58. 123.
    Schuster E, Xu C, Torres N, Morinaga E, Allen CK, Krstic M (2007) Beam matching adaptive control via extremum seeking. Nucl Instrum Methods Phys Res, Sect A, Accel Spectrom Detect Assoc Equip 581:799–815 CrossRefGoogle Scholar
  59. 126.
    Shamma JS, Arslan G (2005) Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria. IEEE Trans Autom Control 53(3):312–327 MathSciNetCrossRefGoogle Scholar
  60. 127.
    Sharma R, Gopal M (2010) Synergizing reinforcement learning and game theory—a new direction for control. Appl Soft Comput 10(3):675–688 CrossRefGoogle Scholar
  61. 128.
    Shitovitz B (1973) Oligopoly in markets with a continuum of traders. Econometrica 41(3):467–501 MathSciNetzbMATHCrossRefGoogle Scholar
  62. 133.
    Stanković MS, Stipanović DM (2009) Stochastic extremum seeking with applications to mobile sensor networks. In: Proceedings of the 2009 American control conference, St. Louis, MA, USA, June 10–12, pp 5622–5627 CrossRefGoogle Scholar
  63. 134.
    Stanković MS, Stipanović DM (2009) Discrete time extremum seeking by autonomous vehicles in a stochastic environment. In: Proceedings of the joint 48th IEEE conference on decision and control and 28th Chinese control conference, Shanghai, China, December 16–18, pp 4541–4546 Google Scholar
  64. 135.
    Stanković MS, Stipanović DM (2010) Extremum seeking under stochastic noise and applications to mobile sensors. Automatica 46:1243–1251 zbMATHCrossRefGoogle Scholar
  65. 136.
    Stanković MS, Johansson KH, Stipanović DM (2010) Distributed seeking of Nash equilibrium in mobile sensor networks. In: Proceedings of IEEE conference on decision and control, Atlanta, GA, USA, December 15–17, pp 5598–5603 Google Scholar
  66. 137.
    Tan Y, Nešić D, Mareels IMY (2006) On non-local stability properties of extremum seeking controllers. Automatica 42:889–903 zbMATHCrossRefGoogle Scholar
  67. 140.
    Wang H-H, Krstic M (2000) Extremum seeking for limit cycle minimization. IEEE Trans Autom Control 45:2432–2437 MathSciNetzbMATHCrossRefGoogle Scholar
  68. 141.
    Wang H-H, Krstic M, Bastin G (1999) Optimizing bioreactors by extremum seeking. Int J Adapt Control Signal Process 13:651–669 zbMATHCrossRefGoogle Scholar
  69. 142.
    Wang H-H, Yeung S, Krstic M (2000) Experimental application of extremum seeking on an axial-flow compressor. IEEE Trans Control Syst Technol 8:300–309 CrossRefGoogle Scholar
  70. 143.
    Wehner W, Schuster E (2009) Stabilization of neoclassical tearing modes in tokamak fusion plasmas via extremum seeking. In: Proceedings of the 3rd IEEE multi-conference on systems and control (MSC 2009), Saint Petersburg, Russia, July 8–10 Google Scholar
  71. 144.
    Wiederhold O, Neuhaus L, King R, Niese W, Enghardt L, Noack BR, Swoboda M (2009) Extensions of extremum-seeking control to improve the aerodynamic performance of axial turbomachines. In: Proceedings of the 39th AIAA fluid dynamics conference, San Antonio, TX, USA Google Scholar
  72. 146.
    Zhang C, Arnold D, Ghods N, Siranosian A, Krstic M (2007) Source seeking with nonholonomic unicycle without position measurement and with tuning of forward velocity. Syst Control Lett 56:245–252 MathSciNetzbMATHCrossRefGoogle Scholar
  73. 148.
    Zhang XT, Dawson DM, Dixon WE, Xian B (2006) Extremum-seeking nonlinear controllers for a human exercise machine. IEEE/ASME Trans Mechatron 11:233–240 CrossRefGoogle Scholar
  74. 150.
    Zhu M, Martinez S (2010) Distributed coverage games for mobile visual sensor networks. SIAM J Control Optim (submitted). Available at

Copyright information

© Springer-Verlag London 2012

Authors and Affiliations

  1. 1.Department of MathematicsSoutheast UniversityNanjingPeople’s Republic of China
  2. 2.Department Mechanical & Aerospace EngineeringUniversity of California, San DiegoLa JollaUSA

Personalised recommendations