Skip to main content
SpringerLink
Log in

Incentive Stackelberg Games for Stochastic Systems

  • Chapter
  • First Online:
Frontiers in Games and Dynamic Games

Part of the book series: Annals of the International Society of Dynamic Games ((AISDG,volume 16))

Abstract

Dynamic games with hierarchical structure have been identified as key components of modern control systems that enable the integration of renewable cooperative and/or non-cooperative control such as distributed multi-agent systems. Although the incentive Stackelberg strategy has been admitted as the hierarchical strategy that induces the behavior of the decision maker as that of the follower, the followers optimize their costs under incentives without a specific information. Therefore, leaders succeed in using the required strategy to induce the behavior of their followers. This concept is considered very useful and reliable in some practical cases. In this survey, incentive Stackelberg games for deterministic and stochastic linear systems with external disturbance are addressed. The induced features of the hierarchical strategy in the considered models, including stochastic systems governed by Itô stochastic differential equation, Markov jump linear systems, and linear parameter varying (LPV) systems, are explained in detail. Furthermore, basic concepts based on the H 2∕H ∞ control setting for the incentive Stackelberg games are reviewed. Next, it is shown that the required set of strategies can be designed by solving higher-order cross-coupled algebraic Riccati-type equations. Finally, as a partial roadmap for the development of the underdeveloped pieces, some open problems are introduced.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 49.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 49.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, Philadelphia: SIAM Series in Classics in Applied Mathematics, 1999.

    MATH  Google Scholar 

  2. V. R. Saksena and J. B. Cruz, Jr, Optimal and near-optimal incentive strategies in the hierarchical control of Markov chains, Automatica, vol. 21, no. 2, pp. 181–191, 1985.

    Article  MathSciNet  Google Scholar 

  3. K. B. Kim, A. Tang and S. H. Low, “A stabilizing AQM based on virtual queue dynamics in supporting TCP with arbitrary delays,” in Proc. 42nd IEEE Conf. Decision and Control, Maui, HI, December 2003, pp. 3665–3670.

    Google Scholar 

  4. R. Kicsiny, Solution for a class of closed-loop leader-follower games with convexity conditions on the payoffs, Annals of Operations Research, vol. 253, no. 1, pp. 405–429, 2017.

    Article  MathSciNet  Google Scholar 

  5. M. Yu and S. H. Hong, Supply-demand balancing for power management in smart grid: A Stackelberg game approach, Applied Energy, vol. 164, pp. 702–710, 2016.

    Article  Google Scholar 

  6. C. I. Chen and J. B. Cruz, Jr., Stackelberg solution for two-person games with biased information patterns, IEEE Trans. Automatic Control, vol. 17, no. 6, pp. 791–798, Dec. 1972.

    Article  Google Scholar 

  7. J. V. Medanic, Closed-loop Stackelberg strategies in linear-quadratic problems, IEEE Trans. Automatic Control, vol. 23, no. 4, pp. 632–637, Aug. 1978.

    Article  Google Scholar 

  8. M. Jungers, E. Trelat and H. Abou-Kandil, Min-max and min-min Stackelberg strategies with closed-loop information structure, J. Dynamical and Control Systems, vol. 17, no. 3, pp. 387–425, 2011.

    Article  MathSciNet  Google Scholar 

  9. A. Bensoussan, S. Chen and S. P. Sethi, The maximum principle for global solutions of stochastic Stackelberg differential games, SIAM Control and Optimization, vol. 53, no. 4, pp. 1956–1981, 2015.

    Article  MathSciNet  Google Scholar 

  10. J. Xu and H. Zhang, Sufficient and necessary open-loop Stackelberg strategy for two-player game with time delay, IEEE Trans. Cybernetics, vol. 46, no. 2, pp. 438–449, Feb. 2016.

    Article  MathSciNet  Google Scholar 

  11. H. Mukaidani and H. Xu, Stackelberg strategies for stochastic systems with multiple followers, Automatica, vol. 53, pp. 53–59, 2015.

    Article  MathSciNet  Google Scholar 

  12. H. Mukaidani and H. Xu, Infinite-horizon linear-quadratic Stackelberg games for discrete-time stochastic systems, Automatica, vol. 76, no. 5, pp. 301–308, 2017.

    Article  MathSciNet  Google Scholar 

  13. Y. C. Ho, Peter B. Luh and G. J. Olsder, A control-theoretic view on incentives, Automatica, vol. 18, no. 2, pp. 167–179, 1982.

    Google Scholar 

  14. T. Basar and H. Selbuz, Closed-loop Stackelberg strategies with applications in the optimal control of multilevel systems, IEEE Trans. Automatic Control, vol. 24, no. 2, pp. 166–178, April 1979.

    Article  MathSciNet  Google Scholar 

  15. B. Tolwinski, Closed-loop Stackelberg solution to multi-stage linear quadratic game, J. Optimization Theory and Applications, vol. 34, no. 3, pp. 485–501, 1981.

    Article  MathSciNet  Google Scholar 

  16. Y. P. Zheng and T. Basar, Existence and derivations of optimal affine incentive schemes for Stackelberg games with partial information: A geometric approach, Int. J. Control, vol. 35, no. 6, pp. 997–1011, 1982.

    Article  MathSciNet  Google Scholar 

  17. Y. P. Zheng, T. Basar and J. B. Cruz Jr., Stackelberg strategies and incentives in multiperson deterministic decision problems, IEEE Trans. Systems, Man, and Cybernetics, vol. 14, no. 1, pp. 10–24, Jan. 1984.

    Article  MathSciNet  Google Scholar 

  18. T. Basar and G. J. Olsder, Team-optimal closed-loop Stackelberg strategies in hierarchical control problems, Automatica, vol. 16, no. 3, pp. 409–414, 1980.

    Article  MathSciNet  Google Scholar 

  19. P. B. Luh, S. C. Chang and T.S. Chang, Solutions and properties of multi-stage Stackelberg games, Automatica, vol. 20, no. 2, pp. 251–256, 1984.

    Article  MathSciNet  Google Scholar 

  20. T.S. Chang and P. B. Luh, Derivation of necessary and sufficient conditions for single-state Stackelberg games via the inducible region concept, IEEE Trans. Automatic Control, vol. 29, no. 1, pp. 63–66, 1984.

    Article  MathSciNet  Google Scholar 

  21. P. B. Luh, T. S. Chang and T. Ning, Three-level Stackelberg decision problems, IEEE Trans. Automatic Control, vol. 29, no. 3, pp. 280–282, 1984.

    Article  Google Scholar 

  22. K. Mizukami and H. Wu, Two-level incentive Stackelberg strategies in LQ differential games with two noncooperative leaders and one follower, Trans. SICE, vol. 23, no. 6, pp. 625–632, 1987.

    Article  Google Scholar 

  23. K. Mizukami and H. Wu, Incentive Stackelberg strategies in linear quadratic differential games with two noncooperative followers, Modelling And Methodology In Social And Economic Systems System Modelling and Optimization Volume 113 of the series Lecture Notes in Control and Information Sciences, pp. 436–445, Berlin Heidelberg: Springer, 1988.

    Google Scholar 

  24. T. Ishida and E. Shimemura, Three-level incentive strategies in differential games, Int. J. Control, vol. 38, no. 6, 1983, pp. 1135–1148.

    Article  MathSciNet  Google Scholar 

  25. T. Basar, Equilibrium strategies in dynamic games with multilevel of hierarchy, Automatica, vol. 17, no. 5, pp. 749–754, 1981.

    Article  MathSciNet  Google Scholar 

  26. M. Li, J. B. Cruz Jr., M. A. Simaan, An approach to discrete-time incentive feedback Stackelberg games, IEEE Trans. Systems, Man, and Cybernetics, vol. 32, no. 4, pp. 10–24, July 2002.

    Google Scholar 

  27. H. Mukaidani, Infinite-horizon team-optimal incentive Stackelberg games for linear stochastic systems, IEICE Trans. Fundamentals of Electronics, Communications and Computer Sciences, vol. E99-A, no. 9, pp. 1721–1725, 2016.

    Article  Google Scholar 

  28. M. Ahmed and H. Mukaidani, “H ∞-constrained incentive Stackelberg game for discrete-time systems with multiple non-cooperative followers,” in Proc. 6th IFAC Workshop on Distributed Estimation and Control in Networked Systems, Tokyo, Japan, September 2016, IFAC-PapersOnLine, 49-22, pp. 262–267.

    Google Scholar 

  29. H. Mukaidani, M. Ahmed, T. Shima and H. Xu, “H ∞ constraint incentive Stackelberg game for discrete-time stochastic systems,” in Proc. American Control Conf., Seattle, WA, May 2017, pp. 5257–5262.

    Google Scholar 

  30. M. Ahmed, H. Mukaidani and T. Shima, “Infinite-horizon multi-leader-follower incentive Stackelberg games for linear stochastic systems with H ∞ constraint,” in SICE Annual Conf., Kanazawa, Japan 2017, pp. 1202–1207.

    Google Scholar 

  31. M. Ahmed, H. Mukaidani and T. Shima, H ∞-constrained incentive Stackelberg games for discrete-time stochastic systems with multiple followers, IET Control Theory and Applications, vol. 11, no. 15, pp. 2475–2485, 2017.

    Article  MathSciNet  Google Scholar 

  32. H. Mukaidani and T. Shima, M. Unno, H. Xu and V. Dragan, “Team-optimal incentive Stackelberg strategies for Markov jump linear stochastic systems with H ∞ constraint,” in Proc. 20th IFAC World Congress, Toulouse, France, July 2017, IFAC-PapersOnLine, 50-1, pp. 3780–3785.

    Google Scholar 

  33. H. Mukaidani, H. Xu and V. Dragan, A stochastic multiple-leader follower incentive Stackelberg strategy for Markov jump linear systems, IEEE Control Systems Letters, vol. 1, no. 2, pp. 250–255, 2017. (See also 56th IEEE Conf. Decision and Control, pp. 3688–3693, Melbourne, Australia, December 2017.)

    Google Scholar 

  34. H. Mukaidani, H. Xu and V. Dragan, Static output-feedback incentive Stackelberg game for discrete-time Markov jump linear stochastic systems with external disturbance, IEEE Control Systems Letters, vol. 2, no. 4, pp. 701–706, 2018. (See also 57th IEEE Conf. Decision and Control, Miami, FL, December 2018.)

    Google Scholar 

  35. H. Mukaidani and H. Xu, Incentive Stackelberg games for stochastic linear systems with H ∞ constraint, IEEE Trans. Cybernetics, vol. 49, no. 4, pp. 1463–1474, 2019.

    Article  Google Scholar 

  36. K. Kawakami, H. Mukaidani, H. Xu and Y. Tanaka, “Incentive Stackelberg-Nash strategy with disturbance attenuation for stochastic LPV systems,” in Proc. the 2018 IEEE Int. Conf. Systems, Man, and Cybernetics, Miyazaki, Japan, October 2018, pp. 3940–3945.

    Google Scholar 

  37. H. Mukaidani, H. Xu, T. Shima and M. Ahmed, “Multi-leader-follower incentive Stackelberg game for infinite-horizon Markov jump linear stochastic systems with H ∞ constraint,” in Proc. the 2018 IEEE Int. Conf. Systems, Man, and Cybernetics, Miyazaki, Japan, October 2018, pp. 3946–3953.

    Google Scholar 

  38. H. Mukaidani and Hua Xu, “Robust incentive Stackelberg games for stochastic LPV systems,” in Proc. 57th IEEE Conf. Decision and Control, Miami, FL, December 2018, pp. 1059–1064.

    Google Scholar 

  39. B. S. Chen and W. Zhang. Stochastic H 2∕H ∞ control with state-dependent noise, IEEE Trans. Automatic Control, vol. 49, no. 1, pp. 45–57, Jan. 2004.

    Article  MathSciNet  Google Scholar 

  40. W. Zhang and B. -S. Chen, On stabilizability and exact observability of stochastic systems with their applications, Automatica, vol. 40, no. 1, pp. 87–94, 2004.

    Article  MathSciNet  Google Scholar 

  41. J. C. Engwerda, LQ Dynamic Optimization and Differential Games, Chichester: John Wiley and Sons, 2005.

    Google Scholar 

  42. H. Mukaidani, Soft-constrained stochastic Nash games for weakly coupled large-scale systems, Automatica, vol. 45, no. 5, pp. 1272–1279, 2009.

    Article  MathSciNet  Google Scholar 

  43. Z. Gajic and M. Qureshi, Lyapunov Matrix Equation in System Stability and Control, San Diego, Academic Press, Mathematics in Science and Engineering Series, 1995.

    MATH  Google Scholar 

  44. M. Sagara, H. Mukaidani and T. Yamamoto, Numerical solution of stochastic Nash games with state-dependent noise for weakly coupled large-scale systems, Applied Mathematics and Computation, vol. 197, no. 2, pp 844–857, 2008.

    Article  MathSciNet  Google Scholar 

  45. V. Dragan and T. Morozan, The linear quadratic optimization problems for a class of linear stochastic systems with multiplicative white noise and Markovian jumping, IEEE Trans. Automatic Control, vol. 49, no. 5, pp. 665–675, 2004.

    Article  MathSciNet  Google Scholar 

  46. Y. Huang, W. Zhang and G. Feng, Infinite-horizon H 2∕H ∞ control for stochastic systems with Markovian jumps, Automatica, vol. 44, no. 3, pp. 857–863, 2008.

    Article  MathSciNet  Google Scholar 

  47. X. Li, X. Y. Zhou and M. A. Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, J. Global Optimization, vol. 27, no. 2–3, pp. 149–175, 2003.

    Article  MathSciNet  Google Scholar 

  48. H. Mukaidani, “Gain-scheduled H ∞ constraint Pareto optimal strategy for stochastic LPV systems with multiple decision makers,” in American Control Conf., Seattle, WA, May 2017, pp. 1097–1102.

    Google Scholar 

  49. W. Zhang and G. Feng, Nonlinear stochastic H 2∕H ∞ control with (x, u, v)-dependent noise: Infinite-horizon case, IEEE Trans. Automatic Control, vol. 53, no. 5, pp 1323–1328, 2008.

    Google Scholar 

  50. M. A. Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls, IEEE Trans. Automatic Control, vol. 45, no. 6, pp 1131–1143, 2000.

    Article  MathSciNet  Google Scholar 

  51. C. -C. Ku and C. -I. Wu, Gain-scheduled H ∞ control for linear parameter varying stochastic systems, J. Dynamic Systems, Measurement, and Control, vol. 137, no. 11, 2015, 111012–1.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Hiroshima University, Higashi-Hiroshima City, Hiroshima, Japan

    Hiroaki Mukaidani

Authors
  1. Hiroaki Mukaidani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiroaki Mukaidani .

Editor information

Editors and Affiliations

  1. SRS Consortium for Advanced Study, Hong Kong Shue Yan University, Hong Kong, Hong Kong

    David Yeung

  2. School of Public Finance and Taxation, Zhejiang University of Finance and Economics, Hangzhou, China

    Shravan Luckraz

  3. School of Economics, University of Nottingham Ningbo China, Ningbo, China

    Chee Kian Leong

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Mukaidani, H. (2020). Incentive Stackelberg Games for Stochastic Systems. In: Yeung, D., Luckraz, S., Leong, C. (eds) Frontiers in Games and Dynamic Games. Annals of the International Society of Dynamic Games, vol 16. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-39789-0_3

Download citation

Publish with us

Policies and ethics

Search

Navigation