Skip to main content

Advertisement

SpringerLink
Log in

Controlled approximation of the value function in stochastic dynamic programming for multi-reservoir systems

  • Original Paper
  • Published:
Computational Management Science Aims and scope Submit manuscript

Abstract

We present a new approach for adaptive approximation of the value function in stochastic dynamic programming. Under convexity assumptions, our method is based on a simplicial partition of the state space. Bounds on the value function provide guidance as to where refinement should be done, if at all. Thus, the method allows for a trade-off between solution time and accuracy. The proposed scheme is experimented in the particular context of hydroelectric production across multiple reservoirs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Barros MT, Tsai FT, Sl Yang, Lopes JE, Yeh WW (2003) Optimization of large-scale hydropower system operations. J Water Res Plan Manag 129(3):178–188

    Article  Google Scholar 

  • Castelletti A, de Rigo D, Rizzoli AE, Soncini-Sessa R, Weber E (2007) Neuro-dynamic programming for designing water reservoir network management policies. Control Eng Pract 15(8):1031–1038

    Article  Google Scholar 

  • Chandramouli V, Raman H (2001) Multireservoir modeling with dynamic programming and neural networks. J Water Res Plan Manag 127(2):89–98

    Article  Google Scholar 

  • Dickinson PJ (2013) On the exhaustivity of simplicial partitioning. J Glob Optim 58.1:1–15

  • Foufoula-Georgiou E, Kitanidis PK (1988) Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems. Water Resour Res 24(8):1345–1359

    Article  ADS  Google Scholar 

  • Gal S (1989) The parameter iteration method in dynamic programming. Manag Sci 35(6):675–684

    Article  MATH  Google Scholar 

  • Hiriart-Urruty JB, Lemaréchal C (2001) Fundamentals of convex analysis. Springer, Berlin

  • Kim YO, Eum HI, Lee EG, Ko IH (2007) Optimizing operational policies of a korean multireservoir system using sampling stochastic dynamic programming with ensemble streamflow prediction. J Water Resour Plan Manag 133(1):4–14

    Article  Google Scholar 

  • Krau S, Émiel G, Merleau J (2014) Une méthode d’échantillonnage adaptatif de l’espace des états pour la programmation dynamique appliquée à la gestion de systèmes hydriques. Seminar presentation, CIRRELT, Montréal, January

  • Lang P, Zéphyr L, Lamond B (2014) Computing the expected value function for a multi-reservoir system with highly correlated inflows. In: Guan Y, Liao H (eds) Proceedings of the industrial and systems engineering research conference (ISERC 2014), Montreal, Canada, May 31–June 3. Online: http://www.xcdsystem.com/iie2014/abstract/finalpapers/I111.pdf

  • Moore DW (1992) Simplical mesh generation with applications. PhD thesis

  • Munos R, Moore A (2002) Variable resolution discretization in optimal control. Mach Learn 49(2–3):291–323

    Article  MATH  Google Scholar 

  • Paulavičius R, Žilinskas J (2013) Simplicial global optimization. Springer, Berlin

  • Philpott AB, Guan Z (2008) On the convergence of stochastic dual dynamic programming and related methods. Oper Res Lett 36(4):450–455

    Article  MATH  MathSciNet  Google Scholar 

  • Powell WB (2009) What you should know about approximate dynamic programming. Naval Res Logist 56(3):239–249

    Article  MATH  MathSciNet  Google Scholar 

  • Rani D, Moreira MM (2010) Simulation-optimization modeling: a survey and potential application in reservoir systems operation. Water Resour Manag 24(6):1107–1138

    Article  Google Scholar 

  • Shawwash ZK, Siu TK, Russell SD (2000) The BC hydro short term hydro scheduling optimization model. IEEE Trans Power Syst 15(3):1125–1131

    Article  Google Scholar 

  • Tilmant A, Pinte D, Goor Q (2008) Assessing marginal water values in multipurpose multireservoir systems via stochastic programming. Water Resour Res 44(12):W12,431

  • Yurtal R, Seckin G, Ardiclioglu G (2005) Hydropower optimization for the lower seyhan system in Turkey using dynamic programming. Water Int 30(4):522–529

    Article  Google Scholar 

  • Zéphyr L (2015) Optimisation stochastique des systémes multi-réservoirs par l’agrégation de scénarios et la programmation dynamique approximative. PhD thesis

Download references

Author information

Authors and Affiliations

  1. Operations and Decision Systems Departement, Université Laval, Pavillon Palasis-Prince, 2325, Rue de la Terrasse, Quebec, QC, G1V 0A6, Canada

    Luckny Zéphyr, Pascal Lang & Bernard F. Lamond

Authors
  1. Luckny Zéphyr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pascal Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernard F. Lamond
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luckny Zéphyr.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zéphyr, L., Lang, P. & Lamond, B.F. Controlled approximation of the value function in stochastic dynamic programming for multi-reservoir systems. Comput Manag Sci 12, 539–557 (2015). https://doi.org/10.1007/s10287-015-0242-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10287-015-0242-1

Keywords

Search

Navigation