Skip to main content

Advertisement

Log in

Stochastic dynamic programming approach to managing power system uncertainty with distributed storage

  • Original Paper
  • Published:
Computational Management Science Aims and scope Submit manuscript

Abstract

Wind integration in power grids is challenging because of the uncertain nature of wind speed. Forecasting errors may have costly consequences. Indeed, power might be purchased at highest prices to meet the load, and in case of surplus, power may be wasted. Energy storage may provide some recourse against the uncertainty of wind generation. Because of their sequential nature, in theory, power scheduling problems may be solved via stochastic dynamic programming. However, this scheme is limited to small networks by the so-called curse of dimensionality. This paper analyzes the management of a network composed of conventional power units and wind turbines through approximate dynamic programming, more precisely stochastic dual dynamic programming. A general power network model with ramping constraints on the conventional generators is considered. The approximate method is tested on several networks of different sizes. The numerical experiments also include comparisons with classical dynamic programming on a small network. The results show that the combination of approximation techniques enables to solve the problem in reasonable time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.allaboutcircuits.com/textbook/alternating-current/chpt-5/susceptance-and-admittance/.

References

  • Abbey C, Joós G (2009) A stochastic optimization approach to rating of energy storage systems in wind-diesel isolated grids. IEEE Trans Power Syst 24:418–426

    Article  Google Scholar 

  • Anderson RN, Boulanger A, Powell WB, Scott W (2011) Adaptive stochastic control for the smart grid. Proc IEEE 99:1098–1115

    Article  Google Scholar 

  • Bazaraa MS, Sherali HD, Shetty CM (2013) Nonlinear programming: theory and algorithms. Wiley, Hoboken

    Google Scholar 

  • Bellman R (1956) Dynamic programming and lagrange multipliers. In: Proceedings of the National Academy of Sciences of the United States of America 42, 767

  • Bellman RE (1957) Some new techniques in the dynamic-programming solution of variational problems. Quart Appl Math 16:295–305

    Article  Google Scholar 

  • Bellman RE, Dreyfus SE (1962) Applied dynamic programming. Princeton university press, Princeton

    Book  Google Scholar 

  • Bertsekas DP, Tsitsiklis JN (1995) Neuro-dynamic programming: an overview. In: Proceedings of the 34th IEEE conference on decision and control. IEEE, vol. 1, pp. 560–564

  • Bradbury K, Pratson L, Patiño-Echeverri D (2014) Economic viability of energy storage systems based on price arbitrage potential in real-time us electricity markets. Appl Energy 114:512–519

    Article  Google Scholar 

  • Castillo A, Gayme DF (2014) Grid-scale energy storage applications in renewable energy integration: a survey. Energy Convers Manag 87:885–894

    Article  Google Scholar 

  • dos Santos Coelho L, Mariani VC (2006) Combining of chaotic differential evolution and quadratic programming for economic dispatch optimization with valve-point effect. IEEE Trans Power Syst 21:989

    Article  Google Scholar 

  • European Commission Impact Assessment (2016). Sustainability of bioenergy, SWD 418. https://ec.europa.eu/energy/sites/ener/files/documents/1_en_impact_assessment_part4_v4_418.pdf

  • Foufoula-Georgiou E, Kitanidis PK (1988) Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems. Water Resour Res 24:1345–1359

    Article  Google Scholar 

  • Garcia-Gonzalez J, la Muela D, Ruiz RM, Santos LM, González AM (2008) Stochastic joint optimization of wind generation and pumped-storage units in an electricity market. IEEE Trans Power Syst 23:460–468

    Article  Google Scholar 

  • Goor Q, Kelman R, Tilmant A (2010) Optimal multipurpose-multireservoir operation model with variable productivity of hydropower plants. J Water Resour Plan Manag 137:258–267

    Article  Google Scholar 

  • Grillo S, Marinelli M, Massucco S, Silvestro F (2012) Optimal management strategy of a battery-based storage system to improve renewable energy integration in distribution networks. IEEE Trans Smart Grid 3:950–958

    Article  Google Scholar 

  • Heussen K, Koch S, Ulbig A, Andersson G (2012) Unified system-level modeling of intermittent renewable energy sources and energy storage for power system operation. IEEE Syst J 6:140–151

    Article  Google Scholar 

  • Homem-de Mello T, De Matos VL, Finardi EC (2011) Sampling strategies and stopping criteria for stochastic dual dynamic programming: a case study in long-term hydrothermal scheduling. Energy Syst 2:1–31

    Article  Google Scholar 

  • Howitt R, Msangi S, Reynaud A, Knapp K (2002) Using polynomial approximations to solve stochastic dynamic programming problems: or a ‘betty crocker’ approach to sdp. University of California, Davis

    Google Scholar 

  • Johnson SA, Stedinger JR, Shoemaker CA, Li Y, Tejada-Guibert JA (1993) Numerical solution of continuous-state dynamic programs using linear and spline interpolation. Oper Res 41:484–500

    Article  Google Scholar 

  • Johri R, Filipi ZS (2011) Self-learning neural controller for hybrid power management using neuro-dynamic programming. Presented at the SAE proceedings, Warrendale, PA, USA, Paper 2011-24-0081

  • Keane M, Wolpin K (1994) The solution and estimation of discrete choice dynamic programming models by simulation and interpolation: Monte Carlo evidence. Rev Econ Stat 76:648–672

  • Kelley JE Jr (1960) The cutting-plane method for solving convex programs. J Soc Ind Appl Math 8:703–712

    Article  Google Scholar 

  • Khani H, Zadeh MRD (2015) Real-time optimal dispatch and economic viability of cryogenic energy storage exploiting arbitrage opportunities in an electricity market. IEEE Trans Smart Grid 6:391–401

    Article  Google Scholar 

  • Kitanidis P (1987) A first-order approximation to stochastic optimal control of reservoirs. Stoch Hydrol Hydraul 1:169–184

    Article  Google Scholar 

  • Korsak A, Larson R (1970) A dynamic programming successive approximations technique with convergence proofs. Automatica 6:253–260

    Article  Google Scholar 

  • Lee J-H, Labadie JW (2007) Stochastic optimization of multireservoir systems via reinforcement learning. Water Resour Res 43:W11408. https://doi.org/10.1029/2006WR005627

    Article  Google Scholar 

  • Lindenberg S (2009) 20% Wind energy by 2030: increasing wind energy’s contribution to US electricity supply. Diane Publishing, Collingdale

    Google Scholar 

  • Löhndorf N, Wozabal D, Minner S (2013) Optimizing trading decisions for hydro storage systems using approximate dual dynamic programming. Oper Res 61:810–823

    Article  Google Scholar 

  • Luh PB, Yu Y, Zhang B, Litvinov E, Zheng T, Zhao F, Zhao J, Wang C (2014) Grid integration of intermittent wind generation: a markovian approach. IEEE Trans Smart Grid 5:732–741

    Article  Google Scholar 

  • Maceira MEP, Duarte V, Penna D, Moraes L, Melo A 2008 Ten years of application of stochastic dual dynamic programming in official and agent studies in Brazil-description of the newave program. In: 16th PSCC, Glasgow, Scotland 14–18

  • Mahlia T, Saktisahdan T, Jannifar A, Hasan M, Matseelar H (2014) A review of available methods and development on energy storage; technology update. Renew Sustain Energy Rev 33:532–545

    Article  Google Scholar 

  • Meibom P, Barth R, Hasche B, Brand H, Weber C, O’Malley M (2011) Stochastic optimization model to study the operational impacts of high wind penetrations in ireland. IEEE Trans Power Syst 26:1367–1379

    Article  Google Scholar 

  • Mokrian P, Stephen M 2006 A stochastic programming framework for the valuation of electricity storage. In: 26th USAEE/IAEE North American Conference. 24–27

  • Momoh J, Zhang Y et al. (2005) Unit commitment using adaptive dynamic programming. In: Proceedings of the 13th international conference on intelligent systems application to power systems

  • Morales-España G, Latorre JM, Ramos A (2013) Tight and compact milp formulation for the thermal unit commitment problem. IEEE Trans Power Syst 28:4897–4908

    Article  Google Scholar 

  • Moura PS, De Almeida AT (2010) The role of demand-side management in the grid integration of wind power. Appl Energy 87:2581–2588

    Article  Google Scholar 

  • Naghibi-Sistani M, Akbarzadeh-Tootoonchi M, Bayaz MJ-D, Rajabi-Mashhadi H (2006) Application of q-learning with temperature variation for bidding strategies in market based power systems. Energy Convers Manage 47:1529–1538

    Article  Google Scholar 

  • NREL (National Renewable Energy Laboratory). http://www.nrel.gov/electricity/transmission/eastern_wind_dataset.html

  • NYISO (New York Independent System Operator). Market and operations. http://www.nyiso.com/public/markets_operations/market_data/load_data/index.jsp

  • Ostrowski J, Anjos MF, Vannelli A (2012) Tight mixed integer linear programming formulations for the unit commitment problem. IEEE Trans Power Syst 1:39–46

    Article  Google Scholar 

  • Papaefthymiou G, Klockl B (2008) MCMC for wind power simulation. IEEE Trans Energy Convers 23:234–240

    Article  Google Scholar 

  • Papavasiliou A, Oren SS (2013) Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network. Oper Res 61:578–592

    Article  Google Scholar 

  • Park J-B, Lee K-S, Shin J-R, Lee KY (2005) A particle swarm optimization for economic dispatch with nonsmooth cost functions. IEEE Trans Power Syst 20:34–42

    Article  Google Scholar 

  • Pereira M (1989) Optimal stochastic operations scheduling of large hydroelectric systems. Int J Electr Power Energy Syst 11:161–169

    Article  Google Scholar 

  • Pereira M, Pinto L (1985) Stochastic optimization of a multireservoir hydroelectric system: a decomposition approach. Water Resour Res 21:779–792

    Article  Google Scholar 

  • Pereira MV, Pinto LM (1991) Multi-stage stochastic optimization applied to energy planning. Math Progr 52:359–375

    Article  Google Scholar 

  • Pereira-Neto A, Unsihuay C, Saavedra O (2005) Efficient evolutionary strategy optimisation procedure to solve the nonconvex economic dispatch problem with generator constraints. IEE Proc Gen Transm Distrib 152:653–660

    Article  Google Scholar 

  • Philbrick CRJ, Kitanidis PK (2001) Improved dynamic programming methods for optimal control of lumped-parameter stochastic systems. Oper Res 49:398–412

    Article  Google Scholar 

  • Philpott AB, de Matos VL (2012) Dynamic sampling algorithms for multi-stage stochastic programs with risk aversion. Eur J Oper Res 218:470–483

    Article  Google Scholar 

  • Philpott AB, Guan Z (2008) On the convergence of stochastic dual dynamic programming and related methods. Oper Res Lett 36:450–455

    Article  Google Scholar 

  • Pinson P et al (2013) Wind energy: forecasting challenges for its operational management. Stat Sci 28:564–585

    Article  Google Scholar 

  • Qiu Q, Pedram M (1999) Dynamic power management based on continuous-time markov decision processes. In: Proceedings of the 36th annual ACM/IEEE design automation conference. ACM, 555–561

  • Renewable energy policy network for the 21st century. 2017. Renewables 2017, Global status report. technical report

  • Sayah S, Zehar K (2008) Modified differential evolution algorithm for optimal power flow with non-smooth cost functions. Energy Convers Manage 49:3036–3042

    Article  Google Scholar 

  • Schoenung S (2011) Energy storage systems cost update

  • Scott W, Powell WB (2012) Approximate dynamic programming for energy storage with new results on instrumental variables and projected bellman errors. Submitted to Operations Research (Under Review)

  • Shapiro A (2011) Analysis of stochastic dual dynamic programming method. Eur J Oper Res 209:63–72

    Article  Google Scholar 

  • Shapiro A, Tekaya W, da Costa JP, Soares MP (2013) Risk neutral and risk averse stochastic dual dynamic programming method. Eur J Oper Res 224:375–391

    Article  Google Scholar 

  • Shapiro JF (1979) Mathematical programming: structures and algorithms. Wiley, New York

    Google Scholar 

  • Song H, Liu C-C, Lawarrée J, Dahlgren RW (2000) Optimal electricity supply bidding by markov decision process. IEEE Trans Power Syst 15:618–624

    Article  Google Scholar 

  • Suberu MY, Mustafa MW, Bashir N (2014) Energy storage systems for renewable energy power sector integration and mitigation of intermittency. Renew Sustain Energy Rev 35:499–514

    Article  Google Scholar 

  • Succar S, Denkenberger DC, Williams RH (2012) Optimization of specific rating for wind turbine arrays coupled to compressed air energy storage. Appl Energy 96:222–234

    Article  Google Scholar 

  • Tan X, Li Q, Wang H (2013) Advances and trends of energy storage technology in microgrid. Int J Electr Power Energy Syst 44:179–191

    Article  Google Scholar 

  • Tan Y, Liu W, Qiu Q (2009) Adaptive power management using reinforcement learning. In: Proceedings of the 2009 international conference on computer-aided design. ACM, 461–467

  • Thatte AA, Xie L, Viassolo DE, Singh S (2013) Risk measure based robust bidding strategy for arbitrage using a wind farm and energy storage. IEEE Trans Smart Grid 4:2191–2199

    Article  Google Scholar 

  • Topaloglu H, Powell WB (2006) Dynamic-programming approximations for stochastic time-staged integer multicommodity-flow problems. INFORMS J Comput 18:31–42

    Article  Google Scholar 

  • Turgeon A (1980) Optimal operation of multireservoir power systems with stochastic inflows. Water Resour Res 16:275–283

    Article  Google Scholar 

  • Wee J-H (2013) A review on carbon dioxide capture and storage technology using coal fly ash. Appl Energy 106:143–151

    Article  Google Scholar 

  • Yang J-S, Chen N (1989) Short term hydrothermal coordination using multi-pass dynamic programming. IEEE Trans Power Syst 4:1050–1056

    Article  Google Scholar 

  • Yu Y, Luh PB, Litvinov E, Zheng T, Zhao J, Zhao F (2015) Grid integration of distributed wind generation: hybrid markovian and interval unit commitment. IEEE Trans Smart Grid 6:3061–3072

    Article  Google Scholar 

  • Zéphyr L, Lang P, Lamond BF (2015) Controlled approximation of the value function in stochastic dynamic programming for multi-reservoir systems. CMS 12:539–557

    Article  Google Scholar 

  • Zéphyr L, Lang P, Lamond BF, Côté P (2017) Approximate stochastic dynamic programming for hydroelectric production planning. Eur J Oper Res 262:586–601

    Article  Google Scholar 

  • Zhou Y, Scheller-Wolf AA, Secomandi N, Smith S (2014) Managing wind-based electricity generation in the presence of storage and transmission capacity. SSRN 1962414

  • Zurn H, Quintana V (1975) Generator maintenance scheduling via successive approximations dynamic programming. IEEE Trans Power Appar Syst 94:665–671

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Science Foundation under grant ECCS-1453615. The authors acknowledge constructive comments from two anonymous referees that helped improve the paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Luckny Zéphyr.

Appendix

Appendix

1.1 Nomenclature

1.1.1 Sets

N :

Buses

L :

Transmission lines

M :

Wind farms

G :

Conventional generators

\(G_n\) :

Conventional generators at bus n

S :

Storage devices

\(O_n\) :

Transmission lines that leave node n

\(I_n\) :

Transmission lines that enter node n

\(\mathbf{R}\) :

Real numbers

\(\mathbf{R}^+\) :

Non negative numbers

1.1.2 Parameters

T :

Length of the planning horizon

t :

Time period

\({\hat{D}}_{nt}\) :

Forecast of the load, in period t at bus n

\({\underline{p}}_g\) :

(resp. \({\bar{p}}_g\)) Lower (resp. upper) bound on generator g output

\({\underline{\lambda }}_g\) :

(resp. \({\bar{\lambda }}_g\)) Ramp down (resp. ramp up) limit on generator g

\({\tilde{W}}_t\) :

Random vector of outputs from the wind turbines in period t

\(w_{mt}\) :

A particular realization of the stochastic process \(\{{\tilde{W}}_{mt}\}\)

\({\bar{e}}_l\) :

Bound on power trough line l

\(B_l\) :

Susceptance of line l

\({\underline{s}}_n\) :

(resp. \({\bar{s}}_n\)) Lower (resp. upper) bound on the level of the storage at bus n

\(c_n\) :

(resp. \(d_n\)) Efficiency coefficient of charging (resp. discharging) of the storage device at bus n

\(\delta _{n}=\) :

\(\left\{ \begin{array}{ll} 1 &{} \hbox {if there is a storage facility at node n},\\ 0 &{} \hbox {otherwise}. \end{array} \right. \)

1.1.3 Decision variables

\(p_{gt}\) :

Power generation from generator g in period t

\(e_{lt}\) :

Power flowing through transmission line l in period t

\(\theta _{nt}\) :

Phase angle of bus n in period t

\(s_{nt}\) :

Level of stored energy at bus n in the beginning of period t

\(\Delta ^+_{nt}\) :

(resp. \(\Delta ^-_{nt}\)) Positive (resp. negative) variation in the level of charge from the beginning through the end of period t

\(\kappa ^+_{nt}\) :

(resp. \(\kappa ^-_{nt}\)) Power excess absorbed (resp. delivered) by the storage unit located at bus n

\(\Delta _{nt}\) :

Power absorbed or delivered by the storage unit at bus n

1.1.4 Functions

\(CP_{gt} (p_{gt})\) :

Cost function of generator g in period t

\(CS_{nt}(\Delta _{nt})\) :

Cost associated with varying the stored energy at bus n

1.1.5 Operators

\({\mathbb {E}}\) :

Mathematical expectation

|X|:

Cardinality of the set X

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zéphyr, L., Anderson, C.L. Stochastic dynamic programming approach to managing power system uncertainty with distributed storage. Comput Manag Sci 15, 87–110 (2018). https://doi.org/10.1007/s10287-017-0297-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10287-017-0297-2

Keywords

Navigation