Energy Systems

, Volume 1, Issue 1, pp 61–77

Optimal day-ahead trading and storage of renewable energies—an approximate dynamic programming approach

Original Paper


A renewable power producer who trades on a day-ahead market sells electricity under supply and price uncertainty. Investments in energy storage mitigate the associated financial risks and allow for decoupling the timing of supply and delivery. This paper introduces a model of the optimal bidding strategy for a hybrid system of renewable power generation and energy storage. We formulate the problem as a continuous-state Markov decision process and present a solution based on approximate dynamic programming. We propose an algorithm that combines approximate policy iteration with Least Squares Policy Evaluation (LSPE) which is used to estimate the weights of a polynomial value function approximation. We find that the approximate policies produce significantly better results for the continuous state space than an optimal discrete policy obtained by linear programming. A numerical analysis of the response surface of rewards on model parameters reveals that supply uncertainty, imbalance costs and a negative correlation of market price and supplies are the main drivers for investments in energy storage. Supply and price autocorrelation, on the other hand, have a negative effect on the value of storage.


Renewable energies Energy storage Day-ahead market Optimal bidding strategy Approximate dynamic programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bathurst, G.N., Strbac, G.: Value of combining energy storage and wind in short-term energy and balancing markets. Electr. Power Syst. Res. 67, 1–8 (2003) CrossRefGoogle Scholar
  2. 2.
    Bathurst, G.N., Weatherill, J., Strbac, G.: Trading wind generation in short term energy markets. IEEE Trans. Power Syst. 17(3), 782–789 (2002) CrossRefGoogle Scholar
  3. 3.
    Bertsekas, D.: Dynamic Programming and Optimal Control, 3rd edn., vol. 2. Athena Scientific, Nashua (2007) Google Scholar
  4. 4.
    Brunetto, C., Tina, G.: Optimal hydrogen storage sizing for wind power plants in day ahead electricity markets. IET Renew. Power Gener. 1(4), 220–226 (2007) CrossRefGoogle Scholar
  5. 5.
    Chen, C.C.P., Tsui, K.-L., Barton, R.R., Meckesheimer, M.: A review on design, modeling and applications of computer experiments. IIE Trans. 38(4), 273–291 (2006) CrossRefGoogle Scholar
  6. 6.
    Fleten, S.E., Kristoffersen, T.K.: Stochastic programming for optimizing bidding strategies of a Nordic hydropower producer. Eur. J. Oper. Res. 181, 916–928 (2007) MATHCrossRefGoogle Scholar
  7. 7.
    García-González, J., De la Muela, R.M.R., Santos, L.M., González, A.M.: Stochastic joint optimization of wind generation and pumped-storage units in an electricity market. IEEE Trans. Power Syst. 23(2), 460–468 (2008) CrossRefGoogle Scholar
  8. 8.
    Graves, F., Jenkin, T., Murphy, D.: Opportunities for electricity storage in deregulating markets. Electr. J. 12(8), 46–56 (1999) CrossRefGoogle Scholar
  9. 9.
    Gujarati, D.N.: Basic Econometrics, 5th edn. McGraw–Hill, New York (2009) Google Scholar
  10. 10.
    Korpås, M., Holen, A.T.: Operation planning of hydrogen storage connected to wind power operating in a power market. IEEE Trans. Energy Convers. 21(3), 599–606 (2006) CrossRefGoogle Scholar
  11. 11.
    Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. J. Mach. Learn. Res. 4, 1107–1149 (2003) CrossRefMathSciNetGoogle Scholar
  12. 12.
    Matevosyan, J., Söder, L.: Minimization of imbalance cost trading wind power on the short-term power market. IEEE Trans. Power Syst. 21(3), 1396–1404 (2006) CrossRefGoogle Scholar
  13. 13.
    Neubarth, J., Woll, O., Weber, C., Gerecht, M.: Beeinflussung der Spotmarktpreise durch Windstromerzeugung. Energiewirtsch. Tagesfr. 56(7), 42–45 (2006) Google Scholar
  14. 14.
    Powell, W.B.: Approximate Dynamic Programming. Wiley, New York (2007) MATHCrossRefGoogle Scholar
  15. 15.
    Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (1994) MATHGoogle Scholar
  16. 16.
    Schainker, R.B.: Executive overview: energy storage options for a sustainable energy future. In: IEEE Power Engineering Society General Meeting, pp. 2309–2314. IEEE Press, New York (2004) Google Scholar
  17. 17.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998) Google Scholar
  18. 18.
    Wen, F., David, A.K.: Strategic bidding in competitive electricity markets: a literature survey. In: IEEE Power Engineering Society Summer Meeting, pp. 2168–2173 (2000) Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Department of Business AdministrationUniversity of ViennaViennaAustria

Personalised recommendations