Skip to main content

Advertisement

SpringerLink
Log in

An adaptive multi-agent-based approach to smart grids control and optimization

  • Original Paper
  • Published:
Energy Systems Aims and scope Submit manuscript

Abstract

In this paper, we describe a reinforcement learning-based approach to power management in smart grids. The scenarios we consider are smart grid settings where renewable power sources (e.g. Photovoltaic panels) have unpredictable variations in power output due, for example, to weather or cloud transient effects. Our approach builds on a multi-agent system (MAS)-based infrastructure for the monitoring and coordination of smart grid environments with renewable power sources and configurable energy storage devices (battery banks). Software agents are responsible for tracking and reporting power flow variations at different points in the grid, and to optimally coordinate the engagement of battery banks (i.e. charge/idle/discharge modes) to maintain energy requirements to end-users. Agents are able to share information and coordinate control actions through a parallel communications infrastructure, and are also capable of learning, from experience, how to improve their response strategies for different operational conditions. In this paper we describe our approach and address some of the challenges associated with the communications infrastructure for distributed coordination. We also present some preliminary results of our first simulations using the GridLAB-D simulation environment, created by the US Department of Energy (DoE) at Pacific Northwest National Laboratory (PNNL).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abbey, C., Robinson, J., Joos, G.: Integrating renewable energy sources and storage into isolated diesel generator supplied electric power systems. In: 13th Power Electronics and Motion Control Conference, 2008. EPE-PEMC 2008, pp. 2178–2183 (2008)

    Chapter  Google Scholar 

  2. Agarwal, Y., Weng, T., Gupta, R.: Micro-systems driving smart energy metering in smart grids. In: DAC’07 (2007)

    Google Scholar 

  3. Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32, 48–77 (2003)

    Article  MathSciNet  Google Scholar 

  4. Caron, S., Kesidis, G.: Incentive-based energy consumption scheduling algorithms for the smart grid. In: First IEEE International Conference on Smart Grid Communications, (SmartGridComm), 2010, pp. 391–396. IEEE, New York (2010)

    Chapter  Google Scholar 

  5. Chassin, D., Schneider, K., Gerkensmeyer, C.: Gridlab-d: an open-source power systems modeling and simulation environment. In: Transmission and Distribution Conference and Exposition, 2008. D. IEEE/PES, pp. 1–5. IEEE, New York (2008). doi:10.1109/TDC.2008.4517260

    Chapter  Google Scholar 

  6. Chinchuluun, A., Pardalos, P., Enkhbat, R.: Optimization and Optimal Control: Theory and Applications, vol. 39. Springer, Berlin (2010)

    Book  MATH  Google Scholar 

  7. Cupp, J., Beehler, M.: Implementing Smart Grid Communications (2008). Burns & McDonnell Marketing Communications

    Google Scholar 

  8. Dagdougui, H., Minciardi, R., Ouammi, A., Sacile, R.: Optimal control of a regional power microgrid network driven by wind and solar energy. In: 2011 IEEE International Systems Conference (SysCon), pp. 86–90 (2011)

    Chapter  Google Scholar 

  9. Dam, Q., Mohagheghi, S., Stoupis, J.: Intelligent demand response scheme for customer side load management. In: Energy 2030 Conference, 2008. ENERGY 2008, pp. 1–7. IEEE, New York (2008)

    Chapter  Google Scholar 

  10. Dave Cliff, J.B.: Minimal Intelligence Agents for Bargaining Behaviors in Market-Based Environments (1997)

    Google Scholar 

  11. Divya, K., Ãstergaard, J.: Battery energy storage technology for power systems: an overview. Electr. Power Syst. Res. 79(4), 511–520 (2009)

    Article  Google Scholar 

  12. Hatami, S., Pedram, M.: Minimizing the electricity bill of cooperative users under a quasi-dynamic pricing model. In: 2010 First IEEE International Conference on Smart Grid Communications (SmartGridComm), pp. 421–426. IEEE, New York (2010)

    Chapter  Google Scholar 

  13. Kaldellis, J., Zafirakis, D.: Optimum energy storage techniques for the improvement of renewable energy sources-based electricity generation economic efficiency. Energy 32(12), 2295–2305 (2007)

    Article  Google Scholar 

  14. Kersting, W.: Radial distribution test feeders. In: Power Engineering Society Winter Meeting, 2001. IEEE, vol. 2, pp. 908–912. IEEE, New York (2001)

    Google Scholar 

  15. Korpaas, M., Holen, A.T., Hildrum, R.: Operation and sizing of energy storage for wind power plants in a market system. Int. J. Electr. Power Energy Syst. 25(8), 599–606 (2003)

    Article  Google Scholar 

  16. Lagorse, J., Paire, D., Miraoui, A.: A multi-agent system for energy management of distributed power sources. Renew. Energy 35(1), 174–182 (2010)

    Article  Google Scholar 

  17. Löhndorf, N., Minner, S.: Optimal day-ahead trading and storage of renewable energies—an approximate dynamic programming approach. Energy Syst. 1, 61–77 (2010)

    Article  Google Scholar 

  18. O’Neill, D., Levorato, M., Goldsmith, A., Mitra, U.: Residential demand response using reinforcement learning. In: 2010 First IEEE International Conference on Smart Grid Communications (SmartGridComm), pp. 409–414. IEEE, New York (2010)

    Chapter  Google Scholar 

  19. Ramchurn, S., Vytelingum, P., Rogers, A., Jennings, N.: Agent-based homeostatic control for green energy in the smart grid. ACM Trans. Intel. Syst. Technol. 2(4) (2011)

  20. Roche, R., Blunier, B., Miraoui, A., Hilaire, V., Koukam, A.: Multi-agent systems for grid energy management: a short review. In: IECON 2010—36th Annual Conference on IEEE Industrial Electronics Society, pp. 3341–3346 (2010)

    Chapter  Google Scholar 

  21. Samadi, P., Mohsenian-Rad, A., Schober, R., Wong, V., Jatskevich, J.: Optimal real-time pricing algorithm based on utility maximization for smart grid. In: 2010 First IEEE International Conference on Smart Grid Communications (SmartGridComm), pp. 415–420. IEEE, New York (2010)

    Chapter  Google Scholar 

  22. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  23. Suvire, G., Mercado, P., Ontiveros, L.: Comparative analysis of energy storage technologies to compensate wind power short-term fluctuations. In: Transmission and Distribution Conference and Exposition: Latin America (T D-LA), 2010 IEEE/PES, pp. 522–528 (2010)

    Chapter  Google Scholar 

  24. Teleke, S., Baran, M., Bhattacharya, S., Huang, A.: Rule-based control of battery energy storage for dispatching intermittent renewable sources. IEEE Trans. Sustain. Energy 1(3), 117–124 (2010)

    Article  Google Scholar 

  25. Vandael, S., De Craemer, K., Boucké, N., Holvoet, T., Deconinck, G.: Decentralized coordination of plug-in hybrid vehicles for imbalance reduction in a Smart Grid. In: Tumer, K., Yolum, P., Sonenberg, E., Stone, P. (eds.) Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2011), pp. 803–810. ifaamas (2011)

    Google Scholar 

  26. Venayagamoorthy, G.: Potentials and promises of computational intelligence for smart grids. In: Power & Energy Society General Meeting, 2009. PES’09, IEEE, pp. 1–6. IEEE, New York (2009)

    Chapter  Google Scholar 

  27. Vytelingum, P., Voice, T., Ramchurn, S., Rogers, A., Jennings, N.: Theoretical and practical foundations of large-scale agent-based micro-storage in the smart grid. J. Art. Intel. Res. 42, 765–813 (2011)

    MATH  Google Scholar 

  28. Vytelingum, P., Voice, T.D., Ramchurn, S.D., Rogers, A., Jennings, N.R.: Agent-based micro-storage management for the smart grid. In: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: vol. 1 AAMAS ’10, pp. 39–46. International Foundation for Autonomous Agents and Multiagent Systems, Richland (2010)

    Google Scholar 

  29. Wei, C., Hu, H., Chen, Q., Yang, G.: Learning agents for storage devices management in the smart grid. In: 2010 International Conference on Computational Intelligence and Software Engineering (CiSE), pp. 1–4. IEEE, New York (2010)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Florida Institute of Technology, 150 W. University Blvd., Melbourne, FL, 32901, USA

    Marco Carvalho

  2. Institute for Human and Machine Cognition, 15 SE Osceola Ave, Ocala, FL, 34471, USA

    Carlos Perez & Adrian Granados

Authors
  1. Marco Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Perez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adrian Granados
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marco Carvalho.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Carvalho, M., Perez, C. & Granados, A. An adaptive multi-agent-based approach to smart grids control and optimization. Energy Syst 3, 61–76 (2012). https://doi.org/10.1007/s12667-012-0054-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12667-012-0054-0

Keywords

Search

Navigation