Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement-learning-based damping control scheme of a PV plant in wide-area measurement system

  • Original Paper
  • Published:
Electrical Engineering Aims and scope Submit manuscript

Abstract

Modern power systems are witnessing a noticeable increase in the integration of low-inertia renewable sources which require robust control schemes to damp out low-frequency oscillations emerged by this expansion. This paper proposes a reinforcement learning (RL)-based controller using a deep deterministic policy gradient (DDPG) algorithm to damp inter-area oscillations. The learning process of the controller is enhanced using a discrete reward-function which is selected to be a reciprocal function of the input error. To allow the agent to drive the total error lower and lower, both the absolute error and integral of error are included in the observation state vector. A two-area system with a solar plant integrated is used as the test system. The controller obtains its global input signal from PMU devices in wide-area measurement systems using frequency information. A comprehensive analysis is presented using several analytical control tools including time-domain simulation, pole-zero plot, mode shape, frequency response, and participation factor map. Furthermore, a package of programs has been developed for this study using MATLAB and Simulink. The communication latency is also included in the design of the controller considering constant and variable practical values of latency. The proposed controller demonstrates its effectiveness in damping inter-area oscillation and improving the system stability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Hashmy Y, Yu Z, Shi D, Weng Y (2020) Wide-area measurement system-based low frequency oscillation damping control through reinforcement learning. IEEE Trans Smart Grid 11(6):5072–5083. https://doi.org/10.1109/TSG.2020.3008364

    Article  Google Scholar 

  2. Abdulrahman I, Belkacemi R, Radman G (2021) Power oscillations damping using wide-area-based solar plant considering adaptive time-delay compensation. Springer J Energy Syst 12:459–489

    Article  Google Scholar 

  3. Cao D et al (2020) Reinforcement learning and its applications in modern power and energy systems: a review. J Modern Power Syst Clean Energy 8(6):1029–1042. https://doi.org/10.35833/MPCE.2020.000552

    Article  Google Scholar 

  4. Sewak M (2019) Deep reinforcement learning frontiers of artificial intelligence. Springer Nature Pte Ltd, Singapore

    Book  MATH  Google Scholar 

  5. Dong H et al (2020) Deep reinforcement learning fundamentals research and applications. Springer Nature Pte Ltd, Springer

    Book  MATH  Google Scholar 

  6. Zhang G, Hu W, Zhao J, Cao D, Chen Z, Blaabjerg F (2021) A novel deep reinforcement learning enabled multi-band PSS for multi-mode oscillation control. IEEE Trans Power Syst 36(4):3794–3797. https://doi.org/10.1109/TPWRS.2021.3067208

    Article  Google Scholar 

  7. Gupta P, Pal A, Vittal V (2022) Coordinated wide-area damping control using deep neural networks and reinforcement learning. IEEE Trans Power Syst. https://doi.org/10.1109/TPWRS.2021.3091940

    Article  Google Scholar 

  8. Lee H et al (2019) Artificial neural network control of battery energy storage system to damp-out inter-area oscillations in power systems. Energies 12:3372. https://doi.org/10.3390/en12173372

    Article  Google Scholar 

  9. Younesi A (2017) Application of reinforcement learning for generating optimal control signal to the IPFC for damping of low- frequency oscillations. Wiley Trans Electr Energ Syst. https://doi.org/10.1002/etep.2488

    Article  Google Scholar 

  10. Liu F et al (2016) Robust wide-area damping controller design for inter-area oscillations with signals’ delay. Wiley Trans Electr Energ Syst 11:206–215

    Google Scholar 

  11. Chen C, Cui M, Li F, Yin S, Wang X (2021) Model-free emergency frequency control based on reinforcement learning. IEEE Trans Industr Inf 17(4):2336–2346

    Article  Google Scholar 

  12. Zhang G et al (2020) Deep reinforcement learning-based approach for proportional resonance power system stabilizer to prevent ultra-low-frequency oscillations. IEEE Trans Smart Grid 11(6):5260–5272. https://doi.org/10.1109/TSG.2020.2997790

    Article  Google Scholar 

  13. Yan Z et al (2020) Deep reinforcement learning-based optimal data-driven control of battery energy storage for power system frequency support. In IET Gen Trans Dis. https://doi.org/10.1049/iet-gtd.2020.0884

    Article  Google Scholar 

  14. Abouheaf M et al (2019) Load frequency regulation for multi-area power system using integral reinforcement learning. IET Gen Transm Distrib. https://doi.org/10.1049/iet-gtd.2019.0218

    Article  Google Scholar 

  15. Zheng Y et al (2021) Power system load frequency active disturbance rejection control via reinforcement learning-based memetic particle swarm optimization. IEEE Access 9:116194–116206. https://doi.org/10.1109/ACCESS.2021.3099904

    Article  Google Scholar 

  16. Yan Z, Xu Y (2019) Data-driven load frequency control for stochastic power systems: a deep reinforcement learning method with continuous action search. IEEE Trans Power Syst 34(2):1653–1656. https://doi.org/10.1109/TPWRS.2018.2881359

    Article  MathSciNet  Google Scholar 

  17. Chen X et al (2021) Reinforcement learning for decision-making and control in power systems: Tutorial, Review, and Vision, in arXiv: 2102.01168

  18. Yang T etal (2020) Reinforcement learning in sustainable energy and electric systems: a survey, in Elsevier Annual Reviews in Control

  19. Zhang Z, Zhang D, Qiu RC (2020) Deep reinforcement learning for power system applications: an overview. CSEE J Power and Energy Syst 6(1):213–225. https://doi.org/10.17775/CSEEJPES.2019.00920

    Article  Google Scholar 

  20. Glavic M (2019) (Deep) Reinforcement learning for electric power system control and related problems: a short review and perspectives, in Elsevier Annual Reviews in Control

  21. Zhang D, Han X, Deng C (2018) Review on the research and practice of deep learning and reinforcement learning in smart grids. CSEE J Power and Energy Syst 4(3):362–370. https://doi.org/10.17775/CSEEJPES.2018.00520

    Article  Google Scholar 

  22. Siraskar R (2021) Reinforcement learning for control of valves. Elsevier Mach Learn Appl. https://doi.org/10.1016/j.mlwa.2021.100030

    Article  Google Scholar 

  23. Abdulrahman I, Radman G (2018) Wide-area-based adaptive neuro-fuzzy SVC controller for damping interarea oscillations. Can J Electric Comput Eng 41(3):133–144

    Article  Google Scholar 

  24. WECC Renewable Energy Modeling Task Force (2014) WECC solar plant dynamic modeling guidelines

  25. Sandia National Laboratories (2013) Generic solar photovoltaic system dynamic simulation model specification”, California

  26. Islam MR, Rahman F, Xu W (2016) Green energy and technology: advances in solar photovoltaic power plants. Springer, Berlin

    Google Scholar 

  27. Pauer P, Pai MA, Chow JH (2017) Power system dynamics and stability, 2nd edn. Wiley, Amsterdam

    Google Scholar 

  28. Kundur P et al (2004) Definition and classification of power system stability IEEE/CIGRE joint task force on stability terms and definitions. IEEE Trans Power Syst 19(3):1387–1401

    Article  Google Scholar 

  29. Pal B, Chaudhuri B (2005) Robust control in power system. Springer, New York

    Google Scholar 

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Technical Information Systems Engineering, Erbil Technical Engineering College, Erbil Polytechnic University, Erbil, 44001, Kurdistan region, Iraq

    Ismael Abdulrahman

Authors
  1. Ismael Abdulrahman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ismael Abdulrahman.

Ethics declarations

Conflict of interest

There are no competing interests in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 64950 kb)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abdulrahman, I. Reinforcement-learning-based damping control scheme of a PV plant in wide-area measurement system. Electr Eng 104, 4213–4225 (2022). https://doi.org/10.1007/s00202-022-01615-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00202-022-01615-3

Keywords

Search

Navigation