Skip to main content
SpringerLink
Log in

IRFLMDNN: hybrid model for PMU data anomaly detection and re-filling with improved random forest and Levenberg Marquardt algorithm optimized dynamic neural network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Phasor Measurement Units (PMU) are capable to generate multi-dimensional time series data, which is one of the most important parts for monitoring power system operation. However, various internal and external factors frequently cause the system to generate anomalous data randomly, so we expect to clean and re-fill the raw PMU data with anomalies to provide support for further advanced applications such as situational awareness, early warning, and dispatch control of the system. Existing methods mostly use classical mathematics and traditional machine learning to analyze PMU data, which makes it difficult to identify the pattern changes of the data under multiple operating conditions in power systems. In this paper, we propose a hybrid model named IRFLMDNN, which consists of an improved CART random forest model and a dynamic neural network optimized by the Levenberg Marquardt algorithm for PMU data anomaly detection and adaptive data re-filling, respectively. Experimental results based on the IEEE 39-node 10-machine New England Power System show that the proposed method has accurate and robust anomaly detection and data refilling performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Phadke AG, Thorp JS (2008) Synchronized phasor measurements and their applications. Springer, New York, p 81

    Book  MATH  Google Scholar 

  2. Phadke AG, Bi T (2018) Phasor measurement units, WAMS, and their applications in protection and control of power systems. J Mod Power Syst Clean Energy 6(4):619–629

    Article  Google Scholar 

  3. Duan G, Yan Y, Xie X, Tao H, Yang D, Li J (2015) Development status quo and tendency of wide area phasor measuring technology. Autom Electr Power Syst 39(1):73–80 (in Chinese)

    Google Scholar 

  4. Xu F, Xue A, Chang N, Kong H, Xu J (2021) Research status and prospects of detection, correction and recovery for abnormal synchrophasor data in power system. Proc CSEE 41(20):6869–6886 (in Chinese)

    Google Scholar 

  5. Zhang Q, Vittal V, Heydt GT, Logic N, Sturgill S (2011) The integrated calibration of synchronized phasor measurement data in power transmission systems. IEEE Trans Power Delivery 26(4):2573–2581

    Article  Google Scholar 

  6. Huang C, Li F, Zhou D, Guo J, Pan Z, Liu Y, Liu Y (2016) Data quality issues for synchrophasor applications part I: a review. J Modern Power Syst Clean Energy 4(3):342–352

    Article  Google Scholar 

  7. Zhang Q, Vittal V, Heydt G, Chakhchoukh Y, Logic N, Sturgill S (2012) The time skew problem in PMU measurements. In: 2012 IEEE Power and energy society general meeting (pp. 1–6). IEEE

  8. Shepard DP, Humphreys TE, Fansler AA (2012) Evaluation of the vulnerability of phasor measurement units to GPS spoofing attacks. Int J Crit Infrastruct Prot 5(3–4):146–153

    Article  Google Scholar 

  9. Zhou M, Wang Y, Srivastava AK, Wu Y, Banerjee P (2018) Ensemble-based algorithm for synchrophasor data anomaly detection. IEEE Trans Smart Grid 10(3):2979–2988

    Article  Google Scholar 

  10. Cai J Fan J (2022) Perturbation learning based anomaly detection. In: The 36th conference on neural information processing systems (pp: 1–18). NeurIPS

  11. Fan J, Chow TW, Qin SJ (2022) Kernel-based statistical process monitoring and fault detection in the presence of missing data. IEEE Trans Ind Inf 18(7):4477–4487

    Article  Google Scholar 

  12. Marquardt DW (1963) An algorithm for least-squares estimation of nonlinear parameters. J Soc Ind Appl Math 11(2):431–441

    Article  MathSciNet  MATH  Google Scholar 

  13. Wang X, Shi D, Wang Z, Xu C, Zhang Q, Zhang X, Yu Z (2017) Online calibration of phasor measurement unit using density-based spatial clustering. IEEE Trans Power Deliv 33(3):1081–1090

    Article  Google Scholar 

  14. Lu X, Shi D, Zhu B, Wang Z, Luo J, Su D, Xu C (2017) PMU assisted power system parameter calibration at Jiangsu electric power company. In: 2017 IEEE Power & Energy Society General Meeting (pp. 1–5). IEEE

  15. Kong H, Xue A, Xu F, Leng S (2019) A novel detection method for abnormal PMU amplitude data obtained in both ends of transmission line. In: 2019 IEEE sustainable power and energy conference (iSPEC) (pp. 2916–2921). IEEE

  16. Wan C, Chen H, Guo M, Liang Z (2016) Wrong data identification and correction for WAMS. In: 2016 IEEE PES Asia-pacific power and energy engineering conference (APPEEC) (pp. 1903–1907). IEEE

  17. Wu M, Xie L (2016) Online detection of low-quality synchrophasor measurements: a data-driven approach. IEEE Trans Power Syst 32(4):2817–2827

    Article  Google Scholar 

  18. Chen Y, Chen W, Yao W, Liao S, Miao S (2016) Rapid identification and recovery of wrong WAMS data. Electr Power Autom Equip 36(12):95–101 (in Chinese)

    Google Scholar 

  19. Vanfretti L, Chow JH, Sarawgi S, Fardanesh B (2010) A phasor-data-based state estimator incorporating phase bias correction. IEEE Trans Power Syst 26(1):111–119

    Article  Google Scholar 

  20. Yao W, Liu Y, Zhou D, Pan Z, Till MJ, Zhao J, Liu Y (2016) Impact of GPS signal loss and its mitigation in power system synchronized measurement devices. IEEE Trans Smart Grid 9(2):1141–1149

    Article  Google Scholar 

  21. Liu Y, Jia Y, Lin Z, Zhang Y, Wang L, Tomsovic K, Liu Y (2011) Impact of GPS signal quality on the performance of phasor measurements. In: 2011 16th international conference on intelligent system applications to power systems (pp.1–6). IEEE

  22. Liu FT, Ting KM, Zhou ZH (2008) Isolation forest. In: 2008 8th IEEE International conference on data mining (pp.413–422). IEEE

  23. Guha S, Mishra N, Roy G, Schrijvers O (2016) Robust random cut forest based anomaly detection on streams. In: International conference on machine learning (pp.2712–2721). PMLR

  24. Xu H, Pang G, Wang Y, Wang Y (2022) Deep isolation forest for anomaly detection. arXiv preprint arXiv:2206.06602

  25. Lin G, Feng X, Guo W, Cui X, Liu S, Jin W, Ding Y (2021) Electricity theft detection based on stacked autoencoder and the undersampling and resampling based random forest algorithm. IEEE Access 9:124044–124058

    Article  Google Scholar 

  26. Bhattacharyya S, Majumder S, Debnath P, Chanda M (2021) Arrhythmic heartbeat classification using ensemble of random forest and support vector machine algorithm. IEEE Trans Artif Intell 2(3):260–268

    Article  Google Scholar 

  27. Wang J, Qiu J, Fang Y, Zhou S (2020) Short term load forecasting based on deep LSTM neural network. Guangdong Electr Power 33(8):62–68 (in Chinese)

    Google Scholar 

  28. Guo Y, Ding Y (2020) Design and implementation of grid information search engine based on knowledge map. University of Chinese Academy of Sciences, Shenyang (in Chinese)

    Google Scholar 

  29. Alhussein M, Aurangzeb K, Haider SI (2020) Hybrid CNN-LSTM model for short-term individual household load forecasting. IEEE Access 8:180544–180557

    Article  Google Scholar 

  30. You Q, Qian Z, Nie Z (2020) Research on filling method of abnormal and missing data of wind turbines. Electr Meas Instrum 57(23):1–8 (in Chinese)

    Google Scholar 

  31. Wang J, Lu Y, Gao Q, Yu Y (2021) Nonlinear predictive control and application in microgrid. J Henan Univ Sci Technol 42(2):46–52 (in Chinese)

    MATH  Google Scholar 

  32. Wan C, Song Y (2021) Theories, methodologies and applications of probabilistic forecasting for probabilistic forecasting for power systems with renewable energy sources. Power Syst Autom 45(1):2–16 (in Chinese)

    Google Scholar 

  33. Bhardwaj R, Vatta S (2013) Implementation of ID3 algorithm. Int J Adv Res Comput Sci Softw Eng 3(6):845–851

    Google Scholar 

  34. Korting TS (2006) C4.5 algorithm and multivariate decision trees. In: Image processing division, national institute for space research–INPE Sao Jose dos Campos–SP, Brazil, 1–5

  35. Denison DGT, Mallick BK, Smith AFM (1998) A bayesian cart algorithm. Biometrika 85(2):363–377

    Article  MathSciNet  MATH  Google Scholar 

  36. Han Y, Huang G, Song S, Yang L, Wang H, Wang Y (2021) Dynamic neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 44(11):7436–7456

    Article  Google Scholar 

  37. Denil M, Shakibi B, Dinh L, Ranzato MA, de Freitas N (2013) Predicting parameters in deep learning. In: Advances in neural information processing Systems, 1–9

  38. Wilamowski BM, Yu H (2010) Improved computation for Levenberg–Marquardt training. IEEE Trans Neural Netw 21(6):930–937

    Article  Google Scholar 

  39. Athay T, Podmore R, Virmani S (1979) A practical method for the direct analysis of transient stability. IEEE Trans Power Appar Syst PAS 98(2):573–584

    Article  Google Scholar 

  40. Son S, Gil MS, Moon YS, Won HS (2016) Anomaly detection of hadoop log data using moving average and 3-sigma. KIPS Trans Softw Data Eng 5(6):283–288

    Article  Google Scholar 

  41. Tian J, Azarian MH, Pecht M (2014) Anomaly detection using self-organizing maps-based k-nearest neighbor algorithm. In: 2014 2nd European Conference of the Prognostics and Health Management Society (ECPHM) (pp.1–9). PHM Society

  42. Huang T, Zhu Y, Zhang Q, Zhu Y, Wang D, Qiu M, Liu L (2013). An lof-based adaptive anomaly detection scheme for cloud computing. In: 2013 IEEE 37th Annual computer software and applications conference workshops (pp. 206–211). IEEE

  43. Erfani SM, Rajasegarar S, Karunasekera S, Leckie C (2016) High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recogn 58:121–134

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the State Key Laboratory of Pyramid Talent Training Project of Beijing University of Civil Engineering and Architecture (JDYC20200324), BUCEA Post Graduate Innovation Project (PG2022132), Security Control and Simulation for Power System and Large Power Generation Equipment (SKLD20M17), Project of Beijing Association of Higher Education (YB2021131), National Natural Science Foundation of China (51407201).

Author information

Authors and Affiliations

  1. School of Mechanical-Electronic and Vehicle Engineering, Beijing University of Civil Engineering and Architecture, Beijing, 100044, China

    Miao Yu, Chenyu Yang, Weize Li, Weijie Du & Jinglin Li

  2. Beijing Engineering Research Center of Monitoring for Construction Safety, Beijing, 100044, China

    Miao Yu, Chenyu Yang, Weize Li, Weijie Du & Jinglin Li

  3. State Key Laboratory of Control and Simulation of Power System and Generation Equipment, Tsinghua University, Beijing, 100084, China

    Miao Yu

Authors
  1. Miao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenyu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weize Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weijie Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinglin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miao Yu.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, M., Yang, C., Li, W. et al. IRFLMDNN: hybrid model for PMU data anomaly detection and re-filling with improved random forest and Levenberg Marquardt algorithm optimized dynamic neural network. Neural Comput & Applic 35, 15563–15572 (2023). https://doi.org/10.1007/s00521-023-08571-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08571-4

Keywords

Search

Navigation