Cluster Computing

, Volume 22, Supplement 4, pp 7831–7839 | Cite as

An on-line detection method for outliers of dynamic unstable measurement data

  • Weixing Su
  • Fang LiuEmail author
  • Jianjun Zhao
  • Maowei He
  • Hanning Chen


Aiming at the characteristics of the vibration data collected by the regulation system during the unstable regulation process and the deficiency of the traditional wavelet anomaly detection method, an on-line anomaly detection method combining the autoregressive and the wavelet analysis is proposed to detect the abnormal data of the regulation system. By introducing the improved robust AR model, this method can overcome the problem that the time and frequency of traditional anomaly detection using wavelet analysis method cannot be well balanced, and ensure the rationality of abnormal value detection of process data. Considering the general parameters of the regulation system is time-varying and has strong dynamic characteristics, the method proposed in this paper has the ability of online detection and real-time updating of parameters to ensure that the control parameters of time-varying control system; In order to avoid the problem that the traditional anomaly detection method needs to set the detection threshold in advance, HMM is introduced to analyze the wavelet coefficients and update the HMM parameters online, which can ensure that the HMM can well reflect the actual distribution of process data anomalies. It is proved that the method of anomaly data detection proposed in this paper is more suitable for the unstable regulation process data and has certain practicability through experiment and application.


Outlier detection Auto-regression Wavelet HMM Time series 



This research is partially supported by National Natural Science Foundation of China under Grant 51607122. This research is partially supported by State Key Laboratory of Process Automation in Mining & Metallurgy/Beijing Key Laboratory of Process Automation in Mining & Metallurgy Research Fund Project BGRIMM-KZSKL-2017-01. This research is partially supported by the basic scientific research business funded projects of Tianjin TJPUZK20170129 and by Tianjin Province Science and Technology projects 16ZLZDZF00150.


  1. 1.
    Liu, S., Hou, Z., Yin, C.: Data-driven modeling for UGI gasification processes via an enhanced genetic BP neural network with link switches [J]. IEEE Trans. Neural Netw. Learn. 27(12), 2718–29 (2016)CrossRefGoogle Scholar
  2. 2.
    Yan, J., Li, K., Bai, E.: Special condition wind power forecasting based on gaussian process and similar historical data [C]. General Meeting of the IEEE-Power-and-Energy-Society, Denver (2015)Google Scholar
  3. 3.
    Bourgin, F., Ramos, M.H.: Investigating the interactions between data assimilation and post-processing in hydrological ensemble forecasting [J]. J. Hydrol. 519, 2775–2784 (2014)CrossRefGoogle Scholar
  4. 4.
    Peng, K., Li, Q., Zhan, K.: Quality-related process monitoring for dynamic non-Gaussian batch process with multi-phase using a new data-driven method [J]. Neurocomputing 214, 318–28 (2016)CrossRefGoogle Scholar
  5. 5.
    Zhang, H., Albin, S.L., Wagner, S.R.: Determining statistical process control baseline periods in long historical data streams [J]. J. Qual. Technol. 42(1), 21–35 (2010)CrossRefGoogle Scholar
  6. 6.
    Keller, S., Popenheim, A.: Analysis process for different polymere electrolyte fuel cell stacks to derive a control stratgy based impedance data [J]. Fuel Cells 14(5), 758–768 (2014)CrossRefGoogle Scholar
  7. 7.
    Grubbs, F.E.: Procedures for detecting outlying observations in samples [J]. Technometrics 11(1), 1–21 (1969)CrossRefGoogle Scholar
  8. 8.
    Barnet, V., Lewis, T.: Outlier in Statistical Data [M]. Wiley, Chichester (1994)Google Scholar
  9. 9.
    Knorr, E.M., Ng R.T.: Finding Intensional Knowledge of Distance-based Outliers [C]. In: Proceedings of the Twenty-Fifth International Conference on Very Large Data Bases, pp. 211–222 (1999)Google Scholar
  10. 10.
    Knorr, E.M., Ng, R.T.: Algorithms for Mining Distance-based Outliers [C]. In: Proceedings of the Twenty-fourth International Conference on Very Large Data Bases, pp. 392–403 (1998)Google Scholar
  11. 11.
    Ramaswamy, S., Rastogi, R., Shim, K.: Efficient algorithms for mining outliers from large data sets [C]. In: Proceeding of the ACM SIGMOD International Conference on Management of Data Dallas, ACM Press, Teas, pp. 427–438 (2000)Google Scholar
  12. 12.
    Seo, H.S.: A sequential outlier detecting method using a clustering algorithm [J]. Korean J. Appl. Stat. 29(4), 699–706 (2016)CrossRefGoogle Scholar
  13. 13.
    Zhang, Q., Wang, C., Zhao, J.: Outlier detecting algorithm based on clustering and local information [J]. J. Jilin Univ. 50(6), 1214–17 (2012)Google Scholar
  14. 14.
    Bharti, S., Pattanaik, K.K.: Gravitational outlier detection for wireless sensor networks [J]. Int. J. Commun. 29(13), 2015–2027 (2016)Google Scholar
  15. 15.
    Su, W., Zhu, Y., Liu, F.: An online outlier detection method based on wavelet technique and robust RBF network [J]. Trans. Inst. Meas. Control 35(8), 1046–1057 (2013)CrossRefGoogle Scholar
  16. 16.
    Pittner, S., Kamarthi, S.V.: Feature extraction from wavelet coefficients for pattern recognition tasks [J]. IEEE Trans. Pattern Anal. Mach. Intell. 21(1), 83–88 (1999)CrossRefGoogle Scholar
  17. 17.
    Mallat, S., Hwang, W.L.: Singularity detection and processing with wavelets [J]. IEEE Trans. Inf. Theory 38(2), 617–642 (1992)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Griffiths, K.R., Hicks, B.J., Keogh, P.S.: Wavelet analysis to decompose a vibration simulation signal to improve pre-distribution testing of packaging [J]. Mech. Syst. Signal Process. 76–77, 780–795 (2016)CrossRefGoogle Scholar
  19. 19.
    Durocher, M., Lee, T.S., Ouarda, T.B.M.J.: Hybrid signal detection approach for hydro-meteorological variables combining EMD and cross-wavelet analysis [J]. Int. J. Climatol. 36(4), 1600–1613 (2016)CrossRefGoogle Scholar
  20. 20.
    Han, J., Kamber, M.: Data Mining Concepts and Techniques [M], vol. 8, pp. 254–257. Machinery Industry Press, Beijing (2001)Google Scholar
  21. 21.
    Takeuchi, J., Yamanishi, K.: A unifying framework for detecting outliers and change points from time series [J]. IEEE Trans. Knowl. Data Eng. 18(4), 482–492 (2006)CrossRefGoogle Scholar
  22. 22.
    Chuanli, Z., Yizhuang, H., Xiaoxu, M., Wenzhe, L., Guoxing, W.: A new approach to detect transformer inrush current by applying wavelet transform [J]. Proc. POWERCON ’98 2, 1040–1044 (1998)CrossRefGoogle Scholar
  23. 23.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition [J]. Proc. IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  24. 24.
    Bilmes, J.A.: What HMMs can do [J]. IEICE Trans. Inf. Syst. E89–D(3), 869–891 (2006)CrossRefGoogle Scholar
  25. 25.
    Hui, L.L.: Implementing the Viterbi Algorithm—Fundamentals and real-rime issues for processor designers. IEEE Signal Process. Mag. 1053–5888, 42–52 (1995)Google Scholar
  26. 26.
    Alexandridis, A., Sarimveis, H., Bafas, G.: Anew algorithm for online structure and parameter adaptation of RBF networks [J]. Neural Netw. 16, 1003–1017 (2003)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  • Weixing Su
    • 1
  • Fang Liu
    • 1
    Email author
  • Jianjun Zhao
    • 2
  • Maowei He
    • 1
  • Hanning Chen
    • 1
  1. 1.School of Computer Science & Software EngineeringTIANJIN Polytechnic UniversityTianjinChina
  2. 2.Bei Jing General Research Institute of Mining & MetallurgyBeijingChina

Personalised recommendations