Skip to main content
SpringerLink
Log in

Networked Fault Detection of Field Equipment from Monitoring System Based on Fusing of Motion Sensing and Appearance Information

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recent development of Internet of Things (IoT) technologies has triggered a soaring application of intelligent monitoring systems. A sensitive online fault detection for industrial equipment is of utmost significance to improve industrial intelligence based on video sensor network. In this paper, we propose the Appearance and Motion SVM (AMSVM), an online fault detection method fusing multimodal information based on One-Class SVM (OCSVM), to monitor the working conditions of unsupervised equipment in the field. It utilizes multimodal features to detect faults in terms of the appearance and motion patterns of equipment. The motion pattern was generated using the OCSVM-encoded histogram of optical flow orientation (HOFO), and meanwhile we employed Local Binary Pattern Histogram (LBPH) to extract texture features to train OCSVM, depicting appearance patterns. Then, decision level information (i.e., appearance and motion patterns) are combined to produce a more precise characteristic for fault detection. The proposed method herein was validated on several industrial video surveillance data set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Transactions on Pattern Analysis & Machine Intelligence 30(3):555–560

    Article  Google Scholar 

  2. Alvar M, Torsello A, Sanchez-Miralles A, Armingol JM (2014) Abnormal behavior detection using dominant sets. Machine Vision & Applications 25(5):1351–1368

    Article  Google Scholar 

  3. Barnich O, Van Droogenbroeck M (2011) ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724

    Article  MathSciNet  Google Scholar 

  4. Benezeth, Y., et al. Abnormal events detection based on spatio-temporal co-occurences. in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on 2012.

  5. Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Elsevier Science Inc. 1145–1159.

  6. Chen, Z., et al. Detecting abnormal behaviors in surveillance videos based on fuzzy clustering and multiple Auto-Encoders. in IEEE International Conference on Multimedia and Expo. 2015.

  7. Cheng, K.W., Y.T. Chen, and W.H. Fang. Video anomaly detection and localization using hierarchical feature representation and Gaussian process regression. in Computer Vision and Pattern Recognition. 2015.

  8. Cong Y, Yuan J, and Liu J (2011) Sparse reconstruction cost for abnormal event detection. in IEEE Conference on Computer Vision and Pattern Recognition

  9. Feng Y, Yuan Y, Lu X (2016) Learning deep event models for crowd anomaly detection. Neurocomputing 219

  10. Foggia, P., et al. Recognizing human actions by a bag of visual words. in IEEE International Conference on Systems, Man, and Cybernetics. 2013.

  11. Gannot I, Litvak D, Zigel Y System for automatic fall detection for elderly people, 2012. US.

  12. García EA, Frank PM (1997) Deterministic nonlinear observer-based approaches to fault diagnosis: a survey. Control Eng Pract 5(5):663–670

    Article  Google Scholar 

  13. Guo W, Feng Z, Ren X (2017) Object tracking using local multiple features and a posterior probability measure. Sensors 17(4):739

    Article  Google Scholar 

  14. Hamid, R., et al. Detection and explanation of anomalous activities: representing activities as bags of event n-grams. in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on 2005.

  15. Horn BKP and Schunck BG (1981) Determining optical flow. in Techniques and Applications of Image Understanding

  16. Hrúz M, Trojanová J, and Železný M (2011) Local binary pattern based features for sign language recognition. Springer-Verlag New York, Inc. 519-526.

  17. Hu Y, Zhang Y, and Davis LS. (2013) Unsupervised abnormal crowd activity detection using semiparametric scan statistic. in Computer Vision and Pattern Recognition Workshops

  18. Jørgensen EK (2014) Fall detection for the elderly using Microsoft Kinect. Department of Engineering Cybernetics

    Google Scholar 

  19. Kim, J. and K. Grauman. Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates. in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. 2009.

  20. Kim H, Lee S, Kim Y, Lee S, Lee D, Ju J, Myung H (2016) Weighted joint-based human behavior recognition algorithm using only depth information for low-cost intelligent video-surveillance system. Expert Syst Appl 45(C):131–141

    Article  Google Scholar 

  21. Kosmopoulos D, Chatzis SP (2010) Robust visual behavior recognition. IEEE Signal Process Mag 27(5):34–45

    Article  Google Scholar 

  22. Kratz L and Nishino K (2009) Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. in Computer Vision and Pattern Recognition. CVPR 2009. IEEE Conference on. 2009.

  23. Li T, Chang H, Wang M, Ni B, Hong R, Yan S (2015) Crowded scene analysis: a survey. Circuits & Systems for Video Technology IEEE Transactions on 25(3):367–386

    Article  Google Scholar 

  24. Li N et al (2015) Anomaly detection in video surveillance via Gaussian process. International Journal of Pattern Recognition & Artificial Intelligence 29(06):150426191333005

    Article  Google Scholar 

  25. Lu C, Shi J, and Jia J. (2014) Abnormal event detection at 150 FPS in MATLAB. in IEEE International Conference on Computer Vision

  26. Ma C, Trung N, Uchiyama H, Nagahara H, Shimada A, Taniguchi RI (2017) Adapting local features for face detection in thermal image. Sensors 17(12):2741

    Article  Google Scholar 

  27. Mabrouk AB, Zagrouba E (2018) Abnormal behavior recognition for intelligent video surveillance systems : a review. Expert Syst Appl 91:480–491

    Article  Google Scholar 

  28. Martínez-Tomás R, et al. (2015) Identification of loitering human behaviour in video surveillance environments. in International Work-Conference on the Interplay Between Natural and Artificial Computation

  29. Nallaivarothayan H, et al. (2014) An MRF based abnormal event detection approach using motion and appearance features. in IEEE International Conference on Advanced Video and Signal Based Surveillance

  30. Nievas EB, et al. (2011) Violence detection in video using computer vision techniques. in International Conference on Computer Analysis of Images and Patterns

  31. Park D, et al. (2018) LiReD: a light-weight real-time fault detection system for edge computing using LSTM recurrent neural networks. Sensors, 18(7).

  32. Popoola OP, Wang K (2012) Video-based abnormal human behavior recognition—a review. IEEE Transactions on Systems Man & Cybernetics Part C 42(6):865–878

    Article  Google Scholar 

  33. Ramos AR, Neto AJDS, Llanes-Santiago O (2018) An approach to fault diagnosis with online detection of novel faults using fuzzy clustering tools. Expert Syst Appl 113

  34. Roshtkhari, M.J. and M.D. Levine. Online dominant and anomalous behavior detection in videos. in Computer Vision and Pattern Recognition. 2013.

  35. Rougier, C., et al. Fall detection from human shape and motion history using video surveillance. in international conference on advanced information NETWORKING and Applications Workshops. 2007.

  36. Schölkopf B et al (2014) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471

    Article  MathSciNet  Google Scholar 

  37. Sonka M, Hlavac V, and Boyle R (1993) Image Processing, analysis, and machine vision. Chapman & Hall Computing. 685–686.

  38. Uy, A.C.P., et al. Machine vision for traffic violation detection system through genetic algorithm. in International Conference on Humanoid, Nanotechnology, Information Technology,communication and Control, Environment and Management. 2016.

  39. Wang T and Snoussi H (2012) Histograms of optical flow orientation for visual abnormal events detection. in IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance

  40. Wang T, Snoussi H (2014) Detection of abnormal visual events via global optical flow orientation histogram. IEEE Transactions on Information Forensics & Security 9(6):988–998

    Article  Google Scholar 

  41. Wang T, Snoussi H (2015) Detection of abnormal events via optical flow feature analysis. Sensors 15(4):7156–7171

    Article  Google Scholar 

  42. Wang T et al (2013) Online least squares one-class support vector machines-based abnormal visual event detection. Sensors (Basel, Switzerland) 13(12):17130–17155

    Article  Google Scholar 

  43. Wang T et al (2017) Video feature descriptor combining motion and appearance cues with length-invariant characteristics. Optik - International Journal for Light and Electron Optics 157

  44. Williamson R, et al. (1999) Support vector method for novelty detection. in International Conference on Neural Information Processing Systems

  45. Wu C et al (2018) A greedy deep learning method for medical disease analysis. IEEE Access 6:1–1

    Article  Google Scholar 

  46. Xu D, et al. (2017) Detecting anomalous events in videos by learning deep representations of appearance and motion. Elsevier Science Inc. 117–127.

  47. Yu L, Sun X, Huang Z (2016) Robust spatial-temporal deep model for multimedia event detection. Neurocomputing 213:48–53

    Article  Google Scholar 

  48. Zhang Y, Lu H, Zhang L, Ruan X (2016) Combining motion and appearance cues for anomaly detection. Pattern Recogn 51(C):443–452

    Article  Google Scholar 

  49. Zhao, B., F.F. Li, and E.P. Xing. Online detection of unusual events in videos via dynamic sparse coding. in IEEE Conference on Computer Vision and Pattern Recognition. 2011.

  50. Zhou C, Huang S, Xiong N, Yang SH, Li H, Qin Y, Li X (2015) Design and analysis of multimodel-based anomaly intrusion detection Systems in Industrial Process Automation. IEEE Transactions on Systems Man & Cybernetics Systems 45(10):1345–1360

    Article  Google Scholar 

  51. Zhu W, Zhong P (2014) A new one-class SVM based on hidden information. Knowl-Based Syst 60(2):35–43

    Article  Google Scholar 

  52. Zin, T.T., et al. A Markov random walk model for loitering people detection. in Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. 2010.

Download references

Funding

This research was supported by the National Key Research and Development Program of China (No. 2018YFC0810204) and Shanghai Science and Technology Innovation Action Plan Project (16,111,107,502, 17,511,107,203) and Shanghai key lab of modern optical system.

Author information

Authors and Affiliations

  1. School of Optical-Electrical & Computer Engineering, University of Shanghai for Science & Technology, Shanghai, 200093, China

    Chunxue Wu, Shengnan Guo & Jun Ai

  2. O’Neill School of Public and Environmental Affairs, Indiana University, Bloomington, IN, 47405, USA

    Yan Wu

  3. Department of Mathematics and Computer Science, Northeastern State University, Tahlequah, OK, 74464, USA

    Neal N. Xiong

Authors
  1. Chunxue Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shengnan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Ai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Neal N. Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunxue Wu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, C., Guo, S., Wu, Y. et al. Networked Fault Detection of Field Equipment from Monitoring System Based on Fusing of Motion Sensing and Appearance Information. Multimed Tools Appl 79, 16319–16348 (2020). https://doi.org/10.1007/s11042-020-08885-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08885-8

Keywords

Search

Navigation