Skip to main content
SpringerLink
Log in

Sniffer-Net: quantitative evaluation of smoke in the wild based on spatial–temporal motion spectrum

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Smoke detection plays an essential role in the wild video surveillance systems for abnormal events warning. In this paper, we introduced a dedicated neural network structure named Sniffer-Net to simultaneously extract smoke dynamic feature robustly and evaluate the smoke concentration accurately. Firstly, we utilize an improved LiteFlowNet to estimate the global optical flow from image sequence. Meanwhile, a Marr–Hildreth method is brought up and fused into this network to distinguish and eliminate occluded regions from global flow map. Then, an evaluation module based on Context-Encoder network is put forward specially to quantify smoke concentration levels. This network, following the improved LiteFlowNet, is modified through replacing the loss function and removing the multiscale scheme and trained to infer approximate smoke optical flow behind occlusion regions. Starting from the statistical view, the irregular RGB/HSV feature spaces are converted into a specific quantitative evaluation space. As a result, the whole evaluation system is responsible to transform the distribution of irregular smoke motion feature into a quantified form of representation. In turn, this transformation endows the system with a novel numerical standard for smoke concentration evaluation. Finally, an accuracy assessment method is applied to compare the results of detected smoke concentration with the human experience prior model, which feedback the accuracy and false detection rate of system algorithm. In the experiments of five smoke datasets, our proposed smoke detection approach is superior to other state-of-the-art methods, and concentration algorithm achieves the satisfactory performance of 97.3% accuracy on some specialized dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Çetin AE, Dimitropoulos K, Gouverneur B et al (2013) Video fire detection—review. Digit Signal Process A Rev J. https://doi.org/10.1016/j.dsp.2013.07.003

    Article  Google Scholar 

  2. Lee CY, Lin CT, Hong CT (2009) Spatio-temporal analysis in smoke detection. In: ICSIPA09—2009 IEEE international conference on signal and image processing applications, conference proceedings

  3. Hui T-W, Tang X, Loy CC (2018) LiteFlowNet: a lightweight convolutional neural network for optical flow estimation, 4–6. https://doi.org/10.1109/CVPR.2018.00936

  4. Ferrari RJ, Zhang H, Kube CR (2007) Real-time detection of steam in video images. Pattern Recognit. https://doi.org/10.1016/j.patcog.2006.07.007

    Article  MATH  Google Scholar 

  5. Maruta H, Nakamura A, Kurokawa F (2010) A new approach for smoke detection with texture analysis and support vector machine. In: IEEE international symposium on industrial electronics

  6. Gunay O, Toreyin BU, Kose K, Cetin AE (2012) Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2012.2183141

    Article  MathSciNet  MATH  Google Scholar 

  7. Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J. https://doi.org/10.1016/j.firesaf.2011.01.001

    Article  Google Scholar 

  8. Zhang H, Sun Y, Liu L et al (2018) ClothingOut: a category-supervised GAN model for clothing segmentation and retrieval. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3691-y

    Article  Google Scholar 

  9. Tian H, Li W, Wang L, Ogunbona P (2012) A novel video-based smoke detection method using image separation. In: Proceedings—IEEE international conference on multimedia and expo

  10. Chen TH, Yin YH, Huang SF, Ye YT (2006) The smoke detection for early fire-alarming system base on video processing. In: Proceedings—2006 international conference on intelligent information hiding and multimedia signal processing, IIH-MSP 2006

  11. Ma L, Wu K, Zhu L (2010) Fire smoke detection in video images using Kalman filter and Gaussian mixture color model. In: Proceedings—international conference on artificial intelligence and computational intelligence, AICI 2010

  12. Kopilovic I, Vagvolgyi B, Sziranyi T (2000) Application of panoramic annular lens for motion analysis tasks: surveillance and smoke detection. In: Proceedings of 15th International Conference on Pattern Recognition ICPR-2000, vol 4, pp 714–717. https://doi.org/10.1109/ICPR.2000.903017

  13. Alamgir N, Nguyen K, Chandran V, Boles W (2018) Combining multi-channel color space with local binary co-occurrence feature descriptors for accurate smoke detection from surveillance videos. Fire Saf J. https://doi.org/10.1016/j.firesaf.2018.09.003

    Article  Google Scholar 

  14. Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec.2008.01.013

    Article  Google Scholar 

  15. Xiong Z, Caballero R, Wang H, et al (2007) Video-based smoke detection: possibilities, techniques, and challenges. In: Proceedings of suppression and detection research and applications

  16. Feiniu Y, et al (2008) Video smoke detection based on accumulation and main motion orientation. J Image Graphics (in Chinese)

  17. Lee CY, Lin CT, Hong CT, Su MT (2012) Smoke detection using spatial and temporal analyses. Int J Innov Comput Inf Control 8:4749–4770

    Google Scholar 

  18. Deldjoo Y, Nazary F, Fotouhi AM (2015) A novel fuzzy-based smoke detection system using dynamic and static smoke features. In: ICEE 2015—proceedings of the 23rd Iranian conference on electrical engineering

  19. Toreyin B, Dedeoglu Y (2005) Wavelet based real-time smoke detection in video. In: 13th European signal processing conference

  20. Surit S, Chatwiriya W (2011) Forest fire smoke detection in video based on digital image processing approach with static and dynamic characteristic analysis. In: Proceedings—1st ACIS/JNU international conference on computers, networks, systems, and industrial engineering, CNSI 2011

  21. Toreyin BU, Dedeoglu Y, Cetin AE (2006) Contour based smoke detection in video using wavelets. In: European signal processing conference

  22. Appana DK, Islam R, Khan SA, Kim JM (2017) A video-based smoke detection using smoke flow pattern and spatial-temporal energy analyses for alarm systems. Inf Sci (N Y). https://doi.org/10.1016/j.ins.2017.08.001

    Article  Google Scholar 

  23. Kolesov I, Karasev P, Tannenbaum A, Haber E (2010) Fire and smoke detection in video with optimal mass transport based optical flow and neural networks. In: Proceedings—international conference on image processing, ICIP

  24. Wang Z, Ben Salah M, Zhang H (2012) Discrete wavelet transform based steam detection with Adaboost. In: 2012 IEEE international conference on information and automation, ICIA 2012

  25. Li X, Chen Z, Wu QMJ, Liu C (2018) 3D parallel fully convolutional networks for real-time video wildfire smoke detection. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2018.2889193

    Article  Google Scholar 

  26. Xu G, Zhang Y, Zhang Q et al (2019) Video smoke detection based on deep saliency network. Fire Saf J 105:277–285. https://doi.org/10.1016/j.firesaf.2019.03.004

    Article  Google Scholar 

  27. Ji Y, Zhang H, Jonathan Wu QM (2018) Saliency detection via conditional adversarial image-to-image network. Neurocomputing. https://doi.org/10.1016/j.neucom.2018.08.013

    Article  Google Scholar 

  28. Yin Z, Shi J (2018) GeoNet: unsupervised learning of dense depth, optical flow and camera pose. https://doi.org/10.1109/CVPR.2018.00212

  29. Tu Z, Xie W, Cao J et al (2017) Variational method for joint optical flow estimation and edge-aware image restoration. Pattern Recognit. https://doi.org/10.1016/j.patcog.2016.10.027

    Article  Google Scholar 

  30. Tu Z, Poppe R, Veltkamp RC (2016) Adaptive guided image filter for warping in variational optical flow computation. Signal Process. https://doi.org/10.1016/j.sigpro.2016.02.018

    Article  Google Scholar 

  31. Yang C, Lu X, Lin Z et al (2017) High-resolution image inpainting using multi-scale neural patch synthesis. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017

  32. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. Clin Toxicol. https://doi.org/10.2507/daaam.scibook.2010.27

    Article  Google Scholar 

  33. Ilg E, Mayer N, Saikia T et al (2017) FlowNet 20: evolution of optical flow estimation with deep networks. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017

  34. Song Y, Yang C, Lin Z et al (2018) Contextual-based image inpainting: infer, match, and translate. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

  35. Ko BC, Cheong KH, Nam JY (2009) Fire detection based on vision sensor and support vector machines. Fire Saf J. https://doi.org/10.1016/j.firesaf.2008.07.006

    Article  Google Scholar 

  36. Grammalidis N, Dimitropoulos K, Cetin E (2017) FIRESENSE database of videos for flame and smoke detection. IEEE Trans Circuits Syst Video Technol 25:339–351. https://doi.org/10.5281/ZENODO.836749

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by National Natural Science Foundation of China (No. 51805312); in part by Shanghai Sailing Program (No. 18YF1409400); in part by Training and Funding Program of Shanghai College Young Teachers (No. ZZGCD15102), and in part by Scientific Research Project of Shanghai University of Engineering Science (No. 2016-19).

Author information

Authors and Affiliations

  1. School of Mechanical and Automotive Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China

    Zeyang Mi, Weiwei Zhang, Xuncheng Wu & Suyun Luo

  2. School of Mechanical and Transportation Engineering, Guangxi University of Science and Technology, Liuzhou, 545006, China

    Qiaoming Gao

Authors
  1. Zeyang Mi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuncheng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiaoming Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Suyun Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiwei Zhang.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mi, Z., Zhang, W., Wu, X. et al. Sniffer-Net: quantitative evaluation of smoke in the wild based on spatial–temporal motion spectrum. Neural Comput & Applic 32, 9165–9180 (2020). https://doi.org/10.1007/s00521-019-04426-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04426-z

Keywords

Search

Navigation