Skip to main content
SpringerLink
Log in

Controllable smoke image generation network based on smoke imaging principle

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Smoke recognition is a critical task in fire prevention and environmental protection. However, existing methods still face the problems of high false alarm rates and low detection rates because of lacking diverse smoke images and enough supervision information. To solve these issues, we propose a controllable smoke image generation network (SGNet) based on smoke imaging principle. Specifically, to enhance training sample diversity, a component separation module, a smoke component fine-tuning module (SFM) and an image synthesis module are designed to integrate smoke imaging principle into deep models to generate controllable and realistic smoke images. The smoke component latent codes in SFM control the smoke component in generated smoke images. Diverse smoke images can be generated by fine-tuning smoke component in smoke images, changing smoke component to diverse assigned smoke component, adding smoke component to background images. Furthermore, to increase supervision information, a three-stage interactive training method is designed to train SGNet using synthetic dataset as well as real dataset for generating smoke images as well as corresponding smoke component and background component images. Extensive experiments show that our SGNet performs better than existing image generation methods. In addition, using the generated samples of SGNet, a new smoke recognition method with sufficient supervision information is designed and achieves the best results with 0.9876 detection rate and 0.0526 false alarm rate in smoke recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Berthelot D, Schumm T, Metz L (2017) BEGAN: boundary equilibrium generative adversarial networks[J]. arXiv preprint, arXiv:1703.10717

  2. Bounliphone W, Belilovsky E, Blaschko MB, Antonoglou I, Gretton A (2015) A test of relative similarity for model selection in generative models[J]. arXiv preprint, arXiv:1511.04581

  3. Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2017) InfoGAN: interpretable representation learning by information maximizing generative adversarial Nets. In Proceedings of the annual conference on neural information processing systems (NIPS), 2180–2188

  4. Chen X, Luo X, Weng J, Luo W, Li H, Tian Q (2021) Multi-view gait image generation for cross-view gait recognition. IEEE Trans Image Process 30:3041–3055

    Article  Google Scholar 

  5. Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In proc. IEEE CVPR, 8789–8797

  6. Fan GF, Yu M, Dong SQ, Yeh YH, Hong WC (2021) Forecasting short-term electricity load using hybrid support vector regression with grey catastrophe and random forest modeling[J]. Util Policy 73:101294

    Article  Google Scholar 

  7. Filonenko A, Hernández DC, Jo KH (2018) Fast smoke detection for video surveillance using CUDA[J]. IEEE Trans Ind Inf 14(2):725–733

    Article  Google Scholar 

  8. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In proceedings of the international conference on neural information processing systems (NIPS), 2:2672–2680

  9. Gu K, Xia Z, Qiao J, Lin W (2019) Deep dual-channel neural network for image-based smoke detection. IEEE Trans Multimed 22(2):311–323

  10. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of Wasserstein GANs. In proceedings of the annual conference on neural information processing systems (NIPS), 5768–5778

  11. Higgins I, Matthey L, Pal A, Burgess C, Glorot X, Botvinick M, Mohamed S, Lerchner A (2017) β-VAE: learning basic visual concepts with a constrained variational framework. In proc. of the 5th international conference on learning representations (ICLR), 1–10

  12. Huang H, Li Z, He R, Sun Z, Tan T (2018) IntroVAE: introspective variational autoencoders for photographic image synthesis. In Proc of the conference on neural information processing systems (NIPS), 31–43

  13. Khan S, Muhammad K, Mumtaz S, Baik SW, de Albuquerque VHC (2019) Energy-efficient deep CNN for smoke detection in foggy IoT environment. IEEE Internet Things J 6(6):9237–9245

    Article  Google Scholar 

  14. Kingma DP, Welling M (2014) Auto-encoding variational bayes. arXiv preprint, arXiv:1312.6114

  15. LopezPaz D, Oquab M (2017) Revisiting classifier two-sample tests. arXiv preprint, arXiv:1610.06545

  16. Nguyen MD, Kim D, Ro S (2018) A video smoke detection algorithm based on cascade classification and deep learning. KSII Trans Int Inf Syst 12(12):6018–6033

  17. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint, arXiv:1511.06434

  18. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In proc. of the conference on medical image computing and computer-assisted intervention (MICCAI), 9351:234–241

  19. Shaham TR, Dekel T, Michaeli T (2019) SinGAN: learning a generative model from a single natural image[C]. In proceedings of ICCV, 4569-4579

  20. Sheng C, Hu B, Meng F, Yin D (2021) Lightweight dual-branch network for vehicle exhausts segmentation. Multimed Tools Appl 80(12):17785–17806

    Article  Google Scholar 

  21. Tao H, Lu X (2020) Smoke vehicle detection based on spatiotemporal bag-of-features and professional convolutional neural network[J]. IEEE Trans Circuits Syst Vid Technol 30(10):3301–3316

    Article  Google Scholar 

  22. Tao H, Lu M, Hu Z, Xin Z, Wang J (2022) Attention-aggregated attribute-aware network with redundancy reduction convolution for video-based industrial smoke emission recognition. IEEE Trans Industrial Inf 18(11):7653–7664

  23. Tao H, Xie C, Wang J, Xin Z. (2022) CENet: a channel-enhanced spatiotemporal network with sufficient supervision information for recognizing industrial smoke emissions[J]. IEEE internet of things journal. 1-10

  24. Tero K, Samuli L, Timo A (2019) A style-based generator architecture for generative adversarial networks. In proceedings of CVPR, 4396–4405

  25. Wen J, Ma H, Luo X (2020) Deep generative smoke simulator: connecting simulated and real data[J]. Vis Comput 36:1385–1399

    Article  Google Scholar 

  26. Xie C, Tao H (2020) Generating realistic smoke images with controllable smoke components[J]. IEEE Access 8:201418–201427

    Article  Google Scholar 

  27. Yuan F, Zhang L, Xia X, et al. (2019) A wave-shaped deep neural network for smoke density estimation[J]. IEEE transactions on image processing, 2301-2313

  28. Yuan F, Zhang L, Wan B, Xia X, Shi J (2019) Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition. Mach Vis Appl 30:345–358

    Article  Google Scholar 

  29. Zhang QX, Lin GH, Zhang YM, Xu G, Wang JJ (2018) Wildland forest fire smoke detection based on faster R-CNN using synthetic smoke images. Proc Conf Fire Sci Fire Protect Eng 211:441–446

    Google Scholar 

  30. Zhu JY, Park T, Isola P, et al. (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. In proceedings of ICCV, 2242-2251

Download references

Acknowledgements

This work was partly supported by the National Natural Science Foundation of China (No. 62102320), and the Fundamental Research Funds for the Central Universities (No. D5000210737.

Author information

Authors and Affiliations

  1. School of Computer Science, Northwestern Polytechnical University, Xi’an, 710129, People’s Republic of China

    Huanjie Tao, Jing Wang & Zhouxin Xin

  2. Engineering Research Center of Embedded System Integration, Ministry of Education, Xi’an, 710129, People’s Republic of China

    Huanjie Tao

  3. National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, Xi’an, 710129, People’s Republic of China

    Huanjie Tao

Authors
  1. Huanjie Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhouxin Xin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huanjie Tao.

Ethics declarations

Conflict of interest

Huanjie Tao declares that he has no conflict of interest. Jing Wang declares that she has no conflict of interest. Zhouxin Xin declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tao, H., Wang, J. & Xin, Z. Controllable smoke image generation network based on smoke imaging principle. Multimed Tools Appl 82, 16057–16079 (2023). https://doi.org/10.1007/s11042-022-14040-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14040-2

Keywords

Search

Navigation