Skip to main content
SpringerLink
Log in

An automatic fire detection system based on deep convolutional neural networks for low-power, resource-constrained devices

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Large-scale fires have been increasingly reported in the news media. These events can cause a variety of irreversible damage, what encourages the search for effective solutions to prevent and fight fires. A promising solution is an automatic system based on computer vision capable of detecting fire in early stages, enabling rapid suppression to mitigate damage, minimizing combat and restoration costs. Currently, the most effective systems are typically based on convolutional neural networks (CNNs). However, these networks are computationally expensive and consume a large amount of memory, usually requiring graphics processing units to operate properly in emergency situations. Thus, we propose a CNN-based fire detector system suitable for low-power, resource-constrained devices. Our approach consists of training a deep detection network and then removing its less important convolutional filters in order to reduce its computational cost while trying to preserve its original performance. Through an investigation of different pruning techniques, our results show that we can reduce the computational cost by up to 83.60% and the memory consumption by up to 83.86% without degrading the system’s performance. A case study was performed on a Raspberry Pi 4 where the results demonstrate the viability of implementing our proposed system on a low-end device.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. All source code is available at [62].

  2. All discussions about computational cost in this work are based on YOLOv4 and Tiny YOLOv4 networks designed for fire and smoke detection (\(C=2\) classes). However, we note that the computational cost increases as C increases.

  3. Obviously, clusters with a single filter do not need to undergo this process.

References

  1. Finney MA (2021) The wildland fire system and challenges for engineering. Fire Safety J 120: 103085 (Fire Safety Science: Proceedings of the 13th International Symposium)

  2. Joglar F, Mowrer F, Modarres M (2005) A probabilistic model for fire detection with applications. Fire Technol 41(3):151–172

  3. Töreyin BU (2018) Smoke detection in compressed video. In: Applications of digital image processing XLI, vol. 10752. International Society for Optics and Photonics, p 1075232

  4. Singh A, Singh H (2012) Forest fire detection through wireless sensor network using type-2 fuzzy system. Int J Comput Appl 52(9)

  5. Remagnino P, Jones GA, Paragios N, Regazzoni CS (2002) Video-based surveillance systems: computer vision and distributed processing

  6. Chen T-H, Wu P-H, Chiou Y-C (2004) An early fire-detection method based on image processing. In: 2004 International conference on image processing, 2004, vol 3. ICIP’04. IEEE, pp 1707–1710

  7. Chen T-H, Yin Y-H, Huang S-F, Ye Y-T (2006) The smoke detection for early fire-alarming system base on video processing. In: 2006 international conference on intelligent information hiding and multimedia. IEEE, pp 427–430

  8. Yu C, Mei Z, Zhang X (2013) A real-time video fire flame and smoke detection algorithm. Proc Eng 62:891–898

  9. Töreyin BU, Dedeoğlu Y, Cetin AE (2005) Wavelet based real-time smoke detection in video. In: 2005 13th European signal processing conference. IEEE, pp 1–4

  10. Marbach G, Loepfe M, Brupbacher T (2006) An image processing technique for fire detection in video images. Fire Saf J 41(4):285–289

    Article  Google Scholar 

  11. Lascio R.D, Greco A, Saggese A, Vento M (2014) Improving fire detection reliability by a combination of videoanalytics. In: International conference image analysis and recognition. Springer, pp 477–484

  12. Celik T, Demirel H, Ozkaramanli H, Uyguroglu M (2007) Fire detection using statistical color model in video sequences. J Vis Commun Image Represent 18(2):176–185

    Article  Google Scholar 

  13. Chunyu Y, Jun F, Jinjun W, Yongming Z (2010) Video fire smoke detection using motion and color features. Fire Technol 46(3):651–663

    Article  Google Scholar 

  14. Cheng X, Wu J, Yuan X, Zhou H (1999) Principles for a video fire detection system. Fire Saf J 33(1):57–69

    Article  Google Scholar 

  15. Yuan F, Fang Z, Wu S, Yang Y, Fang Y (2015) Real-time image smoke detection using staircase searching-based dual threshold adaboost and dynamic analysis. IET Image Proc 9(10):849–856

    Article  Google Scholar 

  16. Schultze T, Kempka T, Willms I (2006) Audio-video fire-detection of open fires. Fire Saf J 41(4):311–314

    Article  Google Scholar 

  17. Morerio P, Marcenaro L, Regazzoni CS, Gera G (2012) Early fire and smoke detection based on colour features and motion analysis. In: 19th International conference on image processing. IEEE, pp 1041–1044

  18. Cheng C, Sun F, Zhou X (2011) One fire detection method using neural networks. Tsinghua Sci Technol 16(1):31–35

    Article  Google Scholar 

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  20. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

  21. Tao C, Zhang J, Wang P (2016) Smoke detection based on deep convolutional neural networks. In: 2016 International conference on industrial informatics-computing technology, intelligent technology, industrial information integration (ICIICII). IEEE, pp 150–153

  22. Yin Z, Wan B, Yuan F, Xia X, Shi J (2017) A deep normalization and convolutional neural network for image smoke detection. IEEE Access 5:18429–18438

    Article  Google Scholar 

  23. Muhammad K, Ahmad J, Baik SW (2018) Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 288:30–42

  24. Govil K, Welch ML, Ball JT, Pennypacker CR (2020) Preliminary results from a wildfire detection system using deep learning on remote camera images. Remote Sensing 12(1):166

  25. Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S et al (2017) Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7310–7311

  26. Malach E, Shalev-Shwartz S (2019) Is deeper better only when shallow is good? Adv Neural Inf Process Syst 32:6429–6438

    Google Scholar 

  27. Muhammad K, Ahmad J, Lv Z, Bellavista P, Yang P, Baik SW (2018) Efficient deep cnn-based fire detection and localization in video surveillance applications. IEEE Trans Syst Man Cybern Syst 49(7):1419–1434

  28. Muhammad K, Ahmad J, Mehmood I, Rho S, Baik SW (2018) Convolutional neural networks based fire detection in surveillance videos. IEEE Access 6:18174–18183

  29. Jadon A, Omama M, Varshney A, Ansari MS, Sharma R (2019) Firenet: a specialized lightweight fire & smoke detection model for real-time iot applications. arXiv preprint arXiv:1905.11922

  30. Muhammad K, Khan S, Elhoseny M, Ahmed SH, Baik SW (2019) Efficient fire detection for uncertain surveillance environment. IEEE Trans Industr Inf 15(5):3113–3122

    Article  Google Scholar 

  31. Jordao A, Akio F, Lie M, Schwartz WR (2021) Stage-wise neural architecture search. In: 2020 25th international conference on pattern recognition (ICPR). IEEE, pp 1985–1992

  32. Han S, Pool J, Tran J, Dally W (2015) Learning both weights and connections for efficient neural network. Adv Neural Inf Process Syst

  33. Rastegari M, Ordonez V, Redmon J, Farhadi A (2016) Xnor-net: Imagenet classification using binary convolutional neural networks. In: European conference on computer vision. Springer, pp 525–542

  34. Zhu S, Duong L.H, Liu W (2020) Xor-net: an efficient computation pipeline for binary neural network inference on edge devices. In: 2020 IEEE 26th international conference on parallel and distributed systems (ICPADS). IEEE, pp 124–131

  35. Howard A.G, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861

  36. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520

  37. Buciluǎ C, Caruana R, Niculescu-Mizil A (2006) Model compression. In: Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining, pp 535–541

  38. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv:1503.02531

  39. Li H, Kadav A, Durdanovic I, Samet H, Graf H.P (2017) Pruning filters for efficient convnets. In: Proceedings of the international conference for learning representations, pp 1–5

  40. He Y, Kang G, Dong X, Fu Y, Yang Y (2018) Soft filter pruning for accelerating deep convolutional neural networks. arXiv:1808.06866

  41. Jordao A, Yamada F, Schwartz WR (2020) Deep network compression based on partial least squares. Neurocomputing 406:234–243

  42. Jordao A, Lie M, Schwartz WR (2020) Discriminative layer pruning for convolutional neural networks. IEEE J Selected Topics Sig Process 14(4):828–837

  43. Mittal D, Bhardwaj S, Khapra MM, Ravindran B (2018) Recovering from random pruning: on the plasticity of deep convolutional neural networks. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 848–857

  44. Ghosh S, Srinivasa S.K, Amon P, Hutter A, Kaup A (2019) Deep network pruning for object detection. In: 2019 IEEE international conference on image processing (ICIP). IEEE, pp 3915–3919

  45. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol. 28, pp 91–99

  46. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You Only Look Once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  47. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271

  48. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. arXiv:1804.02767

  49. Bochkovskiy A, Wang C.-Y, Liao H-YM (2020) YOLOv4: optimal speed and accuracy of object detection

  50. Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2009) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  51. Wang C-Y, Liao H-YM, Wu Y-H, Chen P-Y, Hsieh J-W, Yeh I-H (2020) CSPNet: a new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 390–391

  52. Deng J, Dong W, Socher R, Li L.-J, Li K, Fei-Fei L (2009) ImageNet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 248–255

  53. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  54. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8759–8768

  55. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The PASCAL visual object classes (VOC) challenge. Int J Comput Vision 88(2):303–338

    Article  Google Scholar 

  56. Redmon J, Bochkovskiy A (2013) Darknet: open source neural networks in C. https://git.io/JTICL (Downloaded in January, 2021)

  57. Venâncio PVAB, Rezende TM, Lisboa AC, Barbosa AV (2021) Fire detection based on two-dimensional convolutional neural network and temporal analysis. In: 2021 IEEE Latin American conference on computational intelligence (LA-CCI). IEEE, pp 1–6

  58. Zhang Q-X, Lin G-H, Zhang Y-M, Xu G, Wang J-J (2018) Wildland forest fire smoke detection based on Faster R-CNN using synthetic smoke images. Procedia Eng 211:441–446

  59. Gaia (2018) solutions on demand: D-Fire: an image data set for fire detection. https://git.io/JONna (Downloaded in February 2021)

  60. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) PyTorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037

    Google Scholar 

  61. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  62. de Venâncio PVAB (2021) Pruning techniques of convolutional neural networks implemented in the Darknet framework. https://git.io/JmjNB

  63. Bottou L (1998) Online learning and stochastic approximations. Online Learn Neural Netw 17(9):142

  64. Polyak BT (1964) Some methods of speeding up the convergence of iteration methods. Ussr Comput Math Math Phys 4(5):1–17

  65. Tikhonov A (1963) On solving ill-posed problem and method of regularization. In: Doklady Akademii Nauk USSR, vol 153, pp 501–504

  66. Sadeghi MA, Forsyth D (2014) 30hz object detection with dpm v5. In: European conference on computer vision. Springer, pp 65–79

  67. Wold H (1985) Partial least squares. Encyclopedia of Statistical Sciences, Wiley, pp 581–591

  68. Eckart C (1936) Young G The approximation of one matrix by another of lower rank. Psychometrika 1(3):211–218

    Article  Google Scholar 

  69. Abdi H (2010) Partial least squares regression and projection on latent structure regression (pls regression). Wiley Interdisc Rev Comput Stat 2(1):97–106

    Article  Google Scholar 

  70. Mehmood T, Liland KH, Snipen L, Sæbø S (2012) A review of variable selection methods in partial least squares regression. Chemom Intell Lab Syst 118:62–69

  71. Raspberry Pi (2022) Foundation: Raspberry Pi 4 Model B. https://www.raspberrypi.com/

  72. Chino DY, Avalhais LP, Rodrigues JF, Traina AJ (2015) BoWFire: detection of fire in still images by integrating pixel color and texture analysis. In: 2015 28th SIBGRAPI conference on graphics, patterns and images. IEEE, pp 95–102

  73. Saponara S, Elhanashi A, Gagliardi A (2021) Real-time video fire/smoke detection based on cnn in antifire surveillance systems. J Real-Time Image Proc 18(3):889–900

    Article  Google Scholar 

  74. NVIDIA Developer (2022) NVIDIA Jetson Nano. https://developer.nvidia.com/embedded/jetson-nano-developer-kit/

  75. Veit A, Belongie S (2018) Convolutional networks with adaptive inference graphs. In: Proceedings of the European conference on computer vision (ECCV), pp 3–18

  76. Fan A, Grave E, Joulin A (2020) Reducing transformer depth on demand with structured dropout. In: International conference on learning representations, pp 1–16

Download references

Acknowledgements

Financial support for this work was provided by CEMIG-ANEEL (R&D project D0619), by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) to Adriano Chaves Lisboa (Grant 304506/2020-6), by Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG) to Adriano Vilela Barbosa (Grant APQ-03701-16), and by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).

Author information

Author notes

  1. Adriano C. Lisboa and Adriano V. Barbosa contributed equally to this work.

Authors and Affiliations

  1. Gaia Solutions on Demand, Technological Park, Belo Horizonte, Brazil

    Pedro Vinícius A. B. de Venâncio & Adriano C. Lisboa

  2. Graduate Program in Electrical Engineering, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil

    Pedro Vinícius A. B. de Venâncio & Adriano V. Barbosa

  3. Department of Computer Engineering, Federal Center for Technological Education of Minas Gerais, Belo Horizonte, Brazil

    Adriano C. Lisboa

  4. Department of Electronics Engineering, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil

    Adriano V. Barbosa

Authors
  1. Pedro Vinícius A. B. de Venâncio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adriano C. Lisboa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adriano V. Barbosa
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PVABV: Methodology; Formal analysis; Software; Writing—Original Draft, Visualization. ACL: Formal analysis; Writing—Review and Editing; Validation. AVB: Formal analysis; Supervision; Writing—Review and Editing.

Corresponding author

Correspondence to Pedro Vinícius A. B. de Venâncio.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Venâncio, P.V.A.B., Lisboa, A.C. & Barbosa, A.V. An automatic fire detection system based on deep convolutional neural networks for low-power, resource-constrained devices. Neural Comput & Applic 34, 15349–15368 (2022). https://doi.org/10.1007/s00521-022-07467-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07467-z

Keywords

Search

Navigation