Skip to main content

Robotic Fire Suppression Through Autonomous Feedback Control

Abstract

A computer vision-based autonomous fire suppression system with real-time feedback of fire size and spray direction is presented in this paper. The system has been developed for use in a firefighting robot for close-range, localized fire suppression tasks in enclosed environments. A probabilistic water classification method was developed for segmenting water spray in a pair of IR cameras. Stereo processing was performed to localize points along the spray path for use in yaw and pitch angle estimation. A Golden Section Search with linear least squares optimization was used to determine the optimal pitch angle of the spray position at each sampling time. Kalman filtering was used to remove noise from the angle measurements and obtain a better estimate of the current nozzle orientation. A decision tree was used to determine the correct nozzle positioning mode using image feedback to suppress the fire and accounts for errors in direction, fire size during suppression, and when to adjust the nozzle based on IR feedback. Through implementation of a PI controller, the system is able to correct for unknown disturbances causing erroneous targeting of a localized fire. Experiments are presented with the initial nozzle angled correctly and with forced offsets in the system to set the initial spray position incorrectly in order for the system to correct. Suppression times ranged from 7.2 s to 16.3 s with a standard deviation of 3.9 s and average time of 11.2 s. A total of 12 tests demonstrated performance of the system given a forced offset to the initial nozzle orientation resulting in an error between the spray location and the fire target. Suppression times ranged from 8.1 s to 27.9 s with a mean of 16.9 s and standard deviation of 6.2 s. The proposed system can be implemented on a robotic firefighting platform to autonomously detect a fire, choose a proper manipulation goal and suppress full scale fires given disturbances causing erroneous targeting.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15

References

  1. 1.

    Miyazawa K (2002) Fire robots developed by the Tokyo Fire Department. Adv Robot 16(6):553–556

    Article  Google Scholar 

  2. 2.

    Kim YD, Kim YG, Lee SH, Kang JH, An J (2009) Portable fire evacuation guide robot system. In: IEEE/RSJ international conference on intelligent robots and systems, 2009. IROS 2009. IEEE, pp. 2789–2794

  3. 3.

    Penders J, Alboul L, Witkowski U, Naghsh A, Saez-Pons J, Herbrechtsmeier S, El-Habbal M (2011) A robot swarm assisting a human fire-fighter. Adv Robot 25(1–2):93–117

    Article  Google Scholar 

  4. 4.

    Liljeback P, Stavdahl O, Beitnes A (2006) SnakeFighter-development of a water hydraulic fire fighting snake robot. In: 9th international conference on control, automation, robotics and vision, 2006. ICARCV’06. IEEE, pp 1–6

  5. 5.

    Pack DJ, Avanzato R, Ahlgren DJ, Verner IM (2004) Fire-fighting mobile robotics and interdisciplinary design-comparative perspectives. IEEE Trans Educ 47(3):369–376.

    Article  Google Scholar 

  6. 6.

    Dearie S, Fisher K, Rajala B, Wasson S (2001) Design and construction of a fully autonomous fire fighting robot. In: Proceedings: Electrical insulation conference and electrical manufacturing & coil winding conference, 2001. IEEE, pp 303–310

  7. 7.

    Rehman A, Masood N, Arif S, Shahbaz U, Sarwar F, Maqsood K, Imran M, Pasha MA (2012, October) Autonomous fire extinguishing system. In: 2012 international conference on robotics and artificial intelligence (ICRAI). IEEE, pp. 218–222

  8. 8.

    Khan MJ, Imam MR, Uddin J, Sarkar MR (2012) Automated fire fighting system with smoke and temperature detection. In: 2012 7th international conference on electrical & computer engineering (ICECE). IEEE, pp. 232–235

  9. 9.

    Manjunatha KC, Mohana HS, Vijaya PA (2015) Implementation of computer vision based industrial fire safety automation by using neuro-fuzzy algorithms. Int J Inf Technol Comput Sci (IJITCS) 7(4):14

    Google Scholar 

  10. 10.

    Liu XX, Huang XL, Cao L (2012) The design of intellingent fire monitor system based-on digital image. In: Advanced materials research, vol. 433. Trans Tech Publications, pp. 4178–4183

  11. 11.

    Liu J, Che X (2013) Design of fire targeting device for large space auto tracking jet suppression system. Sci Technol Innov Her 11:243–246.

    Google Scholar 

  12. 12.

    Chen T, Yuan H, Su G, Fan W (2004) An automatic fire searching and suppression system for large spaces. Fire Saf J 39(4):297–307.

    Article  Google Scholar 

  13. 13.

    Xin Y, Thumuluru S, Jiang F, Yin R, Yao B, Zhang K, Liu B (2014) An experimental study of automatic water cannon systems for fire protection of large open spaces. Fire Technol 50(2):233–248.

    Article  Google Scholar 

  14. 14.

    Yuan F (2010) An integrated fire detection and suppression system based on widely available video surveillance. Mach Vis Appl 21(6):941–948.

    Article  Google Scholar 

  15. 15.

    Chen X, Li X (2015) An automatic jet fire extinguishing device based on video. In: 2015 international conference on mechatronics, electronic, industrial and control engineering (MEIC-15). Atlantis Press.

  16. 16.

    Kim JH, Starr JW, Lattimer BY (2015) Firefighting robot stereo infrared vision and radar sensor fusion for imaging through smoke. Fire Technol 51(4):823–845

    Article  Google Scholar 

  17. 17.

    Kim JH, Lattimer BY (2015) Real-time probabilistic classification of fire and smoke using thermal imagery for intelligent firefighting robot. Fire Saf J 72:40–49

    Article  Google Scholar 

  18. 18.

    Töreyin BU, Dedeoğlu Y, Güdükbay,U, Cetin AE (2006) Computer vision based method for real-time fire and flame detection. Pattern Recogn Lett 27(1):49–58

    Article  Google Scholar 

  19. 19.

    Ko B, Cheong KH, Nam JY (2010) Early fire detection algorithm based on irregular patterns of flames and hierarchical Bayesian Networks. Fire Saf J 45(4):262–270

    Article  Google Scholar 

  20. 20.

    Ko BC, Cheong KH, Nam JY (2009) Fire detection based on vision sensor and support vector machines. Fire Saf J 44(3):322–329

    Article  Google Scholar 

  21. 21.

    Celik T, Demirel H, Ozkaramanli H, Uyguroglu M (2007) Fire detection using statistical color model in video sequences. J Vis Commun Image Represent 18(2):176–185

    Article  Google Scholar 

  22. 22.

    Chen TH, Yin YH, Huang SF, Ye YT (2006) The smoke detection for early fire-alarming system base on video processing. In: International conference on intelligent information hiding and multimedia signal processing, 2006. IIH-MSP’06. IEEE

  23. 23.

    Verstockt S, Van Hoecke S, Beji T, Merci B, Gouverneur B, Cetin AE, De Potter P, Van de Walle R (2013) A multi-modal video analysis approach for car park fire detection. Fire Saf J 57:44–57

    Article  Google Scholar 

  24. 24.

    Amon F, Benetis V, Kim J, Hamins A (2004) Development of a performance evaluation facility for fire fighting thermal imagers. In: Defense and security. International Society for Optics and Photonics, pp. 244–252

  25. 25.

    Amon F, Bryner N, Hamins A (2004) Evaluation of thermal imaging cameras used in fire fighting applications. In: Defense and security. International Society for Optics and Photonics, pp 44–53

  26. 26.

    Amon F, Ducharme A (2009) Image frequency analysis for testing of fire service thermal imaging cameras. Fire Technol 45(3):313–322

    Article  Google Scholar 

  27. 27.

    Maxwell FD (1971) A portable IR system for observing fire thru smoke. Fire Technol 7(4):321–331

    Article  Google Scholar 

  28. 28.

    McNeil JG, Lattimer BY (2015) Real-time classification of water spray and leaks for robotic firefighting. Int J Comput Vis Image Process (IJCVIP) 5(1):1–26.

    Article  Google Scholar 

  29. 29.

    Miyashita T, Sugawa O, Imamura T, Kamiya K, Kawaguchi Y (2014) Modeling and analysis of water discharge trajectory with large capacity monitor. Fire Saf J 63:1–8

    Article  Google Scholar 

  30. 30.

    Li YF, Wikander J (2002) Discrete-time sliding mode control of a dc motor and ball-screw driven positioning table. In: Proceedings of the IFAC 15th triennial world congress. Barcelona, pp 2436–2441

  31. 31.

    Wang J, Van Brussel H, Swevers J (2003) Robust perfect tracking control with discrete sliding mode controller. J Dyn Syst Meas Contr 125(1):27–32

    Article  Google Scholar 

  32. 32.

    Drakunov S, Hanchin GD, Su WC, Özgüner Ü (1997) Nonlinear control of a rodless pneumatic servoactuator, or sliding modes versus Coulomb friction. Automatica 33(7):1401–1408

    MathSciNet  Article  MATH  Google Scholar 

  33. 33.

    Southward SC, Radcliffe CJ, MacCluer CR (1991) Robust nonlinear stick-slip friction compensation. J Dyn Syst Meas Contr 113(4):639–645

    Article  MATH  Google Scholar 

  34. 34.

    Patron RSF, Botez RM, Labour D (2012) Vertical profile optimization for the Flight Management System CMA-9000 using the golden section search method. In: IECON 2012-38th annual conference on ieee industrial electronics society. IEEE, pp 5482–5488

  35. 35.

    McNeil JG, Lattimer BY (2016) Autonomous fire suppression system for use in high and low visibility environments by visual servoing. Fire Technol 52:1343

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the Office of Naval Research for funding this research through Grant No. N00014-11-1-0074, scientific monitor Dr. Thomas McKenna. In addition, the authors appreciate the contributions of Dr. Joseph Starr in assisting with stereoscopic IR vision system.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Joshua G. McNeil.

Appendix

Appendix

See Figs. 16, 17, 18, 19, 20, 21, and 22.

Figure 16
figure16

Fire size versus time for Test 4

Figure 17
figure17

IR and visual images from Test 6 a fully developed b fire knocked down c end of test with smoke buildup

Figure 18
figure18

Fire size versus time for Test 6

Figure 19
figure19

Control response for Test 6

Figure 20
figure20

IR and visual images from Test 7: a fully developed, b bottom portion of fire suppressed, c middle portion of fire suppressed, d end of test

Figure 21
figure21

Fire size and target row versus time for Test 7

Figure 22
figure22

Test 16 Visual and IR images at a fully developed fire, b initial off-target spray over and right of fire, c correction of nozzle direction, d suppressed fire

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

McNeil, J.G., Lattimer, B.Y. Robotic Fire Suppression Through Autonomous Feedback Control. Fire Technol 53, 1171–1199 (2017). https://doi.org/10.1007/s10694-016-0623-1

Download citation

Keywords

  • Autonomous fire suppression
  • Fire detection
  • Suppression systems
  • IR stereovision
  • Robotics