A computer vision-based autonomous fire suppression system with real-time feedback of fire size and spray direction is presented in this paper. The system has been developed for use in a firefighting robot for close-range, localized fire suppression tasks in enclosed environments. A probabilistic water classification method was developed for segmenting water spray in a pair of IR cameras. Stereo processing was performed to localize points along the spray path for use in yaw and pitch angle estimation. A Golden Section Search with linear least squares optimization was used to determine the optimal pitch angle of the spray position at each sampling time. Kalman filtering was used to remove noise from the angle measurements and obtain a better estimate of the current nozzle orientation. A decision tree was used to determine the correct nozzle positioning mode using image feedback to suppress the fire and accounts for errors in direction, fire size during suppression, and when to adjust the nozzle based on IR feedback. Through implementation of a PI controller, the system is able to correct for unknown disturbances causing erroneous targeting of a localized fire. Experiments are presented with the initial nozzle angled correctly and with forced offsets in the system to set the initial spray position incorrectly in order for the system to correct. Suppression times ranged from 7.2 s to 16.3 s with a standard deviation of 3.9 s and average time of 11.2 s. A total of 12 tests demonstrated performance of the system given a forced offset to the initial nozzle orientation resulting in an error between the spray location and the fire target. Suppression times ranged from 8.1 s to 27.9 s with a mean of 16.9 s and standard deviation of 6.2 s. The proposed system can be implemented on a robotic firefighting platform to autonomously detect a fire, choose a proper manipulation goal and suppress full scale fires given disturbances causing erroneous targeting.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Miyazawa K (2002) Fire robots developed by the Tokyo Fire Department. Adv Robot 16(6):553–556
Kim YD, Kim YG, Lee SH, Kang JH, An J (2009) Portable fire evacuation guide robot system. In: IEEE/RSJ international conference on intelligent robots and systems, 2009. IROS 2009. IEEE, pp. 2789–2794
Penders J, Alboul L, Witkowski U, Naghsh A, Saez-Pons J, Herbrechtsmeier S, El-Habbal M (2011) A robot swarm assisting a human fire-fighter. Adv Robot 25(1–2):93–117
Liljeback P, Stavdahl O, Beitnes A (2006) SnakeFighter-development of a water hydraulic fire fighting snake robot. In: 9th international conference on control, automation, robotics and vision, 2006. ICARCV’06. IEEE, pp 1–6
Pack DJ, Avanzato R, Ahlgren DJ, Verner IM (2004) Fire-fighting mobile robotics and interdisciplinary design-comparative perspectives. IEEE Trans Educ 47(3):369–376.
Dearie S, Fisher K, Rajala B, Wasson S (2001) Design and construction of a fully autonomous fire fighting robot. In: Proceedings: Electrical insulation conference and electrical manufacturing & coil winding conference, 2001. IEEE, pp 303–310
Rehman A, Masood N, Arif S, Shahbaz U, Sarwar F, Maqsood K, Imran M, Pasha MA (2012, October) Autonomous fire extinguishing system. In: 2012 international conference on robotics and artificial intelligence (ICRAI). IEEE, pp. 218–222
Khan MJ, Imam MR, Uddin J, Sarkar MR (2012) Automated fire fighting system with smoke and temperature detection. In: 2012 7th international conference on electrical & computer engineering (ICECE). IEEE, pp. 232–235
Manjunatha KC, Mohana HS, Vijaya PA (2015) Implementation of computer vision based industrial fire safety automation by using neuro-fuzzy algorithms. Int J Inf Technol Comput Sci (IJITCS) 7(4):14
Liu XX, Huang XL, Cao L (2012) The design of intellingent fire monitor system based-on digital image. In: Advanced materials research, vol. 433. Trans Tech Publications, pp. 4178–4183
Liu J, Che X (2013) Design of fire targeting device for large space auto tracking jet suppression system. Sci Technol Innov Her 11:243–246.
Chen T, Yuan H, Su G, Fan W (2004) An automatic fire searching and suppression system for large spaces. Fire Saf J 39(4):297–307.
Xin Y, Thumuluru S, Jiang F, Yin R, Yao B, Zhang K, Liu B (2014) An experimental study of automatic water cannon systems for fire protection of large open spaces. Fire Technol 50(2):233–248.
Yuan F (2010) An integrated fire detection and suppression system based on widely available video surveillance. Mach Vis Appl 21(6):941–948.
Chen X, Li X (2015) An automatic jet fire extinguishing device based on video. In: 2015 international conference on mechatronics, electronic, industrial and control engineering (MEIC-15). Atlantis Press.
Kim JH, Starr JW, Lattimer BY (2015) Firefighting robot stereo infrared vision and radar sensor fusion for imaging through smoke. Fire Technol 51(4):823–845
Kim JH, Lattimer BY (2015) Real-time probabilistic classification of fire and smoke using thermal imagery for intelligent firefighting robot. Fire Saf J 72:40–49
Töreyin BU, Dedeoğlu Y, Güdükbay,U, Cetin AE (2006) Computer vision based method for real-time fire and flame detection. Pattern Recogn Lett 27(1):49–58
Ko B, Cheong KH, Nam JY (2010) Early fire detection algorithm based on irregular patterns of flames and hierarchical Bayesian Networks. Fire Saf J 45(4):262–270
Ko BC, Cheong KH, Nam JY (2009) Fire detection based on vision sensor and support vector machines. Fire Saf J 44(3):322–329
Celik T, Demirel H, Ozkaramanli H, Uyguroglu M (2007) Fire detection using statistical color model in video sequences. J Vis Commun Image Represent 18(2):176–185
Chen TH, Yin YH, Huang SF, Ye YT (2006) The smoke detection for early fire-alarming system base on video processing. In: International conference on intelligent information hiding and multimedia signal processing, 2006. IIH-MSP’06. IEEE
Verstockt S, Van Hoecke S, Beji T, Merci B, Gouverneur B, Cetin AE, De Potter P, Van de Walle R (2013) A multi-modal video analysis approach for car park fire detection. Fire Saf J 57:44–57
Amon F, Benetis V, Kim J, Hamins A (2004) Development of a performance evaluation facility for fire fighting thermal imagers. In: Defense and security. International Society for Optics and Photonics, pp. 244–252
Amon F, Bryner N, Hamins A (2004) Evaluation of thermal imaging cameras used in fire fighting applications. In: Defense and security. International Society for Optics and Photonics, pp 44–53
Amon F, Ducharme A (2009) Image frequency analysis for testing of fire service thermal imaging cameras. Fire Technol 45(3):313–322
Maxwell FD (1971) A portable IR system for observing fire thru smoke. Fire Technol 7(4):321–331
McNeil JG, Lattimer BY (2015) Real-time classification of water spray and leaks for robotic firefighting. Int J Comput Vis Image Process (IJCVIP) 5(1):1–26.
Miyashita T, Sugawa O, Imamura T, Kamiya K, Kawaguchi Y (2014) Modeling and analysis of water discharge trajectory with large capacity monitor. Fire Saf J 63:1–8
Li YF, Wikander J (2002) Discrete-time sliding mode control of a dc motor and ball-screw driven positioning table. In: Proceedings of the IFAC 15th triennial world congress. Barcelona, pp 2436–2441
Wang J, Van Brussel H, Swevers J (2003) Robust perfect tracking control with discrete sliding mode controller. J Dyn Syst Meas Contr 125(1):27–32
Drakunov S, Hanchin GD, Su WC, Özgüner Ü (1997) Nonlinear control of a rodless pneumatic servoactuator, or sliding modes versus Coulomb friction. Automatica 33(7):1401–1408
Southward SC, Radcliffe CJ, MacCluer CR (1991) Robust nonlinear stick-slip friction compensation. J Dyn Syst Meas Contr 113(4):639–645
Patron RSF, Botez RM, Labour D (2012) Vertical profile optimization for the Flight Management System CMA-9000 using the golden section search method. In: IECON 2012-38th annual conference on ieee industrial electronics society. IEEE, pp 5482–5488
McNeil JG, Lattimer BY (2016) Autonomous fire suppression system for use in high and low visibility environments by visual servoing. Fire Technol 52:1343
The authors would like to thank the Office of Naval Research for funding this research through Grant No. N00014-11-1-0074, scientific monitor Dr. Thomas McKenna. In addition, the authors appreciate the contributions of Dr. Joseph Starr in assisting with stereoscopic IR vision system.
About this article
Cite this article
McNeil, J.G., Lattimer, B.Y. Robotic Fire Suppression Through Autonomous Feedback Control. Fire Technol 53, 1171–1199 (2017). https://doi.org/10.1007/s10694-016-0623-1
- Autonomous fire suppression
- Fire detection
- Suppression systems
- IR stereovision