Skip to main content
SpringerLink
Log in

Forest Fire Localization: From Reinforcement Learning Exploration to a Dynamic Drone Control

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Globalization impacts directly pollution, and one of the side effects is the increasing number of forest fires and their fast propagation. To reduce this phenomenon, several studies have tried to model fire propagation and find an adequate method to predict (or follow) this spread to stop it. However, to the best of our knowledge, most of the developed works do not use an adaptive exploration strategy to follow the fire evolution. Instead, they are mainly focused on observing the evolution and trying to find an adequate spread model to predict fire behavior. Due to the lack of all the exact parameters to predict the fire spread, in this paper, we propose a new exploration strategy based on reinforcement learning using an Unmanned Aerial Vehicle (UAV). The learned exploration strategy is able to predict and follow the fire propagation. To follow the generated waypoints by the exploration strategy, we develop a new robust controller, which can be embedded in a UAV. All these modules are encapsulated and interact with each other using a synchronized architecture. To prove the efficiency of our proposal, we present the obtained results on several simulated forest fire scenarios and compare them with other approaches. Then, We validate our entire architecture and show its efficiency using software in the loop environment using the Robot Operating System (ROS).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Code availability

All code generated or used during the study are available from the corresponding author by request.

References

  1. Deb, P., Moradkhani, H., Abbaszadeh, P., Kiem, A.S., Engström, J., Keellings, D., Sharma, A.: Causes of the widespread 2019–2020 australian bushfire season. Earth’s Future. 8(11), 2020–001671 (2020) https://doi.org/10.1029/2020EF001671https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2020EF001671. e2020EF001671 2020EF001671

  2. Dorrer, G., Yarovoy, S.: Use of agent-based modeling for wildfire situations simulation. In: 2018 3rd Russian-Pacific Conference on Computer Technology and Applications (RPC), pp. 1–4 (2018)

  3. Lin, Z., Liu, H.H.T., Wotton, M.: Kalman filter-based large-scale wildfire monitoring with a system of uavs. IEEE Transactions on Industrial Electronics. 66(1), 606–615 (2019)

    Article  Google Scholar 

  4. Mainetti, L., Patrono, L., Vilei, A.: Evolution of wireless sensor networks towards the internet of things: a survey. In: SoftCOM 2011, 19th International Conference on Software, Telecommunications and Computer Networks, pp. 1–6 (2011)

  5. Gennaro, M.d.: Modélisation de la propagation des grands incendies de forêts et élaboration d’un outil opérationnel d’aide à la lutte tactique. PhD thesis (2017). Thèse de doctorat dirigée par Porterie, Bernard Sciences de l’ingénieur. Energétique Aix-Marseille 2017. http://www.theses.fr/2017AIXM0148

  6. Alexandridis, A., Vakalis, D., Siettos, C.I., Bafas, G.V.: A cellular automata model for forest fire spread prediction: the case of the wildfire that swept through spetses island in 1990, vol. 204, pp. 191–201 (2008). https://doi.org/10.1016/j.amc.2008.06.046 . http://www.sciencedirect.com/science/article/pii/S0096300308004943

  7. Ravankar, A.A., Ravankar, A., Kobayashi, Y., Emaru, T.: Autonomous mapping and exploration with unmanned aerial vehicles using low cost sensors. Proceedings. 4(1) (2019) https://doi.org/10.3390/ecsa-5-05753

  8. Katsev, M., Yershova, A., Tovar, B., Ghrist, R., LaValle, S.M.: Mapping and pursuit-evasion strategies for a simple wall-following robot. IEEE Transactions on Robotics. 27(1), 113–128 (2011). https://doi.org/10.1109/TRO.2010.2095570

    Article  Google Scholar 

  9. Tovar, B., Murrieta-Cid, R., M. LaValle, S.: Distance-optimal navigation in an unknown environment without sensing distances. Robotics, IEEE Transactions on. 23, 506–518 (2007) https://doi.org/10.1109/TRO.2007.898962

  10. Belbachir, A., Gustave, J., Muhammad, N., Zelinka, I.: Toward an exploration-based probabilistic reasoning for a quadrotor. Intell. Serv. Robotics. 14(4), 563–570 (2021). https://doi.org/10.1007/s11370-021-00378-3

    Article  Google Scholar 

  11. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms (2017)

  12. Anastasiou, A., Papaioannou, S., Kolios, P., Panayiotou, C.G.: Model predictive control for multiple castaway tracking with an autonomous aerial agent. In: 2023 European Control Conference (ECC), pp. 1–8 (2023). IEEE

  13. Chahal, J., Belbachir, A., Seghrouchni, A.E.F.: Dynamic interest points: a formalism to identify areas to patrol within a continuous environment. In: Bui, T.X. (ed.) 56th Hawaii International Conference on System Sciences, HICSS 2023, Maui, Hawaii, USA, January 3-6, 2023, pp. 6853–6862. ScholarSpace, ??? (2023). https://hdl.handle.net/10125/103464

  14. Belbachir, A., Escareno, J., Rubio, E., Sossa, H.: Preliminary results on uav-based forest fire localization based on decisional navigation. In: 2015 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 377–382 (2015). https://doi.org/10.1109/RED-UAS.2015.7441030

  15. Bella, S., Belbachir, A., Belalem, G.: A hybrid air-sea cooperative approach combined with a swarm trajectory planning method. Paladyn J. Behav. Robotics. 11(1), 118–139 (2020). https://doi.org/10.1515/pjbr-2020-0006

    Article  Google Scholar 

  16. Gustave, J., Chahal, J., Belbachir, A.: Functional architecture using ros for autonomous uavs. In: Proceedings of the 17th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, pp. 506–512 (2020). https://doi.org/10.5220/0009888305060512 . INSTICC, SciTePress

  17. Belbachir, A., Escareno, J.-A.: Autonomous decisional high-level planning for uavs-based forest-fire localization. In: Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics. ICINCO 2016, pp. 153–159. SCITEPRESS - Science and Technology Publications, Lda, Setubal, PRT (2016). https://doi.org/10.5220/0005972501530159. https://doi.org/10.5220/0005972501530159

  18. Yuan, C., Liu, Z., Zhang, Y.: Uav-based forest fire detection and tracking using image processing techniques. In: 2015 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 639–643 (2015). IEEE

  19. Lu, K., Xu, R., Li, J., Lv, Y., Lin, H., Liu, Y.: A vision-based detection and spatial localization scheme for forest fire inspection from uav. Forests. 13(3), 383 (2022)

    Article  Google Scholar 

  20. Ingrand, F., Lacroix, S., Lemai-Chenevier, S., Py, F.: Decisional autonomy of planetary rovers. Journal of Field Robotics. 24(7), 559–580 (2007). https://doi.org/10.1002/rob.20206, https://arxiv.org/abs/https://onlinelibrary.wiley.com/doi/pdf/10.1002/rob.20206

  21. Gromada, K.A., Stecz, W.M.: Designing a reliable uav architecture operating in a real environment. Applied Sciences. 12(1) (2022) https://doi.org/10.3390/app12010294

  22. Gustave, J., Chahal, J., Belbachir, A.: Functional architecture using ros for autonomous uavs. In: ICINCO: 506-512 (2020)

  23. Bailon-Ruiz, R., Lacroix, S.: Wildfire remote sensing with uavs: A review from the autonomy point of view. In: 2020 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 412–420 (2020). https://doi.org/10.1109/ICUAS48674.2020.9213986

  24. Yamauchi, B.: A frontier-based approach for autonomous exploration. In: In Proceedings of the IEEE International Symposium on Computational Intelligence, Robotics and Automation, pp. 146–151 (1997)

  25. Bachrach, A., He, R., Roy, N.: Autonomous flight in unknown indoor environments. International Journal of Micro Air Vehicles. 1(4), 217–228 (2009). https://doi.org/10.1260/175682909790291492

  26. Basilico, N., Amigoni, F.: Exploration strategies based on multi-criteria decision making for search and rescue autonomous robots. In: AAMAS (2011)

  27. Bouabdallah, S., Noth, A., Siegwart, R.: Pid vs lq control techniques applied to an indoor micro quadrotor. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), vol. 3, pp. 2451–2456 (2004). IEEE

  28. Milhim, A., Zhang, Y., Rabbath, C.-A.: Gain scheduling based pid controller for fault tolerant control of quad-rotor uav. In: AIAA Infotech@ Aerospace 2010, p. 3530 (2010)

  29. Rinaldi, F., Chiesa, S., Quagliotti, F.: Linear quadratic control for quadrotors uavs dynamics and formation flight. Journal of Intelligent & Robotic Systems. 70(1), 203–220 (2013)

    Article  Google Scholar 

  30. Szafranski, G., Czyba, R.: Different approaches of pid control uav type quadrotor. (2011)

  31. Fritsch, O., De Monte, P., Buhl, M., Lohmann, B.: Quasi-static feedback linearization for the translational dynamics of a quadrotor helicopter. In: 2012 American Control Conference (ACC), pp. 125–130 (2012). IEEE

  32. Ha, C., Zuo, Z., Choi, F.B., Lee, D.: Passivity-based adaptive backstepping control of quadrotor-type uavs. Robotics and Autonomous Systems. 62(9), 1305–1315 (2014)

    Article  Google Scholar 

  33. Luque-Vega, L., Castillo-Toledo, B., Loukianov, A.G.: Robust block second order sliding mode control for a quadrotor. Journal of the Franklin Institute. 349(2), 719–739 (2012)

    Article  MathSciNet  Google Scholar 

  34. Islam, S., Liu, P., El Saddik, A.: Nonlinear adaptive control for quadrotor flying vehicle. Nonlinear Dynamics. 78(1), 117–133 (2014)

    Article  MathSciNet  Google Scholar 

  35. Kang, B., Miao, Y., Liu, F., Duan, J., Wang, K., Jiang, S.: A second-order sliding mode controller of quad-rotor uav based on pid sliding mode surface with unbalanced load. Journal of Systems Science and Complexity. 34(2), 520–536 (2021)

    Article  MathSciNet  Google Scholar 

  36. Alvarez-Muñoz, J., Escareno, J., Chevalier, J., Daix, S., Labanni-Igbida, O.: Wind-tolerant event-based adaptive sliding-mode control for vtol rotorcrafts multi-agent systems. IEEE Transactions on Aerospace and Electronic Systems. (2022)

  37. Alvarez-Muñoz, J., Chevalier, J., Castillo-Zamora, J.J., Escareno, J.: Distributed event-based sliding-mode consensus control in dynamic formation for vtol-uavs. In: 2021 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1364–1373 (2021). IEEE

  38. Zhou, C., Tao, H., Chen, Y., Stojanovic, V., Paszke, W.: Robust point-to-point iterative learning control for constrained systems: a minimum energy approach. International Journal of Robust and Nonlinear Control. 32(18), 10139–10161 (2022)

    Article  MathSciNet  Google Scholar 

  39. Zhuang, Z., Tao, H., Chen, Y., Stojanovic, V., Paszke, W.: Iterative learning control for repetitive tasks with randomly varying trial lengths using successive projection. International Journal of Adaptive Control and Signal Processing. 36(5), 1196–1215 (2022)

  40. Dierks, T., Jagannathan, S.: Output feedback control of a quadrotor uav using neural networks. IEEE transactions on neural networks. 21(1), 50–66 (2009)

    Article  Google Scholar 

  41. Sun, C., Liu, M., Liu, C., Feng, X., Wu, H.: An industrial quadrotor uav control method based on fuzzy adaptive linear active disturbance rejection control. Electronics. 10(4), 376 (2021)

    Article  Google Scholar 

  42. Santos, M., Lopez, V., Morata, F.: Intelligent fuzzy controller of a quadrotor. In: 2010 IEEE International Conference on Intelligent Systems and Knowledge Engineering, pp. 141–146 (2010). IEEE

  43. Muliadi, J., Kusumoputro, B.: Neural network control system of uav altitude dynamics and its comparison with the pid control system. Journal of Advanced Transportation. 2018 (2018)

  44. Castillo-Zamora, J.J., Camarillo-Gomez, K.A., Perez-Soto, G.I., Rodriguez-Resendiz, J.: Comparison of pd, pid and sliding-mode position controllers for v-tail quadcopter stability. Ieee Access. 6, 38086–38096 (2018)

  45. Velásquez, W., Munoz-Arcentales, A., Bohnert, T.M., Salvachúa, J.: Wildfire propagation simulation tool using cellular automata and gis. In: 2019 International Symposium on Networks, Computers and Communications (ISNCC), pp. 1–7 (2019)

  46. Viegas, D.: Overview of forest fire propagation research. Fire Safety Science. 10, 95–108 (2011)

    Article  Google Scholar 

  47. Zaitsev, D.A.: A generalized neighborhood for cellular automata. Theoretical Computer Science. 666, 21–35 (2017) https://doi.org/10.1016/j.tcs.2016.11.002

  48. Mousavi, S., Schukat, M., Howley, E.: Deep reinforcement learning: An overview, pp. 426–440 (2018). https://doi.org/10.1007/978-3-319-56991-8_32

  49. Guerrero-Castellanos, J.F., Marchand, N., Hably, A., Lesecq, S., Delamare, J.: Bounded attitude control of rigid bodies: real-time experimentation to a quadrotor mini-helicopter. Control Engineering Practice. 19(8), 790–797 (2011)

    Article  Google Scholar 

  50. Chalanga, A., Kamal, S., Fridman, L.M., Bandyopadhyay, B., Moreno, J.A.: Implementation of super-twisting control: super-twisting and higher order sliding-mode observer-based approaches. IEEE Transactions on Industrial Electronics. 63(6), 3677–3685 (2016)

    Article  Google Scholar 

  51. Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W.M., Donahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., Simonyan, K., Fernando, C., Kavukcuoglu, K.: Population Based Training of Neural Networks (2017)

  52. Liang, E., Liaw, R., Moritz, P., Nishihara, R., Fox, R., Goldberg, K., Gonzalez, J.E., Jordan, M.I., Stoica, I.: RLlib: Abstractions for Distributed Reinforcement Learning (2018)

  53. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  54. Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., Wu, Y.: The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games (2022)

  55. Pham, H.X., La, H.M., Feil-Seifer, D., Deans, M.: A distributed control framework for a team of unmanned aerial vehicles for dynamic wildfire tracking. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6648–6653 (2017). https://doi.org/10.1109/IROS.2017.8206579

Download references

Acknowledgements

This research has received no external support.

Funding

The research has not been funded.

Author information

Authors and Affiliations

  1. Polytechnic Institute of Advanced Sciences (IPSA), 63 Boulevard de Brandebourg, Ivry-sur-Seine, 94200, France

    Jonatan Alvarez, Jamy Chahal, Abdelhak Goudjil, Johvany Gustave & Aybüke Öztürk Suri

  2. Sorbonne Université, LIP6 - UMR 7606 CNRS, Paris, France

    Assia Belbachir & Jamy Chahal

  3. NORCE Norwegian Research Centre, Bergen, Norway

    Assia Belbachir

  4. LyRIDS, ECE Paris, 10 rue Sextius Michel, Paris, France

    Faiza Belbachir

Authors
  1. Jonatan Alvarez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Assia Belbachir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Faiza Belbachir
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jamy Chahal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abdelhak Goudjil
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Johvany Gustave
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Aybüke Öztürk Suri
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, A.B.; methodology, all the authors; software, J.A., J.Ch., A.G. and J.G.; validation, J.A., J.Ch. and J.G; investigation, A.B., F.B., J.Ch., A.G. and A.O.; data curation, A.B., F.B., A.G. and A.O.; writing-original draft preparation, all the authors; writing-review and editing, J.A., A.B., A.G., J.G. and A.O. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Jonatan Alvarez.

Ethics declarations

Ethics Approval/Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Conflict of Interest/Competing Interests

All authors declare that they have no conflicts of interest. The manuscript is approved by all authors for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alvarez, J., Belbachir, A., Belbachir, F. et al. Forest Fire Localization: From Reinforcement Learning Exploration to a Dynamic Drone Control. J Intell Robot Syst 109, 83 (2023). https://doi.org/10.1007/s10846-023-02004-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-023-02004-z

Keywords

Search

Navigation