Skip to main content

Advertisement

SpringerLink
Log in

A Fully-Autonomous Aerial Robot for Search and Rescue Applications in Indoor Environments using Learning-Based Techniques

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Search and Rescue (SAR) missions represent an important challenge in the robotics research field as they usually involve exceedingly variable-nature scenarios which require a high-level of autonomy and versatile decision-making capabilities. This challenge becomes even more relevant in the case of aerial robotic platforms owing to their limited payload and computational capabilities. In this paper, we present a fully-autonomous aerial robotic solution, for executing complex SAR missions in unstructured indoor environments. The proposed system is based on the combination of a complete hardware configuration and a flexible system architecture which allows the execution of high-level missions in a fully unsupervised manner (i.e. without human intervention). In order to obtain flexible and versatile behaviors from the proposed aerial robot, several learning-based capabilities have been integrated for target recognition and interaction. The target recognition capability includes a supervised learning classifier based on a computationally-efficient Convolutional Neural Network (CNN) model trained for target/background classification, while the capability to interact with the target for rescue operations introduces a novel Image-Based Visual Servoing (IBVS) algorithm which integrates a recent deep reinforcement learning method named Deep Deterministic Policy Gradients (DDPG). In order to train the aerial robot for performing IBVS tasks, a reinforcement learning framework has been developed, which integrates a deep reinforcement learning agent (e.g. DDPG) with a Gazebo-based simulator for aerial robotics. The proposed system has been validated in a wide range of simulation flights, using Gazebo and PX4 Software-In-The-Loop, and real flights in cluttered indoor environments, demonstrating the versatility of the proposed system in complex SAR missions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abrahamsen, H.B.: A remotely piloted aircraft system in major incident management: concept and pilot, feasibility study. BMC Emergency Medicine 15(1), 12 (2015)

    Article  MathSciNet  Google Scholar 

  2. Achtelik, M., Bachrach, A., He, R., Prentice, S., Roy, N.: Autonomous Navigation and Exploration of a Quadrotor Helicopter in Gps-Denied Indoor Environments. In: 1st Symposium on Indoor Flight, 2009. Citeseer (2009)

  3. Bachrach, A., He, R., Roy, N.: Autonomous flight in unknown indoor environments. International Journal of Micro Air Vehicles 1(4), 217–228 (2009)

    Article  Google Scholar 

  4. Bavle, H., Sanchez-Lopez, J.L., Rodriguez-Ramos, A., Sampedro, C., Campoy, P.: A Flight Altitude Estimator for Multirotor Uavs in Dynamic and Unstructured Indoor Environments. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS), Pp. 1044–1051. https://doi.org/10.1109/ICUAS.2017.7991467 (2017)

  5. Bejiga, M.B., Zeggada, A., Nouffidj, A., Melgani, F.: A convolutional neural network approach for assisting avalanche search and rescue operations with uav imagery. Remote Sens. 9(2), 100 (2017)

    Article  Google Scholar 

  6. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv:1606.01540 (2016)

  7. Carrio, A., Sampedro, C., Rodriguez-Ramos, A., Campoy, P.: A review of deep learning methods and applications for unmanned aerial vehicles. Journal of Sensors 2017 (2017)

  8. Caruana, R., Niculescu-Mizil, A.: An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on Machine learning, pp. 161–168. ACM (2006)

  9. Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1–27:27 (2011). Software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm

    Article  Google Scholar 

  10. Chaumette, F.: Potential Problems of Stability and Convergence in Image-Based and Position-Based Visual Servoing. In: The Confluence of Vision and Control, pp. 66–78. Springer (1998)

  11. Chaumette, F., Hutchinson, S.: Visual servo control. i. basic approaches. IEEE Robotics Automation Magazine 13(4), 82–90 (2006). https://doi.org/10.1109/MRA.2006.250573

    Article  Google Scholar 

  12. Chaumette, F., Malis, E.: 2 1/2 D Visual Servoing: a Possible Solution to Improve Image-Based and Position-Based Visual Servoings. In: Proceedings of the IEEE International Conference on Robotics and Automation, 2000. ICRA’00, vol. 1, pp 630–635 (2000)

  13. Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp 886–893 (2005)

  14. Doherty, P., Rudol, P.: A Uav Search and Rescue Scenario with Human Body Detection and Geolocalization. In: Australian Conference on Artificial Intelligence, vol. 4830, pp. 1–13. Springer (2007)

  15. Dorigo, M., Colombetti, M.: Robot shaping: an experiment in behavior engineering. MIT press, Cambridge (1998)

    Google Scholar 

  16. Erdos, D., Erdos, A., Watkins, S.E.: An experimental uav system for search and rescue challenge. IEEE Aerosp. Electron. Syst. Mag. 28(5), 32–37 (2013)

    Article  Google Scholar 

  17. Erez, T., Smart, W.D.: What Does Shaping Mean for Computational Reinforcement Learning? In: 7Th IEEE International Conference on Development and Learning, 2008. ICDL 2008, pp 215–219 (2008)

  18. Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)

    MATH  Google Scholar 

  19. Fleck, M.: Usability of lightweight defibrillators for uav delivery. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 3056–3061. ACM (2016)

  20. Furrer, F., Burri, M., Achtelik, M., Siegwart, R.: Rotors-A Modular Gazebo Mav Simulator Framework. In: Robot Operating System (ROS), pp. 595–625. Springer (2016)

  21. Gatteschi, V., Lamberti, F., Paravati, G., Sanna, A., Demartini, C., Lisanti, A., Venezia, G.: New Frontiers of Delivery Services Using Drones: a Prototype System Exploiting a Quadcopter for Autonomous Drug Shipments. In: 2015 IEEE 39Th Annual, Computer Software and Applications Conference (COMPSAC), vol. 2, pp. 920–927 (2015)

  22. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587 (2014)

  23. Grzonka, S., Grisetti, G., BUrgard, W.: A fully autonomous indoor quadrotor. IEEE Trans. Robot. 28 (1), 90–100 (2012)

    Article  Google Scholar 

  24. Kingma, D., Ba, J.: Adam: A method for stochastic optimization arXiv:1412.6980 (2014)

  25. Kohlbrecher, S., Meyer, J., von Stryk, O., Klingauf, U.: A Flexible and Scalable Slam System with Full 3D Motion Estimation. In: Proceedings of the IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR) (2011)

  26. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet Classification with Deep Convolutional Neural Networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  27. Laud, A.D.: Theory and application of reward shaping in reinforcement learning. Tech rep (2004)

  28. Lee, A. X., Levine, S., Abbeel, P.: Learning visual servoing with deep features and fitted q-iteration arXiv:1703.11000 (2017)

  29. Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning arXiv:1509.02971 (2015)

  30. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd:Single Shot Multibox Detector. In: European Conference on Computer Vision, pp. 21–37. Springer (2016)

  31. Malis, E.: Improving Vision-Based Control Using Efficient Second-Order Minimization Techniques. In: Proceedings of the 2004 IEEE International Conference on Robotics and Automation, 2004. ICRA’04, vol. 2, pp 1843–1848 (2004)

  32. Marchand, É., Spindler, F., Chaumette, F.: Visp for visual servoing: a generic software platform with a wide class of robot control skills. IEEE Robot. Autom. Mag. 12(4), 40–52 (2005)

    Article  Google Scholar 

  33. Marder-Eppstein, E., Berger, E., Foote, T., Gerkey, B., Konolige, K.: The Office Marathon: Robust Navigation in an Indoor Office Environment. In: International Conference on Robotics and Automation (2010)

  34. Meier, L., Tanskanen, P., Fraundorfer, F., Pollefeys, M.: Pixhawk: a System for Autonomous Flight Using Onboard Computer Vision. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp 2992–2997 (2011)

  35. Moore, T., Stouch, D.: A generalized extended kalman filter implementation for the robot operating system. In: Proceedings of the 13th International Conference on Intelligent Autonomous Systems (IAS-13). Springer (2014)

  36. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807–814 (2010)

  37. Pestana, J., Mellado-Bataller, I., Sanchez-Lopez, J.L., Fu, C., Mondragón, I.F., Campoy, P.: A general purpose configurable controller for indoors and outdoors gps-denied navigation for multirotor unmanned aerial vehicles. J. Intell. Robot. Syst. 73(1-4), 387–400 (2014)

    Article  Google Scholar 

  38. Polvara, R., Patacchiola, M., Sharma, S., Wan, J., Manning, A., Sutton, R., Cangelosi, A.: Autonomous quadrotor landing using deep reinforcement learning arXiv:1709.03339 (2017)

  39. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an Open-Source Robot Operating System. In: ICRA Workshop on Open Source Software, vol. 3, pp. 5. Kobe, Japan (2009)

  40. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-Cnn: Towards Real-Time Object Detection with Region Proposal Networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  41. Rudol, P., Doherty, P.: Human Body Detection and Geolocalization for Uav Search and Rescue Missions Using Color and Thermal Imagery. In: 2008 IEEE Aerospace Conference, pp. 1–8. IEEE (2008)

  42. Sadeghi, F., Levine, S.: (cad)2rl: Real single-image flight without a single real image arXiv:1611.04201 (2016)

  43. Sampedro, C., Bavle, H., Rodríguez-Ramos, A., Carrio, A., Fernández, R.A.S., Sanchez-Lopez, J.L., Campoy, P.: A Fully-Autonomous Aerial Robotic Solution for the 2016 International Micro Air Vehicle Competition. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS), pp 989–998. IEEE (2017)

  44. Sampedro, C., Bavle, H., Sanchez-Lopez, J.L., Fernandez, R.A.S., Rodriguez-Ramos, A., Molina, M., Campoy, P.: A Flexible and Dynamic Mission Planning Architecture for Uav Swarm Coordination. In: 2016 International Conference on Unmanned Aircraft Systems (ICUAS), pp 355–363. IEEE (2016)

  45. Sanchez-Lopez, J.L., Molina, M., Bavle, H., Sampedro, C., Fernández, R.A.S., Campoy, P.: A multi-layered component-based approach for the development of aerial robotic systems: The aerostack framework. J. Intell. Robot. Syst 88(2–4), 683–709 (2017)

    Article  Google Scholar 

  46. Scherer, J., Yahyanejad, S., Hayat, S., Yanmaz, E., Andre, T., Khan, A., Vukadinovic, V., Bettstetter, C., Hellwagner, H., Rinner, B.: An autonomous multi-uav system for search and rescue. In: Proceedings of the First Workshop on Micro Aerial Vehicle Networks, Systems, and Applications for Civilian Use, pp. 33–38. ACM (2015)

  47. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  48. Sun, J., Li, B., Jiang, Y., Wen, C.Y.: A camera-based target detection and positioning uav system for search and rescue (sar) purposes. Sensors 16(11), 1778 (2016)

    Article  Google Scholar 

  49. Tomic, T., Schmid, K., Lutz, P., Domel, A., Kassecker, M., Mair, E., Grixa, I.L., Ruess, F., Suppa, M., Burschka, D.: Toward a fully autonomous uav: Research platform for indoor and outdoor urban search and rescue. IEEE Robot. Autom. Mag. 19(3), 46–56 (2012)

    Article  Google Scholar 

  50. Xiang, G., Hardy, A., Rajeh, M., Venuthurupalli, L.: Design of the Life-Ring Drone Delivery System for Rip Current Rescue. In: 2016 IEEE Systems and Information Engineering Design Symposium (SIEDS), pp 181–186. IEEE (2016)

  51. Zhang, F., Leitner, J., Milford, M., Upcroft, B., Corke, P.: Towards vision-based deep reinforcement learning for robotic motion control arXiv:1511.03791 (2015)

  52. Zhu, Y., Mottaghi, R., Kolve, E., Lim, J.J., Gupta, A., Fei-Fei, L., Farhadi, A.: Target-Driven Visual Navigation in Indoor Scenes Using Deep Reinforcement Learning. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3357–3364 (2017)

  53. Zingg, S., Scaramuzza, D., Weiss, S., Siegwart, R.: Mav Navigation through Indoor Corridors Using Optical Flow. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 3361–3368 (2010)

Download references

Acknowledgements

This work was supported by the Spanish Ministry of Science (Project DPI2014-60139-R). The LAL UPM and the MONCLOA Campus of International Excellence are also acknowledged for funding the predoctoral contract of the corresponding author.

Author information

Authors and Affiliations

  1. Centre for Automation and Robotics, Universidad Politécnica de Madrid (CSIC-UPM), Madrid, Spain

    Carlos Sampedro, Alejandro Rodriguez-Ramos, Hriday Bavle, Adrian Carrio, Paloma de la Puente & Pascual Campoy

Authors
  1. Carlos Sampedro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alejandro Rodriguez-Ramos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hriday Bavle
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Adrian Carrio
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paloma de la Puente
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pascual Campoy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Sampedro.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sampedro, C., Rodriguez-Ramos, A., Bavle, H. et al. A Fully-Autonomous Aerial Robot for Search and Rescue Applications in Indoor Environments using Learning-Based Techniques. J Intell Robot Syst 95, 601–627 (2019). https://doi.org/10.1007/s10846-018-0898-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0898-1

Keywords

Search

Navigation