Advertisement

SafeUAV: Learning to Estimate Depth and Safe Landing Areas for UAVs from Synthetic Data

  • Alina MarcuEmail author
  • Dragoş Costea
  • Vlad Licăreţ
  • Mihai Pîrvu
  • Emil Sluşanschi
  • Marius Leordeanu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11130)

Abstract

The emergence of relatively low cost UAVs has prompted a global concern about the safe operation of such devices. Since most of them can ‘autonomously’ fly by means of GPS way-points, the lack of a higher logic for emergency scenarios leads to an abundance of incidents involving property or personal injury. In order to tackle this problem, we propose a small, embeddable ConvNet for both depth and safe landing area estimation. Furthermore, since labeled training data in the 3D aerial field is scarce and ground images are unsuitable, we capture a novel synthetic aerial 3D dataset obtained from 3D reconstructions. We use the synthetic data to learn to estimate depth from in-flight images and segment them into ‘safe-landing’ and ‘obstacle’ regions. Our experiments demonstrate compelling results in practice on both synthetic data and real RGB drone footage.

Keywords

UAVs CNNs Depth estimation Safe landing 

Notes

Acknowledgements

This work was supported in part by Romanian Ministry of European Funds, project IAVPLN POC-A1.2.1D-2015-P39-287 and UEFISCDI, under projects PN-III-P4-ID-ERC-2016-0007 and PN-III-P1-1.2-PCCDI-2017-0734. We would also like to express our gratitude to Aurelian Marcu and The Center for Advanced Laser Technologies for providing use GPU training resources.

References

  1. 1.
    Aguilar, W.G., Casaliglla, V.P., Pólit, J.L.: Obstacle avoidance based-visual navigation for micro aerial vehicles. Electronics 6(1), 10 (2017)CrossRefGoogle Scholar
  2. 2.
    Atapour-Abarghouei, A., Breckon, T.P.: Real-time monocular depth estimation using synthetic data with domain adaptation via image style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 18, p. 1 (2018)Google Scholar
  3. 3.
    Aziz, S., Faheem, R.M., Bashir, M., Khalid, A., Yasin, A.: Unmanned aerial vehicle emergency landing site identification system using machine vision. J. Image Graph. 4(1), 36–41 (2016)CrossRefGoogle Scholar
  4. 4.
    Barekatain, M., et al.: Okutama-action: an aerial view video dataset for concurrent human action detection. In: 1st Joint BMTT-PETS Workshop on Tracking and Surveillance, CVPR, pp. 1–8 (2017)Google Scholar
  5. 5.
    Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)
  6. 6.
    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv preprint arXiv:1802.02611 (2018)
  7. 7.
    Cordts, M., et al.: The CityScapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213–3223 (2016)Google Scholar
  8. 8.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)CrossRefGoogle Scholar
  9. 9.
    Google: Google Earth, version 7.3.0 (2018). https://www.google.com/earth/
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  11. 11.
    Hinzmann, T., Stastny, T., Lerma, C.C., Siegwart, R., Gilitschenski, I.: Free LSD: prior-free visual landing site detection for autonomous planes. IEEE Robot. Autom. Lett. 3, 2545–2552 (2018)CrossRefGoogle Scholar
  12. 12.
    Hu, X., Liu, X., He, Z., Zhang, J.: Batch modeling of 3D city based on ESRI CityEngine (2013)Google Scholar
  13. 13.
    Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, p. 3 (2017)Google Scholar
  14. 14.
    Julian, K., Mern, J., Tompa, R.: UAV depth perception from visual, images using a deep convolutional neural network (2017). http://cs231n.stanford.edu/reports/2017/pdfs/200.pdf
  15. 15.
    Kolbe, T.H., Gröger, G., Plümer, L.: CityGML: interoperable access to 3D city models. In: van Oosterom, P., Zlatanova, S., Fendel, E.M. (eds.) Geo-information for disaster management, pp. 883–899. Springer, Heidelberg (2005).  https://doi.org/10.1007/3-540-27468-5_63CrossRefGoogle Scholar
  16. 16.
    Kundu, J.N., Uppala, P.K., Pahuja, A., Babu, R.V.: Adadepth: unsupervised content congruent adaptation for depth estimation. arXiv preprint arXiv:1803.01599 (2018)
  17. 17.
    Leaverton, G.T.: Generation drone: the future of utility O&M. In: Electrical Transmission and Substation Structures 2015, pp. 190–201 (2015)Google Scholar
  18. 18.
    Lee, J.H., Heo, M., Kim, K.R., Kim, C.S.: Single-image depth estimation based on fourier domain analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 330–339 (2018)Google Scholar
  19. 19.
    Li, X.: A software scheme for UAV’s safe landing area discovery. AASRI Procedia 4, 230–235 (2013)CrossRefGoogle Scholar
  20. 20.
    Li, Z., Snavely, N.: MegaDepth: learning single-view depth prediction from internet photos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2041–2050 (2018)Google Scholar
  21. 21.
    Mahjourian, R., Wicke, M., Angelova, A.: Unsupervised learning of depth and ego-motion from monocular video using 3D geometric constraints. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5667–5675 (2018)Google Scholar
  22. 22.
    Marcu, A., Costea, D., Slusanschi, E., Leordeanu, M.: A multi-stage multi-task neural network for aerial scene interpretation and geolocalization. arXiv preprint arXiv:1804.01322 (2018)
  23. 23.
    Paszke, A., et al.: Automatic differentiation in pytorch. In: NIPS-W (2017)Google Scholar
  24. 24.
    Qi, X., Liao, R., Liu, Z., Urtasun, R., Jia, J.: GeoNet: geometric neural network for joint depth and surface normal estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 283–291 (2018)Google Scholar
  25. 25.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. arXiv preprint arXiv:1505.04597 (2015)
  26. 26.
    Tian, Y., Li, X., Wang, K., Wang, F.Y.: Training and testing object detectors with virtual images. IEEE/CAA J. Automatica Sin. 5(2), 539–546 (2018)CrossRefGoogle Scholar
  27. 27.
    Wang, S., et al.: Torontocity: seeing the world with a million eyes. arXiv preprint arXiv:1612.00423 (2016)
  28. 28.
    Xu, D., Ouyang, W., Wang, X., Sebe, N.: Pad-net: multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing. arXiv preprint arXiv:1805.04409 (2018)
  29. 29.
    Zhan, H., Garg, R., Weerasekera, C.S., Li, K., Agarwal, H., Reid, I.: Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 340–349 (2018)Google Scholar
  30. 30.
    Zhu, P., Wen, L., Bian, X., Ling, H., Hu, Q.: Vision meets drones: a challenge. arXiv preprint arXiv:1804.07437 (2018)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute of Mathematics of the Romanian AcademyBucharestRomania
  2. 2.Autonomous SystemsBucharestRomania
  3. 3.University Politehnica of BucharestBucharestRomania

Personalised recommendations