Skip to main content
Log in

Monocular 3D Exploration using Lines-of-Sight and Local Maps

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Nowadays, robots equipped with a single camera, such as micro aerial vehicles (MAVs), are easily found at affordable costs. They can be used in different tasks, including the building of 3D environment maps. For building such maps, monocular simultaneous localization and mapping (SLAM) methods are employed, which usually generate sparse or semi-dense representations that are ill-suited for navigation tasks. We propose a new 3D exploration approach that uses a monocular camera as the only source of information. Our approach transforms a point cloud generated by monocular SLAM into local volumetric maps. These maps are built using the lines-of-sight between points and keyframes, allowing the MAV to navigate safely through the environment. Goal poses are dynamically defined to guide the MAV to explore the environment while avoiding obstacles. Besides that, the proposed approach seeks to determine properly when the environment was entirely explored, preventing that MAV stops before cover all the environment or flies more that is needed. The effectiveness of the proposed approach is evaluated in experiments in two different indoor environments, and show that it is possible to explore an environment using only a MAV equipped with a single monocular camera.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Amigoni, F., Caglioti, V.: An information-based exploration strategy for environment mapping with mobile robots. Robot. Auton. Syst. 58(5), 684–699 (2010). https://doi.org/10.1016/j.robot.2009.11.005

    Article  Google Scholar 

  2. Bhat, S., Meenakshi, M.: Vision based robotic system for military applications–design and real time validation. In: International conference on signal and image processing (ICSIP), pp. 20–25. IEEE. https://doi.org/10.1109/ICSIP.2014.8 (2014)

  3. Bircher, A., Kamel, M., Alexis, K., Oleynikova, H., Siegwart, R.: Receding horizon “next-best-view” planner for 3d exploration. In: International conference on robotics and automation (ICRA), pp. 1462–1468. IEEE. https://doi.org/10.1109/ICRA.2016.7487281 (2016)

  4. Borenstein, J., Koren, Y.: Histogramic in-motion mapping for mobile robot obstacle avoidance. Trans. Robot. Auto. 7(4), 535–539 (1991). https://doi.org/10.1109/70.86083

    Article  Google Scholar 

  5. Brasch, N., Bozic, A., Lallemand, J., Tombari, F.: Semantic monocular slam for highly dynamic environments. In: International conference on intelligent robots and systems (IROS), pp. 393–400. IEEE. https://doi.org/10.1109/IROS.2018.8593828 (2018)

  6. Dunn, E., Frahm, J.M.: Next best view planning for active model improvement. In: British machine vision conference (BMVC), pp. 1–11. https://doi.org/10.5244/C.23.53 (2009)

  7. Engel, J., Schöps, T., Cremers, D.: Lsd-slam: Large-scale direct monocular slam. In: European conference on computer vision (ECCV), pp. 834–849. Springer. https://doi.org/10.1007/978-3-319-10605-2_54(2014)

  8. Faria, M., Maza, I., Viguria, A.: Applying frontier cells based exploration and lazy theta* path planning over single grid-based world representation for autonomous inspection of large 3d structures with an uas. J. Intell. Robot. Syst. 93(1-2), 113–133 (2019). https://doi.org/10.1007/s10846-018-0798-4

    Article  Google Scholar 

  9. Frost, D., Prisacariu, V., Murray, D.: Recovering stable scale in monocular slam using object-supplemented bundle adjustment. Trans. Robot. 34(3), 736–747 (2018). https://doi.org/10.1109/TRO.2018.2820722

    Article  Google Scholar 

  10. Gálvez-López, D., Salas, M., Tardós, J.D., Montiel, J.: Real-time monocular object slam. Robot. Auton. Syst. 75, 435–449 (2016). https://doi.org/10.1016/j.robot.2015.08.009

    Article  Google Scholar 

  11. Han, L., Gao, F., Zhou, B., Shen, S.: Fiesta: Fast incremental euclidean distance fields for online motion planning of aerial robots. In: International conference on intelligent robots and systems (IROS), pp. 4423–4430. IEEE. https://doi.org/10.1109/IROS40897.2019.8968199 (2019)

  12. Hart, P.E., Nilsson, N.J., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. Trans. Syst. Sci. Cybern. 4(2), 100–107 (1968). https://doi.org/10.1109/TSSC.1968.300136

    Article  Google Scholar 

  13. Heng, L., Gotovos, A., Krause, A., Pollefeys, M.: Efficient visual exploration and coverage with a micro aerial vehicle in unknown environments. In: International conference on robotics and automation (ICRA), pp. 1071–1078. IEEE. https://doi.org/10.1109/ICRA.2015.7139309 (2015)

  14. Jorge, V.A., Maffei, R., Franco, G.S., Daltrozo, J., Giambastiani, M., Kolberg, M., Prestes, E.: Ouroboros: Using potential field in unexplored regions to close loops. In: International conference on robotics and automation (ICRA), pp. 2125–2131. IEEE. https://doi.org/10.1109/ICRA.2015.7139479 (2015)

  15. Kanellakis, C., Nikolakopoulos, G.: Survey on computer vision for uavs: Current developments and trends. J. Intell. Robot. Syst. 87(1), 141–168 (2017)

    Article  Google Scholar 

  16. Klein, G., Murray, D.: Parallel tracking and mapping for small ar workspaces. In: International symposium on mixed and augmented reality (ISMAR), pp. 225–234. IEEE. https://doi.org/10.1109/ISMAR.2007.4538852 (2007)

  17. Korb, R., Schöttl, A.: Exploring unstructured environment with frontier trees. J. Intell. Robot. Syst. 91(3-4), 617–628 (2018). https://doi.org/10.1007/s10846-017-0754-8

    Article  Google Scholar 

  18. Lee, S.H., Civera, J.: Loosely-coupled semi-direct monocular slam. Robot. Auto. Lett. 4(2), 399–406 (2018). https://doi.org/10.1109/LRA.2018.2889156

    Article  Google Scholar 

  19. Lim, H., Lim, J., Kim, H.J.: Real-time 6-dof monocular visual slam in a large-scale environment. In: International conference on robotics and automation (ICRA), pp. 1532–1539. IEEE. https://doi.org/10.1109/ICRA.2014.6907055 (2014)

  20. Lukierski, R., Leutenegger, S., Davison, A.J.: Rapid free-space mapping from a single omnidirectional camera. In: European conference on mobile robots (ECMR), pp. 1–8. IEEE. https://doi.org/10.1109/ECMR.2015.7324222(2015)

  21. Madhavan, R., Amory, A., Prestes, E., Guedes, R., Bergamin, A., Neuland, R., Mantelli, M., Kindin, D., Rodrigues, F.A.: The 2017 humanitarian robotics and automation technology challenge [humanitarian technology]. IEEE Robot. Auto. Mag. 24, 127–129 (2017). https://doi.org/10.1109/MRA.2017.2757722

    Article  Google Scholar 

  22. Maffei, R., Jorge, V.A.M., Prestes, E., Kolberg, M.: Integrated exploration using time-based potential rails. In: International conference on robotics and automation (ICRA), pp. 3694–3699. IEEE. https://doi.org/10.1109/ICRA.2014.6907394 (2014)

  23. Maxwell, P., Larkin, D., Lowrance, C.: Turning remote-controlled military systems into autonomous force multipliers. IEEE Potentials 32(6), 39–43 (2013). https://doi.org/10.1109/MPOT.2013.2252240

    Article  Google Scholar 

  24. Mostegel, C., Wendel, A., Bischof, H.: Active monocular localization: Towards autonomous monocular exploration for multirotor mavs. In: International conference on robotics and automation (ICRA), pp. 3848–3855. IEEE. https://doi.org/10.1109/ICRA.2014.6907417 (2014)

  25. Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: ORB-SLAM: A versatile and accurate monocular SLAM system. Trans. Robot. 31(5), 1147–1163 (2015). https://doi.org/10.1109/TRO.2015.2463671

    Article  Google Scholar 

  26. Nex, F., Remondino, F.: Preface: Latest developments, methodologies, and applications based on uav platforms. Drones 3(1), 26 (2019). https://doi.org/10.3390/drones3010026

    Article  Google Scholar 

  27. Oleynikova, H., Burri, M., Taylor, Z., Nieto, J., Siegwart, R., Galceran, E.: Continuous-time trajectory optimization for online uav replanning. In: International conference on intelligent robots and systems (IROS), pp. 5332–5339. IEEE. https://doi.org/10.1109/IROS.2016.7759784 (2016)

  28. Oleynikova, H., Taylor, Z., Fehr, M., Siegwart, R., Nieto, J.: Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning. In: International conference on intelligent robots and systems (IROS), pp. 1366–1373. IEEE. https://doi.org/10.1109/IROS.2017.8202315 (2017)

  29. Oßwald, S., Bennewitz, M., Burgard, W., Stachniss, C.: Speeding-up robot exploration by exploiting background information. Robot. Auto. Lett. 1(2), 716–723 (2016). https://doi.org/10.1109/LRA.2016.2520560

    Article  Google Scholar 

  30. Palazzolo, E., Stachniss, C.: Information-driven autonomous exploration for a vision-based mav. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol. 4, p. 59. Copernicus GmbH. https://doi.org/10.5194/isprs-annals-IV-2-W3-59-2017 (2017)

  31. Papachristos, C., Mascarich, F., Khattak, S., Dang, T., Alexis, K.: Localization uncertainty-aware autonomous exploration and mapping with aerial robots using receding horizon path-planning. Auton. Robot. 2019, 1–31 (2019). https://doi.org/10.1007/s10514-019-09864-1

    Google Scholar 

  32. Prestes, E., Marques, L., Neuland, R., Mantelli, M., Maffei, R., Dogru, S., Prado, J., Macedo, J., Madhavan, R.: The 2016 humanitarian robotics and automation technology challenge [competitions]. IEEE Robot. Auto. Mag. 23(3), 23–24 (2016). https://doi.org/10.1109/MRA.2016.2587921

    Article  Google Scholar 

  33. Pumarola, A., Vakhitov, A., Agudo, A., Sanfeliu, A., Moreno-Noguer, F.: Pl-slam: Real-time monocular visual slam with points and lines. In: International conference on robotics and automation (ICRA), pp. 4503–4508. IEEE. https://doi.org/10.1109/ICRA.2017.7989522 (2017)

  34. Quattrini Li, A., Cipolleschi, R., Giusto, M., Amigoni, F.: A semantically-informed multirobot system for exploration of relevant areas in search and rescue settings. Auton. Robot. 40(4), 581–597 (2016). https://doi.org/10.1007/s10514-015-9480-x

    Article  Google Scholar 

  35. Selin, M., Tiger, M., Duberg, D., Heintz, F., Jensfelt, P.: Efficient autonomous exploration planning of large-scale 3-d environments. IEEE Robot. Auto. Lett. 4(2), 1699–1706 (2019). https://doi.org/10.1109/LRA.2019.2897343

    Article  Google Scholar 

  36. Shade, R., Newman, P.: Choosing where to go: Complete 3d exploration with stereo. In: International conference on robotics and automation, pp. 2806–2811. IEEE. https://doi.org/10.1109/ICRA.2011.5980121 (2011)

  37. Shah, S., Dey, D., Lovett, C., Kapoor, A.: Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In: Field and service robotics, pp. 621–635. Springer (2018)

  38. Shen, S., Michael, N., Kumar, V.: Autonomous indoor 3d exploration with a micro-aerial vehicle. In: International conference on robotics and automation (ICRA), pp. 9–15. IEEE. https://doi.org/10.1109/ICRA.2012.6225146 (2012)

  39. e Silva, Jr, E.P., Engel, P.M., Trevisan, M., Idiart, M.A.: Exploration method using harmonic functions. Robot. Auton. Syst. 40(1), 25–42 (2002). https://doi.org/10.1016/S0921-8890(02)00209-9

    Article  Google Scholar 

  40. Song, S., Jo, S.: Online inspection path planning for autonomous 3d modeling using a micro-aerial vehicle. In: International conference on robotics and automation (ICRA), pp. 6217–6224. IEEE. https://doi.org/10.1109/ICRA.2017.7989737 (2017)

  41. von Stumberg, L., Usenko, V., Engel, J., Stückler, J., Cremers, D.: From monocular slam to autonomous drone exploration. In: European conference on mobile robots (ECMR), pp. 1–8. IEEE. https://doi.org/10.1109/ECMR.2017.8098709 (2017)

  42. Tordesillas, J., Lopez, B.T., Everett, M., How, J.P.: Faster: Fast and safe trajectory planner for flights in unknown environments. In: International conference on intelligent robots and systems (IROS), pp. 1934–1940. IEEE. https://doi.org/10.1109/IROS40897.2019.8968021 (2019)

  43. Vallvé, J., Andrade-Cetto, J.: Potential information fields for mobile robot exploration. Robot. Auton. Syst. 69, 68–79 (2015). https://doi.org/10.1016/j.robot.2014.08.009

    Article  Google Scholar 

  44. Witting, C., Fehr, M., Bähnemann, R., Oleynikova, H., Siegwart, R.: History-aware autonomous exploration in confined environments using mavs. In: International conference on intelligent robots and systems (IROS), pp. 1–9. IEEE. https://doi.org/10.1109/IROS.2018.8594502 (2018)

  45. Yamauchi, B.: A frontier-based approach for autonomous exploration. In: International symposium on computational intelligence in robotics and automation (CIRA), pp. 146–151. IEEE. https://doi.org/10.1109/CIRA.1997.613851 (1997)

  46. Zhou, B., Gao, F., Wang, L., Liu, C., Shen, S.: Robust and efficient quadrotor trajectory generation for fast autonomous flight. Robot. Auto. Lett. 4(4), 3529–3536 (2019). https://doi.org/10.1109/LRA.2019.2927938

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Diego Pittol.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The TITAN Xp used for this research was donated by the NVIDIA Corporation. This study was financed in part by the Brazilian National Council for Scientific and Technological Development (CNPq) and by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pittol, D., Mantelli, M., Maffei, R. et al. Monocular 3D Exploration using Lines-of-Sight and Local Maps. J Intell Robot Syst 100, 465–481 (2020). https://doi.org/10.1007/s10846-020-01208-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-020-01208-x

Keywords

Navigation