Abstract
Numerous work in the past have devoted to solve task-driven navigation, but how to effectively explore unknown environments and serve down-stream tasks received little attention. In this work, we study how agents effectively overcome reward sparsity and achieve efficient autonomous exploration of complex environments with task-agnostic. We proposed a modular hierarchical method to learn and explore the policy of 3D environments, and studied different reward functions and training paradigms. Combining the advantages of multiple reward functions can more effectively avoid the agent getting into trouble. Our experiments in a realistic simulation of visual and physical 3D environment proved that our method was more effective than the past classical methods and end-to-end methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yamauchi, B.: A frontier-based approach for autonomous exploration. In: Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA 1997, Towards New Computational Principles for Robotics and Automation, pp. 146–151. IEEE (1997)
Chen, T., Gupta, S., Gupta, A.: Learning exploration policies for navigation, arXiv preprint arXiv:1903.01959 (2019)
Chaplot, D.S., Gandhi, D., Gupta, S., Gupta, A., Salakhutdinov, R.: Learning to explore using active neural slam, arXiv preprint arXiv:2004.05155 (2020)
Mur-Artal, R., Tardós, J.D.: Orb-slam2: an open-source slam system for monocular, stereo, and RGB-d cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)
Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: Dtam: dense tracking and mapping in real-time. In: 2011 International Conference on Computer Vision, pp. 2320–2327. IEEE (2011)
Kavraki, L.E., Svestka, P., Latombe, J.-C., Overmars, M.H.: Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. Robot. Autom. 12(4), 566–580 (1996)
Bansal, S., Tolani, V., Gupta, S., Malik, J., Tomlin, C.: Combining optimal control and learning for visual navigation in novel environments. In: Conference on Robot Learning, pp. 420–429. PMLR (2020)
Cadena, C., et al.: Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Trans. Robot. 32(6), 1309–1332 (2016)
Martinez-Cantin, R., De Freitas, N., Brochu, E., Castellanos, J., Doucet, A.: A bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot. Autonom. Robot. 27(2), 93–103 (2009)
Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. Proc. Natl. Acad. Sci. 93(4), 1591–1595 (1996)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Savva, M., et al.: Habitat: a platform for embodied AI research. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9339–9347 (2019)
Pathak, D., Agrawal, P., Efros, A.A., Darrell, T.: Curiosity-driven exploration by self-supervised prediction. In: International Conference on Machine Learning, pp. 2778–2787. PMLR (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Beijing HIWING Sci. and Tech. Info Inst
About this paper
Cite this paper
Xue, Y., Chen, W., Zhang, L. (2023). A Hierarchical SLAM Framework Based on Deep Reinforcement Learning for Active Exploration. In: Fu, W., Gu, M., Niu, Y. (eds) Proceedings of 2022 International Conference on Autonomous Unmanned Systems (ICAUS 2022). ICAUS 2022. Lecture Notes in Electrical Engineering, vol 1010. Springer, Singapore. https://doi.org/10.1007/978-981-99-0479-2_87
Download citation
DOI: https://doi.org/10.1007/978-981-99-0479-2_87
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-0478-5
Online ISBN: 978-981-99-0479-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)