Skip to main content
SpringerLink
Log in

Deep Reinforcement Learning-Based Robot Exploration for Constructing Map of Unknown Environment

  • Published:
Information Systems Frontiers Aims and scope Submit manuscript

A Correction to this article was published on 06 June 2022

This article has been updated

Abstract

In traditional environment exploration algorithms, two problems are still waiting to be solved. One is that as the exploration time increases, the robot will repeatedly explore the areas that have been explored. The other is that in order to explore the environment more accurately, the robot will cause slight collisions during the exploration process. In order to solve the two problems, a DQN-based exploration model is proposed, which enables the robot to quickly find the unexplored area in an unknown environment, and designs a DQN-based navigation model to solve the local minima problem generated by the robot during the exploration. Through the switching mechanism of exploration model and navigation model, the robot can quickly complete the exploration task through selecting the modes according to the environment exploration situation. In the experiment results, the difference between the proposed unknown environment exploration method and the previous known-environment exploration methods research is less than 5% under the same exploration time. And in the proposed method, the robot can achieve zero collision and almost zero repeated exploration of the area when it has been trained for 30w rounds. Therefore, it can be seen that the proposed method is more practical than the previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Change history

References

  • Chaisittiporn R. (2018). Service boy robot path planner by deep q network. International Journal of Applied Computer Technology and Information Systems, 8(1), 50–55.

  • Civera, J., Davison, A. J., & Montiel, J. M. (2008). Inverse depth parametrization for monocular slam. IEEE Transactions on Robotics, 24(5), 932–945.

    Article  Google Scholar 

  • Cummins, M., & Newman, P. (2011). Appearance-only slam at large scale with fab-map 2.0. The International Journal of Robotics Research, 30(9), 1100–1123.

    Article  Google Scholar 

  • Dissanayake, M. G., et al. (2001). A solution to the simultaneous localization and map building (slam) problem. IEEE Transactions on Robotics and Automation, 17(3), 229–241.

    Article  Google Scholar 

  • Engel J., Schöps T., & Cremers D. (2014). Lsd-slam: Large-scale direct monocular slam. European conference on computer vision, 834–849.

  • Gálvez-López, D., & Tardos, J. D. (2012). Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics, 28(5), 1188–1197.

    Article  Google Scholar 

  • Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. Autonomous robot vehicles, 396–404.

  • Klein G. & Murray D. (2007). Parallel tracking and mapping for small ar workspaces. 6th IEEE and ACM international symposium on mixed and augmented reality, 225-234.

  • Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018). Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research, 37(4–5), 421–436.

    Article  Google Scholar 

  • Leofante, F., Ábrahám, E., Niemueller, T., et al. (2019). Integrated synthesis and execution of optimal plans for multi-robot systems in logistics. Information Systems Frontiers, 21, 87–107.

    Article  Google Scholar 

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Marchesini E. & Farinelli A. (2020). Genetic DRL for mapless navigation. Proceedings of the 19th international conference on autonomous agents and MultiAgent systems, 1919-1921.

  • Mnih, V., et al. (2015). Human-level control through DRL. Nature, 518(7540), 529–533.

    Article  Google Scholar 

  • Nister, D., & Stewenius, H. (2006). Scalable recognition with a vocabulary tree. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, 2161–2168.

    Google Scholar 

  • Pirita, I., & Katriina, H. (2021). Robot pets as “serious toys”- activating social and emotional experiences of elderly people. Information Systems Frontiers. https://doi.org/10.1007/s10796-021-10175-z

  • Rawin, C. (2018). Service boy robot path planner by deep Q network. International Journal of Applied Computer Technology and Information Systems, 8(1), 50–55.

    Google Scholar 

  • Rosten E. & Drummond T. (2006). Machine learning for high-speed corner detection. European conference on computer vision, 430-443.

  • Rublee E., Rabaud V., Konolige K., & Bradski G. (2011). Orb: An efficient alternative to sift or surf. International conference on computer vision, 2564-2571.

  • Sheshadri, C., Ranjan, C., & Demetris, V. (2021). Usage intention of social robots for domestic purpose: From security, privacy, and legal perspectives. Information Systems Frontiers. https://doi.org/10.1007/s10796-021-10197-7

  • Strasdat, H., Montiel, J. M., & Davison, A. J. (2012). Visual slam: Why filter? Image and Vision Computing, 30(2), 65–77.

    Article  Google Scholar 

  • Tai L., Paolo G., & Liu M. (2017). Virtual-to-real DRL: Continuous control of mobile robots for mapless navigation. IEEE/RSJ international conference on intelligent robots and systems, 31–36.

  • Volos, C. K., Kyprianidis, I. M., & Stouboulos, I. N. (2013). Experimental investigation on coverage performance of a chaotic autonomous mobile robot. Robotics and Autonomous Systems, 61(12), 1314–1322.

    Article  Google Scholar 

  • Xue X., Li Z., Zhang D., & Yan Y. (2019). A DRL method for mobile robot collision avoidance based on double dqn. IEEE 28th international symposium on industrial electronics, 2131-2136.

  • Zhang W. & Zhang Y. (2019). Behavior switch for drl-based robot navigation. IEEE 15th international conference on control and automation, 284-288.

Download references

Funding

This research work is partially supported by Taiwan Ministry of Science and Technology through grant 110–2511-H-006 -011 -MY3 and 109–2511-H-143 -003 -MY2.

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Taitung University, Taitung, Taiwan

    Shih-Yeh Chen

  2. Taiwan Semiconductor Manufacturing Company ltd., Hsinchu, Taiwan

    Qi-Fong He

  3. Department of Engineering Science, National Cheng Kung University, Tainan, Taiwan

    Chin-Feng Lai

Authors
  1. Shih-Yeh Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi-Fong He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chin-Feng Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shih-Yeh Chen.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflicts of interest to report regarding the present study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, SY., He, QF. & Lai, CF. Deep Reinforcement Learning-Based Robot Exploration for Constructing Map of Unknown Environment. Inf Syst Front 26, 63–74 (2024). https://doi.org/10.1007/s10796-021-10218-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10796-021-10218-5

Keywords

Search

Navigation