Skip to main content

Optimal path-planning for mobile robots to find a hidden target in an unknown environment based on machine learning


Using mobile robots in disaster areas can reduce risks and the search time in urban search and rescue operations. Optimal path-planning for mobile robotics can play a key role in the reduction of the search time for rescuing victims. In order to minimize the search time, the shortest path to the target should be determined. In this paper, a new integrated Reinforcement Learning—based method is proposed to search and find a hidden target in an unknown environment in the minimum time. The proposed algorithm is developed in two main phases. Depending on whether or not the mobile robot receives the signal from the hidden target, phases I or II of the proposed algorithm can be carried out. Then, the proposed algorithm is implemented on an e-puck robot in an urban environment which is simulated within Webots software. Finally, to demonstrate the efficiency of the proposed method and to verify it, the computational results from the proposed method are compared with three conventional methods from the literature.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16


  1. Casper J, Murphy RR (2003) Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Trans Syst Man Cybern Part B (Cybern) 33:367–385.

  2. Cormen TH, Stein C, Rivest RL, Leiserson CE (2001) Introduction to algorithms. McGraw-Hill Higher Education, New York

    MATH  Google Scholar 

  3. Faigl J, Macák J, Multi-Goal (2011) Path planning using self-organizing map with navigation functions. In: Proceedings of 19th European symposium on artificial neural networks, Bruges, Belgium, April 27–29

  4. Fakoor M, Kosari A, Jafarzadeh M (2015) Revision on fuzzy artificial potential field for humanoid robot path planning in unknown environment. Int J Adv Mechatron Syst 6:174–183.

    Article  Google Scholar 

  5. Felner A (2011) Position paper: Dijkstra’s algorithm versus uniform cost search or a case against Dijkstra’s algorithm. In: Fourth annual symposium on combinatorial search

  6. Geng N, Gong DW, Zhang Y (2014) PSO-based robot path planning for multisurvivor rescue in limited survival. Time Math Probl Eng 2014:10.

    Google Scholar 

  7. Jafarzadeh H, Gholami S, Bashirzadeh R (2014) A new effective algorithm for on-line robot motion planning. Decis Sci Lett 3:121–130

    Article  Google Scholar 

  8. Khanmohammadi S, Zarrin RS (2011) Intelligent path planning for rescue robot. In: Proceedings of world academy of science, engineering and technology. pp 764–769

  9. Lee L (2015) Robotic search & rescue via online multi-task reinforcement learning. arXiv:1511.08967

  10. Lodeiro-Santiago M, Santos-González I, Caballero-Gil P, Caballero-Gil C (2017) Secure system based on UAV and BLE for improving SAR missions. J Ambient Intell Humaniz Comput.

    Google Scholar 

  11. Majumder S, Kar S (2017) Multi-criteria shortest path for rough graph. J Ambient Intell Humaniz Comput.

    Google Scholar 

  12. Mehlhorn K, Sanders P (2008) Algorithms and data structures: the basic toolbox. Springer, Berlin, Heidelberg

    MATH  Google Scholar 

  13. Mondada F et al. (2009) The e-puck, a robot designed for education in engineering. In: Proceedings of the 9th conference on autonomous robot systems and competitions. vol LIS-CONF-2009-004. IPCB, Castelo Branco. pp 59–65

    Google Scholar 

  14. Pascarella D, Venticinque S, Aversa R, Mattei M, Blasi L (2015) Parallel and distributed computing for UAVs trajectory planning. J Ambient Intell Humaniz Comput 6:773–782.

    Article  Google Scholar 

  15. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Bradford Book. The MIT Press, Cambridge, MA

  16. Williams RL, Wu J (2010) Dynamic obstacle avoidance for an omnidirectional mobile robot. J Robot.

    Google Scholar 

  17. Yamauchi B (1998) Frontier-based exploration using multiple robots. In: Proceedings of the second international conference on Autonomous agents. ACM, pp 47–53

  18. Zeng W, Church RL (2009) Finding shortest paths on real road networks: the case for A*. Int J Geogr Inf Sci 23:531–543.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to S. Khodaygan.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sombolestan, S.M., Rasooli, A. & Khodaygan, S. Optimal path-planning for mobile robots to find a hidden target in an unknown environment based on machine learning. J Ambient Intell Human Comput 10, 1841–1850 (2019).

Download citation


  • Optimal path-planning
  • Machine learning
  • Reinforcement learning
  • Mobile robots