Direct Policy Search with Variable-Length Genetic Algorithm for Single Beacon Cooperative Path Planning

  • Tan Yew Teck
  • Mandar Chitre
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 104)


This paper focuses on Direct Policy Search (DPS) for cooperative path planning of a single beacon vehicle supporting Autonomous Underwater Vehicles (AUVs) performing surveying missions. Due to the lack of availability of GPS signals underwater, the position errors of the AUVs grow with time even though they are equipped with proprioceptive sensors for dead reckoning. One way to minimize this error is to have a moving beacon vehicle with good positioning data transmit its position acoustically from different locations to other AUVs. When the position is received, the AUVs can fuse this data with the range measured from the travel time of acoustic transmission to better estimate their own positions and keep the error bounded. In this work, we address the beacon vehicle’s path planning problem which takes into account the position errors being accumulated by the supported survey AUVs. We represent the path planning policy as state-action mapping and employ Variable-Length Genetic Algorithm (VLGA) to automatically discover the number of representative states and their corresponding action mapping. We show the resultant planned paths using the learned policy are able to keep the position errors of the survey AUVs bounded over the mission time.


Position Error Path Planning Markov Decision Process Dead Reckoning Path Planning Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kinsey, J.C., Eustice, R.M., Whitcomb, L.L.: A survey of underwater vehicle navigation: Recent advances and new challenges. In: IFAC Conference of Manoeuvering and Control of Marine Craft, Lisbon, Portugal (September 2006) (Invited paper)Google Scholar
  2. 2.
    Matos, A., Cruz, N., Martins, A., Lobo Pereira, F.: Development and implementation of a low-cost lbl navigation system for an auv. In: OCEANS 1999 MTS/IEEE. Riding the Crest into the 21st Century, vol. 2, pp. 774–779 (1999)Google Scholar
  3. 3.
    Rigby, P., Pizarro, O., Williams, S.: Towards geo-referenced auv navigation through fusion of usbl and dvl measurements. In: OCEANS 2006, pp. 1–6, 18–21 (2006)Google Scholar
  4. 4.
    Alcocer, A., Oliveira, P., Pascoal, A.: Study and implementation of an ekf gib-based underwater positioning system. Control Engineering Practice 15(6), 689–701 (2007)CrossRefGoogle Scholar
  5. 5.
    Rui, G., Chitre, M.: Cooperative positioning using range-only measurements between two AUVs. In: OCEANS 2010 IEEE - Sydney, pp. 1–6 (May 2010)Google Scholar
  6. 6.
    Bahr, A., Leonard, J.J., Fallon, M.F.: Cooperative localization for autonomous underwater vehicles. The International Journal of Robotics Research 28(6), 714–728 (2009)CrossRefGoogle Scholar
  7. 7.
    Alleyne, J.C.: Position estimation from range only measurements. Master’s thesis, Naval Postgraduate School, Monterey CA (September 2000)Google Scholar
  8. 8.
    Fallon, M.F., Papadopoulos, G., Leonard, J.J., Patrikalakis, N.M.: Cooperative AUV Navigation using a Single Maneuvering Surface Craft. The International Journal of Robotics Research 29(12), 1461–1474 (2010)CrossRefGoogle Scholar
  9. 9.
    Song, T.L.: Observability of target tracking with range-only measurements. IEEE Journal of Oceanic Engineering 24, 383–387 (1999)CrossRefGoogle Scholar
  10. 10.
    Hartsfiel, J.: Single transponder range only navigation geometry (strong) applied to remus autonomous under water vehicles. Master’s thesis. MIT (2005)Google Scholar
  11. 11.
    Gadre, A., Stilwell, D.: Toward underwater navigation based on range measurements from a single location. In: Proceedings of the 2004 IEEE International Conference on Robotics and Automation, ICRA 2004, April-May 1, vol. 5, pp. 4472–4477 (2004)Google Scholar
  12. 12.
    Forney, C., Manii, E., Farris, M., Moline, M., Lowe, C., Clark, C.: Tracking of a tagged leopard shark with an auv: Sensor calibration and state estimation. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 5315–5321 (May 2012)Google Scholar
  13. 13.
    Chitre, M.: Path planning for cooperative underwater range-only navigation using a single beacon. In: 2010 International Conference on Autonomous and Intelligent Systems (AIS), pp. 1–6 (June 2010)Google Scholar
  14. 14.
    Tan, Y.T., Chitre, M.: Single beacon cooperative path planning using cross-entropy method. In: IEEE/MTS OCEANS, Kona, Hawaii (September 2011)Google Scholar
  15. 15.
    Powell, W.B.: Approximate Dynamic Programming: Solving the Curses of Dimensionality, 2nd edn. Wiley Series in Probability and Statistics. Wiley (2011)Google Scholar
  16. 16.
    Tu, J., Yang, S.: Genetic algorithm based path planning for a mobile robot. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA 2003, vol. 1, pp. 1221–1226 (September 2003)Google Scholar
  17. 17.
    Ahn, C.W., Ramakrishna, R.: A genetic algorithm for shortest path routing problem and the sizing of populations. IEEE Transactions on Evolutionary Computation 6, 566–579 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.ARL, Tropical Marine Science Institute and, Department of Electrical and Computer EngineeringNational University of SingaporeSingaporeSingapore

Personalised recommendations