Decision-Theoretic Planning with Person Trajectory Prediction for Social Navigation

  • Ignacio Pérez-Hurtado
  • Jesús Capitán
  • Fernando Caballero
  • Luis Merino
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 418)


Robots navigating in a social way should reason about people intentions when acting. For instance, in applications like robot guidance or meeting with a person, the robot has to consider the goals of the people. Intentions are inherently non-observable, and thus we propose Partially Observable Markov Decision Processes (POMDPs) as a decision-making tool for these applications. One of the issues with POMDPs is that the prediction models are usually handcrafted. In this paper, we use machine learning techniques to build prediction models from observations. A novel technique is employed to discover points of interest (goals) in the environment, and a variant of Growing Hidden Markov Models (GHMMs) is used to learn the transition probabilities of the POMDP. The approach is applied to an autonomous telepresence robot.


Markov decision processes Social robot navigation GHMM 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arras, K.O., Mozos, O.M., Burgard, W.: Using boosted features for the detection of people in 2D range data. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 3402–3407. Roma, April 2007Google Scholar
  2. 2.
    Bai, H., Cai, S., Ye, N., Hsu, D., Lee, W.S.: Intention-aware online POMDP planning for autonomous driving in a crowd. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 454–460, May 2015Google Scholar
  3. 3.
    Foka, A., Trahanias, P.: Probabilistic autonomous robot navigation in dynamic environments with human motion prediction. International Journal of Social Robotics 2(1), 79–94 (2010). CrossRefGoogle Scholar
  4. 4.
    Jocksuch, J., Ritter, H.: An instantaneous topological map for correlated stimuli. In: Proceedings of the International Joint Conference on Neural Networks, vol. 1, pp. 529–534. Washington (1999)Google Scholar
  5. 5.
    Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artificial Intelligence 101, 99–134 (1998)CrossRefMathSciNetzbMATHGoogle Scholar
  6. 6.
    Merino, L., Ballesteros, J., Pérez-Higueras, N., Ramón-Vigo, R., Pérez-Lara, J., Caballero, F.: Robust person guidance by using online POMDPs. In: Armada, M.A., Sanfeliu, A., Ferre, M. (eds.) ROBOT 2013: First Iberian Robotics Con-ference, Advances in Intelligent Systems and Computing, vol. 253, pp. 289–303. Springer International Publishing (2014)Google Scholar
  7. 7.
    Neal, R., Hinton, G.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M. (ed.) Learning in Graphical Models (1998)Google Scholar
  8. 8.
    Ong, S.C., Png, S.W., Hsu, D., Lee, W.S.: POMDPs for robotic tasks with mixed observability. In: Proceedings of the Robotics: Science and Systems Conference. Seattle (2009)Google Scholar
  9. 9.
    Perez-Hurtado, I., Capitan, J., Caballero, F., Merino, L.: An extension of GHMMs for environments with occlusions and automatic goal discovery for person trajectory prediction. In: European Conference on Mobile Robots. p. To appear. Lincoln (2015)Google Scholar
  10. 10.
    Poupart, P.: Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes. Ph.D. thesis, University of Toronto (2005)Google Scholar
  11. 11.
    Shiarlis, K., Messias, J., van Someren, M., Whiteson, S., Kim, J., Vroon, J., Englebienne, G., Truong, K., Evers, V., Perez-Higueras, N., Perez-Hurtado, I., Ramon-Vigo, R., Caballero, F., Merino, L., Shen, J., Petridis, S., Pantic, M., Hedman, L., Scherlund, M., Koster, R., Michel, H.: Teresa: a socially intelligent semi-autonomous telepresence system. In: Workshop on Machine Learning for Social Robotics at ICRA-2015. Seattle (2015)Google Scholar
  12. 12.
    Silver, D., Veness, J.: Monte-Carlo planning in large POMDPs. In: Advances in Neural Information Processing Systems (2010)Google Scholar
  13. 13.
    Somani, A., Ye, N., Hsu, D., Lee, W.S.: DESPOT: Online POMDP planning with regularization. In: Burges, C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, pp. 1772–1780. Curran Associates, Inc. (2013)Google Scholar
  14. 14.
    Tipaldi, G.D., Arras, K.O.: Planning problems for social robots. In: Proc. International Conference on Automated Planning and Scheduling (ICAPS 2011). Freiburgu (2011)Google Scholar
  15. 15.
    Tsui, K., Desai, M., Yanco, H., Uhlik, C.: Exploring use cases for telepresence robots. In: 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 11–18, March 2011Google Scholar
  16. 16.
    Vasquez, D., Fraichard, T., Laugier, C.: Growing Hidden Markov Models: An Incremental Tool for Learning and Predicting Human and Vehicle Motion. The International Journal of Robotics Research 28(11–12), 1486–1506 (2009)CrossRefGoogle Scholar
  17. 17.
    Xie, D., Todorovic, S., Zhu, S.: Inferring "dark matter" and "dark energy" from videos. In: IEEE International Conference on Computer Vision (ICCV), pp. 2224–2231 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Ignacio Pérez-Hurtado
    • 1
  • Jesús Capitán
    • 2
  • Fernando Caballero
    • 2
  • Luis Merino
    • 1
  1. 1.Pablo de Olavide University SevilleSevilleSpain
  2. 2.University of SevilleSevilleSpain

Personalised recommendations