Autonomous Robots

, Volume 39, Issue 1, pp 43–63 | Cite as

Simultaneous people tracking and robot localization in dynamic social spaces

  • Dylan F. Glas
  • Yoichi Morales
  • Takayuki Kanda
  • Hiroshi Ishiguro
  • Norihiro Hagita


Accurate robot localization and people tracking are necessary for deploying service robots in crowded everyday environments such as shopping malls, and features like product displays change over time, making map-based localization using on-board sensors difficult. We propose the use of an external sensor system to track people together with one or more robots. This approach is more robust to occlusions than on-board sensing and is unaffected by changing map features. In our system, laser range finders track people and robots in the environment, and odometry data is used to associate each robot with a tracked entity and correct the robot’s pose. Techniques are also presented for identifying and recovering from tracking errors. Simulation results show that our system can outperform localization using on-board sensors, both in tracking accuracy and in automatic recovery from errors. We demonstrate our system’s effectiveness in simulation, in a controlled experiment in a real shopping mall environment, and in real human–robot interactions with customers in a busy shopping arcade.


Robot localization Social robots Sensor systems  Dynamic environments 



Thanks to the staff of Universal CityWalk Osaka and Apita Town Keihanna for their cooperation in this research, and thanks to Dr. Satoshi Koizumi for his assistance in organizing the field trials. This research was supported in part by the Ministry of Internal Affairs and Communications of Japan, and in part by JST, CREST.


  1. Amarasinghe, D., Mann, G. K. I., & Gosine, R.G. (2008). Integrated laser-camera sensor for the detection and localization of landmarks for robotic applications. In IEEE International Conference on Robotics and Automation, 2008. ICRA 2008, 19–23 May 2008 (pp. 4012–4017). doi: 10.1109/robot.2008.4543827.
  2. Bennewitz, M., Burgard, W., Cielniak, G., & Thrun, S. (2005). Learning motion patterns of people for compliant robot motion. The International Journal of Robotics Research, 24(1), 31–48.CrossRefGoogle Scholar
  3. Bevilacqua, A., Di Stefano, L., & Azzari, P. (2006). People tracking using a time-of-flight depth sensor. In IEEE International Conference on Video and Signal Based Surveillance, 2006. AVSS ’06, Nov. 2006 (pp. 89–89). doi: 10.1109/avss.2006.92.
  4. Billard, A., Ijspeert, A. J., & Martinoli, A. (1999). A multi-robot system for adaptive exploration of a fast-changing environment: Probabilistic modeling and experimental study. Connection Science, 11(3–4), 359–379. doi: 10.1080/095400999116304.CrossRefGoogle Scholar
  5. Blackman, S. S. (2004). Multiple hypothesis tracking for multiple target tracking. Aerospace and Electronic Systems Magazine, IEEE, (Vol. 19, pp. 5–18). doi: 10.1109/MAES.2004.1263228
  6. Borrmann, D., Elseberg, J., Lingemann, K., Nüchter, A., & Hertzberg, J. (2008). Globally consistent 3D mapping with scan matching. Robotics and Autonomous Systems, 56(2), 130–142. doi: 10.1016/j.robot.2007.07.002.CrossRefGoogle Scholar
  7. Brščić, D., Kanda, T., Ikeda, T., & Miyashita, T. (2013). Person tracking in large public spaces using 3-D range sensors. IEEE Transactions on Human-Machine Systems, 43(6), 522–534. doi: 10.1109/thms.2013.2283945.CrossRefGoogle Scholar
  8. Burgard, W., Cremers, A. B., Fox, D., Hahnel, D., Lakemeyer, G., & Schulz, D., et al. (1998). The interactive museum tour-guide robot. In Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, Madison, Wisconsin, United States, (pp. 11–18).Google Scholar
  9. Burgard, W., Moors, M., Stachniss, C., & Schneider, F. E. (2005). Coordinated multi-robot exploration. IEEE Transactions on Robotics, 21(3), 376–386. doi: 10.1109/tro.2004.839232.CrossRefGoogle Scholar
  10. Cui, J., Zha, H., Zhao, H., & Shibasaki, R. (2007). Laser-based detection and tracking of multiple people in crowds. Computer Vision and Image Understanding, 106(2–3), 300–312. doi: 10.1016/j.cviu.2006.07.015.CrossRefGoogle Scholar
  11. Dissanayake, M. W. M. G., Newman, P., Clark, S., Durrant-Whyte, H. F., & Csorba, M. (2001). A solution to the simultaneous localization and map building (SLAM) problem. IEEE Transactions on Robotics and Automation, 17(3), 229–241. doi: 10.1109/70.938381.CrossRefGoogle Scholar
  12. Elfes, A. (1989). Using occupancy grids for mobile robot perception and navigation. Computer, 22(6), 46–57. doi: 10.1109/2.30720.CrossRefGoogle Scholar
  13. Fod, A., Howard, A., & Mataric, M.A.J. (2002). A laser-based people tracker. In Proceedings ICRA ’02. IEEE International Conference on Robotics and Automation, 2002, (Vol. 3, pp. 3024–3029). doi: 10.1109/robot.2002.1013691.
  14. Fox, D. (2003). Adapting the sample size in particle filters through KLD-sampling. The International Journal of Robotics Research, 22, 985–1004. doi: 10.1177/0278364903022012001.CrossRefGoogle Scholar
  15. Fox, D., Burgard, W., & Thrun, S. (1999). Markov localization for mobile robots in dynamic environments. Journal of Artificial Intelligence Research, 2, 391–327.Google Scholar
  16. Gerkey, B., Vaughan, R. T., & Howard, A. (2003). The player/stage project: Tools for multi-robot and distributed sensor systems. In 11th International Conference on Advanced Robotics (ICAR 2003), Coimbra, Portugal, June 2003 (pp. 317–323).Google Scholar
  17. Glas, D. F., Ferreri, F., Miyashita, T., Ishiguro, H., & Hagita, N. (2012a). Automatic calibration of laser range finder positions for pedestrian tracking based on social group detections. Advanced Robotics, 28(9), 573–588. doi: 10.1080/01691864.2013.879272.Google Scholar
  18. Glas, D. F., Kanda, T., Ishiguro, H., & Hagita, N. (2009a). Simultaneous people tracking and localization for social robots using external laser range finders. In Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, St. Louis, MO, USA, 10–15 Oct. 2009 (pp. 846–853). doi: 10.1109/iros.2009.5354198.
  19. Glas, D. F., Kanda, T., Ishiguro, H., & Hagita, N. (2012b). Teleoperation of multiple social robots. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 42(3), 530–544. doi: 10.1109/tsmca.2011.2164243.CrossRefGoogle Scholar
  20. Glas, D. F., Miyashita, T., Ishiguro, H., & Hagita, N. (2009b). Laser-based tracking of human position and orientation using parametric shape modeling. Advanced Robotics, 23(4), 405–428. doi: 10.1163/156855309x408754.CrossRefGoogle Scholar
  21. Glas, D. F., Miyashita, T., Ishiguro, H., & Hagita, N. (2010). Automatic position calibration and sensor displacement detection for networks of laser range finders for human tracking. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 18–22 Oct. 2010 (pp. 2938–2945). doi: 10.1109/iros.2010.5652272.
  22. Gross, H.-M., Boehme, H., Schroeter, C., Müller, S., Koenig, A., & Einhorn, E., et al. (2009). TOOMAS: interactive shopping guide robots in everyday use-final implementation and experiences from long-term field trials. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009. IROS 2009, (pp. 2005–2012): IEEE.Google Scholar
  23. Iwamura, Y., Shiomi, M., Kanda, T., Ishiguro, H., & Hagita, N. (2011). Do elderly people prefer a conversational humanoid as a shopping assistant partner in supermarkets? In Proceedings of the 6th International Conference on Human-Robot Interaction, Lausanne, Switzerland (pp. 449–456). doi: 10.1145/1957656.1957816.
  24. Kanda, T., Glas, D. F., Shiomi, M., & Hagita, N. (2009). Abstracting people’s trajectories for social robots to proactively approach customers. IEEE Transactions on Robotics, 25(6), 1382–1396. doi: 10.1109/tro.2009.2032969.CrossRefGoogle Scholar
  25. Kanda, T., Hirano, T., Eaton, D., & Ishiguro, H. (2004a). Interactive robots as social partners and peer tutors for children: A field trial. Human-Computer Interaction, 19(1), 61–84. doi: 10.1207/s15327051hci1901&2_4.
  26. Kanda, T., Ishiguro, H., Imai, M., & Ono, T. (2004). Development and evaluation of interactive humanoid robots. Proceedings of the IEEE, 92(11), 1839–1850. doi: 10.1109/jproc.2004.835359.CrossRefGoogle Scholar
  27. Kanda, T., Shiomi, M., Miyashita, Z., Ishiguro, H., & Hagita, N. (2010). A communication robot in a shopping mall. IEEE Transactions on Robotics, 26(5), 897–913. doi: 10.1109/tro.2010.2062550.CrossRefGoogle Scholar
  28. Köse, H., & Akın, H. L. (2007). The reverse monte carlo localization algorithm. Robotics and Autonomous Systems, 55(6), 480–489. doi: 10.1016/j.robot.2006.12.007.CrossRefGoogle Scholar
  29. Lee, J.-S., & Chung, W. K. (2010). Robust mobile robot localization in highly non-static environments. Autonomous Robots, 29(1), 1–16. doi: 10.1007/s10514-010-9184-1.CrossRefGoogle Scholar
  30. Montemerlo, M., Thrun, S., & Whittaker, W. (2002). Conditional particle filters for simultaneous mobile robot localization and people-tracking. In Proceedings ICRA ’02. IEEE International Conference on Robotics and Automation, 2002 (Vol. 1, pp. 695–701 vol. 691). doi: 10.1109/robot.2002.1013439.
  31. Moravec, H., & Elfes, A. (1985). High resolution maps from wide angle sonar. In Proceedings 1985 IEEE International Conference on Robotics and Automation, Mar 1985 (Vol. 2, pp. 116–121). doi: 10.1109/robot.1985.1087316.
  32. Mutlu, B., & Forlizzi, J. (2008). Robots in organizations: the role of workflow, social, and environmental factors in human-robot interaction. In Proceedings of the 3rd ACM/IEEE International Conference on Human robot interaction, Amsterdam, The Netherlands (pp. 287–294). doi: 10.1145/1349822.1349860.
  33. Ouellette, R., & Hirasawa, K. (2007). A comparison of SLAM implementations for indoor mobile robots. In Intelligent Robots and Systems, 2007. IROS 2007. IEEE/RSJ International Conference on, Oct. 29 2007-Nov. 2 2007 (pp. 1479–1484). doi: 10.1109/iros.2007.4399575.
  34. Pacchierotti, E., Christensen, H. I., & Jensfelt, P. (2006). Design of an office-guide robot for social interaction studies. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006, 9–15 Oct. 2006 (pp. 4965–4970). doi: 10.1109/iros.2006.282519.
  35. Park, S., Saegusa, R., & Hashimoto, S. (2007). Autonomous navigation of a mobile robot based on passive RFID. In Robot and Human interactive Communication, 2007. RO-MAN 2007. The 16th IEEE International Symposium on, 26–29 Aug. 2007 (pp. 218–223). doi: 10.1109/roman.2007.4415083.
  36. Pizarro, D., Marron, M., Peon, D., Mazo, M., Garcia, J. C., & Sotelo, M. A., et al. (2008). Robot and obstacles localization and tracking with an external camera ring. In IEEE International Conference on Robotics and Automation, 2008. ICRA 2008, 19–23 May 2008 (pp. 516–521). doi: 10.1109/robot.2008.4543259.
  37. Reid, D. B. (1979). An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control (Vol. 24, pp. 843–854). doi: 10.1109/TAC.1979.1102177
  38. Saffiotti, A., Broxvall, M., Gritti, M., LeBlanc, K., Lundh, R., & Rashid, J., et al. (2008). The PEIS-Ecology project: Vision and results. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. IROS 2008, 22–26 Sept. 2008 (pp. 2329–2335). doi: 10.1109/iros.2008.4650962.
  39. Saiki, L. Y. M., Satake, S., Kanda, T., & Hagita, N. (2011). Modeling environments from a route perspective. In Proceedings of the 6th international conference on Human-robot interaction, Lausanne, Switzerland (pp. 441–448). doi: 10.1145/1957656.1957815.
  40. Satake, S., Kanda, T., Glas, D. F., Imai, M., Ishiguro, H., & Hagita, N. (2010). How to approach humans?: Strategies for social robots to initiate interaction. Journal of the Robotics Society of Japan, 28(3), 327–337.CrossRefGoogle Scholar
  41. Schulz, D., Burgard, W., Fox, D., & Cremers, A. B. (2003). People tracking with mobile robots using sample-based joint probabilistic data association filters. The International Journal of Robotics Research, 22(2), 99–116. doi: 10.1177/0278364903022002002.CrossRefGoogle Scholar
  42. Schulz, D., Fox, D., & Hightower, J. (2003). People tracking with anonymous and ID-sensors using Rao-Blackwellised particle filters. In International Joint Conference on Artificial Intelligence (Vol. 18, pp. 921–928).Google Scholar
  43. Siegwart, R., Arras, K. O., Bouabdallah, S., Burnier, D., Froidevaux, G., Greppin, X., et al. (2003). Robox at Expo. 02: A large-scale installation of personal robots. Robotics and Autonomous Systems, 42(3), 203–222. doi: 10.1016/s0921-8890(02)00376-7.CrossRefzbMATHGoogle Scholar
  44. Sisbot, E. A., Alami, R., Simeon, T., Dautenhahn, K., Walters, M., & Woods, S. (2005). Navigation in the presence of humans. In 5th IEEE-RAS International Conference on Humanoid Robots, 2005, 5–5 Dec. 2005 (pp. 181–188). doi: 10.1109/ichr.2005.1573565.
  45. Takahashi, T. (2007). 2D localization of outdoor mobile robots using 3D laser range data. Master’s Thesis, Carnegie Mellon University.Google Scholar
  46. Thrun, S., Bücken, A., Burgard, W., Fox, D., Fröhlinghaus, T., Hennig, D., et al. (1998). Map learning and high-speed navigation in RHINO. In Artificial intelligence and mobile robots (pp. 21–52): MIT Press.Google Scholar
  47. Thrun, S., Fox, D., Burgard, W., & Dallaert, F. (2001). Robust Monte Carlo localization for mobile robots. Artificial Intelligence, 128(1–2), 99–141. doi: 10.1016/s0004-3702(01)00069-8.CrossRefzbMATHGoogle Scholar
  48. Urmson, C., Anhalt, J., Bagnell, D., Baker, C., Bittner, R., Clark, M., et al. (2008). Autonomous driving in urban environments: Boss and the urban challenge. Journal of Field Robotics, 25(8), 425–466.CrossRefGoogle Scholar
  49. Vu, T.-D., Aycard, O., & Appenrodt, N. (2007). Online localization and mapping with moving object tracking in dynamic outdoor environments. In IEEE Intelligent Vehicles Symposium, 2007, 13–15 June 2007 (pp. 190–195). doi: 10.1109/ivs.2007.4290113.
  50. Wang, C.-C., Thorpe, C., & Thrun, S. (2003). Online simultaneous localization and mapping with detection and tracking of moving objects: theory and results from a ground vehicle in crowded urban areas. In Proceedings ICRA ’03. IEEE International Conference on Robotics and Automation, 2003, 14–19 Sept. 2003 (Vol. 1, pp. 842–849 vol. 841). doi: 10.1109/robot.2003.1241698.
  51. Wolf, D. F., & Sukhatme, G. S. (2005). Mobile robot simultaneous localization and mapping in dynamic environments. Autonomous Robots, 19(1), 53–65. doi: 10.1007/s10514-005-0606-4.CrossRefGoogle Scholar
  52. Zhao, H., Chiba, M., Shibasaki, R., Shao, X., Cui, J., & Zha, H. (2008). SLAM in a dynamic large outdoor environment using a laser scanner. In IEEE International Conference on Robotics and Automation, 2008. ICRA 2008, 19–23 May 2008 (pp. 1455–1462). doi: 10.1109/robot.2008.4543407.

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Dylan F. Glas
    • 1
  • Yoichi Morales
    • 1
  • Takayuki Kanda
    • 1
  • Hiroshi Ishiguro
    • 2
  • Norihiro Hagita
    • 1
  1. 1.ATR-IRCKyotoJapan
  2. 2.IRLOsaka UniversityToyonakaJapan

Personalised recommendations