Advertisement

Intelligent Service Robotics

, Volume 10, Issue 4, pp 297–312 | Cite as

A closed-loop approach for tracking a humanoid robot using particle filtering and depth data

  • Pablo Arturo MartínezEmail author
  • Xiao Lin
  • Mario Castelán
  • Josep Casas
  • Gustavo Arechavaleta
Original Research Paper

Abstract

Humanoid robots introduce instabilities during biped march that complicate the process of estimating their position and orientation along time. Tracking humanoid robots may be useful not only in typical applications such as navigation, but in tasks that require benchmarking the multiple processes that involve registering measures about the performance of the humanoid during walking. Small robots represent an additional challenge due to their size and mechanic limitations which may generate unstable swinging while walking. This paper presents a strategy for the active localization of a humanoid robot in environments that are monitored by external devices. The problem is faced using a particle filter method over depth images captured by an RGB-D sensor in order to effectively track the position and orientation of the robot during its march. The tracking stage is coupled with a locomotion system controlling the stepping of the robot toward a given oriented target. We present an integral communication framework between the tracking and the locomotion control of the robot based on the robot operating system, which is capable of achieving real-time locomotion tasks using a NAO humanoid robot.

Keywords

Humanoid robot Tracking RGB-D sensor Particle Filter ROS 

Notes

Acknowledgements

This work has been partially developed in the framework of the project TEC2013-43935-R, financed by the Spanish Ministerio de Economía y Competitividad and the European Regional Development Fund (ERDF). Also, the authors would like to thank Mexican Council of Science and Technology (CONACYT) for the PhD studentship of Pablo A. Martínez and the financial support for the sabbatical leave of Mario Castelán.

Supplementary material

11370_2017_230_MOESM1_ESM.mpg (36 mb)
Supplementary material 1 (mpg 36828 KB)

References

  1. 1.
    Alcantarilla P, Ni K, Bergasa L, Dellaert F (2011) Visibility learning in largescale urban environment. In: IEEE international conference on robotics and automation. Shanghai, ChinaGoogle Scholar
  2. 2.
    Alcantarilla PF, Stasse O, Druon S, Bergasa LM, Dellaert F (2013) How to localize humanoids with a single camera? Auton Robot 38(1–2):47–71CrossRefGoogle Scholar
  3. 3.
    Arechavaleta G, Laumond JP, Hicheur H, Berthoz A (2008) An optimality principle governing human walking. IEEE Trans Robot 24(1):5–14CrossRefGoogle Scholar
  4. 4.
    Castelán M, Arechavaleta G (2009) Approximating the reachable space of human walking paths: a low dimensional linear approach. In: IEEE international conference on humanoid robots. Paris, France, pp 81–86Google Scholar
  5. 5.
    Chestnutt J, Takaoka Y, Suga K, Nishiwaki K, Kuffner J, Kagami S (2009) Biped navigation in rough environments using on-board sensing. In: IEEE international conference on intelligent robots and systems. St. Louis, MO, USAGoogle Scholar
  6. 6.
    Clark R, Wang S, Wen H, Trigoni N, Markham A (2016) Increasing the efficiency of 6-DoF visual localization using multi-modal sensory data. In: 2016 IEEE-RAS 16th international conference on humanoid robots (humanoids). IEEE, pp 973–980Google Scholar
  7. 7.
    Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577Google Scholar
  8. 8.
    Comport AI, Marchand E, Pressigout M, Chaumette F (2006) Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans Vis Comput Graph 12(4):615–628CrossRefGoogle Scholar
  9. 9.
    Davison A, Reid ID, Molton ND, Stasse O (2007) Monoslam: real-time single camera slam. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067CrossRefGoogle Scholar
  10. 10.
    Escande A, Mansard N, Wieber PB (2014) Hierarchical quadratic programming: fat online humanoid-robot motion generation. Int J Robot Res 33(7):1006–1028CrossRefGoogle Scholar
  11. 11.
    Fallon MF, Antone M, Roy N, Teller S (2014) Drift-free humanoid state estimation fusing kinematic, inertial and lidar sensing. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 112–119Google Scholar
  12. 12.
    Ganapathi V, Plagemann C, Koller D, Thrun S (2010) Real time motion capture using a single time-of-flight camera. In: IEEE conference on computer vision and pattern recognition, CVPR, pp 755–762Google Scholar
  13. 13.
    Ganapathi V, Plagemann C, Koller D, Thrun S (2012) Real-time human pose tracking from range data. In: Fitzgibbon A, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Computer vision – ECCV 2012. Lecture Notes in Computer Science, vol 7577. Springer, Berlin, pp 738–751Google Scholar
  14. 14.
    Gordon NJ, Salmond DJ, Smith AF (1993) Novel approach to nonlinear/non-gaussian bayesian state estimation. In: IEE proceedings F (radar and signal processing), vol 140. IET, pp 107–113Google Scholar
  15. 15.
    Gratal X, Smith C, Björkman M, Kragic D (2013) Integrating 3D features and virtual visual servoing for hand-eye and humanoid robot pose estimation. In: IEEE/RAS international conference on humanoids robots, pp 240–245Google Scholar
  16. 16.
    Hansen DW, Hansen MS, Kirschmeyer M, Larsen R, Silvestre D (2008) Cluster tracking with time-of-flight cameras. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW’08, pp 1–6Google Scholar
  17. 17.
    Herdt A, Diedam H, Wieber PB, Dimitrov D, Mombaur K, Diehl M (2010) Online walking motion generation with automatic footstep placement. Adv Robot 24(5–6):719–737CrossRefGoogle Scholar
  18. 18.
    Hornung A, Wurm K, Bennewitz M (2010) Humanoid robot localization in complex indoor environments. In: IEEE/RSJ international conference on intelligent robots and systems. Taipei, Taiwan, pp 1690–1695Google Scholar
  19. 19.
    Klein G, Murray D (2007) Parallel tracking and mapping for small AR workspaces. In: Proceedings 6th IEEE and ACM international symposium on mixed and augmented reality (ISMAR’07). Nara, JapanGoogle Scholar
  20. 20.
    Maier D, Hornung A, Bennewitz M (2012) Real-time navigation in 3D environments based on depth camera data. In: IEEE international conference on humanoid robots. Osaka, Japan, pp 692–697Google Scholar
  21. 21.
    Martínez PA, Castelán M, Arechavaleta G (2016) Vision based persistent localization of a humanoid robot for locomotion tasks. Int J Appl Math Comput Sci 26(3):669MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Martínez PA, Varas D, Castelán M, Camacho M, Marqués F, Arechavaleta G (2014) 3D shape reconstruction from a humanoid generated video sequence. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 699–706Google Scholar
  23. 23.
    Matsumoto Y, Wada T, Nishio S, Miyashita T, Hagita N (2010) Scalable and robust multi-people head tracking by combining distributed multiple sensors. J Intell Serv Robot 3(1):29–36CrossRefGoogle Scholar
  24. 24.
    Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kagami S (2007) GPU-accelerated real-time 3D traking for humanoid locomotion and stair climbing. In: IEEE international conference on intelligent robots and systems. San Diego, USAGoogle Scholar
  25. 25.
    Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kanade T (2006) Online environment reconstruction for biped navigation. In: IEEE international conference on robotics and automation. Orlando, FL, USAGoogle Scholar
  26. 26.
    Migniot C, Ababsa F (2013) 3D human tracking in a top view using depth information recorded by the xtion pro-live camera. In: Advances in visual computing. Springer, Berlin, pp 603–612Google Scholar
  27. 27.
    Nummiaro K, Koller-Meier E, Van Gool L (2002) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110CrossRefzbMATHGoogle Scholar
  28. 28.
    Obwald S, Hornung A, Bennewitz M (2012) Improved proposals for highly accurate localization using range and vision data. In: IEEE/RSJ international conference on intelligent robots and system. Vilamoura, PortugalGoogle Scholar
  29. 29.
    Oriolo G, Paolillo A, Rosa L, Venditelli M (2016) Humanoid odometric localization integrating kinematic, inertial and visual information. Auton Robot 40(5):867–879. doi: 10.1007/s10514-015-9498-0
  30. 30.
    Papadopoulos AV, Bascetta L, Ferretti G (2013) Generation of human walking paths. In: IEEE/RSJ international conference on intelligent robots and systems. Tokyo, Japan, pp 1676–1681Google Scholar
  31. 31.
    Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Underst 108(1):4–18CrossRefGoogle Scholar
  32. 32.
    Puydupin-Jamin AS, Johnson M, Bretl T (2012) A convex approach to inverse optimal control and its application to modeling human locomotion. In: IEEE international conference on robotics and automation. Minnesota, USA, pp 531–536Google Scholar
  33. 33.
    Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: ICRA workshop on open source softwareGoogle Scholar
  34. 34.
    Rauter M (2013) Reliable human detection and tracking in top-view depth images. In: IEEE conference on computer vision and pattern recognition workshops, CVPRW, pp 529–534Google Scholar
  35. 35.
    Siddiqui M, Liao W, Medioni GG (2009) Vision-based short range interaction between a personal service robot and a user. Intell Serv Robot 2(3):113–130CrossRefGoogle Scholar
  36. 36.
    Simon D (2006) Optimal state estimation: Kalman, H infinity, and nonlinear approaches. Wiley, New YorkCrossRefGoogle Scholar
  37. 37.
    Sisbot EA, Marin-Urias LF, Alami R, Siméon T (2007) A human aware mobile robot motion planner. IEEE Trans Robot 23(5):874–883CrossRefGoogle Scholar
  38. 38.
    Sisbot EA, Marin-Urias LF, Broquère X, Sidobre D, Alami R (2010) Synthesizing robot motions adapted to human presence. Int J Soc Robot 2(3):329–343CrossRefGoogle Scholar
  39. 39.
    Spinello L, Arras KO (2011) People detection in RGB-D data. In: IEEE/RSJ international conference on intelligent robots and systems, pp 3838–3843Google Scholar
  40. 40.
    Stasse O, Davison A, Sellaouti R, Yokoi K (2006) Real-time 3D SLAM for a humanoid robot considering pattern generator information. In: IEEE/RSJ international conference on intelligent robots and systems. Beijing, China, pp 348–355Google Scholar
  41. 41.
    Stilman M, Nishiwaki K, Kagami S, Kuffner J (2006) Planning and executing navigation among movable obstacles. In: IEEE international conference on intelligent robots and systems. Beijin, ChinaGoogle Scholar
  42. 42.
    Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW (2000) Bundle adjustment – a modern synthesis. In: Triggs B., Zisserman A., Szeliski R (eds) Vision algorithms: theory and practice. IWVA 1999. Lecture Notes in Computer Science, vol 1883. Springer, Berlin, pp 298–372Google Scholar
  43. 43.
    Wurm KM, Hornung A, Bennewitz M, Stachniss C, Bur-gard W (2010) Octomap: a probabilistic, flexible, and compact 3D map representation for robotic systems. In: IEEE international conference on robotics and automation, workshop on best practice in 3D perception and modeling for mobile manipulation. Anchorage, AlaskaGoogle Scholar
  44. 44.
    Xia L, Chen CC, Aggarwal J (2011) Human detection using depth information by kinect. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW, pp 15–22Google Scholar
  45. 45.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Pablo Arturo Martínez
    • 1
    Email author
  • Xiao Lin
    • 2
  • Mario Castelán
    • 1
  • Josep Casas
    • 2
  • Gustavo Arechavaleta
    • 1
  1. 1.Robotics and Advanced Manufacturing GroupCentro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional (CINVESTAV), Unidad SaltilloRamos ArizpeMexico
  2. 2.Image Processing Group, Department of Signal Theory and CommunicationsUniversitat Politècnica de CatalunyaBarcelonaSpain

Personalised recommendations