Advertisement

Humanoid odometric localization integrating kinematic, inertial and visual information

Abstract

We present a method for odometric localization of humanoid robots using standard sensing equipment, i.e., a monocular camera, an inertial measurement unit (IMU), joint encoders and foot pressure sensors. Data from all these sources are integrated using the prediction-correction paradigm of the Extended Kalman Filter. Position and orientation of the torso, defined as the representative body of the robot, are predicted through kinematic computations based on joint encoder readings; an asynchronous mechanism triggered by the pressure sensors is used to update the placement of the support foot. The correction step of the filter uses as measurements the torso orientation, provided by the IMU, and the head pose, reconstructed by a VSLAM algorithm. The proposed method is validated on the humanoid NAO through two sets of experiments: open-loop motions aimed at assessing the accuracy of localization with respect to a ground truth, and closed-loop motions where the humanoid pose estimates are used in real-time as feedback signals for trajectory control.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Notes

  1. 1.

    https://github.com/Oxford-PTAM/PTAM-GPL.

  2. 2.

    In principle, our localization method can be used on terrain with variable slopes. However, using the NAO built-in locomotion functions, relying on the flat floor assumption, it is only possible to allow very small variations in the slope that are not distinguishable from measurement noise. With sufficiently high slope values the robot falls down.

  3. 3.

    http://www.dis.uniroma1.it/~labrob/research/VisOdoLoc4Hum/software.

References

  1. Ahn, S., Yoon, S., Hyung, S., Kwak, N., & Roh, K. S. (2012). On-board odometry estimation for 3D vision-based SLAM of humanoid robot. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 4006–4012).

  2. Alcantarilla, P., Stasse, O., Druon, S., Bergasa, L., & Dellaert, F. (2013). How to localize humanoids with a single camera? Autonomous Robots, 34(1–2), 47–71.

  3. Chestnutt, J., Takaoka, Y., Suga, K., Nishiwaki, K., Kuffner, J., & Kagami, S. (2009). Biped navigation in rough environments using on-board sensing. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 3543–3548).

  4. Davison, A. J. (2003). Real-time simultaneous localisation and mapping with a single camera. In 9th International Conference on Computer Vision (pp. 1403–1410).

  5. Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007). MonoSLAM: Real-time single camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), 1052–1067.

  6. Garrido-Jurado, S., Muñoz Salinas, R., Madrid-Cuevas, F. J., & Marín-Jiménez, M. J. (2014). Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition, 47(6), 2280–2292.

  7. Hernandez, E., Ibarra, J. M., Neira, J., Cisneros, R., & Lavín, J. E. (2011). Visual SLAM with oriented landmarks and partial odometry. In 21st IEEE International Conference on Electrical Communications and Computers (pp. 39–45).

  8. Hornung, A., Wurm, K. M., & Bennewitz, M. (2010). Humanoid robot localization in complex indoor environments. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1690–1695).

  9. Hornung, A., Osswald, S., Maier, D., & Bennewitz, M. (2014). Monte carlo localization for humanoid robot navigation in complex indoor environments. International Journal of Humanoid Robotics, 11(02), 1441002.

  10. Ido, J., Shimizu, Y., Matsumoto, Y., & Ogasawara, T. (2009). Indoor navigation for a humanoid robot using a view sequence. International Journal of Robotics Research, 28(2), 315–325.

  11. Kelly, A. (2004). Fast and easy systematic and stochastic odometry calibration. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (Vol. 4, 3188–3194).

  12. Klein, G., & Murray, D. (2007). Parallel tracking and mapping for small AR workspaces. In 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (pp. 225–234).

  13. Kwak, N., Stasse, O., Foissotte, T., & Yokoi, K. (2009). 3D grid and particle based SLAM for a humanoid robot. In 2009 9th IEEE-RAS International Conference on Humanoid Robots (pp. 62–67).

  14. Mombaur, K., Truong, A., & Laumond, J.-P. (2010). From human to humanoid locomotion—An inverse optimal control approach. Autonomous Robots, 28, 369–383.

  15. Oriolo, G., Paolillo, A., Rosa, L., & Vendittelli, M. (2012) Vision-based odometric localization for humanoids using a kinematic EKF. In 2012 12th IEEE-RAS International Conference on Humanoid Robots (pp. 153–158).

  16. Oriolo, G., Paolillo, A., Rosa, L., & Vendittelli, M. (2013). Vision-based trajectory control for humanoid navigation. In 2013 13th IEEE-RAS International Conference on Humanoid Robots (pp. 118–123).

  17. Ozawa, R., Takaoka, Y., Kida, Y., Nishiwaki, K., Chestnutt, J., Kuffner, J., Kagami, J., Mizoguch, H., & Inoue, H. (2005). Using visual odometry to create 3D maps for online footstep planning. In 2005 IEEE International Conference on Systems, Man, and Cybernetics (Vol. 3, pp. 2643–2648).

  18. Pretto, A., Menegatti, E., Bennewitz, M., Burgard, W., & Pagello, E. (2009). A visual odometry framework robust to motion blur. In 2009 IEEE International Conference on Robotics and Automation (pp. 2250–2257).

  19. Samson, C. (1993). Time-varying feedback stabilization of car-like wheeled mobile robots. International Journal of Robotics Research, 12(1), 55–64.

  20. Scaramuzza, D., & Fraundorfer, F. (2011). Visual odometry part I: The first 30 years and fundamentals. IEEE Robotics & Automation Magazine, 18(4), 80–92.

  21. Stasse, O., Davison, A. C., Sellaouti, R., & Yokoi, K. (2006). Real-time 3D SLAM for a humanoid robot cosidering pattern generator information. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 348–355).

  22. Takaoka, Y., Kida, Y., Kagami, S., Mizoguchi, H., & Kanade, T. (2004). 3D map building for a humanoid robot by using visual odometry. In 2004 IEEE International Conference on Systems, Man, and Cybernetics (Vol. 5, pp. 4444–4449).

  23. Tellez, R., Ferro, F., Mora, D., Pinyol, D., & Faconti, D. (2008). Autonomous humanoid navigation using laser and odometry data. In 2008 8th IEEE-RAS International Conference on Humanoid Robots (pp. 500–506).

  24. Thompson, S., Kagami, S., & Nishiwaki, K. (2006). Localisation for autonomous humanoid navigation. In 2006 IEEE-RAS International Conference on Humanoid Robots (pp. 13–19).

  25. Truong, T.-V.-A., Flavigne, D., Pettre, J., Mombaur, K., & Laumond, J.-P. (2010). Reactive synthesizing of human locomotion combining nonholonomic and holonomic behaviors. In 3rd IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (pp. 632–637).

  26. Weiss, S., & Siegwart, R. (2011). Real-time metric state estimation for modular vision-inertial systems. In 2011 IEEE International Conference on Robotics and Automation (pp. 4531–4537).

  27. Weiss, S., Scaramuzza, D., & Siegwart, R. (2011). Monocular-SLAM-based navigation for autonomous micro helicopters in GPS-denied environments. Journal of Field Robotics, 28(6), 854–874.

Download references

Author information

Correspondence to Lorenzo Rosa.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 20648 KB)

Supplementary material 1 (mp4 20648 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Oriolo, G., Paolillo, A., Rosa, L. et al. Humanoid odometric localization integrating kinematic, inertial and visual information. Auton Robot 40, 867–879 (2016). https://doi.org/10.1007/s10514-015-9498-0

Download citation

Keywords

  • Humanoid robots
  • Localization
  • Odometry
  • Visual SLAM
  • EKF