Advertisement

Sensor Fusion and State Estimation of the Robot

  • Francesco Nori
  • Silvio Traversaro
  • Maurice Fallon
Reference work entry

Abstract

In this chapter, we review previous estimation and sensor fusion approaches in the field of humanoid robotics. The focus is primarily but not exclusively on state estimation. Humanoids are modelled as free-floating mechanical systems subject to external forces and constrained by whole-body distributed rigid contacts. Besides state estimation, a number of related problems are considered including the estimation of joint torques, contact forces, contact positions, and the center of mass. Previous approaches are discussed with reference to a number of features relevant to humanoid applications: computational complexity, observability analysis, modeling accuracy, and floating-base parameterization. This chapter concludes with a section on future directions and open problems, revisited in the light of previously discussed approaches.

Notes

Acknowledgements

This chapter was supported by the FP7 EU projects CoDyCo (No. 600716 ICT 2011.2.1 Cognitive Systems and Robotics) and Koroibot (No. 611909 ICT-2013.2.1 Cognitive Systems and Robotics).

References

  1. 1.
    P.F. Alcantarilla, O. Stasse, S. Druon, L.M. Bergasa, F. Dellaert, How to localize humanoids with a single camera? Auton. Robot. 34(1), 47–71 (2013). ISSN:1573–7527CrossRefGoogle Scholar
  2. 2.
    Y. Aoustin, F. Plestan, V. Lebastard, Experimental comparison of several posture estimation solutions for biped robot Rabbit, in IEEE International Conference on Robotics and Automation. ICRA 2008, May 2008, pp. 1270–1275. doi:  https://doi.org/10.1109/ROBOT.2008.4543378
  3. 3.
    M. Benallegue, F. Lamiraux, Humanoid flexibility deformation can be efficiently estimated using only inertial measurement units and contact information, in 2014 IEEE-RAS International Conference on Humanoid Robots, Nov 2014, pp. 246–251. doi:  https://doi.org/10.1109/HUMANOIDS.2014.7041367
  4. 4.
    M. Bloesch, M. Hutter, M.H. Hoepflinger, C.D. Remy, C. Gehring, R. Siegwart, State estimation for legged robots – consistent fusion of leg kinematics and IMU, in Robotics: Science and Systems VIII, 2012. doi:  https://doi.org/10.15607/RSS.2012.VIII.003
  5. 5.
    S. Chitta, P. Vemaza, R. Geykhman, D.D. Lee, Proprioceptive localization for a quadrupedal robot on known terrain, in Proceedings 2007 IEEE International Conference on Robotics and Automation, Apr 2007, pp. 4582–4587. doi:  https://doi.org/10.1109/ROBOT.2007.364185
  6. 6.
    C. Ciliberto, L. Fiorio, M. Maggiali, L. Natale, L. Rosasco, G. Metta, G. Sandini, F. Nori, Exploiting global force torque measurements for local compliance estimation in tactile arrays, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sept 2014, pp. 3994–3999. doi:  https://doi.org/10.1109/IROS.2014.6943124
  7. 7.
    A. De Luca, A. Albu-Schaffer, S. Haddadin, G. Hirzinger, Collision detection and safe reaction with the DLR-III lightweight manipulator arm, in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct 2006, pp. 1623–1630. doi:  https://doi.org/10.1109/IROS.2006.282053
  8. 8.
    A. Del Prete, S. Denei, L. Natale, F. Mastrogiovanni, F. Nori, G. Cannata, G. Metta, Skin spatial calibration using force/torque measurements, in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2011), pp. 3694–3700Google Scholar
  9. 9.
    A. Del Prete, L. Natale, F. Nori, G. Metta, Contact force estimations using tactile sensors and force/torque sensors. Human Robot Interaction 2012, 0–2 (2012)Google Scholar
  10. 10.
    A. Del Prete, N. Mansard, O.E. Ramos, O. Stasse, F. Nori, Implementing torque control with high-ratio gear boxes and without joint-torque sensors. Int. J. Humanoid Rob. 13(01), 1550044 (2016)Google Scholar
  11. 11.
    M. Fallon, M. Antone, N. Roy, S. Teller, Drift-free humanoid state estimation fusing kinematic, inertial and LIDAR sensing, in 2014 IEEE-RAS International Conference on Humanoid Robots, Nov 2014, pp. 112–119. doi:  https://doi.org/10.1109/HUMANOIDS.2014.7041346
  12. 12.
    M.F. Fallon, P. Marion, R. Deits, T. Whelan, M. Antone, J. McDonald, R. Tedrake, Continuous humanoid locomotion over uneven terrain using stereo fusion, in 2015 15th IEEE-RAS International Conference on Humanoid Robots (Humanoids), Seoul, Nov 2015Google Scholar
  13. 13.
    R. Featherstone, Rigid Body Dynamics Algorithms (Springer, New York, 2008)CrossRefGoogle Scholar
  14. 14.
    M. Fumagalli, S. Ivaldi, M. Randazzo, L. Natale, G. Metta, G. Sandini, F. Nori, Force feedback exploiting tactile and proximal force/torque sensing. Theory and implementation on the humanoid robot iCub. Auton. Robot. 33(4), 381–398 (2012)CrossRefGoogle Scholar
  15. 15.
    D. Gonzalez-Aguirre, M. Vollert, T. Asfour, R. Dillmann, Robust real-time 6d active visual localization for humanoid robots, in 2014 IEEE International Conference on Robotics and Automation (ICRA), May 2014, pp. 2785–2791. doi:  https://doi.org/10.1109/ICRA.2014.6907258
  16. 16.
    A. Hornung, K.M. Wurm, M. Bennewitz, Humanoid robot localization in complex indoor environments, in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Oct 2010Google Scholar
  17. 17.
    T. Koolen, S. Bertrand, G. Thomas, T. de Boer, T. Wu, J. Smith, J. Englsberger, J. Pratt, Design of a momentum-based control framework and application to the humanoid robot Atlas. Int. J. Humanoid Rob. 13, 1650007 (2016)CrossRefGoogle Scholar
  18. 18.
    E. Krotkov, D. Hackett, L. Jackel, M. Perschbacher, J. Pippine, J. Strauss, G. Pratt, C. Orlowski, The DARPA robotics challenge finals: results and perspectives. J. Field Rob. 34(2), 229–240 (2017). ISSN:1556-4967. doi:  https://doi.org/10.1002/rob.21683CrossRefGoogle Scholar
  19. 19.
    S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Permenter, T. Koolen, P. Marion, R. Tedrake, Optimization-based locomotion planning, estimation, and control design for Atlas. Auton. Robot. 40, 429–455 (2016)Google Scholar
  20. 20.
    K. Lowrey, S. Kolev, Y. Tassa, T. Erez, E. Todorov, Physically-consistent sensor fusion in contact-rich behaviors, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sept 2014, pp. 1656–1662. doi:  https://doi.org/10.1109/IROS.2014.6942777
  21. 21.
    E. Magrini, F. Flacco, A. De Luca, Estimation of contact forces using a virtual force sensor, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sept 2014, pp. 2126–2133. doi:  https://doi.org/10.1109/IROS.2014.6942848
  22. 22.
    L. Manuelli, R. Tedrake, Localizing external contact using proprioceptive sensors: the contact particle filter, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 9–14 Oct 2016. doi:  https://doi.org/10.1109/IROS.2016.7759743
  23. 23.
    M. Mistry, J. Buchli, S. Schaal, Inverse dynamics control of floating base systems using orthogonal decomposition, in 2010 IEEE International Conference on Robotics and Automation (ICRA), May 2010, pp. 3406–3412. doi:  https://doi.org/10.1109/ROBOT.2010.5509646
  24. 24.
    R.M. Murray, Z. Li, S.S. Sastry, S.S. Sastry, A Mathematical Introduction to Robotic Manipulation (CRC Press, Boca Raton, 1994)Google Scholar
  25. 25.
    K. Nishiwaki, J. Chestnutt, S. Kagami, Autonomous navigation of a humanoid robot over unknown rough terrain using a laser range sensor. Int. J. Robot. Res. 31, 1251–1262 (2012)CrossRefGoogle Scholar
  26. 26.
    F. Nori, Inverse, forward and other dynamic computations computationally optimized with sparse matrix factorizations. CoRR, abs/1705.04658, 2017. http://arxiv.org/abs/1705.04658
  27. 27.
    F. Nori, N. Kuppuswamy, S. Traversaro, Simultaneous state and dynamics estimation in articulated structures, in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sept 2015, pp. 3380–3386. doi:  https://doi.org/10.1109/IROS.2015.7353848
  28. 28.
    F. Nori, S. Traversaro, J. Eljaik, F. Romano, A.D. Prete, D. Pucci, iCub whole-body control through force regulation on rigid noncoplanar contacts. Front. Robot. AI 2(6) (2015) ISSN:2296-9144. doi:  https://doi.org/10.3389/frobt.2015.00006. http://www.frontiersin.org/humanoid_robotics/10.3389/frobt.2015.00006/abstract
  29. 29.
    G. Oriolo, A. Paolillo, L. Rosa, M. Vendittelli, Humanoid odometric localization integrating kinematic, inertial and visual information. Auton. Robot. 40(5), 867–879 (2016). ISSN:1573-7527. doi: 10.1007/s10514-015-9498-0CrossRefGoogle Scholar
  30. 30.
    Y.J. Park, W.K. Chung, External torque sensing algorithm for flexible-joint robot based on disturbance observer structure, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sept 2014, pp. 4735–4741. doi:  https://doi.org/10.1109/IROS.2014.6943236
  31. 31.
    J. Park, O. Khatib, Multi-link multi-contact force control for manipulators, in Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Apr 2005, pp. 3613–3618. doi:  https://doi.org/10.1109/ROBOT.2005.1570670
  32. 32.
    A. Petrovskaya, J. Park, O. Khatib, Probabilistic estimation of whole body contacts for multi-contact robot control, in Proceedings 2007 IEEE International Conference on Robotics and Automation, Apr 2007, pp. 568–573. doi:  https://doi.org/10.1109/ROBOT.2007.363047
  33. 33.
    L. Pongsak, M. Okada, Y. Nakamura, Optimal filtering for humanoid robot state estimators, in Proceedings of SICE System Integration Division Annual Conference (SI2002), 2002Google Scholar
  34. 34.
    N. Rotella, M. Bloesch, L. Righetti, S. Schaal, State estimation for a humanoid robot, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2014), pp. 952–958Google Scholar
  35. 35.
    N. Rotella, S. Mason, S. Schaal, L. Righetti, Inertial sensor-based humanoid joint state estimation, in 2016 IEEE International Conference on Robotics and Automation (ICRA), May 2016, pp. 1825–1831. doi:  https://doi.org/10.1109/ICRA.2016.7487328
  36. 36.
    B. Siciliano, L. Sciavicco, Robotics: Modelling, Planning and Control (Springer, London, 2009)Google Scholar
  37. 37.
    V. Sotoudehnejad, A. Takhmar, M.R. Kermani, I.G. Polushin, Counteracting modeling errors for sensitive observer-based manipulator collision detection, in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct 2012, pp. 4315–4320. doi:  https://doi.org/10.1109/IROS.2012.6386198
  38. 38.
    O. Stasse, A.J. Davison, R. Sellaouti, K. Yokoi, Real-time 3D SLAM for a humanoid robot considering pattern generator information, in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006Google Scholar
  39. 39.
    B.J. Stephens, State estimation for force-controlled humanoid balance using simple models in the presence of modeling error, in 2011 IEEE International Conference on Robotics and Automation (ICRA), 2011, pp. 3994–3999Google Scholar
  40. 40.
    S. Thompson, S. Kagami, Humanoid robot localisation using stereo vision, in 5th IEEE-RAS International Conference on Humanoid Robots, Dec 2005, pp. 19–25. doi:  https://doi.org/10.1109/ICHR.2005.1573539
  41. 41.
    L.L. Tien, A. Albu-Schäffer, Friction observer and compensation for control of robots with joint torque measurement, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2008, pp. 22–26Google Scholar
  42. 42.
    S. Traversaro, A. Saccon, Multibody dynamics notation. Technical report, Department of Mechanical Engineering, Eindhoven University of Technology, 2016. http://repository.tue.nl/849895. Report locator DC 2016.064
  43. 43.
    S. Traversaro, A.D. Prete, S. Ivaldi, F. Nori, Inertial parameters identification and joint torques estimation with proximal force/torque sensing, in 2015 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2015), pp. 2105–2110Google Scholar
  44. 44.
    J. Vihonen, J. Honkakorpi, J. Mattila, A. Visa, Geometry-aided angular acceleration sensing of rigid multi-body manipulator using MEMS rate gyros and linear accelerometers, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013Google Scholar
  45. 45.
    T. Wu, Y. Tassa, V. Kumar, J. Movellan, E. Todorov, Stac: simultaneous tracking and calibration, in 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), Oct 2013, pp. 469–476. doi:  https://doi.org/10.1109/HUMANOIDS.2013.7030016
  46. 46.
    B. Xinjilefu, C.G. Atkeson, State estimation of a walking humanoid robot, in IROS (IEEE, 2012), pp. 3693–3699Google Scholar
  47. 47.
    B. Xinjilefu, S. Feng, W. Huang, C. Atkeson, Decoupled state estimation for humanoids using full-body dynamics, in IEEE/RAS 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014Google Scholar
  48. 48.
    B. Xinjilefu, F. Siyuan, C.G. Atkeson, Dynamic state estimation using quadratic programming, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2014), pp. 989–994Google Scholar
  49. 49.
    B. Xinjilefu, S. Feng, C. Atkeson, Center of mass estimator for humanoids and its application in modelling error compensation, fall detection and prevention, in International Conference on Humanoid Robotics, Seoul, Nov 2015Google Scholar
  50. 50.
    B. Xinjilefu, S. Feng, C.G. Atkeson, A distributed mems gyro network for joint velocity estimation, in 2016 IEEE International Conference on Robotics and Automation (ICRA), May 2016, pp. 1879–1884. doi:  https://doi.org/10.1109/ICRA.2016.7487334
  51. 51.
    S. Yoon, S. Hyung, M. Lee, K.S. Roh, S.H. Ahn, A. Gee, P. Bunnun, A. Calway, W.W. Mayol-Cuevas, Real-time 3D simultaneous localization and map-building for a dynamic walking humanoid robot. Auton. Robot. 27(10), 759–772 (2013)Google Scholar
  52. 52.
    H. Zhang, S. Ahmad, G. Liu, Torque estimation for robotic joint with harmonic drive transmission based on position measurements. IEEE Trans. Robot. 31(2), 322–330 (2015). ISSN:1552-3098. doi:  https://doi.org/10.1109/TRO.2015.2402511CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  • Francesco Nori
    • 1
  • Silvio Traversaro
    • 1
  • Maurice Fallon
    • 2
    • 3
  1. 1.iCub Facility, Robotics, Brain and Cognitive Sciences DepartmentIstituto Italiano di TecnologiaGenoaItaly
  2. 2.Oxford Robotics InstituteUniversity of OxfordOxfordUK
  3. 3.Robot Perception GroupUniversity of EdinburghEdinburghUK

Personalised recommendations