Advertisement

Manipulation and Task Execution by Humanoids

  • Kensuke Harada
  • Máximo A. Roa
Reference work entry

Abstract

This chapter is focused on planning and execution of tasks, most notably manipulation, with humanoid robots. The chapter opens with a discussion on different levels of planning: high-level planning for task sequencing and prioritization, medium level for tasks that involve whole-body motions, and low-level planning for grasp and manipulation tasks. The following section discusses in more detail the manipulation tasks, including grasp and motion planning for single- and dual-arm manipulations. Next, the effective exploitation of the multiple degrees of freedom available in a humanoid robot allows the execution of tasks that involve whole-body motions. Finally, higher level plans that define sequences of actions to be executed by the robots are considered, including methods to deal with natural language input for commanding robot actions.

References

  1. 1.
    A. Ajoudani, S.B. Godfrey, M. Catalano, G. Grioli, N.G. Tsagarakis, A. Bicchi, Teleimpedance control of a synergy-driven anthropomorphic hand, in Proceedings of the IEEE International Conference on Intelligent Robots and Systems, 2013, pp. 1985–1991Google Scholar
  2. 2.
    H. Arisumi, J.R. Chardonne, K. Yokoi, Whole body motion of a Humanoid robot for passing through a door – opening a door by impulsive force, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009, pp. 428–434Google Scholar
  3. 3.
    T. Asfour, P. Azad, F. Gyarfas, R. Dillmann, Imitation learning of dual-arm manipulation tasks in humanoid robots. Int. J. Humanoid Robot. 5(2), 183–202 (2008)CrossRefGoogle Scholar
  4. 4.
    N. Banerjee, X. Long, R. Du, F. Polido, S. Feng, C.G. Atkeson, M. Gennert, T. Padir, Human-supervised control of the ATLAS humanoid robot for traversing doors, in Proceedings of the IEEE-RAS International Conference Humanoid Robots, 2015, pp. 722–729Google Scholar
  5. 5.
    D. Berenson, R. Diankov, K. Nishikawi, S. Kagami, J. Kuffner, Grasp planning in complex scenes, in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2007, pp. 42–48Google Scholar
  6. 6.
    D. Berenson, S. Srinivasa, Grasp synthesis in cluttered environments for dexterous hands, Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2008, pp. 189–196Google Scholar
  7. 7.
    D. Berenson, S. Srinivasa, J. Kuffner, Task space regions: a framework for pose-constrained manipulation planning. Int. J. Robot. Res. 30(12), 1435–1460 (2011)CrossRefGoogle Scholar
  8. 8.
    J. Bohg, A. Morales, T. Asfour, D. Kragic, Data-driven grasp synthesis – a survey. IEEE Trans. Robot. 30(2), 289–309 (2014)CrossRefGoogle Scholar
  9. 9.
    J. Borras, T. Asfour, A whole-body pose taxonomy for loco-manipulation tasks, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2015, pp. 1578–1585Google Scholar
  10. 10.
    K. Bouyarmane, A. Kheddar, Humanoid robot locomotion and manipulation step planning. Adv. Robot. 26(10), 1099–1126 (2012)CrossRefGoogle Scholar
  11. 11.
    S. Calinon, A. Billard, Stochastic gesture production and recognition model for a humanoid robot, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005, pp. 2769–2774Google Scholar
  12. 12.
    R. Deimel, O. Brock, A novel type of compliant and underactuated robotic hand for dexterous grasping. Int. J. Robot. Res. 35(1–3), 161–185 (2016)Google Scholar
  13. 13.
    S. Cambon, R. Alami, F. Gravot, A hybrid approach to intricate motion, manipulation and task planning. Int. J. Robot. Res. 28(1), 104–126 (2009)CrossRefGoogle Scholar
  14. 14.
    S. Caron, Q.-C. Pham, Y. Nakamura, Stability of surface contacts for humanoid robots: closed-form formulae of the contact wrench cone for rectangular support areas, in Proceedings of the IEEE International Conference on Robotics and Automation, 2015, pp. 5107–5112Google Scholar
  15. 15.
    S. Chitta, E.G. Jones, M. Ciocarlie, K. Hsiao, Mobile manipulation in unstructured environments: perception, planning, and execution. Robot. Autom. Mag. 19(2), 58–71, (2011)CrossRefGoogle Scholar
  16. 16.
    M. Cognetti, P. Mohammadi, G. Oriolo, M. Vendittelli, Task-oriented whole-body planning for humanoids based on hybrid motion generation, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014, pp. 4071–4076Google Scholar
  17. 17.
    B.A. Dang-Vu, O. Porges, M.A. Roa, Interpreting manipulation actions: from language to execution, in Robot 2015: Second Iberian Robotics Conference, Series Advances in Intelligent Systems and Computing, vol. 417 (Springer, 2015), pp. 175–187Google Scholar
  18. 18.
    A. Dietrich, T. Wimboeck, A. Albu-Schaeffer, G. Hirzinger, Reactive whole-body control: dynamic mobile manipulation using a large number of actuated degrees of freedom. IEEE Robot. Autom. Mag. 19(2), 20–33 (2012)CrossRefGoogle Scholar
  19. 19.
    M.R. Dogar, S. Srinivasa, A framework for push-grasping in clutter, Proceedings of Robotics: Science and Systems, RSS, 2011Google Scholar
  20. 20.
    M.R. Dogar, A. Spielberg, S. Baker, D. Rus, Multi-robot grasp planning for sequential assembly operations, in Proceedings of the IEEE International Conference on Robotics and Automation, 2015, pp. 193–200Google Scholar
  21. 21.
    C. Eppner, O. Brock, Planning grasp strategies that exploit environmental constraints, in Proceedings of the IEEE International Conference on Robotics and Automation, 2015, pp. 4947–4952Google Scholar
  22. 22.
    A. Escande, N. Mansard, P.B. Weiber, Hierachical quadratic programming: fast online humanoid-robot motion generation. Int. J. Robot. Res. 33(7), 1006–1028 (2014)CrossRefGoogle Scholar
  23. 23.
    J. Fasola, M.J. Mataric, Using semantic fields to model dynamic spatial relations in a robot architecture for natural language instruction of service robots, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 143–150Google Scholar
  24. 24.
    M. Gonzalez-Fierro, D. Hernandez-Garcia, T. Nanayakkara, C. Balaguer, Behavior sequencing based on demonstrations: a case of a humanoid opening a door while walking. Adv. Robot. 29(5), 315–329 (2015)CrossRefGoogle Scholar
  25. 25.
    J. Fontanals, B.A. Dang-Vu, O. Porges, J. Rosell, M.A. Roa, Integrated grasp and motion planning using independent contact regions, in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2014, pp. 887–893Google Scholar
  26. 26.
    M. Gienger, M. Toussaint, C. Goerick, Task maps in humanoid robot manipulation, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008, pp. 2758–2764Google Scholar
  27. 27.
    S. Guadarrama, L. Riano, D. Golland, D. Gouhring, Y. Jia, D. Klein, P. Abbeel, T. Darrell, Grounding spatial relations for human-robot interaction, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 1640–1647Google Scholar
  28. 28.
    K. Harada, S. Kajita, K. Kaneko, H. Hirukawa, Dynamics and balance of a humanoid robot during manipulation tasks. IEEE Trans. Robot. 22(3), 568–575 (2006)CrossRefGoogle Scholar
  29. 29.
    K. Harada, S. Kajita, F. Kanehiro, K. Fujiwara, K. Kaneko, K. Yokoi, H. Hirukawa, Real-time planning of humanoid robot’s gait for force controlled manipulation. IEEE/ASME Trans. Mechatron. 12(1), 53–62 (2007)CrossRefGoogle Scholar
  30. 30.
    K. Hauser, T. Bretl, K. Harada, J.C. Latombe, Using motion primitives in probabilistic sample-based planning for humanoid robots, in Algorithmic Foundation of Robotics VII. Springer Tracts in Advanced Robotics, vol. 47, 2008, pp. 507–522Google Scholar
  31. 31.
    K. Hauser, V. Ng-Thowhing, H. Gonzalez-Banos, Multi-modal motion planning for a humanoid robot manipulation task, in Robotics Research, Springer Tracts in Advanced Robotics 66, pp. 307–317 (2010)Google Scholar
  32. 32.
    K. Hauser, J.C. Latombe, Non-gaited humanoid locomotion planning, in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2005, pp. 7–12Google Scholar
  33. 33.
    K. Hauser, J.C. Latombe, Integrating task and PRM motion planning, in Workshop on Bridging the Gap Between Task and Motion Planning, ICAPS, 2009Google Scholar
  34. 34.
    H. Hirukawa, S. Hattori, K. Harada, S. Kajita, K. Kaneko, F. Kanehiro, K. Fujiwara, M. Morisawa, A universal stability criterion of the foot contact of legged robots, in Proceedings of the IEEE International Conference on Robotics and Automation, 2006, pp. 1976–1983Google Scholar
  35. 35.
    M. Horowitz, J. Burdick, Interactive non-prehensile manipulation for grasping via POMDPs, in Proceedings of the IEEE International Conference on Robotics and Automation, 2013, pp. 3242–3249Google Scholar
  36. 36.
    K. Hsiao, L.P. Kaelbling, T. Lozano-Perez, Grasping POMDPs, in Proceedings of the IEEE International Conference on Robotics and Automation, 2007, pp. 4685–4692Google Scholar
  37. 37.
    S.H. Hyon, G. Chen, Passivity-based full-body force control for humanoids and application to dynamic balancing and locomotion, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006, pp. 4915–4922Google Scholar
  38. 38.
    L.P. Kaelbling, T. Lozano-Perez, Integrated task and motion planning in belief space. Int. J. Robot. Res. 32, 1194–1227 (2013)CrossRefGoogle Scholar
  39. 39.
    S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi, H. Hirukawa, Resolved momentum control: humanoid motion planning based on the linear and angular momentum, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003, pp. 1644–1650Google Scholar
  40. 40.
    O. Kanoun, F. Lamiraux, P.B. Wieber, Kinematic control of redundant manipulators: generalizing the task priority framework to inequality tasks. IEEE Trans. Robot. 27(4), 785–792 (2011)CrossRefGoogle Scholar
  41. 41.
    N. Kitaev, I. Mordatch, S. Patil, P. Abbeel, Physics-based trajectory optimization for grasping in cluttered environments, in Proceedings of the IEEE International Conference on Robotics and Automation, 2015, pp. 3102–3109Google Scholar
  42. 42.
    O. Kroemer, C. Daniel, G. Neumann, H. van Hoof, J. Peters, Towards learning hierarchical skills for multi-phase manipulation tasks, in Proceedings of the IEEE International Conference on Robotics and Automation, 2015, pp. 1503–1510Google Scholar
  43. 43.
    J. Kuffner, S.M. Lavalle, RRT-connect: an efficient approach to single-query path planning, in Proceedings of the IEEE International Conference on Robotics and Automation, 2000, pp. 995–1001Google Scholar
  44. 44.
    D. Leidner, A. Dietrich, M. Beetz, A. Albu-Schaeffer, Knowledge-enabled parameterization of whole-body control strategies for compliant service robots. Auton. Robot. 40(3), 519–536 (2016)CrossRefGoogle Scholar
  45. 45.
    M. Levhin, K. Nishiwaki, S. Kagami, M. Stilman, Autonomous environment manipulation to assist humanoid locomotion, in Proceedings of the IEEE International Conference on Robotics and Automation, 2014, pp. 4633–4638Google Scholar
  46. 46.
    N. Mansard, O. Khatib, A. Kheddar, A unified approach to integrate unilateral constraints in the stack of tasks. IEEE Trans. Robot. 25(3), 670–685 (2009)CrossRefGoogle Scholar
  47. 47.
    D. Misra, J. Sung, K. Lee, A. Saxena, Tell me Dave: context-sensitive grounding of natural language to mobile manipulation instructions, in Proceedings of Robotics: Science and Systems, RSS, 2014Google Scholar
  48. 48.
    M. Murooka, S. Noda, S. Nozawa, Y. Kakiuchi, M. Inaba, Manipulation strategy decision and execution based on strategy proving operation for carrying large and heavy objects, in Proceedings of the IEEE International Conference on Robotics and Automation, 2014, pp. 3425–3432Google Scholar
  49. 49.
    S. Nozawa, Y. Kakiuchi, K. Okada, M. Inaba, Controlling the planar motion of a heavy object by pushing with a humanoid robot using dual-arm force control, in Proceedings of the IEEE International Conference on Robotics and Automation, 2012, pp. 1428–1435Google Scholar
  50. 50.
    R. Paul, Problems and research issues associated with the hybrid control of force and displacement, in Proceedings of the IEEE International Conference on Robotics and Automation, 1987, pp. 1966–1971Google Scholar
  51. 51.
    S. Nozawa, I. Kumagai, Y. Kakiuchi, K. Okada, M. Inaba, Humanoid full-body controller adapting constraints in structured objects through updating task-level reference force, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 3417–3424Google Scholar
  52. 52.
    D. Nyga, M. Beetz, Everything robots always wanted to know about housework (but were afraid to ask), in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 243–250Google Scholar
  53. 53.
    K. Okada, M. Kojima, Y. Sagawa, T. Ichino, K. Sato, M. Inaba, Vision based behavior verification system of humanoid robot for daily environment tasks, in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2006, pp. 7–12Google Scholar
  54. 54.
    C. Ott, O. Eiberger, W. Friedl, B. Bauml, U. Hillenbrand, C. Borst, A. Albu-Schaeffer, B. Brunner, H. Hirschmueller, S. Kielhoefer, R. Konietschke, M. Suppa, T. Wimboeck, F. Zacharias, G. Hirzinger, A humanoid two-arm system for dexterous manipulation, in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2006, pp. 276–283Google Scholar
  55. 55.
    J. Pajarinen, V. Kyrki, Robotic manipulation of multiple objects as a POMDP. Artif. Intell. 247, 213–228 (2015)MathSciNetCrossRefGoogle Scholar
  56. 56.
    J. Pan, D. Manocha, Closing the loop between motion planning and task execution using real-time GPU-based planners, in 24th AAAI Conference on Artificial Intelligence, 2010, pp. 43–47Google Scholar
  57. 57.
    O. Porges, T. Stouraitis, C. Borst, M.A. Roa, Reachability and capability analysis for manipulation tasks, in ROBOT2013: First Iberian Robotics Conference. Series Advances in Intelligent Systems and Computing, vol. 253 (Springer, 2014), pp. 703–718Google Scholar
  58. 58.
    K. Ramirez-Amaro, M. Beetz, G. Cheng, Transferring skills to humanoid robots by extracting semantic representations from observations of human activities. Aritf. Intell. 247, 95–118 (2017)MathSciNetCrossRefGoogle Scholar
  59. 59.
    M.A. Roa, R. Suarez, Grasp quality measures: review and performance. Auton. Robot. 38(1), 65–88 (2015)CrossRefGoogle Scholar
  60. 60.
    Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, K. Fujimura, The intelligent ASIMO: system overview and integration, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2002, pp. 2478–2483Google Scholar
  61. 61.
    M. Santello, M. Flanders, J. Soechting, Patterns of hand motion during grasping and the influence of sensory guidance. J. Neurosci. 22(4), 1426–1435 (2002)CrossRefGoogle Scholar
  62. 62.
    L. Sentis, O. Khatib, Synthesis of whole-body behaviors through hierarchical control of behavioral primitives. Int. J. Humanoid Robot. 2(4), 505–518 (2005)CrossRefGoogle Scholar
  63. 63.
    L. Sentis, J. Park, O. Khatib, Compliant control of multicontact and center-of-mass behaviors in humanoid robots. IEEE Trans. Robot. 26(3), 483–501 (2010)CrossRefGoogle Scholar
  64. 64.
    C. Smith, Y. Karayiannidis, L. Nalpantidis, X. Gratal, P. Qi, D.V. Dimarogonas, D. Kragic, Dual arm manipulation – a survey. Robot. Auton. Syst. 60(10), 1340–1353 (2012)Google Scholar
  65. 65.
    S. Srivastava, E. Fang, L. Riano, R. Chitnis, S. Russell, P. Abbeel, Combined task and motion planning through and extensible planner-independent interface layer, in Proceedings of the IEEE International Conference on Robotics and Automation, 2014, pp. 639–646Google Scholar
  66. 66.
    M. Stilman, K. Nishiwaki, S. Kagami, J. Kuffner, Planning and executing navigation among movable obstacles. Adv. Robot. 21(14), 1617–1634 (2007)CrossRefGoogle Scholar
  67. 67.
    T. Sugihara, Y. Nakamura, Whole-body cooperative balancing of humanoid robot using COG Jacobian, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2002, pp. 2575–2580Google Scholar
  68. 68.
    T. Takubo, K. Inoue, T. Arai, Pushing an object considering the hand reflect forces by humanoid robot in dynamic walking, in Proceedings of the IEEE International Conference on Robotics and Automation, 2005, pp. 1706–1711Google Scholar
  69. 69.
    S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A. Banerjee, S. Teller, N. Roy, Understanding natural language commands for robotic navigation and mobile manipulation, in Proceedings of the National Conference on Artificial Intelligence – AAAI, 2011, pp. 1507–1514Google Scholar
  70. 70.
    N. Vahrenkamp, T. Asfour, R. Dillmann, Simultaneous grasp and motion planning. IEEE Robot. Autom. Mag. 19(2), 43–57 (2012)Google Scholar
  71. 71.
    J. Wolfe, H. Marthi, S. Russell, Combined task and motion planning for mobile manipulation, in Proceedings of the 12th International Conference on Automated Planning and Scheduling, 2010, pp. 254–257Google Scholar
  72. 72.
    E. Yoshida, M. Poirier, J.P. Laumond, O. Kanoun, F. Lamiraux, R. Alami, K. Yokoi, Pivoting based manipulation by a humanoid robot. Auton. Robot. 28, 77–88 (2010)CrossRefGoogle Scholar
  73. 73.
    F. Zacharias, C. Borst, S. Wolf, G. Hirzinger, The capability map: a tool to analyze robot arm Workspaces. Int. J. Humanoid Robot. 10(4), 1350031 (2013)CrossRefGoogle Scholar
  74. 74.
    F. Zacharias, D. Leidner, F. Schmidt, C. Borst, G. Hirzinger, Exploiting structure in two-armed manipulation tasks for humanoid robots, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, pp. 5446–5452Google Scholar
  75. 75.
    F. Zacharias, C. Schlette, F. Schmidt, C. Borst, J. Rossmann, G. Hirzinger, Making planned paths look more human-like in humanoid robot manipulation planning, in Proceedings of the IEEE International Conference on Robotics and Automation, 2011, pp. 1192–1198Google Scholar
  76. 76.
    R. Zoliner, M. Pardowitz, S. Knoop, R. Dillmann, Towards cognitive robots: building hierarchical task representations of manipulations from human demonstration, in Proceedings of the IEEE International Conference on Robotics and Automation, 2005, pp. 1535–1540Google Scholar
  77. 77.
    M. Zucker, N. Ratliff, A.D. Dragan, M. Pivtoraiko, M. Klingensmith, C.M. Dellin, J.A. Bagnell, S.S. Srinivasa, CHOMP: covariant hamiltonian optimization for motion planning. Int. J. Robot. Res. 32(9–10), 1164–1193 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Graduate School of Engineering ScienceOsaka UniversityOsakaJapan
  2. 2.Department of Systems Innovation, Graduate School of Engineering ScienceOsaka UniversityToyonakaJapan
  3. 3.Institute of Robotics and MechatronicsGerman Aerospace Center (DLR)WesslingGermany

Personalised recommendations