Autonomous Robots

, Volume 43, Issue 2, pp 469–483 | Cite as

Relaxed-rigidity constraints: kinematic trajectory optimization and collision avoidance for in-grasp manipulation

  • Balakumar SundaralingamEmail author
  • Tucker Hermans
Part of the following topical collections:
  1. Special Issue on Robotics: Science and Systems


This paper proposes a novel approach to performing in-grasp manipulation: the problem of moving an object with reference to the palm from an initial pose to a goal pose without breaking or making contacts. Our method to perform in-grasp manipulation uses kinematic trajectory optimization which requires no knowledge of dynamic properties of the object. We implement our approach on an Allegro robot hand and perform thorough experiments on ten objects from the YCB dataset. The proposed method is general enough to generate motions for most objects the robot can grasp. Experimental results support the feasibillty of its application across a variety of object shapes. We explore the adaptability of our approach to additional task requirements by including collision avoidance and joint space smoothness costs. The grasped object avoids collisions with the environment by the use of a signed distance cost function. We reduce the effects of unmodeled object dynamics by requiring smooth joint trajectories. We additionally compensate for errors encountered during trajectory execution by formulating an object pose feedback controller.


Dexterous manipulation Trajectory optimization Motion planning 



This study was funded partly by National Science Foundation (NSF) (Grant Number 1657596).

Compliance with ethical standards

Conflict of interest

Author Balakumar Sundaralingam has no conflicts of interest. Author Tucker Hermans has no conflicts of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Andrews, S., & Kry, P. G. (2013). Goal directed multi-finger manipulation: Control policies and analysis. Computers & Graphics, 37(7), 830–839.CrossRefGoogle Scholar
  2. Bai, Y., Liu, & C. K. (2014). Dexterous manipulation using both palm and fingers. In IEEE International Conference on Robotics and Automation (ICRA), IEEE (pp. 1560–1565).Google Scholar
  3. Bicchi, A. (2000). Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity. IEEE Transactions Robotics and Automation, 16(6), 652–662.CrossRefGoogle Scholar
  4. Bicchi, A., & Sorrentino, R. (1995). Dexterous manipulation through rolling. IEEE International Conference on Robotics and Automation (ICRA), IEEE, 1, 452–457.Google Scholar
  5. Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2015). The YCB object and model set: Towards common benchmarks for manipulation research. In International conference on advanced robotics (ICAR), IEEE (pp. 510–517).Google Scholar
  6. Carpin, S., Liu, S., Falco, J., & Van Wyk, K. (2016). Multi-fingered robotic grasping: A primer. arXiv preprint arXiv:160706620.
  7. Ciocarlie, M., Goldfeder, C., & Allen, P. (2007). Dexterous grasping via eigengrasps: A low-dimensional approach to a high-complexity problem. In Robotics: Science and systems manipulation workshop-sensing and adapting to the real world, Citeseer.Google Scholar
  8. Cutkosky, M. R., & Wright, P. K. (1986). Friction, stability and the design of robotic fingers. Tohe International Journal of Robotics Research, 5(4), 20–37.CrossRefGoogle Scholar
  9. Fearing, R. S. (1986). Simplified grasping and manipulation with dextrous robot hands. IEEE Transsctions Robotics and Automation, 2(4), 188–195.CrossRefGoogle Scholar
  10. Garcia Cifuentes, C., Issac, J., Wüthrich, M., Schaal, S., & Bohg, J. (2017). Probabilistic articulated real-time tracking for robot manipulation. IEEE Robotics and Automation Letters (RA-L), 2(2), 577–584.CrossRefGoogle Scholar
  11. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F., & Marín-Jiménez, M. (2014). Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition, 47(6), 2280–2292.CrossRefGoogle Scholar
  12. Gill, P. E., Murray, W., & Saunders, M. A. (2005). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM Review, 47(1), 99–131.MathSciNetCrossRefzbMATHGoogle Scholar
  13. Han, L., & Trinkle, J. C. (1998). Dextrous manipulation by rolling and finger gaiting. In IEEE international conference on robotics and automation (ICRA), IEEE (Vol. 1, pp. 730–735).Google Scholar
  14. Han, L., Guan, Y. S., Li, Z., Shi, Q., & Trinkle, J. C. (1997). Dextrous manipulation with rolling contacts. In: IEEE International conference on robotics and automation (ICRA), IEEE (Vol. 2, pp. 992–997).Google Scholar
  15. Hang, K., Li, M., Stork, J. A., Bekiroglu, Y., Pokorny, F. T., Billard, A., et al. (2016). Hierarchical fingertip space: A unified framework for grasp planning and in-hand grasp adaptation. IEEE Transactions on Robotics, 32(4), 960–972.CrossRefGoogle Scholar
  16. Härtl, H. (1995). Dextrous manipulation with multifingered robot hands including rolling and slipping of the fingertips. Robotics and Autonomous Systems, 14(1), 29–53.CrossRefGoogle Scholar
  17. Hertkorn, K., Roa, M. A., & Borst, C. (2013). Planning in-hand object manipulation with multifingered hands considering task constraints. In IEEE International conference on robotics and automation (ICRA), IEEE (pp. 617–624).Google Scholar
  18. Hong, J., Lafferriere, G., Mishra, B., & Tan, X. (1990). Fine manipulation with multifinger hands. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 1568–1573).Google Scholar
  19. Hoof, H. V., Hermans, T., Neumann, G., & Peters, J. (2015). Learning robot in-hand manipulation with tactile features. In Proceedings IEEE/RAS international conference on humanoid robots (Humanoids).Google Scholar
  20. Huynh, D. Q. (2009). Metrics for 3D rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35(2), 155–164.MathSciNetCrossRefGoogle Scholar
  21. Kumar, V., Tassa, Y., Erez, T., & Todorov, E. (2014). Real-time behaviour synthesis for dynamic hand-manipulation. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 6808–6815).Google Scholar
  22. Kumar, V., Todorov, E., & Levine, S. (2016). Optimal control with learned local models: Application to dexterous manipulation. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 378–383).Google Scholar
  23. Li, Z., Hsu, P., & Sastry, S. (1989). Grasping and coordinated manipulation by a multifingered robot hand. International Journal of Robotics Research, 8(4), 33–50.CrossRefGoogle Scholar
  24. Li, Q., Elbrechter, C., Haschke, R., & Ritter, H. (2013). Integrating vision, haptics and proprioception into a feedback controller for in-hand manipulation of unknown objects. In 2013 IEEE/RSJ international conference on intelligent robots and systems, IEEE (pp 2466–2471).Google Scholar
  25. Mamou, K., & Ghorbel, F. (2009). A simple and efficient approach for 3d mesh approximate convex decomposition. In 2009 16th IEEE international conference on image processing (ICIP), IEEE (pp. 3501–3504).Google Scholar
  26. Mordatch, I., Popović, Z., & Todorov, E. (2012). Contact-invariant optimization for hand manipulation. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation, Eurographics Association (pp. 137–144).Google Scholar
  27. Posa, M., Cantu, C., & Tedrake, R. (2014). Direct method for trajectory optimization of rigid bodies through contact. International Journal of Robotics Research, 33(1), 69–81.CrossRefGoogle Scholar
  28. Posa, M., Kuindersma, S., & Tedrake, R. (2016). Optimization and stabilization of trajectories for constrained dynamical systems. In IEEE international conference on robotics and automation (ICRA).Google Scholar
  29. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. (2009). Ros: an open-source robot operating system. In ICRA workshop on open source software, Kobe, Japan (vol 3, p. 5).Google Scholar
  30. Rojas, N., & Dollar, A. M. (2016). Gross motion analysis of fingertip-based within-hand manipulation. IEEE Transactions Robotics and Automation, 32(2), 1009–1016.CrossRefGoogle Scholar
  31. Rus, D. (1992). Dexterous rotations of polyhedra. In IEEE international conference on robotics and automation (ICRA) Google Scholar
  32. Salisbury, J. K., & Craig, J. J. (1982). Articulated hands: Force control and kinematic issues. The International journal of Robotics Research, 1(1), 4–17.CrossRefGoogle Scholar
  33. Salisbury, J. K., & Roth, B. (1983). Kinematic and force analysis of articulated mechanical hands. Journal of Mechanisms, Transmissions, and Automation in Design, 105(1), 35–41.CrossRefGoogle Scholar
  34. Scarcia, U., Hertkorn, K., Melchiorri, C., Palli, G., & Wimböck, T. (2015). Local online planning of coordinated manipulation motion. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 6081–6087).Google Scholar
  35. Schulman, J., Duan, Y., Ho, J., Lee, A., Awwal, I., Bradlow, H., et al. (2014). Motion planning with sequential convex optimization and convex collision checking. Intl Journal of Robotics Research, 33(9), 1251–1270.CrossRefGoogle Scholar
  36. Srinivasa, S. S., Erdmann, M. A., Mason, M. T. (2005). Using projected dynamics to plan dynamic contact manipulation. In IEEE/RSJ international conference on intelligent robots and systems (IROS), IEEE (pp. 3618–3623).Google Scholar
  37. Sundaralingam, B., Hermans, T. (2017). Relaxed-rigidity constraints: In-grasp manipulation using purely kinematic trajectory optimization. In Proceedings of robotics: Science and systems, Cambridge, MA.
  38. Sundaralingam, B., & Hermans, T. (2018). Geometric in-hand regrasp planning: Alternating optimization of finger gaits and in-grasp manipulation. In IEEE international conference on robotics and automation (ICRA).Google Scholar
  39. Toussaint, M. (2017). A tutorial on Newton methods for constrained trajectory optimization and relations to slam, Gaussian process smoothing, optimal control, and probabilistic inference. In Geometric and numerical foundations of movements (pp. 361–392). Berlin: Springer.Google Scholar
  40. Van Den Bergen, G. (2001). Proximity queries and penetration depth computation on 3d game objects. In Game developers conference (Vol. 170).Google Scholar
  41. Zucker, M., Ratliff, N., Dragan, a D, Pivtoraiko, M., Klingensmith, M., Dellin, C. M., et al. (2013). CHOMP: Covariant Hamiltonian optimization for motion planning. The International Journal of Robotics Research, 32(9–10), 1164–1193. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of Utah Robotics Center and the School of ComputingUniversity of UtahSalt Lake CityUSA

Personalised recommendations