This paper proposes a novel approach to performing in-grasp manipulation: the problem of moving an object with reference to the palm from an initial pose to a goal pose without breaking or making contacts. Our method to perform in-grasp manipulation uses kinematic trajectory optimization which requires no knowledge of dynamic properties of the object. We implement our approach on an Allegro robot hand and perform thorough experiments on ten objects from the YCB dataset. The proposed method is general enough to generate motions for most objects the robot can grasp. Experimental results support the feasibillty of its application across a variety of object shapes. We explore the adaptability of our approach to additional task requirements by including collision avoidance and joint space smoothness costs. The grasped object avoids collisions with the environment by the use of a signed distance cost function. We reduce the effects of unmodeled object dynamics by requiring smooth joint trajectories. We additionally compensate for errors encountered during trajectory execution by formulating an object pose feedback controller.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Andrews, S., & Kry, P. G. (2013). Goal directed multi-finger manipulation: Control policies and analysis. Computers & Graphics, 37(7), 830–839.
Bai, Y., Liu, & C. K. (2014). Dexterous manipulation using both palm and fingers. In IEEE International Conference on Robotics and Automation (ICRA), IEEE (pp. 1560–1565).
Bicchi, A. (2000). Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity. IEEE Transactions Robotics and Automation, 16(6), 652–662.
Bicchi, A., & Sorrentino, R. (1995). Dexterous manipulation through rolling. IEEE International Conference on Robotics and Automation (ICRA), IEEE, 1, 452–457.
Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2015). The YCB object and model set: Towards common benchmarks for manipulation research. In International conference on advanced robotics (ICAR), IEEE (pp. 510–517).
Carpin, S., Liu, S., Falco, J., & Van Wyk, K. (2016). Multi-fingered robotic grasping: A primer. arXiv preprint arXiv:160706620.
Ciocarlie, M., Goldfeder, C., & Allen, P. (2007). Dexterous grasping via eigengrasps: A low-dimensional approach to a high-complexity problem. In Robotics: Science and systems manipulation workshop-sensing and adapting to the real world, Citeseer.
Cutkosky, M. R., & Wright, P. K. (1986). Friction, stability and the design of robotic fingers. Tohe International Journal of Robotics Research, 5(4), 20–37.
Fearing, R. S. (1986). Simplified grasping and manipulation with dextrous robot hands. IEEE Transsctions Robotics and Automation, 2(4), 188–195.
Garcia Cifuentes, C., Issac, J., Wüthrich, M., Schaal, S., & Bohg, J. (2017). Probabilistic articulated real-time tracking for robot manipulation. IEEE Robotics and Automation Letters (RA-L), 2(2), 577–584.
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F., & Marín-Jiménez, M. (2014). Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition, 47(6), 2280–2292.
Gill, P. E., Murray, W., & Saunders, M. A. (2005). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM Review, 47(1), 99–131.
Han, L., & Trinkle, J. C. (1998). Dextrous manipulation by rolling and finger gaiting. In IEEE international conference on robotics and automation (ICRA), IEEE (Vol. 1, pp. 730–735).
Han, L., Guan, Y. S., Li, Z., Shi, Q., & Trinkle, J. C. (1997). Dextrous manipulation with rolling contacts. In: IEEE International conference on robotics and automation (ICRA), IEEE (Vol. 2, pp. 992–997).
Hang, K., Li, M., Stork, J. A., Bekiroglu, Y., Pokorny, F. T., Billard, A., et al. (2016). Hierarchical fingertip space: A unified framework for grasp planning and in-hand grasp adaptation. IEEE Transactions on Robotics, 32(4), 960–972.
Härtl, H. (1995). Dextrous manipulation with multifingered robot hands including rolling and slipping of the fingertips. Robotics and Autonomous Systems, 14(1), 29–53.
Hertkorn, K., Roa, M. A., & Borst, C. (2013). Planning in-hand object manipulation with multifingered hands considering task constraints. In IEEE International conference on robotics and automation (ICRA), IEEE (pp. 617–624).
Hong, J., Lafferriere, G., Mishra, B., & Tan, X. (1990). Fine manipulation with multifinger hands. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 1568–1573).
Hoof, H. V., Hermans, T., Neumann, G., & Peters, J. (2015). Learning robot in-hand manipulation with tactile features. In Proceedings IEEE/RAS international conference on humanoid robots (Humanoids).
Huynh, D. Q. (2009). Metrics for 3D rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35(2), 155–164.
Kumar, V., Tassa, Y., Erez, T., & Todorov, E. (2014). Real-time behaviour synthesis for dynamic hand-manipulation. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 6808–6815).
Kumar, V., Todorov, E., & Levine, S. (2016). Optimal control with learned local models: Application to dexterous manipulation. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 378–383).
Li, Z., Hsu, P., & Sastry, S. (1989). Grasping and coordinated manipulation by a multifingered robot hand. International Journal of Robotics Research, 8(4), 33–50.
Li, Q., Elbrechter, C., Haschke, R., & Ritter, H. (2013). Integrating vision, haptics and proprioception into a feedback controller for in-hand manipulation of unknown objects. In 2013 IEEE/RSJ international conference on intelligent robots and systems, IEEE (pp 2466–2471).
Mamou, K., & Ghorbel, F. (2009). A simple and efficient approach for 3d mesh approximate convex decomposition. In 2009 16th IEEE international conference on image processing (ICIP), IEEE (pp. 3501–3504).
Mordatch, I., Popović, Z., & Todorov, E. (2012). Contact-invariant optimization for hand manipulation. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation, Eurographics Association (pp. 137–144).
Posa, M., Cantu, C., & Tedrake, R. (2014). Direct method for trajectory optimization of rigid bodies through contact. International Journal of Robotics Research, 33(1), 69–81.
Posa, M., Kuindersma, S., & Tedrake, R. (2016). Optimization and stabilization of trajectories for constrained dynamical systems. In IEEE international conference on robotics and automation (ICRA).
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. (2009). Ros: an open-source robot operating system. In ICRA workshop on open source software, Kobe, Japan (vol 3, p. 5).
Rojas, N., & Dollar, A. M. (2016). Gross motion analysis of fingertip-based within-hand manipulation. IEEE Transactions Robotics and Automation, 32(2), 1009–1016.
Rus, D. (1992). Dexterous rotations of polyhedra. In IEEE international conference on robotics and automation (ICRA)
Salisbury, J. K., & Craig, J. J. (1982). Articulated hands: Force control and kinematic issues. The International journal of Robotics Research, 1(1), 4–17.
Salisbury, J. K., & Roth, B. (1983). Kinematic and force analysis of articulated mechanical hands. Journal of Mechanisms, Transmissions, and Automation in Design, 105(1), 35–41.
Scarcia, U., Hertkorn, K., Melchiorri, C., Palli, G., & Wimböck, T. (2015). Local online planning of coordinated manipulation motion. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 6081–6087).
Schulman, J., Duan, Y., Ho, J., Lee, A., Awwal, I., Bradlow, H., et al. (2014). Motion planning with sequential convex optimization and convex collision checking. Intl Journal of Robotics Research, 33(9), 1251–1270.
Srinivasa, S. S., Erdmann, M. A., Mason, M. T. (2005). Using projected dynamics to plan dynamic contact manipulation. In IEEE/RSJ international conference on intelligent robots and systems (IROS), IEEE (pp. 3618–3623).
Sundaralingam, B., Hermans, T. (2017). Relaxed-rigidity constraints: In-grasp manipulation using purely kinematic trajectory optimization. In Proceedings of robotics: Science and systems, Cambridge, MA. https://doi.org/10.15607/RSS.2017.XIII.015.
Sundaralingam, B., & Hermans, T. (2018). Geometric in-hand regrasp planning: Alternating optimization of finger gaits and in-grasp manipulation. In IEEE international conference on robotics and automation (ICRA).
Toussaint, M. (2017). A tutorial on Newton methods for constrained trajectory optimization and relations to slam, Gaussian process smoothing, optimal control, and probabilistic inference. In Geometric and numerical foundations of movements (pp. 361–392). Berlin: Springer.
Van Den Bergen, G. (2001). Proximity queries and penetration depth computation on 3d game objects. In Game developers conference (Vol. 170).
Zucker, M., Ratliff, N., Dragan, a D, Pivtoraiko, M., Klingensmith, M., Dellin, C. M., et al. (2013). CHOMP: Covariant Hamiltonian optimization for motion planning. The International Journal of Robotics Research, 32(9–10), 1164–1193. https://doi.org/10.1177/0278364913488805.
This study was funded partly by National Science Foundation (NSF) (Grant Number 1657596).
Conflict of interest
Author Balakumar Sundaralingam has no conflicts of interest. Author Tucker Hermans has no conflicts of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
This is one of several papers published in Autonomous Robots comprising the “Special Issue on Robotics Science and Systems”.
About this article
Cite this article
Sundaralingam, B., Hermans, T. Relaxed-rigidity constraints: kinematic trajectory optimization and collision avoidance for in-grasp manipulation. Auton Robot 43, 469–483 (2019). https://doi.org/10.1007/s10514-018-9772-z
- Dexterous manipulation
- Trajectory optimization
- Motion planning