Advertisement

Planning Multi-fingered Grasps as Probabilistic Inference in a Learned Deep Network

  • Qingkai LuEmail author
  • Kautilya Chenna
  • Balakumar Sundaralingam
  • Tucker Hermans
Conference paper
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 10)

Abstract

We propose a novel approach to multi-fingered grasp planning leveraging learned deep neural network models. We train a convolutional neural network to predict grasp success as a function of both visual information of an object and grasp configuration. We can then formulate grasp planning as inferring the grasp configuration which maximizes the probability of grasp success. We efficiently perform this inference using a gradient-ascent optimization inside the neural network using the backpropagation algorithm. Our work is the first to directly plan high quality multi-fingered grasps in configuration space using a deep neural network without the need of an external planner. We validate our inference method performing both multi-finger and two-finger grasps on real robots. Our experimental results show that our planning method outperforms existing planning methods for neural networks; while offering several other benefits including being data-efficient in learning and fast enough to be deployed in real robotic applications.

Keywords

Grasping grasp planning grasp learning multi-fingered grasping 

Notes

Acknowledgements

Q. Lu and B. Sundaralingam were supported in part by NSF Award #1657596.

References

  1. 1.
    Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis-a survey. IEEE Trans. Robot. 30(2), 289–309 (2014)Google Scholar
  2. 2.
    Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., Dollar, A. M.: The YCB object and model set: towards common benchmarks for manipulation research. In: International Conference on Advanced Robotics (ICAR), pp. 510–517 (2015)Google Scholar
  3. 3.
    Ciocarlie, M., Goldfeder, C., Allen, P.K.: Dimensionality reduction for hand-independent dexterous robotic grasping. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3270–3275 (2007)Google Scholar
  4. 4.
    Dragiev, S., Toussaint, M., Gienger, M.: Gaussian process implicit surfaces for shape estimation and grasping. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2845–2850 (2011)Google Scholar
  5. 5.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  6. 6.
    Grupen, R.A.: Planning grasp strategies for multifingered robot hands. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 646–651 (1991)Google Scholar
  7. 7.
    Gschwandtner, M., Kwitt, R., Uhl, A., Pree, W.: BlenSor: blender sensor simulation toolbox. In: Advances in visual computing, pp. 199–208 (2011)Google Scholar
  8. 8.
    Gualtieri, M., Ten Pas, A., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 598–605 (2016)Google Scholar
  9. 9.
    Johns, E., Leutenegger, S., Davison. A.J.: Deep learning a grasp function for grasping under gripper pose uncertainty. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4461–4468 (2016)Google Scholar
  10. 10.
    Kappler, D., Bohg, J., Schaal, S.: Leveraging big data for grasp planning. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 4304–4311 (2015)Google Scholar
  11. 11.
    Kopicki, M., Detry, R., Adjigble, M., Stolkin, R., Leonardis, A., Wyatt, J.L.: One-shot learning and generation of dexterous grasps for novel objects. Int. J. Robot. Res. (IJRR) 35(8), 959–976 (2016)Google Scholar
  12. 12.
    Kumra, S., Kanan, C.: Robotic grasp detection using deep convolutional neural networks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)Google Scholar
  13. 13.
    Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. Int. J. Robot. Res. (IJRR) 34(4–5), 705–724 (2015)Google Scholar
  14. 14.
    Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Robot. Res. p. 0278364917710318 (2016)Google Scholar
  15. 15.
    Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Goldberg, K.: Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In: Robotics Science and Systems (RSS) (2017)Google Scholar
  16. 16.
    Murray, R.M., Li, Z., Sastry, S.S.: A Mathematical Introduction to Robotic Manipulation. CRC Press, Boca Raton (1994)Google Scholar
  17. 17.
    Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50K tries and 700 robot hours. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413 (2016)Google Scholar
  18. 18.
    Redmon, J., Angelova, A.: Real-time grasp detection using convolutional neural networks. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1316–1322 (2015)Google Scholar
  19. 19.
    Sahbani, A., El-Khoury, S., Bidaud, P.: An overview of 3D object grasp synthesis algorithms. Robot. Auton. Syst. 60(3), 326–336 (2012)Google Scholar
  20. 20.
    Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Robot. Res.(IJRR), 27(2), 157–173 (2008)Google Scholar
  21. 21.
    Saxena, S., Wong, L.L.S., Ng, A.Y.: Learning grasp strategies with partial shape information. In: AAAI National Conference on Artificial Intelligence, pp. 1491–1494 (2008)Google Scholar
  22. 22.
    Singh, A., Sha, J., Narayan, K.S., Achim, T., Abbeel, P.: Bigbird: a large-scale 3D database of object instances. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 509–516 (2014)Google Scholar
  23. 23.
    Varley, J., Weisz, J., Weiss, J., Allen, P.: Generating multi-fingered robotic grasps via deep learning. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4415–4420 (2015)Google Scholar
  24. 24.
    Veres, M., Moussa, M., Taylor, G.W.: Modeling grasp motor imagery through deep conditional generative models. IEEE Robot. Autom. Lett. 2(2), 757–764 (2017)Google Scholar
  25. 25.
    Zhou, Y., Hauser, K.: 6DOF grasp planning by optimizing a deep learning scoring function. In: Robotics: Science and Systems (RSS) Workshop on Revisiting Contact - Turning a Problem into a Solution (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Qingkai Lu
    • 1
    Email author
  • Kautilya Chenna
    • 1
  • Balakumar Sundaralingam
    • 1
  • Tucker Hermans
    • 1
  1. 1.Utah Robotics Center, School of ComputingUniversity of UtahSalt Lake CityUSA

Personalised recommendations