Improving Grasp Performance Using In-Hand Proximity and Contact Sensing

  • Radhen Patel
  • Rebeca Curtis
  • Branden Romero
  • Nikolaus CorrellEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 816)


We describe the grasping and manipulation strategy that we employed at the autonomous track of the Robotic Grasping and Manipulation Competition at IROS 2016. A salient feature of our architecture is the tight coupling between visual (Asus Xtion) and tactile perception (Robotic Materials), to reduce the uncertainty in sensing and actuation. We demonstrate the importance of tactile sensing and reactive control during the final stages of grasping using a Kinova Robotic arm. The set of tools and algorithms for object grasping presented here have been integrated into the open-source Robot Operating System (ROS). We have focused exclusively on the manipulation aspect (Track 1) of the competition as the bin-picking task (Track 2) would require a different perception strategy, focusing more on object identification.


  1. 1.
    Baxter Robot Grippers, Rethink Robotics.
  2. 2.
  3. 3.
    ILIMB User Manual, Touch Bionics.
  4. 4.
  5. 5.
    Balasubramanian, R., Xu, L., Brook, P.D., Smith, J.R., Matsuoka, Y.: Physical human interactive guidance: identifying grasping principles from human-planned grasps. IEEE Trans. Rob. 28(4), 899–910 (2012)CrossRefGoogle Scholar
  6. 6.
    Bohren, J., Rusu, R.B., Jones, E.G., Marder-Eppstein, E., Pantofaru, C., Wise, M., Mösenlechner, L., Meeussen, W., Holzer, S.: Towards autonomous robotic butlers: lessons learned with the PR2. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 5568–5575. IEEE (2011)Google Scholar
  7. 7.
    Bone, G.M., Lambert, A., Edwards, M.: Automated modeling and robotic grasping of unknown three-dimensional objects. In: IEEE International Conference on Robotics and Automation, 2008, ICRA 2008, pp. 292–298. IEEE (2008)Google Scholar
  8. 8.
    Breuer, T., Macedo, G.R.G., Hartanto, R., Hochgeschwender, N., Holz, D., Hegger, F., Jin, Z., Müller, C., Paulus, J., Reckhaus, M., et al.: Johnny: an autonomous service robot for domestic environments. J. Intell. Robot. Syst. 66(1–2), 245–272 (2012)CrossRefGoogle Scholar
  9. 9.
    Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P., Dollar, A.M.: Benchmarking in manipulation research: the YCB object and model set and benchmarking protocols. arXiv preprint arXiv:1502.03143 (2015)
  10. 10.
    Ciocarlie, M.T., Allen, P.K.: Hand posture subspaces for dexterous robotic grasping. Int. J. Robot. Res. 28(7), 851–867 (2009)CrossRefGoogle Scholar
  11. 11.
    Coleman, D., Sucan, I., Chitta, S., Correll, N.: Reducing the barrier to entry of complex robotic software: a moveit! case-study. J. Softw. Eng. Robot. Spec. Issue Best Pract. Robot Softw. Dev. 5(1), 3–16 (2014).
  12. 12.
    Correll, N., Arechiga, N., Bolger, A., Bollini, M., Charrow, B., Clayton, A., Dominguez, F., Donahue, K., Dyar, S., Johnson, L., et al.: Indoor robot gardening: design and implementation. Intel. Serv. Robot. 3(4), 219–232 (2010)CrossRefGoogle Scholar
  13. 13.
    Correll, N., Bekris, K.E., Berenson, D., Brock, O., Causo, A., Hauser, K., Okada, K., Rodriguez, A., Romano, J.M., Wurman, P.R.: Analysis and observations from the first amazon picking challenge. IEEE Trans. Autom. Sci. Eng. (2016)Google Scholar
  14. 14.
    Cox, R., Correll, N.: Merging local and global 3D perception using contact sensing. In: AAAI Spring Symposium on Interactive Multi-Sensory Object Perception for Embodied Agents, Stanford, CA (2017)Google Scholar
  15. 15.
    Cutkosky, M.R., Howe, R.D.: Human grasp choice and robotic grasp analysis. In: Venkataraman, S.T., Iberall, T. (eds.) Dextrous Robot Hands, pp. 5–31. Springer, New York (1990). Scholar
  16. 16.
    Deimel, R., Brock, O.: A novel type of compliant and underactuated robotic hand for dexterous grasping. Int. J. Robot. Res. (2015).
  17. 17.
    Dollar, A.M., Howe, R.D.: The highly adaptive SDM hand: design and performance evaluation. Int. J. Robot. Res. 29(5), 585–597 (2010)CrossRefGoogle Scholar
  18. 18.
    Dune, C., Marchand, E., Collowet, C., Leroux, C.: Active rough shape estimation of unknown objects. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3622–3627. IEEE (2008)Google Scholar
  19. 19.
    Farrow, N., Li, Y., Correll, N.: Morphological and embedded computation in a self-contained soft robotic hand. arXiv preprint arXiv:1605.00354 (2016)
  20. 20.
    Felip, J., Morales, A.: Robust sensor-based grasp primitive for a three-finger robot hand. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1811–1816. IEEE (2009)Google Scholar
  21. 21.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Hsiao, K., Chitta, S., Ciocarlie, M., Jones, E.G.: Contact-reactive grasping of objects with partial shape information. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1228–1235. IEEE (2010)Google Scholar
  23. 23.
    Hsiao, K., Nangeroni, P., Huber, M., Saxena, A., Ng, A.Y.: Reactive grasping using optical proximity sensors. In: IEEE International Conference on Robotics and Automation, 2009, ICRA 2009, pp. 2098–2105. IEEE (2009)Google Scholar
  24. 24.
    Knepper, R.A., Srinivasa, S.S., Mason, M.T.: Hierarchical planning architectures for mobile manipulation tasks in indoor environments. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 1985–1990. IEEE (2010)Google Scholar
  25. 25.
    Kroemer, O., Detry, R., Piater, J., Peters, J.: Combining active learning and reactive control for robot grasping. Robot. Auton. Syst. 58(9), 1105–1116 (2010)CrossRefGoogle Scholar
  26. 26.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  27. 27.
    Marton, Z.c., Pangercic, D., Blodow, N., Kleinehellefort, J., Beetz, M.: General 3D modelling of novel objects from a single view. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3700–3705. IEEE (2010)Google Scholar
  28. 28.
    Miller, A.T., Allen, P.K.: Graspit! a versatile simulator for robotic grasping. IEEE Robot. Autom. Mag. 11(4), 110–122 (2004)CrossRefGoogle Scholar
  29. 29.
    Patel, R., Canardo Alastuey, J., Correll, N.: Improving grasp performance using in-hand proximity and force sensing. In: International Symposium on Experimental Robotics (ISER), Tokyo, Japan (2016)Google Scholar
  30. 30.
    Patel, R., Correll, N.: Integrated force and distance sensing for robotic manipulation using elastomer-embedded commodity proximity sensors. In: Robotics: Science and Systems, Ann Arbor, MN (2016)Google Scholar
  31. 31.
    Rao, D., Le, Q.V., Phoka, T., Quigley, M., Sudsang, A., Ng, A.Y.: Grasping novel objects with depth segmentation. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2578–2585. IEEE (2010)Google Scholar
  32. 32.
    Romano, J.M., Hsiao, K., Niemeyer, G., Chitta, S., Kuchenbecker, K.J.: Human-inspired robotic grasp control with tactile sensing. IEEE Trans. Rob. 27(6), 1067–1079 (2011)CrossRefGoogle Scholar
  33. 33.
    Rusu, R.B., Bradski, G., Thibaux, R., Hsu, J.: Fast 3D recognition and pose using the viewpoint feature histogram. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2155–2162. IEEE (2010)Google Scholar
  34. 34.
    Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–4. IEEE (2011)Google Scholar
  35. 35.
    Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Robot. Res. 27(2), 157–173 (2008)CrossRefGoogle Scholar
  36. 36.
    Srinivasa, S.S., Ferguson, D., Helfrich, C.J., Berenson, D., Collet, A., Diankov, R., Gallagher, G., Hollinger, G., Kuffner, J., Weghe, M.V.: Herb: a home exploring robotic butler. Auton. Robots 28(1), 5–20 (2010)CrossRefGoogle Scholar
  37. 37.
    Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 356–369. Springer, Heidelberg (2010). Scholar
  38. 38.
    Villena-Martínez, V., Fuster-Guilló, A., Azorín-López, J., Saval-Calvo, M., Mora-Pascual, J., Garcia-Rodriguez, J., Garcia-Garcia, A.: A quantitative comparison of calibration methods for RGB-D sensors using different technologies. Sensors 17(2), 243 (2017)CrossRefGoogle Scholar
  39. 39.
    Weisz, J., Allen, P.K.: Pose error robust grasping from contact wrench space metrics. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 557–562. IEEE (2012)Google Scholar
  40. 40.
    Zhang, L., Trinkle, J.C.: The application of particle filtering to grasping acquisition with visual occlusion and tactile sensing. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 3805–3812. IEEE (2012)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Radhen Patel
    • 1
  • Rebeca Curtis
    • 1
  • Branden Romero
    • 1
  • Nikolaus Correll
    • 1
    Email author
  1. 1.University of Colorado BoulderBoulderUSA

Personalised recommendations