Abstract
One of the most important skills for service robots is object manipulation, which is still a challenging task. Since object manipulation is a hard task, it is relevant to know if an object was successfully grasped, avoiding future wrong decisions. Object grasp confirmation is commonly solved by using robotic sensors (infrared, pressure, etc.), but, in many cases, these sensors are not available for all robots. In contrast, depth and RGB sensor are present in almost all service robots. In this work a novel computer vision based method oriented to hand object grasp confirmation is proposed, which uses a deep learning network trained with depth maps. In order to measure the performance of the proposed method, experiments were performed using a single-arm manipulator service robot for both known and unknown objects. Experimental results show that the proposed approach correctly identifies 99% of both classes (object grasped or not grasped) with known objects and \(92\%\) with unknown objects. The grasping confirmation method was added to the Storing Groceries task, for RoboCup@Home competition, improving its time performance.
Sebastián Salazar-Colores (CVU 477758) would like to thank CONACYT (Consejo Nacional de Ciencia y Tecnología) for the financial support of his Ph.D. studies under Scholarship 285651.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A video demonstration is accessible through https://youtu.be/yA5_kS3FlUo.
- 2.
- 3.
Service robot performing grasp confirmation in Storing Groceries partial task https://youtu.be/giTvoMBa1Yo.
References
Allen, P.K., Timcenko, A., Yoshimi, B., Michelman, P.: Automated tracking and grasping of a moving object with a robotic hand-eye system. IEEE Trans. Robot. Autom. 9(2), 152–165 (1993). https://doi.org/10.1109/70.238279
Allen, P., Miller, A., Oh, P., Leibowitz, B.: Using tactile and visual sensing with a robotic hand. In: Proceedings of International Conference on Robotics and Automation, vol. 1, pp. 676–681. IEEE (1997). https://doi.org/10.1109/robot.1997.620114
Allen, P., Miller, A., Oh, P., Leibowitz, B.: Integration of vision, force and tactile sensing for grasping. Int. J. Intell. Mach. 4, 129–149 (1999)
Bicchi, A., Kumar, V.: Robotic grasping and contact: a review. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), vol. 1, pp. 348–353. IEEE, April 2000. https://doi.org/10.1109/ROBOT.2000.844081
Dang, H., Allen, P.K.: Learning grasp stability. In: 2012 IEEE International Conference on Robotics and Automation, pp. 2392–2397. IEEE (2012)
Guo, D., Sun, F., Fang, B., Yang, C., Xi, N.: Robotic grasping using visual and tactile sensing. Inf. Sci. 417, 274–286 (2017). https://doi.org/10.1016/j.ins.2017.07.017. http://www.sciencedirect.com/science/article/pii/S002002551730837X
Hebert, P., Hudson, N., Ma, J., Burdick, J.: Fusion of stereo vision, force-torque, and joint sensors for estimation of in-hand object location. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 5935–5941. IEEE (2011). https://doi.org/10.1109/ICRA.2011.5980185
Heidemann, G., Ritter, H.: Visual checking of grasping positions of a three-fingered robot hand. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) ICANN 2001. LNCS, vol. 2130, pp. 891–898. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44668-0_123
Jara, C.A., Pomares, J., Candelas, F.A., Torres, F.: Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors’ feedback. Sensors 14(1), 1787–1804 (2014)
Konstantinova, J., Stilli, A., Faragasso, A., Althoefer, K.: Fingertip proximity sensor with realtime visual-based calibration. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 170–175, October 2016. https://doi.org/10.1109/IROS.2016.7759051
Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulator and object tracking for in-hand 3D object modeling. Int. J. Robot. Res. 30(11), 1311–1327 (2011). https://doi.org/10.1177/0278364911403178
LeCun, Y., et al.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, pp. 396–404 (1990)
Patel, R., Curtis, R., Romero, B., Correll, N.: Improving grasp performance using in-hand proximity and contact sensing. In: Sun, Y., Falco, J. (eds.) RGMC 2016. CCIS, vol. 816, pp. 146–160. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94568-2_9
Redmon, J., Angelova, A.: Real-time grasp detection using convolutional neural networks. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 1316–1322. IEEE (2015). https://doi.org/10.1109/ICRA.2015.7139361
Roa, M.A., Suárez, R.: Grasp quality measures: review and performance. Auton. Robot. 38(1), 65–88 (2015). https://doi.org/10.1007/s10514-014-9402-3
System, R.O., June 2019. www.ros.org
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Salazar-Colores, S., Méndez-Molina, A., Carrillo-López, D., Escobar-Juárez, E., Morales, E.F., Sucar, L.E. (2019). A Fast and Robust Deep Learning Approach for Hand Object Grasping Confirmation. In: Martínez-Villaseñor, L., Batyrshin, I., Marín-Hernández, A. (eds) Advances in Soft Computing. MICAI 2019. Lecture Notes in Computer Science(), vol 11835. Springer, Cham. https://doi.org/10.1007/978-3-030-33749-0_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-33749-0_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33748-3
Online ISBN: 978-3-030-33749-0
eBook Packages: Computer ScienceComputer Science (R0)