Abstract
Learning to act in unstructured environments such as cluttered piles of objects poses a substantial challenge for manipulation robots. We present a novel neural network-based approach that separates unknown objects in clutter by selecting favourable push actions. Our network is trained from data collected through autonomous interaction of a PR2 robot with randomly organized tabletop scenes. The model is designed to propose meaningful push actions based on over-segmented RGB-D images. We evaluate our approach by singulating up to 8 unknown objects in clutter. We demonstrate that our method enables the robot to perform the task with a high success rate and a low number of required push actions. Our results based on real-world experiments show that our network is able to generalize to novel objects of various sizes and shapes as well as to arbitrary object configurations. Videos of our experiments can be viewed at http://robotpush.cs.uni-freiburg.de.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agrawal, P., Nair, A.V., Abbeel, P., Malik, J., Levine, S.: Learning to poke by poking: Experiential learning of intuitive physics. In: Advances in Neural Information Processing Systems (NIPS) (2016)
Bohg, J., Hausman, K., Sankaran, B., Brock, O., Kragic, D., Schaal, S., Sukhatme, G.: Interactive perception: leveraging action in perception and perception in action. arXiv:1604.03670 (2016)
Boularias, A., Bagnell, J.A.D., Stentz, A.T.: Learning to manipulate unknown objects in clutter by reinforcement. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI) (2015)
Byravan, A., Fox, D.: Se3-nets: learning rigid body motion using deep neural networks. arXiv:1606.02378 (2016)
Chang, L., Smith, J.R., Fox, D.: Interactive singulation of objects from a pile. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2012)
Cosgun, A., Hermans, T., Emeli, V., Stilman, M.: Push planning for object placement on cluttered table surfaces. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2011)
Dogar, M., Srinivasa, S.: A framework for push-grasping in clutter. In: Robotics: Science and Systems VII, vol. 1 (2011)
Finn, C., Levine, S.: Deep visual foresight for planning robot motion. arXiv:1610.00696 (2016)
Gualtieri, M., ten Pas, A., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE (2016)
Gupta, M., Müller, J., Sukhatme, G.S.: Using manipulation primitives for object sorting in cluttered environments. IEEE Trans. Autom. Sci. Eng.12(2) (2015)
Hausman, K., Balint-Benczedi, F., Pangercic, D., Marton, Z.C., Ueda, R., Okada, K., Beetz, M.: Tracking-based interactive segmentation of textureless objects. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2013)
Hermans, T., Li, F., Rehg, J.M., Bobick, A.F.: Learning contact locations for pushing and orienting unknown objects. In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids) (2013)
Hermans, T., Rehg, J.M., Bobick, A.: Guided pushing for object singulation. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2012)
Katz, D., Venkatraman, A., Kazemi, M., Bagnell, J.A., Stentz, A.: Perceiving, learning, and exploiting object affordances for autonomous pile manipulation. Auton. Robot. (2014)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
Koo, S., Lee, D., Kwon, D.S.: Unsupervised object individuation from rgb-d image sequences. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2014)
Kopicki, M., Zurek, S., Stolkin, R., Mörwald, T., Wyatt, J.: Learning to predict how rigid objects behave under simple manipulation. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2011)
Laskey, M., Chuck, C., Lee, J., Mahler, J., Krishnan, S., Jamieson, K., Dragan, A., Goldberg, K.: Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2017)
Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. 17(39) (2016)
Levine, S., Pastor, P., Krizhevsky, A., Quillen, D.: Learning hand-eye coordination for robotic grasping with large-scale data collection. In: International Symposium on Experimental Robotics (ISER) (2016)
Lynch, K.M., Mason, M.T.: Stable pushing: mechanics, controllability, and planning. Int. J. Robot. Res. 15(6) (1996)
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.A., Goldberg, K.: Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv:1703.09312 (2017)
Mees, O., Abdo, N., Mazuran, M., Burgard, W.: Metric learning for generalizing spatial relations to new objects. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)
Pinto, L., Gandhi, D., Han, Y., Park, Y.L., Gupta, A.: The curious robot: learning visual representations via physical interactions. arXiv:1604.01360 (2016)
Pinto, L., Gupta, A.: Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2016)
Richtsfeld, A., Mörwald, T., Prankl, J., Zillich, M., Vincze, M.: Segmentation of unknown objects in indoor environments. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2012)
Schiebener, D., Ude, A., Morimoto, J., Asfour, T., Dillmann, R.: Segmentation and learning of unknown objects through physical interaction. In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids) (2011)
Sucan, I.A., Moll, M., Kavraki, L.E.: The open motion planning library. IEEE Robot. Autom. Mag. 19(4), 72–82 (2012)
Van Hoof, H., Kroemer, O., Peters, J.: Probabilistic segmentation and targeted exploration of objects in cluttered environments. IEEE Trans. Robot.30(5) (2014)
Yu, K.T., Bauza, M., Fazeli, N., Rodriguez, A.: More than a million ways to be pushed. a high-fidelity experimental dataset of planar pushing. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016)
Zhou, J., Paolini, R., Bagnell, J.A., Mason, M.T.: A convex polynomial force-motion model for planar sliding: identification and application. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2016)
Zhu, S., Kimmel, A., Bekris, K.E., Boularias, A.: Model identification via physics engines for improved policy search. arXiv:1710.08893 (2017)
Acknowledgements
This work was partially funded by the German Research Foundation under the priority program Autonomous Learning SPP 1527 and under grant number EXC 108. We thank Seongyong Koo for advice with the baseline method. We further thank Sudhanshu Mittal, Oier Mees and Tim Welschehold for their help and ideas.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Eitel, A., Hauff, N., Burgard, W. (2020). Learning to Singulate Objects Using a Push Proposal Network. In: Amato, N., Hager, G., Thomas, S., Torres-Torriti, M. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 10. Springer, Cham. https://doi.org/10.1007/978-3-030-28619-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-28619-4_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28618-7
Online ISBN: 978-3-030-28619-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)