Advertisement

Learning to Singulate Objects Using a Push Proposal Network

  • Andreas EitelEmail author
  • Nico Hauff
  • Wolfram Burgard
Conference paper
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 10)

Abstract

Learning to act in unstructured environments such as cluttered piles of objects poses a substantial challenge for manipulation robots. We present a novel neural network-based approach that separates unknown objects in clutter by selecting favourable push actions. Our network is trained from data collected through autonomous interaction of a PR2 robot with randomly organized tabletop scenes. The model is designed to propose meaningful push actions based on over-segmented RGB-D images. We evaluate our approach by singulating up to 8 unknown objects in clutter. We demonstrate that our method enables the robot to perform the task with a high success rate and a low number of required push actions. Our results based on real-world experiments show that our network is able to generalize to novel objects of various sizes and shapes as well as to arbitrary object configurations. Videos of our experiments can be viewed at  http://robotpush.cs.uni-freiburg.de.

Notes

Acknowledgements

This work was partially funded by the German Research Foundation under the priority program Autonomous Learning SPP 1527 and under grant number EXC 108. We thank Seongyong Koo for advice with the baseline method. We further thank Sudhanshu Mittal, Oier Mees and Tim Welschehold for their help and ideas.

References

  1. 1.
    Agrawal, P., Nair, A.V., Abbeel, P., Malik, J., Levine, S.: Learning to poke by poking: Experiential learning of intuitive physics. In: Advances in Neural Information Processing Systems (NIPS) (2016)Google Scholar
  2. 2.
    Bohg, J., Hausman, K., Sankaran, B., Brock, O., Kragic, D., Schaal, S., Sukhatme, G.: Interactive perception: leveraging action in perception and perception in action. arXiv:1604.03670 (2016)
  3. 3.
    Boularias, A., Bagnell, J.A.D., Stentz, A.T.: Learning to manipulate unknown objects in clutter by reinforcement. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI) (2015)Google Scholar
  4. 4.
    Byravan, A., Fox, D.: Se3-nets: learning rigid body motion using deep neural networks. arXiv:1606.02378 (2016)
  5. 5.
    Chang, L., Smith, J.R., Fox, D.: Interactive singulation of objects from a pile. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2012)Google Scholar
  6. 6.
    Cosgun, A., Hermans, T., Emeli, V., Stilman, M.: Push planning for object placement on cluttered table surfaces. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2011)Google Scholar
  7. 7.
    Dogar, M., Srinivasa, S.: A framework for push-grasping in clutter. In: Robotics: Science and Systems VII, vol. 1 (2011)Google Scholar
  8. 8.
    Finn, C., Levine, S.: Deep visual foresight for planning robot motion. arXiv:1610.00696 (2016)
  9. 9.
    Gualtieri, M., ten Pas, A., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE (2016)Google Scholar
  10. 10.
    Gupta, M., Müller, J., Sukhatme, G.S.: Using manipulation primitives for object sorting in cluttered environments. IEEE Trans. Autom. Sci. Eng.12(2) (2015)Google Scholar
  11. 11.
    Hausman, K., Balint-Benczedi, F., Pangercic, D., Marton, Z.C., Ueda, R., Okada, K., Beetz, M.: Tracking-based interactive segmentation of textureless objects. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2013)Google Scholar
  12. 12.
    Hermans, T., Li, F., Rehg, J.M., Bobick, A.F.: Learning contact locations for pushing and orienting unknown objects. In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids) (2013)Google Scholar
  13. 13.
    Hermans, T., Rehg, J.M., Bobick, A.: Guided pushing for object singulation. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2012)Google Scholar
  14. 14.
    Katz, D., Venkatraman, A., Kazemi, M., Bagnell, J.A., Stentz, A.: Perceiving, learning, and exploiting object affordances for autonomous pile manipulation. Auton. Robot. (2014)Google Scholar
  15. 15.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
  16. 16.
    Koo, S., Lee, D., Kwon, D.S.: Unsupervised object individuation from rgb-d image sequences. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2014)Google Scholar
  17. 17.
    Kopicki, M., Zurek, S., Stolkin, R., Mörwald, T., Wyatt, J.: Learning to predict how rigid objects behave under simple manipulation. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2011)Google Scholar
  18. 18.
    Laskey, M., Chuck, C., Lee, J., Mahler, J., Krishnan, S., Jamieson, K., Dragan, A., Goldberg, K.: Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2017)Google Scholar
  19. 19.
    Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. 17(39) (2016)Google Scholar
  20. 20.
    Levine, S., Pastor, P., Krizhevsky, A., Quillen, D.: Learning hand-eye coordination for robotic grasping with large-scale data collection. In: International Symposium on Experimental Robotics (ISER) (2016)Google Scholar
  21. 21.
    Lynch, K.M., Mason, M.T.: Stable pushing: mechanics, controllability, and planning. Int. J. Robot. Res. 15(6) (1996)Google Scholar
  22. 22.
    Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.A., Goldberg, K.: Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv:1703.09312 (2017)
  23. 23.
    Mees, O., Abdo, N., Mazuran, M., Burgard, W.: Metric learning for generalizing spatial relations to new objects. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)Google Scholar
  24. 24.
    Pinto, L., Gandhi, D., Han, Y., Park, Y.L., Gupta, A.: The curious robot: learning visual representations via physical interactions. arXiv:1604.01360 (2016)
  25. 25.
    Pinto, L., Gupta, A.: Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2016)Google Scholar
  26. 26.
    Richtsfeld, A., Mörwald, T., Prankl, J., Zillich, M., Vincze, M.: Segmentation of unknown objects in indoor environments. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2012)Google Scholar
  27. 27.
    Schiebener, D., Ude, A., Morimoto, J., Asfour, T., Dillmann, R.: Segmentation and learning of unknown objects through physical interaction. In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids) (2011)Google Scholar
  28. 28.
    Sucan, I.A., Moll, M., Kavraki, L.E.: The open motion planning library. IEEE Robot. Autom. Mag. 19(4), 72–82 (2012)CrossRefGoogle Scholar
  29. 29.
    Van Hoof, H., Kroemer, O., Peters, J.: Probabilistic segmentation and targeted exploration of objects in cluttered environments. IEEE Trans. Robot.30(5) (2014)Google Scholar
  30. 30.
    Yu, K.T., Bauza, M., Fazeli, N., Rodriguez, A.: More than a million ways to be pushed. a high-fidelity experimental dataset of planar pushing. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016)Google Scholar
  31. 31.
    Zhou, J., Paolini, R., Bagnell, J.A., Mason, M.T.: A convex polynomial force-motion model for planar sliding: identification and application. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2016)Google Scholar
  32. 32.
    Zhu, S., Kimmel, A., Bekris, K.E., Boularias, A.: Model identification via physics engines for improved policy search. arXiv:1710.08893 (2017)

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of FreiburgFreiburgGermany

Personalised recommendations