Skip to main content
Book cover

International Symposium on Experimental Robotics

ISER 2020: Experimental Robotics pp 611–621Cite as

Nonprehensile Riemannian Motion Predictive Control

  • 939 Accesses

Part of the Springer Proceedings in Advanced Robotics book series (SPAR,volume 19)

Abstract

Nonprehensile manipulation involves long horizon underactuated object interactions and physical contact with different objects that can inherently introduce a high degree of uncertainty. In this work, we introduce a novel Real-to-Sim reward analysis technique, called Riemannian Motion Predictive Control (RMPC), to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. Our proposed RMPC benefits from Riemannian motion policy and second order dynamic model to compute the acceleration command and control the robot at every location on the surface. Our approach creates a 3D object-level recomposed model of the real scene where we can simulate the effect of different trajectories. We produce a closed-loop controller to reactively push objects in a continuous action space. We evaluate the performance of our RMPC approach by conducting experiments on a real robot platform as well as simulation and compare against several baselines. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-71151-1_54
  • Chapter length: 11 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   269.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-71151-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   349.99
Price excludes VAT (USA)
Hardcover Book
USD   349.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

References

  1. The MIT RACECAR (2016). https://mit-racecar.github.io

  2. Agrawal, P., Nair, A.V., Abbeel, P., Malik, J., Levine, S.: Learning to poke by poking: experiential learning of intuitive physics. In: NeurIPS (2016)

    Google Scholar 

  3. Andrychowicz, O.M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., et al.: Learning dexterous in-hand manipulation. Int. J. Rob. Res. 39, 3–20 (2020)

    CrossRef  Google Scholar 

  4. Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K., et al.: Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In: ICRA (2018)

    Google Scholar 

  5. Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P., Dollar, A.M.: Benchmarking in manipulation research: the YCB object and model set and benchmarking protocols. arXiv:1502.03143 (2015)

  6. Cheng, C.A., Mukadam, M., Issac, J., Birchfield, S., Fox, D., Boots, B., Ratliff, N.: RMPflow: a computational graph for automatic motion policy generation. In: International Workshop on the Algorithmic Foundations of Robotics (2018)

    Google Scholar 

  7. Cosgun, A., Hermans, T., Emeli, V., Stilman, M.: Push planning for object placement on cluttered table surfaces. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2011)

    Google Scholar 

  8. Coumans, E.: Bullet physics simulation. In: ACM SIGGRAPH 2015 Courses (2015)

    Google Scholar 

  9. Hermans, T., Li, F., Rehg, J.M., Bobick, A.F.: Learning contact locations for pushing and orienting unknown objects. In: IEEE-RAS International Conference on Humanoid Robots (Humanoids) (2013)

    Google Scholar 

  10. Izadinia, H., Seitz, S.M.: Scene recomposition by learning-based ICP. In: CVPR (2020)

    Google Scholar 

  11. James, S., Davison, A.J., Johns, E.: Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task. arXiv:1707.02267 (2017)

  12. James, S., Wohlhart, P., Kalakrishnan, M., Kalashnikov, D., Irpan, A., Ibarz, J., Levine, S., Hadsell, R., Bousmalis, K.: Sim-to-real via sim-to-sim: data-efficient robotic grasping via randomized-to-canonical adaptation networks. In: CVPR (2019)

    Google Scholar 

  13. Johnson, A.M., King, J.E., Srinivasa, S.: Convergent planning. IEEE Rob. Autom. Lett. (2016)

    Google Scholar 

  14. Killpack, M.D., Kapusta, A., Kemp, C.C.: Model predictive control for fast reaching in clutter. Auton. Rob. 40, 537–560 (2016)

    Google Scholar 

  15. Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. 17, 1334–1373 (2016)

    Google Scholar 

  16. Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Rob. Res. 37, 421–436 (2018)

    Google Scholar 

  17. Li, J.K., Lee, W.S., Hsu, D.: Push-Net: deep planar pushing for objects with unknown physical properties. In: Robotics: Science and Systems (2018)

    Google Scholar 

  18. Lynch, K.M., Maekawa, H., Tanie, K.: Manipulation and active sensing by pushing using tactile feedback. In: IROS (1992)

    Google Scholar 

  19. Lynch, K.M., Mason, M.T.: Dynamic underactuated nonprehensile manipulation. In: IROS (1996)

    Google Scholar 

  20. Mason, M.T.: Compliance and force control for computer controlled manipulators. IEEE Trans. Syst. Man Cybern. 11, 418–432 (1981)

    Google Scholar 

  21. Nair, A., Chen, D., Agrawal, P., Isola, P., Abbeel, P., Malik, J., Levine, S.: Combining self-supervised learning and imitation for vision-based rope manipulation. In: ICRA (2017)

    Google Scholar 

  22. Olson, E.: AprilTag: a robust and flexible visual fiducial system. In: ICRA (2011)

    Google Scholar 

  23. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS (2019)

    Google Scholar 

  24. Rusu, A.A., Večerík, M., Rothörl, T., Heess, N., Pascanu, R., Hadsell, R.: Sim-to-Real robot learning from pixels with progressive nets. In: CoRL (2017)

    Google Scholar 

  25. Sadeghi, F.: DIViS: Domain invariant visual servoing for collision-free goal reaching. In: RSS (2019)

    Google Scholar 

  26. Sadeghi, F., Levine, S.: CAD2RL: real single-image flight without a single real image. In: RSS (2017)

    Google Scholar 

  27. Sadeghi, F., Toshev, A., Jang, E., Levine, S.: Sim2real viewpoint invariant visual servoing by recurrent control. In: CVPR (2018)

    Google Scholar 

  28. Srinivasa, S.S., Lancaster, P., Michalove, J., Schmittle, M., Rockett, C.S.M., Smith, J.R., Choudhury, S., Mavrogiannis, C., Sadeghi, F.: MuSHR: a low-cost, open-source robotic racecar for education and research. arXiv preprint arXiv:1908.08031 (2019)

  29. Stilman, M., Kuffner, J.J.: Navigation among movable obstacles: real-time reasoning in complex environments. Int. J. Humanoid Rob. 2, 479–503 (2005)

    Google Scholar 

  30. Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., To, T., Cameracci, E., Boochoon, S., Birchfield, S.: Training deep networks with synthetic data: bridging the reality gap by domain randomization. In: CVPRW (2018)

    Google Scholar 

  31. Wang, Y., Boyd, S.: Fast model predictive control using online optimization. IEEE Trans. Control Syst. Technol. 18, 267–278 (2009)

    CrossRef  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the University of Washington Animation Research Labs and Google.

Author information

Authors and Affiliations

  1. University of Washington, Seattle, USA

    Hamid Izadinia, Byron Boots & Steven M. Seitz

Authors
  1. Hamid Izadinia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Byron Boots
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steven M. Seitz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamid Izadinia .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy

    Bruno Siciliano

  2. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Cecilia Laschi

  3. Department of Computer Science, Stanford University, Stanford, CA, USA

    Prof. Oussama Khatib

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Izadinia, H., Boots, B., Seitz, S.M. (2021). Nonprehensile Riemannian Motion Predictive Control. In: Siciliano, B., Laschi, C., Khatib, O. (eds) Experimental Robotics. ISER 2020. Springer Proceedings in Advanced Robotics, vol 19. Springer, Cham. https://doi.org/10.1007/978-3-030-71151-1_54

Download citation