Learning from Experience in Manipulation Planning: Setting the Right Goals

  • Anca D. Dragan
  • Geoffrey J. Gordon
  • Siddhartha S. Srinivasa
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 100)


In this paper, we describe a method of improving trajectory optimization based on predicting good initial guesses from previous experiences. In order to generalize to new situations, we propose a paradigm shift: predicting qualitative attributes of the trajectory that place the initial guess in the basin of attraction of a low-cost solution. We start with a key such attribute, the choice of a goal within a goal set that describes the task, and show the generalization capabilities of our method in extensive experiments on a personal robotics platform.


Target Object Optimal Trajectory Initial Trajectory Good Goal Trajectory Prediction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This material is based upon work supported by DARPA-BAA-10-28, NSF-IIS-0916557, and NSF- EEC-0540865. Thanks to Chris Atkeson and the members of the Personal Robotics Lab for comments and fruitful discussions.


  1. 1.
    A.D. Dragan, N. Ratliff, S.S. Srinivasa, Manipulation planning with goal sets using constrained trajectory optimization, in IEEE International Conference on Robotics and Automation (2011)Google Scholar
  2. 2.
    N. Ratliff, M. Zucker, J.A. Bagnell, S. Srinivasa, CHOMP: gradient optimization techniques for efficient motion planning, in IEEE ICRA (2009), pp. 489–494Google Scholar
  3. 3.
    M.M. Veloso, Ph.D. Learning by analogical reasoning in general problem solving (CMU-CS-92-174) (1992)Google Scholar
  4. 4.
    R. Lopez De Mantaras, D. Mcsherry, D. Bridge, D. Leake, B. Smyth, S. Craw, B. Faltings, Retrieval, reuse, revision and retention in case-based reasoning. Knowl. Eng. Rev. 20(03), 215 (2006)CrossRefGoogle Scholar
  5. 5.
    G. Konidaris, A. Barto, Autonomous shaping: knowledge transfer in reinforcement learning, in Proceedings of the 23rd International Conference on Machine Learning (2006), pp. 489–496Google Scholar
  6. 6.
    M. Stolle, C.G. Atkeson, Policies based on trajectory libraries, in IEEE ICRA (May) (2006), pp. 3344–3349Google Scholar
  7. 7.
    M. Stolle, C.G. Atkeson, Knowledge transfer using local features, in IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning. ADPRL 2007 (2007)Google Scholar
  8. 8.
    M. Stolle, H. Tappeiner, J. Chestnutt, C.G. Atkeson, Transfer of policies based on trajectory libraries, in IEEE/RSJ International Conference on Intelligent Robots and Systems (2007), pp. 2981–2986Google Scholar
  9. 9.
    M. Branicky, R. Knepper, J. Kuffner, Path and trajectory diversity: theory and algorithms, in IEEE International Conference on Robotics and Automation (ICRA) (2008), pp. 1359–1364Google Scholar
  10. 10.
    S. Martin, S. Wright, J. Sheppard, Offline and online evolutionary bi-directional RRT algorithms for efficient re-planning in dynamic environments, in IEEE CASE (2007), pp. 1131–1136Google Scholar
  11. 11.
    N. Jetchev, M. Toussaint, Trajectory prediction, in Proceedings of the 26th Annual International Conference on Machine Learning—ICML 09 (ACM Press, New York, USA, 2009), pp. 1–8Google Scholar
  12. 12.
    D. Dey, T. Liu, B. Sofman, J. Bagnell, Efficient Optimization of Control Libraries. Technical Report (CMU-RI-TR-11-20) (2011)Google Scholar
  13. 13.
    N. Jetchev, M. Toussaint, Trajectory prediction in cluttered voxel environments, in IEEE International Conference on Robotics and Automation—ICRA2010, Anchorage, AK, USA (2010)Google Scholar
  14. 14.
    D.G. Lowe, Object recognition from local scale-invariant features, in Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2(8), pp. 1150–1157 (1999)Google Scholar
  15. 15.
    A. Frome, Y. Singer, J. Malik, Image retrieval and classification using local distance functions, in Advances in Neural Information Processing Systems (NIPS) (2006)Google Scholar
  16. 16.
    C. Lampert, H. Nickisch, S. Harmeling, Learning to detect unseen object classes by between-class attribute transfer, in IEEE Conference on Computer Vision and Pattern Recognition (2009), pp. 951–958Google Scholar
  17. 17.
    A. Farhadi, I. Endres, D. Hoiem, D. Forsyth, Describing objects by their attributes, in IEEE Conference on Computer Vision and Pattern Recognition (2009), pp. 1778–1785Google Scholar
  18. 18.
    M. Palatucci, G. Hinton, D. Pomerleau, T.M. Mitchell, Zero-shot learning with semantic output codes. Adv. Neural Inf. Process. Syst. 22, 1–9 (2009)Google Scholar
  19. 19.
    D. Berenson, S.S. Srinivasa, D. Ferguson, A. Collet, J.J. Kuffner, Manipulation planning with workspace goal regions, in Proceedings of the IEEE ICRA (2009), pp. 618–624Google Scholar
  20. 20.
    J. Blitzer, H. Daume, ICML Tutorial on Domain Adaptation,
  21. 21.
    N. D. Ratliff, J.A. Bagnell, M.A. Zinkevich, Maximum margin planning, in Proceedings of the 23rd International Conference on Machine Learning ICML 06 (2006), pp. 729–736Google Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Anca D. Dragan
    • 1
  • Geoffrey J. Gordon
    • 2
  • Siddhartha S. Srinivasa
    • 1
  1. 1.The Robotics InstituteCarnegie Mellon UniversityPittsburghUSA
  2. 2.Machine Learning DepartmentCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations