Skip to main content
SpringerLink
Log in

An autonomous manipulation system based on force control and optimization

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

In this paper we present an architecture for autonomous manipulation. Our approach is based on the belief that contact interactions during manipulation should be exploited to improve dexterity and that optimizing motion plans is useful to create more robust and repeatable manipulation behaviors. We therefore propose an architecture where state of the art force/torque control and optimization-based motion planning are the core components of the system. We give a detailed description of the modules that constitute the complete system and discuss the challenges inherent to creating such a system. We present experimental results for several grasping and manipulation tasks to demonstrate the performance and robustness of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. A detailed 3-D scan of nine of these objects was provided to each team and for the remaining three objects each team was only informed about the class of object.

References

  • An, C., Atkeson, C., & Hollerbach, J. (1985). Estimation of inertial parameters of rigid body links of manipulators. In IEEE Conference on Decision and Control (pp. 990–995).

  • Asfour, T., Regenstein, K., Azad, P., Schroder, J., Bierbaum, A., Vahrenkamp, N., et al. (2006). Armar-III: An integrated humanoid platform for sensory–motor control. In IEEE-RAS International Conference on Humanoid Robots (pp. 169–175).

  • Atkeson, C., An, C., & Hollerbach, J. (1985). Rigid body load identification for manipulators. In IEEE Conference on Decision and Control (pp. 996–1002).

  • Axelrod, B., & Huang, W. (2012). Improving hand-eye calibration for robotic grasping and manipulation. In IEEE International Conference on Technologies for Practical Robot Applications (pp. 121–126).

  • Bagnell, J. A. D., Cavalcanti, F., Cui, L., Galluzzo, T., Hebert, M., Kazemi, M., et al. (2012). An integrated system for autonomous robotics manipulation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 2955–2962).

  • Balasubramanian, R., Xu, L., Brook, P. D., Smith, J. R., & Matsuoka, Y. (2012). Physical human interactive guidance: Identifying grasping principles from human-planned grasps. IEEE Transactions on Robotics, 28(4), 899–910.

    Article  Google Scholar 

  • Beetz, M., Klank, U., Kresse, I., Maldonado, A., Mösenlechner, L., Pangercic, D., et al. (2011). Robotic roommates making pancakes. In IEEE-RAS International Conference on Humanoid Robots.

  • Berenson, D., Srinivasa, S. S., Ferguson, D., Collet, A., & Kuffner, J. (2009). Manipulation planning with workspace goal regions. In IEEE International Conference on Robotics and Automation.

  • Bicchi, A. (1995). On the closure-properties of robotic grasping. The International Journal of Robotics Research, 14(4), 319–334.

    Article  Google Scholar 

  • Bicchi, A., & Kumar, V. (2000). Robotic grasping and contact: A review. In IEEE International Conference on Robotics and Automation (pp. 348–353).

  • Bohg, J., & Kragic, D. (2010). Learning grasping points with shape context. Robotics and Autonomous Systems, 58, 362–377.

    Article  Google Scholar 

  • Bohg, J., Morales, A., Asfour, T., Kragic, D. (2013). Data-driven grasp synthesis: A survey. In IEEE Transactions on Robotics (in submission).

  • Bruyninckx, H., & De Schutter, J. (1996). Specification of force-controlled actions in the “task frame formalism”: A synthesis. IEEE Transactions on Robotics and Automation, 12(4), 581–589.

    Article  Google Scholar 

  • Calinon, S., Guenter, F., & Billard, A. (2007). On learning, representing, and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 37(2), 286–298.

    Article  Google Scholar 

  • Chiaverini, S. (1997). Singularity-robust task-priority redundancy resolution for real-time kinematic control of robot manipulators. IEEE Transactions on Robotics and Automation, 13(3), 398–410.

    Article  Google Scholar 

  • Chiaverini, S., & Sciavicco, L. (1993). The parallel approach to force/position control of robotic manipulators. IEEE Transactions on Robotics and Automation, 9(4), 361–373.

    Article  Google Scholar 

  • Chiaverini, S., Siciliano, B., & Villani, L. (1999). A survey of robot interaction control schemes with experimental comparison. IEEE/ASME Transactions on Mechatronics, 4(3), 273–285.

    Article  Google Scholar 

  • Chitta, S., Jones, E., Ciocarlie, M., & Hsiao, K. (2012). Perception, planning, and execution for mobile manipulation in unstructured environments. In Special Issue on Mobile Manipulation: IEEE Robotics and Automation Magazine (Vol. 19).

  • Diankov, R., Ratliff, N., Ferguson, D., Srinivasa, S., Kuffner., J., (2008). Bispace planning: Concurrent multi-space exploration. In Robotics: Science and Systems (R:SS).

  • Featherstone, R. (1987). Robot dynamics algorithms. Boston: Kluwer Academic Publishers.

    Google Scholar 

  • Hauser, K., Bretl, T., Latombe, J. C., Harada, K., & Wilcox, B. (2008). Motion planning for legged robots on varied terrain. The International Journal of Robotics Research, 27, 1325– 1349.

    Article  Google Scholar 

  • Hauser, K., & Ng-Thow-Hing, V. (2010). Fast smoothing of manipulator trajectories using optimal bounded-acceleration shortcuts. In IEEE International Conference on Robotics and Automation.

  • Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Asfour, T., & Schaal, S. (2012). Template-based learning of grasp selection. In IEEE International Conference on Robotics and Automation (pp. 2379–2384).

  • Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Bohg, J., Asfour, T., et al. (2013). Learning of grasp selection based on shape-templates. In Autonomous Robots Journal Special Issue: Autonomous Grasping and Manipulation. doi:10.1007/s10514-013-9366-8

  • Hogan, N. (1985). Impedance control: An approach to manipulation. Journal of Dynamic Systems, Measurement, and Control (Transactions of the ASME), 107(1), 1–24.

    Google Scholar 

  • Hollerbach, J., Khalil, W., & Gautier, M. (2008). Model identification. In B. Siciliano & O. Khatib (Eds.), Springer Handbook of Robotics (Chap. 14, pp. 321–344). Berlin: Springer.

  • Hsiao, K., Chitta, S., Ciocarlie, M., & Jones, E. (2010). Contact-reactive grasping of objects with partial shape information. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1228–1235).

  • Hubert, U., Stückler, J., & Behnke, S. (2012). Bayesian calibration of the hand-eye kinematics of an anthropomorphic robot. In IEEE-RAS International Conference on Humanoid Robots.

  • Hudson, N., Howard, T., Ma, J., Jain, A., Bajracharya, M., Myint, S., et al. (2012). End-to-end dexterous manipulation with deliberate interactive estimation. In IEEE International Conference on Robotics and Automation (pp. 2371–2378).

  • Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2003). Learning attractor landscapes for learning motor primitives. In Advances in Neural Information Processing Systems (pp. 1547–1554).

  • Kaelbling, L. P., & Lozano-Pérez, T. (2011). Hierarchical task and motion planning in the now. In IEEE International Conference on Robotics and Automation (pp. 1470–1477).

  • Kalakrishnan, M., Chitta, S., Theodorou, E., Pastor, P., & Schaal, S. (2011). STOMP: Stochastic trajectory optimization for motion planning. In IEEE International Conference on Robotics and Automation (pp. 4569–4574).

  • Kalakrishnan, M., Pastor, P., Righetti, L., & Schaal, S. (2013). Learning objective functions for manipulation. In IEEE International Conference on Robotics and Automation.

  • Kalakrishnan, M., Righetti, L., Pastor, P., & Schaal, S. (2011). Learning force control policies for compliant manipulation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 4639–4644).

  • Kazemi, M., Valois, J. S., Bagnell, J. A., Pollard, N. (2012). Robust object grasping using force compliant motion primitives. In Proceedings of Robotics: Science and Systems.

  • Kröger, T., & Wahl, F. M. (2010). Online trajectory generation: Basic concepts for instantaneous reactions to unforeseen events. IEEE Transactions on Robotics, 26(1), 94–111.

    Article  Google Scholar 

  • Lengagne, S., Mathieu, P., Kheddar, A., & Yoshida, E. (2010). Generation of dynamic motions under continuous constraints: Efficient computation using B-Splines and Taylor polynomials. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 698–703).

  • Li, Y., & Pollard, N. (2005). A shape matching algorithm for synthesizing humanlike enveloping grasps. In IEEE-RAS International Conference on Humanoid Robots (pp. 442–449).

  • Majarena, A. C., Santolaria, J., Samper, D., & Aguilar, J. J. (2010). An overview of kinematic and calibration models using internal/external sensors or constraints to improve the behavior of spatial parallel mechanisms. Sensors, 10(11), 10256–10297.

    Article  Google Scholar 

  • Miller, A., & Allen, P. (2004). Graspit!: A versatile simulator for robotic grasping. IEEE Robotics and Automation Magazine, 11(4), 110–122.

    Article  Google Scholar 

  • Mordatch, I., Todorov, E., & Popović, Z. (2012). Discovery of complex behaviors through contact-invariant optimization. In Eurographics/ ACM SIGGRAPH Symposium on Computer Animation (pp. 1–8).

  • Nakanishi, J., Cory, R., Mistry, M., Peters, J., & Schaal, S. (2008). Operational space control: A theoretical and empirical comparison. The International Journal of Robotics Research, 27(6), 737–757.

    Article  Google Scholar 

  • Niekum, S., Chitta, S., Marthi, B., Osentoski, S., Barto, A. G., (2013). Incremental semantically grounded learning from demonstration. In Robotics: Science and Systems (R:SS).

  • Pastor, P., Kalakrishnan, M., Binney, J., Kelly, J., Righetti, L., Sukhatme, G., et al. (2013). Learning task error models for manipulation. In IEEE International Conference on Robotics and Automation.

  • Pastor, P., Kalakrishnan, M., Chitta, S., Theodorou, E., & Schaal, S. (2011). Skill learning and task outcome prediction for manipulation. In IEEE International Conference on Robotics and Automation (pp. 3828–3834).

  • Pastor, P., Kalakrishnan, M., Meier, F., Stulp, F., Buchli, J., Theodorou, E., et al. (2013). From dynamic movement primitives to associative skill memories. Robotics and Autonomous Systems, 61(4), 351–361.

    Article  Google Scholar 

  • Pastor, P., Kalakrishnan, M., Righetti, L., & Schaal, S. (2012). Towards associative skill memories. In IEEE-RAS International Conference on Humanoid Robots.

  • Pastor, P., Righetti, L., Kalakrishnan, M., & Schaal, S. (2011). Online movement adaptation based on previous sensor experiences. In IEEE International Conference on Robotics and Automation (pp. 365–371).

  • Peters, J., & Schaal, S. (2008). Natural actor critic. Neurocomputing, 71(7–9), 1180–1190.

    Article  Google Scholar 

  • Pradeep, V., Konolige, K., & Berger, E. (2010). Calibrating a multi-arm multi-sensor robot: A bundle adjustment approach. In International Symposium on Experimental Robotics.

  • Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., et al. (2009). Ros: An open-source robot operating system. In IEEE International Conference on Robotics and Automation; Workshop on Open Source Software.

  • Raibert, M., & Craig, J. (1981). Hybrid position/force control of manipulator. Journal of Dynamics Systems, Measurement and Control, 103(2), 126–133.

    Article  Google Scholar 

  • Ratliff, N., Zucker, M., Bagnell, J., & Srinivasa, S. (2009). CHOMP: Gradient optimization techniques for efficient motion planning. In: IEEE International Conference on Robotics and Automation (pp. 12–17).

  • Rusu, R. B., Şucan, I. A., Gerkey, B., Chitta, S., Beetz, M., & Kavraki, L. E. (2009). Real-time perception guided motion planning for a personal robot. In IEEE/RSJ International Conference on Intelligent Robots and Systems.

  • Schaal, S., (2009). The SL simulation and real-time control software package. Technical Report. University of Southern California. http://www-clmc.usc.edu/publications/S/schaal-TRSL.pdf

  • Sciavicco, L., & Siciliano, B. (2000). Modelling and control of robot manipulators. Berlin: Springer-Verlag.

    Book  MATH  Google Scholar 

  • Srinivasa, S., Ferguson, D., Helfrich, C., Berenson, D., Romea, A. C., Diankov, R., et al. (2010). HERB: A home exploring robotic butler. Autonomous Robots, 28(1), 5–20.

    Google Scholar 

  • Stückler, J., Badami, I., Droeschel, D., Gräve, K., Holz, D., McElhone, M., Nieuwenhuisen, M., Schreiber, M., Schwarz, M., Behnke, S., (2013). Nimbro@home: Winning team of the robocup@home competition 2012. RoboCup 2012, Robot Soccer World Cup XVI.

  • Toussaint, M., Plath, N., Lang, T., Jetchev, N., (2010). Integrated motor control, planning, grasping and high-level reasoning in a blocks world using probabilistic inference. In IEEE International Conference on Robotics and Automation (pp. 385–391).

  • Whitney, D. E. (1977). Force feedback control of manipulator fine motions. Journal of Dynamic Systems, Measurement, and Control (Transactions of the ASME), 99(2), 91.

    Article  Google Scholar 

  • Ye, Q. (1988). The signed Euclidean distance transform and its applications. In International Conference on Pattern Recognition (pp. 495–499).

  • Yoshikawa, T. (2000). Force control of robot manipulators. In IEEE International Conference on Robotics and Automation (pp. 220–226).

Download references

Acknowledgments

This research was supported in part by National Science Foundation Grants ECS-0326095, IIS-0535282, IIS-1017134, CNS-0619937, IIS-0917318, CBET-0922784, EECS-0926052, CNS-0960061, the DARPA Program on Autonomous Robotic Manipulation, the Army Research Office, the Okawa Foundation, the ATR Computational Neuroscience Laboratories, and the Max-Planck-Society. The authors would also like to thank the anonymous reviewers for their constructive comments.

Author information

Authors and Affiliations

  1. Computer Science Department, University of Southern California, Los Angeles, CA, 90089, USA

    Ludovic Righetti, Mrinal Kalakrishnan, Peter Pastor, Jonathan Binney, Randolph C. Voorhies, Gaurav S. Sukhatme & Stefan Schaal

  2. Institute for Aerospace Studies, University of Toronto, Toronto, ON, M3H 5T6, Canada

    Jonathan Kelly

  3. Autonomous Motion Department, Max-Planck-Institute for Intelligent Systems, Paul-Ehrlich-Str. 15, 72076, Tübingen, Germany

    Ludovic Righetti & Stefan Schaal

Authors
  1. Ludovic Righetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mrinal Kalakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Pastor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jonathan Binney
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jonathan Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Randolph C. Voorhies
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gaurav S. Sukhatme
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Stefan Schaal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ludovic Righetti.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mov 32813 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Righetti, L., Kalakrishnan, M., Pastor, P. et al. An autonomous manipulation system based on force control and optimization. Auton Robot 36, 11–30 (2014). https://doi.org/10.1007/s10514-013-9365-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-013-9365-9

Keywords

Search

Navigation