Skip to main content
SpringerLink
Log in

Generality and legibility in mobile manipulation

Learning skills for routine tasks

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

This article investigates methods for achieving more general manipulation capabilities for mobile manipulation platforms, which produce legible behavior in human living environments. To achieve generality and legibility, we combine two control mechanisms. First of all, experience- and observation-based learning of skills is applied to routine tasks, so that the repetitive and stereotypical character of everyday activity is exploited. Second, we use planning, reasoning, and search for novel tasks which have no stereotypical solution. We apply these ideas to the learning and use of action-related places, to the model-based visual recognition and localization of objects, and the learning and application of reaching strategies and motions from humans. We demonstrate the integration of these mechanisms into a single low-level control system for autonomous manipulation platforms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Albu-Schaffer, A., Haddadin, S., Ott, Ch., Stemmer, A., Wimbock, T., & Hirzinger, G. (2007). The dlr lightweight robot: design and control concepts for robots in human environments. Industrial Robot: An International Journal, 34(5), 376–385.

    Article  Google Scholar 

  • Asfour, T., Regenstein, K., Azad, P., Schroder, J., Bierbaum, A., Vahrenkamp, N., & Dillmann, R. (2006). Armar-iii: An integrated humanoid platform for sensory-motor control. In Proceedings of the IEEERAS/RSJ international conference on humanoid robots (Humanoids06) (pp. 169–175).

  • Beetz, M., Stulp, F., Radig, B., Bandouch, J., Blodow, N., Dolha, M., Fedrizzi, A., Jain, D., Klank, U., Kresse, I., Maldonado, A., Marton, Z., Mösenlechner, L., Ruiz, F., Rusu, R. B., & Tenorth, M. (2008). The assistive kitchen—a demonstration scenario for cognitive technical systems. In IEEE 17th international symposium on robot and human interactive communication (RO-MAN), Muenchen, Germany. Invited paper.

  • Belker, T. (2004). Plan projection, execution, and learning for mobile robot control. PhD thesis, Department of Applied Computer Science, University of Bonn.

  • Berenson, D., Diankov, R., Nishiwaki, K., Kagami, S., & Kuffner, J. (2007). Grasp planning in complex scenes. In IEEE-RAS international conference on humanoid robots.

  • Berenson, D., Srinivasa, S., Ferguson, D., Romea, A. C., & Kuffner, J. (2009). Manipulation planning with workspace goal regions. In Proceedings of the IEEE international conference on robotics and automation (ICRA).

  • Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration. In Springer handbook of robotics. Berlin: Springer. Chap. 59.

    Google Scholar 

  • Borst, C., Fischer, M., Haidacher, S., Liu, H., & Hirzinger, G. (2003). Dlr hand ii: Experiments and experiences with an anthropomorphic hand. In 2003 IEEE international conference on robotics and automation (ICRA), Taipei, Taiwan, 14–19 September 2003.

  • Calinon, S., Guenter, F., & Billard, A. (2007). On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics, 37(2), 286–298. Special issue on robot learning by observation, demonstration and imitation.

    Article  Google Scholar 

  • Cheng, G., Hyon, S., Morimoto, J., Ude, A., Hale, J. G., Colvin, G., Scroggin, W., & Jacobsen, S. C. (2007). Cb: A humanoid research platform for exploring neuroscience. Journal of Advanced Robotics, 21(10), 1097–1114.

    Article  Google Scholar 

  • Dearden, A., & Demiris, Y. (2005). Learning forward models for robotics. In Proceedings of the nineteenth international joint conference on artificial Intelligence (IJCAI) (pp. 1440–1445).

  • Fedrizzi, A., Moesenlechner, L., Stulp, F., & Beetz, M. (2009). Transformational planning for mobile manipulation based on action-related places. In Proceedings of the international conference on advanced robotics (ICAR).

  • Flash, T., & Hochner, B. (2005). Motor primitives in vertebrates and invertebrates. Current Opinion in Neurobiology, 15, 660–666.

    Article  Google Scholar 

  • Fox, M., Gough, J., & Long, D. (2006). Using learned action models in execution monitoring. In Proceedings of UK planning and scheduling SIG.

  • Gerkey, B., Vaughan, R. T., & Howard, A. (2003). The Player/Stage Project: Tools for multi-robot and distributed sensor systems. In Proceedings of the 11th international conference on advanced robotics (ICAR) (pp. 317–323).

  • Haigh, K. Z. (1998). Situation-dependent learning for interleaved planning and robot execution. Ph.D. thesis, School of Computer Science, Carnegie Mellon University.

  • Harris, C. M., & Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature, 394(20), 780–784.

    Article  Google Scholar 

  • Hoffmann, H., Pastor, P., Park, D.-H., & Schaal, S. (2009). Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance. In IEEE international conference on robotics and automation.

  • Hofhauser, A., Steger, C., & Navab, N. (2008). Harmonic deformation model for edge based template matching. In Third international conference on computer vision theory and applications (Vol. 2, pp. 75–82).

  • Horswill, I. (1996). Integrated systems and naturalistic tasks. In Strategic directions in computing research AI working group.

  • Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002). Movement imitation with nonlinear dynamical systems in humanoid robots. In International conference on robotics and automation (ICRA2002).

  • Jenkins, O., Bodenheimer, R., & Peters, R. (2006). Manipulation manifolds: Explorations into uncovering manifolds in sensory-motor spaces. In International conference on development and learning (ICDL).

  • Julier, S., & Uhlmann, J. K. (1996). A general method for approximating nonlinear transformations of probability distributions (Technical report). Dept. of Engineering Science, University of Oxford.

  • Katz, D., Horrell, E., Yang, Y., Burns, B., Buckley, T., Grishkan, A., Zhylkovskyy, V., Brock, O., & Miller, E. (2006). The UMass mobile manipulator UMan: an experimental platform for autonomous mobile manipulation. In Workshop on manipulation in human environments at robotics: science and systems.

  • Kemp, C., Edsinger, A., & Torres-Jara, E. (2007). Challenges for robot manipulation in human environments. IEEE Robotics and Automation Magazine, 14(1), 20–29.

    Article  Google Scholar 

  • Klank, U., Zia, M. Z., & Beetz, M. (2009). 3D model selection from an Internet database for robotic vision. In International conference on robotics and automation (ICRA).

  • Kragic, D., Björkman, M., Christensen, H. I., & Eklundh, J. O. (2005). Vision for robotic object manipulation in domestic settings. Robotics and Autonomous Systems, 52(1), 85–100.

    Article  Google Scholar 

  • LaValle, S. M. (2006). Planning algorithms. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Lepetit, V., & Fua, P. (2006). Keypoint recognition using randomized trees. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9), 1465–1479.

    Article  Google Scholar 

  • Maciejewski, A. A., & Klein, C. A. (1989). The singular value decomposition: Computation and applications to robotics. International Journal of Robotics Research, 8(6), 63–79.

    Article  Google Scholar 

  • McDermott, D. (1991). A reactive plan language (Research Report YALEU/DCS/RR-864). Yale University.

  • Metta, G., Fitzpatrick, P., & Natale, L. (2006). YARP: Yet Another Robot Platform. International Journal of Advanced Robotics Systems, 3(1). Special issue on Software Development and Integration in Robotics.

  • Okada, K., Kojima, M., Sagawa, Y., Ichino, T., Sato, K., & Inaba, M. (2006). Vision based behavior verification system of humanoid robot for daily environment tasks. In Proceedings of the 6th IEEE-RAS international conference on humanoid robots (humanoids) (pp. 7–12).

  • Osada, R., Funkhouser, T., Chazelle, B., & Dobkin, D. (2002). Shape distributions. ACM Transactions on Graphics (TOG), 21(4), 807–832.

    Article  Google Scholar 

  • Oztop, E., Franklin, D. W., Chaminade, T., & Cheng, G. (2005). Human-humanoid interaction: Is a humanoid robot perceived as a human? International Journal of Humanoid Robotics, 2(4), 537–559.

    Article  Google Scholar 

  • Park, D.-H., Hoffmann, H., & Schaal, S. (2008). Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields. In International conference on humanoid robots.

  • Petrovskaya, A., & Ng, A. Y. (2007). Probabilistic mobile manipulation in dynamic environments, with application to opening doors. In International joint conference on artificial intelligence (IJCAI-07).

  • Roduit, P., Martinoli, A., & Jacot, J. (2007). A quantitative method for comparing trajectories of mobile robots using point distribution models. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2441–2448).

  • Rosenbaum, D. A., Cohen, R. G., Meulenbroek, R. G. J., & Vaughan, J. (2006). Plans for grasping objects. In M. L. Latash & F. Lestienne (Eds.), Motor control and learning (pp. 9–25). New York: Springer.

    Chapter  Google Scholar 

  • Roweis, S., & Saul, L. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500), 2323–2326.

    Article  Google Scholar 

  • Rusu, R. B., Marton, Z. C., Blodow, N., Dolha, M., & Beetz, M. (2008). Towards 3D point cloud based object maps for household environments. Robotics and Autonomous Systems Journal. (Special issue on Semantic Knowledge).

  • Saxena, A., Driemeyer, J., & Ng, A. Y. (2008). Robotic grasping of novel objects using vision. The International Journal of Robotics Research, 27(2), 157.

    Article  Google Scholar 

  • Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), 233–242.

    Article  Google Scholar 

  • Smits, R., De Laet, T., Claes, K., Soetens, P., De Schutter, J., & Bruyninckx, H. (2008). Orocos: A software framework for complex sensor-driven robot tasks. IEEE Robotics and Automation Magazine.

  • Srinivasa, S., Ferguson, D., Weghe, M. V., Diankov, R., Berenson, D., Helfrich, C., & Strasdat, H. (2008). The robotic busboy: Steps towards developing a mobile robotic home assistant. In International conference on intelligent autonomous systems.

  • Stulp, F., & Beetz, M. (2008). Refining the execution of abstract actions with learned action models. Journal of Artificial Intelligence Research (JAIR), 32, June 2008.

  • Stulp, F., Koska, W., Maldonado, A., & Beetz, M. (2007). Seamless execution of action sequences. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 3687–3692).

  • Stulp, F., Kresse, I., Maldonado, A., Ruiz, F., Fedrizzi, A., & Beetz, M. (2009, to appear). Compact models of human reaching motions for robotic control in everyday manipulation tasks. In Proceedings of the 8th international conference on development and learning (ICDL).

  • Tenorth, M., Nyga, D., & Beetz, M. (2009). Understanding and executing instructions for everyday manipulation tasks from the world wide web (Technical report). IAS Group, Technische Universität München, Fakultät für Informatik.

  • Todorov, E. (2004). Optimality principles in sensorimotor control. Nature Neuroscience, 7(9), 907–915.

    Article  Google Scholar 

  • Wiedemann, C., Ulrich, M., & Steger, C. (2008). Recognition and tracking of 3d objects. In G. Rigoll (Ed.), Lecture notes in computer science : Vol. 5096. Pattern recognition (pp. 132–141). Berlin: Springer.

    Chapter  Google Scholar 

  • Wilhelm, A., Huissoon, J., Melek, W., Clark, C., Fuchs, M., & Hirzinger, G. (2007). Design of a wheeled mobile robotic platform with variable footprint. In IEEE/RSJ conference on intelligent robotics and systems.

  • PR2 Robot. (2008). www.willowgarage.com/pages/robots/pr2-overview.

  • Wimmer, M., Stulp, F., Pietzsch, S., & Radig, B. (2008). Learning local objective functions for robust face model fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 30(8), 1357–1370.

    Article  Google Scholar 

  • Wolpert, D., & Ghahramani, Z. (2000). Computational principles of movement neuroscience. Nature Neuroscience Supplement, 3, 1212–1217.

    Article  Google Scholar 

  • Zacharias, F., Borst, Ch., & Hirzinger, G. (2007). Capturing robot workspace structure: representing robot capabilities. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3229–3236).

  • Zacharias, F., Borst, Ch., & Hirzinger, G. (2008). Positioning mobile manipulators to perform constrained linear trajectories. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2578–2584).

Download references

Author information

Authors and Affiliations

  1. Intelligent Autonomous Systems Group, Department of Informatics, Technische Universität München, Boltzmannstr. 3, 85748, Garching bei München, Germany

    Michael Beetz, Freek Stulp, Piotr Esden-Tempski, Andreas Fedrizzi, Ulrich Klank, Ingo Kresse, Alexis Maldonado & Federico Ruiz

Authors
  1. Michael Beetz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Freek Stulp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piotr Esden-Tempski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andreas Fedrizzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ulrich Klank
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ingo Kresse
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alexis Maldonado
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Federico Ruiz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ingo Kresse.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Beetz, M., Stulp, F., Esden-Tempski, P. et al. Generality and legibility in mobile manipulation. Auton Robot 28, 21–44 (2010). https://doi.org/10.1007/s10514-009-9152-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-009-9152-9

Keywords

Search

Navigation