Advertisement

Autonomous Robots

, Volume 6, Issue 2, pp 203–221 | Cite as

GripSee: A Gesture-Controlled Robot for Object Perception and Manipulation

  • Mark Becker
  • Efthimia Kefalea
  • Eric Maël
  • Christoph Von Der Malsburg
  • Mike Pagel
  • Jochen Triesch
  • Jan C. Vorbrüggen
  • Rolf P. Würtz
  • Stefan Zadel
Article

Abstract

We have designed a research platform for a perceptually guided robot, which also serves as a demonstrator for a coming generation of service robots. In order to operate semi-autonomously, these require a capacity for learning about their environment and tasks, and will have to interact directly with their human operators. Thus, they must be supplied with skills in the fields of human-computer interaction, vision, and manipulation. GripSee is able to autonomously grasp and manipulate objects on a table in front of it. The choice of object, the grip to be used, and the desired final position are indicated by an operator using hand gestures. Grasping is performed similar to human behavior: the object is first fixated, then its form, size, orientation, and position are determined, a grip is planned, and finally the object is grasped, moved to a new position, and released. As a final example for useful autonomous behavior we show how the calibration of the robot's image-to-world coordinate transform can be learned from experience, thus making detailed and unstable calibration of this important subsystem superfluous. The integration concepts developed at our institute have led to a flexible library of robot skills that can be easily recombined for a variety of useful behaviors.

service robot human-robot interaction stereo vision gesture recognition hand tracking object recognition fixation grasping grip perception skill behavior 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bergener, T., Bruckhoff, C., Dahm, P., Janßen, H., Joublin, F., and Menzner, R. 1997. Arnold: An anthropomorphic autonomous robot for human environments. In Workshop SOAVE '97, Horst-Michael Groß (Ed.), Ilmenau, Germany, VDI Verlag, Düsseldorf, pp. 25–34.Google Scholar
  2. Bergener, T. and Dahm, P. 1997. A framework for dynamic man-machine interaction implemented on an autonomous mobile robot. In ISIE'97, IEEE International Symposium on Industrial Electronics, IEEE Publications: Piscataway, NJ, pp. SS42-SS47.Google Scholar
  3. Brooks, R.A. 1991. Intelligence without representation. Artificial Intelligence Journal, 47:139–160.Google Scholar
  4. Brooks, R.A. 1997. From earwigs to humans. Robotics and Autonomous Systems, 20(2):291–304.Google Scholar
  5. Cipolla, R. and Hollinghurst, N. 1997. Visually guided grasping in unstructured environments. Robotics and Autonomous Systems, 19(3/4):337–346.Google Scholar
  6. Corbacho, F.J. and Arbib, M.A. 1995. Learning to detour. Adaptive Behavior, 3(4):419–468.Google Scholar
  7. Crowley, J.L. 1995. Integration and control of reactive visual processes. Robotics and Autonomous Systems, 16(1):17–27.Google Scholar
  8. Crowley, J.L. 1996. Vision for man-machine interaction. Robotics and Autonomous Systems, 19(2):347–358.Google Scholar
  9. Dario, P., Guglielmelli, E., Genovese, V., and Toro, M. 1996. Robot assistants: Applications and evolution. Robotics and Autonomous Systems, 18:225–234.Google Scholar
  10. Eckes, C. and Vorbrüggen, J.C. 1996. Combining data-driven and model-based cues for segmentation of video sequences. In Proceedings WCNN96, INNS Press & Lawrence Erlbaum Ass., pp. 868–875.Google Scholar
  11. Fritzke, B. 1995. Incremental learning of local linear mappings. In Proceedings ICANN95. F. Fogelman-Soulié and P. Gallinari (Eds.), Paris, EC2 & Cie, pp. 217–222.Google Scholar
  12. Hara, F. and Kobayashi, H. 1997. State-of-the-art in component technology for an animated face robot-its component technology development for interactive communication with humans. Advanced Robotics, 11(6):585–604.Google Scholar
  13. Hutchinson, S., Hager, G.D., and Corke, P.I. 1996. A tutorial on visual servo control. IEEE Transactions on Robotics and Automation, 12(5):651–670.Google Scholar
  14. Kawamura, K., Pack, R.T., Bishay, M., and Iskarous, M. 1996. Design philosophy for service robots. Robotics and Autonomous Systems, 18(2):109–116.Google Scholar
  15. Kefalea, E. 1998. Object localization and recognition for a grasping robot. In Proceedings of the 24th Annual Conference of the IEEE Industrial Electronics Society (IECON '98), IEEE, pp. 2057–2062.Google Scholar
  16. Kefalea, E. 1999. Flexible object recognition for a grasping robot. Ph.D. Thesis, Ruhr-Universität Bochum, in preparation.Google Scholar
  17. Kefalea, E., Rehse, O., and v.d. Malsburg, C. 1997. Object classification based on contours with elastic graph matching. In Proc. 3rd Int. Workshop on Visual Form, Capri, Italy, World Scientific, pp. 287–297.Google Scholar
  18. Klein C.A. and Huang, C. 1983. Review of pseudoinverse control for use with kinematically redundant manipulators. IEEE Transactions on Systems, Man, and Cybernetics, 13(3):245–250.Google Scholar
  19. Lades, M., Vorbrüggen, J.C., Buhmann, J., Lange, J., von der Malsburg, C., Würtz, R.P., and Konen, W. 1993. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers, 42(3):300–311.Google Scholar
  20. Lourens, T. and Würtz, R.P. 1998. Object recognition by matching symbolic edge graphs. In Computer Vision-ACCV'98, R. Chin and Ting-Chuen Pong (Eds.), volume 1352 of Lecture Notes in Computer Science. Springer Verlag, pp. II-193-II-200.Google Scholar
  21. Maes, P. 1994. Situated agents can have goals. In Designing Autonomous Agents, P. Maes (Ed.), 3rd edition. MIT press, pp. 49–70. Reprinted from Robotics and Autonomous Systems, Vol. 6, Nos. 1 and 2 (June 1990).Google Scholar
  22. Maël, E. 1996. A hierarchical network for learning robust models of kinematic chains. In Artificial Neural Networks-ICANN 96, C. von der Malsburg, J.C. Vorbrüggen, W. von Seelen, and B. Sendhoff (Eds.), volume 1112 of Lecture Notes in Computer Science, Springer Verlag, pp. 615–622.Google Scholar
  23. Mallat, S. and Zhong, S. 1992. Characterization of signals from multiscale edges. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14:710–732.Google Scholar
  24. Pagel, M., Maël, E., and von der Malsburg, C. 1998a. Self calibration of the fixation movement of a stereo camera head. Autonomous Robots, 5:355–367.Google Scholar
  25. Pagel, M., Maël, E., and von der Malsburg, C. 1998b. Self calibration of the fixation movement of a stereo camera head. Machine Learning, 31:169–186.Google Scholar
  26. Paul, R.P. 1981. Robot Manipulators: Mathematics, Programming and Control, MIT Press.Google Scholar
  27. Pauli, J. 1998. Learning to recognize and grasp objects. Autonomous Robots, 5(3/4):407–420.Google Scholar
  28. Rinne, M., Pötzsch, M., and von der Malsburg, C. 1998. Designing Objects for Computer Vision (FLAVOR), in preparation.Google Scholar
  29. Rosenschein, S.J. 1985. Formal theories of knowledge in AI and robotics. New Generation Computing, 3(4):345–357.Google Scholar
  30. Suchmann, L. (Ed.). 1987. Plans and Situated Action, Oxford University Press.Google Scholar
  31. Triesch, J. and von der Malsburg, C. 1996. Robust classification of hand postures against complex backgrounds. In Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, IEEE Computer Society Press, pp. 170–175.Google Scholar
  32. Triesch, J. and von der Malsburg, C. 1998. A gesture interface for robotics. In FG'98, the Third International Conference on Automatic Face and Gesture Recognition, IEEE Computer Society Press, pp. 546–551.Google Scholar
  33. Tsotsos, J.K., Verghese, G., Dickinson, S., Jenkin, M., Jepson, A., Milios, E., Nuflot, F., Stevenson, S., Black, M., Metaxas, D., Culhane, S., Ye, Y., and Mann, R. 1998. PLAYBOT: A visually-guided robot to assist physically disabled children in play. Image and Vision Computing, 16:275–292.Google Scholar
  34. Vorbrüggen, J.C. 1995. Zwei Modelle zur datengetriebenen Segmentierung visueller Daten, Reihe Physik. Verlag Harri Deutsch, Thun, Frankfurt am Main.Google Scholar
  35. Wampler, C.W. 1986. Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man, and Cybernetics, 16(1):93–101.Google Scholar
  36. Wiskott, L. 1996. Labeled Graphs and Dynamic Link Matching for Face Recognition and Scene Analysis, Reihe Physik. Verlag Harri Deutsch, Thun, Frankfurt am Main.Google Scholar
  37. Wiskott, L., Fellous, J.M., Krüger, N., and von der Malsburg, C. 1997. Face recognition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):775–779.Google Scholar
  38. Würtz, R.P. 1997. Object recognition robust under translations, deformations and changes in background. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):769–775.Google Scholar
  39. Würtz, R.P. and Lourens, T. 1997. Corner detection in color images by multiscale combination of end-stopped cortical cells. In Artificial Neural Networks-ICANN '97, Wulfram Gerstner, A. Germond, M. Hasler, and J.-D. Nicoud (Eds.), volume 1327 of Lecture Notes in Computer Science, Springer Verlag, pp. 901–906.Google Scholar
  40. Yau, W.Y. and Wang, H. 1997. Robust hand-eye coordination. Advanced Robotics, 11(1):57–73.Google Scholar
  41. Zadel, S. 1999. Greifen bei Servicerobotern: Sehen und Tastsinn, Lernen und Selbstorganisation. Ph.D. Thesis, Ruhr-Universität Bochum, in preparation.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Mark Becker
    • 1
  • Efthimia Kefalea
    • 1
  • Eric Maël
    • 1
  • Christoph Von Der Malsburg
    • 1
  • Mike Pagel
    • 1
  • Jochen Triesch
    • 1
  • Jan C. Vorbrüggen
    • 1
  • Rolf P. Würtz
    • 1
  • Stefan Zadel
    • 1
  1. 1.Institut für NeuroinformatikRuhr-Universität BochumBochumGermany

Personalised recommendations