Vision-based robot control

  • Peter I. Corke
  • Gregory D. Hager
Conference paper
Part of the Lecture Notes in Control and Information Sciences book series (LNCIS, volume 230)


The topic of vision-based robot control has been investigated for more than 20 years and over that time several major, and well understood, approaches have evolved. This chapter describes the fundamental principles of these methods, and discusses their relative strengths and weaknesses. The discussion emphasizes the interdependence of vision and control, for example, the vision system provides input to the robot control loop, but the vision system may utilize control techniques to track the target. We also discuss issues such as dynamic performance, approaches to image feature extraction, the impact of current technology trends, future applications and research challenges.


Visual Servo Task Space IEEE Trans Robot Image Feature Extraction Laser Rangefinder 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Allen P, Yoshimi B, Timcenko A 1991 Real-time visual servoing. In: Proc 1991 IEEE Int Conf Robot Automat. Sacramento, CA, pp 851–856Google Scholar
  2. [2]
    Blake A, Curwen R, Zisserman A 1993 Affine-invariant contour tracking with automatic control of spatiotemporal scale. In: Proc Int Conf Comp Vis. Berlin, Germany, pp 421–430Google Scholar
  3. [3]
    Brown C 1990 Gaze controls with interactions and delays. IEEE Trans Syst Man Cyber. 20:518–527Google Scholar
  4. [4]
    Castano A, Hutchinson S A 1994 Visual compliance: Task-directed visual servo control. IEEE Trans Robot Automat. 10:334–342Google Scholar
  5. [5]
    Corke P 1993 Visual control of robot manipulators — A review. In: Hashimoto K (ed) Visual Servoing. World Scientific, Singapore, pp 1–31Google Scholar
  6. [6]
    Corke P I 1996 Visual Control of Robots: High-Performance Visual Servoing. Research Studies Press, Taunton, UKGoogle Scholar
  7. [7]
    De Schutter J 1988 Improved force control laws for advanced tracking applications. In: Proc 1988 IEEE Int Conf Robot Automat. Philadelphia, PA, pp 1497–1502Google Scholar
  8. [8]
    Dickmanns E D, Graefe V 1988 Dynamic monocular machine vision. Mach Vis Appl. 1:223–240Google Scholar
  9. [9]
    Espiau B, Chaumette F, Rives P 1992 A new approach to visual servoing in robotics. IEEE Trans Robot Automat. 8:313–326Google Scholar
  10. [10]
    Feddema J, Lee C, Mitchell 1991 Weighted selection of image features for resolved rate visual feedback control. IEEE Trans Robot Automat. 7:31–47Google Scholar
  11. [11]
    Goldreich D, Krauzlis R, Lisberger S 1992 Effect of changing feedback delay on spontaneous oscillations in smooth pursuit eye movements of monkeys. J Neurophys. 67:625–638Google Scholar
  12. [12]
    Hager G D 1997 A modular system for robust hand-eye coordination. IEEE Trans Robot Automat. 13:582–595Google Scholar
  13. [13]
    Hager G D, Chang W-C, Morse A S 1994 Robot hand-eye coordination based on stereo vision. IEEE Contr Syst Mag. 15(1):30–39Google Scholar
  14. [14]
    Hager G D, Toyama K 1996 XVision: Combining image warping and geometric constraints for fast visual tracking. In: Proc 4th Euro Conf Comp Vis. pp 507–517Google Scholar
  15. [15]
    Hashimoto H, Kubota T, Lo W-C, Harashima F 1989 A control scheme of visual servo control of robotic manipulators using artificial neural network. In: Proc IEEE Int Conf Contr Appl. Jerusalem, Israel, pp TA-3-6Google Scholar
  16. [16]
    Hashimoto K, Kimoto T, Ebine T, Kimura H 1991 Manipulator control with image-based visual servo. In: Proc 1991 IEEE Int Conf Robot Automat. Sacramento, CA, pp 2267–2272Google Scholar
  17. [17]
    Hollinghurst N, Cipolla R 1994 Uncalibrated stereo hand eye coordination. Image Vis Comp. 12(3):187–192Google Scholar
  18. [18]
    Hosoda K, Asada M 1994 Versatile visual servoing without knowledge of true Jacobian. In: Proc IEEE Int Work Intel Robot Syst. pp 186–191Google Scholar
  19. [19]
    Huang T S, Netravali A N 1994 Motion and structure from feature correspondences: A review. IEEE Proc. 82:252–268Google Scholar
  20. [20]
    Hutchinson S, Hager G, Corke P 1996 A tutorial on visual servo control. IEEE Trans Robot Automat. 12:651–670Google Scholar
  21. [21]
    Jägersand M, Nelson R 1996 On-line estimation of visual-motor models using active vision. In: Proc ARPA Image Understand Work.Google Scholar
  22. [22]
    Jang W, Bien Z 1991 Feature-based visual servoing of an eye-in-hand robot with improved tracking performance. In: Proc 1991 IEEE Int Conf Robot Automat. Sacramento, CA, pp 2254–2260Google Scholar
  23. [23]
    Kalata P R 1984 The tracking index: A generalized parameter for α-β and α-β-γ target trackers. IEEE Trans Aerosp Electron Syst. 20:174–182Google Scholar
  24. [24]
    Kuperstein M 1988 Generalized neural model for adaptive sensory-motor control of single postures. In: Proc 1988 IEEE Int Conf Robot Automat. Philadelphia, PA, pp 140–143Google Scholar
  25. [25]
    Nelson B, Khosla P K 1993 Increasing the tracking region of an eye-in-hand system by singularity and joint limit avoidance. In: Proc 1993 IEEE Int Conf Robot Automat. Atlanta, GA, vol 3, pp 418–423Google Scholar
  26. [26]
    Nelson B, Khosla P 1994 The resolvability ellipsoid for visual servoing. In: Proc IEEE Conf Comp Vis Patt Recog. pp 829–832Google Scholar
  27. [27]
    Papanikolopoulos N P, Khosla P K 1993 Adaptive robot visual tracking: theory and experiments. IEEE Trans Automat Contr. 38:429–445Google Scholar
  28. [28]
    Papanikolopoulos N, Khosla P, Kanade T 1991 Vision and control techniques for robotic visual tracking. In: Proc 1991 IEEE Int Conf Robot Automat. Sacramento, CA, pp 857–864Google Scholar
  29. [29]
    Rizzi A, Koditschek D 1991 Preliminary experiments in spatial robot juggling. In: Chatila R, Hirzinger G (eds) Experimental Robotics II. Springer-Verlag, London, UK.Google Scholar
  30. [30]
    Robinson D 1987 Why visuomotor systems don't like negative feedback and how they avoid it. In: Arbib M, Hanson A (eds) Vision, Brain and Cooperative Behaviour. MIT Press, Cambridge, MAGoogle Scholar
  31. [31]
    Samson C, Le Borgne M, Espiau B 1992 Robot Control: The Task Function Approach. Clarendon Press, Oxford, UKGoogle Scholar
  32. [32]
    Sanderson A, Weiss L 1983 Adaptive visual servo control of robots. In: Pugh A (ed) Robot Vision. Springer-Verlag, Berlin, Germany, pp 107–116Google Scholar
  33. [33]
    Sharkey P, Murray D 1996 Delays versus performance of visually guided systems. IEE Proc Contr Theo Appl. 143:436–447Google Scholar
  34. [34]
    Shirai Y, Inoue H 1973 Guiding a robot by visual feedback in assembling tasks. Patt Recogn. 5:99–108Google Scholar
  35. [35]
    Skaar S, Brockman W, Hanson R 1987 Camera-space manipulation. Int J Robot Res. 6(4):20–32Google Scholar
  36. [36]
    Tsai R 1986 An efficient and accurate camera calibration technique for 3D machine vision. In: Proc IEEE Conf Comp Vis Patt Recogn. pp 364–374Google Scholar
  37. [37]
    Wilson W 1994 Visual servo control of robots using Kalman filter estimates of robot pose relative to work-pieces. In: Hashimoto K (ed) Visual Servoing. World Scientific, Singapore, pp 71–104Google Scholar

Copyright information

© Springer-Verlag London Limited 1998

Authors and Affiliations

  • Peter I. Corke
    • 1
  • Gregory D. Hager
    • 2
  1. 1.CSIRO Manufacturing Science and TechnologyAustralia
  2. 2.Department of Computer ScienceYale UniversityUSA

Personalised recommendations