Skip to main content
SpringerLink
Log in

Visual Servoing

  • Living reference work entry
  • First Online:
Encyclopedia of Robotics

Synonyms

Vision-based control; Visual feedback

Definition

Visual servoing refers to the use of visual data as input of real-time closed-loop control schemes for controlling the motion of a dynamic system, a robot typically. It can be defined as sensor-based control from a vision sensor and relies on techniques from image processing, computer vision, and control theory.

Overview

Basically, visual servoing consists in using the data provided by one or several cameras so that a dynamic system achieves a task specified by a set of visual constraints (Hutchinson et al., 1996; Chaumette et al., 2016). Such systems are usually robot arms or mobile robots but can also be virtual robots or even a virtual camera. A large variety of positioning tasks, or target tracking tasks, can be considered by controlling from one to all the degrees of freedom (DoF) of the system (Marchand et al., 2005). Whatever the sensor configuration, which can vary from one on-board camera located on the robot...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Agravante DJ, Claudio G, Spindler F, Chaumette F (2016) Visual servoing in an optimization framework for the whole-body control of humanoid robots. IEEE Robot Autom Lett 2(2):608–615

    Google Scholar 

  • Allibert G, Courtial E, Chaumette F (2010) Predictive control for constrained image-based visual servoing. IEEE Trans Robot 26(5):933–939

    Google Scholar 

  • Bakthavatchalam M, Tahri O, Chaumette F (2018) A direct dense visual servoing approach using photometric moments. IEEE Trans Robot 34(5):1226–1239

    Google Scholar 

  • Bateux Q, Marchand E, Leitner J, Chaumette F, Corke P (2018) Training deep neural networks for visual servoing. In: IEEE international conference on robotics and automation (ICRA’18), pp 3307–3314

    Google Scholar 

  • Caron G, Marchand E, Mouaddib EM (2013) Photometric visual servoing for omnidirectional cameras. Auton Robot 35(2):177–193

    Google Scholar 

  • Chaumette F (2004) Image moments: a general and useful set of features for visual servoing. IEEE Trans Robot 20(4):713–723

    Google Scholar 

  • Chaumette F, Hutchinson S, Corke P (2016) Visual servoing, Chapter 34. In: Handbook of robotics, 2nd edn. Berlin: Springer, pp 841–866

    Google Scholar 

  • Chesi G (2009) Visual servoing path planning via homogeneous forms and LMI optimizations. IEEE Trans Robot 25(2):281–291

    Google Scholar 

  • Collewet C, Marchand E (2011) Photometric visual servoing. IEEE Trans Robot 27(4):828–834

    Google Scholar 

  • Corke P, Good M (1996) Dynamic effects in visual closed-loop systems. IEEE Trans Robot Autom 12(5):671–683

    Google Scholar 

  • Crombez N, Mouaddib EM, Caron G, Chaumette F (2019) Visual servoing with photometric Gaussian mixtures as dense features. IEEE Trans Robot 35(1):49–63

    Google Scholar 

  • Dame A, Marchand E (2011) Mutual information-based visual servoing. IEEE Trans Robot 27(5):958–969

    Google Scholar 

  • Deguchi K (2000) A direct interpretation of dynamic images with camera and object motions for vision guided robot control, Int J Comput Vis 37(1):7–20

    MATH  Google Scholar 

  • De Luca A, Oriolo G, Robuffo Giordano P (2008) Feature depth observation for image-based visual servoing: theory and experiments. Int J Robot Res 38(4):422–450; 27(10):1093–1116

    Google Scholar 

  • Duflot LA, Reisenhofer R, Tamadazte B, Andreff N, Krupa A (2019) Wavelet and shearlet-baed image representations for visual servoing. Int J Robot Res 38(4):422–450

    Google Scholar 

  • Espiau B, Chaumette F, Rives P (1992) A new approach to visual servoing in robotics. IEEE Trans Robot Autom 8(3):313–326

    Google Scholar 

  • Ginhoux R, Gangloff J, de Mathelin M, Soler L, Sanchez MA, Marescaux J (2005) Active filtering of physiological motion in robotized surgery using predictive control. IEEE Trans Robot 21(1):67–79

    Google Scholar 

  • Hadj-Abdelkader H, Mezouar Y, Martinet P, Chaumette F (2008) Catadioptric visual servoing from 3D straight lines. IEEE Trans Robot 24(3):652–665

    Google Scholar 

  • Hamel T, Mahony R (2002) Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach. IEEE Trans Robot Autom 18(2):187–198

    Google Scholar 

  • Han S, Censi A, Straw A, Murray R (2010) A bio-plausible design for visual pose stabilization. In: International conference on intelligent robots and systems (IROS’10), pp 5679–5686

    Google Scholar 

  • Hashimoto K, Ebine T, Kimura H (1996) Visual servoing with hand-eye manipulator – Optimal control approach. IEEE Trans Robot Autom 12(5):766–774

    Google Scholar 

  • Hosoda K, Asada M (1994) Versatile visual servoing without knowledge of true jacobian, In: IEEE/RSJ international conference on intelligent robots and systems (IROS’94), pp 186–193

    Google Scholar 

  • Hutchinson S, Hager G, Corke P (1996) A tutorial on visual servo control. IEEE Trans Robot Autom 12(5):651–670

    Google Scholar 

  • Iwatsuki M, Okiyama N (2005) A new formulation of visual servoing based on cylindrical coordinate system. IEEE Trans Robot Autom 21(2):266–273

    Google Scholar 

  • Jägersand M, Fuentes O, Nelson R (1997) Experimental evaluation of uncalibrated visual servoing for precision manipulation. In: IEEE international conference on robotics and automation (ICRA’97), pp 2874–2880

    Google Scholar 

  • Kazemi M, Gupta K, Mehrandezh M (2013) Randomized kinodynamic planning for robust visual servoing. IEEE Trans Robot 29(5):1197–1211

    Google Scholar 

  • Khalil W, Dombre E (2002) Modeling, identification, and control of robots. Oxford: CRC Press

    MATH  Google Scholar 

  • Levine S, Finn C, Darrell T, Abbeel P (2016) End-to-end training of deep visuomotor policies. J Mach Learn Res 17(1):1334–1373

    MathSciNet  MATH  Google Scholar 

  • Lopez-Nicolas G, Guerrero JJ, Sagues C (2010) Visual control through the trifocal tensor for nonholonomic robots. Robot Auton Syst 58(2):216–226

    Google Scholar 

  • Malis E, Chaumette F (2000) 2-1/2D visual servoing with respect to unknown objects through a new estimation scheme of camera displacement. Int J Comput Vis 37(1):79–97

    Google Scholar 

  • Marchand E, Spindler F, Chaumette F (2005) ViSP for visual servoing: a generic software platform with a wide class of robot control skills. IEEE Robot Autom Mag 12(4):40–52

    Google Scholar 

  • Marchand E, Uchiyama H, Spindler F (2016) Pose estimation for augmented reality: a hands-on survey. IEEE Trans Vis Comput Graph 22(12):2633–2651

    Google Scholar 

  • Mariottini GL, Oriolo G, Prattichizo D (2007) Image-based visual servoing for nonholonomic mobile robots using epipolar geometry. IEEE Trans Robot 23(1):87–100

    Google Scholar 

  • Mebarki R, Krupa A, Chaumette F (2010) 2D ultrasound probe complete guidance by visual servoing using image moments. IEEE Trans Robot 26(2):296–306

    Google Scholar 

  • Mebarki R, Lippiello V, Siciliano B (2015) Nonlinear visual control of unmanned aerial vehicles in GPS-denied environments. IEEE Trans Robot 31(4):1004–1017

    Google Scholar 

  • Mezouar Y, Chaumette F (2002) Path planning for robust image-based control. IEEE Trans Robot 22(10):781–804

    Google Scholar 

  • Motyl G, Chaumette F, Gallice J (1992) Coupling a camera and laser stripe in sensor based control. In: Second international symposium on measurement and control in robotics, pp 685–692

    Google Scholar 

  • Nakamura Y, Hanafusa H, Yoshikawa T (1987) Task-priority based redundancy control of robot manipulators. Int J Robot Res 6(2):3–15

    Google Scholar 

  • Nayar S, Nene S, Murase H (1996) Subspace methods for robot vision. IEEE Trans Robot Autom 12(5):750–758

    Google Scholar 

  • Nelson B, Khosla P (1995) Strategies for increasing the tracking region of an eye-in-hand system by singularity and joint limit avoidance. Int J Robot Res 14(3):225–269

    Google Scholar 

  • Pagès J, Collewet C, Chaumette F, Salvi J (2006) Optimizing plane-to-plane positioning tasks by image-based visual servoing and structured light. IEEE Trans Robot 22(5):1000–1010

    Google Scholar 

  • Pandya H, Gaud A, Kumar G, Madhava Krishna K (2019) Instance invariant visual servoing framework for part-aware autonomous vehicle inspection using MAVs. J Field Robot 36(5):892–918

    Google Scholar 

  • Silveira G, Malis E (2012) Direct visual servoing: vision-based estimation and control using only nonmetric information. IEEE Trans Robot 28(4):974–980

    Google Scholar 

  • Suh I (1993) Visual servoing of robot manipulators by fuzzy membership function based neural networks. In: Visual servoing. World scientific series on robotics and automated systems, vol 7, pp 285–315

    Google Scholar 

  • Teulière C, Marchand E (2014) A dense and direct approach to visual servoing using depth maps. IEEE Trans Robot 30(5):1242–1249

    Google Scholar 

  • Thuilot B, Martinet P, Cordesses L, Gallice J (2002) Position based visual servoing: Keeping the object in the field of vision. In: IEEE international conference on robotics and automation (ICRA’02), pp 1624–1629

    Google Scholar 

  • Tsai R, Lenz R (1989) A new technique for fully autonomous and efficient 3D robotics hand-eye calibration. IEEE Trans Robot Autom 5(3):345–358

    Google Scholar 

  • Weiss L, Sanderson A, Neuman C (1987) Dynamic sensor-based control of robots with visual feedback. IEEE J Robot Autom 3(5):404–417

    Google Scholar 

  • Wells G, Venaille C, Torras C (1996) Vision-based robot positioning using neural networks. Image Vis Comput 14(10):715–732

    Google Scholar 

  • Wilson W, Hulls C, Bell G (1996) Relative end-effector control using Cartesian position-based visual servoing. IEEE Trans Robot Autom 12(5):684–696

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Inria, Univ Rennes, CNRS, IRISA, Rennes, France

    François Chaumette

Authors
  1. François Chaumette
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to François Chaumette .

Editor information

Editors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Marcelo H. Ang

  2. Department of Computer Science, Stanford University, Stanford, CA, USA

    Oussama Khatib

  3. Department of Electrical Engineering, University of Naples Federico II, Naples, Italy

    Bruno Siciliano

Section Editor information

  1. Inria, Univ Rennes, CNRS, IRISA, Campus de Beaulieu, 35042, Rennes, France

    François Chaumette

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer-Verlag GmbH Germany, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Chaumette, F. (2020). Visual Servoing. In: Ang, M., Khatib, O., Siciliano, B. (eds) Encyclopedia of Robotics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41610-1_104-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-41610-1_104-1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-41610-1

  • Online ISBN: 978-3-642-41610-1

  • eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation