International Journal of Computer Vision

, Volume 37, Issue 1, pp 79–97 | Cite as

2 1/2 D Visual Servoing with Respect to Unknown Objects Through a New Estimation Scheme of Camera Displacement

  • Ezio Malis
  • François Chaumette


Classical visual servoing techniques need a strong a priori knowledge of the shape and the dimensions of the observed objects. In this paper, we present how the 2 1/2 D visual servoing scheme we have recently developed, can be used with unknown objects characterized by a set of points. Our scheme is based on the estimation of the camera displacement from two views, given by the current and desired images. Since vision-based robotics tasks generally necessitate to be performed at video rate, we focus only on linear algorithms. Classical linear methods are based on the computation of the essential matrix. In this paper, we propose a different method, based on the estimation of the homography matrix related to a virtual plane attached to the object. We show that our method provides a more stable estimation when the epipolar geometry degenerates. This is particularly important in visual servoing to obtain a stable control law, especially near the convergence of the system. Finally, experimental results confirm the improvement in the stability, robustness, and behaviour of our scheme with respect to classical methods.

visual servoing projective geometry homography 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Basri, R., Rivlin, E., and Shimshoni, I. 1998. Visual homing: Surfing on the epipoles. In IEEE Int. Conf. on Computer Vision, Bombay, India, pp. 863–869.Google Scholar
  2. Boufama, B. and Mohr, R. 1995. Epipole and fundamental matrix estimation using the virtual parallax property. In IEEE Int. Conf. on Computer Vision, Cambridge, USA, pp. 1030–1036.Google Scholar
  3. Chaumette, F. 1998. Potential problems of stability and convergence in image-based and position-based visual servoing. In The Confluence of Vision and Control, D. Kriegman, G. Hager, and A. Morse, (Eds.). Springer Verlag, pp. 66–78. LNCIS Series, Vol. 237.Google Scholar
  4. Couapel, B. and Bainian, K. 1995. Stereo vision with the use of a virtual plane in the space. Chinese Journal of Electronics, 4(2):32–39.Google Scholar
  5. Criminisi, A., Reid, I., and Zisserman, A. 1998. Duality, rigidity and planar parallax. In European Conf. on Computer Vision. Freiburg, Germany, Vol. 2, pp. 846–861.Google Scholar
  6. Dementhon, D. and Davis, L.S. 1995. Model-based object pose in 25 lines of code. Int. Journal of Computer Vision, 15(1/2):123–141.Google Scholar
  7. Deriche, R., Zhang, Z., Luong, Q.-T., and Faugeras, O. 1994. Robust recovery of the epipolar geometry for an uncalibrated stereo rig. In European Conf. on Computer Vision, Stockholm, Sweden.Google Scholar
  8. Espiau, B. 1993. Effect of camera calibration errors on visual servoing in robotics. In 3rd Int. Symp. on Experimental Robotics, Kyoto, Japan.Google Scholar
  9. Espiau, B., Chaumette, F., and Rives, P. 1992. A new approach to visual servoing in robotics. IEEE Trans. on Robotics and Automation, 8(3):313–326.Google Scholar
  10. Faugeras, O. 1993. Three-dimensionnal computer vision: A geometric viewpoint. MIT Press, Cambridge, Massachusetts.Google Scholar
  11. Faugeras, O. and Lustman, F. 1988. Motion and structure from motion in a piecewise planar environment. Int. Journal of Pattern Recognition and Artificial Intelligence, 2(3):485–508.Google Scholar
  12. Hartley, R.I. 1992. Estimation of relative camera positions for uncalibrated cameras. In European Conf. on Computer Vision, Santa Margherita Ligure, Italy, pp. 579–587.Google Scholar
  13. Hartley, R.I. 1997. In defense of the eight-point algorithm. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(6):580–593.Google Scholar
  14. Hashimoto, K. 1993. Visual Servoing: Real Time Control of Robot Manipulators Based on Visual Sensory Feedback. World Scientific Press: Singapore. World Scientific Series in Robotics and Automated Systems, Vol. 7.Google Scholar
  15. Hosoda, K. and Asada, M. 1994. Versatile visual servoing without knowledge of true jacobian. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Munchen, Germany.Google Scholar
  16. Huang, T.S. and Faugeras, O. 1989. Some properties of the E matrix in two-view motion estimation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 11(12):1310–1312.Google Scholar
  17. Hutchinson, S., Hager, G.D., and Corke, P.I. 1996. A tutorial on visual servo control. IEEE Trans. on Robotics and Automation, 12(5):651–670.Google Scholar
  18. Irani, M., Anadan, P., and Weinshall, D. 1998. From reference frames to reference planes: multi-view parallax geometry and applications. In European Conf. on Computer Vision, Freiburg, Germany, Vol. 2, pp. 829–845.Google Scholar
  19. Jgersand, M., Fuentes, O., and Nelson, R. 1997. Experimental evaluation of uncalibrated visual servoing for precision manipulation. In IEEE Int. Conf. on Robotics and Automation, Albuquerque, New Mexico, Vol. 3, pp. 2874–2880.Google Scholar
  20. Jerian, C.P. and Jain, R. 1991. Structure from motion—A critical analysis of methods. IEEE Trans. on Systems, Man, and Cybernetics, 21(3):572–588.Google Scholar
  21. Longuet-Higgins, H.C. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133–135.Google Scholar
  22. Longuet-Higgins, H.C. 1984. The reconstruction of a scene from two projections: configurations that defeat the 8-point algorithm. In 1st Conf. on Artificial Intelligence Applications, Denver, pp. 395–397.Google Scholar
  23. Luong, Q.-T. and Faugeras, O. 1996. The fundamental matrix: Theory, algorithms, and stability analysis. Int. Journal of Computer Vision, 17(1):43–75.Google Scholar
  24. Malis, E. 1998. Contributions à la modélisation et à la commande en asservissement visuel. PhD Thesis, Université de Rennes I, IRISA.Google Scholar
  25. Malis, E., Chaumette, F., and Boudet, S. 1999. 2 1/2 D Visual Servoing IEEE Trans. on Robotics and Automation, 15(2):234–246.Google Scholar
  26. Samson, C., Le Borgne, M., and Espiau, B. 1991. Robot Control: the Task Function Approach, Clarendon Press, Oxford, England. Oxford Engineering Science Series, Vol. 22.Google Scholar
  27. Torr, P., Fitzgibbon, A.W., and Zisserman, A. 1998. Maintaining multiple motion model hypotheses over many views to recover matching and structure. In IEEE Int. Conf. on Computer Vision, Bombay, India, pp. 485–491.Google Scholar
  28. Tsai, R.Y. and Huang, T.S. 1984. Uniqueness and estimation of threedimensional motion parameters of rigid objects with curved surfaces. IEEE Trans. on Pattern Analysis and Machine Intelligence, 6(1):13–27.Google Scholar
  29. Weiss, L.E., Sanderson, A.C., and Neuman, C.P. 1987. Dynamic sensor-based control of robots with visual feedback. IEEE Journal of Robotics and Automation, 3(5):404–417.Google Scholar
  30. Wilson, W.J., Hulls, C.C.W., and Bell, G.S. 1996. Relative end-effector control using cartesian position-based visual servoing. IEEE Trans. on Robotics and Automation, 12(5):684–696.Google Scholar
  31. Zhang, Z. and Hanson, A.R. 1995. Scaled euclidean 3D reconstruction based on externally uncalibrated cameras. In IEEE Symp. on Computer Vision, Coral Gables, Florida.Google Scholar
  32. Zhang, Z. 1998. Determining the Epipolar Geometry and its Uncertainty—A Review. Int. Journal of Computer Vision, 27(2):161–195.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Ezio Malis
    • 1
  • François Chaumette
    • 1
  1. 1.IRISA/INRIA RennesRennes-cedexFrance

Personalised recommendations