Solving the robot-world hand-eye(s) calibration problem with iterative methods


Robot-world, hand-eye calibration is the problem of determining the transformation between the robot end-effector and a camera, as well as the transformation between the robot base and the world coordinate system. This relationship has been modeled as \({\mathbf {AX}}={\mathbf {ZB}}\), where \({\mathbf {X}}\) and \({\mathbf {Z}}\) are unknown homogeneous transformation matrices. The successful execution of many robot manipulation tasks depends on determining these matrices accurately, and we are particularly interested in the use of calibration for use in vision tasks. In this work, we describe a collection of methods consisting of two cost function classes, three different parameterizations of rotation components, and separable versus simultaneous formulations. We explore the behavior of this collection of methods on real datasets and simulated datasets and compare to seven other state-of-the-art methods. Our collection of methods returns greater accuracy on many metrics as compared to the state-of-the-art. The collection of methods is extended to the problem of robot-world hand-multiple-eye calibration, and results are shown with two and three cameras mounted on the same robot.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    All of the datasets are available from [14].

  2. 2.

    Various different initial solutions were tested, and there was small or negligible difference in the solution quality versus using an identity matrix for rotation matrices and translation component with all elements zero. For this reason, we conclude that for the experiments covered in this paper, the first class of methods is not sensitive to initial solutions.

  3. 3.

    While use of tables is perhaps not the easiest for the reader, we note that the huge range of values for each of the six error metrics made other types of representation (graphs, etc.) infeasible.

  4. 4.

    Only the first author is mentioned in the tables’ text for brevity.


  1. 1.

    Agarwal, S., Mierle, K., et al.: Ceres solver.

  2. 2.

    Dornaika, F., Horaud, R.: Simultaneous robot-world and hand-eye calibration. IEEE Trans. Rob. Autom. 14(4), 617–622 (1998)

    Article  Google Scholar 

  3. 3.

    Hirsh, R.L., DeSouza, G.N., Kak, A.C.: An iterative approach to the hand-eye and base-world calibration problem. In: Robotics and automation, 2001. Proceedings 2001 ICRA. IEEE international conference on, vol. 3, pp. 2171–2176. IEEE (2001)

  4. 4.

    Horaud, R., Dornaika, F.: Hand-eye calibration. Int. J. Rob. Res. 14(3), 195–210 (1995)

  5. 5.

    Li, A., Lin, W., Defeng, W.: Simultaneous robot-world and hand-eye calibration using dual-quaternions and Kronecker product. Int. J. Phys. Sci. 5(10), 1530–1536 (2010)

    Google Scholar 

  6. 6.

    Lourakis, M.: levmar: Levenberg–Marquardt nonlinear least squares algorithms in C/C++. (July 2004). Accessed 18 Nov 2014

  7. 7.

    Malti, A.: Hand-eye calibration with epipolar constraints: application to endoscopy. Rob. Auton. Syst. 61(2), 161–169 (2013)

  8. 8.

    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11(2), 431–441 (1963)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Opencv. Version 2.4.9

  10. 10.

    Shah, M.: Solving the robot-world/hand-eye calibration problem using the Kronecker product. ASME J. Mech. Robot. 5(3), 031,007–031,007-7 (2013)

    Article  Google Scholar 

  11. 11.

    Shiu, Y.C., Ahmad, S.: Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX = XB. IEEE Trans. Rob. Autom. 5(1), 16–29 (1989)

    Article  Google Scholar 

  12. 12.

    Strobl, K.H., Hirzinger, G.: Optimal hand-eye calibration. In: 2006 IEEE/RSJ international conference on intelligent robots and systems, pp. 4647–4653 (2006). doi:10.1109/IROS.2006.282250

  13. 13.

    Tabb, A.: Shape from silhouette probability maps: reconstruction of thin objects in the presence of silhouette extraction and calibration error. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR) (2013)

  14. 14.

    Tabb, A.: Data from: Solving the robot-world hand-eye(s) calibration problem with iterative methods (2017). doi:10.15482/USDA.ADC/1340592

  15. 15.

    Tabb, A., Ahmad Yousef, K.: Parameterizations for reducing camera reprojection error for robot-world hand-eye calibration. In: IEEE RSJ international conference on intelligent robots and systems, pp. 3030–3037 (2015)

  16. 16.

    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Proceedings of the international workshop on vision algorithms: theory and practice, ICCV ’99, pp. 298–372. Springer, London (2000).

  17. 17.

    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000). doi:10.1109/34.888718

    Article  Google Scholar 

  18. 18.

    Zhuang, H., Roth, Z.S., Sudhakar, R.: Simultaneous robot/world and tool/flange calibration by solving homogeneous transformation equations of the form AX = YB. IEEE Trans. Rob. Autom. 10(4), 549–554 (1994). doi:10.1109/70.313105

    Article  Google Scholar 

Download references


We also would like to thank anonymous reviewers of our previous paper [15] and this paper for their helpful comments.

Author information



Corresponding author

Correspondence to Amy Tabb.

Ethics declarations

Conflict of interest

A. Tabb received a grant from the US National Science Foundation, grant number IOS-1339211.

Additional information

Mention of trade names or commercial products in this publication is solely for the purpose of providing specific information and does not imply recommendation or endorsement by the US Department of Agriculture. USDA is an equal opportunity provider and employer.

A. Tabb acknowledges the support of US National Science Foundation grant number IOS-1339211.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tabb, A., Ahmad Yousef, K.M. Solving the robot-world hand-eye(s) calibration problem with iterative methods. Machine Vision and Applications 28, 569–590 (2017).

Download citation


  • Robot
  • Hand-eye
  • Calibration
  • Reconstruction