Sensor Calibration, Modeling, and Simulation

  • Qianli Ma
  • Gregory S. Chirikjian
Living reference work entry


For humanoid robots, sensors are a critical aspect of the entire robot system because they are responsible for accurately perceiving the environment. Due to the cost of real experiments on humanoids, sensor modeling and simulation play important roles in robotics research. For applications in the real world, sensor calibration is the prerequisite for sensors to function properly, and this chapter begins with a review of general problems in sensor calibration, modeling, and simulation. An often used formulation of sensor calibration in robotics and computer vision is “AX = XB,” where A, X, and B are rigid-body transformations with A and B given from sensor measurements and X is the unknown rigid-body transformation to be calibrated. Many methods have been proposed to solve X given data streams of A and B under different scenarios. This chapter presents the most complete picture of the AX = XB solvers to date. First, a brief overview of the various important sensor calibration techniques is given and problems of interest are highlighted. Then, a detailed review of the various “AX = XB” algorithms is presented. The notations used in different algorithms are unified to show the interconnections between the selected methods in a straightforward way. Finally, the criterion for data selection and various error metrics are introduced, which are of critical importance for evaluating the performance of AX = XB solvers.


Sensor calibration Hand-eye calibration Humanoid robot Review 


  1. 1.
    S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics (MIT press, Cambridge/Mass, 2005)zbMATHGoogle Scholar
  2. 2.
    ARS. Accessed 03 Dec 2017
  3. 3.
    N. Koenig, A. Howard, Design and use paradigms for gazebo, an open-source multi-robot simulator, in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004, pp. 2149–2154Google Scholar
  4. 4.
    C. Aguero, N. Koenig, I. Chen, H. Boyer, S. Peters, J. Hsu, B. Gerkey, S. Paepcke, J. Rivero, J. Manzo, E. Krotkov, G. Pratt, Inside the virtual robotics challenge: simulating real-time robotic disaster response. IEEE Trans. Autom. Sci. Eng. 12(2), 494–506 (2015)CrossRefGoogle Scholar
  5. 5.
    G. Echeverria, S. Lemaignan, A. Degroote, S. Lacroix, M. Karg, P. Koch, C. Lesire, S. Stinckwich, Simulating complex robotic scenarios with morse, in SIMPAR, 2012, pp. 197–208CrossRefGoogle Scholar
  6. 6.
    Webots, Commercial Mobile Robot Simulation Software
  7. 7.
    O. Michel, Webots: professional mobile robot simulation. Int. J. Adv. Robot. Syst. 1(1), 39–42 (2004)CrossRefGoogle Scholar
  8. 8.
    E. Rohmer, S.P. Singh, M. Freese, V-rep: a versatile and scalable robot simulation framework, in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2013), pp. 1321–1326Google Scholar
  9. 9.
    RoboDK, Accessed 03 Dec 2017
  10. 10.
    SimSpark, Accessed 03 Dec 2017
  11. 11.
  12. 12.
    PreScan, Accessed 03 Dec 2017
  13. 13.
    AdasWorks, Accessed 03 Dec 2017
  14. 14.
    Google, Accessed 03 Dec 2017
  15. 15.
    BMW, Accessed 03 Dec 2017
  16. 16.
  17. 17.
  18. 18.
    G. Champleboux, S. Lavallee, R. Szeliski, L. Brunie, From accurate range imaging sensor calibration to accurate model-based 3D object localization, in Computer Vision and Pattern Recognition (IEEE, 1992), pp. 83–89Google Scholar
  19. 19.
    Z. Zhang, R. Deriche, O. Faugeras, Q.-T. Luong, A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artif. Intell. 78(1), 87–119 (1995)CrossRefGoogle Scholar
  20. 20.
    R.I. Hartley, P. Sturm, Triangulation. Comput. Vis. Image Underst. 68(2), 146–157 (1997)CrossRefGoogle Scholar
  21. 21.
    P.H. Torr, A. Zisserman, Mlesac: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78(1), 138–156 (2000)CrossRefGoogle Scholar
  22. 22.
    F. Zhou, G. Zhang, Complete calibration of a structured light stripe vision sensor through planar target of unknown orientations. Image Vis. Comput. 23(1), 59–67 (2005)CrossRefGoogle Scholar
  23. 23.
    V. Bychkovskiy, S. Megerian, D. Estrin, M. Potkonjak, A collaborative approach to in-place sensor calibration, in Proceedings of Second International Workshop Information Processing in Sensor Networks (IPSN 2003), Palo Alto, 22–23 April 2003 (Springer, 2003), pp. 301–316CrossRefGoogle Scholar
  24. 24.
    J.M. Hollerbach, C.W. Wampler, The calibration index and taxonomy for robot kinematic calibration methods. Int. J. Robot. Res. 15(6), 573–591 (1996)CrossRefGoogle Scholar
  25. 25.
    C.G. Atkeson, C.H. An, J.M. Hollerbach, Estimation of inertial parameters of manipulator loads and links. Int. J. Robot. Res. 5(3), 101–119 (1986)CrossRefGoogle Scholar
  26. 26.
    J.J. Craig, P. Hsu, S.S. Sastry, Adaptive control of mechanical manipulators. Int. J. Robot. Res. 6(2), 16–28 (1987)CrossRefGoogle Scholar
  27. 27.
    J.-J.E. Slotine, W. Li, On the adaptive control of robot manipulators. Int. J. Robot. Res. 6(3), 49–59 (1987)CrossRefGoogle Scholar
  28. 28.
    R. Ortega, M.W. Spong, Adaptive motion control of rigid robots: a tutorial. Automatica 25(6), 877–888 (1989)MathSciNetCrossRefGoogle Scholar
  29. 29.
    L.L. Whitcomb, A.A. Rizzi, D.E. Koditschek, Comparative experiments with a new adaptive controller for robot arms. IEEE Trans. Robot. Autom. 9(1), 59–70 (1993)CrossRefGoogle Scholar
  30. 30.
    R. Tsai, Y. Lenz, A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans. Robot. Autom. 5(3), 345–358 (1989)CrossRefGoogle Scholar
  31. 31.
    E. Mair, M. Fleps, M. Suppa, D. Burschka, Spatio-temporal initialization for IMU to camera registration, in IEEE International Conference on Robotics and Biomimetics (IEEE, 2011), pp. 557–564Google Scholar
  32. 32.
    J. Schmidt, F. Vogt, H. Niemann, Robust hand–eye calibration of an endoscopic surgery robot using dual quaternions, in Proceedings of the 25th DAGM Symposium on Pattern Recognition, Magdeburg, Sept 2003 (Springer, 2003), pp. 548–556Google Scholar
  33. 33.
    H. Malm, A. Heyden, A new approach to hand-eye calibration, in International Conference on Pattern Recognition, vol. 1 (IEEE, 2000), pp. 525–529Google Scholar
  34. 34.
    J. Heller, M. Havlena, T. Pajdla, A branch-and-bound algorithm for globally optimal hand-eye calibration, in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1608–1615Google Scholar
  35. 35.
    T. Ruland, T. Pajdla, L. Kruger, Globally optimal hand-eye calibration, in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1035–1042Google Scholar
  36. 36.
    H. Wu, W. Tizzano, T.T. Andersen, N.A. Andersen, O. Ravn, Hand-eye calibration and inverse kinematics of robot arm using neural network, in Robot Intelligence Technology and Applications 2 (Springer, Cham, 2014), pp. 581–591Google Scholar
  37. 37.
    S.-J. Kim, M. Jeong, J. Lee, J. Lee, K. Kim, B. You, S. Oh, Robot head-eye calibration using the minimum variance method in IEEE International Conference on Robotics and Biomimetics (ROBIO) (IEEE, 2010), pp. 1446–1451Google Scholar
  38. 38.
    Y. Liu, Q. Wang, Y. Li, Calibration of a robot hand-eye system with a concentric circles target, in International Bhurban Conference on Applied Sciences and Technology (IEEE, 2015), pp. 204–209Google Scholar
  39. 39.
    Q. Ma, H. Li, G.S. Chirikjian, New probabilistic approaches to the ax = xb hand-eye calibration without correspondence, in 2016 IEEE International Conference on Robotics and Automation (IEEE, 2016), pp. 4365–4371Google Scholar
  40. 40.
    H. Zhuang, Z.S. Roth, R. Sudhakar, Simultaneous robot/world and tool/flange calibration by solving homogeneous transformation equations of the form AX = YB. IEEE Trans. Robot. Autom. 10(4), 549–554 (1994)CrossRefGoogle Scholar
  41. 41.
    F. Dornaika, R. Horaud, Simultaneous robot-world and hand-eye calibration. IEEE Trans. Robot. Autom. 14(4), 617–622 (1998)CrossRefGoogle Scholar
  42. 42.
    R.L. Hirsh, G.N. DeSouza, A.C. Kak, An iterative approach to the hand-eye and base-world calibration problem, in IEEE International Conference on Robotics and Automation, vol. 3 (IEEE, 2001), pp. 2171–2176Google Scholar
  43. 43.
    A. Li, L. Wang, D. Wu, Simultaneous robot-world and hand-eye calibration using dual-quaternions and kronecker product. Int. J. Phys. Sci. 5(10), 1530–1536 (2010)Google Scholar
  44. 44.
    F. Ernst, L. Richter, L. Matthäus, V. Martens, R. Bruder, A. Schlaefer, A. Schweikard, Non-orthogonal tool/flange and robot/world calibration. Int. J. Med. Rob. Comput. Assisted Surg. 8(4), 407–420 (2012)CrossRefGoogle Scholar
  45. 45.
    J. Heller, D. Henrion, T. Pajdla, Hand-eye and robot-world calibration by global polynomial optimization, in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 3157–3164Google Scholar
  46. 46.
    H. Li, Q. Ma, T. Wang, G.S. Chirikjian, Simultaneous hand-eye and robot-world calibration by solving the problem without correspondence. IEEE J. Robot. Autom. Lett. 1(1), 145–152 (2016)CrossRefGoogle Scholar
  47. 47.
    J. Wang, L. Wu, M.Q.-H. Meng, H. Ren, Towards simultaneous coordinate calibrations for cooperative multiple robots, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2014), pp. 410–415Google Scholar
  48. 48.
    Q. Ma, Z. Goh, G.S. Chirikjian, Probabilistic approaches to the AXB = YCZ calibration problem in multi-robot systems, in Proceedings of Robotics: Science and Systems, 2016Google Scholar
  49. 49.
    H. Chen, A screw motion approach to uniqueness analysis of head-eye geometry, in IEEE Conference on Computer Vision and Pattern Recognition Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’91), (IEEE, 1991) pp. 145–151Google Scholar
  50. 50.
    F. Park, B. Martin, Robot sensor calibration: solving AX = XB on the Euclidean group. IEEE Trans. Robot. Autom. 10(5), 717–721 (1994)CrossRefGoogle Scholar
  51. 51.
    Y. Shiu, S. Ahmad, Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX = XB. IEEE Trans. Robot. Autom. 5(1), 16–29 (1989)CrossRefGoogle Scholar
  52. 52.
    J. Chou, M. Kamel, Finding the position and orientation of a sensor on a robot manipulator using quaternions. Int. J. Robot. Res. 10(3), 240–254 (1991)CrossRefGoogle Scholar
  53. 53.
    Y. Shiu, S. Ahmad, Finding the mounting position of a sensor by solving a homogeneous transform equation of the form AX = XB, in IEEE International Conference on Robotics and Automation, vol. 4 (IEEE, 1987), pp. 1666–1671Google Scholar
  54. 54.
    Y. Dai, J. Trumpf, H. Li, N. Barnes, R. Hartley, Rotation averaging with application to camera-rig calibration, in Asian Conference on Computer Vision (Springer, Berlin/Heidelberg, 2009), pp. 335–346Google Scholar
  55. 55.
    M. Shah, R. Eastman, T. Hong, An overview of robot-sensor calibration methods for evaluation of perception systems, in Proceedings of the Workshop on Performance Metrics for Intelligent Systems (ACM, 2012), pp. 15–20Google Scholar
  56. 56.
    I. Fassi, G. Legnani, Hand to sensor calibration: a geometrical interpretation of the matrix equation AX = XB. J. Robot. Syst. 22(9), 497–506 (2005)CrossRefGoogle Scholar
  57. 57.
    Z. Zhao, Y. Liu, Hand-eye calibration based on screw motions, in International Conference on Pattern Recognition, vol. 3 (IEEE, 2006), pp. 1022–1026Google Scholar
  58. 58.
    S. Gwak, J. Kim, F.C. Park, Numerical optimization on the Euclidean group with applications to camera calibration. IEEE Trans. Robot. Autom. 19(1), 65–74 (2003)CrossRefGoogle Scholar
  59. 59.
    J.C. Chou, M. Kamel, Quaternions approach to solve the kinematic equation of rotation, AX = XB, of a sensor-mounted robotic manipulator, in IEEE International Conference on Robotics and Automation (IEEE, 1988), pp. 656–662Google Scholar
  60. 60.
    R. Horaud, F. Dornaika, Hand-eye calibration. Int. J. Robot. Res. 14(3), 195–210 (1995)CrossRefGoogle Scholar
  61. 61.
    K. Daniilidis, E. Bayro-Corrochano, The dual quaternion approach to hand-eye calibration, in International Conference on Pattern Recognition, vol. 1 (IEEE, 1996), pp. 318–322Google Scholar
  62. 62.
    K. Daniilidis, Hand-eye calibration using dual quaternions. Int. J. Robot. Res. 18(3), 286–298 (1999)CrossRefGoogle Scholar
  63. 63.
    N. Andreff, R. Horaud, B. Espiau, On-line hand-eye calibration, in International Conference on 3D Digital Imaging and Modeling (IEEE, 1999), pp. 430–436Google Scholar
  64. 64.
    N. Andreff, R. Horaud, B. Espiau, Robot hand-eye calibration using structure-from-motion. Int. J. Robot. Res. 20(3), 228–248 (2001)CrossRefGoogle Scholar
  65. 65.
    Z. Zhao, Hand-eye calibration using convex optimization, in IEEE International Conference on Robotics and Automation (IEEE, 2011), pp. 2947–2952Google Scholar
  66. 66.
    Y. Seo, Y.-J. Choi, S.W. Lee, A branch-and-bound algorithm for globally optimal calibration of a camera-and-rotation-sensor system, in International Conference on Computer Vision (IEEE, 2009), pp. 1173–1178Google Scholar
  67. 67.
    J. Schmidt, F. Vogt, H. Niemann, Calibration–free hand–eye calibration: a structure–from–motion approach, in Joint Pattern Recognition Symposium (Springer, Berlin/Heidelberg, 2005), pp. 67–74Google Scholar
  68. 68.
    J. Angeles, G. Soucy, F.P. Ferrie, The online solution of the hand-eye problem. IEEE Trans. Robot. Autom. 16(6), 720–731 (2000)CrossRefGoogle Scholar
  69. 69.
    M.K. Ackerman, A. Cheng, E. Boctor, G. Chirikjian, Online ultrasound sensor calibration using gradient descent on the Euclidean group, in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4900–4905Google Scholar
  70. 70.
    M.K. Ackerman, G.S. Chirikjian, A probabilistic solution to the AX = XB problem: sensor calibration without correspondence, in Geometric Science of Information (Springer, Berlin, 2013), pp. 693–701zbMATHGoogle Scholar
  71. 71.
    M.K. Ackerman, A. Cheng, G. Chirikjian, An information-theoretic approach to the correspondence-free AX = XB sensor calibration problem, in IEEE International Conference on Robotics and Automation (IEEE, 2014), pp. 4893–4899Google Scholar
  72. 72.
    B. Horn, Robot Vision (MIT Press, Cambridge/Mass, 1986)Google Scholar
  73. 73.
    V. Lepetit, F. Moreno-Noguer, P. Fua, EPnP: an accurate o(n) solution to the PnP problem. Int. J. Comput. Vis. 81(2), 155–166 (2009)CrossRefGoogle Scholar
  74. 74.
    G.S. Chirikjian, A.B. Kyatkin, Engineering Applications of Noncommutative Harmonic Analysis: with Emphasis on Rotation and Motion Groups (CRC Press, Boca Raton, 2001)zbMATHGoogle Scholar
  75. 75.
    Y. Wang, G.S. Chirikjian, Nonparametric second-order theory of error propagation on motion groups. Int. J. Robot. Res. 27(11–12), 1258–1273 (2008)CrossRefGoogle Scholar
  76. 76.
    M. Ackerman, A. Cheng, B. Shiffman, E. Boctor, G.S. Chirikjian, Sensor calibration with unknown correspondence: solving AX = XB using Euclidean-group invariants, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2013), pp. 1308–1313Google Scholar
  77. 77.
    J.S.F. Vogt, H. Niemann, Vector quantization based data selection for hand-eye calibration, in Proceedings of the Vision, Modeling, and Visualization, Standford, 16–18 Nov 2004 (IOS Press, 2004), p. 21Google Scholar
  78. 78.
    J. Schmidt, H. Niemann, Data selection for hand-eye calibration: a vector quantization approach. Int. J. Robot. Res. 27(9), 1027–1053 (2008)CrossRefGoogle Scholar
  79. 79.
    F. Shi, J. Wang, Y. Liu, An approach to improve online hand-eye calibration, in Pattern Recognition and Image Analysis (Springer, Berlin/Heidelberg, 2005), pp. 647–655Google Scholar
  80. 80.
    J. Zhang, F. Shi, Y. Liu, An adaptive selection of motion for online hand-eye calibration, in Advances in Artificial Intelligence (Springer, Sydney, 2005), pp. 520–529zbMATHGoogle Scholar
  81. 81.
    K.H. Strobl, G. Hirzinger, Optimal hand-eye calibration, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2006), pp. 4647–4653Google Scholar
  82. 82.
    G.S. Chirikjian, Stochastic Models, Information Theory, and Lie Groups, vol. 2 (Birkauser, Boston, 2011)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.Robot and Protein Kinematics Laboratory, Laboratory for Computational Sensing and Robotics, Department of Mechanical EngineeringThe Johns Hopkins UniversityBaltimoreUSA

Personalised recommendations