Building a Kinematic Model of a Robot’s Arm with a Depth Camera

  • Alan Broun
  • Chris Beck
  • Tony Pipe
  • Majid Mirmehdi
  • Chris Melhuish
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7429)


We present a system which first builds and tracks a model of a robot’s hand using a depth camera, and then uses this ability to construct a kinematic model of its own arm using very little prior information. The system is flexible, and easy to integrate with different robots, because the model building process does not require any fiducial markers to be attached to the robot’s hand. To validate the models built by the system we perform a number of experiments. The results of the experiments demonstrate that the hand model built by the system can be tracked with a precision in the order of 1mm, and that the kinematic model is accurate enough for reliably positioning the hand of the robot in camera space.


Point Cloud Kinematic Model Kinematic Chain Iterative Close Point Revolute Joint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bennett, D.J., Hollerbach, J.M.: Kinematic calibration by direct estimation of the Jacobian matrix. Robotics and Automation, 351–357 (1992)Google Scholar
  2. 2.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 239–256 (1992)Google Scholar
  3. 3.
    Chen, Y., Medioni, G.: Object modeling by registration of multiple range images. In: IEEE International Conference on Robotics and Automation, pp. 2724–2729 (1991)Google Scholar
  4. 4.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D Mapping: Using depth cameras for dense 3D modeling of indoor environments. In: 12th International Symposium on Experimental Robotics, ISER (2010)Google Scholar
  5. 5.
    Hersch, M., Sauser, E., Billard, A.: Online learning of the body schema. International Journal of Humanoid Robotics, 161 (2008)Google Scholar
  6. 6.
    Hoffmann, M., Marques, H., Arieta, A.: Body schema in robotics: A review. IEEE Transactions on Autonomous Mental Development, 304–324 (2010)Google Scholar
  7. 7.
    Kijewski, W.: SI Units, Conversion & Measurement Skills (1999)Google Scholar
  8. 8.
    Krainin, M., Henry, P., Ren, X.: Manipulator and object tracking for in hand 3d object modeling. University of Washington, Tech. Rep. UW-CSE-10, pp. 1–34 (2010)Google Scholar
  9. 9.
    Lepetit, V., Fua, P.: Monocular model-based 3D tracking of rigid bbjects: A survey. Foundations and Trends in Computer Graphics and Vision, 1–89 (2005)Google Scholar
  10. 10.
    Mason, M.T.: Mechanics of robotic manipulation. MIT Press (2001)Google Scholar
  11. 11.
    Mooring, B., Driels, M., Roth, Z.: Fundamentals of manipulator calibration. Wiley (1991)Google Scholar
  12. 12.
    Murray, R.M., Li, Z., Sastry, S.S.: A mathematical introduction to robotic manipulation (1994)Google Scholar
  13. 13.
    Pfister, H., Zwicker, M., Van Baar, J., Gross, M.: Surfels: Surface elements as rendering primitives. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 335–342 (2000)Google Scholar
  14. 14.
    Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)Google Scholar
  15. 15.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001)Google Scholar
  16. 16.
    Stone, H., Sanderson, A., Neuman, C.: Arm signature identification. In: IEEE International Conference on Robotics and Automation, pp. 41–48 (1986)Google Scholar
  17. 17.
    Sturm, J., Plagemann, C., Burgard, W.: Body schema learning for robotic manipulators from visual self-perception. In: Journal of Physiology, pp. 220–231 (2009)Google Scholar
  18. 18.
    Sudderth, E.B., Mandel, M.I., Freeman, W.T., Willsky, A.S.: Visual hand tracking using nonparametric belief propagation. In: Conference on Computer Vision and Pattern Recognition Workshop, pp. 189–189 (2004)Google Scholar
  19. 19.
    Weise, T., Wismer, T., Leibe, B., Van Gool, L.: In-hand scanning with online loop closure. In: IEEE 12th International Conference on Computer Vision, pp. 1630–1637 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Alan Broun
    • 1
  • Chris Beck
    • 2
  • Tony Pipe
    • 1
  • Majid Mirmehdi
    • 2
  • Chris Melhuish
    • 1
  1. 1.Bristol Robotics LaboratoryBristolUK
  2. 2.Visual Information LaboratoryUniversity of BristolBristolUK

Personalised recommendations