Markerless Pose Tracking for Augmented Reality

  • Chunrong Yuan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4291)


In this paper a new approach is presented for markerless pose tracking in augmented reality. Using a tracking by detection approach, we estimate the 3D camera pose by detecting natural feature points in each input frame and building correspondences between 2D feature points. Instead of modeling the 3D environment, which is changing constantly and dynamically, we use a virtual square to define a 3D reference coordinate system. Camera pose can hence be estimated relative to it and the calculated 3D pose parameters can be used to render virtual objects into the real world. We propose and implement several strategies for robust matching, pose estimation and refinement. Experimental evaluation has shown that the approach is capable of online pose tracking and augmentation.


Feature Point Augmented Reality Virtual Object Reference Coordinate System World Coordinate System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)MATHGoogle Scholar
  2. 2.
    Szeliski, R., Kang, S.: Recovering 3D shape and motion from image streams using non–linear least squares. In: Technical Report CRL 93/3, Digital Equipment Corporation, Cambridge Research Laboratory (1993)Google Scholar
  3. 3.
    Guo, Y., Hsu, S., Samarasekera, S., Sawhney, H., Kumar, R.: Multi–view 3D analysis with applications for augmented reality and enhanced video visualization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2780–2781 (2000)Google Scholar
  4. 4.
    Cornelis, K., Pollefeys, M., Vergauwen, M., Gool, L.V.: Augmented reality using uncalibrated video sequences. In: Pollefeys, M., Van Gool, L., Zisserman, A., Fitzgibbon, A.W. (eds.) SMILE 2000. LNCS, vol. 2018, pp. 144–160. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  5. 5.
    Shi, J., Tomasi, C.: Good features to track. In: Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 593–600 (1994)Google Scholar
  6. 6.
    Hager, G., Belhumeur, P.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Analysis and Machine Intelligence 20, 1025–1039 (1998)CrossRefGoogle Scholar
  7. 7.
    Jurie, F., Dhome, M.: Real time robust template matching. In: The 13th British Machine Vision Conference, Cardiff, UK, pp. 123–132 (2002)Google Scholar
  8. 8.
    Isard, M., Blake, A.: Condensation — conditional density propagation for visual tracking. Int. Journal of Computer Vision 29, 5–28 (1998)CrossRefGoogle Scholar
  9. 9.
    Jaynes, C., Hou, J.: Temporal registration using a kalman filter for augmented reality. In: Vision Interface, Montreal, Canada (2000)Google Scholar
  10. 10.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel–based object tracking. IEEE Trans. Pattern Analysis and Machine Intelligence 25, 564–577 (2003)CrossRefGoogle Scholar
  11. 11.
    Claus, D., Fitzgibbon, A.W.: Reliable fiducial detection in natural scenes. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 469–480. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video–based augmented reality conferencing system. In: The 2nd Int. Workshop on Augmented Reality, San Francisco, USA, pp. 85–94 (1999)Google Scholar
  13. 13.
    Fiala, M.: ARTAG, a fiducial marker system using digital techniques. In: IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, pp. 590–596 (2005)Google Scholar
  14. 14.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: The 4th Alvey Vision Conference, Manchester, pp. 189–192 (1988)Google Scholar
  15. 15.
    Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In: The 7th European Conference on Computer Vision, Copenhagen, Denmark, pp. 128–142 (2002)Google Scholar
  16. 16.
    Lowe, D.G.: Distinctive image features from scale–invariant keypoints. Int. Journal of Computer Vision 60, 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Ke, Y., Sukthankar, R.: PCA–SIFT: A more distinctive representation for local image descriptors. In: IEEE Conference on Computer Vision and Pattern Recognition, Washington DC, USA, pp. 506–513 (2004)Google Scholar
  18. 18.
    Lepetit, V., Lagger, P., Fua, P.: Randomized trees for real–time keypoint recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, pp. 775–781 (2005)Google Scholar
  19. 19.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Analysis and Machine Intelligence 27, 1625–1630 (2005)CrossRefGoogle Scholar
  20. 20.
    Skrypnyk, I., Lowe, D.: Scene modeling, recognition and tracking with invariant image features. In: Third IEEE and ACM International Symposium on Mixed and Augmented Reality, Arlington, VA, USA, pp. 110–119 (2004)Google Scholar
  21. 21.
    Bleser, G., Pastarmov, Y., Stricker, D.: Real–time 3D camera tracking for industrial augmented reality applications. In: The 13th Int. Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, Plzen, Czech Republic (2005)Google Scholar
  22. 22.
    Lepetit, V., Vacchetti, L., Thalmann, D., Fua, P.: Fully automated and stable registration for augmented reality applications. In: Second Int. Symposium on Mixed and Augmented Reality, Tokyo, Japan, pp. 93–101 (2003)Google Scholar
  23. 23.
    Comport, A.I., Marchand, E., Pressigout, M., Chaumette, F.: Real–time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans. on Visualization and Computer Graphics 12, 615–628 (2006)CrossRefGoogle Scholar
  24. 24.
    Simon, G., Fitzgibbon, A., Zisserman, A.: Markerless tracking using planar structures in the scene. In: Int. Symposium on Augmented Reality, Munich, Germany, pp. 120–128 (2000)Google Scholar
  25. 25.
    Ferrari, V., Tuytelaars, T., Van Gool, L.: Markerless augmented reality with a real–time affine region tracker. In: International Symposium on Mixed and Augmented Reality, New York, USA, pp. 87–96 (2001)Google Scholar
  26. 26.
    Chia, K., Cheok, A., Prince, S.: Online 6DOF augmented reality registration from natural features. In: First Int. Symposium on Mixed and Augmented Reality, Darmstadt, Germany, pp. 305–313 (2002)Google Scholar
  27. 27.
    Najafi, H., Navab, N., Klinker, G.: Automated initialization for marker–less tracking: A sensor fusion approach. In: Third IEEE and ACM International Symposium on Mixed and Augmented Reality, Arlington, VA, USA, pp. 110–119 (2004)Google Scholar
  28. 28.
    Koch, R., Koeser, K., Streckel, B., Evers-Senne, J.: Markerless image–based 3d tracking for real–time augmented reality applications. In: The 7th Int. Workshop on Image analysis for multimedia interactive services, Montreux, Switzerland (2005)Google Scholar
  29. 29.
    Fischler, M., Bolles, C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of ACM 24, 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Horaud, R., Conio, B., Leboulleux, O.: An analytical solution for the perspective–4–point problem. Computer Vision, Graphics, and Image Processing 47, 33–44 (1989)CrossRefGoogle Scholar
  31. 31.
    DeMenthon, D., Davis, L.: Model–based object pose in 25 lines of code. Int. Journal of Computer Vision 15, 123–141 (1995)CrossRefGoogle Scholar
  32. 32.
    Oberkampf, D., De Menthon, D., Davis, L.: Iterative pose estimation using coplanar feature points. Computer Vision and Image Understanding 63, 495–511 (1996)CrossRefGoogle Scholar
  33. 33.
    Schweighofer, G., Pinz, A.: Robust pose estimation from a planar target. In: Technical Report, TR–EMT–2005–01, Graz University of Technology (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Chunrong Yuan
    • 1
  1. 1.Schloss BirlinghovenFraunhofer Institute for Applied Information TechnologySankt AugustinGermany

Personalised recommendations