Journal of Real-Time Image Processing

, Volume 5, Issue 2, pp 129–138 | Cite as

Real-time optical markerless tracking for augmented reality applications

  • Iñigo BarandiaranEmail author
  • Céline Paloc
  • Manuel Graña
Special Issue


Augmented reality (AR) technology consists in adding computer-generated information (2D/3D) to a real video sequence in such a manner that the real and virtual objects appear coexisting in the same world. To get a realistic illusion, the real and virtual objects must be properly aligned with respect to each other, which requires a robust real-time tracking strategy—one of the bottlenecks of AR applications. In this paper, we describe the limitations and advantages of different optical tracking technologies, and we present our customized implementation of both recursive tracking and tracking by detection approaches. The second approach requires the implementation of a classifier and we propose the use of a Random Forest classifier. We evaluated both approaches in the context of an AR application for design review. Some conclusions regarding the performance of each approach are given.


Augmented reality Optical markerless tracking Tracking by detection 



This work has been partially funded under the sixth Framework Programme of the European Union (EU) within the project “IMPROVE”: IST FP6- 004785.


  1. 1.
    Barandiaran, I., Cottez, C., Paloc, C., Graña, M.: Random forest classifier for real-time optical markerless tracking. Proc. VISAPP 2, 559–564 (2008). ISBN:978-989-8111-21-0Google Scholar
  2. 2.
    Barandiaran, I., Cottez, C., Paloc, C., Graña, M.: Comparative evaluation of Random Forest and Fern classifiers for real-time feature matching. In: Proceedings of WSCG, pp. 59–166 (2008). ISBN:978-80-86943-15-2Google Scholar
  3. 3.
    Boffy, A., Tsin, Y., Genc, Y.: Real-time feature matching using adaptative and spatially distributed classification trees. Proc. BMVC 2, 529–539 (2006) Google Scholar
  4. 4.
    Breiman, L.: Random forests. Mach. Learn. J. 45, 5–32 (2001)zbMATHCrossRefGoogle Scholar
  5. 5.
    Chandaria, J., Graham, T., Stricker, D.: The MATRIS project: real-time markerless camera tracking for Augmented Reality and broadcast applications. J. Real Time Image Process. 2, 69–79 (2007). doi: 10.1007/s11554-007-0043-z CrossRefGoogle Scholar
  6. 6.
    Davison, A.J., Mayol, W., Murray, D.W.: Real-time localisation and mapping with wearable active vision. In: Proceedings of ISMAR (2003)Google Scholar
  7. 7.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981) Google Scholar
  8. 8.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, London (2004). ISBN:0521-54051-8Google Scholar
  9. 9.
    Harris, C. Stephens, M.J.: A combined corner and edge detector. In: Alvey Vision Conference, pp. 147–152 (1988)Google Scholar
  10. 10.
    Lepetit, V., Pilet, J., Fua, P.: Point matching as a classification problem for fast and robust object pose estimation. In: Proceedings of CVPR, vol. 2, pp. 244–250 (2004). ISSN: 1063-6919Google Scholar
  11. 11.
    Lepetit, V., Fua, P.: Monocular model-based 3D object tracking of rigid objects: a survey. Found. Trends Comput. Graph. Vis. 1, 1–89 (2005)Google Scholar
  12. 12.
    Lepetit, V., Fua, P.: Keypoint recognition using randomized trees. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006). ISSN: 0162-8828Google Scholar
  13. 13.
    Lowe, D.: Distinctive image features from scale invariants keypoints. Int. J. Comput. Vis. 20(2), 91–110 (2004)CrossRefGoogle Scholar
  14. 14.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L.V.: A comparison of affine region detectors. Int. J. Comput. Vis. 65(1–2), 43–72 (2005). ISSN: 0920-5691Google Scholar
  15. 15.
    Özuysal, M., Fua, P., Lepetit, V.: Feature harvesting for tracking-by-detection. In: Proceedings of European Conference on Computer Vision, pp. 592–605 (2006). ISBN:3-540-33836-5Google Scholar
  16. 16.
    Reitmayr, G., Schmalstieg, D.: OpenTracker: a flexible software design for three-dimensional interaction. Virtual Real. J. 9, 79–92 (2005). doi: 10.1007/s10055-005-0006-2 CrossRefGoogle Scholar
  17. 17.
    Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Proceedings of European Conference on Computer Vision, pp. 430–443 (2006). ISBN:3540338322Google Scholar
  18. 18.
    Vacchetti, L., Lepetit, V., Fua, P.: Combining edge and texture information for real-time accurate 3D camera tracking. In: Proceedings of IEEE and AM International Symposium on Mixed and Augmented Reality, vol. 4, pp. 48–57 (2004). ISBN:0-7695-2191-6Google Scholar
  19. 19.
    Williams, B., Klein, G., Reid, I.: Real-time SLAM relocalisation. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  20. 20.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000). doi: 10.1109/34.888718 CrossRefGoogle Scholar
  21. 21.
    Lepetit, V.: On computer vision for augmented reality. In: Proceedings of the International Symposium on Ubiquitous Virtual Reality, pp. 13–16 (2008)Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Iñigo Barandiaran
    • 1
    Email author
  • Céline Paloc
    • 1
  • Manuel Graña
    • 1
  1. 1.VICOMTechSan SebastiánSpain

Personalised recommendations