Augmented Reality Using Uncalibrated Video Sequences

  • Kurt Cornelis
  • Marc Pollefeys
  • Maarten Vergauwen
  • Luc Van Gool
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2018)


Augmented Reality(AR) aims at merging the real and the virtual in order to enrich a real environment with virtual information. Augmentations range from simple text annotations accompanying real objects to virtual mimics of real-life objects inserted into a real environment. In the latter case the ultimate goal is to make it impossible to differentiate between real and virtual objects. Several problems need to be overcome before realizing this goal. Amongst them are the rigid registration of virtual objects into the real environment, the problem of mutual occlusion of real and virtual objects and the extraction of the illumination distribution of the real environment in order to render the virtual objects with this illumination model. This paper will unfold how we proceeded to implement an Augmented Reality System that registers virtual objects into a totally uncalibrated video sequence of a real environment that may contain some moving parts. The other problems of occlusion and illumination will not be discussed in this paper but are left as future research topics.


Video Sequence Augmented Reality Real Environment Virtual Object Bundle Adjustment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. Azuma: “A Survey of Augmented Reality.” ACM SIGGRAPH’ 95 Course Notes No. 9-Developing Advanced Virtual Reality Applications, (August 1995)Google Scholar
  2. 2.
    P. Beardsley, P. Torr and A. Zisserman: “3D Model Acquisition from Extended Image Sequences” Computer Vision-ECCV’96, Lecture Notes in Computer Science, Vol. 1065, Springer-Verlag, pp. 683–695, 1996Google Scholar
  3. 3.
    M.-O. Berger: “Resolving Occlusion in Augmented Reality: a Contour Based Approach without 3D Reconstruction.”Google Scholar
  4. 4.
    M.-O Berger, G. Simon, S. Petitjean and B. Wrobel-Dautcourt: “Mixing Synthesis and Video Images of Outdoor Environments: Application to the Bridges of Paris. ” ICPR’96, pp. 90–94, 1996Google Scholar
  5. 5.
    D. E. Breen, R. T. Whitaker and E. Rose: “Interactive Occlusion and Collision of Real and Virtual Objects in Augmented Reality.” Technical Report ECRC-95-02, ECRC, Munich, Germany, 1995Google Scholar
  6. 6.
    P. Debevec: “Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography. ” Proceedings SIGGRAPH 98, pp. 189–198, July 1998Google Scholar
  7. 7.
    P. Debevec, Y. Yu, and G. Borshukov: “Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping.” Technical Report UCB//CSD-98-1003, University of California at Berkeley, 1998Google Scholar
  8. 8.
    O. Faugeras, Q.-T. Luong and S. Maybank: “Camera self-calibration: Theory and experiments” Computer Vision-ECCV’92, Lecture Notes in Computer Science, Vol. 588, Springer-Verlag, pp. 321–334, 1992Google Scholar
  9. 9.
    C. Harris and M. Stephens: “A combined corner and edge detector” Fourth Alvey Vision Conference, pp. 147–151, 1988Google Scholar
  10. 10.
    R. Hartley: “Cheirality invariants” Proc. D.A.R.P.A. Image Understanding Work-shop, pp. 743–753, 1993Google Scholar
  11. 11.
    R. Hartley and P. Sturm: “Triangulation” Computer Vision and Image Understanding, 68(2):146–157, 1997CrossRefGoogle Scholar
  12. 12.
    M. Fischler and R. Bolles: “RANdom SAmpling Consensus: a paradigm for model fitting with application to image analysis and automated cartography” Commun. Assoc. Comp. Mach., 24:95–381, 1981MathSciNetGoogle Scholar
  13. 13.
    J. D. Foley, A. Van Dam, S. K. Feiner and J. F. Hughes: “Computer Graphics: principles and practice.” Addison-Wesley, Reading, Massachusetts, 1990Google Scholar
  14. 14.
    A. Fournier: “Illumination Problems in Computer Augmented Reality.” In JournéeINRIA, Analyse/Synthèse d’Images (JASI’94), pp. 1–21, January 1994Google Scholar
  15. 15.
    P. Jancéne, F. Neyret, X. Provot, J. Tarel, J. Vézien, C. Meilhac and A. Verroust: “RES: Computing the Interactions between Real and Virtual Objects in Video Sequences.”Google Scholar
  16. 16.
    K. N. Kutulakos and J. Vallino: “Affine Object Representations for Calibration-Free Augmented Reality.“ IEEE Virtual Reality Annual International Symposium (VRAIS), pp. 25–36, 1996Google Scholar
  17. 17.
    M. Pollefeys: “Self-calibration and metric 3D reconstruction from uncalibrated image sequences” Ph.D. thesis, ESAT-PSI, K.U.Leuven, 1999Google Scholar
  18. 18.
    M. Pollefeys, R. Koch and L. Van Gool: “Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters” International Journal of Computer Vision, Vol.32, Nr.1, pp. 7–25Google Scholar
  19. 19.
    I. Sato, Y. Sato and K. Ikeuchi: “Acquiring a Radiance Distribution to Superimpose Virtual Objects onto a Real Scene.” IEEE Transactions on Visualization and Computer Graphics, Vol.5, No.1, January-March 1999Google Scholar
  20. 20.
    I. Sato, Y. Sato and K. Ikeuchi: “Illumination Distribution from Brightness in Shadows: Adaptive Estimation of Illumination Distribution with Unknown Reflectance Properties in Shadow Regions.” Proceedings of IEEE International Conference on Computer Vision (ICCV’99), pp. 875–882, September 1999Google Scholar
  21. 21.
    W. Schroeder, K. Martin and B. Lorensen: “The Visualization Toolkit 2nd edition. ”Prentice Hall, New Jersey, 1998Google Scholar
  22. 22.
    C. Schütz and H. Hügli: Augmented Reality using Range Images. ” SPIE Photonics West, The Engineering Reality of Virtual Reality 1997, San Jose, 1997Google Scholar
  23. 23.
    Y. Seo, M. H. Ahn and K. S. Hong: “Video Augmentation by Image-based Rendering under the Perspective Camera Model.“Google Scholar
  24. 24.
    B. Triggs: “The Absolute Quadric“ Proc. 1997 Conference on Computer Vision and Pattern Recognition, IEEE Computer Soc. Press, pp. 609–614, 1997Google Scholar
  25. 25.
    M. Tuceryan, D. S. Greer, R. T. Whitaker, D. Breen, C. Crampton, E. Rose and K. H. Ahlers: “Calibration Requirements and Procedures for Augmented Reality.“ IEEE Transactions on Visualization and Computer Graphics, pp. 255–273, Sept 1995Google Scholar
  26. 26.
    T. Tuytelaars and L. Van Gool: “Content-based Image Retrieval based on Local Affinely Invariant Regions.“ Third International Conference on Visual Information Systems, Visual99, pp. 493–500, june 1999Google Scholar
  27. 27.
    M. Uenohara and T. Kanade: Vision-Based Object Registration for Real-Time Image Overlay.” Proceedings CVRMED’95, pp. 14–22, 1995Google Scholar
  28. 28.
    M. Wloka and B. Anderson: “Resolving Occlusion in Augmented Reality.“ ACM Symposium on Interactive 3D Graphics Proceedings, pp. 5–12, April 1995Google Scholar


  1. 1.
    K. Kutulakos and J. Vallino. Calibration-free augmented reality. IEEE Transactionson Computer Graphics, 4(1):1–20, 1998.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Kurt Cornelis
    • 1
  • Marc Pollefeys
    • 1
  • Maarten Vergauwen
    • 1
  • Luc Van Gool
    • 1
  1. 1.K. U. LeuvenESAT-PSILeuvenBelgium

Personalised recommendations