Personal and Ubiquitous Computing

, Volume 7, Issue 5, pp 287–298 | Cite as

Visual registration for unprepared augmented reality environments

  • Ke Xu
  • Simon J. D. Prince
  • Adrian David Cheok
  • Yan Qiu
  • Krishnamoorthy Ganesh Kumar
Original Article


Despite the increasing sophistication of augmented reality (AR) tracking technology, tracking in unprepared environments still remains an enormous challenge according to a recent survey. Most current systems are based on a calculation of the optical flow between the current and previous frames to adjust the label position. Here we present two alternative algorithms based on geometrical image constraints. The first is based on epipolar geometry and provides a general description of the constraints on image flow between two static scenes. The second is based on the calculation of a homography relationship between the current frame and a stored representation of the scene. A homography can exactly describe the image motion when the scene is planar, or when the camera movement is a pure rotation, and provides a good approximation when these conditions are nearly met. We assess all three styles of algorithms with a number of criteria including robustness, speed and accuracy. We demonstrate two real-time AR systems here, which are based on the estimation of homography. One is an outdoor geographical labelling/overlaying system, and the other is an AR Pacman game application.


Augmented reality Fundamental matrix Homography Optical flow Vision based tracking 


  1. 1.
    Azuma R et al. (2001) Recent advances in augmented reality. IEEE Comp Graph Appl 21:34–37Google Scholar
  2. 2.
    Neumann U, You S, Cho Y, Lee J and Park J (1999) Natural feature tracking for augmented reality. IEEE Trans Multi 1(1):53–64Google Scholar
  3. 3.
    Jiang B, You S and Neumann U (2000) Camera tracking for augmented reality media. In: Proceedings of the IEEE International Conference Multimedia Expo 2000, IEEE Computer Society Press, Los Alamitos, pp 1637–1640Google Scholar
  4. 4.
    Park J, Jiang B and Neumann U (1999) Vision-based pose computation: robust and accurate augmented reality tracking. In: Proceedings of the IEEE International Workshop on Augmented Reality, San Francisco, CA, 19–20 October 1999Google Scholar
  5. 5.
    Neumann U, You S, Cho Y, Lee J and Park J (1999) Augmented reality tracking in natural environments. In: Proceedings of the IEEE International Symposium on Mixed Realities, Tokyo, Japan, January 1999Google Scholar
  6. 6.
    Billinghurst M, Kato H and Poupyrev I (2001) The magic book: an interface that moves seamlessly between reality and virtuality. IEEE Comp Graph Appl 21:6–8Google Scholar
  7. 7.
    Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video based augmented reality conferencing system. In: Proceedings of the International Workshop on Augmented Reality (IWAR 1999) IEEE Computer Society Press, Los Alamitos, CA, pp 85–94Google Scholar
  8. 8.
    Satoh K, Anabuki M, Yamamoto H and Tamura H (2001) A hybrid registration method for outdoor augmented reality. In: Proceedings of the 2nd IEEE and ACM International Symposium of Augmented Reality (ISAR'01), New York, NY, 29–30 October 2001Google Scholar
  9. 9.
    Azuma R, Lee JW, Jiang B, Park J, You S and Neumann U (1999) Tracking in unprepared environments for augmented reality systems. IEEE Comp Graph 23:787–793Google Scholar
  10. 10.
    You S, Neumann U and Azuma R (1999) Hybrid inertial and vision tracking for augmented reality registration. In: Proceedings of IEEE Virtual Reality '99, Houston, TX, 13–17 March 1999Google Scholar
  11. 11.
    Harris CJ, Stephens M (1988) A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, Manchester, UK, 31 August– 2 September 1988Google Scholar
  12. 12.
    Fischler MA, Bolles RC (1987) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. In: Fischler MA, Firschein O (eds) Readings in computer vision: issues, problems, principles and paradigms, Morgan Kaufmann, Los Altos, CAGoogle Scholar
  13. 13.
    Hartley R, Zisserman A (2000) Multiple view geometry in computer vision. Cambridge University Press, Cambridge, UKGoogle Scholar
  14. 14.
    Simon G, Fitzgibbon AW and Zisserman A (2000) Markerless tracking using planar structures in the scene. In: Proceedings of the International Symposium on Augmented Reality (ISAR 2000), Munich, Germany, 5–6 October 2000Google Scholar
  15. 15.
    Handeykey Corporation (2003). Cited September 2003Google Scholar
  16. 16.
    You S, Neumann U and Azuma R (1999) Orientation tracking for outdoor augmented reality registration. IEEE Comp Graph Appl 19:36–42Google Scholar
  17. 17.
    Kumar KG, Cheok AD and Prince S (2002) Micro-accelerometer based hardware interfaces for wearable computer mixed reality applications. In: Proceedings of the 6th International Symposium on Wearable Computers (ISWC 2002), Seattle, WA, October 2002Google Scholar

Copyright information

© Springer-Verlag London Limited 2003

Authors and Affiliations

  • Ke Xu
    • 1
  • Simon J. D. Prince
    • 1
  • Adrian David Cheok
    • 1
  • Yan Qiu
    • 1
  • Krishnamoorthy Ganesh Kumar
    • 1
  1. 1.Department of Electrical and Computer EngineeringNational University of SingaporeSingaporeSingapore

Personalised recommendations