Real-Time Spherical Mosaicing Using Whole Image Alignment

  • Steven Lovegrove
  • Andrew J. Davison
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6313)


When a purely rotating camera observes a general scene, overlapping views are related by a parallax-free warp which can be estimated by direct image alignment methods that iterate to optimise photo-consistency. However, building globally consistent mosaics from video has usually been tackled as an off-line task, while sequential methods suitable for real-time implementation have often suffered from long-term drift. In this paper we present a high performance real-time video mosaicing algorithm based on parallel image alignment via ESM (Efficient Second-order Minimisation) and global optimisation of a map of keyframes over the whole viewsphere. We present real-time results for drift-free camera rotation tracking and globally consistent spherical mosaicing from a variety of cameras in real scenes, demonstrating high global accuracy and the ability to track very rapid rotation while maintaining solid 30Hz operation. We also show that automatic camera calibration refinement can be straightforwardly built into our framework.


Image Alignment Camera Intrinsic Parameter Spherical Panorama Live Camera Perceptual Aliasing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Szeliski, R.: Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics and Vision 2, 1–104 (2006)CrossRefGoogle Scholar
  2. 2.
    Brown, M., Lowe, D.G.: Recognising panoramas. In: Proceedings of the International Conference on Computer Vision (ICCV) (2003)Google Scholar
  3. 3.
    Steedly, D., Pal, C., Szeliski, R.: Efficiently stitching large panoramas from video. In: Proceedings of the International Conference on Computer Vision (ICCV) (2005)Google Scholar
  4. 4.
    Szeliski, R., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. ACM Transactions on Graphics (SIGGRAPH) (1997)Google Scholar
  5. 5.
    Morimoto, C., Chellappa, R.: Fast 3D stabilization and mosaic construction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (1997)Google Scholar
  6. 6.
    Civera, J., Davison, A.J., Magallón, J.A., Montiel, J.M.M.: Drift-free real-time mosaicing. International Journal of Computer Vision (IJCV) 81, 128–137 (2009)CrossRefGoogle Scholar
  7. 7.
    Klein, G., Murray, D.W.: Parallel tracking and mapping for small AR workspaces. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR) (2007)Google Scholar
  8. 8.
    Lu, F., Milios, E.: Globally consistent range scan alignment for environment mapping. Autonomous Robots 4, 333–349 (1997)CrossRefGoogle Scholar
  9. 9.
    Gutmann, J.S., Konolige, K.: Incremental mapping of large cyclic environments. In: International Symposium on Computational Intelligence in Robotics and Automation (CIRA) (1999)Google Scholar
  10. 10.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (1981)Google Scholar
  11. 11.
    Malis, E.: Improving vision-based control using efficient second-order minimization techniques. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2004)Google Scholar
  12. 12.
    Mei, C., Benhimane, S., Malis, E., Rives, P.: Efficient homography-based tracking and 3-D reconstruction for single-viewpoint sensors. IEEE Transactions on Robotics (T-RO) 24, 1352–1364 (2008)CrossRefGoogle Scholar
  13. 13.
    Silveira, G., Malis, E., Rives, P.: An efficient direct approach to visual SLAM. IEEE Transactions on Robotics (T-RO) 24, 969–979 (2008)CrossRefGoogle Scholar
  14. 14.
    Agapito, L., Hayman, E., Reid, I.: Self-calibration of rotating and zooming cameras. International Journal of Computer Vision (IJCV) 45, 107–127 (2001)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Steven Lovegrove
    • 1
  • Andrew J. Davison
    • 1
  1. 1.Imperial College LondonLondonUK

Personalised recommendations