VideoPlus: A Method for Capturing the Structure and Appearance of Immersive Environments
This paper describes an approach to capturing the appearance and structure of immersive environments based on the video imagery obtained with an omnidirectional camera system. The scheme proceeds by recovering the 3D positions of a set of point and line features in the world from image correspondences in a small set of key frames in the image sequence. Once the locations of these features have been recovered the position of the camera during every frame in the sequence can be determined by using these recovered features as fiducials and estimating camera pose based on the location of corresponding image features in each frame. The end result of the procedure is an omnidirectional video sequence where every frame is augmented with its pose with respect to an absolute reference frame and a 3D model of the environment composed of point and line features in the scene.
By augmenting the video clip with pose information we provide the viewer with the ability to navigate the image sequence in new and interesting ways. More specifically the user can use the pose information to travel through the video sequence with a trajectory different from the one taken by the original camera operator. This freedom presents the end user with an opportunity to immerse themselves within a remote environment.
KeywordsMobile Robot Point Feature Video Sequence Panoramic Image Camera Frame
Unable to display preview. Download preview PDF.
- 1.Terrance E. Boult. Remote reality via omni-directional imaging. In Scott Grisson, Janet McAndless, Omar Ahmad, Christopher Stapleton, Adele Newton, Celia Pearce, Ryan Ulyate, and Rick Parent, editors, Conference abstracts and applications: SIGGRAPH 98, July 14-21, 1998, Orlando, FL, Computer Graphics, pages 253–253, New York, NY 10036, USA, 1998. ACM Press.Google Scholar
- 2.S. E. Chen. Quicktime vr-an image-based approach to virtual environment navigation. In SIGGRAPH, pages 29–38, August 1995.Google Scholar
- 3.Satyan Coorg and Seth Teller. Automatic extraction of textured vertical facades from pose imagery. Technical report, MIT Computer Graphics Group, January 1998.Google Scholar
- 4.Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Proceedings of SIGGRAPH 96. In Computer Graphics Proceedings, Annual Conference Series, pages 11–21, New Orleans, LA, August 4-9 1996. ACM SIGGRAPH.Google Scholar
- 5.C. Geyer and K. Daniilidis. Catadioptric camera calibration. In International Conference on Computer Vision, pages 398–404, 1999.Google Scholar
- 6.Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael Cohen. The lumigraph. In Proceedings of SIGGRAPH 96. In Computer Graphics Proceedings, Annual Conference Series, pages 31–43, New Orleans, LA, August 4-9 1996. ACM SIGGRAPH.Google Scholar
- 7.Hiroshi Ishiguro, Takeshi Maeda, Takahiro Miyashita, and Saburo Tsuji. A strategy for acquiring an environmental model with panoramic sensing by a mobile robot. In IEEE Int. Conf. on Robotics and Automation, pages 724–729, 1994.Google Scholar
- 8.Hiroshi Ishiguro, Kenji Ueda, and Saburo Tsuji. Omnidirectional visual information for navigating a mobile robot. In IEEE Int. Conf. on Robotics and Automation, pages 799–804, 1993.Google Scholar
- 10.J.E. Dennis Jr. and Robert B. Schnabel. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, 1983.Google Scholar
- 11.Marc Levoy and Pat Hanrahan. Light field rendering. In Proceedings of SIGGRAPH 96. In Computer Graphics Proceedings, Annual Conference Series, pages 31–43, New Orleans, LA, August 4-9 1996. ACM SIGGRAPH.Google Scholar
- 12.Maxime Lhuillier and Long Quan. Image interpolation by joint view triangulation. In Proc. IEEE Conf. on Comp. Vision and Patt. Recog., volume 2, pages 139–145, June 1999.Google Scholar
- 13.A. Lippman. Movie maps: An application of the optical video-disc to computer graphics. In SIGGRAPH, pages 32–42, July 1980.Google Scholar
- 14.B.D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In Proc. 7th International Joint Conference on Artificial Intelligence, 1981.Google Scholar
- 15.Shree Nayar. Catadioptric omnidirectional camera. In Proc. IEEE Conf. on Comp. Vision and Patt. Recog., 1997.Google Scholar
- 16.Heung-Yeung Shum and Li-Wei He. Rendering with concentric mosaics. In SIGGRAPH, pages 299–306, August 1999.Google Scholar
- 17.Tomas Svoboda, Tomas Pajdla, and Vaclav Hlavac. Epipolar geometry for panoramic cameras. In European Conference on Computer Vision, pages 218–232. Springer, 1998.Google Scholar
- 18.R. Szeliski and H. Y. Shum. Creating full ciew panoramic image mosaics and texture-mapped models. In SIGGRAPH, pages 251–258, August 1997.Google Scholar
- 19.Camillo J. Taylor and David J. Kriegman. Minimization on the lie group so(3) and related manifolds. Technical Report 9405, Center for Systems Science, Dept. of Electrical Engineering, Yale University, New Haven, CT, April 1994.Google Scholar
- 20.Camillo J. Taylor and David J. Kriegman. Structure and motion from line segments in multiple images. IEEE Trans. Pattern Anal. Machine Intell., 17-11, November 1995.Google Scholar