Abstract
In this paper, we present a novel image-based rendering technique, which we call manifold hopping. Our technique provides users with perceptually continuous navigation by using only a small number of strategically sampled manifold mosaics or multiperspective panoramas. Manifold hopping has two modes of navigation: moving continuously along any manifold, and discretely between manifolds. An important feature of manifold hopping is that significant data reduction can be achieved without sacrificing output visual fidelity, by carefully adjusting the hopping intervals. A novel view along the manifold is rendered by locally warping a single manifold mosaic using a constant depth assumption, without the need for accurate depth or feature correspondence. The rendering errors caused by manifold hopping can be analyzed in the signed Hough ray space. Experiments with real data demonstrate that we can navigate smoothly in a virtual environment with as little as 88k data compressed from 11 concentric mosaics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adelson, E.H. and Bergen, J. 1991. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing. MIT Press: Cambridge, MA, pp. 3–20.
Avidan, S. and Shashua, A. 1997. Novel view synthesis in tensor space. In Proc. Computer Vision and Pattern Recognition, pp. 1034–1040.
Chai, J.-X., Kang, S.B., and Shum, H.-Y. 2000a. Rendering with non-uniform approximate concentric mosaics. In Proc. ECCV2000 Workshop SMILE2, Dublin, Ireland.
Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000b. Plenotpic sampling. In Proc. SIGGRAPH 2000.
Chang, C., Bishop, G., and Lastra, A. 1999. Ldi tree: A hierarchical representation for image-based rendering. SIGGRAPH'99, pp. 291–298.
Chen, S.E. 1995. QuickTime VR—An image-based approach to virtual environment navigation. Computer Graphics (SIG-GRAPH'95), pp. 29–38.
Chen, S. and Williams, L. 1993. View interpolation for image synthesis. Computer Graphics (SIGGRAPH'93), pp. 279–288.
Gortler, S.J., Grzeszczuk, R., Szeliski, R., and Cohen, M.F. 1996. The lumigraph. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'96 (New Orleans), pp. 43–54. ACM SIGGRAPH.
Gupta, R. and Hartley, R. 1997. Linear pushbroom cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(9):963–975.
Kang, S.B. 1999. A survey of image-based rendering techniques. In Video Metrics, SPIE, vol. 3641, pp. 2–16.
Levoy, M. and Hanrahan, P. 1996. Light field rendering. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'96 (New Orleans), pp. 31–42. ACM SIGGRAPH.
Lippman, A. 1980. Movie maps: An application of the optical videodisc to computer graphics. Computer Graphics (SIG-GRAPH'80), 14(3):32–43.
Mark, W., McMillan, L., and Bishop, G. 1997. Post-rendering 3d warping. In Proc. Symposium on I3D Graphics, pp. 7–16.
McMillan, L. 1999. An image-based approach to three-dimensional computer graphics. Technical Report, Ph.D. Dissertation, UNC Computer Science TR97-013.
McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPH'95), pp. 39–46.
Oliveira, M. and Bishop, G. 1999. Relief textures. Technical Report, UNC Computer Science TR99-015.
Peleg, S. and Herman, J. 1997. Panoramic mosaics by manifold projection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), San Juan, Puerto Rico, pp. 338–343.
Rademacher, P. and 1998. Multiple-center-of-projection images. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'98 (Orlando), pp. 199–206. ACM SIGGRAPH.
Ramasubramanian, M., Pattanaik, S., and Greenberg, D. 1999. Aperceptually based physical error metric for realistic image synthesis. Computer Graphics (SIGGRAPH'99), pp. 73–82.
Seitz, S.M. and Dyer, C.M. 1996. View morphing. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIG-GRAPH'96 (New Orleans), pp. 21–30. ACM SIGGRAPH.
Shade, J., Gortler, S., He, L.-W., and Szeliski, R. 1998. Layered depth images. In Computer Graphics (SIGGRAPH'98) Proceedings, Orlando, pp. 231–242. ACM SIGGRAPH.
Shum, H.-Y. and He, L.-W. 1999. Rendering with concentric mosaics. In Proc. SIGGRAPH 99, pp. 299–306.
Shum, H.-Y. and Szeliski, R. 1999. Stereo reconstruction from multiperspective panoramas. In Proc. Int. Conf. Computer Vision, pp. 14–21.
Wood, D.N. et al. 1997. Multiperspective panoramas for cel animation. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'97 (Los Angeles), pp. 243–250. ACM SIGGRAPH.
Zheng, J.Y. and Tsuji, S. 1990. Panoramic representation of scenes for route understanding. In Proc. of the 10th Int. Conf. Pattern Recognition, pp. 161–167.
Zorin, D. and Barr, A. 1995. Correction of geometric perceptual distortions in pictures. Computer Graphics (SIGGRAPH'95), pp. 257–264.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Shum, HY., Wang, L., Chai, JX. et al. Rendering by Manifold Hopping. International Journal of Computer Vision 50, 185–201 (2002). https://doi.org/10.1023/A:1020398016678
Issue Date:
DOI: https://doi.org/10.1023/A:1020398016678