International Journal of Computer Vision

, Volume 50, Issue 2, pp 185–201 | Cite as

Rendering by Manifold Hopping

  • Heung-Yeung Shum
  • Lifeng Wang
  • Jin-Xiang Chai
  • Xin Tong

Abstract

In this paper, we present a novel image-based rendering technique, which we call manifold hopping. Our technique provides users with perceptually continuous navigation by using only a small number of strategically sampled manifold mosaics or multiperspective panoramas. Manifold hopping has two modes of navigation: moving continuously along any manifold, and discretely between manifolds. An important feature of manifold hopping is that significant data reduction can be achieved without sacrificing output visual fidelity, by carefully adjusting the hopping intervals. A novel view along the manifold is rendered by locally warping a single manifold mosaic using a constant depth assumption, without the need for accurate depth or feature correspondence. The rendering errors caused by manifold hopping can be analyzed in the signed Hough ray space. Experiments with real data demonstrate that we can navigate smoothly in a virtual environment with as little as 88k data compressed from 11 concentric mosaics.

manifold mosaic concentric mosaic perceptually smooth navigation warping plenoptic functions image-based rendering video computing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adelson, E.H. and Bergen, J. 1991. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing. MIT Press: Cambridge, MA, pp. 3–20.Google Scholar
  2. Avidan, S. and Shashua, A. 1997. Novel view synthesis in tensor space. In Proc. Computer Vision and Pattern Recognition, pp. 1034–1040.Google Scholar
  3. Chai, J.-X., Kang, S.B., and Shum, H.-Y. 2000a. Rendering with non-uniform approximate concentric mosaics. In Proc. ECCV2000 Workshop SMILE2, Dublin, Ireland.Google Scholar
  4. Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000b. Plenotpic sampling. In Proc. SIGGRAPH 2000.Google Scholar
  5. Chang, C., Bishop, G., and Lastra, A. 1999. Ldi tree: A hierarchical representation for image-based rendering. SIGGRAPH'99, pp. 291–298.Google Scholar
  6. Chen, S.E. 1995. QuickTime VR—An image-based approach to virtual environment navigation. Computer Graphics (SIG-GRAPH'95), pp. 29–38.Google Scholar
  7. Chen, S. and Williams, L. 1993. View interpolation for image synthesis. Computer Graphics (SIGGRAPH'93), pp. 279–288.Google Scholar
  8. Gortler, S.J., Grzeszczuk, R., Szeliski, R., and Cohen, M.F. 1996. The lumigraph. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'96 (New Orleans), pp. 43–54. ACM SIGGRAPH.Google Scholar
  9. Gupta, R. and Hartley, R. 1997. Linear pushbroom cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(9):963–975.Google Scholar
  10. Kang, S.B. 1999. A survey of image-based rendering techniques. In Video Metrics, SPIE, vol. 3641, pp. 2–16.Google Scholar
  11. Levoy, M. and Hanrahan, P. 1996. Light field rendering. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'96 (New Orleans), pp. 31–42. ACM SIGGRAPH.Google Scholar
  12. Lippman, A. 1980. Movie maps: An application of the optical videodisc to computer graphics. Computer Graphics (SIG-GRAPH'80), 14(3):32–43.Google Scholar
  13. Mark, W., McMillan, L., and Bishop, G. 1997. Post-rendering 3d warping. In Proc. Symposium on I3D Graphics, pp. 7–16.Google Scholar
  14. McMillan, L. 1999. An image-based approach to three-dimensional computer graphics. Technical Report, Ph.D. Dissertation, UNC Computer Science TR97-013.Google Scholar
  15. McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPH'95), pp. 39–46.Google Scholar
  16. Oliveira, M. and Bishop, G. 1999. Relief textures. Technical Report, UNC Computer Science TR99-015.Google Scholar
  17. Peleg, S. and Herman, J. 1997. Panoramic mosaics by manifold projection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), San Juan, Puerto Rico, pp. 338–343.Google Scholar
  18. Rademacher, P. and 1998. Multiple-center-of-projection images. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'98 (Orlando), pp. 199–206. ACM SIGGRAPH.Google Scholar
  19. Ramasubramanian, M., Pattanaik, S., and Greenberg, D. 1999. Aperceptually based physical error metric for realistic image synthesis. Computer Graphics (SIGGRAPH'99), pp. 73–82.Google Scholar
  20. Seitz, S.M. and Dyer, C.M. 1996. View morphing. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIG-GRAPH'96 (New Orleans), pp. 21–30. ACM SIGGRAPH.Google Scholar
  21. Shade, J., Gortler, S., He, L.-W., and Szeliski, R. 1998. Layered depth images. In Computer Graphics (SIGGRAPH'98) Proceedings, Orlando, pp. 231–242. ACM SIGGRAPH.Google Scholar
  22. Shum, H.-Y. and He, L.-W. 1999. Rendering with concentric mosaics. In Proc. SIGGRAPH 99, pp. 299–306.Google Scholar
  23. Shum, H.-Y. and Szeliski, R. 1999. Stereo reconstruction from multiperspective panoramas. In Proc. Int. Conf. Computer Vision, pp. 14–21.Google Scholar
  24. Wood, D.N. et al. 1997. Multiperspective panoramas for cel animation. In Computer Graphics Proceedings, Annual Conference Series, Proc. SIGGRAPH'97 (Los Angeles), pp. 243–250. ACM SIGGRAPH.Google Scholar
  25. Zheng, J.Y. and Tsuji, S. 1990. Panoramic representation of scenes for route understanding. In Proc. of the 10th Int. Conf. Pattern Recognition, pp. 161–167.Google Scholar
  26. Zorin, D. and Barr, A. 1995. Correction of geometric perceptual distortions in pictures. Computer Graphics (SIGGRAPH'95), pp. 257–264.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Heung-Yeung Shum
    • 1
  • Lifeng Wang
    • 1
  • Jin-Xiang Chai
    • 1
  • Xin Tong
    • 1
  1. 1.Microsoft Research, AsiaBeijingP.R. China

Personalised recommendations