Abstract
This paper presents some techniques for synthesizing novel view for a virtual viewpoint from two given views captured at different viewpoints to achieve both high quality and high efficiency. The whole process consists of three passes. The first pass recovers depth map. We formulate it as pixel labelling and propose a bisection approach to solve it. It is accomplished in log2 n (n is the number of depth levels) steps, each of which involves a single graph cut computation. The second pass detects occluded pixels and reasons about their depth. It fits a foreground depth curve and a background depth curve using depth of nearby foreground and background pixels, and then distinguishes foreground and background pixels by minimizing a global energy, which involves only one graph cut computation. The third pass finds for each pixel in the novel view the corresponding pixels in the input views and computes its color. The whole process involves only a small number of graph cut computations, therefore it is efficient. And, visual artifacts in the synthesized view can be removed successfully by correcting depth of the occluded pixels. Experimental results demonstrate that both high quality and high efficiency are achieved by the proposed techniques.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bobick, A., Intille, S., 1999. Large occlusion stereo. Int. J. Computer Vision, 33(3):181–200. [doi:10.1023/A:1008150329890]
Boykov, Y., Veksler, O., Zabih, R., 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. on Pattern Anal. Machine Intell., 23(11):1222–1239. [doi:10.1109/34.969114]
Chai, D.F., 2006. Stereo Matching for Three Dimensional Visual Communication. Ph.D. Thesis, Zhejiang University (in Chinese).
Chen, S.E., Williams, L., 1993. View Interpolation for Image Synthesis. Proc. SIGGRAPH’93, p.279–288. [doi:10.1145/166117.166153]
Criminisi, A., Shotton, J., Blake, A., Torr, P., 2003. Gaze Manipulation for One-to-one Teleconferencing. Proc. Int. Conf. on Computer Vision, p.939–946.
Dhond, U., Aggarwal, J., 1989. Structure from stereo—a review. IEEE Trans. on Systems, Man, and Cybern., 19(6):1489–1510. [doi:10.1109/21.44067]
Geman, S., Geman, D., 1984. Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Trans. on Pattern Anal. Machine Intell., 6(6):721–741.
Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F., 1996. The Lumigraph. Proc. SIGGRAPH’96, p.43–54. [doi:10.1145/237170.237200]
Hartley, R., Zisserman, A., 2000. Multiple View Geometry in Computer Vision. University Press, Cambridge, UK.
Heckbert, P.S., 1986. Survey of texture mapping. IEEE Computer Graphics and Applications, 6(11):56–67.
Kolmogorov, V., Zabih, R., 2001. Visual Correspondence with Occlusions Using Graph Cuts. Proc. Int. Conf. on Computer Vision, p.508–515.
Kolmogorov, V., Zabih, R., 2002. Multi-camera Scene Reconstruction via Graph Cuts. Proc. European Conference on Computer Vision, p.82–96.
Kolmogorov, V., Zabih, R., 2004. What energy functions can be minimized via graph cuts? IEEE Trans. on Pattern Anal. Machine Intell., 26(2):147–159. [doi:10.1109/TPAMI.2004.1262177]
Kutulakos, K., Seitz, S., 2000. A theory of shape by space carving. Int. J. Computer Vision, 38(3):199–218. [doi:10.1023/A:1008191222954]
Levoy, M., Hanrahan, P., 1996. Light Field Rendering. Proc. SIGGRAPH’96, p.31–42. [doi:10.1145/237170.237199]
Mark, W., McMillan, L., Bishop, G., 1997. Post-rendering 3D Warping. Proc. Symposium on Interactive 3D Graphics, p.7–16. [doi:10.1145/253284.253292]
McMillan, L., Bishop, G., 1995. Plenoptic Modeling: An Image-based Rendering System. Proc. SIGGRAPH’95, p.39–46. [doi:10.1145/218380.218398]
Ohta, Y., Kanade, T., 1985. Stereo by intra-and inter-scanline search using dynamic programming. IEEE Trans. on Pattern Anal. Machine Intell., 7(2):139–154.
Sara, R., Bajcsy, R., 1997. On Occluding Contour Artifacts in Stereo Vision. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.852–857. [doi:10.1109/CVPR.1997.609427]
Scharstein, D., Szeliski, R., 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Computer Vision, 47:7–42. [doi:10.1023/A:1014573219977]
Seitz, S., Dyer, C., 1999. Photorealistic scene reconstruction by voxel coloring. Int. J. Computer Vision, 35(2):151–173. [doi:10.1023/A:1008176507526]
Shade, J., Gortler, S., Hey, L., Szeliski, R., 1998. Layered Depth Images. Proc. SIGGRAPH’98, p.231–242. [doi:10.1145/280814.280882]
Shum, H.Y., Kang, S.B., 2000. A Review of Image-based Rendering Techniques. Proc. IEEE/SPIE Visual Communications and Image Processing, p.2–13.
Silva, C., Santos-Victor, J., 2000. Intrinsic Images for Dense Stereo Matching with Occlusions. Proc. European Conference on Computer Vision, p.100–114.
Sun, J., Shum, H., Zheng, N., 2002. Stereo Matching Using Belief Propagation. Proc. European Conference on Computer Vision, p.510–524.
Terzopoulos, D., 1986. Regularization of inverse visual problems involving discontinuities. IEEE Trans. on Pattern Anal. Machine Intell., 8(4):413–424.
Yang, R., Welch, G., Bishop, G., 2002. Real-time Consensus-based Scene Reconstruction Using Commodity Graphics Hardware. Proc. 10th Pacific Conf. on Computer Graphics and Applications, p.225–235. [doi:10.1109/PCCGA.2002.1167864]
Author information
Authors and Affiliations
Additional information
Project (No. 2002CB312101) supported by the National Basic Research Program (973) of China
Rights and permissions
About this article
Cite this article
Chai, Df., Peng, Qs. Stepwise approach for view synthesis. J. Zhejiang Univ. - Sci. A 8, 1218–1226 (2007). https://doi.org/10.1631/jzus.2007.A1218
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.2007.A1218