Advertisement

3D Research

, 3:4 | Cite as

Recent developments in free-viewpoint interpolation for 3DTV

  • Svitlana ZingerEmail author
  • Luat Do
  • Peter H. N. de With
3DR Express

Abstract

Current development of 3D technologies brings 3DTV within reach for the customers. We discuss in this article the recent advancements in free-viewpoint interpolation for 3D video. This technology is still a research topic and many efforts are dedicated to creation, evaluation and improvement of new algorithms. First, we describe a recent DIBR algorithm that allows free-viewpoint interpolation. We perform detailed analysis of its performance and establish the sources of errors that occur in the resulting 3D videos. Our graph presents each DIBR step’s contribution to the overall error. We show that the challenge consists of finding the correct pixel values in the areas of resulting image where warping did not provide any information. The quality of the resulting image is mostly determined by the warping procedure and cannot be considerably changed by post-processing. To accommodate the generation of stereoscopic videos, we discuss the possibilities of creating a free-viewpoint stereoscopic video.

Keywords

3DTV free-viewpoint interpolation depth image based rendering 

References

  1. 1.
    M. Tanimoto (2006) FTV (free-viewpoint television) for 3D scene reproduction and creation, Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop, IEEE Computer Society, pp 172.Google Scholar
  2. 2.
    Y. Mori, N. Fukushima, T. Fujii, M. Tanimoto (2009) View generation with 3-D warping using depth information for FTV, Signal Processing: Image Communication, 24(1–2):65–72.CrossRefGoogle Scholar
  3. 3.
    S. Zinger, D. Ruijters, P. H. N. de With (2009) iGLANCE project: free-viewpoint 3D video, 17th International Conference on Computer Graphics, Visualization and Computer Vision.Google Scholar
  4. 4.
    D. Ruijters, S. Zinger (2009) iGLANCE: transmission to medical high definition autostereoscopic displays, 3DTV Conference: The True Vision — Capture, Transmission and Display of 3D Video.Google Scholar
  5. 5.
    S. Zinger, L. Do, D. Ruijters, P. H. N. de With (2009) iGLANCE: interactive free-viewpoint for 3D TV, 3D Stereo Media conference.Google Scholar
  6. 6.
    Y. Morvan (2009) Acquisition, compression and rendering of depth and texture for multi-view video, Ph.D. dissertation, Eindhoven University of Technology.Google Scholar
  7. 7.
    S. Zinger, L. Do, P. H. N. de With (2010) Free-viewpoint depth image based rendering, Journal of Visual Communication and Image Representation, 21(5–6):533–541.CrossRefGoogle Scholar
  8. 8.
    C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, R. Szeliski (2004) High-quality video view interpolation using a layered representation, ACM SIGGRAPH 2004, pp. 600–608.Google Scholar
  9. 9.
    C. Fehn (2004) Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3DTV, Stereoscopic Displays and Virtual Reality Systems XI, 5291:93–104.Google Scholar
  10. 10.
    A. Smolic, K. Muller, K. Dix, P. Merkle, P. Kauff, T. Wiegand (2008) Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems, IEEE International Conference on Image Processing (ICIP), pp. 2448–2451.Google Scholar
  11. 11.
    Y. Mori, N. Fukushima, T. Yendo, T. Fujii, M. Tanimoto (2009) View generation with 3D warping using depth information for FTV, Signal Processing: Image Communication, 24(1–2):65–72.CrossRefGoogle Scholar
  12. 12.
    K.-J. Oh, Y. Sehoon, Y.-S. Ho (2009) Hole-filling method using depth based in-painting for view synthesis in free viewpoint television (ftv) and 3D video, Picture Coding Symposium.Google Scholar
  13. 13.
    L. McMillan, R. S. Pizer (1997) An image based approach to three-dimensional computer graphics, University of North Carolina, Chapel Hill, Technical Report TR97-013.Google Scholar
  14. 14.
    L. Do, S. Zinger, Y. Morvan, P. H. N. de With (2009) Quality improving techniques in DIBR for free-viewpoint video, 3DTV Conference: The True Vision — Capture, Transmission and Display of 3D Video.Google Scholar
  15. 15.
    L. Do, S. Zinger, P. H. N. de With (2010) Quality improving techniques for free-viewpoint DIBR, IST / SPIE Electronic Imaging, vol 7524.Google Scholar
  16. 16.
    L. Do, S. Zinger, P. H. N. de With (2010) Objective quality analysis for free-viewpoint DIBR, IEEE International Conference on Image Processing (ICIP).Google Scholar
  17. 17.
    Zh. Wang, L. Lu, A. C. Bovik (2004) Video quality assessment based on structural distortion measurement, Signal Processing: Image Communication, 19(2):121–132.CrossRefGoogle Scholar
  18. 18.
    J. Starck, J. Kilner, J. Y. Guillemaut, A. Hilton (2009) Objective quality assessment in free-viewpoint video production, Signal Processing: Image Communication, 24(1–2):3–16.Google Scholar
  19. 19.
    H. Shao, X. Cao, G. Er (2009) Objective quality assessment of depth image based rendering in 3DTV, 3DTV Conference: The True Vision — Capture, Transmission and Display of 3D Video.Google Scholar
  20. 20.
    Y. Morvan, D. Farin, P. H. N. de With (2007) Joint depth/texture bit-allocation for multi-view video compression, Picture Coding Symposium.Google Scholar
  21. 21.
    x264 a free h264/avc encoder. http://developers.videolan.org/x264.html.
  22. 22.
    L. Zhang, W. J. Tam (2005) Stereoscopic image generation based on depth images for 3D TV, IEEE Transactions on Broadcasting, 51(2):191–199.CrossRefGoogle Scholar
  23. 23.
    S. Knorr, M. Kunter, T. Sikora (2008) Stereoscopic 3D from 2D video with super-resolution capability, Signal Processing: Image Communication, 23(9):665–676.CrossRefGoogle Scholar
  24. 24.
    K. Yamaguchi, H. Takemura, K. Yamazawa, N. Yokoya (2000) Real-time generation and presentation of viewdependent binocular stereo images using a sequence of omnidirectional images, International Conference on Pattern Recognition, vol 4, pp 589–593.Google Scholar
  25. 25.
    M. Hori, M. Kanbara, N. Yokoya (2007) Novel stereoscopic view generation by image-based rendering coordinated with depth information, Scandinavian Conference on Image analysis, pp 193–202.Google Scholar
  26. 26.
    A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang (2007) Multiview imaging and 3DTV, IEEE Signal Processing Magazine, 24(6):10–21.CrossRefGoogle Scholar
  27. 27.
    L. Do, S. Zinger, P. H. N. de With (2010) Conversion of free-viewpoint 3D multi-view video for stereoscopic displays, IEEE International Workshop on Hot Topics in 3D, in conjunction with International Conference on Multimedia and Expo (ICME).Google Scholar
  28. 28.
    S. Zinger, L. Do, Y. Gao, P. H. N. de With (2010) Conversion of free-viewpoint 3DTV signals for stereo displays, 31st Symposium on Information Theory in the Benelux, pp 89–96.Google Scholar
  29. 29.
    C. Leung, B. C. Lovell (2003) 3D reconstruction through segmentation of multi-view image sequences, Workshop on Digital Image Computing, vol 1, pp 87–92.Google Scholar
  30. 30.
    S. Zinger, D. Ruijters, L. Do, P. H. N. de With (2011) View Interpolation for Medical Images on Autostereoscopic Displays, IEEE Transactions on Circuits and Systems for Video Technology.Google Scholar

Copyright information

© 3D Display Research Center and Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Svitlana Zinger
    • 1
    Email author
  • Luat Do
    • 1
  • Peter H. N. de With
    • 1
  1. 1.Eindhoven University of TechnologyEindhovenNetherlands

Personalised recommendations