Skip to main content
Log in

A Framework for Representation and Processing of Multi-view Video Using the Concept of Layered Depth Image

  • Published:
The Journal of VLSI Signal Processing Systems for Signal, Image, and Video Technology Aims and scope Submit manuscript

Abstract

The multi-view video is a collection of multiple videos, capturing the same scene at different viewpoints. Since it contains more affluent information than a single video, it can be applied to various applications, such as 3DTV, free viewpoint TV, surveillance, sports matches, and so on. However, the data size of the multi-view video linearly increases as the number of cameras, therefore it is necessary to develop an effective framework to represent, process, and transmit those huge amounts of data. In recent, multi-view video coding is getting lots of attention as efficient video coding technologies are being developed. Although most of multi-view video coding algorithms are based on the state-of-the-art H.264/AVC video coding technology, they do not utilize rich 3-D information. In this paper, we propose a new framework using the concept of layered depth image (LDI), one of the efficient image-based rendering techniques, to efficiently represent and process multi-view video data. We describe how to represent natural multi-view video based on the LDI approach and the overall framework to process those converted data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. Shade, S.J. Gortler, L.W. He, and R. Szeliski, “Layered Depth Images,” in Proc. ACM SIGGRAPH, 1998, pp. 231–242.

  2. S.U. Yoon, S.Y. Kim, E.K. Lee, and Y.S. Ho, “A Framework for Multi-view Video Coding Using Layered Depth Images,” Lect. Notes Comput. Sci. (LNCS), vol. 3767, 2005, pp. 431–442.

    Article  Google Scholar 

  3. S.U. Yoon, S.Y. Kim, and Y.S. Ho, “Preprocessing of Depth and Color Information for Layered Depth Image Coding,” Lect. Notes Comput. Sci. (LNCS), vol. 3333, 2004, pp. 622–699.

    Google Scholar 

  4. H. Shum, S.B. Kang, “A Review of Image-based Rendering Techniques,” in IEEE/SPIE Vis. Commun. Image Process (VCIP), June (2000), pp. 2–13.

  5. H. Shum, S.B. Kang, and S. Chan, “Survey of Image-based Representations and Compression Techniques,” IEEE Trans. Circuits Syst Video Technol., vol. 13, no. 11, 2003, pp. 1020–1037. (November)

    Article  Google Scholar 

  6. C.F. Chang, G. Bishop, and A. Lastra, “LDI tree: A Hierarchical Representation for Image-based Rendering,” in Proc. SIGGRAPH, 1999, pp. 291–298.

  7. R. Namboori, H.C. Teh, and Z. Huang, “An Adaptive Sampling Method for Layered Depth Image,” in Proc. Computer Graphics International, 2004, pp. 206–213.

  8. J. Shade, M.F. Cohen, and D.P. Mitchell, “Tiling Layered Depth Images,” Technical Report, Univ. of Washington, 2000.

  9. H. Kim, S. Kim, B. Koo, and B. Choi, “Layered Depth Image Using Pixel Grouping,” in Proc. Visual Systems and Multimedia (VSMM), 2001, pp. 121–127. (October)

  10. J. Duan and J. Li, “Compression of the LDI,” IEEE Trans. Image Process., vol. 12, no. 3, 2003, pp. 365–372.

    Article  Google Scholar 

  11. Y.H. Im, C.Y. Han, and L.S. Kim, “A Method to Generate Soft Shadows Using Layered Depth Image and Warping,” IEEE Trans. Vis. Comput. Graph., vol. 11, no. 3, 2005, pp. 265–272. (May/June)

    Article  Google Scholar 

  12. C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-quality Video View Interpolation Using a Layered Representation,” in Proc. ACM SIGGRAPH, 2004, pp. 600–608.

  13. T. Kanade, P.W. Rander, and P.J. Narayanan, “Virtualized Reality: Constructing Virtual Worlds from Real Scenes,” MultiMedia Mag., vol. 1, no. 1, 1997, pp. 34–47.

    Google Scholar 

  14. W. Matusik, C. Buehler, L. McMillan, and S.J. Gortler, “Image-based Visual Hulls,” in Proc. of ACM SIGGRAPH, 2000, pp. 369–374.

  15. C. Carranza, C. Theobalt, M.A. Magnor, and H.-P. Seidel, “Free-viewpoint Video of Human Actors,” ACM Trans. Graph., vol. 22, no. 3, 2003, pp. 569–577.

    Article  Google Scholar 

  16. J.C. Yang, M. Everett, C. Buehler, and L. McMillan, “A Real-time Distributed Light Field Camera,” in Eurographics Workshop on Graphics, 2002, pp. 77–85.

  17. ISO/IEC JTC1/SC29/WG11 N6720, “Call for Evidence on Multi-view Video Coding,” 2004.

  18. ISO/IEC JTC1/SC29/WG11 N6999, “Report of the Subjective Quality Evaluation for Multi-view Coding CfE,” 2005.

  19. ISO/IEC JTC1/SC29/WG11 N7327, “Call for Proposals on Multi-view Video Coding,” 2005.

  20. ISO/IEC JTC1/SC29/WG11 N7779, “Subjective Test Results for the CfP on Multi-view Video Coding,” 2006.

  21. L. McMillan, “A List-priority Rendering Algorithm for Redisplaying Projected Surfaces,” UNC Technical Report TR95-005, University of North Carolina, 1995.

  22. ISO/IEC JTC1/SC29/WG11 N7567, “Updated Call for Proposals on Multi-view Video Coding,” 2005.

  23. Interactive Visual Media Group at Microsoft Research, http://research.microsoft.com/vision/InteractiveVisualMediaGroup/3DVideoDownload/.

  24. “A Study on Real-time Extraction of Depth and Disparity Map for Multi-viewpoint Images,” in ETRI Research Report, 2002. (November)

  25. L. McMillan, “An Image-based Approach to Three-dimensional Computer Graphics,” Ph.D. Dissertation, University of North Carolina at Chapel Hill, 1997.

  26. S.U. Yoon, E.K. Lee, S.Y. Kim, Y.S. Ho, K. Yun, S. Cho, and N. Hur, “Inter-camera Coding of Multi-view Video Using Layered Depth Image Representation,” Lect. Notes Comput. Sci. (LNCS), vol. 4261, 2006, pp. 432-441.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Seung-Uk Yoon.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yoon, SU., Lee, EK., Kim, SY. et al. A Framework for Representation and Processing of Multi-view Video Using the Concept of Layered Depth Image. J VLSI Sign Process Syst Sign Image Video Technol 46, 87–102 (2007). https://doi.org/10.1007/s11265-006-0023-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-006-0023-8

Keywords

Navigation