Abstract
The multi-view video is a collection of multiple videos, capturing the same scene at different viewpoints. Since it contains more affluent information than a single video, it can be applied to various applications, such as 3DTV, free viewpoint TV, surveillance, sports matches, and so on. However, the data size of the multi-view video linearly increases as the number of cameras, therefore it is necessary to develop an effective framework to represent, process, and transmit those huge amounts of data. In recent, multi-view video coding is getting lots of attention as efficient video coding technologies are being developed. Although most of multi-view video coding algorithms are based on the state-of-the-art H.264/AVC video coding technology, they do not utilize rich 3-D information. In this paper, we propose a new framework using the concept of layered depth image (LDI), one of the efficient image-based rendering techniques, to efficiently represent and process multi-view video data. We describe how to represent natural multi-view video based on the LDI approach and the overall framework to process those converted data.
Similar content being viewed by others
References
J. Shade, S.J. Gortler, L.W. He, and R. Szeliski, “Layered Depth Images,” in Proc. ACM SIGGRAPH, 1998, pp. 231–242.
S.U. Yoon, S.Y. Kim, E.K. Lee, and Y.S. Ho, “A Framework for Multi-view Video Coding Using Layered Depth Images,” Lect. Notes Comput. Sci. (LNCS), vol. 3767, 2005, pp. 431–442.
S.U. Yoon, S.Y. Kim, and Y.S. Ho, “Preprocessing of Depth and Color Information for Layered Depth Image Coding,” Lect. Notes Comput. Sci. (LNCS), vol. 3333, 2004, pp. 622–699.
H. Shum, S.B. Kang, “A Review of Image-based Rendering Techniques,” in IEEE/SPIE Vis. Commun. Image Process (VCIP), June (2000), pp. 2–13.
H. Shum, S.B. Kang, and S. Chan, “Survey of Image-based Representations and Compression Techniques,” IEEE Trans. Circuits Syst Video Technol., vol. 13, no. 11, 2003, pp. 1020–1037. (November)
C.F. Chang, G. Bishop, and A. Lastra, “LDI tree: A Hierarchical Representation for Image-based Rendering,” in Proc. SIGGRAPH, 1999, pp. 291–298.
R. Namboori, H.C. Teh, and Z. Huang, “An Adaptive Sampling Method for Layered Depth Image,” in Proc. Computer Graphics International, 2004, pp. 206–213.
J. Shade, M.F. Cohen, and D.P. Mitchell, “Tiling Layered Depth Images,” Technical Report, Univ. of Washington, 2000.
H. Kim, S. Kim, B. Koo, and B. Choi, “Layered Depth Image Using Pixel Grouping,” in Proc. Visual Systems and Multimedia (VSMM), 2001, pp. 121–127. (October)
J. Duan and J. Li, “Compression of the LDI,” IEEE Trans. Image Process., vol. 12, no. 3, 2003, pp. 365–372.
Y.H. Im, C.Y. Han, and L.S. Kim, “A Method to Generate Soft Shadows Using Layered Depth Image and Warping,” IEEE Trans. Vis. Comput. Graph., vol. 11, no. 3, 2005, pp. 265–272. (May/June)
C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-quality Video View Interpolation Using a Layered Representation,” in Proc. ACM SIGGRAPH, 2004, pp. 600–608.
T. Kanade, P.W. Rander, and P.J. Narayanan, “Virtualized Reality: Constructing Virtual Worlds from Real Scenes,” MultiMedia Mag., vol. 1, no. 1, 1997, pp. 34–47.
W. Matusik, C. Buehler, L. McMillan, and S.J. Gortler, “Image-based Visual Hulls,” in Proc. of ACM SIGGRAPH, 2000, pp. 369–374.
C. Carranza, C. Theobalt, M.A. Magnor, and H.-P. Seidel, “Free-viewpoint Video of Human Actors,” ACM Trans. Graph., vol. 22, no. 3, 2003, pp. 569–577.
J.C. Yang, M. Everett, C. Buehler, and L. McMillan, “A Real-time Distributed Light Field Camera,” in Eurographics Workshop on Graphics, 2002, pp. 77–85.
ISO/IEC JTC1/SC29/WG11 N6720, “Call for Evidence on Multi-view Video Coding,” 2004.
ISO/IEC JTC1/SC29/WG11 N6999, “Report of the Subjective Quality Evaluation for Multi-view Coding CfE,” 2005.
ISO/IEC JTC1/SC29/WG11 N7327, “Call for Proposals on Multi-view Video Coding,” 2005.
ISO/IEC JTC1/SC29/WG11 N7779, “Subjective Test Results for the CfP on Multi-view Video Coding,” 2006.
L. McMillan, “A List-priority Rendering Algorithm for Redisplaying Projected Surfaces,” UNC Technical Report TR95-005, University of North Carolina, 1995.
ISO/IEC JTC1/SC29/WG11 N7567, “Updated Call for Proposals on Multi-view Video Coding,” 2005.
Interactive Visual Media Group at Microsoft Research, http://research.microsoft.com/vision/InteractiveVisualMediaGroup/3DVideoDownload/.
“A Study on Real-time Extraction of Depth and Disparity Map for Multi-viewpoint Images,” in ETRI Research Report, 2002. (November)
L. McMillan, “An Image-based Approach to Three-dimensional Computer Graphics,” Ph.D. Dissertation, University of North Carolina at Chapel Hill, 1997.
S.U. Yoon, E.K. Lee, S.Y. Kim, Y.S. Ho, K. Yun, S. Cho, and N. Hur, “Inter-camera Coding of Multi-view Video Using Layered Depth Image Representation,” Lect. Notes Comput. Sci. (LNCS), vol. 4261, 2006, pp. 432-441.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yoon, SU., Lee, EK., Kim, SY. et al. A Framework for Representation and Processing of Multi-view Video Using the Concept of Layered Depth Image. J VLSI Sign Process Syst Sign Image Video Technol 46, 87–102 (2007). https://doi.org/10.1007/s11265-006-0023-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-006-0023-8