Abstract
In this chapter, we propose a novel GPU-based algorithm capable of generating free viewpoints from a network of fixed HD video cameras. This free viewpoint TV system consists of two main subsystems: a real-time depth estimation subsystem, which extracts a disparity map from a network of cameras, and a synthetic viewpoint generation subsystem that uses the disparity map to interpolate new views between the cameras. In this system, we use a space-sweep algorithm to estimate depth information, which is amiable to parallel implementation. The viewpoint generation subsystem generates new synthetic images from 3D vertices and renders them from an arbitrary viewpoint specified by the user. Both steps are computationally extensive, but the computations can be easily divided from each other and thus can be efficiently implemented in parallel using CUDA. The framework is tested using publicly available image sequences published by Microsoft. Experimental results are presented.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Tanimoto, M.: Overview of free viewpoint television. Signal Processing: Image Communication 21(6), 454–461 (2006), Special issue on multi-view image processing and its application in image-based rendering
Matusik, W., Buehler, C., Raskar, R., Gortler, S.J., McMillan, L.: Image-based visual hulls. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 369–374. SIGGRAPH ’00. ACM Press/Addison-Wesley Publishing Co, New York, NY. (2000)
Franco, J.S., Boyer, E.: Exact polyhedral visual hulls. In: British Machine Vision Conference (BMVC’03), vol 1, pp. 329–338. Norwich, Royaume-Uni (2003)
Starck, J., Hilton, A.: Surface capture for performance-based animation. IEEE Comput. Graph. Appl. 27(3), 21–31 (2007)
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1362–1376 (2010)
Starck, J., Hilton, A.: Virtual view synthesis of people from multiple view video sequences. Graph. Models 67(6), 600–620 (2005)
Orman N., Kim H., Sakamoto R., Toriyama T., Kogure K., Lindeman R.: GPU-based optimization of a free-viewpoint video system. In: Proceedings of the 2008 symposium on Interactive 3D graphics and games (2008)
Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 43–54. SIGGRAPH ’96. ACM, New York, NY (1996)
Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 31–42. SIGGRAPH ’96. ACM, New York, NY (1996)
Fujii, T., Kimoto, T., Tanimoto, M.: Ray-space coding for 3d visual communication. In: Picture Coding Symposium (PCS), vol. 2, pp. 447–451 (1996)
Mori, Y., Fukushima, N., Yendo, T., Fujii, T., Tanimoto, M.: View generation with 3d warping using depth information for ftv. Image Commun. 24(1–2), 65–72 (2009)
Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23(3), 600–608 (2004)
Yang, R., Welch, G., Bishop, G.: Real-time consensus-based scene reconstruction using commodity graphics hardware. In: Pacific Conference on Computer Graphics and Applications. 225–235 (2002)
Suzuki, K., Fukushima, N., Yendo, T., Tehrani, M., Fujii, T., Tanimoto, M.: Parallel processing method for realtime ftv. In: Picture Coding Symposium (PCS), pp. 330–333 (2010)
Nozick V., Saito H.: On-line free-viewpoint video: From single to multiple view rendering. Int. J. Autom. Comput. 5, 257–267 (2008)
Starck J., Kilner J., Hilton A.: A free-viewpoint renderer. J. Graph. GPU Game Tools 14 57–72 (2009)
Collins, R.: A space-sweep approach to true multi-image matching. In: Computer Vision and Pattern Recognition. IEEE Computer Society Conference on proceedings CVPR ’96, pp. 358–363 (1996)
NVIDIA Corporation: NVIDIA CUDA C programming guide (2012) Version 5.0.
Stam, J.: Stereo imaging with cuda. (2008)
Zhou, X., Boulanger, P.: Radiometric invariant stereo matching based on relative gradients. In: 19th IEEE International Conference on Image Processing (ICIP), pp. 2989–2992 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer Science+Business Media Singapore
About this chapter
Cite this chapter
Shegeda, K., Boulanger, P. (2015). A GPU-Based Real-Time Algorithm for Virtual Viewpoint Rendering from Multi-video. In: Cai, Y., See, S. (eds) GPU Computing and Applications. Springer, Singapore. https://doi.org/10.1007/978-981-287-134-3_11
Download citation
DOI: https://doi.org/10.1007/978-981-287-134-3_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-287-133-6
Online ISBN: 978-981-287-134-3
eBook Packages: EngineeringEngineering (R0)