Advertisement

Generic Content Creation for 3D Displays

Chapter

Abstract

Future 3D productions in the fields of digital signage, commercials, and 3D Television will cope with the problem that they have to address a wide range of different 3D displays, ranging from glasses-based standard stereo displays to auto-stereoscopic multi-view displays or even light-field displays. The challenge will be to serve all these display types with sufficient quality and appealing content. Against this background this chapter discusses flexible solutions for 3D capture, generic 3D representation formats using depth maps, robust methods for reliable depth estimation, required preprocessing of captured multi-view footage, postprocessing of estimated depth maps, and, finally, depth-image-based rendering (DIBR) for creating missing virtual views at the display side.

Keywords

3D display 3D production 3D representation 3D videoconferencing Auto-stereoscopic multi-view display Content creation Depth estimation Depth map Depth-image-based rendering (DIBR) Display-agnostic production Extrapolation Stereo display Stereoscopic video Stereo matching 

References

  1. 1.
    Feldmann I, Schreer O, Kauff P, Schäfer R, Fei Z, Belt HJW, Divorra Escoda Ò (2009) Immersive multi-user 3D video communication. In: Proceedings of international broadcast conference (IBC 2009), Amsterdam, NL, Sept 2009Google Scholar
  2. 2.
    Divorra Escoda O, Civit J, Zuo F, Belt H, Feldmann I, Schreer O, Yellin E, Ijsselsteijn W, van Eijk R, Espinola D, Hagendorf P, Waizennegger W, Braspenning R (2010) Towards 3D-aware telepresence: working on technologies behind the scene. In: Proceedings of ACM conference on computer supported cooperative work (CSCW), new frontiers in telepresence, Savannah, Georgia, USA, 06–10 Feb 2010Google Scholar
  3. 3.
    Feldmann I, Atzpadin N, Schreer O, Pujol-Acolado J-C, Landabaso J-L, Divorra Escoda O (2009) multi-view depth estimation based on visual-hull enhanced hybrid recursive matching for 3D video conference systems. In: Proceedings of 16th international conference on image processing (ICIP 2009), Kairo, Egypt, Nov 2009Google Scholar
  4. 4.
    Waizenegger W, Feldmann I, Schreer O (2011) Real-time patch sweeping for high-quality depth estimation in 3D videoconferencing applications. In: SPIE 2011 conference on real-time image and video processing, San Francisco, CA, USA, 23–27 Jan 2011, Invited PaperGoogle Scholar
  5. 5.
    Pastoor S (1991) 3D-Television: a survey of recent research results on subjective requirements. Signal Process Image Commun 4(1):21–32CrossRefGoogle Scholar
  6. 6.
    IJsselsteijn WA, de Ridder H, Vliegen J (2000) Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain. In: Proceedings of SPIE stereoscopic displays and virtual reality systems, San Jose, Apr 2000Google Scholar
  7. 7.
    Mendiburu B (2008) 3D movie making—stereoscopic digital cinema from script to screen. Elsevier, ISBN: 978-0-240-81137-6Google Scholar
  8. 8.
    Yeh Y–Y, Silverstein LD (1990) Limits of fusion and depth judgement in stereoscopic color pictures. Hum Factors 32(1):45–60Google Scholar
  9. 9.
    Holliman N (2004) Mapping perceived depth to regions of interest in stereoscopic images. In: Stereoscopic displays and applications XV, San Jose, California, Jan 2004Google Scholar
  10. 10.
    Jones G, Lee D, Holliman N, Ezra D (2001) Controlling perceived depth in stereoscopic images. In: Proceedings SPIE stereoscopic displays and virtual reality systems VIII, San Jose, CA, USA, Jan 2001Google Scholar
  11. 11.
    Woods A, Docherty T, Koch R (1993) Image distortions in stereoscopic video systems. Proc. SPIE 1915:36–48CrossRefGoogle Scholar
  12. 12.
    Faubert J (2000) Motion parallax, stereoscopy, and the perception of depth: practical and theoretical issues. In: Proceedings of SPIE three-dimensional video and display: devices and systems, Boston, MA, USA, pp 168–191, Nov 2000Google Scholar
  13. 13.
    Zilly F, Kluger J, Kauff P (2011) Production rules of 3D stereo acquisistion. In: Proceedings of the IEEE (PIEEE), special issue on 3D media and displays, vol 99, issue 4, pp 590–606, Apr 2011Google Scholar
  14. 14.
    Johnson RB, Jacobsen GA (2005) Advances in lenticular lens arrays for visual display. In: Current developments in lens design and optical engineering VI, proceedings of SPIE, vol 5874, Paper 5874-06, San Diego, Aug 2005Google Scholar
  15. 15.
    Börner R (1993) Auto-stereoscopic 3D imaging by front and rear projection and on flat panel displays. Displays 14(1):39–46CrossRefGoogle Scholar
  16. 16.
    Omura K, Shiwa S, Kishino F (1995) Development of lenticular stereoscopic display systems: multiple images for multiple viewers. In: Proceedings of SID 95 digest, pp 761–763Google Scholar
  17. 17.
    Hamagishi G et al (1995) New stereoscopic LC displays without special glasses. Proc Asia Disp 95:921–927Google Scholar
  18. 18.
    Dodgson NA, Moore JR, Lang SR, Martin G, Canepa P (2000) A time sequential multi-projector autostereoscopic display. J Soc Inform Display 8(2):169–176CrossRefGoogle Scholar
  19. 19.
    Lowe D (2004) Distinctive image features from scale invariant keypoints. IJCV 60(2):91–110CrossRefGoogle Scholar
  20. 20.
    Bay H, Ess A, Tuytelaars T, Van Gool L (2008) SURF: speeded up robust features. Comput Vis Image Underst (CVIU) 110(3):346–359CrossRefGoogle Scholar
  21. 21.
    Zilly F, Riechert C, Eisert P, Kauff P (2011) Semantic kernels binarized—a feature descriptor for fast and robust matching. In: Conference on visual media production (CVMP), London, UK, Nov 2011Google Scholar
  22. 22.
    Hartley R, Zisserman A (2000) Multiple view geometry in computer vision. Cambrigde University Press, CambrigdeGoogle Scholar
  23. 23.
    Fischler M, Bolles R (1980) Random sample consensus: a paradigm for model fitting applications to image analysis and automated cartography. In: Proceedings of image understanding workshop, April 1980, pp 71–88Google Scholar
  24. 24.
    Fusiello A, Trucco E, Verri A (2000) A compact algorithm for rectification of stereo pairs. Mach Vis Appl 12(1):16–22CrossRefGoogle Scholar
  25. 25.
    Mallon J, Whelan PF (2005) Projective rectification from the fundamental matrix. Image Vis Comput 23(7):643–650CrossRefGoogle Scholar
  26. 26.
    Wu H-H, Yu Y-H (2005) Projective rectification with reduced geometric distortion for stereo vision and stereoscopic video. J Intell Rob Syst 42:71–94CrossRefGoogle Scholar
  27. 27.
    Scharstein D, Szeliski R (2001) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1/2/3):7–42, Apr–June 2002. Microsoft Research Technical Report MSR-TR-2001-81, Nov 2001Google Scholar
  28. 28.
    Brown MZ, Burschka D, Hager GD (2003) Advances in computational stereo. IEEE Trans Pattern Anal Mach Intell (PAMI) 25(8):993–1008Google Scholar
  29. 29.
    Aschwanden P, Guggenbuhl W (1993) Experimental results from a comparative study on correlation-type registration algorithms. In: Forstner W, Ruwiedel St (eds) Robust computer vision. Wickmann, Karlsruhe, pp 268–289Google Scholar
  30. 30.
    Wegner K, Stankiewicz O (2009) Similarity measures for depth estimation. In: 3DTV conference, the true vision capture transmission and display of 3D videoGoogle Scholar
  31. 31.
    Birchfield S, Tomasi C (1996) Depth discontinuities by pixel-to-pixel stereo. In: Technical report STAN-CS-TR-96-1573, Stanford University, StanfordGoogle Scholar
  32. 32.
    Belhumeur PN (1996) A Bayesian approach to binocular stereopsis. IJCV 19(3):237–260CrossRefGoogle Scholar
  33. 33.
    Cox IJ, Hingorani SL, Rao SB, Maggs BM (1996) A maximum likelihood stereo algorithm. CVIU 63(3):542–567Google Scholar
  34. 34.
    Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239Google Scholar
  35. 35.
    Kolmogorov V, Zabih R (2001) Computing visual correspondence with occlusions using graph cuts. Proc Int Conf Comput Vis 2:508–515Google Scholar
  36. 36.
    Kolmogorov V, Zabih R (2005) Graph cut algorithms for binocular stereo with occlusions. In: Mathematical models in computer vision: the handbook. Springer, New YorkGoogle Scholar
  37. 37.
    Kolmogorov V, Zabih R (2004) What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Mach Intell 26(2):147–159CrossRefGoogle Scholar
  38. 38.
    Bleyer M, Gelautz M (2007) Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions. Signal Process Image Commun 22(2):127–143CrossRefGoogle Scholar
  39. 39.
    Sun J, Shum HY, Zheng NN (2002) Stereo matching using belief propagation. ECCVGoogle Scholar
  40. 40.
    Yang Q, Wang L, Yang R, Wang S, Liao M, Nister D (2006) Real-time global stereo matching using hierachical belief propagation. In: Proceedings of British machine computer vision, 2006Google Scholar
  41. 41.
    Felzenswalb PF, Huttenlocher DP (2006) Efficient belief propagation for early vision. Int J Comput Vis 70(1) (October)Google Scholar
  42. 42.
    Brown MZ, Burschka D, Hager GD (2003) Advances in computational stereo. IEEE Trans Pattern Anal Mach Intell 25(8):993–1008CrossRefGoogle Scholar
  43. 43.
    Kopf J, Cohen M, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. In: Proceedings of the SIGGRAPH conference on ACM Transactions on Graphics, vol 26, no 3Google Scholar
  44. 44.
    Riemens AK, Gangwal OP, Barenbrug B, Berretty R-PM (2009) Joint multi-step joint bilateral depth upsampling. In: Proceedings of SPIE visual communications and image processing, vol 7257, article M, Jan 2009Google Scholar
  45. 45.
    Atzpadin N, Kauff P, Schreer O (2004) Stereo analysis by hybrid recursive matching for real-time immersive video stereo analysis by hybrid recursive matching for real-time immersive video conferencing. In: IEEE Transactions on circuits and systems for video technology, special issue on immersive telecommunications, vol 14, No. 3, pp 321–334, Jan 2004Google Scholar
  46. 46.
    Muscade (MUltimedia SCAlable 3D for Europe), European FP7 research project. http://www.muscade.eu/
  47. 47.
    Ziegler M, Falkenhagen L, ter Horst R, Kalivasd D (1998) Evolution of stereoscopic and three-dimensional video. Signal Process Image Commun 14(1–2):173–1946CrossRefGoogle Scholar
  48. 48.
    Redert A, Op de Beeck M, Fehn C, IJsselsteijn W, Pollefeys M, Van Gool L, Ofek E, Sexton I, Surman P (2002) ATTEST—advanced three-dimensional television systems technologies. In: Proceedings of first international symposium on 3D data processing, visualization, and transmission, Padova, Italy, pp 313–319, June 2002Google Scholar
  49. 49.
    Mohr R, Buschmann R, Falkenhagen L, Van Gool L, Koch R (1998) Cumuli, panorama, and vanguard project overview. In: 3D structure from multiple images of large-scale environments, lecture notes in computer science, vol 1506/1998, pp 1–13. doi:10.1007/3-540-49437-5_1 Google Scholar
  50. 50.
    Fehn C (2004) Depth-image based rendering (DIBR), compression and transmission for a new approach on 3D-TV. In: Proceedings of SPIE stereoscopic display and virtual reality systems XI, San Jose, CA, USA, pp 93–104, Jan 2004Google Scholar
  51. 51.
    Köppel M, Ndjiki-Nya P, Doshkov D, Lakshman H, Merkle P, Mueller K, Wiegand T (2010) Temporally consistent handling of disocclusions with texture synthesis for depth-image-based rendering. In: Proceedings of IEEE ICIP, Hong KongGoogle Scholar
  52. 52.
    Ndjiki-Nya P, Köppel M, Doshkov D, Lakshman H, Merkle P, Mueller K, Wiegand T (2010) Depth-image based rendering with advanced texture synthesis. In: Proceedings of IEEE international conference on multimedia & expo, SingaporeGoogle Scholar
  53. 53.
    3D4YOU, European FP7 research project. http://www.3d4you.eu/
  54. 54.
    Zilly F, Müller M, Kauff P (2010) The stereoscopic analyzer—an image-based assistance tool for stereo shooting and 3D production. In: Proceedings of ICIP 2010, special session on image processing for 3D cinema production, Hong Kong, 26–29 Sept 2010Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Image Processing DepartmentFraunhofer Institute for Telecommunications—Heinrich Hertz InstituteBerlinGermany

Personalised recommendations