Multimedia Tools and Applications

, Volume 44, Issue 1, pp 87–110 | Cite as

Toward cinematizing our daily lives

  • Hansung KimEmail author
  • Ryuuki Sakamoto
  • Itaru Kitahara
  • Tomoji Toriyama
  • Kiyoshi Kogure


We introduce a cinematographic video production system to create movie-like attractive footage from our indoor daily life. Since the system is designed for ordinary users in non-studio environments, it is composed of standard hardware components, provides a simple interface, and works in near real-time of 5 ~ 6 frames/sec. The proposed system reconstructs a visual hull from acquired multiple videos and then generates final videos from the model by referring to the camera shots used in film-making. The proposed method utilizes “Reliability” to compensate for errors that may have occurred in non-studio environments and to produce the most natural scene from the reconstructed model. By using a virtual camera control system, even non-experts can easily convert the 3D model to movies that look as if they were created by experienced filmmakers.


3D video system Cinematized reality Multiple camera system Cinematographic camera control 



This research was supported in part by the National Institute of Information and Communications Technology and KAKENHI(20700122).


  1. 1.
    Ahn J, Byun H (2006) Accurate foreground extraction using graph cut with trimap estimation. Proc. PSIVT, pp. 1185–1194Google Scholar
  2. 2.
    Allili MS, Bouguila N, Ziou D (2007) Finite generalized gaussian mixture modeling and applications to image and video foreground segmentation. Proc CRV, pp. 183–190Google Scholar
  3. 3.
    Arijon D (1991) Grammar of the Film Language. Silman-James PressGoogle Scholar
  4. 4.
    Bresenham J (1965) Algorithm for computer control of a digital plotter. IBM Syst J 4(1):25–30CrossRefGoogle Scholar
  5. 5.
    Elgammal A, Harwood D, Davis LS (2000) Non-parametric model for background subtraction. Proc. ECCV 2:751–767Google Scholar
  6. 6.
    Fabiz F, Cheok A, Wei L, Zhiying Z, Ke X, Prince S, Billinghurst M, Kato H (2005) Live three-dimensional content for augmented reality. IEEE Tans Multimedia 7(3):514–523CrossRefGoogle Scholar
  7. 7.
    Furht B (Ed.) (2008) Encyclopedia of Multimedia, 2nd ed. pp.170–171, SpringerGoogle Scholar
  8. 8.
    Gelasca ED, Ebrahimi T, Karaman M, Sikora T (2006) A Framework for evaluating video object segmentation algorithms. Proc CVPR Workshop, pp. 198Google Scholar
  9. 9.
    Grauman K, Shakhnarovich G, Darrell T (2003) A bayesian approach to image-based visual hull reconstruction. Proc. CVPR, pp. 187–194Google Scholar
  10. 10.
    Gross M, Würmlin S, Naef M, Lamboray E, Spagno C, Kunz A, Koller-Meier E, Svoboda T, Van Gool L, Lang S, Strehlke K, Vande Moere A, Staadt O (2003) Blue-C: a spatially immersive display and 3d video portal for telepresence. Proc. SIGGRAPH, pp. 819–827Google Scholar
  11. 11.
    Guan L, Franco JS, Pollefeys M (2007) 3D Occlusion inference from silhouette cues. Proc CVPR, pp. 1–8Google Scholar
  12. 12.
    Gueziec A, Hummel R (1995) Exploiting triangulated surface extraction using tetrahedral decomposition. IEEE TVCG 1(4):328–342Google Scholar
  13. 13.
    Guillemaut J-Y, Hilton A, Starck J, Kilner J, Grau O (2007) A bayesian framework for simultaneous matting and 3D reconstruction. Proc 3DIM, pp. 167–176Google Scholar
  14. 14.
    Han B, Comaniciu D, Davis L (2004) Sequential kernel density approximation through mode propagation: applications to background modeling. Proc ACCVGoogle Scholar
  15. 15.
    He L, Cohen MF, Salesin DH (1996) The virtual cinematographer: a paradigm for automatic real-time camera control and directing. Proc ICCGIT, pp. 217–224Google Scholar
  16. 16.
    Inoue A, Shigeno H, Okada K, Matsushita Y (2004) Introducing grammar of the film language into automatic shooting for face-to-face meetings. Proc SAINT, pp. 277–280Google Scholar
  17. 17.
    Kanade T, Rander PW, Narayanan PJ (1997) Virtualized reality: constructing virtual worlds from real scenes. IEEE Multimedia 4(1):34–47CrossRefGoogle Scholar
  18. 18.
    Katz SD (2004) Cinematic Motion: Film Directing: A Workshop for Staging Scenes. Michael Wiese Film ProductionsGoogle Scholar
  19. 19.
    Kim H, Kitahara I, Sakamoto R, Kogure K (2006) An Immersive free-viewpoint video system using multiple outer/inner cameras. Proc 3DPVTGoogle Scholar
  20. 20.
    Kim H, Sakamoto R, Kitahara I, Toriyama T, Kogure K (2007a) Reliability-based 3D reconstruction in real environment. Proc. ACM Multimedia pp. 257–260Google Scholar
  21. 21.
    Kim H, Sakamoto R, Kitahara I, Toriyama T, Kogure K (2007b) Robust foreground extraction technique using Gaussian family model and multiple thresholds. Proc ACCVGoogle Scholar
  22. 22.
    Kumar P, Sengupta K, Ranganath S (2000) Real time detection and recognition of human profiles using inexpensive desktop cameras. Proc ICPR, pp.1096–1099Google Scholar
  23. 23.
    Lee JY, Nandi AK (1999) Maximum likelihood parameter estimation of the asymmetric generalized gaussian family of distribution. Proc SPW-HOSGoogle Scholar
  24. 24.
    Lee DS, Hull JJ, Erol B (2003) A bayesian framework for gaussian mixture background modeling. Proc ICIP 3:973–976Google Scholar
  25. 25.
    Li M, Magnor M, Seidel HP (2003) Improved hardware-accelerated visual hull rendering. Proc Vision, Modeling, and VisualizationGoogle Scholar
  26. 26.
    Lorensen WE, Cline HE (1987) Marching cubes: a high resolution 3d surface constructing algorithm. Proc SIGGRAPH, pp.163–169.Google Scholar
  27. 27.
    Magnor MA (2005) Video-Based Rendering. A K PetersGoogle Scholar
  28. 28.
    Matsuyama T, Wu X, Takai T, Wada T (2004) Real-time dynamic 3-D object shape reconstruction and high-fidelity texture mapping for 3-D video. IEEE Trans CSVT 14(3):357–369Google Scholar
  29. 29.
    Mittal A, Paragios N (2004) Motion-based background subtraction using adaptive kernel density estimation. Proc CVPR, pp. 302–309Google Scholar
  30. 30.
    Nobuhara S, Matsuyama T (2006) Deformable mesh model for complex multi-object 3D motion estimation from multi-viewpoint video. Proc 3DPVT, pp.264–271Google Scholar
  31. 31.
    Piccardi M (2004) Background subtraction techniques: a review. Proc IEEE SMC 4:3099–3104Google Scholar
  32. 32.
    Porikli F, Tuzel O (2003) Human body tracking by adaptive background models and mean-shift analysis. Proc Pets-ICVSGoogle Scholar
  33. 33.
    Ronfard R, Taubin G (2007) Introducing 3D cinematography. IEEE CG&A 27(3):18–20Google Scholar
  34. 34.
    Sormann M, Zach C, Karner K (2006) Graph cut based multiple view segmentation for 3D reconstruction. Proc 3DPVT, pp. 1085–1092Google Scholar
  35. 35.
    Starck J, Hilton A (2007) Surface capture for performance-based animation. IEEE CG&A 27(3):21–31Google Scholar
  36. 36.
    Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. Proc CVPR, 246–252Google Scholar
  37. 37.
    Takai T, Nobuhara S, Yoshimoto H, Matsuyama T (2006) 3D video technologies: capturing high fidelity full 3D shape, motion, and texture. Proc ISMAR WorkshopGoogle Scholar
  38. 38.
    Tenmoku R, Ichikari R, Shibata F, Kimura A, Tamura H (2007) Design and prototype implementation of Mr pre-visualization workflow. Proc ISMAR Workshop for Mixed Reality Entertainment and ArtGoogle Scholar
  39. 39.
    Triggs B, McLauchlan P, Hartley R, Fitzgibbon A (1999) bundle adjustment—a modern synthesis. Proc International Workshop on Vision Algorithms: Theory and Practice, pp.298–372Google Scholar
  40. 40.
    Tuzel O, Porikli F, Meer P (2005) A bayesian approach to background modeling. Proc IEEE MVIV 3:58–63Google Scholar
  41. 41.
    Ueda M, Arita D, Taniguchi R (2004) Real-time free-viewpoint video generation using multiple cameras and a PC-cluster. Proc PCM, LNCS 3331, pp. 418–425Google Scholar
  42. 42.
    Wren C, Azarbayejani A, Darrell T, Pentland AP (1997) Pfinder: real-time tracking of the human body. IEEE Trans. PAMI 19(7):780–785Google Scholar
  43. 43.
    Yamazaki S, Sagawa R, Kawasaki H, Ikeuchi K, Sakauchi M (2002) Microfacet billboarding. Proc Eurographics Workshop on Rendering, pp. 175–186Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Hansung Kim
    • 1
    Email author
  • Ryuuki Sakamoto
    • 2
  • Itaru Kitahara
    • 3
  • Tomoji Toriyama
    • 4
  • Kiyoshi Kogure
    • 5
  1. 1.Centre for VisionSpeech and Signal Processing in University of SurreyGuildfordUK
  2. 2.Wakayama UniversityWakayamaJapan
  3. 3.Department of Intelligent Interaction TechnologiesUniversity of TsukubaTsukubaJapan
  4. 4.Toyama Prefectural UniversityImizuJapan
  5. 5.Knowledge Science LaboratoriesATRKyotoJapan

Personalised recommendations