Computer Vision Technology Applied to MR-Based Pre-visualization in Filmmaking

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6469)


In this talk, we introduce the outline of gThe MR-PreViz Projecth performed in Japan. In the pre-production process of filmmaking, PreViz, pre-visualizing the desired scene by CGI, is used as a new technique. In its advanced approach, we propose MR-PreViz to utilize mixed reality technology as in current PreViz. MR-PreViz makes it possible to merge the real background and the computer-generated humans and creatures in an open set or at an outdoor location. Computer vision technologies are required for many aspects of MR-PreViz. For capturing an actor’s action, we applied 3D Video, which is a technology that allows one to reconstruct an image seen from any viewpoint in real time from video images taken by multiple cameras. As the other application of CV, we developed a vision based camera tracking method. The method collects environmental information required for tracking efficiently using a structure-from-motion technique before the shooting. Additionally, we developed a relighting technique for lighting design of MR-PreViz movie.


Feature Point Virtual Object Mixed Reality Omnidirectional Image Light Stage 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tamura, H., Yamamoto, H., Katayama, A.: Mixed reality: Future dreams seen at the border between real and virtual worlds. IEEE Computer Graphics & Applications 21, 64–70 (2001)CrossRefGoogle Scholar
  2. 2.
    Tenmoku, R., Ichikari, R., Shibata, F., Kimura, A., Tamura, H.: Design and prototype implementation of MR pre-visualization workflow. In: DVD-ROM Proc. Int. Workshop on Mixed Reality Technology for Filmmaking, pp. 26–30 (2006)Google Scholar
  3. 3.
    Ichikari, R., Tenmoku, R., Shibata, F., Ohshima, T., Tamura, H.: Mixed reality pre-visualization for filmmaking: On-set camera-work authoring and action rehearsal. Int. J. Virtual Reality 7, 25–32 (2008)Google Scholar
  4. 4.
    Starck, J., Maki, A., Nobuhara, S., Hilton, A., Matsuyama, T.: The multiple-camera 3-D production studio. IEEE Trans. on Circuits and Systems for Video Technology 19, 856–869 (2009)CrossRefGoogle Scholar
  5. 5.
    Yamaguchi, T., Nobuhara, S., Matsuyama, T.: Cell-based object tracking method for 3D shape reconstruction using multi-viewpoint active cameras. In: IEEE Int. Workshop on Visual Surveillance, VS 2009 (2009)Google Scholar
  6. 6.
    Mukasa, T., Miyamoto, A., Nobuhara, S., Maki, A., Matsuyama, T.: Complex human motion estimation using visibility. In: IEEE Int. Conf. Series on Automatic Face and Gesture Recognition (2008)Google Scholar
  7. 7.
    Klein, G., Murray, D.: Parallel tracking and ma pping for small AR workspaces. In: Proc. ISMAR, pp. 225–234 (2007)Google Scholar
  8. 8.
    Taketomi, T., Sato, T., Yokoya, N.: Real-time geometric registration using feature landmark database for augmented reality applications. In: Proc. SPIE (2009) 723804–723804–9Google Scholar
  9. 9.
    Sato, T., Ikeda, S., Yokoya, N.: Extrinsic camera parameter recovery from multiple image sequences captured by an omni-directional multi-camera system. In: Proc. European Conf. on Computer Vision, vol. 2, pp. 326–340 (2004)Google Scholar
  10. 10.
    Toishita, W., Momoda, Y., Tenmoku, R., Shibata, F., Tamura, H., Taketomi, T., Sato, T., Yokoya, N.: A novel approach to on-site camera calibration and tracking for MR pre-visualization procedure. In: Proc. Human-Computer Interaction International (HCI 2009), pp. 492–502 (2009)Google Scholar
  11. 11.
    Debevec, P.: Virtual cinematography: Relighting through computation. IEEE Computer 39, 57–65 (2006)CrossRefGoogle Scholar
  12. 12.
    Ichikari, R., Hatano, R., Ohshima, T., Shibata, F., Tamura, H.: Designing cinematic lighting by relighting in MR-based pre-visualization. In: ACM SIGGRAPH ASIA 2009 Posters (2009)Google Scholar
  13. 13.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.College of Information Scienece and EngineeringRitsumeikan UniversityJapan
  2. 2.Graduate School of InformaticsKyoto UniversityJapan
  3. 3.Graduate School of Information ScienceNara Institute of Science and TechnologyJapan

Personalised recommendations