Dynamic Projection Mapping on Multiple Non-rigid Moving Objects for Stage Performance Applications
In this paper we have proposed and developed a projection mapping system that can project 3D images on multiple moving objects that change their positions and shapes. Although projection mapping has become a new type of image projection method, so far most of the projection mappings have focused on the projection of 3D images on objects such as buildings that stay static without changing their shapes. If projection mapping on non-rigid moving objects such as human performers is realized, application area of projection mapping will become far wider.
To demonstrate this, a simple test of integrating projection mapping and Noh performance was carried out. Although Noh performance has been considered difficult to understand for beginners and foreigners because of its minimalism, by projecting 2D images that suggest inner emotion of a main character and also ongoing story, the evaluation result show that this new type of Noh performance could make it easier to be understood. This means that the integration of projection mapping and various stage performances could open-up new possibilities for various kinds of performing arts to add their values.
To realize this, we have proposed a projection mapping system consisting of multiple depth sensors and multiple projectors. In the first step, using multiple depth sensors the position and shape of multiple objects are detected in real time and the obtained 3D models of the multiple objects are sent to the projection phase. In the projection phase, the images to be projected on the objects are rendered on the obtained 3D model of the object. Then using multiple projectors real-time 3D projection mapping is carried out on the multiple moving objects. The system is now under development and demonstration of actual Noh performance using this technology will be achieved.
KeywordsProjection mapping Depth sensor Moving object Performing arts Noh performance Japanese culture Inner emotion Media art
- 1.Murayama, S., Torii, I., Ishii, N.: Development of projection mapping with utility of digital signage. In: IIAI 3rd International Conference on Advanced Applied Informatics, pp. 895–900 (2014)Google Scholar
- 5.Maniello, D.: Augmented Reality in Public Spaces Basic Techniques for Video Mapping, vol. 1. Paperback (2015)Google Scholar
- 6.Tosa, N., Nakatsu, R., Yunian, P.: Creation of media art utilizing fluid dynamics. In: 2017 International Conference on Culture and Computing, pp. 130–135 (2017)Google Scholar
- 7.Yunian, P, Liang, Z., Nakatsu, R., Tosa, N.: A study of variable control of Sound Vibration Form (SVF) for media art creation. In: 2017 International Conference on Culture and Computing, pp. 137–142 (2017)Google Scholar
- 8.Yasumura, T.: Rimpa: Decorative Japanese Painting. PIE Books (2011)Google Scholar
- 9.Tosa, N., Nakatsu, R., Yunian, P.: Projection mapping celebrating RIMPA 400th Anniversary. In: 2015 International Conference on Culture and Computing (2015)Google Scholar
- 10.Sadler, A.L.: Japanese Plays: Classic Noh, Kyogen and Kabuki Works. Tuttle Publishing, Clarendon (2011)Google Scholar
- 11.Raskar, R., Welch, G., Low, K.L., Bandyopadhyay, D.: Shader lamps: animating real objects with image-based illumination. In: Gortler, S.J., Myszkowski, K. (eds.) Rendering Techniques 2001. Eurographics. Springer, Vienna (2001)Google Scholar
- 14.Gay-Bellile, V., Bartoli, A., Sayd, P.: Deformable surface augmentation in spite of self-occlusions. In: Proceedings of 6th IEEE ACM International Symposium on Mixed Augmented Reality, pp. 235–238 (2007)Google Scholar
- 15.Uchiyama, H., Marchand, E.: Object detection and pose tracking for augmented reality: recent approaches. In: Proceedings of 18th Korea-Japan Joint Workshop Frontiers Computer Vision, pp. 1–8 (2012)Google Scholar
- 16.Park, H., Park, J.-I.: Invisible marker based augmented reality system. In: Proceedings of SPIE Visual Communication Image Processing, Art. No. 59601 (2006)Google Scholar
- 17.Guskov, I.: Efficient tracking of regular patterns on non-ridig geometry. In: Proceedings of 16th International Conference on Pattern Recognition, vol. 2, pp. 1057–1060 (2002)Google Scholar
- 19.Jones, B.R., Benko, H., Ofek, E., Wilson, A.D.: Illumiroom: peripheral projected illusions for interactive experiences. In: ACM SIGGRAPH 2013 Emerging Technologies, SIGGRAPH 2013, p. 7:1. ACM, New York (2013)Google Scholar