Abstract
We developed a 3D archive system for Japanese traditional performing arts. The system generates sequences of 3D actor models of the performances from multi-view video by using a graph-cuts algorithm and stores them with CG background models and related information. The system can show a scene from any viewpoint as follows; the 3D actor model is integrated with the background model and the integrated model is projected to a viewpoint that the user indicates with a viewpoint controller.
A challenge of generating the actor models is how to reconstruct thin or slender parts. Japanese traditional costumes for performances include slender parts such as long sleeves, fans and strings that may be manipulated during the performance. The graph-cuts algorithm is a powerful 3D reconstruction tool but it tends to cut off those parts because it uses an energy-minimization process. Hence, the search for a way to reconstruct such parts is important for us to preserve these arts for future generations. We therefore devised an adaptive erosion method that works on the visual hull and applied it to the graph-cuts algorithm to extract interior nodes in the thin parts and to prevent the thin parts from being cut off. Another tendency of the reconstruction method using the graph-cuts algorithm is over-shrinkage of the reconstructed models. This arises because the energy can also be reduced by cutting inside the true surface. To avoid this tendency, we applied a silhouette-rim constraint defined by the number of the silhouette-rims passing through each node.
By applying the adaptive erosion process and the silhouette-rim constraint, we succeeded in constructing a virtual performance with costumes including thin parts. This paper presents the results of the 3D reconstruction using the proposed method and some outputs of the 3D archive system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H., & Thrun, S. (2008). Performance capture from sparse multi-view video. In Proc. of SIGGRAPH.
Banno, A., Masuda, T., Oishi, T., & Ikeuchi, K. (2008). Flying laser range sensor for large-scale site-modeling and its applications in Bayon digital archival project. International Journal of Computer Vision (IJCV), 78(2–3), 207–222.
Boykov, Y., & Kolmogorov, V. (2003). Computing geodesics and minimal surfaces via graph cuts. In IEEE ICCV (pp. 26–33).
Debevec, P., Borshukov, G., & Yu, Y. (1998). Efficient view-dependent image-based rendering with projective texture-mapping. In 9th Eurographics rendering workshop (pp. 105–116).
Eisemann, M., Decker, B., Magnor, M., Bekaert, P., Aguiar, E., Ahmed, N., Theobalt, C., & Sellent, A. (2008). Floating textures. Eurographics, 27(2).
Furukawa, Y., & Ponce, J. (2007). Accurate, dense, and robust multi-view stereopsis. In IEEE conference on computer vision and pattern recognition (pp. 1–8).
Goldlucke, B., & Magnor, M. (2003). Real-time microfacet billboarding for free-viewpoint video rendering. In Proc. of ICIP (Vol. 3, pp. 713–716).
Hisatomi, K., Tomiyama, K., Katayama, M., & Iwadate, Y. (2009). A video production method using dynamic 3D models and its application to making a scene of crowd. SMPTE Motion Imaging Journal, 118(7), 29–36.
Katayama, M., Orihara, Y., Tomiyama, K., & Iwadate, Y. (2006). Archiving performances of Japanese traditional performing arts with a dynamic 3D model. In Eva digital cultural heritage for tourism (pp. 51–58).
Kazhdan, M., Bolitho, M., & Hoppe, H. (2006). Poisson surface reconstruction. In Eurographics symposium on geometry processing (pp. 61–70).
Kenmochi, Y., Kotani, K., & Imiya, A. (1999). Marching cubes method with connectivity. In Proc. of ICIP (Vol. 4, pp. 361–365).
Lorensen, W., & Cline, H. (1987). Marching cubes: a high resolution 3D surface construction algorithm. Computer Graphics, 21(4), 163–169.
Nobuhara, S., & Matsuyama, T. Deformable mesh model for complex multi-object 3D motion estimation from multi-viewpoint video, 3DPVT (2006).
Orzan, A., & Hasenfratz, J. M. (2005). Omnidirectional texturing of human actors from multiple view video sequences. In Romanian conference on computer-human interaction (pp. 133–136).
Sande, K. (2004). A practical setup for voxel coloring using off-the-shelf components, University of Amsterdam Bachelor thesis.
Seitz, S., & Dyer, C. (1999). Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision, 35(2), 151–173.
Sinha, S., Mordohai, P., & Pollefeys, M. (2007). Multi-view stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In International conference on computer vision (ICCV).
Starck, J., Miller, G., & Hilton, A. (2006). Volumetric stereo with silhouette and feature constraints. In British machine vision conference.
Tomiyama, K., Katayama, M., Orihara, Y., & Iwadate, Y. (2005). Arbitrary viewpoint images for performances of Japanese traditional art. In The 2nd European conference on visual production (pp. 68–75).
Tran, S., & Davis, L. (2006). 3D surface reconstruction using graph cuts with surface constraints. In Proc. of ECCV (pp. 218–231).
Tsai, R. Y. (1987). A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal of Robotics and Automation, RA-3(4), 323–344.
Vlasic, D., Baran, I., Matusik, W., & Popovic, J. (2008). Articulated mesh animation from multi-view silhouettes. ACM SIGGRAPH, 27, 3.
Vogiatzis, G., Torr, P., & Cipolla, R. (2005). Multi-view stereo via volumetric graph-cuts. In Proc. of CVPR (pp. 391–398).
Vogiatzis, G., Hernandez, C., Torr, P., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2241–2246.
Introduction to Noh & Kyogen website http://www2.ntj.jac.go.jp/unesco/noh/en/.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hisatomi, K., Katayama, M., Tomiyama, K. et al. 3D Archive System for Traditional Performing Arts. Int J Comput Vis 94, 78–88 (2011). https://doi.org/10.1007/s11263-011-0434-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-011-0434-2