Performance-Driven Hybrid Full-Body Character Control for Navigation and Interaction in Virtual Environments
This paper presents a hybrid character control interface that provides the ability to synthesize in real-time a variety of actions based on the user’s performance capture. The proposed methodology enables three different performance interaction modules: the performance animation control that enables the direct mapping of the user’s pose to the character, the motion controller that synthesizes the desired motion of the character based on an activity recognition methodology, and the hybrid control that lies within the performance animation and the motion controller. With the methodology presented, the user will have the freedom to interact within the virtual environment, as well as the ability to manipulate the character and to synthesize a variety of actions that cannot be performed directly by him/her, but which the system synthesizes. Therefore, the user is able to interact with the virtual environment in a more sophisticated fashion. This paper presents examples of different scenarios based on the three different full-body character control methodologies.
KeywordsCharacter animation Hybrid controller Navigation Object manipulation Virtual reality interaction
Supplementary material 1 (mp4 21354 KB)
- 1.Microsoft Kinect Motion Capture System, from http://www.microsoft.com/en-us/kinectforwindows/. Accessed 12 2016.
- 2.Assus Xtion Motion Capture Device, from http://www.asus.com/Multimedia/Xtion_PRO/. Accessed 12 2016.
- 6.Sarris, N., & Strintzis, M. G. (2005). 3D modeling and animation: Synthesis and analysis techniques for the human body. Hershey: IGI Global.Google Scholar
- 9.Mousas, C., & Anagnostopoulos, C.-N. (2015). CHASE: Character animation scripting environment. In Virtual Reality Interaction and Physical Simulation, pp. 55–62Google Scholar
- 10.Mousas, C., & Anagnostopoulos, C.-N. (2015). Character animation scripting environment. Encyclopedia of computer graphics and games. Berlin: Springer.Google Scholar
- 12.Thorne, M., Burke, D., & van de Panne, M. (2007). Motion doodles: An interface for sketching character motion. In ACM SIGGRAPH 2007 courses, p. 24.Google Scholar
- 13.Davis, J., Agrawala, M., Chuang, E., Popović, Z., Salesin, D. (2003). A sketching interface for articulated figure animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 320–328.Google Scholar
- 15.Liu, H., Wei, X., Chai, J., Ha, I., Rhee, T. (2011) Realtime human motion control with a small number of inertial sensors. In Symposium on Interactive 3D Graphics and Games, pp. 133–140.Google Scholar
- 16.Bleiweiss, A., Eshar, D., Kutliroff, G., Lerner, A., Oshrat, Y., & Yanai, Y. (2010). Enhanced interactive gaming by blending full-body tracking and gesture animation. In ACM SIGGRAPH ASIA 2010 Sketches, p. 34.Google Scholar
- 17.Ouzounis, C., Mousas, C., Anagnostopoulos, C.-N., & Newbury, P. (2015). Using personalized finger gestures for navigating virtual characters. In Virtual Reality Interaction and Physical Simulation, pp. 5–14.Google Scholar
- 20.Kovar, L., & Gleicher, M. (2003). Flexible automatic motion blending with registration curves. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 214–224.Google Scholar
- 21.Park, S. I., Shin, H. J., & Shin, S. Y. (2002). On-line locomotion generation based on motion blending. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 105–111.Google Scholar
- 24.Mousas, C., & Newbury, P. (2012). Real-time motion synthesis for multiple goal-directed tasks using motion layers. In Virtual Reality Interaction and Physical Simulation, pp. 79–85.Google Scholar
- 25.Witkin, A., & Popović, Z. (1995). Motion warping. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp. 105–108.Google Scholar
- 26.Gleicher, M. (1998). Retargetting motion to new characters. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 33–42.Google Scholar
- 27.Feng, A. W., Xu, Y., & Shapiro, A. (2012). An example-based motion synthesis technique for locomotion and object manipulation. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 95–102.Google Scholar
- 31.Slyper, R., Hodgins, J. K. (2008). Action capture with accelerometers. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 193–199.Google Scholar
- 32.Numaguchi, N., Nakazawa, A., Shiratori, T., & Hodgins, J. K. (2011). A puppet interface for retrieval of motion capture data. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 157–166.Google Scholar
- 33.Yin, K., & Pai, D. K. (2003). Footsee: An interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 329–338.Google Scholar
- 34.Misumi, H., Fujimura, W., Kosaka, T., Hattori, M., & Shirai, A. (2011). GAMIC: Exaggerated real time character animation control method for full-body gesture interaction systems. In SIGGRAPH Posters, p. 5.Google Scholar
- 36.Mousas, C., Newbury, P., & Anagnostopoulos, C.-N. (2014). Data-driven motion reconstruction using local regression models. In Artificial Intelligence Applications and Innovations, pp. 364–374.Google Scholar
- 37.Mousas, C., Newbury, P., & Anagnostopoulos, C.-N. (2014). Evaluating the covariance matrix constraints for data-driven statistical human motion reconstruction. In Spring Conference on Computer Graphics, pp. 99–106.Google Scholar
- 38.Mousas, C., Newbury, P., & Anagnostopoulos, C.-N. (2014). Efficient hand-over motion reconstruction. In International Conference on Computer Graphics, Visualization and Computer Vision, pp. 111–120.Google Scholar
- 45.Seol, Y., O’Sullivan, C., & Lee, J. (2013). Creature features: Online motion puppetry for non-human characters. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 213–221.Google Scholar
- 47.Powerset Algorithm, from http://rosettacode.org/wiki/Power_set. Accessed 12 2016.
- 49.Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., & Moore, R. (2011). Real-time human pose recognition in parts from single depth images. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304.Google Scholar
- 50.CMU Graphics Lab Motion Capture Database, from http://mocap.cs.cmu.edu/. Accessed 12 2016.
- 52.He, Z., & Jin, L. (2009). Activity recognition from acceleration data based on discrete consine transform and SVM. In IEEE International Conference on Systems, Man and Cybernetics, pp. 5041–5044.Google Scholar
- 54.Shoulson, A., Marshak, N., Kapadia, M., & Badler, N. I. (2013). ADAPT: The agent development and prototyping testbed. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 9–18.Google Scholar
- 55.Thiebaux, M., Marsella, S., Marshall, A. N., & Kallmann, M. (2008). Smartbody: Behavior realization for embodied conversational agents. In Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Vol. 1, pp. 151–158.Google Scholar
- 57.Liang, X., Hoyet, L., Geng, W., & Multon, F. (2010). Responsive action generation by physically-based motion retrieval and adaptation. In Motion in Games, pp. 313–324.Google Scholar
- 58.Al-Asqhar, R. A., Komura, T., & Choi, M. G. (2013). Relationship descriptors for interactive motion adaptation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 45–53.Google Scholar
- 59.Dam, P., Braz, P., Raposo, A. (2013). A study of nnavigation and selection techniques in virtual environments using microsoft kinect®. In International Conference on Virtual, Augmented and Mixed Reality, pp. 139–148.Google Scholar
- 60.Mousas, C. (2017). Towards developing an easy-to-use scripting environment for animating virtual characters. arXiv preprint arXiv:1702.03246.