Abstract
Recently, interactive character control models based on neural network have become a hot research topic in computer graphics and motion synthesis. A real-time interactive character control model with two substructures called gated neural network is proposed in this paper. In the first part of the model, a gated network is used to calculate the expert weights based on the user’s input parameters and the character’s current pose dynamically. Mixture of experts is used in choosing different control strategies for user input. The character posture is controlled by selecting mode-adjustment or phase-adjustment. In the second part, a simple neural network is used to adjust the character’s state through additional input parameters on different landscapes and to synthesize the final character motion. Experimental results show that the model can improve the performance of character control.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bengio E, Bacon P L, Pineau J et al (2015) Conditional computation in neural networks for faster models[J]. arXiv preprint arXiv:1511.06297
Chu M, Thuerey N (2017) Data-driven synthesis of smoke flows with CNN-based feature descriptors[J]. ACM Transactions on Graphics (TOG) 36(4):69
Coros S, Beaudoin P, Yin KK et al (2008) Synthesis of constrained walking skills[C]. Acm Siggraph Asia. Singapore: ACM 1–9
Diba A, Sharma V, Van Gool L (2017) Deep temporal linear encoding networks[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. Honolulu 2329–2338
Du Y, Wang W, Wang L (2015) Hierarchical recurrent neural network for skeleton based action recognition[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. Boston 1110–1118
Fragkiadaki K, Levine S, Felsen P et al (2015) Recurrent network models for human dynamics[C]. Computer vision (ICCV), 2015 IEEE international conference on. Santiago: IEEE 4346–4354
Holden D, Komura T, Saito J (2017) Phase-functioned neural networks for character control[J]. ACM Transactions on Graphics (TOG) 36(4):42
Holden D, Saito J, Komura T (2016) A deep learning framework for character motion synthesis and editing[J]. ACM Transactions on Graphics (TOG) 35(4):138
Holden D, Saito J, Komura T et al (2015) Learning motion manifolds with convolutional autoencoders[C]. SIGGRAPH Asia 2015 Technical briefs. Kobe: ACM 18
Huang TC, Huang YJ, Lin WC (2013) Real-time horse gait synthesis[J]. Computer Animation and Virtual Worlds 24(2):87–95
Lau M, Kuffner JJ (2005) Behavior planning for character animation[C]. Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on computer animation. Los Angeles: ACM 271–280
Lee Y, Wampler K, Bernstein G et al (2010) Motion fields for interactive character locomotion[C]. ACM transactions on graphics (TOG). New York: ACM 29(6): 138.
Liu L, Hodgins J (2017) Learning to schedule control fragments for physics-based characters using deep q-learning[J]. ACM Transactions on Graphics (TOG) 36(3):29
Liu L, Yin KK, van de Panne M et al (2010) Sampling-based contact-rich motion control[C]. ACM transactions on graphics (TOG). Seoul: ACM 29(4):128
Min J, Chai J (2012) Motion graphs++: a compact generative model for semantic motion analysis and synthesis[J]. ACM Transactions on Graphics (TOG) 31(6):153
Mittelman R, Kuipers B, Savarese S et al (2014) Structured recurrent temporal restricted boltzmann machines[C]. International conference on machine learning. Beijing: JMLR II-1647
Mordatch I, Lowrey K, Andrew G et al (2015) Interactive control of diverse complex characters with neural networks[C]. Adv Neural Inf Process Syst Montreal 3132–3140.
Peng XB, Berseth G, Van de Panne M (2016) Terrain-adaptive locomotion skills using deep reinforcement learning[J]. ACM Transactions on Graphics (TOG) 35(4):81
Peng XB, Berseth G, Yin K et al (2017) DeepLoco: dynamic locomotion skills using hierarchical deep reinforcement learning[J]. ACM Trans Graph 36(4):1–13
Safonova A, Hodgins JK (2007) Construction and optimal search of interpolated motion graphs[J]. ACM Trans Graph 26(3):106
Salimans T, Goodfellow I, Zaremba W et al (2016) Improved techniques for training gans[C]. Advances in neural information processing systems. Barcelona 2234–2242
Wang X, Chen L, Jing J et al (2016) Human motion capture data retrieval based on semantic thumbnail[J]. Multimed Tools Appl 75(19):11723–11740
Wang Z, Mao T, Jiang H, Xia S (2010) Guarder: virtual drilling system for crowd evacuation under emergency scheme——guarder[J]. Journal of Computer Research and Development 6:969–978
Zha M, Pan Z, Zhang M (2016) A motion segmentation based algorithm of human motion alignment[C]// Asian simulation conference. Springer, Singapore, pp 660–670
Zhang H, Starke S, Komura T et al (2018) Mode-adaptive neural networks for quadruped motion control[J]. ACM Transactions on Graphics (TOG) 37(4):145
Zhu D, Wang Z (2009) Human animation synthesis based on primitive movement—guarder[J]. Journal of Computer Research and Development 4:610–617
Acknowledgements
The work is supported by National Science and Technology Innovation 2030 Major Project (2018AAA0100703) of the Ministry of Science and Technology of China, the National Natural Science Foundation of China (No. 61672451, No. 61303142), and Provincial Key Research and Development Plan of Zhejiang Province (No. 2019C03137).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, X., Jiang, X., Regedzai, G.R. et al. Gated neural network framework for interactive character control. Multimed Tools Appl 80, 16229–16246 (2021). https://doi.org/10.1007/s11042-020-08792-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-08792-y