Abstract
In the field of character animation, recent work has shown that data-driven reinforcement learning (RL) methods can address issues such as the difficulty of crafting reward functions, and train agents that can portray generalisable social behaviours. However, particularly when portraying subtle movements, these agents have shown a propensity for noticeable artefacts, that may have an adverse perceptual effect. Thus, for these agents to be effectively used in applications where they would interact with humans, the likelihood of these artefacts need to be minimised. In this paper, we present a novel architecture for agents to learn latent dynamics in a more efficient manner, while maintaining modelling flexibility and performance, and reduce the occurrence of noticeable artefacts when generating animation. Furthermore, we introduce a mean-sampling technique when applying learned latent stochastic dynamics to improve the stability of trained model-based RL agents.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adobe: Mixamo (2020). http://www.mixamo.com. Accessed 30 Jun 2021
Asadi-Aghbolaghi, M., et al.: Deep learning for action and gesture recognition in image sequences: a survey. In: Escalera, S., Guyon, I., Athitsos, V. (eds.) Gesture Recognition. TSSCML, pp. 539–578. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57021-1_19
Chakraborty, B.K., Sarma, D., Bhuyan, M.K., MacDorman, K.F.: Review of constraints on vision-based gesture recognition for human-computer interaction. IET Comput. Vis. 12(1), 3–15 (2017)
Cho, K., et al.: Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv arXiv:1406.1078 (2014)
Gamage, V., Ennis, C., Ross, R.: Data-driven reinforcement learning for virtual character animation control. arXiv preprint arXiv:2104.06358 (2021)
Ha, D., Schmidhuber, J.: World models. arXiv arXiv:1803.10122 (2018)
Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International Conference on Machine Learning, pp. 1861–1870. PMLR (2018)
Hafner, D., et al.: Learning latent dynamics for planning from pixels. In: International Conference on Machine Learning, pp. 2555–2565. PMLR (2019)
Hershey, J.R., Olsen, P.A.: Approximating the kullback leibler divergence between gaussian mixture models. In: 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP 2007, vol. 4, pp. IV-317. IEEE (2007)
Holden, D., Komura, T., Saito, J.: Phase-functioned neural networks for character control. ACM Trans. Graph. (TOG) 36(4), 42 (2017)
Holden, D., Saito, J., Komura, T.: A deep learning framework for character motion synthesis and editing. ACM Trans. Graph. (TOG) 35(4), 1–11 (2016)
Klein, A., Yumak, Z., Beij, A., van der Stappen, A.F.: Data-driven gaze animation using recurrent neural networks. In: Motion, Interaction and Games, pp. 1–11 (2019)
Latoschik, M.E., Roth, D., Gall, D., Achenbach, J., Waltemate, T., Botsch, M.: The effect of avatar realism in immersive social virtual realities. In: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology, pp. 1–10 (2017)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. arXiv arXiv:1509.02971 (2015)
Liu, L., Hodgins, J.: Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning. ACM Trans. Graph. (TOG) 37(4), 142 (2018)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, pp. 807–814 (2010)
Parent, R.: Computer Animation: Algorithms and Techniques. Elsevier, Amsterdam (2012)
Pavllo, D., Feichtenhofer, C., Auli, M., Grangier, D.: Modeling human motion with quaternion-based neural networks. Int. J. Comput. Vis. 1–18 (2019)
Peng, X.B., Abbeel, P., Levine, S., van de Panne, M.: Deepmimic: example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph. (TOG) 37(4), 1–14 (2018)
Peng, X.B., Berseth, G., Yin, K., Van De Panne, M.: Deeploco: dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Trans. Graph. (TOG) 36(4), 1–13 (2017)
Wu, Y., et al.: Effects of virtual human animation on emotion contagion in simulated inter-personal experiences. IEEE Trans. Vis. Comput. Graph. 20(4), 626–635 (2014)
Zhang, H., Starke, S., Komura, T., Saito, J.: Mode-adaptive neural networks for quadruped motion control. ACM Trans. Graph. (TOG) 37(4), 1–11 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gamage, V., Ennis, C., Ross, R. (2021). Latent Dynamics for Artefact-Free Character Animation via Data-Driven Reinforcement Learning. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12894. Springer, Cham. https://doi.org/10.1007/978-3-030-86380-7_55
Download citation
DOI: https://doi.org/10.1007/978-3-030-86380-7_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86379-1
Online ISBN: 978-3-030-86380-7
eBook Packages: Computer ScienceComputer Science (R0)