Encyclopedia of Computer Graphics and Games

Living Edition
| Editors: Newton Lee

Interacting with a Fully Simulated Self-Balancing Bipedal Character in Augmented and Virtual Reality

Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-08234-9_220-1



Simulated characters offer rich and realistic interactions with users and dynamic environments. They can be thought of as compliant robots in the real world. Hence, for every unique push or perturbation, the character responds equally in a unique and realistic fashion. As the result, the immersion for the user is greatly increased and all the more powerful. This article provides insights on how to make compelling interactions with a a self-balancing bipedal character in Virtual and Augmented Reality settings. It also describes a general method to interface a simulated character with a traditional skeleton, used in modern game engines such as Unity and Unreal Engine – thereby making simulated characters more accessible.


High-level game engines such as Unity and Unreal Enginemake easily accessible...


Motion Clips Kinematic Skeleton (KS) Explicit Transfer Function Quality Simulation Target Pose 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.


  1. Abe, Y., da Silva, M., Popović, J.: Multiobjective control with frictional contacts. In Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’07, pp. 249–258. Eurographics Association (2007)Google Scholar
  2. Arikan, O., Forsyth, D.A., O’Brien, J.F.: Pushing people around. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’05, pp. 59–66. ACM (2005)Google Scholar
  3. Chai, J., Hodgins, J.K.: Performance animation from low-dimensional control signals. In ACM SIGGRAPH 2005 Papers, SIGGRAPH ’05, pp. 686–696. ACM (2005)Google Scholar
  4. Choi, K.-J., Ko, H.-S.: On-line motion retargetting. In Proceedings of the 7th Pacific Conference on Computer Graphics and Applications, PG ’99. IEEE Computer Society (1999)Google Scholar
  5. Clavet, S.: Motion matching and the road to next-gen animation. In Proceedings of GDC (2016)Google Scholar
  6. Coros, S., Beaudoin, P., van de Panne, M.: Generalized biped walking control. In ACM SIGGRAPH 2010 Papers, SIGGRAPH ’10, pp. 130:1–130:9. ACM (2010)Google Scholar
  7. Coros, S., Karpathy, A., Jones, B., Reveret, L., van de Panne, M.: Locomotion skills for simulated quadrupeds. In ACM SIGGRAPH 2011 Papers, SIGGRAPH ’11, pp. 59:1–59:12. ACM (2011)Google Scholar
  8. Da Silva, M., Abe, Y., Popović, J.: Simulation of human motion data using short-horizon model-predictive control. Comput. Graph. Forum. 27(2), 371–380 (2008)CrossRefGoogle Scholar
  9. de Lasa, M., Mordatch, I., Hertzmann, A.: Feature-based locomotion controllers. In ACM SIGGRAPH 2010 Papers, SIGGRAPH ’10, pp. 131:1–131:10. ACM (2010)Google Scholar
  10. Dontcheva, M., Yngve, G., Popović, Z.: Layered acting for character animation. In ACM SIGGRAPH 2003 Papers, SIGGRAPH ’03, pp. 409–416. ACM (2003)Google Scholar
  11. Gleicher, M.: Motion editing with spacetime constraints. In Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D ’97, pp. 139–ff. ACM (1997)Google Scholar
  12. Hodgins, J.K., Wooten, W.L., Brogan, D.C., O’Brien, J.F.: Animating human athletics. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’95, pp. 71–78 (1995)Google Scholar
  13. Holden, D., Komura, T., Saito, J.: Phase-functioned neural networks for character control. ACM Trans. Graph. 36(4), 1–42 (2017)CrossRefGoogle Scholar
  14. Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’02, pp. 473–482. ACM (2002)Google Scholar
  15. Lee, Y., Wampler, K., Bernstein, G., Popović, J., Popović, Z.: Motion fields for interactive character locomotion. In ACM SIGGRAPH Asia 2010 Papers, SIGGRAPH ASIA ’10, pp. 138:1–138:8. ACM (2010)Google Scholar
  16. Levine, S., Wang, J.M., Haraux, A., Popović, Z., Koltun, V.: Continuous character control with low-dimensional embeddings. ACM Trans. Graph. 31(4), 1–28 (2012)CrossRefGoogle Scholar
  17. Liu, L., Van De Panne, M., Yin, K.: Guided learning of control graphs for physics-based characters. ACM Trans. Graph. 35(3), 1–29 (2016)Google Scholar
  18. Macchietto, A., Zordan, V., Shelton, C.R.: Momentum control for balance. In ACM SIGGRAPH 2009 Papers, SIGGRAPH ’09, pp. 80:1–80:8. ACM (2009)Google Scholar
  19. Mordatch, I., de Lasa, M., Hertzmann, A.: Robust physics-based locomotion using low-dimensional planning. In ACM SIGGRAPH 2010 Papers, SIGGRAPH ’10, pp. 71:1–71:8. ACM (2010)Google Scholar
  20. Peng, X.B., Berseth, G., van de Panne, M.: Terrain-adaptive locomotion skills using deep reinforcement learning. ACM Trans. Graph. 35(4), 1–81 (2016)Google Scholar
  21. Peng, X.B., Berseth, G., Yin, K., van de Panne, M.: Deeploco: dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Trans. Graph. 36(4), 1–41 (2017)CrossRefGoogle Scholar
  22. Rabbani, A.H., van de Panne, M., Kry, P.G.: Anticipatory balance control. In Proceedings of the Seventh International Conference on Motion in Games, MIG ’14, pp. 71–76. ACM (2014)Google Scholar
  23. Rhodin, H., Tompkin, J., In Kim, K., Varanasi, K., Seidel, H.-P., Theobalt, C.: Interactive motion mapping for real-time character control. Comput. Graph. Forum. 33(2), 273–282 (2014)CrossRefGoogle Scholar
  24. Rose, C., Cohen, M.F., Bodenheimer, B.: Verbs and adverbs: multidimensional motion interpolation. IEEE Comput. Graph. Appl. 18(5), 32–40 (1998)CrossRefGoogle Scholar
  25. Seol, Y., O’Sullivan, C., Lee, J.: Creature features: Online motion puppetry for non-human characters. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’13, pp. 213–221. ACM (2013)Google Scholar
  26. Tan, J., Yuting, G., Karen Liu, C., Turk, G.: Learning bicycle stunts. ACM Trans. Graph. 33(4), 1–50 (2014)CrossRefGoogle Scholar
  27. Yamane, K., Ariki, Y., and Hodgins, J.: Animating non-humanoid characters with human motion data. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’10, pp. 169–178. Eurographics Association (2010)Google Scholar
  28. Yin, K.K., Loken, K., van de Panne, M.. Simbicon: Simple biped locomotion control. In ACM SIGGRAPH 2007 Papers, SIGGRAPH ’07. ACM (2007)Google Scholar

Authors and Affiliations

  1. 1.ETH ZurichZürichSwitzerland
  2. 2.Disney ResearchZürichSwitzerland