The Visual Computer

, Volume 31, Issue 5, pp 497–512 | Cite as

Adaptive motion synthesis for virtual characters: a survey

  • Shihui Guo
  • Richard Southern
  • Jian Chang
  • David Greer
  • Jian Jun Zhang
Survey

Abstract

Character motion synthesis is the process of artificially generating natural motion for a virtual character. In film, motion synthesis can be used to generate difficult or dangerous stunts without putting performers at risk. In computer games and virtual reality, motion synthesis enriches the player or participant experience by allowing for unscripted and emergent character behavior. In each of these applications the ability to adapt to changes to environmental conditions or to the character in a smooth and natural manner, while still conforming with user-specified constraints, determines the utility of a method to animators and industry practitioners. This focus on adaptation capability distinguishes our survey from other reviews which focus on general technology developments. Three main methodologies (example-based; simulation-based and hybrid) are summarised and evaluated using compound metrics: adaptivity, naturalness and controllability. By assessing existing techniques according to this classification we are able to determine how well a method corresponds to users’ expectations. We discuss optimization strategies commonly used in motion synthesis literature, and also contemporary perspectives from biology which give us a deeper insight into this problem. We also present observations and reflections from industry practitioners to reveal the operational constraints of character motion synthesis techniques. Our discussion and review presents a unique insight into the subject, and provide essential guidance when selecting appropriate methods to design an adaptive motion controller.

Keywords

Computer animation Character motion synthesis 

References

  1. 1.
    Abe, Y., da Silva, M., Popović, J.: Multiobjective control with frictional contacts. In: Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’07, pp. 249–258. Eurographics Association, Aire-la-Ville, Switzerland (2007)Google Scholar
  2. 2.
    Adam, A.: Identifying humans by their walk and generating new motions using hidden markov models. In: The University of British Columbia, Topics in AI (2008)Google Scholar
  3. 3.
    Al-Asqhar, R.A., Komura, T., Choi, M.G.: Relationship descriptors for interactive motion adaptation. In: Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’13, pp. 45–53. ACM, New York (2013)Google Scholar
  4. 4.
    Al Borno, M., de Lasa, M., Hertzmann, A.: Trajectory optimization for full-body movements with complex contacts. IEEE Trans. Vis. Comput. Graph. 19(8), 1405–1414 (2013)CrossRefGoogle Scholar
  5. 5.
    Alexander, R.: Optima for Animals. Princeton University Press, Princeton (1996)Google Scholar
  6. 6.
    Autodesk: The new art of virtual moviemaking. Tech. rep. (2008)Google Scholar
  7. 7.
    Bai, Y., Siu, K., Liu, C.K.: Synthesis of concurrent object manipulation tasks. ACM Trans. Graph. 31(6), 156:1–156:9 (2012)Google Scholar
  8. 8.
    Carvalho, S.R., Boulic, R., Vidal, C.A., Thalmann, D.: Latent motion spaces for full-body motion editing. Vis. Comput. 29(3), 171–188 (2013)CrossRefGoogle Scholar
  9. 9.
    Chai, J., Hodgins, J.K.: Constraint-based motion optimization using a statistical dynamic model. In: ACM SIGGRAPH 2007 papers, pp. 8 (2007)Google Scholar
  10. 10.
    Choi, K.J., Ko, H.S.: On-line motion retargetting. In: Proceedings of the Seventh Pacific Conference on Computer Graphics and Applications, 1999, pp. 32–42 (1999)Google Scholar
  11. 11.
    Choi, M.G., Kim, M., Hyun, K.L., Lee, J.: Deformable motion: squeezing into cluttered environments. Comput. Graph. Forum 30(2), 445–453 (2011)CrossRefGoogle Scholar
  12. 12.
    Coros, S., Beaudoin, P., van de Panne, M.: Robust task-based control policies for physics-based characters. In: ACM SIGGRAPH Asia 2009 papers, SIGGRAPH Asia ’09, pp. 170:1–170:9. ACM, New York (2009)Google Scholar
  13. 13.
    Coros, S., Beaudoin, P., van de Panne, M.: Generalized biped walking control. ACM Trans. Graph. (TOG) 29(4), 130 (2010)CrossRefGoogle Scholar
  14. 14.
    Coros, S., Beaudoin, P., Yin, K.K., van de Pann, M.: Synthesis of constrained walking skills. ACM Trans. Graph. 27, 113:1–113:9 (2008)Google Scholar
  15. 15.
    Coros, S., Karpathy, A., Jones, B., Reveret, L., van de Panne, M.: Locomotion skills for simulated quadrupeds. In: ACM SIGGRAPH 2011 papers, SIGGRAPH ’11, pp. 59:1–59:12. ACM, New York (2011)Google Scholar
  16. 16.
    Faloutsos, P., van de Panne, M., Terzopoulos, D.: Composable controllers for physics-based character animation. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’01, pp. 251–260. ACM, New York (2001)Google Scholar
  17. 17.
    Fang, A.C., Pollard, N.S.: Efficient synthesis of physically valid human motion. ACM Trans. Graph. (TOG) 22(3), 417–426 (2003)CrossRefGoogle Scholar
  18. 18.
    Feldman, A.G.: Once more on the equilibrium-point hypothesis(\(\lambda \) model) for motor control. J. Mot. Behav. 18(1), 17–54 (1986)CrossRefGoogle Scholar
  19. 19.
    Feng, A.W., Xu, Y., Shapiro, A.: An example-based motion synthesis technique for locomotion and object manipulation. In: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. I3D ’12, pp. 95–102. ACM, New York (2012)Google Scholar
  20. 20.
    Geijtenbeek, T., van de Panne, M., van der Stappen, A.F.: Flexible muscle-based locomotion for bipedal creatures. ACM Trans. Graph. (TOG) 32(6), 206 (2013)CrossRefGoogle Scholar
  21. 21.
    Geijtenbeek, T., Pronost, N.: Interactive character animation using simulated physics: a state-of-the-art review. Comp. Graph. Forum 31(8), 2492–2515 (2012)CrossRefGoogle Scholar
  22. 22.
    Ghahramani, Z.: An introduction to hidden markov models and bayesian networks. Int. J. Pattern Recognit. Artif. Intell. 15(01), 9–42 (2001)CrossRefGoogle Scholar
  23. 23.
    Gleicher, M.: Retargetting motion to new characters. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 42 (1998)Google Scholar
  24. 24.
    Grochow, K., Martin, S.L., Hertzmann, A., Popović, Z.: Style-based inverse kinematics. ACM Trans. Graph. (TOG) 23(3), 522–531 (2004)CrossRefGoogle Scholar
  25. 25.
    Guo, S., Chang, J., Cao, Y., Zhang, J.: A novel locomotion synthesis and optimisation framework for insects. Comput. Graph. 38, 78–85 (2014)CrossRefGoogle Scholar
  26. 26.
    Hecker, C., Raabe, B., Enslow, R.W., DeWeese, J., Maynard, J., van Prooijen, K.: Real-time motion retargeting to highly varied user-created morphologies. ACM Trans. Graph. (TOG) 27(3), 1–11 (2008)CrossRefGoogle Scholar
  27. 27.
    Hertzmann, A., Zordan, V.: Physics-based characters. Comput. Graph. Appl. IEEE 31(4), 20–21 (2011)CrossRefGoogle Scholar
  28. 28.
    Herzeg, I.: Crysis2: Getting more interactivity out of animation data. Tech. rep., Game Developers Conference (2011)Google Scholar
  29. 29.
    Ho, E.S.L., Komura, T., Tai, C.L.: Spatial relationship preserving character motion adaptation. ACM Trans. Graph. 29(4), 1–8 (2010)CrossRefGoogle Scholar
  30. 30.
    Ikemoto, L., Arikan, O., Forsyth, D.: Knowing when to put your foot down. In: Proceedings of the 2006 Symposium on Interactive 3D Graphics and Games. I3D ’06, pp. 49–53. ACM, New York (2006)Google Scholar
  31. 31.
    Jain, S., Liu, C.K.: Interactive synthesis of human-object interaction. In: Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’09, pp. 47–53. ACM, New York (2009)Google Scholar
  32. 32.
    Jain, S., Liu, C.K.: Controlling physics-based characters using soft contacts. ACM Trans. Graph. (SIGGRAPH Asia) 30, 163:1–163:10 (2011)Google Scholar
  33. 33.
    Jain, S., Ye, Y., Liu, C.K.: Optimization-based interactive motion synthesis. ACM Trans. Graph. 28, 10:1–10:12 (2009)Google Scholar
  34. 34.
    Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. ACM Trans. Graph. 21, 473–482 (2002)CrossRefGoogle Scholar
  35. 35.
    Kulpa, R., Multon, F., Arnaldi, B.: Morphology-independent representation of motions for interactive human-like animation. In: Computer Graphics Forum, vol. 24, pp. 343–351 (2005)Google Scholar
  36. 36.
    de Lasa, M., Mordatch, I., Hertzmann, A.: Feature-based locomotion controllers. ACM Trans. Graph. (TOG) 29(4), 131 (2010)Google Scholar
  37. 37.
    Latash, M.: Neurophysiological Basis of Movement. Human Kinetics Publishers, Champaign (2008)Google Scholar
  38. 38.
    Lau, M., Bar-Joseph, Z., Kuffner, J.: Modeling spatial and temporal variation in motion data. ACM Trans. Graph. (TOG) 28(5), 1–10 (2009)CrossRefGoogle Scholar
  39. 39.
    Lee, J., Shin, S.Y.: A hierarchical approach to interactive motion editing for human-like figures. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 48 (1999)Google Scholar
  40. 40.
    Lee, Y., Kim, S., Lee, J.: Data-driven biped control. ACM Trans. Graph. 29, 129:1–129:8 (2010)Google Scholar
  41. 41.
    Liu, C.K., Popović, Z.: Synthesis of complex dynamic character motion from simple animations. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques—SIGGRAPH ’02, pp. 408. San Antonio, Texas (2002)Google Scholar
  42. 42.
    Liu, F., Southern, R., Guo, S., Yang, X., Zhang, J.: Motion adaptation with motor invariant theory. IEEE Trans. Cybern. 43(3), 1131–1145 (2013)CrossRefGoogle Scholar
  43. 43.
    Liu, L., Yin, K., van de Panne, M., Guo, B.: Terrain runner: control, parameterization, composition, and planning for highly dynamic motions. ACM Trans. Graph. 31(6), 154:1–154:10 (2012)Google Scholar
  44. 44.
    Ma, W., Xia, S., Hodgins, J., Yang, X., Li, C., Wang, Z.: Modeling style and variation in human motion. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 21–30. Eurographics Association (2010)Google Scholar
  45. 45.
    Macchietto, A., Zordan, V., Shelton, C.R.: Momentum control for balance. ACM Trans. Graph. 28, 80:1–80:8 (2009)Google Scholar
  46. 46.
    McIntyre, J., Zago, M., Berthoz, A., Lacquaniti, F., et al.: Does the brain model newton’s laws? Nat. Neurosci. 4(7), 693–694 (2001)CrossRefGoogle Scholar
  47. 47.
    Min, J., Chen, Y.L., Chai, J.: Interactive generation of human animation with deformable motion models. ACM Trans. Graph. 29, 9:1–9:12 (2009)Google Scholar
  48. 48.
    Monzani, J.S., Baerlocher, P., Boulic, R., Thalmann, D.: Using an intermediate skeleton and inverse kinematics for motion retargeting. In: Computer Graphics Forum, vol. 19, pp. 11–19 (2000)Google Scholar
  49. 49.
    Mordatch, I., de Lasa, M., Hertzmann, A.: Robust physics-based locomotion using low-dimensional planning. ACM Trans. Graph. 29, 71:1–71:8 (2010)Google Scholar
  50. 50.
    Muico, U., Popović, J., Popović, Z.: Composite control of physically simulated characters. ACM Trans. Graph. 30, 16:1–16:11 (2011)Google Scholar
  51. 51.
    Mukai, T., Kuriyama, S.: Geostatistical motion interpolation. ACM Trans. Graph. 24, 1062–1070 (2005)CrossRefGoogle Scholar
  52. 52.
    Pejsa, T., Pandzic, I.: State of the art in example-based motion synthesis for virtual characters in interactive applications. Comput. Graph. Forum 29(1), 202–226 (2010)CrossRefGoogle Scholar
  53. 53.
    Popović, Z., Witkin, A.: Physically based motion transformation. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques—SIGGRAPH ’99, pp. 11–20 (1999)Google Scholar
  54. 54.
    Pratt, J., Chew, C., Torres, A., Dilworth, P., Pratt, G.: Virtual model control: an intuitive approach for bipedal locomotion. Int. J. Robot. Res. 20(2), 129 (2001)CrossRefGoogle Scholar
  55. 55.
    Pronost, N., Multon, F., Li, Q., Geng, W., Kulpa, R., Dumont, G.: Morphology independent motion retrieval and control. Int. J. 100, 1 (2009)Google Scholar
  56. 56.
    Qinxin, Y.: A decision network framework for the behavioral animation of virtual humans. Ph.D. thesis, University of Toronto, Ontario, Toronto, Canada (2007)Google Scholar
  57. 57.
    Rother, D.D.: Bayesian network applications in molecular biology, computer graphics and computer vision. Ph.D. thesis, University of Minnesota, Minneapolis, MN, USA (2008)Google Scholar
  58. 58.
    Safonova, A., Hodgins, J.K., Pollard, N.S.: Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM SIGGRAPH 2004 Papers. SIGGRAPH ’04, pp. 514–521. ACM, New York (2004)Google Scholar
  59. 59.
    Seol, Y., O’Sullivan, C., Lee, J.: Creature features: Online motion puppetry for non-human characters. In: Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’13, pp. 213–221. ACM, New York (2013)Google Scholar
  60. 60.
    Shadmehr, R., Mussa-Ivaldi, F.: Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14(5), 3208–3224 (1994)Google Scholar
  61. 61.
    Sharon, D., van de Panne, M.: Synthesis of controllers for stylized planar bipedal walking. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005, pp. 2387–2392 (2005)Google Scholar
  62. 62.
    Shin, H.J., Kovar, L., Gleicher, M.: Physical touch-up of human motions. In: Proceedings of the 11th Pacific Conference on Computer Graphics and Applications. PG ’03, pp. 194–203. IEEE Computer Society, Washington, DC (2003)Google Scholar
  63. 63.
    Shin, H.J., Lee, J., Shin, S.Y., Gleicher, M.: Computer puppetry: an importance-based approach. ACM Trans. Graph. 20, 67–94 (2001)Google Scholar
  64. 64.
    Shiratori, T., Coley, B., Cham, R., Hodgins, J.K.: Simulating balance recovery responses to trips based on biomechanical principles. In: Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’09, pp. 37–46. ACM, New York (2009)Google Scholar
  65. 65.
    Shum, H.P.H., Komura, T., Yamazaki, S.: Simulating multiple character interactions with collaborative and adversarial goals. IEEE Trans. Vis. Comput. Graph. 99(PrePrints) (2010)Google Scholar
  66. 66.
    da Silva, M., Durand, F., Popović, J.: Linear bellman combination for control of character animation. ACM Trans. Graph. 28, pp. 82:1–82:10 (2009)Google Scholar
  67. 67.
    Sok, K.W., Kim, M., Lee, J.: Simulating biped behaviors from human motion data. In: ACM SIGGRAPH 2007 papers, SIGGRAPH ’07. ACM, New York (2007)Google Scholar
  68. 68.
    Sok, K.W., Yamane, K., Lee, J., Hodgins, J.: Editing dynamic human motions via momentum and force. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’10, pp. 11–20. Eurographics Association, Aire-la-Ville, Switzerland (2010)Google Scholar
  69. 69.
    Spong, M., Bullo, F.: Controlled symmetries and passive walking. IEEE Trans. Autom. Control 50(7), 1025–1031 (2005)CrossRefMathSciNetGoogle Scholar
  70. 70.
    Sun, H.C., Metaxas, D.N.: Automating gait generation. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’01, pp. 261–270. ACM, New York (2001)Google Scholar
  71. 71.
    Taesoo, K., Young-Sang, C., Sang, I.P., Sung Yong, S.: Two-character motion analysis and synthesis. IEEE Trans. Vis. Comput. Graph. 14, 707–720 (2008)CrossRefGoogle Scholar
  72. 72.
    Taga, G., Yamaguchi, Y., Shimizu, H.: Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biol. Cybern. 65(3), 147–159 (1991)CrossRefMATHGoogle Scholar
  73. 73.
    Tak, S., Ko, H.S.: A physically-based motion retargeting filter. ACM Trans. Graph. 24, 98–117 (2005)CrossRefGoogle Scholar
  74. 74.
    Takahashi, C., Scheidt, R., Reinkensmeyer, D.: Impedance control and internal model formation when reaching in a randomly varying dynamical environment. J. Neurophysiol. 86(2), 1047 (2001)Google Scholar
  75. 75.
    Tan, J., Gu, Y., Turk, G., Liu, C.K.: Articulated swimming creatures. In: ACM SIGGRAPH 2011 papers, SIGGRAPH ’11, pp. 58:1–58:12. ACM, New York (2011)Google Scholar
  76. 76.
    Tsai, Y.Y., Lin, W.C., Cheng, K., Lee, J., Lee, T.Y.: Real-time physics-based 3d biped character animation using an inverted pendulum model. IEEE Trans. Vis. Comput. Graph. 16(2), 325–337 (2010)CrossRefGoogle Scholar
  77. 77.
    Wampler, K., Andersen, E., Herbst, E., Lee, Y., Popović, Z.: Character animation in two-player adversarial games. ACM Trans. Graph. 29, 26:1–26:13 (2010)Google Scholar
  78. 78.
    Wampler, K., Popović, Z.: Optimal gait and form for animal locomotion. In: ACM SIGGRAPH 2009 papers, SIGGRAPH ’09, pp. 60:1–60:8. ACM, New York (2009)Google Scholar
  79. 79.
    Wang, J., Fleet, D., Hertzmann, A.: Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. (TOG) 29(4), 73 (2010)Google Scholar
  80. 80.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Optimizing walking controllers. In: ACM SIGGRAPH Asia 2009 papers, SIGGRAPH Asia ’09, pp. 168:1–168:8. ACM, New York (2009)Google Scholar
  81. 81.
    Wang, J.M., Hamner, S.R., Delp, S.L., Koltun, V.: Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph. 31(4), 25:1–25:11 (2012)Google Scholar
  82. 82.
    Wei, X., Min, J., Chai, J.: Physically valid statistical models for human motion generation. ACM Trans. Graph. (TOG) 30(3), 19 (2011)CrossRefGoogle Scholar
  83. 83.
    Williamson, M.: Robot arm control exploiting natural dynamics. Ph.D. thesis, Massachusetts Institute of Technology (1999)Google Scholar
  84. 84.
    Witkin, A., Kass, M.: Spacetime constraints. ACM SIGGRAPH 88. In: Computer Graphics (Conference Proceedings), pp. 159–168. ACM, New York (1988)Google Scholar
  85. 85.
    Wu, C.C., Zordan, V.: Goal-directed stepping with momentum control. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’10, pp. 113–118. Eurographics Association, Aire-la-Ville, Switzerland (2010)Google Scholar
  86. 86.
    Wu, J.c., Popović, Z.: Terrain-adaptive bipedal locomotion control. ACM Trans. Graph. 29, 72:1–72:10 (2010)Google Scholar
  87. 87.
    Yamane, K., Ariki, Y., Hodgins, J.: Animating non-humanoid characters with human motion data. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer, Animation, pp. 169–178 (2010)Google Scholar
  88. 88.
    Yamane, K., Nakamura, Y.: Dynamics filter-concept and implementation of online motion generator for human figures. In: IEEE International Conference on Robotics and Automation, 2000. Proceedings. ICRA’00, vol. 1, pp. 688–694 (2000)Google Scholar
  89. 89.
    Ye, Y., Liu, C.K.: Animating responsive characters with dynamic constraints in near-unactuated coordinates. In: ACM SIGGRAPH Asia 2008 papers, SIGGRAPH Asia ’08, pp. 112:1–112:5. ACM, New York (2008)Google Scholar
  90. 90.
    Ye, Y., Liu, C.K.: Synthesis of responsive motion using a dynamic model. In: Computer Graphics Forum, vol. 29, pp. 555–562 (2010)Google Scholar
  91. 91.
    Ye, Y., Liu, C.K.: Synthesis of detailed hand manipulations using contact sampling. ACM Trans. Graph. 31(4), 41:1–41:10 (2012)Google Scholar
  92. 92.
    Yin, K., Coros, S., Beaudoin, P., van de Panne, M.: Continuation methods for adapting simulated skills. ACM Trans. Graph. 27, 81:1–81:7 (2008)Google Scholar
  93. 93.
    Yin, K., Loken, K., van de Panne, M.: Simbicon: simple biped locomotion control. In: ACM SIGGRAPH 2007 papers, SIGGRAPH ’07. ACM, New York (2007)Google Scholar
  94. 94.
    Zehr, E., Stein, R.: What functions do reflexes serve during human locomotion? Prog. Neurobiol. 58(2), 185–205 (1999)CrossRefGoogle Scholar
  95. 95.
    Zordan, V., Macchietto, A., Medina, J., Soriano, M., Wu, C.C.: Interactive dynamic response for games. In: Proceedings of the 2007 ACM SIGGRAPH Symposium on Video Games. Sandbox ’07, pp. 9–14. ACM, New York (2007)Google Scholar
  96. 96.
    Zordan, V.B., Hodgins, J.K.: Motion capture-driven simulations that hit and react. In: Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. SCA ’02, pp. 89–96. ACM, New York (2002)Google Scholar
  97. 97.
    Zordan, V.B., Majkowska, A., Chiu, B., Fast, M.: Dynamic response for motion capture animation. ACM Trans. Graph. 24, 697–701 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Shihui Guo
    • 1
  • Richard Southern
    • 1
  • Jian Chang
    • 1
  • David Greer
    • 1
  • Jian Jun Zhang
    • 1
  1. 1.National Centre for Computer AnimationBournemouth UniversityBournemouthUK

Personalised recommendations