Continuous Control of Style and Style Transitions through Linear Interpolation in Hidden Markov Model Based Walk Synthesis

  • Joëlle Tilmanne
  • Thierry Dutoit
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7380)

Abstract

We present a Hidden Markov Model (HMM) based stylistic walk synthesizer, where the synthesized styles are combinations or exaggerations of the walk styles present in the training database. Our synthesizer is also capable of generating walk sequences with controlled style transitions. In a first stage, Hidden Markov Models of eleven different gait styles are trained, using a database of motion capture walk sequences. In a second stage, the probability density functions inside the stylistic models are interpolated or extrapolated in order to synthesize walks with styles or style intensities that were not present in the training database. A continuous model of the style parameter space is thus constructed around the eleven original walk styles. Qualitative user evaluation of the synthesized sequences showed that the naturalness of motions is preserved after linear interpolation between styles and that evaluators are sensitive to the interpolation factor.

Keywords

HMM gait style control synthesis motion capture 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Brand, M., Hertzmann, A.: Style machines. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 183–192 (2000)Google Scholar
  2. 2.
    Bruderlin, A., Williams, L.: Motion signal processing. In: SIGGRAPH 1995 Proceedings, pp. 97–104 (1995)Google Scholar
  3. 3.
    Calinon, S., Guenter, F., Billard, A.: On Learning, Representing, and Generalizing a Task in a Humanoid Robot. IEEE Transactions on Systems, Man and Cybernetics 37(2), 286–298 (2007)CrossRefGoogle Scholar
  4. 4.
    Chiu, C., Marsella, S.: A style controller for generating virtual human behaviors. In: Proceedings of AAMAS 2011, The 10th International Conference on Autonomous Agents and Multiagent Systems, vol. 3, pp. 1023–1030 (2011)Google Scholar
  5. 5.
    Elgammal, A., Lee, C.S.: The Role of Manifold Learning in Human Motion Analysis Human Motion Understanding, Modeling, Capture and Animation, pp. 1–29 (2008)Google Scholar
  6. 6.
    Forbes, K., Fiume, E.: An efficient search algorithm for motion data using weighted PCA. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 67–76 (2005)Google Scholar
  7. 7.
    Forsyth, D.A., Arikan, O., Ikemoto, L., O‘Brien, J., Ramanan, D.: Computational Studies of Human Motion: Part 1, Tracking and Motion Synthesis. Foundations and Trends in Computer Graphics and Vision 1(2/3) (2006)Google Scholar
  8. 8.
    Geng, W., Yu, G.: Reuse of Motion Capture Data in Animation: A Review. In: Kumar, V., Gavrilova, M.L., Tan, C.J.K., L’Ecuyer, P. (eds.) ICCSA 2003. LNCS, vol. 2669, pp. 620–629. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Glardon, P., Boulic, R., Thalmann, D.: PCA-based walking engine using motion capture data. In: Computer Graphics International, pp. 292–298 (2004)Google Scholar
  10. 10.
    Glardon, P., Boulic, R., Thalmann, D.: A Coherent Locomotion Engine Extrapolating Beyond Experimental Data. In: Proceedings of Computer Animation and Social Agent (CASA), Geneva, Switzerland, pp. 73–84 (2004)Google Scholar
  11. 11.
    Grassia, F.S.: Practical parameterization of rotations using the exponential map. Journal of Graphics Tools 3, 29–48 (1998)CrossRefGoogle Scholar
  12. 12.
    Grudzinski, T.: Exploiting Quaternion PCA in Virtual Character Motion Analysis. In: Bolc, L., Kulikowski, J.L., Wojciechowski, K. (eds.) ICCVG 2008. LNCS, vol. 5337, pp. 420–429. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  13. 13.
    Hsu, E., Pulli, K., Popovic, J.: Style Translation for Human Motion. In: SIGGRAPH 2005 Proceedings, pp. 1082–1089 (2005)Google Scholar
  14. 14.
    HTS working group: The HMM-based speech synthesis system (HTS) Version 2.1, http://hts.sp.nitech.ac.jp/ (accessed 2010)
  15. 15.
    IGS-190. Animazoo (2010), http://www.animazoo.com
  16. 16.
    Johnson, M.P.: Exploiting quaternions to support expressive interactive character motion. PhD Thesis (2002)Google Scholar
  17. 17.
    Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer Series in Statistic. Springer, New York (2002)MATHGoogle Scholar
  18. 18.
    Lau, M., Bar-Joseph, Z., Kuffner, J.: Modeling Spatial and Temporal Variation in Motion Data. ACM Transactions on Graphics (SIGGRAPH ASIA ) 28(5), 171 (2009)Google Scholar
  19. 19.
    Li, Y., Wang, T., Shum, H.: Motion texture: a two-level statistical model for character motion synthesis. In: Proc. of SIGGRAPH 2002, New York, NY, USA, pp. 465–472 (2002)Google Scholar
  20. 20.
    Lu, X., Hsu, R.-L., Jain, A.K., Kamgar-Parsi, B., Kamgar-Parsi, B.: Face Recognition with 3D Model-Based Synthesis. In: Zhang, D., Jain, A.K. (eds.) ICBA 2004. LNCS, vol. 3072, pp. 139–146. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  21. 21.
    Menache, A.: Understanding motion Capture for Computer Animation and Video Games. Morgan Kauffman Publishers Inc., San Francisco (1999)Google Scholar
  22. 22.
    Min, J., Chan, Y., Chai, J.: Interactive generation of human animation with deformable motion models. ACM Trans. Graph. 29(1), 9:1–9:12 (2009)CrossRefGoogle Scholar
  23. 23.
    Min, J., Liu, H., Chai, J.: Synthesis and editing of personalized stylistic human motion. In: Proceedings of SI3D, pp. 39–46 (2010)Google Scholar
  24. 24.
    Pejsa, T., Pandzic, I.S.: State of the Art in Example-Based Motion Synthesis for Virtual Characters in Interactive Applications. Computer Graphics Forum 29(1), 202–226 (2010)CrossRefGoogle Scholar
  25. 25.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proc. of IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  26. 26.
    Rose, C., Cohen, M.F., Bodenheimer, B.: Verbs and Adverbs: Multidimensional Motion Interpolation. IEEE Comput. Graph. Appl. 18(5), 32–40 (1998)CrossRefGoogle Scholar
  27. 27.
    Shapiro, A., Cao, Y., Faloutsos, P.: Style components. In: Proceedings of Graphics Interface, Quebec, Canada, pp. 33–39 (2006)Google Scholar
  28. 28.
    Shoemake, K.: Animating Rotations with Quaternion Curves. In: Proc. of SIGGRAPH 2005, San Francisco, vol. 19(3), pp. 245–254 (1985)Google Scholar
  29. 29.
    Tanco, L.M., Hilton, A.: Realistic synthesis of novel human movements from a database of motion capture examples. In: Proc. of the Workshop on Human Motion (HUMO 2000), Washington DC, USA, pp. 137–142 (2000)Google Scholar
  30. 30.
    Taylor, G.W., Hinton, G.E.: Factored conditional restricted Boltzmann Machines for modeling motion style. In: ICML 2009 Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1025–1032 (2009)Google Scholar
  31. 31.
    Tilmanne, J., Dutoit, T.: Expressive Gait Synthesis Using PCA and Gaussian Modeling. In: Boulic, R., Chrysanthou, Y., Komura, T. (eds.) MIG 2010. LNCS, vol. 6459, pp. 363–374. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  32. 32.
    Tilmanne, J., Moinet, A., Dutoit, T.: Stylistic gait synthesis based on hidden Markov models. EURASIP Journal on Advances in Signal Processing, 72 (2012)Google Scholar
  33. 33.
    Tilmanne, J., Sebbe, R., Dutoit, T.: A Database for Stylistic Human Gait Modeling and Synthesis. In: Proceedings of the eNTERFACE 2008 Workshop on Multimodal Interfaces, Paris, France, pp. 91–94 (2008)Google Scholar
  34. 34.
    Toda, T., Tokuda, K.: A Speech Parameter Generation Algorithm Considering Global Variance for HMM-Based Speech Synthesis. IEICE-Transactions on Information and Systems 90(5), 816–824 (2007)CrossRefGoogle Scholar
  35. 35.
    Tokuda, K., Yoshimura, T., Masuko, T., Kobayashi, T., Kitamura, T.: Speech parameter generation algorithms for HMM-based speech synthesis. In: Proc. of ICASSP (June 2000)Google Scholar
  36. 36.
    Troje, N.F.: Retrieving information from human movement patterns. In: Understanding Events: How Humans See, Represent, and Act on Events, pp. 308–334. Oxford University Press (2008)Google Scholar
  37. 37.
    Unuma, M., Anjyo, K., Takeuchi, R.: Fourier principles for emotion-based human figure animation. In: SIGGRAPH 1995 Proceedings, pp. 91–96 (1995)Google Scholar
  38. 38.
    Urtasun, R., Glardon, P., Boulic, R., Thalmann, D., Fua, P.: Style-based Motion Synthesis. Computer Graphics Forum 23(4), 799–812 (2004)CrossRefGoogle Scholar
  39. 39.
    Wang, Y., Liu, Z., Zhou, L.: Automatic 3D Motion Synthesis with Time-Striding Hidden Markov Model. In: Yeung, D.S., Liu, Z.-Q., Wang, X.-Z., Yan, H. (eds.) ICMLC 2005. LNCS (LNAI), vol. 3930, pp. 558–567. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  40. 40.
    Wang, Y., Liu, Z., Zhou, L.: Learning Style-directed Dynamics of Human Motion for Automatic Motion Synthesis. In: IEEE Conference on Systems, Man, and Cybernetics 2006, Taiwan, pp. 4428–4433 (2006)Google Scholar
  41. 41.
    Wang, Y., Xie, L., Liu, Z., Zhou, L.: The SOMN-HMM Model and Its Application to Automatic Synthesis of 3D Character Animation. In: IEEE Conference on Systems, Man, and Cybernetics 2006, Taiwan, pp. 4948–4952 (2006)Google Scholar
  42. 42.
    Wecker, L., Samavati, F., Gavrilova, M.: A multiresolution approach to iris synthesis. Computers & Graphics 34(4), 468–478 (2010)CrossRefGoogle Scholar
  43. 43.
    Yamagishi, J., Nose, T., Zen, H., Ling, Z.H., Toda, T., Tokuda, K., King, S., Renals, S.: Robust speaker-adaptive HMM-based text-to-speech synthesis. IEEE Transactions on Audio, Speech, and Language Processing 17(6), 1208–1230 (2009)CrossRefGoogle Scholar
  44. 44.
    Yamagishi, J., Kobayashi, T.: Average-voice-based speech synthesis using HSMM-based speaker adaptation and adaptive training. IEICE TRANSACTIONS on Information and Systems 90(2), 533–543 (2007)CrossRefGoogle Scholar
  45. 45.
    Yamagishi, J., Kobayashi, T., Nakano, Y., Ogata, K., Isogai, J.: Analysis of speaker adaptation algorithms for HMM-based speech synthesis and a constrained SMAPLR adaptation algorithm. IEEE Transactions on Audio, Speech, and Language Processing 17(1), 66–83 (2009)CrossRefGoogle Scholar
  46. 46.
    Yamazaki, T., Niwase, N., Yamagishi, J., Kobayashi, T.: HumanWalking Motion Synthesis Based on Multiple Regression Hidden Semi-Markov Model. In: 2005 International Conference on Cyberworlds (CW 2005), pp. 445–452 (2005)Google Scholar
  47. 47.
    Yoshimura, T., Tokuda, K., Masuko, T., Kobayashi, T., Kitamura, T.: Duration modeling for HMM-based speech synthesis. In: Fifth International Conference on Spoken Language Processing (ICSLP), pp. 29–32 (1998)Google Scholar
  48. 48.
    Young, S., Evermann, G., Gales, M., Hain, T., Kershaw, D., Liu, X., Moore, G., Odell, J., Ollason, D., Povey, D., Valtchev, V., Woodland, P.: The HTK Book (for HTK Version 3.4) (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Joëlle Tilmanne
    • 1
  • Thierry Dutoit
    • 1
  1. 1.Numediart Institute / TCTS Lab.University of Mons (UMons)MonsBelgium

Personalised recommendations