Learning Human Action Sequence Style from Video for Transfer to 3D Game Characters

  • XiaoLong Chen
  • Kaustubha Mendhurwar
  • Sudhir Mudur
  • Thiruvengadam Radhakrishnan
  • Prabir Bhattacharya
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6459)


In this paper, we present an innovative framework for a 3D game character to adopt human action sequence style by learning from videos. The framework is demonstrated for kickboxing, and can be applied to other activities in which individual style includes improvisation of the sequence in which a set of basic actions are performed. A video database of a number of actors performing the basic kickboxing actions is used for feature word vocabulary creation using 3D SIFT descriptors computed for salient points on the silhouette. Next an SVM classifier is trained to recognize actions at frame level. Then an individual actor’s action sequence is gathered automatically from the actor’s kickboxing videos and an HMM structure is trained. The HMM, equipped with the basic repertoire of 3D actions created just once, drives the action level behavior of a 3D game character.


3D SIFT HMM Motion Capture SVM 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hogue, A., Gill, S., Jenkin, M.: Automated Avatar Creation for 3D Games. In: Future Play, Toronto, Canada, pp. 174–180 (2007)Google Scholar
  2. 2.
    Hou, J., Wanga, X., Xua, F., Nguyena, V.D., Wua, L.: Humanoid personalized avatar through multiple natural language processing. World Academy of Science, Engineering and Technology 59, 230–235 (2009)Google Scholar
  3. 3.
    Sucontphunt, T., Deng, Z., Neumann, U.: Crafting personalized facial avatars using editable portrait & photograph example. In: IEEE Virtual Reality Conference, Lafayette, LA, USA, pp. 259–260 (2009)Google Scholar
  4. 4.
    Li, Y., Wang, T., Shum, H.: Motion texture- a two level statistical model for character motion synthesis. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH, San Antonio, Texas, USA, pp. 465–472 (2002)Google Scholar
  5. 5.
    Brand, M., Hertzmann, A.: Style machines. In: Proceedings of the 27th annual conference on Computer Graphics and Interactive Techniques, SIGGRAPH, New Orleans, Louisiana, USA, pp. 183–192 (2000)Google Scholar
  6. 6.
    Hsu, E., Pulli, K., Popović, J.: Style translation for human motion. In: Proceedings of the 32nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH, Los Angeles, USA, pp. 1082–1089 (2005)Google Scholar
  7. 7.
    Yin, K., Pai, D.: Footsee- an interactive animation system. In: Proceedings of the 2003 ACM, SIGGRAPH/Eurographics Symposium on Computer Animation, San Diego, California, USA, pp. 329–338 (2003)Google Scholar
  8. 8.
    Chai, J., Hodgins, J.: Performance animation from low-dimensional control signals. ACM Transaction on Graphics (24), 686–696 (2005)CrossRefGoogle Scholar
  9. 9.
    Oshita, M., Yoshiya, T.: Learning motion rules for autonomous characters from control logs using support vector machine. In: International Conference on Computer Animation and Social Agents, Saint-Malo, France (2010) (to appear)Google Scholar
  10. 10.
    Ofli, F., Erzin, E., Yemez, Y., Tekalp, A., Erdem, C.: Unsupervised dance figure analysis from video for dancing avatar animation. In: IEEE International Conference on Image Processing, San Diego, CA, USA, pp. 1484–1487 (2008)Google Scholar
  11. 11.
    Poppe, R.: A survey on vision-based human action recognition. Image and Vision Computing (28), 976–990 (2010)CrossRefGoogle Scholar
  12. 12.
    Turaga, P., Chellapa, R., Subramhanian, V., Udrea, O.: Machine recognition of human activities- A Survey. IEEE Transactions on Circuits and Systems for Video Technology (18), 1473–1488 (2008)CrossRefGoogle Scholar
  13. 13.
    Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: ECCV Workshop on Statistical Learning in Computer Vision, Prague, Czech Republic, pp. 59–74 (2004)Google Scholar
  14. 14.
    Lowe, D.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal Computer Vision 2(60), 91–110 (2004)CrossRefGoogle Scholar
  15. 15.
    Scovanner, P., Ali, S., Shah, M.: A 3-dimensional sift descriptor and its application to action recognition. In: ACM Multimedia, Augsburg, Germany, pp. 357–360 (2007)Google Scholar
  16. 16.
    Klaser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3D-gradients. In: British Machine Vision Conference, Leeds, UK, pp. 995–1004 (2008)Google Scholar
  17. 17.
    Gillies, M.: Learning finite-state machine controllers from motion capture data. IEEE Transactions on Computational Intelligence and AI in Games 1(1), 63–72 (2009)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Kovar, L., Gleicher, M., Pighin, F.: Motion Graphs. ACM Transactions on Graphics 21(3), 473–482 (2002)CrossRefGoogle Scholar
  19. 19.
    Arikan, O., Forsyth, D.: Interactive motion generation from examples. ACM Transaction on Graphics 21(3), 483–490 (2002)CrossRefzbMATHGoogle Scholar
  20. 20.
    Zordan, V., Majkowska, A., Chiu, B., Fast, M.: Dynamic response for motion capture animation. ACM Transaction on Graphics 24(3), 697–701 (2005)CrossRefGoogle Scholar
  21. 21.
    Niebles, J., Wang, H., Li, F.: Unsupervised learning of human action categories using Spatial-Temporal words. In: British Machine Vision Conference, Edinburgh, UK (2006)Google Scholar
  22. 22.
    Shapewrap Motion Capture System, (retrieved)
  23. 23.
  24. 24.
    Suryavanshi, B.S., Shiri, N., Mudur, S.P.: An Efficient Technique for Mining Usage Profiles Using Relational Fuzzy Subtractive Clustering. In: IEEE WIRI (Web Information Retrieval and Integration), Washington, DC, pp. 23–29 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • XiaoLong Chen
    • 1
  • Kaustubha Mendhurwar
    • 1
  • Sudhir Mudur
    • 1
  • Thiruvengadam Radhakrishnan
    • 1
  • Prabir Bhattacharya
    • 2
  1. 1.Department of Computer Science and Software EngineeringConcordia UniversityMontrealCanada
  2. 2.Department of Computer Science, College of Engineering and Applied SciencesUniversity of CincinnatiCincinnatiUSA

Personalised recommendations