Arm-Hand Behaviours Modelling: From Attention to Imitation

  • Sean R. F. Fanello
  • Ilaria Gori
  • Fiora Pirri
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6454)


We present a new and original method for modelling arm-hand actions, learning and recognition. We use an incremental approach to separate the arm-hand action recognition problem into three levels. The lower level exploits bottom-up attention to select the region of interest, and attention is specifically tuned towards human motion. The middle level serves to classify action primitives exploiting motion features as descriptors. Each of the primitives is modelled by a Mixture of Gaussian, and it is recognised by a complete, real time and robust recognition system. The higher level system combines sequences of primitives using deterministic finite automata. The contribution of the paper is a compositional based model for arm-hand behaviours allowing a robot to learn new actions in a one time shot demonstration of the action execution.


gesture recognition action segmentation human motion analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mitra, S., Acharya, T.: Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C 37(3), 311–324 (2007)CrossRefGoogle Scholar
  2. 2.
    Wildes, R.P., Bergen, J.R.: Qualitative spatiotemporal analysis using an oriented energy representation. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 768–784. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  3. 3.
    Adelson, E.H., Bergen, J.R.: Spatiotemporal energy models for the perception of motion. J. of the Optical Society of America A 2(2), 284–299 (1985)CrossRefGoogle Scholar
  4. 4.
    Braddick, O., OBrien, J., Wattam-Bell, J., Atkinson, J., Turner, R.: Form and motion coherence activate independent, but not dorsal/ventral segregated, networks in the human brain. Current Biology 10, 731–734 (2000)CrossRefGoogle Scholar
  5. 5.
    Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 104(2-3), 90–126 (2006)CrossRefGoogle Scholar
  6. 6.
    Poppe, R.: A survey on vision-based human action recognition. Image and Vision Computing 28, 976–990 (2010)CrossRefGoogle Scholar
  7. 7.
    Aggarwal, J.K., Cai, Q.: Human motion analysis: A review. Computer Vision and Image Understanding 73, 428–440 (1999)CrossRefGoogle Scholar
  8. 8.
    Bobick, A.F.: Movement, activity, and action: the role of knowledge in the perception of motion. Philosophical Transactions of the Royal Society of London 352, 1257–1265 (1997)CrossRefGoogle Scholar
  9. 9.
    Forsyth, D.A., Arikan, O., Ikemoto, L., O’Brien, J.F., Ramanan, D.: Computational studies of human motion: Part 1, tracking and motion synthesis. Foundations and Trends in Computer Graphics and Vision 1(2/3) (2005)Google Scholar
  10. 10.
    Krüger, V., Kragic, D., Geib, C.: The meaning of action a review on action recognition and mapping. Advanced Robotics 21, 1473–1501 (2007)Google Scholar
  11. 11.
    Casile, A., Dayan, E., Caggiano, V., Hendler, T., Flash, T., Giese, M.A.: Neuronal enc. of human kinematic invariants during action obs. Cereb Cortex 20(7), 1647–1655 (2010)CrossRefGoogle Scholar
  12. 12.
    Gabor, D.: Theory of communication. J. IEE 93(26, Part III), 429–460 (1946)Google Scholar
  13. 13.
    Daugman, J.G.: Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America 2(7), 1160–1169 (1985)CrossRefGoogle Scholar
  14. 14.
    Jones, J.P., Palmer, L.A.: An evaluation of the two-dimensional gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology 58, 1233–1258 (1987)Google Scholar
  15. 15.
    Daugman, J.G.: Complete discrete 2-d Gabor tansforms by neural networks for image analysis and compression. IEEE Trans. on ASSP 36(7), 1169–1179 (1988)CrossRefzbMATHGoogle Scholar
  16. 16.
    Wasserman, L.: All of Nonparametric Statistics. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  17. 17.
    Heeger, D.J.: Optical flow using spatiotemporal filters. International Journal of Computer Vision 1(4), 279–302 (1988)CrossRefGoogle Scholar
  18. 18.
    Watson, A.B., Ahumada, A.J.J.: Model of human visual-motion sensing. Journal of the Optical Society of America A: Optics, Image Science, and Vision 2(2), 322–342 (1985)CrossRefGoogle Scholar
  19. 19.
    Horn, B.K.P., Shunk, B.G.: Determining optical flow. Art. Intel. 17, 185–203 (1981)CrossRefGoogle Scholar
  20. 20.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. of DARPA Imaging Understanding Work, pp. 121–130 (1981)Google Scholar
  21. 21.
    Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Real-time optic flow computation with variational methods. In: Petkov, N., Westenberg, M.A. (eds.) CAIP 2003. LNCS, vol. 2756, pp. 222–229. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  22. 22.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  23. 23.
    Luxburg, U.V.: A tutorial on spectral clustering. Statistics and Comp. 14, 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Hopcroft, J., Ullman, J.: Introduction to Automata Theory Languages and Computation. Addison-Wesley, Reading (1979)zbMATHGoogle Scholar
  25. 25.
    Oncina, J., García, P.: Identifying regular languages in polynomial time. World Scientific Publishing, Singapore (1992)Google Scholar
  26. 26.
    Rabin, M.O.: Probabilistic automata. Information and Control 6(3), 230–245 (1963)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Vidal, E., Thollard, F., de la Higuera, C., Casacuberta, F., Carrasco, R.C.: Probabilistic finite-state machines-part i-ii. IEEE Trans. Pattern Anal. Mach. Intell. 27(7), 1013–1039 (2005)CrossRefGoogle Scholar
  28. 28.
    Dupont, P., Denis, F., Esposito, Y.: Links between probabilistic automata and hidden markov models: probability distributions, learning models and induction algorithms. Pattern Recognition 38(9), 1349–1371 (2005)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Sean R. F. Fanello
    • 1
  • Ilaria Gori
    • 1
  • Fiora Pirri
    • 1
  1. 1.Dipartimento di Informatica e SistemisticaSapienza Università di RomaRomaItaly

Personalised recommendations