Skip to main content
SpringerLink
Log in

Motion-Based Representations for Trajectory-Based Hand Gestures: A Brief Overview

  • Conference paper
  • First Online:
Proceedings of International Conference on Data, Electronics and Computing (ICDEC 2022)

Part of the book series: Algorithms for Intelligent Systems ((AIS))

Included in the following conference series:

  • 47 Accesses

Abstract

Action/gesture representation especially modeling of actions/gestures has a special role in the recognition process. Here, in this paper, we would primarily look for motion-based hand gesture representations which are widely used but less talked about topics. Model-based and appearance-based methods are the two primary techniques for hand gesture representation. Apart from these two, motion-based approaches have gained quite impressive performance in various applications. Many researchers generally include motion-based methods in appearance-based methods. But here we want to discuss the motion-based methods separately with special attention representing hand gestures. Most of the representations generally depend on the shape, size, and color of the body/body part. But these may vary depending on many factors, e.g., illumination variation, image resolution, skin color, clothing, etc. But motion estimation should be independent of these factors. Optical flow and motion templates are the two major motion-based representation schemes that can be used directly to describe human gestures/actions. The main benefits of these techniques are basically their simplicity, ease of implementation, competitive performance, and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahad MAR, Tan JK, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23(2):255–281

    Article  Google Scholar 

  2. Akita K (1984) Image sequence analysis of real world human motion. Pattern Recognit 17(1):73–83

    Article  Google Scholar 

  3. Alon J, Athitsos V, Yuan Q, Sclaroff S (2009) A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Trans Pattern Anal Mach Intell 31(9):1685–1699

    Article  Google Scholar 

  4. Arunraj M, Srinivasan A, Juliet AV (2018) Online action recognition from RGB-D cameras based on reduced basis decomposition. J Real-Time Image Process 1–16

    Google Scholar 

  5. Barros P, Magg S, Weber C, Wermter S (2014) A multichannel convolutional neural network for hand posture recognition. In: International conference on artificial neural networks. Springer, Berlin, pp 403–410

    Google Scholar 

  6. Bhuyan MK, Kumar DA, MacDorman KF, Iwahori Y (2014) A novel set of features for continuous hand gesture recognition. J Multimodal User Interfaces 8(4):333–343

    Article  Google Scholar 

  7. Bilen H, Fernando B, Gavves E, Vedaldi A (2017) Action recognition with dynamic image networks. IEEE Trans Pattern Anal Mach Intell

    Google Scholar 

  8. Bilen H, Fernando B, Gavves E, Vedaldi A, Gould S (2016) Dynamic image networks for action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3034–3042

    Google Scholar 

  9. Bobick AF, Davis JW (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267

    Article  Google Scholar 

  10. Brox T, Bruhn A, Papenberg N, Weickert J (2004) High accuracy optical flow estimation based on a theory for warping. In: European conference on computer vision. Springer, Berlin, pp 25–36

    Google Scholar 

  11. Brox T, Malik J (2011) Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell 33(3):500–513

    Article  Google Scholar 

  12. Chakraborty BK, Sarma D, Bhuyan M, MacDorman KF (2017) Review of constraints on vision-based gesture recognition for human-computer interaction. IET Comput Vis 12(1):3–15

    Article  Google Scholar 

  13. Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: European conference on computer vision. Springer, Berlin, pp 428–441

    Google Scholar 

  14. Escalante HJ, Guyon I, Athitsos V, Jangyodsuk P, Wan J (2017) Principal motion components for one-shot gesture recognition. Pattern Anal Appl 20(1):167–182

    Article  MathSciNet  Google Scholar 

  15. Farnebäck G (2003) Two-frame motion estimation based on polynomial expansion. In: Scandinavian conference on image analysis. Springer, Berlin, pp 363–370

    Google Scholar 

  16. Fernando B, Anderson P, Hutter M, Gould S (2016) Discriminative hierarchical rank pooling for activity recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1924–1932

    Google Scholar 

  17. Fernando B, Gavves E, Oramas J, Ghodrati A, Tuytelaars T (2017) Rank pooling for action recognition. IEEE Trans Pattern Anal Mach Intell 39(4):773–787

    Article  Google Scholar 

  18. Goncalves L, Di Bernardo E, Ursella E, Perona P (1995) Monocular tracking of the human arm in 3d

    Google Scholar 

  19. Horn BK, Schunck BG (1981) Determining optical flow. Artif Intell 17(1–3):185–203

    Article  MATH  Google Scholar 

  20. Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8(2):179–187

    Article  MATH  Google Scholar 

  21. Kavyasree V, Sarma D, Gupta P, Bhuyan M (2020) Deep network-based hand gesture recognition using optical flow guided trajectory images. In: 2020 IEEE applied signal processing conference (ASPCON). IEEE, pp 252–256

    Google Scholar 

  22. Keskin C, Kıraç F, Kara YE, Akarun L. Real time hand pose estimation using depth sensors. In: Consumer depth cameras for computer vision. Springer, Berlin, pp 119–137

    Google Scholar 

  23. Kim H et al (2013) Novel and efficient pedestrian detection using bidirectional PCA. Pattern Recognit 46(8):2220–2227

    Article  MathSciNet  Google Scholar 

  24. Lan Z, Lin M, Li X, Hauptmann AG, Raj B (2015) Beyond gaussian pyramid: multi-skip feature stacking for action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 204–212

    Google Scholar 

  25. Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: IEEE conference on computer vision and pattern recognition. CVPR 2008. IEEE, pp 1–8

    Google Scholar 

  26. Lee HK, Kim JH (1999) An HMM-based threshold model approach for gesture recognition. IEEE Trans Pattern Anal Mach Intell 21(10):961–973

    Article  Google Scholar 

  27. Lucas BD, Kanade T et al (1981) An iterative image registration technique with an application to stereo vision

    Google Scholar 

  28. Mahbub U, Imtiaz H, Ahad MAR (2011) An optical flow based approach for action recognition. In: 14th International conference on computer and information technology (ICCIT 2011). IEEE, pp 646–651

    Google Scholar 

  29. Malagón-Borja L, Fuentes O (2009) Object detection using image reconstruction with PCA. Image Vis Comput 27(1–2):2–9

    Article  Google Scholar 

  30. Pavlovic VI, Sharma R, Huang TS (1997) Visual interpretation of hand gestures for human-computer interaction: a review. IEEE Trans Pattern Anal Mach Intell 19(7):677–695

    Article  Google Scholar 

  31. Rehg JM, Kanade T (1995) Model-based tracking of self-occluding articulated objects. In: Fifth international conference on computer vision. Proceedings. IEEE, pp 612–617

    Google Scholar 

  32. Sarma D, Bhuyan M (2021) Methods, databases and recent advancement of vision-based hand gesture recognition for HCI systems: a review. SN Comput Sci 2(6):1–40

    Article  Google Scholar 

  33. Sarma D, Bhuyan M (2022) Hand detection by two-level segmentation with double-tracking and gesture recognition using deep-features. Sens Imaging 23(1):1–29

    Article  Google Scholar 

  34. Sarma D, Bhuyan MK (2018) Hand gesture recognition using deep network through trajectory-to-contour based images. In: 15th IEEE India council international conference (INDICON), pp 1–6

    Google Scholar 

  35. Sarma D, Bhuyan MK (2020) Optical flow guided motion template for hand gesture recognition. In: Proceedings of the 2nd IEEE conference on applied signal processing (ASPCON)

    Google Scholar 

  36. Sarma D, Kavyasree V, Bhuyan M (2020) Two-stream fusion model for dynamic hand gesture recognition using 3D-CNN and 2D-CNN optical flow guided motion template. arXiv:2007.08847

  37. Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from single depth images. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1297–1304

    Google Scholar 

  38. Srivastava N, Mansimov E, Salakhudinov R (2015) Unsupervised learning of video representations using LSTMs. In: International conference on machine learning, pp 843–852

    Google Scholar 

  39. Tsai DM, Chiu WY, Lee MH (2015) Optical flow-motion history image (OF-MHI) for action recognition. Signal Image Video Process 9(8):1897–1906

    Article  Google Scholar 

  40. Wilson AD, Bobick AF (1995) Learning visual behavior for gesture analysis. In: International symposium on computer vision, 1995. Proceedings. IEEE, pp 229–234

    Google Scholar 

  41. Wixson L (2000) Detecting salient motion by accumulating directionally-consistent flow. IEEE Trans Pattern Anal Mach Intell 22(8):774–780

    Article  Google Scholar 

  42. Xu H, Li L, Fang M, Zhang F (2018) Movement human actions recognition based on machine learning. Int J Online Biomed Eng (IJOE) 14(04):193–210

    Article  Google Scholar 

  43. Yacoob Y, Davis LS (1996) Recognizing human facial expressions from long image sequences using optical flow. IEEE Trans Pattern Anal Mach Intell 18(6):636–642

    Article  Google Scholar 

  44. Yamato J, Ohya J, Ishii K (1992) Recognizing human action in time-sequential images using hidden Markov model. In: IEEE computer society conference on computer vision and pattern recognition, 1992. Proceedings CVPR’92. IEEE, pp 379–385

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Electrical Engineering, Indian Institute of Technology (IIT) Guwahati, Guwahati, Assam, 781039, India

    Debajit Sarma & M. K. Bhuyan

  2. Department of Electronics and Communication Engineering, Tezpur University, Tezpur, Assam, 784028, India

    Trishna Barman

  3. Department of Computer Science, Chubu University, Kasugai, 487-8501, Japan

    Yuji Iwahori

Authors
  1. Debajit Sarma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trishna Barman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. K. Bhuyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuji Iwahori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Debajit Sarma .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Jadavpur University, Kolkata, India

    Nibaran Das

  2. Department of Electronics and Communication Engineering, North Eastern Hill University, Shillong, India

    Juwesh Binong

  3. Faculty of Informatics and Management, University of Hradec Kralove, Hradec Králové, Czech Republic

    Ondrej Krejcar

  4. Department of Computer Science and Engineering, Jadavpur University, Kolkata, India

    Debotosh Bhattacharjee

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sarma, D., Barman, T., Bhuyan, M.K., Iwahori, Y. (2023). Motion-Based Representations for Trajectory-Based Hand Gestures: A Brief Overview. In: Das, N., Binong, J., Krejcar, O., Bhattacharjee, D. (eds) Proceedings of International Conference on Data, Electronics and Computing. ICDEC 2022. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-99-1509-5_14

Download citation

Publish with us

Policies and ethics

Search

Navigation