Multiple Hypothesis Tracking with Sign Language Hand Motion Constraints

  • Mark BorgEmail author
  • Kenneth P. Camilleri
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9257)


In this paper, we propose to incorporate prior knowledge from sign language linguistic models about the motion of the hands within a multiple hypothesis tracking framework. A critical component for automated visual sign language recognition is the tracking of the signer’s hands, especially when faced with frequent and persistent occlusions and complex hand interactions. Hand motion constraints identified by sign language phonological models, such as the hand symmetry condition, are used as part of the data association process. Initial experimental results show the validity of the proposed approach.


Sign langage recognition MHT Tracking 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    von Agris, et al.: Recent developments in visual sign language recognition. Universal Access in the Information Society 6(4), 323–362 (2007)CrossRefGoogle Scholar
  2. 2.
    Sandler, W., Lillo-Martin, D.: Sign Language and Linguistic Universals. Cambridge Univ. Press (2006)Google Scholar
  3. 3.
    Bar-shalom, Y., Daum, F., Huang, J.: The Probabilistic Data Association Filter. IEEE Control Syst. Mag., 82–100 (2009)Google Scholar
  4. 4.
    Blackman, S.: Multiple Hypothesis Tracking For Multiple Target Tracking. IEEE Aerosp. Electron. Syst. Mag. 19(1), 5–18 (2004)CrossRefGoogle Scholar
  5. 5.
    Roshtkhar, M., et al.: Multiple Object Tracking Using Local Motion Patterns. BMVC (2014)Google Scholar
  6. 6.
    Amer, A.: Voting-based simultaneous tracking of multiple video objects. IEEE Trans. Circuits Syst. Video Technol. 15(11), 1448–1462 (2005)CrossRefGoogle Scholar
  7. 7.
    Shafique, K., Shah, M.: A Non-Iterative Greedy Algorithm for Multi-frame Point Correspondence. IEEE Proc. Int. Conf. Comp. Vision, 110–115 (2003)Google Scholar
  8. 8.
    Klinger, A., et al.: A Dynamic Bayes Network for Visual Pedestrian Tracking. ISPRS 40(3), 145–150 (2014)Google Scholar
  9. 9.
    Reid, D.: An algorithm for tracking multiple targets. IEEE Trans. Automat. Contr. 24(6), 843–854 (1979)CrossRefGoogle Scholar
  10. 10.
    Yilmaz, A.: Object Tracking: A Survey. ACM Comput. Surv. 38(4), 1–45 (2006)CrossRefGoogle Scholar
  11. 11.
    Ragland, K., Tharcis, P.: A Survey on Object Detection, Classification and Tracking Methods. IJERT 3(11), 622–628 (2014)Google Scholar
  12. 12.
    Thirde, D., et al.: Robust Real-Time Tracking for Visual Surveillance. EURASIP J. Advances in Signal Proc 2007(1), 1–23 (2007)Google Scholar
  13. 13.
    Ying, L., Xu, C., Guo, W.: Extended MHT algorithm for multiple object tracking. In: Proc. Int. Conf. Internet Multimedia Computing and Service (2012)Google Scholar
  14. 14.
    Lao, Y., Zheng, Y.: Tracking highly correlated targets through statistical multiplexing. Image and Vision Computing 29(12), 803–817 (2011)CrossRefGoogle Scholar
  15. 15.
    Veenman, C., Reinders, M., Backer, E.: Resolving motion correspondence for densely moving points. Trans. PAMI 23(1), 54–72 (2001)CrossRefGoogle Scholar
  16. 16.
    Antunes, D., et al.: A Library for Implementing the Multiple Hypothesis Tracking Algorithm. CoRR 23(1), 54–72 (2011)Google Scholar
  17. 17.
    Cox, I.: Miller, M: On Finding Ranked Assignments With Application to Multi-Target Tracking & Motion Correspondence. Trans. Aerosp. Electron. Syst. (1996)Google Scholar
  18. 18.
    Wimmer, M.: Adaptive skin color classificator. In: Proc. GVIP, pp. 324–327 (2005)Google Scholar
  19. 19.
    Tomasi, C., Shi, J.: Good Features to Track. In: Proc. CVPR, pp. 593–600 (1994)Google Scholar
  20. 20.
    Doermann, D: Tools and Techniques for Video Performances Evaluation (2000)Google Scholar
  21. 21.
    Crasborn, O., et al.: ECHO Data Set for Sign Language of the Netherlands (NGT) (2004)Google Scholar
  22. 22.
    Arras, K., et al.: Efficient People Tracking in Laser Range Data using a Multi-Hypothesis Leg-Tracker with Adaptive Occlusion Probabilities. Int. Conf. on Robotics and Automation (2008)Google Scholar
  23. 23.
    Kasturi, R., et al.: Framework for performance evaluation of face, text, and vehicle detection and tracking in video. IEEE PAMI 31(2), 319–336 (2009)CrossRefGoogle Scholar
  24. 24.
    Amditis, A., et al.: Multiple Hypothesis Tracking Implementation, 199–220 (2012)Google Scholar
  25. 25.
    Wu, S., Hong, L.: Hand tracking in a natural conversational environment by the interacting multiple model and probabilistic data association (IMM-PDA) algorithm. Pattern Recognition 38(11), 199–220 (2005)CrossRefGoogle Scholar
  26. 26.
    Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: Proc. Int. Conf. Image Processing (2002)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Systems and Control Engineering, Faculty of EngineeringUniversity of MaltaMsidaMalta

Personalised recommendations