Clustering and Classification of Time Series Representing Sign Language Words

  • Mariusz Oszust
  • Marian Wysocki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7895)


The paper considers time series with known class labels representing 101 words of Polish sign language (PSL) performed many times in front of a camera. Three clustering algorithms: K–means, K–medoids and Minimum Entropy Clustering (MEC) are compared. Preliminary partitioning of the data set is performed with help of immune based optimisation. Some time series representations and different clustering quality indices are considered. It is shown that clustering is able to reveal existing natural division. Moreover, it gives an opportunity to learn the issues of processing large number of multidimensional data and to identify potential problems which may occur in automatic classification of signed expressions. Results of ten–fold cross–validation tests for nearest neighbour classification are also given.


clustering of time series dynamic time warping cluster validation immune based optimisation sign language recognition computer vision 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Achtert, E., Bernecker, T., Kriegel, H.-P., Schubert, E., Zimek, A.: Elki in time: Elki 0.2 for the performance evaluation of distance measures for time series. In: Mamoulis, N., Seidl, T., Pedersen, T.B., Torp, K., Assent, I. (eds.) SSTD 2009. LNCS, vol. 5644, pp. 436–440. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Ankerst, M., Breunig, M.M., Kriegel, H.-P., Sander, J.: Optics: ordering points to identify the clustering structure. SIGMOD Record 28(2), 49–60 (1999)CrossRefGoogle Scholar
  3. 3.
    Athitsos, V., Neidle, C., Sclaroff, S., Nash, J., Stefan, R., Thangali, A., Wang, H., Yuan, Q.: Large lexicon project: American sign language video corpus and sign language indexing/retrieval algorithms (2010)Google Scholar
  4. 4.
    Awad, G., Han, J., Sutherland, A.: Novel boosting framework for subunit–based sign language recognition. In: Proceedings of the 16th IEEE International Conference on Image Processing, ICIP 2009, pp. 2693–2696. IEEE Press, Piscataway (2009)Google Scholar
  5. 5.
    Chen, Y., Garcia, E.K., Gupta, M.R., Rahimi, A., Cazzanti, L.: Similarity–based Classification: Concepts and Algorithms. Journal of Machine Learning Research, 747–776 (2009)Google Scholar
  6. 6.
    Cooper, H.: Sign Language Recognition: Generalising to More Complex Corpora. Centre For Vision Speech and Signal Processing. PhD thesis, University Of Surrey (2010)Google Scholar
  7. 7.
    Davidson, I., Satyanarayana, A.: Speeding up k-means Clustering by Bootstrap Averaging. In: Proceedings of the IEEE ICDM Workshop on Clustering Large Data Sets, pp. 16–25 (2003)Google Scholar
  8. 8.
    de Castro, L.N., Von Zuben, F.J.: Learning and optimization using the clonal selection principle. IEEE Transactions on Evolutionary Computation 6(3), 239–251 (2002)CrossRefGoogle Scholar
  9. 9.
    Elkan, C.: Using the Triangle Inequality to Accelerate k-Means. In: ICML, pp. 147–153. AAAI Press (2003)Google Scholar
  10. 10.
    Flasinski, M., Myslinski, S.: On the use of graph parsing for recognition of isolated hand postures of Polish sign language. Pattern Recognition 43(6), 2249–2264 (2010)CrossRefGoogle Scholar
  11. 11.
    Frahling, G., Sohler, C.: A fast k–means implementation using coresets. In: Symposium on Computational Geometry, pp. 135–143 (2006)Google Scholar
  12. 12.
    Fu, T.-C.: A review on time series data mining. Engineering Applications of Artificial Intelligence 24(1), 164–181 (2011)CrossRefGoogle Scholar
  13. 13.
    Hein, A., Kirste, T.: Unsupervised detection of motion primitives in very high dimensional sensor data. In: Gottfried, B., Aghajan, H.K. (eds.) Proceedings of the 5th Workshop on Behaviour Monitoring and Interpretation, BMI 2010, Karlsruhe, Germany. CEUR Workshop Proceedings (2010)Google Scholar
  14. 14.
    Hendzel, J.K.: Polish Sign Language Dictionary. Publishing House Pojezierze (1986) (in polish)Google Scholar
  15. 15.
    Keogh, E., Lin, J.: Hot sax: Efficiently finding the most unusual time series subsequence. In: Fifth IEEE International Conference on Data Mining, pp. 226–233 (2005)Google Scholar
  16. 16.
    Kosmidou, V., Petrantonakis, P., Hadjileontiadis, L.J.: Enhanced sign language recognition using weighted intrinsic-mode entropy and signer’s level of deafness. IEEE Transactions on Systems, Man, and Cybernetics, Part B 41(6), 1531–1543 (2011)CrossRefGoogle Scholar
  17. 17.
    Kriegel, H.-P., Kröger, P., Zimek, A.: Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering. ACM The Transactions on Knowledge Discovery from Data 3(1), 1–58 (2009)CrossRefGoogle Scholar
  18. 18.
    Li, H., Zhang, K., Jiang, T.: Minimum entropy clustering and applications to gene expression analysis. In: Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference, CSB 2004, Washington, DC, pp. 142–151 (2004)Google Scholar
  19. 19.
    Liao, W.T.: Clustering of time series data a survey. Pattern Recognition 38(11), 1857–1874 (2005)zbMATHCrossRefGoogle Scholar
  20. 20.
    Lin, D.J., Le, V., Huang, T.S.: Human–computer interaction. Visual Analysis of Humans, pp. 493–510 (2011)Google Scholar
  21. 21.
    Maulik, U., Bandyopadhyay, S.: Performance evaluation of some clustering algorithms and validity indices. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(12), 1650–1654 (2002)CrossRefGoogle Scholar
  22. 22.
    MEC: Minimum entropy clustering Java package, (accessed August 10, 2012)
  23. 23.
    Ong, S.C.W., Ranganath, S.: Automatic sign language analysis: A survey and the future beyond lexical meaning. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(6), 873–891 (2005)CrossRefGoogle Scholar
  24. 24.
    Oszust, M., Wysocki, M.: Modelling and recognition of signed expressions using subunits obtained by data–driven approach. In: Ramsay, A., Agre, G. (eds.) AIMSA 2012. LNCS, vol. 7557, pp. 315–324. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  25. 25.
    Szczepankowski, B.: Sign language at School. WSiP (1988) (in polish)Google Scholar
  26. 26.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press (2008)Google Scholar
  27. 27.
    Tomaszewski, P.: Visual phonology of Polish Sign Language. Publishing House Matrix (2010) (in polish)Google Scholar
  28. 28.
    Trojanowski, K., Wierzchon, S.: Immune-based algorithms for dynamic optimization. Information Sciences 179(10), 1495–1515 (2009)CrossRefGoogle Scholar
  29. 29.
    Tseng, V.S., Chen, C.-H., Huang, P.-C., Hong, T.-P.: Cluster-based genetic segmentation of time series with DWT. Pattern Recognition Letters 30(13), 1190–1197 (2009)CrossRefGoogle Scholar
  30. 30.
    Vogler, C., Metaxas, D.N.: Toward scalability in ASL recognition: Breaking down signs into phonemes. In: Braffort, A., Gibet, S., Teil, D., Gherbi, R., Richardson, J. (eds.) GW 1999. LNCS (LNAI), vol. 1739, pp. 211–224. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  31. 31.
    Wang, Q., Chen, X., Zhang, L.-G., Wang, C., Gao, W.: Viewpoint invariant sign language recognition. Computer Vision and Image Understanding 108(1-2), 87–97 (2007)CrossRefGoogle Scholar
  32. 32.
    Warmuth, M.K., Kuzmin, D.: Randomized online PCA algorithms with regret bounds that are logarithmic in the dimension. Journal of Machine Learning Research 9, 2287–2320 (2008)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Wysocki, M., Kapuscinski, T., Marnik, J., Oszust, M.: Vision–based Hand Gesture Recognition. Rzeszow University of Technology Publishing House (2011) (in polish)Google Scholar
  34. 34.
    Xu, R., Wunsch, D.: Clustering. Wiley-IEEE Press (2009)Google Scholar
  35. 35.
    Zahedi, M., Manashty, A.R.: Robust sign language recognition system using ToF depth cameras. CoRR (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Mariusz Oszust
    • 1
  • Marian Wysocki
    • 1
  1. 1.Department of Computer and Control EngineeringRzeszow University of TechnologyRzeszowPoland

Personalised recommendations