Rotation invariant hand-drawn symbol recognition based on a dynamic time warping model

  • Alicia FornésEmail author
  • Josep Lladós
  • Gemma Sánchez
  • Dimosthenis Karatzas
Full Paper


One of the major difficulties of handwriting symbol recognition is the high variability among symbols because of the different writer styles. In this paper, we introduce a robust approach for describing and recognizing hand-drawn symbols tolerant to these writer style differences. This method, which is invariant to scale and rotation, is based on the dynamic time warping (DTW) algorithm. The symbols are described by vector sequences, a variation of the DTW distance is used for computing the matching distance, and K-Nearest Neighbor is used to classify them. Our approach has been evaluated in two benchmarking scenarios consisting of hand-drawn symbols. Compared with state-of-the-art methods for symbol recognition, our method shows higher tolerance to the irregular deformations induced by hand-drawn strokes.


Document analysis Graphics recognition Symbol recognition Handwriting recognition Sequence alignment 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bartolini I., Ciaccia P., Patella M.: Warp: accurate retrieval of shapes using phase of fourier descriptors and time warping distance. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 142–147 (2005)CrossRefGoogle Scholar
  2. 2.
    Belongie S., Malik J., Puzicha J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)CrossRefGoogle Scholar
  3. 3.
    Bunke H.: Attributed programmed graph grammars and their application to schematic diagram interpretation. IEEE Trans. Pattern Anal. Mach. Intell. 4(6), 574–582 (1982)zbMATHCrossRefGoogle Scholar
  4. 4.
    Chan K.F., Yeung D.Y.: Mathematical expression recognition: a survey. Int. J. Document Anal. Recognit. 3(1), 3–15 (2000)CrossRefGoogle Scholar
  5. 5.
    Chou K.S., Fan K.C., Fan T.I.: Peripheral and global features for use in coarse classification of Chinese characters. Pattern Recognit. 30(3), 483–489 (1997)CrossRefGoogle Scholar
  6. 6.
    Fonseca M.J., Pimentel C., Jorge J.A.: CALI: an online scribble recognizer for calligraphic interfaces. In: AAAI Spring Symposium on Sketch Understanding, pp. 51–58. (2002)Google Scholar
  7. 7.
    Fornés, A., Lladós, J., Sánchez, G.: Primitive segmentation in old handwritten music scores. In: Liu, W., Lladós, J. (eds.) Graphics Recognition: Ten Years Review and Future Perspectives, vol. 3926 of Lecture Notes in Computer Science, pp. 279–290. Springer (2006)Google Scholar
  8. 8.
    Garain U., Chaudhuri B.B.: Recognition of online handwritten mathematical expressions. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 34(6), 2366–2376 (2004)CrossRefGoogle Scholar
  9. 9.
    Golubitsky, O., Watt, S.M.: Online recognition of multi-stroke symbols with orthogonal series. International Conference on Document Analysis and Recognition, vol. 2, pp. 1265–1269 (2009)Google Scholar
  10. 10.
    Hse, H., Newton, A.R.: Sketched symbol recognition using Zernike moments. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 1, pp. 367–370 (2004)Google Scholar
  11. 11.
    Hu, N., Dannenberg, R.B., Tzanetakis, G.: Polyphonic audio matching and alignment for music retrieval. In: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 185–188. New Paltz, New York, October (2003)Google Scholar
  12. 12.
    Keogh E., Ratanamahatana C.A.: Exact indexing of dynamic time warping. Knowl. Inf. Syst. 7(3), 358–386 (2005)CrossRefGoogle Scholar
  13. 13.
    Khotanzad A., Hong Y.H.: Invariant image recognition by Zernike moments. IEEE Trans. Pattern Anal. Mach. Intell. 12(5), 489–497 (1990)CrossRefGoogle Scholar
  14. 14.
    Kim, W.Y., Kim, Y.S.: A new region-based shape descriptor. Technical report, Hanyang University and Konan Technology (1999)Google Scholar
  15. 15.
    Kornfield, E.M., Manmatha, R., Allan, J.: Text alignment with handwritten documents. In Document Image Analysis for Libraries, pp. 195–209. IEEE Computer Society, Washington, DC, USA, (2004)Google Scholar
  16. 16.
    Kruskal J.B., Liberman M.: The symmetric time-warping problem: from continuous to discrete. In: Sankoff, D., Kruskal, J.B. (eds) Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison., pp. 125–161. Addison-Wesley Publishing Co, Reading, MA (1983)Google Scholar
  17. 17.
    Khurshid, K., Faure, C., Vincent, N.: Fusion of word spotting and spatial information for figure caption retrieval in historical document images. 10th International Conference on Document Analysis and Recognition, vol. 1, pp. 266–270 (2009)Google Scholar
  18. 18.
    Levin, E., Pieraccini, R.: Dynamic planar warping for optical character recognition. In IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 149–152 (1992)Google Scholar
  19. 19.
    Liang, S., Sun, Z., Li, B.: Sketch retrieval based on spatial relations. Proceedings of International Conference on Computer Graphics, Imaging and Visualization, Beijing, China, pp. 24–29 (2005)Google Scholar
  20. 20.
    Liu, C.L.: Normalization-cooperated gradient feature extraction for handwritten character recognition. IEEE Trans. Pattern Anal. Mach. Intell., pp. 1465–1469 (2007)Google Scholar
  21. 21.
    Lladós, J., Valveny, E., Sánchez, G., Martí, E.: Symbol recognition: current advances and perspectives. In Lecture Notes in Computer Science, vol. 2390, pp. 104–128. Springer (2002)Google Scholar
  22. 22.
    Lladós, J., Martí, E., Villanueva, J.J.: Symbol recognition by error-tolerant subgraph matching between region adjacency graphs. In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23(10), pp. 1137–1143, October (2001)Google Scholar
  23. 23.
    Marti U.V., Bunke H.: Using a statistical language model to improve the performance of an hmm-based cursive handwriting recognition system. Int. J. Pattern Recogni. Artif. Intell. 15, 65–90 (2001)CrossRefGoogle Scholar
  24. 24.
    Mas, J., Jorge, J.A., Sánchez, G., Lladós, J.: Representing and parsing sketched symbols using adjacency grammars and a grid-directed parser. In: Ogier eds, J.M., Liu, W., Lladós, J. (eds.), Graphics Recognition: Recent Advances and New Opportunities, Lecture Notes in Computer Science, vol. 5046, pp. 176–187. Springer-Verlag (2008)Google Scholar
  25. 25.
    Miyao H., Maruyama M.: An online handwritten music symbol recognition system. Int. J. Document Anal. Recogni. 9(1), 49–58 (2007)CrossRefGoogle Scholar
  26. 26.
    Mokhtarian F., Mackworth A.K.: Scale-based description and recognition of planar curves and two-dimensional shapes. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 34–43 (1986)CrossRefGoogle Scholar
  27. 27.
    Müller S., Rigoll G.: Engineering Drawing Database Retrieval Using Statistical Pattern Spotting Techniques. Graphics Recogni. Recent Adv. Lecture Notes Comput. Sci. 1941, 246–255 (2000)Google Scholar
  28. 28.
    Niels, R., Vuurpijl, L.: Using dynamic time warping for intuitive handwriting recognition. Advances in Graphonomics, Proceedings of the 12th Conference of the International Graphonomics Society, pp. 217–221 (2005)Google Scholar
  29. 29.
    Orio, N., Schwarz, D.: Alignment of monophonic and polyphonic music to a score. In San Francisco International Computer Music Association, editor, Proceedings of the International Computer Music Conference, pp. 155–158, Havana, Cuba, September (2001)Google Scholar
  30. 30.
    Parker, J., Pivovarov, J., Royko, D.: Vector templates for symbol recognition. In: International Conference on Pattern Recognition, vol. 15, pp. 602–605 (2000)Google Scholar
  31. 31.
    Peng, B., Liu, Y., Wenyin, L., Huang, G.: Sketch recognition based on topological spatial relationship. In: Structural, Syntactic, and Statistical Pattern Recognition: Lecture Notes in Computer Science, vol. 3138, pp. 434–443. Springer-Verlag (2004)Google Scholar
  32. 32.
    Rabiner L.R.: A tutorial on hidden Markov models and selected applications inspeech recognition. Proc. IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  33. 33.
    Rath, T.M. Manmatha, R.: Word image matching using dynamic time warping. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 521–527 Madison, WI, June 18–20 (2003)Google Scholar
  34. 34.
    Rusiñol M., Borràs A., Lladós J.: Relational indexing of vectorial primitives for symbol spotting in line-drawing images. Pattern Recogni. Lett. 31(3), 188–201 (2010)CrossRefGoogle Scholar
  35. 35.
    Sanchez, G., Valveny, E., Llados, J., Romeu, J. Mas, Lozano. N.: A platform to extract knowledge from graphic documents. Application to an architectural sketch understanding scenario. In: Dengel, A. Marinai, S., (eds.), Document Analysis Systems VI, Lecture Notes in Computer Science, vol. 3163, pp. 389–400, Florence—Italy, 2004, Springer-VerlagGoogle Scholar
  36. 36.
    Shi, Y., Li, H.Y., Soong, F.K.: A unified framework for symbol segmentation and recognition of handwritten mathematical expressions. Ninth International Conference on Document Analysis and Recognition, vol. 2, 854–858, Sept. (2007)Google Scholar
  37. 37.
    Tabbone, S., Wendling, L.: Technical symbols recognition using the two-dimensional radon transform. In Proceedings of the 16th International Conference on Pattern Recognition, vol. 3, pp. 200–203 (2002)Google Scholar
  38. 38.
    Uchida, S., Sakoe, H.: A monotonic and continuous two-dimensional warping based on dynamic programming. In Proceedings of 14th International Conference on Pattern Recognition, vol. 1, pp. 521–524, (1998)Google Scholar
  39. 39.
    Uchida S., Sakoe H.: A survey of elastic matching techniques for handwritten character recognition. IEICE Trans. Inf. Syst. E88-D, 1781–1790 (2005)CrossRefGoogle Scholar
  40. 40.
    Valveny, E., Marti, E.: Hand-drawn symbol recognition in graphic documents using deformable template matching and a bayesian framework. Proceedings of the 15th International Conference on Pattern Recognition, vol. 2, pp. 239–242 (2000)Google Scholar
  41. 41.
    Vinciarelli, A., Bengio, S., Bunke, H.: Offline recognition of unconstrained handwritten texts using HMMs and statistical language models. IEEE Trans. Pattern Anal. Mach. Intell. pp. 709–720 (2004)Google Scholar
  42. 42.
    Vuori V., Laaksonen J., Oja E., Kangas J.: Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters. Int. J. Document Anal. Recogni. 3(3), 150–159 (2001)CrossRefGoogle Scholar
  43. 43.
    Wilfong G., Sinden F., Ruedisueli L.: On-line recognition of handwritten symbols. IEEE Trans. Pattern Anal. Mach. Intell. 18(9), 935–940 (1996)CrossRefGoogle Scholar
  44. 44.
    Xin, G., Cuiyun, L., Jihong, P., Weixin, X.: HMM based online hand-drawn graphic symbol recognition. In Proceedings of the 6th International Conference on Signal Processing, vol. 2, pp. 1067–1070 (2002)Google Scholar
  45. 45.
    Yang S.: Symbol recognition via statistical integration of pixel-level constraint histograms: a new descriptor. IEEE Trans. Pattern Anal. Mach. Intell. 27(2), 278–281 (2005)CrossRefGoogle Scholar
  46. 46.
    Zhang, D.S., Lu, G.: Generic Fourier descriptor for shape-based image retrieval. In Proceedings of the IEEE International Conference on Multimedia and Expo, vol. 1, pp. 425–428 (2002)Google Scholar
  47. 47.
    Zhang, W., Liu, W.: A new syntactic approach to graphic symbol recognition. International Conference on Document Analysis and Recognition vol. 1, pp. 516–520 (2007)Google Scholar
  48. 48.
    Zhang W., Liu W., Zhang K.: Symbol recognition with kernel density matching. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2020–2024 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Alicia Fornés
    • 1
    Email author
  • Josep Lladós
    • 1
  • Gemma Sánchez
    • 1
  • Dimosthenis Karatzas
    • 1
  1. 1.Computer Vision Center, Department of Computer ScienceUniversitat Autònoma de BarcelonaBellaterraSpain

Personalised recommendations