Statistical Relational Learning for Handwriting Recognition

  • Arti ShivramEmail author
  • Tushar Khot
  • Sriraam Natarajan
  • Venu Govindaraju
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9046)


We introduce a novel application of handwriting recognition for Statistical Relational Learning. The proposed framework captures the intrinsic structure of handwriting by modeling fundamental character shape representations and their relationships using first-order logic. Our framework consists of three stages, (1) character extraction (2) feature generation and (3) class label prediction. In the character extraction stage, handwriting trajectory data is decoded into characters. Following this, character features (predicates) are defined across multiple levels - global, local and aggregated. Finally, a relational One-vs-All classifier is learned using relational functional gradient boosting (RFGB). We evaluate our approach on two datasets and demonstrate comparable accuracy to a well-established, meticulously engineered approach in the handwriting recognition paradigm.


  1. 1.
    Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning. MIT Press, Cambridge (2007)zbMATHGoogle Scholar
  2. 2.
    Riedel, S., Chun, H.W., Takagi, T., Tsujii, J.: A Markov logic approach to bio-molecular event extraction. In: Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task, Association for Computational Linguistics, pp. 41–49 (2009)Google Scholar
  3. 3.
    Poon, H., Vanderwende, L.: Joint inference for knowledge extraction from biomedical literature. In: ACL, Association for Computational Linguistics, pp. 813–821 (2010)Google Scholar
  4. 4.
    Yoshikawa, K., Riedel, S., Asahara, M., Matsumoto, Y.: Jointly identifying temporal relations with Markov logic. In: ACL, pp. 405–413 (2009)Google Scholar
  5. 5.
    Poon, H., Domingos, P.: Unsupervised semantic parsing. In: EMNLP, Association for Computational Linguistics, pp. 1–10 (2009)Google Scholar
  6. 6.
    Davis, J., Burnside, E.S., de Castro Dutra, I., Page, D., Ramakrishnan, R., Costa, V.S., Shavlik, J.W.: View learning for statistical relational learning: with an application to mammography. In: IJCAI, DTIC Document, pp. 677–683 (2005)Google Scholar
  7. 7.
    Davis, J., Ong, I.M., Struyf, J., Burnside, E.S., Page, D., Costa, V.S.: Change of representation for statistical relational learning. In: IJCAI, pp. 2719–2726 (2007)Google Scholar
  8. 8.
    Davis, J., Burnside, E., de Castro Dutra, I., Page, D.L., Santos Costa, V.: An integrated approach to learning Bayesian networks of rules. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 84–95. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  9. 9.
    Natarajan, S., Khot, T., Kersting, K., Gutmann, B., Shavlik, J.: Gradient-based boosting for statistical relational learning: the relational dependency network case. Mach. Learn. 86(1), 25–56 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Weiss, J., Natarajan, S., Peissig, P., McCarty, C., Page, D.: Statistical relational learning to predict primary myocardial infarction from electronic health records. In: AI Magazine (2012)Google Scholar
  11. 11.
    Natarajan, S., Kersting, K., Ip, E., Jacobs, D., Carr, J.: Early prediction of coronary artery calcification levels using machine learning. In: Innovative Applications in AI (2013)Google Scholar
  12. 12.
    Natarajan, S., Saha, B., Joshi, S., Edwards, A., Khot, T., Davenport, E.M., Kersting, K., Whitlow, C.T., Maldjian, J.A.: Relational learning helps in three-way classification of Alzheimer patients from structural magnetic resonance images of the brain. Int. J. Mach. Learn. Cybern. 5(5), 659–669 (2013)CrossRefGoogle Scholar
  13. 13.
    Antanas, L., van Otterlo, M., Mogrovejo, O., Antonio, J., Tuytelaars, T., De Raedt, L.: A relational distance-based framework for hierarchical image understanding. In: Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods, vol. 2, pp. 206–218 (2012)Google Scholar
  14. 14.
    Antanas, L., Hoffmann, M., Frasconi, P., Tuytelaars, T., De Raedt, L.: A relational kernel-based approach to scene classification. In: 2013 IEEE Workshop on Applications of Computer Vision (WACV), pp. 133–139, IEEE (2013)Google Scholar
  15. 15.
    Riedel, S., McClosky, D., Surdeanu, M., McCallum, A., Manning, C.D.: Model combination for event extraction in bionlp 2011. In: Proceedings of the BioNLP Shared Task 2011 Workshop, Association for Computational Linguistics, pp. 51–55 (2011)Google Scholar
  16. 16.
    Dietterich, T.G.: Machine learning for sequential data: a review. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 15–30. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  17. 17.
    Graves, A., Liwicki, M., Bunke, H., Schmidhuber, J., Fernández, S.: Unconstrained on-line handwriting recognition with recurrent neural networks. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems 20, pp. 577–584. MIT Press, Cambridge (2007)Google Scholar
  18. 18.
    Liwicki, M., Bunke, H., et al.: HMM-based on-line recognition of handwritten whiteboard notes. In: Tenth International Workshop on Frontiers in Handwriting Recognition (2006)Google Scholar
  19. 19.
    Shivram, A., Zhu, B., Setlur, S., Nakagawa, M., Govindaraju, V.: Segmentation based online word recognition: a conditional random field driven beam search strategy. In: 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 852–856, IEEE (2013)Google Scholar
  20. 20.
    Guberman, S.A., Lossev, I., Pashintsev, A.V.: Method and apparatus for recognizing cursive writing from sequential input information, 17 May 1994. US Patent 5,313,527Google Scholar
  21. 21.
    Li, X., Yeung, D.Y.: On-line handwritten alphanumeric character recognition using dominant points in strokes. Pattern Recogn. 30(1), 31–44 (1997)CrossRefGoogle Scholar
  22. 22.
    Plamondon, R., Maarse, F.J.: An evaluation of motor models of handwriting. IEEE Trans. Syst. Man Cybern. 19(5), 1060–1072 (1989)CrossRefGoogle Scholar
  23. 23.
    Parizeau, M., Plamondon, R.: A fuzzy-syntactic approach to allograph modeling for cursive script recognition. IEEE Trans. Pattern Anal. Mach. Intell. 17(7), 702–712 (1995)CrossRefGoogle Scholar
  24. 24.
    Malaviya, A., Peters, L.: Fuzzy handwriting description language: Fohdel. Pattern Recogn. 33(1), 119–131 (2000)CrossRefGoogle Scholar
  25. 25.
    Ziino, D., Amin, A., Sammut, C.: Recognition of hand printed Latin characters using machine learning. In: Proceedings of the Third International Conference on Document Analysis and Recognition 1995, vol. 2, pp. 1098–1102, IEEE (1995)Google Scholar
  26. 26.
    Amin, A., Sammut, C., Sum, K.: Learning to recognize hand-printed Chinese characters using inductive logic programming. Int. J. Pattern Recogn. Artif. Intell. 10(07), 829–847 (1996)CrossRefGoogle Scholar
  27. 27.
    Manke, S., Finke, M., Waibel, A.: NPen++: a writer independent, large vocabulary on-line cursive handwriting recognition system. In: Proceedings of the Third International Conference on Document Analysis and Recognition 1995, vol. 1, pp. 403–408, August 1995Google Scholar
  28. 28.
    Neville, J., Jensen, D.: Relational dependency networks. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning. MIT Press, Cambridge (2007)Google Scholar
  29. 29.
    Domingos, P., Lowd, D.: Markov logic: an interface layer for artificial intelligence. Synth. Lect. Artif. Intell. Mach. Learn. 3(1), 1–155 (2009)CrossRefzbMATHGoogle Scholar
  30. 30.
    Kersting, K., De Raedt, L.: Bayesian logic programming: theory and tool. In: Getoor, L., Taskar, B. (eds.) Statistical Relational Learning, p. 291. MIT Press, Cambridge (2007)Google Scholar
  31. 31.
    Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: IJCAI, vol. 99, pp. 1300–1309 (1999)Google Scholar
  32. 32.
    Singla, P., Domingos, P.: Discriminative training of Markov logic networks. In: AAAI, vol. 5, pp. 868–873 (2005)Google Scholar
  33. 33.
    Lowd, D., Domingos, P.: Recursive random fields. In: IJCAI, pp. 950–955 (2007)Google Scholar
  34. 34.
    Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Connell, S.D.: Online handwriting recognition using multiple pattern class models. Ph.D. thesis (2000)Google Scholar
  36. 36.
    Malaviya, A., Peters, L.: Fuzzy feature description of handwriting patterns. Pattern Recogn. 30(10), 1591–1604 (1997)CrossRefzbMATHGoogle Scholar
  37. 37.
    Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn. 44(8), 1761–1776 (2011)CrossRefGoogle Scholar
  38. 38.
    Shivram, A., Ramaiah, C., Setlur, S., Govindaraju, V.: IBM_UB_1: a dual mode unconstrained English handwriting dataset. In: Proceedings of the Twelfth International Conference on Document Analysis and Recognition 2013, pp. 13–17, (2013)Google Scholar
  39. 39.
    Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., Janet, S.: Unipen project of on-line data exchange and recognizer benchmarks. In: Proceedings of the 12th IAPR International. Conference on Pattern Recognition 1994. Vol. 2-Conference B: Computer Vision &; Image Processing, vol. 2, pp. 29–33, IEEE (1994)Google Scholar
  40. 40.
    Liu, C.L., Sako, H., Fujisawa, H.: Performance evaluation of pattern classifiers for handwritten character recognition. Int. J. Doc. Anal. Recogn. 4(3), 191–204 (2002)CrossRefGoogle Scholar
  41. 41.
    Zhu, B., Shivram, A., Setlur, S., Govindaraju, V., Nakagawa, M.: Online handwritten cursive word recognition using segmentation-free MRF in combination with P2DBMN-MQDF. In: 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 349–353, IEEE (2013)Google Scholar
  42. 42.
    Cao, J., Shridhar, M., Kimura, F., Ahmadi, M.: Statistical and neural classification of handwritten numerals: a comparative study. In: 11th IAPR International Conference on Pattern Recognition 1992, vol. II. Conference B: Pattern Recognition Methodology and Systems, Proceedings, pp. 643–646, IEEE (1992)Google Scholar
  43. 43.

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Arti Shivram
    • 1
    Email author
  • Tushar Khot
    • 2
  • Sriraam Natarajan
    • 3
  • Venu Govindaraju
    • 1
  1. 1.University at Buffalo - SUNYBuffaloUSA
  2. 2.University of Wisconsin-MadisonMadisonUSA
  3. 3.Indiana UniversityBloomingtonUSA

Personalised recommendations