A sigma-lognormal model-based approach to generating large synthetic online handwriting sample databases

  • Ujjwal Bhattacharya
  • Réjean Plamondon
  • Souvik Dutta Chowdhury
  • Pankaj Goyal
  • Swapan K. Parui
Original Paper

Abstract

This article describes a methodology to generate a large database of synthetic samples from a small set of original online handwriting specimens. The overall paradigm is based on the Kinematic Theory of rapid human movements and its sigma-lognormal model. The principal contributions of the present study include (i) development of a strategy for sigma-lognormal model-based generation of synthetic samples from real online handwriting samples of arbitrary scripts captured by arbitrary relevant devices and (ii) verification of the structural similarities, including the naturalness of such synthetic prototypes, through various human perception experiments, computer evaluations and statistical hypothesis testing. A database consisting of a large number of online synthetic handwritten word samples is used to train and evaluate the performance of three existing automatic online handwriting recognition systems. Training based on a combined set of original and synthetic samples improves the recognition accuracies on the test set. A combined training set is useful irrespective of the nature of the feature set used (online, offline or combined). Although the proposed method has primarily been developed and applied to the design of an online handwriting sample database of a popular Indian script, Bangla, it can be applied to the generation of large databases of any arbitrary script for example: English, Chinese and Arabic.

Keywords

Online handwriting Synthetic training samples Sigma-lognormal model 

References

  1. 1.
    Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., Janet, S.: UNIPEN project of online data exchange and recognizer benchmarks. In: Proceedings on 12th ICPR, pp. 29–33 (1994)Google Scholar
  2. 2.
    Forman, G., Cohen, I.: Learning from little: comparison of classifiers given little training. In: Proceedings on 8th European Conference on PKDD, pp. 161–172 (2004)Google Scholar
  3. 3.
    Lim, J.J., Salakhutdinov, R., Torralba, A.: Transfer learning by borrowing examples for multiclass object detection. In: Proceedings on NIPS, pp. 118–126 (2011)Google Scholar
  4. 4.
    Fink, M.: Object classification from a single example utilizing class relevance metrics. In: Proceeding on NIPS, pp. 449–456 (2004)Google Scholar
  5. 5.
    Varga, T., Bunke, H.: Comparing natural and synthetic training data for off-line cursive handwriting recognition. In: Proceeding on IWFHR, pp. 221–225 (2004)Google Scholar
  6. 6.
    Plamondon, R.: A kinematic theory of rapid human movements. Part I: movement representation and generation. Biol. Cybern. 72(4), 295–307 (1995)CrossRefMATHGoogle Scholar
  7. 7.
    Plamondon, R.: A kinematic theory of rapid human movements: Part II: movement time and control. Biol. Cybern. 72(4), 309–320 (1995)CrossRefMATHGoogle Scholar
  8. 8.
    Baird, H.S.: Document image defect models. In: Baird, H.S., et al. (eds.) Structured Document Image Analysis, pp. 546–556. Springer, New York (1992)CrossRefGoogle Scholar
  9. 9.
    Nonnemaker, J.: The safe use of synthetic data in classification. Ph.D. thesis, Lehigh University (2008)Google Scholar
  10. 10.
    Gader, P.D., Khabou, M.A.: Automatic feature generation for handwritten digit recognition. IEEE Trans. Pattern Anal. Mach. Intell. 18(12), 1256–1261 (1996)CrossRefGoogle Scholar
  11. 11.
    Xu, S., Jiang, H., Jin, T., Lau, F.C.M., Pan, Y.: Generation of Chinese calligraphic writings with style imitation. IEEE Intell. Syst. 24(2), 44–53 (2009)CrossRefGoogle Scholar
  12. 12.
    Varga, T., Bunke, H.: Generation of synthetic training data for an HMM-based handwriting recognition system. In: Proceeding on 7th ICDAR, pp. 618–622 (2003)Google Scholar
  13. 13.
    Mori, M., Suzuki, A., Shio, A., Ohtsuka, S.: Generating new samples from handwritten numerals based on point correspondence. In: Proceeding on 7th IWFHR, pp. 281–290 (2000)Google Scholar
  14. 14.
    Cano, J., Perez-Cortes, J., Arlandis, J., Llobet, R.: Training set expansion in handwritten character recognition. In: Proceeding on 9th SSPR, LNCS 2396, pp. 548–556 (2002)Google Scholar
  15. 15.
    Varga, T., Kilchhofer, D., Bunke, H.: Template-based synthetic handwriting generation for the training of recognition systems. In: Proceeding on 12th IGS, pp. 206–211 (2005)Google Scholar
  16. 16.
    Thomas, A., Rusu, A., Govindaraju, V.: Generation and performance evaluation of synthetic handwritten CAPTCHAs. Pattern Recognit. 42, 3365–3373 (2009)CrossRefMATHGoogle Scholar
  17. 17.
    Chowriappa, A., Rodrigues, R.N., Kesavadas, T., Govindaraju, V., Bisantz, A.: Generation of handwriting by active shape modeling and global local approximation (GLA) adaptation. In: Proceeding on ICFHR, pp. 206–211 (2010)Google Scholar
  18. 18.
    Bayoudh, S., Anquetil, E., Miclet, L., Mouchère, H.: Synthetic online handwriting generation by distortions and analogy. In: Proceeding on 13th IGS, pp. 10–13 (2007)Google Scholar
  19. 19.
    Galbally, J., Fierrez, J., Martinez-Diaz, M., Ortega-Garcia, J.: Improving the enrollment in dynamic signature verification with synthetic samples. In: Proceeding on 10th ICDAR, pp. 1295–1299 (2009)Google Scholar
  20. 20.
    Lin, Z., Wan, L.: Style-preserving English handwriting synthesis. Pattern Recognit. 40, 2097–2109 (2007)CrossRefMATHGoogle Scholar
  21. 21.
    Wang, J., Wu, C., Xu, Y.-Q., Shum, H.-Y.: Combining shape and physical models for online cursive handwriting synthesis. Int. J. Doc. Anal. Recognit. 7, 219–227 (2005)CrossRefGoogle Scholar
  22. 22.
    Choi, H., Cho, C.J., Kim, J.H.: Writer dependent online handwriting generation with Bayesian network. In: 9th IWFHR, pp. 130–135 (2004)Google Scholar
  23. 23.
    Chang, W.D., Shin, J.: A statistical handwriting model for style-preserving and variable character synthesis. Int. J. Doc. Anal. Recognit. 15(1), 1–19 (2012)CrossRefGoogle Scholar
  24. 24.
    Choi, H., Kim, J.H.: Probabilistic synthesis of personal-style handwriting. In: IEICE Transactions on Information and Systems, E92-D:4, pp. 653–661 (2009)Google Scholar
  25. 25.
    Lee, D.H., Cho, H.-G.: A new synthesizing method for handwriting Korean scripts. Int. J. Pattern Recognit. Artif. Intell. 12(1), 46–61 (1998)Google Scholar
  26. 26.
    Graves, A.: Generating sequences with recurrent neural networks. arXiv:1308.0850 [cs.NE] (2013)
  27. 27.
    Rabasse, C., Guest, R., Fairhurst, C.: A new method for the synthesis of signature data with natural variability. IEEE Trans. Syst. Man Cybern. B 38(3), 691–699 (2008)CrossRefGoogle Scholar
  28. 28.
    Popel, D.V.: Signature Analysis, Verification and Synthesis in Pervasive Environment, Synthesis and Analysis in Biometrics, pp. 31–63. World Scientific, Singapore (2007)Google Scholar
  29. 29.
    Galbally, J., Plamondon, R., Fierrez, J., Ortega-Garcia, J.: Synthetic on-line signature generation, part I: methodology and algorithms. Pattern Recognit. 45(7), 2610–2621 (2012)CrossRefGoogle Scholar
  30. 30.
    Galbally, J., Fierrez, J., Ortega-Garcia, J., Plamondon, R.: Synthetic on-line signature generation, part II: experimental validation. Pattern Recognit. 45(7), 2622–2632 (2012)CrossRefGoogle Scholar
  31. 31.
    Almaksour, A., Anquetil, E., Plamondon, R., O’Reilly, C.: Synthetic handwritten gesture generation using sigma-lognormal model for evolving handwriting classifier. In: Proceeding on 15th IGS, pp. 98–101 (2011)Google Scholar
  32. 32.
    Grossberg, S., Paine, R.W.: A neural model of corticocerebellar interactions during attentive imitation and predictive learning of sequential handwriting movements. Neural Netw. 13, 999–1046 (2000)CrossRefGoogle Scholar
  33. 33.
    Schomaker, L.R.B.: Simulation and recognition of handwriting movement: a vertical approach to modeling human motor behavior. Ph.D. thesis, Nijmegen University (1991)Google Scholar
  34. 34.
    Plamondon, R., Guefali, W.: The generation of handwriting with delta lognormal synergies. Biol. Cybern. 78, 119–132 (1998)CrossRefMATHGoogle Scholar
  35. 35.
    Gangadhar, G., Joseph, D., Chakravarthy, V.S.: An oscillatory neuromotor model of handwriting generation. Int. J. Doc. Anal. Recognit. 10(2), 69–84 (2007)CrossRefGoogle Scholar
  36. 36.
    Edelman, S., Flash, T.: A model of handwriting. Biol. Cybern. 57, 25–36 (1987)CrossRefGoogle Scholar
  37. 37.
    Wada, Y., Kawato, M.: A theory for cursive handwriting based on the minimization principle. Biol. Cybern. 73(1), 3–13 (1995)CrossRefMATHGoogle Scholar
  38. 38.
    Wada, Y., Kasuga, H., Surnita, K.: An evolutionary approach for the generation of diversiform characters using a handwriting model. In: Proceeding on 16th ICPR, vol. 1, pp. 131–134 (2002)Google Scholar
  39. 39.
    Ltaief, M., Bezine, H., Alimi, A.M.: A neuro-beta-elliptic model for handwriting generation movements. In: Proceeding of 13th ICFHR, pp. 803–808 (2012)Google Scholar
  40. 40.
    Bezine, H., Alimi, A.M., Sherkat, N.: Generation and analysis of handwriting script with the beta-elliptic model. In: Proceeding on 9th ICFHR, pp. 515–520 (2004)Google Scholar
  41. 41.
    Ghosh, D., Shivaprasad, A.P.: An analytical approach for generation of artificial hand-printed character database from given generative models. Pattern Recognit. 32(6), 907–920 (1999)CrossRefGoogle Scholar
  42. 42.
    Woch, A., Plamondon, R., O’Reilly, C.: Kinematic characteristics of successful movement primitives in young and older subjects: a delta-lognormal comparison. Hum. Mov. Sci. 30(1), 1–17 (2011)CrossRefGoogle Scholar
  43. 43.
    Plamondon, R., O’Reilly, C., Ouellet-Plamondon, C.: Strokes against strokes—strokes for strides. Pattern Recognit. 47, 929–944 (2014)CrossRefGoogle Scholar
  44. 44.
    Plamondon, R., Djioua, M.: A multi-level representation paradigm for handwriting stroke generation. Hum. Mov. Sci. 25(4–5), 586–607 (2006)CrossRefGoogle Scholar
  45. 45.
    Djioua, M., Plamondon, R.: An interactive system for the automatic generation of huge handwriting databases from a few specimens. In: Proceeding on 19th ICPR, pp. 1–4 (2008)Google Scholar
  46. 46.
    O’Reilly, C., Plamondon, R.: Development of a sigma-lognormal representation for online signatures. Pattern Recognit. 42(12), 3324–3337 (2009)CrossRefMATHGoogle Scholar
  47. 47.
    Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, Berlin (2005). (2nd Printing, 2008)MATHGoogle Scholar
  48. 48.
    Chowdhury, S.D., Bhattacharya, U., Parui, S.K.: Levenshtein distance metric based holistic handwritten word recognition. In: Proceeding on 4th MOCR. ACM (2013)Google Scholar
  49. 49.
    Mohiuddin, S.k., Bhattacharya, U., Parui, S.K.: Unconstrained Bangla online handwriting recognition based on MLP and SVM. In: Proceeding on J-MOCR-AND. ACM (2011)Google Scholar
  50. 50.
    Bhattacharya, U., Nigam, A., Rawat, Y.S., Parui, S.K.: An analytic scheme for online handwritten Bangla cursive word recognition. In: Proceeding on 11th ICFHR, pp. 320–325 (2008)Google Scholar
  51. 51.
    Jaeger, S., Manke, S., Reichert, J., Waibel, A.: Online handwriting recognition: the NPen++ recognizer. Int. J. Doc. Anal. Recognit. 3(3), 169–180 (2001)CrossRefGoogle Scholar
  52. 52.
    Samanta, O., Bhattacharya, U., Parui, S.K.: Smoothing of HMM parameters for efficient recognition of online handwriting. Pattern Recognit. 47(11), 3614–3629 (2014)CrossRefGoogle Scholar
  53. 53.
  54. 54.
    El Abed, H., Márgner, V., Kherallah, M., Alimi, A.M.: ICDAR 2009 online Arabic handwriting recognition competition. In: Proceeding on 10th ICFHR, pp. 1388–1392 (2009)Google Scholar
  55. 55.
    Jin, L., Gao, Y., Liu, G., Li, Y., Ding, K.: Scut-couch2009—a comprehensive online unconstrained Chinese handwriting database and benchmark evaluation. Int. J. Doc. Anal. Recognit. 14(1), 53–64 (2011)CrossRefGoogle Scholar
  56. 56.

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Ujjwal Bhattacharya
    • 1
  • Réjean Plamondon
    • 2
  • Souvik Dutta Chowdhury
    • 1
  • Pankaj Goyal
    • 3
  • Swapan K. Parui
    • 1
  1. 1.Indian Statistical InstituteKolkataIndia
  2. 2.École Polytechnique de MontréalMontréalCanada
  3. 3.Heritage Institute of TechnologyKolkataIndia

Personalised recommendations