Skip to main content
SpringerLink
Log in

Handwriting perceptual classification and synthesis using discriminate HMMs and progressive iterative approximation

  • IAPR-MedPRAI
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper handles the problem of online handwriting synthesis. Indeed, this work presents a probabilistic model using the embedded hidden Markov models (HMMs) for the classification and modeling of perceptual sequences. At first, we start with a vector of perceptual points as input seeking a class of basic shape probability as output. In fact, these perceptual points are necessary for the drawing and the recovering of each basic shape where each one is designed with an HMM built and trained with its components. Each path through these possibilities of control points represents an observation that serves as input for the following step. Secondly, the already detected sequences of observations which represent a segment formed an initial HMM and the concatenation of multiple ones leads to a global HMM. To classify a global HMM, we should codify it by searching the best path of initial HMM. The best path is obtained by computing the maximum of likelihood of the different basic shapes. In order to synthesize the handwritten trace, and to recover the best control points sequences, we investigated the progressive iterative approximation. The performance of the proposed model was assessed using samples of scripts extracted from IRONOFF and MAYASTROON databases. In fact, these samples served for the generation of the set of control points used for the HMMs training models. In experiments, good quantitative agreement and approximation were found for the generated trajectories and a more reduced representation of the scripts models was designed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Alvaro F, Sanchez J-A, Benedi J-M (2014) Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Pattern Recogn Lett 35:58–67

    Google Scholar 

  2. Aycard O, Mari JF, Washington R (2004) Learning to automatically detect features for mobile robots using second-order hidden Markov models. Int J Adv Robot Syst 1(4):29

    Google Scholar 

  3. Bag S, Bhowmick P, Harit G (2011) Recognition of Bengali handwritten characters using skeletal convexity and dynamic programming. In: International conference on emerging applications of information technology, pp 265–268

  4. Bag S, Bhowmick P, Harit G (2012) Detection of structural concavities in character images—a writer-independent approach. In: First Indo-Japan conference, PerMIn, pp 260–268

  5. Berio D, Akten M, Leymarie F, Grierson M, Plamondon R (2017) Calligraphic stylization learning with a physiologically plausible model of movement and recurrent neural networks. In: 4th International conference on movement computing, MOCO’2017, https://doi.org/10.1145/3077981.3078049

  6. Bezine H, Alimi AM (2013) Development of an Arabic handwriting learning educational system. Int J Softw Eng Appl 4(2):33–49

    Google Scholar 

  7. Bezine H, Alimi AM (2016) Analysis and synthesis of handwriting movements via the enhanced Beta-elliptic model. In: International conference on systems, signals and devices, pp 295–300

  8. Bezine H, Ghanmi W, Alimi MA (2014) A HMM model based on perceptual codes for on-line handwriting generation. In: International conference on advances cognitive technologies and applications, Cognitive’ Italy, pp 126–132

  9. Bezine H, Kefi M, et Alimi AM (2007) On the Beta-elliptic model for the control of the human arm movement. Int J Pattern Recognit Artif Intell 1(21):5–19

    Google Scholar 

  10. Bouaziz S, Magnan A (2007) Contribution of the visual perception and graphic production systems to the copying of complex geometrical drawings: a developmental study. Cognit Dev 22(1):5–15. https://doi.org/10.1016/j.cogdev.2006.10.002

    Article  Google Scholar 

  11. Bullock D, Grossberg S, Mannes C (1993) A neural network model for cursive script production. Biol Cybern 70:15–28

    MATH  Google Scholar 

  12. Chang WD, Shin J (2012) A statistical handwriting model for style-preserving and variable character synthesis. Int J Doc Anal Recognit 15:1–19

    Google Scholar 

  13. Chatzis S, Kosmopoulos D, Papadourakis GA (2016) non stationary hidden Markov model with approximately infinitely-long time-dependencies. Int J Artif Intell Tools 25(5):51–62

    Google Scholar 

  14. Choi H, Kim J (2003) Generation of handwritten characters with Bayesian network based on-line handwriting recognizers. In: Proceedings of ICDAR’03, England, pp 995–999

  15. Choi T, Li M, Fu K, Lin L (2018) Music sequence prediction with mixture Hidden Markov models. arXiv preprint arXiv:1809.00842

  16. Cui Y, Mousas C (2018) Master of puppets: an animation-by-demonstration computer puppetry authoring framework. 3D Res 9(1):1–14

    Google Scholar 

  17. Danna J, Fontaine M, Paz-Villagran V, Gondre C, Thoret E, Aramaki M, Kronland MR, Ystad S, Velay J-L (2015) The effect of real-time auditory feedback on learning new characters. Hum Mov Sci 43:216–228

    Google Scholar 

  18. Dean TA, SinghS S, Jasra A, Peters GW (2014) Parameter estimation for hidden Markov models with intractable likelihoods. J Stat 41(4):970–987

    MathSciNet  MATH  Google Scholar 

  19. Ding S, Bian W, Liao H, Sun T, Xue Y (2017) Combining Gabor filtering and classification dictionaries learning for fingerprint enhancement. IET Biom 6(6):438–447

    Google Scholar 

  20. Ding S, Zhao X, Xu H, Zhu Q, Xue Y (2018) NSCT-PCNN image fusion based on image gradient motivation. IET Comput Vis 12(4):377–383

    Google Scholar 

  21. Flash T, Hogan N (1985) Moving gracefully: quantitative theories of motor coordination. Neuroscience 10(4):170–174

    Google Scholar 

  22. Forney G (1972) The viterbi algorithm. Proc IEEE 61(3):268–278

    MathSciNet  Google Scholar 

  23. Gangadhar G, Chakravarthy VS, Joseph D (2007) An oscillatory neuromotor model of handwriting generation. Int J Doc Anal Recognit (IJDAR) 10(2):69–84. https://doi.org/10.1007/s10032-007-0046

    Article  Google Scholar 

  24. Gilet E (2009) Modélisation bayésienne d’une boucle perception-action: Application à la lecture et à l’écriture. Grenoble, France, Ph.D., University Joseph Fourie

  25. Gilloux M (1994) Hidden Markov models in handwriting recognition. Springer, Berlin, pp 264–288

    Google Scholar 

  26. Graves A (2014) Generating sequences with recurrent neural networks. Neural Evol Comput 14:1–43

    Google Scholar 

  27. Grossberg S, Paine RW (2000) A neural model of corticocerebellar interactions during attentive imitation and predictive learning of sequential handwriting movements. Neural Netw 2:999–1046

    Google Scholar 

  28. Guyon I (1996) Handwriting synthesis from handwritten glyphs. In: Proceedings of IWFHR’96, England, pp 309–312

  29. Hollerbach J (1981) An oscillation theory of handwriting. Biol Cybern 39:139–156

    Google Scholar 

  30. Hu L, Zanibbi R (2011) Segmenting handwritten math symbols using AdaBoost and multi-scale shape context features. In: International conference on document analysis and recognition ICDAR’2011, pp 1180–1184

  31. Hu Z, Xu Y, Huang L, Leung H (2009) A Chinese handwriting system with automatic error detection. Int J Softw Spec Issue Adv Distance Learn Technol 4(2):101–107

    Google Scholar 

  32. Ioannidou ZS, Theodoropoulou MC, Papandreou NC, Willis JH, Hamodrakas SJ (2014) Cutprotfam-pred: detection and classification of putative structural cuticular proteins from sequence alone based on profile hidden Markov models. Insect Biochem Mol Biol 52:51–59

    Google Scholar 

  33. Jawahar CV, Balasubramanian A, Nambo AM (2009) Retrieval of online handwriting by synthesis and matching. Int J Pattern Recogn 42:1445–1457

    MATH  Google Scholar 

  34. Kalveram KT (1998) A neural oscillator model learning given trajectories. Motor Control and Human skill: A multi-disciplinary perspective, 127–140

  35. Kherallah M, Haddad L, Alimi AM (2009) A new approach for online Arabic hand-writing recognition. In: Proceedings of 2nd international conference on Arabic language resources and tools, pp 22–23

  36. Kuo S, Agazzi O (1994) Keywords spotting in poorly printed documents using psuedo 2-D hidden Markov models. IEEE Trans Pattern Anal Mach Intell 16:842–848

    Google Scholar 

  37. Langrock R, Kneib T, Sohn A, DeRuiter SL (2015) Non parametric inference in hidden Markov models using p-splines. Biometrics 71(2):520–528

    MathSciNet  MATH  Google Scholar 

  38. Lee Y-S, Cho S-B (2011). Activity recognition using hierarchical hidden Markov models on a smartphone with 3D accelerometer. In: International conference on hybrid artificial intelligence systems, pp 460–467

  39. Lin Z, Wan L (2007) Style-preserving English handwriting synthesis. Pattern Recog. 40(7):2097–2109

    MathSciNet  MATH  Google Scholar 

  40. Ltaief M, Bezine H, Alimi MA (2012) A neuro-Beta-elliptic model for handwriting generation movements. In: International conference on frontiers in handwriting recognition, ICFHR’2012, Italy, pp 799–803

  41. Ltaief M, Bezine H, Alimi MA (2016) A Spiking neural network model for complex handwriting movements generation. Int J Comput Sci Inf Sec IJCSIS 14(7):319–327

    Google Scholar 

  42. Ltaief M, Njah S, Bezine H, Alimi MA (2012) Genetic algorithms for perceptual codes extraction. Int J Intell Learn Syst Appl 4:256–265

    Google Scholar 

  43. Malaviya A, Peters L, Camposano R (1993) A fuzzy online handwriting recognition system: FOHRES. In: International conference on fuzzy theory and technology, USA, pp 1–15

  44. Mari J, Fohr D, Junqua J (1996) A second-order hmm for high performance word and phoneme-based continuous speech recognition. Int Conf Acoust Speech Signal Process 1:435–438

    Google Scholar 

  45. Mousas C (2017) Full-body locomotion reconstruction of virtual characters using a single inertial measurement unit. Sensors 17:11. https://doi.org/10.3390/s17112589

    Article  Google Scholar 

  46. Mousas C (2018) Performance-driven dance motion control of a virtual partner character. In: International conference on virtual reality and 3D user interfaces, pp 57– 64

  47. Mousas C, Anagnostopoulos C-N (2017) Real-time performance-driven finger motion synthesis. Comput Gr 65:1–11

    Google Scholar 

  48. Njah S, Nouma B, Bezine H, Alimi AM (2012) MAYASTROUN: a multi language handwriting database. In: International conference on frontiers in handwriting recognition ICFHR’2012, Italy, pp 308–312

  49. Plamondon R (1989) A handwriting model based on differential geometry. In: Plamondon R, Suen CY, Simner M (eds) Computer recognition and human production of handwriting. World Scientific Publisher, Singapore, pp 179–192

    Google Scholar 

  50. Plamondon R, Guerfali W (1998) The generation of handwriting with delta-lognormal synergies. Biol Cybern 78:119–132

    MATH  Google Scholar 

  51. Qiao S, Shen D, Wang X, Han N, Zhu W (2015) A self-adaptive parameter selection trajectory prediction approach via hidden Markov models. IEEE Trans Intell Trans Syst 16(1):284–296

    Google Scholar 

  52. Rabiner L (1989) A tutorial on HMM and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Google Scholar 

  53. Ramaiah C, Plamondon R, Govindaraju V (2014) A sigma-lognormal model for handwritten text CAPTCHA generation. In: Proceedings of international conference on pattern recognition, ICPR’2014, pp 250–254

  54. Rémi C, Frelicot C, Courtellemont P (2002) Automatic analysis of the structuring of children’s drawing and writing. Pattern Recogn 35(5):1059–1069

    MATH  Google Scholar 

  55. Rusu A, Govindaraju V (2004) Handwritten CAPTCHA: using the difference in the abilities of humans and machines in reading handwritten words. In: Proceedings of IWFHR’2004, pp 586–591

  56. Schomaker L (1991) Simulation and recognition of handwriting movements: a vertical approach to modeling Human motor behavior. Netherlands, Dissertation, University Nijmegen

  57. Senatore R, Marcelli A (2012) A neural scheme for procedural motor learning of handwriting. In: International conference on frontiers in handwriting recognition. ICFHR’2012, pp 659–666

  58. Shao L, Zhou H (1996) Curve fitting with Bezier cubics. Gr Models Image Process 58:223–228

    Google Scholar 

  59. Shi D, Elliott RJ, Chen T (2016) Event-based state estimation of discrete-state hidden Markov models. Automatica 65:12–26

    MathSciNet  MATH  Google Scholar 

  60. Simonnet D, Anquetil E, Bouillon M (2017) Multi-criteria handwriting quality analysis with online fuzzy models. Pattern Recogn 69:310–324

    Google Scholar 

  61. Sin B-K, Kim J (1998) Network-based approach to Korean handwriting analysis. Int J Pattern Recogn Artif Intell 12(2):233–249

    Google Scholar 

  62. Song C, Qu Z, Blumm N, Barabasi A-L (2010) Limits of predictability in human mobility. Science 327:1018–1021

    MathSciNet  MATH  Google Scholar 

  63. Srihari SN, Cha S-H, Arora H, Lee S (2002) Individuality of handwriting. J Forensic Sci 44(4):856–872

    Google Scholar 

  64. Tamposis IA, Theodoropoulou MC, Tsirigos KD, Bagos PG (2018) Extending hidden Markov models to allow conditioning on previous observations. J Bioinf Comput Biol 1:2. https://doi.org/10.1142/S0219720018500191

    Article  Google Scholar 

  65. Taweechai N, Natasha D (2013) Approximating handwritten curves using progressive-iterative approximation. In: 10th IEEE international conference computer graphic, imaging and visualisation, USA, pp 17–22

  66. Taweechai N, Natasha D (2012) Approximating online handwriting image by Bezier curves. In: 10th IEEE international conference on computer graphic, imaging and visualisation (CGIV), USA, pp 33–37

  67. Thammano A, Rugkunchon S (2006) A neural network model for online handwritten mathematical symbol recognition. In: International conference on intelligent computing, pp 292–298

  68. Uno M, Suzuki R, Kawato M (1989) Minimum muscle-tension change model which reproduces human arm movement. In: 4th symposium on biological and physiological engineering, pp 299–305

  69. Viard-Gaudin C, Lallican PM, Knerr S (1999) The ireste on/off (IRONOFF) dual handwriting database. In: International conference on document analysis and recognition. https://doi.org/10.1109/icdar.1999.791823

  70. Wada Y, Kawato M (2004) A via-point time optimization algorithm for complex sequential trajectory formation. Neural Netw 17:353–364

    MATH  Google Scholar 

  71. Wada Y, Ohkawa K, Sumita K (2001). Generation of diversity form characters using a computational handwriting model and a genetic algorithm. In: ICANN’01: LNCS, Springer, Hedelberg. vol 2130, pp 1217–1224

  72. Wang J, Wu C, Xu Q-Y, Shum Y-H (2004) Combining shape and physical models for online cursive handwriting synthesis. Int J Doc Anal Recogn 7(4):1433–2833

    Google Scholar 

  73. Wang J, Wu C, Xu Y, Shum H, Ji L (2002) Learning-based cursive handwriting synthesis. In: Proceedings of IWFHR’2002, pp 157–162

  74. Wheeler TJ, Clements J, Finn RD (2014) Skylign: a tool for creating informative, interactive logos representing sequence alignments and profile hidden Markov models. BMC Bioinf 15:1. https://doi.org/10.1186/1471-2105-15-7

    Article  Google Scholar 

  75. Yanhong L, David LO, Zheng Q (2007) Similarity measures between intuitionistic fuzzy (vague) sets: a comparative analysis. Pattern Recogn Lett 28:278–285

    Google Scholar 

  76. Zeng K, Ding S, Jia W (2019) Single image super-resolution using a polymorphic parallel CNN. Appl Intell 49(1):292–300. https://doi.org/10.1007/s10489-018-1270-7

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUB 01/UR/11/02 program.

Author information

Authors and Affiliations

  1. REGIM-Lab: Research Group on Intelligent Machines Laboratory, National School of Engineers, University of Sfax, BP 1173, 3038, Sfax, Tunisia

    Hala Bezine & Adel M. Alimi

Authors
  1. Hala Bezine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adel M. Alimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hala Bezine.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bezine, H., Alimi, A.M. Handwriting perceptual classification and synthesis using discriminate HMMs and progressive iterative approximation. Neural Comput & Applic 32, 16549–16570 (2020). https://doi.org/10.1007/s00521-019-04206-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04206-9

Keywords

Search

Navigation