Perturbation Models for Generating Synthetic Training Data in Handwriting Recognition

  • Tamás Varga
  • Horst Bunke
Part of the Studies in Computational Intelligence book series (SCI, volume 90)

In this chapter, the use of synthetic training data for handwriting recognition is studied. After an overview of the previous works related to the field, the authors' main results regarding this research area are presented and discussed, including a perturbation model for the generation of synthetic text lines from existing cursively handwritten lines of text produced by human writers. The goal of synthetic text line generation is to improve the performance of an off-line cursive handwriting recognition system by providing it with additional training data. It can be expected that by adding synthetic training data the variability of the training set improves, which leads to a higher recognition rate. On the other hand, synthetic training data may bias a recognizer towards unnatural handwriting styles, which could lead to a deterioration of the recognition rate. The proposed perturbation model is evaluated under several experimental conditions, and it is shown that significant improvement of the recognition performance is possible even when the original training set is large and the text lines are provided by a large number of different writers.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bunke, H.: Recognition of Cursive Roman Handwriting - Past, Present and Future. In: Proc. 7th Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland (2003) 448-459CrossRefGoogle Scholar
  2. 2.
    Plamondon, R., Srihari, S.: On-line and Off-line Handwriting Recognition: A Comprehensive Survey. IEEE Trans. on Pattern Analysis and Machine Intelli-gence 22(1) (2000) 63-84CrossRefGoogle Scholar
  3. 3.
    Srihari, S.: Handwritten Address Interpretation: a Task of Many Pattern Recog- nition Problems. Int. Journal of Pattern Recognition and Artificial Intelligence 14(5)(2000) 663-674CrossRefGoogle Scholar
  4. 4.
    . Impedovo, S., Wang, P., Bunke, H., eds.: Automatic Bankcheck Processing. World Scientific (1997)Google Scholar
  5. 5.
    Gopisetty, S., Lorie, R., Mao, J., Mohiuddin, M., Sorin, A., Yair, E.: Auto-mated Forms-processing Software and Services. IBM Journal of Research and Development 40(2) (1996) 211-230CrossRefGoogle Scholar
  6. 6.
    Ye, X., Cheriet, M., Suen, C.: A Generic Method of Cleaning and Enhancing Handwritten Data from Business Forms. Int. Journal on Document Analysis and Recognition 4(2) (2001) 84-96CrossRefGoogle Scholar
  7. 7.
    Arica, N., Yarman-Vural, F.: An Overview of Character Recognition Focused on Off-line Handwriting. IEEE Trans. on Systems, Man, and Cybernetics - Part C: Applications and Reviews 31(2) (2001) 216-233CrossRefGoogle Scholar
  8. 8.
    . Mori, S., Suen, C., Yamamoto, K.: Historical Review of OCR Research and Development. In O’Gorman, L., Kasturi, R., eds.: Document Image Analysis. IEEE Computer Society Press (1995) 244-273Google Scholar
  9. 9.
    Simon, J.C.: Off-line Cursive Word Recognition. Proceedings of the IEEE 80(7) (1992) 1150-1161CrossRefGoogle Scholar
  10. 10.
    Steinherz, T., Rivlin, E., Intrator, N.: Offline Cursive Script Word Recognition - a Survey. Int. Journal on Document Analysis and Recognition 2(2) (1999) 90-110Google Scholar
  11. 11.
    Vinciarelli, A.: A Survey on Off-line Cursive Word Recognition. Pattern Recog- nition 35(7) (2002) 1433-1446MATHCrossRefGoogle Scholar
  12. 12.
    Marti, U.V., Bunke, H.: Using a Statistical Language Model to Improve the Performance of an HMM-based Cursive Handwriting Recognition System. Int. Journal of Pattern Recognition and Artificial Intelligence 15(1) (2001) 65-90CrossRefGoogle Scholar
  13. 13.
    Kittler, J., Hatef, M., Duin, R., Matas, J.: On Combining Classifiers. IEEE Trans. on Pattern Analysis and Machine Intelligence 20(3) (1998) 226-239CrossRefGoogle Scholar
  14. 14.
    Roli, F., Kittler, J., Windeatt, T., eds.: Proc. 5th Int. Workshop on Multiple Classifier Systems, Cagliari, Italy, Springer (2004)Google Scholar
  15. 15.
    .Kuncheva, L.: Combining Pattern Classifiers: Methods and Algorithms. Wiley- Interscience (2004)Google Scholar
  16. 16.
    Lorette, G.: Handwriting Recognition or Reading? - What is the Situation at the Dawn of the 3rd Millenium? Int. Journal on Document Analysis and Recognition 2(1) (1999) 2-12CrossRefGoogle Scholar
  17. 17.
    Rosenfeld, R.: Two Decades of Statistical Language Modeling: Where do We Go from Here? Proc. of the IEEE 88(8) (2000) 1270-1278CrossRefGoogle Scholar
  18. 18.
    . Elliman, D., Sherkat, N.: A Truthing Tool for Generating a Database of Cursive Words. In: Proc. 6th Int. Conf. on Document Analysis and Recognition, Seattle, WA, USA (2001) 1255-1262Google Scholar
  19. 19.
    . Guyon, I., Haralick, R., Hull, J., Phillips, I.: Data Sets for OCR and Document Image Understanding Research. In Bunke, H., Wang, P., eds.: Handbook of Character Recognition and Document Image Analysis. World Scientific (1997) 779-799Google Scholar
  20. 20.
    Marti, U.V., Bunke, H.: The IAM-Database: an English Sentence Database for Off-line Handwriting Recognition. Int. Journal on Document Analysis and Recognition 5(1) (2002) 39-46MATHCrossRefGoogle Scholar
  21. 21.
    Cano, J., Pérez-Cortes, J., Arlandis, J., Llobet, R.: Training Set Expansion in Handwritten Character Recognition. In: Proc. 9th SSPR / 4th SPR, Windsor, Ontario, Canada (2002) 548-556Google Scholar
  22. 22.
    Günter, S., Bunke, H.: Multiple Classifier Systems in Offline Handwritten Word Recognition - On the Influence of Training Set and Vocabulary Size. Int. Journal of Pattern Recognition and Artificial Intelligence 18(7) (2004) 1302-1320Google Scholar
  23. 23.
    Rowley, H., Goyal, M., Bennett, J.: The Effect of Large Training Set Sizes on Online Japanese Kanji and English Cursive Recognizers. In: Proc. 8th Int. Workshop on Frontiers in Handwriting Recognition, Niagara-on-the-Lake, Ontario, Canada (2002) 36-40CrossRefGoogle Scholar
  24. 24.
    Velek, O., Nakagawa, M.: The Impact of Large Training Sets on the Recogni-tion Rate of Off-line Japanese Kanji Character Classifiers. In: Proc. 5th IAPR Workshop on Document Analysis Systems, Princeton, New Jersey, USA (2002) 106-109Google Scholar
  25. 25.
    Varga, T., Bunke, H.: Generation of Synthetic Training Data for an HMM-based Handwriting Recognition System. In: Proc. 7th Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland (2003) 618-622CrossRefGoogle Scholar
  26. 26.
    Varga, T., Bunke, H.: Off-line Handwritten Textline Recognition Using a Mix-ture of Natural and Synthetic Training Data. In: Proc. 17th Int. Conf. on Pattern Recognition, Cambridge, United Kingdom (2004) 545-549CrossRefGoogle Scholar
  27. 27.
    Kasturi, R., O’Gorman, L., Govindaraju, V.: Document Image Analysis: A Primer. Sadhana 27(1) (2002) 3-22CrossRefGoogle Scholar
  28. 28.
    Nagy, G., Nartker, T., Rice, S.: Optical Character Recognition: An Illustrated Guide to the Frontier. In: Proc. IS&T/SPIE Symposium on Electronic Imaging: Science and Technology. Volume 3967., San Jose, CA, USA (2000) 58-69Google Scholar
  29. 29.
    Rice, S., Jenkins, F., Nartker, T.: The Fifth Annual Test of OCR Accuracy. Technical Report ISRI-TR-96-01, University of Nevada, Las Vegas, Nevada, USA (1996)Google Scholar
  30. 30.
    Baird, H.: The State of the Art of Document Image Degradation Modeling. In: Proc. 4th IAPR Workshop on Document Analysis Systems, Rio de Janeiro, Brasil (2000) 1-13Google Scholar
  31. 31.
    . Stork, D.: Toward a Computational Theory of Data Acquisition and Truthing. In Helmbold, D., Williamson, B., eds.: Computational Learning Theory. Volume 2111 of Lecture Notes in Computer Science. Springer (2001) 194-207Google Scholar
  32. 32.
    Vuurpijl, L., Niels, R., Erp, M., Schomaker, L., Ratzlaff, E.: Verifying the UNIPEN devset. In: Proc. 9th Int. Workshop on Frontiers in Handwriting Recognition, Kokubunji, Tokyo, Japan (2004) 586-591CrossRefGoogle Scholar
  33. 33.
    . Baird, H.: Document Image Defect Models. In Baird, H., Bunke, H., Yamamoto, K., eds.: Structured Document Image Analysis. Springer (1992) 546-556Google Scholar
  34. 34.
    Doermann, D., Yao, S.: Generating Synthetic Data for Text Analysis Systems. In: Proc. 4th Annual Symposium on Document Analysis and Information Re-trieval, Las Vegas, Nevada, USA (1995) 449-467Google Scholar
  35. 35.
    Kanungo, T., Haralick, R., Phillips, I.: Global and Local Document Degrada-tion Models. In: Proc. 2nd Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan (1993) 730-734CrossRefGoogle Scholar
  36. 36.
    Baird, H.: Document Image Defect Models and their Uses. In: Proc. 2nd Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan (1993) 62-67CrossRefGoogle Scholar
  37. 37.
    Ho, T., Baird, H.: Large-Scale Simulation Studies in Image Pattern Recognition. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(10) (1997) 1067-1079Google Scholar
  38. 38.
    Märgner, V., Pechwitz, M.: Synthetic Data for Arabic OCR System Develop-ment. In: Proc. 6th Int. Conf. on Document Analysis and Recognition, Seattle, WA, USA (2001) 1159-1163CrossRefGoogle Scholar
  39. 39.
    Baird, H., Fossey, R.: A 100-Font Classifier. In: Proc. 1st Int. Conf. on Document Analysis and Recognition, St.Malo, France (1991) 332-340Google Scholar
  40. 40.
    Baird, H., Nagy, G.: A Self-Correcting 100-Font Classifier. In: Proc. IS&T/SPIE Symposium on Electronic Imaging: Science and Technology. Volume 2181., San Jose, California, USA (1994) 106-115Google Scholar
  41. 41.
    Ho, T., Baird, H.: Evaluation of OCR Accuracy Using Synthetic Data. In: Proc. 4th Annual Symposium on Document Analysis and Information Retrieval, Las Vegas, Nevada, USA (1995) 413-422Google Scholar
  42. 42.
    Helmers, M., Bunke, H.: Generation and Use of Synthetic Training Data in Cursive Handwriting Recognition. In: Proc. 1st Iberian Conf. on Pattern Recog-nition and Image Analysis, Puerto de Andratx, Mallorca, Spain (2003) 336-345Google Scholar
  43. 43.
    . Drucker, H., Schapire, R., Simard, P.: Improving Performance in Neural Net- works Using a Boosting Algorithm. In Hanson et. al., S., ed.: Advances in Neural Information Processing Systems 5. Morgan Kaufmann (1993) 42-49Google Scholar
  44. 44.
    Ha, T., Bunke, H.: Off-line Handwritten Numeral Recognition by Perturba- tion Method. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(5) (1997) 535-539CrossRefGoogle Scholar
  45. 45.
    Mao, J., Mohiuddin, K.: Improving OCR Performance Using Character Degra-dation Models and Boosting Algorithm. Pattern Recognition Letters 18 (1997) 1415-1419CrossRefGoogle Scholar
  46. 46.
    Mori, M., Suzuki, A., Shio, A., Ohtsuka, S.: Generating New Samples from Handwritten Numerals based on Point Correspondence. In: Proc. 7th Int. Work-shop on Frontiers in Handwriting Recognition, Amsterdam, The Netherlands (2000) 281-290Google Scholar
  47. 47.
    . Kaufmann, G., Bunke, H., Ha, T.: Recognition of Cursively Handwritten Words Using a Combined Normalization/Perturbation Approach. In Downton, A., Impedovo, S., eds.: Progress in Handwriting Recognition. World Scientific (1997) 21-28Google Scholar
  48. 48.
    Setlur, S., Govindaraju, V.: Generating Manifold Samples from a Handwritten Word. Pattern Recognition Letters 15 (1994) 901-905CrossRefGoogle Scholar
  49. 49.
    Velek, O., Liu, C.L., Nakagawa, M.: Generating Realistic Kanji Character Im-ages from On-line Patterns. In: Proc. 6th Int. Conf. on Document Analysis and Recognition, Seattle, WA, USA (2001) 556-560CrossRefGoogle Scholar
  50. 50.
    Govindan, V., Shivaprasad, A.: Artificial Database for Character Recognition Research. Pattern Recognition Letters 12(10) (1991) 645-648CrossRefGoogle Scholar
  51. 51.
    Tung, C., Lee, H.: Performance Analysis of an OCR System via a Handwritten Character Generator. Pattern Recognition 27(2) (1994) 221-232CrossRefGoogle Scholar
  52. 52.
    . Simard, P., Cun, Y., Denker, J.: Efficient Pattern Recognition Using a New Transformation Distance. In et al., S.H., ed.: Advances in Neural Information Processing Systems 5. Morgan Kaufmann (1993) 50-58Google Scholar
  53. 53.
    Guyon, I.: Handwriting Synthesis from Handwritten Glyphs. In: Proc. 5th Int. Workshop Frontiers in Handwriting Recognition, Essex, United Kingdom (1996) 309-312Google Scholar
  54. 54.
    Choi, H., Cho, S., Kim, J.: Generation of Handwritten Characters with Bayesian Network based On-line Handwriting Recognizers. In: Proc. 7th Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland (2003) 995-1001CrossRefGoogle Scholar
  55. 55.
    Choi, H., Cho, S., Kim, J.: Writer-Dependent Online Handwriting Generation with Bayesian Networks. In: Proc. 9th Int. Workshop on Frontiers in Handwrit-ing Recognition, Kokubunji, Tokyo, Japan (2004) 130-135CrossRefGoogle Scholar
  56. 56.
    Wang, J., Wu, C., Xu, Y.Q., Shum, H.Y., Ji, L.: Learning based Cursive Hand-writing Synthesis. In: Proc. 8th Int. Workshop on Frontiers in Handwriting Recognition, Niagara-on-the-Lake, Ontario, Canada (2002) 157-162CrossRefGoogle Scholar
  57. 57.
    Lee, D.H., Cho, H.G.: A New Synthesizing Method for Handwriting Korean Scripts. Int. Journal of Pattern Recognition and Artificial Intelligence 12(1) (1998) 46-61Google Scholar
  58. 58.
    Plamondon, R., Guerfali, W.: The Generation of Handwriting with Delta-lognormal Synergies. Biological Cybernetics 78 (1998) 119-132MATHCrossRefGoogle Scholar
  59. 59.
    Baird, H., Coates, A., Fateman, R.: PessimalPrint: a Reverse Turing Test. Int. Journal on Document Analysis and Recognition 5(2-3) (2003) 158-163CrossRefGoogle Scholar
  60. 60.
    . Ahn, L., Blum, M., Hopper, N., Langford, J.: The CAPTCHA Web Page. http://www.captcha.net (2000)
  61. 61.
    . Ahn, L., Blum, M., Hopper, N., Langford, J.: CAPTCHA: Telling Humans and Computers Apart. In Biham, E., ed.: Advances in Cryptology. Volume 2656 of Lecture Notes in Computer Science. Springer (2003) 294-311Google Scholar
  62. 62.
    Ahn, L., Blum, M., Langford, J.: Telling Humans and Computers Apart Au- tomatically - How Lazy Cryptographers Do AI. Communications of the ACM 47(2)(2004) 57-60Google Scholar
  63. 63.
    Baird, H., Luk, M.: Protecting Websites with Reading-based CAPTCHAs. In: Proc. 2nd Int. Web Document Analysis Workshop, Edinburgh, Scotland (2003) 53-56Google Scholar
  64. 64.
    Baird, H., Riopka, T.: ScatterType: a Reading CAPTCHA Resistant to Segmen-tation Attack. In: Proc. 12th SPIE/IS&T Document Recognition and Retrieval Conference. Volume 5676., San Jose, California, USA (2005)Google Scholar
  65. 65.
    Chew, M., Baird, H.: BaffleText: a Human Interactive Proof. In: Proc. 10th SPIE/IS&T Document Recognition and Retrieval Conference. Volume 5010., Santa Clara, California, USA (2003) 305-316Google Scholar
  66. 66.
    . Lillibridge, M., Abadi, M., Bharat, K., Broder, A.: Method for Selectively Re-stricting Access to Computer Systems. U.S. Patent No. 6,195,698 (2000)Google Scholar
  67. 67.
    Simard, P., Szeliski, R., Couvreur, J., Calinov, I.: Using Character Recognition and Segmentation to Tell Computer from Humans. In: Proc. 7th Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland (2003) 418-423CrossRefGoogle Scholar
  68. 68.
    Rusu, A., Govindaraju, V.: Handwritten CAPTCHA: Using the Difference in the Abilities of Humans and Machines in Reading Handwritten Words. In: 9th Int. Workshop on Frontiers in Handwriting Recognition, Kokubunji, Tokyo, Japan (2004) 226-231CrossRefGoogle Scholar
  69. 69.
    Manzanera, A., Bernard, T.: Improved Low Complexity Fully Parallel Thinning Algorithm. In: Proc. 10th Int. Conf. on Image Analysis and Processing, Venice, Italy (1999) 215-220Google Scholar
  70. 70.
    . Soille, P.: Morphological Image Analysis. Springer (1999)Google Scholar
  71. 71.
    Rabiner, L.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2) (1989) 257-286CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Tamás Varga
    • 1
  • Horst Bunke
    • 1
  1. 1.Institute of Computer Science and Applied Mathematics (IAM)University of BernBernSwitzerland

Personalised recommendations