MCMC Based Generative Adversarial Networks for Handwritten Numeral Augmentation

  • He Zhang
  • Chunbo Luo
  • Xingrui Yu
  • Peng Ren
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 463)


In this paper, we propose a novel data augmentation framework for handwritten numerals by incorporating the probabilistic learning and the generative adversarial learning. First, we simply transform numeral images from spatial space into vector space. The Gaussian based Markov probabilistic model is then developed for simulating synthetic numeral vectors given limited handwritten samples. Next, the simulated data are used to pre-train the generative adversarial networks (GANs), which initializes their parameters to fit the general distribution of numeral features. Finally, we adopt the real handwritten numerals to fine-tune the GANs, which greatly increases the authenticity of generated numeral samples. In this case, the outputs of the GANs can be employed to augment original numeral datasets for training the follow-up inference models. Considering that all simulation and augmentation are operated in 1-D vector space, the proposed augmentation framework is more computationally efficient than those based on 2-D images. Extensive experimental results demonstrate that our proposed augmentation framework achieves improved recognition accuracy.


Data augmentation Probabilistic model Generative adversarial learning Handwritten numeral classification 



This work was supported by grants from the Chinese Scholarship Council (CSC) program.


  1. 1.
    Ha, T.M., Bunke, H.: Off-Line, handwritten numeral recognition by perturbation method. IEEE Trans. Pattern Anal. Mach. Intell. 19, 535–539 (1997)Google Scholar
  2. 2.
    Wu, Y.C., Yin, F., Liu, C.L.: Evaluation of geometric context models for handwritten numeral string recognition. In: 14th ICFHR, pp. 193–198. IEEE Press, Greece (2014)Google Scholar
  3. 3.
    Gelfand, A.E., Smith, A.F.: Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85, 398–409 (1990)Google Scholar
  4. 4.
    Andrieu, C., de Freitas, N., Doucet, A., Jordan, M.I.: An introduction to MCMC for machine learning. Mach. Learn. 50, 5–43 (2003)Google Scholar
  5. 5.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: 27th NIPS, pp. 2672–2680. Curran Associates Inc., Canada (2014)Google Scholar
  6. 6.
    Denton, E.L., Chintala, S., Fergus, R.: Deep generative image models using a laplacian pyramid of adversarial networks. In: 28th NIPS, pp. 1486–1494. Curran Associates Inc., Canada (2015)Google Scholar
  7. 7.
    Metropolis, N., Ulam, S.: The Monte Carlo method. J. Am. Stat. Assoc. 44, 335–341 (1949)Google Scholar
  8. 8.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)Google Scholar
  9. 9.
    Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A. A.: Context encoders: feature learning by inpainting. In: 2016 CVPR, pp. 2536–2544. IEEE Press, America (2016)Google Scholar
  10. 10.
    Dai, B., Lin, D., Urtasun, R., Fidler, S.: Towards Diverse and Natural Image Descriptions via a Conditional GAN. arXiv preprint arXiv:1703.06029 (2017)
  11. 11.
    Arjovsky, M., Bottou, L.: Towards Principled Methods for Training Generative Adversarial Networks. arXiv preprint arXiv:1701.04862 (2017)
  12. 12.
    Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1703.06029 (2017)
  13. 13.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, New York (2012)Google Scholar
  14. 14.
    Kingma, D., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980 (2014)
  15. 15.
    Sutskever, I., Martens, J., Dahl, G.E., Hinton, G.E.: On the importance of initialization and momentum in deep learning. In: 30th ICML, pp. 1139–1147. PMLR, America (2013)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.College of Engineering, Mathematics and Physical SciencesUniversity of ExeterExeterUK
  2. 2.College of Information and Control EngineeringChina University of Petroleum (East China)QingdaoChina

Personalised recommendations