Enhanced Non-linear Features for On-line Handwriting Recognition Using Deep Learning
Conventionally, a deep neural network (DNN) is trained to predict probabilities of class labels. Recently, DNN has shown great success in many pattern recognition tasks such as speech recognition and handwritten digit recognition. To take advantage of its great learning power, we propose and build an on-line handwriting recognition system for French strings, which applies a DNN for non-linear feature transformation before training the character models. When a DNN is predicting class labels, its hidden layer outputs can be regarded as a better representation of the original features extracted from handwriting trajectory data. In this paper, we demonstrate that the proposed system can achieve a relative character error rate (CER) reduction of about 28.5% when being compared to a conventional system without feature transformation. We also notice that the CER could be further reduced by 3.3% relatively when the transformed features are used along with the original features.
KeywordsDeep neural network (DNN) On-line handwriting recognition Feature transformation Hidden Markov model (HMM) Gaussian mixture model (GMM)
Unable to display preview. Download preview PDF.
- 2.Santosh, K., Nattee, C., et al.: A comprehensive survey on on-line handwriting recognition technology and its real application to the nepalese natural handwriting. Kathmandu University Journal of Science, Engineering, and Technology 5(I), 31–55 (2009)Google Scholar
- 4.Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. In: 2011 IEEE International Conference on Document Analysis and Recognition (ICDAR), pp. 1135–1139 (2011)Google Scholar
- 5.Liwicki, M., Graves, A., Bunke, H., Schmidhuber, J.: A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In: Proc. 9th Int. Conf. on Document Analysis and Recognition, vol. 1, pp. 367–371 (2007)Google Scholar
- 6.Seide, F., Li, G., Yu, D.: Conversational speech transcription using context- dependent deep neural networks. In: Interspeech, pp. 437–440 (2011)Google Scholar
- 7.Deng, L., Yu, D.: DEEP LEARNING: Methods and Applications. NOW Publishers (2014)Google Scholar
- 9.Mitchell, M.: An introduction to genetic algorithms. MIT Press (1998)Google Scholar
- 10.Povey, D., Woodland, P.C.: Minimum phone error and I-smoothing for improved discriminative training. In: 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 1, pp. 105–108 (2002)Google Scholar