Kernel PCA for HMM-Based Cursive Handwriting Recognition

  • Andreas Fischer
  • Horst Bunke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5702)


In this paper, we propose Kernel Principal Component Analysis as a feature selection method for offline cursive handwriting recognition based on Hidden Markov Models. In contrast to formerly used feature selection methods, namely standard Principal Component Analysis and Independent Component Analysis, nonlinearity is achieved by making use of a radial basis function kernel. In an experimental study we demonstrate that the proposed nonlinear method has a great potential to improve cursive handwriting recognition systems and is able to significantly outperform linear feature selection methods. We consider two diverse datasets of isolated handwritten words for the experimental evaluation, the first consisting of modern English words, and the second consisting of medieval Middle High German words.


Independent Component Analysis Feature Selection Method Independent Component Analysis Kernel Principal Component Analysis Word Error Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Plamondon, R., Srihari, S.: Online and off-line handwriting recognition: A comprehensive survey. IEEE Trans. Pattern Analysis and Machine Intelligence 22, 63–84 (2000)CrossRefGoogle Scholar
  2. 2.
    Marti, U.V., Bunke, H.: Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. Journal of Pattern Recognition and Art. Intelligence 15, 65–90 (2001)CrossRefGoogle Scholar
  3. 3.
    Vinciarelli, A., Bengio, S.: Off-line cursive word recognition using continuous density HMMs trained with PCA and ICA features. In: 16th Int. Conf. on Pattern Recognition, vol. 3, pp. 81–84 (2002)Google Scholar
  4. 4.
    Oja, E.: Independent component analysis: algorithms and applications. Neural Networks 13, 411–430 (2000)CrossRefGoogle Scholar
  5. 5.
    Müller, K., Mika, S., Rätsch, G., Tsuda, K., Schölkopf, B.: An introduction to kernel-based learning algorithms. Neural Networks 12(2), 181–202 (2001)CrossRefGoogle Scholar
  6. 6.
    Schölkopf, B., Smola, A., Müller, K.R.: Kernel principal component analysis. In: Advances in Kernel Method – Support Vector Learning, pp. 327–352. MIT Press, Cambridge (1999)Google Scholar
  7. 7.
    Liu, Y.H., Chen, Y.T., Lu, S.S.: Face detection using kernel PCA and imbalanced SVM. In: Jiao, L., Wang, L., Gao, X.-b., Liu, J., Wu, F. (eds.) ICNC 2006. LNCS, vol. 4221, pp. 351–360. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Ekinci, M., Aykut, M.: Palmprint recognition by applying wavelet-based kernel PCA. Journal of Computer Science and Technology 23(5), 851–861 (2008)CrossRefGoogle Scholar
  9. 9.
    Shi, D., Ong, Y.S., Tan, E.C.: Handwritten chinese character recognition using kernel active handwriting model. In: Proc. IEEE Int. Conf. on Systems, Man and Cybernetics, vol. 1, pp. 251–255. IEEE, Los Alamitos (2003)Google Scholar
  10. 10.
    Takiguchi, T., Ariki, Y.: Robust feature extraction using kernel PCA. In: Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, pp. 509–512. IEEE, Los Alamitos (2006)Google Scholar
  11. 11.
    Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for off-line handwriting recognition. Int. Journal on Document Analysis and Recognition 5, 39–46 (2002)zbMATHCrossRefGoogle Scholar
  12. 12.
    Wüthrich, M., Liwicki, M., Fischer, A., Indermühle, E., Bunke, H., Viehhauser, G., Stolz, M.: Lanugage model integration for the recognition of handwritten medieval documents. Accepted for publication in Proc. IEEE Int. Conf. on Document Analysis and Recognition (2009)Google Scholar
  13. 13.
    Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–285 (1989)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Andreas Fischer
    • 1
  • Horst Bunke
    • 1
  1. 1.Institute of Computer Science and Applied MathematicsUniversity of BernBernSwitzerland

Personalised recommendations