Embedded Prototype Subspace Classification: A Subspace Learning Framework

  • Anders HastEmail author
  • Mats Lind
  • Ekta Vats
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11679)


Handwritten text recognition is a daunting task, due to complex characteristics of handwritten letters. Deep learning based methods have achieved significant advances in recognizing challenging handwritten texts because of its ability to learn and accurately classify intricate patterns. However, there are some limitations of deep learning, such as lack of well-defined mathematical model, black-box learning mechanism, etc., which pose challenges. This paper aims at going beyond the black-box learning and proposes a novel learning framework called as Embedded Prototype Subspace Classification, that is based on the well-known subspace method, to recognise handwritten letters in a fast and efficient manner. The effectiveness of the proposed framework is empirically evaluated on popular datasets using standard evaluation measures.


Handwritten text Subspaces Deep learning t-SNE 


  1. 1.
    Carbon, M., Hallin, M., Tat Tran, L.: Kernel density estimation for random fields: the l 1 theory. J. Nonparametric Stat. 6(2–3), 157–170 (1996)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., Ha, D.: Deep learning for classical Japanese literature. CoRR abs/1812.01718 (2018).
  3. 3.
    Cohen, G., Afshar, S., Tapson, J., van Schaik, A.: EMNIST: an extension of MNIST to handwritten letters. CoRR abs/1702.05373 (2017).
  4. 4.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002). Scholar
  5. 5.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893, June 2005.
  6. 6.
    Ding, S., Zhao, H., Zhang, Y., Xu, X., Nie, R.: Extreme learning machine: algorithm, theory and applications. Artif. Intell. Rev. 44(1), 103–115 (2015). Scholar
  7. 7.
    Domhan, T., Springenberg, J.T., Hutter, F.: Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In: Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI 2015, pp. 3460–3468. AAAI Press (2015).
  8. 8.
    Dutta, K., Krishnan, P., Mathew, M., Jawahar, C.: Improving CNN-RNN hybrid networks for handwriting recognition. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 80–85. IEEE (2018)Google Scholar
  9. 9.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)Google Scholar
  10. 10.
    Ilievski, I., Akhtar, T., Feng, J., Shoemaker, C.: Efficient hyperparameter optimization for deep learning algorithms using deterministic RBF surrogates (2017).
  11. 11.
    Jamieson, K., Talwalkar, A.: Non-stochastic best arm identification and hyperparameter optimization. In: Gretton, A., Robert, C.C. (eds.) Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 51, pp. 240–248. PMLR, Cadiz, Spain, 09–11 May 2016.
  12. 12.
    Kohonen, T., Lehtiö, P., Rovamo, J., Hyvärinen, J., Bry, K., Vainio, L.: A principle of neural associative memory. Neuroscience 2(6), 1065–1076 (1977). Scholar
  13. 13.
    Kohonen, T., Oja, E.: Fast adaptive formation of orthogonalizing filters and associative memory in recurrent networks of neuron-like elements. Biol. Cybern. 21(2), 85–95 (1976). Scholar
  14. 14.
    Kohonen, T., Reuhkala, E., Mäkisara, K., Vainio, L.: Associative recall of images. Biol. Cybern. 22(3), 159–168 (1976). Scholar
  15. 15.
    Krishnan, P., Dutta, K., Jawahar, C.: Word spotting and recognition using deep embedding. In: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS), pp. 1–6. IEEE (2018)Google Scholar
  16. 16.
    Laaksonen, J., Aksela, M., Oja, E., Kangas, J.: Adaptive local subspace classifier in on-line recognition of handwritten characters. In: IJCNN 1999 International Joint Conference on Neural Networks. Proceedings (Cat. No. 99CH36339), vol. 4, pp. 2812–2815, July 1999.
  17. 17.
    Laaksonen, J.: Subspace classifiers in recognition of handwritten digits, 07 May 1997.
  18. 18.
    LeCun, Y., Cortes, C., Burges, C.: Mnist handwritten digit database. AT&T Labs (2010).
  19. 19.
    Läuter, H.: Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman & Hall, London, p. 175 (1986). £12.—. Biometrical Journal 30(7), 876–877 (1988). Scholar
  20. 20.
    Maaten, L.v.d., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)Google Scholar
  21. 21.
    McInnes, L., Healy, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv e-prints, February 2018CrossRefGoogle Scholar
  22. 22.
    Oja, E., Kohonen, T.: The subspace learning algorithm as a formalism for pattern recognition and neural networks. In: IEEE 1988 International Conference on Neural Networks, vol. 1, pp. 277–284, July 1988.
  23. 23.
    Rauber, P.E., Fadel, S.G., Falcão, A.X., Telea, A.C.: Visualizing the hidden activity of artificial neural networks. IEEE Trans. Vis. Comput. Graph. 23(1), 101–110 (2017). Scholar
  24. 24.
    Roerdink, J.B., Meijster, A.: The watershed transform: definitions, algorithms and parallelization strategies. Fundam. Inf. 41(1,2), 187–228 (2000). Scholar
  25. 25.
    van Schaik, A., Tapson, J.: Online and adaptive pseudoinverse solutions for ELM weights. Neurocomputing 149(PA), 233–238 (2015). Scholar
  26. 26.
    Shapshak, P.: Artificial intelligence and brain. Bioinformation 14(1), 38 (2018)CrossRefGoogle Scholar
  27. 27.
    Shores, T.S.: Applied Linear Algebra and Matrix Analysis, p. 104. Springer Publisher, New York (2007). Scholar
  28. 28.
    Sudholt, S., Fink, G.A.: PHOCNET: a deep convolutional neural network for word spotting in handwritten documents. In: ICFHR, pp. 277–282. IEEE Computer Society (2016)Google Scholar
  29. 29.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991). Scholar
  30. 30.
    Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13(6), 583–598 (1991). Scholar
  31. 31.
    Watanabe, S., Pakvasa, N.: Subspace method in pattern recognition. In: 1st International Joint Conference on Pattern Recognition, Washington DC, pp. 25–32 (1973)Google Scholar
  32. 32.
    Watanabe, W., Lambert, P.F., Kulikowski, C.A., Buxto, J.L., Walker, R.: Evaluation and selection of variables in pattern recognition, vol. 2, pp. 91–122. Academic Press, New York (1967)Google Scholar
  33. 33.
    Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. Chemom. Intell. Lab. Syst. 2(1–3), 37–52 (1987)CrossRefGoogle Scholar
  34. 34.
    Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. CoRR abs/1708.07747 (2017).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Division of Visual Information and Interaction, Department of Information TechnologyUppsala UniversityUppsalaSweden

Personalised recommendations