Farsi/Arabic handwritten digit recognition based on ensemble of SVD classifiers and reliable multi-phase PSO combination rule

Original Paper

Abstract

The problem of handwritten digit recognition has long been an open problem in the field of pattern classification and of great importance in industry. The heart of the problem lies within the ability to design an efficient algorithm that can recognize digits written and submitted by users via a tablet, scanner, and other digital devices. From an engineering point of view, it is desirable to achieve a good performance within limited resources. To this end, we have developed a new approach for handwritten digit recognition that uses a small number of patterns for training phase. To improve the overall performance achieved in classification task, the literature suggests combining the decision of multiple classifiers rather than using the output of the best classifier in the ensemble; so, in this new approach, an ensemble of classifiers is used for the recognition of handwritten digit. The classifiers used in proposed system are based on singular value decomposition (SVD) algorithm. The experimental results and the literature show that the SVD algorithm is suitable for solving sparse matrices such as handwritten digit. The decisions obtained by SVD classifiers are combined by a novel proposed combination rule which we named reliable multi-phase particle swarm optimization. We call the method “Reliable” because we have introduced a novel reliability parameter which is applied to tackle the problem of PSO being trapped in local minima. In comparison with previous methods, one of the significant advantages of the proposed method is that it is not sensitive to the size of training set. Unlike other methods, the proposed method uses just 15 % of the dataset as a training set, while other methods usually use (60–75) % of the whole dataset as the training set. To evaluate the proposed method, we tested our algorithm on Farsi/Arabic handwritten digit dataset. What makes the recognition of the handwritten Farsi/Arabic digits more challenging is that some of the digits can be legally written in different shapes. Therefore, 6000 hard samples (600 samples per class) are chosen by K-nearest neighbor algorithm from the HODA dataset which is a standard Farsi/Arabic digit dataset. Experimental results have shown that the proposed method is fast, accurate, and robust against the local minima of PSO. Finally, the proposed method is compared with state of the art methods and some ensemble classifier based on MLP, RBF, and ANFIS with various combination rules.

Keywords

Classifiers combination Two-dimensional PCA (2DPCA) Singular value decomposition (SVD) Particle swarm optimization (PSO) Reliability 

References

  1. 1.
    Abdi, M.J., Salimi, H.: Farsi handwriting recognition with mixture of RBF experts based on particle swarm optimization. Int. J. Inf. Sci. Comput. Math. 2, 129–136 (2010)Google Scholar
  2. 2.
    Alaei, A., Pal, U., Nagabhushan, P.: Using modified contour features and SVM based classifier for the recognition of Persian/Arabic handwritten numerals, 2009 Seventh International Conference on Advances in Pattern RecognitionGoogle Scholar
  3. 3.
    Anasuya Devi, H.K.: “Thresholding. A pixel-level image processing methodology preprocessing technique for an OCR system for the Brahmi script”. Ancient Asia (2009)Google Scholar
  4. 4.
    Bellman, R.: Introduction to Matrix Analysis, 2nd edn. McGraw-Hill, NY (1970)MATHGoogle Scholar
  5. 5.
    Broumandnia, A., Shanbehzadeh, J., RezakhahVarnoosfaderani, M.: Persian/Arabic handwritten word recognition using M-band packet wavelet transform. Image Vis. Comput. Elsevier 26, 829–842 (2008)CrossRefGoogle Scholar
  6. 6.
    Brown, G.: Diversity in neural network ensembles. PhD thesis, University of Birmingham; September (2003)Google Scholar
  7. 7.
    Chandra, A, Yao, X.: DIVACE: diverse and accurate ensemble learning algorithm. In: Proceeding of the international conference on intelligent data engineering and automated learning (IDEAL: lecture notes in computer science, vol. 3117. Berlin Springer 2004, 619–25 (2004)Google Scholar
  8. 8.
    Chen, C.H., Wang, P.S.P.: Handbook of Pattern Recognition and Computer Vision, 3rd edn. World Scientific, Singapore (2005)CrossRefMATHGoogle Scholar
  9. 9.
    Cordella, L.P., Foggia, P., Sansone, C., Tortorella, F., Vento, M.: Reliability parmeters to improve combination strategies in multi-expert systems. Pattern Anal. Appl. 2, 205–214 (1999)CrossRefGoogle Scholar
  10. 10.
    Dehghan, M., Faez, K.: Farsi handwritten character recognition with moment invariants. In: Proceedings of 13th International Conference on Digital Signal Processing. Vol. 2, 507–510 (1997)Google Scholar
  11. 11.
    Dimauro, G., Impedovo, S., Pirlo, G., Salzo, A.: Automatic bankcheck processing: a new engineered system. Mach. Percept. Artif. Intell. 28, 5–42 (1997)CrossRefGoogle Scholar
  12. 12.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, New York (2001)MATHGoogle Scholar
  13. 13.
    Ebrahimpour, R., Esmkhani, A., Faridi, S.: Farsi handwritten digit recognition based on mixture of RBF experts. IEICE Electron. Express 7(14), 1014–1019 (2010)CrossRefGoogle Scholar
  14. 14.
    Eldén, L.: Matrix Methods in Data Mining and Pattern Recognition. SIAM, Philadelphia 3, 4 (2007)Google Scholar
  15. 15.
    Fang, Y., Tan, T., Wang, Y.: Fusion of global and local features for face verification. In: 16th International conference on pattern recognition (ICPR 2002), vol. 2, pp. 382–385 (2002)Google Scholar
  16. 16.
    Foley, D.H.: Considerations of sample and feature size. IEEE Trans. Inf. Theory 18(5), 618–626 (1972)CrossRefMATHGoogle Scholar
  17. 17.
    Frade, F., De la Torre, F., Gross, R., Baker, S., Kumar, V.: Representational oriented component analysis (ROCA) for face recognition with one sample image per training class. In: Proceedings, IEEE Conference on Computer Vision and Pattern Recognition vol. 2, pp. 266–273 (2005)Google Scholar
  18. 18.
    Hanmandlu, M., Grover, J., Madasu, V.K., Vasikarla, S.: “Input fuzzy modeling for the recognition of handwritten Hindi numeral”. International conference on informational technology, vol. 2, pp. 208–213 (2007)Google Scholar
  19. 19.
    Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993–1001 (1990)CrossRefGoogle Scholar
  20. 20.
    Harifi, A., Aghagolzadeh, A.: A new pattern for handwritten Persian/Arabic digit recognition. J. Inf. Technol. 3, 249–252 (2004)Google Scholar
  21. 21.
  22. 22.
    Jain, A.K., Chandrasekaran, B.: Dimensionality and sample size considerations in pattern recognition practice. In: Krishnaiah, P., Kanal, L. (eds.) Handbook of Statistics volume 2, pp. 835–855. Amsterdam, North Holland (1982)Google Scholar
  23. 23.
    Jang, J.-S.R.: ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans. Syst. Man Cybern. 23(3), 665–685 (1993)CrossRefGoogle Scholar
  24. 24.
    Jolliffe, I.T.: Principal Component Analysis. Springer, New York (1986)CrossRefGoogle Scholar
  25. 25.
    Kennedy, J, Eberhart, RC.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks (ICNN 1995), vol. 4, pp. 1942–1948 (1995)Google Scholar
  26. 26.
    Keysers, D., Deselaers, T., Gollan, C., Ney, H.: Deformation models for image recognition. IEEE Trans. Patt. Anal. Mach. Intell. 29(8), 1422–1435 (2007)CrossRefGoogle Scholar
  27. 27.
    Khosravi, H., Kabir, E.: Introducing a very large dataset of handwritten Farsi digit and a study on their varieties. Pattern Recognit. Lett. 28, 1133–1141 (2007)CrossRefGoogle Scholar
  28. 28.
    Kim, C., Oh, J., Choi, C.-H.: Combined subspace method using global and local features for face recognition. In: Proceedings of the International Joint Conference on, Neural Networks, pp. 2030–2035 (2005)Google Scholar
  29. 29.
    Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)CrossRefGoogle Scholar
  30. 30.
    Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion an experimental comparison. Pattern Recognit. 34(2), 299–314 (2001)CrossRefMATHGoogle Scholar
  31. 31.
    Lauer, F., Suen, C.Y., Bloch, G.: A trainable feature extractor for handwritten digit recognition. Pattern Recognit. 40(6), 1816–1824 (2007)CrossRefMATHGoogle Scholar
  32. 32.
    LeCun, Y., Bottou, L., Bengio, Y.: Reading checks with graph transformer networks. In: Proceedings IEEE International Conference on Acoustics, Speech, and Signal Processing vol. 1, 151–154 (1997)Google Scholar
  33. 33.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (November 1998)Google Scholar
  34. 34.
    Liu, C.-L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: benchmarking of the state-of the- art techniques. Pattern Recognit. 36(10), 2271–2285 (2003)CrossRefMATHGoogle Scholar
  35. 35.
    Liu, C.-L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: investigation of normalization and feature extraction techniques. Pattern Recognit. 37, 256–279 (2004)Google Scholar
  36. 36.
    Liu, C.-L., Sako, H.: Class-specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition. Pattern Recognit. 39(4), 669–681 (2006)CrossRefMATHGoogle Scholar
  37. 37.
    Liu, C.L., Suen, C.Y.: A new benchmark on the recognition of handwritten Bangla and Farsi numeral characters. Pattern Recognit. 42, 3287–3295 (2008)CrossRefGoogle Scholar
  38. 38.
    Marc’Aurelio, R., Poultney, C., Chopra, S., LeCun, Y.: Efficient learning of sparse representations with an energy based model. In MIT. Press, editor, Proceedings Advances in Neural Information Processing Systems, (2006)Google Scholar
  39. 39.
    Hosseini, H.M.M., Bouzerdoum, A.: A Combined Method for Persian and Arabic Handwritten Digit Recognition, Australian New Zealand Conference on Intelligent Information System, pp. 80–83 (1996)Google Scholar
  40. 40.
    Mori, S., Suen, C.Y, Yamamoto, K: “Historical review of OCR research and developing”. In: Proceedings IEEE vol. 80, July, pp. 1029–1058 (1992)Google Scholar
  41. 41.
    Mori, S., Nishida, H., Yamada, H.: Optical Character Recognition. Wiley, New York (1999)Google Scholar
  42. 42.
    Mowlaei, A., Faez, K.: Recognition of isolated handwritten Persian/Arabic characters and numerals using support vector machines. In: Proceedings IEEE 13th Workshop on Neural Networks for Signal Processing, pp. 547–554 (2003)Google Scholar
  43. 43.
    Mowlaei, A., Faez, K., Haghighat, A.: Feature extraction with wavelet transform for recognition of isolated handwritten Farsi/Arabic characters and numerals. Digit. Signal Process. 2, 923–926 (2002)Google Scholar
  44. 44.
    Mozaffari, S., Faez, K., Ziaratban, M.: “Structural Decomposition and Statistical description of Farsi/Arabic handwritten numeric characters. In: Proceedings of the 8th International Conference on Document Analysis and Recognition vol. 1, 237–241 (2005)Google Scholar
  45. 45.
    Nabavi-Karizi, SH., Abadi, M., Kabir, E.: A PSO-based weighting method for linear combination of neural networks. Comput. Electr. Eng. doi:10.1016/j.comeleceng2008.04.006 (2008)
  46. 46.
    OToole, A.J., Abdi, H.: Low-dimensional representation of faces in higher dimensions of the face space. Opt. Soc. Am. 10, 411 (1993)Google Scholar
  47. 47.
    Pan, W.M., Bui, T.D., Suen, C.Y.: Isolated handwritten Farsi numerals recognition using sparse and over-complete representations. In: 10th International Conference on Document Analysis and Recognition. 586–590 (2009)Google Scholar
  48. 48.
    Sanguansat, P.: Two-dimensional principal component analysis and its extensions, principal component analysis. In: Sanguansat, P. (ed.), ISBN: 978-953-51-0195-6, InTech, Available from: http://www.intechopen.com/books/principal-component-analysis/2dpca-and-its-extensions (2012)
  49. 49.
    Shi, Y.: Particle swarm optimization. IEEE Connect. 2(1), 8–13 (2004)Google Scholar
  50. 50.
    Rajashekararadhya, S.V., Ranjan, P.V., ManjunathAradhya, V.N.:“Isolated handwritten Kannada and Tamil numeral recognition: a novel approach”, First International Conference on Emerging Trends in Engineering and Technology ICETET, No. 8, pp. 1192–1195 (2008)Google Scholar
  51. 51.
    Raudys, S.J., Pikelis, V.: On dimensionality, sample size, classification error, and complexity of classification algorithms in pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2, 243–251 (1980)Google Scholar
  52. 52.
    Raudys, S.J., Jain, A.K.: Small sample size effects in statistical pattern recognition: recommendations for practitioners. IEEE Trans. Pattern Anal. Mach. Intell. 13(3), 252–264 (1991)CrossRefGoogle Scholar
  53. 53.
    Sadri, J., Suen, C.Y., Bui, T.D.: “Application of support vector machines for recognition of handwritten Arabic/Persian digits”. In: Proceedings of the 2nd Conference on Machine Vision and Image Processing & Applications. vol. 1, pp. 300–307 (2003)Google Scholar
  54. 54.
    Sadri, J., Suen, C.Y., Bui, T.D.: “Application of support vector machines for recognition of handwritten Arabic/Persian digits. In: Proceedings of the 2nd Conference on Machine Vision and Image Processing & Applications. vol. 1, 300–307 (2003)Google Scholar
  55. 55.
    Savas, B., Elden, L.: Handwritten digit classification using higher order singular value decomposition. Pattern Recognit. 40, 993–1003 (2007)CrossRefMATHGoogle Scholar
  56. 56.
    Shi, M., Fujisawa, Y., Wakabayashi, T., Kimura, F.: Handwritten numeral recognition using gradient and curvature of gray scale image. Pattern Recognit. 35(10), 2051–2059 (2002)Google Scholar
  57. 57.
    Shirali-Shahreza, M.H., Faez, K., Khotanzad, A.: Recognition of hand-written Persian/Arabic numerals by shadow coding and an edited probabilistic neural network. Proc. Int. Conf. Image Process. 3, 436–439 (1995)Google Scholar
  58. 58.
    Simard, P.Y., Steinkraus, D., Platt, J. : Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings International Conference on Document Analysis and Recognition (ICDAR), pp. 958–962, (2003)Google Scholar
  59. 59.
    Simard, P.Y., LeCun, Y.A., Denker, J.S., Victorri, B.: Transformation invariance in pattern recognition tangent distance and tangent propagation. Intern. J. Imag. Syst. Technol. 11(3), 181–197 (2000)Google Scholar
  60. 60.
    Sirovich, L., Kirby, M.: Low-dimensional procedure for characterization of human faces. J. Opt. Soc. Am. 4, 519–524 (1987)Google Scholar
  61. 61.
    Soltanzadeh, H., Rahmati, M.: Recognition of persian handwritten digits using image profiles of multiple orientations. Pattern Recognit. Lett. 25(14), 1569–1576 (2004)Google Scholar
  62. 62.
    Srihari, S., Keubert, E.: Integration of handwritten address interpretation technology into the United States Postal Service remote computer reader system. In: Proceedings Fourth International Conference on Document Analysis and Recognition, vol. 2, 892–896 (1997)Google Scholar
  63. 63.
    Suen, C., Liu, K., Strathy, N.: Sorting and recognizing cheques and financial documents. In: Proceedings of third IAPR workshop on document analysis systems, pp 1–18, (1998)Google Scholar
  64. 64.
    Tan, X., Chen, S., Zhou, Z., Zhang, F.: Face recognition from a single image perperson: a survey. Pattern Recognit. 39, 1725–1745 (2006)CrossRefMATHGoogle Scholar
  65. 65.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)CrossRefGoogle Scholar
  66. 66.
    Wshah, S., Shi, Z., Govindaraju, V.: “Segmentation of Arabic handwriting based on both contour and skeleton segmentation”, 10th International Conference on Document Analysis and Recognition, (2009)Google Scholar
  67. 67.
    Yang, J., Zhang, D., Frangi, A.F., Yang, J.Y.: “Two-dimensional PCA: a new approach to appearance-based face representation and recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 131–137 (2004) Google Scholar
  68. 68.
    Zhang, L., Tjondronegoro, D.: Selecting, optimizing and fusing ’Salient’ gabor features for facial expression recognition. In: Leung, C.S., Lee, M., Chan, J.H. (eds.) Neural Information Processing. LNCS, vol. 5863, pp. 724–732. Springer, Heidelberg (2009)Google Scholar
  69. 69.
    Zhang, D., Zhou, Z.-H.: (2D)\(^{2}\)PCA: 2-directional 2-dimensional PCA for efficient face representation and recognition. Neurocomputing 69(1–3), 224–231 (2005)CrossRefGoogle Scholar
  70. 70.
    Zhang, P., Bui, T., Suen, C.: A novel cascade ensemble classifier system with a high recognition performance on handwritten digits. Pattern Recognit. 40(12), 3415–3429 (2007)CrossRefMATHGoogle Scholar
  71. 71.
    Ziaratban, M., Faez, K., Faradji, F.: “Language-based feature extraction using template-matching in Farsi/Arabic handwritten numeral recognition. In: Proceedings of 9th International Conference on Document Analysis and Recognition. vol. 1, 297–301 (2007)Google Scholar
  72. 72.
    Ziaratbanv, M., Faez, K., Faradji, F.: Language-based feature extraction using template-matching in Farsi/Arabichandwritten numeral recognition. In Proceedings Ninth International Conference on Document Analysis and Recognition, vol. 1, pp. 297–301, (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.School of Mathematics and Computer ScienceUniversity of TehranTehranIran
  2. 2.Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations