Advertisement

A Novel Pattern Rejection Criterion Based on Multiple Classifiers

  • Wei-Na Wang
  • Xu-Yao Zhang
  • Ching Y. Suen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7872)

Abstract

Aiming at improving the reliability of a recognition system, this paper presents a novel SVM-based rejection measurement (SVMM) and voting based combination methods of multiple classifier system (MCS) for pattern rejection. Compared with the previous heuristic designed criteria, SVMM is more straight-forward and can make use of much more information from the training data. The voting based combination methods for rejection is a preliminary attempt to adopt MCS for rejection. Comparison of SVMM with other well-known rejection criteria proves that it achieves the highest performance. Two different methods (structural modification and dataset re-sampling) are used to build MCSs. The basic classifier is the convolution neural network (CNN) which has achieved promising performances in numerous applications. Rejection based on MCS is then evaluated on MNIST and CENPARMI digit databases. Specifically, different rejection criteria (FRM, FTRM and SVMM) are individually combined with MCS for pattern rejection. Experimental results indicate that these combinations improve the rejection performance consistently and MCS built by dataset re-sampling works better than that with structural modification in rejection.

Keywords

Rejection criterion SVMM MCS CNN soft voting handwritten digit recognition 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lauer, F., Suen, C.Y., Bloch, G.: A trainable feature extractor for handwritten digit recognition. Pattern Recognition 40(6), 1816–1824 (2007)zbMATHCrossRefGoogle Scholar
  2. 2.
    Zhang, P., Bui, T.D., Suen, C.Y.: A novel cascade ensemble classifier system with a high recognition performance on handwritten digits. Pattern Recognition 40(12), 3415–3429 (2007)zbMATHCrossRefGoogle Scholar
  3. 3.
    Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practice for convolutional neural networks applied to visual document analysis. In: Int’l Conf. on Document Analysis and Recognition, pp. 958–962 (2003)Google Scholar
  4. 4.
    Niu, X.X., Suen, C.Y.: A novel hybrid CNN-SVM classifier for recognition handwritten digits. Pattern Recognition 45(4), 1318–1325 (2012)CrossRefGoogle Scholar
  5. 5.
    Liu, C.L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: investigation of normalization and feature extractor techniques. Pattern Recognition 37(2), 265–279 (2004)zbMATHCrossRefGoogle Scholar
  6. 6.
    He, C.L., Lam, L., Suen, C.Y.: A novel rejection measurement in handwritten numeral recognition on linear discriminant analysis. In: Int’l Conf. on Document Analysis and Recognition, pp. 451–455 (2009)Google Scholar
  7. 7.
    He, C.L., Lam, L., Suen, C.Y.: Rejection measurement based on linear discriminant analysis for document recognition. Int’l J. of Document Analysis and Recognition 14(3), 263–272 (2011)CrossRefGoogle Scholar
  8. 8.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machine. Journal ACM Transactions on Intelligent Systems and Technology 2(3), 27:1–27:27 (2011)CrossRefGoogle Scholar
  9. 9.
    Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. In: Int’l Conf. on Document Analysis and Recognition, pp. 1135–1139 (2011)Google Scholar
  10. 10.
    Wu, C., Fan, W., He, Y., Sun, J., Naoi, S.: Cascaded heterogeneous convolution neural networks for handwritten digit recognition. In: Int’l Conf. on Pattern Recognition, pp. 657–660 (2012)Google Scholar
  11. 11.
    Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Int’l Conf. Machine Learning, pp. 148–156 (1996)Google Scholar
  13. 13.
    Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  14. 14.
    Zhang, P., Bui, T.D., Suen, C.Y.: A novel cascade ensemble classifier system with a high recognition performance on handwritten digits. Pattern Recognition 40(12), 3415–3429 (2007)zbMATHCrossRefGoogle Scholar
  15. 15.
    He, C.L., Suen, C.Y.: A hybrid multiple classifier system of unconstrained handwritten numeral recognition. Pattern Recognition and Image Analysis 17(4), 608–611 (2007)CrossRefGoogle Scholar
  16. 16.
    Suen, C.Y., Lam, L.: Multiple classifier combination methodologies for different output levels. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 52–66. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  17. 17.
    Lam, L., Suen, C.Y.: Optimal combinations of pattern classifiers. Pattern Recognition Letters 16(9), 945–954 (1995)CrossRefGoogle Scholar
  18. 18.
    Suen, C.Y., et al.: Computer recognition of unconstrained handwritten numerals. Proc. IEEE 80(7), 1162–1180 (1992)CrossRefGoogle Scholar
  19. 19.
    LeCun, Y.: The MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist
  20. 20.
    Yuan, A., Bai, G., Jiao, L., Liu, Y.: Offline handwritten English character recognition based on convolutional neural network. In: Int’l Workshop Document Analysis Systems, pp. 125–129 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Wei-Na Wang
    • 1
  • Xu-Yao Zhang
    • 2
  • Ching Y. Suen
    • 1
  1. 1.CENPARMIConcordia UniversityMontrealCanada
  2. 2.Institute of AutomationChinese Academy of SciencesBeijingChina

Personalised recommendations