Partial Discriminative Training of Neural Networks for Classification of Overlapping Classes

  • Cheng-Lin Liu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5064)


In applications such as character recognition, some classes are heavily overlapped but are not necessarily to be separated. For classification of such overlapping classes, either discriminating between them or merging them into a metaclass does not satisfy. Merging the overlapping classes into a metaclass implies that within-metaclass substitution is considered as correct classification. For such classification problems, I propose a partial discriminative training (PDT) scheme for neural networks, in which, a training pattern of an overlapping class is used as a positive sample of its labeled class, and neither positive nor negative sample for its allied classes (classes overlapping with the labeled class). In experiments of handwritten letter recognition using neural networks and support vector machines, the PDT scheme mostly outperforms cross-training (a scheme for multi-labeled classification), ordinary discriminative training and metaclass classification.


  1. 1.
    Lu, B.-L., Ito, M.: Task decomposition and modular combination based on class relations: a modular neural network for pattern classification. IEEE Trans. Neural Networks 10(5), 1244–1256 (1999)CrossRefGoogle Scholar
  2. 2.
    Podolak, I.T.: Hierarchical classifier with overlapping class groups. Expert Systems with Applications 34(1), 673–682 (2008)CrossRefGoogle Scholar
  3. 3.
    Boutell, M.R., Luo, J., Shen, X., Browm, C.M.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  4. 4.
    Tsoumakas, G., Katakis, I.: Multi-label classification: an overview. Int. J. Data Warehousing and Mining 3(3), 1–13 (2007)Google Scholar
  5. 5.
    Camastra, F., Spinetti, M., Vinciarelli, A.: Offline cursive character challenge: a new benchmark for machine learning and pattern recognition algorithms. In: Proc. 18th ICPR, Hong Kong, pp. 913–916 (2006)Google Scholar
  6. 6.
    Camastra, F.: A SVM-based cursive character recognizer. Pattern Recognition 40(12), 3721–3727 (2007)CrossRefzbMATHGoogle Scholar
  7. 7.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, London (1990)zbMATHGoogle Scholar
  8. 8.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
  9. 9.
    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Knowledge Discovery and Data Mining 2(2), 1–43 (1998)Google Scholar
  10. 10.
    Koerich, A.L.: Unconstrained handwritten character recognition using different classification strategies. In: Gori, M., Marinai, S. (eds.) Proc. 1st IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pp. 52–56 (2003)Google Scholar
  11. 11.
    Blumenstein, M., Liu, X.Y., Verma, B.: An investigation of the modified direction feature for cursive character recognition. Pattern Recognition 40(2), 376–388 (2007)CrossRefzbMATHGoogle Scholar
  12. 12.
    Shürmann, J.: Pattern Classification: A Unified View of Statistical and Neural Approaches. Wiley Interscience, Chichester (1996)Google Scholar
  13. 13.
    Kreßel, U., Schürmann, J.: Pattern classification techniques based on function approximation. In: Bunke, H., Wang, P.S.P. (eds.) Handbook of Character Recognition and Document Image Analysis, pp. 49–78. World Scientific, Singapore (1997)Google Scholar
  14. 14.
    Liu, C.-L., Sako, H.: Class-specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition. Pattern Recognition 39(4), 669–681 (2006)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Cheng-Lin Liu
    • 1
  1. 1.National Laboratory of Pattern Recognition (NLPR) Institute of AutomationChinese Academy of SciencesBeijingP.R. China

Personalised recommendations