Classifier Ensembles for Vector Space Embedding of Graphs

  • Kaspar Riesen
  • Horst Bunke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4472)


Classifier ensembles aim at a more accurate classification than single classifiers. Different approaches to building classifier ensembles have been proposed in the statistical pattern recognition literature. However, in structural pattern recognition, classifier ensembles have been rarely used. In this paper we introduce a general methodology for creating structural classifier ensembles. Our representation formalism is based on graphs and includes strings and trees as special cases. In the proposed approach we make use of graph embedding in real vector spaces by means of prototype selection. Since we use randomized prototype selection, it is possible to generate n different vector sets out of the same underlying graph set. Thus, one can train an individual base classifier for each vector set und combine the results of the classifiers in an appropriate way. We use extended support vector machines for classification and combine them by means of three different methods. In experiments on semi-artificial and real data we show that it is possible to outperform the classification accuracy obtained by single classifier systems in the original graph domain as well as in the embedding vector spaces.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kuncheva, L.: Combining Pattern Classifiers: Methods and Algorithms. John Wiley, Chichester (2004)MATHGoogle Scholar
  2. 2.
    Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)MathSciNetMATHGoogle Scholar
  3. 3.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. on Pattern Analysis ans Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  4. 4.
    Freund, Y., Shapire, R.E.: A decision theoretic generalization of online learning and application to boosting. Journal of Computer and Systems Sciences 55, 119–139 (1997)CrossRefMATHGoogle Scholar
  5. 5.
    Conte, D., et al.: Thirty years of graph matching in pattern recognition. Int. Journal of Pattern Recognition and Artificial Intelligence 18(3), 265–298 (2004)CrossRefGoogle Scholar
  6. 6.
    Bianchini, M., et al.: Recursive processing of cyclic graphs. IEEE Transactions on Neural Networks 17(1), 10–18 (2006)CrossRefGoogle Scholar
  7. 7.
    Neuhaus, M., Bunke, H.: Edit distance based kernel functions for structural pattern classification. In: Pattern Recognition, pp. 1852–1863 (2006)Google Scholar
  8. 8.
    Gärtner, T., Lloyd, J., Flach, P.: Kernels and distances for structured data. Machine Learning 57(3), 205–232 (2004)CrossRefMATHGoogle Scholar
  9. 9.
    Marcialis, G.L., Roli, F., Serrau, A.: Fusion of statistical and structural fingerprint classifiers. In: Kittler, J., Nixon, M.S. (eds.) AVBPA 2003. LNCS, vol. 2688, pp. 310–317. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. 10.
    Neuhaus, M., Bunke, H.: Graph-based multiple classifier systems – a data level fusion approach. In: Roli, F., Vitulano, S. (eds.) ICIAP 2005. LNCS, vol. 3617, pp. 479–487. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Schenker, A., et al.: Building graph-based classifier ensembles by random node selection. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 214–222. Springer, Heidelberg (2004)Google Scholar
  12. 12.
    Pekalska, E., Duin, R., Paclik, P.: Prototype selection for dissimilarity-based classifiers. Pattern Recognition 39(2), 189–208 (2006)CrossRefMATHGoogle Scholar
  13. 13.
    Spillmann, B., et al.: Transforming strings to vector spaces using prototype selection. In: Yeung, D.-Y., et al. (eds.) SSPR 2006 and SPR 2006. LNCS, vol. 4109, pp. 287–296. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Riesen, K., Neuhaus, M., Bunke, H.: Graph embedding in vector spaces by means of prototype selection. Submitted.Google Scholar
  15. 15.
    Wilson, R.C., Hancock, E.R., Luo, B.: Pattern vectors from algebraic graph theory. IEEE Trans. on Pattern Analysis ans Machine Intelligence 27(7), 1112–1124 (2005)CrossRefGoogle Scholar
  16. 16.
    Robles-Kelly, A., Hancock, E.R.: A riemannian approach to graph embedding. Pattern Recognition 40, 1024–1056 (2007)Google Scholar
  17. 17.
    Sanfeliu, A., Fu, K.S.: A distance measure between attributed relational graphs for pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics (Part B) 13(3), 353–363 (1983)MATHGoogle Scholar
  18. 18.
    Bunke, H., Allermann, G.: Inexact graph matching for structural pattern recognition. Pattern Recognition Letters 1, 245–253 (1983)CrossRefMATHGoogle Scholar
  19. 19.
    Duin, R., Pekalska, E.: The Dissimilarity Representations for Pattern Recognition: Foundations and Applications. World Scientific, Singapore (2005)Google Scholar
  20. 20.
    Burges, C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2(2), 121–167 (1998)CrossRefGoogle Scholar
  21. 21.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar
  22. 22.
    Chang, C.C., Lin, C.J.: LIBSVM: A Library for Support Vector Machines (2001), Software available at
  23. 23.
    Wu, C.F.J., Lin, C.J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 5, 975–1005 (2004)MathSciNetGoogle Scholar
  24. 24.
    Kittler, J., et al.: On combining classifiers. IEEE Trans. on Pattern Analysis ans Machine Intelligence 20(3), 226–239 (1998)CrossRefGoogle Scholar
  25. 25.
    Pudil, P., Novovicova, J., Kittler, J.: Floating search methods in feature-selection. PRL 15(11), 1119–1125 (1994)Google Scholar
  26. 26.
    Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  27. 27.
    Le Saux, B., Bunke, H.: Feature selection for graph-based image classifiers. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3523, pp. 147–154. Springer, Heidelberg (2005)Google Scholar
  28. 28.
    Watson, C.I., Wilson, C.L.: NIST special database 4, fingerprint database. National Institute of Standards and Technology (March 1992)Google Scholar
  29. 29.
    Neuhaus, M., Bunke, H.: A graph matching based approach to fingerprint classification using directional variance. In: Kanade, T., Jain, A., Ratha, N.K. (eds.) AVBPA 2005. LNCS, vol. 3546, pp. 191–200. Springer, Heidelberg (2005)Google Scholar
  30. 30.
    Development Therapeutics Program DTP. Aids antiviral screen (2004),

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Kaspar Riesen
    • 1
  • Horst Bunke
    • 1
  1. 1.Department of Computer Science, University of Bern, Neubrückstrasse 10, CH-3012 BernSwitzerland

Personalised recommendations