Improving Accuracy and Speed of Optimum-Path Forest Classifier Using Combination of Disjoint Training Subsets

  • Moacir P. PontiJr.
  • João P. Papa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6713)


The Optimum-Path Forest (OPF) classifier is a recent and promising method for pattern recognition, with a fast training algorithm and good accuracy results. Therefore, the investigation of a combining method for this kind of classifier can be important for many applications. In this paper we report a fast method to combine OPF-based classifiers trained with disjoint training subsets. Given a fixed number of subsets, the algorithm chooses random samples, without replacement, from the original training set. Each subset accuracy is improved by a learning procedure. The final decision is given by majority vote. Experiments with simulated and real data sets showed that the proposed combining method is more efficient and effective than naive approach provided some conditions. It was also showed that OPF training step runs faster for a series of small subsets than for the whole training set. The combining scheme was also designed to support parallel or distributed processing, speeding up the procedure even more.


Optimum-Path Forest classifier distributed combination of classifiers pasting small votes 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L.: Bagging predictors. Machine Learning Journal 2(24), 123–140 (1996)zbMATHGoogle Scholar
  2. 2.
    Breiman, L.: Pasting small votes for classification in large databases and on-line. Machine Learning 36, 85–103 (1999)CrossRefGoogle Scholar
  3. 3.
    Breve, F.A., Ponti Jr., M.P., Mascarenhas, N.D.A.: Multilayer perceptron classifier combination for identification of materials on noisy soil science multispectral images. In: XX Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2007), pp. 239–244. IEEE, Belo Horizonte (2007)CrossRefGoogle Scholar
  4. 4.
    Brown, G., Kuncheva, L.I.: “Good” and “Bad” Diversity in Majority Vote Ensembles. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 124–133. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Chawla, N.V., Hall, L.O., Bowyer, K.W., Moore Jr., T.E.: Distributed pasting of small votes. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 52–62. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010),
  7. 7.
    Freund, T.: Boosting: a weak learning algorithm by majority. Information and Computation 121(2), 256–285 (1995)CrossRefzbMATHGoogle Scholar
  8. 8.
    Ho, T.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  9. 9.
    Kuncheva, L., Whitaker, C., Shipp, C.A., Duin, R.: Limits on the majority vote accuracy in classifier fusion. Pattern Analysis and Applications 6, 22–31 (2003)CrossRefzbMATHGoogle Scholar
  10. 10.
    Lee, W.J., Duin, R.: A labelled graph based multiple classifier system. In: Benediktsson, J.A., Kittler, J., Roli, F. (eds.) MCS 2009. LNCS, vol. 5519, pp. 201–210. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Li, J., Wang, J.Z.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans Pattern Analysis and Machine Intelligence 25(9), 1075–1088 (2003)CrossRefGoogle Scholar
  12. 12.
    Papa, J.P., Falcão, A.X., Suzuki, C.T.N.: LibOPF: a library for optimum-path forest (OPF) classifiers. (2009),
  13. 13.
    Papa, J.P., Falcão, A.X., Suzuki, C.T.N.: Supervised pattern classification based on optimum-path forest. Int. J. Imaging Systems and Technology 19(2), 120–131 (2009)CrossRefGoogle Scholar
  14. 14.
    Schenker, A., Bunke, H., Last, M., Kandel, A.: Building graph-based classifier ensembles by random node selection. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 214–222. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  15. 15.
    Skurichina, M., Duin, R.P.W.: Bagging, boosting and the random subspace method for linear classifiers. Pattern Analysis and Applications 5, 121–135 (2002)CrossRefzbMATHGoogle Scholar
  16. 16.
    Woods, K., Kegelmeyer Jr., W., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Trans Pattern Analysis and Machine Intelligence 19(4), 405–410 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Moacir P. PontiJr.
    • 1
  • João P. Papa
    • 2
  1. 1.Institute of Mathematical and Computer SciencesUniversity of São Paulo (ICMC/USP)São CarlosBrazil
  2. 2.Department of ComputingUNESP — Univ Estadual PaulistaBauruBrazil

Personalised recommendations