Managing Monotonicity in Classification by a Pruned Random Forest

  • Sergio González
  • Francisco Herrera
  • Salvador García
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9375)

Abstract

In ordinal monotonic classification problems, the class variable should increase according to a subset of explanatory variables. Standard classifiers do not guarantee to produce model that satisfy the monotonicity constraints. Some algorithms have been developed to manage this issue, such as decision trees which have modified the growing and pruning mechanisms. In this contribution we study the suitability of using these mechanisms in the generation of Random Forests. We introduce a simple ensemble pruning mechanism based on the degree of monotonicity. After an exhaustive experimental analysis, we deduce that a Random Forest applied over these problems is able to achieve a slightly better predictive performance than standard algorithms.

Keywords

Monotonic classification Decision tree induction Random forest Ensemble pruning 

References

  1. 1.
    Ben-David, A., Sterling, L., Pao, Y.H.: Learning, classification of monotonic ordinal concepts. Comput. Intell. 5, 45–49 (1989)CrossRefGoogle Scholar
  2. 2.
    Kotłowski, W., Słowiński, R.: On nonparametric ordinal classification with monotonicity constraints. IEEE Trans. Knowl. Data Eng. 25, 2576–2589 (2013)CrossRefGoogle Scholar
  3. 3.
    Rokach, L., Maimon, O.: Data Mining with Decision Trees: Theory and Applications, 2nd edn. World Scientific, Singapore (2014)CrossRefMATHGoogle Scholar
  4. 4.
    Furnkranz, J., Gamberger, D., Lavrac, N.: Foundations of Rule Learning. Springer, Heidelberg (2012)CrossRefMATHGoogle Scholar
  5. 5.
    Wozniak, M., Graña, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Inf. Fusion 16, 3–17 (2014)CrossRefGoogle Scholar
  6. 6.
    Martínez-Muñoz, G., Hernández-Lobato, D., Suárez, A.: An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans. Pattern Anal. Mach. Intell. 31, 245–259 (2009)CrossRefGoogle Scholar
  7. 7.
    Ben-David, A.: Monotonicity maintenance in information-theoretic machine learning algorithms. Mach. Learn. 19, 29–43 (1995)Google Scholar
  8. 8.
    Alcala-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J. Multiple Valued Logic Soft Comput. 17, 255–287 (2011)Google Scholar
  9. 9.
    Duivesteijn, W., Feelders, A.: Nearest neighbour classification with monotonicity constraints. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 301–316. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  10. 10.
    Xia, F., Zhang, W., Li, F., Yang, Y.: Ranking with decision tree. Knowl. Inf. Syst. 17, 381–395 (2008)CrossRefGoogle Scholar
  11. 11.
    Ben-David, A.: Automatic generation of symbolic multiattribute ordinal knowledge-based DSSs: methodology and applications. Decis. Sci. 23, 1357–1372 (1992)CrossRefGoogle Scholar
  12. 12.
    Lievens, S., Baets, B.D., Cao-Van, K.: A probabilistic framework for the design of instance-based supervised ranking algorithms in an ordinal setting. Ann. Operational Res. 163, 115–142 (2008)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Japkowicz, N., Shah, M. (eds.): Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, Cambridge (2011) MATHGoogle Scholar
  14. 14.
    García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf. Sci. 180, 2044–2064 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Sergio González
    • 1
  • Francisco Herrera
    • 1
  • Salvador García
    • 1
  1. 1.Department of Computer Science and Artificial IntelligenceUniversity of GranadaGranadaSpain

Personalised recommendations