A Framework for Analytical Approaches to Combine Interpretable Models
Analytic approaches to combine interpretable models, although presented in different contexts, can be generalized to highlight the components that can be specialized. We propose a framework that structures the combination process, formalizes the problems that can be solved in alternative ways and evaluates the combined models based on their predictive ability to replace the base ones, without loss of interpretability. The framework is illustrated with a case study using data from the University of Porto, Portugal, where experiments were carried out. The results show that grouping base models by scientific areas, ordering by the number of variables and intersecting their underlying rules creates conditions for the combined models to outperform them.
KeywordsKnowledge generalization Interpretable models Prediction of performance Decision tree merging C5.0
This work is funded by projects “NORTE-07-0124-FEDER-000059” and “NORTE-07-0124-FEDER-000057”, financed by the North Portugal Regional Operational Programme (ON.2 - O Novo Norte), under the National Strategic Reference Framework (NSRF), through the European Regional Development Fund (ERDF), and by national funds, through the Portuguese funding agency, Fundação para a Ciência e a Tecnologia (FCT).
- 1.Andrzejak, A., Langner, F., Zabala, S.: Interpretable models from distributed data via merging of decision trees. In: Proceedings of the 2013 IEEE Symposium on Computational Intelligence and Data Mining. IEEE (2013)Google Scholar
- 2.Bursteinas, B., Long, J.: Merging distributed classifiers. In: Proceedings of the 5th World Multiconference on Systemics, Cybernetics and Informatics (2001)Google Scholar
- 5.Hall, L., Chawla, N., Bowyer, K.: Combining decision trees learned in parallel. In: Working Notes of the KDD-97 Workshop on Distributed Data Mining, pp. 10–15 (1998)Google Scholar
- 8.Kohavi, R., Quinlan, R.: Data mining tasks and methods: classification: decision-tree discovery. In: Handbook of Data Mining and Knowledge Discovery, pp. 267–276. Oxford University Press Inc., New York (1999)Google Scholar
- 9.Kuhn, M., Weston, S., Coulter, N., Quinlan, J.: C50: C5.0 decision trees and rule-based models. R package version 0.1.0-16 (2014)Google Scholar
- 13.Provost, F.J., Hennessy, D.N.: Scaling up: distributed machine learning with cooperation. In: Proceedings of the 13th National Conference on Artificial Intelligence, pp. 74–79 (1996)Google Scholar
- 14.Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
- 18.Williams, G.: Inducing and combining multiple decision trees. Ph.D. thesis, Australian National University (1990)Google Scholar