Distant diversity in dynamic class prediction
- 58 Downloads
Instead of using the same ensemble for all data instances, recent studies have focused on dynamic ensembles in which a new ensemble is chosen from a pool of classifiers for each new data instance. Classifiers agreement in the region where a new data instance resides in has been considered as a major factor in dynamic ensembles. We postulate that the classifiers chosen for a dynamic ensemble should behave similarly in the region in which the new instance resides, but differently outside of this area. In other words, we hypothesize that high local accuracy, combined with high diversity in other regions, is desirable. To verify the validity of this hypothesis we propose two approaches. The first approach focuses on finding the k-nearest data instances to the new instance, which then defines a neighborhood, and maximizes simultaneously local accuracy and distant diversity, based on data instances outside of the neighborhood. The second method makes use of an alternative definition of the neighborhood: all data instances are in the neighborhood. However, the importance of data instances for accuracy and diversity depends on the distance to the new instance. We demonstrate through several experiments that the distance-based diversity and accuracy outperform all benchmark methods.
KeywordsDynamic ensemble Classification Diversity Local accuracy
We presented this study at the INFORMS 2015 Data Mining & Analytics Workshop and got invited by the co-chairs of the workshop to submit to this special issue.
- Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.Google Scholar
- Brown, G., & Kuncheva, L. I. (2010). “Good” and “bad” diversity in majority vote ensembles. In N. Gayar, J. Kittler, & F. Roli (eds.), Multiple classifier systems, lecture notes in computer science (Vol. 5997, pp. 124–133). Berlin, Heidelberg: Springer. doi: 10.1007/978-3-642-12127-2-13.
- Cavalin, P. R., Sabourin, R., & Suen, C. Y. (2010). Dynamic selection of ensembles of classifiers using contextual information. In N. Gayar, J. Kittler, & F. Roli (Eds.), Multiple classifier systems, lecture notes in computer science (Vol. 5997, pp. 145–154). Berlin, Heidelberg: Springer. doi: 10.1007/978-3-642-12127-2-15.
- Chang, C. C., & Lin, C. J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.Google Scholar
- Didaci, L., & Giacinto, G. (2004). Dynamic classifier selection by adaptive K-nearest-neighbourhood rule. In F. Roli, J. Kittler & T. Windeatt (Eds.), Multiple classifier systems: Proceedings of the 5th International workshop, MCS 2004, Cagliari, Italy, June 9–11 (pp. 174–183). Berlin: Springer.Google Scholar
- Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm. In International workshop on machine learning (Vol. 96, pp. 148–156). Morgan Kaufmann.Google Scholar
- Hsu, K. W., & Srivastava, J. (2010). Relationship between diversity and correlation in multi-classifier systems. In M. J. Zaki, J. X. Yu, B. Ravindran, & V. Pudi (Eds.), Advances in knowledge discovery and data mining, lecture notes in computer science (Vol. 6119, pp. 500–506). Berlin, Heidelberg: Springer. doi: 10.1007/978-3-642-13672-6-47.
- Kohavi, R., & Wolpert, D. H. (1996). Bias plus variance decomposition for zero-one loss functions. In Machine learning: Proceedings of the thirteenth international (pp. 275–283).Google Scholar
- Margineantu, D. D., & Dietterich, T. G. (1997). Pruning adaptive boosting. International Workshop on Machine Learning, 97, 211–218.Google Scholar
- Yaşar Sağlam, C., & Street, W. N. (2014). Dynamic class prediction with classifier based distance measure. In Conferences in research and practice in information technology (CRPIT): Proceedings of The twelfth Australasian data mining conference, ICML-04 (Vol. 158).Google Scholar