Diversity and Locality in Multi-Component, Multi-Layer Predictive Systems: A Mutual Information Based Approach
This paper discusses the effect of locality and diversity among the base models of a Multi-Components Multi-Layer Predictive System (MCMLPS). A new ensemble method is introduced, where in the proposed architecture, the data instances are assigned to local regions using a conditional mutual information based on the similarity of their features. Furthermore, the outputs of the base models are weighted by this similarity metric. The proposed architecture has been tested on a number of data sets and its performance was compared to four benchmark algorithms. Moreover, the effect of changing three parameters of the proposed architecture has been tested and compared.
KeywordsEnsemble diversity Ensemble methods Local learning Conditional mutual information Feature selection
- 12.Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/ml
- 19.Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5(2), 197–227 (1990)Google Scholar
- 21.Xue, F., Subbu, R., Bonissone, P.: Locally weighted fusion of multiple predictive models. In: International Joint Conference on Neural Networks, 2006. IJCNN’06, pp. 2137–2143. IEEE (2006)Google Scholar