Classification by Cluster Analysis: A New Meta-Learning Based Approach

  • Anna Jurek
  • Yaxin Bi
  • Shengli Wu
  • Chris Nugent
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6713)


Combination of multiple classifiers, commonly referred to as an classifier ensemble, has previously demonstrated the ability to improve classification accuracy in many application domains. One popular approach to building such a combination of classifiers is known as stacking and is based on a meta-learning approach. In this work we investigate a modified version of stacking based on cluster analysis. Instances from a validation set are firstly classified by all base classifiers. The classified results are then grouped into a number of clusters. Two instances are considered as being similar if they are correctly/incorrectly classified to the same class by the same group of classifiers. When classifying a new instance, the approach attempts to find the cluster to which it is closest. The method outperformed individual classifiers, classification by a clustering method and the majority voting method.


Combining Classifiers Stacking Ensembles Clustering Meta-Learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bi, Y., McClean, S., Anderson, T.: On Combining Multiple Classifiers Using an Evidential Approach. Artificial Intelligence, 324–329 (2006)Google Scholar
  2. 2.
    Danesh, A., Moshiri, B., Fatemi, O.: Improve Text Classification Accuracy based on Classifier Fusion Methods. Information Fusion pp. 1–6 (2007)Google Scholar
  3. 3.
    Dzeroski, S., Zenko, B.: Is Combining Classifiers with Stacking Better than Selecting the Best One? Machine Learning, 255–273 (2004)Google Scholar
  4. 4.
    Furnkranz, J.: Pairwise Classification as an Ensemble Technique. In: 13th European Conference on Machine Learning, pp. 97–110. Springer, London (2002)Google Scholar
  5. 5.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA Data Mining Software: An Update SIGKDD Explorations, vol. 11(1) (2009)Google Scholar
  6. 6.
    Kuncheva, L.I., Whitaker, C.J.: Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy. Machine Learning, 181–207 (2004)Google Scholar
  7. 7.
    Menahem, E., Rokach, L., Elovici, Y.: Troika - An improved stacking schema for classification tasks. Information Sciences, 4097–4122 (2009)Google Scholar
  8. 8.
    Rodriguez, J., Maudes, J.: Boosting recombined weak classifiers. Pattern Recognition Letters, 1049–1059 (2008)Google Scholar
  9. 9.
    Rokach, L.: Ensemble-based classifiers. Artificial Intelligence Review, 1–39 (2010)Google Scholar
  10. 10.
    Seewald, A.K.: How to Make Stacking Better and Faster While Also Taking Care of an Unknown Weakness. In: 19th International Conference on Machine Learning, pp. 554–561. Morgan Kaufmann Publishers, San Francisco (2002)Google Scholar
  11. 11.
    Todorovski, L., Dzeroski, S.: Combining Multiple Models with Meta Decision Trees. In: 4th European Conference on Principles of Data Mining and Knowledge Discovery, pp. 54–64. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  12. 12.
    Ting, K.M., Witten, I.F.: Issues in stacked generalization. Artificaial Intelligence Research, 271–289 (1999)Google Scholar
  13. 13.
    UC Irvine Machine Learning Repository,
  14. 14.
    Zenko, B., Todorovski, L., Dzeroski, S.: A comparison of stacking with MDTs to bagging, boosting, and other stacking methods. In: Int. Conference on Data Mining, pp. 669–670 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Anna Jurek
    • 1
  • Yaxin Bi
    • 1
  • Shengli Wu
    • 1
  • Chris Nugent
    • 1
  1. 1.School of Computing and MathematicsUniversity of UlsterNewtownabbeyUK

Personalised recommendations