Advertisement

Frontiers of Computer Science

, Volume 14, Issue 2, pp 241–258 | Cite as

A survey on ensemble learning

  • Xibin Dong
  • Zhiwen YuEmail author
  • Wenming Cao
  • Yifan Shi
  • Qianli Ma
Review Article
  • 155 Downloads

Abstract

Despite significant successes achieved in knowledge discovery, traditional machine learning methods may fail to obtain satisfactory performances when dealing with complex data, such as imbalanced, high-dimensional, noisy data, etc. The reason behind is that it is difficult for these methods to capture multiple characteristics and underlying structure of data. In this context, it becomes an important topic in the data mining field that how to effectively construct an efficient knowledge discovery and mining model. Ensemble learning, as one research hot spot, aims to integrate data fusion, data modeling, and data mining into a unified framework. Specifically, ensemble learning firstly extracts a set of features with a variety of transformations. Based on these learned features, multiple learning algorithms are utilized to produce weak predictive results. Finally, ensemble learning fuses the informative knowledge from the above results obtained to achieve knowledge discovery and better predictive performance via voting schemes in an adaptive way. In this paper, we review the research progress of the mainstream approaches of ensemble learning and classify them based on different characteristics. In addition, we present challenges and possible research directions for each mainstream approach of ensemble learning, and we also give an extra introduction for the combination of ensemble learning with other machine learning hot spots such as deep learning, reinforcement learning, etc.

Keywords

ensemble learning supervised ensemble classification semi-supervised ensemble classification clustering ensemble semi-supervised clustering ensemble 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

The authors are grateful for the constructive advice received from the anonymous reviewers of this paper. The work described in this paper was partially funded by grants from the National Natural Science Foundation of China (Grant Nos. 61722205, 61751205, 61572199, 61502174, 61872148, and U1611461), the grant from the key research and development program of Guangdong province of China (2018B010107002), the grants from Science and Technology Planning Project of Guangdong Province, China (2016A050503015, 2017A030313355), and the grant from the Guangzhou science and technology planning project (201704030051).

Supplementary material

11704_2019_8208_MOESM1_ESM.pdf (610 kb)
A survey on ensemble learning

References

  1. 1.
    Zhou Z H. Ensemble Methods: Foundations and Algorithms. Chapman and Hall/CRC, 2012Google Scholar
  2. 2.
    Dasarathy B V, Sheela B V. A composite classifier system design: concepts and methodology. Proceedings of the IEEE, 1979, 67(5): 708–713Google Scholar
  3. 3.
    Kearns M. Learning boolean formulae or finite automata is as hard as factoring. Technical Report TR-14-88 Harvard University Aikem Computation Laboratory, 1988Google Scholar
  4. 4.
    Schapire, Robert E. The strength of weak learnability. Machine Learning, 1990, 5(2): 197–227Google Scholar
  5. 5.
    Breiman L. Bagging predictors. Machine Learning, 1996, 24(2): 123–140zbMATHGoogle Scholar
  6. 6.
    Hastie T, Rosset S, Zhu J, Zou H. Multi-class adaboost. Statistics and its Interface, 2009, 2(3): 349–360MathSciNetzbMATHGoogle Scholar
  7. 7.
    Breiman L. Random forests. Machine Learning, 2001, 45(1): 5–32zbMATHGoogle Scholar
  8. 8.
    Ho T K. Random decision forests. In: Proceedings of the 3rd International Conference on Document Analysis and Recognition. 1995, 278–282Google Scholar
  9. 9.
    Friedman J H. Stochastic gradient boosting. Computational Statistics and Data Analysis, 2002, 38(4): 367–378MathSciNetzbMATHGoogle Scholar
  10. 10.
    Garcia-Pedrajas N. Constructing ensembles of classifiers by means of weighted instance selection. IEEE Transactions on Neural Networks, 2009, 20(2): 258–277Google Scholar
  11. 11.
    Garcia-Pedrajas N, Maudes-Raedo J, Garcia-Osorio C, Rodriguez-Díez J J, Linden D E, Johnston SJ. Supervised subspace projections for constructing ensembles of classifiers. Information Sciences, 2012, 193(11): 1–21Google Scholar
  12. 12.
    Kuncheva L I, Rodriguez J J, Plumpton C O, Linden D E, Johnston SJ. Random subspace ensembles for FMRI classification. IEEE Transactions on Medical Imaging, 2010, 29(2): 531–542Google Scholar
  13. 13.
    Ye Y, Wu Q, Huang J Z, Ng M K, Li X. Stratified sampling for feature subspace selection in random forests for high dimensional data. Pattern Recognition, 2013, 46(3): 769–787Google Scholar
  14. 14.
    Bryll R, Gutierrez-Osuna R, Quek F. Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets. Pattern Recognition, 2003, 36(6): 1291–1302zbMATHGoogle Scholar
  15. 15.
    Blum A, Mitchell T. Combining labeled and unlabeled data with co-training. In: Proceedings of the 11th Annual Conference on Computational Learning Theory. 1998, 92–100Google Scholar
  16. 16.
    Wang J, Luo S W, Zeng XH. A random subspace method for co-training. In: Proceedings of 2008 IEEE International Joint Conference on Neural Networks. 2008, 195–200Google Scholar
  17. 17.
    Yaslan Y, Cataltepe Z. Co-training with relevant random subspaces. Neurocomputing, 2010, 73(10–12): 1652–1661Google Scholar
  18. 18.
    Zhang J, Zhang D. A novel ensemble construction method for multi-view data using random cross-view correlation between within-class examples. Pattern Recognition, 2011, 44(6): 1162–1171zbMATHGoogle Scholar
  19. 19.
    Guo Y, Jiao L, Wang S, Liu F, Rong K, Xiong T. A novel dynamic rough subspace based selective ensemble. Pattern Recognition, 2015, 48(5): 1638–1652Google Scholar
  20. 20.
    Windeatt T, Duangsoithong R, Smith R. Embedded feature ranking for ensemble MLP classifiers. IEEE Transactions on Neural Networks, 2011, 22(6): 988–994Google Scholar
  21. 21.
    Rodriguez J J, Kuncheva L I, Alonso CJ. Rotation forest: a new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(10): 1619–1630Google Scholar
  22. 22.
    Takemura A, Shimizu A, Hamamoto K. Discrimination of breast tumors in ultrasonic images using an ensemble classifier based on the AdaBoost algorithm with feature selection. IEEE Transactions on Medical Imaging, 2010, 29(3): 598–609Google Scholar
  23. 23.
    Amasyali M F, Ersoy OK. Classifier ensembles with the extended space forest. IEEE Transactions on Knowledge and Data Engineering, 2013, 26(3): 549–562Google Scholar
  24. 24.
    Polikar R, Depasquale J, Mohammed H S, Brown G, Kuncheva LI. Learn++.MF: a random subspace approach for the missing feature problem. Pattern Recognition, 2010, 43(11): 3817–3832zbMATHGoogle Scholar
  25. 25.
    Nanni L, Lumini A. Evolved feature weighting for random subspace classifier. IEEE Transactions on Neural Networks, 2008, 19(2): 363–366Google Scholar
  26. 26.
    Kennedy J, Eberhart RC. A discrete binary version of the particle swarm optimization algorithm. Computational Cybernatics and Simulation, 1997, 5(1): 4104–4108Google Scholar
  27. 27.
    Zhou Z H, Tang W. Selective ensemble of decision trees. In: Proceedings of International Workshop on Rough Sets, Fuzzy Sets, Data Mining, and Granular-Soft Computing. 2003, 476–483Google Scholar
  28. 28.
    Diao R, Chao F, Peng T, Snooke N, Shen Q. Feature selection inspired classifier ensemble reduction. IEEE Transactions on Cybernetics, 2014, 44(8): 1259–1268Google Scholar
  29. 29.
    Yu Z, Wang D, You J, Wong H S, Wu S, Zhang J, Han G. Progressive subspace ensemble learning. Pattern Recognition, 2016, 60: 692–705Google Scholar
  30. 30.
    Yu Z, Wang D, Zhao Z, Chen C P, You J, Wong H S, Zhang J. Hybrid incremental ensemble learning for noisy real-world data classification. IEEE Transactions on Cybernetics, 2017, 99: 1–14Google Scholar
  31. 31.
    Dos Santos E M, Sabourin R, Maupin P. A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognition, 2008, 41(10): 2993–3009zbMATHGoogle Scholar
  32. 32.
    Hernández-Lobato D, Martínez-Muñoz G, Suárez A. Statistical instance-based pruning in ensembles of independent classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 364–369Google Scholar
  33. 33.
    Martínez-Muñoz G, Hernández-Lobato D, Suárez A. An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 245–259Google Scholar
  34. 34.
    De Stefano C, Folino G, Fontanella F, Di Freca AS. Using bayesian networks for selecting classifiers in GP ensembles. Information Sciences, 2014, 258: 200–216MathSciNetGoogle Scholar
  35. 35.
    Rahman A, Verma B. Novel layered clustering-based approach for generating ensemble of classifiers. IEEE Transactions on Neural Networks, 2011, 22(5): 781–792Google Scholar
  36. 36.
    Verma B, Rahman A. Cluster-oriented ensemble classifier: impact of multicluster characterization on ensemble classifier learning. IEEE Transactions on Knowledge and Data Engineering, 2012, 24(4): 605–618Google Scholar
  37. 37.
    Zhang L, Suganthan PN. Oblique decision tree ensemble via multi-surface proximal support vector machine. IEEE Transactions on Cybernetics, 2015, 45(10): 2165–2176Google Scholar
  38. 38.
    Tan P J, Dowe DL. Decision forests with oblique decision trees. In: Proceedings of Mexican International Conference on Artificial Intelligence. 2006, 593–603Google Scholar
  39. 39.
    Zhou Z H, Wu J, Tang W. Ensembling neural networks: many could be better than all. Artificial Intelligence, 2002, 137(1–2): 239–263MathSciNetzbMATHGoogle Scholar
  40. 40.
    Yu Z, Chen H, Liu J, You J, Leung H, Han G. Hybrid k-nearest neighbor classifier. IEEE Transactions on Cybernetics, 2016, 46(6): 1263–1275Google Scholar
  41. 41.
    Li H, Wen G, Yu Z, Zhou T. Random subspace evidence classifier. Neurocomputing, 2013, 110(13): 62–69Google Scholar
  42. 42.
    Hernández-Lobato D, Martínez-Muñoz G, Suárez A. How large should ensembles of classifiers be? Pattern Recognition, 2013, 46(5): 1323–1336zbMATHGoogle Scholar
  43. 43.
    Wang X Z, Xing H J, Li Y, Hua Q, Dong C R, Pedrycz W. A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Transactions on Fuzzy Systems, 2015, 23(5): 1638–1654Google Scholar
  44. 44.
    Kuncheva LI. A bound on kappa-error diagrams for analysis of classifier ensembles. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(3): 494–501Google Scholar
  45. 45.
    Gao W, Zhou ZH. Approximation stability and boosting. In: Proceedings of International Conference on Algorithmic Learning Theory. 2010, 59–73Google Scholar
  46. 46.
    Yin X C, Huang K, Hao H W, Iqbal K, Wang ZB. A novel classifier ensemble method with sparsity and diversity. Neurocomputing, 2014, 134: 214–221Google Scholar
  47. 47.
    Zhang L, Suganthan PN. Random forests with ensemble of feature spaces. Pattern Recognition, 2014, 47(10): 3429–3437Google Scholar
  48. 48.
    Li N, Yu Y, Zhou ZH. Diversity regularized ensemble pruning. In: Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2012, 330–345Google Scholar
  49. 49.
    Zhang D, Chen S, Zhou Z H, Yang Q. Constraint projections for ensemble learning. In: Proceedings of the 23rd National Conference on Artifical Intelligence-Volume 2. 2008, 758–763Google Scholar
  50. 50.
    Zhou Z H, Li N. Multi-information ensemble diversity. In: Proceedings of International Workshop on Multiple Classifier Systems. 2010, 134–144Google Scholar
  51. 51.
    Sun T, Zhou ZH. Structural diversity for decision tree ensemble learning. Frontiers of Computer Science, 2018, 12(3): 560–570MathSciNetGoogle Scholar
  52. 52.
    Mao S, Jiao L, Xiong L, Gou S, Chen B, Yeung SK. Weighted classifier ensemble based on quadratic form. Pattern Recognition, 2015, 48(5): 1688–1706zbMATHGoogle Scholar
  53. 53.
    Yu Z, Wang Z, You J, Zhang J, Liu J, Wong H S, Han G. A new kind of nonparametric test for statistical comparison of multiple classifiers over multiple datasets. IEEE Transactions on Cybernetics, 2017, 47(12): 4418–4431Google Scholar
  54. 54.
    Kim K J, Cho SB. An evolutionary algorithm approach to optimal ensemble classifiers for DNA microarray data analysis. IEEE Transactions on Evolutionary Computation, 2008, 12(3): 377–388Google Scholar
  55. 55.
    Qian C, Yu Y, Zhou ZH. Pareto ensemble pruning. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015Google Scholar
  56. 56.
    Zhou Z H, Feng J. Deep forest: towards an alternative to deep neural networks. 2017, arXiv preprint arXiv:1702.08835Google Scholar
  57. 57.
    Feng J, Zhou ZH. AutoEncoder by forest. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018Google Scholar
  58. 58.
    Zhang Y L, Zhou J, Zheng W, Feng J, Li L, Liu Z, Zhou ZH. Distributed deep forest and its application to automatic detection of cash-out fraud. 2018, arXiv preprint arXiv:1805.04234Google Scholar
  59. 59.
    Feng J, Yu Y, Zhou ZH. Multi-layered gradient boosting decision trees. In: Proceedings of Advances in Neural Information Processing Systems. 2018, 3555–3565Google Scholar
  60. 60.
    Pang M, Ting K M, Zhao P, Zhou ZH. Improving deep forest by confidence screening. In: Proceedings of the 18th IEEE International Conference on Data Mining. 2018, 1194–1199Google Scholar
  61. 61.
    Yu Z, Li L, Liu J, Han G. Hybrid adaptive classifier ensemble. IEEE Transactions on Cybernetics, 2015, 45(2): 177–190Google Scholar
  62. 62.
    Zhou Z H, Zhang ML. Solving multi-instance problems with classifier ensemble based on constructive clustering. Knowledge and Information Systems, 2007, 11(2): 155–170Google Scholar
  63. 63.
    Zhu X, Zhang P, Lin X, Shi Y. Active learning from stream data using optimal weight classifier ensemble. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2010, 40(6): 1607–1621Google Scholar
  64. 64.
    Brzezinski D, Stefanowski J. Reacting to different types of concept drift: the accuracy updated ensemble algorithm. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(1): 81–94Google Scholar
  65. 65.
    Muhlbaier M D, Topalis A, Polikar R. Learn++.NC: combining ensemble of classifiers with dynamically weighted consult-and-vote for efficient incremental learning of new classes. IEEE Transactions on Neural Networks, 2009, 20(1): 152–168Google Scholar
  66. 66.
    Xiao J, He C, Jiang X, Liu D. A dynamic classifier ensemble selection approach for noise data. Information Sciences, 2010, 180(18): 3402–3421Google Scholar
  67. 67.
    Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F. A review on ensembles for the class imbalance problem: bagging, boosting, and hybrid-based approaches. IEEE Transactions on Systems Man and Cybernetics Part C, 2012, 42(4): 463–484Google Scholar
  68. 68.
    Liu X Y, Wu J, Zhou ZH. Exploratory under-sampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2009, 39(2): 539–550Google Scholar
  69. 69.
    Sun B, Chen H, Wang J, Xie H. Evolutionary under-sampling based bagging ensemble method for imbalanced data classification. Frontiers of Computer Science, 2018, 12(2): 331–350Google Scholar
  70. 70.
    Li Q, Li G, Niu W, Cao Y, Chang L, Tan J, Guo L. Boosting imbal-anced data learning with wiener process oversampling. Frontiers of Computer Science, 2017, 11(5): 836–851zbMATHGoogle Scholar
  71. 71.
    Abawajy J H, Kelarev A, Chowdhury M. Large iterative multitier ensemble classifiers for security of big data. IEEE Transactions on Emerging Topics in Computing, 2014, 2(3): 352–363Google Scholar
  72. 72.
    Li N, Zhou ZH. Selective ensemble of classifier chains. In: Proceedings of International Workshop on Multiple Classifier Systems. 2013, 146–156Google Scholar
  73. 73.
    Li N, Jiang Y, Zhou ZH. Multi-label selective ensemble. In: Proceedings of International Workshop on Multiple Classifier Systems. 2015, 76–88Google Scholar
  74. 74.
    Yu Z, Deng Z, Wong H S, Tan L. Identifying protein-kinase-specific phosphorylation sites based on the Bagging-AdaBoost ensemble approach. IEEE Transactions on Nanobioscience, 2010, 9(2): 132–143Google Scholar
  75. 75.
    Yu D J, Hu J, Yang J, Shen H B, Tang J, Yang JY. Designing template-free predictor for targeting protein-ligand binding sites with classifier ensemble and spatial clustering. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2013, 10(4): 994–1008Google Scholar
  76. 76.
    Yu G, Rangwala H, Domeniconi C, Zhang G, Yu Z. Protein function prediction using multilabel ensemble classification. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2013, 10(4): 1Google Scholar
  77. 77.
    Daliri MR. Combining extreme learning machines using support vector machines for breast tissue classification. Computer Methods in Biomechanics and Biomedical Engineering, 2015, 18(2): 185–191Google Scholar
  78. 78.
    Oliveira L, Nunes U, Peixoto P. On exploration of classifier ensemble synergism in pedestrian detection. IEEE Transactions on Intelligent Transportation Systems, 2010, 11(1): 16–27Google Scholar
  79. 79.
    Xu Y, Cao X, Qiao H. An efficient tree classifier ensemble-based approach for pedestrian detection. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2011, 41(1): 107–117Google Scholar
  80. 80.
    Zhang B. Reliable classification of vehicle types based on cascade classifier ensembles. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(1): 322–332Google Scholar
  81. 81.
    Sun S, Zhang C. The selective random subspace predictor for traffic flow forecasting. IEEE Transactions on Intelligent Transportation Systems, 2007, 8(2): 367–373Google Scholar
  82. 82.
    Su Y, Shan S, Chen X, Gao W. Hierarchical ensemble of global and local classifiers for face recognition. IEEE Transactions on Image Processing, 2009, 18(8): 1885–1896MathSciNetzbMATHGoogle Scholar
  83. 83.
    Zhang P, Bui T D, Suen CY. A novel cascade ensemble classifier system with a high recognition performance on handwritten digits. Pattern Recognition, 2007, 40(12): 3415–3429zbMATHGoogle Scholar
  84. 84.
    Xu X S, Xue X, Zhou ZH. Ensemble multi-instance multi-label learning approach for video annotation task. In: Proceedings of the 19th ACM International Conference on Multimedia. 2011, 1153–1156Google Scholar
  85. 85.
    Hautamaki V, Kinnunen T, Sedlák F, Lee K A, Ma B, Li H. Sparse classifier fusion for speaker verification. IEEE Transactions on Audio Speech and Language Processing, 2013, 21(8): 1622–1631Google Scholar
  86. 86.
    Guan Y, Li C T, Roli F. On reducing the effect of covariate factors in gait recognition: a classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(7): 1521–1528Google Scholar
  87. 87.
    Tao D, Tang X, Li X, Wu X. Asymmetric bagging and random sub-space for support vector machines-based relevance feedback in image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(7): 1088–1099Google Scholar
  88. 88.
    Hu W, Hu W, Maybank S. AdaBoost-based algorithm for network intrusion detection. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008, 38(2): 577–583Google Scholar
  89. 89.
    Zhang P, Zhu X, Shi Y, Guo L, Wu X. Robust ensemble learning for mining noisy data streams. Decision Support Systems, 2011, 50(2): 469–479Google Scholar
  90. 90.
    Yu L, Wang S, Lai KK. Developing an SVM-based ensemble learning system for customer risk identification collaborating with customer relationship management. Frontiers of Computer Science, 2010, 4(2): 196–203Google Scholar
  91. 91.
    Fersini E, Messina E, Pozzi FA. Sentiment analysis: Bayesian ensemble learning. Decision Support Systems, 2014, 68: 26–38Google Scholar
  92. 92.
    Yu G, Zhang G, Yu Z, Domeniconi C, You J, Han G. Semi-supervised ensemble classification in subspaces. Applied Soft Computing, 2012, 12(5): 1511–1522Google Scholar
  93. 93.
    Yu Z, Zhang Y, Chen C L P, You J, Wong H S, Dai D, Wu S, Zhang J. Multiobjective semisupervised classifier ensemble. IEEE Transactions on Cybernetics, 2019, 49(6): 2280–2293Google Scholar
  94. 94.
    Gharroudi O, Elghazel H, Aussem A. A semi-supervised ensemble approach for multi-label learning. In: Proceedings of the 16th IEEE International Conference on Data Mining Workshops (ICDMW). 2016, 1197–1204Google Scholar
  95. 95.
    Lu X, Zhang J, Li T, Zhang Y. Hyperspectral image classification based on semi-supervised rotation forest. Remote Sensing, 2017, 9(9): 924Google Scholar
  96. 96.
    Wang S, Chen K. Ensemble learning with active data selection for semi-supervised pattern classification. In: Proceedings of 2007 International Joint Conference on Neural Networks. 2007, 355–360Google Scholar
  97. 97.
    Soares R G F, Chen H, Yao X. A cluster-based semi-supervised ensemble for multiclass classification. IEEE Transactions on Emerging Topics in Computational Intelligence, 2017, 1(6): 408–420Google Scholar
  98. 98.
    Woo H, Park CH. Semi-supervised ensemble learning using label propagation. In: Proceedings of the 12th IEEE International Conference on Computer and Information Technology. 2012, 421–426Google Scholar
  99. 99.
    Zhang M L, Zhou ZH. Exploiting unlabeled data to enhance ensemble diversity. Data Mining and Knowledge Discovery, 2013, 26(1): 98–129MathSciNetzbMATHGoogle Scholar
  100. 100.
    Alves M, Bazzan A L C, Recamonde-Mendoza M. Social-training: ensemble learning with voting aggregation for semi-supervised classification tasks. In: Proceedings of 2017 Brazilian Conference on Intelligent Systems (BRACIS). 2017, 7–12Google Scholar
  101. 101.
    Yu Z, Lu Y, Zhang J, You J, Wong H S, Wang Y, Han G. Progressive semi-supervised learning of multiple classifiers. IEEE Transactions on Cybernetics, 2018, 48(2): 689–702Google Scholar
  102. 102.
    Hosseini M J, Gholipour A, Beigy H. An ensemble of cluster-based classifiers for semi-supervised classification of non-stationary data streams. Knowledge and Information Systems, 2016, 46(3): 567–597Google Scholar
  103. 103.
    Wang Y, Li T. Improving semi-supervised co-forest algorithm in evolving data streams. Applied Intelligence, 2018, 48(10): 3248–3262Google Scholar
  104. 104.
    Yu Z, Zhang Y, You J, Chen C P, Wong H S, Han G, Zhang J. Adaptive semi-supervised classifier ensemble for high dimensional data classification. IEEE Transactions on Cybernetics, 2019, 49(2): 366–379Google Scholar
  105. 105.
    Li M, Zhou ZH. Improve computer-aided diagnosis with machine learning techniques using undiagnosed samples. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2007, 37(6): 1088–1098Google Scholar
  106. 106.
    Guz U, Cuendet S, Hakkani-Tur D, Tur G. Multi-view semi-supervised learning for dialog act segmentation of speech. IEEE Transactions on Audio Speech and Language Processing, 2010, 18(2): 320–329Google Scholar
  107. 107.
    Shi L, Ma X, Xi L, Duan Q, Zhao J. Rough set and ensemble learning based semi-supervised algorithm for text classification. Expert Systems with Applications, 2011, 38(5): 6300–6306Google Scholar
  108. 108.
    Abdelgayed T S, Morsi W G, Sidhu TS. Fault detection and classification based on co-training of semi-supervised machine learning. IEEE Transactions on Industrial Electronics, 2018, 65(2): 1595–1605Google Scholar
  109. 109.
    Saydali S, Parvin H, Safaei AA. Classifier ensemble by semi-supervised learning: local aggregation methodology. In: Proceedings of International Doctoral Workshop on Mathematical and Engineering Methods in Computer Science. 2015, 119–132Google Scholar
  110. 110.
    Shao W, Tian X. Semi-supervised selective ensemble learning based on distance to model for nonlinear soft sensor development. Neuro-computing, 2017, 222: 91–104Google Scholar
  111. 111.
    Ahmed I, Ali R, Guan D, Lee Y K, Lee S, Chung T. Semi-supervised learning using frequent itemset and ensemble learning for SMS classification. Expert Systems with Applications, 2015, 42(3): 1065–1073Google Scholar
  112. 112.
    Strehl A, Ghosh J. Cluster ensembles: a knowledge reuse framework for combining partitionings. Journal of Machine Learning Research, 2002, 3(3): 583–617zbMATHGoogle Scholar
  113. 113.
    Yang F, Li X, Li Q, Li T. Exploring the diversity in cluster ensemble generation: random sampling and random projection. Expert Systems with Applications, 2014, 41(10): 4844–4866Google Scholar
  114. 114.
    Wu O, Hu W, Maybank S J, Zhu M, Li B. Efficient clustering aggregation based on data fragments. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(3): 913–926Google Scholar
  115. 115.
    Franek L, Jiang X. Ensemble clustering by means of clustering embedding in vector spaces. Pattern Recognition, 2014, 47(2): 833–842zbMATHGoogle Scholar
  116. 116.
    Yu Z, Wong H S, Wang H. Graph-based consensus clustering for class discovery from gene expression data. Bioinformatics, 2007, 23(21): 2888–2896Google Scholar
  117. 117.
    Yu Z, Wong H S, You J, Yu G, Han G. Hybrid cluster ensemble framework based on the random combination of data transformation operators. Pattern Recognition, 2012, 45(5): 1826–1837zbMATHGoogle Scholar
  118. 118.
    Yu Z, Li L, You J, Wong H S, Han G. SC3: triple spectral clustering-based consensus clustering framework for class discovery from cancer gene expression profiles. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2012, 9(6): 1751–1765Google Scholar
  119. 119.
    Yu Z, Chen H, You J, Han G, Li L. Hybrid fuzzy cluster ensemble framework for tumor clustering from biomolecular data. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2013, 10(3): 657–670Google Scholar
  120. 120.
    Yu Z, Li L, Liu J, Zhang J, Han G. Adaptive noise immune cluster ensemble using affinity propagation. IEEE Transactions on Knowledge and Data Engineering, 2015, 27(12): 3176–3189Google Scholar
  121. 121.
    Ayad H G, Kamel MS. On voting-based consensus of cluster ensembles. Pattern Recognition, 2010, 43(5): 1943–1953zbMATHGoogle Scholar
  122. 122.
    Zhang S, Wong H S, Shen Y. Generalized adjusted rand indices for cluster ensembles. Pattern Recognition, 2012, 45(6): 2214–2226zbMATHGoogle Scholar
  123. 123.
    Fred A L N, Jain AK. Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(6): 835–850Google Scholar
  124. 124.
    Lourenco A, Fred A L N, Jain AK. On the scalability of evidence accumulation clustering. In: Proceedings of the 20th International Conference on Pattern Recognition. 2010, 782–785Google Scholar
  125. 125.
    Amasyali M F, Ersoy O. The performance factors of clustering ensembles. In: Proceedings of the 16th IEEE Signal Processing, Communication and Applications Conference. 2008, 1–4Google Scholar
  126. 126.
    Fern X Z, Brodley CE. Random projection for high dimensional data clustering: a cluster ensemble approach. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03). 2003, 186–193Google Scholar
  127. 127.
    Kuncheva L I, Whitaker CJ. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 2003, 51(2): 181–207zbMATHGoogle Scholar
  128. 128.
    Kuncheva L I, Vetrov DP. Evaluation of stability of k-means cluster ensembles with respect to random initialization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(11): 1798–1808Google Scholar
  129. 129.
    Shi Y, Yu Z, Chen C L P, You J, Wong H S, Wang Y D, Zhang J. Transfer clustering ensemble selection. IEEE Transactions on Cybernetics, 2018, PP(99): 1–14Google Scholar
  130. 130.
    Topchy A P, Law M H C, Jain A K, Fred AL. Analysis of consensus partition in cluster ensemble. In: Proceedings of the 4th IEEE International Conference on Data Mining (ICDM’04). 2004, 225–232Google Scholar
  131. 131.
    Wang T. CA-tree: a hierarchical structure for efficient and scalable coassociation-based cluster ensembles. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2011, 41(3): 686–698Google Scholar
  132. 132.
    Hore P, Hall L O, Goldgof DB. A scalable framework for cluster ensembles. Pattern Recognition, 2009, 42(5): 676–688zbMATHGoogle Scholar
  133. 133.
    Fern X Z, Lin W. Cluster ensemble selection. Statistical Analysis and Data Mining: The ASA Data Science Journal, 2008, 1(3): 128–141MathSciNetGoogle Scholar
  134. 134.
    Azimi J, Fern X. Adaptive cluster ensemble selection. In: Proceedings of the 21st International Joint Conference on Artificial Intelligence. 2009, 992–997Google Scholar
  135. 135.
    Wang X, Han D, Han C. Rough set based cluster ensemble selection. In: Proceedings of the 16th International Conference on Information Fusion. 2013, 438–444Google Scholar
  136. 136.
    Yu Z, Li L, Gao Y, You J, Liu J, Wong H S, Han G. Hybrid clustering solution selection strategy. Pattern Recognition, 2014, 47(10): 3362–3375Google Scholar
  137. 137.
    Yu Z, You J, Wong H S, Han G. From cluster ensemble to structure ensemble. Information Sciences, 2012, 198: 81–99zbMATHGoogle Scholar
  138. 138.
    Yu Z, Li L, Wong H S, You J, Han G, Gao Y, Yu G. Probabilistic cluster structure ensemble. Information Sciences, 2014, 267(5): 16–34MathSciNetGoogle Scholar
  139. 139.
    Yu Z, Zhu X, Wong H S, You J, Zhang J, Han G. Distribution-based cluster structure selection. IEEE Transactions on Cybernetics, 2017, 47(11): 3554–3567Google Scholar
  140. 140.
    Yang Y, Jiang J. HMM-based hybrid meta-clustering ensemble for temporal data. Knowledge-Based Systems, 2014, 56: 299–310Google Scholar
  141. 141.
    Yang Y, Chen K. Temporal data clustering via weighted clustering ensemble with different representations. IEEE Transactions on Knowledge and Data Engineering, 2010, 23(2): 307–320MathSciNetGoogle Scholar
  142. 142.
    Yu Z, Wong HS. Class discovery from gene expression data based on perturbation and cluster ensemble. IEEE Transactions on Nanobio-science, 2009, 8(2): 147–160Google Scholar
  143. 143.
    Yu Z, Chen H, You J, Liu J, Wong H S, Han G, Li L. Adaptive fuzzy consensus clustering framework for clustering analysis of cancer data. IEEE/ACM Transactions on Computational Biology and Bioinfor-matics, 2015, 12(4): 887–901Google Scholar
  144. 144.
    Avogadri R, Valentini G. Fuzzy ensemble clustering based on random projections for DNA microarray data analysis. Artificial Intelligence in Medicine, 2009, 45(2): 173–183Google Scholar
  145. 145.
    Mimaroglu S, Aksehirli E. DICLENS: divisive clustering ensemble with automatic cluster number. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2012, 9(2): 408–420Google Scholar
  146. 146.
    Alush A, Goldberger J. Ensemble segmentation using efficient integer linear programming. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(10): 1966–1977Google Scholar
  147. 147.
    Li H, Meng F, Wu Q, Luo B. Unsupervised multiclass region coseg-mentation via ensemble clustering and energy minimization. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(5): 789–801Google Scholar
  148. 148.
    Zhang X, Jiao L, Liu F, Bo L, Gong M. Spectral clustering ensemble applied to SAR image segmentation. IEEE Transactions on Geo-science and Remote Sensing, 2008, 46(7): 2126–2136Google Scholar
  149. 149.
    Jia J, Liu B, Jiao L. Soft spectral clustering ensemble applied to image segmentation. Frontiers of Computer Science, 2011, 5(1): 66–78MathSciNetGoogle Scholar
  150. 150.
    Rafiee G, Dlay S S, Woo WL. Region-of-interest extraction in low depth of field images using ensemble clustering and difference of Gaussian approaches. Pattern Recognition, 2013, 46(10): 2685–2699Google Scholar
  151. 151.
    Huang X, Zheng X, Yuan W, Wang F, Zhu S. Enhanced clustering of biomedical documents using ensemble non-negative matrix factorization. Information Sciences, 2011, 181(11): 2293–2302Google Scholar
  152. 152.
    Bassiou N, Moschou V, Kotropoulos C. Speaker diarization exploiting the eigengap criterion and cluster ensembles. IEEE Transactions on Audio Speech and Language Processing, 2010, 18(8): 2134–2144Google Scholar
  153. 153.
    Zhuang W, Ye Y, Chen Y, Li T. Ensemble clustering for internet security applications. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2012, 42(6): 1784–1796Google Scholar
  154. 154.
    Tsai C F, Hung C. Cluster ensembles in collaborative filtering recommendation. Applied Soft Computing, 2012, 12(4): 1417–1425Google Scholar
  155. 155.
    Yu Z, Luo P, You J, Wong H S, Leung H, Wu S, Zhang J, Han G. Incremental semi-supervised clustering ensemble for high dimensional data clustering. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(3): 701–714Google Scholar
  156. 156.
    Yu Z, Kuang Z, Liu J, Chen H, Zhang J, You J, Wong H S, Han G. Adaptive ensembling of semi-supervised clustering solutions. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(8): 1577–1590Google Scholar
  157. 157.
    Wei S, Li Z, Zhang C. Combined constraint-based with metric-based in semi-supervised clustering ensemble. International Journal of Machine Learning and Cybernetics, 2018, 9(7): 1085–1100Google Scholar
  158. 158.
    Karypis G, Han E H S, Kumar V. Chameleon: hierarchical clustering using dynamic modeling. Computer, 1999, 32(8): 68–75Google Scholar
  159. 159.
    Xiao W, Yang Y, Wang H, Li T, Xing H. Semi-supervised hierarchical clustering ensemble and its application. Neurocomputing, 2016, 173: 1362–1376Google Scholar
  160. 160.
    Zhou Z H, Tang W. Clusterer ensemble. Knowledge-Based Systems, 2006, 19(1): 77–83Google Scholar
  161. 161.
    Zhang J, Yang Y, Wang H, Mahmood A, Huang F. Semi-supervised clustering ensemble based on collaborative training. In: Proceedings of International Conference on Rough Sets and Knowledge Technology. 2012, 450–455Google Scholar
  162. 162.
    Zhou Z H, Li M. Tri-training: exploiting unlabeled data using three classifiers. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(11): 1529–1541Google Scholar
  163. 163.
    Wang H, Yang D, Qi J. Semi-supervised cluster ensemble based on normal mutual information. Energy Procedia, 2011, 13: 1673–1677Google Scholar
  164. 164.
    Yu Z, Luo P, Liu J, Wong H S, You J, Han G, Zhang J. Semi-supervised ensemble clustering based on selected constraint projection. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(12): 2394–2407Google Scholar
  165. 165.
    Yang Y, Teng F, Li T, Wang H, Zhang Q. Parallel semi-supervised multi-ant colonies clustering ensemble based on mapreduce methodology. IEEE Transactions on Cloud Computing, 2018, 6(3): 857–867Google Scholar
  166. 166.
    Iqbal A M, Moh’D A, Khan Z. Semi-supervised clustering ensemble by voting. Computer Science, 2012, 2(9): 33–40Google Scholar
  167. 167.
    Chen D, Yang Y, Wang H, Mahmood A. Convergence analysis of semi-supervised clustering ensemble. In: Proceedings of International Conference on Information Science and Technology. 2014, 783–788Google Scholar
  168. 168.
    Yan B, Domeniconi C. Subspace metric ensembles for semi-supervised clustering of high dimensional data. In: Proceedings of European Conference on Machine Learning. 2006, 509–520Google Scholar
  169. 169.
    Mahmood A, Li T, Yang Y, Wang H, Afzal M. Semi-supervised clustering ensemble for Web video categorization. In: Proceedings of International Workshop on Multiple Classifier Systems. 2013, 190–200Google Scholar
  170. 170.
    Mahmood A, Li T, Yang Y, Wang H, Afzal M. Semi-supervised evolutionary ensembles for web video categorization. Knowledge-Based Systems, 2015, 76: 53–66Google Scholar
  171. 171.
    Junaidi A, Fink GA. A semi-supervised ensemble learning approach for character labeling with minimal human effort. In: Proceedings of 2011 International Conference on Document Analysis and Recognition. 2011, 259–263Google Scholar
  172. 172.
    Yu Z, Wongb H S, You J, Yang Q, Liao H. Knowledge based cluster ensemble for cancer discovery from biomolecular data. IEEE Transactions on Nanobioscience, 2011, 10(2): 76–85Google Scholar
  173. 173.
    Yu Z, Chen H, You J, Wong H S, Liu J, Li L, Han G. Double selection based semi-supervised clustering ensemble for tumor clustering from gene expression profiles. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2014, 11(4): 727–740Google Scholar
  174. 174.
    Krogh A, Vedelsby J. Neural network ensembles, cross validation and active learning. In: Proceedings of the 7th International Conference on Neural Information Processing Systems. 1994, 231–238Google Scholar
  175. 175.
    Yin Z, Zhao M, Wang Y, Yang J, Zhang J. Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Computer Methods and Programs in Biomedicine, 2017, 140: 93–110Google Scholar
  176. 176.
    Kumar A, Kim J, Lyndon D, Fulham M, Feng D. An ensemble of fine-tuned convolutional neural networks for medical image classification. IEEE Journal of Biomedical and Health Informatics, 2017, 21(1): 31–40Google Scholar
  177. 177.
    Liu W, Zhang M, Luo Z, Cai Y. An ensemble deep learning method for vehicle type classification on visual traffic surveillance sensors. IEEE Access, 2017, 5: 24417–24425Google Scholar
  178. 178.
    Kandaswamy C, Silva L M, Alexandre L A, Santos JM. Deep transfer learning ensemble for classification. In: Proceedings of International Work-Conference on Artificial Neural Networks. 2015, 335–348Google Scholar
  179. 179.
    Nozza D, Fersini E, Messina E. Deep learning and ensemble methods for domain adaptation. In: Proceedings of the 28th IEEE International Conference on Tools with Artificial Intelligence (ICTAI). 2016, 184–189Google Scholar
  180. 180.
    Liu X, Liu Z, Wang G, Cai Z, Zhang H. Ensemble transfer learning algorithm. IEEE Access, 2018, 6: 2389–2396Google Scholar
  181. 181.
    Brys T, Harutyunyan A, Vrancx P, Nowé A, Taylor ME. Multi-objectivization and ensembles of shapings in reinforcement learning. Neurocomputing, 2017, 263: 48–59Google Scholar
  182. 182.
    Chen X L, Cao L, Li C X, Xu Z X, Lai J. Ensemble network architecture for deep reinforcement learning. Mathematical Problems in Engineering, 2018, 2018: 1–6Google Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Xibin Dong
    • 1
  • Zhiwen Yu
    • 1
    Email author
  • Wenming Cao
    • 2
  • Yifan Shi
    • 1
  • Qianli Ma
    • 1
  1. 1.School of Computer Science and EngineeringSouth China University of TechnologyGuangzhouChina
  2. 2.Department of Computer ScienceCity University of Hong KongHong Kong SARChina

Personalised recommendations