On the Creation of Diverse Ensembles for Nonstationary Environments Using Bio-inspired Heuristics

  • Jesus L. LoboEmail author
  • Javier Del Ser
  • Esther Villar-Rodriguez
  • Miren Nekane Bilbao
  • Sancho Salcedo-Sanz
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 514)


Recently the relevance of adaptive models for dynamic data environments has turned into a hot topic due to the vast number of scenarios generating nonstationary data streams. When a change (concept drift) in data distribution occurs, the ensembles of models trained over these data sources are obsolete and do not adapt suitably to the new distribution of the data. Although most of the research on the field is focused on the detection of this drift to re-train the ensemble, it is widely known the importance of the diversity in the ensemble shortly after the drift in order to reduce the initial drop in accuracy. In a Big Data scenario in which data can be huge (and also the number of past models), achieving the most diverse ensemble implies the calculus of all possible combinations of models, which is not an easy task to carry out quickly in the long term. This challenge can be formulated as an optimization problem, for which bio-inspired algorithms can play one of the key roles in these adaptive algorithms. Precisely this is the goal of this manuscript: to validate the relevance of the diversity right after drifts, and to unveil how to achieve a highly diverse ensemble by using a self-learning optimization technique.


Concept drift Diversity Bioinspired optimization 



This work has been supported by the Basque Government through the ELKARTEK program (ref. KK-2015/0000080, BID3A project) and BID3ABI project.


  1. 1.
    Atzori, L., Iera, A., Morabito, G.: The internet of things: a survey. Comput. Netw. 54(15), 2787–2805 (2010)CrossRefzbMATHGoogle Scholar
  2. 2.
    Minku, L.L., White, A.P., Yao, X.: The impact of diversity on online ensemble learning in the presence of concept drift. IEEE Trans. Knowl. Data Eng. 22(5), 730–742 (2010)CrossRefGoogle Scholar
  3. 3.
    Minku, L.L., Yao, X.: DDD: a new ensemble approach for dealing with concept drift. IEEE Trans. Knowl. Data Eng. 24(4), 619–633 (2012)CrossRefGoogle Scholar
  4. 4.
    Geem, Z.W., Kim, J.H., Loganathan, G.: A new heuristic optimization algorithm: harmony search. Simulation 76(2), 60–68 (2001)CrossRefGoogle Scholar
  5. 5.
    Pandi, V.R., Panigrahi, B.K., Das, S., Cui, Z.: Dynamic economic load dispatch with wind energy using modified harmony search. Int. J. Bio-inspired Comput. 2(3/4), 282–289 (2010)CrossRefGoogle Scholar
  6. 6.
    Salcedo-Sanz, S., Pastor-Sánchez, A., Del Ser, J., Prieto, L., Geem, Z.W.: A Coral reefs optimization algorithm with harmony search operators for accurate wind speed prediction. Renew. Energy 75, 93–101 (2015)CrossRefGoogle Scholar
  7. 7.
    Scalabrin, M.H., Parpinelli, R.S., Benítez, C.M., Lopes, H.S.: Population-based harmony search using GPU applied to protein structure prediction. Int. J. Comput. Sci. Eng. 9(1/2), 106 (2014)CrossRefGoogle Scholar
  8. 8.
    Zhang, R., Hanzo, L.: Iterative multiuser detection and channel decoding for DS-CDMA using harmony search. IEEE Signal Process. Lett. 16(10), 917–920 (2009)CrossRefGoogle Scholar
  9. 9.
    Manjarres, D., Del Ser, J., Gil-Lopez, S., Vecchio, M., Landa-Torres, I., Lopez-Valcarce, R.: A novel heuristic approach for distance- and connectivity-based multihop node localization in wireless sensor networks. Soft. Comput. 17(1), 17–28 (2013)CrossRefGoogle Scholar
  10. 10.
    Manjarres, D., Landa-Torres, I., Gil-Lopez, S., Del Ser, J., Bilbao, M.N., Salcedo-Sanz, S., Geem, Z.W.: A survey on applications of the harmony search algorithm. Eng. Appl. Artif. Intell. 26(8), 1818–1831 (2013)CrossRefGoogle Scholar
  11. 11.
    Geem, Z.W., Tseng, C.L., Williams, J.C.: Harmony search algorithms for water and environmental systems. In: Geem, Z.W. (ed.) Music-Inspired Harmony Search Algorithm, vol. 191, pp. 113–127. Springer, Heidelberg (2009)Google Scholar
  12. 12.
    Karimi, Z., Abolhassani, H., Beigy, H.: A new method of mining data streams using harmony search. J. Intell. Inf. Syst. 39(2), 491–511 (2012)CrossRefGoogle Scholar
  13. 13.
    Bilbao, M.N., Del Ser, J., Salcedo-Sanz, S., Casanova-Mateo, C.: On the application of multi-objective harmony search heuristics to the predictive deployment of firefighting aircrafts: a realistic case study. Int. J. Bioinspired Comput. 7(5), 270–284 (2015)CrossRefGoogle Scholar
  14. 14.
    Z̆liobaitė, J., Pechenizkiy, M., Gama, J.: An overview of concept drift applications. In: Japkowicz, N., Stefanowski, J. (eds.) Big Data Analysis: New Algorithms for a New Society, vol. 16, pp. 91–114. Springer, Cham (2016)Google Scholar
  15. 15.
    Gama, J., Zliobaite, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. (CSUR) 46(4), 1–37 (2014)CrossRefzbMATHGoogle Scholar
  16. 16.
    Ditzler, G., Polikar, R., Chawla, N.: An incremental learning algorithm for non-stationary environments and class imbalance. In: International Conference on Pattern Recognition (ICPR), pp. 2997–3000 (2010)Google Scholar
  17. 17.
    Ditterrich, T.G.: Machine learning research: four current directions. Artif. Intell. Mag. 4, 97–136 (1997)Google Scholar
  18. 18.
    Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles. Mach. Learn. 51(2), 181–207 (2003)CrossRefzbMATHGoogle Scholar
  19. 19.
    Yule, G.U.: On the association of attributes in statistics: with illustrations from the material of the childhood society, & c. In: Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, vol. 194, pp. 257–319 (1900)Google Scholar
  20. 20.
    Street, W.N., Kim, Y.: A streaming ensemble algorithm (SEA) for large-scale classification. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 377–382 (2001)Google Scholar
  21. 21.
    Grossberg, S.: Nonlinear neural networks: principles, mechanisms, and architectures. Neural Netw. 1(1), 17–61 (1988)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  • Jesus L. Lobo
    • 1
    Email author
  • Javier Del Ser
    • 1
    • 2
    • 3
  • Esther Villar-Rodriguez
    • 1
  • Miren Nekane Bilbao
    • 2
  • Sancho Salcedo-Sanz
    • 4
  1. 1.TECNALIADerioSpain
  2. 2.University of the Basque Country UPV/EHUBilbaoSpain
  3. 3.Basque Center for Applied Mathematics (BCAM)BilbaoSpain
  4. 4.Universidad de AlcaláAlcalá de HenaresSpain

Personalised recommendations