Evolutionary Computation and Big Data: Key Challenges and Future Directions

  • Shi ChengEmail author
  • Bin Liu
  • Yuhui Shi
  • Yaochu Jin
  • Bin Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9714)


Over the past few years, big data analytics has received increasing attention in all most all scientific research fields. This paper discusses the synergies between big data and evolutionary computation (EC) algorithms, including swarm intelligence and evolutionary algorithms. We will discuss the combination of big data analytics and EC algorithms, such as the application of EC algorithms to solving big data analysis problems and the use of data analysis methods for designing new EC algorithms or improving the performance of EC algorithms. Based on the combination of EC algorithms and data mining techniques, we understand better the insights of data analytics, and design more efficient algorithms to solve real-world big data analytics problems. Also, the weakness and strength of EC algorithms could be analyzed via the data analytics along the optimization process, a crucial entity in EC algorithms. Key challenges and future directions in combining big data and EC algorithms are discussed.


Big data analytics Data science Evolutionary algorithms Evolutionary computation Swarm intelligence 



This work is partially supported by National Natural Science Foundation of China under Grant Numbers 60975080, 61273367, 61571238, and 61302158.


  1. 1.
    Abraham, A., Grosan, C., Ramos, V. (eds.): Swarm Intelligence in Data Mining, Studies in Computational Intelligence, vol. 34. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  2. 2.
    Alexander, F.J., Hoisie, A., Szalay, A.: Big data. Comput. Sci. Eng. 13(6), 10–13 (2011)CrossRefGoogle Scholar
  3. 3.
    Bellman, R.: Adaptive Control Processes: A guided Tour. Princeton University Press, Princeton (1961)CrossRefzbMATHGoogle Scholar
  4. 4.
    Bui, L.T., Michalewicz, Z., Parkinson, E., Abello, M.B.: Adaptation in dynamic environments: a case study in mission planning. IEEE Trans. Evol. Comput. 16(2), 190–209 (2012)CrossRefGoogle Scholar
  5. 5.
    Chai, T., Jin, Y., Sendhoff, B.: Evolutionary complex engineering optimization: opportunities and challenges. IEEE Comput. Intell. Mag. 8(3), 12–15 (2013)CrossRefGoogle Scholar
  6. 6.
    Cheng, S., Shi, Y., Qin, Q., Bai, R.: Swarm intelligence in big data analytics. In: Yin, H., Tang, K., Gao, Y., Klawonn, F., Lee, M., Weise, T., Li, B., Yao, X. (eds.) IDEAL 2013. LNCS, vol. 8206, pp. 417–426. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
    Cheng, S., Zhang, Q., Qin, Q.: Big data analytic with swarm intelligence. Ind. Manag. Data Syst. 116(4) (2016, in press)Google Scholar
  8. 8.
    Coello, C.A.C., Lamont, G.B., Veldhuizen, D.A.V.: Evolutionary Algorithms for Solving Multi-Objective Problems. Genetic and Evolutionary Computation Series, 2nd edn. Springer, New York (2007)zbMATHGoogle Scholar
  9. 9.
    Coello, C.A.C., Dehuri, S., Ghosh, S. (eds.): Swarm Intelligence for Multi-objective Problems in Data Mining, Studies in Computational Intelligence, vol. 242. Springer, Heidelberg (2009)Google Scholar
  10. 10.
    Dhar, V.: Data science and prediction. Commun. ACM 56(12), 64–73 (2013)CrossRefGoogle Scholar
  11. 11.
    Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)CrossRefGoogle Scholar
  12. 12.
    Donoho, D.L.: 50 years of data science. Technical report, Stanford University September 2015Google Scholar
  13. 13.
    Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)zbMATHGoogle Scholar
  14. 14.
    Eberhart, R., Shi, Y.: Computational Intelligence: Concepts to Implementations. Morgan Kaufmann Publisher, San Francisco (2007)CrossRefzbMATHGoogle Scholar
  15. 15.
    Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Mag. 17(3), 37–54 (1996)Google Scholar
  16. 16.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics, 2nd edn. Springer, New York (2009)CrossRefzbMATHGoogle Scholar
  17. 17.
    Hauschild, M., Pelikan, M.: An introduction and survey of estimation of distribution algorithms. Swarm Evol. Comput. 1(3), 111–128 (2011)CrossRefGoogle Scholar
  18. 18.
    Hu, J., Fu, M.C., Marcus, S.I.: A model reference adaptive search method for global optimization. Oper. Res. 55(3), 549–568 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments - a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)CrossRefGoogle Scholar
  20. 20.
    Jin, Y., Hammer, B.: Computational intelligence in big data. IEEE Comput. Intell. Mag. 9(3), 12–13 (2014)CrossRefGoogle Scholar
  21. 21.
    Jin, Y., Sendhoff, B.: A systems approach to evolutionary multiobjective structural optimization and beyond. IEEE Comput. Intell. Mag. 4(3), 62–76 (2009)CrossRefGoogle Scholar
  22. 22.
    Kennedy, J., Eberhart, R., Shi, Y.: Swarm Intelligence. Morgan Kaufmann Publisher, San Francisco (2001)Google Scholar
  23. 23.
    Kim, Y.S.: Multi-objective clustering with data- and human-driven metrics. J. Comput. Inf. Syst. 51(4), 64–73 (2011)Google Scholar
  24. 24.
    Lee, J.A., Verleysen, M.: Nonlinear Dimensionality Reduction. Information Science and Statistics. Springer, New York (2007)CrossRefzbMATHGoogle Scholar
  25. 25.
    Li, L., Tang, K.: History-based topological speciation for multimodal optimization. IEEE Trans. Evol. Comput. 19(1), 136–150 (2015)CrossRefGoogle Scholar
  26. 26.
    Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity. Technical report, McKinsey Global Institute, May 2011Google Scholar
  27. 27.
    Morrison, R.W., De Jong, K.A.: A test problem generator for non-stationary environments. In: Proceedings of the 1999 Congress on Evolutionary Computation (CEC 1999), vol. 3, pp. 2047–2053, July 1999Google Scholar
  28. 28.
    Pelikan, M., Goldberg, D.E., Lobo, F.G.: A survey of optimization by building and using probabilistic models. Comput. Optim. Appl. 21(1), 5–20 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Rajaraman, A., Leskovec, J., Ullman, J.D.: Mining of Massive Datasets. Cambridge University Press, Cambridge (2012)Google Scholar
  30. 30.
    Shi, Y.: Brain storm optimization algorithm. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) ICSI 2011, Part I. LNCS, vol. 6728, pp. 303–309. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  31. 31.
    Shi, Y.: An optimization algorithm based on brainstorming process. Int. J. Swarm Intell. Res. (IJSIR) 2(4), 35–62 (2011)CrossRefGoogle Scholar
  32. 32.
    Yang, P., Tang, K., Lu, X.: Improving estimation of distribution algorithm on multimodal problems by detecting promising areas. IEEE Trans. Cybern. 45(8), 1438–1449 (2015)CrossRefGoogle Scholar
  33. 33.
    Yang, S., Li, C.: A clustering particle swarm optimizer for locating and tracking multiple optima in dynamic environments. IEEE Trans. Evol. Comput. 14(6), 959–974 (2010)CrossRefGoogle Scholar
  34. 34.
    Yang, Z., Tang, K., Yao, X.: Differential evolution for high-dimensional function optimization. In: Proceedings of 2007 IEEE Congress on Evolutionary Computation (CEC 2007), pp. 35231–3530. IEEE (2007)Google Scholar
  35. 35.
    Yang, Z., Tang, K., Yao, X.: Scalability of generalized adaptive differential evolution for large-scale continuous optimization. Soft. Comput. 15(11), 2141–2155 (2011)CrossRefGoogle Scholar
  36. 36.
    Zhou, Z.H., Chawla, N.V., Jin, Y., Williams, G.J.: Big data opportunities and challenges: discussions from data analytics perspectives. IEEE Comput. Intell. Mag. 9(4), 62–74 (2014)CrossRefGoogle Scholar
  37. 37.
    Zlochin, M., Birattari, M., Meuleau, N., Dorigo, M.: Model-based search for combinatorial optimization: a critical survey. Ann. Oper. Res. 131, 373–395 (2004)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Shi Cheng
    • 1
    Email author
  • Bin Liu
    • 2
  • Yuhui Shi
    • 3
  • Yaochu Jin
    • 4
  • Bin Li
    • 5
  1. 1.School of Computer ScienceShaanxi Normal UniversityXi’anChina
  2. 2.School of Computer Science and TechnologyNanjing University of Posts and TelecommunicationsNanjingChina
  3. 3.Department of Electrical and Electronic EngineeringXi’an Jiaotong-Liverpool UniversitySuzhouChina
  4. 4.Department of ComputingUniversity of SurreySurreyUK
  5. 5.University of Science and Technology of ChinaHefeiChina

Personalised recommendations