Skip to main content
SpringerLink
Log in

Metaheuristics for Continuous Optimization of High-Dimensional Problems: State of the Art and Perspectives

  • Chapter
  • First Online:
Book cover Big Data Optimization: Recent Developments and Challenges

Part of the book series: Studies in Big Data ((SBD,volume 18))

Abstract

The age of big data brings new opportunities in many relevant fields, as well as new research challenges. Among the latter, there is the need for more effective and efficient optimization techniques, able to address problems with hundreds, thousands, and even millions of continuous variables. Over the last decade, researchers have developed various improvements of existing metaheuristics for tacking high-dimensional optimization problems, such as hybridizations, local search and parameter adaptation. Another effective strategy is the cooperative coevolutionary approach, which performs a decomposition of the search space in order to obtain sub-problems of smaller size. Moreover, in some cases such powerful search algorithms have been used with high performance computing to address, within reasonable run times, very high-dimensional optimization problems. Nevertheless, despite the significant amount of research already carried out, there are still many open research issues and room for significant improvements. In order to provide a picture of the state of the art in the field of high-dimensional continuous optimization, this chapter describes the most successful algorithms presented in the recent literature, also outlining relevant trends and identifying possible future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abbass, H.: The self-adaptive pareto differential evolution algorithm. In: Proceedings of the 2002 Congress on Evolutionary Computation, 2002. CEC’02, vol. 1, pp. 831–836 (2002)

    Google Scholar 

  2. Auger, A., Hansen, N., Mauny, N., Ros, R., Schoenauer, M.: Bio-inspired continuous optimization: the coming of age. Piscataway, NJ, USA, invited talk at CEC2007 (2007)

    Google Scholar 

  3. Blecic, I., Cecchini, A., Trunfio, G.A.: Fast and accurate optimization of a GPU-accelerated CA urban model through cooperative coevolutionary particle swarms. Proc. Comput. Sci. 29, 1631–1643 (2014)

    Article  Google Scholar 

  4. Blecic, I., Cecchini, A., Trunfio, G.A.: How much past to see the future: a computational study in calibrating urban cellular automata. Int. J. Geogr. Inf. Sci. 29(3), 349–374 (2015)

    Article  Google Scholar 

  5. Brest, J., Boskovic, B., Greiner, S., Zumer, V., Maucec, M.S.: Performance comparison of self-adaptive and adaptive differential evolution algorithms. Soft Comput. 11(7), 617–629 (2007)

    Article  MATH  Google Scholar 

  6. Brest, J., Boskovic, B., Zamuda, A., Fister, I., Maucec, M.: Self-adaptive differential evolution algorithm with a small and varying population size. In: 2012 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8 (2012)

    Google Scholar 

  7. Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V.: Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans. Evol. Comput. 10(6), 646–657 (2006)

    Article  Google Scholar 

  8. Brest, J., Maucec, M.S.: Self-adaptive differential evolution algorithm using population size reduction and three strategies. Soft Comput. 15(11), 2157–2174 (2011)

    Article  Google Scholar 

  9. Brest, J., Zumer, V., Maucec, M.: Self-adaptive differential evolution algorithm in constrained real-parameter optimization. In: IEEE Congress on Evolutionary Computation, 2006. CEC 2006, pp. 215–222 (2006)

    Google Scholar 

  10. Chai, T., Jin, Y., Sendhoff, B.: Evolutionary complex engineering optimization: opportunities and challenges. IEEE Comput. Intell. Mag. 8(3), 12–15 (2013)

    Article  Google Scholar 

  11. Chen, W., Weise, T., Yang, Z., Tang, K.: Large-scale global optimization using cooperative coevolution with variable interaction learning. In: Parallel Problem Solving from Nature. PPSN XI, Lecture Notes in Computer Science, vol. 6239, pp. 300–309. Springer, Berlin, Heidelberg (2010)

    Google Scholar 

  12. Cheng, S., Shi, Y., Qin, Q., Bai, R.: Swarm intelligence in big data analytics. In: Yin, H., Tang, K., Gao, Y., Klawonn, F., Lee, M., Weise, T., Li, B., Yao, X. (eds.) Intelligent Data Engineering and Automated Learning—IDEAL 2013. Lecture Notes in Computer Science, vol. 8206, pp. 417–426. Springer, Berlin, Heidelberg (2013)

    Google Scholar 

  13. Cheng, S., Ting, T., Yang, X.S.: Large-scale global optimization via swarm intelligence. In: Koziel, S., Leifsson, L., Yang, X.S. (eds.) Solving Computationally Expensive Engineering Problems, Springer Proceedings in Mathematics & Statistics, vol. 97, pp. 241–253. Springer International Publishing (2014)

    Google Scholar 

  14. Doerner, K., Hartl, R.F., Reimann, M.: Cooperative ant colonies for optimizing resource allocation in transportation. In: Proceedings of the EvoWorkshops on Applications of Evolutionary Computing, pp. 70–79. Springer-Verlag (2001)

    Google Scholar 

  15. El-Abd, M., Kamel, M.S.: A taxonomy of cooperative particle swarm optimizers. Int. J. Comput. Intell. Res. 4 (2008)

    Google Scholar 

  16. Ergun, H., Van Hertem, D., Belmans, R.: Transmission system topology optimization for large-scale offshore wind integration. IEEE Trans. Sustain. Energy 3(4), 908–917 (2012)

    Article  Google Scholar 

  17. Eshelman, L.J., Schaffer, J.D.: Real-coded genetic algorithm and interval schemata. In: Foundation of Genetic Algorithms, pp. 187–202 (1993)

    Google Scholar 

  18. Esmin, A.A., Coelho, R., Matwin, S.: A review on particle swarm optimization algorithm and its variants to clustering high-dimensional data. Artif. Intell. Rev. 1–23 (2013)

    Google Scholar 

  19. Fernandes, C., Rosa, A.: A study on non-random mating and varying population size in genetic algorithms using a royal road function. In: Proceedings of the 2001 Congress on Evolutionary Computation, 2001, vol. 1, pp. 60–66 (2001)

    Google Scholar 

  20. Fernandes, C., Rosa, A.: Self-adjusting the intensity of assortative mating in genetic algorithms. Soft Comput. 12(10), 955–979 (2008)

    Article  Google Scholar 

  21. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001)

    Article  Google Scholar 

  22. Hinterding, R.: Gaussian mutation and self-adaption for numeric genetic algorithms. In: IEEE International Conference on Evolutionary Computation, 1995, vol. 1, pp. 384–389 (1995)

    Google Scholar 

  23. Huang, V., Qin, A., Suganthan, P.: Self-adaptive differential evolution algorithm for constrained real-parameter optimization. In: IEEE Congress on Evolutionary Computation, 2006. CEC 2006, pp. 17–24 (2006)

    Google Scholar 

  24. Lastra, M., Molina, D., Bentez, J.M.: A high performance memetic algorithm for extremely high-dimensional problems. Inf. Sci. 293, 35–58 (2015)

    Article  Google Scholar 

  25. LaTorre, A.: A framework for hybrid dynamic evolutionary algorithms: multiple offspring sampling (MOS). Ph.D. thesis, Universidad Politecnica de Madrid (2009)

    Google Scholar 

  26. LaTorre, A., Muelas, S., Pea, J.M.: A comprehensive comparison of large scale global optimizers. Inf. Sci. (in press) (2015)

    Google Scholar 

  27. LaTorre, A., Muelas, S., Peña, J.M.: A mos-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test. Soft Comput. 15(11), 2187–2199 (2011)

    Article  Google Scholar 

  28. LaTorre, A., Muelas, S., Pena, J.M.: Multiple offspring sampling in large scale global optimization. In: 2012 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8 (2012)

    Google Scholar 

  29. LaTorre, A., Muelas, S., Pena, J.M.: Large scale global optimization: experimental results with MOS-based hybrid algorithms. In: 2013 IEEE Congress on Evolutionary Computation (CEC), pp. 2742–2749 (2013)

    Google Scholar 

  30. Li, X., Yao, X.: Cooperatively coevolving particle swarms for large scale optimization. IEEE Trans. Evol. Comput. 16(2), 210–224 (2012)

    Article  MathSciNet  Google Scholar 

  31. Liu, J., Lampinen, J.: A fuzzy adaptive differential evolution algorithm. Soft Comput. 9(6), 448–462 (2005)

    Article  MATH  Google Scholar 

  32. Liu, Y., Yao, X., Zhao, Q.: Scaling up fast evolutionary programming with cooperative coevolution. In: Proceedings of the 2001 Congress on Evolutionary Computation, Seoul, Korea, pp. 1101–1108 (2001)

    Google Scholar 

  33. Lu, Y., Wang, S., Li, S., Zhou, C.: Particle swarm optimizer for variable weighting in clustering high-dimensional data. Mach. Learn. 82(1), 43–70 (2011)

    Article  MathSciNet  Google Scholar 

  34. Mahdavi, S., Shiri, M.E., Rahnamayan, S.: Metaheuristics in large-scale global continues optimization: a survey. Inf. Sci. 295, 407–428 (2015)

    Article  MathSciNet  Google Scholar 

  35. Molina, D., Lozano, M., García-Martínez, C., Herrera, F.: Memetic algorithms for continuous optimisation based on local search chains. Evol. Comput. 18(1), 27–63 (2010)

    Article  Google Scholar 

  36. Molina, D., Lozano, M., Herrera, F.: Ma-sw-chains: Memetic algorithm based on local search chains for large scale continuous global optimization. In: 2010 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8 (2010)

    Google Scholar 

  37. Molina, D., Lozano, M., Sánchez, A.M., Herrera, F.: Memetic algorithms based on local search chains for large scale continuous optimisation problems: Ma-ssw-chains. Soft Comput. 15(11), 2201–2220 (2011)

    Article  Google Scholar 

  38. Moscato, P.: On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithms. Technical Report, Caltech Concurrent Computation Program Report 826, Caltech, Pasadena, California (1989)

    Google Scholar 

  39. Moscato, P.: New ideas in optimization. In: Memetic Algorithms: A Short Introduction, pp. 219–234. McGraw-Hill Ltd., UK, Maidenhead, UK, England (1999)

    Google Scholar 

  40. Mühlenbein, H., Schlierkamp-Voosen, D.: Predictive models for the breeder genetic algorithm I. continuous parameter optimization. Evol. Comput. 1(1), 25–49 (1993)

    Article  Google Scholar 

  41. Omidvar, M.N., Li, X., Mei, Y., Yao, X.: Cooperative co-evolution with differential grouping for large scale optimization. IEEE Trans. Evol. Comput. 18(3), 378–393 (2014)

    Article  Google Scholar 

  42. Omidvar, M.N., Li, X., Tang, K.: Designing benchmark problems for large-scale continuous optimization. Inf. Sci. (in press) (2015)

    Google Scholar 

  43. Omidvar, M.N., Li, X., Yang, Z., Yao, X.: Cooperative co-evolution for large scale optimization through more frequent random grouping. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 1–8. IEEE (2010)

    Google Scholar 

  44. Omidvar, M.N., Li, X., Yao, X.: Cooperative co-evolution with delta grouping for large scale non-separable function optimization. In: IEEE Congress on Evolutionary Computation, pp. 1–8 (2010)

    Google Scholar 

  45. Omidvar, M.N., Li, X., Yao, X.: Smart use of computational resources based on contribution for cooperative co-evolutionary algorithms. Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation. GECCO’11, pp. 1115–1122. ACM, New York, NY, USA (2011)

    Google Scholar 

  46. Omidvar, M.N., Mei, Y., Li, X.: Effective decomposition of large-scale separable continuous functions for cooperative co-evolutionary algorithms. In: Proceedings of the IEEE Congress on Evolutionary Computation. IEEE (2014)

    Google Scholar 

  47. Parsopoulos, K.E.: Parallel cooperative micro-particle swarm optimization: a master-slave model. Appl. Soft Comput. 12(11), 3552–3579 (2012)

    Google Scholar 

  48. Potter, M.A., De Jong, K.A.: A cooperative coevolutionary approach to function optimization. In: Proceedings of the International Conference on Evolutionary Computation. The Third Conference on Parallel Problem Solving from Nature: Parallel Problem Solving from Nature, PPSN III, pp. 249–257. Springer-Verlag (1994)

    Google Scholar 

  49. Potter, M.A., De Jong, K.A.: Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol. Comput. 8(1), 1–29 (2000)

    Google Scholar 

  50. Qin, A., Huang, V., Suganthan, P.: Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans. Evol. Comput. 13(2), 398–417 (2009)

    Article  Google Scholar 

  51. Qin, A.K., Suganthan, P.N.: Self-adaptive differential evolution algorithm for numerical optimization. In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2005, 2–4 Sept 2005, Edinburgh, UK, pp. 1785–1791. IEEE (2005)

    Google Scholar 

  52. Ray, T., Yao, X.: A cooperative coevolutionary algorithm with correlation based adaptive variable partitioning. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 983–989. IEEE (2009)

    Google Scholar 

  53. Salomon, R.: Reevaluating genetic algorithm performance under coordinate rotation of benchmark functions—a survey of some theoretical and practical aspects of genetic algorithms. BioSystems 39, 263–278 (1995)

    Article  Google Scholar 

  54. Snchez-Ante, G., Ramos, F., Frausto, J.: Cooperative simulated annealing for path planning in multi-robot systems. MICAI 2000: Advances in Artificial Intelligence. LNCS, vol. 1793, pp. 148–157. Springer, Berlin, Heidelberg (2000)

    Google Scholar 

  55. Solis, F.J., Wets, R.J.B.: Minimization by Random Search Techniques. Math. Oper. Res. 6(1), 19–30 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  56. Storn, R., Price, K.: Differential evolution a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)

    Google Scholar 

  57. Sun, L., Yoshida, S., Cheng, X., Liang, Y.: A cooperative particle swarm optimizer with statistical variable interdependence learning. Inf. Sci. 186(1), 20–39 (2012)

    Article  MathSciNet  Google Scholar 

  58. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (1998)

    Google Scholar 

  59. Takahashi, M., Kita, H.: A crossover operator using independent component analysis for real-coded genetic algorithms. In: Proceedings of the 2001 Congress on Evolutionary Computation, 2001, vol. 1, pp. 643–649 (2001)

    Google Scholar 

  60. Talbi, E.G.: A taxonomy of hybrid metaheuristics. J. Heuristics 8(5), 541–564 (2002)

    Article  Google Scholar 

  61. Tang, K., Li, X., Suganthan, P.N., Yang, Z., Weise, T.: Benchmark functions for the CEC’2010 special session and competition on large-scale global optimization. http://nical.ustc.edu.cn/cec10ss.php

  62. Tang, K., Yang, Z., Weise, T.: Special session on evolutionary computation for large scale global optimization at 2012 IEEE World Congress on Computational Intelligence (cec@wcci-2012). Technical report, Hefei, Anhui, China: University of Science and Technology of China (USTC), School of Computer Science and Technology, Nature Inspired Computation and Applications Laboratory (NICAL) (2012)

    Google Scholar 

  63. Tang, K., Yao, X., Suganthan, P., MacNish, C., Chen, Y., Chen, C., Yang, Z.: Benchmark functions for the CEC’2008 special session and competition on large scale global optimization

    Google Scholar 

  64. Teo, J.: Exploring dynamic self-adaptive populations in differential evolution. Soft Comput. 10(8), 673–686 (2006)

    Article  Google Scholar 

  65. Thomas, S., Jin, Y.: Reconstructing biological gene regulatory networks: where optimization meets big data. Evol. Intell. 7(1), 29–47 (2014)

    Article  Google Scholar 

  66. Trunfio, G.A.: Enhancing the firefly algorithm through a cooperative coevolutionary approach: an empirical study on benchmark optimisation problems. IJBIC 6(2), 108–125 (2014)

    Article  Google Scholar 

  67. Trunfio, G.A.: A cooperative coevolutionary differential evolution algorithm with adaptive subcomponents. Proc. Comput. Sci. 51, 834–844 (2015)

    Article  Google Scholar 

  68. Tseng, L.Y., Chen, C.: Multiple trajectory search for large scale global optimization. In: IEEE Congress on Evolutionary Computation, 2008. CEC 2008 (IEEE World Congress on Computational Intelligence), pp. 3052–3059 (2008)

    Google Scholar 

  69. van den Bergh, F., Engelbrecht, A.P.: A cooperative approach to particle swarm optimization. IEEE Trans. Evol. Comput. 8(3), 225–239 (2004)

    Google Scholar 

  70. Wang, Y., Huang, J., Dong, W.S., Yan, J.C., Tian, C.H., Li, M., Mo, W.T.: Two-stage based ensemble optimization framework for large-scale global optimization. Eur. J. Oper. Res. 228(2), 308–320 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  71. Wang, Y., Li, B.: Two-stage based ensemble optimization for large-scale global optimization. In: 2010 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8 (2010)

    Google Scholar 

  72. Weicker, K., Weicker, N.: On the improvement of coevolutionary optimizers by learning variable interdependencies. In: 1999 Congress on Evolutionary Computation, pp. 1627–1632. IEEE Service Center, Piscataway, NJ (1999)

    Google Scholar 

  73. Xue, F., Sanderson, A., Bonissone, P., Graves, R.: Fuzzy logic controlled multi-objective differential evolution. In: The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ’05, pp. 720–725 (2005)

    Google Scholar 

  74. Yang, X.S.: Firefly algorithms for multimodal optimization. In: Stochastic Algorithms: Foundations and Applications, 5th International Symposium, SAGA 2009, Sapporo, Japan, 26–28 Oct 2009. Proceedings, LNCS, vol. 5792, pp. 169–178. Springer (2009)

    Google Scholar 

  75. Yang, Z., Tang, K., Yao, X.: Large scale evolutionary optimization using cooperative coevolution. Inf. Sci. 178(15), 2985–2999 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  76. Yang, Z., Tang, K., Yao, X.: Multilevel cooperative coevolution for large scale optimization. In: IEEE Congress on Evolutionary Computation, pp. 1663–1670. IEEE (2008)

    Google Scholar 

  77. Yang, Z., Tang, K., Yao, X.: Self-adaptive differential evolution with neighborhood search. In: IEEE Congress on Evolutionary Computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence), pp. 1110–1116 (2008)

    Google Scholar 

  78. Yang, Z., Tang, K., Yao, X.: Scalability of generalized adaptive differential evolution for large-scale continuous optimization. Soft Comput. 15(11), 2141–2155 (2011)

    Article  Google Scholar 

  79. Zhang, J., Sanderson, A.: Jade: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13(5), 945–958 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DADU, University of Sassari, Alghero, Italy

    Giuseppe A. Trunfio

Authors
  1. Giuseppe A. Trunfio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giuseppe A. Trunfio .

Editor information

Editors and Affiliations

  1. Aston Business School, Aston University, Birmingham, United Kingdom

    Ali Emrouznejad

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Trunfio, G.A. (2016). Metaheuristics for Continuous Optimization of High-Dimensional Problems: State of the Art and Perspectives. In: Emrouznejad, A. (eds) Big Data Optimization: Recent Developments and Challenges. Studies in Big Data, vol 18. Springer, Cham. https://doi.org/10.1007/978-3-319-30265-2_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-30265-2_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-30263-8

  • Online ISBN: 978-3-319-30265-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation