Evolutionary Intelligence

, Volume 9, Issue 4, pp 203–220 | Cite as

Towards a new Praxis in optinformatics targeting knowledge re-use in evolutionary computation: simultaneous problem learning and optimization

  • D. Lim
  • Y. S. Ong
  • A. GuptaEmail author
  • C. K. Goh
  • P. S. Dutta
Research Paper


As the field of evolutionary optimization continues to expand, it is becoming increasingly common to incorporate various machine learning approaches, such as clustering, classification, and regression models, to improve algorithmic efficiency. However, we note that although problem learning is popularly used in improving the ongoing optimization process, little effort is ever made in extracting re-usable domain knowledge. In other words, the acquired knowledge is seldom transferred and exploited for future design exercises. Focusing on evolutionary optimization, in this paper we investigate the concept of simultaneous problem learning and optimization inspired by the following notions: (1) that prior/dynamically acquired knowledge can enhance the effectiveness of evolutionary search, and (2) that evolution can be geared towards gathering crucial knowledge about the underlying problem. Taking benchmark functions as well as an engineering (process) design problem into consideration, we demonstrate the efficacy of a novel classifier-assisted constrained EA towards simultaneous evolutionary search and problem learning.


Optinformatics Evolutionary computation Learning Knowledge transfer Constrained optimization 



This work was conducted within the Rolls-Royce@NTU Corporate Lab with support from the National Research Foundation (NRF) Singapore under the Corp Lab@University Scheme.


  1. 1.
    Aranha C, Iba H (2009) The memetic tree-based genetic algorithm and its application to portfolio optimization. Memet Comput 1(2):139–151CrossRefGoogle Scholar
  2. 2.
    Becerra RL, Coello CAC (2006) Cultured differential evolution for constrained optimization. Comput Methods Appl Mech Eng 195(33):4303–4322MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Bosman PA, Thierens D (1999) An algorithmic framework for density estimation based evolutionary algorithms. Utrecht University Repository, NetherlandsGoogle Scholar
  4. 4.
    Branke J, Nguyen S, Pickardt CW, Zhang M (2016) Automated design of production scheduling heuristics: a review. IEEE Trans Evol Comput 20(1):110–124CrossRefGoogle Scholar
  5. 5.
    Cagnina LC, Esquivel SC, Coello CAC (2011) Solving constrained optimization problems with a hybrid particle swarm optimization algorithm. Eng Optim 43(8):843–866MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chen Q, Xue B, Zhang M (2015) Generalisation and domain adaptation in GP with gradient descent for symbolic regression. In: 2015 IEEE congress on evolutionary computation (CEC), IEEE, pp 1137–1144Google Scholar
  7. 7.
    Chen X, Ong YS, Lim MH, Tan KC (2011) A multi-facet survey on memetic computation. IEEE Trans Evol Comput 15(5):591CrossRefGoogle Scholar
  8. 8.
    Elliott L, Ingham DB, Kyne AG, Mera NS, Pourkashanian M, Wilson CW (2004) An informed operator based genetic algorithm for tuning the reaction rate parameters of chemical kinetics mechanisms. In: Genetic and evolutionary computation conference, Springer Berlin Heidelberg, pp 945–956Google Scholar
  9. 9.
    Feng L, Ong YS, Lim MH, Tsang IW (2015) Memetic search with interdomain learning: a realization between CVRP and CARP. IEEE Trans Evol Comput 19(5):644–658CrossRefGoogle Scholar
  10. 10.
    Feng L, Ong YS, Tan AH, Tsang IW (2015) Memes as building blocks: a case study on evolutionary optimization+ transfer learning for routing problems. Memet Comput 7(3):159–180CrossRefGoogle Scholar
  11. 11.
    Giannakoglou KC, Papadimitriou DI, Kampolis IC (2006) Aerodynamic shape design using evolutionary algorithms and new gradient-assisted metamodels. Comput Methods Appl Mech Eng 195(44):6312–6329zbMATHCrossRefGoogle Scholar
  12. 12.
    Gupta A, Kelly P (2013) Optimal Galerkin finite element methods for non-isothermal liquid composite moulding process simulations. Int J Heat Mass Transf 64:609–622CrossRefGoogle Scholar
  13. 13.
    Gupta A, Kelly PA, Bickerton S, Walbran WA (2012) Simulating the effect of temperature elevation on clamping force requirements during rigid-tool liquid composite moulding processes. Compos A Appl Sci Manuf 43(12):2221–2229CrossRefGoogle Scholar
  14. 14.
    Gupta A, Kelly PA, Ehrgott M, Bickerton S (2013) A surrogate model based evolutionary game-theoretic approach for optimizing non-isothermal compression RTM processes. Compos Sci Technol 84:92–100CrossRefGoogle Scholar
  15. 15.
    Gupta A, Mańdziuk J, Ong YS (2015) Evolutionary multitasking in bi-level optimization. Complex Intell Syst 1(1–4):83–95CrossRefGoogle Scholar
  16. 16.
    Gupta A, Ong YS, Feng L (2016) Multifactorial evolution: toward evolutionary multitasking. IEEE Trans Evol Comput 20(3):343–357CrossRefGoogle Scholar
  17. 17.
    Gupta A, Ong YS, Feng L, Tan KC (2016) Multiobjective multifactorial optimization in evolutionary multitasking. IEEE Trans Cybern. doi: 10.1109/TCYB.2016.2554622
  18. 18.
    Han J, Pei J, Kamber M (2011) Data mining: concepts and techniques. Elsevier, AmsterdamzbMATHGoogle Scholar
  19. 19.
    Handoko SD, Kwoh CK, Ong YS (2010) Feasibility structure modeling: an effective chaperone for constrained memetic algorithms. IEEE Trans Evol Comput 14(5):740–758CrossRefGoogle Scholar
  20. 20.
    Hasan SK, Sarker R, Essam D, Cornforth D (2009) Memetic algorithms for solving job-shop scheduling problems. Memet Comput 1(1):69–83CrossRefGoogle Scholar
  21. 21.
    Holland JH (1962) Outline for a logical theory of adaptive systems. J ACM 9(3):297–314zbMATHCrossRefGoogle Scholar
  22. 22.
    Iqbal M, Browne WN, Zhang M (2014) Reusing building blocks of extracted knowledge to solve complex, large-scale boolean problems. IEEE Trans Evol Comput 18(4):465–480CrossRefGoogle Scholar
  23. 23.
    Iqbal M, Browne WN, Zhang M (2015) Extending XCS with cyclic graphs for scalability on complex Boolean problems. Evolut Comput, Early Access article. doi: 10.1162/EVCO_a_00167
  24. 24.
    Jin X, Reynolds RG (1999) Using knowledge-based evolutionary computation to solve nonlinear constraint optimization problems: a cultural algorithm approach. In: Proceedings of the 1999 congress on evolutionary computation, CEC 99, vol 3. IEEEGoogle Scholar
  25. 25.
    Jin Y, Sendhoff B (2002) Fitness approximation in evolutionary computation—a survey. In: Proceedings of the genetic and evolutionary computation conference, Morgan Kaufmann Publishers Inc., pp 1105–1112Google Scholar
  26. 26.
    Jin Y, Sendhoff B (2004) Reducing fitness evaluations using clustering techniques and neural network ensembles. In: Genetic and evolutionary computation conference, Springer Berlin Heidelberg, pp 688–699Google Scholar
  27. 27.
    Johnson CG, Cardalda JJR (2002) Genetic algorithms in visual art and music. Leonardo 35(2):175–184CrossRefGoogle Scholar
  28. 28.
    Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13(4):455–492MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Jourdan L, Dhaenens C, Talbi EG (2006) Using datamining techniques to help metaheuristics: a short survey. In: International workshop on hybrid metaheuristics, Springer Berlin Heidelberg, pp 57–69Google Scholar
  30. 30.
    Kamal MR, Sourour S (1973) Kinetics and thermal characterization of thermoset cure. Polym Eng Sci 13(1):59–64CrossRefGoogle Scholar
  31. 31.
    Kim HS, Cho SB (2001) An efficient genetic algorithm with less fitness evaluation by clustering. In: Proceedings of the 2001 congress on evolutionary computation, vol 2. IEEE, pp 887–894Google Scholar
  32. 32.
    Knowles J (2006) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10(1):50–66CrossRefGoogle Scholar
  33. 33.
    Lameijer EW, Bäck T, Kok JN, Ijzerman AP (2005) Evolutionary algorithms in drug design. Nat Comput 4(3):177–243MathSciNetCrossRefGoogle Scholar
  34. 34.
    Larranaga P, Lozano JA (2002) Estimation of distribution algorithms: a new tool for evolutionary computation, vol 2. Springer, BerlinzbMATHGoogle Scholar
  35. 35.
    Le MN, Ong YS, Nguyen QH (2008) Optinformatics for schema analysis of binary genetic algorithms. In: Proceedings of the 10th annual conference on genetic and evolutionary computation, ACM, pp 1121–1122Google Scholar
  36. 36.
    Leifsson L, Koziel S (2016) Surrogate modelling and optimization using shape-preserving response prediction: a review. Eng Optim 48(3):476–496CrossRefGoogle Scholar
  37. 37.
    Lim D, Jin Y, Ong YS, Sendhoff B (2010) Generalizing surrogate-assisted evolutionary computation. IEEE Trans Evol Comput 14(3):329–355CrossRefGoogle Scholar
  38. 38.
    Lim D, Ong YS, Jin Y, Sendhoff B (2007) A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation. In: Proceedings of the 9th annual conference on genetic and evolutionary computation, ACM, pp 1288–1295Google Scholar
  39. 39.
    Meuth R, Lim MH, Ong YS, Wunsch DC II (2009) A proposition on memes and meta-memes in computing for higher-order learning. Memet Comput 1(2):85–100CrossRefGoogle Scholar
  40. 40.
    Mezura-Montes E, Coello CAC (2005) A simple multimembered evolution strategy to solve constrained optimization problems. IEEE Trans Evol Comput 9(1):1–17zbMATHCrossRefGoogle Scholar
  41. 41.
    Michalski RS (2000) Learnable evolution model: evolutionary processes guided by machine learning. Mach Learn 38(1–2):9–40zbMATHCrossRefGoogle Scholar
  42. 42.
    Mitchell TM (1999) Machine learning and data mining. Commun ACM 42(11):30–36CrossRefGoogle Scholar
  43. 43.
    Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithmsC3P. Caltech Concurr Comput Progr Rep 826:1989Google Scholar
  44. 44.
    Muñoz E, Cadenas JM, Ong YS, Acampora G (2016) Memetic music composition. IEEE Trans Evol Comput 20(1):1–15CrossRefGoogle Scholar
  45. 45.
    Okabe T, Jin Y, Sendoff B, Olhofer M (2004) Voronoi-based estimation of distribution algorithm for multi-objective optimization. In: Congress on evolutionary computation, 2004. CEC2004. vol 2. IEEE, pp 1594–1601Google Scholar
  46. 46.
    Ong YS, Gupta A (2016) Evolutionary multitasking: a computer science view of cognitive multitasking. Cogn Comput 8(2):125–142CrossRefGoogle Scholar
  47. 47.
    Ong YS, Keane AJ (2004) Meta-Lamarckian learning in memetic algorithms. IEEE Trans Evol Comput 8(2):99–110CrossRefGoogle Scholar
  48. 48.
    Ong YS, Lim MH, Chen X (2010) Research frontier-memetic computation—past, present & future. IEEE Comput Intell Mag 5(2):24CrossRefGoogle Scholar
  49. 49.
    Ong YS, Nair PB, Keane AJ (2003) Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA J 41(4):687–696CrossRefGoogle Scholar
  50. 50.
    Pelikan M, Goldberg DE, Cantú-Paz E (1999) BOA: The Bayesian optimization algorithm. In: Proceedings of the 1st annual conference on genetic and evolutionary computation, vol 1. Morgan Kaufmann Publishers Inc., pp 525–532Google Scholar
  51. 51.
    Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106Google Scholar
  52. 52.
    Quinlan JR (2014) C4. 5: programs for machine learning. Elsevier, AmsterdamGoogle Scholar
  53. 53.
    Ramsey CL, Grefenstette JJ (1993) Case-based initialization of genetic algorithms. In: ICGA, pp 84–91Google Scholar
  54. 54.
    Rasheed K, Hirsh H (2000) Informed operators: speeding up genetic-algorithm-based design optimization using reduced models. In: Proceedings of the 2nd annual conference on genetic and evolutionary computation, Morgan Kaufmann Publishers Inc., pp 628–635Google Scholar
  55. 55.
    Rechenberg I (1965) Cybernetic solution path of an experimental problem (Royal Aircraft Establishment Translation No. 1122, B.F. Toms, Trans). Ministry of Aviation, Royal Aircraft Establishment, Farnsborough HantsGoogle Scholar
  56. 56.
    Reynolds RG, Michalewicz Z, Cavaretta MJ (1995) Using cultural algorithms for constraint handling in GENOCOP. In: Evolutionary programming, pp 289–305Google Scholar
  57. 57.
    Runarsson TP, Yao X (2000) Stochastic ranking for constrained evolutionary optimization. IEEE Trans Evol Comput 4(3):284–294CrossRefGoogle Scholar
  58. 58.
    Sun G, Li G, Gong Z, He G, Li Q (2011) Radial basis functional model for multi-objective sheet metal forming optimization. Eng Optim 43(12):1351–1366MathSciNetCrossRefGoogle Scholar
  59. 59.
    Tsutsui S, Pelikan M, Goldberg DE (2001) Probabilistic model-building genetic algorithms using marginal histograms in continuous domain. In: Proceedings of the international conference on knowledge-based and intelligent information and engineering systems, pp 112–121Google Scholar
  60. 60.
    Walbran WA (2011) Experimental validation of local and global force simulations for rigid tool liquid composite moulding processes. Doctoral dissertation, ResearchSpace@ AucklandGoogle Scholar
  61. 61.
    Wang Y, Cai Z, Guo G, Zhou Y (2007) Multiobjective optimization and hybrid evolutionary algorithm to solve constrained optimization problems. IEEE Trans Syst Man Cybern Part B 37(3):560–575CrossRefGoogle Scholar
  62. 62.
    Wang Y, Cai Z, Zhou Y, Zeng W (2008) An adaptive tradeoff model for constrained evolutionary optimization. IEEE Trans Evol Comput 12(1):80–92CrossRefGoogle Scholar
  63. 63.
    Zhang Q, Liu W, Tsang E, Virginas B (2010) Expensive multiobjective optimization by MOEA/D with Gaussian process model. IEEE Trans Evol Comput 14(3):456–474CrossRefGoogle Scholar
  64. 64.
    Zhang Q, Zhou A, Jin Y (2008) RM-MEDA: A regularity model-based multiobjective estimation of distribution algorithm. IEEE Trans Evol Comput 12(1):41–63CrossRefGoogle Scholar
  65. 65.
    Zhou Z, Ong YS, Nair PB, Keane AJ, Lum KY (2007) Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Trans Syst Man Cybern Part C 37(1):66–76CrossRefGoogle Scholar
  66. 66.
    Zhou Z, Ong YS, Nguyen MH, Lim D (2005) A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm. In 2005 IEEE congress on evolutionary computation, vol 3. IEEE, pp 2832–2839Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • D. Lim
    • 1
  • Y. S. Ong
    • 2
  • A. Gupta
    • 2
    Email author
  • C. K. Goh
    • 2
  • P. S. Dutta
    • 2
  1. 1.Computational Intelligence Lab, School of Computer Science and EngineeringNanyang Technological UniversitySingaporeSingapore
  2. 2.Rolls-Royce@NTU Corporate Lab c/o, School of Computer Science and EngineeringNanyang Technological UniversitySingaporeSingapore

Personalised recommendations