Advertisement

Soft Computing

, Volume 17, Issue 12, pp 2327–2348 | Cite as

A hybrid multi-population framework for dynamic environments combining online and offline learning

  • Gönül Uludağ
  • Berna Kiraz
  • A. Şima Etaner-Uyar
  • Ender ÖzcanEmail author
Methodologies and Application

Abstract

Population based incremental learning algorithms and selection hyper-heuristics are highly adaptive methods which can handle different types of dynamism that may occur while a given problem is being solved. In this study, we present an approach based on a multi-population framework hybridizing these methods to solve dynamic environment problems. A key feature of the hybrid approach is the utilization of offline and online learning methods at successive stages. The performance of our approach along with the influence of different heuristic selection methods used within the selection hyper-heuristic is investigated over a range of dynamic environments produced by a well known benchmark generator as well as a real world problem, referred to as the Unit Commitment Problem. The empirical results show that the proposed approach using a particular hyper-heuristic outperforms some of the best known approaches in literature on the dynamic environment problems dealt with.

Keywords

Heuristic Metaheuristic Hyper-heuristic Estimation of distribution algorithm Dynamic environment 

Abbreviations

EDA

Estimation of distribution algorithms

HH-EDA2

Hyper-heuristic based dual population EDA

PBIL

Population based incremental learning

PBIL2

Dual population PBIL

PBILr

PBIL with restart

PBIL2r

PBIL2 with restart

MPBILr

Memory-based PBIL with restart

MPBIL2r

Dual population memory-based PBIL with restart

Sentinel8

Sentinel-based genetic algorithm with 8 sentinels

Sentinel16

Sentinel-based genetic algorithm with 16 sentinels

DUF

Decomposable unitation-based function

MW

Megawatt

UCP

Unit commitment problem

CL

Cycle length

HF

High frequency

HS

High severity

LF

Low frequency

LS

Low severity

MF

Medium frequency

MS

Medium severity

VHS

Very high severity

Notes

Acknowledgments

This work is supported in part by the EPSRC, grant EP/F033214/1 (The LANCS Initiative Postdoctoral Training Scheme) and Berna Kiraz is supported by the TUBITAK 2211-National Scholarship Programme for PhD students.

References

  1. Baluja S (1994) Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA, Tech. repGoogle Scholar
  2. Barlow GJ, Smith SF (2009) Using memory models to improve adaptive efficiency in dynamic problems. In: IEEE symposium on computational intelligence in scheduling, CISCHED, pp 7–14Google Scholar
  3. Bosman PAN (2005) Learning, anticipation and time-deception in evolutionary online dynamic optimization. In: Proceedings of the 2005 workshops on genetic and evolutionary computation, ACM, GECCO ’05, pp 39–47Google Scholar
  4. Branke J (1999) Memory enhanced evolutionary algorithms for changing optimization problems. In: IEEE congress on evolutionary computation CEC 99, vol 3, pp 1875–1882Google Scholar
  5. Branke J (2002) Evolutionary optimization in dynamic environments. Kluwer, NorwellGoogle Scholar
  6. Branke J, Kaussler T, Schmidt C, Schmeck H (2000) A multi-population approach to dynamic optimization problems. In: 4th International conference on adaptive computing in design and manufacture (ACDM 2000). Springer, Berlin, pp 299–308Google Scholar
  7. Burke E, Kendall G (eds) (2005) Search methodologies: introductory tutorials in optimization and decision support techniques. Springer, BerlinGoogle Scholar
  8. Burke EK, Gendreau M, Hyde MR, Kendall G, Ochoa G, Özcan E, Qu R (2013) Hyper-heuristics: A survey of the state of the art. J Oper Res Soc. doi: 10.1057/jors.2013.71
  9. Cao Y, Luo W, (2010) A novel updating strategy for associative memory scheme in cyclic dynamic environments. In: 3rd International workshop on advanced computational intelligence (IWACI), (2010), Suzhou, Jiangsu, pp 32–39Google Scholar
  10. Chakhlevitch K, Cowling P (2008) Hyperheuristics: recent developments. Adaptive and multilevel metaheuristics. In: Cotta C, Sevaux M, Sirensen K (eds) Studies in computational intelligence, vol 136. Springer, Berlin, pp 3–29Google Scholar
  11. Chen X, Ong YS, Lim MH, Tan KC (2011) A multi-facet survey on memetic computation. IEEE Trans Evolut Comput 15(5):591–607CrossRefGoogle Scholar
  12. Cheng H, Yang S (2010) Multi-population genetic algorithms with immigrants scheme for dynamic shortest path routing problems in mobile ad hoc networks. EvoApplications (1):562–571Google Scholar
  13. Cowling PI, Kendall G, Soubeiga E (2000) A hyper-heuristic approach to scheduling a sales summit. In: 3rd International conference on practice and theory of automated timetabling III, PATAT 2000, Springer, Berlin, vol 2079Google Scholar
  14. Crowston WB, Glover F, Thompson GL, Trawick JD (1963) Probabilistic and parametric learning combinations of local job shop scheduling rules. ONR Research memorandum, GSIA, Carnegie Mellon University, Pittsburgh, p 117Google Scholar
  15. Cruz C, Gonzalez J, Pelta D (2011) Optimization in dynamic environments: a survey on problems, methods and measures. Soft Comput Fusion Found Methodol Appl 15:1427–1448Google Scholar
  16. Fernandes CM, Lima C, Rosa AC (2008a) Umdas for dynamic optimization problems. In: Proceedings of the 10th conference on genetic and evolutionary computation, ACM, GECCO ’08, pp 399–406Google Scholar
  17. Fernandes CM, Lima C, Rosa AC (2008b) Umdas for dynamic optimization problems. In: Proceedings of the 10th annual conference on Genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’08, pp 399–406Google Scholar
  18. Fisher H, Thompson GL (1963) Probabilistic learning combinations of local job-shop scheduling rules. In: Thompson GL (ed) Muth JF Industrial Scheduling. Prentice-Hall, Englewood Cliffs, pp 225–251Google Scholar
  19. Ghosh A, Muehlenbein H (2004) Univariate marginal distribution algorithms for non-stationary optimization problems. Int J Know Based Intell Eng Syst 8(3):129–138Google Scholar
  20. Goncalves AR, Zuben FJV (2011) Online learning in estimation of distribution algorithms for dynamic environments. In : IEEE Congress on Evolutionary Computation, pp 62–69Google Scholar
  21. Iacca G, Neri F, Mininno E, Ong YS, Lim MH (2012) Ockham’s razor in memetic computing: three stage optimal memetic exploration. Inf Sci 188:17–43MathSciNetCrossRefGoogle Scholar
  22. Jin Y, Branke J (2005) Evolutionary optimization in uncertain environments-a survey. IEEE Trans Evolut Comp 9(3):303–317CrossRefGoogle Scholar
  23. Kiraz B, Topcuoglu HR (2010) Hyper-heuristic approaches for the dynamic generalized assignment problem. In: 10th International conference on intelligent systems design and Applications (ISDA), 2010, pp 1487–1492Google Scholar
  24. Kiraz B, Etaner-Uyar AŞ, Özcan E (2011) An investigation of selection hyper-heuristics in dynamic environments. EvoApplications 2011. In: Lecture notes in computer science, vol 6624. Springer, Berlin, pp 314–323Google Scholar
  25. Kiraz B, Etaner-Uyar AŞ, Özcan E (2013) Selection hyper-heuristics in dynamic environments. J Oper Res Soc. doi: 10.1057/jors.2013.24
  26. Kiraz B, Etaner-Uyar AŞ, Özcan E (2013) An ant-based selection hyper-heuristic for dynamic environments. EvoApplications 2013. In: Lecture notes in computer science, vol 7835. Springer, Berlin, pp 626–635Google Scholar
  27. Kobliha M, Schwarz J, Očenášek J (2006) Bayesian optimization algorithms for dynamic problems. EvoWorkshops. In: Lecture notes in computer science, vol 3907. Springer, Berlin, pp 800–804Google Scholar
  28. Larrañaga P, Lozano JA (eds) (2002) Estimation of distribution algorithms: a new tool for evolutionary computation. Kluwer, BostonGoogle Scholar
  29. Lewis J, Hart E, Ritchie G (1998) A comparison of dominance mechanisms and simple mutation on nonstationary problems. In: Proceedings of parallel problem solving from nature, pp 139–148Google Scholar
  30. Li X, Mabu S, Mainali M, Hirasawa K, (2011) Probabilistic model building genetic network programming using multiple probability vectors. In: TENCON, (2010) IEEE Region 10 Conference. IEEE Region Conference, Fukuoka, pp 1398–1403Google Scholar
  31. Maturana J, Riff MC (2007) Solving the short-term electrical generation scheduling problem by an adaptive evolutionary approach. Eur J Oper Res 179:677–691CrossRefzbMATHGoogle Scholar
  32. Morrison RW (2004) Designing evolutionary algorithms for dynamic environments. Springer, BerlinGoogle Scholar
  33. Nareyek A (2004) Choosing search heuristics by non-stationary reinforcement learning. In: Metaheuristics. Kluwer Academic Publishers, Norwell, pp 523–544Google Scholar
  34. Neri F, Cotta C (2012) Memetic algorithms and memeting computing optimization: a literature review. Swarm Evolut Comput 2:1–14CrossRefGoogle Scholar
  35. Neri F, Cotta C, Moscato P (eds) (2012) Handbook of Memetic Algorithms. In: Studies in computational intelligence. Springer, BerlinGoogle Scholar
  36. Özcan E, Bilgin B, Korkmaz EE (2008) A comprehensive analysis of hyper-heuristics. Intell Data Anal 12:3–23Google Scholar
  37. Özcan E, Misir M, Ochoa G, Burke EK (2010) A reinforcement learning—great-deluge hyper-heuristic for examination timetabling. Int J Appl Metaheuristic Comput 1(1):39–59CrossRefGoogle Scholar
  38. Peng X, Gao X, Yang S (2011) Environment identification-based memory scheme for estimation of distribution algorithms in dynamic environments. Soft Comput 15:311–326CrossRefGoogle Scholar
  39. Ross P (2005) Hyper-heuristics. In: Burke EK, Kendall G (eds) Search methodologies: introductory tutorials in optimization and decision support techniques, vol 17. Springer, Berlin. pp 529–556Google Scholar
  40. Saramourtsis A, Damousis J, Bakirtzis A, Dokopoulos P (1996) Genetic algorithm solution to the economic dispatch problem. IEE Proc C 141(4):377–382Google Scholar
  41. Shakya S, Oliveira F, Owusu G (2007) An application of eda and ga to dynamic pricing. In: Proceedings of the 9th annual conference on genetic and evolutionary computation, New York, NY, USA, GECCO ’07, pp 585–592Google Scholar
  42. Simões A, Costa E (2008a) Evolutionary algorithms for dynamic environments: Prediction using linear regression and markov chains. In: Proceedings of the 10th international conference on parallel problem solving from nature: PPSN X. Springer, Berlin, pp 306–315.Google Scholar
  43. Simões A, Costa E (2008b) Evolutionary algorithms for dynamic environments: prediction using linear regression and markov chains. Tech. rep, Coimbra, PortugalGoogle Scholar
  44. Simões A, Costa E (2009a) Improving prediction in evolutionary algorithms for dynamic environments. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’09Google Scholar
  45. Simões A, Costa E (2009b) Prediction in evolutionary algorithms for dynamic environments using markov chains and nonlinear regression. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’09, pp 883–890Google Scholar
  46. Uludağ G, Kiraz B, Etaner-Uyar AŞ, Özcan E (2012) A framework to hybridise PBIL and a hyper-heuristic for dynamic environments. In: PPSN 2012: 12th International conference on parallel problem solving from nature, vol 7492. Springer, Berlin, pp 358–367Google Scholar
  47. Uludağ G, Kiraz B, Etaner-Uyar AŞ, Özcan E (2012b) Heuristic selection in a multi-phase hybrid approach for dynamic environments. In: 12th UK workshop on computational intelligence, Edinburgh, Scotland, UKCI12Google Scholar
  48. Ursem RK (2000) Multinational GA optimization techniques in dynamic environments. In: Proceedings of the Genetic Evol. Comput. Conf, pp 19–26Google Scholar
  49. Uyar Ş, Harmanci E (2005) A new population based adaptive domination change mechanism for diploid genetic algorithms in dynamic environments. Soft Comput 9(11):803–814CrossRefzbMATHGoogle Scholar
  50. Uyar Ş, Türkay B (2008) Evolutionary algorithms for the unit commitment problem. Turk J Electr Eng 16(3):239–255Google Scholar
  51. Uyar Ş, Türkay B, Keleş A (2011) A novel differential evolution application to short-term electrical power generation scheduling. Electr Power Energy Syst 33:1236–1242Google Scholar
  52. Valenzuela J, Smith AE (2002) A seeded memetic algorithm for large unit commitment problems. J Heuristics 8:173–195CrossRefGoogle Scholar
  53. Wineberg M, Oppacher F (2000) Enhancing the GA’s ability to cope with dynamic environments. In: Whitley (ed) Genetic and evolutionary computation conference. Morgan Kaufmann, Burlington, pp 3–10Google Scholar
  54. Wood AJ, Wollenberg BF (1996) Power generation, operation and control, 2nd edn. Wiley, New YorkGoogle Scholar
  55. Wu Y, Wang Y, Liu X (2010a) Multi-population based univariate marginal distribution algorithm for dynamic optimization problems. J Intell Robotic Syst 59(2):127–144Google Scholar
  56. Wu Y, Wang Y, Liu X, Ye J (2010b) Multi-population and diffusion umda for dynamic multimodal problems. J Syst Eng Electron 21(5):777–783Google Scholar
  57. Xingguang P, Demin X, Fubin Z (2011) On the effect of environment-triggered population diversity compensation methods for memory enhanced umda. In: Proceedings of the 30th Chinese control conference, pp 5430–5435Google Scholar
  58. Yang S (2004) Constructing dynamic test environments for genetic algorithms based on problem difficulty. In: In Proceedings of the 2004 congress on, evolutionary computation, pp 1262–1269Google Scholar
  59. Yang S (2005a) Memory-enhanced univariate marginal distribution algorithms for dynamic optimization problems. In: Proceedings of the 2005 congress on, evolutionary computation pp 2560–2567 Google Scholar
  60. Yang S (2005b) Population-based incremental learning with memory scheme for changing environments. In: Proceedings of the 2005 conference on genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’05, pp 711–718Google Scholar
  61. Yang S (2007) Explicit memory schemes for evolutionary algorithms in dynamic environments. In: Evolutionary computation in dynamic and uncertain environments, chap 1, pp 3–28Google Scholar
  62. Yang S, Richter H (2009) Hyper-learning for population-based incremental learning in dynamic environments. In: Proc. 2009 Congr. Evol. Comput, pp 682–689Google Scholar
  63. Yang S, Yao X (2005) Experimental study on population-based incremental learning algorithms for dynamic optimization problems. Soft Comput 9(11):815–834CrossRefzbMATHGoogle Scholar
  64. Yang S, Yao X (2008) Population-based incremental learning with associative memory for dynamic environments. IEEE Trans Evolut Comp 12:542–561CrossRefGoogle Scholar
  65. Yang S, Ong YS, Jin Y (eds) (2007) Evolutionary computation in dynamic and uncertain environments. In: Studies in computational intelligence, vol 51. Springer, BerlinGoogle Scholar
  66. Yuan B, Orlowska ME, Sadiq SW (2008) Extending a class of continuous estimation of distribution algorithms to dynamic problems. Optim Lett 2(3):433–443MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Gönül Uludağ
    • 1
  • Berna Kiraz
    • 1
  • A. Şima Etaner-Uyar
    • 2
  • Ender Özcan
    • 3
    Email author
  1. 1.Institute of Science and TechnologyIstanbul Technical UniversityIstanbulTurkey
  2. 2.Department of Computer EngineeringIstanbul Technical UniversityIstanbulTurkey
  3. 3.School of Computer ScienceUniversity of NottinghamNottinghamUK

Personalised recommendations