Self-adaptation for Individual Self-aware Computing Systems

  • Martina MaggioEmail author
  • Tarek Abdelzaher
  • Lukas Esterle
  • Holger Giese
  • Jeffrey O. Kephart
  • Ole J. Mengshoel
  • Alessandro V. Papadopoulos
  • Anders Robertsson
  • Katinka Wolter


This chapter discusses the role of self-awareness for adaptation at the individual level, when one single entity receives inputs both from itself or some of its components and from the external environment and uses the input to adjust to the current conditions. The chapter reviews the most widely used techniques for self-adaptation and identifies the role of self-awareness for each of the techniques and the metrics used to evaluate these techniques. Finally, we pave the way toward the following chapter, which discusses multiple entity adaptation, by introducing the interaction of different self-adaptation techniques at the level of the single individual.


Utility Function Reinforcement Learning Adaptation Strategy Reward Function Reinforcement Learning Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This work was partially supported by the Swedish Research Council (VR) for the projects “Cloud Control” and “Power and temperature control for large-scale computing infrastructures,” and through the LCCC Linnaeus and ELLIIT Excellence Centers.


  1. 1.
    Karl Johan Åström and Bo Bernhardsson. Comparison of Riemann and Lebesque sampling for first order stochastic systems. In Proceedings of the 41st IEEE Conference on Decision and Control, 2002.Google Scholar
  2. 2.
    Karl Johan Åström and Richard M. Murray. Feedback systems: an introduction for scientists and engineers. Princeton university press, 2010.Google Scholar
  3. 3.
    Peter Auer, Nicol Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235–256, 2002.Google Scholar
  4. 4.
    Josep Ll. Berral, Íñigo Goiri, Ramón Nou, Ferran Julià, Jordi Guitart, Ricard Gavaldà, and Jordi Torres. Towards energy-aware scheduling in data centers using machine learning. In Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking, e-Energy ’10, pages 215–224, New York, NY, USA, 2010. ACM.Google Scholar
  5. 5.
    Donald A Berry and Bert Fristedt. Bandit problems: sequential allocation of experiments (Monographs on statistics and applied probability). Springer, 1985.Google Scholar
  6. 6.
    Marco C. Campi, Simone Garatti, and Maria Prandini. The scenario approach for systems and control design. Annual Reviews in Control, 33(2):149–157, 2009.Google Scholar
  7. 7.
    Eva Darulova and Viktor Kuncak. Sound compilation of reals. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, pages 235–248, New York, NY, USA, 2014. ACM.Google Scholar
  8. 8.
    Nicolás D’Ippolito. Synthesis of event-based controllers: A software engineering challenge. In Proceedings of the 34th International Conference on Software Engineering, pages 1547–1550. IEEE Press, 2012.Google Scholar
  9. 9.
    Xavier Dutreilh, Aurélien Moreau, Jacques Malenfant, Nicolas Rivierre, and Isis Truck. From data center resource allocation to control theory and back. In Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing, CLOUD ’10, pages 410–417, Washington, DC, USA, 2010. IEEE Computer Society.Google Scholar
  10. 10.
    Eyal Even-Dar and Yishay Mansour. Learning rates for Q-learning. Journal of Machine Learning Research, 5:1–25, December 2004.Google Scholar
  11. 11.
    F. Farahnakian, P. Liljeberg, and J. Plosila. Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In Parallel, Distributed and Network-Based Processing (PDP), 2014 22nd Euromicro International Conference on, pages 500–507, Feb 2014.Google Scholar
  12. 12.
    Robert France and Bernhard Rumpe. Model-driven development of complex software: A research roadmap. In 2007 Future of Software Engineering, FOSE ’07, pages 37–54, Washington, DC, USA, 2007. IEEE Computer Society.Google Scholar
  13. 13.
    Martin Franzle and Christian Herde. HySAT: An efficient proof engine for bounded model checking of hybrid systems. Formal Methods in System Design, 30(3):179–198, 2007.Google Scholar
  14. 14.
    Severino F. Galán and Ole J. Mengshoel. Generalized crowding for genetic algorithms. In Genetic and Evolutionary Computation Conference, GECCO 2010, Proceedings, Portland, Oregon, USA, July 7-11, 2010, pages 775–782, 2010.Google Scholar
  15. 15.
    Sicun Gao, Soonho Kong, and E.M. Clarke. Satisfiability modulo ODEs. In Formal Methods in Computer-Aided Design (FMCAD), 2013, pages 105–112, Oct 2013.Google Scholar
  16. 16.
    David Garlan, Shang-Wen Cheng, An-Cheng Huang, Bradley Schmerl, and Peter Steenkiste. Rainbow: Architecture-based self-adaptation with reusable infrastructure. Computer, 37(10):46–54, October 2004.Google Scholar
  17. 17.
    Jin Heo and Tarek Abdelzaher. Adaptguard: Guarding adaptive systems from instability. In Proceedings of the 6th International Conference on Autonomic Computing, ICAC ’09, pages 77–86, New York, NY, USA, 2009. ACM.Google Scholar
  18. 18.
    Zhihao Jiang and R. Mangharam. Modeling cardiac pacemaker malfunctions with the virtual heart model. In Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pages 263–266, Aug 2011.Google Scholar
  19. 19.
    Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. Reinforcement learning: a survey. Journal of Artificial Intelligence Research, 4:237–285, 1996.Google Scholar
  20. 20.
    Jeffrey O Kephart and William E Walsh. An artificial intelligence perspective on autonomic computing policies. In Policies for Distributed Systems and Networks, 2004. POLICY 2004. Proceedings. Fifth IEEE International Workshop on, pages 3–12. IEEE, 2004.Google Scholar
  21. 21.
    M.M.H. Khan, Jin Heo, Shen Li, and T. Abdelzaher. Understanding vicious cycles in server clusters. In Distributed Computing Systems (ICDCS), 2011 31st International Conference on, pages 645–654, June 2011.Google Scholar
  22. 22.
    Cristian Klein, Alessandro Vittorio Papadopoulos, Manfred Dellkrantz, Jonas Dürango, Martina Maggio, Karl-Erik Årzén, Francisco Hernández-Rodriguez, and Erik Elmroth. Improving cloud service resilience using brownout-aware load-balancing. In Reliable Distributed Systems (SRDS), 2014 IEEE 33rd International Symposium on, pages 31–40, Oct 2014.Google Scholar
  23. 23.
    Ian Doré Landau and Gianluca Zito. Digital Control Systems: Design, Identification and Implementation. Communications and Control Engineering. Springer-Verlag London, 2006.Google Scholar
  24. 24.
    Shen Li, Shaohan Hu, Raghu Ganti, Mudhakar Srivatsa, and Tarek Abdelzaher. Pyro: A spatial-temporal big-data storage system. In 2015 USENIX Annual Technical Conference (USENIX ATC 15), pages 97–109, Santa Clara, CA, July 2015. USENIX Association.Google Scholar
  25. 25.
    Pattie Maes. Concepts and experiments in computational reflection. In Conference Proceedings on Object-oriented Programming Systems, Languages and Applications, OOPSLA ’87, pages 147–155, New York, NY, USA, 1987. ACM.Google Scholar
  26. 26.
    Ole J. Mengshoel, Severino F. Galán, and Antonio de Dios. Adaptive generalized crowding for genetic algorithms. Inf. Sci., 258:140–159, 2014.Google Scholar
  27. 27.
    Ole J. Mengshoel and David E. Goldberg. The crowding approach to niching in genetic algorithms. Evol. Comput., 16(3):315–354, September 2008.Google Scholar
  28. 28.
    Ole J. Mengshoel, Abe Ishihara, and Erik Reed. Reactive bayesian network computation using feedback control: An empirical study. In Proceedings of the Ninth UAI Bayesian Modeling Applications Workshop, Catalina Island, United States, August 18, 2012, pages 44–54, 2012.Google Scholar
  29. 29.
    Ole J. Mengshoel, Dan Roth, and David C. Wilkins. Portfolios in stochastic local search: Efficiently computing most probable explanations in bayesian networks. J. Autom. Reasoning, 46(2):103–160, 2011.Google Scholar
  30. 30.
    Alessandro Vittorio Papadopoulos, Ahmed Ali-Eldin, Karl-Erik Årzén, Johan Tordsson, and Erik Elmroth. PEAS: A performance evaluation framework for auto-scaling strategies in cloud applications. ACM Transactions on Modeling and Performance Evaluation of Computing Systems, 2016.Google Scholar
  31. 31.
    L. Parolini, B. Sinopoli, B.H. Krogh, and Zhikui Wang. A cyber-physical systems approach to data center modeling and control for energy efficiency. Proceedings of the IEEE, 100(1):254–268, Jan 2012.Google Scholar
  32. 32.
    Wenyi Qian, Xin Peng, Bihuan Chen, John Mylopoulos, Huanhuan Wang, and Wenyun Zhao. Rationalism with a dose of empiricism: Case-based reasoning for requirements-driven self-adaptation. In Requirements Engineering Conference (RE), 2014 IEEE 22nd International, pages 113–122, Aug 2014.Google Scholar
  33. 33.
    E. Reed, A. Ishihara, and O. Mengshoel. Adaptive control of bayesian network computation. In Resilient Control Systems (ISRCS), 2012 5th International Symposium on, pages 106–111, Aug 2012.Google Scholar
  34. 34.
    Francesca Rossi, Peter van Beek, and Toby Walsh. Handbook of Constraint Programming (Foundations of Artificial Intelligence). Elsevier Science Inc., New York, NY, USA, 2006.Google Scholar
  35. 35.
    Jun Shi, Ole J. Mengshoel, and Dipan K. Pal. Feedback control for multi-modal optimization using genetic algorithms. In Genetic and Evolutionary Computation Conference, GECCO ’14, Vancouver, BC, Canada, July 12-16, 2014, pages 839–846, 2014.Google Scholar
  36. 36.
    Filippo Sironi, Martina Maggio, Riccardo Cattaneo, Giovanni Francesco Del Nero, Donatella Sciuto, and Marco Domenico Santambrogio. ThermOS: System support for dynamic thermal management of chip multi-processors. In Proceedings of the 22Nd International Conference on Parallel Architectures and Compilation Techniques, PACT ’13, pages 41–50, Piscataway, NJ, USA, 2013. IEEE Press.Google Scholar
  37. 37.
    Richard S. Sutton and Andrew G. Barto. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998.Google Scholar
  38. 38.
    G. Tesauro, N.K. Jong, R. Das, and M.N. Bennani. A hybrid reinforcement learning approach to autonomic resource allocation. In Autonomic Computing, 2006. ICAC ’06. IEEE International Conference on, pages 65–73, June 2006.Google Scholar
  39. 39.
    Gerald Tesauro, Rajarshi Das, William E Walsh, and Jeffrey O Kephart. Utility-function-driven resource allocation in autonomic systems. In null, pages 342–343. IEEE, 2005.Google Scholar
  40. 40.
    Matthias Tichy and Holger Giese. A self-optimizing run-time architecture for configurable dependability of services. In Rogério de Lemos, Cristina Gacek, and Alexander Romanovsky, editors, Architecting Dependable Systems II, volume 3069 of Lecture Notes in Computer Science, pages 25–50. Springer Berlin Heidelberg, 2004.Google Scholar
  41. 41.
    K. Van Moffaert, T. Brys, A. Chandra, L. Esterle, P.R. Lewis, and A. Nowe. A novel adaptive weight selection algorithm for multi-objective multi-agent reinforcement learning. In Neural Networks (IJCNN), 2014 International Joint Conference on, pages 2306–2314, July 2014.Google Scholar
  42. 42.
    Thomas Vogel, Andreas Seibel, and Holger Giese. The role of models and megamodels at runtime. In Proceedings of the 2010 International Conference on Models in Software Engineering, MODELS’10, pages 224–238, Berlin, Heidelberg, 2011. Springer-Verlag.Google Scholar
  43. 43.
    Christopher J.C.H. Watkins and Peter Dayan. Q-learning. Machine Learning, 8(3-4):279–292, 1992.Google Scholar
  44. 44.
    R.R. Yager and L.A. Zadeh. An Introduction to Fuzzy Logic Applications in Intelligent Systems. The Springer International Series in Engineering and Computer Science. Springer US, 2012.Google Scholar
  45. 45.
    Jingling Yuan, Xing Jiang, Luo Zhong, and Hui Yu. Energy aware resource scheduling algorithm for data center using reinforcement learning. In Intelligent Computation Technology and Automation (ICICTA), 2012 Fifth International Conference on, pages 435–438, Jan 2012.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Martina Maggio
    • 1
    Email author
  • Tarek Abdelzaher
    • 2
  • Lukas Esterle
    • 3
  • Holger Giese
    • 4
  • Jeffrey O. Kephart
    • 5
  • Ole J. Mengshoel
    • 6
  • Alessandro V. Papadopoulos
    • 1
  • Anders Robertsson
    • 1
  • Katinka Wolter
    • 7
  1. 1.Department of Automatic ControlLund UniversityLundSweden
  2. 2.Department of Computer ScienceUniversity of Illinois at Urbana ChampaignUrbanaUSA
  3. 3.Vienna University of TechnologyViennaAustria
  4. 4.Hasso-Plattner-Institut fr Softwaresystemtechnik GmbHPotsdamGermany
  5. 5.Thomas J. Watson Research CenterHawthorneUSA
  6. 6.Department of Electrical and Computer EngineeringCarnegie Mellon UniversityPittsburghUSA
  7. 7.Institute of Computer Science, Freie Universität BerlinBerlinGermany

Personalised recommendations