A Q-Leaning-Based On-Line Planning Approach to Autonomous Architecture Discovery for Self-managed Software

  • Dongsun Kim
  • Sooyong Park
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5333)


Two key concepts for architecture-based self-managed software are flexibility and autonomy. Recent discussion have focused on flexibility in self-management, but the software engineering community has not been paying attention to autonomy as much as flexibility in self-management. In this paper, we focus on achieving the autonomy of software systems by on-line planning in which a software system can decide an appropriate plan in the presence of change, evaluate the result of the plan, and learn the result. Our approach applies Q-leaning, which is one of the reinforcement learning techniques, to self-managed systems. The paper presents a case study to illustrate the approach. The result of the case study shows that our approach is effective for self-management.


Evaluation Step Architectural Change Software Engineering Community Reinforcement Learning Technique Root Goal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Laddaga, R.: Active software. In: Robertson, P., Shrobe, H.E., Laddaga, R. (eds.) IWSAS 2000. LNCS, vol. 1936, pp. 11–26. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  2. 2.
    Garlan, D., Kramer, J., Wolf, A. (eds.): WOSS 2002: Proceedings of the first workshop on Self-healing systems. ACM Press, New York (2002)Google Scholar
  3. 3.
    Kramer, J., Magee, J.: Self-managed systems: an architectural challenge. In: FOSE 2007: 2007 Future of Software Engineering, pp. 259–268. IEEE Computer Society Press, Washington (2007)Google Scholar
  4. 4.
    Oreizy, P., Medvidovic, N., Taylor, R.N.: Architecture-based runtime software evolution. In: ICSE 1998: Proceedings of the 20th international conference on Software engineering, pp. 177–186. IEEE Computer Society Press, Washington (1998)Google Scholar
  5. 5.
    Floch, J., Hallsteinsen, S.O., Stav, E., Eliassen, F., Lund, K., Gjørven, E.: Using architecture models for runtime adaptability. IEEE Software 23(2), 62–70 (2006)CrossRefGoogle Scholar
  6. 6.
    Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD thesis. Cambridge University, Cambridge (1989)Google Scholar
  7. 7.
    Shin, M.E.: Self-healing components in robust software architecture for concurrent and distributed systems. Sci. Comput. Program. 57(1), 27–44 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Garlan, D., Cheng, S.W., Huang, A.C., Schmerl, B.R., Steenkiste, P.: Rainbow: Architecture-based self-adaptation with reusable infrastructure. IEEE Computer 37(10), 46–54 (2004)CrossRefGoogle Scholar
  9. 9.
    Klein, G.: Flexecution as a paradigm for replanning, part 1. IEEE Intelligent Systems 22(5), 79–83 (2007)CrossRefGoogle Scholar
  10. 10.
    Klein, G.: Flexecution, part 2: Understanding and supporting flexible execution. IEEE Intelligent Systems 22(6), 108–112 (2007)CrossRefGoogle Scholar
  11. 11.
    Harman, M., Jones, B.F.: Search-based software engineering. Information & Software Technology 43(14), 833–839 (2001)CrossRefGoogle Scholar
  12. 12.
    van Lamsweerde, A.: Goal-oriented requirements engineering: A guided tour. In: 5th IEEE International Symposium on Requirements Engineering (RE 2001), Toronto, Canada, August 27-31, 2001, p. 249 (2001)Google Scholar
  13. 13.
    Sutcliffe, A.G., Maiden, N.A.M., Minocha, S., Manuel, D.: Supporting scenario-based requirements engineering. IEEE Trans. Software Eng. 24(12), 1072–1088 (1998)CrossRefGoogle Scholar
  14. 14.
    Rolland, C., Souveyet, C., Achour, C.B.: Guiding goal modeling using scenarios. IEEE Trans. Software Eng. 24(12), 1055–1071 (1998)CrossRefGoogle Scholar
  15. 15.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (1998)Google Scholar
  16. 16.

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Dongsun Kim
    • 1
  • Sooyong Park
    • 1
  1. 1.Department of Computer Science and EngineeringSogang UniversitySeoulRepublic of Korea

Personalised recommendations