Automated Software Engineering

, Volume 25, Issue 4, pp 875–915 | Cite as

Developing and evolving a DSL-based approach for runtime monitoring of systems of systems

  • Rick RabiserEmail author
  • Jürgen Thanhofer-Pilisch
  • Michael Vierhauser
  • Paul Grünbacher
  • Alexander Egyed


Complex software-intensive systems are often described as systems of systems (SoS) due to their heterogeneous architectural elements. As SoS behavior is often only understandable during operation, runtime monitoring is needed to detect deviations from requirements. Today, while diverse monitoring approaches exist, most do not provide what is needed to monitor SoS, e.g., support for dynamically defining and deploying diverse checks across multiple systems. In this paper we report on our experiences of developing, applying, and evolving an approach for monitoring an SoS in the domain of industrial automation software, that is based on a domain-specific language (DSL). We first describe our initial approach to dynamically define and check constraints in SoS at runtime and then motivate and describe its evolution based on requirements elicited in an industry collaboration project. We furthermore describe solutions we have developed to support the evolution of our approach, i.e., a code generation approach and a framework to automate testing the DSL after changes. We evaluate the expressiveness and scalability of our new DSL-based approach using an industrial SoS. We also discuss lessons we learned. Our results show that while developing a DSL-based approach is a good solution to support industrial users, one must prepare the approach for evolution, by making it extensible and adaptable to future scenarios. Particularly, support for automated (re-)generation of tools and code after changes and automated testing are essential.


Systems of systems Requirements monitoring Constraint checking Domain-specific languages DSL evolution 



This work has been supported by the Christian Doppler Forschungsgesellschaft (Grant No. CDL MEVSS), Austria and Primetals Technologies. Michael Vierhauser’s work was funded by the Austrian Science Fund (FWF) under Grant No. J3998-N31. We want to thank Thomas Krismayer for his feedback and his help in setting up the evaluation environment.


  1. Aktug, I., Dam, M., Gurov, D.: Provably correct runtime monitoring. In: Formal Methods (FM’08), pp. 262–277. Springer (2008)Google Scholar
  2. Albuquerque, D., Cafeo, B., Garcia, A., Barbosa, S., Abrahão, S., Ribeiro, A.: Quantifying usability of domain-specific languages: an empirical study on software maintenance. J. Syst. Softw. 101, 245–259 (2015)CrossRefGoogle Scholar
  3. Autili, M., Grunske, L., Lumpe, M., Pelliccione, P., Tang, A.: Aligning qualitative, real-time, and probabilistic property specification patterns using a structured English grammar. IEEE Trans. Softw. Eng. 41(7), 620–638 (2015)CrossRefGoogle Scholar
  4. Baresi, L., Guinea, S.: Event-based multi-level service monitoring. In: Proceedings of the 20th International Conference on Web Services, pp. 83–90. IEEE (2013)Google Scholar
  5. Bauer, A., Leucker, M., Schallhart, C.: Monitoring of real-time properties. In: Foundations of Software Technology and Theoretical Computer Science, pp. 260–272. Springer (2006)Google Scholar
  6. Bertolino, A., Calabrò, A., Lonetti, F., Di Marco, A., Sabetta, A.: Towards a model-driven infrastructure for runtime monitoring. In: Software Engineering for Resilient Systems, pp. 130–144. Springer (2011)Google Scholar
  7. Bubak, M., Funika, W., Smetek, M., Kiliański, Z., Wismüller, R.: Event handling in the J-OCM monitoring system. In: Parallel Processing and Applied Mathematics, pp. 344–351. Springer (2004)Google Scholar
  8. Bures, T., Hnetynka, P., Plasil, F.: Strengthening architectures of smart CPS by modeling them as runtime product-lines. In: Proceedings of the 17th International ACM SIGSOFT Symposium on Component-Based Software Engineering, pp. 91–96. ACM (2014)Google Scholar
  9. Cailliau, A., van Lamsweerde, A.: Runtime monitoring and resolution of probabilistic obstacles to system goals. In: 2017 IEEE/ACM 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, pp. 1–11. IEEE (2017)Google Scholar
  10. Calinescu, R., Ghezzi, C., Kwiatkowska, M.Z., Mirandola, R.: Self-adaptive software needs quantitative verification at runtime. Commun. ACM 55(9), 69–77 (2012)CrossRefGoogle Scholar
  11. Cazzola, W., Poletti, D.: DSL evolution through composition. In: Proceedings of the 7th Workshop on Reflection, AOP and Meta-Data for Software Evolution, pp. 6:1–6:6. ACM (2010)Google Scholar
  12. Chen, F., d’Amorim, M., Roşu, G.: A formal monitoring-based framework for software development and analysis. In: Formal Methods and Software Engineering, pp. 357–372. Springer (2004)Google Scholar
  13. De Geest, G., Vermolen, S., Van Deursen, A., Visser, E.: Generating version convertors for domain-specific languages. In: Proceedings of the 15th Working Conference on Reverse Engineering, pp. 197–201. IEEE (2008)Google Scholar
  14. Degueule, T., Combemale, B., Blouin, A., Barais, O.: Reusing legacy DSLs with melange. In: Proceedings of the Workshop on Domain-Specific Modeling, pp. 45–46. ACM (2015)Google Scholar
  15. Delgado, N., Gates, A.Q., Roach, S.: A taxonomy and catalog of runtime software-fault monitoring tools. IEEE Trans. Softw. Eng. 30(12), 859–872 (2004)CrossRefGoogle Scholar
  16. Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Proceedings of the International Conference on Software Engineering, pp. 411–420. IEEE (1999)Google Scholar
  17. Egyed, A.: Instant consistency checking for the UML. In: Proceedings of the 28th International Conference on Software Engineering, pp. 381–390. ACM (2006)Google Scholar
  18. Eichelberger, H., Schmid, K.: Flexible resource monitoring of Java programs. J. Syst. Softw. 93, 163–186 (2014)CrossRefGoogle Scholar
  19. Fickas, S., Feather, M.S.: Requirements monitoring in dynamic environments. In: Proceedings of the 2nd IEEE International Symposium on Requirements Engineering, pp. 140–147. IEEE (1995)Google Scholar
  20. Ghezzi, C., Mocci, A., Sangiorgio, M.: Runtime monitoring of component changes with Spy@Runtime. In: 34th International Conference on Software Engineering, pp. 1403–1406. IEEE (2012)Google Scholar
  21. Gunadi, H., Tiu, A.: Efficient runtime monitoring with metric temporal logic: a case study in the Android operating system. In: Proceedings Formal Methods 2014, pp. 296–311. Springer (2014)Google Scholar
  22. Hermans, F., Pinzger, M., van Deursen, A.: Domain-specific languages in practice: a user study on the success factors. In: Schürr, A., Selic, B. (eds.) Model Driven Engineering Languages and Systems, pp. 423–437. Springer, Berlin (2009)CrossRefGoogle Scholar
  23. Izquierdo, J.L.C., Cabot, J.: Community-driven language development. In: Proceedings of the 2012 4th International Workshop on Modeling in Software Engineering, pp. 29–35 (2012)Google Scholar
  24. Juergens, E., Pizka, M.: The language evolver lever—tool demonstration. Electr. Notes Theor. Comput. Sci. 164(2), 55–60 (2006)CrossRefGoogle Scholar
  25. Kanstrén, T.: A systematic review and taxonomy of runtime invariance in software behaviour. Int. J. Adv. Softw. 4(3 and 4), 256–274 (2011)Google Scholar
  26. Kim, M., Viswanathan, M., Kannan, S., Lee, I., Sokolsky, O.: Java-MaC: a run-time assurance approach for Java programs. Form. Methods Syst. Des. 24(2), 129–155 (2004)CrossRefGoogle Scholar
  27. Kiviluoma, K., Koskinen, J., Mikkonen, T.: Run-time monitoring of architecturally significant behaviors using behavioral profiles and aspects. In: Proceedings of the 2006 International Symposium on Software Testing and Analysis, pp. 181–190. ACM (2006)Google Scholar
  28. Krismayer, T., Rabiser, R., Grünbacher, P.: Mining constraints for event-based monitoring in systems of systems. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, pp. 826–831. IEEE (2017)Google Scholar
  29. Luckham, D.C.: Event Processing for Business: Organizing the Real-Time Enterprise. Wiley, New York (2011)Google Scholar
  30. Ludwig, T., Wismueller, R., Sundcram, V., Bode, A.: OMIS: On-line Monitoring Interface Specification (v. 2.0). Technical report TUM-I9733, Technische Universität München (1997)Google Scholar
  31. Maiden, N.: Monitoring our requirements. IEEE Softw. 30(1), 16–17 (2013)CrossRefGoogle Scholar
  32. Maier, M.W.: Architecting principles for systems-of-systems. Syst. Eng. 1(4), 267–284 (1998)CrossRefGoogle Scholar
  33. Mansouri-Samani, M., Sloman, M.: Monitoring distributed systems. IEEE Netw. 7(6), 20–30 (1993)CrossRefGoogle Scholar
  34. Meyers, B., Vangheluwe, H.: A framework for evolution of modelling languages. Sci. Comput. Program. 76(12), 1223–1246 (2011)CrossRefGoogle Scholar
  35. Montali, M., Maggi, F.M., Chesani, F., Mello, P., van der Aalst, W.M.: Monitoring business constraints with the event calculus. ACM Trans. Intell. Syst. Technol. 5(1), 17:1–17:30 (2014)Google Scholar
  36. Muccini, H., Polini, A., Ricci, F., Bertolino, A.: Monitoring architectural properties in dynamic component-based systems. In: Component-Based Software Engineering, LNCS 4608, pp. 124–139. Springer (2007)Google Scholar
  37. Nielsen, C.B., Larsen, P.G., Fitzgerald, J., Woodcock, J., Peleska, J.: Systems of systems engineering: basic concepts, model-based techniques, and research directions. ACM Comput. Surv. 48(2), 18:1–18:41 (2015)CrossRefGoogle Scholar
  38. Nikolov, N., Rossini, A., Kritikos, K.: Integration of DSLs and migration of models: a case study in the cloud computing domain. Procedia Comput. Sci. 68, 53–66 (2015)CrossRefGoogle Scholar
  39. Paschke, A.: RBSLA—a declarative rule-based service level agreement language based on RuleML. In: International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce, vol. 2, pp. 308–314. IEEE (2005)Google Scholar
  40. Pizka, M., Jurgens, E.: Automating language evolution. In: Proceedings of the 1st Joint IEEE/IFIP Symposium on Theoretical Aspects of Software Engineering, pp. 305–315. IEEE (2007)Google Scholar
  41. Rabiser, R., Vierhauser, M., Grünbacher, P.: Assessing the usefulness of a requirements monitoring tool: a study involving industrial software engineers. In: Proceedings of the 38th International Conference on Software Engineering, Companion, pp. 122–131. ACM (2016)Google Scholar
  42. Rabiser, R., Vierhauser, M., Grünbacher, P.: Variability management for a runtime monitoring infrastructure. In: Proceedings of the 9th International Workshop on Variability Modelling of Software-Intensive Systems, pp. 35–42. ACM (2015)Google Scholar
  43. Rabiser, R., Guinea, S., Vierhauser, M., Baresi, L., Grünbacher, P.: A comparison framework for runtime monitoring approaches. J. Syst. Softw. 125(March), 309–321 (2017)CrossRefGoogle Scholar
  44. Robinson, W.N.: A requirements monitoring framework for enterprise systems. Requir. Eng. 11(1), 17–41 (2006)CrossRefGoogle Scholar
  45. Robinson, W.N.: Extended OCL for goal monitoring. Electron. Commun. Eur. Assoc. Softw. Sci. Technol. 9(1), 1–12 (2008)MathSciNetGoogle Scholar
  46. Sammapun, U., Lee, I., Sokolsky, O.: RT-MaC: runtime monitoring and checking of quantitative and probabilistic properties. In: Proceedings of the 11th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, pp. 147–153. IEEE (2005)Google Scholar
  47. Schmidt, M., Wider, A., Scheidgen, M., Fischer, J., von Klinski, S.: Refactorings in language development with asymmetric bidirectional model transformations. In: Proceedings of the 16th International SDL Forum on Model-Driven Dependability Engineering, pp. 222–238. Springer (2013)Google Scholar
  48. Skene, J., Emmerich, W.: Engineering runtime requirements-monitoring systems using MDA technologies. In: Nicola, R.D., Sangiorgi, D. (eds.) Trustworthy Global Computing, pp. 319–333. Springer (2005). zbMATHGoogle Scholar
  49. Spanoudakis, G., Kloukinas, C., Mahbub, K.: The SERENITY runtime monitoring framework. In: Kokolakis, S., Gómez, A.M., Spanoudakis, G. (eds.) Security and Dependability for Ambient Intelligence, pp. 213–237. Springer (2009). Google Scholar
  50. Spinellis, D.: Notable design patterns for domain-specific languages. J. Syst. Softw. 56(1), 91–99 (2001)CrossRefGoogle Scholar
  51. Thanhofer-Pilisch, J., Lang, A., Vierhauser, M., Rabiser, R.: A systematic mapping study on DSL evolution. In: Proceedings of the 43rd Euromicro Conference on Software Engineering and Advanced Applications, pp. 149–156. IEEE (2017)Google Scholar
  52. van den Bos, J., van der Storm, T.: A case study in evidence-based DSL evolution. In: Proceedings of the 9th European Conference on Modelling Foundations and Applications, pp. 207–219. Springer (2013)Google Scholar
  53. Van Deursen, A., Klint, P., Visser, J.: Domain-specific languages: an annotated bibliography. ACM SIGPLAN Not. 35(6), 26–36 (2000)CrossRefGoogle Scholar
  54. van Hoorn, A., Waller, J., Hasselbring, W.: Kieker: A framework for application performance monitoring and dynamic software analysis. In: Proceedings of the 3rd Joint International Conference on Performance Engineering, pp. 247–248. ACM (2012)Google Scholar
  55. van Lamsweerde, A.: Requirements Engineering: From System Goals to UML Models to Software Specifications. Wiley, New York (2009)Google Scholar
  56. Vermolen, S., Visser, E.: Heterogeneous coupled evolution of software languages. In: Proceedings of the 11th International Conference on Model Driven Engineering Languages and Systems, pp. 630–644. Springer (2008)Google Scholar
  57. Vierhauser, M., Grünbacher, P., Egyed, A., Rabiser, R., Heider, W.: Flexible and scalable consistency checking on product line variability models. In: Proceedings of the International Conference on Automated Software Engineering, pp. 63–72. ACM (2010)Google Scholar
  58. Vierhauser, M., Grünbacher, P., Heider, W., Holl, G., Lettner, D.: Applying a consistency checking framework for heterogeneous models and artifacts in industrial product lines. In: Model Driven Engineering Languages and Systems, pp. 531–545. Springer (2012)Google Scholar
  59. Vierhauser, M., Rabiser, R., Cleland-Huang, J.: From requirements monitoring to diagnosis support in system of systems. In: Proceedings of the 23rd International Working Conference on Requirements Engineering: Foundation for Software Quality, pp. 181–187. Springer (2017)Google Scholar
  60. Vierhauser, M., Rabiser, R., Grünbacher, P., Egyed, A.: Developing a DSL-based approach for event-based monitoring of systems of systems: experiences and lessons learned. In: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering, pp. 715–725. IEEE (2015)Google Scholar
  61. Vierhauser, M., Rabiser, R., Grünbacher, P.: A case study on testing, commissioning, and operation of very-large-scale software systems. In: Proceedings of the 36th International Conference on Software Engineering, Companion, pp. 125–134. ACM (2012)Google Scholar
  62. Vierhauser, M., Rabiser, R., Grünbacher, P.: Requirements monitoring frameworks: a systematic review. Inf. Softw. Technol. 80, 89–109 (2016a)CrossRefGoogle Scholar
  63. Vierhauser, M., Rabiser, R., Grünbacher, P., Seyerlehner, K., Wallner, S., Zeisel, H.: ReMinds: a flexible runtime monitoring framework for systems of systems. J. Syst. Softw. 112, 123–136 (2016b)CrossRefGoogle Scholar
  64. Viswanathan, M., Kim, M.: Foundations for the run-time monitoring of reactive systems—fundamentals of the MaC language. In: Liu, Z., Araki, K. (eds.) Theoretical Aspects of Computing, 1st International ICTAC Colloquium, (Revised Selected Papers), Lecture Notes in Computer Science 3407, pp. 543–556. Springer (2005)Google Scholar
  65. Voelter, M., Visser, E.: Product line engineering using domain-specific languages. In: Proceedings of the 15th International Software Product Line Conference, IEEE CS, pp. 70–79 (2011)Google Scholar
  66. Völz, M., Koldehofe, B., Rothermel, K.: Supporting strong reliability for distributed complex event processing systems. In: Proceedings of the 13th International Conference on High Performance Computing & Communication, pp. 477–486. IEEE (2011)Google Scholar
  67. Whittle, J., Sawyer, P., Bencomo, N., Cheng, B., Bruel, J.M.: RELAX: a language to address uncertainty in self-adaptive systems requirement. Requir. Eng. 15(2), 177–196 (2010)CrossRefGoogle Scholar
  68. Wieringa, R., Maiden, N., Mead, N., Rolland, C.: Requirements engineering paper classification and evaluation criteria: a proposal and a discussion. Requir. Eng. 11(1), 102–107 (2006)CrossRefGoogle Scholar
  69. Zhang, P., Li, B., Muccini, H., Sun, M.: An approach to monitor scenario-based temporal properties in Web service compositions. In: Advanced Web and Network Technologies, and Applications, pp. 144–154. Springer (2008)Google Scholar
  70. Zhang, P., Li, W., Wan, D., Grunske, L.: Monitoring of probabilistic timed property sequence charts. Softw. Pract. Exp. 41(7), 841–866 (2011)CrossRefGoogle Scholar
  71. Zschaler, S., Kolovos, D.S., Drivalos, N., Paige, R.F., Rashid, A.: Domain-specific metamodelling languages for software language engineering. In: Proceedings of the 2nd International Conference on Software Language Engineering, pp. 334–353. Springer (2009)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Christian Doppler Lab. MEVSSJohannes Kepler University LinzLinzAustria
  2. 2.Computer Science and EngineeringUniversity of Notre DameNotre DameUSA
  3. 3.Institute for Software Systems EngineeringJohannes Kepler University LinzLinzAustria

Personalised recommendations