Towards Automated Defect Analysis Using Execution Traces of Scenario-Based Models
Modern software systems are so complex that at times engineers find it difficult to understand why a system behaves as it does under certain conditions, and, in particular, which conditions trigger specific behavior. This adds a significant burden to tasks like debugging or maintenance. Scenario-based specifications can mitigate some of the problems engineers face thanks to the scenarios’ intuitiveness, executability and amenability to formal methods such as verification and synthesis. However, as a specification grows it becomes increasingly difficult to fully comprehend the interplay of all scenarios, thus again making it difficult to understand the final system’s behavior. Therefore we propose a (semi-)automatic trace analysis method. It incorporates automatic techniques for identifying interesting traces, or subsequences within traces, from large sets of long execution traces. Developers are interested in knowing whether certain specification properties hold: if a property holds, what are possible executions which are evidence of this? If a property does not hold, what are examples that violate it? Scenario-based specifications are well-suited to generating meaningful execution traces due to being based on events which are meaningful in the domain of the system under design. A key observation we made was that interesting properties of a trace are often encoded in just one or very few scenarios. These concise scenarios, describing desired or forbidden behavior, are often already part of the specification or should be added to it as they encode implicitly made assumptions.
This paper incorporates and substantially extends the material of the paper published in MODELSWARD 2018 “Towards Systematic and Automatic Handling of Execution Traces Associated with Scenario-based Models” .
KeywordsSoftware engineering System engineering Scenario-based programming Behavioral programming Abstraction Debugging Program repair Execution trace Event log
This work has been funded in part by grants from the German-Israeli Foundation for Scientific Research and Development (GIF), from the Minerva foundation, and from the Israel Science Foundation (ISF).
- 1.Greenyer, J., Gritzner, D., Harel, D., Marron, A.: Towards systematic and automatic handling of execution traces associated with scenario-based models. In: Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development (MODELSWARD), pp. 560–566 (2018)Google Scholar
- 4.Greenyer, J., et al.: Scenarios@run.time - distributed execution of specifications on IoT-connected robots. In: 10th International Workshop on Models@Run.Time (MRT), Co-located with MODELS 2015, CEUR Workshop Proceedings (2015)Google Scholar
- 5.Greenyer, J., Gritzner, D., Katz, G., Marron, A.: Scenario-based modeling and synthesis for reactive systems with dynamic system structure in scenariotools. In: Proceedings of the MoDELS 2016 Demo and Poster Sessions, Co-located with ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MoDELS), CEUR (2016)Google Scholar
- 6.Gritzner, D., Greenyer, J.: Controller synthesis and PCL code generation from scenario-based GR (1) robot specifications. In: Proceedings of the 4th Workshop on Model-Driven Robot Software Engineering (MORSE 2017), Co-located with Software Technologies: Applications and Foundations (STAF) (2017)Google Scholar
- 8.Harel, D., Katz, G., Marelly, R., Marron, A.: An initial wise development environment for behavioral models. In: Proceedings of the 4th International Conference on Model-Driven Engineering and Software Development (MODELSWARD), pp. 600–612 (2016)Google Scholar
- 9.Hamou-Lhadj, A., Lethbridge, T.: Summarizing the content of large traces to facilitate the understanding of the behaviour of a software system. In: 14th IEEE International Conference on Program Comprehension (ICPC), pp. 181–190. IEEE (2006)Google Scholar
- 10.Braun, E., Amyot, D., Lethbridge, T.C.: Generating software documentation in use case maps from filtered execution traces. In: Fischer, J., Scheidgen, M., Schieferdecker, I., Reed, R. (eds.) SDL 2015. LNCS, vol. 9369, pp. 177–192. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24912-4_13CrossRefGoogle Scholar
- 11.Noda, K., Kobayashi, T., Toda, T., Atsumi, N.: Identifying core objects for trace summarization using reference relations and access analysis. In: 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC). IEEE (2017)Google Scholar
- 12.Bertero, C., Roy, M., Sauvanaud, C., Trédan, G.: Experience report: log mining using natural language processing and application to anomaly detection. In: 28th International Symposium on Software Reliability Engineering (ISSRE) (2017)Google Scholar
- 15.Harel, D., Katz, G., Marron, A., Weiss, G.: Non-intrusive repair of safety and liveness violations in reactive programs. Trans. Comput. Collect. Intell. (TCCI) 16, 1–33 (2014)Google Scholar
- 16.Marron, A.: A reactive specification formalism for enhancing system development, analysis and adaptivity. In: 15th ACM-IEEE International Conference on Formal Methods and Models for System Design (MEMCODE) (2017)Google Scholar