Editorial to the theme section on model-based testing
- 215 Downloads
This theme on model-based testing (MBT) was organized in the context of advances in model-based testing (A-MOST) workshop series. Now in its fourteenth edition, this workshop covers all aspects of MBT from theoretical developments to industrial implementations. Following the twelfth edition in Chicago (2016), we invited the MBT community to submit their latest and finest research in the field via an open call. We selected seven articles that are presented in this theme section.
Model-based testing (MBT) continues to be an important research area, where new approaches, methods and tools make MBT techniques deployable and useful for industrial applications. Models and different abstractions can ease comprehension of complex systems and ease testing and automation. The execution of software using test cases or sequences derived from models (manually or automatically), is often referred to as MBT. MBT is an encouraging scientific and industrial trend capable of coping with the growing software system complexity. Modeling requires substantial investment, and MBT solutions leverage this investment. The testing models may have been adapted from system design models or might have been devised specifically to support testing. Naturally, significant benefits can be obtained when test generation is automated. Interestingly, practitioners also report that the modeling process is of value, as it often highlights issues with the system requirements.
The industrial-scale software demands generally require a model-based construction of software, which in turn results in systems with compositions of independent and reusable actors. In this engineering paradigm, complex system functionality arises out of the composition of many component services. For these systems, model-based testing may significantly improve component acceptance and move component integration testing toward a canonical validation and certification of complete systems. Automation of software development and software testing on the basis of executable models and simulation, promise significant reductions in fault-removal cost and development time. As a consequence, changes in the processes of the requirement analysis, development and testing are in need. These needs demand combined efforts from both research and industry in order to reach a broadly accepted solution.
MBT research and practice have been the focus of the Advances in Model-based testing (A-MOST) workshop series for fourteen years. Following the twelfth edition in Chicago (2016), we invited the MBT community to submit their latest and finest research in the field via an open call. This call was popular as it yielded 29 submissions of which seven were accepted at the end of the review process. This number is one of the indicators of the MBT dynamism.
2 Selected articles
In Model-based testing of software for automation systems using heuristics and coverage criterion, Leandro Dias Silva et al. focus on programmable logic controllers (PLC) testing of Petrobras industrial processes in Brazil. In particular, they provide a testing criterion called “At Least N” that guarantees that test covers at least N times the effects of the choice of input values for the logic controller. This criterion is implemented on a tool called Gungnir which generates timed automata models from PLC specifications, generates tests from the environment model, calculates the effects and analyzes the coverage.
In Toward testing from finite state machines with symbolic inputs and outputs, Alexandre Petrenko removes the limitation of having finite sets of inputs (and outputs) by working symbolically. After having introduced the symbolic input/output finite state machine (SIOFSM), the author defines several relations such as equivalence, isomorphism and distinguishability between SIOFSMs and discuss the impact of these relations with respect to fault coverage. This article lays the theoretical foundations for a new line of research on MBT with very large or potentially infinite inputs and outputs.
In A method for testing and validating executable statechart models, Tom Mens et al. lift test-driven development and behavior-driven development at the model level to improve the quality of statecharts produced during the modeling phase. This method is supported by an open-source statechart interpreter in Python that can check for runtime violation of contracts, support the simulation of scenarios and integrate with unit testing frameworks.
In Temporal property patterns for model-based testing from UML/OCL, Frédéric Dadeau et al. introduce the notion of temporal property patterns in OCL to support dedicated coverage-based test generation from UML models. They additionally use mutation to tackle corner cases. A tool-supported assistance is provided to validation engineers to navigate across the temporal patterns depending on their needs.
In Property-based testing of web services by deriving properties from business-rule models, Bernhard Aichernig et al. apply property-based testing, i.e., trying to generate random inputs that falsify a property, to business rules models. Their proposition automatically translates XML business rules models into extended finite state machines to generate random sequences of arbitrary sizes checking whether the postcondition of any path holds.
In Testing concurrent user behavior of synchronous web applications with Petri nets, Sunitha Thummala et al. use Petri nets to model the behavior of concurrent web applications. They then define several criteria for test generation such as the access of the same data within multiple sessions.
Finally, in Supporting inheritance hierarchy changes in model-based regression test selection, Mohammed Al-Refai et al. focus on model-based regression testing and in particular the impact of inheritance hierarchy changes in the selection of tests. The method can be applied during runtime adaptation and show either similar or better results than its code-based counterparts.
We are greatly indebted to Jeff Gray, Bernhard Rumpe and Martin Schindler for their support along the revisions. Also, we would like to thank the anonymous reviewers for their hard work and of course, congratulate the authors for the high quality of their submissions.