1 Software tools in systems engineering

Model-driven engineering (MDE) plays an important role in the design of critical embedded and cyber-physical systems in various application domains such as automotive, avionics or telecommunication. MDE tools aim to simultaneously improve quality and decrease costs by early validation by highlighting conceptual design flaws well before traditional testing phases in accordance with the correct-by-construction principle. Furthermore, they improve productivity of engineers by automatically synthesizing different design artifacts (source code, configuration tables, test cases, fault trees, etc.) necessitated by certification standards (like DO-178C [117], DO-330 [116] or ISO 26262[78]).

Certain shares in the software tool market of systems engineering are dominated by very few industrial tools (e.g., MATLAB Simulink, Dymola, DOORS, MagicDraw) each of which typically provides advanced support for certain development stages (requirements engineering, simulation, allocation, test generation, etc). To protect their intellectual property rights, these tools are of closed nature, which implies huge tool integration costs for system integrators (such as airframers or car manufacturers). On the other hand, recent initiatives (such as PolarSys, OpenModelica) have started to promote open language standards and the systematic use of open-source software components in tools for critical systems to reduce licensing costs and risks of vendor lock-in.

Certification standards of critical cyber-physical systems require that software tools used for developing such critical system are validated with the same scrutiny as the system under design by software tool qualification [87, 116], especially, when no further human checking is carried out on the outputs of such tools. Software tool qualification distinguishes between design tools which, by definition, may introduce new errors to the system and verification tools which may fail to reveal existing errors of the system [87] or falsely reduce or simplify the verification process itself by automation [116].

Unsurprisingly, software tool qualification is extremely costly due to high algorithmic complexity [97], tightly coupled architecture and unexpected feature interaction of such tools [86]. In fact, many companies rather opt for using tools just as design aids to highlight errors quickly, and then, they carry out the traditional verification and validation process by thorough simulation and testing [32, 156]. Anyhow, systematic software engineering techniques to simultaneously improve quality and reduce the costs of software tool qualification would be highly beneficial. Existing software engineering practices may guarantee the quality of the system itself, but they frequently fail to ensure the quality of the software tool used in systems engineering [141]. Furthermore, the rapid increase in the size and complexity of systems models introduces significant scalability challenges for these tools [84].

Software language engineering aims to provide foundations, techniques and tools for domain-specific modeling languages to capture the models. Model transformation engineering aims to systematically develop queries and transformations used in automated code generators, simulators or debuggers to process these models. Of course, seamless integration of these techniques is needed when developing industrial tools.

The VIATRA open-source software project provides advanced support for incremental and reactive model transformations [20, 141] built on top of incremental graph queries to assist the systematic development of novel tools for systems engineering. This invited paper extends previous papers [20, 39, 137, 141, 142, 148] to provide an overview on the history and evolution of the VIATRA model transformation framework. It first started as part of the MSc research project of the first author in 1999 [146, 147], then evolved into a Prolog-based model transformation engine [39, 148] (Sect. 3) followed by an open-source Eclipse project founded in 2005 [142] and used in various European projects (Sect. 4). Since 2010, the VIATRA family includes IncQuery [22, 137] which supports the scalable incremental evaluation of graph queries over large models of heterogeneous technological spaces (Sect. 5.2). The current industrial release of VIATRA [20] (Sects. 5.3, 5.4) is a reactive and incremental transformation engine built on top of graph queries following several principles of reactive programming [13] and active databases [106]. We overview the main features of each major release along an open case study influenced by an industrial project (Sect. 2) which uses reactive transformations for model-based deployment with run-time models. We present selected academic and industrial applications of the VIATRA family and summarize related work in a historical context.

2 Motivating example

As a motivating example, we investigate model deployment transformations for dynamic and self-adaptive systems frequently considered in the context of smart cyber-physical systems (CPS) [92]. The source domain describes a high-level generic infrastructure where applications (services) are dynamically allocated to connected hosts. The target domain represents low-level system deployment configuration with stateful applications deployed on hosts. Traceability links between the source and target models are also persisted as models to comply with traceability requirements of CPSs.

The full case study presents a complex challenge including (1) continuous validation of well-formedness constraints, (2) a model synchronization scenario, i.e., a model-to-model (M2M) transformation from the CPS model to a deployment model and (3) a code generation scenario from the deployment model to Java code.

Fig. 1
figure 1

Metamodels extracts of source, target and traceability domains

Full size image

In a real design tool, some of these steps can be addressed by batch (on-demand) transformations which are initiated explicitly by the engineer, while others are defined as live (reactive) transformations which are triggered automatically by certain changes in the underlying model. Some transformation steps can be target incremental which only update or move target model elements instead of regenerating them from scratch. Furthermore, source incremental transformation steps traverse or query exactly those source elements which are relevant for change detection and propagation [68, 123].

Due to data and control dependencies, the different transformation phases heavily depend upon each other. For instance, continuous validation of constraints has to be suspended, while a transformation is running; otherwise, constraint violations may be unintentionally detected in an incomplete state. In a traditional MDE toolchain, separate tool features (e.g., plugins) would be used to describe the various phases, requiring an external orchestrator to facilitate the coordination. Complex features in real MDE tools (like model indexing or file operations) add further complexity to the integration of tool features. Needless to say that such orchestrators are extremely hard to develop and debug.

Metamodels We present simplified fragments of the metamodels in Fig. 1 to provide better focus for our paper. The source (CPS) domain (Fig. 1a) contains classes (nodes) HostInstances and AppInstances, respectively, typed by HostTypes and AppTypes as denoted by the corresponding instances references (edges). AppInstances are allocated to a HostInstance captured by allocatedTo references. In the target (Deployment) domain (Fig. 1b), DeploymentHosts and DeploymentApplications are derived from their counterparts in the CPS model, but hosted applications are deployed directly under their hosts (see reference apps). The containment hierarchy is defined along references marked by black diamonds (at the container end).

Finally, the mappings between the two domains are persisted in a simple traceability model using CPS2DEPTrace elements and src and trg references.

Instance models A sample source model is depicted in the top part of Fig. 2 with one AppType and HostType, three AppInstances (app1, app2 and app3) and two HostInstances (host1 and host2). Application instances app1 and app2 are running, while app3 is stopped. Moreover, app1 is allocated to host1, while app3 is allocated to host2. Note that the containment hierarchy of elements is not explicitly depicted in the instance models.

Fig. 2
figure 2

Sample source and target models

Full size image

Its corresponding target deployment model is illustrated in the bottom part of Fig. 2 with two DeploymentHosts (dh1 and dh2) each containing a DeploymentApp (da1 and da3). Traceability is represented in a separated model with four elements of type CPS2DEPTrace linking the corresponding elements of the source and target domains and a top-level trace CPS2DEP linking the models themselves.

A transformation problem Below, we only overview the M2M scenario, which aims to illustrate the specification of transformations with different levels of incrementality, such as batch transformations, reactive/live transformations and traceability-driven transformations. Initially, we derive a deployment model from the CPS model, and then, incremental model transformations propagate changes observed in the CPS model to the deployment model as well as to the traceability model.

The informal transformation rules are the following:

  1. 1.

    The root element of a CPS model root is mapped to the root of the Deployment model (after clearing all existing traceability elements).

  2. 2.

    Each HostInstance in the source model is mapped to a DeploymentHost in the target model and connected to the root of the Deployment model.

  3. 3.

    Each allocated AppInstance in the source model is mapped to a DeploymentApp in the target model and deployed to its DeploymentHost.

This transformation problem will be used in the sequel to exemplify and compare the different versions of the Viatra framework. However, the reader is encouraged to check the complete source code, documentation and performance evaluation results available from https://github.com/IncQueryLabs/incquery-examples-cps.

3 VIATRA1: a Prolog-based transformation framework

3.1 Motivation

The motivation for a well-founded model transformation framework came from the HIDE project [28] which aimed to carry out model-based evaluation of functional and dependability attributes of the system under design by formal methods. Assuming that system models are captured in high-level languages (e.g., in UML as in HIDE [28, 39] or using languages such as SysML, AADL, BPMN), the key idea was to carry out early systematic formal analysis of design models by generating appropriate mathematical models by automated model transformations (MT). Precise formal analysis retrieves a list of problems, which can be back-annotated to the high-level engineering models to allow system designer to make corrections prior to investing in manual coding for implementation. This way, formal methods are hidden by automated model transformations which project system models into various mathematical domains [28, 134]. Furthermore, the source code of the target system can be derived by automatic code generation.

3.2 Key innovations and features

The initial concepts of [146, 147] continued as the Ph.D. project of the first author to develop the first version of the Viatra model transformation framework published in two key papers [39, 148]. The key innovative features of Viatra1 included the following:

  1. (a)

    XMI-based model export/import Viatra1 imported and exported models serialized in accordance with the XMI 1.0 standard for arbitrary MOF-based metamodels—5 years before EMF achieving the same for Ecore metamodels [45].

  2. (b)

    Model transformations by graph transformation Transformations were formally captured by graph transformation (GT) rules [47, 115], which offer a rule and pattern-based approach for manipulating graph-based models. In GT, the preconditions of applying a rule are captured by the left-hand side (LHS) graph, while the right-hand side (RHS) graph declaratively captures how to rewrite the graph as a result of the rule application.

    While GT had already been a well-established formal conceptual framework [115] with several applications and tool support [47], using GT for capturing language semantics or model-to-model transformations was a new idea in 2000 reported in [48, 50, 147, 148]. Triple graph grammars were proposed in [119] as a concept for bidirectional transformations, although most of tool support was still ongoing work at that time.

  3. (c)

    UML as visual syntax The concrete syntax of GT rules was defined by UML Class diagrams. Transformation programs were assembled from rules using UML Activity Diagrams and core control primitives (e.g., as-long-as-possible mode). Rational Rose was used as a UML editor for modeling transformation rules and exporting them also in an XMI format.

    Embedding graph transformation rules into the UML language was investigated by FUJABA [50] or GReAT [14] with significantly higher maturity compared to Viatra1 . UML was used as a transformation language later, e.g., in [5, 82].

  4. (d)

    Auto-generated transformation code Viatra1 took a compiled approach by automatically generating Prolog code from GT rules captured as UML models. To be more precise, the target language of the compilation was a internal domain-specific language (DSL) over Prolog. The compilation process consisted of a sequence of model transformation steps [139], and it used only metamodel-level information to derive a local-search-based traversal for edges of graph patterns. Some manual modification of the auto-generated Prolog code was needed in case of complex, recursive transformation rules.

    By that time, a similar compiler-based approach was already taken by several graph transformation tools including PROGRES [120], FUJABA [50, 102] or OPTIMIX [10].

  5. (e)

    Prolog as transformation engine Model transformations were executed using SWI-Prolog as the underlying engine. Debugging of transformations was carried out on the Prolog-engine level. Logging of transformations was limited to reporting the number of rule applications during a transformation run.

    Prolog has remained a popular platform for model transformations as demonstrated by [6, 118].

Fig. 3
figure 3

A graph transformation rule (in UML)

Full size image

Example A graph transformation rule for mapping- allocated applications to the deployment platform is depicted in Fig. 3. When a AppInstance is found which is allocated to a HostInstance but not yet mapped to a DeploymentApp (identified by the lack of traceability structures), then the rule creates a new DeploymentApp and linked with a CPS2DEPTrace traceability node and src and trg traceability edges.

In its UML notation, a Viatra1 rule is specified by three UML packages (for the LHS, the RHS and a negative condition NEG) containing classes and associations between them where (1) each model element (node or edge) is identified by a unique name (defining a corresponding variable) and (2) the types of model elements are defined by appropriate stereotypes. If several classes (or associations) appear in different packages with identical names, it refers to the same model element.

For example, <<DeploymentHost>>DH appears both in the LHS and the RHS packages; thus, it defines a node which needs to be matched in the LHS and preserved when applying the rule. However, edge <<apps>> E2 only appears in the RHS package; thus, a new edge is created by the rule with a unique identifier resolved to E2.

Viatra1 automatically generated the following Prolog code from rule appInst2deployedAppR (listed in Fig. 4). The code generator used a simple depth first search for ordering the Prolog clauses of the fully declarative LHS always starting the search at a designated node of the LHS.

Fig. 4
figure 4

Generated Prolog code for rule of Fig. 3

Full size image

Prolog meta-programming and the cut (!) construct were intensively used for checking attributes and nodes already visited (e.g., as part of node1). Control structures of transformation programs were restricted to a few constructs (see Fig. 5).

Fig. 5
figure 5

Generic Viatra1 library in Prolog (extract)

Full size image

3.3 Application in projects

The first complex model transformations implemented in Viatra1 were focused around challenges of the HIDE project [28], and numerous follow-up national projects. Key transformations included the following:

  • SC2Promela mapping UML statecharts to Promela for functional behavior analysis by model checking [94];

  • SCComplete completeness analysis of UML statecharts using Viatra1 and Prolog [105]

  • UML2DFN mapping UML component diagrams to dataflow networks for fault modeling and fault propagation analysis [29];

  • SC2SPN mapping UML statecharts to stochastic Petri nets for reliability analysis [41].

3.4 Software engineering aspects

The Viatra1 transformation framework was developed by the first author as part of his Ph.D. project without having a real software engineering process behind the project. Taking the semiformal high-level specification of transformations, he was the developer for most of the model transformations as well since (1) developing transformation rules by UML profiles within Rational Rose had severe usability issues, and (2) Prolog as an internal transformation language and execution engine turned out to be an obstacle for many colleagues.

4 VIATRA2: a model transformation framework in Eclipse

4.1 Motivation

The development of Viatra2 was started in early 2004 to serve as a general-purpose model transformation engineering framework that aims at supporting the entire lifecycle, i.e., the specification, design, execution, validation and maintenance of transformations within and between various modeling languages and domains [142, 145]. A specific goal was to provide foundations for precise model transformations aiming to bridge engineering and formal mathematical domains. The birth of Viatra2 was also triggered by the open call for proposals for the QVT standard [104], although it took a significantly different approach.

4.2 Key innovations and features

  1. (a)

    Model management Viatra2 was built on the VPM metamodeling approach [143, 144] which facilitated a unified treatment of classes and objects similarly to clabjects to support multilevel metamodeling [11, 12] along arbitrary (fluid) metalevels. Viatra2 introduced the concepts of a model space which provided uniform storage and handling of metamodels, models and transformation models. Unlike in contemporary EMF-based tools which stored metamodels in a separate registry and generated Java code from metamodel-level information, VPM provided a single namespace (registry) for models, metamodels and transformations with fully qualified names and offered a generic model manipulation library without additional code generation.

  2. (b)

    Transformation language The Viatra2 framework offered a rule- and pattern-based transformation language for manipulating graph models by combining graph transformation and abstract state machines (ASMs) into a single specification paradigm [142]. This textual language provided advanced constructs for querying (e.g., recursive graph patterns [152]), unidirectional graph transformation rules for elementary model manipulations, and complex transformation programs captured by ASMs. As further key innovation, generic and meta-transformations were introduced in [142, 145] which are known nowadays as higher-order transformations [8, 133]. Popular graph transformation tools developed in parallel with Viatra2 included AGG [49], ATOM3 [93], FUJABA [102], GReAT [14], GrGen.Net [55], MOFLON [7], VMTS [99]. Popular model transformation languages within the Eclipse framework were ATL [80], Epsilon [83], GEMS [127], or Tefkat [95]. Several of these projects later evolved to components of the top-level Eclipse Modeling project.

  3. (c)

    Graph query and transformation engine Graph queries could initially be evaluated on instance models using local-search-based pattern matching [33]. First search plans were statically generated using metamodel-level information (and magic sets for recursive patterns [152]), and later, we experimented with model-level adaptive search plans [153]. As a key innovation, we developed incremental graph pattern matching by using a custom [154] and Rete-based [52] caching mechanism in [109]. Moreover, a hybrid approach for combining incremental and local search techniques was proposed in [21]. Viatra2 also had a live model transformation engine [109] where rules are permanently loaded and they immediately react to changes in the underlying models in order to support change-driven transformations [24, 112]. Several efficient search plan-based techniques have been proposed since then for GrGEN [55], EMF models [150], adaptive search plans [55] as well as for the current Viatra [34]. Moreover, incremental model transformation techniques were proposed, e.g., for Tefkat [62], ATL [81] and in triple graph grammar tools [56, 69].

  4. (d)

    Transformation plugins While most transformations were executed by the Viatra2 transformation engine, extensive research has been dedicated to generate standalone transformation plugins from high-level specifications using the Viatra 2 transformation language. These plugins could be embedded and executed in industrial platforms without the need for the Viatra 2 engine. Target platforms included relational databases [23, 151] or Enterprise Java Beans [16, 149].

  5. (e)

    Add-ons The Viatra2 transformation started to serve as a core for high-level features and add-on. Most notable examples include the VIATRA-DSM framework [111] which aimed to support the development of custom domain-specific languages and simulators. However, unlike the Graphical Modeling Framework [128], it avoided the use of code generators by providing a customizable generic core framework with real-time reflection to changes in language specifications. Interestingly, similar concepts are used in the Sirius framework [131] nowadays. Viatra2 also served as core engine for solving constraint satisfaction [74, 75] and design space exploration [65, 66] problems with complex structural constraints directly over models. A key challenge here is to find consistent sequences of rule applications leading to a designated target state fulfilling goal constraints. Incremental pattern matching was beneficial to quickly identify constraint violations during traversal. Viatra2 also served as the conceptual basis for a stochastic simulator for GT systems [135]. State space exploration over graph models has also been investigated in model checkers for GT systems (e.g., Groove [113, 114], Augur [85]). Nowadays, rule-based design space exploration approaches following related ideas include [4, 44, 51, 54].

Example As a sample transformation rule of Viatra2 , we present in Fig. 6 how to derive DeploymentApp elements for each AppInstance allocated to a HostInstance (which was discussed for Viatra1 transformations in Fig. 3 and Fig. 4).

Fig. 6
figure 6

A graph transformation rule in Viatra2

Full size image

This GT rule appInst2deployedAppR also uses explicit traceability models to identify which elements are already mapped during transformation by introducing graph pattern mappedElement. The LHS of the rule is also a graph pattern which consists of local structural constraints (e.g., AppInstance(App) and pattern composition using the predefined mappedElement pattern in both positive and negative way: We check that Host is already mapped to a DeploymentHost, while App is not yet mapped as requested by a negative application condition. The complete graph pattern is composed and flattened at compile time by the Viatra2 engine (also arranging the actual predicates in an efficient order using search plans). This time the action of the transformation rule is defined in an imperative way by using model manipulation operations with abstract state machine constructs.

An extract from a sample transformation program assembled by using abstract state machine constructs is listed in Fig. 7. The transformation first handles the root CPS element of the source model (identified by variable CPSModel) by applying rule cps2deploymentR. Then, it applies rule hostInst2deployedHostR for all HostInstances within CPSModel. Finally, rule appInst2deployedAppR (detailed in Fig. 6) is initiated to handle (allocated) AppInstances.

4.3 Selected applications

The Viatra2 framework served as a key underlying model transformation technology of several European projects for dependable embedded systems and service-oriented applications including DECOS, DIANA, MOGENTES and SENSORIA and the INDEXYS project within the industry-driven ARTEMIS platform. Academic and industrial partners in these projects became the first end users of the Viatra2 framework. Regular usage of the framework has been reported at ARCS and TU Vienna (Austria), University of Leicester (UK), LMU Munich (Germany), TU Kaiserslautern (Germany), University of Pisa (Italy), University of Waterloo (Canada), Georgia University of Technology and NASA (USA).

Fig. 7
figure 7

A transformation program in VIATRA2

Full size image

4.3.1 Transformations for service-oriented computing

The SENSORIA European project developed a comprehensive, model-driven approach for service engineering including (1) novel languages for service modeling, (2) qualitative and quantitative techniques for service analysis, (3) automated mechanisms for model-driven service deployment and (4) transformations for legacy service re-engineering. Model transformation served as a key technology for model-driven service engineering by bridging different languages and tools in the context of service-oriented applications. Various model transformations were developed in the scope of the project:

  • Automated formal analysis of BPEL processes The consistency of business processes captured using the standard BPEL notation [103] was formally analyzed by the SAL model checker [19], which exhaustively investigates all potential execution paths of a dynamic behavioral model to decide if a designated property (requirement) holds or not. SAL models were automatically derived by a complex model transformation [88, 89].

  • Back-annotation of model checking results to BPEL processes As a reverse problem, back-annotation of the results retrieved by the SAL model checker to the BPEL model of service engineers was carried out [63] by a mapping between traces captured by change-driven transformations [112].

  • Model-driven performability analysis. Performability is a nonfunctional system-level parameter, which aims to assess the cost of using fault-tolerant techniques in terms of performance. We developed a model-driven performability analysis approach [59] by mapping UML-based service models to formal process model for the PEPA framework [58]. The system-level performability model was assembled from a library of core performability components driven by system-level UML component diagrams.

  • Model-driven service deployment The derivation of configuration descriptors required for service deployment was automated in [57, 60, 90] by a chain of generic model transformations [145]. This approach was successfully adapted to a number of standard service platforms including the application interface specification of the service availability forum, the web service description language (WSDL), WS-reliable messaging, the IBM RAMP platform, and Rampart and Sandesha configuration files for the Apache Axis2 framework.

4.3.2 Transformations for critical embedded systems

The Viatra2 model transformation framework has been intensively used for providing tool support for developing and verifying critical embedded systems in numerous European research projects such as DECOS, DIANA, MOGENTES, INDEXYS and SecureChange.

  • Model-driven tool integration Model transformations served a key role in tool integration scenarios to bridge a variety of off-the-shelf industrial tools where tool integration scenarios were driven by the underlying development process [15, 67].

  • Model-driven development tools A user-guided interactive model transformation chain [15] served as the foundation for model-driven tools in the automotive and avionics domain aiming to support the development of configuration tables for hardware-software allocation [67, 76].

4.4 Software engineering aspects

To facilitate the increased involvement of graduate students, Viatra2 was implemented in Java as a plugin of the new open source Eclipse framework. In September 2005, Viatra2 became an open-source component of Generative Modeling Tools (GMT) project, which was a joint incubator of several model transformation technologies including ATL [80] or Epsilon [83].

The Viatra2 project has been initially led by the first author, while the first contribution of Viatra2 was developed by one Ph.D. student and 3 MSc students. Since then, Viatra2 has continuously been developed until mid of 2013 with 6 major releases altogether involving a total of 8 Ph.D. students, including all co-authors of the paper, András Balogh, Gergely Varró and Dániel Tóth and well over 20 MSc student throughout the years.

5 VIATRA3: a reactive transformation platform built on incremental queries

5.1 Motivation

By 2010, it was clear to us that the major strength of Viatra2 is its query language and incremental evaluation engine, while its most severe practical limitation is the lack of seamless support for models captured using the industrial Eclipse Modeling Framework (EMF) [45]. An industrial tool development project carried out in collaboration with OptXware Ltd. further revealed that writing queries and validation rules for EMF models was especially cumbersome due to, e.g., problematic testability of embedded loops, complex navigation expressions and export–import functionality.

This led us to focus most of our research and development efforts to incremental model queries over EMF models [22] by giving birth to the EMF-IncQuery framework in 2010— and decelerating the development of Viatra until late 2012. Later Viatra has become a reactive and live model transformation platform exploiting the incremental query evaluation provided by IncQuery. In 2015, EMF-IncQuery successfully made transition out of the incubation phase at the Eclipse Foundation. From April 2016, the two projects were unified (where EMF-IncQuery became “Viatra Query”), and the full Viatra release leaves the incubation phase.

5.2 Incremental model queries: key features

IncQuery started as an open-source Eclipse project to define declarative graph queries over EMF models [45] without manual coding and execute them efficiently using incremental graph pattern matching techniques over an imperative programming language such as Java. The main features of IncQuery include:

  1. 1.

    High-level declarative graph query language The query language of IncQuery [25, 137] and thus VIATRA3 conceptually builds upon the query language of VIATRA2, but its type system is tightly integrated with EMF models and offers many powerful language shortcuts. A main conceptual extension is the introduction of transitive closure over edges defined as arbitrary binary relations [23].

  2. 2.

    Incremental query engine IncQuery offers a highly efficient engine to evaluate queries over models with millions of elements [22, 137, 138] by adapting Rete networks [52] to change notifications sent by EMF models. To decrease memory consumption, a query can also be evaluated using local-search-based techniques [34] which exploit model-specific search plans [150].

  3. 3.

    Integrated development environment. The advanced IDE of IncQuery [137] enables to construct and validate model queries supported by state-of-the-art Xtext tooling with syntax highlighting, auto-completion, type checking, traceability between models and queries, debugger or incremental compilation.

  4. 4.

    Integration with EMF tools The modular architecture of IncQuery enables easy integration with existing EMF- based modeling tools and applications [137] such as Papyrus UML [130], Capella [108], Sirius [131] or Artop [1]. The primary use case for model queries is to support the live validation of well-formedness constraints of a domain in order to highlight and report inconsistencies as soon as they are introduced by engineers. Additional main use cases include advanced support for incremental maintenance of base model indexers [137], derived features [110], soft traceability links [64] or incremental view maintenance [43]. In all these cases, language-level annotations eliminate manual coding for integration. Note that while we illustrate IncQuery and Viatra in the context of EMF models, core queries and transformations are regular Java programs which have been successfully adapted to other technological spaces (outside Eclipse).

Detailed scalability assessment of IncQuery is carried out in numerous papers for validation of well-formedness constraints [22, 137], detection of source code anti-patterns [138] or maintenance of soft traceability links [64] over models with 10 million elements.

Related work Other query technologies in the context of EMF include EMF Model Query 2 [129], which provides simple query primitives for selecting model elements that satisfy a set of conditions. The OCL development environment of the Eclipse OCL project [46] provides different ways to edit OCL constraints: an Xtext-based editor for file-based editing, an embedded editor inside Ecore model editors. The Epsilon Validation Language is dedicated to support the construction of validation rules within the Epsilon family [83], while the Acceleo Query Language (AQL) is heavily used within the Sirius project[131] to populate views from underlying models. In addition, some academic approaches support incremental constraint evaluation over models [27, 35, 61, 136].

Example The definition of a sample well-formedness constraint [141] for checking valid allocations of application instances to host instances is listed in Fig. 8. The query notAllocatedButRunning captures an erroneous situation for allocation when an application app is running, but not allocated to a host instance (using another graph pattern allocatedApplication by negative composition). When checking this constraint on the source instance model depicted in Fig. 9, app2 is the only ApplInstance which matches the pattern (thus violates the constraint) since app1 is allocated to a host instance ht1, while app3 is stopped.

Fig. 8
figure 8

Sample queries for well-formedness constraints (adapted from [77])

Full size image
Fig. 9
figure 9

Erroneous situation

Full size image

By using a @Constraint annotation, a query will be automatically integrated into an EMF-based model editor. As a result, an error marker will immediately be placed on the model whenever this consistency constraint is violated, which is removed automatically once the source of the problem is corrected (e.g., the application instance is stopped or allocated to a host instance).

5.3 A reactive transformation platform: key features

Viatra is a reactive, event-driven model transformation platform [20] built on top of incremental graph queries where transformations are executed continuously as reactions to changes in the underlying model. The main features of the Viatra projectFootnote 1 are as follows:

  1. 1.

    Reactive transformation framework Viatra adopts the principles of reactive programming [13] and active database systems [106]. The core concept of reactive programming is event-driven behavior: Components are connected to event sources, and their behavior is determined by the event instances observed on event streams. Compared to sequential programming, the benefits of reactive programming are remarkable when continuous interaction with the environment has to be maintained by the application based on external events without a priori knowledge on their sequence. Viatra has proven to be an efficient execution platform for incremental transformations in [77, 107].

  2. 2.

    Internal DSL as transformation language Viatra uses an internal DSL for specifying both batch and event-driven, reactive transformations, which is an advanced API over Java and Xtend [132]. The specification of a Viatra transformation program contains (1) rule specifications consisting of model queries, which serve as a precondition to the transformation, and actions, which typically prescribe model manipulations. Then, (2) execution schemas are defined to orchestrate the reactive behavior.

  3. 3.

    Complex event processing Viatra supports complex event processing (CEP) [42] over EMF models to detect complex sequences of (hierarchical) events and specify reactions upon them. Event hierarchies are constituted from external events (that appear on the event stream), notifications of elementary model changes and aggregated model changes identified by changes in the result set of queries.

  4. 4.

    Rule-based design space exploration The current Viatra framework natively supports rule-based design space exploration over EMF models [66] to explore design candidates as graph model which satisfy multiple criteria over states and trajectories and evolve along certain operations. As a key innovation, it extends previous rule-based DSE concepts with multi-objective optimization [4].

  5. 5.

    Model obfuscator and secure model access In order to remove sensitive information from a confidential model provided by an industrial partner (e.g., to create a bug report), Viatra supports model obfuscation which changes sensitive data stored in models to randomly generated names. Ongoing work carried out partly in the context of the MONDO European projectFootnote 2 aims to support collaborative model-driven engineering by providing (1) secure model access to model fragments compliant with high-level policies, (2) secure bidirectional model transformations using lenses and (3) property-based locks [37].

Related work Various EMF-based model transformation tools provide support for specifying, executing and evaluation of transformations including frameworks such as ATL [80], Henshin [9] or eMoflon [2]. Many industrial applications rely on Xtend[132] as a code generation and transformation language based on Java. Epsilon [83] provides the Epsilon Transformation Language and the low-level Epsilon Object Language with an advanced execution platform.

However, relatively few tools support event-driven or reactive transformations: New features of ATL include target incremental computation [81] combined into the ReactiveATL transformation engine. Generic low-level transformation engines include EMFTVM [155] or T-Core [124]. SIGMA [91] provides Scala-embedded DSLs that map Epsilon model transformation functionality directly into Scala as a specification language, and the JVM as the execution platform. A detailed overview of the state-of-the-art for rule-based design space exploration frameworks is provided in [66]

Example A reactive and event-driven version of our sample transformation rule of Fig. 3 is illustrated in Fig. 10. The application of the rule has double effects: (1) When an AppInstance gets allocated to a HostInstance in the source model, then it creates the corresponding DeploymentApp in the target model, and (2) when a mapped AppInstance is no longer allocated to a HostInstance in the source model, then it removes the corresponding DeploymentApp from the target model.

The execution of (1) is triggered by the appearance of a new match of its precondition pattern applicationInstance, while the execution of (2) is initiated when an existing match of the same pattern disappears.

Fig. 10
figure 10

A reactive transformation rule

Full size image

Let us now assume that the source CPS model changes by removing the allocatedTo edge between app3 and h2 (see Fig. 11a). Due to this change, an existing match of pattern applicationInstance disappears, which correspondingly triggers the application of the reactive rule appInstanceRule. As a result, the corresponding DeploymentApp is removed from the target model together with the traceability structures to yield the transformation result depicted in Fig. 11b.

Fig. 11
figure 11

Effect of reactive transformations upon change

Full size image

5.4 Reactive execution architecture of Viatra

A reactive transformation program in Viatra consists of two parts. First, the rule specifications are defined by a precondition and some actions. A precondition is most frequently captured as a query over a given model(s) or alternatively by detections of complex events, while actions include model manipulations and control structures. As a key innovation, the activation of rules is clearly separated from the execution of their actions by defining execution schemas are defined in order to orchestrate the reactive behavior, e.g., to resolve conflicts and schedule rule activations. Now, we briefly describe the behavior of core components of the event-driven virtual machine (EVM) of Viatra in Fig. 12.

Fig. 12
figure 12

The Viatra architecture for reactive model transformations

Full size image

5.4.1 Events and activation lifecycles

Events Activations (i.e., matches) and executions of reactive transformation rules are triggered by events, which are either (1) controlled events which are initiated explicitly by a transformation program, a transaction or the user, (2) observed events which are caused by external behavior, and the exact time of their occurrence may not be determined by the transformation program. Such observed events include notifications upon elementary model changes, event sequences or updated results of model queries. In both category of events, detection of events is followed by the firing of a corresponding rule along specific parameter bindings.

Activation lifecycles Reactive transformation rules react to events in accordance with their activation lifecycles, which is a state transition system reflecting the current state of a rule activation. An activation lifecycle consists of different (1) phases and (2) event-triggered transitions between such phases. Optionally, (3) a transition may be associated with a job, which represents the executable actions of a input rule specification. Figure 13 presents two typical activation lifecycles for event-driven and batch transformations.

Fig. 13
figure 13

Typical rule lifecycles [20]. a Event-driven transformation. b Batch execution

Full size image

To unify the behavior of model transformations over the reactive Viatra platform, activations of both event-driven and batch transformations are executed as reactive programs. For instance, the enabled phase of an activation lifecycle represents reactions to observed events, while the firing of the actual reactive jobs is tied to controlled events. A library of most common lifecycles is readily available in Viatra which can be customized in Java by transformation developers.

5.4.2 Agenda, scheduler, conflict resolver

Scheduler External observed events influence activation phases according to the lifecycle, and the job to be executed (if any) is determined by the active phase. However, it is the scheduler component that determines when the EVM can fire these controlled scheduling events to actually execute the jobs.

Practical examples of scheduling events include (1) the signal of the query engine indicating that the update of query results has concluded after an elementary model manipulation; (2) the successful commit of a model editing transaction; or (3) some combination of the former events with a timer. The choice of scheduling event has to take into account the following factors:

  • By default, most rules are executed as soon as possible; thus, their effects are observable by the user or available for further transformation steps.

  • However, certain rules may require that the models is consistent (or inconsistent in case of quick fix rules), while others may be inefficient to execute while a large-scale transaction is incomplete.

Scheduling events offer better control over consistently executing different features compared to traditional plugins and still provide a high level of flexibility.

Event-driven rules may explicitly invoke other rules, which is a direct rule dependency. However, indirect rule dependency may also exist when model manipulation in a job causes observed changes which, in turn, enable activations and trigger the scheduler.

Agenda The agenda stores the current phases (states) of all activations of each rule. Its role is dual: It helps maintain the phase of activations in reaction to events, and it supplies the set of rule activations being in an enabled phase, i.e., activations that can be fired. The core behavior of EVM is intrinsically tied to the agenda: In case of an observed or controlled event, the rule activation corresponding to the specific event will change phase according to the transition in the lifecycle model defined for the rule that starts at the current phase and it is labeled with the event type. Afterward, if there is a job associated with the transition, it is invoked with the activation providing the input parameters.

As the set of all possible activations is practically infinite (as each rule parameter may point to any memory address), the implementation considers only those activations that are currently not in their initial phase. This makes the agenda finitely representable, since a finite number of events may have moved only a finite number of activations out of their initial phase.

Conflict resolution At any point in time, the rules may have multiple activations in an enabled state, which is called a conflict. If the transformation is to invoke a rule firing, a single enabled activation has to be selected from the conflicting ones (mainly due to the single-threaded manipulation of EMF models). This selection can be done manually by the transformation code, but EVM also provides an automated mechanism to delegate this decision to a custom (user-specified) conflict resolver. Built-in strategies include FIFO, LIFO, fair random choice, rule priority (with a secondary conflict resolver within priority levels), and interactive choice (e.g., selection on the user interface), but it is possible to implement arbitrary conflict resolution strategies.

5.4.3 Execution primitives

Model manipulation primitives The createChild method in Fig. 10 is one of the model manipulation primitives provided by the Viatra framework to enable more concise transformation definitions by hiding EMF-related details, e.g., transaction handling. The common operations supported by the framework are the following.Footnote 3

  1. (A)

    (createModel, Class) Creates an object of the corresponding Class, and places it as the root of the selected Model (called Resource in EMF).

  2. (B)

    createChild(Object, Reference, Class) Creates an object of type Class to be contained by its parent Object along the specified Reference.

  3. (C)

    addTo(SObject, StructuralFeature, TObject) Adds a noncontainment reference or attribute of type StructuralFeature from the source SObject to the target TObject (list semantics).

  4. (D)

    set(SObject, StructuralFeature, TObject) Sets the value of a single-valued StructuralFeature (reference or attribute) of SObject to the TObject.

  5. (E)

    remove(Object) Removes the Object from the model including an implicit removal of all dangling references pointing to the Object.

  6. (F)

    move(FObject, TObject, StructuralFeature) Moves element FObject to a new container TObject along containment StructuralFeature and removes it from the old one.

Control structures for batch execution In many practical cases of batch transformations, it is worth controlling the behavior of rule execution after firing a rule on a specific activation by explicit control structures and semantic modifiers such as the following:

  1. (a)

    fireOne Selects an enabled activation of the given rule nondeterministically and fires the rule on it.

  2. (b)

    fireAllCurrent Fires all of the currently enabled activations of the given rules (in the order specified by the conflict resolver). Newly enabled activations are ignored. If one of the selected activations gets disabled during the execution, it will not be fired.

  3. (c)

    fireAsLongAsPossible Repeatedly performs fireOne as long as there are enabled activations to fire.

  4. (d)

    fireUntil(exitCondition) Repeatedly performs fireOne as long as there are any enabled activations to fire, and the exit condition evaluates to false.

The role of these control structures is similar to the old Viatra versions (see Figs. 5 and 7).

5.5 Selected applications

The IncQuery and Viatra frameworks have actively been used in different industrial and academic projects carried out by various researchers and practitioners. Below we provide a short overview of selected applications of these frameworks within our own projects.

  1. (a)

    Dedicated systems engineering toolchains A recent project industrial aimed to define a model-driven approach and tool chain for the synthesis of complex, integrated MATLAB Simulink models capable of simulating the software and hardware architecture of an aircraft [64, 70]. We actively developed industrial toolchains for other application domains (e.g., automotive, telecommunication) on contractual basis, which heavily built on incremental model queries and transformations.

  2. (b)

    Integration of MATLAB Simulink and EMF models The Massif (MATLAB Simulink Integration Framework for Eclipse)Footnote 4 framework [70, 73] provides a bidirectional bridge between MATLAB Simulink models and their EMF model counterpart via the MATLAB API. Massif was initiated within an industrial project with Embraer and then continued as part of the CONCERTO European ARTEMIS-JU project.

  3. (c)

    Formal validation of DSLs Designing advanced tooling for a new DSL is an error-prone task as it is surprisingly easy to introduce contradicting or incomplete well-formedness constraints for a language. DSL-level validation of language specifications is carried out in [121, 122] by using back-end logic solvers where derived features and well-formedness constraints are captured by queries. This technique derives small instance models as proofs of consistency by composing required model fragments.

  4. (d)

    Incremental code generators Incremental code generators [72, 140] aim to avoid complete regeneration in case of small changes by exploiting incremental queries and transformations. Viatra allows incorporating incremental transformations on different levels of granularity (resource, model fragment, model element) and information stored in traceability models can also be customized. Source incremental computation provided by Viatra turns out to be an efficient technique also for code generation.

  5. (e)

    Custom views by queries Incremental recomputation of graphical views [43, 71] can be also be driven by reactive transformations. Each node and edge in a graphical view is defined as a query (or a transformation in a more complex case), and the view is automatically (and incrementally) recalculated upon each change in the source model. Viatra is flexible enough to aggregate multiple model elements in the underlying model and represent them as a single node in the view.

  6. (f)

    Live movement detection Live detection of human gestures and movements is carried out in [42] by using streaming transformations and complex event processing (CEP) [101]. A live model forms the basis of calculations which is updated rapidly (25 times each second), and respective model changes are turned into events. Finally, relevant situations are detected as event sequences by CEP techniques.

5.6 Software engineering aspects

The significant growth in the complexity of the projects necessitated to better distribute responsibilities. Since its foundation in 2010, István Ráth has been the project lead for the IncQuery project (now together with Ábel Hegedüs), Zoltán Ujhelyi has become the project lead of Viatra in 2012, while Massif is led by Ákos Horváth. Viatra is expected to join the yearly release cycles imposed by the Eclipse Foundation in 2017. Consequently, committers are now dominated by professional software engineers (working for industrial companies), and novel features of the next release are highly influenced by customers and tool vendors. Moreover, the development team is international as Viatra regularly receives contributors from several countries.

However, novel innovative or high-risk components (e.g., the local search engine, complex event processing, design space exploration) are still dominantly prototyped by researchers or students (working for academia). This synergy enables to accelerate the transfer of academic results into industry while also providing highly challenging usage scenarios for researchers.

6 Summary and future work

6.1 Summary

In the past 16 years, the three generations of the Viatra model transformation framework have continuously served as a general means to design complex mappings within and between DSLs. As a summary, Fig. 14 presents the main features, specificities and differences between the various major Viatra versions. More extensive survey of model transformation tools are available in [40, 69, 79].

Fig. 14
figure 14

Feature overview of the VIATRA family (partly adapted from [79])

Full size image

In our practice, particularly deep and complex transformations are those that need to bridge large conceptual gaps between different formalisms, such as mapping high-level engineering models to low-level mathematical models in order to carry out formal verification and validation. Despite the wide range of existing languages and formalisms, our experience shows that many underlying concepts of transformation design have a common conceptual basis.

Moreover, an increasing number of scenarios with industrial relevance necessitated to use transformations (1) for developing new domain-specific (or general-purpose) modeling tools within the open- source Eclipse framework or (2) for integrating existing tools into complex, end-to-end toolchains. Checker tools for early validations or code generators for automated synthesis of various engineering artifacts have also been major application scenarios in our practice. The continuous advances of different model transformation techniques and tools have turned the science of transformation design into an engineering tasks.

However, there is still much to do to decrease the extreme costs of tool qualification, especially in the context of critical embedded and cyber-physical systems. While some transformations are now precisely specified on the semantic level (using techniques such as semantic anchoring [38] or formalism transformation graphs [100]), it is still not possible to guarantee that the actual implementation for the integration of different tools or interaction of tool features is free of flaws. Scalability of engineering tools is also a major challenge [84]. Testing of customized DSL tools is still in its infancy due to the lack of automatically synthesized consistent and large model instances with given diversity and coverage criteria. Industry-driven tool integration initiatives (like OSLC [3]) cover a broad spectrum of processes, but fail to address many of the underlying technical challenges. Altogether, developing software tools in an open component architecture still lacks scientific foundations.

6.2 Future of modeling tools?

The lack of such foundations may become severe in the context of smart cyber-physical systems (CPS) [36, 96, 126], which are open, interconnected and highly distributed complex systems expected to consist of 50 billion smart objects by 2020 [31]. They will integrate simple sensors and actuators to the Internet-of-Things (IoT) to exploit low latency cloudlets as in edge/ fog computing [30]. High-level web services may exploit the immense computation power of cloud computing. But a CPS also interconnects critical infrastructures and systems (such as cars, medical devices, aircrafts) with tightly integrated real-time computing platforms and physical systems [36] where a system failure may result in major financial loss, severe damage or even casualties.

For smart CPS, the distinction between design-time and run-time models [26] is more and more blurred [18, 96] which gives birth to run-time modeling frameworks (like Kevoree [53]). Thus, incremental query and transformation techniques will likely be used as part of the underlying middleware, which triggers further open research challenges such as how to support the development of scalable future tools for smart and trusted CPS where run-time models are directly connected to the system and interact with it in close synergy. While still providing various consistency guarantees for tool qualification purposes, we foresee that future modeling frameworks will support the following tasks:

  1. 1.

    Interaction with the underlying system and its context;

  2. 2.

    Identification of critical situations and triggering reactions at run-time;

  3. 3.

    Deployment of run-time models as services over heterogeneous platforms;

  4. 4.

    Hierarchical abstractions over time and structure.

As ongoing research in the MONDO project, we have started the development of IncQuery-D to support the incremental evaluation of graph queries in a distributed environment by distributing the nodes of Rete networks [125]. Models can be captured by different graph representations and stored in existing graph storages (e.g., RDF triple stores, distributed storage frameworks like Hadoop or Spark). The IncQuery-D framework is an independent query layer on top of such storages to efficiently support incremental reevaluation over large and evolving graph data. This frameworks is intended to support run-time checks of cyber-physical systems and detect relevant situations in IoT applications.

Related ongoing work [17] aims to address the run-time verification [98] of rich and high-level specifications (such as graph queries or complex event processing languages) which is being extended toward over heterogeneous and distributed platforms which include smart IoT devices with limited resources, mobile phones as well as cloud-based computations. This technique was used as part of a complex demonstrator for the Eclipse IoT Challenge 2016.Footnote 5