1 Introduction

Model-driven engineering (MDE) is recognized as an established approach for developing complex software systems [1] bringing advantages during software system’s development. However, its adoption is currently hindered by a range of factors [2], including poor tool support and its usability [3], social and organizational issues, and a mismatch between technical and research requirements [2].

As developers engage in the intricate task of modeling, whether for data, systems, or simulations, evidence from practice shows that their experiences diverge from conventional academic practices or guidelines. In [4], the authors first introduce the term User eXperience (UX) for MDE or Modeler Experience (MX). MX goes beyond the traditional definition of usability to include experience that includes individual feelings, such as emotions, affects, motivations, and values in the process of modeling, playing a crucial role in the success and adoption of MDE approaches. This paper takes the initial steps toward developing a theory of MX, as called for in the original work by Abrahão et al. [4].

Taking the above definition one step further, we argue that MX is not only about the modeling language, tools, or individual perspectives, but also about how modeling is embedded in the organization and the mindset of the individuals. It can, therefore, be a tool to address the mindset barriers reported by Kalantari and Lethbridge [5].

A fundamental insight is that modeler experience depends on the concrete context and circumstances in which modeling is used. This is in line with the principles of Context-Driven Software Engineering, where Briand et al. [6] advocate for the importance of considering contextual factors, whether human (e.g., modelers background and experience), organizational (e.g., time and cost constraints) or domain-related (e.g., level of criticality, compliance with standards) when introducing a specific approach in an industrial setting. For example, organizations that use model-based systems engineering (MBSE), e.g., to build automotive products, have a very different approach and mindset toward modeling than organizations where modeling is only done informally and not enforced by process descriptions or similar measures. Therefore, another objective of this work is to identify scenarios where modeling has proven successful, based on existing literature and our collective experience on the topic, and to distill future usage scenarios based on successful industrial practices.

Based on the work of Bucchiarone et al. [7], we regard the modeling success stories of MBSE, low-code, and informal modeling and add infrastructure-as-code as an additional success story. To gain a better understanding of informal modeling, we differentiate semi-formal modeling from it. Overall, we are thus regarding five success stories in this paper:

  • Infrastructure-as-Code (IaC) aligns with the definition of modeling as the creation of system representations, where the models are embedded within the code itself. According to Madni et al. [8], a model can take various forms, including modeling languages, algorithms, equations, and parametric curves. This perspective offers a unified approach to both modeling and programming, suggesting that “programs are models,” where code is viewed as a less abstract, textual model [9]. In this context, users are not necessarily aware that they are modeling. Domain-specific languages (DSLs) are mostly implemented as textual languages and take advantage of features of the usual tools/IDEs for programming, like automation and version control. Although these languages are not agnostic of the specificities of different tools, one might argue that IaC is already an abstraction with inherent structure and intentionality, reflecting a specific system deployment. We add IaC as it is a success story for the use of domain-specific languages, especially in a programming context.

  • Low-code is an emerging paradigm combining modeling and graphical programming. There are claims that low-code is not MDE according to [10], or that it is [11]. In this work, we follow the latter. Low-code platforms are modeling environments in which a user combines different pre-defined building blocks into an overall workflow. They are often used to model and automate simple processes or to create dashboards to visualize data. Examples of such platforms are Node-REDFootnote 1 or Outsystems.Footnote 2

  • MBSE in the Automotive Domain uses models intensively for systematically developing and analyzing complex automotive systems architectures with dedicated modeling frameworks. In this domain, the most commonly used languages are UML and SysML to represent the structure of the software architecture at a high level of abstraction and to describe the system’s behavior via state machines. Stateflow/Simulink models are also used to represent subsystems which involve I/O and control design.

  • Informal Modeling produces models which are sketches with no or little structure that have no direct or automatic translation to code. The main objective is to gain a collective understanding of some problem domain, while engaging in intensive communication.

  • Semi-formal Modeling produces models which are possibly created with CASE tools or diagrammatic tools such as draw.io, sometimes with limited automatic analysis or code generation capabilities.

The primary objective of this expert voice is to introduce factors that contribute to MX and their interrelations in the four identified modeling success stories. Moreover, it outlines how the identified factors impact the modeling success stories. This work results from the week-long GI-Dagstuhl Seminar on “Human Factors in Model-driven Engineering” attended by researchers and practitioners experts in the topics of model-driven engineering and human factors [12]. The ultimate objective of this work is to contribute to designing and developing better MDE tools by understanding the MX factors that ultimately hinder MDE adoption.

This paper is structured as follows. In Sect. 2, we define and detail the five modeling success stories mentioned before. In Sect. 3, we introduce the relevant factors that influence MX. In Sect. 4, we analyze the five success stories in light of the factors identified. Finally, in Sect. 5, we discuss the results and outline the research challenges and opportunities related to MX.

Fig. 1
figure 1

MX factors and their relations

Full size image

2 Selected modeling success stories

Bucchiarone et al. [7] introduce three modeling success stories as diverse but archetypal instances of successful applications of modeling. Due to their diversity, they exhibit a broad set of characteristics that influence the modeler’s experience differently. Therefore, it is important to elicit the characteristics of each success story and analyze the extent to which they influence MX. As stated above, we also consider infrastructure-as-code as a modeling success story and consider it as such in the following.

Table 1 presents the characteristics that define each modeling success story. These characteristics help to understand the differences and commonalities of the identified success stories. At this stage, we do not consider the list of characteristics to be complete but sufficient to show that the success stories are different enough to warrant a separate treatment.

Table 1 Overview of the characteristics of the modeling success stories based on Bucchiarone et al. [7]
Full size table

The mapping in Table 1 shows that the chosen success stories already provide a significant diversity in terms of the characteristics we have selected. This means they provide a sufficient difference regarding what modeler experience means in each of them and, ultimately, how they impact the factors of modeler experience as we describe in Sect. 4.

3 Factors of modeler experience

In this section, we first describe the methodology we have followed to identify a set of factors that affect the modeler’s experience. We then present these factors grouped by inherent factors, technical factors, and non-technical factors.

Methodology

We conducted a focus group with ten modeling experts from academia and industry to better understand the factors contributing to MX. The starting point of the focus group was a discussion of modeling success stories that can be observed in the industry. These success stories were defined in smaller groups consisting of two to three focus group participants. Each group identified several characteristics they deemed relevant for the success story described in Sect. 2.

While these characteristics were useful to distinguish the success stories and are used in Sect. 2, the group quickly realized that they are not suitable as descriptors of modeler experience since they are not focused on the modeler. Therefore, as a second step, the group engaged in a discussion that yielded factors more tailored toward the modeler: required training, maintainability, immediate benefits, integration in the programming ecosystem, and reduced friction between modeling and programming. Upon further reflection, the group identified that some of them were goals and others were very hard to measure, even qualitatively.

Therefore, the group engaged in a brainstorming session using the “1-2-4-all” technique: first, individuals brainstormed relevant factors on their own; second, two individuals came together as a group, discussed their respective factors, and consolidated them into a new list; third, four participants got together and consolidated again; and finally, the entire group discussed the different lists of factors and agreed on a final list of factors.

As an additional step, the focus group participants then discussed how the different factors relate to each other (cf. Fig. 1). While originally trying to identify positive and negative influences, this idea was abandoned at one point in favor of a more generic notion of relationship. It is unnecessary to distinguish cases in which a factor can positively or negatively influence another.

The result of these intense discussions during the focus groups is the set of factors described below.

Inherent factors

An inherent factor is based on the characteristics of the problem to address. The complexity of the language is inherent in the complexity of the problem. Language choice is also based on the domain and, ultimately, on the problem that needs to be solved. The chosen language needs to be able to address that problem and needs to have the necessary complexity to describe relevant issues in the problem domain.

  • Language complexity is a measure of effort to solve a modeling problem using a modeling language. Perceived language complexity has been listed as an impacting factor in [4]. In their seminal work, France et al. [13] highlight the challenge of Managing Language Complexity faced by practitioners. Researchers have subsequently employed this classification to assess and prioritize challenges practitioners encounter in software modeling [14].

  • Language choice is the process of selecting, extending, or defining a set of modeling languages that provide suitable domain-specific abstractions. Vendor lock-in creates barriers to MX since it limits interoperability. It should, therefore, be included as a criterion in the selection process. Language choice is also influenced by the system domain, “a sphere of knowledge, influence, or activity. The subject area to which the user applies a program is the domain of the software.” [15] According to a literature review [16], the language used directly influences MX. Modeling Languages usually also cover multiple view points [17]. The need for modeling viewpoints was emphasized for practitioners and is evaluated based on its support for multiple viewpoints and large viewpoint management [18].

Technical factors

These factors concern the way modelers work with the chosen languages and integrate them into their workflows. Specifically, we identified four technical factors:

  • Integration is the conceptual and technical capability of the modeling approach to fit into existing development processes and development platforms. Whittle et al. [19] categorize this issue under “Practical Applicability/Challenges of Applying Tools in Practice.” This is captured in two sub-factors: 1) chaining tools together, addressing the ease or difficulty of using multiple tools for end-to-end functionalities, and 2) flexibility of tools, evaluating a tool’s adaptability to various processes, tools, and working methods without imposing strict processes or requiring additional tools [19]. Mohagheghi et al. [20] report that three of four companies mentioned significant efforts to integrate MDE into the existing processes. The domain was identified as a contributing factor toward adoption. Participants voiced that MDE seems most beneficial for bigger companies, projects, or companies with product lines or similar projects. The vision paper [21] identifies integration into DevOps workflows as an important step toward adopting MDE. They focused specifically on the domain of cyber-physical systems, which would include the automotive industry.

  • Tool UX: “A person’s perceptions and responses that result from the use and/or anticipated use” of the modeling tool [22]. Ease of use and maturity of tools are two aspects mentioned by Mohagheghi et al. [20]. Both aspects can be categorized as Tool UX, where tool maturity is likewise closely related to integration and technical features. Interestingly, the authors found that companies with lower adoption of MDE considered maturity of tools to be worse than those with higher adoption. Some participants suggested enhancing the UI of modeling software to improve Tool UX. In their systematic literature review, Kalantari and Lethbridge [16] classified and documented MX issues into five distinct categories: utility, usability, reliability, emotional, and marketing. Utility and usability are related to ease of use, while reliability corresponds with the maturity of tools.

  • Versioning is the ability to track and merge model changes, facilitating model maintenance, and is one of the main enablers of Collaboration, as stated by Pietron [23]. This suggests that versioning is also an important factor of MX. The inadequacy of version management support has been identified as a drawback of Model-Based Engineering (MBE) tools within the embedded systems domain [24]. The technical challenge of combining versioning with blended modeling is addressed by Exelmans et al. [25]. Collaboration typically alternates between two modes: asynchronous collaboration, where the work is divided, and synchronous collaboration, to obtain a shared understanding [26]. In the literature, support for collaboration has consistently emerged as a desired attribute of modeling tools for practitioners [18, 27,28,29,30]. In the more general field of software engineering, collaborative tools positively impact productivity [31, 32].

  • Technical Capabilities result from the combination of modeling language and modeling tool. They describe how the models can be used in the development process, e.g., reason about properties of the modeled system via simulation or formal analysis or to generate downstream artifacts. For example, Liebel et al. [33] present results from a survey about the technical capabilities used in model-based engineering in the embedded systems domain. Störrle [34] presents results on a survey for which purposes models are used when modeling informally.

Non-technical factors

These factors relate to modeling outside the problem domain and its technical use. Specifically, we identified four non-technical factors:

  • Modeler-intrinsic motivation are those factors that lead modelers to use modeling, in particular, the perceived benefits and positive emotions they experience. Intrinsic motivation is the inherent motivational source that drives individuals to engage in activities they personally find compelling. This stands in contrast to extrinsic motivation, where the incentive originates from an external source rather than the inherent appeal of the task [35]. In software development, empirical research suggests that intrinsic motivation significantly predicts developers’ overall experience [36]. On the other hand, a lack of perceived benefit of modeling has been noted as one of the key mindset barriers for adopting MDE [5].

  • Organization-intrinsic motivation are the factors inside the organization that lead the organization and the modelers to adopt and foster modeling, in particular, because of perceived benefits for productivity, product quality, cost, or collective well-being. The benefits of software modeling practices have been widely discussed in the literature. A survey on modeling practices in the embedded software industry underscores cost savings, shortened development time, reusability, and improved quality as key motivations for adopting MDE [37]. An empirical assessment noted the benefits of MDE in communication and control within a project [38]. The assessment also noted that MDE adoption is affected by culture, expertise, and evangelism within the organization. MDE adoption requires organizational changes, notably, the need for a modeling ‘champion’ and carefully choosing initial projects for applying MDE [38]. Vogelsang et al. [39] noted the importance of managing expectations when adopting MDE.

  • Organization-extrinsic motivation are the factors outside the organization which influence adoption, maturity, and approach of/to modeling, e.g., existing standards, regulations, tool availability and maturity, or customer demands. Adhering to regulations is one of the strong drivers for MDE adoption in the embedded systems industry [39]. The Unified Modeling Language (UML) has been cited as de facto standard for software modeling, with a lot of tools available that support not only model creation and code generation but also viewpoint management, verification, etc. [18]. However, it has been noted that existing research on modeling does not address quality issues reported in industrial context [40].

  • Training includes all factors that are related to the skills and knowledge of modelers in using the selected modeling languages and the tools used to create, manipulate, analyze, and use the models. Training and education have been mentioned as an important factor affecting MDE adoption [38]. Insufficient training resources and support, on the one hand, [5], and the substantial effort required for developer training, on the other [39, 41], are two significant factors in this context. Lack of fundamentals in MDE and education issues have been noted as major current problems in MDE [1]. Moreover, perceived competence, defined as an individual’s subjective judgment regarding their own skills and performance, is identified as a crucial factor contributing to intrinsic motivation [36]. This recognition underlines the interconnected nature of factors in this domain, emphasizing that effective training impacts proficiency and influences individuals’ intrinsic motivation in model-driven practices.

An important aspect of the different motivations is the willingness to take risks on the individual and organizational level. A small organization might be more willing to take a risk with its modeling approach and the tools used than a larger organization because it is more driven by the need to innovate and cannot afford a less risky but more expensive solution.

4 Applying MX to the modeling success stories

In this section we highlight how the different MX factors apply to the modeling success stories introduced previously. We structure each section according to the different groups of factors. Table 2 also shows the importance of the different factors to the different success stories.

Table 2 Mapping of the identified factors to the modeling success stories
Full size table

4.1 Infrastructure-as-code

Inherent factors Infrastructure-as-code is typically used at the beginning of the development process when an infrastructure is needed to execute a developed software. This infrastructure is often managed by a single or only a few DevOps Engineers and evolves as the development process continues. The language depends on the chosen cloud provider or container orchestration tool, so the choice is limited. While it is simple to configure a CRUD web application, due to reusability, community support, and extensive documentation, configuring specialized distributed systems may require more complex features of IaC languages.

Technical factors As IaC is usually written in an IDE (with additional language support), it directly benefits from many features such as versioning, auto-completion, formatting, and testability. IaC is, therefore, highly integrated into the development workflow. Command line interfaces or simple graphical interfaces are then used for provisioning and setting up infrastructure from IaC-configurations, which provides developers with a familiar tool’s UX.

Non-technical factors Infrastructure-as-code is adopted by DevOps engineers to reduce repeated work, improve debugging capabilities, and improve consistency across similar software systems. Organizations further aim to reduce costs, gain flexibility, and improve documentation of their systems. However, depending on previous workflows and infrastructure management, the change to IaC can be demanding and requires training for operations engineers and developers.

4.2 Low-code

Inherent factors Mature low-code platforms support the entire application lifecycle management. They do use proprietary modeling languages, though, so language choice is limited by the chosen tool. Different platforms specialize in different use cases: Node-RED, e.g., is marketed as a tool for data processing and visualization for embedded systems whereas Microsoft’s PowerBI platform is geared toward business process automation and visualization. With these use cases come different feature sets, different language complexity, different levels of extensibility and different support options that modelers have to take into consideration.

Technical factors Low-code platforms provide their own development environment. In many cases, these environments include collaboration and versioning features as well as features to automatically run, test, provision, and deploy the application using cloud-native concepts. They are separate from other IDEs developers might be using for other parts of the system, however. Usability of these tools is generally quite good [42] since the visual aspect of the environment is one of the most important features and many low-code platforms are specifically aiming at non-professionals.

Non-technical factors The switch to low-code platforms is often motivated by organization-wide decisions, e.g., to move to Microsoft PowerBI to modernize legacy systems. We cannot discern situations in which organization would be pushed to adopt such systems due to external circumstances. Depending on the use case, the barrier of entry for such platforms can be significant, but documentation and training material are exhaustive and there are a variety of training options.

4.3 MBSE

Inherent factors The modeling success story in the Model-Based Systems Engineering domain is characterized by extensively using highly sophisticated and standardized modeling languages. Examples are AUTOSAR for modeling the hardware/software architecture, and Matlab/Simulink/Stateflow for modeling the software’s behavior. Due to the ability to generate production code from the models, the modeling languages are highly complex and provide many technical features. The language choice in that domain is restricted as complex value networks between OEMs and suppliers require extensive artifact exchange and, thus, language standardization.

Technical factors In addition to the code generation capabilities, the tools also provide extensive analyses and simulation capabilities. Furthermore, the tools support versioning and integrating software artifacts, e.g., an OEM integrates components from various suppliers. Tool UX is often rather low as the tools are expert and niche tools in addition to the complex languages and the number of technical features.

Non-technical factors The restrictions of the domain highly influence the non-technical factors of the modeling experience in the MBSE success story. Modeling languages and the modeling success story are restricted by organization-extrinsic motivation, e.g., standards or clients require companies to use certain modeling languages and corresponding tools. Furthermore, the company-internal and company-external collaboration is dependent on using the same modeling languages and conforming modeling tools.

4.4 Informal and semi-formal modeling

Inherent factors Informal modeling mostly occurs early in the design process, where conformance to any standardized notation is of minimal concern, as opposed to (creative) expressiveness and a free flow of ideas. Often, the created sketches evolve into (semi-)formal models later on.

Technical factors Much of the informal modeling occurs with analog media such as pen and paper or whiteboards. Although digital alternatives (e.g., electronic whiteboards) exist, their adoption rate remains low in practice, despite technical advantages such as automated persistence and remote collaboration. We think this remains due to poor Tool UX (e.g., responsiveness, viewing angles, accuracy, ...) of these solutions compared to analog media, and we conclude this factor is of utmost importance for informal modeling.

Non-technical factors Informal modeling, and its low-tech nature, comes natural to engineers, and has been done even since before it was considered a form of ‘modeling’. Workforce is intrinsically motivated, and this motivation does not depend on organization-intrinsic or -extrinsic factors. It is often a collaborative activity.

4.5 Comparing the success stories

When comparing the different modeling success stories, we see they are in different places on the spectrum of MX factors. We illustrate the diversity of modeling success stories w.r.t. language complexity and technical capability of the tooling as an example. We believe there are different sweet spots for different modeling workflows: for instance, model-based systems engineering reaps benefits from being formal and, therefore, having high language complexity; this requires tools with high technical capability, which are also very complex to use. Informal modeling, on the other hand, uses languages with very low complexity and requires tools with less technical capabilities, e.g., to reason about models. Both modeling success stories are in stark contrast to UML and its tools in the early days when language complexity was high, but at the same time, rather informal, and the tools had low technical capability.

When using other factors, the different success stories will be positioned differently. For instance, when contrasting language choice and organization-external motivation, MBSE with a high motivation and few available languages poses fewer issues than informal modeling where languages are often made-up on the spot and syntax might not be understood between different companies nor even across teams. This illustrates that our factors are able to capture interesting trade-offs and that a more systematic investigation of these trade-offs in future research is necessary.

5 Conclusion and future work

We defined and explored the concept of modeler experience (MX) in the context of different modeling success stories: infrastructure-as-code, model-based systems engineering (MBSE), and informal and semi-formal modeling. MX, which encompasses factors such as usability, motivation, integration, and language complexity, highlights the dynamics between practitioners and modeling tools. One of the contributions of this paper is the delineation of technical and non-technical MX factors and the characterization of these factors in different modeling success stories. In addition, we show how the proposed MX factors differ between those modeling stories. By examining these success stories in light of MX factors, the paper underscores the contextual nature of modeling practices and the need for tailored approaches to effectively address modelers’ needs.

Although this paper lays a solid foundation for understanding MX, there are still many potential research paths to be explored in the future:

The interplay between MX factors and the success of modeling activities suggests the need for a holistic approach to tool design and workflow integration. Moving forward, research should focus on empirically validating the identified MX factors across diverse contexts and exploring the impact of different workflows on adopting and sustaining modeling practices.

Further empirical studies are essential to quantify the relative importance of each MX factor and how they interact with one another. These studies could take the form of longitudinal research within organizations that are transitioning to or already utilizing model-driven approaches, as well as experimental studies comparing different tool sets and methodologies. Additionally, there is an opportunity to conduct cross-sectional analyses comparing MX across domains, such as automotive, aerospace, healthcare, and finance, where modeling plays a crucial role.

Another promising direction is the development and refinement of modeling tools that incorporate the principles of human-centered design. By focusing on the end-user experience, tool developers can create more intuitive interfaces, improve the integration with existing development ecosystems, and enhance collaborative features. This direction also calls for an iterative design process, where feedback from modelers is continuously incorporated into tool enhancements.

Case studies can provide in-depth insights into the practical challenges of addressing MX in specific contexts and the benefits of modeling in industry. By examining specific cases in detail, researchers can identify best practices, common pitfalls, and innovative uses of modeling tools that may not be evident through other research methods.

In conclusion, by continuing to explore and address the various dimensions of MX, we will serve both academic research and practical tool development, guiding efforts toward powerful models and tools that are a pleasure to use, thereby enhancing productivity and satisfaction for modelers worldwide.