1 Introduction

The term “Robot” was coined by the Czech writer Karel Čapek in 1921 in a science-fiction play. One century later, much of this science-fiction has become reality. Nowadays, a robot might be defined as “an intelligent device with a certain degree of autonomy that contains sensors, control systems, manipulators, power supplies and software all working together to perform the required tasks” [1]. In this paper, we focus on mobile robots, which are robotic systems carried around by locomotive elements and which can operate in various environments (aquatic, aerial, or terrestrial). The operational environment decides the locomotive elements. Robotics comes in many diverse shapes, which are increasingly becoming part of our lives: automated vehicles bring us to work [2] where we cooperate with industrial robots [3] or service robots [4].

Despite robots becoming ubiquitous, robotics is one of the most challenging fields of cyber-physical systems’ research and engineering. The H2020 Multi-Annual Robotics Roadmap ICT-2016Footnote 1 describes the production of software for robotic systems as a “craftsmanship in building robotic systems instead of following established engineering processes”. On the same line, a previous study identifies one of the main challenges of robotics as promoting “a shift towards well-defined engineering approaches able to stimulate component supply-chains and significantly impact the robotics marketplace” [1].

The successful engineering of non-trivial robotics systems demands the interdisciplinary collaboration of experts from different domains, such as mechanical and electrical engineering, artificial intelligence, manipulation, Human-Robot Interaction (HRI), and systems engineering. In all of these domains, experts traditionally employ various kinds of models to describe or prescribe parts of their views on the robots under development. For instance, in mechanical engineering, these models may be CAD geometries or differential equations, in path planning, these may be the environment geometries, in artificial intelligence, these may be artificial neural networks, in HRI these may be user interface models, and in systems engineering, these may be SysML [5] diagrams.

Due to this heterogeneity research and industry have brought forth a multitude of model-driven solutions to engineering the software of Mobile Robot Systems (MRS) [6], such as modelling tools and languages supporting the development of robotics solutions focusing on kinematics, dynamics, sensing, motion planning, force control, reasoning, and architectures. These solutions range from very general modelling techniques, including architecture modelling [7] (e.g. SmartMDSD [8] or Orocos [9]) or mission planning [10, 11], to specific application domains, such as manufacturing [12], to solutions focusing on individual platforms [13]. This wealth of platforms, application domains, modelling and techniques warrants investigating the state-of-the-art systematically to support researchers and practitioners in identifying and selecting model-driven technologies to facilitating engineering of mobile robot systems.

Although the use of model-driven engineering (MDE) in robotics is deemed promising [14], there is no systematic evidence on which kinds of robots are supported by existing MDE approaches, which type of support models provide, how existing MDE approaches are validated, and how tools support existing MDE approaches. To better understand all these aspects, this paper provides a map of software engineering research in MDE for mobile robotic systems, which can guide researchers and practitioners from robotics and software engineering through the research landscape and help to shed light on future challenges. It is important to highlight that in this work, we explicitly focus on software aspects, and we do not dig into the interplay of software and hardware. This would require a separate study and most probably it would demand dedicated (MDE) approaches and tools, software aspects and specifically in software engineering for robotics [15]. This scopes the work and helps us in identifying the papers to be included in this mapping study. The side effect of this choice would be not providing a deep analysis of the interplay of software and hardware, which would require a separate study. For example, the report on the DARPA Urban Challenge presented in [16] is potentially interesting since it describes the model-based design of an autonomous ground vehicle, but it focusses primarily on systems/control models, rather than software. Also, in this work we do not focus on what needs to be done during runtime in order to guarantee important properties, like reliability, availability, maintainability and safety, as for instance discussed in [17]. It is also important to point out that by construction, this study only considers published papers that are captured by our search and selection process. This would imply that potentially interesting studies and approaches might be left out of the selection, like the ESA TASTE toolchain,Footnote 2 and 20-Sim, an MDE toolchain for modelling, verification, code generation, and simulation of dynamic systems including mobile robots.Footnote 3

In this work, we aim at answering the following research questions:

  1. RQ1:

    What are the publication trends of research in MDE for MRSs?

  2. RQ2:

    Which types of MRS are supported by existing MDE approaches?

  3. RQ3:

    How do existing MDE approaches support the engineering of MRSs?

  4. RQ4:

    How are researchers validating existing MDE approaches for MRSs?

  5. RQ5:

    What does the landscape of tools supporting MDE for MRSs look like?

To answer these research questions, we performed a systematic mapping study [18, 19]. A mapping study research methodology permits, in fact, to identify, classify, and evaluate focus, trends, and open challenges.

The main contributions of this paper are:

  1. 1.

    An overview of the academic venues and forums where the community disseminate the research ideas and results in the field, together with the density of scientific interest in MDE for MRSs.

  2. 2.

    The types and characteristics of MRSs engineered using MDE approaches.

  3. 3.

    An overview on how MDE approaches support the engineering of MRSs, especially focusing on employed modelling languages and model transformations. Moreover, we describe the aspects of MRSs that are engineered by MDE approaches, such as behaviour, movement, image recognition, and pathfinding.

  4. 4.

    A precise characterization of the types of performed validations, as well as the characteristics of MRSs used for the evaluation of MDE approaches for MRS.

  5. 5.

    A description of the tools and tool types that support MDE approaches for MRSs.

  6. 6.

    A replication package for independently assessing and replicating the study.

Outline of the paper. The remainder of the paper is structured as follows. Section 2 details the background of our research before, Sect. 3 details the design of our study. Afterwards, Sect. 4 presents its results, and Sect. 5 uncovers relations between these. Section 6 discusses results, relations, and their implications. Subsequently, Sect. 7 debates threats to validity, and Sect. 8 highlights related work. Section 9 closes the paper.

2 Model-driven engineering and robotics

In the last couple of decades, the amount and the importance of software and software-intensive systems have increased significantly and their complexity has increased exponentially. The importance of software has increased exponentially along with the complexity of software-intensive systems. Thus, software engineers have begun to leverage models to reduce the conceptual gap [20] that arises from using different abstractions in the problem domains at hand, such as robotics, rather than in the solution domain.

Robotics is a domain driven by four challenges [21]:

  1. 1.

    lack of standardization,

  2. 2.

    its multi-disciplinary nature complicating reuse,

  3. 3.

    uncertain environments, and

  4. 4.

    the lack of sufficiently expressive, domain-specific languages.

Consequently, robotics has motivated software engineering research for decades and many frameworks and toolchains to mitigate the challenges in robotics have been developed. Many approaches to software engineering for robotics aim to improve encapsulation and reuse of robotics software solutions by applying component-based software engineering [22] to robotics [23]. For instance, Orca [24], OROCOS [9], and the Robot Operating System (ROS) [25] contribute solutions to modularize and componentize robotics software architectures. Similarly, the trend towards distributed, cloud-based systems has affected robotics at well [26]. To foster creation of distributed multi-robotics systems that interact through the cloud, various methods [27], platforms [28], and tools have been developed [29].

In line with the fourth challenge, MDE has been identified [30] as a core technology to support developers when designing robotic systems. Consequently, robotics has become a domain of original research on model-driven engineering [31].

Model-driven engineering [31,32,33] is an umbrella term for software development methodologies that leverage (domain-specific) models as primary development artefacts to reduce this gap. To support efficient engineering with such models, these must be machine-processable, which demands corresponding notations, e.g. in the form of meta-models [34, 35] or grammars [36] to describe corresponding domain-specific languages (DSLs).

On one hand, models corresponding to different domain-specific notations can be created, edited, and understood, by experts of the different domains contributing to the development of modern software-intensive systems. On the other hand, such models can be processed by sophisticated modelling tools and infrastructures provided by software engineering experts to perform more complex analyses and transformations automatically—thus liberating domain experts from the need to become software engineering experts themselves. These different types of models can be identified as descriptive models, those mainly used for documentation, and prescriptive models, those mostly used for development [37].

In the last years, several papers have been published presenting how industries use MDE and, therefore discussing the importance of modelling. For instance, the works by Kuhn et al. [38] and Aranda et al. [39] report about the use of MDE at General Motors and the work in [40] describes the experience of introducing MDE at Motorola.

These studies show somehow contradictory findings. The work by Liebel et al. [41] shows that modelling can be suitable for embedded systems, and, given that one understands the consequences of adopting modelling, the study by Vallecillo et al. [42] convinces us that modelling is ready for industry. A different point of view is suggested by other studies. The work by Gorschek et al. [43] shows that there is a lack of use of modelling in industry. Similar findings can be found in the work by Petre et al. [44]. The work by Heldal et al. [37] points out that in order to understand these contradictory messages, it is important to distinguish between descriptive models and prescriptive models. In fact, this study shows that there are substantial differences in the use, importance during the development, maintenance, and management of these two types of models.

The work by Ciccozzi et al. [45] focuses on mission-critical Internet of Things systems, which are exposed to uncertainty and variability and which require high availability, reliability, safety, security, regulatory compliance, scalability, and serviceability. The work discusses how MDE can help meet the technical challenges. The work by Wortmann et al. [46] focuses on modelling in the smart factory of Industry 4.0 and identified various modelling techniques being used that are tailored specifically to that domain. Successful applications of MDE have been reported in the context of software engineering for various domains [31, 47, 48].

Model-driven engineering for robotics comes in diverse shapes and guises. For instance, several works have been proposed in the last years on the use of architecture modelling languages and techniques for robotic software systems [49,50,51]. The research reported in [49] contributes the SmartSoft modelling language for software architectures and behaviour of robotic systems, which is supported by a fully automated code generation toolchain to produce executable robotics applications. The BRICS component model presented in [51] also aims to ease modelling of robotics software architectures by reusing tried and tested component models as black boxes. The work presented in [52] combines multiple modelling languages to describe robot capabilities, mission goals, and software architectures from which applications able to interface SmartSoft and ROS [25] can be generated.

Other research in robotics focuses on modelling robotic missions. For instance, by proposing better comprehensible mission DSLs that facilitate their definition by non-programmers [10, 53] or through identification of specification patterns to further ease robot mission planning [54, 55]. The application of block-based modelling languages also is subject of is addressed by ongoing research in robotics too [56,57,58]. While being less domain-specific than mission planning languages, this research trend holds promise for easing robot programming in the future.

Overall, there is a plethora of publications on the application of model-based and model-driven techniques to robotics, there only are few studies on this.

Summarizing, the use of MDE in robotics looks promising as testified by many initiatives, works, and statements made by organizations. However, there is no reproducible evidence on which types of robots are supported by existing MDE approaches, which type of support models provide, how existing MDE approaches are validated, and how tools support existing MDE approaches. To better understand all these aspects, we performed this mapping study that will be detailed in the remainder of the paper.

3 Study design

We designed and performed this study following the established guidelines for systematic mapping studies [18, 19]. Based on these, this section introduces the core aspects of the design of our mapping study. Specifically, we present the research goal and questions driving the work and describe the various steps of our research method.

A complete replication package is publicly availableFootnote 4 for independent replication and verification of our study. The replication package includes the raw data of our search and selection phase, the list of selected primary studies, the raw data extracted from each primary study, and the R scripts we developed for data exploration and analysis.

3.1 Research goal and questions

The goal of this study is to identify, classify, and evaluate trends, focus, and open challenges in existing research in MDE for MRSs. We are interested in the points of view of both researchers and practitioners.

With this goal in mind, we applied the population intervention comparison outcome context (PICOC) criterion in accordance with [59], and a set of research questions that address different aspects of the state-of-the-art of MDE for MRSs. Our PICOC criterion definition is the following:

  • POPULATION: Mobile Robot Systems

  • INTERVENTION: Model-Driven Engineering approaches for Mobile Robot Systems

  • COMPARISON: not applicable

  • OUTCOMES: a classification of the primary studies that reflects the current state-of-the-art of MDE for MRSs

  • CONTEXT: academic peer-reviewed publications with a software engineering perspective

The research questions to be answered for achieving our goal are:

  1. RQ1:

    What are the publication trends of research in MDE for MRSs? Rationale: Research in academia is a multi-faceted and dynamic ecosystem, where several research groups investigate specific scientific problems over time with different levels of independence and different methodologies, as well as targeting different forums and venues for dissemination and communication. Outcome: A snapshot of (1) the set of academic venues and forums where research ideas and results concerning MDE for MRSs, and (2) the density of scientific interest in MDE for MRSs.

  2. RQ2:

    Which types of MRS are supported by existing MDE approaches? Rationale: There are several types of MRS that have been engineered using MDE approaches. Outcome: The set of characteristics of MRSs engineered using MDE approaches.

  3. RQ3:

    How do existing MDE approaches support the engineering of MRSs? Rationale: Different MDE approaches for MRSs focus on different engineering aspects as well as different characteristics of MRSs. Outcomes: (1) A mapping of how MDE approaches support the engineering of MRSs, with special attention to: entailed modelling languages and automation mechanisms (i.e. model transformations), and (2) the set of aspects of MRSs that are engineered by MDE approaches (e.g. behaviour, movement, image recognition, pathfinding, architecture).

  4. RQ4:

    How are researchers validating existing MDE approaches for MRSs? Rationale: MDE approaches for MSRs can have different characteristics and support different types of MRSs. Outcome: A precise characterization of (1) the types of validation applied to the approaches and (2) the types of MRSs used as subjects during the validation of the approaches.

    1. RQ4.1:

      Which types of validation have been applied to existing MDE approaches for MRSs? Rationale: Research results can undergo different types of validation and evaluation, as described by Wieringa et al [60]. Outcome: A map that illustrates how MDE approaches for MRSs have been validated or evaluated (e.g. industrial case-studies, experiments in a lab, etc.).

    2. RQ4.2:

      Which types of MRS have been used when evaluating existing MDE approaches for MSRs? Rationale: There exist several types of MRS that have been used when evaluating MDE approaches for MRSs. Outcome: The set of characteristics of MRSs used for the evaluation of MDE approaches for MSRs.

  5. RQ5:

    What does the landscape of tools supporting MDE for MRSs look like? Rationale: One of the pillars of MDE is the provision of automated mechanisms for alleviating engineering effort. Automation is provided by specific tools. Outcome: A catalogue of tools (and tool types) supporting MDE approaches for MRSs.

3.2 Search and selection process

Figure 1 illustrates the main phases of the search and selection process of this study, together with the number of potentially relevant studies involved in each phase.

Fig. 1
figure 1

Our search and selection process comprises five phases to search, identify, and process relevant publications

1. Initial search. In this phase, we perform automatic searches on an heterogenous set of electronic databases and indexing systems [61]. As suggested by Kitchenham et al. [61], to cover as much potentially relevant literature as possible, we choose multiple data sources; in our case they are five of the largest and most complete scientific databases and indexing systems in software engineering: IEEE Xplore Digital Library, ACM Digital Library, SCOPUS, Science Direct, and Web of Science. The search is performed considering title, abstract, and keywords of the considered entries.

figure g

The search string used for querying the above-mentioned data sources is shown in Listing 3.2. In order to keep our focus centred around the main goal of the study, the search string has been created by considering our goal and research questions. Specifically, our search string is composed of three main components linked by a logical and relationship, each targeting a specific facet of MRSs: the first component (line 1 in the listing) targets the mobility aspect of MRSs, the second component (line 3) targets the robotics domain, and the third component (line 5) targets the model-driven engineering discipline. In order to be reasonably confident about the coverage of our search string, within each of its three components we (1) include keywords related to its corresponding facet, based on the pilot studies and known primary studies, (2) add synonyms to the already added terms, (3) add alternative spellings and acronyms of the already added terms, and finally (4) link all the added terms via a logical or relationship. To ensure consistency, the search strings is always applied to the title, abstract and keywords of papers. Note that the search string was adapted and, in some cases, broken down into multiple strings to comply to the constraints of individual databases or indexing systems (refer to the replication package for more details). This phase produced an initial corpus of 2809 potentially relevant studies.

2. Merging and impurity removal. Due to the nature of the electronic databases and indexing systems, the search results can also include items that were not research papers, such as complete conference and workshop proceedings, international standards, textbooks, editorials, and similar. Consequently, in this phase, we manually remove these results as well as duplicates to obtain a coherent corpus of 2541 potentially relevant research studies. Eventually, the remaining papers from the various databases are combined into a single dataset.

3. Application of selection criteria. In this phase, we consider all the potentially relevant studies and filter them according to a set of well-defined selection criteria. The used selection criteria are reported below.

Inclusion criteria

  1. I1:

    Studies applying MDE methods or techniques for MRSs. We consider the definition of methodology, given by [62], as a set of methods and techniques, possibly supported by a tool.

  2. I2:

    Studies focusing on MDE for MRSs analysed from a software engineering for roboticsFootnote 5 perspective. For example, we do not consider studies that cite MDE as a background, but actually focus on other aspects, such as mechanical or electrical engineering.

  3. l3:

    Studies including a kind of evaluation or validation of the proposed MDE method or technique. For example, real-world software projects, empirical examples, case study, experiment, formal analysis.

  4. I4:

    Peer-reviewed studies: journal papers, workshop papers, conference papers.

  5. I5:

    Studies written in English.

  6. I6:

    Studies are available in full text.

Exclusion criteria

  1. E1:

    Secondary literature studies.

  2. E2:

    Tutorial papers, short papers, poster papers, editorials, books, keynotes, tutorial summaries, tool demonstrations and panel discussions, books, introductory papers for books and workshops, technical reports, and any non-peer-reviewed publication.

  3. E3:

    Studies not in the scope of software engineering for robotics.

  4. E4:

    Studies published before 2000. Since MDE is generally regarded to be born after the OMG standard for the model-driven architecture proposed in 2000 [63].

To be included in the corpus of primary studies, a potentially relevant study must fulfil all inclusion criteria and none of the exclusion criteria. For each study, an adaptive reading depth was applied in order to carry out the selection process in a time-efficient and objective manner [18]. Specifically, in the first round, the title of the study is considered, enabling us to discard all those studies that clearly do not fall in the domain of MDE for MRSs. In the second and third round, we consider (1) the abstract and keywords and (2) introduction and conclusions of the study, respectively. Finally, if a solid decision about the selection of the study was not taken yet, the full text of the study is inspected.

In order to reduce possible biases, three researchers were involved in this phase of the study. Firstly, we piloted our selection procedure by having three researchers independently assessing a random sample of ten potentially relevant studies; for all ten pilot studies, all researchers agreed (i.e. one included study and nine excluded). Based on the promising results, two researchers proceeded with the independent assessment of all the other potentially relevant studies. Following the suggestion by Wohlin et al. [64], we computed the inter-researcher agreement using the Cohen Kappa statistic and reported it as a quality assessment of this stage of our study. We achieved a value of 0.84 for the Cohen Kappa statistic, which is a good indication of the objectivity in which the selection has been performed which, being \(\ge 0.80\), indicates successful objectivity of our selection process according to Wohlin et al. [64]. Finally, all conflicts were resolved by discussing them with a third researcher, who played the role of arbiter so to avoid endless discussions [65].

The selection procedure described above reduced the corpus to 69 primary studies.

4. Snowballing To mitigate potential bias with respect to the construct validity of the study, we complement the previously described automatic search with a snowballing activity [66]. The main goal of this phase is to enlarge the set of potentially relevant studies by considering each study selected in the previous phases and focusing on those papers that either cited or were cited by it. Note that we perform a closed recursive backward and forward snowballing activity, as shown in [67].

After snowballing, we identify 97 primary studies (and we use them for answering RQ1). However, among those 97 primary studies, some of them were about the same approach, simple extensions of preliminary studies (already included in our work), and similar minor variations. Consequently, we collapsed those 97 studies into 80 unique primary studies

for research questions investigating the content of the studies (e.g. approaches, robot types, applied MDE methods, etc.).

3.3 Data extraction

Through this activity, we create a data extraction form to be used to collect data extracted from our primary studies. The form is composed of five facets, each of which addressing a specific research question. Specifically, for answering RQ1 (i.e. research question about publication trends), we consider standard information, such as title, authors, and publication details of each study. As for the other research questions, (i.e. RQ2, RQ3, RQ4, RQ5), we follow a systematic process based on the keywording technique [18] for: (1) defining the parameters of each facet of the data extraction form, and (2) extracting data from the primary studies accordingly. The main benefit of the keywording technique is that it allows us to effectively create an extraction form that fits existing studies and tools, and which takes their characteristics into account [18]. The steps of this process are:

  1. 1.

    Identify keywords and concepts. We collect keywords and concepts by reading the full text of each pilot study. After all the pilot studies were analysed, the extracted keywords and concepts were combined together to clearly identify the context, nature, and contribution of research on MDE for MRSs. The output of this stage was the full set of keywords and concepts extracted from the pilot studies.

  2. 2.

    Cluster keywords and define parameters. We perform a clustering operation on collected keywords and concepts in order to organize them according to the identified characteristics. The clustering operation is very similar to the sorting phase of the grounded theory methodology [68]. Starting from all the keywords and concepts collected in the previous phase, we iteratively group them until saturation of all the concepts is achieved. The output of this stage is the initial data extraction form containing all the identified parameters, each of which represents a specific characteristic of an MDE approach for MRSs. The following steps are performed individually for each primary study.

  3. 3.

    Extract data from the current study. We first extract information about the current primary study to be analysed and then collect information based on the parameters of the data extraction form. In some cases, a primary study can have multiple values for a single parameter. For example, multiple automation techniques can be used within the same approach (e.g. model transformation and automated analysis), multiple types of robots can be used when validating an approach (e.g. aerial and aquatic robots together), and so on. When needed, this aspect is highlighted in the paper. Finally, we collect any kind of additional information that is deemed relevant but that does not fit within the data extraction form.

  4. 4.

    Refine the data extraction form. We review the collected additional information because of one of these reasons:

    • The collected information is not interpreted correctly. In this case, the collected information is refined.

    • The parameters of the data extraction form are not representative enough for the considered primary study. In this case, the data extraction form is refined so that it better fits the collected information. Previously analysed primary studies are reanalysed according to the refined data extraction form.

The above process is complete when all primary studies are analysed. Once we have a complete data extraction form, we isolate the facet related to the technical characteristics of the approaches (i.e. the facet related to RQ3). This is then analysed further to establish any relationships between the parameters, and also to determine the multiplicity and possible values of each parameter. This process is performed through several iterations and leads to the definition of a classification framework that can be used by researchers and practitioners for classifying, comparing, and evaluating solutions for MDE of MRSs in an objective manner. The classification framework is summarized in Table 1.

Table 1 Classification framework of our study follows its research questions

3.4 Data synthesis

We designed and performed the data synthesis activity by following the recommendations by Cruzes et al. [69]. More specifically, our data synthesis activity is divided into two activities: vertical analysis and orthogonal analysis.

In vertical analysis, we analyse the extracted data to find trends and collect information about each parameter of our classification framework. In this case, we apply the line of argument synthesis [64]; that is, we first analyse each primary study individually in order to document it and to tabulate its main features with respect to each specific parameter of the data extraction form. Next, we analyse the set of studies as a whole, in order to reason about potential patterns and trends. We present the results of this vertical analysis in Sect. 4.

In the orthogonal analysis, we analyse the extracted data to explore it and to uncover the existence of possible interesting relations between data pertaining to different aspects of our research. For this purpose, we use contingency tables analysis.Footnote 6 The results of the orthogonal analysis are presented in Sect. 5.

In both cases, we perform a combination of content analysis [70] (mainly for categorizing and coding approaches under broad thematic categories) and narrative synthesis [71] (mainly for detailed explanation and interpretation of the findings coming from the content analysis).

4 Results

In this section, we present the results of this study grouped by research question. For each research question, we report both quantitative data and an interpretation of the obtained results. Primary studies are identified with a unique code, Px (where \(1\le x \le 97\), see Appendix A for the complete list).

4.1 Publication trends (RQ1)

For answering RQ1, we consider all selected publications after the snowballing procedure, independently of the merging operation we performed on studies about the same MDE approach. So, here we are considering a total of 97 publications, including both the final set of 80 primary studies and the 17 studies which have been merged during the data extraction phase (see Sect. 3.2). We merge those studies to prevent skewing with quantitative results without missing potential unique details carried by these redundant studies.

In addition to standard bibliographic data like authors, title, and year (they are all available in the replication package), for each of the 97 publications, we consider the following information: publication venue, research type, and contribution type.

Fig. 2
figure 2

Publication venues of primary studies

As shown in Fig. 2, the most targeted publication venues are conferences (64/97), followed by journals (19/97), workshops (12/97), and book chapters (2/97).

In Table 2, we report the publication venues which have been targeted at least twice by researchers (the last row of the table is an aggregate of all the publication venues with less than two publications). SIMPAR and IROS are by far the most targeted publication venues (11/97 and 6/97, respectively), followed by IRC (3/97); those top-3 targeted venues are specific to the robotics domain. The other targeted venues are either on software engineering (i.e. ASE, EASE, ACM SE), at the intersection of software engineering and robotics (i.e. JOSER and RoSE), or about industrial automation technologies (i.e. INDIN). We can consider the venues mentioned above as good candidates for future publications on MDE approaches for MRSs. Interestingly, the IEEE International Conference on Robotics and Automation (ICRA) does not appear among the most targeted publication venues despite being the most important conference for robotics. We conjecture that this is due to the fact that researchers working on MDE for robotic systems tend to publish in software engineering venues, from our perspective missing a potential chance of impacting the robotics state of the art and practice. Finally, we notice that a large portion of publication venues have been targeted by researchers in MDE for robotics only once. This may be an indication of the scientific interest around MDE techniques across many orthogonal research areas and domains, such as aerospace, multiagent systems, artificial intelligence, communication and networking, self-adaptive systems, systems engineering. Alternatively, this may be due those research works not gaining momentum in these venues or that the results were very specific and hard to build upon. Future analysis of the field could detail this investigation.

Table 2 Targeted publication venues

Regarding research types, we classify each primary study into one or more of the 6 categories reported in Table 3. Those categories have been proposed by Wieringa et al. [60], and we chose them because (1) they are widely used and accepted in various secondary studies in software engineering (e.g. [72,73,74,75]), and (2) they can be identified in the primary studies in a cost-effective manner [18].

Table 3 Categories of research types (adapted from [18])

As shown in Fig. 3, the majority of primary studies involve a solution proposal (83/97), immediately followed by validation research.Footnote 7 The “solution + validation research” is a very recurrent pattern among our primary studies and this unveils one of the major points of improvement for the MDE community working on robotics: proposed approached are very often not validated in practice (e.g. via an industrial case study or using robots in a production environment). If on one side, we can understand that when a new approach is proposed, the first immediate step is to prove its applicability or feasibility in a controlled environment, on the other we call the community conducting research on MDE for robotics to close the loop from a methodological perspective and apply their proposed approaches in practice. This activity will surely help the overall research field in terms of credibility, industrial adoption and, more importantly, it can potentially trigger a fruitful wave of collaborations between academics and industrial practitioners.

Fig. 3
figure 3

Research types of primary studies. Among our primary studies, there are no occurrences of philosophical papers and opinion papers

The five studies performing evaluation research and the four experience papers provide evidence that some steps are already being done towards the industrial application of MDE approaches for MRSs. As an example of successful evaluation research, P19 presents a modelling infrastructure based on a set of declarative DSLs for specifying and executing robotics tasks. Models conforming to those DSLs are transformed into a component-based implementation integrated with a third-party planner for suitably executing the modelled tasks. The whole approach has been applied for a week in the Katharinen Hospital in Stuttgart, where the realized system faced daily hospital routines and operated between real patients and caregivers.

We characterize the contribution types of each publication by reusing the categorization proposed in Petersen et al. [18]. According to such categorization, each primary study can provide one or more of the contribution types presented in Table 4.

As shown in Fig. 4, the main contribution types are method (97 occurrences), tool (71 occurrences), and model (58 occurrences). This result does not come as a surprise since the MDE paradigm is fundamentally rooted in the (regulated and/or formalized) use and manipulation of models, predominantly via automated tool support (e.g. for models transformation, analysis, and interpretation).

Open items are discussed in 4 primary studies (P74, P75, P84, and P89). Those studies have been published between 2015 and 2017 and the described open items are mostly about how the proposed method and technique can be extended and/or enhanced. The lack of papers exclusively dealing with the elaboration of challenges in MDE for mobile robotic systems may be seen as an indication of the actual need of scoping studies like this one, which may help researchers in objectively identifying open research gaps in the field.

Table 4 Categories of contribution types (adapted from [18])

Four primary studies are contributing specific metrics for engineering MRSs via MDE techniques (P1, P2, P43, P53). For example, the authors of P43 present an approach supporting the integration of safety techniques within an MDE-based process for robotic systems. The proposed approach is realized as a methodology and an Eclipse-based framework in which safety engineers can use the concepts and metrics of the IEC61508 standard for functional safety [76]. The framework is able to automatically assess the safety of robotic systems [1] in their early development phases. By quoting the authors of P43, the use of the proposed methodology [fills] the gap between system modelling and safety assessment tools and helps to better cope with system engineering time and cost constraints. Even though we found only four primary studies proposing metrics for MRSs, this can be seen as a positive trait of the MDE community since it shows the interest in the measurement and assessment of MRSs from some perspectives of interest (e.g. safety by design).

4.2 Types of supported MRSs (RQ2)

This research question focuses on the types of robots supported by the analysed MDE approaches for MRSs and on whether the engineered approaches involve either multiple robots or a single robot.

For extracting the types of considered robots,we considered only studies where the authors explicitly claimed that their proposed approach is specific to robots with certain characteristics (e.g. unmanned autonomous vehicles—P40) and clustered such studies according to the mentioned type of robot (e.g. aerial robots).

Based on our data extraction strategy, we identified five categories of robots, namely terrestrial robots, aerial robots, aquatic robots, space robots, and genericFootnote 8 robots that are addressed in the corpus. In addition, we keep track of those primary studies where the proposed MDE approach is generic, i.e. it is not tailored to any specific type of robot and thus in principle, it is applicable to any robotic system.

Fig. 4
figure 4

Contribution types of primary studies

Fig. 5
figure 5

Types of robots

Figure 5 shows that the vast majority of contributed approaches is related to generic robots (51/80 primary studies). This result can be seen as an indication of the high level of flexibility of MDE methods and techniques, which can be applied to different types of MRSs, while their inner mechanisms remain at a high level of abstraction. The flexibility of generic MDE approaches is also confirmed in practice by the fact that among them, we can find two of the five studies performing evaluation research and one of the four experience papers (see Fig. 3). This is followed by terrestrial robots (21/80 primary studies), which includes three studies on humanoid robots.

In some cases, generic approaches trade the independence from the types of robots with the types of supported missions. For example, the modelling platform described in P11 achieves genericity by means of three domain-specific modelling languages: MML (Monitoring Mission Language), RL (Robot Language), and BL (Mission Behaviour Language). MML allows non-technical operators to specify missions as sequences of tasks, task dependencies, forks/joins, areas, and obstacles; RL allows robot engineers to extend the generic platform and specify the technical characteristics of new types of robots (e.g. hardware configuration, sensors); BL permits to specify the chain of atomic movements of each individual robot declared in MML and RL models. The languages above have been designed so to allow the platform to automatically generate BL models from MML and RL models, thus (1) relieving the operator from the intricacy of low-level details of the individual robots and (2) making the whole platform independent of the types of used robots. Here, the trade-off lies in the fact that P11 supports the definition of only civilian monitoring missions, e.g. monitoring of forestry, control of pollution emissions, surveillance of public events.

Among the approaches supporting specific types of robots, terrestrial robots are the most supported ones (16/80 primary studies), followed by other types of approaches with much lower occurrences. For example, P44 proposes a domain-specific modelling language for formally specifying constraint-based programming tasks. The DSL is specific for service robots and provides constructs such as robotic scene, location, object, etc. Models conforming to such a DSL can be used as starting points for automatic model verification and code generation targeting different platforms (both hardware and software). The DSL comes with a textual concrete syntax and promotes reuse and component-based development of robotic systems by grouping potentially reusable functionalities.

No approaches were specific to aquatic robotic systems (e.g. underwater robots). Nevertheless, aquatic robots have been used twice as subjects during the validation of generic MDE approaches (see Sect. 4.4.2). This is a further confirmation of the flexibility and applicability of MDE methods and techniques in the robotics field.

When dealing with the cardinality of robots, we classify each primary study according to whether its proposed approach assumes that the system is composed of multiple robots or a single robot.

Fig. 6
figure 6

Cardinality of robots

As shown in Fig. 6, More than two thirds of the approaches support single robots only (61/80 primary studies) but there is also a non-negligible number of studies supporting systems with multiple robots (19/80 primary studies).

This result is promising since supporting more than one robot introduces additional challenges, such as robots coordination and synchronization, communication issues (e.g. path loss), communication overhead, etc. We speculate that this result may be an indication that MDE techniques can support engineers in abstracting from and managing the intrinsic variability emerging when managing multiple different robots. For example, P16 proposes an OWL-based ontology of mobile robotics capabilities, a UML-based DSL for describing robotic systems with variation points which can be filled by third-party entities, and a code generator targeting several robotic middlewares (e.g. OROCOS [9]) and simulators (e.g. Blender MORSE [77]).

Moreover, MDE techniques can also help to engineer in enabling better reasoning on the high-level problems emerging when dealing with multiple robots, such as the formation of robot ensembles at run-time. For example, P7 proposes a framework for programming distributed mobile robots in such a way that collisions are prevented, and each individual robot moves in the environment according to optimal motion plans. The approach presented in P7 is based on a provably correct multi-robot motion planner which is decentralized, asynchronous, and reactive to dynamically generated task requests. At the core of the motion planner lies P, a state-machine-based DSL for defining event-driven asynchronous programs for the robots. P specifications are verified at design-time using the Zing model checker [78]. Finally, C code is automatically generated from P specifications and directly deployed as ROS [25] nodes in the system.

4.3 Applied MDE methods and techniques (RQ3)

4.3.1 Types of modelling language

Observing the types of modelling language used by the analysed approaches, it is not surprising that DSLs, both non-UML and UML-based, own the stage. In some cases, different types of languages are used in combination in order to enable specific kinds of analysis or modelling different aspects of the MRS. Interestingly, some studies report on experiences with different languages for solving a specific problem. An example is P13, where UML Real-time, derived from the ROOM language [79], is compared to AADL [80] for modelling and timing analysis of complex (swarming) problems. In this specific study, the authors suggest that AADL and UML real-time were both suitable for their purpose, so the choice on which one to prefer highly depends on the tools implementing them (Fig. 7).

Fig. 7
figure 7

Types of modelling languages applied to MRS

4.3.2 UML-based languages

Although representing almost half of the languages mentioned in the analysed approaches, UML-based languages are rarely explicitly enclosed in a so-called UML profile. The most often used UML profiles are SysML [81] and SmartSoft (cf. Table 5). In P10 for instance, the authors exploit the SmartMDSD toolchain [82], composed of a SmartSoft implementation in terms of UML profiles for graphical modelling and an orthogonal Xtext-based language for textual modelling. It is somewhat interesting to note that the most recent (i.e. youngest) UML profile of those encountered in our study is RobotML, introduced 15 years ago; this seems to point out a trend of rather reusing established profiles in combination with other DSLs than designing new profiles tailored for specific application domains. This is in line with the wealth of DSLs identified for specific robotics challenges (kinematics, architecture, navigation, etc) [83].

Table 5 UML profiles applied to MRS

4.3.3 Engineered aspects

Figure 8 provides an overview of the main keywords we collected for the engineered aspects in the primary studies. We decided to report the raw keywords because they can help the reader in having a bird eye view of the various research themes and schools of thought that MDE researchers follow when dealing with MSRs. As shown in the figure, primary studies exhibit a relatively high fragmentation in terms of the aspects of MRSs engineered by the proposed approaches. Authors are considering aspects such as a component-based perspective on the MRS (eight occurrences), its software architecture (seven occurrences), separation of concerns (four occurrences), safety (four occurrences), system variability (four occurrences), software product lines (three occurrences), etc. This result is expected since several research groups are currently active in applying MDE techniques on MRSs and are targeting publication venues belonging to various computing domains (see Sect. 4.1). We can also notice primary studies investigating on less-explored research venues, such as methods for managing the uncertainty of units of measure at runtime (P1), monitoring functional requirements at runtime (P20), the composition of languages and code generators (P23) and methods for systematically testing autonomous robots (P67). We invite the reader to use those studies as starting points for understanding what MDE techniques are capable of, in addition to more canonical applications like code generation, mission specification, abstraction from the robotic platform, etc. For a complete overview, the interested reader can refer to our replication package, where the engineered aspect of each primary study is available.

Fig. 8
figure 8

Overview of the keywords related to the engineered aspects mentioned in the primary studies

We also provide a more abstract characterization of the primary studies based on three cross-cutting aspects emerged from our analysis. The three engineered aspects of MRSs that we encountered in the analysed papers are: structure of the MRS, behaviour of the MRS, and navigation system. Since this study is targeting mobile robotic systems, we highlight navigation, which could be considered a particular type of behaviour. Behaviour includes sensing, actuation, and multi-robot interaction. It is quite surprising that only 3 approaches (P13, P16, P76) provide full coverage of these aspects in their proposed solutions. As surprising is the fact that more than 10% of the analysed papers only focus on modelling structural information of MRSs. Overall, behavioural details are the ones that are modelled the most, both for automatic generation of production code and for model-based analysis (Figs. 9, 10).

Fig. 9
figure 9

Cross-cutting aspects of the primary studies

4.3.4 Supported automation

We encountered four different types of automated mechanisms supported by MDE approaches: model transformation, code generation, model-based analysis, and models@runtime.Footnote 9

Model transformation and code generation are, as we expected, the most common automation mechanisms carried out from models; only in 3 cases, neither of the two is provided. In 11 cases, it is not specified whether code generation is achieved via transformations, while in 25 cases transformations are employed for other purposes than code generation (14 for model-based analysis and 9 for models@runtime). Only 1 approach (P20) exploits models@runtime in combination with model-based analysis.

Fig. 10
figure 10

Automated MDE mechanisms for MRS

4.4 Validation of MDE approaches for MRSs (RQ4)

The objective of RQ4 is to provide a map about (1) how MDE approaches for MMRSs have been validated (RQ4.1–Sect. 4.4.1) and (ii) the types of robots employed during the validation of each approach (RQ4.2—Sect. 4.4.2).

4.4.1 Types of validation (RQ4.1)

The intuition behind RQ4.1 is that approaches that have been (rigorously) validated in industrial/realistic settings are more likely to be adopted in the industry [84]. For example, a validation carried out only by showing an example of applying proposed approach (e.g. P6) has a much lower degree of realism with respect to a validation performed using real robots deployed in the field (e.g. P76).

Fig. 11
figure 11

Types of performed validation

As shown in Fig. 11, a large portion of primary studies relies on the least rigours and non-systematic type of validation, i.e. just showing an example of the application of the proposed approach (38/80). This result might be exploited by our community in order to raise discussion on how to carrying out research. On one side, it is important for consolidated research results to provide rigorous, replicable, and realistic evidence about the performance of the proposed approach. This would probably increase the likeness to have impact either in academia or industry. On the other side, it is important to consider that early concept papers might not be able to provide a mature validation. It is important to do not rule out these types of works, since they are an important part of the exchange of ideas in science. For what concerns consolidated research results, it would be probably important to strive towards performing well-designed empirical experiments on their proposed approaches, preferably in realistic scenarios and with real robots. This result indicates the need of providing more rigorous, replicable, and realistic evidence for consolidated research results to have a stronger impact both in academia and industry. Nonetheless, it is important to still consider the usefulness of concept and visionary papers, for which providing a mature validation is not feasible. These works are in fact an important part means for exchange of ideas in science, also for practitioners.

Some research groups are already moving towards this direction, with 23 primary studies reporting about empirical experiments carried out on the MRS (23/80), with the deployment of the MRS in the context of a real project (12/80) or even by applying the case study research methodology (2/80) [85]. As an example of realistic validation, the authors of P5 report about their experience in applying their proposed approach on a youBot mobile manipulation robot participating in the RoboCup@Work competition [86].

Also, ten primary studies report about a simulation-based validation of their proposed approach. In those cases, the following simulators are used: GazeboFootnote 10 (P7, P34), MATLABFootnote 11 (P13, P59, P71), SimSpark [87] (P30), Player/Stage [88] (P39), JSBSim [89] (P11), and custom simulators (P15, P73).

4.4.2 Types of MSRs used as validation subjects (RQ4.2)

While RQ2 focuses on the characteristics of the robots supported by the approach itself (e.g. the approach presented in P59 works only for swarms of aerial robots), here we focus on the type of robots actually employed while validating the proposed approach.

Terrestrial robots (59/80) are the most often used types of robots when validating MDE approaches for MRSs (see Fig. 12). This result is expected since terrestrial robots are the ones with the most controllable dynamics. Also, the usage of terrestrial robots is facilitated by the fact that many types of research and educational terrestrial robots exist in the market, such as the Pioneer 3-DX platformFootnote 12 (e.g. used in P26), the iRobot CreateFootnote 13 (e.g. used in P2), Lego MindstormsFootnote 14 (e.g. used in P78), and similar robots. Nonetheless, in many primary studies, a full robotic prototype has been designed and realized from scratch (i.e. from hardware design up to the application level), such as in P41 (a service robot), P15 (a rover), and P33 (self-driving car).

Fig. 12
figure 12

Types of robots used as subject for validation

Aerial robots have been used in 11 primary studies, both in single-robot missions (e.g. P14) and swarm-oriented ones (e.g. P59). Differently from terrestrial robots, among those 11 studies, we can mostly find custom-built vehicles, like in P24 (fixed-wing aircraft) and P40 (quadrotor with Arduino-based controller). The effort for designing, developing, and properly testing autonomous flying vehicles is significant. Therefore, it would be desirable to have a shared hardware-software platform for aerial robots, instead of having different research groups building their own ones. This would open many opportunities for scaling up our research, including:

  • reusing already built and well-maintained flying vehicles,

  • having a common (possibly benchmarked) hardware platform across different research groups so to easily compare independently-achieved research results,

  • reusing different (open-source) software and hardware modules across research projects (e.g. the PX4 autopilot firmware),Footnote 15 and

  • collaborating with companies already using the same platform for their commercial products, thus potentially leading to a smoother technological transfer of the obtained research results.

In 7 cases, validation has been performed on generic mobile robots. In those cases, the validation of the approach has been performed on synthetically built examples focusing on robotics in general, without specifying the specific type of robot and with a focus on specific aspects of the system (e.g. navigation, planning). If on the one side, we understand that for some studies having a validation on a generic mobile robot is practical (e.g. in case of very preliminary results), then on the other side, we invite researchers to spend effort in the validation of their approaches towards using realistic robotic systems. More realism helps in terms of adoptability of the proposed approach and provides more evidence about the characteristics of the approach, especially when dealing with run-time properties such as performance, energy consumption, and safety.

Fig. 13
figure 13

Cardinality of robots used as subject for validation

Table 6 Language workbenches used to build DSLs for MRS

About the cardinality of the robots used as subjects for validation (Fig. 13), we notice a strong imbalance in favour of single robots (64/80) at the expense of multiple robots (16/80). This result is partially due to the fact that the majority of primary studies are dedicated to single-robot systems (see Sect. 4.2). However, there are 12 primary studies whose proposed approach is applicable to multi-robot systems, but their validation is focusing on a single robot (e.g. P1, P12, P20). After a closer inspection of all those studies, all of them focus on a specific aspect of a MRS (e.g. how software components communicate—P12), independently of the number of robots involved in the mission (e.g. in P12 software components can be deployed flexibly on both single and multiple robots).

4.5 Tool support (RQ5)

There is a plethora of tools used for various aspects of MDE of MRSs. Interestingly, some of the approaches focus on the language workbench used to develop a specific DSL for MDE of MRSs. Among them, the eclipse modelling framework [90] (EMF), for defining metamodel-based DSLs, is the most popular workbench, followed by Xtext [91] and MontiCore [92], both for textual DSLs. Besides language workbenches (see Table 6), the analysed approaches rely on tools of all kinds, from model transformation suites (e.g. Acceleo)Footnote 16 and execution semantics for metamodels (e.g. Kermeta3)Footnote 17 to specialized IDEs (e.g. RoboStudio [93]), web-based mission planners (e.g. FLYAQ [10]) and analysis tools (e.g. RTEdge [94]). Among the mentioned supporting tools (74 in total), 29 are based on the eclipse ecosystem.

5 Orthogonal analysis

The orthogonal analysis investigates possible correlations between related concepts of the classification framework. To this end, we cross-tabulated and grouped the data related to these concepts, compared the data relating to the different pairs of concepts, and discussed findings of interest. This section describes these findings. Moreover, we investigated correlations between other pairs of concepts not discussed below. We found these to confirm the trends of the vertical analysis. However, all observations are available from the replication package, which allows further investigation of the data.

5.1 Modelling languages and research contribution type

In our corpus, DSLs, variants of UML, and other, nondescript, modelling techniques are used as objects in different types of research contributions, including novel methods, metrics, models, open items, and tools. By relating both facets, we aim to understand whether DSLs or UML-based modelling languages are specific to a certain research type. To this end, we compare both dimensions as depicted in 14. As some publications employ multiple modelling techniques for one or more contribution types, the sum of numbers in a column or in a row can exceed the number of publications of our corpus. For instance, the research presented in study P93 contributes novel abstractions based on UML, a method for using these, and a metric to evaluate their use. Consequently, it adds three individual entries to our cross-tabulation as presented in 14. Overall, the research addressed by publications included in our corpus contributes 186 of such entries.

Overall DSLs make up 121 of these entries, whereas UML variants only make up 65. Hence, DSL-based publications contribute to 2.6 research types on average and publications reporting research with UML variants contribute to 2.2 research types on average. DSLs are used to contribute methods and models most often. In contrast, UML-based languages are used to contribute methods and tools most often (Fig. 14).

Fig. 14
figure 14

Modelling language kinds relative to research contribution types

From our findings, the focus on DSLs for model type research might be due to the heterogeneity of the field which demands more different modelling techniques than the purely functional and discrete UML diagrams. And while specifying such techniques generally is possible via UML profiles, the appearance of powerful language workbenches might have raised the number of contributions providing stand-alone domain-specific languages for specific challenges in mobile robotics. In the future, we expect to see more publications contributing specific modelling languages to mobile robotics, which would be in line with a general trend towards DSLs in robotics [95].

This might entail a diversification of robotics DSLs into DSLs for specific kinds of systems in the context of specific application scenarios (e.g. DSLs for search-and-rescue missions with aerial robots or DSLs for modelling logistics operations for terrestrial robots). We expect that most of these will be realized as external DSLs built on top of powerful language workbenches (cf. Table 6).

5.2 Multi-robot support and supported robot types

As presented in Fig. 5, the majority of research on MDE for MRS aims to be generic with respect to the supported robot types 51 out of 80 publications). This also holds for major European research projects in this discipline, which, e.g. aim for establishing best practices for modelling different robotics concerns in a generic fashionFootnote 18 or for enabling modular model-driven development in robotics in general.Footnote 19

When considering approaches specific to distinct types of robots, we found that research most often focuses on single generic mobile robots (reported 37x), single terrestrial robots (19x), and generic multi-robot systems (14x). Relatively, the small number of approaches supporting terrestrial multi-robot systems is surprising (Fig. 15).

Fig. 15
figure 15

Multi-robot support relative to supported robot types

While these findings being are too small to assert significance, the relatively small number of terrestrial multi-robot systems might indicate that the approaches towards MDE for generic MRS demand further evaluation with specific robot types or that the generic multi-robot approaches are good enough to subsume multi-robot research for specific robot kinds.

5.3 Modelling language kinds and engineered aspects

To better understand how modelling languages are applied to the different engineered aspects, we cross-tabulated these relative to each other as well. As some publications addressed multiple aspects with multiple kinds of modelling languages, the overall number of 86 findings is slightly higher than the number of publications in our corpus.

Our observations show that DSLs and UML are used differently to address the engineered aspects of MRS. Where aspects were mentioned, they were addressed by DSLs more often (reported 62timesx) than by UML (34x).

Contributions leveraging DSLs focus more often on structure and behaviour (together 54/62), whereas navigation is significantly less often addressed (7/62) by DSLs. In contrast, contributions leveraging UML relatively more often focus on navigation (7/34) (Figs. 16, 17).

Fig. 16
figure 16

Engineered aspect relative to employed modelling language kinds

Our data cannot explain why this is the case. It could be that navigation is more of a cross-cutting concern that addresses both structure and behaviour, for which the combination of respective UML diagrams might be better suitable that creating a novel DSL that captures both from scratch.

With the large majority (64%) of DSL-related contributions being developed starting 2014, we expect more contributions on DSLs for MRS navigation in the future.

5.4 Engineered aspects and automation support

To uncover how the supported engineered aspects related to the provided automation support, we also cross-tabulated these relative to each other. Again, publications can produce multiple entries to this cross-tabulation, e.g. where code generation is applied to behaviour and structure, two entries are counted.

We found that the engineering aspects of MRS also relate differently to the supported automation techniques. Overall, the publications our corpus report on leveraging various automation techniques 184timesx to address behaviour, navigation, or structure. According to these reports, code generation (reported 86timesx) and model transformation (72x) are supported most often, whereas models at runtime (20x) and model analyses (6x) are supported significantly less often.

Fig. 17
figure 17

Engineered aspects relative to the provided automation support

As implied by the use of DSLs and UML to address these engineering aspects, also the automation support for behaviour (84x) and structure (71x) is more prevalent than for navigation (29x). Where behavioural, navigational, and structural modelling contributions are similarly often supported by code generation and model transformation, navigation rarely is relatively more supported by models at runtime.

The lack of models at runtime support for navigation is surprising as navigation demands for some kind of modelling (robot, environment) at runtime. This might indicate that much of navigation still is implemented with traditional programming languages. Overall, MDE for MRS strongly focuses on producing solutions via code generation of model transformation.

We assume that this might be due to the large amounts of infrastructure code that even a simple robotics application needs.

The increasing popularity of a few robotics software platforms (such as ROS [25] or SmartSoft (P10)) that take care of infrastructure challenges, might liberate software engineering research capabilities to focus on the added value of analyses of MRS structure, behaviour, or other aspects.

6 Discussion

We conducted this systematic literature study to gain insights on MDE for mobile robotic systems. To this end, we formulated five research questions (see Sect. 3). This section reports our observations on the main findings,

which are described in Sect. 4, and orthogonal analysis, which is described in Sect. 5.

Regarding the publication trends (RQ1), we found that most contributions to MDE for MRS are published at conferences, followed by journals, and workshops. This is probably in line with any software engineering publication. Most publications occur at specialist venues dedicated to the intersection of robotics engineering and software engineering, such as SIMPAR, IROS, or IRC (cf. Table 2). This focus is confirmed by the little number of publications appeared in venues purely dedicated to robotics (e.g. the International Conference on Robotics and Automation (ICRA)), model-driven software engineering (e.g. International Conference on Model-Driven Engineering Languages and Systems (MODELS)), or software engineering in general (e.g. International Conference on Software Engineering, (ICSE)). However, we can observe an increasing interest in the software engineering community, and we expect that in the near future in the program of software engineering conferences like ICSE there will be dedicated sessions to robotics.

In fact, many of the challenges identified in the corpus (e.g. modelling of structure, behaviour, navigation) are general enough to warrant publication and presentation at less focused venues, including conferences such as ICRA, MODELS, ICSE, ESEC/FSE, and journals such as Journal on Software and Systems Modelling (SoSyM), IEEE Transactions on Software Engineering (TSE), ACM Transactions on Software Engineering and Methodologies (ToSEM), Automated Software Engineering (ASE), Journal of Systems and Software (JSS), International Journal of Robotics Research (IJRR), IEEE Transactions on Robotics (TOR), or Journal of Field Robotics (JFR). The organization of workshops, such as the workshop on Robotics Software Engineering (RoSE) or the workshop on Model-Driven Robotics Software Engineering (MORSE), at the non-specialist top conferences (ICSE, ICRA, ASE, ESEC/FSE) or the organization of special issues in the related journals are also contributing to maximize outreach, dissemination, and exposure of the community’s research.

Research in MDE for MRS largely focuses on solution-seeking research [96], contributing methods and tools in the form of solution and validation research. This is a common trend shown in many mapping studies in the field [1, 46, 97,98,99]. However, the relatively large number of publications contributing tools (cf. Table 4), appears to be remarkable.

We can probably find a reason for that in the nature of research on MDE for MRS. In fact, this type of research focuses on answering research questions centred around robotic systems and in producing results that are relevant to the community. Therefore, there is a need for tools that automate the proposed methodologies and make it usable in practice. The next step will be on making these tools actually usable in everyday life and transferable into industrial practice, for instance providing guarantees for safety aspects [1].

On the supported types of considered robots (RQ2), we found a strong focus on research about single generic terrestrial robots. This is not surprising, since single generic terrestrial robots require less maintenance and operation effort with respect to other types of robots. For many research challenges in MDE for MRS, e.g. cognitive robot behaviour, modelling and generating robotic software architectures, implementing robust navigation, multi-robot systems, as well as aerial systems or aquatic systems, introduce additional challenges that might, if not in the focus, prevent advancing the original research challenges.

Despite focusing on single robot systems, a significant number of the studies of our corpus also addresses MDE for multi-robot systems. This might be due to the improved accessibility of individual platforms (e.g. usable drones have become fairly inexpensive) since this enables novel methods to evaluate systems of collaborative or competitive robotics in a wide range of scenarios, such as e.g. in smart factories or for disaster relief.

Moreover, the majority of contributions focuses on MDE for generic mobile robot types (e.g. modelling of robot capabilities (P16)), which generally should support transferring these results to other related domains, such as smart manufacturing systems (which also need elaborate behaviours, structures, as well as object recognition and reasoning capabilities). We believe that this transfer can yield valuable insights into the concurrent challenges specific to MDE for MRS and could motivate future research in this direction.

We want to highlight the need for better validation to clearly identify the limits, scope, and qualities of the proposed solutions [1, 100]. It can happen that contributions to

single-robot systems might scale to multiple-robot systems, or that generic solutions might actually be less generic than expected.

We believe that this is an important step for having a large adoption in industrial practice.

Concerning the applied MDE methods and techniques (RQ3), we found that many publications contribute DSLs, either defined externally or as UML profiles (e.g. RobotML, SmartSoft)—to address challenges regarding behaviour, structure, and navigation. These also include UML profiles not tailored directly to robotics (such as MARTE or MechatronicUML). While this suggests that (tailored) UML generally can be useful to facilitate engineering of MRS, we found that contributions involving

DSLs significantly more often (50 contributions) present new models to describe solutions than contributions involving UML (13 contributions),

whereas contributions involving UML more often focus on new methods (54 contributions) than contributions involving DSLs (34 contributions).

As new models usually contribute new ways of conceptualizing things, whereas methods focus on doing things, this might entail that for innovative robotic aspects, DSLs suit better than UML and its profiles.

Also, this would explain why there are significantly more contributions leveraging explicit DSLs than UML. This seems to be a specific characteristic of this domain, which differ, for instance, from for instance, does not

MDE in smart manufacturing [46]. We, therefore, suggest intensifying research on DSLs for specific MRS challenges but advocate keeping the interconnection of all three MRS engineering challenges (behaviour, navigation, structure) in mind when developing such languages to enable their integration with DSLs for interconnected purposes.

Moreover, we observe a strong focus on addressing behavioural and structural challenges instead of navigational challenges. This might be caused by the fact that navigational challenges are relatively uniform across different types of robotic systems, e.g. there need to be sensors and actuators connected to other components realizing the high-level behaviour and most platforms need to take care of local and global navigation issues using laser-based or camera-based orientation.

We also observed a strong focus on constructive automation techniques, i.e. code generation and model transformations over analyses or models at runtime. This is in line with our observation on the constructive nature (solution-seeking) of MRS research (and, hence, the relative abundance of tool contributions). From our experience, one of the major challenges in robotics are configuration and orchestration of the vast numbers of components that need to properly interact to successfully deploy even relatively simple MRS. Not leveraging the main strengths of models, which are in communication and analysis [31] through automated analyses supporting this, hence, is surprising. However, with this increasing wealth of models and modelling techniques becoming available, we suggest that future research also focuses on analyses for, e.g. the reliable or explainable behaviour of robotic systems that support the adoption of MRS outside of the laboratory environments they often are confined to.

Research on MDE for MRS is mostly validated (RQ4) by example using single terrestrial robots. This often has the form of non-systematic validation with little rigour, e.g. by giving examples for its application.

We assume this is due to the complexity of performing rigorous experiments that systematically cover (or rule out) all potentially relevant aspects of the system under investigation and its environment. This implies a need of modelling solutions able to grasp and represent various engineering aspects and stakeholders’ concerns, including hardware, mechanics, software, and even the environment, to facilitate simulations and the definition of rigorous experiments.

The choice of terrestrial, single robot, evaluation platforms follows the trends on the kinds of MRS addressed and, hence, is not surprising.

Indeed, research on generic mobile robots should be evaluated with different kinds of MRS. This application of generic solutions to specific robot types might provide insights into limitations not considered when conceiving generic modelling techniques (as often, “the devil is in the details”). In particular, aerial robots have become inexpensive enough to consider as alternatives to terrestrial robots for validation. Indeed, the rigorous experimental evaluation with aquatic and space robots requires to access (realistic) operation environments (e.g. oceans, lakes, space); in this sense, terrestrial robots can be operated more easily in realistic environments.

The main pillars of MDE are abstraction and automation. Consequently, we expect MDE for MRS to provide tools build upon existing automation tools

(RQ5). DSLs are quintessential to MDE leveraging automation. This is in line with many studies of our corpus providing novel models or methods using DSLs. Many of these studies leverage sophisticated language engineering tools, such as EMF [90], MontiCore [92], and Xtext [91], which appear to be the most prominent language workbenches to support DSL engineering for MRS.

Moreover, we have found many studies leveraging MDE by using models (e.g. to describe robot structure, behaviour, or navigation) instead of developing novel DSLs. For the use of models, we have found specific IDEs ranging from full-fledged IDEs, such as RobotStudio [93], to web-based mission planners, such as FLYAQ [10], to very specific analysis tools, such as RTEdge [94].

Generally, aside from language workbenches, however, and few DSLs (such as SmartSoft or RobotML) there seems to be little tool reuse in MDE for MRS. Our data does not indicate whether this is due to the heterogeneity of research on MDE for MRS, although the focus on terrestrial single robot systems appears to contradict this assumption, or due to the complexity of the different technological spaces that these more established tools reside in. Moreover, this might be due to established robotics middleware (such as ROS) not being model-driven but instead focusing on traditional programming. Consequently, when reusing such a middleware, there is little “MDE tool” reuse. To uncover why model-driven middleware solutions are used less often, we suggest further research targeting the replication and evaluation of approaches within their, to be explicated, validation frames.

7 Threats to validity

In the following, we discuss the potential threats to the validity of this study as well as the activities we carried out to mitigate them.

7.1 External validity

External validity refers to the generalizability of our results [64].

The generalizability of our results strongly depends on the extent to which the selected primary studies are representative of the research landscape about MDE approaches for mobile robotic systems. This potential threat to validity is mitigated by targeting multiple data sources, i.e. ACM Digital Library, IEEE Xplore, Scopus, Science Direct, and Web of Science. The rationale for having multiple data sources is that their provided results complement each other in terms of coverage of the state of the art. The used data sources cover the area of software engineering well [101, 102] and are five of the largest and most complete scientific databases and indexing systems in software engineering. Moreover, we further complement the results of the automatic searches by performing a closed recursive backward and forward snowballing activity.

Moreover, only studies published in the English language have been selected in our selection process (see I5 inclusion criterion in Sect. 3.2). This decision may result in a possible threat to validity because potentially important primary studies published in other languages have not been selected in our research. However, the English language is the most widely used language for scientific papers, so this bias can be reasonably considered as minimal.

In this study, we exclusively target peer-reviewed studies (e.g. journal or conference papers), leaving out the grey literature (e.g. white papers , technical reports, blog posts). This potential threat to validity is intrinsic to our study design because we target high-quality scientific papers, and thus undergoing a rigorous peer-reviewed publication process is an accepted requirement for this kind of scientific works. The same line of reasoning applies for our E2 exclusion criterion, dealing with the exclusion of tutorial papers, short papers, poster papers, editorials, etc., since they do not contain enough information for answering our research questions.

7.2 Internal validity

Internal validity refers to the level of influence that extraneous variables may have on the design of the study [64].

In this case, having a rigorously defined protocol with a rigorous data extraction form helped in mitigating biases related to the internal validity of our research. Also, for what concerns the data analysis validity, the threats are minimal since we employed only descriptive statistics when dealing with quantitative data. When considering qualitative data, we systematically applied the keywording method for transforming qualitative data into quantitative data. Finally, ten primary studies have been randomly selected and two researchers checked whether the results were consistent, independently from the researcher performing the extraction; moreover, each disagreement has been discussed and resolved, together with a third researcher, when needed.

Moreover, we formalized the design of this study into a detailed research protocol beforehand and thoroughly discussed it prior to the actual execution of the study. Our research protocol (and thus the whole study) is based on well-accepted guidelines for systematic literature reviews and mapping studies  [19, 64, 103].

To allow independent replication and verification of our study, a complete replication package is publicly available. The replication package includes the review protocol, the raw data of our search and selection phase, the complete list of selected primary studies, the raw data extracted from each primary study, and the R scripts we developed for data exploration and analysis.

7.3 Construct validity

Construct validity refers to the selection of the primary studies with respect to the research questions we aim to answer [64].

As already discussed in Sect. 7.1, the initial automated search has been performed on six different data sources and complemented with the snowballing procedure. In this context, one potential threat to validity may be caused by a malformed search string. We mitigated this potential threat to validity by (1) following a rigorous process for defining it (see Sect. 3.2) and (2) piloting the search string in preliminary queries using IEEE Xplore.

After having collected all relevant studies from the automatic search, we rigorously screened them in a collaborative fashion and according to well-documented inclusion and exclusion criteria (see Sect. 3.2).

7.4 Conclusion validity

Conclusion validity refers to the relationship between the extracted data and the obtained results [64].

First of all, third-party researchers can verify and check the obtained results by replicating our study independently from our results. This is possible thanks to the detailed research protocol and the complete replication package, which are both available publicly.

Another potential threat to conclusion validity may lie in the definition of the classification framework. Indeed, other researchers may identify classification frameworks with different facets and parameters, thus potentially leading to totally different perspectives on MDE for MRSs and different results. We mitigated this potential threat to validity by (1) carefully documenting the process we followed for building the classification framework, (2) having the data extraction process conducted by multiple researchers in collaboration, and (3) making the raw extracted data available for independent verification.

8 Related work

The interest on software engineering for robotic systems is clearly growing and contributions range from discussing the general trends in software engineering for robotics [104], over benefits of MDE for specific platforms [105], to developing test-benches for MDE approaches [106] to contributing MDE-based reuse processes [107] or specific software engineering techniques [108] to robotics. The works by Brugali et al. [14, 21, 109] review the use of Component-based Software Engineering (CBSE) and MDE in robotics. A state of the art of service robotic engineering might be found in [15]. The study provides a comprehensive picture of (1) the practices applied by robotics industrial and academic practitioners, including processes, paradigms, languages, tools, frameworks, and reuse practices, (2) the distinguishing characteristics of robotics software engineering, and (3) recurrent challenges usually faced, together with adopted solutions. One of the highlighted challenges of the robotic domain is the lack of reusable solutions for recurrent problems in application development. In practice, integration is typically performed in an ad-hoc way, while being complicated by the wide range of disciplines practitioners working to develop a full-working robotic system have.

Also, the body of literature on systematic studies on robotics in relation to programming and developing software for it is rich and ranges from mapping studies on software architectures for robotic systems [110] to systematic literature reviews on teaching robot programming to novices [111]. Pons et al. [112] carried out a, fairly shallow, systematic review on the application of existing software engineering techniques to robotics. Feitosa et al. [113] reported on a mapping study on embedded software engineering for mobile robots. While they were seeking potential application of a set of known software engineering techniques to robotics, in this paper, we report on how a specific branch of software engineering, MDE, is used for mobile robots. Others focus on other relevant aspects of robotics, such as safety for mobile robots [1], safety certification [114], and the state of DSLs for robotics [83].

The literature study on the state-of-the-art of DSLs in robotics [83] investigates MDE for robotics through the lens of the RoboCup@Work competition. The authors classify 41 publications and focus on specific robotics subdomains related, such as structure modelling, planning, manipulation, or coordination. As such, the study has a limited scope of nine venues specific to modelling in robotics and focuses on DSLs, i.e. does not query the selected databases for the use of UML in this context. In the study, the authors find an increasing number of DSL contributions to robotics starting in 2010, which is in line with our finding that the youngest UML profile in the field was introduced 15 years ago.

There also are research results on MDE for robotics in other studies. For instance, in a study on model-driven techniques in automation, Yang et al. [115], identify the OpenRTM robotics middleware [116] being used in this context. While being interesting, the survey does not investigate the platforms, automation techniques, or further MDE-related questions in the context of robotics.

A systematic mapping study on software architectures for robotic systems is presented by Ahmad et al. [117]. In this survey over 56 publications, the authors investigate architecture modelling techniques, applied modelling languages, architectural concerns, and application domain these architecture modelling techniques are applied to. The authors find that UML component diagrams are the most popular modelling language for robotics architectures but do neither investigate which automation support is used, nor which tools are employed.

A study on formal specification and verification of the behaviour of autonomous robotic systems and related tools analyses 63 publications found with Google Scholar [118]. The study does not focus on MDE but identifies the formalisms applied to the verification of robotic systems (e.g. temporal logic or state-transition descriptions) and presents the model checking tools applied to robotics.

Hence, to the best of our knowledge, a systematic study on the broad use of MDE for mobile robots has not been carried out before.

9 Conclusions

This paper presents the design, conduction, and results of a systematic mapping study on model-driven engineering techniques for mobile robotic systems. Specifically, by following well-established guidelines on systematic literature reviews and mapping studies [19, 61, 103], we identified an initial set of 2809 potentially relevant studies and then we performed a rigorous selection process, eventually leading to a final set of 97 primary studies. We analysed each of the 97 primary studies and extracted relevant data for each of them in order to provide a map of MDE approaches for MRSs. The data extraction phase was supported by an incrementally-defined classification framework, which can also be used by future researchers either for putting an already-existing approach into context or to understand scientifically interesting contribution they may make with their own approaches.

The main results of the study are summarised in the following (and expanded throughout the paper):

  • Researchers tend to publish in specialized scientific venues at the intersection of software engineering and robotics, even though many approaches are generally applicable.

  • Researchers largely focus on solution proposals and contribute novel MDE methods and tools for mobile robotic systems.

  • There is a strong scientific interest on single general-purpose terrestrial robots.

  • The number of MDE approaches for multiple robots is growing.

  • There is a strong need for better validating MDE approaches for MRSs in order to sharply identify their limits, scope, applicability, and characteristics.

  • The used modelling languages are more domain-specific than generic (e.g. UML-based) with a focus on both structural and behavioural aspects of the modelled robotic systems.

  • We observed a strong focus on code generation and model transformations over model-based analysis or the usage of models at runtime.

  • Researchers tend to validate MDE approaches for MRSs by applying them to simple examples, often involving single terrestrial robots.

  • Sound and rigorous empirical experiments are not yet widely carried out, potentially hampering the transferability of the proposed approaches into real industrial contexts.

  • The most used language engineering tools are EMF, MontiCore, and Xtext. However, we observed very little MDE tools reuse for MRSs.

By presenting and discussing the above-mentioned results, this study provides a comprehensive and replicable map of the state of the art about MDE approaches for MRSs. The results of this study form a solid foundation for future research in the area and will (hopefully) lead to the development of new (and better) MDE methods and techniques for the mobile robotic systems of the future.