In the following we will present our results with respect to the three research questions. Whenever we use quotes, we identify the interviewee and the area according to Table 2.
RQ1: How and why are models currently used in automotive RE?
We break down our findings regarding the current use of models in RE (RQ1) into the nature of existing models in RE (summarised in Table 3) and the purpose of these models (summarised in Table 4).
Nature of models in RE
We find numerous different models that are existing today, reflecting the heterogeneous nature of large automotive projects and the variety of our two case companies and three application areas. An overview over the main themes we extracted from the interview data is given in Table 3.
Supporting the findings of many publications in the area of embedded systems, e.g. [24, 25, 50], multiple interviewees from both companies stated that textual requirements are the norm (T1.1.1) in most areas and that only few models exist. On a high level of abstraction, and outside of software development, these textual requirements are typically stored in Word or Powerpoint documents. On lower levels of abstraction, closer to architecture and software development, specific tools are used that allow for advanced features, such as versioning or traceability.
While the requirements themselves might be textual, several interviewees stated that the entire specification is an emerging structural model (T1.1.2) due to existing trace links. That is, the model is not actively modelled by someone, but emerges through creation of traces. In the case of Company A, a systems engineering tool is used that supports different element types, e.g. requirements and logical design components, and relationships between them, e.g. a requirement is fulfilled by a design component, in the form of a customisable meta-model. Similarly, in Company B, there was an ongoing initiative at the time of the interviews to move to model-based documentation in the form of SysML. In SysML, requirements are a specific type that contains a textual description, i.e. while the requirements themselves remain textual, they can be linked via defined relationships.
As a step towards formalisation, interviewees mentioned the use of templates (T1.1.3), keywords, or sentence patterns in order to describe requirements. However, in most cases the interviewees stated that these templates are either not enforced or had already been abandoned. The reason for abandoning these templates were often stated to be that their meaning deteriorated over time or that they were simply not followed.
[..] there were a couple of terms specified that were not followed and, due to that, (it) has broken down in what it actually means, [..] it has broken down in principle.—A7, EmbSE
In both companies, interviewees mentioned that sketches of behaviour are common (T1.1.4). For example, one interviewee at Company A stated:
And sometimes when we have pictures, it’s mostly like state diagrams and so on. So you say ’OK, A, B, C, and what are the transitions between that?’.—A4 EmbSE
Similarly, several interviewees at Company B also stated that sketches of state machines or flow diagrams are not uncommon. In particular, two interviewees from SysE stated that diagrams of the control flow or other behaviour diagrams close to implementation are widespread.
Formal structural models describing the system (T1.1.5) are used in both companies, e.g. to define the logical architecture with software and hardware components, and their connections. While these models are primarily architecture models, they are often connected to RE, e.g. as a basis for structuring the requirements specification or as building blocks that are used to define behaviour requirements.
Finally, multiple interviewees in both companies stated that some formal models describing requirements exist, e.g. in the form of UML models. However, these are typically used by experts or proponents of modelling only.
But it (the model) is not fully used. This is here the problem. Some use it, some gurus.—B4, AppSE
An exception is SysE, in which formal diagrams are commonly drawn for safety-critical applications.
Beyond the current practice, there are ongoing attempts to introduce formal models for RE at both companies. At Company A, there are initiatives to entirely abolish textual specifications on lower levels of requirements abstraction in favour of Simulink. At Company B, there are piloting efforts to introduce model-based systems engineering using SysML in parts of the company.
Purpose of models in RE
Similar to the variety of different models existing in the two case companies, our interviewees named multiple different purposes to use models during RE. An overview over the main themes we extracted from the interview data is given in Table 4.
At Company A, requirements models were named by five different people as a support to verification activities (T1.2.1) later on, e.g. in terms of sketches of behaviour.
[..] it’s used for test and verification. I mean those pictures, it’s not just for writing down a specification and have it for later use when we do a new model, it’s used later on also to verify that we have gotten the right things.—A5, EmbSE
For semi-formal and informal notations, interviewees mentioned models as a tool to support communication (T1.2.2).
Our focus is clearly on cross-domain communication. Definitely.—B2, SysE
Additionally, several interviewees brought up formalisation as a tool to guide and streamline RE (T1.2.3). For example, in Company B one interviewee mentioned a defined vocabulary and process as a way to guide engineers.
[..] it helps [..] that you concentrate on ’What is the function I want to fulfil? What is the behaviour [..]?’ And that contributes to a lot of thoughts that you would never have if you would simply write it down in a Word document.—B1, SysE
Finally, several interviewees mentioned that different kinds of models are currently used as a way to handle increasing complexity (T1.2.4). This is especially true for the structural models close to architecture, e.g. interface and signal descriptions that can be used to generate skeleton code or documentation. However, also tracing was mentioned several times as a means to understand changes, support product variants, or support verification activities. On higher levels of abstraction, use case models were brought up specifically in Company B as a way to understand the existing system functionality and allow for easier impact analysis when new feature or change requests are incoming from customers.
RQ2: What is the perceived potential of models in automotive RE?
We identified five themes with respect to the use of models in automotive RE in our data. These are depicted in Table 5. Several of these overlap with the current use of models at the case companies. In the following, we describe these themes in detail.
Note that, in our data, when we asked about the potential of models in RE, interviewees referred to current use, past use, and potential future use of models. Therefore, by perceived potential of models, we refer to all three cases.
Formal analysis and execution (T2.1)
In theory, MBE enables practitioners to increase quality of the resulting system by automating repetitive tasks, e.g. through code generation, or by running a number of analyses on existing models, e.g. through model checking.
Interestingly, only three interviewees in Company B and none in Company A raised formal analysis of requirements models as a potential use case. One interviewee mentioned that code generation should be used to generate skeleton code of interface models (see Sect. 5.2.5). Two interviewees in Company B stated that it would be beneficial to have use case models in order to automatically check for existing behaviour.
What does the product contain already? What do we need to develop newly? Here it would be cool if we had the existing products as a model.—B3, AppSE
In Company A, the importance of models for early validation with customers and other stakeholders was raised repeatedly. Three interviewees stated that they would like to use executable models to simulate parts of the system behaviour. This could then be used to demonstrate early concepts or to establish a common understanding in a group of engineers or stakeholders.
I think information management is key. [..] I do believe in using models for testing. To have things that you can show people, that you can work with. That you can already try the specification or the model.—A2, EmbSE
One interviewee expressed the belief that the company should go directly from high-level textual requirements to executable models.
I think we should have (high-level requirements) in whatever Word document that is needed and then we go directly to a model where we can execute the model and simulate the behaviour and the functionality.—A3, EmbSE
One of the reasons for having executable requirements models is, according to another interviewee at Company A, that faults can easily be injected and the resulting behaviour assessed.
In contrast to formal analyses of models, all but one interviewee in Company B stated that they would like to use requirements models for communication purposes only, especially among people with different backgrounds.
On cross-discipline level. [..] There I think it makes most sense, as more and more experts and disciplines emerge [..], but the connections, that’s the point.—B2, SysE
While models that are used for communication could possibly also be reused for formal analyses, one interviewee stressed the importance of informal models for communication.
Well, models. When you say you work with pictures, with sketches, graphics, that definitely helps. But flow diagrams, when you are that far then usually everybody understood it already. The problems come before that.—B5, AppSE
Interestingly, none of the interviewees in Company A pointed out models as a means to communicate only.
Manage system evolution (T2.3)
Evolution was mentioned several times by interviewees in both case companies. As both companies build products that are maintained for a long period of time and that often build on already existing products, evolution is an important aspect in their daily work.
One interviewee in each company suggested that requirements models could be useful for regression testing. This could be in the form of executable models similar to Behaviour Driven Development . That is, requirements would be expressed in terms of executable models that contain acceptance criteria or similar information that could be used for testing later on.
Similar to understanding the impact of new feature requests, as discussed previously, several interviewees stated that it could be useful to have a structural model of the system’s interfaces for change impact analysis. In fact, one interviewee in AppSE stated that they used to have a model of all the use cases in the past. While this model had been discarded in the new agile process, the interviewee expressed the need for a similar solution.
[..] there is the work in progress and there is the product as it is. [..] we created a functional model of natural language use cases. This means the product is structured into its main functions [..], a tree with use cases in the leaves. And this represents the current state of the product. Which use cases are there? And I have to maintain that, because the work in progress [..] extends the tree, it changes the tree. But if there is no tree [..], then I don’t know how you do that.—B3, AppSE
Finally, one interviewee mentioned that requirements models could be used in areas where computer tools are still missing and repeated work is necessary. In these areas, it could be useful to build a library of reusable model elements that can be used by engineers to reduce the effort and errors.
[..] we plan to use MBE not as a individual method in small islands, but using synergies and reuse. On the long term we would like to build up a library of standard elements that can be reused.—B2, SysE
Provide context (T2.4)
In our interviews, lack of context information in requirements specifications was mentioned by multiple interviewees as problematic (see also ). That is, engineers lack information of how a requirement fits into the system context, which other components it affects or it is affected by, and who is working on them. In particular for subcontractors, this information is often not available, which makes it difficult to interpret and understand requirements.
In line with this challenge, two interviewees in Company A suggested to have structural models that show how functions or subsystems are connected in the overall product. As examples, the interviewees named classical architectural models such as component models with in- and outgoing signals, but also higher-level structural models that show the relation between requirements. Such models could then be used for actually understanding the requirements better, but also to identify circular references or for change impact analyses as mentioned before.
As a last theme, interviewees in both companies mentioned the importance of models to standardise and streamline the process in the organisation.
Two interviewees in Company B suggested a formal model syntax, e.g. as a profile for SysML. This would force engineers to think deeply about what they want to express, instead of just writing it down in natural language.
Often, the problem is not the existence of a requirements documents, but the method to create this document. [..] That is, every specification looks slightly different [..]. And model-based means in this case a strong use of SysML as a language, which brings the advantage that people are more aware of the fact that they express something in a more formal way that before.—B1, SysE.
In a similar direction, one more interviewee from Company B suggested to use defined terms or model elements as a means of standardisation. This would allow to check for consistency on syntax level, have a uniform (graphical) representation, and create different views of the model for different stakeholders.
Finally, four interviewees brought up the need for interface models to structure the work. These would allow a clearer separation of subsystems, but also of the system to its environment. As discussed in Sect. 5.1, these models already exist to some extent in both companies and, while their main purpose is architecture and design, they do connect to RE, e.g. for structuring of requirements.
RQ3: Why are models not used during automotive RE?
While models are clearly already used to some extent during RE in the automotive domain, we identified several obstacles that restrict a wider use of models during this important phase of systems engineering. In the following, we discuss these in detail. An overview is given in Table 6.
Mirroring the large focus on tooling in related work on MBE, we identified several challenges with respect to modelling tooling in our data. These relate to the customisation of tools, the use of multiple tools in an integrated way, and the extraction of information from tools.
Additionally to tooling challenges, we identified several themes that can be seen as general challenges in the area of modelling and model use.
Shortcomings of modelling tools
In both companies, interviewees stated that a variety of different tools is needed, meaning that these tools need to be connect and interoperate (T3.1). Five interviewees stated that it would be unrealistic to assume that a company can use a single tool in the same way across different organisation units and especially disciplines. For example, one interviewee in Company B stated:
One size fits all won’t work for Company B. Definitely not. [..] as I already said in systems engineering we are in the mechatronic world...we wouldn’t be interested to re-build existing UML models for software for SysML. It also wouldn’t help us.—B2, SysE
Similarly, interviewees in Company A refused the idea to use Simulink as a general purpose tool for requirements modelling, as other interviewees in the same company proposed.
I think Simulink is really good for some parts. It’s absolutely, really wrong for HMI parts. And it’s not really quite super good for state machines. So the thing is that this modelling has to be very flexible —A6, EmbSE
Similarly, tools need to be customisable (T3.2) to fit existing processes. However, according to the interviewees, the possibility to customise existing modelling tools is currently limited.
[..] they are all in their core 10, 15 years old, or older. [..] that means you have big problems customising these tools in an easy way. That is, directly highlighting certain menus, hiding certain features.—B2, SysE
Finally, six interviewees in both companies stressed the need to be able to make information stored in models more accessible (T3.3). In this context, one interviewee stated:
Most users don’t care about diagrams. They are concretely interested in ’I’m responsible for this component, and this requirements, or this function. I’d simply like to have a list, whatever, a matrix view, which tells me what the function is, what attributes it has, [..], which interfaces [..].’ [..] I’m not interested in the whole model. That is, the gap between the expert who creates the model and the group [..] which interacts or works with the model later.—B1, SysE
Concretely, interviewees expressed the need for tables, layouting, easy model navigation, and tracing capabilities in modelling tools.
Despite these challenges, it is important to point out that several interviewees stated that tooling is not the main problem in the introduction of modelling in RE.
[..] most of the functionality that we have today in (Requirements tool) is good enough...I see few tooling problems, very few tooling problems. There is a lack of information thinking. But that’s very much linked to the key concept of how the product is documented.—A6, EmbSE
Complexity and effort of modelling
In both companies, the high effort (T3.4) to create and maintain requirements models was brought up. One interviewee stated that it would be too much effort to reach complete formalisation, but suggested that it would be feasible to reach a large amount of formalisation with little effort. Therefore, formalising parts of the overall requirements specifications should be aimed for. In particular, one interviewee in Company A added that it is difficult to express non-functional requirements as models.
A similar aspect raised by the interviewees was that natural language is needed in addition to models, as these quickly become very complex (T3.5).
I think you still need to write what the purpose of the function would be. But probably not like the exact logical behaviour.—A2, EmbSE
In particular, interviewees in Company A saw the risk that there is a large amount of domain and modelling knowledge needed to understand existing models, meaning that recipients of the requirements models would need additional explanations in natural language.
Organisation and process aspects of modelling
One of the large risks brought up by eight interviewees was the risk of accidental complexity and design (T3.6) when creating requirements models. One interviewee stated that it is easy to go into too much detail when creating models.
It should be on higher level. [..] usually if you do a model and you want simulate it and so on, then you go into too much detail. [..] So to avoid these kind of problems, for the high-level requirements I think it is enough with textual requirements.—A3, EmbSE
Another interviewee added that this level of detail would require requirements engineers to have detailed development knowledge to be able to create requirements models.
While many interviewees felt that they should distinguish between requirements and design, a position also common in RE literature, they found this difficult to achieve in practice. In Company A, they even removed one abstraction layer in their requirements in the past, as engineer had difficulties to distinguish them in practice.
What we have now is sort of a merge of the analysis and the design. It’s more towards the design, but it’s sort of a mix. Because it was too complex for users to grasp how to distinguish between analysis and design.—A5, EmbSE
However, one interviewee in Company B stated that, overall, the right level of abstraction would simply be an iterative learning process.
In the first step, it will always end up wrong. Either too little or too much or something else. [..] You will approximate this iteratively. What is needed? What is meaningful?—B1, SysE
In addition to more technical concerns, several interviewees stated that their organisations are not mature enough (T3.7) to introduce modelling during RE. This was stated both with respect to the necessary skills to actually create and understand models, and with respect to the process currently used in the companies. Three interviewees stated that engineers would currently lack the necessary skills for modelling.
I mean because the persons that write requirements for (vehicles) know about (vehicles). [..] So how many (domain experts) know Simulink? I don’t know.—A8, EmbSE
On an organisation level, several interviewees stated that there would either be a lack of support from management, or a lack of understanding of the actual problem that requirements models would aim to solve.
Technically, everything works. It’s only that companies like Company B [..] have to figure out ’What do we want to fix?’. [..] Sometimes, they don’t even have a real understanding of their problem, leave alone a solution.—B2, SysE
Specifically, interviewees named the integration of requirements modelling into an interdisciplinary work environment, a suitable process for using requirements models, and the right education and training strategy.
Finally, interviewees stated that there might be resistance from employees (T3.8) when introducing models during RE. While several interviewees in Company B stated that employees with different education levels and backgrounds had been successfully trained in using models, in particular SysML, they expressed concerns that modelling would be a paradigm shift in the company. Additionally, one interviewee added that there could be resistance specifically when employees see their roles changing or even fear that their expertise is no longer needed.
I think a lot is simply a human factor. There are for example also those who have the overview. They sometimes don’t want it (the change), as they realise that the privilege they’re having, what defines their role, is in danger.—B2, SysE