Executing automotive test workflows with BPMN and web service orchestration in software-defined car manufacturing

Abstract

Automotive original equipment manufacturers (OEMs) are facing the challenge of keeping their cars innovative for the customer by integrating new advanced software functions, e.g., to provide higher levels of automated driving. Nonetheless, internal business processes should stay efficient to support the transformation despite the increasing complexity caused by new information and car technologies. For engineering and production processes where interdisciplinary teams of engineers, software developers, and workers interact along the car life cycle, the concept of executable process models offers an opportunity for work streamlining and process automation by web service orchestration. This paper will shed light on using executable process models under the cross-industry Business Process Model and Notation (BPMN) standard, along with the example of automotive test and diagnostic processes in smart manufacturing. We introduce a lean approach that includes the enterprise IT architecture for process modeling and the product car for process execution. Finally, we present our evaluation of the approach through a focus group with OEM experts.

1 Introduction to automotive trends

To be attractive to customers, cars include a rising number of software-assisted functions, which are essential for all levels of vehicle autonomy. The implementation of these functions implies that the number of software components in the car is also increasing and thus, original equipment manufacturers (OEMs) develop new underlying software and electrical & electronic (E/E) architectures. Supportive of this, automotive development partnerships such as AUTOSAR GbR and ASAM e.V. publish new standards for automotive software development or diagnostics for software-defined cars to provide a modern and flexible set of tools, concepts, and technologies.

Fig. 1

Interdisciplinary workflow during car manufacturing represented in BPMN notation

E.g., the AUTOSAR Adaptive Platform supports a runtime environment for separated hardware and software in electrical control units (ECUs) as well as for central car clusters including high-performance computers (HPCs) [1]. ASAM e.V. proposes software-oriented vehicle diagnostics (SOVD) for diagnosing and communicating with cars over a RESTful HTTP API [2].

The technological change within the car influences the OEMs’ internal engineering and production processes as well. On the one hand, processes are adapted to be flexible to respond to agile changes in the production process arising from new car software updates. On the other hand, internal OEM processes should be streamlined to provide an additional budget for the digital transformation. In this context, the term software-defined Manufacturing aims to decouple physical production hardware from the control software, which requires a service-oriented, modular architecture with virtualization and abstraction layers [3].

Business process management can serve with proven techniques from non-automotive industries to overcome these challenges: the Business Process Model and Notation (BPMN) 2.0 is an established cross-industry standard to graph business processes [4]. The standard includes execution semantics, which allows the definition of directly executable process models that are processed with a so-called process engine. In Fig. 1, an interdisciplinary car testing lifecycle during manufacturing work is illustrated with the BPMN standard. Four roles on specification and scheduling, testing, vehicle connection, and shopfloor and car engineering interact in a time-directed workflow to perform interdisciplinary work w.r.t. to a car. The question arises of bridging the business-IT gap between such a business process model and the execution of car-related tasks. This approach can form the basis for flexible and intelligent manufacturing [5]. Executing process models in the planning level of the automation pyramid is typically achieved by enterprise architecture engines. There is the particular challenge of incorporating the car network, including HPCs, for runtime critical applications, i.e., the field level with process engines. Aside from the technical components, user acceptance of the technology is also essential.

The underlying research question of this work is as follows:

• How can executable business process models for automotive testing and diagnostic processes improve software-defined manufacturing?

In doing so, we also investigate the five principles for conceptual modeling, i.e., user scenario, modeling language, modeling tool, people, and utility of the model [6].

The main contribution of this paper is a BPMN-based process automation approach to bridge the business-IT gap between engineering and production in automotive manufacturing systems. Based on automotive expert knowledge, we identify criteria for software artifacts to execute test process models in interdisciplinary work in and around a software-defined car. We will give an architectural blueprint w.r.t. seamless execution of internal engineering processes for car testing and diagnostics in flexible manufacturing with BPMN. The approach is evaluated by an automotive focus group and is therefore intended as an inspiration for end-to-end ecosystems to achieve quick adaption times.

The remainder of this paper is structured as follows: in Sect. 2, the industrial scope is stated and technical requirements are derived. Related work is reviewed and evaluated w.r.t. the requirements in the executable BPMN environment (see Sect. 3). The solution approach is presented in Sect. 4 and the respective artifact characteristics are described in detail. The approach is evaluated with a focus group discussion in Sect. 5. The conclusion of this work is given in Sect. 6.

2 Automotive scope and requirements

We have a closer look at the automotive scope based on industrial case studies from the car manufacturing domain and derive requirements for the software artifact creation.

Fig. 2

Relevant automotive processes in this paper with screenshots of test specification, implementation and execution

The following case studies give a detailed view of the workflow from Fig. 1.

2.1 Case studies in automotive business process management

E/E car components are tested and commissioned in car production and, if necessary, flashed and coded with software. For this purpose, the underlying diagnostic tests are specified and programmed into a sequence of tasks (see the general setup of Fig. 2). This test specification is an interdisciplinary manual process for engineers from development, production, and after-sales that results in a textual specification document. This specification is, in general, valid for a car’s lifetime, i.e., $>10$ years, including repair workshops. A diagnostic test is a test to identify the cause of a fault, such as a test to see if the door lights illuminate when the door is opened.

The process is implemented in source code with variants for several car series. In the car development phase, these implementations are executed individually. The interaction of production staff is required for manual execution and rework. For a selected assembly line section in volume production, the relevant tests are planned and scheduled by computer scientists for the assembly line so that executing the tests on the car requires a minimum run time under predefined boundary conditions. For any given car, the set of relevant tests is then performed on an assembly line section with worker-car interaction. There are standardized interfaces for vehicle diagnostics, such as the SOVD API [2].

In the event of an error during execution, the manufacturing rework process is initiated. Therefore, the worker makes a computer-aided decision as to whether the failed test is due to a mechanical cause, such as a defective component, or whether there is a functional error, such as the incorrect coding of an ECU. A decision is made if the rework can be performed inline on the assembly line or if the car must be sent for central rework to better understand and reproduce the error. In the daily shop floor meeting, workshop employees and car engineers study shopfloor reports, e.g., on a car line with errors and test times from the previous production day.

An exemplary test is the control of the door exit light. There, an ECU session is established via diagnostics, the door exit light is checked, and the session is closed again. The case studies are shown in Fig. 2 with screenshots of the initially mentioned function test of the door exit light.

2.2 Requirements

Group discussions in testing projects with automotive experts highlighted seven relevant requirements for developing the concept of executable process models in software-defined manufacturing. The requirements are also identified from further work by König et al. [7] and the technological and data related challenges listed in the work of Bickelhaupt et al. [8].

• 1. Executing business processes is no new approach. We demand the re-use of existing cross-industry methods, i.e., even in industries outside the automotive domain ($R_{\text {re-use}}$).

• 2. The developed approach should be integrated into large enterprise IT scopes with OEMs and their suppliers ($R_{IT}$). Testing specific applications as planning and scheduling should be supported, which means that algorithmic solutions must be integrated, too.

• 3. To reduce complexity in internal digital processes and respective IT systems, the test specification document should also serve as an executable artifact. The approach should support the execution of test specifications such that the specification and implementation are no longer separated ($R_{seamless}$).

• 4. As the diagnostic web service calls in the car are supposed to be routed over SOVD for software-defined cars [2], the parameterization of the API should be directly integrated into the process specification ($R_{API}$). The approach should support API Management for automotive standards as SOVD and benefit from web technologies.

• 5. Programming knowledge should no longer be required for process specification and execution. The approach should not require programming knowledge for defining schedule descriptions and visual representations are preferred ($R_{\text {no-program}}$).

• 6. The concept should assist assembly line workers during rework ($R_{analysis}$). Therefore, a simple analysis of failed tests to find root causes should be supported.

• 7. As the considered work in the business process is along interdisciplinary teams along the product life cycle, new employees are involved over time. Thus, new employees should be able to learn the approach quickly, i.e., standards already known from other industries, universities, or online tutorials are preferable ($R_{low}$).

3 Related work

Based on the automotive case studies and the requirements presented, we assess our work into the related work w.r.t. business process management in software-defined manufacturing.

The nomenclatures “Software-defined Car” and “Software-defined Manufacturing” are predominantly driven by research projects, which are publicly-funded by the German Federal Ministry for Economic Affairs and Climate Action. For instance, there is a research project named SofDCar for Software-Defined Cars, as well as SDM4FZI for Software-Defined Manufacturing. The projects aim to improve vehicular and automation systems by making the functionalities of vehicular and manufacturing hardware definable through software. Car diagnosis can be implemented over-the-air for autonomous driving functions, whereas production systems are intended to be more reconfigurable. In the production environment, particularly Oechsle et al. deal with the independence of vendor-specific applications [9], however, the focus is on a general real-time capable architecture in manufacturing. The proposed deployment and orchestration workflow is not specific to a modeling language and lacks a concrete implementation.

In the next step, relevant modeling languages are discussed. On the one hand, in the context of the automotive industry and orchestrating test workflows, there exists the ISO standard OTX [10]. OTX is the standard to describe diagnostics and testing sequences for the automotive industry based on the UDS protocol [11]. As OTX is focused on the automotive onboard diagnostics setting with UDS, it does not allow including human tasks, and is less feature-rich than BPMN. This means especially, there is no task element that supports web services.

On the other hand, UML [12] and SysML [13] are languages to model the design and structure of software systems with various diagram types. They are not intended to model business processes and execute these with process engines.

Event-driven process chain (EPC) is a modeling language to graphically illustrate business processes with flowcharts [14]. It is not standardized or used worldwide and moreover, not indented to be executed through a process engine.

To summarize, in the context of manufacturing, Garcıa-Domınguez et al. provide a comparison with further modeling notations [15].

As we are interested in a domain-independent modeling language with standardized basis exchange format and execution semantics, we focus on BPMN, as already touched by König et al. [16]. In the following, we have a look at existing work on executing BPMN process models in manufacturing lines and on flexible orchestration of services.

Kocher et al. discuss modeling and executing production processes with capabilities and skills using ontologies and BPMN [17]. Modeling and executing IoT-enhanced business processes through BPMN and microservices with the use of ontologies for low-level real-world data is discussed by Valderez et al. [18]. We do not focus on ontologies and there is no focus on scheduling or hardware-related process execution.

Erasmus et al. discuss the application of business fragments and process models with BPMN for the specification of manufacturing operations [19]. The presented HORSE system discusses the execution of models as well, but there is no explicit focus on the manufacturing of cars.

Traganos et al. study end-to-end production process orchestration for smart printing factories [20]. The presented architecture focuses on a cyber-physical system with BPMN for modeling and executing purposes.

The general development of process execution support for high-tech manufacturing processes is discussed by Vanderfeesten et al. [21].

Roller and Engesser discuss the application of BPMN for automotive plant simulation [22].

Dubani et al. present a BPM-based framework for modeling and executing automotive business processes [23]. The framework combines BPMN and BPEL, which is no wide-applied industry standard anymore.

Hasic et al. study the execution of IoT Processes in BPMN 2.0 and identify remaining challenges [24].

Neubauer et al. [25] present subject-oriented business process management (S-BPM) with respect to high and low-level control over a common communication model.

A novel framework to automatically generate executable web services from BPMN models is presented by Zafar et al. [26]. The focus lies on Java web services.

Schäffer et al. present a process-driven approach within the engineering domain by combining BPMN with process engines [27]. The focus is on the product emergence process (PEP) that includes the subprocesses of product development, production planning, and production.

Stiehl et al. illustrate process-driven application with BPMN are illustrated and exploit the full functionality of BPMN 2.0 to model and execute cross-organizational differentiating processes [28].

In terms of flexible orchestration, a decision notation needs to be introduced. Funk creates a low-code business process execution platform with Python, BPMN, and Decision Model and Notation (DMN, OMG [29]) [30], which does not map to a flexible scheduling setting. Peinl and Perak study BPMN and DMN for easy customizing of manufacturing execution systems [31]. Nevertheless, the application of decision tables alone is not sufficient for the overall concept, as it is already highlighted by König et al. in terms of production workflows in manufacturing lines [7]. So we can use an optimization solver as discussed by König et al. [32].

Fig. 3

Artefact overview w.r.t. the automation diabolo for improved test management (originally from Vogel-Heuser et al. [33])

To the best of our knowledge, there is no related work that studies the end-to-end application of executable business process models with BPMN in the context of car testing in software-defined manufacturing.

4 Approach on executable BPMN in software-defined car manufacturing

Based on the requirements and the identified research gap, we formulate an approach for executing automotive tests modeled with BPMN in software-defined manufacturing in the following. The approach includes an architectural proposal and refers to appropriate software artifacts to realize the system.

4.1 High and low-level test management with BPMN

The artifact overview is presented in Fig. 3. Following the automation diabolo model of Vogel-Heuser et al. [33] and the work of Neubauer et al. [25], the automotive architecture model for testing in software-defined manufacturing is divided into three layers. High-level control along automotive business processes is depicted in the upper rhombus and refers to an off-car architecture in the enterprise IT architecture (more details in Sect. 4.2). Low-level control along automotive production processes occurs in the lower gray rhombus in-car and refers to the in-vehicle architecture (details follow in Sect. 4.3). The familiar interface is the standard BPMN XML exchange format, illustrated as a box in the middle. The artifact overview builds the basis for an automotive workflow management system.

The respective transformation of the processes w.r.t. users and production strategy are displayed in matrix form in Fig. 4.

Engineers specify and implement a test for diagnostics or functionality checking by using the notation BPMN. The notation element Service Task can be used to interact with the car diagnostics API for web service calls. Worker interaction during execution is represented with a User Task. A decision based on Data Objects can be illustrated with Exclusive Gateways. Subprocesses can be represented as fragments in Call Activities.

Scheduling of tests is achieved by orchestrating BPMN models w.r.t. an automated optimization solution, e.g., as presented by König et al. [32].

Executing the diagnostic test with an automotive process engine takes a particular role. In this step, the concept leaves the horizontal level of engineering processes and carries out executable BPMN in diagnostic procedures in-car (see Fig. 3).

Rework and analysis documentation can be highlighted in the BPMN process, e.g., by a heatmap. In the following, we discuss details on high-level control in the upper rhombus of Fig. 3.

Fig. 4

Transformation matrix for consistent interdisciplinary process management

The conceptual model gives a detailed understanding of the highlighted components and relates to the central class of the automotive automation model associated with a web service OrchstrationContainer (see Fig. 5).

The software system orchestration container is built upon the concept of a process model (PM) graph [34]. It builds the formal basis for BPMN process models. A ProcessModelGraph instance instantiates a software system for a specific CarSystemConfiguration at a test location w.r.t. to the Assembly planning. The car system configuration depends on the car‘s software and hardware (see Fig. 5).

4.2 Details on high level control in enterprise IT systems

The underlying use case is on modeling tests for production diagnostics. An evaluated guideline exists on how to model these workflows with BPMN published by König et al. [35]. The DMN standard is integrated to represent decision via a decision table. A test management process model is deployed via the OEM backend onto the vehicle onboard HPC and stored in the BPMN off-car Process Model Storage as standard BPMN serialized in XML format (see Fig. 3).

Moreover, constraints for test scheduling must be planned as proposed by König et al. [32].

After reviewing various commercial modeling tools, the authors decided to develop their modeling tool prototype because the modeling of the data flow with data objects as data inputs and outputs, as well as directed associations could not take place explicitly with just one tool. In addition, the integration of the interface to the vehicle was to be solved more flexibly between the modeling tool and engine. Thus, the modeling prototype should trigger the explicit modeling of data flows, the deployment, and the execution of process models in the car and support the integration of BPMN templates developed explicitly for the automotive sector. A screenshot of this software prototype is shown in Fig. 8.

Since the use of Service tasks is of particular interest in the application area of car diagnostics, e.g., to invoke diagnostics-related API methods of a car’s SOVD API, the modeling tool allows to specify service calls against any HTTP API generically. Therefore, it enables employees to create and enrich required API requests based on modeled data objects and extract data from API responses through defined transformation expressions.

On the other hand, the user can also integrate existing SOVD OpenAPI templates. For this purpose, a particular BPMN template is offered for the automotive industry. Thus, the API methods specified within an OpenAPI document can be directly converted into a user-friendly BPMN template containing all the required information for calling the respective API method. These templates can then be offered within the palette of the modeling tool to enable employees to add them to their process models via drag and drop. In the Appendix, we propose a BPMN Service Task template. It is an excerpt from an evaluation study. It utilized a transformation for the API call. Data field assignments are also defined in the BPMN 2.0 standard but are not realized in the present modeling tool.

Fig. 5

Conceptual model of the overall approach

Monitoring and reporting of BPMN processes and their executions are available in customized tools. Since we do not have any specific requirements regarding their application in the context of test management, we do not focus on these aspects in detail.

4.3 Details on low-level control in vehicle networks

The BPMN process models have been deployed from remote access over the OEM backend to the vehicle network into the storage of the BPMN in-car process model. Now, the event managing component operates an instance of process models into a test schedule dependent on the respective test constraints for the car configuration. This belongs to the OrchestrationContainer of Fig. 5.

The BPMN process execution engine is integrated into the car network and communicates with the diagnostics framework during software-defined manufacturing. In contrast to existing industry applications, e.g., as studied by Rücker and Freund [36], there runs only one process instance at a time within one process engine in a car. In car production, APIs must meet strict millisecond latency requirements for tasks like window scaling or test bench communication, often for safety or certification reasons. Consequently, individual workflow engines are deployed in each car to execute diagnostic processes specific to that vehicle within the necessary latency limits. A low footprint in terms of memory and CPU consumption, as well as fast execution of processes. As figured out in an automotive runtime comparison framework [16], the process engine is written in C++. This choice is underlined by the work of AUTOSAR GbR [1]. This paper does not take a deeper look into the process engine as details regarding a car-tailored process engine are presented in König et al. [16].

To enable flexible orchestration of test processes and their constraints to each other in the car, the method of adaptive rescheduling is used. This allows for a new schedule to be recalculated even during runtime if a more optimal solution should be available due to time shifts of past tests. Results of a respective quantitative experiment are presented in König et al. [37].

The illustrated business-car-related system architecture of Fig. 3 supports the interdisciplinary work during the engineering phase of software-defined cars by multi-disciplinary teams composed of engineers, IT architects, programmers, process experts, and data analysts.

5 Evaluation and discussion

The approach presented in Sect. 4 is evaluated in two steps: a BPMN modeling tool and in-car execution engine can prove the end-to-end integration for tests. In the following, we call this an integration test. The overall approach to executable BPMN is discussed in a focus group with OEM experts.

5.1 OEM evaluation setting

The system used for experimental evaluation is part of an automotive OEM-internal development setting for production technology. An employee specifies and develops tests on a personal notebook and further executes the tests via a car tester over the onboard diagnostics interface, the prototypical evaluation setting was intended to be similar.

The automotive BPMN modeling tool runs on a personal notebook. The BPMN Process Models are deployed onto a car diagnostics tester that includes an assembler of an automotive HPC.

The car tester dongle is a HPC emulator. In Fig. 6a, the device is illustrated itself, whereas, in Fig. 6b, the device is plugged into the OBD socket. The starting test setting is illustrated in Fig. 6c. After the successful integration and setup of all components, the scenario was also successfully applied and tested in an actual assembly line production context.

The BPMN Process Execution Engine is outlined in the work of König et al. [16], and the Orchestration Service is validated by König et al. [37]; both are containerized as Docker containers on the HPC emulator. The in-car components are accessed via REST APIs with HTTP clients. So they can be applied for the ASAM standard SOVD.

Fig. 6

The OEM evaluation set up

5.2 Integration test for BPMN modeling tool and car BPMN engine

Using the OEM evaluation setting, we perform a so-called integration test to show the implementation and execution of a BPMN process model in the car with the provided software tool components.

To do this, the function test for the car door light was chosen as it checks basic diagnostic routines and includes different hierarchy levels within the overall workflow.

Fig. 7

BPMN models of the door exit light functionality for automotive modeling tool and engine integration test—screenshots are taken from the modeling tool that includes pixels in the canvas

In Fig. 7a, the test is illustrated as a workflow of Call Activities for three subprocesses, which are further described in the Fig. 7b–d. The subprocesses receive data inputs Car and ECU, which are parameters for the API calls so that the test can be parameterized and executed for several cars and door light ECU variants. Within the first subprocess in Fig. 7b, a diagnostic security token is generated via a Service Task API call. The token is saved as a data object for further Service Tasks within the subprocess and provided as data output to the following subprocesses in Fig. 7c and 7d. Further automotive-specific steps are performed. In Fig. 7c, the light is activated within the time duration of 5s. If the door exit light works, the test produces the visual output illustrated in Fig. 2. In the subprocess on postprocessing in Fig. 7d, the ECU error memory is checked, and finally, the process is terminated.

The realization for two car variants showed that the integration of BPMN in high and low-level test management is fulfilled.

5.3 Focus group validation

A focus group validation has been conducted to evaluate the approach to executable BPMN with the involved user group.

Focus group members The focus group consisted of eight employees of an OEM, including one employee with an expert background in production planning, three employees involved in specification and implementation of testing and diagnostics, two employees engaged in test schedule programming, one employee working on failure analysis during shopfloor, and one employee being a production test system developer. One of the eight experts had experience less than a year in his current job. Two of the eight experts were women. The moderator of the focus group was one of the authors of this paper.

The focus group participants were already involved in the product development of the automotive BPMN modeling tool during two workshops of totally 3.5h to create a user-friendly artifact. The focus group tasks were communicated online to the participants, who had three weeks to complete the tasks and provide a questionnaire of the approach in the tool Mural. Afterwards, a hybrid 1.5h workshop with discussion has been conducted.

The engineering processes of test sequence specification and programming, schedule generation, execution in the vehicle, and the idea of an end-to-end ecosystem were evaluated concerning the parameters: Applicability of the BPMN-based approach, Expressiveness of BPMN models for the automotive testing setting, and Feasibility of the modeling and executing setup. The rating could be made in five levels: very good, good, neutral, poor, and very poor. In addition, three free-text columns were added to the questionnaire: Name advantages of the solution, Name challenges of the solution and, if applicable, suggestions for addressing them (w.r.t. tooling, concept...), and Comments. The full questionnaire is shown in the Appendix.

Fig. 8

Illustration of BPMN data handling and processing during a focus group evaluation task

Modeling tasks The first modeling task was to understand a software module test to control the fan in the car seat (see Fig. 8). Three functional changes were then to be made in functional parameters. In addition, a subprocess invocation had to be modeled via a Call Activity.

In Fig. 9, further process models are illustrated. The employees were presented with one example each for the integration of BPMN script tasks (see Fig. 9a) and a representation of error (see Fig. 9b) handling in the context of test bench communication.

In Fig. 9a, it is shown how a user can integrate an external source code script via the BPMN element Script Task. The authors have included this process model in the evaluation because there were concerns in preliminary discussions with focus group participants that, e.g., measurement evaluations could be mapped within BPMN.

In Fig. 9b, an example of error handling events in a process model is shown, which executes a test routine as a subprocess via a Call Activity and contains two error cases: When the door is opened, a test stand goes into idle mode. Otherwise, the test is terminated.

Results and discussion In the subsequent discussion meeting during the on-site workshop, the results were discussed with the participants. The aggregated results of the questionnaire are presented in Fig. 10. At first glance, most results range from neutral to good.

The evaluation result regarding the applicability of the concept from the perspective of the focus group participants is shown in Fig. 10a. One participant found the applicability to be very good in terms of test specification and programming, six participants found it good, and one participant was undecided in the scores very good and good (0.5 points each).

Fig. 9

Interpretation tasks during focus group evaluation

The advantages of the solution are that no programming is required and that functions can be reused in the company through templates. Moreover, the modeling approach makes it possible to track an error and its propagation through the process model graphically. A graphical overview is actually always advantageous, the participants said. In summary, simple test sequences can be mapped well. Challenges in the current tooling were cited as the naming being incomprehensible, which the authors justified by the fact that the modeling tool is a software prototype. One employee noted that in the case of complex examples, he considered source code programming to be clearer. Another coworker saw in addition the construct of the Script Tasks as suitable, if complex contents can be merged in external scripts.

For schedule programming, the concept is rated as very good by two participants, good by five participants and one participant rated it as poor.

The advantage of the solution is that no programming is required and the graphical representation provides a good overview of the processes. Nevertheless, in case of flexible manufacturing, an automated concept, as proposed in the paper of König et al. [37], is preferable. No concrete challenges were named. Critiques were that there is much initial effort since existing specifications and program code would have to be modeled from scratch in BPMN 2.0. The current prototype lacks visualization for error detection. In this context, the current prototype of the modeling tool was discussed, which should be improved in terms of user experience, e.g., labels in the modeling tool often need to be clarified for the users. The participants can think of assistance functions that flow creation is partly automated.

In the in-vehicle execution, two participants rate the approach as very good, three as good and three as neutral.

It was noted that only those defects can be visualized graphically that are also related to the modeled processes. A higher level of detail would still have to be analyzed. According to the employees, the concept is a milestone away from the current process. Great potential is seen in the execution of safeguards, not only on the car, but also, e.g., in software-in-the-loop, hardware-in-the-loop, or functional mock-ups.

Regarding an end-to-end ecosystem, two participants rate the concept as good, five are undecided and one participant rates it as poor.

It is seen as a necessary foundation that all relevant stakeholders, including suppliers, need the same modeling tool and execution engine, e.g., as further development of the ASAM standards.

In the next part, the evaluation regarding the expressiveness of BPMN models were evaluated. The results are presented in Fig. 10b.

One participant rates the expressiveness of the models for test specification and programming as very good, three participants evaluate it as good, one participant is neutral and two participants evaluate it as poor. One participant finds good and poor portions, but does not want to rate neutral, resulting in 0.5 points each for good and poor.

The possibility of creating functionalities quickly by modeling is seen as an advantage. The parameterization of REST calls in Service Tasks must be made easier for users. The authors have countered that the modeling tool is a prototype that is intended to map the basic functionalities and does not represent a finished end product. One challenge is the confusion caused by long strings in the parameterization. Basically, it would be necessary to work with sub-routines in order to maintain an overview. Here, more visual concepts are needed on how to understand a complex process without clicking back and forth too much.

For the test schedule generation, one participant sees the significance as very good, four participants as good, two participants as poor and one participant as very poor.

Basically the same benefits and challenges were mentioned as for test creation.

Regarding the in-car execution, the participants rate the informative value with $50\%$ as good, $25\%$ neutral and $25\%$ as poor.

It was noted that for engineers, testing on the car involves working with a laptop. Consequently, a smaller monitor is available as when working on a desk, where more complex models can be clearly displayed in the modeling tool.

For end-to-end ecosystem design along car development, production and after sales, one participant rates very good, three participants rate good, and four participants are neutral.

In the context of interdisciplinary collaboration, the concept is considered to be quickly understood. If modifications are expected in the program, expert knowledge is required.

The third and last part of the evaluation concerns the feasibility with the goal of automating executable process models with BPMN (see Fig. 10c). Two participants rate the feasibility in the environment of test specification and programming as very good, one participant as good, four participants as neutral and one participant as poor.

In principle, the feasibility is good to neutral. In the case of special cases with many parameters and sub-menus, the challenges lie in maintaining an overview.

Regarding test schedule generation, three participants give the rating very good, one participant good, three participants neutral and one participant poor.

To achieve a well-structured flow, the test sequences must be well modularized. If all BPMN components are allowed at the flow level, automatic optimization can be difficult to implement. Reference is made by stakeholders to the concept of König et al. [32].

Regarding the feasibility for automation in the vehicle, two participants give the rating very good, two participants give the rating good and four participants are neutral.

Regarding the feasibility of executable BPMN in the ecosystem, one participant gives the rating very good, one participant good, five participants are neutral in their rating and one participant gives the rating poor.

Fig. 10

Cumulative ratings among 8 experts as part of the focus group—line color blue for area of test sequence, orange for test schedule, grey for in-car execution and yellow for extrapolation to OEM-wide testing workflows

It was noted that all stakeholders from development, production and after sales need the same processes and tools. If all parties involved the new standard, the greatest added value is created.

In the further discussion, it was noted that templates for standard processes with nested structure with 10–15 elements are practical. There should be the possibility to group BPMN elements by color. Staff would also like to be able to store comments. The authors noted that this is possible in the BPMN standard, but is not part of the modeling prototype. A debugging concept with breakpoints is desired, as is a simulation of test cases. A connection to repositories like GitHub is desired to observe development statuses. A concept for user training is demanded.

5.4 Assessment of the requirements

We presented and discussed the findings of the integration test and the focus group evaluation w.r.t. the approach on executable BPMN with flexible in-car orchestration in software-defined manufacturing in Sect. 5.3. The assessment of the requirements from Sect. 2.2 gives more details on the slightly positive evaluation.

Requirement $R_{\text {re-use}}$ is fulfilled as the integration test’s result shows and it is a plus for the focus group participants that the notation is not only automotive-specific. In a next step, templates for execution should be defined. The integration test showed that the general approach set up works and thus, requirement $R_{IT}$ is fulfilled. It would be a plus, if car development and after sales domains applied the approach as well. Via the integration test and the results of the focus group,the requirement $R_{seamless}$ is fulfilled in the automotive manufacturing domain. The seamless integration was the most significant point for the focus group participants. Requirement $R_{API}$ is partly fulfilled as there is a need for better user experience as the focus group evaluation showed. There was a lack in user experience and given templates. As a result, the participants had problems to quickly understand the appraoch. Moreover, requirement $R_{\text {no-program}}$ is fulfilled. This was a main requirement for the employees as production processes get more complicated because of the integration of more and more software artifact into the business processes. Especially workers in automotive shopfloor highlight the benefit of visual analysis, so requirement $R_{analysis}$ is fulfilled. This helps the workers to easily understand what the current problems are, e.g., during shopfloor meetings. Engineers confirm that the ramp-up time for new employees can be shortened by the cross-industry approach on executable BPMN. Nevertheless, quantitative metrics on time reduction are missing and should be studied. So, $R_{low}$ is partly fulfilled.

Regarding the research question, executable business process models with BPMN build a solid basis for automotive testing and diagnostic processes in software-defined manufacturing.

Transferability The approach is applicable for run time critical application scenarios. This means the non-automotive core components of the approach cannot only be integrated into vehicle networks, but also, e.g., on programmable logic controllers to execute testing processes seamlessly. The transfer to areas outside the automotive test and diagnostic processes was not examined in this paper.

6 Conclusion and outlook

Lean process automation for software-defined car manufacturing is the basis for a digital ecosystem. The approach of this paper implies the application of business process management with execution semantic approaches to bridge operational and information technology for automotive testing processes for software-defined cars in manufacturing. As one result continuous engineering processes are enabled without vendor-specific software frameworks. The cross-industry BPMN standard plays a crucial role, as business processes can be modeled graphically and executed in enterprise IT systems and cars with an in-car process engine.

In this work, the contribution of executing automotive test workflows with BPMN and web service orchestration in interdisciplinary software-defined car manufacturing is highlighted through an evaluated approach at an OEM. The presented system allows for seamless modeling and execution of manufacturing processes in the enterprise architecture and the car. It includes flexible orchestration of web services to manage car variants and interact with modern automotive diagnosis interfaces.

The approach is validated with respective software artifacts and a software-defined car-integrated case study. A focus group evaluation with automotive experts highlights the benefits of seamless internal processes and shows ideas for further research. User experience design for the modeling tool is essential when employees in interdiscriplinary teams collaborate in complex systems. With respect to the software-defined car manufacturing lines, a production-specific web service interface should be developed to further integrate information technologies into car architectures.

Moreover, the concept should be evaluated across all diagnosis domains, i.e., car development, production and after sales. A study across OEMs, e.g., as part of a future research project, could show more benefits on standardization of the software-enabling technology that was presented in this paper.

Appendix

See Table 1.

Table 1 Questionnaire form as basis for the focus group discussion