The application of the two selected process variant modeling approaches, namely DDM and Provop, in our action study provided various insights about both the benefits of using a variant modeling approach instead of following standard process modeling approaches and the challenges related to these approaches. Table 6 provides a comparison of the characteristics of the approaches and the resulting outputs. In total, the team spent 28 h in seven sessions on applying DDM, whereas 23 h in six sessions were needed for applying Provop.
In the rest of this section, we provide our findings from the application of the two selected process variant modeling approaches. This includes the evaluation of the business value of applying a variant management approach in Sect. 6.1, and evaluation of the two approaches against the identified user needs and requirements in Sect. 6.2. Lastly, we present the selection guidelines identified based on these evaluations in Sect. 6.3.
Evaluation of the value of applying a variant modeling approach
The team evaluated the business value to be achieved by switching from the current way of process modeling to a specific process model variant management approach. For this purpose, the team calculated the expected Return on Investment (ROI) for DDM and Provop, as presented in Table 7. We describe the details of this table below.
Variant creation 4S works on average with nine customers per year, two of them being added as a new customer, and seven being existing customers. For these existing customers, only 30% of the effort is spent on defining new processes. 4S analysts works on around eight different process areas per customer for which variant modeling is performed. Around 90 h of effort are spent by 4S analysts for creating a new variant. As a result, \((2 + 7 \cdot 0.3) \cdot 8 \cdot 90 = 2952\) h are spend in total for variant creation.
For both DDM and Provop, a reduction is expected in the variant creation time. For DDM, a reduction of 49% is expected. For Provop, a reduction of 64% is expected.
Maintenance As mentioned, 4S works with nine customers per year. Two of these are new and do not require any maintenance. For the seven existing customers, 30% of the effort is spent on variant creation. The remaining 70% are on maintenance. As mentioned, 4S works on eight different process areas. 4S analysts spend on average 54 h of effort per process area. As a result, \((7 \cdot 0.7) \cdot 8 \cdot 54 = 2116.8\) h are spent in total on maintenance per year.
Similar to the variant creation, also in the maintenance of variants a reduction in effort is expected by the team. For DDM, this reduction is expected to be 12%. For Provop, a reduction of 27% is expected.
Transformation For transforming from the current way of working to either DDM or Provop, two kinds of costs are involved: (1) transforming the current set of variants, and (2) training the analysts of 4S to become proficient with either DDM or Provop.
In the past five years, 4S has worked with 17 customers. For each customer, a variant has been created for each of the aforementioned eight process areas. For DDM, it is expected that, on average, the transformation of one variant takes around 32, 4 h. For Provop, this is 25, 2 h. Currently 20 analysts are employed by 4S. 4S expects that a total of 24 and 32 training hours are needed for DDM and Provop respectively. In total, the transformation for DDM is \((17 \cdot 8 \cdot 32,4) + (20 \cdot 24) = 4886,4\) h and for Provop, this is \((17 \cdot 8 \cdot 25,2) + (20 \cdot 32) = 4067,2\) h.
Costs and return on investment The costs are calculated based on an hourly wage of 50 Turkish Lira (TL), which is about € 12,50. Considering the yearly savings to be obtained in variant creation and maintenance activities, it is calculated that by using DDM, 4S will start to gain benefits in 2.9 years for the investment of transformation costs, whereas this number is 1.7 years for Provop. Although this calculation is specific to 4S and all values are prone to change in specific cases, it shows that considerable business value can be obtained by organizations in short-term by applying any process variant management approach.
Evaluation of the approaches with respect to user needs and requirements
In this section, we evaluate the application of the two approaches with respect to the user needs identified in Sect. 2.2 and the requirements specified in Sect. 3.1. We drop three of the requirements, namely Req. 1 domain independence, Req. 2 conceptual process type, and Req. 4 validation, since the definition of the approaches already supports these requirements and there were no further arguments by the team. We keep the four remaining requirements, control-flow (Req. 3), hierarchy (Req. 5), process modeling language (Req. 6), and decision support (Req. 7) that deserve further discussion about the extent the approaches satisfy them. Table 8 provides a summary of the evaluation. In this table, a benefit and/or advantage provided by an approach is indicated by a “+” sign, whereas a challenge is expressed by a “−” sign before the item. We further elaborate on each user need/requirement below.
Table 8 Comparison of the approaches based on the user needs and requirements User Need 1Establishing a knowledge-base Both Provop and DDM were observed to provide a knowledge-base by combining the process knowledge from multiple customers in an integrated model. This was found to be an improvement with respect to the current way of working where 4S analysts developed and maintained process models of each customer separately. The team found it relatively easier to read Provop’s base process as a reference. The base process provided essential information about the process without the need to use the option list. Moreover, the option list proved to be an effective tool to discover detailed similarities between the variants by means of operations grouped for multiple variants. Other interesting characteristics of control-flow variations were also realized by means of the option list. For example, although three variants included the create asset activity, each of them used this activity in the control-flow in a different way. Thus, three different operations had to be defined for inserting the same activity in different ways. The team found it non-intuitive to interpret the variation map of DDM, so it was hard to use it as a knowledge-base. However, main variation map included high-level activities, and provided a more summarized view that was detailed through lower levels.
In conformance with this evaluation, the team estimated a drop in the variant creation activities for both approaches, but more for Provop as seen in Table 7. Such an effort saving is foreseen since the analysts can use the created models as a knowledge-base while developing new process models.
User Need 2Maintenance Since both approaches provided an integrated environment that facilitates the impact analysis for changes, the team evaluated that maintenance effort to be lowered with respect to the current status. Maintaining DDM variation maps was evaluated to be harder with respect to Provop’s base process as there may be multiple subprocesses defined for variants of the same activity. A generic change may require the analysts to check each subprocess and apply the change in a consistent way. The team tried to alleviate this issue by applying their solution to integrate subprocess variant models that are referenced from multiple processes. Provop outputs were found easier to maintain than the outputs of DDM as changes can be applied on a single model. Provop was found to facilitate the maintenance of specific changes in addition to generic ones due to this fact. However, special attention needs to be given to maintain consistency with the option list. Accordingly, the team estimated the maintenance effort to be lowered for both approaches and more for Provop (Table 7).
User Need 3Reuse By providing an integrated knowledge-base from which similar variants can be looked up and a new variant can be configured, the team evaluated that both approaches can facilitate systematic reuse of process knowledge in comparison with the current approach. The team found DDM’s variation map more flexible than the Provop’s base process for defining a new process, as they can see all alternatives together with the constraints on the map. Business drivers helped the team to identify potentially similar variants in the existing models. However, Provop was found to be more practical for configuration-based reuse as described in the decision support item below. The support for variant modeling of hierarchical processes was found to be another aspect that affected reuse, which is discussed below. The expected improvements on the reuse were reflected in the effort estimation for variant creation in Table 7.
User Need 4Variability in a hierarchical structure and Requirement 5Hierarchy This user need emerged with the implementation of a process variant management approach, since BPMN natively supports the organization of process models in a hierarchical structure. The team evaluated that the application of both DDM and Provop requires additional consideration and effort for managing hierarchical process models. Although DDM by definition supports variant modeling in hierarchies, in practice the team encountered a problem to develop variant models for subprocesses that are used by multiple process variants. Among the solutions identified, the team applied the one which increased the reusability on low-level process variants. For Provop, the team could apply the native subprocess definition mechanism of BPMN. In this case, the team realized that they need to represent activities that are invisible in the base process because they are inserted by options. To establish a consistent hierarchical structure in the repository, the team placed such “standalone” activities in the base process. Moreover, another issue that arose with the application of Provop in a hierarchical structure is that, special attention must be paid to the consistency of adjustment points and events between different process levels.
Requirement 6Process modeling language 4S preferred to use a variability management approach based on BPMN since it was the notation used in the company. This requirement was met by both approaches. However, for applying Provop, it was necessary to add specific symbols representing adjustment points on top of standard BPMN models. Moreover, the team had to tailor the use of BPMN for specific purposes (such as standalone activity symbols for invisible activities). In summary, Provop required some changes over standard BPMN notation that the analysts need to learn and deal with. On the other hand, standard BPMN constructs were sufficient to use DDM.
Based on the metrics in Table 6, Provop seems to produce process models with less complexity in terms of BPMN constructs. This may enhance the understandability of the approach in comparison with DDM [48]. However, there is an extra artifact, the list of options, required to read and customize the Provop base process. The use of an extra artifact can cause cognitive difficulty due to the split attention effect [34]. Nevertheless, the team indicated that it was easier for them to read the Provop’s base process with respect to DDM’s variation map, and “picturize” how the adjustments may be conducted even without seeing the option list.
Table 9 Process variant modeling approach selection guidelines together with related user need (N) and requirement (R) for each item (refined from [14]) Requirement 3Control-flow The team found DDM relatively more flexible in defining control-flow variations, because the variants were developed as distinct subprocess models. The team experienced some limitations in modeling control-flow variability in Provop, such as inserting optional and consecutive activities, deleting process fragments, and dealing with possible configuration conflicts. The team had to extend and limit the application of the approach to resolve those limitations. An important limitation introduced by the team’s decision was avoiding the use of the fragment insertion feature of Provop. While this decision may have increased the complexity of the base process, it alleviated the maintenance effort related to process fragments. In the end, as Provop required the team to handle control-flow variations in detail, it also enabled them to understand detailed differences between variants from this perspective. The transformation costs in Table 7 were estimated based on the characteristics discussed under this and the previous item (Req. 3 and 6). The overall transformation costs of Provop is estimated to be lower, whereas the training costs are foreseen to be higher.
Requirement 7Decision support We evaluate the decision support provided by the approaches in two dimensions: the development of variant models, and configuring the models for process variants. For the first dimension, the team appreciated the support of DDM to identify business and syntactic drivers. In this way, they could better understand their business context. This benefit became even more prominent with the emergence of new drivers at low levels. Provop provides policies to define the base process. However, the team had to update the initial base process prepared according to such policies. Moreover, the team found the option design a challenging task which can also end up in different design alternatives. The approach provided hardly any guidance on how to design them.
For the configuration dimension, the team found it easier to use the Provop base process for the configuration of process variants. Similar change operations grouped in the option list decreased the complexity to generate a variant, and made it easier to configure a variant without much knowledge of the customer. The variation map of DDM does not provide any information on variants, one needs to have specialized knowledge for this task. The team suggested to add the driver names causing variations in related edges of the variation map to alleviate this problem. Overall, this requirement is strongly related with the user needs of establishing a knowledge-base and reuse, as decision support features of the approaches facilitated the conduct of the activities related to these needs. Both approaches were evaluated to outperform the current modeling approach.
Selection guidelines for process variant modeling approaches
Based on the evaluations, the team defined the guidelines in Table 9 for variant modeling approach selection. For each guideline, we refer to its source, the evaluated user needs and requirements that lead to its definition.
The highlight of DDM was its capability to embed all variant information in a single type of artifact, process model, while following standard BPMN notation and rules. The variation drivers based on business context provided a clear mechanism to identify and reuse variants. Based on DDM’s notable properties, guidelines 1, 5 and 8 were identified. DDM may struggle if low-level variations are to be analyzed in detail and if variation starts at the top-level of the process hierarchy. Provop comes forward with its support to discover control-flow variations in fine detail, and accordingly provide a robust configuration and maintenance mechanism. Consequently, guidelines 2, 3, 4, and 6 were identified. These features come with the difficulties in consistently defining and maintaining artifacts other than the process models. Although such challenges may be even more prominent in a hierarchical process structure, Provop may be applied similar to standard BPMN way in those cases. This brings us to the identification of the guideline 7.
To identify these guidelines, in conformance with the participatory action research method, we worked on the problem of selecting a suitable process variant modeling approach in a specific case for two approaches. This work allowed us to obtain first-hand and deep knowledge on the problem. Eventually, the meticulous identification of user needs and requirements and in-depth analysis of the approach applications may show the way for other organizations to implement process variant modeling approaches in their own context. Although these user needs and requirements are not directly generalizable to other organizations, they point out the concerns to focus on for approach selection. Similarly, the guidelines identified here for the two approaches potentially represent key points in variant modeling, which organizations can use to assess their situation and evaluate other approaches as well.