1 Introduction

Quality management helps companies to improve their business processes, performance, and competitiveness (Flynn et al. 1995; Samson and Terziovski 1999). The positive benefits of quality management have persuaded many companies to implement quality management systems (Peris-Ortiz and Alvarez-Garcia 2014). This has led to the emergence of a number of process improvement related quality management methods or frameworks, such as Six Sigma, Total Quality Management (TQM), business process reengineering/management, and relevant standards, such as the ISO 9000 family of quality management system standards (ISO 2015; Evans and Lindsay 2016).

One of the quality management frameworks that has seen a significant increase in usage over the last few decades is Six Sigma. Six Sigma uses a set of quality principles and techniques to minimize the number of defects in a process (George 2002; Pyzdek 2003). It was originally developed and used by Motorola in the 1980s and it has now branched out to many industry sectors (Pande and Holpp 2002; Tjahjono et al. 2010). The objective of Six Sigma is to identify and remove errors in business processes and, thereby, the causes of variability in processes. In order to do so, Six Sigma practitioners often use a project methodology known as ‘Define-Measure-Analyze-Improve-Control’ (DMAIC) (Pande and Holpp 2002).

There is an abundance of research on the use and effectiveness of Six Sigma in several business domains, such as healthcare (Bedgood 2017; Antony et al. 2018), construction (Siddiqui et al. 2016; Karakhan 2017), supply chain management (Madhani 2016; Hong 2017), public sector (Antony et al. 2017), and education (LeMahieu et al. 2017; Lu et al. 2017; Nadeau 2017).

For reducing variations in business processes, Six Sigma often employs statistical methods to quantify changes in process performance. However, the data used for such an analysis is usually collected manually through, for instance, surveys and observations, which makes Six Sigma a costly and time-consuming endeavor (Park and Kang 2016; van der Aalst 2016). Moreover, such approaches often involve subjectivity as they rely heavily on the knowledge of those involved in the analysis, who often reflect and confirm their own view of the processes and relevant problems (Johannsen et al. 2011). It is also difficult to capture all the complexity and variations in process executions using such approaches (Fluxicon 2019).

Information systems and devices record and store large amounts of data (Bücker et al. 2016). Specifically, event data provides records of the operational steps that have been executed in the past. Process mining is an emerging discipline that offers a set of techniques to gain data-based insights from recorded event data and to conduct further analysis in order to support process improvements (van der Aalst 2016). Process mining acts, on the one hand, between computational intelligence and data mining, and, on the other hand, between process modeling and analysis, to serve as an enabling technology for process-oriented quality management methods and frameworks (van der Aalst et al. 2012a, b; Harmon 2018).

The existing literature confirms process mining’s potential to enrich the set of techniques that practitioners can use in Six Sigma programs (van der Aalst et al. 2016; Valle et al. 2017; Garcia et al. 2019). However, current research in this field provides very limited guidance in this direction (van Geffen and Niks 2013; van der Aalst 2016). In particular, a structured method or a guideline is missing that helps organizations to discover how and when process mining techniques can be used to support Six Sigma activities. This leads to the following research objective of this study:

To develop a guideline to support organizations in systematically using process mining techniques aligned with the DMAIC model of the Six Sigma.

In order to address this research objective, we followed the design science research (DSR) methodology (Gregor and Hevner 2013) to develop a suitable guideline as a design artifact, namely Process Mining for Six Sigma (PMSS). Following a structured literature review, we identified as main requirements that PMSS ought to support practitioners in the effective use of process mining techniques in their Six Sigma initiatives. PMSS guides organizations through different stages of process improvement and offers a direct link between different phases of the DMAIC model of Six Sigma and existing process mining activities. To help ensure the utility, PMSS was developed in close collaboration with Six Sigma and process mining experts, both researchers and practitioners.

Following the DSR method, the initial version of PMSS was developed by taking an existing process mining methodology as a basis. The initial version was refined based on the insights gathered from domain experts in semi-structured interviews. In addition, a prototypical tool support, which is integrated into a commercial platform, was developed to complement and further increase the usability of the guideline. The adapted version of PMSS and the tool support were refined iteratively using expert interviews, focus group demonstrations, and usability tests. The validity of the final version of PMSS and the tool support were evaluated through a series of demonstration sessions with both process mining and Six Sigma experts. The results of our qualitative evaluation show that PMSS is considered an easy to use and useful guideline for organizations when performing Six Sigma activities.

The remainder of this paper is structured as follows: Sect. 2 gives a brief background of the Six Sigma framework and the concept of process mining. Section 3 discusses related work on the use of process mining techniques in Six Sigma initiatives. Section 4 presents the research design that we followed when developing, demonstrating, and evaluating PMSS. In Sect. 5, the final version of PMSS, including the tool support, is briefly described. The evaluation of PMSS by means of the demonstration sessions and interviews is discussed in Sect. 6. Finally, in Sect. 7, we conclude with the discussions of contributions, limitations and future research directions.

2 Background

2.1 Six Sigma

The term Six Sigma refers to a set of tools, techniques, and methods which aim to improve the quality of processes within businesses (George 2002; Pyzdek 2003). It was developed and introduced by Motorola in the early 1980s and afterward adopted by many other companies (Tjahjono et al. 2010). In order to minimize the number of defects (this refers to the cases that do not produce a desired outcome), Six Sigma aims to improve business processes in such a way that the standard deviation of a process is so small that any value within six standard deviations of the mean can be considered as non-defective.

The σ in Six Sigma refers to the standard deviation of a normal distribution. The values that lie within a single standard deviation of the mean belong to the one-sigma level. A process that runs at one sigma has less than 690,000 defective cases per million cases – in other words, at least 31% of the cases are handled properly (Pyzdek 2003). At two-sigma level, this value goes up to 69.2%. The higher the sigma level, the lower the defective cases. Motorola eventually settled on the six-sigma level, which indicates that 99.99966% of the cases are handled correctly and thus the process has as few as 3.4 defective cases per million cases.

In order to minimize the sigma of a process to such an extent that the process runs at six-sigma, practitioners can use a myriad of tools and techniques. These tools and techniques are most often applied through the DMAIC project methodology (Define-Measure-Analyze-Improve-Control) (De Mast and Lokkerbol 2012). The steps of the DMAIC model aim to Define the goals of the improvement activity, Measure the existing process to establish a baseline on how the process currently performs, Analyze it to identify improvement opportunities, Improve the process accordingly, and Control it to ensure that the improvement was effective (Pyzdek 2003). These steps are looked at in further detail in Sect. 5.1.

2.2 Process Mining

Process mining has emerged as a research field that incorporates a set of techniques which enable an organization to gain data-based insights into their processes and support process improvements (van der Aalst 2016). The starting point for Process Mining is an event log. An information system controls real-world business processes and records events which are stored in the event logs. An event refers to an activity (i.e., a defined step in a process) which took place at a particular time, and is related to a particular case (i.e., process instance) (van der Aalst et al. 2012b). Additional information, such as the resource (e.g., person or device) executing the activity, or cost incurred in the execution for a single instance of the activity, can also be stored in the event log (van der Aalst et al. 2012b; van der Aalst 2016). The process mining techniques can be used to analyze these event logs and generate fact-based representations of business processes. Three classes of analysis can be distinguished in process mining: process discovery, conformance checking, and process enhancement.

Process discovery takes as input an event log and generates a corresponding process model. Various techniques can be used for extracting a process model from the raw event log (van der Aalst and Dustdar 2012). An example is the inductive algorithm, which discovers a set of sound block-structured process models from any given event log (Leemans et al. 2013). For a more complete overview of different process discovery techniques, the reader is referred to (De Weerdt et al. 2012; Augusto et al. 2019). Process discovery techniques can be used to generate the structure of the process, to find out about the routing probabilities, to determine the most frequent path in the process, and to discover the distribution of cases over paths (Ailenei et al. 2012).

Conformance checking compares a designed process model with the corresponding event log. This comparison shows where the assumed process model deviates from the real-world process as retrieved from the event logs (van der Aalst and Dustdar 2012; Carmona et al. 2018). Domain knowledge is often required to locate and explain these deviations and measure their severity (van der Aalst 2010). Typical conformance checking use cases include finding exceptions from the normal path, determining the degree to which the business rules and regulations are followed, and measuring the level of compliance to a reference model (Ailenei et al. 2012).

Process Enhancement helps to extend or improve an existing process model with additional information about the process present in the event log. Where conformance checking is limited to measuring the alignment of the process and the event log, the goal of enhancement is to improve or extend the process to gain additional insights (van der Aalst et al. 2012b). For instance, process mining tools can extend a model to show bottlenecks, service levels, throughput times, and frequencies (van der Aalst and Dustdar 2012; van der Aalst et al. 2012b; van der Aalst 2016).

3 Related Work

In this section, we provide a brief overview of the methodologies proposed to guide process mining and improvement projects, and present the results of our systematic review of relevant works that discuss the application of process mining techniques in Six Sigma initiatives.

3.1 Process Mining Methodologies

In order to structurally implement the different types of process mining in a process improvement project, researchers have sought to provide guidance by developing process mining methodologies, such as the Process Diagnostics Method (Bozkaya et al. 2009), the Methodology for Business Process Analysis in Healthcare (Rebuge and Ferreira 2012), the Process Mining Methodology for Emergency Room Processes (Rojas et al. 2017), the Life-cycle Model (L*) (van der Aalst 2016), and the Process Mining Project Methodology (PM2) (van Eck et al. 2015). The first three methodologies are to a large extent for the specific needs of the healthcare domain. The L* covers a large set of mining techniques, while focusing mainly on the discovery and analysis of structured processes.

PM2 aims to overcome the limitations of the former methodologies by providing a framework covering the majority of the mining techniques that can serve as guidance for process mining initiatives in diverse business domains. The methodology incorporates the following steps: planning, data extraction, data processing, mining & analysis, evaluation, and process improvement & support. The steps of data processing, mining & analysis, and evaluation, take place in several iterations. Although PM2 provides guidance for the general use of process mining techniques, it does not explicitly consider the use of process mining in a larger context of quality management – in particular, in the context of process-improvement focused management strategies, such as Six Sigma or Total Quality Management (van Cruchten and Weigand 2018).

3.2 Six Sigma and Process Mining

In order to gain an accurate understanding of the state-of-the art research on the works that discuss the application of process mining techniques in Six Sigma initiatives, we conducted a structured literature review (SLR) following the guidelines proposed by Kitchenham and Charters (2007). We applied a keyword search strategy over a set of academic digital libraries to locate relevant studies published until October 2019. Table 1 shows the list of digital libraries in which the keyword search was conducted. The following keyword string was applied to all fields of the publications (i.e., title, keywords, abstracts, and full text where available): “Six Sigma” and “process mining”

Table 1 Digital libraries and search results
Full size table

The initial search resulted in 941 publications in total (including duplicates that appear in multiple libraries). In order to identify the studies that are directly relevant for our research topic, we defined inclusion and exclusion criteria. For the context, we included papers that are published (1) in English, and (2) in an academic journal, conference proceeding, or a chapter in a scientific book. We excluded publications in the grey literature, i.e., those without bibliographic information (such as publication date/type, volume and issue numbers), working papers, white papers, books, or reports. As for the relevance, we included papers that discuss the use of process mining in six sigma initiatives or present a work that shows how process mining techniques were applied in such initiatives.

A thorough review of all publications resulted in 16 primary studies. Figure 1 presents the distribution of these studies by type and year.

Fig. 1
figure 1

Distribution of primary studies by year and type

Full size image

In the list of primary studies, we have distinguished between two categories of studies. The first category of studies endorses the use of process mining techniques in Six Sigma initiatives and discusses the benefits. Ten out of 16 studies belong to this category: (Hess 2006; van der Aalst and Dustdar 2012; van der Aalst et al. 2012b, 2016; Aguirre et al. 2013; Tomašević and Slović 2013; Sebu and Ciocarlie 2014, 2015; Park and Kang 2016; Valle et al. 2017).

The second category of studies are relatively recent publications that present discussions of the application of process mining techniques in Six Sigma initiatives. In addition to discussing the benefits, these studies propose approaches to how mining techniques can be applied within the DMAIC model. Six out of 16 publications are of this type: (van Geffen and Niks 2013; Dogan and Gurcan 2018; van Kollenburg and Wouters 2018; Boersma et al. 2019; Dahlin et al. 2019; Gupta et al. 2019). In the paragraphs below, we discuss these works in more detail.

The study by van Geffen et al. (2013) presents a case study that discusses how process mining techniques can be used to support different phases of the Six Sigma DMAIC model. It shows that using process mining techniques within the DMAIC model in a particular organizational setting can decrease a complete rundown duration from 9 to 12 weeks to 4-6 weeks. Although the study confirms the potential benefits of using process mining as a tool in a Six Sigma program, it does not aim to introduce a structured guideline on how process mining can be methodically used within the phases of the DMAIC model.

In the study by Dogan and Gurcan (2018), the authors discuss the applicability of specific classes of process mining techniques in the DMAIC phases. While the process discovery techniques are considered useful in the Define and Analyze phases, conformance checking is deemed applicable in the Measure and Control phases. For the Improve phase, enhancement techniques are proposed.

van Kollenburg and Wouters (2018) also argue for the usefulness of process mining in continuous and sustainable improvements, and propose applications of process mining in the DMAIC cycle. For the Define phase, the authors argue that the goals and scope of the project can be determined using the real data from process executions. Process mining offers a quick and objective analysis of process performance in the Measure and Analyze phases. It helps to identify opportunities for the Improve phase. Finally in the Control phase, using process mining as an established approach in the daily management of operations gives the opportunity to sustain the improvements.

Boersma et al. (2019) discuss how the discovery, conformance and enhancement techniques of process mining can be used in the DMAIC cycle in the healthcare domain. For the initial phases, they show the efficiency of process mining techniques over qualitative methods, such as interviews. In the Analyze phase, conformance checking is proposed for checking the extent to which the existing processes comply with internal and external guidelines. In the Control phase, process mining is considered useful for checking adherence to the new (improved) processes and identifying deviations. Enhancement techniques that enrich the discovered process model with additional perspectives (such as the time, cost, and resource utilization), are proposed for the Control phase to facilitate in-depth monitoring.

Dahlin et al. (2019) propose a set of steps depicting how process mining can be applied to improve healthcare processes. Although this study does not refer to the DMAIC model, it discusses how process mining can be compared to the process mapping technique – as an approach commonly used in Six Sigma initiatives – and what benefits can be achieved when they are combined. Consequently, the authors call for future research that should investigate how process mining can be integrated into organization-wide quality improvement initiatives.

Finally, Gupta et al. (2019) discuss the role of data analytics techniques – including process mining – in the DMAIC cycle. Similar to the proposition by Dogan and Gurcan (2018), the process discovery techniques are considered useful in the Define phase, while conformance checking is regarded applicable in the Measure phase.

In brief, the existing body of research considers the use of process mining techniques within Six Sigma initiatives as useful and sound, and the linking of such techniques to existing concepts and methods beneficial are found to provide an enriched understanding of processes. Therefore, a structured guideline that designates how process mining can be systematically used along the DMAIC cycle would facilitate the organizational efforts to realize the claimed benefits.

4 Research Design

The main goal of this research is to develop a new artifact that supports Six Sigma practitioners to perform their activities with the use of Process Mining techniques. This artifact is referred to as Process Mining for Six Sigma (PMSS) and consists of three parts: a high-level graphical overview, explanatory text in the form of tables, and tool support.

The design science research (DSR) methodology is well-suited for studies that aim to develop and evaluate such artifacts (Gregor and Hevner 2013). We followed the DSR process proposed by Peffers et al. (2007) that includes the following activities (executed in this nominal sequence): identifying the problem, defining requirements of a solution, designing and developing the artifact, demonstrating the artifact in a suitable context, and evaluating it with respect to a set of criteria. In following this process, we have performed a number of iterations between the design & development, and demonstration. The process that we followed is depicted in Fig. 2.

Fig. 2
figure 2

Research process

Full size image

As a first step, we conducted a literature review on Six Sigma, on process mining methodologies, and on the use of related techniques in Six Sigma initiatives (step 1). Driven by the findings of the first step, the requirements of the solution were identified (step 2). An initial version of PMSS was developed based on these requirements (step 3). Through interviews, feedback on the initial and intermediate versions of PMSS was obtained (step 4), which was later used to iteratively adapt the guideline (step 5). In addition, the tool support was developed for PMSS (step 6) based on the input gathered from the interviews and from the brainstorm sessions we organized with process mining and Six Sigma experts (step 7).

The adapted version of the guideline (v2) together with the tool support was demonstrated to the experts for gathering feedback to further enhance PMSS (step 8). The guideline and tool support were enhanced accordingly (step 9). This was followed by the usability tests with domain experts, focusing on the usability of the tool support (step 10). Based on the feedback gathered, PMSS was finalized (step 11).

The interviews and usability tests took place during the construction of PMSS. Therefore, they were ex-ante evaluations; that is, they involved the assessment of an un-instantiated artifact to show its potential (Venable et al. 2016), (Venable and Pries-heje 2012).

The final version of PMSS was demonstrated to both Six Sigma and process mining experts, who were then interviewed to provide feedback on the utility of PMSS (step 12). To evaluate the utility (i.e., how useful and easy-to-use the practitioners consider the artifact is), they were asked to respond to a questionnaire that involves Technology Acceptance Model (TAM) constructs (step 13). The interviews and survey took place after PMSS was developed; thus, they were ex-post evaluations (Venable et al. 2016).

In the next sub-sections, we provide more detail on the steps of the research design.

4.1 Problem Identification

The literature acknowledges the applicability and potential usefulness of process mining techniques in the DMAIC cycle of Six Sigma initiatives. As we have indicated above, however, the existing literature lacks a method or a guideline that is explicitly dedicated to how these techniques can be systematically used in the DMAIC cycle. This research gap drove the formulation of our research objective as presented in the first section of this paper. This study aims to develop a guideline that shows how DMAIC activities of the Six Sigma initiatives can be supported with process mining techniques, thereby bridging the gap between Six Sigma and the field of process mining.

4.2 Define Requirements of the Solution

To fulfill the above mentioned research gap, we identified the requirements of the solution that can address our research objective (Peffers et al. 2007). Since the DMAIC model is the de-facto methodology used in improvement projects by Six Sigma practitioners (Pyzdek 2003), the guideline should focus on explaining how process mining techniques can be aligned with the phases of the DMAIC model. Furthermore, the activities that can be taken within each step in PMSS should be explained in order to provide a systematic guideline for practitioners. To address these needs, the following requirements are specified:

R1.:

PMSS shall be aligned with DMAIC and show where in the model process mining techniques can be employed

R2.:

The tasks incorporated within PMSS shall be thoroughly explained in order to effectively guide their enactment by practitioners

In order to effectively use process mining techniques in each DMAIC phase, PMSS should take a process mining methodology as a basis and align it with the DMAIC phases. In addition, PMSS should be business-driven to support (relatively) large and complex projects where the domain knowledge is limited (Suriadi et al. 2013). The guideline should also be domain-independent rendering it to be applicable to any process, and should support iterative analyses of processes (Suriadi et al. 2013; van Eck et al. 2015; Dijkman et al. 2019). PMSS should, therefore, fulfill the following requirements:

R3.:

PMSS shall be based on a process mining methodology

R4.:

PMSS shall take business domain knowledge into account

R5.:

PMSS shall be domain-independent

R6.:

PMSS shall support iterative process analysis

Furthermore, since Six Sigma practitioners will use process mining techniques to support their DMAIC activities, a complementary tool support for the guideline and use of these techniques is necessary. Incorporating all functionality within a single platform would help reducing its complexity (Lepofsky 2015) and thus make it easier for Six Sigma practitioners to use process mining techniques. To fulfill this need, we defined the following requirement:

R7.:

PMSS shall include complementary tool support which helps Six Sigma practitioners to use available and appropriate process mining techniques in a single platform

4.3 Develop the Initial Version of PMSS

In Sect. 3, we discuss the process mining methodologies that are proposed in the academic literature to provide guidance in the use of mining techniques. However, these methodologies lack explicit consideration of a larger organizational objective of quality management and also lack an evident link between these methodologies and the phases of the quality management frameworks – in our case, Six Sigma. Therefore, for the initial version of PMSS, we decided to take a process mining methodology with guidance of techniques as a basis and map it to the phases of the DMAIC model of Six Sigma to align it with the phases and objectives of quality management.

For this initial version, we took the process mining methodology of PM2 as a basis, since it was developed as an improvement of former methodologies (van der Aalst 2016), (van Eck et al. 2015). PM2 is domain-independent, has an explicit consideration of the business level requirements, and is designed to support iterative analysis of processes. Hence, we mapped the PM2 activities to the phases of the DMAIC model (as given in Table 2).

Table 2 DMAIC and PM2 activities mapped
Full size table

As a next step, we performed a series of interviews to evaluate (ex-ante) the initial version of PMSS – i.e., the mapping between DMAIC and PM2 – as described in the next section.

4.4 Interviews with Experts

We conducted interviews with 13 field experts, with the objective of investigating how the domain experts would decide to use process mining in the phases of the DMAIC model. A second goal was to evaluate the extent to which the initial version of PMSS was aligned with their views and can effectively provide structural guidance in the use of these techniques.

Conducting interviews with a group that has a wide coverage of expertise in related topics enhances the internal validity of the semi-structured interviews (Gibbert and Wicki 2008). Therefore, the interviewees were selected so that they had expertise in at least one of the two fields, i.e., process mining and Six Sigma, to ensure capturing the views from both fields. We also distinguished experts with respect to their roles in related fields (i.e., Six Sigma practitioner, process mining tool developer, consultant, and C-level/project manager, academic, etc.). The interviewee profile with respect to these roles and their level of expertise in related fields are presented in Table 3.

Table 3 The interviewees and their level of expertise in related fields
Full size table

As depicted in the table, 11 interviewees considered themselves experts in process mining, while 8 were considered either experts or knowledgeable in Six Sigma (note that a number of interviewees had considerable experience and knowledge in both fields). The interviewees had at most 12 and as a minimum 2 years of experience in their main field of expertise.

The interviews were semi-structured; that is, they featured a number of predefined questions as well as room to deviate for open discussions. The questions were driven by a case, which involved a realistic process and a scenario where the interviewee was expected to go through the DMAIC phases for improving the process. While going through an example case, the experts were asked to indicate in detail various process mining techniques that they would consider employing at each DMAIC phase. The final part of the discussions with the experts focused on:

  • whether the PMSS can be useful in practice,

  • whether its components are correctly structured,

  • whether it is complete in terms of the components, and

  • whether the level of granularity in the description of components is sufficient to guide its use.

Based on the feedback retrieved from the experts, PMSS was updated and refined as a second version (v2). This version incorporated a high-level visual overview of adapted PM2 activities mapped to DMAIC phases and explanatory text describing what exactly each activity entails in terms of the process mining techniques and tasks. (We describe the final version of PMSS in Sect. 5.) The feedback from experts gathered at this stage also served as a point of departure for the design and development of PMSS tool support as discussed in the next sub-section.

The interviews were recorded in order to reduce the risk of their subjective interpretation (Blumberg et al. 2008). This made it possible to transcribe interviews into text without leaving certain claims unnoticed, and provided the opportunity for the interviewees to review and approve the points made (Dul and Hak 2008).

4.5 Design and Development of the Tool Support, and Brainstorm Sessions

The feedback from the interviewees allowed us to identify the classes of process mining techniques and particular graphical indicators that can be useful at each phase of the DMAIC model and which formed the requirements to be fulfilled by PMSS tool support. To address these requirements, we iteratively designed and developed the tool support, and conducted a number of brainstorm sessions with 2 experts on process mining and data visualization to ensure that the tool support was relevant and addressed the requirements set forth. The choice for the brainstorming method was motivated by its recognized potential to generate ideas, find solutions to specific problems, and support conceptual user interface designs by generating alternatives (Wilson 2013).

The first expert had 12 years of experience in process mining and 17 years in data visualization, while the second expert had 4 years of experience in data visualization and 3 years in process mining. Both experts were also knowledgeable about Six Sigma programs and the DMAIC model. Together with the experts, we identified the features and related user interfaces that should be supported by the tool. In particular, we identified the process mining techniques that should be supported and should be made available to the users at each DMAIC phase. The experts were involved during the development to help ensure that the requirements originated from the expert interviews in the previous step were addressed, and that the tool support would be useful for its intended audience.

There are a number of process mining tool platforms available for use in practice and research (van der Aalst 2016). Therefore, instead of developing a new tool to support specific process mining techniques, we took an existing platform – ProcessGold, that features several process mining techniques – and extended it with additional (functional and visual) layers to support PMSS. The ProcessGold platform (https://processgold.com/en/) is an enterprise solution for process mining and can be used to develop web-based process mining applications. That makes it an appropriate platform for the purpose of this research. In Sects. 5.2 and 5.3, we elaborate the tool support with a number of user interface elements that present relevant and applicable graphical indicators whose development was based on the use of the process mining techniques.

4.6 Focus Group session with Experts for Demonstration and Feedback

PMSS (v2) and the first prototype of the tool were demonstrated in a focus group setting to a group of experts in order to gather feedback about how PMSS and the prototype tool can further be improved. The group consisted of 24 experts with a range of years of experience and fields of expertise in relation to process mining, Six Sigma and process improvement in general. The majority of the group members (22) were practitioners in the process improvement field (i.e., process mining tool developers and sales representatives, process improvement consultants and C-level managers) and 2 were academics with expertise both in process mining and Six Sigma. The 2-h session with the group resulted in several points of improvement that were incorporated in a new version (v3) of PMSS and its tool support.

4.7 Usability Tests with Domain Experts

To assess the usability of PMSS and the tool support, we conducted usability tests with 12 practitioners. A usability test aims to gather a better understanding of how real users interact with a product and to improve the product based on the results (Nielsen 1994). It aims to identify areas where potential users struggle with a product and thus provide feedback for improving its design. Several types of usability tests have been distinguished in the literature, and we chose to conduct informal usability tests, as they are suitable for relatively small-scale prototypical implementations (Barnum 2011). These informal tests are in the category of formative usability testing and provide the product developers with a list of findings to analyze and fix, in addition to features that users liked.

Among the 12 participants of these tests, 5 were Six Sigma experts (black belts), while the others had expertise on process mining and improvement in general. The average number of years of experience of these experts were 4 years for Six Sigma experts (maximum of 14, and minimum of 1 year) and 7 years (maximum of 17, and minimum of 2 years) for process mining and improvement experts.

Aligned with the usability test protocol (Barnum 2011), the participants – in individual sessions – went through a task-based scenario that was created for these tests. The scenario incorporated a realistic case of a process improvement setting with specific steps that implicate the use of PMSS and its tool support. Adopting a think-aloud process, we encouraged participants to share their thoughts while going through the scenario and performing the steps using PMSS and its tool support.

By going through the scenario, the participants provided feedback on the elements of PMSS and elaborated on its usability. Based on this feedback, PMSS and the tool support were finalized (v4). The feedback from each participant was recorded and incorporated in the functionalities to be supported by the tool and in its user interfaces. We present the results of these efforts in the description of the tool support (Sect. 5.2) and its demonstration (Sect. 5.3). The next section focuses on the final version of PMSS.

5 PMSS and the Tool Support

In this section, we first present an overview of PMSS and discuss the activities that need to be taken for each guideline step. In Sect. 5.2, we briefly introduce the tool support and we illustrate the application of PMSS on an example business case in Sect. 5.3.

5.1 Elements of PMSS

A graphical representation of PMSS is shown in Fig. 3. The bottom part serves as a legend. The white blocks indicate the steps that are taken in order to conduct a process improvement project. As mentioned before, these steps are based on the process mining methodology of PM2 (van Eck et al. 2015) (which fulfills the requirement R3 as presented in Sect. 4.2). The goal of PMSS is to guide Six Sigma practitioners in using process mining to support the DMAIC of Six Sigma, and potentially make it more efficient/effective in order to perform related DMAIC activities. To do so, the steps of PMSS have been aligned with the phases of the DMAIC model (fulfilling R1). However, as a result, the steps no longer flow in the same order as in PM2.

Fig. 3
figure 3

Process mining for Six Sigma (PMSS)

Full size image

As depicted in Fig. 3, PMSS consists of eight steps in the main flow that are explicitly positioned within the five phases of DMAIC: (Define) Planning, preliminary data preparation, exploratory mining & analysis, (Measure) data preparation, (Analyze) explanatory mining & analysis, (Improve) process improvement, and (Control) monitoring and evaluation. These steps and phases are depicted in sequence with a feedback loop connecting the end phase to the first to represent the cyclic nature of the DMAIC model. However, this depicts an ideal flow and PMSS presumes iterations between steps and phases (fulfilling R6) to reflect the case in real-life Six Sigma initiatives. Inheriting the properties of PM2, PMSS is designed to be applicable for the improvement of processes in any business domain (fulfilling R5).

The guideline contains a resource layer showing the roles that act as the leading role for each step. Three roles are at the core of PMSS: Data analysts, process analysts, and business user. Briefly put, a data analyst collects, processes, and uses the data related to process executions. The process analyst is an (IT) professional specialized in analyzing business processes and workflows with the objective of finding opportunities for improvement. The business user is an abstract role representing those that are knowledgeable of organizational processes, such as business unit managers, process owners, change managers. Each PMSS step is assigned to one of these roles as being responsible (leading), although the work at each step requires a team with members representing all three roles which are closely collaborating and making use of the expertise and viewpoint brought along by each one of them.

Table 4 provides more details regarding each PMSS step with a brief list of corresponding inputs, outputs, activities, and the responsible role (fulfilling R2). (Further details regarding these activities of each step is available at: https://goo.gl/nBm3e5). Section 5.3 demonstrates a number of steps alongside an example business case.

Table 4 The resource responsible, the input, the output, and the activities of each step
Full size table

In the paragraphs below, we elaborate more on the PMSS steps that have been defined and enhanced based on the feedback gathered from field experts along the DMAIC phases. (Where necessary, we explicitly refer to the feedback gathered from the experts we interviewed to justify specific design decisions made for PMSS).

5.1.1 Define Phase

The Define phase incorporates three steps. In the Planning, the business goals and questions are identified, the processes to be analyzed and improved are selected, and a project team is established. In order to help identify appropriate business goals and accompanying business questions (that are related to one or more aspects of business processes, i.e., quality, time, resource, costs), the preliminary data preparation is performed, so that a brief overview of the process can be gathered on the event data in the exploratory mining & analysis step. However, before considering the event data, it is important to have clear business goals and questions that you expect to be addressed in the following steps. If there is no clear orientation for the improvement project, it is likely to fail (as maintained by PMC1 and PMM2; please refer to Table 3 for abbreviations).

The input for the planning step comprises a broad spectrum of information regarding the organization, such as its business processes and major issues faced regarding these processes [PMD1], a selection of processes that are known to contain issues and require attention for improvement [PMC3], and first insights gained from exploratory mining and analysis. As a result of a set of activities in this step, the business goals and questions are defined, processes to be improved and supporting systems are identified, the project team with members that bring different perspectives to the process execution is composed, and a preliminary description of the business case is defined. The leading role in this step is the business user acting as the project owner and contributing with the domain knowledge and relevant context information (satisfying R4) to ensure that the initiative starts in the right direction [PMC1, PMS1]. Although these activities in the planning step are depicted in an ideal sequence, they are iterative as their actual execution unfolds.

The objective in the preliminary data preparation step is to provide data for the coming exploratory data analysis, and in turn to facilitate the planning step in better identifying business problems and building a business case for the initiative. As depicted in Fig. 3, this step is a special form of data preparation. At the Define phase, process mining can provide insights into the current process-related problems and complement the traditional techniques applied in this phase. Therefore, there is a need for a set of quickly-performed data preparation activities to provide sufficient input for the subsequent exploratory mining and analysis. Hence, we differentiate this set of preliminary data preparation activities, which would quickly offer data that can allow process analysis to be conducted with a wider lens, and the data preparation activities performed in the Measure phase, which aim to provide enriched data regarding KPIs that are focused on the identified business problem and scope to enable more in-depth explanatory mining & analysis for performance measurement.

The (preliminary) data preparation is comprised of three sub-steps: (preliminary) data extraction, where the process execution data is extracted from selected information systems; processing, where the data is prepared so that it can be stored in an event log and can be loaded into a process mining tool; and verification, where the data is reviewed for correctness and validity to ensure that it correctly depicts the actual process. In the data processing sub-step, the good practices include the definition of an audit trail of the changes made to the data and the description of the data types in the log [PMM1]. At this step, the leading role is the data analyst [SSP1 and SSP2].

The exploratory mining & analysis step is a special form of mining & analysis that is carried out with exploratory motives to support the identification of process-related business problems. It takes the data originating from the previous step as input and incorporates three families of techniques (that we discussed in Sect. 2): process discovery, conformance checking and process analysis (covering also enhancement techniques). In addition to process mining techniques, the process analysis incorporates statistical methods and techniques (such as Pareto analysis, histograms, descriptive statistics (Pyzdek 2003)). Such techniques are often used in Six Sigma initiatives to support traditional process analysis. The importance of traditional process analysis alongside process mining is stressed by multiple studies in the process mining field in which process analysis makes up for a large share of the actual reasoning (Berger 2017; Ryu et al. 2017; Smith and Day 2017). Note that the mining & analysis does not enforce any predefined order for the use of process discovery, conformance checking, and process analysis techniques.

The feedback arrow from the exploratory mining & analysis step to planning indicates that the insights gained from the exploratory analysis can serve as input for establishing the business goals. The process analyst ensures that the activities in this step are properly carried out (van Eck et al. 2015).

The three steps in the Define phase, planning, preliminary data preparation, and exploratory mining & analysis can be conducted iteratively until the business goals for the improvement project are clearly defined. The define phase also clarifies which additional execution-related information should be extracted from the information systems.

5.1.2 Measure Phase

Based on the specification of the required additional information to be extracted from the information systems, in the data preparation step of the Measure phase, process data and metrics relevant to the business problem are retrieved, a baseline is established, and current process performance is determined. This is achieved through the sub-steps of data extraction, processing and verification. The objective is to extract additional process execution-related data from the information systems of the organization, to process it to obtain a clear, filtered, and enriched event log, and to verify that the data is correct and valid and can be used as input to the next step, i.e., the explanatory mining & analysis. The leading role for this step is the data analyst [SSP1 and SSP2].

5.1.3 Analyze Phase

In the Analyze phase, a closer look is taken at the data through explanatory mining & analysis. In this phase, the team guided by the process analyst performs detailed analyses of the data with the aim to detect potential causes for the problems identified in the previous phase (Define), and identify improvement opportunities that can be acted upon in the next phase (Improve).

The explanatory mining and analysis step comprises three sub-activities: process discovery, conformance checking, and process analysis (as indicated in the legend by the block titled mining and analysis in Fig. 3). Although the techniques used in this phase are the same as those used in the exploratory mining & analysis step, the analyses are driven by identified business questions aiming to address the improvement goals. In turn, the level of analysis at this step is deeper and intense. The main input constitutes the event logs created in the data preparation step, the audit trail, the data description, and business problems.

5.1.4 Improve Phase

The opportunities identified in the previous step are addressed in the Improvement phase. In this phase, process mining can be useful in detecting the likely impact of alternative improvement actions and in selecting those that are likely to bring the highest impact when implemented.

As the actual implementation of the process improvement action takes place on the business side, the business user takes the leading role in this step, and implements and manages the required changes in the processes [PMC1, PMC2, PMM1, PMM2] taking the improvement opportunities found in the previous step as input. The deliverable of this step is subsequently improvements or so-called process changes towards business goals.

5.1.5 Control Phase

In the Control phase, the process mining techniques can be used to monitor the predefined process performance indicators to help evaluate if the implemented changes have yielded the expected results. In this phase, we can distinguish two steps: monitoring, where the process execution is monitored with respect to the performance indicators, and evaluation, where the organization determines if the impact of the changes depicts the expected results and whether the process has been successfully improved. As a result of monitoring, new findings may also emerge leading to new business problems and in turn a re-initiation of the DMAIC cycle. As we mentioned above, although Fig. 3 depicts a single feedback loop from the Control to the Define phase, PMSS assumes iterations that allow traversing between any prior phase.

In the monitoring step, the business user (e.g., the change manager) is the leading role to identify and prioritize the process performance indicators (and metrics) that are of importance to the organization and thus should be monitored [PMS1]. In addition, since it is important to have domain knowledge at this point and since the evaluation happens on the business side [PMM2], the business user (e.g., improvement project leader) is responsible for evaluating the values of the indicators that are monitored in order to assess if the process improvements have yielded the expected result [PMM2].

5.2 The Tool Support

As indicated in R7 (Sect. 4.2), a complementary tool support that allows Six Sigma practitioners to use available process mining techniques in a single platform is important for the usefulness of PMSS. As described in Sect. 4.5, we developed the tool support for PMSS as an extension to the ProcessGold platform. One of the unique features of the ProcessGold platform is its ability to add tags to cases in order to group them according to certain relevant properties. For example, in the ProcessGold platform the cases with certain issues can be tagged so that they can later be located using filters for the tag. This has made it a suitable candidate platform for our tool support.

For each of the phases of the DMAIC model, one or more dashboards have been created based on the draft interface sketches developed during the brainstorm sessions with potential users and experts (Sect. 4.5). These sketches were refined during the focus group sessions and usability tests performed with domain experts (Sects. 4.5 and 4.6) and during the demonstration and evaluation sessions that we discuss in the next section. Each dashboard contains visualizations that support the initial analysis relevant for each phase. When further analysis is needed, the visualizations link to that part of the application that supports more elaborate analyses.

As an example, a screenshot of the dashboard created for the Define phase is presented in Fig. 4. Each dashboard element (numbered 1–3 in the Fig. 4) addresses a certain need to fulfill a particular user objective that we gathered through the brainstorm sessions. For instance, as maintained by all experts except one, the most prominent mining technique that can be used for exploratory mining & analysis at this phase is process discovery, which reflects how the actual process execution took place and what bottlenecks can potentially be identified in the process. Therefore, to check for deviations in the process, the first dashboard element in Fig. 4 (#1) is placed showing a discovered process graph tuned with respect to the number of execution paths.

Fig. 4
figure 4

Screenshot of the dashboard for the define phase

Full size image

As also supported by 4 experts (PMD1, PMD2, PMC1, PMC2), particularly when the goal of the project is to improve the performance of a business process, process analysis (or enhancement) techniques can be used to analyze the performance through indicators, such as throughput time, waiting time, or total cost. Hence, the second dashboard element in Fig. 4 (#2) presents the average completion times of different case types in order to check if cases with certain properties in common have different average processing times. In relation to that, the third element (#3) shows the extent to which certain activities or resources contribute to that average. From these dashboard elements, it is possible to navigate to that part of the application that supports more elaborate analyses of the information shown in the charts.

A detailed explanation of the tool support and reasoning regarding specific design decisions is available at: https://goo.gl/LPZJpC.

5.3 Demonstration by Means of a Business Case

In this section, we provide a brief demonstration of the PMSS and tool support by going through a real-life business case that involves an organization (ABC Inc.) adopting Six Sigma quality management framework and following the DMAIC model.

In the Define phase, ABC performs the planning, preliminary data preparation, and exploratory mining & analysis activities. For the planning, ABC considers a number of complaints that it has received from its suppliers about invoicing, and conducts short discussions with a number of employees that are involved in the related process with the main objective of defining the scope for the initiative and the goals that it aims to attain after improvement actions are implemented. As result, ABC considers focusing on its invoicing process, which is deemed to suffer from long processing times. It sets up a team involving a number business users, internal data and process analysts, and a team leader appointed as the project/process owner.

The planning is facilitated by the preliminary data preparation and exploratory mining & analysis activities in order to provide a better understanding of the business problem in the invoicing process. Relevant data about the invoicing is extracted from ABC’s current enterprise information system, prepared and loaded into the PMSS tool support for the subsequent exploratory mining and analysis. At this stage, the data requirements are kept at a minimum to expedite the mining and analysis. (For the sake of brevity, we do not show the screens of the application components that show the loading of the data).

Figure 4 (given in the section above) shows the screen for the Define dashboard of this business case. The first screen element shows the process graph for the invoicing process. At the bottom of the second element, we can see that the average processing time is 4 days when all 1855 process execution instances (cases) are considered. The avg. processing time is highest for medium invoice cases (9 days) and lowest for employee declaration cases (2.3 h).

Checking the process graph, the team notices considerable repetitions of certain activities as a sign of excessive rework (i.e., loops and self-loops in the process graph) which potentially lead to longer processing times. For further analysis, the team selects one of the arrows that indicates a repetition of a certain activity (i.e., the activity ‘final check of invoice’). Figure 5 shows the updated dashboard screen when the process instances (cases) with repetition of this activity are selected by the user. As can be seen in the figure, the average processing time for the cases where this activity is repeated is twice as long (8 days) as the average time when all cases stored in the tool is considered.

Fig. 5
figure 5

Process instances with repeated activity selected

Full size image

Performing similar analyses on other activity repetitions (and possibly a more elaborate analysis of the information shown in the charts) confirms the initial considerations regarding the potential cause of long processing times. Having gained an initial understanding of how the process has been executed, its performance and potential problems, the team closes the Define phase and moves onwards to the next DMAIC phase.

At the Measure phase, the data analyst takes the lead to extract additional process execution-related data from ABC’s enterprise system for enriching the event log (e.g., with data from an extended period of time and other type of cases), and verify that the data is correct and valid. Figure 6 shows the dashboard for the Measure phase. The data analyst uses the dashboard to check if the data loaded into the application is correct (for instance, by comparing the numbers of cases, activities, and users with those in ABC’s enterprise system from which the data has been extracted). Furthermore, the open and closed cases are rechecked to ensure that the cases started and closed as expected.

Fig. 6
figure 6

Measure dashboard

Full size image

At the top right of Fig. 6, the number of activity repetitions per case is shown. The fact that, on average, each activity in a case is repeated 1.14 times, provides a further suggestion regarding the cause of the problem.

When the team has verified the correctness of the data, they move to the Analyze phase. Figure 7 shows the dashboard for this phase, which consists of two views: the overview and statistics. In the overview (as shown in Fig. 7), the cases that are problematic are compared to the other cases. In the case of ABC’s invoicing process, problem cases refer to those process executions that feature activity repetitions, and, in turn, rework. Hence, the overview dashboard shown in Fig. 7 is tailored to reflect the Six Sigma waste problem at hand, i.e., rework (as opposed to, for instance, inefficiency).

Fig. 7
figure 7

Analyze dashboard – overview tab

Full size image

The overview dashboard features a process graph (element #1 in Fig. 7) to determine if the paths of the problem cases with rework (in purple) deviate significantly from the paths of other cases (in orange). In addition, the team analyzes a number of graphs each differentiated for problem cases and other cases. They analyze the average processing times for different cases (#2), the resources involved in the process to check if a particular resource is a major contributor to the problem (#3), and the average number of times an activity occurs in the cases (#4). The graph in dashboard element #5 shows the progress of ‘defects per million cases’ (DPMO) – a metric that is of particular importance for the Six Sigma initiatives. The graph in element #6 shows the workload, i.e., the percentage of cases that are considered problematic with regard to the repetitions.

The team sees that the average processing time (#2) is higher for the problem cases than for the other cases for majority of the case types. The same situation holds for the number of activity repetitions per case (#3). The DPMO metric, which is expected to be 3.4 for Six Sigma, is considerably higher – around 175 k.

Furtheron in the analyze phase, the team performs additional investigations and analyses supported by a number of dashboard elements and related graphs with the aim to locate the causes that contribute most to the problem, and hence to decide on the improvement alternatives to act upon in the improve phase.

In order to keep this section concise, we present the further demonstration steps on the following page: http://tiny.cc/pqesdz.

6 Evaluation of PMSS

In Sect. 4, we described how the initial and intermediate versions of PMSS were evaluated ex-ante during its development through expert interviews (Sect. 4.4), brainstorm sessions (Sect. 4.5), a focus group session (Sect. 4.6), and usability tests (Sect. 4.7). The ex-post evaluation of the final version of PMSS guideline and tool support focused on its validity and utility.

In order to collect evidence for the validity of the artifact (i.e., the extent to which it is applicable and can be used for its intended purpose (Gregor and Hevner 2013)), we held demonstration sessions with domain experts and interviewed them to obtain their opinion on the instantiated artifact. Next, we asked the experts of the demonstration sessions to respond to a questionnaire to evaluate PMSS’ utility, i.e., how useful and easy-to-use they considered the guideline and tool support to be for Six Sigma initiatives. In the sub-sections below, we elaborate on the demonstration sessions and the conduct of the survey, and discuss our findings.

6.1 Demonstration Sessions and Interviews for Validation

The demonstration sessions (step 12 in Fig. 2) and follow-up interviews were conducted with 11 experts. Table 5 shows the profile of the experts with respect to their roles and level of expertise in related fields. Five experts were certified (black-belt or master black-belt) on Six Sigma, and (at least) knowledgeable about process mining. The remaining six were process mining experts, four of them also with considerable expertise in Six Sigma. Note that seven experts in this list were also participants of the first interview sessions (which we describe in Sect. 4.4 and list in Table 3). Similar to the case for the initial set of interviews, we aimed at choosing experts that have different roles in Six Sigma initiatives and/or process mining projects in order to enhance the internal validity of the inferences we obtain from the follow-up interviews (Gibbert and Wicki 2008).

Table 5 The experts in the demonstration sessions and their level of expertise in related fields
Full size table

In the first part of the demonstration sessions, we presented and discussed the final version of PMSS and tool support. After a brief presentation and discussion of the PMSS visual guideline (Fig. 3) and how it should be interpreted, the details regarding each step and phase (i.e., Table 4 and additional details that are provided as external sources for this paper at https://goo.gl/nBm3e5) were discussed. Next, the tool support was demonstrated by going through a realistic case scenario, and participants were interviewed for their opinion on the method and tool support regarding its validity and completeness. During the demonstration sessions and follow-up interviews, experts also suggested minor changes for the guideline and tool-support, which were incorporated in the final version.

The experts displayed an agreement that a standard operating procedure as presented in the guideline is useful to help Six Sigma practitioners to perform their activities in an improvement project more effectively and efficiently with the help of process mining. Below, we include a number of quotes from interviews.

Six Sigma practitioners analyze in a very structured way and also want to know how process mining can contribute to that. So, when we are able to present that, in a structured and predefined way, then I think that those people will benefit from that. They will also be convinced of process mining in general. Therefore, I think it’s quite useful to have it. [PMC3]

I think it’s definitely useful. It helps people to better understand what they have to do everywhere and it guides them through it. I think that guidance is very very important. [SSP5]

Another point of agreement among the majority of experts was the guideline’s ease of use. They indicated that the guideline might be hard to grasp at first sight – particularly for those new to these fields -, but that the level of granularity and complexity engrained in the guideline is needed to ensure usefulness and still remain relatively easy-to-use.

Everything can always become much easier. But, easiness will also lead to the loss of some functionality as well. I’m not sure if I’m, at this point, willing to give that up, [PMM2]

I think the guideline is easy to use, it’s a bit overwhelming at the beginning maybe. But, you definitely need all the things that are in there. There is nothing that I would leave out when I look at it. [PMS1]

Furthermore, the experts indicated that they would recommend PMSS to Six Sigma practitioners. Not only do they think that this structured way of conducting process mining is useful for Six Sigma practitioners, but that it can be useful for organizations that have not adopted Six Sigma principles.

(This) is a structured way of doing Six Sigma using process mining. These are two fields that can really benefit from each other. [PMS1]

I’d definitely recommend the guideline to Six Sigma practitioners. Also to non-Six Sigma guys. …. guys who are not that familiar with Six Sigma can make use of it. That’s quite nice I think. [PMC3]

The experts also indicated a number of points for improvement that we accepted as tasks for future work. The main point of criticism of the guideline was the lack of clarity regarding the relation between process mining techniques and the traditional Six Sigma tools. It is argued that the way the process mining techniques can be used in combination with the traditional Six Sigma tools should be made more explicit. Many Six Sigma experts commented on this point:

From a master black belt perspective, I still need to integrate your guideline in the traditional (Lean) Six Sigma guideline because I have to train and coach my black belts in the entire process. For me, they cannot be separate. [SSP1]

In addition, the experts suggested incorporating additional (statistical) traditional Six Sigma techniques into the tool support, such as the distribution plots and control charts, which would help making it more useful and more likely to be embraced by Six Sigma practitioners. Supporting different perspectives with respect to common process problems was also brought up as another point of improvement regarding the tool support. Instead of having generic dashboards applicable to any kind of process problems, dashboards that are tailored to a specific process problem (e.g., rework) could be generated:

The guideline is self-explanatory and the tool support is very easy to use. With the addition that it would be even easier to use if you added some templates on traditional lean analysis: waiting time, lead time, rework, etc. [SSP1]

6.2 Survey

The experts who joined the demonstration sessions and were interviewed, were asked to participate in a short survey to express their opinion on the utility of PMSS (step 13 in Fig. 3). From the 11 experts that were interviewed, 9 participated, which resulted in a participation rate of 82%.

The questionnaire was composed by taking as a basis the Technology Acceptance Model – TAM (Davis 1989; Venkatesh and Davis 2000). The TAM and its derivatives, e.g., (Venkatesh et al. 2003), have been used as a theoretical basis for several empirical studies in the information systems field, including the acceptance of IS methods and models (e.g., Turner et al. 2010; Stojanov et al. 2015; Schriek et al. 2016; Turetken and Grefen 2017; Dikici et al. 2018; Turetken et al. 2019). The original TAM has three primary constructs: perceived ease of use, perceived usefulness, and intention to use (Davis 1989). Perceived usefulness refers to users’ perception of the utility of the design artefact in providing gains to its user (Venkatesh et al. 2003). Perceived ease of use refers to “the degree to which a person believes that using a particular design artefact will be free from physical or mental effort”. Finally, intention to use can be defined as the extent to which a person intends to use a particular design artefact. Intention to use is the most proximal antecedent to the artefact use and believed to be determined by perceived usefulness and ease of use (Davis 1989; Venkatesh and Davis 2000).

The constructs of TAM are operationalized using multiple indicators, and their reliability and validity have been rigorously evaluated (Davis 1989). In line with (Venkatesh and Davis 2000), we used 4 indicators for perceived usefulness and ease of use, and 2 for intention to use. To accommodate this research, the wording of the indicators (items) were modified in accordance with the approach followed in (Moody 2003). In addition, 3 statements were negated in order to prevent monotonous responses. The participants were asked to express their level of agreement with each statement on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree).

The questions and the experts’ responses are presented in Fig. 8. The results indicate a positive view of the experts towards the measured constructs, i.e., perceived usefulness (Q1–Q4), perceived ease of use (Q5–Q8), and intention to use (Q9–Q10).

Fig. 8
figure 8

Results of the survey about perceived usefulness, perceived ease of use, and intention to use of PMSS

Full size image

The positive tendency regarding the ‘perceived usefulness’ statements indicate that the experts considered the guideline to be useful. Particularly the indicators (items) regarding the perceived usefulness link closely to the objective of our research study, and the majority of experts indicated a positive view on these indicators and agreed that PMSS can support organizations that adopt Six Sigma principles to perform their activities more effectively and efficiently with the help of process mining techniques.

The positive outlook also held for the ‘perceived ease of use’. The majority of the experts believed that the guideline was easy to use and that they would be able to become skillful at doing so. The responses to the ‘intention to use’ indicators also showed a positive outcome. However, only few experts seemed to ‘strongly agree’ with the statements. Furthermore, the experts mostly had a neutral opinion on preferring the guideline over another approach of using process mining techniques to support Six Sigma related activities. Only a single expert among the expert group exhibited a relatively negative attitude to the guideline and indicated this in the responses. However, overall, the responses of the experts indicate a generally positive attitude towards using PMSS.

7 Conclusions

Six Sigma focuses on improving business processes by statistically quantifying process performance changes, but data used for such analyses is typically collected manually, which makes Six Sigma a costly and time-consuming endeavor (Pyzdek 2003; van der Aalst 2016). In the past few years, process mining has proven to be a useful technique to conduct process analysis in improvement projects in a potentially time-efficient way (Garcia et al. 2019). Hence, process mining can serve as an important support technology for process improvement frameworks, such as Six Sigma (van der Aalst and Dustdar 2012; van der Aalst et al. 2012b; van der Aalst 2016; Harmon 2018).

Despite the agreed potential, there is no research on how process mining can be systematically incorporated into the DMAIC model of Six Sigma (van Geffen and Niks 2013; Sebu and Ciocarlie 2014; van der Aalst et al. 2016; Valle et al. 2017). No structured guideline exists that specifies how and when process mining can support Six Sigma related activities and lead to a potentially more efficient and effective performance of these improvement activities. This research aimed to fill this research gap by developing PMSS and its tool support.

This work has implications for both research and practice. It not only extends the body of knowledge in the fields of process mining and Six Sigma, but also helps to close the gap between them, thereby contributing to the broad field of quality management. It is an initial and explicit step to bridge these fields, which will contribute to future research regarding the design and development of process mining tools and techniques that are tuned to the needs of Six Sigma practitioners. This work contributes to the process mining research with an attempt to structure its application, and therefore paves the way for exploiting the capabilities and potential of process mining algorithms, tools and methods that have been widely researched over the last decade (Thiede et al. 2018).

This works contributes also to the practice of DMAIC-based process improvement. PMSS offers Six Sigma practitioners a guideline that extends their existing standard operating procedure and will potentially increase the efficiency and effectiveness of process improving efforts in their organizations. PMSS can also serve as a generic process mining methodology for organizations willing to follow a structured approach for improving processes with the help of process mining techniques. The tool support that has been developed using a commercially available enterprise solution and its qualitative evaluation with domain experts demonstrates its potential to be deployed as a support for Six Sigma practitioners. Recent initiatives in practice prove the importance of using process mining to provide Six Sigma practitioner with new perspectives and tools to find root causes quickly (Fluxicon 2019; Gartner 2019).

7.1 Limitations and Future Work

Our work has a number of limitations which also drive our future work. PMSS has been ex-post validated through interviews with domain experts, to whom the artifact (PMSS and tool support) was explained and demonstrated with an example business case. After the demonstrations, the experts were invited to indicate their opinion on the validity and utility of the artifact. Although this method of evaluation is valid and reliable, it would be considerably more conclusive if PMSS and the tool were used by the experts in real-life settings to perform Six Sigma activities. This would provide a stronger confirmation of the validity and utility of the artifact. Our future work will incorporate these evaluation techniques that involve applications by practitioners in real-life settings to improve the artifact and strengthen the conclusions regarding its validity and usefulness. Similarly, it will include performing controlled experiments in real-life business settings with groups of practitioners to test the tangible influence and value of using the PMSS over not using it at all.

The Six Sigma experts that we interviewed indicated that the relationship between the process mining tools and techniques and the traditional techniques used in Six Sigma initiatives (e.g., distribution plots, control charts) can be further clarified and potentially integrated. There is a need to explicitly describe how the tools and techniques of process mining can complement those that are commonly employed in DMAIC phases, and to incorporate this structure into the standard operating procedure of the DMAIC model. This research work has taken process mining as a point of departure and incorporated it into the DMAIC phases. Future work should also proceed from the Six Sigma’s perspective, and aim at exploring how and when traditional tools and techniques of Six Sigma can be complemented and supported by process mining tools and techniques. Similarly, some of these Six Sigma techniques can be combined in such a way that together they make process mining insights more comprehensive for Six Sigma practitioners. In addition, PMSS is currently intended as complementary to the DMAIC toolkit, while future work can also consider incorporating traditional tools and techniques within PMSS to support all DMAIC activities.

The tool support that we developed is prototypical and suffers from the limitations that are described above (e.g., limited support for traditional Six Sigma techniques). Hence, it requires further work to streamline and improve the functionalities offered. Several improvement points were raised in expert interviews that provide valuable input for future work. For instance, enhancing the dashboards with respect to certain types of business problems (e.g., increasing throughput, increasing resource utilization) was considered to be a useful feature for the tool support. The current dashboards purposely have generic designs and are not designed to support custom elements for a particular business problem type. Allowing users to select a particular problem to focus on for improvement and populating the dashboards with elements that are relevant to that particular problem is considered as a useful feature for the new version of the tool.

Finally, we used a commercially available enterprise process mining solution to host the tool support and made used of the functionality available for process mining within a single platform. However, the developed guideline and related concepts are general and can act as a basis for the implementation of a tool support in other platforms. Hence, for future work, the constructs on which the dashboards are created can be further generalized so that it becomes possible to use publicly available process mining platforms, such as ProM (van der Aalst 2016).