Keywords

1 Introduction

At the turn of the decade, the logistics of operations in hospitals and healthcare centers have been severely disrupted worldwide by the COVID-19 pandemic. Its impact has been profound and damaging in all aspects of life, but in no context it has been more damaging than in healthcare: the safety and well-being of physicians and medical personnel, the supply chain of drugs and equipment, and the capacity of hospitals were all challenged by the pandemic.

One of the most critical points for healthcare systems involved in the treatment process is the management of COVID-19 patients needing acute and respiratory care. Therefore, healthcare organizations are increasingly pushed to improve the efficiency of care processes and the resource management for such category of patients. One way to attain such improvement is to leverage historical data from information systems of hospitals. These data can be then cleaned and analyzed, to individuate non-compliant behavior and inefficiencies in the care process.

The aim of our work is to analyze the care process for the COVID-19 patients treated at the Intensive Care Unit (ICU) ward of the Uniklinik Aachen hospital in Germany, in order to identify divergences or anomalies within the process. To do so, our work intends to develop an executable process model representing the clinical guidelines for the treatment of COVID-19 patients and evaluate the adherence of the observed behavior (recorded by the information system of the hospital) to such guidelines.

The STAKOB guidelinesFootnote 1 (“Ständigen Arbeitskreis der Kompetenz- und Behandlungszentren für Krankheiten durch hochpathogene Erreger”, “Permanent working group of competence and treatment centers for diseases caused by highly pathogenic agents”) are widely accepted and recognized protocols for the treatment of COVID-19, compiled and verified by a large consensus of medical scientists, physicians, and research institutions. They provide a comprehensive overview of recommendations on the management of hospitalized COVID-19 patients. The process model was obtained starting from such guidelines, and was validated by the physicians working in the intensive and intermediate care unit of the Uniklinik. We openly share the resulting BPMN model, as well as the related documentation. The conformance with the guidelines was assessed by using process mining techniques. The results provide hospital managers with information about the main deviations and/or anomalies in the process and their possible causes. In addition, they suggest improvements to make the process more compliant, cost-effective, and performant.

The remainder of the paper is structured as follows. Section 2 explores related work and sets the context of our research. Section 3 lays out the methodology we employed in our case study. Section 4 illustrates the results of our case study. Finally, Sect. 5 concludes the paper.

2 Related Work

The global effort to fight the pandemic has stimulated the adoption of new technologies in healthcare practice [7]. An area where this effect has been radical is the digitization of healthcare processes, both medical and administrative. Data recording and availability have improved during the years of the pandemic. Stakeholders realized that data are a valuable information source to support the management and improvement of healthcare processes [9]. In addition, the reliance of medical personnel on digital support systems is now much more significant. Fields of science that have recently shown to be particularly promising when applied to healthcare operations are the process sciences, and specifically Business Process Management (BPM) and process mining [9]. This is mainly due to the characteristics of healthcare process, which are complex and flexible and involve a multidisciplinary team [9, 13]. Particularly, process mining has emerged as a suitable approach to analyze, discover, improve, and manage real-life and complex processes, by extracting knowledge from event logs [1]. Currently, process scientists have gathered event data on the process of treatment for COVID-19 and leveraged process mining techniques to obtain insights on various aspects of the healthcare process [3, 12, 15] or on how other business processes have been impacted by the disruption caused by COVID-19 [17].

Among process mining techniques, conformance checking aims to measure the adherence of a (discovered or known) process with a given set of data, or vice-versa [6]. Conformance checking helps medics to understand major deviations from clinical guidelines, as well as to identify areas for improvement in practices and protocols [9]. Some studies have applied these techniques in different healthcare contexts, such as oncology [14]. However, no studies have addressed the compliance analysis on the care process of COVID-19 patients in a real-life scenario. To do so, it is essential to have a normative model, reflecting clinical guidelines and protocols, that can be interpreted by machines. Currently, executable process models representing the guidelines for the treatment of COVID-19 patients are still absent and needed, given the uncertainty and variability of the disease.

3 Methodology

The methodology conducted in this study consists of the following three main steps, also shown in Fig. 1:

  • Development of a normative model based on the STAKOB guidelines. A normative model is a process model that reflects and implements rules, guidelines, and policies of the process, mandated by process owners or other supervisory bodies. This phase involves (i) the analysis of the STAKOB documentation and interview with ICU physicians, (ii) the development of the model from the guidelines, and (iii) the validation of the model with ICU physicians.

  • Data collection and preparation, which involves the extraction and preprocessing of event data, gathered from the information system of the hospital. The event log is refined by removing duplicate and irrelevant data, handling missing data, and detecting outliers to ensure data reliability.

  • Conformance checking, which involves the use of conformance checking techniques to compare the normative model with the event logs for the three COVID-19 waves and determine whether the behavior observed in practice conforms to the documented process.

Fig. 1.
figure 1

Case study methodology. Our work measures the deviation between the expected and real behavior of the COVID-19 treatment process, respectively represented by the STAKOB guidelines, and by the COVAS dataset.

Full size image

3.1 Development of a Normative Model Based on the STAKOB Guidelines

The STAKOB guidelines provide information on the disease and its related symptoms, and describe the diagnostic and treatment activities to be performed on COVID-19 patients and the therapies to be administered. The treatment of COVID-19 patients requires a multi-disciplinary approach: in addition to intensive care physicians and nurses, specialists in infectious diseases and infection control must also be part of the team [8]. The guidelines guide the operations of the medical team involved in the inpatient care of COVID-19 patients, but are also intended to provide information for individuals and/or organizations directly involved in this topic.

To make the guidelines interpretable by machines—and thus suitable for conformance checking—we developed a normative process model of the STAKOB guidelines in the BPMN language using the Signavio toolFootnote 2. The choice of the BPMN standard is due to its ability to be executable but, at the same time, easy to understand by physicians and practitioners. The BPMN model of the STAKOB guidelines was validated by using a qualitative approach. Specifically, the model was presented and discussed with three physicians working in the intensive and intermediate care unit of the Uniklinik during three meetings. During the meetings, several refinements were applied to the model, until it was approved by all.

3.2 Data Collection and Preparation

We collected and pre-processed data of COVID-19 patients monitored in the context of the COVID-19 Aachen Study (COVAS). The log contains event information regarding COVID-19 patients treated by the Uniklinik between January 2020 and June 2021. Events (patient admittance, symptoms, treatments, drug administration) are labeled with the date, creating timestamps with a coarseness at the day level. While here we exclusively focus on process mining, the COVAS dataset has also been analyzed in the context of explainable AI [16].

Data were gathered from the information system of the hospital. The initial database consisted of 269 cases, 33 activity labels, 210 variants, and 3542 events. Before the analysis, we refined the raw event log, to guarantee its quality. Data cleaning and preparation were executed with Python and included: (i) outliers and incomplete cases removal based on the number of hospitalization days, (ii) less significant activities abstraction, and (iii) filtering of infrequent variants. As an example, we removed the cases with a duration of more than 70 days: this value was validated with the doctors, according to whom durations longer than 70 days may be due to registration delays. In the end, the refined event log consisted of 187 patient cases, 32 activities, 135 variants, and 2397 events.

To evaluate the adherence of the COVAS dataset to the normative model during the three COVID-19 waves, we split the dataset into three sub-event logs. As illustrated in the next sections, this is done with the goal of examining how treatment operations for COVID-19 change between infection waves with respect to the adherence to the STAKOB guidelines. As shown by the dotted chart of the event log in Fig. 2, the three waves can be clearly identified. Such a choice of wave separation was also supported by the literature [5].

Fig. 2.
figure 2

Dotted chart of the COVAS event log. The cases are sorted by the first recorded event, which is highlighted in orange. Every blue dot corresponds to a recorded event. The vertical dashed lines separate the first, second, and third COVID-19 waves, based on the knowledge of physicians. (Color figure online)

Full size image

The event log of the first wave contains 106 cases and 1410 events. The average duration of the process is 25.38 days. The log of the second wave contains 59 cases and 892 events, with an average duration of 22.42 days. The log of the third wave contains 22 cases and 282 events, with an average duration of 16.38 days.

3.3 Conformance Checking

For each sub-event log, we applied conformance checking techniques to identify deviations within the process. Specifically, we utilized the plug-in “Replay a Log on Petri Net for Conformance Analysis” as implemented on ProM, with standard setting parameters. The choice is due to the fact that alignment-based techniques can exactly pinpoint where deviations are observed [1, 2].

The alignment-based technique allowed to estimate a global conformance measure, which quantifies the overall conformance of the model and event log, and local diagnostics, which identify points where the model and event log do not agree. In the first case, we calculated fitness, which measures “the proportion of behavior in the event log possible according to the model” [1]. In the second case, we estimated for each activity within the model the following [4]:

  • the number of “moves on log”: Occurrences of an activity in the trace cannot be mapped to any enabled activity in the process model.

  • the number of “moves on model”: Occurrences of an enabled activity in the process model cannot be mapped to any event in the trace sequence.

  • the number of “synchronous moves”: Occurrences of an activity belonging to a trace can be mapped to occurrences of an enabled activity in the process model.

4 Results

In this section, we presented the results from the development of the normative model and the conformance checking analysis.

4.1 Normative Model

The developed normative model consists of 3 sub-processes, 23 activities and approximately 36 gateways (XOR, AND and OR). Figure 3 shows a section of the model.

Fig. 3.
figure 3

A section of the STAKOB COVID-19 model, depicting some activities related to the ICU operations for COVID-19 patients.

Full size image

The model clearly underlines the fact that the treatment of hospitalized patients with COVID-19 is complex and is characterized by several pursuable pathways (see the presence of XOR and OR gateways). It also requires the collaboration of different departments and specialists. More in detail, the care treatment includes an antibiotic/drug therapy phase and, if necessary, an oxygenation phase. At this point, if the patient’s health condition deteriorates, the transfer to the ICU is planned (partially shown in Fig. 3). In the ICU, the patient may undergo mechanical ventilation, ECMO (ExtraCorporeal Membrane Oxygenation) or pronation in addition to the medical therapy. A section of the sub-process showing the respiratory support for the patient can be seen in Fig. 4. Recovery and subsequent discharge are confirmed by two negative COVID-19 tests.

Fig. 4.
figure 4

A section of the STAKOB COVID-19 model, depicting some activities related to the respiration support operations for COVID-19 patients.

Full size image

The full model is openly available on GitHubFootnote 3. It is rendered in the XML export format of the BPMN standardFootnote 4. The folder also contains a PDF depicting the entire model, a license declaration, and an addendum describing the model schematic in more detail.

4.2 Conformance Checking Results

COVID-19 First Wave Results. For the first wave, the fitness between the model and the data is 0.69; some trace variants are not reproduced by the model. This may be due to the variability of the process (health conditions vary from patient to patient). In addition, the coarseness of the timestamps in the dataset has an impact: events are recorded at the date level, so the order in which they are recorded may vary in some instances. Table 1 shows the results of the conformance checking for the first wave. Specifically, for each activity, it shows the misalignments between the normative model and the event log.

Table 1. Results of conformance checking alignments with the STAKOB model for the patient sub-log corresponding to the first COVID-19 wave. For each activity in the log, we show the count of moves on log, moves on model, and synchronous moves.
Full size table

Several misalignments can be observed. In particular:

  • The HiFlo Start and HiFlo End activities (corresponding to high flow oxygenation) present 19 moves on model and one synchronous move. This means that, although it is required by the guidelines, this activity is only performed in one case. This indicates that, given the patient’s condition, the physicians may have seen fit to skip this treatment.

  • There are several tasks that have both moves on model and moves on log. This means that these tasks often deviate from the normative model (in some cases they are present in the model but not in reality, in others vice-versa). This may be due to the variability of patients’ conditions and the lack of familiarity with COVID-19 and its standardized treatment, since this data was recorded in the early days of the pandemic. For example, the guidelines suggest that the Discharge ICU should occur after ventilation and pronation, while in reality, in some cases, it occurs before. Thus, many activities occur while the patient is hospitalized, but not still formally admitted to the ICU.

  • Some activities present only moves on log and synchronous moves, i.e., they are present in reality but at times not in the normative model. This means that they are performed at different times than the guidelines suggest. For example, Admission ICU may be anticipated because of a particularly critical course not foreseen by the physicians or be delayed because no space in ICU is available at that time; or Prone End (the interruption of the treatment of pronation) may be brought forward because of the negative effects on the patient, e.g., the appearance of pressure sores. Alternatively, pronation may be delayed because the patient has not achieved optimal arterial blood oxygenation.

COVID-19 Second Wave Results. For the log of the second wave, the fitness with the STAKOB model is 0.66. Table 2 shows the results of conformance checking for the second wave.

Table 2. Results of conformance checking alignments with the STAKOB model for the patient sub-log corresponding to the second COVID-19 wave. For each activity in the log, we show the count of moves on log, moves on model, and synchronous moves.
Full size table

In the second wave, Hospitalization is only performed after the onset of symptoms, as suggested by the guidelines. However, deviations are also encountered. As in the first wave, the most affected activities are End Of Fever, Admission ICU and Discharge ICU, and Last Oxygen Day, which have both moves on log and moves on model. This may be related to the mutability of the disease becoming difficult to manage with common protocols and the variability of the patients’ conditions. Compared to the first wave, the use of drugs has changed. In particular, a new drug is being administered, i.e., Dexamethasone, and the use of Remdesivir is increased. The administration of both drugs has moves on log mismatches, indicating that the physicians needed to administer such treatments more frequently than recommended. The former is also used in patients who do not require intensive care, contrary to what the guidelines suggest. The second, which is preferred for non-critical hospitalized patients, is also used in intensive care. In addition, high flow oxygenation is rarely performed here, despite being included in the guidelines.

COVID-19 Third Wave Results. The fitness between the log and the model is 0.69 for the third COVID-19 wave. Table 3 shows the results of conformance checking for the third wave.

Table 3. Results of conformance checking alignments with the STAKOB model for the patient sub-log corresponding to the third COVID-19 wave. For each activity in the log, we show the count of moves on log, moves on model, and synchronous moves.
Full size table

The physicians’ experience and familiarity with the disease appear to have increased. However, many of the misaligned activities have similar behavior to those performed during past waves. Note that the ECMO treatment has zero values in all columns. This is because it is not performed in the third wave (unlike the first two). Since ECMO is the most invasive oxygenation treatment, this may be due to the fact that the severity of the patients’ condition has decreased.

To summarize, alignments-based techniques make it possible to detect and analyze process deviations, providing useful insights for physicians. Furthermore, in the three waves, most activities remained misaligned, while some moved closer to the guidelines’ suggestion. This shows that the process is highly variable and specific care pathways are required for each patient, which do not always coincide with those stated in the guidelines.

5 Conclusion

Our work aimed to analyze the care process for COVID-19 patients, bringing to light deviations from the clinical guidelines. Specifically, the work proposed a normative model bases on the STAKOB guidelines, which can be interpreted by software tools (e.g., process mining software). The BPMN model is openly accessible to any analyst, and can also be loaded into any commercial software supporting the BPMN standard, like Celonis and Signavio. This addresses the need for computer-interpretable and usable guidelines in healthcare, particularly for the treatment of COVID-19 patients [10]. In addition, the work provided physicians with guidance on the management of COVID-19 patients, highlighting deviations and critical points in the three infection waves.

The contributions of our work are:

  • One of the first attempts to apply a process mining-based methodology for the analysis of process deviations in a real, complex, and uncertain healthcare context, like the recent and ongoing COVID-19 pandemic.

  • The development of a normative model that can advise physicians in the treatment of COVID-19 patients by providing specific guidelines and procedures to follow. This is helpful in dealing with the uncertainty and complexity of healthcare operations brought about by the pandemic. In addition, the model can be used as input for the development of a decision support system, which alerts in real-time in case of violations of the guidelines.

  • The extraction of valuable insights for physicians regarding the main deviations and the related causes in the COVID-19 patient care process. This knowledge is crucial for improving the process and ensuring service quality and patient satisfaction, e.g., better management of drug administration (when to administer and how often), more targeted execution of certain treatments—e.g., pronation—(who to treat and when to do it), and execution of treatments suggested by guidelines but never performed in reality that can enhance the care pathway and reduce hospitalization time (such as high flow oxygenation).

The work presents some open questions and directions for future research. The limited size, especially for the third wave, and the coarseness of the timestamps in the dataset may impact the results. To address this issue, a possible option is to weigh the results of analyses using the probability of specific orderings of events in traces [11]. Furthermore, the physician’s consensus on both the validity of the STAKOB model and the interpretation of the conformance checking results can definitely be enlarged, by soliciting the expert opinion of a larger group of medics. As future developments, we plan to: (i) extend the research and collect new data from other German hospitals, in order to generalize the results and identify best practices in the treatment of COVID-19 patients; (ii) improve the validation of results; (iii) actively involve physicians in the analysis of deviations, using qualitative approaches such as interviews and field observations; (iv) conduct a more extensive comparative analysis based on process mining, including a structural model comparison, concept drift, and performance analysis.