Keywords

1 Introduction

Process mining techniques can be used to measure the level of compliance by comparing event data to a de jure or normative model [19]. In the healthcare domain, normative models can be extracted from clinical guidelines and protocols. Deviations from clinical guidelines occur frequently as healthcare processes are intrinsically highly variable - a challenge described in the process mining for healthcare (PM4H) manifesto [16]. In healthcare it is sometimes very important to deviate from guidelines for the safety of the patient. Other distinctive characteristics of healthcare processes include the need to consider contextual information during analysis and be aware of process changes brought about by advances in medicine and technology.

Changes in healthcare processes can also be caused by external factors that are unplanned, for example the COVID-19 pandemic, or planned, for example the implementation of a new hospital IT system. We propose a method to examine the impact of these planned and unplanned factors on patient care by analysing pathway changes using process mining techniques. We are building on an approach for checking conformance of event logs to discovered models to detect sudden process changes that was originally developed and validated against synthetic data [5]. The method proposed in this paper investigates process changes due to known perturbations to real-life care pathways using process mining techniques including checking conformance to normative models.

Distinctive characteristics of healthcare such as high variability and frequent process changes lead to certain key challenges in mining healthcare processes identified in the PM4H manifesto [16]. Challenge C2 in the manifesto describes the need for novel techniques for checking conformance of healthcare processes to available clinical guidelines. This is relevant to our study as we compare real-life care pathways in event logs with normative models by checking conformance. As highlighted in challenge C3 of the PM4H manifesto, changes in healthcare processes over time due to factors such as seasonal changes or the introduction of a new work system should also be considered. This challenge directly affects our study on the impacts of two major perturbations on care pathways.

We studied patient pathways at Bradford Royal Infirmary (BRI) which is a public hospital in Bradford, UK. During the period of our study there were two potential sources of perturbations. These were the COVID-19 pandemic and the near co-incident implementation of a new Command Centre approach to patient-flow management. A study protocol had been designed [11] to evaluate impacts of the newly implemented Command Centre system on patient safety and healthcare delivery. However, shortly after the Command Centre was introduced, COVID-19 disrupted the hospital activities along with the rest of world. Thus, we have a unique opportunity to study effects of two co-incident sources of perturbations on hospital processes, one of which was planned and the other was unplanned.

The Command Centre approach was based on a new IT system and a corresponding redesign of patient-flow management processes and implemented in a series of planned interventions. The Command Centre aims to improve healthcare delivery by providing relevant information to assist staff in making real-time complex decisions [6]. Designated staff monitor continuously updated hospital information summarised on a wall of high-resolution screens through applications called ‘tiles’. The ‘tiles’ present process status data from established hospital information systems used across the different units of the hospital. The Command Centre software uses rule-based algorithms to warn about impending bottlenecks and other patient safety risks to support optimised patient care and effective management of resources. The intention of the Command Centre approach is to provide centralised surveillance of hospital patient flow to a team of people empowered to manage that flow in the best interests of patients and the hospital, following approaches that are well established in other industries such as air traffic control centres at airports.

The COVID-19 pandemic has impacted healthcare around the world, notably with a great reduction in the use of healthcare services [15] as available resources were allocated to the high demand of care for COVID-19 patients. The re-prioritisation of resources to meet the challenges of the pandemic affected normal care processes. For instance, in England, major disruptions to the pathway for colorectal cancer diagnosis led to a considerable reduction in detection of the disease in April 2020 [14]. In the city of Bradford, UK, surveys for studying the pandemic’s impacts on families showed increases in mental health issues in adults as well as children [3]. Our hypothesis is that the impact of external disturbances on care processes can be detected and measured in event log data extracted from BRI’s Electronic Health Records (EHR) data. In this paper, we propose a method for identifying and measuring impacts of disruptive events by building on previously established approaches of detecting process changes and apply this method on a real-life case study.

1.1 Related Work

This process mining work is part of a larger project based on the study protocol [11] to evaluate impacts of the Command Centre at BRI. Our focus is on a subsection of the study protocol aiming to analyse effects of the Command Centre on patient journeys using process-mining techniques. Investigation of patient flow is expected to contribute towards assessing the installation of the Command Centre under the hypothesis that productivity, associated processes and patient outcomes are influenced by patient flow. The study protocol also hypothesised that the recording of hospital data is influenced by the Command Centre’s installation. Studies on quality of data, patient flow and patient outcomes throughout the phases of the Command Centre’s installation are proposed in the protocol.

In related work, Mebrahtu et al. [13] investigated quality of data and patient flow to test the hypothesis that the Command Centre positively impacts recorded data and flow of patients through the hospital. They considered five time periods based on different interventions involved in the Command Centre’s installation as shown in Fig. 1. They also explored A &E patient records for missing timestamps of certain events and the relative occurrence of the valid A &E pathway to assess the Command Centre’s impact on data quality. To study the impact on patient flow, time intervals between selected timestamped events recorded for A &E patients were analysed. They observed no notable improvements to A &E patient flow and quality of data suggesting the Command Centre had no measurable impact. A drawback of the investigation was that the COVID-19 pandemic that disrupted normal hospital function occurred nearly co-incidentally with the launch of the Command Centre. This suggests the need for further research to understand how command centres may influence data quality and patient journeys in hospital settings.

1.2 Study Design

In this paper, we aim to extend the previous work by including an investigation of the impacts on patient pathways using process mining. Process mining methods have been previously used in describing hospital journey of patients in different healthcare domains. For instance, process mining has been used to discover that few patients undergoing chemotherapy followed an ideal care pathway [1]. A study of A &E pathways using process mining attributed the reason for longer stays in the department to a loop in the pathway [17]. This paper proposes a method aiming to measure and investigate process changes associated with disruptive events which is demonstrated using a case study of in-hospital care pathways.

Fig. 1.
figure 1

Timeline of the study period indicating the interventions involved in installing the Command Centre.

Full size image

Our study focuses on the perturbations to A &E and maternity pathways through BRI arising from COVID-19 and the co-incident implementation of a new patient-flow management system. We aim to add to the previous investigation of A &E pathways through the hospital described in the related work. The choice of maternity pathways is justified by the ease of identifying maternity patients in the data and of obtaining a predetermined normative model as maternity processes can be expected to be reasonably consistent in nature. Moreover, maternity patients were possibly the least affected by COVID-19 measures that prevented other patients from accessing timely medical treatment. Exploring impacts of two simultaneous perturbation sources on hospital pathways presents a unique opportunity to reflect on the challenges of PM4H. Through this case study, we demonstrate a method to measure the impact of perturbations on the quality of hospital service in the context of in-hospital care pathways.

2 Methods

2.1 \({{\textbf {PM}}}^2\) for Exploring Impacts on Care Pathways

We followed the \(\text {PM}^2\) process mining methodology [4] as adapted by Kurniati et al. [10] to analyse process changes using a multi-level approach. The first two stages of \(\text {PM}^2\) are focused on formulating basic research questions followed by two stages of analysis which delve deeper into the objectives while the last two stages focus on process improvements and real-world implementation. In our investigation we applied stages 1 to 4 of \(\text {PM}^2\) but not stages 5 and 6 as process improvement and clinical intervention were out of the scope of this work. For the multi-level approach, we focused on model and trace-level process comparisons to explore impacts of potential perturbations on care pathways.

For Stage 1 (Planning), our research questions were drawn based on the process mining subsection of the study protocol [11] to evaluate impacts of the new Command Centre at BRI. Previous related studies of impacts on patient flow [13] and patient safety [12] were also included in framing the process mining objectives. Research questions were further adapted to studying A &E and maternity pathways during the time period covering the two perturbations. In Stage 2 (Extraction), selection of data attributes relevant to the patient pathways was guided by previous related work [13], advice from clinicians and our understanding of attribute labels with the help of public information resources. A clinician familiar with the study data was part of the project team, while other clinicians were engaged as interviewees as described in Stage 2 of the ClearPath method [9].

In the next stage, Stage 3 (Data Processing), we used patient admission as the case identifier to create event logs for in-hospital care pathways. The models that were discovered in Stage 4 (Mining and Analysis) identified the key activities for building normative models based on clinical advice. Our main analysis involved process comparisons over the period of interest by studying durations at the trace-level and conformance and precision between event logs and normative models at the model-level. The conformance is measured by checking the proportion of traces in the event log that fit the model. Precision is a measure obtained by comparing the set of traces that are allowed by the model with the set of traces in the event log fitting the model. Thus, precision is not a meaningful measure for unfit traces [2]. In this paper, the precision was calculated only for traces that were fitting the respective process model.

For obtaining normative models for A &E and maternity pathways, we followed a framework proposed by Grüger et al. [7] for the non-trivial transformation of clinical guidelines into computer-interpretable process models. The framework known as the Clinical Guideline Knowledge for Process Mining (CGK4PM) comprises five steps that include identification of required key inputs, conceptualisation through workshops with stakeholders, formalisation to obtain a semi-formal process representation of guidelines, implementation by translating into a selected process modelling language and finally the testing step for verifying and validating the implemented model.

For the first step of identification, clinical guidelines were selected based on discovered process models and discussions with clinicians. We skipped the conceptualisation step due to resource constraints and instead we consulted clinicians for the transformation of clinical guidelines into semi-formal process models. This was achieved by identifying relevant activities through process discovery and consulting clinicians on the expected order of events for patients who were progressing well. The process representation of clinical guidelines was then translated into Business Process Model and Notation (BPMN). The BPMN model was verified by conformance checking and reviewed by clinicians for validation.

2.2 Data

The data source for our case study was the EHR data from BRI. A data extract was provided for our study by the Connected Bradford [18] data linkage project. The Connected Bradford project brings together data covering a wide range of factors influencing population health for the Bradford region through data linkage. In particular we used the summary of activity produced as a part of integrating hospital data to Secondary User Services (SUS) which is created as a data feed to the national data warehouse for healthcare data in England, augmented by timings from a data feed used to drive the Command Centre tiles. The data extract included information on A &E patients, outpatients and inpatients along with diagnoses, procedures, surgeries, prescriptions and some patient demographics.

Our study uses the A &E and inpatient data during the period from January 2018 to August 2021. The A &E timing data that was used in this study came from the Command Centre system which recorded 100% of the data starting only from September 2020. However, it did also include a small amount of data (approximately 20% of attendances, which may not be a representative sample) from prior to this point. Thus, the findings from the A &E data in this study cannot be interpreted.

Fig. 2.
figure 2

A &E models.

Full size image

For analysing A &E pathways, we identified 193,772 A &E attendances that occurred during the period of study. For analysing maternity pathways, we selected admissions of patients who registered in one of the ‘maternity wards’ which included a birth centre for uncomplicated labour cases, a labour ward for patients needing specialist care during labour, two maternity operating theatres and two wards for patients needing care before or after birth, referred to in this paper as ‘natal wards’. A total of 18,076 maternity admissions were identified for analysis of which 16,905 resulted in the delivery of a newborn baby with a recorded timestamp. Data on admissions, ward stays and delivery timestamps were distributed across three tables which were linked through admission and patient identifiers.

2.3 Tools

The summary of activity data from BRI was made accessible on the cloud by the Connected Bradford service via Google Cloud Platform (GCP). Accessing data was possible through the relational database management system BigQuery provided by GCP. Timestamped data was extracted to RStudio Server Pro storage using SQL-based queries for the analysis. Event logs and process maps in the form of directly-follows graphs were generated using the open-source process mining platform called bupaR [8] in the RStudio Server Pro environment. An open-source Python package known as PM4Py [2] was used for conformance checking against normative models.

Fig. 3.
figure 3

Metrics depicting process changes associated with A &E pathways.

Full size image

Directly-follows graphs discovered from the event logs identified activities relevant for building normative models following the CGK4PM framework. BPMN diagrams of normative models reviewed by clinicians were drawn using an online tool (https://demo.BPMN.io) and uploaded to Jupyter notebook for conversion into Petri nets using the PM4Py package. The resulting Petri nets were used for checking conformance of event logs by token-based replay to obtain percentage of traces in the event log that fit the normative model.

3 Results

These are the research questions that were identified during Stage 1 (Planning) of the \(\text {PM}^2\) methodology: Q1. What are the discovered pathways for A &E attendance and maternity admissions in BRI? Q2. What are the normative models for A &E and maternity pathways in BRI? Q3. Can process changes due to potential perturbations be identified and measured in discovered pathways using normative models? In Stage 2 (Extraction), we selected timestamps of arrival, assessment, treatment and check-out for A &E attendance. For maternity admissions, we selected the timestamps of admission, ward stays in any specialty, and discharge.

During Stage 3 (Data Processing), the event logs were filtered for the time period of interest. We filtered out A &E attendances that did not have arrival and check-out as the start and end points respectively. Two maternity patients with inconsistent timestamps and one admission with two simultaneous ward stays were excluded from the analysis. The maternity event log was enriched with information on the time of childbirth by including the delivery timestamp.

In Stage 4 (Mining and Analysis), directly-follows graphs for A &E and maternity pathways were obtained over the entire period of study, using the processmapR package in bupaR, to answer Q1. To analyse process changes at the trace-level, we examined the mean and median length of A &E attendances and maternity admissions. The CGK4PM framework was followed to obtain normative models for A &E (see Fig. 2a) and maternity pathways to answer Q2. Although the notation in the normative models suggests typical clinical rationale for transitions, it is to be noted that these do not depict the ideal pathway for every scenario.

For process change analysis at the model-level, event logs were checked for conformance and precision against process models. For A &E pathways, the conformance and precision between monthly event logs and two process models, namely the normative model (see Fig. 2a) and a model based on the discovered process map (see Fig. 2b), are shown in Figs. 3a and 3b. The conformance and precision between monthly event logs and the normative maternity model are shown in Fig. 4. For obtaining monthly values of conformance and precision, event logs for each month were selected by including cases that started within the month. It was found that for both A &E and maternity, the conformance between event logs and corresponding normative models reduced during the period after COVID-19 measures were introduced (April 2020 to August 2021) compared to pre-pandemic times (April 2018 to February 2020). No significant difference in the precision was observed in these two periods for both A &E and maternity pathways. To answer Q3, the trace-level analysis of durations showed that the lowest median length of stay over the period of study occurred in April 2020 for both A &E and maternity pathways. It was also observed that the mean length of stay reduced after national pandemic measures were introduced for both A &E and maternity compared to pre-pandemic times.

4 Discussion

In this case study we have added conformance checking against the normative model as an extension to the multi-level approach of Kurniati et al. [10] to address challenge C2 of PM4H [16]. Changes in healthcare processes over time could be analysed by this method but identifying the cause of process changes requires further research. For obtaining a clearer picture of the impacts of the two perturbations, other inherent influences such as seasonal factors need to be controlled for as described in challenge C3. Since one of the perturbations was planned, while the other was unplanned, there is scope for further research to try to differentiate the impacts of the two perturbations.

In this study, the most noticeable process change was the drop in conformance of A &E pathways to the normative model in April 2020 (see blue curve in Fig. 3a) following the introduction of nationwide pandemic measures in March 2020. For the normative A &E model, the precision is high throughout (see red curve in Fig. 3a). For the model based on the discovered process map, the conformance remains high throughout (see blue curve in Fig. 3b), while the precision is lower and changes significantly during the ‘Third intervention’ period (see red curve in Fig. 3b).

From conformance values with respect to the normative A &E model, we can see some disturbances in the pathways in April 2020. The precision values between monthly A &E event logs and the model based on the discovered process showed fluctuating behaviour during ‘Third intervention’ period. The extra path in the A &E model based on the discovered process map (see Fig. 2b), indicating an alternative way of working, captured deviations that were not detected by the normative model. Thus, the model based on the discovered process map contains a useful level of complexity which is the occurrence of the activity ‘Assessed’ after ‘Treated’. This only shows what has been recorded by the IT system but not necessarily what happens in the A &E department and is thus not part of the normative model. On discussion with clinicians, assessment might be recorded after treatment due to a technical need to progress with the treatment. This suggests that it might be a regular feature of work in practice to record assessment after the event.

The drop in conformance in April 2020 and the changes in precision during the ‘Third intervention’ period could be detected using the identified metrics. Further research using the full A &E dataset and discussion with clinical experts is required to accurately identify causes of the process changes. If the detected perturbations can be attributed to external disturbances, the ability to capture them could be implemented in information systems to warn about disruptions to the normal workflow. The integration of process mining into process aware information systems would enable perturbations to be detected in real time allowing clinicians and hospital managers to identify and react to adverse events taking place in the hospital.

As described before, the reasonably consistent nature of maternity pathways can be seen in the more stable values of conformance and precision (see Fig. 4) over the period of study. The significant differences in the impact on A &E and maternity pathways under the influence of the same external perturbation sources might be attributed to the contrasting nature of the two pathways. A &E pathways are very dynamic whereas maternity pathways are often predetermined. Through this study we have demonstrated that process changes due to complex perturbations can be detected using process mining techniques. In future work, the frequency of traces following a selected sequence of activities over the time period of interest can be studied at the trace-level [10]. The care pathways can also be studied at the activity-level for further investigation of the process changes.

Fig. 4.
figure 4

Conformance and precision between monthly event logs and the normative maternity model.

Full size image

Since only a subset of the data for A &E attendances was available until August 2020, we are not in a position to draw conclusions about the length of stay and the overall conformance and precision measures of the A &E pathway. We have demonstrated the proposed method but cannot state that all our results are representative since we do not know the bias. Further work would be necessary to rerun the analysis on the complete set of data. However, the full dataset was not available at the time of writing.

5 Conclusion

We used the proposed method to identify process changes at the trace and model levels. Causes of identified changes cannot be determined with confidence as the perturbations were nearly co-incident. In further research, the results may be compared with another hospital that did not implement a change in the management system during the same period.