1 Introduction

The Design Science research paradigm has received significant attention in the Information Systems (IS) literature during the most recent decade [25]. Following the theory, which has its roots in the engineering discipline, a mathematical basis for designing information systems provides opportunities for the quantitative evaluation of an IT artifact and features proving optimality of its design [14]. More recently, a number of scholars from IS and related fields have started to use theories, methods, and tools from neuroscience and psycho-physiology to model and better understand human cognition, emotion, and behavior in IS design [23]. This paper attempts to bridge the gaps between the Design Science paradigm and Cognitive Sciences by formulating a mathematical model of the cognitive aspects within IS design and showcasing its applicability to healthcare information systems. Thus, gradually achieving the unfulfilled promises of health information technology [17] through redesigning systems to align with end user needs is the objective of our paper.

1.1 Design of hospital information systems

The design of effective and efficient business processes through information and communication technologies (ICT) has received increasing attention and healthcare is no exception [4, 5]. More significantly, along with process efficiency, improving patient safety and quality of care, particularly through ICT, have been the target of recent research [2, 7]. Furthermore, as detailed data is increasingly made available through clinical information systems and health information exchange platforms at the individual level [27, 29], the efficient management of tasks for individual patients becomes critical to improve workflow efficiency and quality of care, simultaneously.

Hospital information systems play an important role in improving the delivery of health care services. On the macro-level, evaluating the impact of health IT is still much needed [1]. In addition, useful insights have to be generated by studying health IT at the micro-level [1] where Computerized Physician Order Entry (CPOE) has proven to be effective in increasing patient safety and reducing medication errors and costs [18]. Factors such as functionality and ease of use can vary significantly across these type of systems [1].

1.2 Order set usage in hospital information systems

Within CPOE, order set optimization has been the subject of recent research [12, 29]. Order sets support physicians in high risk situations by serving as expert-recommended guidelines, reducing prescribing time by making complex ordering easier, and increasing physician compliance with the current best practice. For instance, the “Asthma order set No. 12” shown in Fig. 1 groups together order items for Asthma patients. The order set is typically prescribed upon a patient’s admission to the inpatient setting. Each order item in an order set can be defaulted ON or OFF according to clinical relevance and frequency of use. An order item can be part of multiple order sets. Despite the benefits of order sets, historical data indicate a tremendous variability in order set usage by physicians, driven largely by the diversity in patient population, physician experience, and system usability.

Fig. 1
figure 1

Asthma order set No. 12

Full size image

To illustrate the use of the order set given in Fig. 1, assume we want to minimize the number of mouse clicks associated with the prescription of orders and assume we have three Asthma patients who require, respectively:

  1. 1.

    Ventilator Circuit Change, Subsequent Mechanical Ventilation, and Arterial Blood Gas Draw (ICU Resp)

  2. 2.

    Subsequent Mechanical Ventilation, and Arterial Blood Gas Draw (ICU Resp)

  3. 3.

    Elevate

The enumeration of patients’ order items reveals that the first, second and third patient requires three, two and one order items, respectively. The optimal solution which minimizes mouse clicks is as follows: We prescribe the first patient the “Asthma Order Set No. 12” (1 click). This means that all defaulted ON items are prescribed to the patient. We also prescribe the second patient the order set (1 click). However, we deselect “Ventilator Circuit Change” (1 click) because the patient does not require this order. Finally, we prescribe the third patient the item “Elevate” (1 click) without prescribing him/her the order set. This is also called “a la carte” prescription because the order item does not come from an order set. The result is a total number of 4 mouse clicks.

It is worth noting that the same number of clicks would have been achieved if the second patient was prescribed the items “Subsequent Mechanical Ventilation” and “Arterial Blood Gas Draw (ICU Resp)” without assigning him/her to the order set. In other words, these two order items could have been prescribed “a la carte”.

When using CPOE systems, clinicians can search for particular orders by typing the order names and the search result includes all a la carte orders and order sets that match the keyword because order set usage is not mandatory. A la carte orders are individual orders that physicians choose to enter without using order sets. Intuitively, ordering a la carte items takes more time compared to order sets because they have to be searched for and entered one by one. Some orders are standalone items and a la carte is the only way to prescribe them. Yet, reasons for ordering a la carte items instead of order set items mainly come from a physician’s disagreement with order set content, unfamiliarity with order sets, inconsistency of order set content with current best practices, or, at times, a simple need for only one or two orders. Ordering efficiency decreases when order sets contain items that do not match the workflow or the patient’s condition, forcing physicians to go through long lists of orders to determine each item’s relevance to particular patients, and eventually rely on a la carte orders which are time-consuming and subject to errors [29]. Size of order sets range from 2 to more than 50 unique items.

1.3 Using design science to address cognitive workload in order set optimization

Following Hevner et al. [14] who state that if design solutions can be formulated appropriately and posed mathematically, standard Operations Research (OR) methods can be used to determine an optimal solution for the specified end conditions. The approaches developed in this paper come into play in this context, namely addressing the challenges of order set generation using OR methods, specifically Combinatorial Optimization, that allows us to create order sets from usage data with the objective of minimizing physicians’ cognitive workload. Physicians’ cognitive workload can be decomposed into the following six components: i) assigning patients to order sets, ii) deselecting non-required order items from order sets, iii) deselecting order items which are prescribed multiple times, iv) ordering items a la carte, v) confirming defaulted ON and vi) confirming defaulted OFF order items in order sets after assigning order sets to patients.

To tackle the problem of incorporating cognitive workload into order set optimization such that we bridge Design Science with Cognitive Sciences, we develop a mathematical model to minimize cognitive workload associated with order prescription in CPOE. The problem, model formulation, computational study and decision support system developed in this paper substantially extend and generalize Gartner et al. [12]’s approach, particularly going beyond optimizing only the mouse clicks associated with order prescription by incorporating physicians’cognitive workload into the current study.

Furthermore, we execute the model on more realistic test instances, and embed the model and solution method in a prototype decision support system that can help practitioners analyze the composition of order sets. Moreover, our theoretical insights can be seen as a generalization of Gartner et al. [12]’s and we show that our theoretical insights are confirmed in our experiments. Our cognitive workload minimization model is then decomposed given non-overlapping fixed time intervals. However, disentangling the K −means based cluster generation from the patients’ order assignment problem now leads to a different objective function and constraints as compared to the ones of Gartner et al. [12]. Our evaluation study is carried out using order item data on Asthma patients with moderate complexity, which includes more than seven times the patient population as compared to the one of Gartner et al. [12]. We compare the current order set configuration of a major pediatric hospital with the exact and heuristic solutions on several performance metrics and at multiple levels of detail.

The results of comparing the mathematical program with the current order set configuration in the hospital indicates that cognitive workload can be reduced by about 20.2% by allowing 1 to 5 order sets, respectively. The comparison of the K −means based decomposition with the hospital’s current configuration reveals a cognitive workload reduction of about 19.5%, also by allowing 1 to 5 order sets, respectively. In order to facilitate the comparison of current order set usage with order set optimization results, which is important when demonstrating the new approaches to clinicians, we develop a graphical user interface (GUI)-based decision support system. This approach not only allows us to apply Design Science methods to health care information systems but also to push the frontier towards bridging the gap with Cognitive Sciences, and, further, test our models and solution approaches for other critical health conditions that are managed in the inpatient setting [29].

The remainder of this paper is structured as follows. In the next section, we position our paper within related work in this area. We then provide a formal description of the problem, the model formulation, and structural properties. A computational study is provided in order to demonstrate the effectiveness of our approach based on data from a major hospital in the United States. In that section, we describe our evaluation metrics and present the results. We finally summarize our paper in the conclusion section and provide an outline of future research on this problem.

2 Related work

Four fields are relevant for the positioning of this research: i) Cognitive workload addressed in the development of Information Systems, ii) IT, analytics and workload impacts on health care performance in high risk situations, iii) order set optimization and data-driven product development, and, iv) Operations Research (OR) applied to clustering and health care management.

2.1 Cognitive workload addressed in IS development

Cognitive information is processed in the lateral pre-frontal cortex of the human brain [11] and has been reviewed in combination with IS design by Eppler and Mengis [10]. Setting the research agenda towards using neuro-physiological tools in the development of IS, Dimoka et al. [8] review literature on the reduction of users’ information and cognitive overload. They conclude that overload arises i) from having too much information when a person is performing a task or ii) from the difficulty in inferring what information is required for the task. Order set optimization can be seen as addressing both problems. Too much information encoded in an order set can overload a clinician because he may confirm or de-select too many order items. On the other hand, inferring what information is required for prescribing orders to a patient may also incur workload and order sets can help guide a clinician toward best-practice or evidence-based guidelines. While other research has been done to measure cognitive workload of IS users performing a task, Zhang et al. [29] approached measuring cognitive workload as time spent on each task in such as selecting or confirming order items. Our study incorporates their results into the cognitive workload minimization model. The mathematical program can, however, be extended towards incorporating brain activity measures (rather than time) associated with choosing an order set or order item.

2.2 IT, analytics and workload impacts on performance in high risk health care situations

IT-enabled processes in a pharmacological setting in which high risk situations can occur because of noisy data are improved by Bai et al. [4]. Providing IT-enabled decision support for these situations is similar to the focus of our study where order sets can provide guidance in high risk prescription and order management processes. We employ analytic approaches for the efficient assignment of order items to patients. Documentation tasks of clinicians are studied by Powell et al. [22] who report that over-worked clinicians document less and therefore hospital revenues are reduced. These studies demonstrate that intelligently designed information technology solutions can support the efficient delivery of healthcare processes, with substantial impact on the economics of healthcare delivery.

2.3 Order set optimization and data-driven product development

Related work on order set optimization includes Gartner et al. [12] and the references therein. The authors develop a mathematical program to reduce the physical workload of mouse clicks through order set improvement. The objective of our paper is, however, to provide optimal and heuristic approaches to minimize cognitive workload associated with order prescription. Physicians’ cognitive workload can be decomposed into the following six components: i) assigning patients to order sets, ii) deselecting non-required order items from order sets, iii) deselecting order items which are prescribed multiple times, iv) ordering items a la carte, v) confirming defaulted ON and vi) confirming defaulted OFF order items in order sets after assigning order sets to patients. The composition of order sets is dependent on the cognitive workload associated with confirming and de-selecting order items from order sets (see Fig. 1 in Section 1.2). As a consequence, our aim is to provide structural insights into the problem, such as which cognitive workload coefficients lead to a valid lower bound on order set size. Related to order set optimization is the paper of Jiao et al. [16] who provide a review of product design approaches including applications of clustering methods. More recently, Lei and Moon [19] developed a market-driven product design approach by applying K–means clustering in the automobile industry. This is similar to our work since we develop order sets based on patient demand.

2.4 OR applied to clustering and health care

The heuristic and optimal development of order sets can be seen as a generalization of clustering problems. Recent literature reviews that combine data mining and OR including clustering applications are Baesens et al. [3], Olafsson et al. [21] and Meisel and Mattfeld [20]. Clustering approaches are reviewed by Jain [15]. The effectiveness of applying mathematical programming to clustering is demonstrated by Hansen and Jaumard [13]. This is highly relevant for our work because we apply mathematical programming for order set optimization. One of the first binary programs to model clustering problems is based on Vinod [24] which is related to our modelling approach because we employ binary decision variables to assign order items to clusters, among others. Focusing on healthcare, Cardoen et al. [6] and Dobson et al. [9] group medical items for surgeries which can be seen as a clustering problem. A major difference is that we have a time interval-dependent demand function which captures the patients’ length of stay in the hospital. With respect to the solution methodology, we can decompose the problem and solve subproblems to optimality or heuristically using K–means. Another difference is that we have designed and implemented a GUI-based decision support system for clinicians with the aim of optimizing cognitive workload reduction.

As a conclusion of our literature review, our study can be considered to be the first to consider the problem of cognitive workload reduction in clinical information systems and successfully employ mathematical programming to address this for order set optimization. In addition, we incorporate the model into an exact and a heuristic decomposition approach and prove structural properties. Our approaches allow us to solve real-world test instances to optimality which is unique in the order set optimization literature. Also, the development of a GUI-based decision support tool which allows practitioners to compare current order set usage with heuristic and optimal order set development is entirely new.

3 Model formulation

In what follows, we provide a concise problem description followed by a mathematical model that clusters order items to which patients are assigned. We will use the following terms as synonyms: activities, items, orders, procedures and treatments. Similarly, clusters and order sets are used as synonyms.

3.1 Problem Description

When patients arrive at the hospital and are treated over a planning horizon, we wish to assign the patients’ treatments to clusters which represent sets of order items. Unlike a la carte order placement, where users need to apply a mouse click every time to select an individual order, default ON items are automatically selected when an order set is chosen. With additional clicks, users can add default OFF items to the selection or deselect default ON items from the order placement, as indicated in Fig. 1 in Section 1. In what follows, we start with the definition of the general parameters for building clusters and then turn to patient-related parameters as well as cognitive workload parameters for the assignment of patients’ activities to order sets, and for selecting order items a la carte, among others.

3.1.1 Sets and indices for time intervals, order sets, order items and patients

We have a set of time intervals \(\mathcal {H}:=\{1,2,\ldots , H\}\) with H denoting the last interval. For example, [22;24] denotes a time interval between 22 and 24 h after admission of the patient to the inpatient setting. Intervals \(h,h'\in \mathcal {H}\) are non-overlapping. Order sets are indexed by the set \(\mathcal {O}:=\{1,2,\ldots ,O\}\) with the maximum number of order sets denoted by O. For example, O = 5 order sets can be created in each time interval. Order item demand at time interval \(h\in \mathcal {H}\) is denoted by set \(\mathcal {I}_{h}:=\{1,2,\ldots ,I_{h}\}\) in which I h is the biggest index of order items observed at time interval \(h\in \mathcal {H}\). Patients are denoted by set \(\mathcal {P}:=\{1,2,\ldots ,P\}\) in which P is the last index of all patients. Patient demand at time interval \(h\in \mathcal {H}\) is denoted by \(\mathcal {P}_{h}\subset \mathcal {P}\). We observe activities that are required for patient p at time interval h and we denote this subset by \(\mathcal {I}_{p,h}\subset \mathcal {I}_{h}\).

3.1.2 Cognitive workload (CW)

We break down CW into i) the cognitive workload when patients are assigned to order sets, ii) CW associated with the selection of additional order items iii) CW associated with the deselection of non-required order items and iv) CW associated with the confirmation of patients’ required and non-required order items.

Cognitive order set selection workload:

When an order set is assigned to a patient, we denote the CW associated with its selection as c os.

Cognitive item-specific selection workload:

If additional order items are required (in addition to the activities in an order set), cognitive workload of c off,on arise for each additional activity. Cognitive workload for adding an a la carte item is denoted by c alc.

Cognitive item-specific deselection workload:

We denote c off,non-req as cognitive workload when an order item that is part of an order set must be deselected for that particular patient because it is not required. Sometimes, patients may be assigned to multiple order sets. In that case, it can happen that order items are prescribed multiple times and cognitive workload associated with the deselection of items that are prescribed multiple times are denoted by c off,mult.

Cognitive order confirmation workload:

We denote c conf,on as cognitive workload when an order item is confirmed as default ON and assigned an individual patient. Similarly, we denote c conf,off as cognitive workload when an order item is confirmed as default OFF and therefore not assigned to the patient.

We will now introduce the decision variables, the objective function and the constraints to model the problem. The decision variables are shown in Table 1. We denote the \(a^{\text {off}}_{h,i,o}\) and \(a^{\text {on}}_{h,i,o}\) variables as ‘clustering variables’ because in each order set o, they will provide information regarding which order item is defaulted-off and -on, respectively. All other variables will mainly be used for assigning patients to order sets or performing decisions on the patients’ item level to determine the physician’s cognitive workload in the objective function.

Table 1 Overview of decision variables
Full size table

The model minimizes the cognitive workload for selecting a la carte items and order sets. Moreover, it accounts for de-selecting orders from order sets to which orders are assigned because they are either not required or assigned more than one time. Finally, we penalize cognitive workload associated with de-selecting defaulted ON items, selecting defaulted OFF items and confirming items.

$$ \begin{array}{ll} &\text{minimize} \qquad z = \sum\limits_{h\in\mathcal{H}} \sum\limits_{p\in\mathcal{P}_{h}} \sum\limits_{i\in\mathcal{I}_{p,h}} c^{\text{alc}} \cdot x^{\text{alc}}_{h,p,i} + \sum\limits_{h\in\mathcal{H}} \sum\limits_{p\in\mathcal{P}_{h}} \sum\limits_{o\in\mathcal{O}} c^{\text{os}} \cdot x^{\text{os}}_{h,p,o}\\ &\qquad\qquad\qquad\quad\;\; +\sum\limits_{h\in\mathcal{H}}\sum\limits_{p\in\mathcal{P}_{h}} \left( c^{\mathrm{off,non-req}} \cdot \sum\limits_{i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h}} \sum\limits_{o\in\mathcal{O}} x^{\mathrm{on,off}}_{h,p,i,o} + c^{\mathrm{off,mult}} \cdot \sum\limits_{i\in\mathcal{I}_{h}} x^{\mathrm{m,on}}_{h,p,i}\right)\\ &\qquad\qquad\qquad\quad\;\; + \sum\limits_{h\in\mathcal{H}}\sum\limits_{p\in\mathcal{P}_{h}} c^{\mathrm{off,on}} \cdot \sum\limits_{i\in\mathcal{I}_{p,h}} \sum\limits_{o\in\mathcal{O}} x^{\mathrm{off,on}}_{h,p,i,o} + \sum\limits_{h\in\mathcal{H}} \sum\limits_{p\in\mathcal{P}_{h}} \sum\limits_{i\in\mathcal{I}_{p,h}} \sum\limits_{o\in\mathcal{O}} c^{\mathrm{conf,on}} \cdot x^{\mathrm{conf,on}}_{h,p,i,o}\\ &\qquad\qquad\qquad\quad\;\; + \sum\limits_{h\in\mathcal{H}} \sum\limits_{p\in\mathcal{P}_{h}} \sum\limits_{i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h}} \sum\limits_{o\in\mathcal{O}} c^{\mathrm{conf,off}} \cdot x^{\mathrm{conf,off}}_{h,p,i,o} \end{array} $$
(1)

subject to

$$\begin{array}{@{}rcl@{}} && x^{\text{alc}}_{h,p,i} + \sum\limits_{o\in\mathcal{O}} \left( x^{\mathrm{conf,on}}_{h,p,i,o} + x^{\text{off,on}}_{h,p,i,o} \right) = 1 \quad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, i\in\mathcal{I}_{p,h} \end{array} $$
(2)
$$\begin{array}{@{}rcl@{}} && x^{\text{os}}_{h,p,o} + a^{\text{on}}_{h,i,o} - x^{\mathrm{on,off}}_{h,p,i,o} \leq 1 \qquad\qquad\,\,\,\,\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, o\in\mathcal{O},i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h} \end{array} $$
(3)
$$\begin{array}{@{}rcl@{}} && x^{\text{os}}_{h,p,o} + a^{\text{off}}_{h,i,o} - x^{\mathrm{conf,off}}_{h,p,i,o} \leq 1 \qquad\qquad\, \forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, o\in\mathcal{O},i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h} \end{array} $$
(4)
$$\begin{array}{@{}rcl@{}} &&a^{\text{off}}_{h,i,o} - x^{\text{off,on}}_{h,p,i,o} \geq 0 \qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h},o\in\mathcal{O},i\in\mathcal{I}_{p,h} \end{array} $$
(5)
$$\begin{array}{@{}rcl@{}} &&a^{\text{on}}_{h,i,o} - x^{\mathrm{on,off}}_{h,p,i,o} \geq 0 \qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, o\in\mathcal{O},i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h} \end{array} $$
(6)
$$\begin{array}{@{}rcl@{}} &&x^{\mathrm{os,on}}_{h,p,i,o} \geq x^{\text{os}}_{h,p,o} + a^{\text{on}}_{h,i,o} - 1 \qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, o\in\mathcal{O},i\in\mathcal{I}_{p,h} \end{array} $$
(7)
$$\begin{array}{@{}rcl@{}} &&x^{\text{m,on}}_{h,p,i} \geq \sum\limits_{o\in\mathcal{O}} x^{\mathrm{os,on}}_{h,p,i,o} - 1\qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h},i\in\mathcal{I}_{p,h} \end{array} $$
(8)
$$\begin{array}{@{}rcl@{}} &&a^{\text{on}}_{h,i,o} - x^{\mathrm{conf,on}}_{h,p,i,o} \geq 0 \qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h},o\in\mathcal{O},i\in\mathcal{I}_{p,h} \end{array} $$
(9)
$$\begin{array}{@{}rcl@{}} &&a^{\text{off}}_{h,i,o} - x^{\mathrm{conf,off}}_{h,p,i,o} \geq 0 \qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h},o\in\mathcal{O},i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h} \end{array} $$
(10)
$$\begin{array}{@{}rcl@{}} &&a^{\text{on}}_{h,i,o} + a^{\text{off}}_{h,i,o} \leq 1\qquad\qquad\forall h\in\mathcal{H}, i\in\mathcal{I}_{h},o\in\mathcal{O} \end{array} $$
(11)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{h,p,o} - x^{\text{off,on}}_{h,p,i,o} \geq 0\qquad\qquad\forall h\in\mathcal{H},p\in\mathcal{P}_{h},i\in\mathcal{I}_{h,p},o\in\mathcal{O} \end{array} $$
(12)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{h,p,o} - x^{\mathrm{conf,on}}_{h,p,i,o} \geq 0\qquad\qquad\forall h\in\mathcal{H},p\in\mathcal{P}_{h},i\in\mathcal{I}_{h,p},o\in\mathcal{O} \end{array} $$
(13)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{h,p,o} - x^{\mathrm{on,off}}_{h,p,i,o} \geq 0\qquad\qquad\forall h\in\mathcal{H},p\in\mathcal{P}_{h},i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h},o\in\mathcal{O} \end{array} $$
(14)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{h,p,o} - x^{\mathrm{conf,off}}_{h,p,i,o} \geq 0\qquad\qquad\forall h\in\mathcal{H},p\in\mathcal{P}_{h},i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h},o\in\mathcal{O} \end{array} $$
(15)
$$\begin{array}{@{}rcl@{}} &&a^{\text{on}}_{h,i,o}, a^{\text{off}}_{h,i,o}\in \{0,1\}\qquad\qquad\forall h\in\mathcal{H}, i\in\mathcal{I}_{h}, o\in\mathcal{O} \end{array} $$
(16)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{h,p,o} \in \{0,1\}\qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, o\in\mathcal{O} \end{array} $$
(17)
$$\begin{array}{@{}rcl@{}} &&x^{\text{alc}}_{h,p,i}, x^{\text{m,on}}_{h,p,i} \in \{0,1\}\qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, i\in\mathcal{I}_{p,h} \end{array} $$
(18)
$$\begin{array}{@{}rcl@{}} &&x^{\mathrm{conf,off}}_{h,p,i,o}, x^{\mathrm{on,off}}_{h,p,i,o} \in \{0,1\}\qquad\qquad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h},o\in\mathcal{O} \end{array} $$
(19)
$$\begin{array}{@{}rcl@{}} &&x^{\mathrm{conf,on}}_{h,p,i,o}, x^{\mathrm{os,on}}_{h,p,i,o}, x^{\text{off,on}}_{h,p,i,o} \in \{0,1\}\qquad\quad\forall h\in\mathcal{H}, p\in\mathcal{P}_{h}, i\in\mathcal{I}_{p,h},o\in\mathcal{O} \end{array} $$
(20)

Objective function (1) minimizes cognitive workload for selecting patients’ order items from a la carte, assigning patients to order sets, deselecting defaulted ON order items from order sets, selecting defaulted OFF order items from order sets, confirming defaulted ON order items within order sets and confirming defaulted OFF order items within order sets. We will denote the different terms of the objective function as z alc, z os, z off,non-req, z off,mult, z off,on, z conf,on and z conf,off. Constraints (2) ensure that each patient’s required order item is either selected a la carte or it is selected from order sets. If it is selected from order sets, the order item is confirmed defaulted ON or it is switched on because it is defaulted OFF. Constraints (3) ensure that if a patient is assigned to an order set and a non-required order item is defaulted ON, then it has to be de-selected. Constraints (4) ensure that if a patient is assigned to an order set and a non-required order item is defaulted OFF, then it has to be confirmed to be OFF. Constraints (5) ensure that if a patient’s order item is switched ON from defaulted OFF, it has to be defaulted OFF in the corresponding order set. Constraints (6) ensure that if a patient’s non-required order item is switched OFF from defaulted ON, it has to be defaulted ON in the corresponding order set. Constraints (7) ensure that if the patient is assigned to an order set and the order item is defaulted ON, the \(x^{\mathrm {os,on}}_{h,p,i,o}\)-variables have to be 1. Using these variables, Constraints (8) ensure that if the patient’s required order item is selected multiple times, it has to be counted by the auxiliary decision variables. We assume that there is a mechanism in the information system which detects whether or not the item is prescribed more than once. The user then decides through one click with a cognitive workload c off,mult that all over-prescribed items are de-selected automatically in order sets. Constraints (9) ensure that a patient’s required order item can only be confirmed on if it is defaulted ON in the corresponding order set. Constraints (10) ensure that a patient’s required order item can only be confirmed OFF if it is defaulted OFF in the corresponding order set. Constraints (11) ensure that an order item cannot be defaulted ON and defaulted OFF in the same order set at the same time interval. Constraints (12) ensure that if a patient’s order item is switched on from defaulted OFF in an order set, the patient has to be assigned to the corresponding order set. Constraints (13) ensure that if a patient’s order item is switched OFF from defaulted ON in an order set, the patient has to be assigned to the corresponding order set. Constraints (14) ensure that if a patient’s defaulted ON order item is switched OFF, the patient has to be assigned to the corresponding order set. Constraints (15) ensure that if a patient’s defaulted OFF order item is confirmed OFF, the patient has to be assigned to the corresponding order set. Equations 16–20 are the decision variables and their domain.

3.2 Dominance properties and fixing variables

Our model simplifies to the one presented in Gartner et al. [12] if the coefficients of the last three objective function terms are equal to zero and the remaining coefficients are equal to 1. This means that each a la carte and order set selection is weighted by 1 which corresponds to one mouse click. Similarly, if an order item is not required in a prescribed order set or if it is prescribed multiple times, its deselection is also weighted by 1 mouse click. As a result, the simplified objective function is as shown in Eq. 21.

$$\begin{array}{@{}rcl@{}} \textnormal{minimize} \;\; z &\,=\,\!& \sum\limits_{h\in\mathcal{H}} \sum\limits_{p\in\mathcal{P}_{h}} \sum\limits_{i\in\mathcal{I}_{p,h}} x^{\text{alc}}_{h,p,i} +\! \sum\limits_{h\in\mathcal{H}} \sum\limits_{p\in\mathcal{P}_{h}} \sum\limits_{o\in\mathcal{O}} x^{\text{os}}_{h,p,o} \\ && +\!\sum\limits_{h\in\mathcal{H}}\sum\limits_{p\in\mathcal{P}_{h}} \left( \sum\limits_{i\in\mathcal{I}_{h}:i\notin\mathcal{I}_{p,h}} \sum\limits_{o\in\mathcal{O}} x^{\mathrm{on,off}}_{h,p,i,o} \,+\,\! \sum\limits_{i\in\mathcal{I}_{h}} x^{\text{m,on}}_{h,p,i}\right)\\ \end{array} $$
(21)

However, if the cognitive workload coefficients are not equal to zero which Section 4.3 will reveal, objective function (1) has to be used. In the following, we show dominance properties for the cognitive workload model.

Proposition 1

If c alc , c os , and c conf,on ≥ 0, and, for patient p, inequalities \(c^{\text {alc}} \leq c^{\text {os}} + |\mathcal {I}_{p,{h}}| \cdot c^{\text {conf,on}}\) and \(c^{\text {alc}} \leq c^{\text {os}} + |\mathcal {I}_{p,{h}}| \cdot c^{\text {off,on}}\) hold true, the assignment of all patient p’s order items to a la carte is non-dominated. If all patients have this property, none of the order set assignment, confirmation, or switching variables are required.

An example can be constructed as follows: Assume, we have the cost coefficients as given in Section 4.3 and we solve a subproblem with demand as follows: \(\mathcal {I} := \{\)0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32 }. The individual patient demand is: \(\mathcal {I}_{0} := \{0,1,2,3,4,5\}\), \(\mathcal {I}_{1} := \{6,7,8\}\), \(\mathcal {I}_{2} := \{9,10\}\), \(\mathcal {I}_{3} := \{11,12,13,14\}\), \(\mathcal {I}_{4} := \{1,15,16\}\), \(\mathcal {I}_{5} := \{17\}\), \(\mathcal {I}_{6} := \{17\}\), \(\mathcal {I}_{7} := \{17\}\), \(\mathcal {I}_{8} := \{18,19,20,21\}\), \(\mathcal {I}_{9} := \{22,23,24,25,26,27,28,29,30,31,32\}\). Then, fetching all order items from a la carte is optimal because even designing one order set for patient p = 9 who requires \(|\mathcal {I}_{9}|=11\) order items will not lead to a better objective function value as compared to selecting all the patient’s order items from a la carte.

Proposition 2

Switching order items from defaulted OFF to ON is dominated if c off,on > c alc≥ 0. As a consequence, decision variables \(x^{\text {off,on}}_{h,p,i,o}\) can be fixed to zero.

This property will be confirmed in our computational study where c off,on > c alc (see Section 4.3). As a consequence, we can also remove the defaulted OFF variables \(a^{\text {off}}_{h,i,o}\) and modify the corresponding constraints in the model.

3.3 The time-interval-based MIP decomposition approach

We build on the work of Gartner et al. [12] where, similar to our cognitive workload optimization model, time interval-connectivity is absent. However, the subproblems are different as we will show in the remainder of this subsection.

The time-interval-based MIP decomposition (TID) approach is provided by Algorithm 1.

figure a

In line 1–3, we solve the MIP subproblems of each time interval using model formulation (22)–(41). A problem in a time interval can still be challenging as our computational results will reveal. This is where the MIP decomposition can be used as a heuristic by letting the solver execute for a pre-specified amount of time or until an LP-relaxation gap is reached. Finally, we sum up all subproblems’ objective function values to one global objective function value. The subproblem for each time interval reads as follows where we can safely remove index h from the decision variables and constraints:

$$\begin{array}{@{}rcl@{}} \text{minimize} \quad z &=& \sum\limits_{p\in\mathcal{P}} \sum\limits_{i\in\mathcal{I}_{p}} c^{\text{alc}} \cdot x^{\text{alc}}_{p,i} + \sum\limits_{p\in\mathcal{P}} \sum\limits_{o\in\mathcal{O}} c^{\text{os}} \cdot x^{\text{os}}_{p,o}\\ && +\sum\limits_{p\in\mathcal{P}} \left( c^{\text{off,non-req}} \cdot \sum\limits_{i\in\mathcal{I}:i\notin\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} x^{\mathrm{on,off}}_{p,i,o} + c^{\text{off,mult}} \cdot \sum\limits_{i\in\mathcal{I}} x^{\text{m,on}}_{p,i}\right)\\ && + \sum\limits_{p\in\mathcal{P}} c^{\text{off,on}} \cdot \sum\limits_{i\in\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} x^{\text{off,on}}_{p,i,o} + \sum\limits_{p\in\mathcal{P}} \sum\limits_{i\in\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} c^{\mathrm{conf,on}} \cdot x^{\mathrm{conf,on}}_{p,i,o}\\ && + \sum\limits_{p\in\mathcal{P}} \sum\limits_{i\in\mathcal{I}:i\notin\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} c^{\mathrm{conf,off}} \cdot x^{\mathrm{conf,off}}_{p,i,o} \end{array} $$
(22)

subject to

$$\begin{array}{@{}rcl@{}} & &x^{\text{alc}}_{p,i} + \sum\limits_{o\in\mathcal{O}} \left( x^{\mathrm{conf,on}}_{p,i,o} + x^{\text{off,on}}_{p,i,o} \right) = 1 \qquad\forall p\in\mathcal{P}, i\in\mathcal{I}_{p} \end{array} $$
(23)
$$\begin{array}{@{}rcl@{}} & &x^{\text{os}}_{p,o} + a^{\text{on}}_{i,o} - x^{\mathrm{on,off}}_{p,i,o} \leq 1 \qquad\qquad \forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(24)
$$\begin{array}{@{}rcl@{}} & &x^{\text{os}}_{p,o} + a^{\text{off}}_{i,o} - x^{\mathrm{conf,off}}_{p,i,o} \leq 1 \qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(25)
$$\begin{array}{@{}rcl@{}} & &a^{\text{off}}_{i,o} - x^{\text{off,on}}_{p,i,o} \geq 0 \qquad\qquad\forall p\in\mathcal{P},o\in\mathcal{O},i\in\mathcal{I}_{p} \end{array} $$
(26)
$$\begin{array}{@{}rcl@{}} & &a^{\text{on}}_{i,o} - x^{\mathrm{on,off}}_{p,i,o} \geq 0 \qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(27)
$$\begin{array}{@{}rcl@{}} & &x^{\mathrm{os,on}}_{p,i,o} \geq x^{\text{os}}_{p,o} + a^{\text{on}}_{i,o} - 1 \qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}_{p} \end{array} $$
(28)
$$\begin{array}{@{}rcl@{}} & &x^{\text{m,on}}_{p,i} \geq \sum\limits_{o\in\mathcal{O}} x^{\mathrm{os,on}}_{p,i,o} - 1\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}_{p} \end{array} $$
(29)
$$\begin{array}{@{}rcl@{}} & &a^{\text{on}}_{i,o} - x^{\mathrm{conf,on}}_{p,i,o} \geq 0 \qquad\qquad\forall p\in\mathcal{P},o\in\mathcal{O},i\in\mathcal{I}_{p} \end{array} $$
(30)
$$\begin{array}{@{}rcl@{}} & &a^{\text{off}}_{i,o} - x^{\mathrm{conf,off}}_{p,i,o} \geq 0 \qquad\qquad\forall p\in\mathcal{P},o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(31)
$$\begin{array}{@{}rcl@{}} & &a^{\text{on}}_{i,o} + a^{\text{off}}_{i,o} \leq 1\qquad\qquad\forall i\in\mathcal{I},o\in\mathcal{O} \end{array} $$
(32)
$$\begin{array}{@{}rcl@{}} & &x^{\text{os}}_{p,o} - x^{\text{off,on}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(33)
$$\begin{array}{@{}rcl@{}} & &x^{\text{os}}_{p,o} - x^{\mathrm{conf,on}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(34)
$$\begin{array}{@{}rcl@{}} & &x^{\text{os}}_{p,o} - x^{\mathrm{on,off}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}:i\notin\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(35)
$$\begin{array}{@{}rcl@{}} & &x^{\text{os}}_{p,o} - x^{\mathrm{conf,off}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}:i\notin\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(36)
$$\begin{array}{@{}rcl@{}} & &a^{\text{on}}_{i,o}, a^{\text{off}}_{i,o}\in \{0,1\}\qquad\qquad\forall h\in\mathcal{H}, i\in\mathcal{I}, o\in\mathcal{O} \end{array} $$
(37)
$$\begin{array}{@{}rcl@{}} & &x^{\text{os}}_{p,o} \in \{0,1\}\qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O} \end{array} $$
(38)
$$\begin{array}{@{}rcl@{}} & &x^{\text{alc}}_{p,i}, x^{\text{m,on}}_{p,i} \in \{0,1\}\qquad\qquad\forall p\in\mathcal{P}, i\in\mathcal{I}_{p} \end{array} $$
(39)
$$\begin{array}{@{}rcl@{}} & &x^{\mathrm{conf,off}}_{p,i,o}, x^{\mathrm{on,off}}_{p,i,o} \in \{0,1\}\qquad\qquad\forall p\in\mathcal{P}, i\in\mathcal{I}:i\notin\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(40)
$$\begin{array}{@{}rcl@{}} & &x^{\mathrm{conf,on}}_{p,i,o}, x^{\mathrm{os,on}}_{p,i,o}, x^{\text{off,on}}_{p,i,o} \in \{0,1\}\qquad\quad\forall p\in\mathcal{P}, i\in\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(41)

3.4 The time-interval-based K −means heuristic approach

A similar idea is followed by the K −means based, heuristic, time-interval decomposition approach. However, it not only decomposes the MIP into time intervals but it also disentangles the clustering decisions from the assignment decisions represented by the a and x decision variables, respectively. Again, this approach is an extension of Gartner et al. [12] because we have a cognitive workload optimization objective leading to a different assignment problem where patients are assigned (by x-variables) to the order sets which are now generated by the K −means algorithm. Algorithm 2 gives an overview of the heuristic.

figure b

We can determine the K–means cluster centroids shown in Line 2 by any implementation of Vinod [24]’s well known K −means algorithm where the Euclidean distance can be used to measure and improve the distance between the centroids and the orders required for each instance (patient). Once we have found the centroids, in the matrix χ i,o , we insert them as parameters into the following subproblem which we solve for each interval individually (note that, again, we remove the interval index h):

$$\begin{array}{@{}rcl@{}} \text{minimize} \quad z &=& \sum\limits_{p\in\mathcal{P}} \sum\limits_{i\in\mathcal{I}_{p}} c^{\text{alc}} \cdot x^{\text{alc}}_{p,i} + \sum\limits_{p\in\mathcal{P}} \sum\limits_{o\in\mathcal{O}} c^{\text{os}} \cdot x^{\text{os}}_{p,o}\\ &&+\sum\limits_{p\in\mathcal{P}} \left( c^{\text{off,non-req}} \cdot \sum\limits_{i\in\mathcal{I}:i\notin\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} x^{\mathrm{on,off}}_{p,i,o}\right.\\ && \qquad\quad \left. +~c^{\text{off,mult}} \cdot \sum\limits_{i\in\mathcal{I}} x^{\text{m,on}}_{p,i}\right)\\ && + \sum\limits_{p\in\mathcal{P}} c^{\text{off,on}} \cdot \sum\limits_{i\in\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} x^{\text{off,on}}_{p,i,o}\\ && + \sum\limits_{p\in\mathcal{P}} \sum\limits_{i\in\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} c^{\mathrm{conf,on}} \cdot x^{\mathrm{conf,on}}_{p,i,o}\\ && + \sum\limits_{p\in\mathcal{P}} \sum\limits_{i\in\mathcal{I}:i\notin\mathcal{I}_{p}} \sum\limits_{o\in\mathcal{O}} c^{\mathrm{conf,off}} \cdot x^{\mathrm{conf,off}}_{p,i,o} \end{array} $$
(42)

subject to

$$\begin{array}{@{}rcl@{}} x^{\text{alc}}_{p,i} + \sum\limits_{o\in\mathcal{O}} \left( x^{\mathrm{conf,on}}_{p,i,o} + x^{\text{off,on}}_{p,i,o} \right) = 1 \qquad\forall p\in\mathcal{P}, i\in\mathcal{I}_{p} \end{array} $$
(43)
$$\begin{array}{@{}rcl@{}} && x^{\text{os}}_{p,o} - x^{\mathrm{on,off}}_{p,i,o} \leq 1 - \chi_{i,o} \qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(44)
$$\begin{array}{@{}rcl@{}} && x^{\text{os}}_{p,o} + a^{\text{off}}_{i,o} - x^{\mathrm{conf,off}}_{p,i,o} \leq 1 \qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(45)
$$\begin{array}{@{}rcl@{}} &&a^{\text{off}}_{i,o} - x^{\text{off,on}}_{p,i,o} \geq 0 \qquad\qquad\forall p\in\mathcal{P},o\in\mathcal{O},i\in\mathcal{I}_{p} \end{array} $$
(46)
$$\begin{array}{@{}rcl@{}} && x^{\mathrm{on,off}}_{p,i,o} \leq \chi_{i,o} \qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(47)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{p,o} - x^{\mathrm{os,on}}_{p,i,o} \leq 1 - \chi_{i,o} \qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O},i\in\mathcal{I}_{p} \end{array} $$
(48)
$$\begin{array}{@{}rcl@{}} &&x^{\text{m,on}}_{p,i} \geq \sum\limits_{o\in\mathcal{O}} x^{\mathrm{os,on}}_{p,i,o} - 1\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}_{p} \end{array} $$
(49)
$$\begin{array}{@{}rcl@{}} && x^{\mathrm{conf,on}}_{p,i,o} \leq \chi_{i,o} \qquad\qquad\forall p\in\mathcal{P},o\in\mathcal{O},i\in\mathcal{I}_{p} \end{array} $$
(50)
$$\begin{array}{@{}rcl@{}} &&a^{\text{off}}_{i,o} - x^{\mathrm{conf,off}}_{p,i,o} \geq 0 \qquad\qquad\forall p\in\mathcal{P},o\in\mathcal{O},i\in\mathcal{I}:i\notin\mathcal{I}_{p} \end{array} $$
(51)
$$\begin{array}{@{}rcl@{}} && a^{\text{off}}_{i,o} \leq 1 - \chi_{i,o}\qquad\qquad\forall i\in\mathcal{I},o\in\mathcal{O} \end{array} $$
(52)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{p,o} - x^{\text{off,on}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(53)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{p,o} - x^{\mathrm{conf,on}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(54)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{p,o} - x^{\mathrm{on,off}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}:i\notin\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(55)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{p,o} - x^{\mathrm{conf,off}}_{p,i,o} \geq 0\qquad\qquad\forall p\in\mathcal{P},i\in\mathcal{I}:i\notin\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(56)
$$\begin{array}{@{}rcl@{}} &&a^{\text{off}}_{i,o}\in \{0,1\}\qquad\qquad\forall h\in\mathcal{H}, i\in\mathcal{I}, o\in\mathcal{O} \end{array} $$
(57)
$$\begin{array}{@{}rcl@{}} &&x^{\text{os}}_{p,o} \in \{0,1\}\qquad\qquad\forall p\in\mathcal{P}, o\in\mathcal{O} \end{array} $$
(58)
$$\begin{array}{@{}rcl@{}} &&x^{\text{alc}}_{p,i}, x^{\text{m,on}}_{p,i} \in \{0,1\}\qquad\qquad\forall p\in\mathcal{P}, i\in\mathcal{I}_{p} \end{array} $$
(59)
$$\begin{array}{@{}rcl@{}} &&x^{\mathrm{conf,off}}_{p,i,o}, x^{\mathrm{on,off}}_{p,i,o} \in \{0,1\}\qquad\qquad\forall p\in\mathcal{P}, i\in\mathcal{I}:i\notin\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(60)
$$\begin{array}{@{}rcl@{}} &&x^{\mathrm{conf,on}}_{p,i,o}, x^{\mathrm{os,on}}_{p,i,o}, x^{\text{off,on}}_{p,i,o} \in \{0,1\}\qquad\quad\forall p\in\mathcal{P}, i\in\mathcal{I}_{p},o\in\mathcal{O} \end{array} $$
(61)

As can be observed, for example, Constraints (44) are similar to Constraints (24) with the exception that the defaulted ON variables become now a parameter, not a decision variable. This holds true for other constraints in which a on-variables are used.

4 Experimental analysis using data on patients suffering from a chronic condition

In the following, we provide an experimental investigation of the presented methods. We first give an overview of the data employed for our study, followed by an analysis of computation times and the optimality gap. We then turn to the physician’s cognitive workload analysis of the hospital’s current solution and compare it with our optimal and heuristic approach. The results are broken down by different metrics and levels of detail.

4.1 Data

We evaluated our approaches on data from a major U.S. pediatric hospital and focused on Asthma patients with moderate complexity. In total, we observed 106 patients who were prescribed 9141 order items within 24 hours before and after admission to the inpatient setting. In the current system, 32 unique order sets were used for this condition along with a la carte orders while the total number of unique order items in the entire CPOE system adds up to 3335. We joined usage data from the current CPOE system with data from the electronic medical record. In doing so, we obtained time stamps for the current order set assignments and patient demand, among others. This allows us to generate all parameters for our exact and heuristic approaches and to compare the solution with the physicians’ current cognitive workload.

4.2 Computational complexity and optimality gap analysis

All computations were performed on an Intel Core i7-4700MQ CPU with 32 GB RAM running Windows 7 operating system. The models were coded in Java in an ILOG Concert environment. The solver used was ILOG CPLEX 12.7 (64 bit) and we used the K–means algorithm as implemented in WEKA [26]. We chose to split the planning horizon into H = 9 intervals as follows: [−24,−4.45], [−4.45,−2], [−2,0] , [0,1], [1,2] , [2,5] , [5,10] , [10,15] and [15,24] hours with respect to each patient’s admission time point. In setting these intervals, it is guaranteed that at least 71 patients require at least 125 distinct orders in each interval. Another motivation not to choose smaller time intervals is to prevent clinicians from having to revise the order sets too frequently.

4.3 Cognitive workload definition

The six key components of physicians’ cognitive workload are listed in Section 2.3. We assume that the objective function weights, or cognitive cost coefficients, are independent of the patient, order set, time interval and physician. The coefficients are obtained through a survey of 15 respondents including physicians and nurses. A paper-based survey was given to subjects during an order set development workshop and a Biomedical Informatics conference, where subjects completed the survey on the spot in about 5 minutes [28]. The survey contains 6 questions with sub-questions, asking the subjects to estimate the time to pick an order set while treating patients. More specifically, the subjects were asked to identify the time it takes them to 1) choose a default-ON item, 2) not choose a default-ON item, 3) choose a default-OFF item, and 4) leave a default-OFF item as off under large, mid-size, and small order sets. According to the survey results, there are some defaulted-ON items in the general lab order sets which should not be given to patients with a particular diagnosis. Therefore, close attention has to be paid to the defaulted-ON items to ensure the safety of patients, and thus the cognitive workload associated with leaving a defaulted-ON item as ON is higher than what it would be in an ideal situation. Contrarily, physicians often have to go through a list of defaulted-OFF items to find items that patients need because being defaulted-OFF does not necessarily guarantee clinical irrelevance either. Therefore, the cognitive workload for leaving those as OFF is also higher than an ideal situation.

Following the results from the study of Zhang [28], we chose to set the cognitive workload coefficients as follows [29]: c alc = 1.1, c os = 1.1, c off,non-req = 1.3, c off,mult = 1.3, c conf,on = 1.0c conf,off = 1.1 and c off,on = 1.4. The figures reveal that all CW coefficients are greater than or equal to one. However, since c off,on ≥ c alc, switching order items from default OFF to ON is dominated by a la carte assignment.

To evaluate the computational complexity of the order set optimization problem, we vary the number of order sets by setting O = 1,2 and 5. Table 2 shows the computational results where we limit the computation times of each of the MIP subproblems in the MIP decomposition to 3,600s. Once we solve the MIP decomposition with its 9 subproblems, we store the computation time and set it as a limit for the full MIP formulation given in Eqs. 1–20. This ensures comparability of the computational results.

Table 2 Computation time analysis results for cognitive workload minimization
Full size table

The figures reveal that in the case of O = 1 order set, running Model (1)–(20) which is the MIP without decomposition, 528s are required for an 0.08% optimality gap. However, with one order set generated in each interval and the same computation time, the decomposed MIP can be solved to optimality. The heuristic decomposition solves the same problem more than 60 times faster as compared to the MIP decomposition. The drawback is, however, that the optimality gap between the MIP with decomposition and the K −means based heuristic is 0.29%.

In the case of O = 2 order sets, and the MIP decomposition, we observed that two subproblems couldn’t be solved within 3,600s time. A more detailed analysis revealed that the LP relaxation gap across all 9 subproblems is 0.3%. Comparing the MIP decomposition results with the heuristic decomposition results, we have a gap in the objective function value of 39.5. In other words, using the MIP, the objective function value of the K −means approach can be improved by 0.47% with the drawback of longer computation times.

In the case of O = 5 order sets, we observed that in the case of the MIP decomposition, 4 subproblems couldn’t be solved to optimality within the 3,600s time limit of each subproblem. One explanation for this phenomenon is the increase in the model size handed over to CPLEX. Now, comparing the MIP decomposition results with the heuristic results, we have a gap in the objective function value of 51.2.

Overall, the results show that all computation times increase with increasing number of order sets O. The increase in computation times is, however, substantial in the case of the MIP approaches either used as a full model formulation or as a time-index based decomposition version. We also observe that the number of constraints is larger for the K −means-based approach as compared to the MIP-based decomposition approach. One explanation for this phenomenon is that, for simplicity, we fixed the clustering decision variables by additional constraints rather than incorporating them as constants into the subproblems.

4.4 Cognitive workload analysis

We now turn to the cognitive workload analysis results which are broken down by i) the current configuration in the hospital, ii) the MIP decomposition approach and iii) the K −means based approach.

4.4.1 Cognitive workload analysis of the hospital’s current configuration

Figure 2 provides an analysis of cognitive physician workload in the current configuration of the hospital’s CPOE system.

Fig. 2
figure 2

Cognitive workload associated with current order set usage

Full size image

The figure reveals that a la carte selections contribute to 52.6% of the entire cognitive workload. The results confirm a trend observed in Gartner et al. [12]’s small sample results where the main contributor of physical workload was identified as a la carte selections. However, since this study goes beyond the work of Gartner et al. [12], we focus on a detailed evaluation of cognitive workload associated with the confirmation of order items. A closer analysis revealed that 28.5% of the total cognitive workload is attributed to the confirmation of order items wich are defaulted ON in order sets.

4.4.2 Cognitive workload analysis of the MIP approach

The figures from the MIP approach with time-interval-based decomposition are shown in Figs. 3, 4 and 5 where the number of order sets is bounded by O = 1,2 and 5, respectively. Figure 3 reveals a substantial drop-off between the cognitive workload observed with the current hospital’s configuration and the MIP decomposition approach already by allowing for O = 1 order set. Here, the total potential to reduce cognitive workload is 19.7%.

Fig. 3
figure 3

Cognitive workload associated with order sets built using the MIP decomposition approach and O = 1 order set

Full size image
Fig. 4
figure 4

Cognitive workload associated with order sets built using the MIP decomposition approach and O = 2 order sets

Full size image
Fig. 5
figure 5

Cognitive workload associated with order sets built using the MIP decomposition approach and O = 5 order sets

Full size image

Now, with setting O = 2 order sets, cognitive workload can be reduced further which is shown in Fig. 4.

The results when O = 5 order sets are generated are shown in Fig. 5. The figure reveals that the cognitive workload drops to a value of CW = 8,386.3 which is a 20.2% reduction as compared to the hospital’s current configuration where the cognitive workload is CW = 10,080.2, shown in Fig. 2.

The results from the MIP decomposition approach reveal that cognitive workload from selecting order items a la carte is the biggest driver in cognitive workload. In none of the results did we observe a switching between defaulted OFF to ON, which confirms our Proposition 2. Finally, the second largest driver in cognitive workload is the confirmation of defaulted ON order items which contributes to 14.0% of the cognitive workload when the number of order sets is set to O = 5.

4.4.3 Cognitive workload analysis of the K −means based approach

The cognitive workload analysis for the K −means based approach are shown in Figs. 6, 7 and 8. Figure 6 reveals a remarkable pattern that in all but the admission time interval [0,1], none of the patients were assigned to order sets. A more detailed analysis of the admission time interval revealed that of the 100 patients who were admitted, 12 patients were assigned to the order set with a total number of 120 order items confirmed defaulted ON. This is a valuable insight because, obviously, decomposing the MIP into the K −means based clustering problem that determines the a variables and a MIP-based assignment problem that determines the x variables fails. A closer look into the order set composition revealed a pattern that only in the admission time interval, the number of items defaulted ON is 13. All other order sets consist of less than 10 order items defaulted ON. Therefore, assigning patients to order sets is dominated by a la carte decisions, detailed in Proposition 2.

Fig. 6
figure 6

Cognitive workload associated with order sets built using the K −means based approach and O = 1 order set

Full size image
Fig. 7
figure 7

Cognitive workload associated with order sets built using the K −means based approach and O = 2 order sets

Full size image
Fig. 8
figure 8

Cognitive workload associated with order sets built using the K −means based approach and O = 5 order sets

Full size image

The results when O = 2 order sets are built are shown in Fig. 7. The figure shows a similar pattern as compared to the case where O = 1 order set is built using K −means algorithm: Only in 4 out of 9 time intervals, patients are assigned to order sets.

The case where O = 5 order sets are generated is shown in Fig. 8. The barplot shows that only in time interval [10,15] no order sets are assigned to patients.

As a summary of our K −means-based heuristic decomposition, we can conclude that the cognitive workload can be reduced by 19.3% to 19.5% by allowing 1 to 5 order sets in each time interval, respectively.

4.5 Order set size and number of a la carte selections

For each time interval h, we now report the average order set size (OSS h ) and a la carte selection count (ALC h ). The average order set size metric is computed as follows:

$$ \text{OSS}_{h}=\frac{\sum\limits_{i\in\mathcal{I}_{h}}\sum\limits_{o\in\mathcal{O}} a^{\text{on}}_{h,i,o}}{|\mathcal{O}|}. $$
(62)

Since the \(a^{\text {on}}_{h,i,o}\)-variables are represented by the constants χ h,i,o in each of the subproblem h of the K −means approach, we calculate the order set size for this approach as:

$$ \text{OSS}_{h}=\frac{\sum\limits_{i\in\mathcal{I}_{h}}\sum\limits_{o\in\mathcal{O}} \chi_{h,i,o}}{|\mathcal{O}|}. $$
(63)

The a la carte selection count is determined by

$$ \text{ALC}_{h}=\sum\limits_{p\in\mathcal{P}_{h}}\sum\limits_{i\in\mathcal{I}_{h,p}} x^{\text{alc}}_{h,p,i}. $$
(64)

for each time interval \(h\in \mathcal {H}\).

We don’t report how many order items are defaulted OFF because the dominance properties of the objective function coefficients prevents order items defaulted OFF in order sets simply because the objective function coefficients associated with switching items from default OFF to ON are greater or equal to the objective function coefficients associated with a la carte selection. Table 3 shows the results.

Table 3 Average order set size (OSS h ) and number of a la carte selections (ALC h ) for \(|\mathcal {O}|=5\) order sets
Full size table

Remarkably, the K −means approach comes up with bigger order sets as compared to the current configuration in the hospital. Another observation is that the MIP approach comes up with a substantially bigger OSS as compared to the K −means and the current approach. This confirms Proposition 1 because a valid lower bound on the number of items in an order set can be determined by comparing the a la carte with the order set selection cost coefficients. This result extends the study of Gartner et al. [12] by demonstrating how the structural properties hold in the experimental analysis.

4.6 Detailed analysis of the MIP decomposition

In what follows, we break down the MIP decomposition results into computational and solution characteritics as given in Table 4.

Table 4 Computational complexity and results for each subproblem for \(|\mathcal {O}|=5\) order sets
Full size table

The results reveal an interesting pattern. In interval h = 8, the average order set size comes up to 13.8 which is very close to our lower bound on order set size given the cognitive workload coefficients set in Section 4.3. A more detailed analysis of the x −variables revealed that only 4 patients were assigned to order sets and each patient was assigned to a different order set. A closer look into the patient demand showed that these 4 patients were the only ones requiring 12 or more order items. Moreover, the size of the 5th order set which was never assigned to any patient had only 9 order items defaulted ON.

5 A platform for order set optimization

To demonstrate the effectiveness of heuristic and optimal order set optimization for hospital practitioners, we developed a Java-based order set optimization platform. It features a graphical user interface (GUI) in which the current order set configuration in the collaborating hospital’s CPOE system can be displayed and explored. Important metrics such as order set size (OSS h ) are calculated for each interval h, automatically. Another important feature is that order sets generated by our heuristic and optimal approaches can be calculated and displayed. Figures 9 and 10 show the platform which is divided horizontally by the physical and cognitive workload-oriented order set optimization. Since this paper focuses on cognitive workload minimization, the corresponding tab will be examined in greater detail. The panel within this tab is divided into three sub-tabs displaying the hospital’s current setting, heuristic and optimal order set results. For simplicity, we focus on O = 1.

Fig. 9
figure 9

Current order set usage

Full size image
Fig. 10
figure 10

Overview of the optimal order set for O = 1

Full size image

Figure 9 shows the current order set usage in the ‘current solution’ tab and the user can choose between the different time intervals by clicking ‘analyze next interval’.

Subsequently, the user can select the desired order set by clicking ‘analyze next order set’. Once the user has reached the last order set, denoted by O, the counter starts over from order set number 1 in that interval. Similarly, once the last interval is reached, the next click on ‘analyze next interval’ brings the user back to the first interval. The screenshot reveals that interval [-24;-4.45] and order set number 3 are selected. As can be seen, this order set consists of multiple defaulted ON order items and if the user scrolls further down the list, he will see that, in fact, all order items are defaulted ON. For example, a non-ICU venous blood gas test is defaulted ON with item ID 667.

Figure 10 shows the optimal order set configuration where 37 order items are defaulted ON as marked by the [X] in the table’s ‘default-setting’ column. This observation is consistent with the findings from Table 3 where the average number of a la carte clicks is 1,029 which, multiplied by c alc = 1.1, results in the cognitive a la carte cost of 1,131.9 as shown in the decision support tool.

6 Discussion and limitations

In this study, we incorporated the cognitive workload coefficients of Zhang [28] into our model where the cognitive workload was measured for Asthma patients. However, to generalize our study, the cognitive workload coefficients may be different for prescribing order items to patients being treated for other types of conditions, such as acute or surgical conditions.

Another limitation is that the cognitive workload may be physician-specific which were averaged and normalized by Zhang [28]. The results of our model are applicable when the number of patients treated by physicians who have, on average, a cognitive workload structure similar to those we have used. However, in extreme cases, where one physician has a high cognitive workload associated with, say, a la carte selections and that physician treats the majority of the patients, then different results are to be expected. This can, however, be addressed by extending the model to include i) clinician-dependent cognitive workload coefficients, and ii) subsets of patients that are treated by that physician and summing up the cognitive workload in the objective function across clinicians.

7 Summary and conclusions

In this paper, we have tackled the problem of cognitive workload minimization for order set optimization to improve Hospital Information Systems design. Building on the pillars of Design and Cognitive Sciences theory, we bridge both paradigms by mathematical modelling of the cognitive aspects within IS design in the context of our study on optimizing order sets in clinical information systems. Showcasing the applicability of our approach to a real-world problem, our results from comparing a mathematical program with the current order set configuration in the hospital indicates that cognitive workload can be reduced by about 20.2% by allowing 1 to 5 order sets, respectively. The comparison of the K −means based decomposition with the hospital’s current configuration reveals a cognitive workload reduction of about 19.5%, also by allowing 1 to 5 order sets, respectively. We developed a graphical user interface to the optimization platform which allows practitioners to compare current order set usage with optimal and heuristic order set results. Extending and validating our experimental study towards other clinical conditions and reviewing results of order set compositions with clinicians are the next steps in this research.