FormalPara Key Points

We evaluated a drug safety algorithm that detects patients receiving "triple whammy" (TW) therapy at risk of acute kidney injury by considering additional risk factors such as age, kidney function and drug dosage.

The algorithm detected 11 out of 12 patients who developed acute kidney injury during TW therapy. It had a sensitivity of 88.3%, a specificity of 99.7% and a physician acceptance of 77.7%.

The TW algorithm is capable of detecting patients with risk of acute kidney injury and may contribute to the prevention of this serious adverse drug event.

1 Introduction

The term triple whammy (TW) was first introduced in 2000 and refers to the triple-combination of a non-steroidal anti-inflammatory drug (NSAID), a diuretic, and an angiotensin-converting enzyme inhibitor (ACEI) or an angiotensin receptor antagonist (ARA) [1]. TW prescriptions increase the risk of pre-renal acute kidney injury (AKI), particularly in the elderly and at the beginning of TW treatment [2]. The underlying mechanism is a combination of effects on the kidney: renal afferent arteriolar vasoconstriction caused by inhibition of prostaglandin synthesis (from NSAIDs); efferent arteriolar vasodilation (from ACEI/ARAs); and hypovolemia (from diuretics) [3,4,5]. A 2013 case-control study from Lapi et al. revealed that TW increases AKI risk by 31% [6], though this study relied on hospital discharge data in which AKI is often under-reported [7, 8]; more recent studies suggest an even higher risk for AKI with TW therapy. Depending on the setting and the patient group, AKI occurs in 0.9–22.0% of inpatients on TW therapy [2, 6, 9]. In an adverse drug-event reporting database, the median time to onset of this adverse drug reaction was 8 days [10]. The risk of AKI increases with advanced age, pre-existing renal impairment and impaired myogenic response [7, 11], and AKI is associated with prolonged hospitalisation, renal morbidity, and mortality [12, 13].

Clinical decision support systems (CDSSs) provide clinicians with knowledge and patient-specific information to improve medication safety [14]. Medication errors occur in nearly 6% of drug administrations, and CDSSs are a promising approach to reducing medication-related problems [15]. However, the utility of CDSSs for the reduction of medication errors is diminished by alert fatigue, which is caused by alerts that are too frequent, often irrelevant, or even false [16, 17]. Alert fatigue can even lead to fatal events if a drug-drug interaction alert for a potentially life-threatening contraindication is overridden [18]; studies have shown inappropriately high override rates for geriatric and renal alerts [19]. Therefore, developers are moving towards more specific and contextual alerts [20, 21].

Many CDSSs detect only dual drug-drug interactions [22]. However, most patients are treated with more than just two medications, which creates the potential for more complex drug interactions [23]. According to a Swiss study, 18–25% of people over the age of 65 years regularly take five or more medications, making polypharmacy a widespread phenomenon among the elderly [24]. Polypharmacy increases the risk of adverse events [25]. The TW depicts a pharmacodynamic drug-drug interaction between three drug classes. As such, this risky drug combination is often not detected by conventional CDSSs. Several specific algorithms have been developed for primary care [26,27,28,29,30]. Notably, like many drug-drug interactions, a TW prescription is not strictly contraindicated. It can be administered with caution, depending on the clinical situation and the progression of renal function. A CDSS with too sensitive alerts leads to alert fatigue; therefore, more specific algorithms are needed.

We have developed an algorithm and implemented it into our hospital's electronic health record (EHR). In this study, we analysed patients with TW prescriptions in our hospital over the course of one year to assess the performance of a CDSS in detecting patients at risk for AKI. Performance was further evaluated in terms of sensitivity, specificity, and acceptance rate. Special attention was given to patients who experienced an AKI under TW prescription.

2 Methods

2.1 Setting

A retrospective cross-sectional study was conducted in a tertiary care hospital with 497 beds in Switzerland. The hospital uses an in-house CDSS (KPharm) developed by an interdisciplinary team of physicians and pharmacists and implemented directly into the hospital's EHR (KISIM™ by CISTEC, www.cistec.com), which has been described in more detail elsewhere [31]. As of December 2022, the CDSS consisted of 20 different algorithms that all allow for multiple alerts.

The TW algorithm is designed to detect TW prescriptions in patients at risk for AKI. TW drugs are identified via the anatomical therapeutic chemical (ATC) classification system by the World Health Organization. Apart from the current medication prescriptions, the algorithm considers the patient's age and estimated glomerular filtration rate (eGFR) according to the Chronic Kidney Disease Epidemiology Collaboration Formula (CKD-EPI). Prescriptions for on-demand medications are included. Some medication is only considered if the daily dose is above a specific threshold (Online Resource 3, see the electronic supplementary material). In this case, on demand medication is not included in the calculation of the daily dose. The algorithm consists of five alerts (Fig. 1):

  • Alert 1: Triple Whammy; eGFR < 30 ml/min/1.73m2

  • Alert 2: Triple Whammy; eGFR 30–60 ml/min/1.73m2

  • Alert 3: Triple Whammy; age ≥ 75 years

  • Alert 4: Triple Whammy; no current creatinine value

  • Alert 5: Error alert—dose calculation of TW drugs not possible by the system (this alert ensures that no TW is missed because the dose was entered incorrectly)

Fig. 1
figure 1

Triple Whammy Algorithm. Decision pathway of the algorithm for triggering each of the five alerts. The defined thresholds can be found in online resource 3. Alert Number 5, an error message, is not displayed in the figure. It is produced if a dose calculation failed. ACEI Angiotensin-converting enzyme inhibitor, ARA angiotensin receptor antagonist, eGFR estimated glomerular filtration rate according to CKD-EPI, NSAID non-steroidal anti-inflammatory drugs

Full size image

When an alert is triggered, it is first directed to a clinical pharmacist who evaluates the clinical relevance of each alert before either dismissing the alert or sending an intervention message via the EHR to the prescribing physician. Alerts for non-urgent situations may be paused (watchful waiting for 1–3 days), after which a new alert may be generated for the clinical pharmacist to revisit.

The intervention is displayed in the patient's chart as a non-interruptive message that can be seen by all healthcare professionals treating the patient. The message contains information about the patient's age, sex, renal function, the medications involved, and a recommendation for action. The message does not require a response. The prescribing physician can either accept or dismiss the alert to stop it from being displayed. The physician may also choose to keep the alert in the patient's chart as a reminder. The algorithm runs through all patient records once per hour. If the criteria that triggered the alert are no longer valid, the alert is automatically ended by the CDSS.

2.2 Data

For the quantitative analysis, we used routinely collected data from patients hospitalised during 1 Janaury 2021–31 December 2021. Patients at the hospital sign a general consent for the further use of health-related data. We excluded patients aged < 18 years and those who rejected general consent. We identified all patients who either received a TW (documented administration of all the drug classes on one calendar day) or received a TW alert (Fig. 2). The hospital laboratory automatically provides an eGFR with each creatinine measurement, which was also extracted from the EHR. Jupyter Notebook (v.6.1.5) with Python (v.3.9.2) was used for data aggregation, cleaning, and analysis. The Ethics Committee of North-Western and Central Switzerland EKNZ approved the study (Project-ID: EKNZ 2021-01379).

Fig. 2
figure 2

Patient selection. AKI acute kidney injury, TW Triple Whammy

Full size image

2.3 Patients' Characteristics

Descriptive statistics were used to characterise all patients who received a TW, a TW alert, or experienced AKI under a TW prescription in the 2021 calendar year. Patient characteristics were summarised using counts and proportions where appropriate. The duration of TW therapy was compared between the groups of patients receiving TW therapy and patients receiving a TW alert using an unpaired t test. A p value of < 0.05 was considered significant. We also captured the quantities and proportions of medications involved in TW therapy.

Patients with AKI during TW therapy were identified by an increase in serum creatinine to ≥ 1.5 times baseline within 7 days [32]; this is the standard defined by the non-profit organisation Kidney Disease: Improving Global Outcomes (KDIGO). We used the last creatinine measurement before each day with TW treatment as a baseline and matched the highest creatinine measurement in the consecutive 7 days to detect AKI. A pharmacist and a physician reviewed the patients' demographics, etiopathology, chronology of events and continuation of TW medication, and outcome to assess causality as either possible, likely, certain and unlikely according to the World Health Organization Uppsala Monitoring Centre (WHO-UMC) system for standardised case-causality assessment [33].

2.4 Performance of Algorithm

2.4.1 Sensitivity and Specificity

To determine the sensitivity and specificity of the algorithm, we included all inpatients who received at least one drug during hospitalisation. Patients with error messages (Alert 5, n = 6) were excluded from the sensitivity and specificity analysis. We mimicked all relevant aspects of the TW algorithm to identify patients at risk in the retrospective dataset of documented drug administrations, and patients were then assigned according to a pre-defined decision pathway (Online Resource 1, see the electronic supplementary material). Since the daily dose was not available as structured data, this aspect could not be mimicked. Patients who received at least one TW alert were classified as either 'true positive' or 'false positive', depending on the status of the alert. Patients who received multiple alerts, at least one of which resulted in an intervention, were classified as 'true positive'. If the alert was automatically ended by the CDSS during the patient's stay because the criteria that triggered the alert were no longer valid, it was considered 'true positive' as appropriate action was taken regardless of the alert. Patients discharged prior to status determination were reassessed by a pharmacist who reviewed the patient’s medical records. The decision was made on the premise of whether, if the pharmacist had seen the patient before discharge, they would have made an intervention.

2.5 Acceptance Rates

The frequencies of Alerts 1–5 and their corresponding outcomes are reported as counts and percentages. The clinical pharmacist intervention rate is the percentage of processed alerts in which an intervention message was sent. Alerts that were automatically closed by the CDSS were not included. A team of clinical pharmacists followed up on all interventions that resulted from TW alerts. An intervention was defined as accepted if the responsible physician at least partially complied with the suggested recommendation. For example, a recommendation to either discontinue the NSAID or to monitor renal function more closely was considered accepted if the suggested monitoring was carried out. The period during which an intervention was deemed accepted relied on the recommended course of action in the message and its time of transmission. If discontinuation of medication was recommended, we anticipated medication modification on the same day or, if the message was dispatched in the late afternoon, the following morning. In some cases, the acceptance could not be determined, e.g., because the patient was discharged soon after receiving the intervention message. The physician compliance rate was calculated including all TW interventions with known outcomes.

3 Results

3.1 Patient Characteristics

The incidence of TW administration in 2021 was 1.4% (290 of 21,332 patients). Of these, 4.1% (12 of 290) experienced AKI with a possible or likely causal relationship to TW therapy. TW alerts were generated for 216 individual patients (Fig. 2). The characteristics of the patient groups are shown in Table 1.

Table 1 Characteristics of patients receiving Triple Whammy (TW) therapy, receiving a TW alert, and experiencing Acute Kidney Injury (AKI) during TW therapy
Full size table

The median age of the patients who received TW therapy was 70 years (range: 31–96 years), and patients who received a TW alert were older on average than those who did develop AKI under TW therapy. Sex distributions were similar for patients with TW and those receiving an alert, while most patients who experienced AKI were male (n = 9, 75%). Patients on TW therapy received this combination for a median of 3 days (range: 1–23 days). Although there is substantial overlap between patients receiving TW and patients receiving a TW alert, the mean duration of TW therapy was shorter in those receiving a TW alert (3.0 days vs. 1.7 days, p < 0.05). Nineteen patients had an accepted intervention message and subsequently never received TW therapy, implying that TW therapy was prevented by the timely alert. Patients who experienced AKI during TW therapy received this combination for a mean of 3.6 days (1–11 days) until at least one drug was discontinued, and these patients had longer hospital stays compared to other patients. The mean time from first day of TW treatment until onset of AKI during TW therapy was 2.8 days (standard deviation: 1.7 days). In all groups, most patients were admitted to a surgical ward where urological procedures were the most common surgeries.

On the first day of TW therapy, the 290 patients received a total of 791 drugs from one of the three TW categories. The most commonly prescribed NSAID was ibuprofen (n = 198). Over half of all prescribed single-component diuretics were torasemide (n = 96), and the most commonly prescribed single-component ACEI was lisinopril (n = 64). The drugs used are shown in Online Resource 2 with their frequencies. Patients may receive TW therapy with only two prescriptions if, for example, they received an ACEI and a diuretic as a fixed-dose combination (=polypill); this was the case for 106 patients. Others (n = 2) received up to five TW drugs: an ACEI along with two different diuretics and two different NSAIDs.

3.2 Acute Kidney Injury

Overall, we identified 15 patients with AKI after TW administration. Causality assessments determined that TW administration was ‘likely’ causal for AKI in four patients, 'possible' in eight patients, and 'unlikely' in three patients. For the cases rated 'unlikely', there was an unplausible time relationship between TW therapy and AKI onset or another strong explanation for AKI. Consequently, they were not considered to be AKI cases for our analysis.

In Fig. 3, we show the progression of creatinine measurements over time for the four patients with a likely causal relationship; in patients 2 and 3, there was a marked increase in creatinine measurements shortly after the addition of an NSAID to the regimen (other relevant risk factors were absent). Patients 1 and 4 had less frequent creatinine measurements, and AKI diagnosis was not possible until day 5. Patient 4 had a positive de-challenge and a positive re-challenge that qualifies for a ‘certain’ causality assessment [33], but we downgraded this assessment because other risk factors were present. In eight patients, the causal relationship was classified as possible, which means that there was a close chronological relationship, but other risk factors were present. The potential creatinine falsifier trimethoprim sulfamethoxazole was prescribed at therapeutic doses to four of these patients. Five patients received other antibiotics or had an inflammatory condition that might contribute to the development of AKI. In two patients on the neurology ward, we suspected sarcopenia as a possible alternative risk factor. These two patients had only mild reductions in eGFR and were also discharged with a TW prescription, so we could not assess the de-challenge. Four patients received TW on the first day of hospitalisation, whereas the remaining patients were started on a NSAID later. In general, the medications of the AKI patients reflected the frequency of use of the respective drug group in our hospital. Interestingly, three patients had celecoxib as an NSAID, which is not part of the hospital's medication inventory. For five out of 12 patients, the NSAID was not given as a fixed regimen, but on demand.

Fig. 3
figure 3

Acute Kidney Injury during Triple Whammy therapy. Development of creatinine measurements in patients with likely causal relationship between acute kidney injury and Triple Whammy therapy. Each red dot represents a day on which all three components (medications) of the Triple Whammy were administered

Full size image

Of the 12 patients with likely or possible causality, 11 were detected by the TW algorithm. The patient who was not detected was younger than our threshold (age < 75 years) with an eGFR that remained above 60 ml/min/1.73 m2 even after creatinine levels increased. Clinical pharmacists paused the first alerts for four patients who subsequently experienced AKI. Intervention messages were written in nine cases, one patient was discharged prior to assessment by a clinical pharmacist, and one alert was self-limiting as the prescribing physicians discontinued the TW prescription before pharmacist intervention.

3.3 Performance of the Algorithm

3.3.1 Sensitivity and Specificity

A total of 21,326 patients were evaluated and classified according to the predefined decision pathway (Online Resource 1, see the electronic supplementary material). Of the patients who received an alert (n = 210), most were 'true positives' with at least one alert resulting in a message or self-resolution (n = 144). Sixty-six patients were classified as 'false positives' with unnecessary alerts that did not require intervention. A retrospective analysis identified 19 patients classified as 'false negatives' because they received a TW and had at least one risk factor when the TW was administered. Of these false-negative patients, 15 should have triggered Alert 2, three should have triggered Alert 3, and one should have triggered Alert 4. From these findings, we calculated a specificity of 99.7% and a sensitivity of 88.3%.

3.3.2 Acceptance Rate

The TW algorithm generated 343 alerts in 216 patients in 2021. No alerts were generated in 162 days of the year, and alerts did not exceed five per day. The majority of the 216 patients received only one TW alert. One patient received seven TW alerts in 2021. Alerts 1–5 were generated in varying numbers; Alert 2 was most frequent (n = 111, 32.4%) and Alert 5 was least frequent (n = 10, 2.9%).

Eighty-nine alerts were automatically closed by the CDSS, leaving 254 alerts for evaluation by clinical pharmacists (Fig. 4a). An intervention was forwarded to the physician in 43.3% (n = 110) of cases, mostly in written form and once via telephone. Notably, 48.8% (n = 124) of alerts were paused (1–3 days) for re-evaluation. Only 7.9% (n = 20) of the TW alerts were flagged as irrelevant. Alert 2 had the highest intervention rate at 43.2% (n = 48 of 111). Alert 3 was the most frequently paused alert at 46.2% (n = 36 of 78).

Fig. 4
figure 4

Processing of alerts. Quantity of alerts per alert number that were processed by the clinical pharmacists in a year (a) and the number of resulting interventions that were forwarded to the physicians (b)

Full size image

The outcome of the intervention could be evaluated for 94 interventions. Physicians implemented 73 interventions, and 21 interventions were not accepted, resulting in a compliance rate of 77.7%. Compliance rates differed for each alert (Fig. 4b); Alert 1 compliance rates were highest at 93.8% (n = 15 of 16), followed by Alert 2 (82.9%, n = 34 of 41), Alert 3 (70.0%, n = 14 of 20), and Alert 4 (53.3%, n = 8 of 15). Alert 5 relates to erroneous prescriptions of TW drugs not allowing a dose calculation by our system; this alert resulted in two interventions for missing dosing regimen information. Both interventions were resolved, resulting in a 100% compliance rate for Alert 5.

4 Discussion

In this study, we evaluated the performance of a CDSS to detect patients who receive TW therapy and are at risk for AKI. For this purpose, we identified all patients in our hospital receiving TW therapy over the course of 1 year. The hospital TW incidence rate (1.4%) is comparable with other findings; in a group of geriatric (ages 64–75 years), multi-morbid patients treated with polypharmacy in primary care, 2.5% of patients received a TW alert [30]. Given the predefined parameters of the algorithm, it is not surprising that the patients who received alerts were older on average and had lower baseline renal function than the TW patient group as a whole. Patients who developed AKI during TW had a higher eGFR at baseline. This is probably an artefact resulting from our definition of AKI as a 1.5-fold increase in creatinine from baseline and our definition of the baseline as the last measurement before each day of TW therapy; patients with elevated baseline creatinine levels prior to TW therapy are less likely to be identified as having AKI. Nonetheless, TW therapy is not ideal for these patients, and they should be identified by a drug safety alert. A promising but not yet validated approach to tackle this problem is the use of a dynamic-adjusted kidney function that puts weight on deteriorating kidney function parameters[34]. TW medication contained fixed-dose combination medications in 106 of the cases; physicians may not be aware that by prescribing two formulations, they are infact adding three potentially nephrotoxic agents to their patient's medication.

Twelve out of 290 patients (4.1%) developed AKI during TW therapy, which is within the reported range of 0.9–22.0% [2, 6, 9]. The onset of AKI was rapid; in two patients, AKI occurred within 1 day, which is more rapid than the median time of 8 days reported by Kunitsu et al. [10]. This difference could result from the use of different AKI definitions or more frequent creatinine measurements in our hospital, which allow us to diagnose AKI earlier. Most patients who experienced AKI were detected by the algorithm, but the adverse reaction could not be prevented in time; clinical pharmacists often paused these initial alerts and may have underestimated the risk of rapid onset AKI from TW. These pharmacists are aware of the workload of ward physicians and aim to provide only relevant alerts. We have since trained our clinical pharmacists to inform physicians at an earlier stage.

The performance of the TW algorithm was evaluated in terms of sensitivity, specificity, and acceptance rate. The overall sensitivity of 88.3% aligns with the recommendations that a CDSS should maximise specificity while sensitivity is kept above 75% [35]. In a scoping review, the clinical validation of several CDSSs for drug-related problems revealed a sensitivity of 28%–85% and a specificity of 42–75% [36], though these were not studies of CDSS for TW management. A general acceptance rate above 60% is suggested for action-oriented interventions [37]; our TW algorithm compliance rate was 77.7%, which compares favourably with this standard and with the acceptance rates reported for primary care TW CDSSs (30–82%) [26, 28, 29]. The acceptance rate differed between the alerts and was highest within patients with a kidney function below 30 ml/min/1.73m2. This is line with other findings that alert acceptance correlates with the assessment of alert relevance by care providers [38]. A further comparison to the aforementioned primary care TW CDSSs is difficult, firstly, because not all details of the algorithms were published and secondly, because the interpretation of the performance parameters relies heavily on the setting in which the algorithms are used.

With respect to the performance of the algorithm in our hospital, the false positive and false negative patients were especially of interest. The algorithm creates the alerts based on active drug prescriptions. To identify the TW patients, we used a dataset of documented drug administrations. It is therefore possible for patients to have received an alert based on active prescriptions, but ultimately never received TW therapy. This confirms our strategy to have non-interrupting alerts. While false positive alerts lead to over-alerting and alert fatigue, false negative alerts cause clinicians to miss patients that require attention. We discovered 19 patients that received TW therapy and had an additional risk factor, but received no TW alert. A common reason for failing to raise an alert was that the patient had a documented drug administration without a corresponding drug prescription. This is possible in our EHR and happens rarely, for example if immediate pain relief is needed and the nurse gets verbal permission to administer an NSAID. Since the algorithm relies on drug prescriptions, such patients cannot be detected. For other patients, we were not able to identify the reasons.

TW therapy is not contraindicated per se, and its harmful effects are dose-dependent, which is why we set daily dose thresholds for specific drugs based on our experience in order to avoid alert fatigue. As we included all drugs administrations of any dosage into our retrospective identification of TW patients, we were able to verify the practicality of these thresholds. Since 11 out of 12 AKI patients received an alert, we consider our threshold reasonable and do not consider a change in the thresholds at the moment. We do however consider lowering the age threshold from 75 years to 65 years, as the patient that was missed fell within that age range.

4.1 Limitations

There are several limitations to this analysis. Our study took place in 2021 during the coronavirus pandemic, which substantially altered daily routines and inpatient populations. However, the study was conducted with data that reflect the real situation in a Swiss tertiary hospital.

As we did not use data from patients who refused general consent, the prevalence of TW and AKI, as well as the sensitivity and specificity analysis of the algorithm, may differ slightly from a comprehensive patient population. Since the incidence of TW therapy is rather low, the specificity may be inflated and could be lower in a population with a higher incidence. Further, since we defined our baseline for the creatinine value at the start of TW therapy, AKI that occurred at a later stage during TW therapy might not have been detected, which may have led to underreporting. However, since the mean duration of hospitalisation of TW patients was 8 days, the likelihood is rather small. The diagnosis of AKI was not verified by a trained nephrologist but was determined strictly according to the KDIGO guidelines [30].

Regarding the sensitivity and specificity analysis, our case classifications and alert assessments were made by several clinical pharmacists and may vary slightly by pharmacist, time of day, or specific workload; we established and communicated a general gold standard to minimize this variability.

As there were no pre-implementation data, we cannot draw any conclusions about the effectiveness of this CDSS in preventing AKI during TW therapy. We identified 19 patients with an accepted TW intervention that pre-empted the administration of the drug combination, thus protecting these vulnerable patients against the risk of TW. We could not quantify the benefit of stopping one of the TW drugs earlier or monitoring creatinine levels more closely.

5 Conclusion

By limiting alerts to older patients with declining kidney function, the TW algorithm is highly sensitive and specific in detecting patients at risk for AKI. Clinical pharmacists were too hesitant to forward the alerts to the prescribing physicians, though the acceptance rates among physicians were high. The algorithm may help to prevent AKI in TW patients in the future.