Abstract
Background
Inappropriate prescribing is associated with negative patient outcomes. In hospitalized patients, the use of Clinical Decision Support Systems (CDSSs) may reduce inappropriate prescribing and thereby improve patient-related outcomes. However, recently published large clinical trials (OPERAM and SENATOR) have shown negative results on the use of CDSSs and patient outcomes and strikingly low acceptance of recommendations.
Objective
The purpose of the present study was to investigate the use of a CDSS in a real-life clinical setting of hospitalized older patients. As such, we report on the real-life pattern of this in-hospital implemented CDSS, including (i) whether generated alerts were resolved; (ii) whether a recorded action by the pharmacist led to an improved number of resolved alerts; and (iii) the natural course of generated alerts, in particular of those in the non-intervention group; as these data are largely lacking in current studies.
Methods
Hospitalized patients, aged 60 years and older, admitted to Zuyderland Medical Centre, the Netherlands, in 2018 were included. The evaluation of the CDSS was investigated using a database used for standard care. Alongside demographic and clinical data, we also collected the total numbers of CDSS alerts, the number of alerts ‘handled’ by the pharmacist, those that resulted in an action by the pharmacist, and finally the outcome of the alerts at day 1 and day 3 after the alert was generated.
Results
A total of 3574 unique hospitalized patients, mean age 76.7 (SD 8.3) years and 53% female, were included. From these patients, 8073 alerts were generated, of which 7907 (97.9% of total) were handled by the pharmacist (day 1). In 51.6% of the alerts handled by the pharmacist, an action was initiated, resulting in 36.1% of the alerts resolved after day 1, compared with 27.3% if the pharmacist did not perform an action (p < 0.001). On day 3, in 52.6% of the alerts an action by the pharmacist was initiated, resulting in 62.4% resolved alerts, compared with 48.0% when no action was performed (p < 0.001). In the category renal function, the percentages differed significantly between an action versus no action of the pharmacist at day 1 and at day 3 (16.6% vs 10.6%, p < 0.001 [day 1]; 29.8% vs 19.4%, p < 0.001 [day 3]).
Conclusion
This study demonstrates the pattern and natural course of clinical alerts of an in-hospital implemented CDSS in a real-life clinical setting of hospitalized older patients. Besides the already known beneficial effect of actions by pharmacists, we have also shown that many alerts become resolved without any specific intervention. As such, our study provides an important insight into the spontaneous course of resolved alerts, since these data are currently lacking in the literature.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
It is important to investigate the real-life pattern and natural course of clinical alerts of Clinical Decision Support Systems (CDSSs). |
Our data may contribute to the current literature, as they allow researchers to more carefully interpret negative study results of clinical trials investigating CDSSs. |
Further research is needed to completely understand the effectiveness of CDSS interventions and our proposed ‘natural course of clinical alerts’ may be an important parameter in assessing the quality of clinical rules and thereby a potential target to optimize CDSSs. |
1 Introduction
The population is ageing rapidly and as a result multimorbidity and associated polypharmacy is an increasing health risk leading to considerable mortality and morbidity [1,2,3]. As such, 30% of emergency department visits and hospital admissions in older age are attributable to side effects and inappropriate prescription of medicine, since the risk of inappropriate medication use, adverse drug events (ADE) and medication-related problems (MRPs) is increasing [4,5,6]. These risks increase even further when older patients are hospitalized, at least in part due to the fact that acutely ill older patients are often exposed to new prescriptions by multiple prescribers [7, 8]. Thus, a variety of interventions to reduce MRPs in older hospitalized individuals has been studied with widely varying impact [3, 9]. Most of these interventions studied the participation of pharmacists to ideally prevent, and otherwise reduce the impact of MRPs, although in recent years the involvement of a clinical decision support system (CDSS) has emerged, demonstrating the possibility of reducing potentially inappropriate prescribing in hospitalized older patients [7, 9,10,11].
Recently, two large clinical trials, the OPERAM (OPtimising thERapy to prevent Avoidable hospital admissions in Multimorbid older people) trial and the SENATOR (Software ENgine for the Assessment and optimization of drug and non-drug Therapy in Older peRsons) trial investigated medication optimization supported by a CDSS including the Screening Tool of Older Persons’ Prescriptions (STOPP) and Screening Tool to Alert to Right Treatment (START) recommendations [3, 12]. However, both studies found no significant differences in incidence of adverse drug reactions, drug-related hospital admissions and mortality between the intervention and control arms, and only a low proportion of the recommendations accepted by the pharmacotherapy team or attending physicians [12, 13]. Potential explanations for these low acceptance rates were the fact that recommendations were deemed of low clinical relevance by clinicians as well as variable attitudes to the intervention and/or participation in clinical trials and patient-specific factors [14, 15].
The ultimate goal of a CDSS is to improve patient-related outcomes, however, as long as only low proportions of recommendations are accepted and rules with low clinical relevance are used in CDSSs, it is unlikely this goal will be achieved [13, 16, 17]. These specific factors, presumably leading to low response rates and alert fatigue, have not been recorded in the initial trials evaluating CDSSs. Therefore, the purpose of the present study was to report on existing knowledge gaps in a real-life clinical setting of hospitalized older patients, in order to describe, evaluate and optimize the use of the CDSS in clinical practice. To this end, we investigated the real-life pattern of an in-hospital implemented CDSS, including (i) whether generated alerts were resolved; (ii) whether a recorded action by the pharmacist led to an improved number of resolved alerts; and (iii) description of the natural course of generated alerts, in particular of those in the non-intervention group, as these data are largely lacking in current studies.
2 Methods
2.1 System Details, Description of CDSS System, Clinical Rule Reporter (CRR)
The CDSS used in Zuyderland Medical Centre, a large teaching hospital in the Netherlands, is the Clinical Rule Reporter (CRR). The CRR has been implemented in daily practice since 2016 and is mainly based on guidance for medication-related laboratory testing, dosing support in patients with renal impairment and guidance for optimal use for antibiotics and anticoagulation [18, 19]. It is used for medication surveillance using demographic, medication and laboratory data from admitted patients and contains several clinical rules [19]. The CDSS in this study consisted of 80 rules (supplementary data table 1, see electronic supplementary material [ESM]).
The CDSS analyses the medication from all admitted patients on a daily basis. The clinical pharmacist receives (per patient) a report in which all alerts are given. The clinical pharmacist then assesses whether further action is indicated according to a distinct rule. For this study, we defined this action as ‘an action by the pharmacist’ when a rule was discussed extensively (i.e. consultation between pharmacist and physician) or when an intervention was performed by the pharmacist or by the physician (after consultation).
2.2 Patients
We included patients aged 60 years and older, hospitalized in 2018, from January 01, 2018 to December 31, 2018, in Zuyderland Medical Centre, the Netherlands. Patients admitted to rehabilitation wards and short stay departments were excluded. Demographic data (age and sex) were collected. The evaluation of the CRR was investigated in a retrospective study using a database used for standard care, which is why this study did not require ethical approval.
2.3 Data Collection
The data of the generated alerts for the included patients and their management were extracted to Qlik Sense version September 2020 SR1. Qlik Sense is a tool which visualizes data in an interactive way. All rules were categorized as one of the following: renal function, potassium, antibiotics (intravenous [IV] to oral), antibiotics (long use), opioids/laxatives, anticoagulant therapy and unknown lab value. Alongside the number of unique patients, the following data was collected in order to evaluate the use of the CRR: (i) the total number of alerts, (ii) the number of alerts ‘handled’ by the pharmacist, (iii) the number of alerts resulting in an action of the pharmacist and (iv) the outcome of the alert—described as ‘green’ (resolved), ‘red’ (unresolved) and ‘unknown’. We also calculated the percentage of resolved and unresolved alerts, after excluding the ‘unknown’ alerts. The latter were excluded because it is unknown whether an ‘unknown’ alert was actually resolved by discontinuing the medication (and thus the rule that generated the alert did not apply anymore) or because the patient was discharged.
2.4 Statistical Analyses
Descriptive statistics were used to describe the study population and were presented as mean (SD) or median (IQR) as appropriate. To test the differences in percentages in the groups with or without an action by the pharmacist an unpaired student’s t-test was used. Statistical analyses were performed using SPSS Statistics v25 (IBM, Armonk, NY, USA).
3 Results
3.1 Baseline Characteristics
In 2018, anonymized data from 3574 unique hospitalized patients were included for the current analyses. The mean age was 76.7 (SD 8.3) years and 53% were female. From these patients, in total 8073 alerts were generated, of which 7907 (97.9% of total) were handled by the pharmacist (day 1). The patient characteristics and subdivision of the clinical alerts per category and whether the pharmacist performed or did not perform an action are described in Table 1. The percentages of actions performed varied between the different rule categories from 33.8% (opioids/laxatives) to 94% (anticoagulant therapy), respectively (Table 1).
In 4083 (51.6%) of the handled alerts, an action was performed by the pharmacist, resulting in 1297 (36.1% after excluding ‘unknown’) resolved alerts after day 1, while when the pharmacist did not perform an action, which was the case in 3824 alerts, 915 (27.3% after excluding ‘unknown’) alerts were resolved (36.1% vs 27.3%, p < 0.001). These results are schematically summarized in Fig. 1.
For the day 3 evaluation, 5750 handled alerts were analysed. In 3025 (52.6%) alerts, an action by the pharmacist was performed, resulting in 1242 (62.4% after excluding ‘unknown’) resolved alerts after day 3, while when no action was performed, which was the case in 2725 alerts, 772 (48.0% after excluding ‘unknown’) alerts were resolved (62.4% vs 48.0%, p < 0.001). These results are schematically summarized in Fig. 2.
3.2 Rule Categories
When we analysed the different rule categories separately, we found that only in the category ‘renal function’ did the percentages differ significantly between an action versus no action of the pharmacist (16.6% vs 10.6%, p < 0.001). In other rule categories, we observed small positive effects when an action was performed by the pharmacist, that is, in the categories potassium, antibiotics (both IV to oral, and long use) and anticoagulant therapy; however, these differences did not differ statistically significant (Fig. 3).
On day 3, similar results were observed, with a significant difference between those with and without an action of the pharmacist only found in the category ‘renal function’ (29.8% vs 19.4%, p < 0.001) (Fig. 4).
4 Discussion
This study demonstrates the real-life pattern and natural course of clinical alerts of an in-hospital implemented CDSS. As such, we have shown that when the pharmacist did not perform an action, 27% and 48% of the generated alerts are resolved after 1 and 3 days, respectively. These percentages improved to 36% and 62% on day 1 and 3 when an action by the pharmacist was performed. Also, we found that the percentages of resolved alerts varied widely between different rule categories.
The finding that actions taken by pharmacists in response to the generated alerts were beneficial and led to an improved total number of resolved alerts was not surprising, since this has been studied extensively in the past [20, 21]. However, this study reveals that many alerts are resolved without a specific intervention by the pharmacist. Our study therefore may provide important insight into the null effect of CDSS in the OPERAM and SENATOR trials, since alerts that are resolved without intervention are also addressed per usual care and therefore are unlikely to improve clinical outcomes [3, 12]. We speculate that these ‘false positive’ alerts may distract from potentially more clinically relevant alerts and therefore lead to a reduced effect of CDSS interventions. We believe that the CDSS may be particularly useful for rare MRPs with, for instance, uncommon high-risk medications, and that therefore the initial clinical trials may have been underpowered to detect the impact of a CDSS on these events.
Our observational study shows some reassuring findings, given the total number of resolved alerts on one hand, but also the total number of resolved alerts in specific categories on the other hand, which are generally considered as more clinically relevant. Most notable are the number of resolved ‘anticoagulant therapy’ and ‘potassium’ alerts, which may indicate that clinicians themselves are perfectly able to determine the clinical relevance and also prioritization of these alerts. Moreover, this is substantiated by the low number of resolved ‘opioid + laxative’ alerts, an otherwise clinically relevant recommendation, which is likely to have been deemed of lower priority in the acute setting.
The discrepancy in acceptance of recommendations between our real-life clinical data and data from recent trials may be explained by several factors. First, in the two largest clinical trials to date, START/STOPP recommendations were used and therefore alert fatigue may have occurred due to lower prioritization of otherwise clinically relevant recommendations [3, 12, 14, 15]. This observation contrasts with our set of clinical rules that are largely directly clinically relevant and also of greater priority in the hospital setting, although our figures also show that clinicians seem to be perfectly able to prioritize this themselves. Second, variable attitudes towards the intervention itself (both OPERAM and SENATOR used software-generated recommendations) or participation in trials by investigators or participants might have played a role, while these factors play no significant role when implemented in clinical practice. Third, in the SENATOR trial, specific exclusion criteria, such as admission to a geriatric ward, were applied that not only limited the generalizability of the findings, but also excluded a large proportion of the population concerned, namely the geriatric population [12, 22]. Nevertheless, since OPERAM had only minimal exclusion criteria, it is unlikely this explains this discrepancy completely [3, 23].
We have shown that different rule categories show varying percentages of resolved alerts. When investigating the impact of computerized interventions, acceptance rates (or agreement with recommendations) vary widely between different studies and interventions [7]. Recommendations may not be accepted by clinicians as the recommendation might be deemed of low clinical relevance, patients may insist on medication continuation, or possible interactions may be monitored instead of direct medication discontinuation [7, 24,25,26].
Our study has several strengths and limitations. Strengths include the large sample size of 3754 unique patients and the real-life clinical setting of an already implemented CDSS. Our study also has some limitations. First, our definition of ‘action taken by the pharmacist’ is broad, including when a rule was discussed extensively (i.e. consultation between pharmacist and physician) or when an intervention was performed by the pharmacist or the physician (after consultation). This might increase the effect of the pharmacist because some of the ‘pharmacist interventions’ were performed by the physician. Second, our study is limited by its retrospective and observational design. Third, in 2018 our CDSS only consisted of 80 clinical rules. As such, clinically relevant, but also well -known rules (such as START/STOPP) have not been included in this version of the CDSS, making this study difficult to interpret in the current field of studies investigating generic CDSSs. Fourth, although this is a relatively large study, we only had access to a limited set of clinical data and were therefore unable to investigate the impact of clinical predictors on the percentage of alerts resolved. Despite this, we have included a cohort of patients in which MRPs are of particular interest, namely the geriatric population, often excluded in clinical trials.
5 Conclusions
This study demonstrates the pattern and natural course of clinical alerts of an in-hospital implemented CDSS in a real-life clinical setting of hospitalized older patients. Next to the beneficial effect of actions by pharmacists, we have also shown that many alerts are resolved without specific interventions. As such, our study provides insight into the spontaneous course of resolved alerts, since these data are currently lacking in the literature. Further research is needed to completely understand the effectiveness of CDSS interventions and our proposed ‘natural course of clinical alerts’ may be an important parameter in assessing the quality of clinical rules and thereby a potential target to optimize CDSS.
References
Fortin M, Hudon C, Hafferty J, et al. Prevalence estimates of multimorbidity: a comparative study of two sources. BMC Health Serv Res. 2010;10:1.
Aubert CE, Streit S, Da Costa BR, et al. Polypharmacy and specific comorobidities in university primary care settings. Eur J Intern Med. 2016;35:35–42.
Blum MR, Sallevelt BTGM, Spinewine A, et al. Optimizing therapy to prevent avoidable hospital admissions in multimorbid older adults (OPERAM): cluster randomised controlled trial. BMJ. 2021;13(374): n1585.
Thorell K, Midlöv P, Fastbom J, et al. Importance of potentially inappropriate medications, number of chronic conditions and medications for the risk of hospitalisation in elderly in Sweden: a case-control study. BMJ Open. 2019;9(9): e029477.
Luttikhuis HM, Blomaard LC, van der Kaaij MAE, et al. Geriatric characteristics and the risk of drug-related hospital admissions in older Emergency Department patients. Eur Geriatr Med. 2022;13(2):329–37.
Haag JD, Bellamkonda VR, Perinpam L, Peters BJ, Sunga KL, Gross CL, Dierkhising RA, Baudoin MR, Rudis MI. Prevalence and Categorization of Drug-Related Problems in the Emergency Department. J Emerg Med. 2022;63(2):192–9.
Dalton K, O’Brien G, O’Mahony D, et al. Computerised interventions designed to reduce potentially inappropriate prescribing in hospitalised older adults: a systematic review and meta-analysis. Age Ageing. 2018;47(5):670–8.
Page RL 2nd, Linnebur SA, Bryant LL, et al. Inappropriate prescribing in the hospitalized elderly patient: defining the problem, evaluation tools, and possible solutions. Clin Interv Aging. 2010;7(5):75–87.
Rankin A, Cadogan CA, Patterson SM, et al. Interventions to improve the appropriate use of polypharmacy for older people. Cochrane Database Syst Rev. 2018;9(9):CD008165.
Mulder-Wildemors LGM, Heringa M, Floor-Schreudering A, et al. Reducing inappropriate drug use in older patients by use of clinical decision support in community pharmacy: a mixed-methods evaluation. Drugs Aging. 2020;37(2):115–23.
O’Sullivan D, O’Mahony D, O’Connor MN, et al. The impact of a structured pharmacist intervention on the appropriateness of prescribing in older hospitalized patients. Drugs Aging. 2014;31(6):471–81.
O’Mahony D, Gudmundsson A, Soiza RL, et al. Prevention of adverse drug reactions in hospitalized older patients with multi-morbidity and polypharmacy: the SENATOR* randomized controlled clinical trial. Age Ageing. 2020;49(4):605–14.
Sallevelt BTGM, Huibers CJA, Heij JMJO, et al. Frequency and acceptance of clinical decision support system-generated STOPP/START signals for hospitalised older patients with polypharmacy and multimorbidity. Drugs Aging. 2022;39(1):59–73.
Dalton K, O’Mahony D, Cullinan S, et al. Factors affecting prescriber implementation of computer-generated medication recommendations in the SENATOR trial: a qualitative study. Drugs Aging. 2020;37(9):703–13.
Dalton K, Curtin D, O’Mahony D, et al. Computer-generated STOPP/START recommendations for hospitalised older adults: evaluation of the relationship between clinical relevance and rate of implementation in the SENATOR trial. Age Ageing. 2020;49(4):615–21.
O’Sullivan D, O’Mahony D, O’Connor MN, et al. Prevention of adverse drug reactions in hospitalised older patients using a software-supported structured pharmacist intervention: a cluster randomised controlled trial. Drugs Aging. 2016;33(1):63–73.
Olakotan OO, Mohd YM. The appropriateness of clinical decision support systems alerts in supporting clinical workflows: a systematic review. Health Inform J. 2021;27(2):14604582211007536.
de Wit HA, Hurkens KP, Mestres Gonzalvo C, et al. The support of medication reviews in hospitalised patients using a clinical decision support system. Springerplus. 2016;5(1):871.
de Wit HA, Mestres Gonzalvo C, Cardenas J, et al. Evaluation of clinical rules in a standalone pharmacy based clinical decision support system for hospitalized and nursing home patients. Int J Med Inform. 2015;84(6):396–405.
Arvisais K, Bergeron-Wolff S, Bouffard C, et al. A pharmacist-physician intervention model using a computerized alert system to reduce high-risk medication use in elderly inpatients. Drugs Aging. 2015;32(8):663–70.
Bankes D, Pizzolato K, Finnel S, et al. Medication-related problems identified by pharmacists in an enhanced medication therapy management model. Am J Manag Care. 2021;27(16 Suppl):S292–9.
Lavan AH, O’Mahony D, Gallagher P, et al. The effect of SENATOR (Software ENgine for the Assessment and optimisation of drug and non-drug Therapy in Older peRsons) on incident adverse drug reactions (ADRs) in an older hospital cohort—trial protocol. BMC Geriatr. 2019;19(1):40.
Adam L, Moutzouri E, Baumgartner C, et al. Rationale and design of OPtimising thERapy to prevent avoidable hospital admissions in Multimorbid older people (OPERAM): a cluster randomised controlled trial. BMJ Open. 2019;9(6): e026769.
Griffey RT, Lo HG, Burdick E, et al. Guided medication dosing for elderly emergency patients using real-time, computerized decision support. J Am Med Inform Assoc. 2012;19(1):86–93.
Terrell KM, Perkins AJ, Dexter PR, et al. Computerized decision support to reduce potentially inappropriate prescribing to older emergency department patients: a randomized, controlled trial. J Am Geriatr Soc. 2009;57(8):1388–94.
Mattison ML, Afonso KA, Ngo LH, et al. Preventing potentially inappropriate medication use in hospitalized older patients with a computerized provider order entry warning system. Arch Intern Med. 2010;170(15):1331–6.
Acknowledgements
The authors would like to thank Bram de Kort from ICON Healthcare and Folkert Botma from Panacea informatics for their assistance to extract the data to Qlik Sense.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
No funding.
Conflicts of interest
All authors declare that they have no conflict of interest.
Availability of data and material
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval
Not applicable. The study did not require ethical approval since it involved the retrospective analysis of a database used for standard care.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Code availability
Not applicable.
Author contributions
AL,BS and HK designed the study, analyses and interpreted the data; BL and HK obtained the data, AL and BS wrote the first draft; critical revision of the manuscript for important intellectual content was performed by all authors (AL, DK, AZ, VM, KH, NN, BL, HK, BS). DK and AZ contributed equally to this manuscript. All authors read and approved the final manuscript.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Linkens, A.E.M.J.H., Kurstjens, D., Zwietering, N.A. et al. Clinical Decision Support Systems in Hospitalized Older Patients: An Exploratory Analysis in a Real-Life Clinical Setting. Drugs - Real World Outcomes 10, 363–370 (2023). https://doi.org/10.1007/s40801-023-00365-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40801-023-00365-3