To investigate the application value of “electronic alerts” (“e-alerts”) for acute kidney injury (AKI) among high-risk wards of hospitals.
A prospective, randomized, controlled study was conducted. We developed an e-alert system for AKI and ran the system in intensive care units and divisions focusing on cardiovascular disease. The e-alert system diagnosed AKI automatically based on serum creatinine levels. Patients were assigned randomly to an e-alert group (467 patients) or non-e-alert group (408 patients). Only the e-alert group could receive pop-up messages.
The sensitivity, specificity, Youden Index and accuracy of the AKI e-alert system were 99.8, 97.7, 97.5 and 98.1%, respectively. The prevalence of the diagnosis for AKI and expanded-AKI (AKI or multiple-organ failure) in the e-alert group was higher than that in the non-e-alert group (AKI 7.9 and 2.7%, P = 0.001; expanded-AKI 16.3 and 6.1%, P < 0.001). The prevalence of nephrology consultation in the e-alert group was higher than that in the non-e-alert group (9.0 and 3.7%, P = 0.001). There was no significant difference in the prevalence dialysis, rehabilitation of renal function or death in the two groups.
The e-alert system described here was a reliable tool to make an accurate diagnosis of AKI.
Approximately 21% of hospitalized patients have acute kidney injury (AKI) . A multicenter epidemiologic study showed that 1.4–2.9 million hospitalized patients developed AKI in China in 2013 . AKI is associated with adverse outcomes. AKI may develop into chronic kidney disease (CKD) and even end-stage renal disease. AKI is associated with mortality and increased health-care costs and is a financial burden to families and societies [3, 4]. Small changes to serum levels of creatinine (sCr) are not detected by clinicians, so early intervention is unlikely .
In 2015, The International Society of Nephrology (ISN) proposed the “0 by 25” initiative for AKI. “Electronic alerts” (“e-alerts”) are key interventions in patients with AKI . In 2016, the Acute Dialysis Quality Initiative (ADQI) stated the importance of an early diagnosis of AKI and suggested to build e-alerts to improve the diagnosis of AKI . Some studies have shown that e-alerts can affect the behavior of clinicians, reduce the use of nephrotoxic drugs and improve the prognosis of AKI . Wilson et al.  found that e-alerts for AKI did not improve clinical outcomes among hospitalized patients. The value of e-alerts for AKI is not clear. Here, we evaluated the clinical value of e-alerts for AKI patients.
This was a prospective, randomized, controlled study. This study is registered with clinicaltrials.gov (NCT02793167). Ethical approval of the study protocol was obtained from Guangdong General Hospital (GDREC2016164H; Guangdong, China).
The e-alert system was based on automatic measurement of sCr levels using computer software. The e-alert was based on Kidney Disease: Improving Global Outcomes (KDIGO) criteria. Patients were divided randomly into two groups: “e-alert” and “non-e-alert.” Only the e-alert group could receive pop-up windows on dashboard of the instrument.
During the study, trained researchers prospectively collected the clinical data of all patients: diagnosis upon admission and discharge from hospital; medical history; consultation records; dialysis records; final outcome in hospital.
Interpretation and related definitions of AKI
In the present study, three classifications of AKI were used. The first was “e-alert-confirmed,” whereby the diagnosis was based on the sCr value. The second was “researcher-confirmed AKI,” whereby the researcher confirmed AKI. This type of diagnosis was considered the “gold standard” because e-alert data and clinical data were satisfied simultaneously. The final classification type was “discharge-diagnosis AKI,” which was based on medical records, and was one of the main endpoints of our study.
In this system, each sCr value could be compared with the baseline sCr value. AKI was diagnosed on the e-alert according to 2012 KDIGO-AKI guidelines . The baseline sCr value was based on three rules: (1) a sCr value had been obtained in the last 2 days, and the latest value was higher by 26.5 µmol/L than the baseline sCr value; (2) a sCr value had not been obtained in the last 2 days, but a sCr value had been obtained in the last 7 days and, using the lowest value as the baseline, the latest sCr value was 50% higher than the baseline sCr value; (3) a sCr value had not been obtained in the last 7 days, but a sCr value had been obtained in the last 30, 90 or 365 days; using the lowest sCr value in the above time range as the baseline, the latest sCr value was 50% higher than the baseline sCr value. The AKI stage was classified as 1, 2 or 3 according to the maximum value of sCr (Table 1).
The clinical data of all cases were collected prospectively by trained researchers. Researchers confirmed AKI after ruling out the following conditions in the e-alert: (1) baseline sCr > 353.6 µmol/L; (2) a history of stage-5 chronic kidney disease (CKD) or maintenance hemodialysis; (3) kidney transplantation; (4) amputation; (5) no clinical evidence to support a diagnosis of AKI.
For non-e-alert patients, the researchers screened for AKI-related disease processes (e.g., oliguria) and confirmed them to be non-AKI. The sensitivity, specificity, Youden Index (sensitivity + specificity − 1) and accuracy of e-alerts were calculated using researcher-confirmed AKI as the gold standard.
Discharge-diagnosis AKI was judged according to the International Classification of Disease (tenth revision, clinical modification, ICD-10). Relevant codes were N17 and N10 x00 (acute renal tubular interstitial nephritis), N14.102 (contrast nephropathy), N18.80001 (acute exacerbation of CKD), N99.000 (kidney failure after operation) and N99.001 (kidney failure after surgery). Two ICDs for multiple-organ dysfunction syndrome (MODS) were included in discharge-diagnosis AKI: R65.101 (infectious multiple-organ dysfunction syndrome) and R65.301 (multiple-organ dysfunction syndrome). The diagnostic prevalence of AKI and expanded-AKI was obtained by calculating the ratio of discharge-diagnosis AKI and researcher-confirmed AKI.
The recovery of kidney function among discharged patients was defined as shown in Table 2.
Development of the e-alert system for AKI
The e-alert system was developed by the divisions of Nephrology and Information Management of Guangdong General Hospital (Patent Application 201610001950.5). The e-alert system consisted of a patient filter, sCr extractor, AKI automatic interpretation, random-number generation and distribution, and pop-up window generator. The system generated the random allocation sequence according to the time of acute kidney injury.
The working principle of the e-alert system was ingenious. First, the system screened adult patients aged ≥ 18 years who could be alerted. Upon hospitalization, the system would compare the sCr values (including the results of previous hospitalizations or outpatient visits). Then, the system made a diagnosis of AKI according to KDIGO criteria. Based on the generation of random numbers, the system divided patients into an e-alert group and non-e-alert group. Only the e-alert group would receive pop-up windows (Fig. 1). The participants were blinded after assignment to interventions.
Operation of the e-alert for AKI
The e-alert system was run in intensive care units (ICUs), cardiology divisions (eight divisions), coronary care units (CCUs) and cardiac surgery divisions (four wards) in Guangdong General Hospital from July 1 to November 31, 2016.
Analyses were done using SPSS v20.0 (IBM, Armonk, NY, USA). Continuous data are presented as the mean (standard deviation) or median (interquartile range) as appropriate, and categorical variables as n (%). We compared groups using the Student’s t test for continuous variables and χ2 test for categorical variables. For ordered classification variables and non-normally distributed measurement data, we used the rank sum test. All tests were two-tailed, and P < 0.05 was considered significant.
During the operation of the e-alert system, 5308 patients were screened in our hospital. There were 975 (18.3%) e-alerts according to 2012 KDIGO criteria. At random, 513 patients received an e-alert, and the remainder (462 patients) did not. According to the exclusion criteria, 467 patients (in the e-alert group) and 408 patients (in the non-e-alert group) were researcher-confirmed AKI. From 4333 non-e-alert patients, clinical evidence (e.g., oliguria) was used to diagnose AKI in two patients (Fig. 2).
There was no significant difference between the e-alert group and non-e-alert group with regard to sex, age, age group, baseline sCr value or Charlson score (Table 3).
Authenticity of e-alerts
Using researcher-confirmed AKI as the gold standard, we evaluated the authenticity of the e-alert system for AKI. The total number of e-alerts was 976, of which 875 were researcher-confirmed AKI. The total number of non-e-alerts was 4333, of which 2 were researcher-confirmed AKI. Evaluation of e-alerts revealed the sensitivity to be 99.8%, specificity to be 97.7%, Youden Index to be 97.5% and accuracy to be 98.1%.
Value of e-alerts for improving the prevalence of AKI diagnosis
The prevalence of the diagnosis of AKI in the e-alert group was higher than that for the non-e-alert group, and the difference was significant (7.9 and 2.7%, P = 0.001). The prevalence of the diagnosis of expanded-AKI was also higher in the e-alert group than that in the non-e-alert group, and the difference was significant (16.3 and 6.1%, P < 0.001). In patients with stage-1 AKI, the prevalence of the diagnosis of AKI and expanded-AKI was higher in the e-alert group than in the non-alert group (AKI: 3.5 and 0.7%, P = 0.048; expanded-AKI: 7.7 and 1.8%, P = 0.003). However, for stage-2 and stage-3 AKI, a significant difference was not observed in the two groups (AKI: P = 1.000, 0.181; expanded-AKI: P = 0.555, 0.306). In a ward-stratified study, e-alerts improved the prevalence of the diagnosis of AKI and expanded-AKI in cardiology divisions (AKI: 5.0 and 0.0%, P = 0.011; expanded-AKI: 8.3 and 0.0%, P < 0.001) and cardiac surgery divisions (AKI: 9.0 and 2.7%, P = 0.019; expanded-AKI: 17.5 and 6.8%, P = 0.004). There was no significant difference in the ICU or CCU (AKI: P = 0.298, 0.714; expanded-AKI: P = 0.091, 0.349) (Table 4).
Relationship between e-alerts and outcome
The prevalence of nephrology consultation in the e-alert group was higher than that in the non-e-alert group (9.0 and 3.7%, P = 0.001). There was no significant difference in the prevalence of dialysis, rehabilitation of renal function or death in the two groups (P = 0.885, 0.382 and 0.160) (Table 5).
AKI is a common and serious clinical syndrome, especially in the ICU. Due to the insufficiency of cardiac function, the use of contrast agents and extracorporeal circulation in cardiac surgery,  cardiology divisions become the “frontline” in the battle against AKI.
The ISN and ADQI suggested that e-alerts should be constructed to resolve the early diagnosis of AKI. However, Wilson et al.  found that e-alerts did not improve the clinical endpoints of patients with AKI. One of the limitations they considered was that the role of e-alerts in different wards might be different. Therefore, we applied e-alerts to the high-risk wards of AKI and evaluated its value.
An accurate and timely diagnosis is prerequisites for effective interventions. In ICUs, if AKI or other types of organ dysfunction occur, clinicians are more likely to diagnose MODS. Hence, we included MODS in the diagnosis of expanded-AKI. The prevalence of the diagnosis for AKI and expanded-AKI was higher in the e-alert group than that of the non-alert group. In 2013, AKI was documented in 1.4–2.9 million hospitalized patients . The e-alert used in the present study increased the prevalence of the diagnosis of AKI from 2.7 to 7.9%. This means that it can reduce the missed diagnosis of 72,800–150,800 cases nationwide per year in China.
Research has shown that even the clinical manifestations of low-level AKI or “transient” AKI are associated with dialysis, cardiovascular disease and death . In the present study, e-alerts were more effective in improving the prevalence of the diagnosis of patients with stage-1 AKI. AKI can occur in different hospital divisions. The AKI-related knowledge of physicians in different specialties is not optimal, especially in non-ICU and surgery divisions. The preliminary results of our study showed that e-alerts can improve the prevalence of the diagnosis in cardiology and cardiac surgery divisions.
In the present study, use of e-alerts reduced the prevalence of the missed diagnosis of AKI, but the overall prevalence was very low. There was no improvement in the prevalence of dialysis or renal-function recovery in survivors. There could have been three main reasons for this observation. First, the high prevalence of AKI in high-risk wards was not matched with monitoring of renal function, especially in non-ICU and surgery wards. Second, some clinicians did not understand the meaning of e-alerts. This information system is convenient for medical management, but, with increasing use of e-alerts for different diseases, “e-alert fatigue” may occur . Third, nephrologists did not intervene actively. Early intervention and follow-up by nephrologists can improve the prognosis of AKI . An alert system alone is not adequate to improve the effectiveness of AKI management. Therefore, it is necessary to strengthen the education of clinicians. Through meetings and lectures, we can explain the purpose, importance and methods of the present study.
There were two study limitations. First, we lacked follow-up data such as long-term rehospitalization and other adverse outcomes. Second, because preadmission and in hospital medical records are not combined in most Chinese hospitals, it is difficult to distinguish community-acquired AKI with e-alert.
The present study evaluated, in a preliminary fashion, the value of application of e-alerts in AKI high-risk wards. e-alerts reduced the prevalence of a missed diagnosis of AKI to some extent, but there was no improvement in the main endpoints. Our research team intends to strengthen the training of physicians in relevant divisions of our hospital. In the future, we will reevaluate the value of e-alerts based not only on the inhospital outcomes, but also on long-term prognosis.
Susantitaphong P, Cruz DN, Cerda J et al (2013) World incidence of AKI: a meta-analysis. Clin J Am Soc Nephrol 8(9):1482–1493
Yang L, Xing G, Wang L et al (2015) Acute kidney injury in China: a cross-sectional survey. Lancet 386(10002):1465–1471
Coca SG, Singanamala S, Parikh CR (2012) Chronic kidney disease after acute kidney injury: a systematic review and meta-analysis. Kidney Int 81(5):442–448
Kerr M, Bedford M, Matthews B et al (2014) The economic impact of acute kidney injury in England. Nephrol Dial Transplant 29(7):1362–1368
Aitken E, Carruthers C, Gall L et al (2013) Acute kidney injury: outcomes and quality of care. QJM 106(4):323–332
Mehta RL, Cerda J, Burdmann EA et al (2015) International Society of Nephrology’s 0 by 25 initiative for acute kidney injury (zero preventable deaths by 2025): a human rights case for nephrology. Lancet 385(9987):2616–2643
James MT, Hobson CE, Darmon M et al (2016) Applications for detection of acute kidney injury using electronic medical records and clinical information systems: workgroup statements from the 15th ADQI consensus conference. Can J Kidney Health Dis 3:100
Mccoy AB, Waitman LR, Gadd CS et al (2010) A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am J Kidney Dis 56(5):832–841
Wilson FP, Shashaty M, Testani J et al (2015) Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomised controlled trial. Lancet 385(9981):1966–1974
Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group (2012) KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl 2(1):1–13
Uchino S, Bellomo R, Bagshaw SM et al (2010) Transient azotaemia is associated with a high risk of death in hospitalized patients. Nephrol Dial Transplant 25(6):1833–1839
van der Sijs H, Aarts J, Vulto A et al (2006) Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 13(2):138–147
Harel Z, Wald R, Bargman JM et al (2013) Nephrologist follow-up improves all-cause mortality of severe acute kidney injury survivors. Kidney Int 83(5):901–908
This work was funded by Science and Technology Program of Guangzhou, China (201604020037).
Conflict of interest
The authors declare that they have no competing interests.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Wu, Y., Chen, Y., Li, S. et al. Value of electronic alerts for acute kidney injury in high-risk wards: a pilot randomized controlled trial. Int Urol Nephrol 50, 1483–1488 (2018). https://doi.org/10.1007/s11255-018-1836-7
- Acute kidney injury
- Information system