Introduction

Handheld blood glucose meters or department-based blood gas analyzers are currently the preferred methods to measure blood glucose levels in intensive care unit (ICU) patients [1,2]. These intermittent glucose-monitoring techniques have variable accuracies [3] but foremost lack useful trending because of the interval between consecutive measurements. Continuous glucose monitoring (CGM) is suggested to increase practicalities and safety of insulin titration in ICU patients [1,4], in particular when targeting normal or near-normal blood glucose levels when hypoglycemic episodes can be expected [5-13].

Glucose oxidase technique-based interstitial CGM devices have been used before in diabetic patients outside the ICU setting [14]. It is uncertain, however, whether interstitial CGM devices are point accurate in critically ill patients [1]. An altered relationship between blood and interstitial fluid glucose levels during critical illness could affect the point accuracy of interstitial CGM to reflect the blood glucose level [15]. Several interstitial CGM sensor systems originally designed for non-ICU patients have been tested in the ICU setting in recent years [16-28]. Medtronic MiniMed (Medtronic Inc., Northridge, CA, USA) developed the Sentrino Continuous Glucose Management System, an interstitial CGM device that was especially designed for use in critically ill patients. This device was improved from previous models by creating the processor cable and pole-mounted monitor and by four sensing elements designed to increase responsiveness to glucose changes and to limit the influence from drug interactions.

The aim of this study was to test its point accuracy and reliability in a mixed medical-surgical ICU. We hypothesized that the device would provide an accurate reflection of the blood glucose level in ICU patients treated according to a local guideline for blood glucose control targeting blood glucose levels between 90 and 144 mg/dL. In addition, we determined its reliability, including duration of the device start-up, the need for calibration, skips in data acquisition, and number of and reasons for premature disconnections.

Methods

Study design and informed consent

This was an investigator-initiated observational trial. The Institutional Review Board of the Academic Medical Center (Amsterdam, The Netherlands) approved the study protocol (study ID: NL41498.018.12). Medtronic MiniMed provided three devices for the duration of the trial and the necessary sensors but had no influence on study design or study reporting. Patients or next of kin had to provide written informed consent before the start of any study-related procedure.

Study population

Patients were recruited between October 2012 and February 2014 in a 30-bed mixed medical-surgical ICU of a large university hospital (Academic Medical Center). Patients were eligible for inclusion if they were at least 18 years old and had an anticipated life expectancy of more than 96 hours. Patients were excluded from participation if they had a platelet count of less than 30 × 109/L, had participated in a trial testing an investigational product or treatment within the past 30 days, were pregnant, or had a suspected or diagnosed medical condition which in the opinion of the investigators prevented the patient from completing the study.

Glucose control

ICU nurses performed glucose control with insulin by following a local guideline for blood glucose control targeting a blood glucose level between 90 and 144 mg/dL [29]. Insulin titration adjustments were based on sliding scales. The local guideline for blood glucose control dictated nurses to perform blood glucose measurements at least every 4 hours and more frequently if blood glucose levels were out of range or were expected to change rapidly. For details, see Additional file 1.

During the study, ICU nurses were not allowed to change insulin infusion rate based on the readings by the investigational device. However, they were allowed to perform additional blood glucose measurements if the device suggested rapid changes in the glucose level or when there was a trend toward hypoglycemia.

The investigational device

The disposable glucose sensors of the device were glucose oxidase-based; each sensor had two probes, and each probe had two sensing elements. The individual measurement results were combined and displayed on the device monitor every minute. The signal was transmitted through the processor cable to the monitor. It was a single-patient single-use sensor, which could be used for up to 72 hours. The processor cable was reusable.

The sensor was inserted into the subcutaneous tissue by using two parallel introducer needles. The two needles automatically retracted when the introducer hub was pulled away from the sensor base; the sensor probes remained in the subcutis. Each new sensor needed calibration by using blood glucose levels after insertion and initialization and after 1 hour and 2 hours; thereafter, repeated calibrations were performed every 8 hours.

Study procedures

Sensors were inserted into the subcutis of the thigh. Successive sensors could be used for 72 hours, depending on length of stay in the ICU, but never for longer than 30 days. Arterial blood glucose levels were measured by using RapidLab 1265 blood gas analyzers (Siemens Healthcare Diagnostics, The Hague, The Netherlands), which were used for calibrations of the device. Not only did ICU nurses provide the mandatory calibration blood glucose levels, but also the routinely obtained blood glucose levels (that is, blood glucose measurements which were not requested by the device for calibrations but were taken by the nurses as dictated by the local guideline for blood glucose control) were entered into the device as well. Therefore, these measurements were also used for calibrations of the device. If the device displayed a message requesting an additional non-routine calibration to resolve a sensor performance issue (that is, a ‘Poor Sensor Signal’ alert), the nurses were permitted to disregard manufacturer recommendation and remove sensors rather than enter the requested calibration.

Each day, the place of insertion was photographed and inspected for redness, bruises, and swelling. In case the patient was awake, we questioned the patient whether it was painful. Every item could be scored as ‘none’, ‘minor’, or ‘major’.

Power calculation

We intended to enroll 50 patients to assess accuracy of the CGM device. With 50 patients, we expected to have at least 40,000 subcutaneous CGM device results and at least 1,200 blood glucose level measurements with the RapidLab 1265. Considering previous studies testing point accuracy, we assumed we would have a sufficiently high number of paired samples to enable evaluation of the point accuracy of the device.

Analysis plan

The glucose data collected with each new sensor were downloaded from the device after use in a patient; the arterial blood glucose levels were downloaded from the patient data management system. The arterial blood glucose levels in the patient data management system were compared with the entries for calibrations into the device. In case of an entry error, defined as a difference between the arterial blood glucose level in the patient data management system and the calibration entry of more than 9 mg/dL, the correct blood glucose level was used in the accuracy analysis. The subsequent pairs, though, were excluded from the accuracy analysis since these were influenced by the preceding entry.

For reporting point accuracy, we used analytical and clinical accuracy measures: that is, Bland-Altman plot with bias and limits of agreement (bias ± 1.96 × standard deviation of the bias) [30], glucose prediction errors, and Clarke error grid analyses [31]. According to International Organization for Standardization (ISO) criteria, 95% of the paired measurements should be within the glucose prediction error criteria; the consensus is that 95% of the values should be in zones A and 5% in zones B of Clarke error grid analyses. Finally, we expressed the linearity between the device glucose results and blood glucose results by the Pearson correlation coefficient and coefficient of determination, R2.

In a post hoc analysis, we also report point accuracy according to the recently published consensus recommendations [1]. In this round-table meeting of ICU experts in blood glucose control, it was recommended to always report the mean absolute relative difference (MARD) when testing a CGM device, where MARD values should be less than 14%; values of more than 18% should be considered to represent poor accuracy [32]. We added the MARD as a post hoc analysis. Furthermore, we analyzed the point accuracy following the recently published surveillance error grid [33]. For more details, see Additional file 1.

We also reported reasons for early disconnection, defined as the removal of a sensor before 72 hours. For details, see Additional file 1. The time between calibrations using an incorrect glucose value entry and the next calibration was extracted from the total connection time of the device. Definitions of the metrics used to assess device reliability, including those suggested by recent consensus recommendations [1], are described in Additional file 1.

Statistical analysis

We reported data as mean (± standard deviation) or median (interquartile range, or IQR) where appropriate. To be considered for the statistical analysis, each patient needed to have at least four comparative blood glucose results for accuracy analysis. However, the excluded patients remained included in the reliability analysis.

In a post hoc analysis, we used a linear mixed model to determine which variables influence the accuracy of the device. In addition, we stratified the accuracy results by diabetic status. For a detailed description of this model, see Additional file 1. Analyses were performed by using R (version: 2.15.1; R Foundation for Statistical Computing, Vienna, Austria).

Results

Patients and sensors

We included 50 patients. In total, we used 105 sensors (median of 1 (IQR 1 to 3) sensor per patient) with a total connection time of 4,639 hours (median of 46 (IQR 27 to 134) hours per patient and median of 46 (IQR 22 to 69) hours per sensor). Five patients from whom we could not collect the minimum number of four consecutive comparative blood draws were excluded from the point accuracy analysis. A CONSORT (Consolidated Standards of Reporting Trials) diagram is provided in Figure 1. Patient characteristics and metrics of glucose control are shown in Tables 1 and 2.

Figure 1
figure 1

CONSORT (Consolidated Standards of Reporting Trials) diagram of the study.

Full size image
Table 1 Patient characteristics
Full size table
Table 2 Measures of blood glucose control in patients included in point accuracy analysis
Full size table

We could not inspect the insertion site of three sensors. Major bruises were observed in 3 out of 102 inspected insertion sensor sites; 10 minor bruises were seen. Major redness of the skin was observed in 7 out of 102 insertion sensor sites, and minor redness in 6 out of 102 insertion sites. Swelling of the skin was never seen, and none of the conscious patients mentioned pain at the sensor insertion site.

Point accuracy

We collected 929 comparative samples (11 (IQR 7 to 28) samples per patient). Bland-Altman plot, glucose prediction error grid, and Clarke error grid are presented in Figure 2. The surveillance error grid is presented in Figure 3. The Pearson correlation coefficient was 0.81; the R2 was 0.65. The MARD was 14.8%.

Figure 2
figure 2

Bland-Altman plot with bias and limits of agreement (bias ± 1.96 × standard deviation of the bias), glucose prediction errors, and Clarke error grid analyses.

Full size image
Figure 3
figure 3

Surveillance error grid with risk scores.

Full size image

Fifty-eight percent of the device results were within 12.5% of the arterial blood glucose results (or within 10 mg/dL for results of less than 99 mg/dL), and 75% were within 20% of the arterial blood glucose results.

In the linear mixed model, only history of diabetes (P = 0.02) and number of calibrations per sensor (P = 0.04) affected the absolute difference between blood glucose and device result. Per each new calibration, the absolute difference decreased by 1.4% (standard error of 0.006%), meaning that the sensor performance increased. The effect of a history of diabetes was larger since an increase by 34.3% (standard error of 13.0%) in the absolute difference was found when comparing patients with a history of diabetes and patients without diabetes. In addition, we stratified the accuracy results by diabetic status; results are shown in Additional file 1: Figure S1 and Table S2. For detailed results of the multivariate random intercept model, see Additional file 1.

Reliability of the continuous glucose-monitoring device

Start-up time after placing a new sensor was 49 (IQR 43 to 58) minutes. The number of skips in data acquisition was low, resulting in availability of real-time data during 95% (IQR 89% to 98%) of the connection time per sensor. Table 3 summarizes reliability metrics of the investigational device.

Table 3 Device reliability
Full size table

Out of 105 sensors, 60 were removed before 72 hours; the reasons for removal and the connection times of sensors are shown in Table 4. Out of 105 sensors, 42 were removed before 72 hours after insertion for reasons other than ICU discharge or death, and 36 sensors were removed because of an unresolved ‘Poor Sensor Signal’ alert or a device error (19 with no attempt to resolve).

Table 4 Sensors removed less than 72 hours
Full size table

Discussion

We determined the point accuracy and reliability of a device specifically designed for continuous real-time monitoring of interstitial glucose levels in critically ill patients. The analytic point accuracy of the device was low in a typical cohort of patients from a mixed medical-surgical ICU, according to ISO criteria and consensus recommendations. The clinical point accuracy was low according to Clarke error grid analysis but better according to surveillance error grid analysis. The device had few downtimes, but one third of the sensors were removed prematurely because of sensor- or device-related problems.

The present findings are in line with results from a previous trial testing the same device in cardiac surgery patients [34]. In that study, the mean absolute relative difference was 12.2% with 95% real-time data. Similar results come from studies testing other devices for interstitial glucose monitoring that were originally designed for use in non-critically ill patients. Those studies were performed in cardiac surgery patients [21,24,35], surgery patients [26], patients with neurologic emergencies [27], and non-surgical patients [16,22,25], and only two reported more favorable accuracy results [21,22]. Taken together, these data suggest that point accuracy of interstitial glucose monitoring cannot replace blood glucose level measurements.

In contrast to our findings, a previous publication by Brunner et al. [18] suggests a better point accuracy of another interstitial CGM device in critically ill patients. This report combined data of two separate trials in medical ICU patients [19,36]. The tested device in that study was from the same manufacturer but was not specifically designed for use in critically ill patients. In addition, the sensor was used for up to 72 hours and never replaced. One important difference with the present study was that the sensors were placed exclusively under the skin of the abdomen in patients included in these two trials. In most other trials, sensors were inserted under the skin of the abdomen [16,18,22,24,26,28], thigh [25,26], or shoulder [21]. Reported point accuracies do not suggest superiority of one of these sites. Certainly, there could be other unknown and unreported factors that could have resulted in the differences in performance.

We performed a mixed linear model to determine which factors could have influenced the point accuracy of the tested sensor. Rank order of measurement and presence of a history of diabetes affected the accuracy. The finding that rank order of measurement improved sensor performance is not new [18] and certainly is not surprising: more calibrations may always increase accuracy of a sensor. A history of diabetes was the most important variable influencing point accuracy and deteriorated sensor performance by 34%. As yet, this effect remains unexplained. It could be that microcirculation alteration in patients with diabetes affects interstitial glucose level. However, in previous studies with interstitial devices, diabetes was not found to be significantly associated with poor sensor accuracy in critically ill and cardiac surgery patients [16,18,28]. Moreover, in a recent study in cardiac surgery patients, an impaired microcirculation did not affect accuracy of two interstitial glucose sensors from two different manufacturers [28]. The difference found between patients with and without diabetes might also be related to glucose variability. Patients with diabetes will have more glucose variability compared with patients without diabetes. Thereby, when the focus is percentage difference, a greater disparity could be found when variability differences are compared.

It should be stressed that we compared interstitial glucose measurements with glucose levels in arterial blood samples, which are far from comparable. Indeed, the interstitial glucose level is dependent on several factors other than the blood glucose level, such as the speed of glucose diffusion from blood to interstitial spaces, as well as the rate of glucose uptake by subcutaneous cells [37]. Importantly, these factors are not constant, particularly in critically ill patients. Furthermore, there is a time lag between interstitial glucose and blood glucose measurement [37]. Studies suggested that the interstitial glucose level decreases before the blood glucose decreases [37,38], although this was not confirmed in other studies [39]. It is probably very difficult, if not impossible, to correct for factors causing a difference between interstitial and arterial blood glucose levels. Moreover, it is unknown whether differences between arterial and interstitial glucose levels are physiological.

Nevertheless, subcutaneous glucose monitoring could have advantages. One potential advantage is that continuous monitoring of interstitial glucose levels enables detection of trends in the blood glucose level [32]. This could allow earlier responses to a rise or a decline of the blood glucose level. In both cases, knowledge of the direction of the trend may be more valuable than the exact blood glucose level.

It is clear that the tested device can never replace blood glucose measurements. First, initial calibrations are always necessary, as are calibrations every 8 hours thereafter. As nurses were allowed to perform additional blood glucose measurements and as we asked them to insert the values into the investigational device monitor where they were used for additional calibrations, the number of calibrations in this study was higher than mandated. In fact, this could have improved the accuracy of the investigational device: it is possible that with fewer calibrations, point accuracy becomes worse.

Our trial has several strengths and weaknesses. Strengths include the fact that we were able to use the sensors for several days in the participating patients. Moreover, we used accurate blood gas analyzer measurements for comparisons as well as for the calibrations. Furthermore, we were able to test the device in a typical mixed medical-surgical ICU. Weaknesses include the small sample size and the single-center design of the trial. Furthermore, we did not collect as many samples as we expected. A more important limitation of our trial, though, is that the vast majority of blood glucose levels were in a narrow range, preventing us from drawing firm conclusions regarding accuracy in the hypoglycemic range. Although the ICU nurses were not allowed to change insulin infusion rates, they could have anticipated hypoglycemia by performing new blood glucose measurements earlier than dictated by the local guideline for blood glucose control, allowing them to respond earlier to, for example, hypoglycemia. Still, some hypoglycemic events occurred, probably because not all nurses were paying attention to the readings of the investigational device. In addition, nurses could have noted that its point accuracy was not always good, so they could have mistrusted the device readings. Finally, we cannot exclude the possibility that hypoglycemia can occur even with the use of CGM. The latter possibility will be the subject of a planned trial. An accuracy analysis limitation was that the assessment focused on percentage difference comparisons between the continuous sensor and discrete reference points, evaluated by standards meant for discrete measurements for dosing. Another important limitation is that trend accuracy was not evaluated. Trending is the most interesting endpoint but mandates very short intervals (that is, a short as 15 minutes) between blood glucose reference measurements [32,40]. Trend accuracy should and will be evaluated in future studies.

Notably, length of stay in the ICU and sensor connection time were far from similar. This was caused by the fact that sensors could not be used before informed consent was obtained. Thus, we may have missed an important phase of glucose control (that is, the first day or days of stay in the ICU). In addition, because of sensor- or device-related factors, one third of the sensors were removed before sensor life ended. This is an important problem for the reliability of the device. However, nurses did not always attempt to solve sensor- or device-related problems that could have been solved. During conduct of the trial, they were always allowed to remove the sensor because of ‘Poor Sensor Signal’ alerts or recurrent alarms. With increasing device-specific experience, it could be that there are fewer early removals.

Conclusions

The point accuracy of a device designed for continuous real-time monitoring of the interstitial glucose level did not meet the ISO15197 standard or the recent consensus guidance for discrete glucose measurement for dosing when used on critically on critically ill patients admitted to a mixed medical-surgical ICU. Although this device is not a replacement for current blood gas analyzer measurements, a real-time system may be used for trend guidance on timely reference measurement for insulin adjustment. The device had few downtimes, but one third of the sensors were removed prematurely because of unresolved sensor- or device-related problems.

Key messages

  • An interstitial glucose sensor system in critically ill patients cannot replace blood glucose level measurements but may provide important trend information for glucose management.

  • Sensors are frequently removed prematurely for a variety of reasons.