Introduction

Self-monitoring of blood glucose (SMBG) is an important component of a comprehensive diabetes care plan and can help people with diabetes achieve glycemic control [1, 2]. The accuracy of SMBG results is an important consideration because these results are often used to make diabetes management decisions [3–7]. SMBG helps detect hypoglycemia and hyperglycemia and helps reduce glycemic variability, which has been shown to have an impact on diabetes complications [8–10]. Because SMBG results can be used to guide the self-care practices of people with diabetes, it is important for the blood glucose monitoring system (BGMS) to provide accurate results [3–5]. Performance evaluations of BGMSs for regulatory purposes may include the use of criteria of the International Organization for Standardization (ISO 15197:2013 [11]). Furthermore, Parkes-Consensus Error Grid analysis [12] may be used to evaluate the clinical impact of the error in measurement of blood glucose.

ISO guidelines were developed based on the need to establish requirements that result in acceptable performance of BGMSs and to specify procedures for demonstrating conformance. According to ISO guidelines, accuracy is based on the absolute difference of BGMS results from the reference result in lower glucose ranges, and the absolute relative (percent) difference of BGMS results from the reference result in higher glucose ranges; ISO requirements state that ≥95% of results shall fall within specified margins of error [11, 13]. Specifically, the ISO 15197:2003 standard required that ≥95% of results were within ±15 mg/dL (±0.8 mmol/L) or ±20% of the reference result for samples with glucose concentrations <75 mg/dL (<4.2 mmol/L) and ≥75 mg/dL (≥4.2 mmol/L), respectively [13]. Tighter accuracy criteria were adopted with the ISO 15197:2013 standard, which requires that ≥95% of results fall within ±15 mg/dL (±0.8 mmol/L) or ±15% of the reference result for samples with glucose concentrations <100 mg/dL (<5.6 mmol/L) and ≥100 mg/dL (≥5.6 mmol/L), respectively [11]. However, even though commercially available BGMSs have met the accuracy criteria required to be permitted to enter a specific regional market, differences exist in overall performance [14–18].

While individual system performance evaluations commonly use ISO criteria to assess accuracy, other types of analyses are well suited for evaluating comparative accuracy among multiple meters. For example, mean absolute relative difference (MARD) is a useful measure for comparing the accuracy of multiple meters in a single study, as it represents the mean difference from the laboratory reference value of all (100%) the glucose measurements obtained [19, 20]. Thus, a lower MARD value indicates better accuracy when comparing multiple BGMSs.

The CONTOUR®PLUS (CP) BGMS uses test strips with a flavin adenine dinucleotide-glucose dehydrogenase enzyme, a proprietary electron mediator and an advanced algorithm, which, in combination, form the technical basis for the system’s accuracy. This study compared the accuracy of the CP BGMS, as measured by MARD, with 4 other BGMSs (Accu-Chek® Active [ACA], Accu-Chek® Performa [ACP], FreeStyle Freedom™ [FF], and OneTouch® SelectSimple™ [OTSS]). The primary endpoint of the study was to evaluate differences in accuracy between the CP BGMS and the 4 other BGMSs across the overall tested glucose range; additional objectives were to evaluate differences in accuracy between the BGMSs in the low (≤80 mg/dL [≤4.4 mmol/L]), middle (81–180 mg/dL [4.5–10.0 mmol/L]), and high (>180 mg/dL [>10.0 mmol/L]) glucose ranges.

Methods

Study Design

This sponsor-investigator study was conducted by trained personnel in a Bayer facility where industry standards related to Good Clinical Practice and Good Laboratory Practice were followed. Institutional Review Board approval was obtained from the Allendale Investigational Review Board for the protocol, informed consent forms, and all study documents requiring such approval. Each subject completed the informed consent process before participating in the study. The study enrolled subjects aged ≥18 years with diabetes (>90%) and without diabetes (≤10%). The study consisted of a single study visit, approximately 30 min in length. Subjects were excluded from the study if they had a blood-borne infection, hemophilia or other bleeding disorder, or were pregnant.

Study staff performed a single fingerstick on each subject and tested all 5 BGMSs (CP [Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY, USA], ACA [Roche Diagnostics, Indianapolis, IN, USA], ACP [Roche Diagnostics, Indianapolis, IN, USA], FF [Abbott Diabetes Care, Inc, Alameda, CA, USA], and OTSS [LifeScan, Inc., Milpitas, CA, USA]) directly from the subject’s fingertip, wiping the finger in between each BGMS test. Second and third fingerstick blood samples taken from each subject were collected in microtubes containing lithium heparin anticoagulant. These samples were modified to achieve blood glucose concentrations throughout the measuring range. Blood samples were glycolyzed in a 32°C water bath for a maximum of 10 h to lower the glucose concentration (mean ± standard deviation [SD] duration 3.79 ± 1.53 h; range 0.30–7.33 h), and standard glucose solution was added to raise the glucose concentration.

For testing the modified blood samples, a fresh drop of blood was removed from the tube just prior to each meter test (after being placed on Parafilm or similar) and was tested promptly to avoid evaporation. All blood samples were tested on the 5 BGMSs using a test order rotation throughout the study. A single test strip lot per meter system was used for testing all samples.

After the meter tests, the remaining blood in the tube was immediately centrifuged for testing on a Yellow Springs Instruments (YSI) laboratory glucose analyzer (YSI Life Sciences, Inc., Yellow Springs, OH, USA). The time from the first meter test to centrifugation was not to exceed 15 min (mean ± SD interval 2.3 ± 1.2 min; range 1–7 min). The accuracy and precision of the 2 YSI analyzers were monitored throughout the study using 6 serum traceability control levels that spanned the range from 23.5 mg/dL (1.3 mmol/L) to 585 mg/dL (32.5 mmol/L). The target glucose levels for the controls had been previously determined using a reference method traceable to the National Institute of Standards and Technology Standard Reference Material 965a Glucose in Frozen Human Serum (aqueous New England Reagents Laboratory Glucose Standards) [21].

Assessments and Analyses

An analysis of variance (ANOVA) was performed for the absolute value of the relative percent difference between BGMS and reference results to compare the MARD between meter systems for all tested samples (both modified and unmodified) across a number of blood glucose ranges (≤80 mg/dL [≤4.4 mmol/L], 81–180 mg/dL [4.5–10.0 mmol/L], >180 mg/dL [>10.0 mmol/L], and across the overall range). Glucose oxidase–based systems (e.g., OTSS) are sensitive to oxygen in the sample, and sample modification may affect the concentration of oxygen in the sample. Thus, MARD analysis was also performed for unmodified samples only.

Samples were not included in the analyses if they did not fall within the hematocrit range (30–55%) or glucose concentration range (20–500 mg/dL [1.1–27.8 mmol/L] as measured by YSI) to accommodate the combined specifications of all 5 BGMSs. The majority of BGMSs in this study claimed a low glucose limit of 20 mg/dL (1.1 mmol/L) in their labeling (all BGMSs except CP and ACA, both of which have a low glucose limit of 10 mg/dL [0.6 mmol/L]).

P values comparing each BGMS to the CP BGMS were determined using post-ANOVA contrasts. P values <0.0125 are considered significant (Bonferroni adjustment to α). For each BGMS, a modified Bland-Altman plot of the difference of BGMS results from the YSI reference results was constructed. ISO 15197:2013 accuracy criteria [11] were represented by dashed lines. The points were differentiated by unique symbols, denoting whether the blood samples were modified or unmodified. Measured glucose values most similar to YSI reference values were indicated by points nearest the horizontal line (y = 0). To assess clinical accuracy, Parkes-Consensus Error Grid analysis [12] was performed for each BGMS. Adverse events (AEs) were monitored throughout the study.

Compliance With Ethics Guidelines

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000 and 2008. Informed consent was obtained from all subjects for inclusion in the study. Institutional Review Board approval was obtained from the Allendale Investigational Review Board.

Results

Subjects

A total of 106 subjects aged 18–84 years were enrolled and completed the study. Most subjects (n = 90) had type 2 diabetes, 8 subjects had type 1 diabetes, 2 subjects had diabetes of unknown type, and 6 subjects did not have diabetes (Table 1).

Table 1 Subject demographic and baseline characteristics
Full size table

Blood Samples

For each BGMS, 105 unmodified blood samples and 209 modified capillary blood samples were analyzed. Unmodified and modified blood samples from 1 subject were not included in the accuracy analyses because hematocrit was >55%, which is outside the hematocrit range that is specified in the labeling for 2 of the 4 BGMSs tested (ACA and OTSS). The third sample from another subject was glycolyzed, resulting in a YSI value of 12.3 mg/dL (0.68 mmol/L). This sample was, therefore, not evaluable because its YSI value was below the minimum glucose limit of 20 mg/dL (1.1 mmol/L) for all BGMSs combined.

The overall glucose concentration range of analyzed samples was 27 mg/dL (1.5 mmol/L) to 460 mg/dL (25.5 mmol/L) as measured by YSI. The hematocrit range of the blood samples was 34.5% to 55.0%, with a mean of 43.3%. In 1 case of testing of the FF BGMS, even though the glucose concentration of the blood sample was greater than 20 mg/dL (1.1 mmol/L) as measured by the YSI analyzer, the meter did not produce a numerical reading; rather, a “low” message was displayed. As per protocol, a result of 20 mg/dL (1.1 mmol/L) was assigned to include the result in numerical analyses. The largest numerical value representing the lower limit of the glucose concentration range for any of the meters tested in the study was 20 mg/dL (1.1 mmol/L); thus, setting the value to 20 mg/dL (1.1 mmol/L) assigned the meter that produced the non-numerical result the smallest possible error relative to the YSI value. The remaining BGMSs tested in the study displayed a numerical result for all samples.

Accuracy

Mean absolute relative difference comparisons to the CP BGMS are shown in Table 2. Analysis of the primary endpoint showed that the CP BGMS had a statistically significantly lower MARD than the other BGMSs across the overall glucose range tested (27–460 mg/dL [1.5–25.5 mmol/L] as measured by YSI). In the high glucose range (>180 mg/dL [>10.0 mmol/L]), the CP BGMS had a statistically significantly lower MARD than all BGMSs tested. In the low (≤80 mg/dL [≤4.4 mmol/L]) and middle (81–180 mg/dL [4.5–10.0 mmol/L]) glucose ranges, the CP BGMS had a lower MARD than all BGMSs tested and was statistically significantly lower than all systems except for ACP.

Table 2 MARD comparisons to the CP BGMS
Full size table

The MARD analysis using only unmodified samples revealed that the CP BGMS had a lower MARD than all BGMSs tested in the overall glucose range (45–460 mg/dL [2.5–25.5 mmol/L] as measured by YSI) and was statistically significantly lower than all systems except for the ACA BGMS (Table 2).

Modified Bland-Altman plots for each BGMS are shown in Fig. 1a–e, with the YSI value on the x-axis and the signed difference between the meter result and the YSI result on the y-axis.

Fig. 1
figure 1

Modified Bland-Altman plots of the difference of BGMS results from laboratory reference results for a CP, b ACA, c ACP, d FF, and e OTSS. BGMS blood glucose monitoring system, CP CONTOUR®PLUS, ACA Accu-Chek® Active, ACP Accu-Chek® Performa, FF FreeStyle Freedom™, OTSS OneTouch® SelectSimple™, YSI Yellow Springs Instruments. Dashed lines ±15 mg/dL (±0.8 mmol/L) or ±15% of the reference result for samples with YSI glucose concentrations <100 mg/dL (<5.6 mmol/L) and ≥100 mg/dL (≥5.6 mmol/L), respectively

Full size image

By Parkes-Consensus Error Grid analysis, 99.7% (313/314) of CP BGMS results, 98.1% (308/314) of ACA BGMS results, and 99.4% (312/314) of ACP BGMS results were within Zone A, with the remainder within Zone B (Table 3). For the FF BGMS and the OTSS BGMS, 94.3% (296/314) of results were within Zone A, with the remainder within Zone B (Table 3).

Table 3 Parkes-Consensus Error Grid analysis of BGMS results
Full size table

Safety

There were 3 mild, anticipated, non–device-related AEs; the 3 subjects had hypoglycemia (blood glucose value <60 mg/dL [<3.3 mmol/L]). All AEs were managed, resolved, and documented.

Discussion

Using an accurate BGMS may help people with diabetes to use their blood glucose values from SMBG to make better informed decisions about their diabetes management [3–7]. MARD analysis is useful for comparing the accuracy of multiple meters in a single study [19]. In this study, the CP BGMS had a statistically significantly lower MARD in the overall glucose range (27–460 mg/dL [1.5–25.5 mmol/L]) than all BGMSs tested. The CP BGMS also had the lowest MARD in the low (≤80 mg/dL [≤4.4 mmol/L]), middle (81–180 mg/dL [4.5–10.0 mmol/L]), and high (>180 mg/dL [>10.0 mmol/L]) glucose ranges, and was statistically significantly lower than the other BGMSs, except for the ACP BGMS in the low and middle glucose ranges.

Parkes-Consensus Error Grid analysis can be used to assess the clinical impact of inaccurate blood glucose results. In this study, all BGMS results were within Zone A or Zone B of the Parkes-Consensus Error Grid, with the majority (>94%) within Zone A across groups. Zone A of the Parkes-Consensus Error Grid indicates no effect on clinical action, and Zone B indicates altered clinical action with little or no effect on clinical outcome [12]. There were no results within Zones C, D, or E; results in these zones would indicate altered clinical action with increasingly severe effects on clinical outcome.

In addition to the choice of meters for evaluation, comparative analyses of meter accuracy may involve differences in study design. Such differences should be considered when assessing the results of comparative analyses. For example, meters may be compared based on their ability to fulfill ISO accuracy criteria or using MARD analysis. Being continuously valued (i.e., a decimal number), MARD facilitates the use of powerful methods for comparing multiple meters simultaneously (e.g., ANOVA). Furthermore, as the clinical risk associated with meter inaccuracy varies depending on blood glucose concentration, evaluation of meter accuracy in the low, middle, and high glucose ranges in addition to the overall glucose range provides a more complete assessment of accuracy than does evaluation in the overall glucose range only.

As patients with diabetes often have comorbidities such as hyperlipidemia or chronic kidney disease and may also be taking multiple medications [22, 23], health care providers and people with diabetes should also be aware of agents that could interfere with the accuracy of blood glucose results (Table 4). For example, maltose can interfere with some SMBG systems, leading to anomalously high glucose measurements that could mask hypoglycemia or give an inaccurate indication of hyperglycemia [3]. Maltose can be found in certain immunoglobulin products, and it is a metabolic by-product of the icodextrin that is used in peritoneal dialysis.

Table 4 Comparison of BGMS specifications
Full size table

Chronic diseases, including diabetes, present serious health concerns, and their prevalence is increasing in low-, middle-, and high-income countries [24]. Relatively simple and cost-effective steps can be taken to prevent or significantly delay the onset of diabetes and its complications [25]. For people with diabetes, treatment and clinical monitoring to achieve glycemic and metabolic control are core components of effective diabetes care [25]. However, contributing factors to suboptimal glycemic control include underutilization of blood glucose data and the use of glycated hemoglobin as the sole measure of glycemic control [26]. SMBG is an integral component of diabetes management and helps people with diabetes to attain better blood glucose control [1, 2], especially when they are educated about how to act on their results [5, 9, 27, 28]. Along with the importance of health care providers educating patients about proper SMBG technique, advances in SMBG technology can help minimize inaccuracy of blood glucose results [5, 9, 20].

The primary objective of this study was evaluation of all samples (unmodified and modified); however, it is recognized that glucose oxidase–based systems (e.g., OTSS) are sensitive to interference from oxygen in the sample [3, 20, 29, 30], as may occur during sample modification. In light of this recognition, a sub-analysis on unmodified samples was performed, yielding similar results and suggesting limited impact of blood sample modification on results derived from glucose oxidase–based systems in this study.

A potential limitation of this study is that there is a finite number of BGMSs that can realistically be compared in a single study. It is possible that the results of the study may have been different if another group of BGMSs had been selected. However, based on the design of the study, it would not have been feasible to conduct the study using a larger group of BGMSs, as it was important to minimize the time elapsed from the first meter test to the last meter test of a given blood sample to ensure accurate comparisons. For this study, BGMSs were selected to represent the most relevant comparators for the CP BGMS. Another potential limitation of the study is that only 1 lot of test strips was used, a noteworthy consideration given that variation between test strip lots may impact the apparent accuracy of a BGMS [14]. At the same time, however, using 1 test strip lot reduces variability/noise in the results.

Conclusions

The results of this study showed the CP BGMS had a lower mean difference from the reference value than all other systems tested across all glucose ranges. The technologic aspects of the system itself are contributors to the accuracy of the CP BGMS observed in this study. As people with diabetes learn more about how the use of their blood glucose data can enable improved glycemic control, use of the most accurate BGMS may further contribute a significant impact toward helping people manage their diabetes more effectively by making better informed decisions.