# Utilizing distributional analytics and electronic records to assess timeliness of inpatient blood glucose monitoring in non-critical care wards

**Part of the following topical collections:**

## Abstract

### Background

Regular and timely monitoring of blood glucose (BG) levels in hospitalized patients with diabetes mellitus is crucial to optimizing inpatient glycaemic control. However, methods to quantify timeliness as a measurement of quality of care are lacking. We propose an analytical approach that utilizes BG measurements from electronic records to assess adherence to an inpatient BG monitoring protocol in hospital wards.

### Methods

We applied our proposed analytical approach to electronic records obtained from 24 non-critical care wards in November and December 2013 from a tertiary care hospital in Singapore. We applied distributional analytics to evaluate daily adherence to BG monitoring timings. A one-sample Kolmogorov-Smirnov (1S-KS) test was performed to test daily BG timings against non-adherence represented by the uniform distribution. This test was performed among wards with high power, determined through simulation. The 1S-KS test was coupled with visualization via the cumulative distribution function (cdf) plot and a two-sample Kolmogorov-Smirnov (2S-KS) test, enabling comparison of the BG timing distributions between two consecutive days. We also applied mixture modelling to identify the key features in daily BG timings.

### Results

We found that 11 out of the 24 wards had high power. Among these wards, 1S-KS test with cdf plots indicated adherence to BG monitoring protocols. Integrating both 1S-KS and 2S-KS information within a moving window consisting of two consecutive days did not suggest frequent potential change from or towards non-adherence to protocol. From mixture modelling among wards with high power, we consistently identified four components with high concentration of BG measurements taken before mealtimes and around bedtime. This agnostic analysis provided additional evidence that the wards were adherent to BG monitoring protocols.

### Conclusions

We demonstrated the utility of our proposed analytical approach as a monitoring tool. It provided information to healthcare providers regarding the timeliness of daily BG measurements. From the real data application, there were empirical evidences suggesting adherence of BG timings to protocol among wards with adequate power for assessing timeliness. Our approach is extendable to other areas of healthcare where timeliness of patient care processes is important.

## Keywords

Distributional analytics Timeliness Quality of care Diabetes mellitus Inpatient Electronic medical records## Abbreviations

- 1S-KS
one-sample Kolmogorov-Smirnov

- 2S-KS
two-sample Kolmogorov-Smirnov

- ADA
American Diabetes Association

- BG
blood glucose

- BIC
Bayesian information criterion

- Cdf
cumulative distribution function

- DM
diabetes mellitus

- ICL
integrated classification criterion

- O&G
obstetrics & gynecology

- POC
point-of-care

- SD
standard deviation

## Background

Regular monitoring of blood glucose (BG) in hospitalized patients is an important component of inpatient diabetes mellitus (DM) care. The American Diabetes Association (ADA) recommends monitoring blood glucose (BG) four times per day (i.e., before meals and at bedtime) in hospitalized patients with DM. If the patient is fasting or receiving continuous enteral or parenteral nutrition, the recommended BG monitoring frequency is once every 4 to 6 h. BG monitoring is performed every 30 min to 2 hourly, if a patient is on an intravenous insulin infusion [1, 2, 3, 4].

Timely measurement of BG facilitates the delivery of inpatient DM care, and allows treatment regimens to be revised to achieve optimal glycaemic control. For example, pre-prandial BG measurement supports clinical decision-making by enabling healthcare providers to prescribe an appropriate dose of supplemental insulin to correct for pre-meal hyperglycaemia. Therefore, a holistic assessment of the quality of inpatient DM care should include an evaluation of compliance to BG monitoring timings. However, methodologies assessing timeliness are not well-established.

A study by Buchs and colleagues assessed the compliance to BG monitoring protocols by using a pie chart to display the proportion of BG timings in a specific portion of time, such as, pre-meals and post-meals [5]. The majority of BG measurements (approximately 75 %) occurred before mealtimes over a 6-month period. However, this method aggregated data over a long period which did not facilitate further evaluation of circumstances surrounding non-adherence, if any was detected. This would limit its utility in a real-world setting.

Continuous evaluation of BG measurements can now be conducted with the advent of electronic medical records [6]. We propose using distributional analytics as a surveillance tool to provide high-resolution empirical evidence of adherence to clinical protocols where timeliness is an important factor [7]. Our proposed analytical approach will address the following: (i) assessment of daily adherence to a BG monitoring protocol by ward over a pre-specified period, and (ii) detection of wards and days exhibiting potential non-adherence or changes in patient care processes.

## Methods

We examined all point-of care (POC) BG measurements performed in 24 non-critical care wards in a 1000-bed tertiary care hospital, National University Hospital, from November to December 2013. Capillary BG measurements were performed using POC glucose meter Accu-Chek Inform II (Roche, Basel Switzerland) and stored in a central laboratory database. The BG data downloaded from the central database contains de-identified patient identifier, BG value, patient location (i.e., ward), date and time of BG measurement. The specialty of the ward indicates the predominant type of patients in the ward. We did not collect additional data including mealtimes and patient demographics, which reflected the current set-up of the laboratory database.

We analyzed all the POC BG timings during the 2-month period. BG timings were converted to hours according to the standard 24-h clock. In the hospital where we conducted the study, the mealtimes are targeted around 0800, 1200 and 1800 h, while the bedtime is targeted around 2200 h.

The majority of inpatients requiring BG monitoring are eating regular meals (i.e., not fasted). Hence, the ideal BG timings should be distributed with multi-modes within a day, where the modes should occur before mealtimes and at bedtime. An extreme contrast would be a uniform distribution in the BG timing, which reflects a complete lack of adherence to ADA-recommended timings for patients eating regular meals. Hence, a simple way to assess potential adherence to BG monitoring protocol is to perform a one-sample Kolmogorov-Smirnov (1S-KS) test that detects deviation of BG timings from a continuous uniform distribution between 0 and 24 h [8], where a significant p-value (i.e., *p*-value < 0.05) suggests potential adherence to protocol. For presentation purposes only, we ranked wards in a decreasing order according to the proportions of days with significant deviations from the uniform distribution over the 2-month period and ties were broken with the median p-value. We plotted the boxplots of p-values for each ward stratified by medical specialty to investigate for potential differences between specialties.

*p*-values < 0.05) [13]. The simulation procedure for each BG timing in a day was as follows:

- 1.
We generated a random number from a uniform distribution, i.e.,

*u*~*U*(0, 1) - 2.
We identified the closest observed BG timings interval, (

*t*_{1,}*t*_{2}], on that day, such that,*F*(*t*_{1}) <*u*≤*F*(*t*_{2}), where*F*(·) is the empirical cdf of BG timings, then \( {t}_{simulate}=\left({t}_2-{t}_1\right)\frac{u-F\left({t}_1\right)}{F\left({t}_2\right)-F\left({t}_1\right)}+{t}_1. \)

We gathered the estimated power from simulation across all days to compute the median and mean power for each ward. Wards with average or median power more than or equal to 90 % were considered adequately powered for assessing adherence. To understand the interplay among effect size, sample size and p-value across all wards in the hospital, we also regressed –log_{10}(*p*-value) of 1S-KS test on standardized effect size and sample size, with interaction between the two standardized quantities. The effect size is the 1S-KS test statistic, \( D={ \sup}_y\left|F(y)-\frac{1}{24}y\right| \), where *y* denotes the BG timing and *D* quantifies the magnitude of deviation between the empirical cdf of BG timing and the uniform distribution.

To complete the assessment of BG monitoring protocol with 1S-KS, we corroborated the p-value findings visually with the cdf plots to inspect for features that we would expect from a ward that had been compliant with the protocol. Testing against a uniform distribution is based on a simple assumption of an extreme non-adherence behavior. However, other non-adherent behaviours may exhibit distributions of BG timings that differ from a uniform distribution resulting in statistically significant findings with 1S-KS tests too. To rely less on the parametric assumption of the uniform distribution, we also proposed to test the BG timing distribution of the current day against the previous day via a two-sample Kolmogorov-Smirnov (2S-KS) test [14, 15]. A significant p-value from a 2S-KS test suggests a statistically significant difference in BG timing distributions between two consecutive days.

*y*

_{ ij }denotes the j-th BG timing on the i-th day for

*i*= 1, …, 61, and

*j*= 1, …,

*n*

_{ i }(where

*n*

_{ i }is the sample size on the i-th day). Assuming there are K components,

*π*

_{ k }is the component probability assigned to the k-th component, and

*f*(⋅) denotes the normal density function with component-constant mean,

*u*

_{ k }, and variance,

*σ*

_{ k }

^{2}, where

*= (*

**μ***μ*

_{1},

*μ*

_{2}, …,

*μ*

_{ K }) ' and

**σ**^{2}= (

*σ*

_{1}

^{2},

*σ*

_{2}

^{2}, …,

*σ*

_{ K }

^{2}) '. We assume a multinomial logit model for the component probabilities with independent variable,

**w**_{ ij }= (

*I*

_{ij2}, …,

*I*

_{ ijd }, …,

*I*

_{ij61}) ', where

*I*

_{ ijd }takes a value of 1 if the i-th day is the d-th day in this 2-month period:

*= (*

**α**

**α**_{ k }

^{'})

_{k = 1,…,K}

^{'}and

**α**_{1}≡0. So the full log-likelihood function for all BG timing in a ward is:

We used the integrated classification likelihood (ICL) as the criterion for model selection. It is a more robust criterion than Bayesian information criterion (BIC) in the presence of violation in model assumptions [16]. Hence, if the wards with high power were adherent to the BG monitoring protocol, there should be only four components with small standard deviation (SD), which we had arbitrary taken it to be SD < 1, and the mean values of these components should be prior mealtimes and around bedtime. We summarized the mean and variance estimates for each component with SD <1 using the minimum, median and maximum statistics, and plotted the mean ± 1.96 × SD of each component for all wards. For the probability estimates, we first took the average probability estimates within each ward and reported their minimum, median and maximum for each component with SD < 1. For components with SD ≥ 1, their minimum, median and maximum statistics were reported collectively for mean, variance and probability estimates.

We used the R statistical software program to analyze the BG data. R packages, *stats, Matching* and *flexmix* were used to perform the simulation (R functions: *runif* and *approx*), KS-tests (R functions: *ks.test* and *ks.boot*) and mixture modelling (R functions: *stepFlexmix* and *FLXPmultinom*) [14, 17, 18].

## Results

There was a total of 73,182 BG measurements in 23,221 patient-days from 3,673 patients during the period November to December 2013. Among the 23,221 patient-days, 9.4 % of patient-days had more than four BG measurements, 42.3 % of them had four BG measurements, and 48.3 % had less than four BG measurements.

_{10}(

*p*-values) keeping standardized effect size fixed at 1, which is about 99.7 % reduction in p-values, and similarly a one unit increase in the standardized effect size alone could lead to an increase of 2.41 in –log

_{10}(

*p*-values) keeping standardized sample size fixed at 1, which is about 99.6 % reduction in p-values (see Table 1). Hence, the change in p-value that was attributable to sample size and effect size respectively was comparable.

The relationship of –log_{10}(*p*-values) with effect size and sample size

Beta (95 % CI) | |
---|---|

Standardized effect size | 1.52 (1.5, 1.54) |

Standardized sample size | 1.67 (1.66, 1.69) |

Interaction between the two standardized quantities | 0.89 (0.87, 0.9) |

Coefficient of determination ( | 97.2 % |

In Fig. 1, the majority of 2S-KS test p-values were insignificant. For the scenarios where two consecutive days had significantly different BG timing distributions, they contained the days with less than four pronounced steps observed in Fig. 2 previously. Further exploration among wards with high power by utilizing both the 1S-KS and 2S-KS *p*-values within a two consecutive days moving window did not suggest frequent potential change from or towards non-adherence to protocol (i.e., a significant 2S-KS p-value, and only one significant 1S-KS *p*-value) as only three occurrences across all wards over the 2-month period had transitions from non-adherence to adherence or vice versa.

Mean, standard deviation and mixture probability estimates from the mixture models among wards with high power

Components with SD < 1 | Components with SD ≥1 | ||||
---|---|---|---|---|---|

First | Second | Third | Fourth | Others | |

Mean (converted to hours according to 24-h clock) | 7.23 (6.29, 7.52) | 11.38 (11.11, 11.61) | 17.18 (17.02, 17.37) | 21.94 (21.57, 22.26) | 11.22 (4.38, 21.56) |

Standard deviation (hours) | 0.28 (0.13, 0.75) | 0.34 (0.19, 0.44) | 0.25 (0.13, 0.38) | 0.26 (0.21, 0.93) | 5.94 (1.58, 6.55) |

Mixture probabilities | 0.22 (0.18, 0.29) | 0.18 (0.16, 0.2) | 0.22 (0.19, 0.23) | 0.2 (0.15, 0.23) | 0.12 (0.06, 0.22) |

As for the SD estimates, the median across the four components were between 0.25 and 0.34 h (i.e., for each component, 95 % of the BG measurements were approximately within an hour, or an hour and 20 min interval). For components with SD ≥ 1, the range of the total number of components were 1 and 3. These components may potentially represent the BG measurements taken from patients with hypoglycaemia, or patients who were fasting or receiving insulin infusions. The minimum SD for these components was around 1.6, suggesting 95 % of the BG measurements were within a time interval ≥ 6 h. The majority of the probability estimates for each component were distributed almost equally across the components, except for a few cases where the components with large SD had smaller probabilities when compared to the four components with small SD.

## Discussion

In this paper, we applied distributional analytics-based methodology, mixture modelling and visualization to assess adherence to an inpatient BG monitoring protocol in non-critical care wards. To alleviate the occurrence of false positives due to the interplay of small sample size and effect size in the 1S-KS test, we performed a simulation study to identify wards where the daily total number of BG measurements had a mean or median power greater than or equal to 90 % over the 2-month period (i.e., wards with high power). Restricting our assessment to these wards, we further corroborated the 1S-KS test findings using: (1) visualizations with cdf plots, (2) 2S-KS tests, and (3) mixture modelling.

The BG timing distribution of wards with high power exhibited four modes before three mealtimes and around bedtime. From the cdf plots, we observed four pronounced steps and from the mixture modelling, we observed exactly four components with small standard deviation. This phenomenon concurred with our expectation of a ward following ADA monitoring recommendations for patients eating regular meals. In particular, the cdf plots and the mean ± 1.96SD intervals from mixture modelling allowed us to identify the time interval where majority of BG measurements was taken, which was not possible using the pie chart with aggregated BG data [5]. These visualizations facilitate further investigations of non-adherence by healthcare administrators by identifying problematic timings during the day. This highlights the importance of pairing statistics with visualization to deliver actionable information. When we combined both the 1S-KS and 2S-KS tests within a 2-day moving window over the 2-month period, there was a low occurrence of change from or towards non-adherence to protocol.

Although we could only draw conclusions for wards with high power to minimize false negatives, we could accrue a sufficient number of BG timings for those wards with moderate power by aggregating the BG data over a two-day window. For example, the 6-th medical ward in Fig. 1 had median power close to 90 % (i.e., 87.3 %) and median effect size close to the wards with high power. We proposed to combine the BG timing of the current day of interest with its previous day to obtain a larger sample size while assuming the patient care process was the same between two consecutive days. With a two-day window, we obtained six other wards with an average or median power greater than 90 %, and only the last ward in the first five specialties and the two O&G wards in Fig. 1 were not adequately powered. The results on adherence for these six additional wards with moderate power were similar to the 11 wards with high power. In particular, we identified 20 out of 360 instances = 60 2-day windows × 6 wards were potentially non-adherent to BG protocol and there were no occurrence of change from or towards non-adherence to protocol.

As our proposed approach uses ward level as the unit of analysis, it captures the correlation of BG timings within a patient through the marginal distribution of BG timings from all patients in the ward. Although we had identified some days without BG measurements, these occurrences were low and sporadic, and therefore unlikely to affect our findings.

Our proposed approach provided an analytical way to alert healthcare administrators of potential non-adherence to protocol for a specific day and ward. However, to extend our approach to continuous surveillance through daily monitoring, we proposed to modify the criterion for wards with high power based on the estimated power for each day instead of the mean or median power over the entire 2-month period. This will be useful for a large hospital and will open up the opportunity to monitor adherence on a daily time-scale.

## Conclusion

In this paper, we have proposed a way to assess adherence to BG monitoring protocols using electronic BG records. To reduce false negatives from our proposed assessment with 1S-KS test, we used concepts from power calculation and simulation to determine the wards that are adequately powered with empirical data. By investigating the p-values of KS tests, cdf plots and mixture modelling, we found that wards with high power were adherent to the BG monitoring protocol.

In summary, our approach leverages on the distributional analytics and the availability of electronic records of laboratory data to provide a practical surveillance tool for identifying potential non-adherence to clinical workflow. Our approach is also applicable to other areas of healthcare where timeliness of patient care processes is paramount, for example, medication administration or timed blood investigations.

### Ethics approval and consent to participate

The investigations were carried out in accordance with the ethical codes and guidelines of the Nuremberg Code (1946), the Declaration of Helsinki (1964), the Belmont Report (1979) and the Singapore Guideline for Good Clinical Practice (1998), and were approved by the National Healthcare Group Domain Specific Review Board, including waiver of informed consent.

### Consent for publication

Not applicable.

### Availability of data and materials

The dataset supporting the conclusions of this article is available in: http://blog.nus.edu.sg/dasa/bgdataset1.

## Notes

### Funding

This work was supported by the Centre for Health Services and Policy Research SBRO14/NS01G from the National University Health Systems Pte Ltd, National University of Singapore (NUS) Start-up Grant (WBS: R-608-000-059-133), and Ministry of Education Singapore Academic Research Funds Tier 1 grant (WBS: R148000195112).

## References

- 1.Moghissi ES et al. American Association of Clinical Endocrinologists and American Diabetes Association consensus statement on inpatient glycemic control. Diabetes Care. 2009;32(6):1119–31.CrossRefPubMedPubMedCentralGoogle Scholar
- 2.Thomas P, Inzucchi SE. An internet service supporting quality assessment of inpatient glycemic control. J Diabetes Sci Technol. 2008;2(3):402–8.CrossRefPubMedPubMedCentralGoogle Scholar
- 3.Goldberg PA et al. “Glucometrics”-Assessing the quality of inpatient glucose management. Diabetes Technol Ther. 2006;8(5):560–9.CrossRefPubMedGoogle Scholar
- 4.Cook CB et al. Assessing inpatient glycemic control: what are the next steps? J Diabetes Sci Technol. 2012;6(2):421–7.CrossRefPubMedPubMedCentralGoogle Scholar
- 5.Buchs AE, Rapoport MJ. Institutional glucometrics to determine glucose control as practiced by general medicine wards. IMAJ-Isr Med Assoc J. 2010;12(8):463.Google Scholar
- 6.Cook CB et al. Inpatient point-of-care bedside glucose testing: preliminary data on use of connectivity informatics to measure hospital glycemic control. Diabetes Technol Ther. 2007;9(6):493–500.CrossRefPubMedGoogle Scholar
- 7.Pivovarov R et al. Temporal trends of hemoglobin A1c testing. J Am Med Inform Assoc. 2014;21(6):1038–44.CrossRefPubMedPubMedCentralGoogle Scholar
- 8.Massey Jr FJ. The Kolmogorov-Smirnov test for goodness of fit. J Am Stat Assoc. 1951;46(253):68–78.CrossRefGoogle Scholar
- 9.Goldstein R. Power and sample size via MS/PC-DOS computers. Am Stat. 1989;43(4):253–60.Google Scholar
- 10.Manly BF. Randomization, bootstrap and Monte Carlo methods in biology. Boca Raton, Florida: CRC Press. 2006;70Google Scholar
- 11.Rubinstein RY and DP Kroese. Simulation and the Monte Carlo method. Hoboken, New Jersey: John Wiley & Sons 2011;707.Google Scholar
- 12.L’Ecuyer P. Non-uniform Random Variate Generations. In: International Encyclopedia of Statistical Science. Berlin, Heidelberg: Springer. 2011; p. 991-995.Google Scholar
- 13.Davis P.J. Interpolation and approximation. Mineola, New York: Dover publications, Inc. 1975.Google Scholar
- 14.Abadie A. Bootstrap tests for distributional treatment effects in instrumental variable models. J Am Stat Assoc. 2002;97(457):284–92.CrossRefGoogle Scholar
- 15.Sekhon, JS. Multivariate and propensity score matching software with automated balance optimization: the matching package for R. Journal of Statistical Software, Forthcoming, 2008.Google Scholar
- 16.Biernacki C, Celeux G, Govaert G. Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans Pattern Anal Mach Intell. 2000;22(7):719–25.CrossRefGoogle Scholar
- 17.Leisch F. Flexmix: A general framework for finite mixture models and latent glass regression in R. 2004.Google Scholar
- 18.Team RC. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. p. 2014. ISBN 3-900051-07-0.Google Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.