FormalPara Key Points

In a simulated setting, the use of a dose calculator integrated with a web-based paediatric formulary reduced the estimated marginal mean probability of a calculation error from 39 to 24%. Paediatric healthcare professionals therefore may benefit from using this technology.

At the same time, digital solutions for dose calculation should be used with due caution as they may introduce risks as well. Special attention is needed for correct selection of parameters and transferring the calculation results to other information systems.

1 Introduction

Among all paediatric prescribing errors, dosing errors are the most common, accounting for 2.2–36.5% of all prescribing errors [1,2,3,4,5,6,7]. Incorrect dosing is thought to be caused by the complexity of paediatric prescribing, as nearly all drugs have varying dose recommendations based on the child’s age, weight or body surface area [8]. Furthermore, drugs are diluted and manipulated to meet the need for small doses. In addition, clear dosing guidance is lacking for off-label drugs [5,6,7,8]. Of all dosing errors, calculation errors are the most common in neonatal and paediatric patients. Davis et al. and Kirk et al. report error rates varying from 8.4 to 28.2% [4, 9]. Studies by Rowe et al. and Glover and Sussmane confirm that healthcare professionals have difficulties calculating the correct dose [10, 11]. Kaushal et al. show that 34% of all potential adverse drug events (ADEs) in paediatric inpatients involve incorrect dosing [6]. Therefore, there is an urgent need to reduce the number of calculating errors.

The availability of digitized paediatric dosing guidelines of the Dutch Paediatric Formulary [12] provided the opportunity to develop a website-integrated dose calculator that combines dosing recommendations of the formulary with patient variables. A manuscript describing the development of the calculator has been accepted for publication [13]. In a simulated setting, we tested the ability of the calculator to reduce calculation errors.

2 Methods

2.1 Study Setting

This performance study was designed as a non-randomized, comparative simulation trial comparing the odds ratios for calculation errors in a standardized calculation assessment in a control group versus an intervention group.

2.2 Participants

All users of the Dutch paediatric formulary were invited through the formulary’s homepage to voluntarily participate in a calculation assessment. Participants were categorized by their profession—physicians, pharmacists or other professionals (nurses/pharmacy technicians)—in order of date of registration.

Personal data other than age, profession and IP address were not collected. The simulation study was not subject to Institutional Review Board approval according to the Dutch Medical Research Involving Human Subjects Act (WMO).

2.3 Sample Size Calculation and Group Allocation

Although participants were not strictly randomly allocated to one of the groups, we applied equation 4 for cluster randomized trials described by Hayes and Bennett [14] to calculate the power of the study, in order to address the multiple dichotomous outcomes of each respondent. This calculation resulted in a minimum sample size of 34 per group to show a 50% reduction in overall error rate with a power of 80% and a significance level of 0.05, using a coefficient of variation of 0.6 when each subject performed 23 calculations. Based on the study of Rowe et al. we estimated the a priori error rate at 10% [10].

Two hundred and thirty-eight users registered to participate. Numbers 1–25 of each profession group were allocated to the control group; numbers 26–50 of each profession group were allocated to the calculator group. Numbers from 51 onwards of each group (88 out of 238) were excused. Anticipating a 50% non-response, we invited 75 participants per study group (25 physicians, 25 pharmacists and 25 other professionals) to achieve a minimal inclusion of 34 participants per study group.

2.4 Intervention

The control group was instructed to perform a calculation assessment with conventional tools (i.e. a pocket calculator) and the dosing recommendations as listed on the Dutch Paediatric Formulary website. The intervention group was instructed to perform the same calculation assessment using the website-integrated dose calculator of the Dutch Paediatric Formulary.

2.5 Calculation Assessment

The calculation assessment consisted of 14 case descriptions with either one or two calculations per case (23 calculation items in total). The cases covered the paediatric age range from neonate to adolescent; the selected drugs were regularly used drugs and different calculation challenges were presented: dosing based on milligrams per kilogram, on milligram per square meter of body surface area, on International Units (IU) per kilogram, respecting the maximum dose and using weight of a premature neonate in grams instead of kilograms. The control group and intervention group each completed the same assessment. The cases were presented in a random order. Participants were instructed to always use the lowest dose of a dose range and to provide the calculation result in the specified dose unit. Specific instructions on rounding were not provided. The calculation assessment was designed as an online questionnaire using the Survey Gizmo online platform. The survey could be completed at any place and time at the discretion of the respondent. To mimic daily practices with stressful circumstances and to prevent meticulous (re-)calculations, the time for the calculating tasks was limited to 2 min per case. If a calculation was not completed within 2 min, the participant was automatically directed to the next question. Participants in the calculator group were instructed to read the online instruction manual or watch the online instruction tutorial on the use of the calculator before starting the assessment. Participants in both groups were advised to open two different web browsers for the purpose of completing the assessment in Survey Gizmo and simultaneously consulting the Paediatric Formulary website. Participants were informed that the Survey Gizmo assessment could not be interrupted and completed afterwards. Multiple completions were identified by tracking of the IP address. To get accustomed to the procedure, the assessment started with a dummy question.

The control group completed the assessment prior to the online launch of the calculator (May–June 2015). The intervention group completed the assessment after the launch of the calculator on September 7, 2015 (September 25, 2015–February 04, 2016).

2.6 Data Analysis and Statistics

Survey results were included in the data analyses if six calculation items or more had been completed. Of duplicate IP addresses, the survey with the highest number of completed calculation items was included in the analysis.

The primary outcome parameter was a dichotomous variable indicating correct or erroneous calculation outcome. An erroneous calculation was defined as any deviation from the correct outcome plus or minus 0.05 units of dosing to account for minor rounding errors. Calculation outcomes not provided within the set time frame of 2 min qualified as missing data in the dataset. Any exceedance of the absolute maximum dose was considered to be an erroneous calculation.

The definition of error for the primary outcome was very strict and did not reflect clinical practice, where a 10% deviation from the calculated dose is usually accepted and often even needed to enable administration of specific formulations. Therefore, we performed a secondary analysis addressing the clinical relevance of the error. In this analysis, a calculation error was defined as a ≥ 10% deviation from the correct outcome.

The primary and secondary outcomes were analysed using generalized estimated equations (GEE) logistic regression analysis (i.e. a GEE model with a binomial error distribution and a logit link) to account for missing data and within-subject correlations. The independent variables in the GEE model were the calculation item (to account for the difficulty of the calculation), the group (manual or calculator) and the interaction effect between the independent variables. The results are reported as (a) the estimated marginal mean probabilities of a calculation error and (b) the odds ratios (ORs) for the correct outcome obtained with the website-integrated calculator compared with that obtained with manual calculation. The estimated marginal mean probabilities are the predicted probabilities of a calculation error adjusted for the effects of covariates and missing data. Due to the presence of an interaction effect, the ORs of group (calculator versus manual calculation) vary by calculation item.

Demographic data were analysed using percentages for categorical data (profession) and median and interquartile ranges for age. For each clinically relevant error, we tried to reproduce the erroneous calculation outcome by manual recalculation, thus retrieving the cause of the error. All causes for error were described and scored using percentages for categorical data. Furthermore, the number of tenfold errors per group and per cause of error were evaluated. Statistical analysis on the cause of errors was not performed, due to the limited numbers per cause of error.

IBM SPSS version 21 was used for all analyses.

3 Results

3.1 Participants and Assessment

The participant groups were similar in age and profession (Table 1). Participants who did not report their profession were listed as profession ‘unknown’.

Table 1 Characteristics of the study population
Full size table

3.2 Reduction of Errors

The estimated mean difference in calculation error between the groups was 14.6% (95% CI 3.1–26.2; p = 0.013). In an analysis taking into account the clinical relevance of the error, the estimated mean difference decreased to 7% (95% CI 2.2–16.3; p = 0.137) (Table 2).

Table 2 Estimated marginal mean probability of a calculation error per group
Full size table

Due to the presence of an interaction effect, the ORs of the group (calculator versus manual calculation) varied by calculation item, thus representing the difficulty of the calculation item.

The OR for correct outcome when using the website-integrated dose calculator (instead of manual calculation) was statistically significant for eight items (items 1, 4, 8, 11, 12, 18, 19 and 20) (Table 3). These items may be labelled ‘difficult’ or error-prone calculations. Errors in items 1 and 19 were related to exceeding the maximum dose above a pre-specified weight. Items 4, 8, 12, 18 and 19 all required a conversion of a dose from milligrams to millilitres. In item 11, many participants in the manual group (27/31) entered the single dose instead of the requested daily dose. When corrected for clinical relevance (Table 3), the use of the website-integrated dose calculator was associated with significant ORs for items 1, 8, 11, 17 and 19 only. Item 17 (calculation of lactulose dose) shows a significant OR for correct calculation outcome in favour of manual calculation. Item 17 required participants to enter the outcome in milligrams while many participants in the website-integrated dose calculator group entered the outcome in grams, which was the unit of dosing provided by the website-integrated dose calculator.

Table 3 Odds ratios by calculation item for correct outcome using the calculator compared with manual calculation in a generalized estimated equations (GEE) model
Full size table

3.3 Qualitative Aspects of the Calculation Errors

Missing data comprised 12.9% of the calculations in the manual group and 18.4% of the calculations in the calculator group; these had been completed within the set time limit of 2 min. Eleven respondents in the manual group completed all 23 calculation items versus nine respondents in the calculator group (Table 4).

Table 4 Comparison of completion rate of calculations per group
Full size table

Causes of the erroneous calculations are presented in Table 5. Participants in both groups were likely to act by their clinical experience rather than instructions provided (‘incompliant with instructions’). For example, all respondents were instructed to always select the lowest dose of a dose range. For the amoxicillin case (Table 3, items 3 and 4), this would imply selection of 40 mg/kg/day out of the 40–90 mg/kg/day range. However, respondents tended to calculate the dose of amoxicillin based on the regularly used dose of 50 mg/kg/day. For some erroneous calculations, we could not retrieve the causes by manual recalculation (‘Calculation error not specified’). In line with the finding of significant ORs for calculation items, the percentage that exceeded the maximum dose in the calculator group was lower than that of the manual group (Table 5). Table 5 also confirms the number of errors in the calculator group originating from the use of the incorrect unit of dosing (item 17). The website-integrated dose calculator requires participants to select the indication and route of administration from a pre-specified list. Wrong selection from these drop-down lists (also known as juxtaposition error), typo/transcription errors and non-specified errors are more frequent in the calculator group, while exceeding the maximum dose and wrong correction for age are more frequent in the manual group.

Table 5 Comparison of rate of types of errors per group
Full size table

The percentage of tenfold errors in the calculator group was higher than that in the manual group (Table 4). Wrong transcription of the dosing unit—12 out of 19 errors in item 17—and other transcription errors accounted for the 77% of tenfold errors in the calculator group (Table 6).

Table 6 Comparison of rate of types of tenfold errors per group
Full size table

4 Discussion

Our data show that in a simulated setting the probability of a calculation error made by healthcare professionals is significantly lower when they use a website-integrated dosing calculator instead of a pocket calculator. The ORs for correct calculation suggest that the use of the website-integrated dose calculator is most effective in preventing the absolute maximum daily dose being exceeded and in converting a dose in milligrams to a dose in millilitres. The qualitative analysis, however, did not show a large reduction in the percentage of milligram to millilitre conversion errors with the use of the website-integrated dosing calculator. Therefore, the conversion step may not be the primary cause of erroneous outcome in these calculation items. Instead, the error is likely to be caused by calculation steps prior to the milligram to millilitre conversion. Significant ORs, indicating difficulty of the calculations, were found for common drugs such as paracetamol, ferrous fumarate and ranitidine.

Published error rates for incorrect dosing in children vary from 11.3% of all prescription errors (n = 391) in paediatric inpatients [15] to 36.5% of all prescription errors (n = 192, concerning dosages that do not fall within 25% of the recommended dose) by junior doctors completing a prescribing competency assessment [4]. Our results for manual calculation suggest that percentages for incorrect dosing are more likely to be on the higher end of this range. The high error rate found in our study may be the consequence of the strict definition for erroneous calculation: any deviation exceeding the correct outcome by 0.05 mg or 0.05 mL in both directions was considered an error. In clinical practice, a 10% deviation from the calculated dose is usually accepted and often even needed to enable administration of specific formulations. In the secondary analysis, in which we took into account a 10% or greater deviation from the correct answer, the estimated mean probability decreased from 27 to 19%, but the mean difference between groups was no longer significant (p = 0.137).

Although the probability of error rate was reduced from 39 to 24% (and 27 to 19% when accepting 10% deviation) we are surprised to find a still high number of errors with the use of the website-integrated dose calculator. The error analyses reveal that the nature of errors is different in both groups. The website-integrated dose calculator provides a good technical solution to prevent the absolute maximum dose being exceeded, incorrect milligram to millilitre conversions and wrong correction for age. At the same time, however, it introduces dosing errors based on juxtaposition (wrong selection from drop-down lists) and transcription, explaining the remaining error rate of 24% in the calculator group. These errors occur despite the programmed corrective and preventive actions aimed at detection of erroneous selection of parameters and incorrect data entry. The high number of tenfold errors in the calculator group is explained by dosing unit errors and typo/transcription errors, together accounting for 77% of all tenfold errors. The tenfold dosing unit error is likely to be caused by the design of item 17 of the assessment, where the calculator provided the dose in grams, but the assessment required the outcome in milligrams. Differences in dosing units between systems and the need for transcription are likely to occur outside a simulation setting as well and may lead to major dosing errors.

The participants in the manual calculation group completed the survey before the website-integrated dose calculator was available on the Formulary’s website. Participants in the calculator group, however, might have used manual calculations instead of using the website-integrated dose calculator. Four participants in the calculator group provided erroneous answers that by no means could have been generated with the website-integrated dose calculator considering the technical specifications (exceeding the maximum dose, multiplication of a fixed dose by weight, Table 5), even when instructions were not followed or incorrect parameters were selected. These four participants accounted for 52 of the 159 calculation errors in the calculator group (33%), which may imply a greater favourable effect of using the website-integrated dose than our results suggest.

Limitations of our assessment may consist of the deviations from daily practice, the need to transcribe calculation results, the limited time in addition to the need to switch between multiple computer displays and the lack of supervision during the assessment. Furthermore, the written instructions on use of the calculator as well as the instructions for the assessment did not ensure the correct use of the calculator in the simulated assessment. The simulation setting therefore may have induced errors that are less likely to occur when using the calculator congruent to everyday clinical practice. From September 2015 to June 2017, a beta version was made available, and users were asked to use it cautiously and report any problems. Errors like the ones encountered in the study were not reported during this period. Still, underreporting is a recognized limitation of spontaneous reporting systems. Currently, the website-integrated dose calculator is being used more than 30,000 times a month. Having received several reports on suspected problems with the calculator, none of the reports identified a malfunction of the calculator. Therefore, we have confidence in the safety of the calculator in everyday practice.

Although computerized dose calculation is advocated as a major approach to prevent paediatric calculation errors [1, 16, 17], our study shows that this technology does not completely rule out dosing errors and in fact may generate new types of errors. Several other studies have identified similar unintended consequences of the implementation of health information technologies [18,19,20,21,22]. Healthcare professionals should, therefore, use these technologies with due caution. Nonetheless, our findings confirm the findings of Kirk et al., that the computerized dose calculation can help reduce calculation errors [9]. A print option for the calculation was installed to enable calculation checks. Connecting the website-integrated dose calculator to computerized physician order entry systems may further reduce calculation errors caused by transcription.

5 Conclusion

Our study shows that a dose calculator as an add-on to a web-based paediatric formulary can reduce calculation errors, but it may introduce new errors based on transcription errors and the wrong selection of parameters from drop-down lists. Therefore, dosing calculators should be developed and used with special attention paid to selection and transcription errors.