Background

Emergency Departments (EDs) worldwide are confronted with overcrowding due to physician shortage and a steady growth of patients visiting the ED [1, 2]. Long waiting times in the ED lead to unsatisfied patients. Moreover, crowding causes late diagnoses with the possibility of unnecessary complications. To cope with these problems, Advanced Nurse Practitioners (ANPs) and Physician Assistants (PAs) were introduced to take over certain diagnostic tasks from doctors to alleviate the physicians of some of the workload in the ED [3]. These so-called midlevel practitioners have proven to be of good value in the everyday practice of the ED by treating certain injuries and conditions while maintaining a good standard of care and patient satisfaction [4–6].

However, there are also some disadvantages to the development: training is relatively long and expensive and trainees are withdrawn from the already tight nurses pool [7, 8]. To resolve these disadvantages and maintain the benefits of these practitioners, we conceived a new concept: the Specialized Emergency Nurse (SEN). Regular emergency nurses received an injury-specific course to assess and treat minor injuries themselves according to a protocol. SENs are trained to be flexible employees capable of treating common injuries alongside their regular nursing duties.

The first injury to test this method on was the ankle sprain, because it is a common injury for which validated clinical decision rules exist that could easily be implemented in an algorithm (Ottawa Ankle and Foot Rules) [9–13].

Before this study, the ability of SENs to clinically assess ankle and foot injuries was compared with junior emergency physicians (house officers [HOs]) in an interobserver agreement study [14]. Furthermore, the accuracy of SENs in interpreting the accompanying radiographs was studied [15]. The results of both studies were promising to such a degree that a randomized controlled trial, the SEN-trial, was set up to assess the ability and consequences of SENs diagnosing and treating patients with an ankle or foot injury compared with HOs. Outcome measures considered in the trial were accuracy of assessment, patient satisfaction, and waiting time. The clinical part of the study revealed comparable accuracy results between the SENs and HOs. The SEN group scored significantly better than the HO group on patient satisfaction and waiting times. These results are presented in a separate paper [16].

At present, no literature is available about the cost-effectiveness of deploying regular nurses to perform diagnostic and treatment tasks after a short, injury-specific course. ANPs and PAs, however, have been proven to be more expensive than HOs [17]. The results of the economic evaluation that was performed alongside the SEN trial are presented in the current paper.

Methods

Study design

An economic evaluation was conducted alongside a prospective randomized controlled trial (SEN-trial) comparing House Officers (HOs) and Specialized Emergency Nurses (SENs) in their assessment of ankle and foot injuries in the ED. Assessment was performed by both groups according to a protocol based on the Ottawa Ankle and Foot rules. Effectiveness and cost-effectiveness were investigated. The protocol was approved by the scientific committee and medical ethics committee of the VU University Medical Center and was carried out in accordance with the Declaration of Helsinki (1989) of the World Medical Association.

Observer groups

Out of 32 certified emergency nurses, 16 volunteered to participate in this trial. The mean age of the nurses was 36 years (range, 26–56 years). The mean clinical experience in the ED for the nurses was 5 years (range, 6 months–12 years).

Before the study started, the nurses were trained in the anatomy and biomechanics (trauma mechanisms) of the ankle and foot, and were taught how to interpret the accompanying radiographs for the detection of acute fractures in a 2-day course, provided by a surgeon and a radiologist. Successful completion of the course led to the SEN qualification. Furthermore, all HOs, 24 in total, participated in the study. The HOs' mean age was 28 years (range, 26–30 years) and their mean clinical experience (in an ED) was 1 year (range, 6 months–1.5 years).

Study population

Patients were randomly assigned to 1 of the 2 treatment groups by computer, which allocated patients unstratified into the 2 observer groups in blocks of 20. All consecutive patients who came to the ED with an ankle or foot injury were invited to enter the study. Exclusion criteria were: age younger than 18 years or older than 65 years; trauma sustained more than 48 hours before presentation; mental or physical conditions known to complicate assessment of the injury; ankle/foot injuries as part of a more severe (poly)trauma; and prior injury to the ipsilateral ankle/foot that required surgery. Written informed consent was obtained from all patients. For more details on the trial, we refer to the clinical paper [16].

Clinical outcome measures

The included patients visited the outpatient clinic again after one week. One surgeon reassessed all patients to establish the definitive diagnosis (gold standard). The primary outcome measures were the accuracy parameters calculated as the sensitivity and specificity of both observer groups. These parameters were found by comparing the results of both observer groups to the gold standard (surgeon that reviewed the radiographs and reassessed ankle stability). The secondary outcome measure was patient satisfaction. The clinical outcome measures and their assessment are extensively described in the clinical paper [16].

Assessment of resource utilization

Data regarding costs directly related to the treatment of ankle and foot injuries were collected for each patient in the trial. The following data were taken into account: number of X-rays performed; use of crutches and duration of use; initial treatment (cast, pressure bandage or tubigrip); admittance to the hospital and duration of hospitalization; operation together with the used materials and duration of operation. Most cost parameters were collected at initial presentation at the ED. Few cost parameters (crutch use and hospitalization elsewhere) were collected at the control visit or by phone (for patients who did not return for the control visit). Data on crutch use were collected for the last 115 patients of the study. These data were used to estimate group means that were consequently imputed for the entire group. Other costs within the health care sector consisted of those associated with setting up the 2-day training course as well as providing it. Furthermore, the costs associated with the time SENs and HOs spent on patients in both groups were included. This was done by prospectively measuring the time spent on clinically assessing the patient as well as the time spent on interpreting the accompanying radiograph (if made). For practical reasons, these data were collected for a selection of patients and imputed for both groups.

The time loss patients experienced during the ED visit (indirect cost of productivity loss) was also taken into account. Data were prospectively collected concerning the time spent in the waiting room of the ED; time between clinical assessment and radiograph (if made), and time spent waiting between hearing the diagnosis and final treatment. The time spent in the waiting room was collected for all patients. The waiting time between clinical assessment and radiograph, and time spent waiting between hearing the diagnosis and final treatment in the ED were, because of practical reasons, collected in a random selection of patients (n = 18). The mean of these data were consequently imputed per group for the rest of the study population (Table 1).

Table 1 Mean (SD) invested time spans per treatment group
Full size table

Valuation of health care consumption; unit costs

The economic evaluation was conducted from a societal perspective. The study was carried out from August 2004 to March 2005. Therefore, 2004 prices were used. For the most relevant cost items, cost prices for the VU University Medical Center were calculated. These cost prices reflect costs of real resource use and include overhead costs. Items with a negligible influence (because of low average numbers) were valued according to the Handbook for economic evaluation in the Netherlands [18]. Table 2 shows the unit cost prices that were used. Waiting time was valued using shadow prices that value time spent at unpaid work or informal care. This shadow price equals the tariff for hiring someone, which was € 8.30 an hour.

Table 2 Costs per unit health care resource used in the economic evaluation of SEN (year 2004)
Full size table

Statistical analysis

Statistical analysis was carried out according to the intention-to-treat principle. A total of 512 patients were included in the trial: 263 were randomized to the SEN group and 249 to the HO group. The total costs were computed by multiplying resource data by cost prices. The difference between total costs in the SEN group and the HO group were consequently computed and the 95% confidence interval (95% CI) for this difference was calculated. As cost data are typically skewed, confidence intervals for cost differences cannot be estimated with conventional methods that assume normality. To avoid distributional assumptions, we applied the non-parametric bootstrap [19, 20]. Basically, in the non-parametric bootstrap, samples of the same size as the original dataset are drawn by sampling with replacement from the observed data. These bootstrap samples can be used to estimate standard errors and confidence intervals. To obtain 95% confidence intervals for cost differences, we performed a non-parametric bootstrap with a 1000 replications [21].

For the cost-effectiveness analysis, the difference in total cost between the two treatment groups was compared with the difference in sensitivity and specificity. In other words, the increase (or decrease) in total costs per missed diagnosis (false positives and negatives) for the SEN group versus the HO group was calculated and constitutes the cost-effectiveness ratio.

Uncertainty around the cost-effectiveness ratios was estimated using the bias corrected and accelerated bootstrapping method (1000 replications) and presented on a cost-effectiveness plane [21, 22].

Results

Clinical outcomes

SENs are capable of assessing and treating patients with an ankle or foot trauma at least as accurate as HOs. The percentage of false positives and false negatives in the SEN group was 7.4% compared with 8.6% in the HO group. For false negatives alone, these percentages were 2.9% and 4.7%, respectively. These differences were not statistically significant. Moreover, SENs accomplished these results with better patient satisfaction results than did the HOs. Furthermore, waiting times were decreased for patients with this type of injury. Full details on the clinical outcomes are presented in a separate paper [16].

Resource use

Table 1 summarizes the time measurements. No significant differences were found between treatment groups with regard to the invested time of either SEN or HO. Also, as expected, the waiting time for patients between clinical assessment and radiograph (if made) were not different between groups. This waiting period depends on the pressure of activities at the radiology department and is in no way associated with the observer group. Finally, the time spent in the waiting room of the ED and the time spent between hearing the diagnosis and receiving final treatment (eg, application of pressure bandage) was significantly longer in the HO group.

Table 3 lists the mean utilization of health care resources for both treatment groups. No significant differences were seen between the two treatment groups for any of the health care resources measured (X-ray indication; use of crutches and duration of use; treatment modality (cast, pressure bandage or tubigrip); hospital admittance/duration of hospitalization; and duration of operation. However, the waiting time for first assessment by a treatment officer was significantly longer in the HO group.

Table 3 Mean (SD) healthcare utilisation per treatment group
Full size table

Costs

Table 4 shows the mean (standard deviation) costs for the two treatment groups. Direct health care costs were not significantly different between the two groups. These included the costs associated with educating the SENs. Costs outside the health care sector were also not significantly different between groups. As such, the total direct costs were similar in both treatment groups. A substantial part of the direct health care costs was attributable to hospitalization and operation. Note that the indication for hospitalization and operation is set by a supervising surgeon and is, therefore, independent of the treatment group. Furthermore, the mean time spent in the waiting room was significantly shorter in the SEN group compared with the HO group. However, in absolute values, these costs were very small in comparison to the total costs. The total costs were € 186 (SD € 623) per patient in the SEN group and € 153 (SD € 529) per patient in the HO group. The difference in total costs was € 33 (95% CI: – € 84 to € 155). This difference could be explained by the difference in hospital admittance/operation and training costs for the SENs, and was not statistically significant.

Table 4 Mean (SD) costs for treatment group for all patients
Full size table

Cost-effectiveness

Table 5 shows the total costs and effects. Table 6 displays the differences in total costs and the differences in effect together with the incremental cost-effectiveness ratios. The effect differences were 1.2% for the false negatives and positives combined and 1.8% for the false negatives alone. Considering the cost-difference between treatment groups of € 33, the incremental cost-effectiveness ratio for reduction of false negatives and positives was € 27 and the incremental cost-effectiveness ratio for reduction of false negatives alone was € 18. In other words, SEN cost an additional € 27 per false positive or negative that is avoided. Furthermore, SEN cost an additional € 18 per false negative (missed fracture/total collateral ligament rupture) that is avoided.

Table 5 Mean costs and effects by treatment group for all patients (missing data imputed)
Full size table
Table 6 Mean cost and effect differences between treatment groups and cost-effectiveness ratios for all patients (missing data imputed)
Full size table

Figure 1 shows the cost-effectiveness plane for true positives gained. 65 percent of the cost-effect pairs lie above the x-axis, the area where SEN deployment is associated with higher costs. Furthermore, 85 percent of cost-effect pairs lie right to the y-axis, the area where SEN deployment is associated with more effect. Note that the differences are small and the spreading is close to zero.

Figure 1
figure 1

Cost-effectiveness plane. Cost-effect pairs are displayed as dots (coordinates) in the grid. On the Y-axis, incremental costs are displayed; on the X-axis, the difference in sensitivity is displayed between SENs and physicians.

Full size image

Discussion

Differences in total costs between the SEN group and HO group were not statistically significant. In daily practice, this means that we seem to have found an attainable solution to ED crowding that reduces waiting times, increases patient satisfaction and reduces workload for emergency physicians without increasing the costs. SEN cost only € 27 per false positive or negative that is avoided. The small difference in total costs in favour of the HO group is largely attributable to the slight (non-significant) difference in number of hospital admittances/operations, that accounts for larger costs in the SEN group. These costs are random since hospital admittance/operation is not a decision made by the observer, though is made by a supervising surgeon. Furthermore, the cost difference could partly be attributed to the costs of educating SENs. These costs are overestimated, since most of these costs were made for developing the educational programme and are not needed when a second group of nurses is trained. In addition, once a nurse has received training, the more patients he or she treats, the lower are the additional costs per patient.

Only few earlier publications on the costs and cost-effectiveness of alternative solutions to the problem of ED crowding are available. In the case of ANPs/PAs, one important publication revealed that ANP/PA delivered care was associated with higher costs than the standard care provided [17]. In light of these findings by Sakr et al., it seems that the SEN concept might constitute an attractive alternative to the currently available midlevel practitioners. SENs provide the benefits of deploying nurses to do a physicians' job as do ANPs/PAs: shorter waiting periods, decreased workload for emergency physicians and increased patient satisfaction while maintaining excellent diagnostic accuracy [16]. However, the costs associated with deploying SENs seem to be lower than those associated with the deployment of ANPs/PAs. The latter are more expensive because of the higher wages and costly educational programmes involved. Furthermore, it is important to note that the same results would have been seen when HOs/Emergency Physicians would be more expensive. This is due to the fact that the costs associated with income of the observer constitute only a small part of the total costs and therefore are irrelevant to the overall costs and consequently the cost difference.

Strengths and limitations of the study

For a critical appraisal of the design of the study, we refer to the clinical paper [16]. With respect to the collection of information on utilization of some resources and (waiting/assessment) times, some limitations should be mentioned. Most resources were collected for all patients. However, as for crutch use, these data were collected for the last 115 patients and imputed for the rest of the groups. With regard to invested times by patients, the overall time spent in the ED as well as time spent in the waiting room were collected for all patients. However, waiting times between hearing the diagnosis and final treatment were collected for a limited number (N = 18) of patients per group and then imputed for both observer groups. As for the invested assessment times by the observers, these times were all recorded in a limited number of patients (N = 18) and then imputed for both observer groups. When imputing the mean values from the mentioned data, the variance was ignored. Obviously, it would have been better if we had collected these data for all patients. However, the sensitivity analysis performed later, resulted in similar results and therefore, the method used seems justified. Furthermore, since the effect is an underestimation of the variance, an increase in the variance will only affirm the observation that cost differences between the groups are not significant and therefore will not influence the conclusions drawn.

For the remaining resource variables, very few data were missing and considering the large sample size, estimates are considered to be accurate. With over 500 patients being included, the study is well-powered. Finally, all data were collected prospectively, contributing to the strength with which conclusions can be drawn from the presented results.

Conclusion

Considering the results of this study and keeping in mind the results of the accompanying clinical trial, costs and effects are equal for both the SEN group and the HO group. SENs cost only € 27 per false positive or negative and only € 18 per false negative that is avoided. Considering the benefits of the SEN-concept in terms of decreased workload for the physicians working in the ED, the increased patient satisfaction and shorter waiting times in the SEN group, SENs appear to be an attractive solution to the problem of ED crowding.