Introduction

Current recommendations by the American Academy of Orthopaedic Surgeons (AAOS) are to obtain a confirmatory test in patients for whom carpal tunnel surgery is being considered. The AAOS, however, does not specify which test should be used [17]. A recent meta-analysis [9] showed comparable sensitivity and specificity of ultrasound and electrodiagnostic testing (EDX) in the diagnosis of carpal tunnel syndrome (CTS). Despite this evidence, there is an anecdotal bias toward the preferential use of EDX over ultrasound to confirm clinical findings in the diagnosis of CTS. Changes in the healthcare environment have placed a high premium on the cost-effectiveness of diagnostic and therapeutic procedures. With more than 3.8 million Americans evaluated by a physician for CTS in 2003, the burden of diagnostic testing for this condition is immense [17].

EDX, defined as a combination of nerve conduction studies and electromyography, has been used as the gold standard for the diagnosis of CTS [2, 7, 9, 15, 24, 28, 32–34]. The American Association of Electrodiagnostic Medicine (AAEM) has defined a standard protocol for EDX to detect focal nerve conduction slowing and changes in muscle innervation in patients suspected of having CTS [15]. EDX has a high specificity [9, 25], but also a false-negative rate of 10% to 20% [14, 16, 19, 25].

The use of ultrasound for the diagnosis of CTS relies on measurement of the cross-sectional area of the median nerve at the inlet of the carpal tunnel, with many authors defining a positive test as a cross-sectional area greater than 9 mm2 (range, 9–15 mm2) [1, 2, 7, 8, 18, 20–23, 26–28, 31–35]. The reported sensitivity and specificity of ultrasound varies from 57% to 98% and 51% to 100%, respectively with a false negative rate of approximately 20% [9]. Ultrasound is operator-dependent and unable to differentiate distal from proximal compression [9, 24]. We found the charge for diagnostic ultrasound of the upper extremity for CTS was less than for EDX based on internal Medicare charges supplied by our billing department. We therefore presumed it may be more cost-effective for all patients with clinical findings of CTS, who are determined by their surgeon to require confirmatory testing, to undergo ultrasound as a first-line diagnostic test. In the case of a positive ultrasound, the diagnosis would be considered confirmed and treatment would follow the standard of care. In the case of a negative test, with clinical findings suggestive of CTS, EDX then would be performed to confirm or refute the results of the ultrasound.

We therefore asked whether ultrasound as a first-line diagnostic test is more cost-effective than using EDX alone or using ultrasound alone: (1) when used by a general practitioner; and (2) when used by a specialist.

Patients and Methods

A fictional population of 38,000 patients, based on data from the United States Department of Health and Human Services Health Care Cost and Utilization Project (HCUP) [31], being referred for confirmatory testing for presumed diagnosis of CTS was created and each patient was assigned a probability of having true positive, false positive, true negative, and true positive ultrasound and EDX tests over an expected range of sensitivity and specificity values using Monte Carlo methods. Monte Carlo methods are a class of computational algorithms that use repeated random sampling of an expected range of probabilities to compute results for situations such as the modeling of risk and cost overruns. These methods are beneficial in situations with many variables, such as our current model, and allows for the uncertainty seen in clinical practice. The likelihood of these outcomes and the sensitivity and specificity data were defined based on a recent meta-analysis [9] and the study by Graham [11]. Using this statistical method allows the creation of a fictional population that more accurately reflects clinical practice. For example, the sensitivity and specificity of any given test can vary over the 95% confidence intervals of the test. As every clinician has experienced, most patients do not experience results that exactly mirror the mean or median, but rather fall within a spectrum on either side. The fictional population of 38,000 patients was created one time, followed by the repeated random selection, over a uniform distribution, of a group of 100 patients from this population to simulate the clinical spectrum that might be seen in the office by a practitioner. The data from the repeated random samplings then was aggregated and confidence intervals calculated.

This Monte Carlo analysis relies heavily on pretest assumptions (Table 1), which are defined below. The prevalence of CTS in patients sent by general practitioners for confirmatory testing was set at 60%, as defined by Bland and Rudolfer [4]. A specialist is defined as a physician to whom patients with clinical suspicion of CTS are referred, by other physicians, for evaluation and management. The prevalence of CTS in patients sent by specialists for confirmatory testing was set at 89%, as defined by Szabo et al. [30] and Grundberg [13]. The sensitivity and specificity of ultrasound [9] were set at 80.2% and 78.7%, respectively, and the sensitivity and specificity of EDX [11] were set at 69% and 97%, respectively.

Table 1 Assumptions made for the cost-effectiveness analysis with mean and 95% confidence intervals, when available
Full size table

The charge of each diagnostic test was based on internal Medicare charge data provided by the billing department at one of our hospitals. Based on these data, ultrasound was assigned a charge of $204 per test and EDX was assigned a charge of $250 per test. A charge also was assigned for a false-positive diagnosis, based on loss of wages attributable to missed work. Cowan et al. [6] found that patients are expected to return to work at an average of 11.8 days for light duty and 18.9 days for full duty after carpal tunnel release. For the purposes of this analysis, the average time to return to work was chosen as a mean of 10 days (range, 1–80 days) and each patient was assigned a value using Monte Carlo methods. The Social Security Administration National Wage Index for 2010 was $41,673 per year [9, 29]. The salaries of the fictional patients in our group were varied randomly from $20,000 to $250,000 per year with the average set at $41,673. The charge for a false-negative test was set at half of the average salary per week to account for decreased productivity at work and the need for additional tests and physician office visits. The fictional patients were again assigned a value for the number of missed days with an average of 10 days (range, 1–50 days).

We used an algorithm for testing the use of ultrasound as a first-line test, with results confirmed or refuted by EDX in the case of a negative test (Fig. 1). The fictional patients with clinical findings suggestive of CTS, who were determined by their surgeon to require additional confirmatory testing, underwent ultrasound of the carpal tunnel. In the event of a negative test, the patient was referred for EDX to confirm or refute the negative ultrasound. In this clinical algorithm, a positive ultrasound resulted in no additional testing. A true-positive result, defined by clinical improvement after surgery, was assigned a charge of $204. A false-positive result, defined by lack of clinical improvement after surgery, was assigned a charge of $204 plus charge per week of missed work (as described above). A negative ultrasound was followed by EDX to confirm or refute the findings of the ultrasound. A true-negative ultrasound, defined by a concordant negative EDX, was assigned a charge of $204 plus $250 (total of $454). A false-negative ultrasound, defined by a positive EDX, was assigned a charge of $204 plus $250 plus charge of half salary per week of decreased productivity (as described above).

Fig. 1
figure 1

The flow diagram shows the algorithm using ultrasound as a first-line test, confirmed by EDX if the ultrasound was negative.

Full size image

In the EDX-alone scenario (EDX was used as the only test, no ultrasound tests were ordered), a true-positive test was assigned a charge of $250. A true-negative test was assigned a charge of $250. A false-positive test was assigned a charge of $250 plus charge per week of missed work (as described above). A false-negative test was assigned a charge of $250 plus half salary per week (as described above). In the ultrasound-alone scenario (EDX was not ordered to confirm or deny the results of ultrasound), a true-positive test was assigned a charge of $204. A true-negative test was assigned a charge of $204. A false-positive test was assigned a charge of $204 plus charge per week of missed work (as described above). A false-negative test was assigned a charge of $204 plus half salary per week (as described above).

Groups of 100 fictional patients from the fictional sample population of 38,000 were repeatedly chosen and the average charge per patient was calculated with 95% confidence intervals. The false-negative, false-positive, true-positive, and true-negative values were calculated with 95% confidence intervals. The simulated trials were generated using R 2.15.1 (R Foundation for Statistical Computing, Vienna Austria; www.r-project.org).

Results

In the general practitioner scenario, the use of ultrasound alone (Table 2) as the diagnostic test for CTS, resulted in an average charge of $476.30 per patient. The use of EDX alone, the most commonly used method in most practices, resulted in an average charge of $400.30 per patient. The difference in charges between the use of EDX alone and ultrasound alone was $76 per patient in favor of EDX. The average charge for the use of ultrasound as a first-line diagnostic test followed by EDX for confirmation of a negative ultrasound test was $562.90 per patient. The use of ultrasound alone had a false-positive rate of 8.5% and false-negative rate of 11.9%. The use of EDX alone had a false-positive rate of 1.4% and false-negative rate of 18.9%. The use of ultrasound as a first-line test, confirmed by EDX if needed, had a false-positive rate of 9.6% and false-negative rate of 3.7%.

Table 2 False-positive rate, false-negative rate, and mean charges in the generalist scenario*
Full size table

In the specialist scenario, the most cost-effective strategy (Table 3) was the use of ultrasound alone as the diagnostic test for CTS, with an average charge of $367.80 per patient. The use of EDX alone, the most commonly used method in most practices, resulted in an average charge of $428.30 per patient. The difference in charge between the use of EDX alone and ultrasound alone was $60.50 per patient in favor of ultrasound. The average charge for the use of ultrasound as a first-line diagnostic test followed by EDX for confirmation of a negative ultrasound test was $369.50 per patient, nearly identical to the use of ultrasound alone. The use of ultrasound alone had a false-positive rate of 2.5% and false-negative rate of 17.5%. The use of EDX alone had a false-positive rate of 0.3% and false-negative rate of 27.7%. The use of ultrasound as a first-line test, confirmed by EDX if needed, had a false-positive rate of 2.7% and false-negative rate of 5.5%.

Table 3 False-positive rate, false-negative rate, and mean charges in the specialist scenario*
Full size table

Discussion

Ultrasound has been proposed as an alternative test for the diagnosis of CTS as a result of its potential cost savings, time savings, and improved patient satisfaction resulting from less discomfort [3, 7]. Sensitivity and specificity of ultrasound, when using clinical diagnosis as the gold standard (77.3% and 92.8%, respectively), were comparable to the sensitivity and specificity of EDX (69% and 97%, respectively) [9, 11]. Despite this evidence, a bias still exists favoring the use of EDX for confirming the diagnosis of CTS. We therefore asked whether ultrasound as a first-line diagnostic test is more cost-effective than using EDX alone or using ultrasound alone: (1) when used by a general practitioner; and (2) when used by a specialist.

There are several limitations to this study. First, this analysis is a simulation based on randomly assigning probabilities of true-negative, false-negative, true-positive, and false-positive outcomes of the diagnostic tests over an expected range of values as previously reported [9, 11]. It may not accurately reflect true clinical practice based on the assumptions that we have made regarding the prevalence of CTS in the two clinical scenarios [4, 13]. The sensitivity and specificity of ultrasound used in our analysis were based on a reference standard of clinical diagnosis [9]. The use of sensitivity and specificity values based on EDX as the reference standard could change the findings. The assignment of charges to each outcome (true-positive, true-negative, false-positive, and true-positive) is another limitation of the study. The charges of the diagnostic tests were based on internal Medicare charges from one of our institutions. Medicare charges vary among institutions, however, the ratio between the charges is likely to be similar. The charge assigned to a false-positive test was meant to penalize a confirmatory test for incorrectly diagnosing CTS. The determination of a false-positive test in clinical practice may be difficult and hard to confirm, as successful surgery for CTS release is an imperfect measure because patient symptoms often are multifactorial and all symptoms may not improve with surgery [5, 10]. We did not attempt to quantify the legal and ethical costs of performing surgery for an incorrect diagnosis (false-positive test). The cost of missed work could vary greatly depending on the population of the surgeon and the salaries of the patients, although we have attempted to model for this effect. Patients with desk jobs would be more likely to return to work sooner than patients with manual labor jobs, although we controlled for this variable by varying the time to return to work over a range of 1 to 80 days, with a skew toward the left and earlier return to work at an average of 10 days. Another limitation is that in clinical practice, the clinician performing the confirmatory EDX may not be blinded to the results of the ultrasound, which could induce bias.

Although EDX alone was the most cost-effective strategy in the general practitioner population, the false-negative rate of nearly 18.9% is unacceptably high. Interestingly, the use of ultrasound as a first-line test confirmed by EDX, if necessary, resulted in a low false negative rate (3.7%). Given the lower rate of false positives and cost-savings, EDX remains the preferred test for general practitioners. A charge savings of $76 per patient becomes relevant as 3.8 million Americans are seen by a physician each year for evaluation of CTS [17]. The prevalence of patients, sent from the general practitioner’s office for confirmatory diagnostic testing, who had a true diagnosis of CTS possibly could be increased by standardized use of a clinical diagnostic tool as proposed by Graham et al. [11, 12].

In the specialist scenario, the false-negative rate of 17.5% for ultrasound alone is also unacceptably high, although not as high as the false negative rate of EDX alone. EDX alone had the lowest false positive rate of any scenario (0.3%). The use of ultrasound as a first-line test, followed by EDX in the case of a negative ultrasound, had the best combination of confirming the diagnosis (false-negative rate, 2.7%) while maintaining an acceptable rate of false-positive tests (5.5%). The use ultrasound as a first-line test in this scenario resulted in a cost savings of $1.70 per patient.

We found the use of diagnostic ultrasound as a first-line test for confirmation of a clinical diagnosis of CTS to be a cost-effective strategy in the hands of a specialist. Although first-line ultrasound was associated with improved false-negative rates over the use of EDX alone in the general practitioner model, it resulted in a substantial increase in charges over the use of EDX alone. Our data suggest ultrasound is a cost-effective option for confirming the clinical diagnosis of carpal tunnel syndrome in the specialist’s office.