Introduction

Inappropriate use of diagnostic imaging tests may result in unnecessary workup because of false-positive test results, unnecessary radiation exposure, inconvenience and discomfort to patients and increasing health-care costs. Results of studies in the USA and the Netherlands suggest that about 30–40% of patients do not receive health-care services according to present scientific evidence, and about 20–25% of health services provided is not necessary or is potentially harmful [1]. High-quality health care, however, implies a practice that is consistent with evidence-based medicine that provides a framework on which clinicians can base their practice [2, 3].

In diagnostic breast imaging, mammography is the standard imaging tool and can be followed by breast ultrasonography when clinically indicated. In the last 2 decades, the contribution of ultrasonography in the imaging process has improved. Evidence from prospective clinical studies shows that selective use of ultrasonography as an adjunct to mammography can enhance the imaging sensitivity and specificity of breast cancer diagnostics [4, 5]. Based on these studies, evidence-based guidelines were developed, according to which breast ultrasonography can be used more efficiently and appropriately. Practice according to these guidelines is expected to lead to a better accuracy of diagnostic breast imaging.

In order to encourage evidence-based practice and improve guideline adherence by radiologists, investments can be made in implementation interventions. In order to assess the need for such implementation activities, a good analysis of actual performance and the target group and setting is important. A measurement of current practice identifies where it does match and does not match the desired routines proposed by a guideline [6]. However, no empirical data were available yet on the variability of the application of breast ultrasonography in daily practice by radiologists in the Netherlands.

Therefore, the aim of this study was to determine the degree of adherence of radiologists with the proposed guidelines in daily clinical practice. Furthermore, the consequences of deviating from the guidelines were studied in terms of clinical outcome measures.

Patients and methods

Study population

A retrospective multicenter study was performed in two university hospitals (Maastricht University Hospital, Leiden University Medical Center) and four teaching hospitals (Atrium Medical Center Heerlen, Viecuri Medical Center Venlo, Catharina Hospital Tilburg and Sint Elisabeth Hospital Eindhoven) with a special interest in breast cancer diagnostics. The medical ethics committee of each hospital approved the study. In order to take the privacy of these hospitals into account, the data in this study were coded and reported anonymously.

All hospitals perform mammography and have directly accessible ultrasonography equipment.

All patients referred for mammography to the radiology departments of the participating hospitals during 4 months in 2004 were eligible for the study. The months May, June, October and November were selected to represent common clinical practice allowing for possible seasonal effects. Patients were included in the study if they were female, older than 30 years of age and if the first imaging technique applied was mammography. Patients were excluded from the study if they had a prior history of breast cancer, had breast implants or if the reason for referral was not mentioned or fell beyond the scope of the guidelines. If a patient underwent mammography more then once during the study period, additional mammographic examinations were included as new cases.

Guidelines

Current practice was compared to recommended practice from guidelines. The following set of guidelines for the use of breast ultrasonography was based on scientific evidence from the literature [4, 5, 7], Dutch national guidelines for breast cancer screening and diagnosis [8], practice guidelines from the American College of Radiology [9] and expert opinion of breast radiologists in the participating hospitals in this study:

Breast ultrasonography is recommended as an adjunct to mammography and clinical breast examination in:

  1. 1.

    patients referred for a palpable breast mass (guideline 1);

  2. 2.

    patients referred with an abnormal screening mammogram from the national breast cancer screening program (guideline 2);

  3. 3.

    patients with an abnormal diagnostic mammogram, defined as a mammographic new or increased mass, architectural distortion or asymmetric density that requires further diagnostic evaluation (guideline 3);

Breast ultrasonography is not recommended as an adjunct to mammography and clinical breast examination in:

  1. 4.

    patients referred for follow-up of a (probably) benign breast abnormality (guideline 4);

  2. 5.

    patients referred for symptomatic complaints, such as pain, skin or nipple abnormalities (guideline 5);

  3. 6.

    patients referred for positive family history or anxiety for breast cancer (guideline 6).

The basic principle of these guidelines was the reason for referral for mammography (guidelines 1–2–4–5–6). In patients with no indication for breast ultrasonography based on their reason for referral (guidelines 4–5–6), the findings at the initial diagnostic mammography could still provide this indication (guideline 3). A flow chart of the procedures and guidelines is shown in Fig. 1.

Fig. 1
figure 1

Guidelines

Data collection

Medical records of all eligible patients were collected and analyzed by the principal investigator (LV). For all patients meeting the inclusion criteria the subsequent items were scored: age, reason for referral, detected abnormalities at mammography, imaging techniques used, imaging conclusion (of mammography alone or combined with ultrasonography when available) and the BI-RADS classification. The BI-RADS lexicon is a standardized terminology in mammography and ultrasonography reporting and uses a grading scale of increasing suspicion for breast malignancy [10]. A final diagnostic conclusion was defined positive if a BI-RADS classification score of 3, 4 or 5 was ascribed, a follow-up imaging examination was recommended or fine-needle aspiration cytology or core-needle biopsy was indicated.

The gold standard for the presence or absence of breast cancer was determined by pathology results from histological breast tissue examination from biopsies or surgical excisions after a follow-up of 12 months. Pathology data were retrieved from the nationwide network and registry of histo- and cytopathology in the Netherlands (PALGA), to which all Dutch hospital pathology departments are linked. Breast cancer status was considered negative when no pathologic condition of the breast was reported in the PALGA system.

Outcome measures

Adherence rates

The degree of adherence to the guidelines was reflected by the percentage of patients for whom the radiologists were complying with the proposed guidelines. The adherence rate was calculated by dividing the number of patients referred for breast imaging who were treated by the radiologist according to the guidelines by the total number of patients referred for breast imaging. Next to the overall adherence rate per hospital, adherence rates were calculated for guidelines 1–2–3 (ultrasonography is recommended), guidelines 4–5–6 (ultrasonography is not recommended), per guideline per hospital and per radiologist (performing at least 50 mammograms).

Clinical outcome

The following clinical outcome measures were calculated by linking the diagnostic imaging conclusions with the pathology data: numbers of true-positive diagnostic conclusions, true-negative diagnostic conclusions, false-positive diagnostic conclusions and false-negative diagnostic conclusions. All false-negative and false-positive imaging results were reviewed, and the nature of the diagnostic conclusions was classified [11].

Furthermore, breast cancer prevalence rates, imaging sensitivity and specificity as well as the positive predictive value (PPV), negative predictive value (NPV) and diagnostic odds ratios (DOR’s) were reported. The DOR combines the sensitivity and specificity into one estimate and can be derived by dividing the odds of a positive test result among diseased persons by the odds of a positive test result among non-diseased persons ((sensitivity/(1-sens)) * (specificity/(1-spec))) [12].

Analysis

Pearson’s correlation coefficients were calculated to analyze the strength of the linear relationship between the following variables: adherence rate for guidelines 1–2–3 and adherence rate for guidelines 4–5–6 at the radiologist level; adherence rate for guidelines 1–2–3 and guidelines 4–5–6 and the corresponding DORs at the hospital level. Furthermore, the influence of patient age and breast cancer prevalence on the adherence rates was studied by regression analysis. Statistical analyses were performed using the software package SPSS 13.0.

Results

General

Of the 9,172 patients referred for mammography and assessed for eligibility, 4,915 patients were excluded because of a prior history of breast cancer (n = 2,948), a reason for referral beyond the scope of the guidelines (n = 769), ultrasonography being the first or only imaging technique (n = 616), an incomplete patient record (n = 331) or age younger than 30 years (n = 251). Consequently, 4,257 patients were included in the study.

The number of included patients per center in the study period varied between 522 and 1,117 patients (mean, n = 710). The mean age of the total study population was 50 years and varied between 48–52 years among the hospitals.

Adherence rates

Overall, breast ultrasonography was applied as an adjunct to mammography in 2,272 patients (53%). The distribution of the study population over the six guidelines and the corresponding adherence rates are demonstrated in Table 1. Of all patients, 33% was referred for a palpable breast mass (n = 1,408), 30% was referred for a family history or anxiety for breast cancer (n = 1,245), 17% was referred because of symptomatic breast complaints (n = 725), 9% was referred for the follow-up of a known benign breast abnormality (n = 362) and 7% was referred from the national breast cancer screening program (n = 305).

Table 1 Adherence rates per hospital per guideline (GL) and per set of guidelines (GL1–3: ultrasonography is recommended and GL4–6: ultrasonography is not recommended)

Furthermore, in 212 patients (5%) the indication for breast ultrasonography was caused by an abnormal prior diagnostic mammography, specified as guideline 3. The reason for referral of these patients was a family history or anxiety for breast cancer in 79 cases (37%), the follow-up of a known benign breast abnormality in 57 patients (27%), pain in 46 cases (22%) and nipple or skin abnormalities in 30 patients (14%). Patients with abnormal mammograms who were referred with a palpable breast mass or an abnormal screening mammogram were not included in guideline 3, but fell under guidelines 1 and 2, respectively (Fig. 1).

High adherence rates (74%-100%, mean: 95%) were observed for the different guidelines that advise ultrasonography (guidelines 1–2–3), whereas lower adherence rates (44%–99%, mean: 81%) were found for the different guidelines that do advise against additional ultrasonography (guidelines 4–5–6). A negative correlation indicates a mutual dependence within the guidelines, as adhering well on guidelines 1–2–3 implicates lower adherence on guidelines 4–5–6; Pearson correlation coefficient was -0.57; p = 0.001.

Furthermore, the overall guideline adherence rates per hospital varied between 79% and 92% (mean: 87%). The adherence rates for the different participating radiologists within the hospitals were relatively consistent; the differences were not statistically significant (results not shown).

In order to study the influence of patient age and breast cancer prevalence on the adherence rates, regression analysis was performed. Results show that age was a statistically significant predictor of the adherence rate for both guidelines 1–2–3 (β = −0.001, p = 0.03) and guidelines 4–5–6 (β = 0.002, p = 0.017). Breast cancer prevalence was also a predictor of the adherence rates for guidelines 1–2–3 (β = 0.82, p = 0.15) and for guidelines 4–5–6 (β = −5.59, p = 0.5), although its influence was not statistically significant.

Clinical outcome

The clinical outcome measures of this study are reported in Table 2. Overall, breast cancer was detected in 323 patients (7.6%); 307 cases were found under guidelines 1–2–3 and 16 cases were found under guidelines 4–5–6.

Table 2 Clinical outcome measures

For the patients in guidelines 1–2–3, the breast imaging sensitivity ranged from 95–100% (mean = 98%), and specificity ranged from 71–95% (mean = 88%). Furthermore, for the patients in guidelines 4–5–6, the breast imaging sensitivity ranged from 20–50% (mean = 25%) and specificity ranged from 93–100% (mean = 98%). Combining the sensitivity and specificity values in a DOR showed higher logDOR values for the imaging results in guidelines 1–2–3 (range = 4.2–8.0; mean = 6.1) compared to guidelines 4–5–6 (range = 1.5–5.2; mean = 2.9). Similarly, the PPV ranged from 56–81% (mean = 60%) in guidelines 1–2–3 compared to 3–60% (mean = 9%) in guidelines 4–5–6. The NPV ranged from 99–100% for both groups of guidelines.

Among the participating hospitals the overall sensitivity and specificity of diagnostic breast imaging varied between 91–97% and 84–98%, respectively. Local breast cancer prevalence rates ranged from 5.0% to 9.9%.

The relationship between guideline adherence rates and clinical outcome was studied by correlation coefficients. The hospital adherence rate for guidelines 1–2–3 was positively correlated with logDOR, but this was not statistically significant (Pearson correlation coefficient = 0.42; p = 0.40). On the other hand, the adherence rate for guidelines 4–5–6 was negatively correlated with logDOR (Pearson correlation coefficient = −0.69; p = 0.13). Furthermore, it was shown that the hospitals with the lowest guideline adherence rate for guidelines 4–5–6 had the highest corresponding logDOR (see hospital B and F in Table 2).

Overall, in 4,257 patients 17 false-negative (0.4%) and 240 false-positive (5.6%) imaging conclusions were found. In Tables 3 and 4 the characteristics of these cases are described.

Table 3 False-negative imaging results (n = 17)
Table 4 False-positive imaging results (n = 240)

In 11 cases of the false-negative imaging conclusions, the mammographic signs of malignancy were overlooked or misinterpreted, whereas in the other 6 cases the malignancy was undetectable, occult or had non-specific signs of abnormality. The majority (82%) of the false-positive imaging results were due to misinterpretation of the lesion, an asymmetric density or an architectural distortion.

In 95 patients, radiologists failed to adhere to guidelines 1–2–3 (breast ultrasonography was indicated but not performed), but this course of action did not lead to false-negative imaging results. In 442 patients, radiologists deviated from guidelines 4–5–6 (breast ultrasonography was not indicated but performed), which resulted in five additional false-positive results and one true-positive finding, which was classified as benign by mammography.

Discussion

This study demonstrated that in the two university hospitals and four teaching hospitals the application of additional breast ultrasonography in daily clinical practice corresponded well with the guidelines proposed. The overall guideline adherence was 87% and varied among the hospitals between 79% and 91%. The mean adherence with guidelines 1–2–3, which recommend additional ultrasonography in patients referred for palpable breast masses and abnormal screening and diagnostic mammograms, was 95%. Furthermore, the mean adherence with guidelines 4–5–6, which do advise against breast ultrasonography in patients referred for other breast symptoms, follow-up of a known benign abnormality, a family history or anxiety of breast cancer, was 81%.

It is likely that a radiologist who is more reserved in applying breast ultrasonography shows relatively lower adherence rates for guidelines 1–2–3 and relatively higher adherence rates for guidelines 4–5–6 as a consequence. Furthermore, a radiologist who is more sensitive to defensive diagnostic imaging will apply ultrasonography more often and will show higher adherence rates for guidelines 1–2–3 and lower adherence rates for guidelines 4–5–6. This mutual dependence within the guidelines was illustrated by a negative correlation between the adherence rates of the two subgroups of guidelines.

Although no scientific literature is available on benchmark adherence rates of radiologists with guidelines on diagnostic breast imaging, the results of this study can be compared to studies on other physicians’ adherence rates with clinical guidelines. Burstin et al. [13] reported an adherence rate of 60.4% with different process-of-care guidelines in emergency medicine. Furthermore, in a meta-analysis on 23 studies, Grilli and Lomas [14] reported a mean adherence rate of 54.5% with 147 clinical recommendations. Another study in the Netherlands on 70 evidence-based guidelines in family medicine found a mean adherence rate of 67% (range: 34%–100%) [15]. It is obvious that the adherence rates found in the current study were much higher than in the studies reported.

The overall breast cancer prevalence (7.6%) found in this study was comparable to the prevalence reported by Duijm et al. (5%), Flobbe et al. (6%) and Zonderland et al. (7%) [4, 5, 16]. There was a substantial variation in prevalence among the hospitals, with lower prevalences found in hospitals C and D. Although the distribution of patients over the different guidelines was comparable for all hospitals, the difference in prevalence could be explained by a variation in the composition of the study population. In hospital C, the proportion of malignancies was relatively low among patients referred for a palpable mass (6.9% versus 11.8% on average) and patients with an abnormal screening mammogram (21.3% versus 38.5% on average). In hospital D, the proportion of malignancies for patients with an abnormal diagnostic mammogram (guideline 3) was low (3.8% versus 14.8% on average).

Although the correlation between the adherence rates for guidelines 1–2–3 and guidelines 4–5–6 with the corresponding breast cancer prevalence was not statistically significant, the difference between both regression coefficients (+0.82 and -5.59, respectively) does indicate that more frequent use of breast ultrasonography is related to a higher breast cancer prevalence.

The sensitivity of breast imaging varied among the hospitals between 91% and 97% (mean: 95%), and the specificity varied between 84% and 98% (mean: 94%). These rates are comparable to the imaging accuracy found by Flobbe et al., Zonderland et al. and Duijm et al. [5, 16, 17].

Overall, breast ultrasonography was applied as an adjunct to mammography in 53% of all patients included. Based on the guidelines in this study, its performance was recommended in 43% of this patient population (1,925/4,257). It should be noted, however, that, as evidence-based guidelines are meant to direct professionals in their decision-making rather than to prescribe compulsory actions, an aim of 100% guideline adherence would be unrealistic and undesirable.

Non-adherence with the guidelines that recommend breast ultrasonography is expected to lead to an increase in the number of missed breast cancer diagnoses, or false-negative imaging results. Furthermore, non-adherence with the guidelines that advise against ultrasonography would have an effect on the number of false-positive results. In this patient population, breast ultrasonography was underused in 95 patients, but this did not result in false-negative results. Furthermore, breast ultrasonography was overused in 442 patients, which resulted in five false-positive and one additional true-positive result. These numbers are low compared to other studies and could be partly explained by a difference in study design and methodology [4, 18].

Although ultrasonography is a relatively cheap and accessible imaging test and its overuse did not lead to many false-positive results here, there is a concern that the radiologists with lower adherence to guidelines 4–5–6 would induce a general overuse of imaging tests due to defensive diagnostics. This could lead to large numbers of unnecessary time- and labor-intensive and costly radiological procedures. For example, the pursuit of false-positive CT findings in lungs is at best costly, anxiety producing and involves 2 years of repeated CT scans. At worst, it will lead to painful, costly and potentially risky major surgical procedures [19].

It was assumed that the guideline adherence rate was positively correlated with the breast imaging performance. This was confirmed for guidelines 1–2–3; for guidelines 4–5–6, a negative correlation was found between adherence and the DOR. These results suggest that performing more breast ultrasonography would generally lead to better imaging performance. This finding confirms prior study results by Flobbe et al., who report the sensitivity increasing from 91.5% to 96.9% and the specificity increasing from 87% to 94.8% by applying breast ultrasonography in all patients referred for mammography. In clinical practice, however, breast ultrasonography should be restricted mainly to the patient groups in guidelines 1–2–3, which will lead to a more efficient, time- and labor-saving and cost-effective use of the imaging test [4, 5, 7].

Furthermore, results showed a negative relationship between age and guideline adherence for guidelines 1–2–3 and a positive relationship between age and guideline adherence for guidelines 4–5–6, indicating that the use of ultrasonography decreases by age. Several studies have reported lower sensitivity for mammography at younger age [20, 21] and a related higher additional value for ultrasonography [22]. Furthermore, the effectiveness of whole breast ultrasonography in women with dense breast tissue and negative mammograms [2325] may have favored the performance of ultrasonography in younger women, since breast density declines by age.

A possible methodological limitation of this study is the retrospective nature of data collection. The lacking of abnormalities in mammogram reports could either be because they were actually not existing or because they were not registered. Furthermore, some breast findings were excluded from analysis because of missing data (e.g., breast density scores and comparison with previous mammograms). A prospective study design with standardized data collection would prevent this study bias. Secondly, a selection bias may occur since only hospitals with a special interest in breast cancer diagnostics were involved. Inclusion of other hospitals could possibly lead to a lower level of guideline adherence.

In this study the measurement of current practice was performed in order to assess the need for guideline implementation activities, by which evidence-based practice could be encouraged. The high adherence rates, however, indicate that these specific guidelines in this setting are less suitable for investments by guideline implementation activities. However, publication of these study results leads to dissemination of the guidelines among radiologists and other relevant target groups, which can be considered as a guideline implementation strategy by itself. An increased awareness of radiologists of current practice and better knowledge of the guidelines could then lead to a further enhancement of the adherence rates.

Nevertheless, this study has revealed new and valuable information about current radiology practice in relation to the proposed guidelines. The results show that the guidelines for appropriate use of breast ultrasonography that have been developed seem workable and feasible in clinical practice. It can be concluded that current daily practice of diagnostic breast imaging corresponded to a great extent to these guidelines proposed.