Background

Each year some 3.6 million women in England have a cervical smear taken following an invitation from the national screening programme[1]. The proportion of these that do not contain material suitable for analysis and, are therefore deemed inadequate[1], has risen from around 6% in the early 1990s to 9.7% in 2000–1[1]. Women who have an inadequate smear must be retested, in line with national guidelines, and women who have three successive inadequate smears are referred for colposcopy[2]. Inadequate smears are therefore a source of distress to women, and a waste of resources in general practices, clinics and cytology laboratories[1].

There is wide variation in the proportion of inadequate smears between providers. This paper analyses this variation using Walter Shewhart's theory of variation which classifies variation as either emanating from a "common cause" or a "special cause" and gives guidance on the appropriate action required to address these two types of variation and so deliver continual improvement.

Common cause variation is expected variation attributable to "chance". It is part of every process and affects everyone in that process. To reduce common cause variation we need to fundamentally change the underlying process. In contrast, special cause variation is exceptional variation not attributable to "chance", but arising from special circumstances and therefore not affecting everyone in that process. Special cause variation can produce exceptionally good or bad results. To reduce unfavourable special cause variation we need to identify the special cause and act on it. In the case of favourable special cause variation, we likewise need to investigate and learn the lessons therein for the benefit of others.

Shewhart's methods have been widely used in industry [3, 4], in laboratory settings [5], and in communicable diseases control [6, 7]. The use of control charts has been suggested in clinical governance and health care performance monitoring [8, 9]. This paper reports an investigation of variation in the proportion of inadequate smears in South Staffordshire Health Authority using control charts.

Methods

Cervical cytology data routinely collected by the Health Authority from the laboratories serving the district was obtained for all 100 general practices in South Staffordshire. The data included the number of smears taken and the number inadequate for the calendar years 2000 and 2001.

We investigated two distinct sources of variation – variation by general practice and variation by cytology laboratory. We undertook these analyses using P-charts. P-charts are one member of the family of control charts and are designed to be used with binomial data (inadequate smear – yes/no) which is expressed as a proportion of the sample size. We sorted our data by the total number of smears (ie sample size) and plotted our P-charts with respect to this order. This has two advantages. Firstly the resulting P-charts show the impact of sample size on the control limits, and secondly the resulting control limits are easy on the eye because they appear like a funnel. Such plots have been advocated in health care [10].

National comparative data, which included the proportion of inadequate smears, the proportion of smears classified as mild dyskariosis, severe dyskariosis and the positive predictive value of a smear, was obtained for all 156 laboratories in England for the years 2000–2001[1]. The relationship between the proportion of smears classified as inadequate by a laboratory and the positive predictive value of a smear in that laboratory was investigated by linear regression analysis. The relationship between the proportion of smears classified as inadequate and the proportion of smears classified as negative (no dyskariosis), mild dyskariosis, moderate dyskariosis or severe dyskariosis was investigated by step-down multivariate regression. All analysis was carried out using SPSS 11.0. For the step-down analysis the proportion of inadequate smears was the dependent variable and the proportion categorised as negative, borderline, mild, moderate and severe dyskariosis were independent variables. The regression was repeated with non-significant variables eliminated in a stepwise fashion.

Results

Analysis of inadequate smears by general practice

Inter-general practice variation in the proportion of inadequate smears was explored using a P-chart for our 100 general practices (Figure 1). There is wide variation in the proportion of inadequate smears between general practices, with 23% of general practices showing evidence of special cause variation – 12% showing low special cause variation and 11% showing high special cause variation. However, the majority of practices (77%) are consistent with common cause variation.

Figure 1
figure 1

Control chart of the proportion of inadequate smears across the 100 general practices in the health authority.

Full size image

Analysis of inadequate smears by cytology laboratory

We also explored the inter-laboratory variation in the proportion of inadequate smears. The results are plotted on a P-chart (Figure 2). Laboratories 1 and 2 show evidence of special cause variation with exceptionally high proportions of inadequate smears. Laboratories 3 and 4 show special cause variation with very low proportions of inadequate smears.

Figure 2
figure 2

Control chart of the proportion of inadequate smears across laboratories (1–6) in the health authority.

Full size image

Analysis of inadequate smears by general practice and cytology laboratory

To provide further insight into the interaction between the two sources of variation (general practice and cytology laboratory), we analysed the proportion of inadequate for each general practice by cytology laboratories. We produced six P-charts one for each laboratory (Figure 3). The mean proportion of inadequate smears ranged from 12.7% (laboratory 1) to 6.7% (laboratory 6).

Figure 3
figure 3

Control chart of the proportion of inadequate smears across GP practices in the health authority sub-grouped by laboratory. Each panel shows the control chart for a laboratory (1–6 in panel header). For each cytology laboratory, the mean proportion of inadequate smears is shown by a dotted horizontal line with accompanying upper and lower control limits.

Full size image

Two of the six practices served by laboratory 1 show special cause variation. Six of the 40 practices served by laboratory 2 show special cause variation. One of the seven practices served by laboratory 3 shows special cause variation. Four of the 41 practices served by laboratory 4 show special cause variation. Practices served by laboratory 5 and by laboratory 6 show common cause variation.

Further analysis of inadequate smear rates

It not clear whether a high or low inadequate rate is preferable; either too many smears are being rejected unnecessarily or inadequate smears are being analysed inappropriately. To investigate this further data were obtained on the results of laboratories nationally[1].

No relationship was found between the proportion of smears reported as inadequate and the positive predictive value of a smear in all 156 English laboratories [P = NS]. The positive predictive value of a smear is the percentage of referrals following a smear that have grade 2 cervical intraepithelial neoplasia or worse[1]. The step-down multivariate regression found only the proportion of smears classified as negative to predict the proportion classified as inadequate (beta coefficient = -0.319; P < 0.001). However, this explained little of the variation in the rates reported as inadequate (R squared = 0.082), suggesting that laboratories classifying larger proportions of smears as inadequate do not report higher proportions of moderate or severe dyskaryosis.

Discussion

Our study shows that there is wide variation in the proportion of inadequate smears amongst our 100 general practices, with 23% showing evidence of special cause variation which merit further investigation to identify possible causes. However the vast majority of general practices (77%) are consistent with common cause variation, which, according to Shewhart's theory of variation, is best addressed by introducing fundamental changes to the underlying process.

The National Institute for Clinical Excellence (NICE) has recently recommended the adoption of liquid based cytology (LBC), in part because it is associated with significantly lower proportions of inadequate smears from 9% to 1.6% [11]. This constitutes a fundamental change in the cytology process. However, the inadequate smear rates reported were based on only three pilot site laboratories, the evaluation of which states that the long term reduction of the rate of inadequate smears cannot be assessed from the data reported[12]. The systematic review that informed the NICE decision also cites studies that show an overall mean inadequate rate of 1.4% (5% and 95% percentile 0.3% and 9.1%) for conventional cytology and 0.8% (5% and 95% percentile 0.1% and 5.5%) for LBC[13]. These studies also show great heterogeneity with either method and were not consistent with the current mean inadequate rate of 9% in NHS practice. This is reflected in NICE's statement that although LBC offers a decrease in the proportion of inadequate specimens "the literature reveals a wide and overlapping range in this proportion with both conventional smears and liquid-based methods."[11]

Although LBC technology is potentially associated with significant reductions in the proportion of inadequate smears, we cannot be confident that the impact of change will be readily discernable in general practices/laboratories because (a) it is not clear that the results of the pilot evaluations are generalisable to practices/laboratories with special cause variation, and (b) there is potential interaction between the new technology and the existing but as yet unidentified special cause variation. As a first step we would recommend that special causes at these general practices/laboratories be identified (and where appropriate eliminated) before introducing changes to the underlying process. The following guidance has been recommended [8] in investigating special cause variation – check the data, check the case-mix, check the process, check the resources and finally check the individuals involved. The full benefits of LBC will only be realised if its use in practice is optimised. Under the philosophy of continual improvement, control charts, even in the presence of very low inadequate rates, offer a way of doing this because a key feature of the control chart methodology is that "special cause" variation is, by definition, economic to find and remove[3].

Our approach differs from the current methods employed in the NHS to analyse variations in inadequate smear rates. At present, quality assurance targets are set from the 10th to the 90th percentiles of the distributions for laboratories nationally[1]. However, this fails to recognise that results from different laboratories cannot be assumed to be consistent with common cause variation, even in the 10th–90th percentile, as we have shown. In addition, a target based approach seems to focus attention on those laboratories who have "missed the target", implying that there is no "need" for "on target" laboratories to take action. In sharp contrast Shewhart's approach requires action for both special and common cause variation and so is able to support efforts for continual improvement.

We have undertaken a comparative analysis between general practices and laboratories. It would be very useful to investigate how the proportion of inadequate smears change over time (eg quarterly), for each laboratory and for each general practice. This longitudinal analysis would complement the cross-sectional analysis and provide a more complete picture of the variation in the proportion of inadequate smears.

Conclusion

Control charts offer an action-oriented way of analysing cervical smear data. By highlighting areas for further investigation they act as a tool for hypothesis generation about the causes of variation. In this way they direct us towards the most efficient action that will achieve quality improvement.

This study has shown that control charts can be used to analyse routine cervical data at a health authority level. Two distinct sources of variation in the inadequate smear data can be identified; that associated with laboratories and that associated with GP practices.