Introduction

The Canadian National Breast Cancer Screening Study (CNBSS) is one of eight large-scale randomized controlled trials on mammography screening. Compared to the other studies, it is an outlier.

Following its first publication in 1992 [1, 2], the quality, randomization, and design of this study was highly debated in the scientific community. The CNBSS is an outlier among the eight usually cited randomized controlled trials (RCTs), including those with comparable follow-up [3], which showed an average mortality reduction for invited versus non-invited women aged 50–69 years of 23 % and for examined ages about 20 % [4–11].

It contradicts the results of today’s quality-assured mammography screening programmes and of the present Canadian screening programme, which report a mortality reduction ranging around 40 % for participating versus nonparticipating women [12–18].

However, it has been classified as one of two studies of high quality in Cochrane reviews, published by Goetzsche since 2000 [6, 19]. A 25-year follow-up of the CNBSS, with added data from two different studies, was published recently [20]. It claimed that mammography screening, performed annually for 5 years (from 1980 to 1985) in the age-group 40–59 years showed no benefit for breast cancer-specific mortality over clinical examination. Taking this outcome seriously, mammography screening should be stopped immediately.

Is this a reasonable conclusion for ongoing mammography screening programs? To answer this question, critical details of the CNBSS, which may have been lost, forgotten, or suppressed, are reconsidered. Another unasked question concerns its applicability to the present situation and to quality-assured screening programmes following European guidelines.

Materials and methods

A systematic search concerning the CNBSS was conducted, which was followed by thematic analysis of study findings and a narrative synthesis. The search included articles from 1981 to 2014 and was performed using PubMed and Embase databases. The following adapted MeSH-Terms were used: “CNBSS”, “Canadian National Breast Screening Study”, “NBSS”, and “Breast cancer”. The selected studies were limited to articles published in English. Papers, comments, editorials, letters, reports, and handbooks dealing with the subject were included.

A total of 240 articles (PubMed n = 129, Embase n = 111) were identified. Additionally, a manual search and cited reference search generated another 41 articles leading to a total of 281 articles, of which a title and abstract screening was executed by two authors independently. They identified those articles, which treated specific structural, procedural and evaluation issues of the CNBSS. One hundred and twenty-nine potentially relevant articles were saved for full text screening, while 53 articles were duplicates, 97 articles did not meet the inclusion criteria, and two articles were not available. These 129 articles were screened for the following topics: inclusion criteria, randomisation process, contamination of the control group, quality of mammograms, mammographic reading, and diagnostic chain. Forty-three articles were excluded, as they did not meet these inclusion criteria. Finally, 86 articles were selected for review (Fig. 1).

Fig. 1
figure 1

Flow diagram of systematic literature search

Full size image

Results

Sixteen of the 86 eligible articles mentioned > 1 of the debated issues, but argued neither for nor against the quality of the study. Thirty-seven articles of various authors, including 15 articles (co-)authored by five radiological reviewersFootnote 1 [21–29] (Kopans, Feig, Logan, Sickles, Tabar) one advisor (Moskowitz), and the responsible physicist [30] (Yaffe) published on “severe flaws” of the study. Thirty-three articles, of which 27 were (co-)authored by the two principal investigators, Miller or Baines, defended the study (Fig. 2). Two of the radiological reviewers resigned during the study.

Fig. 2
figure 2

This chart shows the number of publications which criticized or defended the CNBSS. With very few exceptions, only the main investigators defended the trial. Numerous authors criticized it. Seven of these authors were involved with review or quality assurance of the CNBSS. These authors are mentioned explicitly. Some published several articles concerning the CNBSS. Further publications mentioned some of the issues, but did not comment them. The list of references can be reviewed in the electronic supplementary material (ESM)

Full size image

The following issues were debated (Table 1):

Table 1 Issues concerning design, quality assurance, and evaluation of the CNBSS: all cited literature
Full size table

  • Randomisation: Randomisation was performed after a clinical breast examination (CBE) by the principal investigators at each site [28, 31, 32]. This contradicted the initial study design [33], but was documented in the handbook of operation [34] and by an external investigation [32]. While Baines states that “the center coordinators … were blind to the CBE”, other authors including external reviewers of the study reported the opposite [28, 34]. Also, the coordinator of one of the CNBSS sites was removed from her position because of suspected subversion of the randomisation process [31].

    An external investigation [32] for possible fraud stated that in 12 out of 15 sites “…the nurses and probably the coordinators were aware of the findings of the clinical examination when the allocation was made”. In addition, more alterations of the allocation book were found in the mammography screening arm than in the control group ( >100 unexplained alterations) [28, 31, 32, 34]. This investigation, which mainly concentrated on checking erasements of participants’ names in the allocation book, did “not uncover credible evidence of subversion” [32]. However, Boyd [31] and others (e.g. Kopans, Burhenne) pointed out that various other easy possibilities of subversion existed that could not be excluded with absent blinding.

    The external investigation stated, considering that “…referral would not have ensured mammography. The charge has been made that there remained a motive for the examiner or coordinator to subvert the randomisation, if for clinical or other reasons he or she believed that the subject should…have a mammogram” [32].

    Unfortunately, a severe imbalance of the distribution of advanced cancers (>4 involved lymph nodes) was noted among those women screened in the first round < age 50 years. Nineteen women with far advanced cancers (>4 involved lymph nodes) were allocated to the screening group versus five in the control group, and more women with prior breast cancer were reported in the screening group (n = 8 versus n = 1) [28, 31, 34–36].

    While Baines argues that more than 50 other variables (“…demographic and risk factors”) showed “virtually identical distribution across control and study groups”, Kopans pointed out that “….shifting a much smaller number of advanced cancers to the study group would substantially affect mortality…..without producing a demographic imbalance”.

    A bias of randomisation must also be suspected when comparing the hazard ratios of the mammography arm in the prevalence round versus subsequent rounds: 1.47 versus 0.9 [20].

    Goetzsche investigated none of the above issues of the randomisation process. Instead, he classified the CNBSS as one of two “high quality” RCTs using one formal criterion: he rated individual randomisation higher than cluster randomisation (where demographic regions are invited/not invited as opposed to individuals).

    Comparable to the Malmö study, which also used individual randomisation, a high proportion of cross-over was reported: 26 % of the women in the control group underwent mammography [37].

  • Inclusion criteria: Women with palpable tumours were included in the CNBSS [1, 2]. This fact alone calls for scrutinizing the inclusion criteria. In the mammography arm of the CNBSS 68 % of breast cancers were already palpable. Furthermore, women with previous breast cancers were included [28, 31], participants whose prognosis may have already be determined by the prior disease.

  • Quality of the screening procedure: Miller claimed that all equipment was new and that a quality assurance was in place [11, 6]. The responsible physicist stated [30], “quality was far below state of the art – even at that time (early 1980s). Problems … from inadequate equipment, … inappropriate imaging technique and lack of….specialized training….”. This was confirmed by several others including external reviewers [28, 29, 34], who specified that “training for mammography technologists or radiologists and a quality assurance were not present”. The study had initially been overseen by two highly renowned advisors (S. Feig and W. A. Logan). They resigned from the study after 3 years due to unacceptable quality. Subsequent external review rated image quality as acceptable in < 40 % of the cases during 1980-1984, in 1985 in about 60 %, and in 1986-7 in < 85 %. Problems included incomplete inclusion of the breast tissue, unsharp images, low image contrast, over- or underexposed images [28, 38] (Fig. 3). Overall, a very large number of interval cancers (143/575) resulted in this annual screening trial. It exceeds the rate, which in modern quality-assured mammography screening is considered acceptable for year one of the interval, by a factor of about 3. According to the reviewers, the above problems of poor image quality, inadequate training of personnel and readers may have accounted for the majority of the excess interval cancers. A subgroup analysis by Moskowitz [39] reported an increase of the detection rate with improving image quality at the end of the study, supporting a correlation with image quality.

    Fig. 3
    figure 3

    Images demonstrating the significant changes of technology between 1984, 1987 (CNBSS-study), and a follow-up mammogram of the same patient of 1993. Even though the later technology is still far from present contrast resolution, it becomes obvious that on the former mammograms almost no structures can be discerned in 80 % of the breast making detection of both masses and microcalcifications almost impossible. Images reproduced courtesy of Dr. Roberta Jong, Sunnybrook Health Sciences Centre, Toronto Canada

    Full size image

Finally, it has been estimated that 25 % of the recommended biopsies were not performed. One of the surgeons was convinced that non palpable lesions detected by mammography did not require biopsy [1, 28].

Discussion and conclusion

The above summary of the published results demonstrates several debatable issues concerning the CNBSS:

  • It has to be emphasized that by definition, screening should address asymptomatic women [40]. Current screening is population-based and aims to invite asymptomatic women. The prevalent round of the Canadian trial had a very high proportion of palpable cancers (partly skewed by the nature of their recruitment strategy). Thus, their results are not applicable to current practice where there is uniform access to high quality symptomatic services for women with symptoms. By accepting palpable tumours (most probably advanced and with worse prognosis), the results are skewed: the overall numbers of cancers are artificially high while the palpable cancers cannot contribute to an improved mortality reduction. Thus, the screening effect will be considerably diluted and underestimated. This leads also to underpowered statistics [28, 31].

    The documented process of randomisation did not warrant blinding. Thus, any person involved in the study could subvert the randomisation. Also, the probability of subversion was enhanced since mammography was not necessarily offered to women in the control group. We do not assume that the principal investigators committed any fraud. However, they could not have prevented subversion with the chosen protocol. The disproportional distribution of far advanced stages (cancers with > 4 involved lymph nodes) in the prevalence screen < age 50 years is highly significant and supports the doubts concerning correct randomisation. An even distribution of demographic and risk factors cannot exclude a bias toward late stage cancers, which may severely impact on the assessment of mortality reduction and calculation of overdiagnoses.

  • Long-term mortality reduction was calculated from a maximum of five annual rounds. Because of the short overall duration and continuing entrance of first round screenees, the maximum screening effect could not be reached for many of the participants. Mortality reduction was calculated based on cumulative rates of a mixed trial participation of one to five rounds during up to 5 years. This might lead to a substantial underestimation of the true screening effect compared to a screening programme following approved guidelines (in which participants undergo approximately 10 complete rounds in 20 years) [41].

  • The higher evidence classification of individual versus cluster randomized studies is correct in principle. But for screening trials, where non-invited and invited women cannot be blinded, individual randomisation may lead to a much higher contamination of the control group than in a cluster randomized setting. Thus, in the CNBSS, as in other individual randomized screening trials, a substantial underestimation of the screening effect through contamination cannot be ruled out.

  • The fact that none of the radiological reviewers including the responsible physicist considered the quality sufficient is highly concerning, as is the described lack of technologist and reader training and the high rate of interval cancers. Two reviewers resigned during the study. How can a method be tested if it is not properly performed and interpreted? What is the value of the results?

  • The fact that recommended biopsies of mammographically detected abnormalities were not systematically performed is likely to have distorted the results. What effect is expected from early detection if suspicious findings are not followed by adequate assessment and therapy?

Both obvious and probable protocol deficiencies are likely to have had an impact on the results counteracting a possible effect of mammography screening on breast cancer mortality and distorting estimates on overdiagnosis. As appropriate randomisation is one of the key validity criteria for RCTs, a study with that kind of violation should not be rated as a high quality study. Whether evidence from such a trial can be used at all must be questioned.

Even if the raised concerns were insignificant and the results were valid, the question remains whether the results and conclusions from a screening trial performed in 1980-85 are applicable to or useful for the assessment of present screening programmes. The most appropriate answer is obviously “no”.

First, the age range in the CNBSS was 40–59 years, which does not apply to the age range of most mammography screening programs today (50–69 or 74 years), as recommended in National and European Guidelines. Because of the lower incidence of breast cancer at younger ages, the absolute effect is lower. Today we know that mammography quality is even more important in the younger age group due to more difficult detection within dense breast tissue. Secondly, the mammographic technique and quality assurance of the complete chain from screening to screen reading, assessment, and treatment in modern population-based mammography screening programs is almost completely different from the CNBSS [42]. Abnormalities are routinely assessed using state of the art minimally invasive methods, and treatment is increasingly standardized and adapted to the stage at detection and the aggressiveness of the cancer. Finally the improved quality today clearly has increased the sensitivity and specificity of mammography screening. The result of 68 % of palpable breast cancers in the mammography arm (average size 1.9 cm) would today be unacceptable for annual (!) screening.

What can we conclude from this short review?

Probable deficits of randomisation and of proper application of the test cannot be repaired by performing a follow-up study. The results of such a study remain biased.

We do not want to discount the CNBSS which represents an enormous and exceptional effort in the 1980s, and we took note of its size, the time, and circumstances when it was conducted. Also, the important question asked in this trial was different from all other screening trials.

However, the chosen methodology led to obvious biases with significant impact on the results, especially when long-term results are considered. Furthermore, it is more than obvious that the setting of mammography in the CNBSS is not comparable to present mammography screening programs. Therefore, using the CNBSS as “highest evidence” to assess the effects of modern mammography screening programs of the new millennium is not scientifically justified.

Considering the fact that properly performed cohort studies and nested case-control studies with appropriate consideration of length time bias demonstrate a much higher effect of mortality reduction the assumption that the null effect of the CNBSS is “due to availability of chemotherapy” [20] is unproven and highly speculative.

It is an unanswered question, why high-ranking journals, like the BMJ, and representatives of evidence-based medicine close their eyes to these arguments [43] and still refer to the CNBSS as “superior” evidence (against mammography screening).

Taking the CNBSS as an example, the authors want to point out how evidence in the field of breast cancer screening has systematically been omitted, distorted, or inappropriately used over the last decades.

When using CNBSS data, opponents of screening mammography [6, 19, 44, 45] ignore or misinterpret an important part of the existing evidence. The consequence of this recommendation is “waiting until a cancer becomes palpable”. This means that contrary to early detection, women would present at a stage that usually requires aggressive treatment including chemotherapy and more often axillary dissection. There is no doubt that evidence shows that the earlier the stage of breast cancer at diagnosis, the better the prognosis.

In conclusion, the comparison of the settings of the CNBSS with the setting of modern mammography screening is akin to comparing apples with pears. Drawing conclusions from the CNBSS for today’s quality-assured population based screening programmes is an act of negligence.

What we need today is the continuous evaluation of the ongoing mammography screening programmes, including, but not only, breast cancer mortality as an outcome.