Introduction

The identification of BRCA1 and BRCA2 mutations has dramatically changed the landscape of breast cancer in the past decade. Testing of these genes has become an important part of clinical practice. Mutations in either of these genes results in increased risk of breast and ovarian cancer, accounting for 5% to 10% of breast cancers and 10% to 15% of ovarian cancers [1–4]. BRCA1 is mutated more frequently in families with both breast and ovarian cancer [5, 6] and more rarely in families with male breast cancer where BRCA2 is predominant [7, 8].

Genetic testing, however, is expensive and may be associated with adverse psychological effects not only to the patient but also family members [9, 10]. Family history of breast cancer is not uncommon, but BRCA mutations are relatively rare. Establishing an efficient way to identify a “high-risk group” accurately for genetic testing is important for patient care. However, the prevalence of germline BRCA mutations in these “high-risk families” is estimated at 13–19% [11, 12]. These low figures lead to the development of models (such as BOADICEA [13], BRCAPRO [14–16], Myriad [11], Couch (also known as PENN) [17], Shattuck-Eidens [18], and Manchester [19]) that can assess the pre-test probability of identifying a BRCA1 or BRCA2 mutation and enable efficient targeting of genetic testing.

Although these models were built by using data from Caucasian populations, they are being used in clinical practice to assess the risk of BRCA1/2 mutation carriage in patients of other ethnic backgrounds [20–22] and have had variable accuracy for African Americans and Hispanics in the United States in different studies [20–23]. Indeed, the chance of carrying a genetic mutation varies between different races and is most common in Ashkenazi Jewish cohorts [24]. For ethnic populations where limited genetic studies have been undertaken, variants of unknown significance detected may be benign mutation changes but some may in fact be characterized to be pathogenic when analyzed further at an RNA level [25]. Hence, evaluation of the performance of these models in different ethnic groups has been performed so that individuals who are at risk can be accurately identified and be offered intensive surveillance and preventative measures.

Although still less-tested than in Caucasian populations, increasing numbers of Asian cohorts are being clinically tested both in Asia and in western countries, such as the United States [26–30]. A few studies have evaluated these models in a mixture of different Asian or Chinese cohorts [31–33]. Most of these studies found an underprediction of BRCA2 mutations with a comparable discriminative ability as for Caucasians. Moreover, none of the studies separately analyzed the accuracy of the use of these models in prediction of mutation carriage for males [32, 34], and therefore, there is still limited reporting of gender-specific prediction.

The purpose of our study was to compare the four commonly used BRCA1/2 mutation prediction models (BOADICEA, BRCAPRO, Myriad II, Couch and Shattuck-Eidens) to determine the likelihood of finding a BRCA1, BRCA2, or combined BRCA1/2 gene mutation in patients residing in Hong Kong, who are mainly Southern China origin Chinese, and to determine if these models can perform accurately for males.

Materials and methods

Study population

Participants were recruited through the prospective database at The Hong Kong Hereditary and High Risk Breast Cancer Family Registry (www.asiabreastregistry.com), which was established in March 2007. Protocols of the study were approved by Institutional Review Board of the participating research centers. The Registry collects data from high-risk probands and families referred to the Hong Kong Hereditary and High-Risk Breast Cancer Programme for consideration of genetic testing. Female breast and ovarian cancer patients were accrued based on age of onset, family history suggestive of hereditary predisposition, bilateral breast cancer status, and male breast cancer patients and were recruited from public and private hospitals and centers covering all areas of Hong Kong. Additional details of accrual were published in a previous study [29]. An epidemiological questionnaire, pedigree information about breast, ovarian, and/or other cancers of the first-, second-, and third-degree relatives of each proband was obtained. Unknown ages and unknown year at death were assumed to be 25 years between each generation [35].

The CancerGene software program (CaGene 4.3, The University of Texas Southwestern Medical Center, Dallas, TX) was used to calculate the probability of BRCA1, BRCA2, and BRCA1/2 mutation carriage from BRCAPRO, Myriad II, and Couch and Shattuck-Eidens models. BOADICEA risk model was calculated through https://pluto.srl.cam.ac.uk/cgi-bin/bd1/v1/bd.cgi using the most updated software available. Couch and Shattuck-Eidens models were only calculated in female participants.

All calculations were performed for male and female patients separately to evaluate how accurately BRCAPRO, Myriad II, Couch and Shattuck-Eidens, and BOADICEA models predicted risks for male and female independently. Independent t tests were used to compare any difference in mean score computed by the five carrier prediction algorithms between patients with and without BRCA mutation. Pearson χ2 goodness of fit test was used to compare the number of mutations predicted by these risk models with the actual number of mutation detected.

The area under the receiver operating characteristic (ROC) curve with 95% confidence interval for each model was used to determine discrimination. ROC was evaluated to compare the ability of these models to distinguish between patients with and without a mutation and to measure the overall performance of each model. The closer the area under the ROC curve (AUC) is to 1, the better the overall performance of the model. A model with an AUC value of 1 is one that is perfectly accurate, whereas an AUC of 0.5 indicates no discriminating ability. The reference line distinguishes subjects who carry the BRCA mutation versus those who do not by pure chance. The resulting ROC curve would fall along this diagonal line, which is referred to as the chance diagonal. The empirical estimates of the sensitivity and specificity for positive BRCA status were calculated at the conventional testing thresholds of 10 and 20 for BRCAPRO, Myriad II, and BOADICEA models.

Fisher’s exact test was used to test for significance for small sample sizes. All tests were two-sided, and p values < 0.05 were considered statistically significant difference. All statistical analyses were performed with the SPSS for Windows Release 16.0 (SPSS Inc., Chicago, IL).

Results

A total of 310 probands (285 females and 25 males) were recruited. All were of Chinese ancestry. Among them, 267 (86.1%) individuals (247 females and 20 males) were noncarriers and 43 (13.9%) individuals (38 females and 5 males) were mutation carriers. Among 285 female probands, most were breast cancer patients (98%), and 247 (86.7%), 15 (5.3%), and 23 (8.1%) were noncarriers, BRCA1, and BRCA2 carriers respectively. Among 25 male probands, 20 (80%) and 5 (20%) were noncarriers and carriers respectively. All male mutation carriers had BRCA2 mutations. Table 1 shows the characteristics of the study population.

Table 1 Personal and family history of the probands (N = 310)
Full size table

Table 2 shows the mean and median predicted probabilities of mutation carriage for BRCA mutation carriers and noncarriers for female patients. We found that there was significant difference in mean predicted probability by all models for female BRCA carriers versus noncarriers. BRCA mutation carriers generally had higher mean scores than noncarriers. Table 3 shows that for males, none of the models showed any significant difference in mean predicted probability between BRCA carriers and BRCA noncarriers, although BRCA mutation carriers had higher model scores overall.

Table 2 Difference in scoring systems between BRCA-positive and BRCA-negative using two independent t statistics: females
Full size table
Table 3 Difference in scoring systems between BRCA-positive and BRCA-negative using two independent t statistics: males
Full size table

Observed and expected numbers of mutation carriers by predicted carrier probability using the models are shown in Tables 4, 5, and 6. BRCAPRO predictions are seen in Table 4. In females, 16 BRCA1 and 25 BRCA2 mutation carriers were predicted and 15 and 23 were observed respectively. For a total BRCA mutation prediction of 41 carriers when 38 were observed (p for goodness of fit = 0.91), this model performed the closest predicted carrier probability. In males, three, four, and three carriers were predicted using BRCAPRO, Myriad II, and BOADICEA models respectively and five were observed.

Table 4 Observed and expected number of mutation by predicted carrier probability under BRCAPRO: females and males
Full size table
Table 5 Observed and expected number of mutation by predicted carrier probability under Myriad II: females and males
Full size table
Table 6 Observed and expected number of mutation by predicted carrier probability under BOADICEA: females and males
Full size table

As shown in Table 5, Myriad II predicted 32 female BRCA mutation carriers compared with the 38 observed (p = 0.444). For males, four were predicted and five were observed (p = 1.000). This was the best predictive model for males.

BOADICEA (Table 6) predicted 13 BRCA1 and 19 BRCA2 (32 in total) mutation carriers compared with 15 and 23 observed (38 in total) respectively (p = 0.723). In male probands, BOADICEA predicted three BRCA2 mutation carriers compared with five observed (p = 0.702).

For female probands, BRCAPRO (Table 4) tended to underestimate the number of BRCA1 and BRCA2 carriers at carrier probabilities <20%, but overestimated those ≥20%. Myriad II (Table 5) underestimated the number of BRCA1/2 mutation carriers for all carrier probabilities. BOADICEA (Table 6) underestimated the number of BRCA1 and BRCA2 carriers for carrier probabilities <40%, but overestimated those ≥40%. All models underestimated the expected number of males.

For BRCA1/2 prediction in females (Fig. 1a), the AUC was 0.79 using BRCAPRO, 0.72 using Myriad II, and 0.8 using BOADICEA. BRCAPRO had greatest AUC in BRCA1 specific (0.93) prediction compared with other models, and the same BRCA2 specific (0.73) prediction in females as BOADICEA. Conversely, the AUC was 0.8 using BOADICEA for BRCA1/2 mutation prediction in females and 0.87 in males (Fig. 1b); both were the highest scores compared with other models. Overall, BOADICEA had the highest discriminating power in females and males. Figures 1c and d show ROC curves for different models, comparing female BRCA1 carriers and noncarriers and BRCA2 carriers and noncarriers respectively. Figure 1e illustrates ROC curves for different models comparing male BRCA2 carriers and noncarriers.

Fig. 1
figure 1

a ROC curves among difference models in female comparing BRCA carriers and noncarriers. Best model: BOADICEA, ROC = 0.8, p < 0.001. b ROC curves among difference models in male comparing BRCA carriers and noncarriers. Best model: BOADICEA, ROC = 0.87, p = 0.013. c ROC curves among difference models in female comparing BRCA1 and non-BRCA1 carriers. Best model: BRCAPRO, ROC = 0.93, p < 0.001. d ROC curves among difference models in female comparing BRCA2 and non-BRCA2 carriers. Best model: BOADICEA, ROC = 0.73, p < 0.001. e ROC curves among difference models in male comparing BRCA2 and non-BRCA2 carriers. Best model: BOADICEA, ROC = 0.87, p = 0.013

Full size image

Performance of BRCAPRO, Myriad II, Couch and Shattuck-Eidens, and BOADICEA models at conventional thresholds of 10% and 20% is shown in Table 7. In females, the highest sensitivity at both 10% and 20% cutoffs was achieved by BRCAPRO for BRCA1/2 mutations combined (73.7 and 57.9), BRCA1 mutations (86.7 and 66.7), and BRCA2 mutations (43.5 and 34.8), but its specificity was slightly lower than BOADICEA.

Table 7 Performance of BRCAPRO, Myriad II, Couch and Shattuck-Eidens, and BOADICEA at conventional thresholds of 10% and 20%
Full size table

In males, BOADICEA had similar sensitivity compared with BRCAPRO at both 10% and 20% cutoffs (60 and 40) in BRCA1/2 combined and BRCA2 (60 and 40) but a higher specificity for BRCA1/2 combined (80 and 100 vs. 750 and 95) and BRCA2 specificity (85 and 100 vs. 75 and 95). Myriad II generally had a lower sensitivity and specificity at both 10% and 20% cutoffs except for a slightly high specificity at 20% in BRCA1/2 combined in females (92.3). Couch and Shattuck-Eidens model had inferior sensitivities and specificities overall in our cohort.

Discussion

Hong Kong, being the southern part of China, is a unique place to study hereditary breast cancers in Chinese with >80% of the study population originating from southern China [29, 36]. Moreover, the one-child policy in Mainland China is not practiced in Hong Kong, enabling larger family structures for analysis, which is relevant because limited familial history has been reported to result in underestimation of mutation carriers by various prediction models [33, 37]. In 2008, more than 2,600 new breast cancer cases were diagnosed (Hong Kong Cancer Registry) and it ranked the third most common cancer after lung and colorectal malignancies and the most common cancer in females. An improved understanding of hereditary breast cancer and more accurate selection of patients for genetic testing will have important implications for economic health policies.

A previous study performed by our group found 12.8% of clinically high-risk Chinese probands with breast and/or ovarian cancers carried a deleterious BRCA mutation, of which 60.7% were BRCA2 mutations [29]. This is a higher percentage of BRCA2 mutations compared with that of most Caucasian cohorts where studies have found that the prevalence of BRCA1 mutations ranges from 6.9% to 8.3% in European and American Caucasians compared with BRCA2 mutation prevalence of 5.2% to 5.9% [38–40], although consistent with other findings in Asian countries and a study performed by our group in Asian Americans [33, 41–43]. Mutations tend to be population-specific so different ethnic cohorts are likely to have a different spectrum of mutations and also different founder mutations [20, 44, 45]. In fact in a previous study, we reported a BRCA2 founder mutation in our cohort, which has accounted for the larger proportions of BRCA2 mutation carriers in our locality [36]. All of these differences are likely to result in inaccuracies in the use of existing prediction models, which have been designed based on Caucasian cohorts.

Asians comprise 57% of the world’s population and Chinese represent the largest group in the Asia continent [46], many of whom reside in western countries, such as the United States, where 4.2% of the population are Asian Americans. Therefore, Asian-specific studies and the accuracy of risk prediction models for use in this group would be of clinical relevance worldwide.

Our study indicated that BOADICEA is most accurate in predicting the numbers of BRCA1/2 mutation carriers combined compared with the other models with an AUC of 0.8 in females. It also had the closest predicted carrier probability for both male and female cohorts. Both BOADICEA and BRCAPRO models predicted the BRCA1 mutation carriage more accurately compared with the BRCA2 mutation carriage, although BRCA1 mutation carriage was better predicted by BRCAPRO: AUC was 0.87 (BOADICEA) and 0.93 (BRCAPRO), respectively.

Overall for the BRCAPRO and BOADICEA models, the AUC of 0.8 for BRCA1/2 mutations combined were higher than previous reports by Rao et al. (0.725) [34], Euhus et al. (0.712) [47], Marroni et al. (0.757) [48], Antoniou et al. (0.76) [49], and Kurian et al. (0.71 for Asians, 0.77 for whites) [33]. Possible reasons for predictions being higher in our study may be attributed to the differences in the prevalence of mutations, differences in mutation spectrum, and penetrance. Most other studies performed in Asians comprised a mixture of different Asian ethnic groups, including Vietnamese, Koreans, Filipinos, Malays, and Indians, whereas ours study was limited to southern Chinese. Moreover, a prediction model’s accuracy is dependent of the proband’s own account of family cancer history [50]. The reporting of family history may have differed in other studies where three generations of family history may not have been elicited which may also explain the differences in the estimation of BRCA mutations using these prediction models between studies [51]. Another possible explanation for the differences is the use of gender-specific analysis in our study. For the few males we included in our study, we found that the models did not have any different predictive probability between BRCA2 mutation carriers and noncarriers and they all underpredicted BRCA2 mutations by nearly twofold (there were no BRCA1 carriers). Risk prediction models should be used with caution in males because there is a lack of studies to assess the accuracy of the use of such models in male cohorts alone. One Caucasian study did find that BODICEA 5.0 can achieve a prediction sensitivity of 0.8 for BRCA1/2 and 0.63 for BRCA2 at 10% threshold [52]. A larger cohort of male probands is necessary to allow further confirmation of our studies’ findings. Consistent with previous reported studies, we found that both BOADCIEA and BRCAPRO models underestimated the number of mutations carriers at a lower threshold and overestimated at a higher threshold [53].

Recent studies have found that accuracy of risk prediction models can be improved by incorporating pathologic information into the algorithm [54]. Moreover, the use of risk-reduction strategies can affect the apparent penetrance of mutations and affect the prediction accuracy. Our previous studies found that approximately 20–30% of women with BRCA mutation elected for prophylactic contralateral mastectomy and salpingo-oophorectomy [55], including unaffected family members of probands who have found themselves to be mutation carriers. Revised versions of the BRCAPRO model incorporate such information [56] and may improve on the prediction accuracy that we report here.

The strength of this study is its representation of the broader Hong Kong population, because most cancer genetics referrals are seen at our institution through the referral to the Hong Kong Hereditary and High-Risk Breast Cancer Programme. Unlike many prior studies, complete genetic testing, including full gene sequencing and multiplex ligation-dependent probe amplifications (MLPA), was performed on all patients to minimize the chance of false negatives. Limitations include a clinic-based cohort rather than population-based setting, and a relatively small sample size especially for males, given the rarity of their disease.

We found underestimation of BRCA2 mutations in this Chinese cohort by standard mutation prediction models, despite relatively satisfactory discriminative ability; given that BRCA2 mutations are more prevalent in Asian and male cohorts, this finding has clinical significance. Further studies in larger cohorts, including Asians and males, are indicated, with the goal of developing an accurate predictive model specific to these populations and of targeting genetic testing more accurately for optimal patient care.