Background

Many studies have demonstrated the ability of using gene-expression "signatures" derived from DNA microarray data to define cancer subtypes, predict disease recurrence, and guide treatment decisions. In breast cancer, van't Veer et al. [1] derived a 70-gene profile to predict a patient's risk of developing distant metastases. Perou et al. [2] and Sorlie et al. [3] developed an intrinsic-subtype signature that classifies breast tumors into molecular subtypes showing distinct differences in prognosis. From a cancer biology perspective, Chang et al. [4] studied the links between the wound healing process and cancer progression. Based on the expression pattern of a wound-response signature of 512 genes, they classify a tumor to have either activated or quiescent response and found this to be a significant prognostic predictor of tumor metastasis. These are promising results and a few of these signatures have begun to be assessed in clinical settings. Two questions have often been asked: (1) are these signatures identifying the same set of genes and (2) will they generate similar prediction performance when tested in new data sets?

The answer to the first question has been discouraging. Any pair of these signatures share only a few common genes. Possible reasons have been suggested including the differences in patient cohort characteristics (such as the distribution of age or stage of the disease), lack of comparability and reproducibility of the data generated using different microarray platforms, and varying statistical procedures used to generate the gene list. Nevertheless, Ein-Dor et al. [5] showed that the inconsistency still exists when eliminating all three differences. In particular, the authors repeated the same analysis in a single data set and identified many lists of genes equally predictive of the outcome. Any two of these gene lists share only a small number of genes. In another study by Son et al. [6], the authors reported that any randomly selected subgroup of around 100 differentially expressed genes generates similar hierarchical clustering results in the same data set.

Ein-Dor et al. [7] further suggested perhaps the main source of the problem lies in the small sample size and large number of genes the signatures were derived from. For several published breast cancer data sets, the authors estimated that several thousands of samples would be needed to achieve a typical gene overlap of 50%. On the other hand, the problem is compounded by analyzing and interpreting genes in isolation. A common approach to gene selection involves selecting a handful of top-ranking genes that best differentiate sample classes (such as tumor vs. normal tissue) or are most predictive of clinical outcome. The univariate selection procedure ignores correlation between genes. The biological and statistical validity of such assumption seems tenuous. As a result, gene-set based approaches have emerged in recent years to identify sets of biologically related genes that are deregulated as a group. Examples of gene-set analysis include the Gene Set Enrichment Analysis (GSEA) [8], Significance Analysis of Function and Expression (SAFE) [9], and the globaltest package [10] These methods focus on groups of genes that share common biological functions such as cell cycle regulation; metabolic or signaling pathways defined by Gene Ontology (GO); online databases such as BioCarta, KEGG and signaling data base; or a literature-defined gene set subject to experimental perturbations such as a drug treatment or an oncogene-activation. In addition, Rhodes et al. [11] introduced a Molecular Concepts Map (MCM) providing an expanded analytic framework to explore the network of relationships among biologically related gene sets.

The motivation of this study came from a recent paper by Fan et al. [12], which addresses the second question described above. The authors demonstrated a high degree of prediction concordance of five breast cancer gene-signatures despite minimal gene-wise overlap. In an independent data set of 295 tumors, the authors showed that the intrinsic subtypes [3] of basal-like, HER2+/ER-, and luminal B were consistently classified as poor 70-gene profile [13] prognosis, activated wound response [4] and high recurrence score [14]. It raises the question that perhaps the gene-overlap is not the most relevant measure of robustness and reproducibility of the gene-signatures. There may be common themes shared across these signatures that are not apparent on the individual gene level. As an example, the cell cycle gene Cyclin E1 (CCNE1) was included in the 70-gene profile while Cyclin E2 (CCNE2) in the intrinsic subtype signature. The two signatures apparently share commonality in the activation of the Cyclin family genes. For another example, ERBB2 and EGFR are both receptor tyrosine kinase involved in estrogen pathway. Inclusion of one or the other in two different signatures apparently converges at the pathway level both indicating the activation of the estrogen-signaling pathway.

In this study, we assess the potential functional convergence of these gene-signatures on the basis of activated oncogenic pathways. This involves first annotating each gene-signature to identify significantly enriched functional modules (e.g., cell growth, response to estrogen, myb-regulated pathways, etc.). Definition of the modules can be based on Gene Ontology (GO) terms, online pathway databases such as BioCarta and KEGG, or literature-defined concepts. In the next step, the overlapping functional modules are obtained by intersecting the annotated sets. We investigated six breast cancer signatures (four of which were compared in Fan et al. [12]) that share high prediction concordance. We found eighteen common modules including estrogen-signaling, responses to tamoxifen treatment, and BRCA1 expression. The degree of the functional overlap across the six BR-signatures is highly significant (P = 0.0002) under a bootstrapped null distribution.

Results and Discussion

Prediction concordance across five breast-cancer gene-signatures

In a similar fashion as in [12], we cross-tabulated the prediction results of the gene-signatures listed in Table 1 in the 295 breast cancer patients in the van de Vijver study [15]. In Table 2, all the signatures consistently classified the basal-like and HER2/ER- subtype tumors as having high risk of recurrence outcome. The 70-gene profile and wound-response signatures both classify luminal B subtype to be a low risk group, while the meta-signature classifies the luminal A and the normal-like subtypes as low risk groups. Overall, the signatures showed a certain degree of prediction concordance. The kappa coefficient measuring the classification agreement across the signatures is estimated to be 0.67.

Table 1 Breast cancer gene-signatures.
Full size table
Table 2 Classification concordance of the breast cancer gene signatures (Kappa coefficient = 0.67)
Full size table

Common "oncogenic" sets underpinning breast cancer outcome prediction

For pairs of the six signatures, there is a fair amount of overlapping literature concepts (MCMs). Many of the overlaps are highly significant (Figure 1A). For example, there is a set of 142 enriched MCM modules shared between the 70-gene profile and the wound-response signature (P < 0.00001) while only two genes were identified by both. Furthermore, signatures 1–3 showed marginal significance in metabolic and signaling pathway overlaps (Figure 1B).

Figure 1
figure 1

Pair-wise functional overlap of the six breast cancer gene-signatures. 1. 70-gene profile 2. Wound response 3. Intrinsic subtype 4. Meta90 5. Recurrence score 6. Wang ER+ profile. A. The number of overlapping literature-defined oncogenic concepts (MCM) and the corresponding P-value heatmap indicating the significance of the overlap under bootstrapped null distribution. B. The number of overlapping pathway sets (MsigDB) and the corresponding P-value heatmap.

Full size image

We found 18 common MCMs (P = 0.0002) and 5 common metabolic and signaling pathways (P = 0.04) across signatures 1–4. Table 3 and 4 list these common sets ordered by the overall significance of enrichment (summarized hypergeometric test P-value adjusted for multiple testing). Among the top are deregulated genes in androgen-sensitive prostate cancer cell lines in response to MSA (MCM 258), Myb-regulated transcriptional changes in the estrogen-dependent human breast cancer cell line MCF7 (MCM 458), several MCMs comprising responsive genes upon antiestrogen hormonal treatment (MCM 691, 379, 375, 673). Clearly a dominant common characteristic underpinning the four breast-cancer signatures is closely related to the estrogen-receptor status of the tumor which is a main prognostic factor in breast cancer. Another common prognostic set of interest is response to BRCA1 expression (MCM513), which many studies have shown a characteristic of sporadic basal-like cancers.

Table 3 Eighteen common literature-defined oncogenic concepts (MCM) across the four breast cancer gene signatures (significance of overlap, P = 0.0002)
Full size table
Table 4 Five common pathway sets (MsigDB) across the four breast cancer gene signatures (significance of overlap, P = 0.04).
Full size table

Table 4 listed the five common metabolic and signaling pathways using the functional subset of the MsigDB annotation data. All of the signatures apparently enlisted genes customized on a commercial array platform that represent the breast cancer estrogen signaling pathway [see Additional File 1].

For the gene signatures listed in Table 1, it should be pointed out that they were constructed using different types of endpoints, along with differing supervised learning algorithms. In attempting to combine results across the signatures, we make the assumption that there exists an underlying tumorigenetic mechanism that manifests itself in terms of the endpoints used by the different authors. One such mechanism might be tumor metastasis.

ER-positive relapse signatures

Both ER+ relapse-signatures showed evidence of E2F activation, response to Interleukin-6 (IL6), and activation of insulin-signaling pathways, some of which have been reported in the literature to be specific to ER+ disease [see Additional Files 2 and 3]. For example, studies have shown in estrogen-sensitive breast cancer cell lines, the widely used antiestrogen tamoxifen treatment inhibits insulin-signaling. The degree of such inhibition can reflect the effectiveness of the tamoxifen treatment and thus correlate with a patient's risk of recurrence [16, 17].

Conclusion

Cancer gene-expression signatures derived from microarray experiments are beginning to be tested in clinical trials, while the exact biology that enables these gene-signatures to accurately predict tumor metastasis and patient survival is unclear. Microarray experiments are often limited in power by the small number of samples used to derive a panel of prognostic genes relative to the large number of features on the array. In addition, sets of biologically-related genes are often co-regulated while many feature selection procedures are univariate in nature. As a result, gene-signatures developed by different studies typically share very few common components. A recent study showed high prediction concordance of several breast cancer gene-signatures despite minimal overlap in gene identity. It gave main motivation to investigate common oncogenic themes that may not be apparent at the individual gene level. This study explored this hypothesis by evaluating the functional overlap of the signatures on the basis of annotated gene sets. When the gene signatures are mapped to the deregulated pathway space, two things become clear. First, there is a significant degree of functional overlap in oncogenic and prognostic pathways. Second, many of these common pathways provide plausible explanations of tumor biology through which these signatures predict patient outcome. There are several conclusions to be gleaned from this study. First, this work explains why independent signatures that appear to perform equally well at predicting patient prognosis show minimal overlap in gene membership. This is because such genes are different members of pathways and processes that are relevant to prognosis. Thus, the lack of gene overlap found between the various signatures listed in Table 1 should not be considered problematic. The implication of our study is that most of these signatures will do well in clinical trials given that they seem to be picking up the same pathway signals. We can thus be assured that the gene lists found by different investigators are consistent, even if they do not contain the same genes.

Second, the results have suggested that the interpretability and delineation of how diverse cancer gene expression signatures work are more likely attainable at the pathway level rather than the individual gene level. On the other hand, as many studies have already suggested so, feature selection methods need to be based on biologically related gene sets that are deregulated as a group [8–11]. However, it is not a straightforward task to construct a prognostic signature based on pathways that are composed of overlapping sets of genes. New statistical methods need to be established in this area. This is beyond the scope of the study and is currently under investigation.

Methods

Table 1 lists the six BR-signatures that are compared in this study. Fan et al. [10] showed high prediction concordance of signature 1–3 and signature 5. In addition, a 90-gene meta-signature [13] is included. This signature was derived in a meta-analysis framework by integrating four microarray data sets, which included the van't Veer data set and the Sorlie data set. Another signature included here is the subset of 60-gene profile from Wang et al. [18] that was derived in tumors with estrogen receptor (ER) positive status. The recurrence-score signature [14] is also an ER+ disease signature that has been shown in a clinical trial to be able to identify patients with very low risk of recurrence on hormone therapy using tamoxifen alone, and do not require adjuvant chemotherapy.

Annotation

The set of signature genes were annotated using two different annotation sources:

Literature-defined module

A collection of 661 literature-defined modules from the Molecular Concept (MCM) database MCM that focuses on human cancer studies. These include gene sets from peer-reviewed publications using microarrays to study gene expression changes subject to experimental perturbation such as drug treatment or candidate gene activation.

Pathway module

The functional subsets from the molecular signature database or MSigDB GSEA, including modules representing metabolic and signaling pathways imported from online pathway databases such as BioCarta [19], signalling pathway database [20] and the Kyoto Encyclopedia of Genes and Genomes (KEGG) [21].

Enrichment analysis was performed using hypergeometric tests. In particular, the procedure tests the significance of the proportion of module genes (e.g., estrogen pathway) in the signature being greater than the "population"-proportion of the module genes in the experimental set from which the signature was selected. Multiple testing was adjusted by using the Benjamini-Hochberg procedure [22].

Notation and methods

For a set of K gene-signatures, let n i be the number of genes in signature i and N i be the total number of genes in the experimental set from which the signature genes were selected. Furthermore, let J = 661 or 552 denote the number of literature-defined concepts and the number of metabolic and signaling pathways in the two annotation database MCM and MsigDB respectively. For a gene signature, we first perform a module enrichment analysis using a hypergeometric test. As mentioned earlier, the basic idea is to test whether the proportion of the module genes in the signature of size n i is significantly larger than the "population"-fraction of the module genes in the experimental set of size N i . The jth module is enriched in the ith signature if the hypergeometric test p-value is less than 0.3. Across the K signatures under comparison, this threshold correspond to a p-value of less than 0.05 under a conventional meta-analysis of combining the hypergeometric p-values − 2 ∑ i = 1 K log P i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeyOeI0IaeGOmaiZaaabCaeaacyGGSbaBcqGGVbWBcqGGNbWzcqWGqbaudaWgaaWcbaGaemyAaKgabeaaaeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGlbWsa0GaeyyeIuoaaaa@3B29@ across the four signatures based on a chi-square distribution with 2K degrees of freedom. Let X ij be the indicator variable where X ij = 1 if the jth module is enriched in the ith (i = 1, ..., K) signature and X ij = 0 otherwise. As a result,

m i = ∑ j = 1 J X i j MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyBa02aaSbaaSqaaiabdMgaPbqabaGccqGH9aqpdaWdXbqaaiabdIfaynaaBaaaleaacqWGPbqAcqWGQbGAaeqaaaqaaiabdQgaQjabg2da9iabigdaXaqaaiabdQeakbqdcqGHris5aaaa@3AE1@

is the total number of enriched modules in signature i. Then for the set of K signatures, the amount of functional overlap is Y = ∑ j = 1 J ∏ i = 1 K X i j MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemywaKLaeyypa0ZaaabCaeaadaqeWbqaaiabdIfaynaaBaaaleaacqWGPbqAcqWGQbGAaeqaaaqaaiabdMgaPjabg2da9iabigdaXaqaaiabdUealbqdcqGHpis1aaWcbaGaemOAaOMaeyypa0JaeGymaedabaGaemOsaOeaniabggHiLdaaaa@3F7B@ .

The significance of overlap is defined as P (Y > y obs) under a bootstrapped null distribution. The bootstrap procedure is described elsewhere [see Additional File 4].

We used B = 100,000 in the procedure. The bootstrapped null distribution of Y preserves 1) potential correlation of the signature size n i and the number of enriched modules m i , and 2) the module-module dependence due to the one-to-many mapping of a gene to the annotation data.