Background

High-throughput expression profiling is a major tool for functional annotation of sequenced genomes. SAGEmap [1] and NCI60 cDNA microarray [2] datasets have been incorporated into the Ensembl and UCSC human genome browsers. Large-scale expression data have also been used to construct a transcriptional profile of adult skeletal muscle [3]; to identify genes preferentially expressed in prostate [4], granulocytes [5] and thyroid [6]; to discover markers of pathological states such as cancer [7], and for whole genome structure analysis such as identification of clusters of similarly expressed genes [8, 9].

Numerous public databases have been created to store large-scale expression data. These databases are, however, heterogeneous in format, sample annotation, and statistical post-processing of experimental results. Furthermore, the different databases store data from different experimental technologies. A cross-platform comparison between cDNA and oligonucleotide microarrays has demonstrated that data generated by different platforms may have poor correlation [10]. We investigated whether three public gene expression databases: the Gene Expression Atlas published by Su et al. [11], SAGEmap [1] and TissueInfo [12] (representing respectively oligonucleotide microarray, Serial Analysis of Gene Expression – SAGE, and Expressed Sequence Tags – ESTs) are internally consistent and complementary when used to compare expression profiles of human genes within six normal tissue categories, which were represented in the three databases.

The scope of this study is integration of expression data and meta-analysis of datasets. Our aim is to evaluate the congruence of publicly available expression datasets. It was not our intention to provide a technical benchmarking of different expression platforms, because such a benchmarking exercise would require controlled microarray, SAGE and EST experiments starting with identical RNA samples derived from the same tissue specimen, and would be beyond the resources of most laboratories.

Results and discussion

Preferential Expression Measure

We devised a Preferential Expression Measure (PEM) to score differential expression of genes in tissues. PEM describes the expression of a gene in a given tissue in relation to its expression in all tissues. Therefore, PEM controls for the fact that some genes are highly expressed across many tissues (housekeeping genes), and has the virtue of reporting a negative value for under-expressed genes, and a positive value for over-expressed genes. Large positive PEM scores for a gene in a particular tissue indicate that the gene is unusually highly expressed in that tissue, relative to its expression in other tissues. Large negative PEM scores indicate repression of a gene in a tissue.

For SAGEmap and TissueInfo we define PEM as log10(o/e), where o is the observed SAGE or EST tag count for a gene in a given tissue, and e the expected tag count under the null hypothesis of uniform expression in all tissues. e is calculated as (G * N/T) where G is the total number of tags ascribed to the gene, N is the total number of tags for the tissue, and T is the total number of tags in the dataset.

For example, carbonic anhydrase XI (UniGene cluster Hs.22777) is linked to a SAGE tag GTCGCTGAGA. This tag occurs 132 times in all normal tissue SAGEmap libraries. The total number of SAGE tags in the normal brain tissue category is 326,481, and the total number of tags in all normal tissue categories is 1,077,231. Therefore, if the distribution of this tag was uniform across all libraries, it would be expected to occur ~40 times (132 * 326,481/1,077,231) in brain libraries (expected count). The actual count in brain libraries is 124 (observed count) bringing the PEM value for this tag in brain to log10(o/e) = log10(132/40) = 0.49.

For microarray experiments, where the raw data is a continuous variable (a relative intensity of a gene-specific fluorescent signal) as opposed to a tag count, we defined PEM as log10(S/A), where S is the specific tissue signal for a gene and A is the arithmetic mean signal for the gene across all tissues.

Reproducibility and internal consistency

A major concern with all types of high-throughput expression data is reproducibility and internal consistency. To investigate this, for each of the three databases, we compared replicate experiments (where available) or equivalent tissues and measured the Pearson correlation coefficient for relative fluorescent signal intensity values assigned to the same probe in a pair of experiments. Gene Expression Atlas oligonucleotide microarray data had good correlation between repeat hybridizations of the same RNA sample (mean r2 = 0.94 ± 0.04; N = 45 pairwise comparisons) and, as expected, slightly lower values for repeat hybridizations of different RNA preparations from the same tissue type (mean r2 = 0.87 ± 0.06; N = 17 pairwise comparisons).

Internal correlations for SAGEmap and TissueInfo were lower. Counts of identical tags were correlated in pairwise SAGEmap library comparisons yielding mean r2 values of 0.51 ± 0.25, N = 15 for brain; 0.26 ± 0.08, N = 6 for prostate; 0.89, N = 2 for vascular endothelium; 0.27, N = 2 for ovary; 0.83, N = 2 for kidney; and 0.97, N = 2 for pancreas.

For TissueInfo, we included tumour tissues to maximize the size of datasets and, therefore, minimize the sampling error. Counts of EST tags linked to the same UniGene cluster in pools of libraries were correlated. Comparison of brain TissueInfo libraries comprising 154,214 EST tags from normal tissue and 111,317 from tumour tissue, revealed an r2 of only 0.27.

Correlations among tissue profiles from Gene Expression Atlas, SAGEmap and TissueInfo

To compare different databases we grouped libraries into higher-level tissue categories, and calculated Pearson correlation coefficients for PEM scores for categories that were represented in all three databases. Tumour tissue libraries were not used in any of the interdatabase comparisons. UniGene was used as the common gene index to link entries from the three databases. There were six tissues available for comparison (Table 1). Significant positive correlations were found for brain, prostate and vascular endothelium (Figure 1) but not for ovary, kidney, and pancreas.

Table 1 r2 values for Pearson correlations between PEM scores computed from Gene Expression Atlas (GEA), SAGEmap and TissueInfo expression profiles for six human tissue categories, for which data were available in the three databases.
Full size table
Figure 1
figure 1

Correlations between the three databases Correlations between Gene Expression Atlas (GEA), SAGEmap, and TissueInfo preferential expression measures (PEM) for brain, prostate and vascular endothelium. Trend lines for linear regression and the corresponding r2 values are shown for each pairwise comparison.

Full size image

Because of sampling error, the total number of sequenced tags is a key determinant of reliability of SAGE and EST expression profiles. In our analysis, brain is the tissue category with the highest number of both SAGE and EST tags (Table 2) and accordingly shows the strongest correlations of PEM values among oligonucleotide microarray, SAGEmap, and TissueInfo (Table 1). Prostate, the tissue category with the second highest total number of tags, showed good correlation between SAGEmap and TissueInfo, but the correlation between the tag-based databases and oligonucleotide microarray data was weaker (r2 of 0.09 and 0.19 for SAGEmap and TissueInfo respectively).

Table 2 Number of SAGE and EST tags in tissue categories derived from SAGEmap and TissueInfo.
Full size table

We suggest that the relative complexity of a tissue is another important factor for comparison of high-throughput expression profiles deposited in public databases. Complex tissues consist of many cell types that may be mixed in different proportions in different samples. Expression profiles obtained from these tissues will be heterogeneous and consequently require a large number of tags and repeated hybridizations to detect signal in the noise. This is shown by the relatively strong correlations seen in SAGEmap/TissueInfo and SAGEmap/microarray correlations for vascular endothelium despite the small number of tags available (in comparison to brain, about 3-fold fewer SAGE tags and 18-fold fewer EST tags). SAGEmap and dbEST/TissueInfo vascular endothelium libraries are derived from in vitro cultures or homogenous primary isolates of vascular endothelial cells [13].

Correlations of PEM scores obtained from the three databases for kidney, pancreas, and ovary had very low r2 values (Table 1). However, measuring the correlation coefficient of PEM scores is a very rigorous test of congruence, because it not only requires that two datasets being compared identify the same sets of genes as being over- or under-expressed in the tissue, but also that the genes deviate from uniform expression to a similar degree. This may be an unrealistic expectation; for example, it is apparent from Figure 1 that PEM scores for SAGEmap in brain never exceed +1.19 (i.e., approximately 15-fold over-representation), which is probably due to the limited size of the SAGE database. A simpler test is to divide the plots into quadrants and measure the fraction of genes for which the two methods being compared give PEM scores with the same sign. For example, for brain tissue the PEM scores from SAGEmap and TissueInfo data are both positive for 134 genes, both negative for 184 genes, and in disagreement for only 28 genes (i.e., one method identifies the gene as significantly over-expressed in brain, and the other identifies it as significantly under-expressed) (Figure 1). Across all six tissues and three datasets, the PEM signs are in agreement for more than 50% of genes (the null expectation) in 13 of the 18 comparisons made (Table 1).

Tissue-specific genes

We showed previously that integrating two expression databases (SAGEmap and dbEST) provided accurate predictions of vascular endothelium-specific expression that were verified experimentally, whereas neither method alone was reliable [13]. Similarly, genes for which Gene Expression Atlas, SAGEmap and TissueInfo analyses all yield positive PEM scores should be very strong candidates for being tissue-specific. To test this hypothesis, we integrated PEM scores from the three types of expression profile to search for putative brain-specific genes. The 50 top-scoring genes were investigated through LocusLink and OMIM and of these 47 proved to be previously known brain-specific or preferentially brain expressed genes (Table 3). Three genes were novel, one of which was highly similar to a known brain-specific inorganic phosphate co-transporter. This confirms the very high reliability of integrated predictions of tissue-specific expression. The bottom-scoring ten genes for brain are also listed in Table 1. They have negative PEM values indicating under-representation in the brain transcriptome. Four of these genes are epithelial markers (EPCAM, KRT5, KRT18, and KRT19).

Table 3 Top 50 brain-specific and bottom ten brain underexpressed genes (in italics) identified by integrated PEM scores from Gene Expression Atlas, SAGEmap and TissueInfo.
Full size table

The top ten prostate-specific genes are listed in Table 4. As with the analysis for brain, combining Gene Expression Atlas, SAGEmap and TissueInfo data appears to identify prostate-specific genes successfully. Five genes were previously known to be expressed specifically by prostate epithelium (MSMB, ACPP, KLK3, AMD1 and RDH11). The other five are muscle-specific genes. This is not unexpected since the fibromuscular stroma surrounding prostatic glands is known to account for about half of the volume of the prostate [14].

Table 4 Top ten prostate-specific genes identified by integrated PEM scores from Gene Expression Atlas, SAGEmap and TissueInfo.
Full size table

Conclusions

We show here that data from Gene Expression Atlas, SAGEmap and TissueInfo can be integrated when libraries are grouped into higher-level tissue categories and genes are mapped between datasets using the UniGene gene index. After integration, expression profiles between tissue categories represented in multiple databases can be compared using a measure of differential expression, PEM. Internal consistency of PEM scores is high for Gene Expression Atlas but poorer in the tissue categories derived from SAGEmap and TissueInfo. Between datasets, PEM values correlate relatively well when large numbers of SAGE and EST tags are available (brain, prostate) or when tissue cellular composition is simple (vascular endothelium), but in the other tissue categories examined the correlation is low. The usefulness of the integrated dataset is demonstrated by the accuracy with which brain- and prostate-specific genes could be identified. This suggests that similar accuracy could be achieved for other tissues if oligonucleotide microarray, SAGE, and EST experiments were performed on RNAs from the same well-annotated tissue samples of relatively simple cellular composition. This would also enable direct comparison and benchmarking of expression profiles obtained using different technologies. The integrated dataset, code and database schema are available on request from L.H.

Methods

Gene Expression Atlas

Affymetrix U95A oligonucleotide microarray [15] data for 12,587 consensus probes in 101 human tissue samples were downloaded from the Gene Expression Atlas website http://expression.gnf.org/data_public_U95.gz. Image analysis and normalization has been performed using Genechip 3.2 software (Affymetrix, Santa Clara, CA) by Su et al. [11]. The libraries were grouped into 47 higher-level tissue categories (averaging across duplicates and triplicates). Tumour libraries were grouped as separate to normal tissues and not used for further analysis.

SAGEmap

Tag frequencies for 132 SAGEmap libraries [1] were obtained from the project ftp site ftp://ftp.ncbi.nih.gov/pub/sage. After downloading, the libraries were grouped into 15 higher-level tissue categories (brain, prostate, kidney, colon, pancreas, ovary, breast, skin, peritoneum, stomach, blood, lung, liver, heart and vascular endothelium) and annotated according to their disease status (91 tumour tissues, 37 normal tissue libraries and four immortalised cell lines). Unless stated otherwise, only normal tissue libraries (total of 1,077,231 tags) were used for further analysis.

TissueInfo

The TissueInfo [12] database links ESTs from dbEST to the tissue where there are expressed by assigning a tissue-key to each EST entry. The dataset was downloaded from the TissueInfo website http://icb.mssm.edu/tissueinfo/local-inst.xml, which is updated daily, on 27 February 2002. Our aim was to group EST into major tissues with sufficient numbers of ESTs for statistically meaningful analysis. Therefore, some TissueInfo categories were grouped together into higher-level tissue categories. For example hypothalamus was also annotated as brain. We excluded ESTs that were annotated as mitochondrial libraries, tumour libraries, or multiple tissues. This procedure resulted in 48 higher-level tissue categories, 27 of which had EST counts in excess of 10,000 (the number of ESTs we accepted as the threshold for dataset size). We also included vascular endothelium (8,435 ESTs) because of the high number of SAGE tags available for this category (110,460) and its simple cellular composition.

Mapping

Six higher-level tissue categories were available for comparison between Gene Expression Atlas, SAGEmap, and TissueInfo (Table 1). An integrated MySQL database was set up and gene identifiers from the UniGene gene index were used to link expression data from the three datasets. Only UniGene entries with more than one EST (62,008 entries) were considered as valid clusters. UniGene mapping was provided as part of both SAGEmap and TissueInfo datasets. To map oligonucleotide microarray probes, probe consensus sequences were searched against longest representative sequence of each UniGene cluster using BLASTN with the cutoff parameter E = 10e-20. Alignments were accepted if the percentage identity was higher than 94% and length was at least 99 bp or 90% of the length of the query, or when percentage identity was 100% and length more than 49 bp [11].

When linking UniGene clusters to SAGEmap data, UniGenes mapping to more than SAGE tag sequence were excluded (25% of clusters). The size of the Unigene dataset linked to SAGEmap was, therefore, reduced from 21,224 to 15,872 clusters. Similarly, Unigene clusters mapping to more than one Affymetrix probe were excluded (22% of clusters). The size of the Unigene dataset linked to Gene Expression Atlas data was, therefore, reduced from 9,402 to 7,375 clusters. The overlaps between UniGenes linked to GEA and SAGEmap, GEA and TissueInfo, and SAGEmap and TissueInfo were 2742, 6758 and 15872 clusters respectively. Therefore:

- 63% and 8% of UniGene clusters linked to GEA could not be linked to SAGEmap and TissueInfo respectively;

- 83% and 0% of UniGene clusters linked to SAGEmap could not be linked to GEA and TissueInfo respectively;

- 74% and 89% of UniGene clusters linked to TissueInfo could not be linked to SAGEmap and GEA respectively.

Statistics

Gene expression can be estimated by counting gene-specific SAGE and EST tags but, as a stochastic process, these estimates are subject to sampling error. The size of the sampled dataset can have a dramatic influence on reliability of the data. Two independent groups have established χ2 as the best statistical test to use in tag sampling experiments [16, 17]. We excluded from subsequent comparative analysis all tissue/gene data for which the expected number of tags (e) was less than 5, which is the lower limit of reliability for the χ2 statistic. We used χ2 to filter the PEM score for both tag-based methods: unless significantly different from expectation (with P = 0.05) we concluded that we had no confidence in either the sign or the magnitude of the apparent deviation from uniform expression. A relatively low significance threshold of 0.05 was used to maximize the number of values available for inter-database comparison. Because χ2 test is only used on a case-by-case basis for preliminary selection of informative datapoints, a multiple testing correction such as Bonferroni correction was not applicable.

To compare the results from any two databases for a given tissue, we plotted the PEM scores for all genes for which data was available from the two experiments, but excluding SAGE and EST data with non-significant χ2. We calculated the correlation coefficient (r) and report r2, which represents the proportion of the variability in the data that is explained by the correlation.

Database versions

Gene Expression Atlas, Affymetrix human Genechip U95A http://expression.gnf.org

SAGEmap, January 2002 http://www.ncbi.nlm.nih.gov/SAGE/

TissueInfo, 27 February 2002 http://icb.mssm.edu/crt/tissueinfowebservice.xml

UniGene, Build 148 http://www.ncbi.nlm.nih.gov/UniGene/