Robust PCA based method for discovering differentially expressed genes

Abstract

How to identify a set of genes that are relevant to a key biological process is an important issue in current molecular biology. In this paper, we propose a novel method to discover differentially expressed genes based on robust principal component analysis (RPCA). In our method, we treat the differentially and non-differentially expressed genes as perturbation signals S and low-rank matrix A, respectively. Perturbation signals S can be recovered from the gene expression data by using RPCA. To discover the differentially expressed genes associated with special biological progresses or functions, the scheme is given as follows. Firstly, the matrix D of expression data is decomposed into two adding matrices A and S by using RPCA. Secondly, the differentially expressed genes are identified based on matrix S. Finally, the differentially expressed genes are evaluated by the tools based on Gene Ontology. A larger number of experiments on hypothetical and real gene expression data are also provided and the experimental results show that our method is efficient and effective.

Background

One of the challenges in current molecular biology is how to find the genes associated with key cellular processes. Up to date, using microarray technology, these genes associated with a special biological process have been detected more comprehensively than ever before.

DNA microarray technology has enabled high-throughput genome-wide measurements of gene transcript levels [1, 2], which is promising in providing insight into biological processes involved in gene regulation [3]. It allows researchers to measure the expression levels of thousands of genes simultaneously in a microarray experiment. Gene expression data usually contain thousands of genes (sometimes more than 10,000 genes), and yet only a small number of samples (usually less than 100 samples). Gene expression is believed to be regulated by a small number of factors (compared to the total number of genes), which act together to maintain the steady-state abundance of specific mRNAs. Some of these factors could represent the binding of one (or more) transcription factor(s) (TFs) to the promoter region(s) of the gene [4]. So, it can be assumed that the genes associated with a biological process are influenced only by a small subset of TFs [5]. Although the expression levels of thousands of genes are measured simultaneously, only a small number of genes are relevant to a special biological process. Therefore, it is important how to find a set of genes that are relevant to a biological process.

Various methods have been proposed for identifying differentially expressed genes from gene expression data. These methods can be roughly divided into two categories: univariate feature selection (UFS) and multivariate feature selection (MFS). The commonest scheme of UFS is utilized as follows. First, a score for each gene is independently calculated. Then the genes with high scores were selected [6]. The main virtues of UFS are simple, interpretable and fast. However, UFS has some drawbacks. For example, if each gene is independently selected from gene expression data, a large part of the mutual information contained in the data will be lost.

To overcome the drawbacks of UFS, the methods of MFS use all the features simultaneously to select the genes. So far, many mathematical methods for MFS, such as principal component analysis (PCA), independent component analysis (ICA), nonnegative matrix factorization (NMF), lasso logistic regression (LLR) and penalized matrix decomposition (PMD), have been devised to analyze gene expression data. For example, Lee et al. applied PCA to analyze gene expression data [7]. Liu et al. proposed a method of weighting principal components by singular values to select characteristic genes [8]. Probabilistic PCA was used to analyze gene expression data by Nyamundanda et al. [9]. Huang et al. used ICA to analyze gene expression data [10]. NMF was used to select the gene by Zheng et al. [11]. Liu et al. used LLR to select characteristic gene using gene expression data [12]. In [13], Witten et al. proposed penalized matrix decomposition (PMD), which was used to extract plant core genes by Liu et al. [14]. However, the brittleness of these methods with respect to grossly corrupted observations often puts its validity in jeopardy.

Recently, a new method for matrix recovery, namely robust PCA, has been introduced in the field of signal processing [15]. The problem of matrix recovery can be described as follows, assume that all the data points are stacked as column vectors of a matrix $D$, and the matrix (approximately) have low rank:

$$D=A_0+S_0,$$

(1)

where $A_0$ has low-rank and $S_0$ is a small perturbation matrix. The robust PCA proposed by Candes et al. can recover a low-rank matrix $A_0$ from highly corrupted measurements $D$[15]. Here, the entries in $S_0$ can have arbitrary large magnitude, and their support is assumed to be sparse but unknown.

Although the method has been successfully applied to model background from surveillance video and to remove shadows from face images [15], it's validity for gene expression data analysis is still need to be studied. The gene expression data all lie near some low-dimensional subspace [16], so it is natural to treat these genes data of non-differential expression as approximately low rank. As mentioned above, only a small number of genes are relevant to a biological process, so these genes with differential expression can be treated as sparse perturbation signals.

In this paper, based on robust PCA, a novel method is proposed for identifying differentially expressed genes. The differentially and non-differentially expressed genes are treated as perturbation signals $S$ and low-rank matrix $A$. Firstly, the matrix $D$ of expression data is decomposed into two adding matrices $A$ and $S$ by using RPCA. Secondly, the differentially expressed genes are discovered according to the matrix $S$. Finally, the differentially expressed genes are evaluated by the tools based on Gene Ontology. The main contributions of our work are as follows: firstly, it proposes, for the first time, the idea and method based on RPCA for discovery of differentially expressed genes; secondly, it provides a larger number of experiments of gene selection.

Methods

The definition of Robust PCA (RPCA)

This subsection simply introduces robust PCA (RPCA) proposed by Candes et al. [15]. Let ${||A||}_{*}:={\sum }_i{\sigma }_i\left(A\right)$ denote the nuclear norm of the matrix $A$, that is, the sum of its singular values, and let ${||S||}_1:={\sum }_{ij}\left|S_{ij}\right|$ denote the $L_1$-norm of $S$. Supposing that $D$ denotes the observation matrix given by Eq.(1), RPCA solves the following optimization problem:

$$\begin{array}{cc}\hfill \mathsf{\text{minimize}}\hfill & \hfill {||A||}_{*}+\lambda {||S||}_1\hfill \\ \hfill \mathsf{\text{subject to}}\hfill & \hfill D=A+S\hfill \end{array},$$

(2)

where $\lambda$ is a positive regulation parameter. Due to the ability to exactly recover underlying low-rank structure in the data, even in the presence of large errors or outliers, this optimization is referred to as Robust Principal Component Analysis (RPCA).

For the RPCA problem Eq.(2), a Lagrange multiplier Y is introduced to remove the equality constraint. According to [17], the augmented Lagrange multiplier method on the Lagrangian function can be applied:

$$L\left(A,S,Y,\mu \right)={||A||}_{*}+\lambda {||S||}_1+〈Y,D-A-S〉+\frac{\mu }{2}{||D-A-S||}_F^2,$$

(3)

where $\mu$ is a positive scalar and ${||•||}_F^2$ denotes the Frobenius norm. Lin et al. gave a method for solving the RPCA problem, which is referred to as the inexact ALM (IALM) method [17]. The details of this algorithm can be seen in [17].

The RPCA model of gene expression data

Considering the matrix $D$ of gene expression data with size $m\times n$, each row of $D$ represents the transcriptional responses of a gene in all the $n$ samples, and each column of $D$ represents the expression levels of all the $m$ genes in one sample. Without loss of generality, $m\gg n$, so it is a classical small-sample-size problem.

Our goal of using RPCA to model the microarray data is to identify these significant genes. As mentioned in Introduction, it is reasonable to view the significant genes as sparse signals, so the differential ones are viewed as the sparse perturbation signals $S$ and the non-differential ones as the low-rank matrix $A$. Consequently, the genes of differential expression can be identified according to the perturbation signals $S$. The RPCA model of microarray data is shown in Figure 1. The white and yellow blocks denote zero and near-zero in Figure 1. Red and blue blocks denote the perturbation signals. As shown in Figure 1, the matrix $S$ of differentially expressed genes (red or blue block) can be recovered from the matrix $D$ of gene expression data.

Figure 1

The RPCA model of microarray data. The white and yellow blocks denote zero and near-zero in this figure. Red and blue blocks denote the perturbation signals.

Suppose the matrix decomposition $D=A+S$ has been done by using RPCA. By choosing the appropriate parameter $\lambda$, the sparse perturbation matrix $S$ can be obtained, i.e., most of entries in $S$ are zero or near-zero (as white and yellow blocks shown in Figure 1). The genes corresponding to non-zero entries can be considered as ones of differential expression.

Identification of differentially expressed genes

After observation matrix has been decomposed by using RPCA, sparse perturbation matrix $S$ can be obtained. Therefore the differentially expressed genes can be identified according to sparse matrix $S$.

Denote the perturbation vector associated with $i$-th sample as:

$$S_i={\left[s_{1i},s_{2i},\cdots ,s_{mi}\right]}^T,\begin{array}{c}\hfill i=1,\cdots ,n\hfill \end{array}.$$

(4)

Then the sparse matrix $S$ can be expressed as follows:

$$S=\left[S_1,\cdots ,S_n\right].$$

(5)

So the sparse matrix $S$ can be denoted as:

$$S=\left[\begin{array}{cccc}\hfill s_{11}\hfill & \hfill s_{12}\hfill & \hfill \cdots \hfill & \hfill s_{1n}\hfill \\ \hfill s_{21}\hfill & \hfill s_{22}\hfill & \hfill \cdots \hfill & \hfill s_{2n}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill s_{m1}\hfill & \hfill s_{m2}\hfill & \hfill \cdots \hfill & \hfill s_{mn}\hfill \end{array}\right].$$

(6)

The differentially expressed genes can be classified into two categories: up-and down-regulated ones [18], which are reflected by the positive and negative entries in the sparse matrix $S$. Here, to discover the differentially expressed genes, only the absolute value of entries in $S$ need to be considered. Then the following two steps are executed: firstly, the absolute values of entries in the sparse matrix $S$ are find out; secondly, to get the evaluating vector $\tilde{S}$, the matrix is summed by rows. Mathematically, it can be expressed as follows:

$$\tilde{S}={\left[\begin{array}{ccc}\hfill {\displaystyle \sum _{i=1}^n}\left|s_{1i}\right|\hfill & \hfill \cdots \hfill & \hfill {\displaystyle \sum _{i=1}^n}\left|s_{mi}\right|\hfill \end{array}\right]}^T.$$

(7)

Consequently, to obtain the new evaluating vector $\mathit{Ŝ}$, which is sorted in descending order. Without loss of generality, suppose that the first $c_1$ entries in $\mathit{Ŝ}$ are non-zero, that is,

$$\mathit{Ŝ}={\left[\begin{array}{cc}\hfill \begin{array}{ccc}\hfill {\mathit{ŝ}}_1,\hfill & \hfill \cdots ,\hfill & \hfill {\mathit{ŝ}}_{c_1},\hfill \end{array}\hfill & \hfill \underset{m-c_1}{\underbrace{\begin{array}{ccc}\hfill 0,\hfill & \hfill \cdots ,\hfill & \hfill 0\hfill \end{array}}}\hfill \end{array}\right]}^T.$$

(8)

Generally, the larger the element in $\mathit{Ŝ}$ is, the more differential the gene is. So, the genes associated with only the first $num$ ($num\le c_1$) entries in $\mathit{Ŝ}$ are picked out as differentially expressed ones.

Results and discussion

This section gives the experimental results. Firstly, in the first subsection, hypothetical data are exploited to clarify how to set the parameter$\lambda$. Secondly, in the second subsection, our method is compared with the following methods on the real gene expression data of plants responding to abiotic stresses: (a) PMD method using the left singular vectors $\left\{u_k\right\}$ to identify the differentially expressed genes (proposed by Witten et al. [13]); (b) SPCA method using all the PCs of SPCA (proposed by Journée et al. [19]) to identify the differentially expressed genes. Finally, in the third subsection, the three methods are compared on the real gene expression data of colon tumor.

Experimental results on hypothetical data

Matrices randomly generated will be used for the simulation experiments. The true solution is denoted by the ordered pairs $\left(A^{*},S^{*}\right)$, which are generated by using the method in [17]. The rank-r matrix $A^{*}\in R^{m\times n}$ is generated as $A^{*}=L{R}^T$, where $L$ and $R$ are independent $m\times r$ and $n\times r$ matrices, respectively. Elements of $L$ and $R$ are i.i.d. Gaussian random variables with zero mean and unit variance. $S^{*}\in {\left\{-1,0,1\right\}}^{m\times n}$ is a sparse matrix whose support is chosen uniformly at random, and whose non-zero entries are i.i.d. uniformly in the space $R^{m\times n}$. $\mu$ denotes the sparse degree of matrix $S^{*}$, which is defined as the number of non-zero entries divided by the number of all the entries. The matrix $D=A^{*}+S^{*}$ is the input data to the RPCA. To evaluate the identification performance of RPCA, $Ac{c}_S$ denotes the recognition accuracy of matrix $S$, which is defined as follows.

$$Ac{c}_S=\frac{NumberofcorrectidentifiedentriesinS}{NumberofentriesinS},$$

(9)

where correct identified entries mean that the identified entries in $S$ approximately equal to the ones in $S^{*}$.

In [17, 20], a fixed regulation parameter $\lambda ={\left(c*\text{max}\left(m,n\right)\right)}^{-1/2}$ is used, where $c=1.0$. In order to clarify how to set $\lambda$, the following two different cases are considered: first, $m=n$; second, $m>n$, the small-size-sample problem.

Results while m=n

In this experiment, let $m=n=\begin{array}{ccc}\hfill 500,\hfill & \hfill 1000\hfill & \hfill \begin{array}{cc}\hfill or\hfill & \hfill 2000\hfill \end{array}\hfill \end{array}$, $rankn=\begin{array}{ccc}\hfill 0.05\hfill & \hfill or\hfill & \hfill 0.1\hfill \end{array}$, $\begin{array}{cc}\hfill \begin{array}{cc}\hfill \mu =0.05\hfill & \hfill or\hfill \end{array}\hfill & \hfill 0.1\hfill \end{array}$. Table 1 lists the recognition results with different $c$. As Table 1 listed, when $c=0.2$, the recognition accuracy $Ac{c}_S$ can be achieved above 90%. When $c\ge 0.3$, the matrix $S$ can be completely identified, i.e. $Ac{c}_S=\mathsf{\text{1}}00\%$.

Table 1 The recognition accuracy $Ac{c}_S$ with different $c$

Results while m>n

In this experiment, let $m=10000$,$rank=\begin{array}{ccc}\hfill 5\hfill & \hfill or\hfill & \hfill 10\hfill \end{array}$, $\begin{array}{cc}\hfill \begin{array}{cc}\hfill \mu =0.05\hfill & \hfill or\hfill \end{array}\hfill & \hfill 0.1\hfill \end{array}$ and $n$ increase from 10 to 100 with an interval 10. Table 2, 3, 4, 5 list the results. As tables 2 and 3 listed with $rank=5$, when $n\ge 20$, the recognition accuracy $Ac{c}_S$ can be achieved above 90%. As tables 4 and 5 listed with $rank=10$, when $n\ge 30$, the recognition accuracy $Ac{c}_S$ can be achieved above 90%. In words, to achieve the recognition accuracy $Ac{c}_S$ above 90%, $n$ must be equal to or larger than three times of $rank$ ($n\ge 3*rank$). As tables 2, 3, 4, 5 listed, by rows, the larger the number of column $n$ is, the higher the recognition accuracy $Ac{c}_S$ can be achieved.

Table 2 The recognition accuracy $Ac{c}_S$ with $rank=5$ and $\mu =0.05$

Table 3 The recognition accuracy $Ac{c}_S$ with $rank=5$ and $\mu =0.1$

Table 4 The recognition accuracy $Ac{c}_S$ with $rank=10$ and $\mu =0.05$

Table 5 The recognition accuracy $Ac{c}_S$ with $rank=10$ and $\mu =0.1$

Now, we investigate how different $c$ influences the recovery accuracy $Ac{c}_S$. For example, when $n=40$, Figure 2 shows the recognition accuracy $Ac{c}_S$ with different $c$. As shown in Figure 2, when $c=0.3$, the recognition of matrix $S$ can reach highest accuracy. With $c$ increasing, the recovery accuracy $Ac{c}_S$ drops. For example, when $c=1.0$, s3 and s4 are degraded to 90%.

Figure 2

The recognition accuracy of matrix $S$ with different $c$. s1 denotes the recognition accuracy series with $rank=5$ and $\mu =0.05$. s2 denotes the recognition accuracy series with $rank=5$ and $\mu =0.1$. s3 denotes the recognition accuracy series with $rank=10$ and $\mu =0.05$. s4 denotes the recognition accuracy series with $rank=10$ and $\mu =0.1$.

From these experiments, a conclusion can be drawn that when the optimal empirical value of $\lambda$ is given as: $\lambda ={\left(0.3*\text{max}\left(m,n\right)\right)}^{-1/2}$, where the size of data matrix $D$ is $m\times n$, the highest identification accuracy $Ac{c}_S$ can be obtained.

Experimental results on gene expression data of plants responding to abiotic stresses

Along with other two state-of-the-art methods, namely PMD and SPCA, used as comparison, three methods, including RPCA, are used to discover the differentially expressed genes responding to abiotic stresses based on real gene expression data.

Data source

The raw data were downloaded from NASCArrays [http://affy.arabidopsis.info/] [21], which include two classes: roots and shoots in each stress. The reference numbers are: control, NASCArrays-137; cold stress, NASCArrays-138; osmotic stress, NASCArrays-139; salt stress, NASCArrays-140; drought stress, NASCArrays-141; UV-B light stress, NASCArrays-144; heat stress, NASCArrays-146. Table 6 lists the sample number of each stress type. There are 22810 genes in each sample. The data are adjusted for background of optical noise using the GC-RMA software by Wu et al. [22] and normalized using quartile normalization. The results of GC-RMA are gathered in a matrix for further processed.

Table 6 The sample number of each stress type in the raw data

Selection of the parameters

In this paper, for PMD method, the $L_1$-norm of $u$ is taken as the penalty function, i.e. ${||u||}_1\le {\alpha }_1$. Because of $1\le {\alpha }_1\le \sqrt{m}$, let ${\alpha }_1=\alpha *\sqrt{m}$, where $1\sqrt{m}\le \alpha \le 1$. For simplicity, let $p=1$, that is, only one factor is used. The results with $L_1$-norm (${||z||}_1={\sum }_i\left|z_i\right|$) and $L_0$-norm (${||z||}_0$, i.e. the number of nonzero coefficients, or cardinality) penalty in SPCA are similar, which is also shown in [19], so $L_0$-norm penalty and the parameter $\gamma$ are taken in SPCA. For a fair comparison, 500 genes are roughly selected by these methods via choosing appropriate parameters $\alpha$ and $\gamma$ of the two methods, PMD and SPCA, which are listed in Table 7 for different data set. As the first subsection of experiments mentioned, while $c=0.3$, RPCA gives the optimization results. Then, according to methods section, the first 500 genes are selected.

Table 7 The values of $\alpha$ and $\gamma$ on different data set

Gene ontology (GO) analysis

Recently, many tools have been developed for the functional analysis of large lists of genes [23, 24]. Most of them focus on the evaluation of Gene Ontology (GO) annotations. GOTermFinder is a web-based tool that finds the significant GO terms shared among a list of genes, helping us discover what these genes may have in common. The analysis of GOTermFinder provides significant information for the biological interpretation of high-throughput experiments.

In this subsection, the genes identified by these methods, RPCA, PMD and SPCA, are sent to GOTermFinder [24], which is publicly available at http://go.princeton.edu/cgi-bin/GOTermFinder. Its threshold parameters are set as following: minimum number of gene products = 2 and maximum P-value = 0.01. Here, the key results are shown. Table 8 lists the terms of Response to abiotic stimulus (GO:0009628), whose background frequency in TAIR set is 1539/29556 (5.2%). Response to abiotic stimulus is the ancestor term of all the abiotic stresses. In GOTermFinder, a p-value is calculated using the hyper-geometric distribution, its details can be seen in [24]. Sample frequency denotes the number of genes hit in the selected genes, such as 107/500 denotes 107 genes associated with the GO term in 500 ones selected by these methods. As listed in Table 8, all the three experimented methods, PMD, SPCA and RPCA, can extract the significant genes with very lower P-value, as well as very higher sample frequency. In Table 8, the superior results are in bold type. In the twelve items, there is only one of them (cold on root) that PMD is equal to our method. In other items, our method is superior to SPCA and PMD.

Table 8 Response to abiotic stimulus (GO:0009628)

Figure 3 shows the sample frequency of response to abiotic stimulus (GO:0009628) given by the three methods. From Figure 3(a), RPCA method outperforms others in all the data sets of shoot samples with six different stresses. Figure 3(b) shows that only in cold-stress data set of root samples, PMD is equal to our method and they are superior to SPCA. In other data sets, our method is superior to the others.

Figure 3

The sample frequency of response to abiotic stimulus.

The characteristic terms are listed in Table 9, in which the superior results are in bold type. As listed in Table 9, PMD method outperforms SPCA and our method in three items, such as drought in shoot, salt in root and cold in root, among the whole items. However, it shows that, on one of the twelve items (osmotic in shoot), our method has the same competitive result as PMD, while both methods are superior to SPCA. In other eight items, our method excels PMD and SPCA methods. In addition, on all the characteristic items, our method has superiority over SPCA.

Table 9 Characteristic terms selected from GO by algorithms

From the results of experiments, it can be concluded that our method is efficient and effective.

Experimental results on colon data

The three methods, SPCA, PMD and RPCA, are compared on colon cancer data set [25]. Colon cancer is the fourth most common cancer for males and females and the second most frequent cause of death.

Data source

The raw data were downloaded from http://genomics-pubs.princeton.edu/oncology/affydata/I2000.html, which include gene expression levels for 2000 gene and contain 40 tumor and 22 normal tissue samples.

Selection of the parameters

In this subsection, for PMD method, the $L_1$-norm of $u$ is taken as the penalty function, i.e. ${||u||}_1\le {\alpha }_1$. Let ${\alpha }_1=\alpha *\sqrt{m}$, where $1\sqrt{m}\le \alpha \le 1$. For SPCA method, let $p=1$, that is, only one factor is used. $L_0$-norm penalty and the parameter $\gamma$ are taken in SPCA. For a fair comparison, 100 genes are roughly selected by these methods via choosing appropriate parameters. PMD and SPCA use the parameters $\alpha =\mathsf{\text{0}}\mathsf{\text{.2351}}$ and $\gamma =\mathsf{\text{0}}\mathsf{\text{.4306}}$ on colon data set, respectively. As the first subsection of experiments mentioned, while $c=0.3$, RPCA gives the optimization results. Then, according to Methods section,the first 100 genes are selected using our method.

Gene ontology (GO) analysis

The genes identified by these methods, RPCA, PMD and SPCA, are evaluated by using AmiGO [26]. Its threshold parameters are set as following: minimum number of gene products = 2 and maximum P-value = 0.1. A number of lines of evidence suggest that immune, stimulus and tumor have affinity, so Table 10 lists the key results: the terms of Response to stimulus (GO:0050896) and Immune system process (GO:0002376). As listed in Table 10, RPCA outperforms its competitive methods with higher sample frequency.

Table 10 Characteristic terms selected from GO on colon data

Function analysis

Table 11 lists the top 30 genes selected by using RPCA. To further study the biology functions of the selected genes, we also make the network analysis of the top 100 genes selected by our algorithm using the GeneMANIA tool [27] on the Web sitehttp://genemania.org/. The result is listed in Table 12. From the table it can be seen that there are 215 genes of this chip participating in the cytokine-mediated signalling pathway, in which there are 21 genes discovered by our method. This pathway has the lowest p-value. It is considered as the most probable pathway with these top 100 genes. Recent findings also indicate that cytokine receptors can regulate immune cell functions by transcription-independent mechanisms [28]. Some other pathways with the most significance are also listed in Table 12.

Table 11 The top 30 genes of colon data selected by RPCA

Table 12 Pathway analysis of the top 100 genes selected by RPCA on colon data

Conclusion

In this paper, a novel RPCA-based method of discovering differentially expressed genes was proposed. It combined RPCA and sparsity of gene differential expression to provide an efficient and effective approach for gene identification. Our method mainly included the following two steps: firstly, the matrix $S$ of differential expression was discovered from gene expression data matrix by using robust PCA; secondly, the differentially expressed genes were discovered according to matrix $S$. The experimental results on real gene data showed that our method outperformed the other state-of-the-art methods. In future, we will focus on the biological meaning of the differentially expressed genes.