Discovering functional impacts of miRNAs in cancers using a causal deep learning model
Micro-RNAs (miRNAs) play a significant role in regulating gene expression under physiological and pathological conditions such as cancers. However, it remains a challenging problem to discover the target messenger RNAs (mRNAs) of a miRNA in a data driven fashion. On one hand, sequence-based methods for predicting miRNA targets tend to make too many false positive calls. On the other hand, analyzing expression correlation between miRNAs and mRNAs cannot establish whether relationship between a pair of correlated miRNA and mRNA is causal.
In this study, we designed a deep learning model, referred to as miRNA causal deep net (mCADET), which aims to explicitly represent two types of statistical relationships between miRNAs and mRNAs: correlation resulting from confounded co-regulation and correlation as a result of causal regulation. The model utilizes a deep neural network to simulate transcription mechanism that leads to co-expression of miRNA and mRNA, and, in addition, it also contains directed edges from miRNAs to mRNAs to capture causal relationships among them.
We trained the mCADET model using pan-cancer miRNA and mRNA data from The Cancer Genome Atlas (TCGA) project to investigate mechanism of co-expression and causal interactions between miRNAs and mRNAs. Quantitative analyses of the results indicate that the mCADET significantly outperforms conventional deep learning models when modeling combined miRNA and mRNA expression data, indicating its superior capability of capturing the high-order statistical structures in the data. Qualitative analysis of predicted targets of miRNAs indicate that predictions by mCADET agree well with existing knowledge. Finally, the predictions by mCADET have a significantly lower false discovery rate and better overall accuracy in comparison to sequence-based and correlation-based methods when comparing to experimental results.
The mCADET model can simultaneously infer the states of cellular signaling system regulating co-expression of miRNAs and mRNAs, while capturing their causal relationships in a data-driven fashion.
KeywordsDeep learning Causal discovery miRNA and mRNA
Deep belief network
miRNA causal deep net
Positive predictive value
Restricted boltzmann machines
The cancer genome atlas
Regulated expression of miRNAs and their regulatory functions
Micro-RNAs are a class of small RNAs, about 22 nucleotides in length and involved in post-transcriptional and translation regulation of gene-expression either by direct cleavage of mRNA or translational repression . In the last decade, studies show that the dysfunction/dysregulation of certain miRNAs are involved in the development and progression of many diseases. Particularly, the role of miRNA in cancer has drawn attention in last decade. Studies demonstrated that the dysregulation of miRNA expression could lead to human cancers . The mechanisms include amplification or deletion of miRNA genes , abnormal transcriptional control of miRNAs , dysregulated epigenetic changes [5, 6] and deficiencies in the miRNA biogenesis machinery . The loss of tumor suppressor miRNAs or overexpression of oncogenic miRNAs can lead to breast cancer tumorigenesis or metastasis [8, 9].
To gain a better understanding of the roles of miRNAs in normal biological processes and in the development of disease, it is important to accurately identify which genes are targeted by each miRNA. Since it is infeasible to perform biological experiments for such a large number of miRNAs, computational methods play an important role in studying miRNA, and numerous computational methods have been developed for predicting targets of miRNAs. To predict the interaction between miRNA and mRNA, many tools have been developed using different algorithms [10, 11, 12, 13, 14], although two main approaches dominate the field. One approach is the sequence-based miRNA target prediction, which models the complementary sequence similarity between miRNA and mRNA and structural stability of the putative duplex to predict whether a mRNA is a target of a miRNA [12, 15, 16, 17]. Give a miRNA sequence dataset (e.g., miRBase , StarBase ), sequence-based models can be used to scan whole mRNA transcriptome to predict potential targets. However, these methods have been shown to have a high rate of false positives and false negatives . This is mainly because sequence similarity is not sufficient to predict the folding of RNA duplexes and their interaction with the proteins involved in miRNA-mediated regulation [19, 20].
Another common approach of studying miRNA and mRNA relationship is the correlation-based miRNA target prediction. Based on miRNA and mRNA expression data collected from a cohort of biological samples, correlation-based methods search for anti-correlation relationships between pairs of miRNA and mRNA as potential regulator-target pairs. Different databases based on correlation analyses are available, e.g., mirCox . However, an anti-correlation between a pair of miRNA and mRNA does not necessarily represent a causal relationship. It is not uncommon that a signaling pathway may simultaneously regulate expression of a miRNA and a set of mRNAs, which may lead to confounded correlation. As it is often said: correlation does not entail causality. Thus more rigorous causal inference methods are needed to infer the causal relationships between miRNA and mRNA.
In this study, we present a novel method of studying the statistical relationships between mRNAs and miRNAs by analyzing large-scale data from TCGA. Our method integrates two complementary machine learning frameworks: deep learning and causal inference. Our model, referred to as miRNA causal deep net (mCADET), consists of deep neural network that can capture the transcriptomic machine that controls expression of both miRNA and mRNAs to capture the statistical structure resulting from co-regulation, and, in addition, it includes directed edges from miRNA to mRNA to capture the potential causal relationships between miRNA and mRNA. We show that this integrative approach can significantly outperform the sequence-based and correlation-based methods in predicting targets of miRNAs.
The miRNA and mRNA expression data were obtained and downloaded from TCGA consortium website (https://cancergenome.nih.gov/). For the breast cancer (BRCA), 1218 mRNA sequencing samples were downloaded with 20,531 mRNAs, and 1701 miRNA sequencing samples were downloaded with 1918 microRNAs, which includes duplicate and triplicate samples. We further identified tumor samples with both mRNA and miRNA measurements. We discretized (binary) the expression data by comparing the expression values of mRNA and miRNAs from tumor samples with those from normal samples profiled by TCGA using the 3-fold change. Finally, we merged the miRNA and mRNA dataset into a 757 ×22,449 (samples × combined miRNA and mRNA) binary matrix for model training.
Several open-resource miRNA-target databases are used in this paper including miRTarBase (an experimentally validated database) , and miRDB (a sequence based prediction database and the prediction tool used is MirTarget V3) [23, 24]. TMREC  and TTRUST  were used to look up the TF-miRNA and TF-mRNA interactions separately to help find the common TF regulating both a particular miRNA and a particular mRNA.
An autoencoder uses multiple layers of latent variables (hidden nodes) to capture compositional statistical structures in a distributed manner, such that each layer captures the structure of different degrees of abstraction. As shown in Fig. 1b, an autoencoder contains one visible (input) layer and one or more hidden layers. To efficiently train the autoencoder, we treat it as a series of two-layered restricted Boltzmann machines (RBM) stacked on top of each other . The inference of the states of hidden node and learning of model parameters are performed by learning the RBM stacks in a bottom-up fashion, which is followed by a global optimization of generative parameters using the back-propagation algorithm . As shown in our previous studies [29, 30], autoencoders are capable of capturing signals regulating gene expressions under different settings. For example, the first hidden layer could represent the transcription factors (TFs).
To further capture the causal relationships between miRNAs and mRNAs, we designed a deep belief network (DBN) model containing directed edges from miRNA to mRNAs. We pre-train the model by treating the observed variables (miRNA and mRNA) and first hidden layer above them as a semi-RBM. The semi-RBM between the first two layers allows the edges from miRNA visible nodes to mRNA visible nodes to reflect the regulatory role of miRNA in gene expression. The layers above the second layer still use the traditional RBMs as shown in Fig. 1b. The followings show how the positive phase and negative phase perform. Under this setting, when the correlation between a pair of miRNA and mRNA are competently explained by co-regulation or causal edges, regularization techniques (discussed later) potentially constrain the model to pick one dominant mechanism to represent the correlation.
Semi-RBM positive phase
where vmrna & mirnai represents the binary state of ith visible unit of mRNA and miRNA; hj represents the state of jth hidden unit; bj represents the bias for the jth hidden unit; Wij represents the weight between the ith visible unit of mRNA and miRNA and the jth hidden unit.
Semi-RBM negative phase
where vmirna _ k represents the kth visible unit of miRNA; Wkj represents the weight between the kth visible unit of miRNA and the jth hidden unit; amirna _ k represents the bias for the kth visible unit of miRNA; vmrna _ o represents the oth visible unit of mRNA; amrna _ o represents the bias for the oth visible unit of mRNA; Woj represents the weight between oth visible unit of mRNA and the jth hidden unit; πok represents the weight between oth visible unit of mRNA and the kth visible unit of miRNA. cbind represents the combination of mRNA and miRNA for the same sample.
Semi-RBM parameter update
More details of the algorithm and pseudo code for training a traditional autoencoder were discussed in both literature and our previous work [29, 30]. The backpropagation process is the same as the standard one except that we update the visible units of miRNA using the hidden units in the first hidden layer only, but use both hidden units in the first hidden layer and the visible units of miRNA to update the visible units of mRNA.
The number of miRNA and mRNA training samples is limited compared with the dimension of miRNA and mRNA features. Therefore, we applied the techniques of regularization to the first two hidden layers to reduce the risk of overfitting. When we train a traditional RBM model, each hidden unit is fully connected to each observed unit and a non-zero weight is usually assigned to each pair of observed unit and hidden unit. However, in the real cases of cellular regulatory mechanism, the change in mRNA and miRNA expression is often induced by a small number of biological components such as TFs or pathways. This enables one to specify that only a certain percent of hidden units have a high probability to be set to 1 (“on”) by adding a penalization term to the optimization function. In this model, regularization was only added to the first hidden layer . During the RBM training within an autoencoder, the optimization of the sparse RBM minimizes the negative log-likelihood of the data with the addition of the regularization term .
We trained models under several configurations. We trained a deep belief network (DBN) model without edges from miRNAs to mRNAs, the mCADET model with edges from miRNAs to mRNAs but without regularization, and the mCADET model with edges from miRNAs to mRNAs and with regularization (different sparsity ratios), and finally a the mCADET model with random permutation of miRNAs and mRNAs across tumors where the statistical relationships between miRNAs and mRNAs were fully disrupted. We compared the reconstruction errors for these models to choose the best model. The reconstruction errors are the difference between the raw input data and the reconstructed input predicted from the model. The model with the best reconstruction error is chosen to conduct the evaluations of biological representations.
We used the experimental results from miRTarBase  as “gold standard” to evaluate the prediction by our models and those by sequence-based and correlation-based methods. The accuracies of regulatory miRNA-mRNA pairs predicted by mCADET models were compared with ones predicted by correlation-based and sequence-based analysis separately. For the correlation analysis, we run pair-wise linear regression on our dataset to identify pairs with statistically significance, corrected by false discovery. For the sequence-based analysis, we used the predicted miRNA-mRNA interaction acquired from the miRDB database . Since it is difficult to assess the true negative rate, we mainly concentrate on evaluating models’ capabilities in identifying the true positives, i.e., the recall and positive predictive value (PPV) of each method.
To test whether the PPVs for two methods are significantly different from each other, we calculate the z-score as follows,
where p represents the PPV for mCADET-based analysis, correlation-based analysis and sequence-based analysis separately; n represents the number of predicted mRNAs. The z value between mCADET versus sequence-based model and the z value between mCADET versus correlation-based analysis were calculated separately. To quantitatively evaluate the difference between precision and recall for two groups, we calculated the F1 score that evaluates weighted average of precision and recall .
Finally, we also validated predicted mRNA-miRNA pairs by checking the agreement with literatures.
The reconstruction errors for experimentally validated miRNA-targets only
miR374a (414 targets)
miR15b (265 targets)
miR190 (428 targets)
The following interesting observations are noteworthy: 1) Adding causal edges between miRNA and mRNA improves the capability of a model to capture the overall statistical structures of data. Indicating that these potential causal edges enabled the model to capture the statistical relationships between miRNA and mRNAs that cannot be fully explained by co-regulation. 2) Right amount of regularization enhances the capability of model to capture the statistical structures. This is reflected by the fact that model1_semi-spa_0.2 outperforms models without regularization (model2_semi_no_spa) and the model with too few parameters (model3_semi_spa_0.1 that only allow 10% of the hidden nodes to be active).
Statistical evaluation of predicted miRNA-mRNA interaction
In the deep learning model, the weights of the direct edges from miRNAs to mRNAs reflect the strength of interaction between miRNA and mRNA. Therefore, we used the weights trained by the deep learning models to perform the miRNA-mRNA interaction analysis. We only keep the top 5% absolute weights for each miRNA and get a corresponding mRNA list.
The sensitivity, positive predictive value (PPV) and F1 score of mCADET-based, sequence-based and correlation-based prediction for miRNA-374a separately
We used z-score to test the significant difference of PPVs for each two analyses. After the calculation, the z value and p-value between model-based analysis and correlation-based analysis is 2.67 and 0.045 respectively. The z value and p-value between model-based analysis and sequence-based analysis is 3.25 and 0.038 respectively. Both of z scores are bigger than 1.96  and p-values are less than 0.05, which shows that each of two groups is significantly different from each other. We could also conclude from Table 2 that the performance of our mCADET model is better than sequenced-based and correlation-based model. Besides, the sensitivity of the baseline DBN is 0.61 compared with the mCADET 0.64 and the PPV of the baseline DBN is 0.28 compared with the mCADET 0.43.
mCADET provides insights of different mechanisms for correlation between miRNAs and mRNAs
As shown in previous sections, adding causal edges in mCADET model can enhance the capability of the model to capture the overall statistical.
Structures including both miRNAs and mRNAs. Comparison to experimentally validated targets indicates that mCADET is more likely to capture the causal relationships between miRNAs and mRNAs. We further examined the examples that strong correlations between miRNAs and mRNAs are explained by different mechanisms in the results learned from the mCADET model.
Interaction network of miRNA-targets
In addition to detect common targets of a miRNA family, our model can also detect the common functional impacts of distinct miRNAs. For example, our model can detect that CCND1 is the shared target of miR-17 and miR-20a . More such relationships can be found in a broader analysis of our results, which are not shown in Fig. 5.
Literature-based evaluation of predicted miRNA-mRNA interactions
Examples of predicted miRNA-mRNA pairs validated by literature
Invasion and metastasis
MiR-146a suppresses tumor growth and progression by targeting EGFR pathway and in ap-ERK-dependent manner in castration-resistant prostate cancer
MiR-146b-5p suppresses EGFR expression and reduces in vitro migration and invasion of glioma
miR-335 orchestrates cell proliferation, migration and differentiation in human mesenchymal stem cells
Cell cycle, cellular proliferation
The miR-17-5p microRNA is a key regulator of the G1/S phase cell cycle transition
Cell cycle, cellular proliferation
MicroRNAs MiR-17, MiR-20a, and MiR-106b act in concert to modulate E2F activity on cell cycle arrest during neuronal lineage differentiation of USSC
MicroRNA-374a activates Wnt/ß-catenin signaling to promote breast cancer metastasis
MicroRNA-374b Suppresses Proliferation and Promotes Apoptosis in T-cell Lymphoblastic Lymphoma by Repressing AKT1 and Wnt-16
Cellular proliferation, survival and growth
MicroRNA-374a activates Wnt/ß-catenin signaling to promote breast cancer metastasis
Increased miR-374b promotes cell proliferation and the production of aberrant glycosylated lgA1 in B cells of lgA nephropathy
MicroRNAs play a significant role in regulating gene expression under physiological and pathological conditions. In particular, genomic alterations (amplification/deletion) of miRNAs in cancers have significant impacts on cancer development, disease progression, and therapy responses [38, 39, 40]. Thus, revealing the functional impacts of miRNAs in cancer will advance cancer biology. As shown in this report, previous methods of identifying targets of miRNAs have significant limitations. By combining deep learning and causal inference, the reported mCADET model ac-hieved significantly identifying targets of miRNAs. Particularly, the improved PPV will convince cancer biologists to carry out validation experiments with much high confidence, thus helping to advance cancer biology.
The reported mCADET model is motivated by biological insights of the data related to miRNAs and mRNAs. It combines the strength of deep learning and causal inference in solving this important biological problem. The superior performance of the model reflects the importance of integrating biological insights with advance machine learning technology. Possible future improvement of the model includes (but not necessarily limited to) combining genomic alteration data to map hidden nodes to concrete biological entities as we did in mining yeast gene expression data .
In this study, we investigated the utility of the mCADET model to simultaneously infer the states of cellular signaling system regulating co-expression of miRNAs and mRNAs, while capturing their causal relationships in a data-driven fashion. This model can be used by miRNA researchers to systematically search for miRNAs that play significant roles in cancers and understand their disease mechanism which, we anticipate, will make significant advances in cancer biology, beyond what are reported here.
The authors would like to thank Mr. Sanghoon Lee for discussion and suggestions.
This work and publication have been partially supported by the U54HG008540 and R01LM012011.
Availability of the data and materials
The Matlab code is available upon request.
About this supplement
This article has been published as part of BMC Medical Genomics Volume 11 Supplement 6, 2018: Proceedings of the 29th International Conference on Genome Informatics (GIW 2018): medical genomics. The full contents of the supplement are available online at https://bmcmedgenomics.biomedcentral.com/articles/supplements/volume-11-supplement-6.
Conceived and designed the experiments: LC, XL; Performed the experiments: LC; Analyzed the data: LC; Wrote the paper: LC, XL. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 5.Iorio MV, Croce CM. MicroRNA dysregulation in cancer: diagnostics, monitoring and therapeutics. A comprehensive review (vol 4, pg 143, 2012). Embo Mol Med. 2017;9(6):852–2.Google Scholar
- 15.Riffo-Campos AL, Riquelme I, Brebi-Mieville P. Tools for sequence-based miRNA target prediction: what to choose? Int J Mol Sci. 2016;17(12).Google Scholar
- 20.Li X, Quon G, Lipshitz HD, Morris Q. Predicting in vivo binding sites of RNA-binding proteins using mRNA secondary structure. Rna-a Publication of the Rna. Society. 2010;16(6):1096–107.Google Scholar
- 26.Han H, Shim H, Shin D, Shim JE, Ko Y, Shin J, Kim H, Cho A, Lee EKT, Kim H, et al. TRRUST: a reference database of human transcriptional regulatory interactions. Sci Rep. 2015;5.Google Scholar
- 27.Salakhutdinov R, Mnih A, Hinton GE. Restricted Boltzmann Machines for Collaborative Filtering. Proceedings of the 24th international conference on. Mach Learn. 2007:791–8.Google Scholar
- 29.Chen L, Cai C, Chen V, Lu X. Trans-species learning of cellular signaling systems with bimodal deep belief networks. Bioinformatics. 2015.Google Scholar
- 31.Chen LJ, Cai CH, Chen V, Lu XH. Learning a hierarchical representation of the yeast transcriptomic machinery using an autoencoder model. Bmc Bioinformatics. 2016;17.Google Scholar
- 32.H Lee CE, Ng AY. Sparse deep belief net model for visual area V2. In: NIPS: 2008; 2008.Google Scholar
- 35.Cloonan N, Brown MK, Steptoe AL, Wani S, Chan WL, Forrest AR, Kolle G, Gabrielli B, Grimmond SM. The miR-17-5p microRNA is a key regulator of the G1/S phase cell cycle transition. Genome Biol. 2008;9(8).Google Scholar
- 37.Mihailovich M, Bremang M, Spadotto V, Musiani D, Vitale E, Varano G, Zambelli F, Mancuso FM, Cairns DA, Pavesi G, et al. miR-17-92 fine-tunes MYC expression and function to ensure optimal B cell lymphoma growth. Nat Commun. 2015;6.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.