Lasso Granger Causal Models: Some Strategies and Their Efficiency for Gene Expression Regulatory Networks

  • Kateřina Hlaváčková-SchindlerEmail author
  • Sergiy PereverzyevJr.
Part of the Studies in Computational Intelligence book series (SCI, volume 538)


The detection of causality in gene regulatory networks from experimental data, such as gene expression measurements, is a challenging problem. Granger causality, based on a vector autoregressive model, is one of the most popular methods for uncovering the temporal dependencies between time series, and so it can be used for estimating the causal relationships between the genes in the network. The application of multivariate Granger causality to the networks with a big number of variables (genes) requires a variable selection procedure. For fighting with lack of informative data, the so called regularization procedures are applied. Lasso method is a well known example of such a procedure and the multivariate Granger causality method with the Lasso is called Graphical Lasso Granger method. It is widely accepted that the Graphical Lasso Granger method with an inappropriate parameter setting tends to select too many causal relationships, which leads to spurious results. In our previous work, we proposed a thresholding strategy for Graphical Lasso Granger method, called two-level-thresholding and demonstrated how the variable over-selection of the Graphical Lasso Granger method can be overcome. Thus, an appropriate thresholding, i.e. an appropriate choice of the thresholding parameter, is crucial for the accuracy of the Graphical Lasso Granger method. In this paper, we compare the performance of the Graphical Lasso Granger method with an appropriate thresholding to two other Lasso Granger methods (the regular Lasso Granger method and Copula Granger method) as well as to the method combining ordinary differential equations with dynamic Bayesian Networks. The comparison of the methods is done on the gene expression data of the human cancer cell line for a regulatory network of nineteen selected genes. We test the causal detection ability of these methods with respect to the selected benchmark network and compare the performance of the mentioned methods on various statistical measures. The discussed methods apply a dynamic decision making. They are scalable and can be easily extended to networks with a higher number of genes. In our tests, the best method with respect to the precision and computational cost turns out to be the Graphical Lasso Granger method with two-level-thresholding. Although the discussed algorithms were motivated by problems coming from genetics, they can be also applied to other real-world problems dealing with interactions in a multi-agent system.


Bayesian Network Adjacency Matrix Gene Regulatory Network Granger Causality Ordinary Differential Equation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The first author gratefully acknowledges the partial support by the research grant GACR 13-13502S of the Grant Agency of the Czech Republic (Czech Science Foundation).


  1. 1.
    Abramowitz, M., Stegun, I.A.: Handbook of mathematical functions with formulas, graphs, and mathematical tables, 9th printing. Dover, New York (1972)Google Scholar
  2. 2.
    Äijö, T., Lahdesmäki, H.: Learning gene regulatory networks from gene expression measurements using non-parametric molecular kinetics. Bioinformatics 25(22), 2937–2944 (2009)CrossRefGoogle Scholar
  3. 3.
    Arnold, A., Liu, Y., Abe, N.: Temporal causal modeling with graphical Granger methods. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2007)Google Scholar
  4. 4.
    Bahadori, T., Y. Liu, Y.: An examination of large-scale Granger causality inference. SIAM Conference on Data Mining (2013)Google Scholar
  5. 5.
    Bansal, M., Della Gatta, G.: Inference of gene regulatory networks and compound mode of action from time course gene expression profiles. Bioinformatics 22, 815822 (2006)CrossRefGoogle Scholar
  6. 6.
    Bansal, M., Belcastro, V., Ambesi-Impiombato, A., di Bernardo, D.: How to infer gene networks from expression profiles. Mol. Syst. Biol. 3, 78 (2007)CrossRefGoogle Scholar
  7. 7.
    Barenco, M., et al.: Ranked prediction of p53 targets using hidden variable dynamic modeling. Genome Biol. 7, R25 (2006)CrossRefGoogle Scholar
  8. 8.
    Bauer, F., Reiß, M.: Regularization independent of the noise level: an analysis of quasi-optimality. Inverse Probl. 24, 5 (2008)Google Scholar
  9. 9.
    Biological General Repository for Interaction Datasets, Biogrid 3.2Google Scholar
  10. 10.
    Cao, J., Zhao, H.: Estimating dynamic models for gene regulation networks. Bioinformatics 24, 1619–1624 (2008)CrossRefGoogle Scholar
  11. 11.
    Caraiani, P.: Using complex networks to characterize international business cycles. PLoS ONE 8(3), 58109 (2013)CrossRefGoogle Scholar
  12. 12.
    Cooper, G.F.: The computational complexity of probabilistic inference using Bayesian belief networks. Artif. Intell. 42, 393–405 (1990)CrossRefzbMATHGoogle Scholar
  13. 13.
    Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004)CrossRefzbMATHGoogle Scholar
  14. 14.
    Ebert-Uphoff, I., Deng, Y.: Causal discovery for climate research using graphical models. J. Clim. 25, 5648–5665 (2012)CrossRefGoogle Scholar
  15. 15.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer Academic, Dordrecht (1996)CrossRefzbMATHGoogle Scholar
  16. 16.
    Fornasier, M.: Theoretical Foundations and Numerical Methods for Sparse Recovery. de Gruyter, Berlin (2010)CrossRefzbMATHGoogle Scholar
  17. 17.
    Fujita, A., Sato, J.R., Garay-Malpartida, H.M., Yamaguchi, R., Miyano, S., Ferreira, C.E.: Modeling gene expression regulatory networks with the sparse vector autoregressive model. BMC Syst. Biol. 1, 37 (2007)CrossRefGoogle Scholar
  18. 18.
    Granger, C.W.J.: Investigating causal relations by econometric and cross-spectral methods. Econometrica 37, 424–438 (1969)CrossRefGoogle Scholar
  19. 19.
    Grasmair, M., Haltmeier, M., Scherzer, O.: Sparse regularization with \(l^{q}\) penalty term. J. Inverse Probl. 24(5), 13 (2008)MathSciNetGoogle Scholar
  20. 20.
    Hasings, C., Mosteller, F., Tukey, J.W., Winsor, C.P.: Low moments for small samples: a comparative study of order statistics. Ann. Math. Stat. 18, 413–426 (1947)CrossRefGoogle Scholar
  21. 21.
    Hlaváčková-Schindler, K., Bouzari, H.: Granger Lasso causal models in high dimensions: application to gene expression regulatory networks, In: The Proceedings of EVML/PKDD 2013, SCALE, Prague (2013)Google Scholar
  22. 22.
    Jensen, F.V.: An Introduction to Bayesian Networks. UCL Press, London (1996)Google Scholar
  23. 23.
    Kindermann, S., Neubauer, A.: On the convergence of the quasioptimality criterion for (iterated) Tikhonov regularization. Inverse Probl. Imaging 2(2), 291–299 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Li, X., Rao, S., Jiang, W., Li, C., Xiao, Y., Guo, Z., Zhang, Q., Wang, L., Du, L., Li, J., Li, L., Zhang, T., Wang, Q.K.: Discovery of time-delayed gene regulatory networks based on temporal gene expression profiling. BMC Bioinform. 7, 26 (2006)CrossRefGoogle Scholar
  25. 25.
    Liu, H., Lafferty, J.D., Wasserman, T.: The nonparanormal: semiparametric estimation of high dimensional undirected graphs. J. Mach. Learn. Res. 10, 2295–2328 (2009)zbMATHMathSciNetGoogle Scholar
  26. 26.
    Lorenz, D.A., Maass, P., Pham, Q.M.: Gradient descent for Tikhonov functionals with sparsity constraints: theory and numerical comparison of step size rules. Electron. Trans. Numer. Anal. 39, 437–463 (2012)zbMATHMathSciNetGoogle Scholar
  27. 27.
    Lozano, A.C., Abe, N., Liu, Y., Rosset, S.: Grouped graphical Granger modeling for gene expression regulatory networks discovery. ISMB 25, i110–i118 (2009)Google Scholar
  28. 28.
    Marinazzo, D., Pellicoro, M., Stramaglia, S.: Kernel-Granger causality and the analysis of dynamic networks. Phys. Rev. E 77, 056215 (2008)CrossRefGoogle Scholar
  29. 29.
    Marinazzo, D., Pellicoro, M., Stramaglia, S.: Causal information approach to partial conditioning in multivariate data sets. Comput. Math. Methods Med. 2012, 8 (2012)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Paluš, M., Komárek, V., Procházka, T., Hrnčír, Z., Štěrbová, K.: Synchronization and information flow in EEGs of epileptic patients. IEEE Eng. Med. Biol. Mag. 20(5), 65–71 (2001)CrossRefGoogle Scholar
  31. 31.
    Pearl, J.: Probabilistic reasoning in intelligent systems. Morgan Kaufmann, San Mateo (1988)Google Scholar
  32. 32.
    Pereverzev, S., Schock, E.: On the adaptive selection of the parameter in regularization of ill-posed problems. SIAM J. Numer. Anal. 43, 2060–2076 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  33. 33.
    Pereverzyev Jr, S., Hlaváčková-Schindler, K.: Graphical Lasso Granger method with two-level-thresholding for recovering causality networks, Research Report, 09/13. Leopold Franzens Universität Innsbruck, Department of Applied Mathematics (2013)Google Scholar
  34. 34.
    Ramlau, R., Teschke, G.: A Tikhonov-based projection iteration for nonlinear ill-posed problems with sparsity constraints. J. Numer. Math. 104(2), 177–203 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  35. 35.
    Sambo, F., Camillo, B.D., Toffolo, G.: CNET: an algorithm for reverse engineering of causal gene networks, NETTAB2008. Varenna, Italy (2008)Google Scholar
  36. 36.
    Seth, A.K.: Causal connectivity of evolved neural networks during behavior. Netw.-Comput. Neural Syst. 16(1), 35–54 (2005)CrossRefMathSciNetGoogle Scholar
  37. 37.
    Shmulevich, I., Dougherty, E.R., Kim, S., Zhang, W.: Probabilistic Boolean networks: a rule-based uncertainty model for gene regulatory networks. Bioinformatics 18(2), 261–274 (2002)CrossRefGoogle Scholar
  38. 38.
    Shojaie, A., Michalidis, G.: Discovering graphical Granger causality using the truncating lasso penalty. Bioinformatics 26(18), i517–i523 (2010)CrossRefGoogle Scholar
  39. 39.
    Shojaie, A., Basu, S. Michalidis, G.: Adaptive thresholding for reconstructing regulatory networks from time course gene expression data (2011).
  40. 40.
    Steinhaeuser, K., Ganguly, A.R., Chawla, N.V.: Multivariate and multiscale dependence in the global climate system revealed through complex networks. Clim. Dyn. 39, 889–895 (2012)CrossRefGoogle Scholar
  41. 41.
    Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. B 58, 267–288 (1996)zbMATHMathSciNetGoogle Scholar
  42. 42.
    Tikhonov, A.N., Glasko, V.B.: Use of the regularization method in non-linear problems. Scmmp 5, 93–107 (1965)Google Scholar
  43. 43.
  44. 44.
    Whitfield, M.L., Sherlock, G., Saldanha, A.J., Murray, J.I., Ball, C.A., Alexander, K.E., Matese, J.C., Perou, C.M., Hurt, M.M., Brown, P.O., Botstein, D.: Identification of genes periodically expressed in the human cell cycle and their expression in tumors. Mol. Biol. Cell 13(6), 1977–2000 (2002)CrossRefGoogle Scholar
  45. 45.
    Wiener, N.: The theory of prediction. In: Beckenbach, E.F. (ed.) Modern Mathematics for Engineers. McGraw-Hill, New York (1956)Google Scholar
  46. 46.
    Wikipedia, Causality, The Free Encyclopedia (2013)Google Scholar
  47. 47.
    Yu, J., Smith, V.A., Wang, P.P., Hartemink, A.J., Jarvis, E.D.: Advances to Bayesian network inference for generating causal networks from observational biological data. Bioinformatics 20, 35943603 (2004)Google Scholar
  48. 48.
    Zou, M., Conzen, S.D.: A new dynamic Bayesian network (DBN) approach for identifying gene regulatory networks from time course microarray data. Bioinformatics 21, 7179 (2005)CrossRefGoogle Scholar
  49. 49.
    Zou, C., Feng, J.: Granger causality vs dynamic Bayesian network inference: a comparative study. BMC Bioinform. 10, 122 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Kateřina Hlaváčková-Schindler
    • 1
    Email author
  • Sergiy PereverzyevJr.
    • 2
  1. 1.Department of Adaptive Systems, Institute of Information Theory and AutomationAcademy of Sciences of the Czech RepublicPragueCzech Republic
  2. 2.Applied Mathematics Group, Department of MathematicsUniversity of InnsbruckInnsbruckAustria

Personalised recommendations