Skip to main content

Gene Networks Inference by Reinforcement Learning

  • Conference paper
  • First Online:
Advances in Bioinformatics and Computational Biology (BSB 2023)

Part of the book series: Lecture Notes in Computer Science ((LNBI,volume 13954))

Included in the following conference series:

  • 186 Accesses

Abstract

Gene Regulatory Networks inference from gene expression data is an important problem in systems biology field, involving the estimation of gene-gene indirect dependencies and the regulatory functions among these interactions to provide a model that explains the gene expression dataset. The main goal is to comprehend the global molecular mechanisms underlying diseases for the development of medical treatments and drugs. However, such a problem is considered an open problem, since it is difficult to obtain a satisfactory estimation of the dependencies given a very limited number of samples subject to experimental noises. Many gene networks inference methods exist in the literature, where some of them use heuristics or model based algorithms to find interesting networks that explain the data by codifying whole networks as solutions. However, in general, these models are slow, not scalable to real sized networks (thousands of genes), or require many parameters, the knowledge from an specialist or a large number of samples to be feasible. Reinforcement Learning is an adaptable goal oriented approach that does not require large labeled datasets and many parameters; can give good quality solutions in a feasible execution time; and can work automatically without the need of a specialist for a long time. Therefore, we here propose a way to adapt Reinforcement Learning to the Gene Regulatory Networks inference domain in order to get networks with quality comparable to one achieved by exhaustive search, but in much smaller execution time. Our experimental evaluation shows that our proposal is promising in learning and successfully finding good solutions across different tasks automatically in a reasonable time. However, scalabilty to networks with thousands of genes remains as limitation of our RL approach due to excessive memory consuming, although we foresee some possible improvements that could deal with this limitation in future versions of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akutsu, T., Miyano, S., Kuhara, S., et al.: Identification of genetic networks from a small number of gene expression patterns under the Boolean network model. In: Proceedings of the Pacific Symposium on Biocomputing (PSB), vol. 4, pp. 17–28 (1999)

    Google Scholar 

  2. Anastassiou, D.: Computational analysis of the synergy among multiple interacting genes. Mole. Syst. Biol. 3, 83 (2007)

    Article  Google Scholar 

  3. Barrera, J., et al.: Constructing probabilistic genetic networks of Plasmodium falciparum from dynamical expression signals of the intraerythrocytic development cycle. In: McConnell, P., Lin, S.M., Hurban, P. (eds.) Methods of Microarray Data Analysis, pp. 11–26. Springer, Boston, MA (2007). https://doi.org/10.1007/978-0-387-34569-7_2

  4. Bonini, R., Da Silva, F.L., Glatt, R., Spina, E., Costa, A.H.R.: A framework to discover and reuse object-oriented options in reinforcement learning. In: 2018 7th Brazilian Conference on Intelligent Systems (BRACIS), pp. 109–114. IEEE (2018)

    Google Scholar 

  5. Bonini, R.C., Silva, F.L., Spina, E., Costa, A.H.R.: Using options to accelerate learning of new tasks according to human preferences. In: AAAI Workshop Human-Machine Collaborative Learning, pp. 1–8 (2017)

    Google Scholar 

  6. Brazhnik, P., Fuente, A., Mendes, P.: Gene networks: how to put the function in genomics. Trends Biotechnol. 20(11), 467–472 (2002)

    Article  CAS  PubMed  Google Scholar 

  7. Cover, T.M., Van-Campenhout, J.M.: On the possible orderings in the measurement selection problem. IEEE Trans. Syst. Man Cybern. 7(9), 657–661 (1977)

    Article  Google Scholar 

  8. Da Silva, F.L., Nishida, C.E., Roijers, D.M., Costa, A.H.R.: Coordination of electric vehicle charging through multiagent reinforcement learning. IEEE Trans. Smart Grid 11(3), 2347–2356 (2019)

    Article  Google Scholar 

  9. De Jong, H.: Modeling and simulation of genetic regulatory systems: a literature review. J. Comput. Biol. 9(1), 67–103 (2002)

    Article  PubMed  Google Scholar 

  10. D’haeseleer, P., Liang, S., Somgyi, R.: Tutorial: gene expression data analysis and modeling. In: Pacific Symposium on Biocomputing. Hawaii, January 1999

    Google Scholar 

  11. Dougherty, E.R., Xiao, Y.: Design of probabilistic Boolean networks under the requirement of contextual data consistency. IEEE Trans. Signal Process. 54(9), 3603–3613 (2006)

    Article  Google Scholar 

  12. Eberwine, J., Sul, J., Bartfai, T., Kim, J.: The promise of single-cell sequencing. Nat. Methods 11, 25–27 (2014)

    Article  CAS  PubMed  Google Scholar 

  13. Erdös, P., Rényi, A.: On random graphs. Publ. Math. Debrecen 6, 290–297 (1959)

    Article  Google Scholar 

  14. Hecker, M., Lambeck, S., Toepfer, S., van Someren, E., Guthke, R.: Gene regulatory network inference: data integration in dynamic models-a review. Biosystems 96, 86–103 (2009)

    Article  CAS  PubMed  Google Scholar 

  15. Jacomini, R.S., Martins-Jr, D.C., Silva, F.L., Costa, A.H.R.: GeNICE: a novel framework for gene network inference by clustering, exhaustive search, and multivariate analysis. J. Comput. Biol. 24(8), 809–830 (2017)

    Article  CAS  Google Scholar 

  16. Jimenez, R.D., Martins-Jr, D.C., Santos, C.S.: One genetic algorithm per gene to infer gene networks from expression data. Netw. Modeling Anal. Health Inform. Bioinform. 4, 1–22 (2015)

    Google Scholar 

  17. Kauffman, S.A.: Homeostasis and differentiation in random genetic control networks. Nature 224(215), 177–178 (1969)

    Article  CAS  PubMed  Google Scholar 

  18. Liang, S., Fuhrman, S., Somogyi, R.: Reveal, a general reverse engineering algorithm for inference of genetic network architectures. In: Pacific Symposium on Biocomputing, vol. 3, pp. 18–29 (1998)

    Google Scholar 

  19. Lopes, F.M., Martins-Jr, D.C., Barrera, J., Cesar-Jr, R.M.: A feature selection technique for inference of graphs from their known topological properties: revealing scale-free gene regulatory networks. Inf. Sci. 272, 1–15 (2014)

    Article  Google Scholar 

  20. Marbach, D., Prill, R.J., Schaffter, T., Mattiussi, C., Floreano, D., Stolovitzky, G.: Revealing strengths and weaknesses of methods for gene network inference. Proc. Natl. Acad. Sci. 107(14), 6286–6291 (2010)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Martins-Jr, D.C., Braga-Neto, U., Hashimoto, R.F., Dougherty, E.R., Bittner, M.L.: Intrinsically multivariate predictive genes. IEEE J. Sel. Top. Signal Process. 2(3), 424–439 (2008)

    Article  Google Scholar 

  22. Nam, D., Seo, S., Kim, S.: An efficient top-down search algorithm for learning Boolean networks of gene expression. Mach. Learn. 65, 229–245 (2006)

    Article  Google Scholar 

  23. Pratapa, A., Jalihal, A.P., Law, J.N., Bharadwaj, A., Murali, T.: Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nat. Methods 17(2), 147–154 (2020)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Shalon, D., Smith, S.J., Brown, P.O.: A DNA microarray system for analyzing complex DNA samples using two-color fluorescent probe hybridization. Genome Res. 6, 639–45 (1996)

    Article  CAS  PubMed  Google Scholar 

  25. Shmulevich, I., Dougherty, E.R., Kim, S., Zhang, W.: Probabilistic Boolean networks: a rule-based uncertainty model for gene regulatory networks. Bioinformatics 18(2), 261–274 (2002)

    Article  CAS  PubMed  Google Scholar 

  26. Silva, F.L., Taylor, M.E., Costa, A.H.R.: Autonomously reusing knowledge in multiagent reinforcement learning. In: IJCAI (2018)

    Google Scholar 

  27. Snoep, J.L., Westerhoff, H.V.: From isolation to integration, a systems biology approach for building the silicon cell. Top. Curr. Genet. 13, 13–30 (2005)

    Article  CAS  Google Scholar 

  28. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 1st edn. MIT Press, Cambridge (1998)

    Google Scholar 

  29. Velculescu, V.E., Zhang, L., Vogelstein, B., Kinzler, K.W.: Serial analysis of gene expression. Science 270, 484–487 (1995)

    Article  CAS  PubMed  Google Scholar 

  30. Wang, Z., Gerstein, M., Snyder, M.: RNA-SEQ: a revolutionary tool for transcriptomics. Nat. Rev. Genet. 10(1), 57–63 (2009)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)

    Article  Google Scholar 

  32. Yerudkar, A., Chatzaroulas, E., Del Vecchio, C., Moschoyiannis, S.: Sampled-data control of probabilistic Boolean control networks: a deep reinforcement learning approach. Inf. Sci. 619, 374–389 (2023)

    Article  Google Scholar 

  33. Zhang, Y., Chang, X., Liu, X.: Inference of gene regulatory networks using pseudo-time series data. Bioinformatics 37(16), 2423–2431 (2021)

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgment

We are grateful for the financial support from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brazil (CAPES) - Finance Code 001 and Fundação de Amparo a Pesquisa do Estado de São Paulo (FAPESP), grants 2018/18560-6 and 2018/21934-5. We also thank Felipe Leno da Silva for important technical discussions about RL.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rodrigo Cesar Bonini .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cesar Bonini, R., Correa Martins-Jr, D. (2023). Gene Networks Inference by Reinforcement Learning. In: Reis, M.S., de Melo-Minardi, R.C. (eds) Advances in Bioinformatics and Computational Biology. BSB 2023. Lecture Notes in Computer Science(), vol 13954. Springer, Cham. https://doi.org/10.1007/978-3-031-42715-2_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42715-2_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42714-5

  • Online ISBN: 978-3-031-42715-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics