Abstract
While exponential growth in public genomic data can afford great insights into biological processes underlying diseases, a lack of structured metadata often impedes its timely discovery for analysis. In the Gene Expression Omnibus, for example, descriptions of genomic samples lack structure, with different terminology (such as “breast cancer”, “breast tumor”, and “malignant neoplasm of breast”) used to express the same concept. To remedy this, we learn models to extract salient information from this textual metadata. Rather than treating the problem as classification or named entity recognition, we model it as machine translation, leveraging state-of-the-art sequence-to-sequence (seq2seq) models to directly map unstructured input into a structured text format. The application of such models greatly simplifies training and allows for imputation of output fields that are implied but never explicitly mentioned in the input text.
We experiment with two types of seq2seq models: an LSTM with attention and a transformer (in particular GPT-2), noting that the latter outperforms both the former and also a multi-label classification approach based on a similar transformer architecture (RoBERTa). The GPT-2 model showed a surprising ability to predict attributes with a large set of possible values, often inferring the correct value for unmentioned attributes. The models were evaluated in both homogeneous and heterogenous training/testing environments, indicating the efficacy of the transformer-based seq2seq approach for real data integration applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Companies currently offer complete genome sequencing for under 600USD (e.g. https://www.veritasgenetics.com/myGenome) with costs expected to fall.
- 2.
NCBI E-utilities [17] provide a federated search engine supporting information on experimental protocols, but lack functionality regarding characteristics of the sample, such as species of origin, age, gender, tissue, mutations, disease state, etc.
- 3.
Information regarding the submission of high-throughput sequences is provided at https://www.ncbi.nlm.nih.gov/geo/info/seq.html.
- 4.
The data model centers on the item of experimental data, with views describing biological elements, technology used, management aspects, and extraction parameters.
- 5.
K562 is a widely known cell line originally extracted from tissue belonging to a 53 year-old woman affected by myeloid leukemia.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
Pre-processing was motivated by the fact that important character ngrams often appear in sequences separated by special characters, e.g., “RH_RRE2_14028”.
- 12.
- 13.
The input in this case was: microRNA profile of case NPC362656 survival status (1-death,0-survival): 0; gender (1-male,2-female): 1; age (years): 56; ...
References
Abadi, M., Agarwal, A., Barham, P., et al.: Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)
Barrett, T., Wilhite, S.E., Ledoux, P., et al.: NCBI GEO: archive for functional genomics data sets-update. Nucleic Acids Res. 41(D1), D991–D995 (2012)
Bernasconi, A., Canakoglu, A., Masseroli, M., et al.: META-BASE: a novel architecture for large-scale genomic metadata integration. IEEE/ACM Trans. Comput. Biol. Bioinform. https://doi.org/10.1109/TCBB.2020.2998954
Bernasconi, A., Canakoglu, A., Masseroli, M., et al.: The road towards data integration in human genomics: players, steps and interactions. Briefings in Bioinform. 22(1), 30–44 (2021). https://doi.org/10.1093/bib/bbaa080
Bernasconi, A., Ceri, S., Campi, A., Masseroli, M.: Conceptual modeling for genomics: building an integrated repository of open data. In: Mayr, H.C., Guizzardi, G., Ma, H., Pastor, O. (eds.) ER 2017. LNCS, vol. 10650, pp. 325–339. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69904-2_26
Bodenreider, O.: Biomedical ontologies in action: role in knowledge management, data integration and decision support. Yearbook of Medical Informatics, p. 67 (2008)
Canakoglu, A., Bernasconi, A., Colombo, A., et al.: GenoSurf: metadata drivensemantic search system for integrated genomic datasets. Database 2019 (2019)
Davis, C.A., Hitz, B.C., Sloan, C.A., et al.: The encyclopedia of DNA elements (ENCODE): data portal update. Nucleic Acids Res. 46(D1), D794–D801 (2017)
Devlin, J., Chang, M.W., Lee, K., et al.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186 (2019)
Ellis, S.E., Collado-Torres, L., Jaffe, A., et al.: Improving the value of public RNA-seq expression data by phenotype prediction. Nucleic Acids Res. 46(9), e54–e54 (2018)
Galeota, E., Kishore, K., Pelizzola, M.: Ontology-driven integrative analysis of omics data through onassis. Sci. Rep. 10(1), 1–9 (2020)
Giles, C.B., Brown, C.A., Ripperger, M., et al.: ALE: automated label extraction from GEO metadata. BMC Bioinform. 18(14), 509 (2017)
Guo, Z., Tzvetkova, B., Bassik, J.M., et al.: RNASeqMetaDB: a database and web server for navigating metadata of publicly available mouse RNA-Seq datasets. Bioinformatics 31(24), 4038–4040 (2015)
Hadley, D., Pan, J., El-Sayed, O., et al.: Precision annotation of digital samples in NCBI’s Gene Expression Omnibus. Sci. Data 4, 170125 (2017)
Hong, E.L., Sloan, C.A., Chan, E.T., et al.: Principles of metadata organization at the ENCODE data coordination center. Database 2016 (2016)
Huang, C.C., Lu, Z.: Community challenges in biomedical text mining over 10 years: success, failure and the future. Briefings Bioinform. 17(1), 132–144 (2016)
Kans, J.: Entrez direct: E-utilities on the unix command line. In: Entrez Programming Utilities Help [Internet]. National Center for Biotechnology Information (US) (2020)
Kundaje, A., Meuleman, W., Ernst, J., et al.: Integrative analysis of 111 reference human epigenomes. Nature 518(7539), 317 (2015)
Li, J., Tseng, C.S., Federico, A., et al.: SFMetaDB: a comprehensive annotation of mouse RNA splicing factor RNA-Seq datasets. Database 2017 (2017)
Li, Z., Li, J., Yu, P.: GEOMetaCuration: a web-based application for accurate manual curation of Gene Expression Omnibus metadata. Database J. Biol. Databases Curation 2018 (2018)
Liu, Y., Ott, M., Goyal, N., et al.: RoBERTa: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Lonsdale, J., Thomas, J., Salvatore, M., et al.: The genotype-tissue expression (GTEx) project. Nat. Genet. 45(6), 580 (2013)
Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1412–1421 (2015)
Musen, M.A., Sansone, S.A., Cheung, K.H., et al.: CEDAR: semantic web technology to support open science. In: Companion Proceedings of the The Web Conference 2018, pp. 427–428. International World Wide Web Conferences Steering Committee (2018)
Posch, L., Panahiazar, M., Dumontier, M., et al.: Predicting structured metadata from unstructured metadata. Database 2016 (2016)
Radford, A., Wu, J., Child, R., et al.: Language models are unsupervised multitask learners. OpenAI Blog. 1(8), 9 (2019)
Genomes Project Consortium: A global reference for human genetic variation. Nature 526(7571), 68 (2015)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wang, Z., Lachmann, A., Ma’ayan, A.: Mining data and metadata from the Gene Expression Omnibus. Biophys. Rev. 11(1), 103–110 (2019)
Wang, Z., Monteiro, C.D., Jagodnik, K.M., et al.: Extraction and analysis of signatures from the Gene Expression Omnibus by the crowd. Nature Commun. 7(1), 1–11 (2016)
Weinstein, J.N., Collisson, E.A., Mills, G.B., et al.: The cancer genome atlas pan-cancer analysis project. Nat. Genet. 45(10), 1113 (2013)
Zaveri, A., Hu, W., Dumontier, M.: MetaCrowd: crowdsourcing biomedical metadata quality assessment. Hum. Comput. 6(1), 98–112 (2019)
Zheng, R., Wan, C., Mei, S., et al.: Cistrome Data Browser: expanded datasets and new tools for gene regulatory analysis. Nucleic Acids Res. 47(D1), D729–D735 (2018)
Zhu, Y., Davis, S., Stephens, R., et al.: GEOmetadb: powerful alternative search engine for the Gene Expression Omnibus. Bioinformatics 24(23), 2798–2800 (2008)
Acknowledgments
This research is funded by the ERC Advanced Grant 693174 GeCo.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cannizzaro, G., Leone, M., Bernasconi, A., Canakoglu, A., Carman, M.J. (2021). Automated Integration of Genomic Metadata with Sequence-to-Sequence Models. In: Dong, Y., Ifrim, G., Mladenić, D., Saunders, C., Van Hoecke, S. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12461. Springer, Cham. https://doi.org/10.1007/978-3-030-67670-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-67670-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67669-8
Online ISBN: 978-3-030-67670-4
eBook Packages: Computer ScienceComputer Science (R0)
