Encyclopedia of Big Data Technologies

Living Edition
| Editors: Sherif Sakr, Albert Zomaya

Big Data Technologies for DNA Sequencing

Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-63962-8_32-1



DNA sequencing is a modern technique for the precise determination of the order of nucleotides within a DNA molecule. Using this technique a huge amount of raw data is generated in life sciences.


Genome analyses play an important role in different applications in the life sciences ranging from animal breeding to personalized medicine. The technological advancements in DNA sequencing lead to vast amounts of genome data being produced and processed on a daily basis. This chapter provides an overview of the big data challenges in the area of DNA sequencing and discusses several data management solutions.

Next-generation sequencing (NGS) technologies make it possible for life scientists to produce huge amounts of DNA sequence data in a short period of time (Stephens et al. 2015). Using these technologies, in recent years thousands of genomes and short DNA sequence reads for humans, plants, animals, and microbes have been collected...

This is a preview of subscription content, log in to check access


  1. Becker H (2011) Pflanzenzüchtung. UTB basics. UTB GmbHGoogle Scholar
  2. Bonfield JK, Mahoney MV (2013) Compression of FASTQ and SAM format sequencing data. PLoS One 8(3):e59190CrossRefGoogle Scholar
  3. Cao MD, Ganesamoorthy D, Elliott AG, Zhang H, Cooper MA, Coin LJ (2016) Streaming algorithms for identification of pathogens and antibiotic resistance potential from real-time minion tm sequencing. GigaScience 5(1):32CrossRefGoogle Scholar
  4. Carlson R (2003) The pace and proliferation of biological technologies. Biosecur Bioterror Biodefense Strategy Pract Sci 1(3):203–214Google Scholar
  5. Christley S, Lu Y, Li C, Xie X (2008) Human genomes as email attachments. Bioinformatics 25(2):274–275CrossRefGoogle Scholar
  6. Chung WC, Chen CC, Ho JM, Lin CY, Hsu WL, Wang YC, Lee DT, Lai F, Huang CW, Chang YJ (2014) Clouddoe: a user-friendly tool for deploying Hadoop clouds and analyzing high-throughput sequencing data with MapReduce. PloS one 9(6):e98146CrossRefGoogle Scholar
  7. Dorok S, Breß S, Teubner J, Läpple H, Saake G, Markl V (2017) Efficiently storing and analyzing genome data in database systems. Datenbank-Spektrum 17(2):139–154CrossRefGoogle Scholar
  8. Fiannaca A, La Rosa M, La Paglia L, Messina A, Urso A (2016) Biographdb: a new graphdb collecting heterogeneous data for bioinformatics analysis. In: Proceedings of BIOTECHNOGoogle Scholar
  9. Have CT, Jensen LJ (2013) Are graph databases ready for bioinformatics? Bioinformatics 29(24):3107CrossRefGoogle Scholar
  10. Jain M, Fiddes IT, Miga KH, Olsen HE, Paten B, Akeson M (2015) Improved data analysis for the minion nanopore sequencer. Nat Methods 12(4):351–356CrossRefGoogle Scholar
  11. Loman NJ, Watson M (2015) Successful test launch for nanopore sequencing. Nat methods 12(4):303CrossRefGoogle Scholar
  12. Martínez H, Barrachina S, Castillo M, Tárraga J, Medina I, Dopazo J, Quintana-Ortí ES (2015) Scalable RNA sequencing onclusters of multicore processors. Trustcom/BigDataSE/ISPA 3:190–195Google Scholar
  13. Mielczarek M, Szyda J (2016) Review of alignment and SNP calling algorithms for next-generation sequencing data. J Appl Genet 57(1):71–79. https://doi.org/10.1007/s13353-015-0292-7 CrossRefGoogle Scholar
  14. Mushtaq H, Liu F, Costa C, Liu G, Hofstee P, Al-Ars Z (2017) Sparkga: a spark framework for cost effective, fast and accurate dna analysis at scale. In: Proceedings of the 8th ACM international conference on bioinformatics, computational biology, and health informatics. ACM, pp 148–157Google Scholar
  15. Patro R, Duggal G, Love MI, Irizarry RA, Kingsford C (2017) Salmon provides fast and bias-aware quantification of transcript expression. Nat Methods 14(4):417–419CrossRefGoogle Scholar
  16. Pedersen E, Bongo LA (2016) Big biological data management. In: Pop F, Kolodziej J, Martino BD (eds) Resource management for big data platforms. Computer communications and networks. Springer, Heidelberg, pp 265–277Google Scholar
  17. Popitsch N, von Haeseler A (2012) NGC: lossless and lossy compression of aligned high-throughput sequencing data. Nucleic Acids Res 41(1):e27–e27CrossRefGoogle Scholar
  18. Salavert Torres J, Blanquer Espert I, Tomas Dominguez A, Hernendez V, Medina I, Terraga J, Dopazo J (2012) Using GPUs for the exact alignment of short-read genetic sequences by means of the burrows-wheeler transform. IEEE/ACM Trans Comput Biol Bioinform (TCBB) 9(4):1245–1256CrossRefGoogle Scholar
  19. Stephens ZD, Lee SY, Faghri F, Campbell RH, Zhai C, Efron MJ, Iyer R, Schatz MC, Sinha S, Robinson GE (2015) Big data: astronomical or genomical? PLoS Biol 13(7):e1002195CrossRefGoogle Scholar
  20. Taylor RC (2010) An overview of the hadoop/mapreduce/hbase framework and its current applications in bioinformatics. BMC Bioinform 11(12):S1CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Institute of Computer ScienceGeorg-August UniversityGöttingenGermany
  2. 2.Department of Breeding InformaticsGeorg-August UniversityGöttingenGermany

Section editors and affiliations

  • Kamran Munir
    • 1
  • Antonio Pescapè
    • 2
  1. 1.Computer Science and Creative TechnologiesUniversity of the West of EnglandBristolUnited Kingdom
  2. 2.Department of Electrical Engineering and Information TechnologyUniversity of Napoli Federico IINapoliItaly