Advertisement

Parallelizing Big De Bruijn Graph Traversal for Genome Assembly on GPU Clusters

  • Shuang QiuEmail author
  • Zonghao Feng
  • Qiong Luo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11448)

Abstract

De Bruijn graph traversal is a critical step in de novo assemblers. It uses the graph structure to analyze genome sequences and is both memory space intensive and time consuming. To improve the efficiency, we develop ParaGraph, which parallelizes De Bruijn graph traversal on a cluster of GPU-equipped computer nodes. With effective vertex partitioning and fine-grained parallel algorithms, ParaGraph utilizes all cores of each CPU and GPU, all CPUs and GPUs in a computer node, and all computer nodes of a cluster. Our results show that ParaGraph is able to traverse billion-node graphs within three minutes on a cluster of six GPU-equipped computer nodes. It is an order of magnitude faster than the state-of-the-art shared memory based assemblers, and more than five times faster than the current distributed assemblers.

References

  1. 1.
    Avery, C.: Giraph: large-scale graph processing infrastructure on Hadoop. In: Proceedings of the Hadoop Summit. Santa Clara, vol. 11, pp. 5–9 (2011)Google Scholar
  2. 2.
    Chikhi, R., Limasset, A., Medvedev, P.: Compacting de bruijn graphs from sequencing data quickly and in low memory. Bioinformatics 32(12), i201–i208 (2016)CrossRefGoogle Scholar
  3. 3.
    Li, Y., Kamousi, P., Han, F., Yang, S., Yan, X., Suri, S.: Memory efficient minimum substring partitioning. In: Proceedings of the VLDB Endowment, vol. 6, pp. 169–180. VLDB Endowment (2013)Google Scholar
  4. 4.
    Luo, R., et al.: Soapdenovo2: an empirically improved memory-efficient short-read de novo assembler. Gigascience 1(1), 18 (2012)CrossRefGoogle Scholar
  5. 5.
    Meng, J., Seo, S., Balaji, P., Wei, Y., Wang, B., Feng, S.: Swap-assembler 2: optimization of de novo genome assembler at extreme scale. In: 2016 45th International Conference on Parallel Processing (ICPP), pp. 195–204. IEEE (2016)Google Scholar
  6. 6.
    Minkin, I., Pham, S., Medvedev, P.: Twopaco: an efficient algorithm to build the compacted de bruijn graph from many complete genomes. Bioinformatics 33(24), 4024–4032 (2016)Google Scholar
  7. 7.
    Qiu, S., Luo, Q.: Parallelizing big de bruijn graph construction on heterogeneous processors. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 1431–1441. IEEE (2017)Google Scholar
  8. 8.
    Yan, D., Chen, H., Cheng, J., Cai, Z., Shao, B.: Scalable de novo genome assembly using pregel. arXiv preprint arXiv:1801.04453 (2018)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.The Hong Kong University of Science and TechnologyClear Water BayHong Kong

Personalised recommendations