Skip to main content
SpringerLink
Log in

Toil enables reproducible, open source, big biomedical data analyses

  • Correspondence
  • Published:

From Nature Biotechnology

View current issue Submit your manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1: RNA-seq pipeline and expression concordance.
Figure 2: Costs and core usage.

References

  1. Weinstein, J.N. et al. Nat. Genet. 45, 1113–1120 (2013).

    Article  Google Scholar 

  2. Zhang, J. et al. Database. http://dx.doi.org/10.1093/database/bar026 (2011)

  3. Siva, N. Lancet 385, 103–104 (2015).

    Article  Google Scholar 

  4. McKenna, A. et al. Genome Res. 20, 1297–1303 (2010).

    Article  CAS  Google Scholar 

  5. UNC Bioinformatics. TCGA mRNA-seq pipeline for UNC data. https://webshare.bioinf.unc.edu/public/mRNAseq_TCGA/UNC_mRNAseq_summary.pdf (2013).

  6. Albrecht, M., Michael, A., Patrick, D., Peter, B. & Douglas, T. in Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies (SWEET '12) 1. ACM (Association of Computing Machinery. http://dx.doi.org/10.1145/2443416.2443417 (2012).

    Google Scholar 

  7. Bernhardsson, E. & Frieder, E. Luigi. Github https://github.com/spotify/luigi (2016).

    Google Scholar 

  8. Goecks, J., Nekrutenko, A. & Taylor, J. Genome Biol. 11, R86 (2010).

    Article  Google Scholar 

  9. UCSC. Xena http://xena.ucsc.edu (2016).

  10. Amstutz, P. Common workflow language. Github https://github.com/common-workflow-language/common-workflow-language (2016).

    Google Scholar 

  11. Frazer, S. Workflow description language. Github https://github.com/broadinstitute/wdl (2014).

    Google Scholar 

  12. Vivian, J. Toil scripts. Github https://github.com/BD2KGenomics/toil-scripts/tree/master/src/toil_scripts (2016).

    Google Scholar 

  13. Apache Software Foundation. Apache Spark http://spark.apache.org/ (2017).

  14. Massie, M. et al. ADAM: genomics formats and processing patterns for cloud scale computing. University of California, Berkeley, Technical Report No. UCB/EECS-2013-207 (2013).

  15. Gentzsch, W. in Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid 35–36 http://dx.doi.org/10.1109/ccgrid.2001.923173 (IEEE, 2001).

    Book  Google Scholar 

  16. Yoo, A.B., Jette, M.A. & Mark, G. in Lecture Notes in Computer Science 44–60 (2003) Springer, Berlin, Heidelberg.

    Google Scholar 

  17. Apache Software Foundation. Apache Mesos http://mesos.apache.org/

  18. GTEx Consortium. Science 348, 648–660 (2015).

  19. Dobin, A. et al. Bioinformatics 29, 15–21 (2013).

    Article  CAS  Google Scholar 

  20. Li, B. & Dewey, C.N. BMC Bioinformatics 12, 323 (2011).

    Article  CAS  Google Scholar 

  21. Bray, N.L., Pimentel, H., Melsted, P. & Pachter, L. Nat. Biotechnol. 34, 525–527 (2016).

    Article  CAS  Google Scholar 

  22. Barker, A.D. et al. Clin. Pharmacol. Ther. 86, 97–100 (2009).

    Article  CAS  Google Scholar 

  23. Kent, W.J. et al. Genome Res. 12, 996–1006 (2002).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

This work was supported by (BD2K) the National Human Genome Research Institute of the National Institutes of Health award no. 5U54HG007990 and (Cloud Pilot) the National Cancer Institute of the National Institutes of Health under the Broad Institute subaward no. 5417071-5500000716. The UCSC Genome Browser work was supported by the NHGRI award 5U41HG002371 (Corporate Sponsors). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or our corporate sponsors.

Author information

Authors and Affiliations

  1. Computational Genomics Lab, UC Santa Cruz Genomics Institute, University of California Santa Cruz, Santa Cruz, California, USA

    John Vivian, Arjun Arkal Rao, Christopher Ketchum, Joel Armstrong, Adam Novak, Jacob Pfeil, Jake Narkizian, Alden D Deran, Audrey Musselman-Brown, Hannes Schmidt, Brian Craft, Mary Goldman, Kate Rosenbloom, Melissa Cline, Brian O'Connor, W James Kent, Jingchun Zhu, David Haussler & Benedict Paten

  2. AMP Lab, University of California Berkeley, Berkeley, California, USA

    Frank Austin Nothaft, David A Patterson & Anthony D Joseph

  3. UC Berkeley ASPIRE Lab, Berkeley, Berkeley, California, USA

    Frank Austin Nothaft, David A Patterson & Anthony D Joseph

  4. Curoverse, Somerville, Massachusetts, USA

    Peter Amstutz & Sasha Zaranek

  5. Broad Institute of Harvard and Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts, USA

    Megan Hanna, Chet Birger & Gad Getz

Authors
  1. John Vivian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arjun Arkal Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frank Austin Nothaft
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christopher Ketchum
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joel Armstrong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Adam Novak
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jacob Pfeil
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jake Narkizian
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Alden D Deran
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Audrey Musselman-Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Hannes Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Peter Amstutz
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Brian Craft
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Mary Goldman
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Kate Rosenbloom
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Melissa Cline
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Brian O'Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Megan Hanna
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Chet Birger
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. W James Kent
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. David A Patterson
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Anthony D Joseph
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Jingchun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Sasha Zaranek
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Gad Getz
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. David Haussler
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Benedict Paten
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benedict Paten.

Ethics declarations

Competing interests

The authors received support from AWS, Microsoft, and Google.

Supplementary information

Supplementary Figures and Texts

Supplementary Notes 1–9, Supplementary Figures 1–4, Supplementary Tables 1 (PDF 2647 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vivian, J., Rao, A., Nothaft, F. et al. Toil enables reproducible, open source, big biomedical data analyses. Nat Biotechnol 35, 314–316 (2017). https://doi.org/10.1038/nbt.3772

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt.3772

  • Springer Nature America, Inc.

This article is cited by

Search

Navigation