How Many Bootstrap Replicates Are Necessary?

  • Nicholas D. Pattengale
  • Masoud Alipour
  • Olaf R. P. Bininda-Emonds
  • Bernard M. E. Moret
  • Alexandros Stamatakis
Conference paper

DOI: 10.1007/978-3-642-02008-7_13

Volume 5541 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Pattengale N.D., Alipour M., Bininda-Emonds O.R.P., Moret B.M.E., Stamatakis A. (2009) How Many Bootstrap Replicates Are Necessary?. In: Batzoglou S. (eds) Research in Computational Molecular Biology. RECOMB 2009. Lecture Notes in Computer Science, vol 5541. Springer, Berlin, Heidelberg

Abstract

Phylogenetic Bootstrapping (BS) is a standard technique for inferring confidence values on phylogenetic trees that is based on reconstructing many trees from minor variations of the input data, trees called replicates. BS is used with all phylogenetic reconstruction approaches, but we focus here on the most popular, Maximum Likelihood (ML). Because ML inference is so computationally demanding, it has proved too expensive to date to assess the impact of the number of replicates used in BS on the quality of the support values. For the same reason, a rather small number (typically 100) of BS replicates are computed in real-world studies. Stamatakis et al. recently introduced a BS algorithm that is 1–2 orders of magnitude faster than previous techniques, while yielding qualitatively comparable support values, making an experimental study possible.

In this paper, we propose stopping criteria, that is, thresholds computed at runtime to determine when enough replicates have been generated, and report on the first large-scale experimental study to assess the effect of the number of replicates on the quality of support values, including the performance of our proposed criteria. We run our tests on 17 diverse real-world DNA, single-gene as well as multi-gene, datasets, that include between 125 and 2,554 sequences. We find that our stopping criteria typically stop computations after 100–500 replicates (although the most conservative criterion may continue for several thousand replicates) while producing support values that correlate at better than 99.5% with the reference values on the best ML trees. Significantly, we also find that the stopping criteria can recommend very different numbers of replicates for different datasets of comparable sizes.

Our results are thus two-fold: (i) they give the first experimental assessment of the effect of the number of BS replicates on the quality of support values returned through bootstrapping; and (ii) they validate our proposals for stopping criteria. Practitioners will no longer have to enter a guess nor worry about the quality of support values; moreover, with most counts of replicates in the 100–500 range, robust BS under ML inference becomes computationally practical for most datasets. The complete test suite is available at http://lcbb.epfl.ch/BS.tar.bz2 and BS with our stopping criteria is included in RAxML 7.1.0.

Keywords

Phylogenetic Inference Maximum Likelihood Bootstrap Support Value Stopping Criterion Bootstopping 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Nicholas D. Pattengale
    • 1
  • Masoud Alipour
    • 2
  • Olaf R. P. Bininda-Emonds
    • 3
  • Bernard M. E. Moret
    • 2
    • 4
  • Alexandros Stamatakis
    • 5
  1. 1.Department of Computer ScienceUniversity of New MexicoAlbuquerqueUSA
  2. 2.Laboratory for Computational Biology and BioinformaticsEPFL (École Polytechnique Fédérale de Lausanne)Switzerland
  3. 3.AG Systematik und Evolutionsbiologie, Institut für Biologie und UmweltwissenschaftenUniversity of OldenburgGermany
  4. 4.Swiss Institute of BioinformaticsLausanneSwitzerland
  5. 5.The Exelixis Lab, Department of Computer ScienceTechnische Universität MünchenGermany