Comparison of methods for detecting bottlenecks from microsatellite loci

Abstract

This paper describes simulation tests to compare methods for detecting recent bottlenecks using microsatellite data. This study considers both type I error (detecting a bottleneck when there wasn’t one) and type II error (failing to detect a bottleneck when there was one) under a variety of scenarios. The two most promising methods were the range in allele size conditioned on the number of alleles, M _k , and heterozygosity given the number of alleles, H _k , under a two-phase mutation model; in most of the simulations one of these two methods had the lowest type I and type II error relative to other methods. M _k was the method most likely to correctly identify a bottleneck when a bottleneck lasted several generations, the population had made a demographic recovery, and mutation rates were high or pre-bottleneck population sizes were large. On the other hand H _k was most likely to correctly identify a bottleneck when a bottleneck was more recent and less severe and when mutation rates were low or pre-bottleneck population sizes were small. Both methods were prone to type I errors when assumptions of the model were violated, but it may be easier to design a conservative heterozygosity test than a conservative ratio test.

Appendix

The transition between an equilibrium population to a bottlenecked population required the transition from a coalescent simulation to a full Wright–Fisher simulation. To do this, I generated a sample of size 2N _b using the coalescent simulation. This sample represented the entire population during the first generation of the bottleneck.

Simulating the recovery from a bottleneck to a large population size required the transition from the full Wright–Fisher simulation to the coalescent simulation. This transition was more complicated because coalescent simulations keep track of only the sample and ancestors of the sample backward in time while full Wright–Fisher simulations require information on the entire population and work forward in time. First, using the size of the sample and the time since the bottleneck as input parameters, I used a coalescent simulation to determine the number of ancestors, n, of the sample, that were present immediately after recovery from the bottleneck. Thus, the coalescent simulation of the recovered population was stopped after a specific amount of time, T, unlike the pre-bottleneck simulation, which was stopped when all lineages coalesced to a single common ancestor. Allele sizes of these n ancestors were then determined by sampling (with replacement) n individuals from the bottlenecked population (the last generation of the full Wright–Fisher simulation). Finally, allele sizes for the sample from the recovered population were determined by the mutations and coalescent events dictated by the coalescent simulation that determined n.