The Practical Evaluation of DNA Barcode Efficacy

Abstract

This chapter describes a workflow for measuring the efficacy of a barcode in identifying species. First, assemble individual sequence databases corresponding to each barcode marker. A controlled collection of taxonomic data is preferable to GenBank data, because GenBank data can be problematic, particularly when comparing barcodes based on more than one marker. To ensure proper controls when evaluating species identification, specimens not having a sequence in every marker database should be discarded. Second, select a computer algorithm for assigning species to barcode sequences. No algorithm has yet improved notably on assigning a specimen to the species of its nearest neighbor within a barcode database. Because global sequence alignments (e.g., with the Needleman–Wunsch algorithm, or some related algorithm) examine entire barcode sequences, they generally produce better species assignments than local sequence alignments (e.g., with BLAST). No neighboring method (e.g., global sequence similarity, global sequence distance, or evolutionary distance based on a global alignment) has yet shown a notable superiority in identifying species. Finally, “the probability of correct identification” (PCI) provides an appropriate measurement of barcode efficacy. The overall PCI for a data set is the average of the species PCIs, taken over all species in the data set. This chapter states explicitly how to calculate PCI, how to estimate its statistical sampling error, and how to use data on PCR failure to set limits on how much improvements in PCR technology can improve species identification.

Appendix

For a barcode with several markers, each of which can have a failed PCR, specimen identification ultimately relies on the markers with a successful PCR. To quantify the identification process, number the markers $\left\{1,2,\mathrm{...},m\right\}$, and consider any subset $M$ of $\left\{1,2,\mathrm{...},m\right\}$. For a particular specimen, let the probability that $M$ is the subset of markers with PCR success be denoted by ${s}_{M}$, and let the PCI for the barcode based on the marker subset $M$ be ${p}_{M}$. A species PCI $p$ can then be calculated from the values of ${s}_{M}$ and ${p}_{M}$ (although the calculation depends on the definition of species PCI: see Section 2.3 for various definitions.)

One very reasonable definition of the PCR-adjusted species PCI is the average $p={\displaystyle {\sum }_{(M)}{p}_{M}{s}_{M}}$. For the case of a barcode based on a single marker, e.g., $M$is a subset of $\left\{1\right\}$, i.e., the empty set $\left\{\right\}$or $\left\{1\right\}$. Because the empty set$\left\{\right\}$corresponds to a complete absence of information about a specimen, the corresponding PCI is ${p}_{\left\{\right\}}=0$, so $p={p}_{\left\{\right\}}{s}_{\left\{\right\}}+{p}_{\left\{1\right\}}{s}_{\left\{1\right\}}={p}_{\left\{1\right\}}{s}_{\left\{1\right\}}$, which agrees with the formula for the PCR-adjusted PCI in the main text, for a barcode based on a single marker.