Skip to main content
Log in

Some applications of genetics in statistical ecology

  • Original Paper
  • Published:
AStA Advances in Statistical Analysis Aims and scope Submit manuscript


Genetic data are in widespread use in ecological research, and an understanding of this type of data and its uses and interpretations will soon be an imperative for ecological statisticians. Here, we provide an introduction to the subject, intended for statisticians who have no previous knowledge of genetics. Although there are numerous types of genetic data, we restrict attention to multilocus genotype data from microsatellite loci. We look at two application areas in wide use: investigating population structure using genetic assignment and related techniques; and using genotype data in capture–recapture studies for estimating population size and demographic parameters. In each case, we outline the conceptual framework and draw attention to both the strengths and weaknesses of existing approaches to analysis and interpretation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others


  • Adams, A.L., van Heezik, Y., Dickinson, K.J.M., Robertson, B.C.: Identifying eradication units in an invasive mammalian pest species. Biol. Invasions 16, 1481–1496 (2014)

    Google Scholar 

  • Anderson, E.C., Waples, R.S., Kalinowski, S.T.: An improved method for predicting the accuracy of genetic stock identification. Can. J. Fish. Aquat. Sci. 65, 1475–1486 (2008)

    Article  Google Scholar 

  • Bagasra, A., Nathan, H.W., Mitchell, M.S., Russell, J.C.: Tracking invasive rat movements with a systemic biomarker. N. Z. J. Ecol. 40, 267–272 (2016)

    Article  Google Scholar 

  • Barker, R.J., Schofield, M.R., Wright, J.A., Frantz, A.C., Stevens, C.: Closed-population capture-recapture modeling of samples drawn one at a time. Biometrics 70, 775–782 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Baudouin, L., Lebrun, P.: An operational Bayesian approach for the identification of sexually reproduced cross-fertilized populations using molecular markers. Acta Hortic. 546, 81–94 (2001)

    Article  Google Scholar 

  • Bell, E.A., Bell, B.D., Merton, D.: The legacy of Big South Cape: rat irruption to rat eradication. N. Z. J. Ecol. 40, 205–211 (2016)

  • Bravington, M.V., Grewe, P.G., Davies, C.R.: Fishery-independent estimate of spawning biomass of Southern Bluefin Tuna through identification of close-kin using genetic markers. FRDC Report 2007/034, CSIRO, Australia (2014)

  • Bravington, M.V., Skaug, H.J., Anderson, E.C.: Close-kin mark-recapture. Statist. Sci. 31, 259–274 (2016)

  • Carroll, E.L., Patenaude, N.J., Childerhouse, S.J., Kraus, S.D., Fewster, R.M., Baker, C.S.: Abundance of the New Zealand subantarctic southern right whale population estimated from photo-identification and genotype mark-recapture. Mar. Biol. 158, 2565–2575 (2011)

    Article  Google Scholar 

  • Carroll, E.L., Childerhouse, S.J., Fewster, R.M., Patenaude, N.J., Steel, D., Dunshea, G., Boren, L., Baker, C.S.: Accounting for female reproductive cycles in a superpopulation capture-recapture framework: application to southern right whales (Eubalaena australis). Ecol. Appl. 23, 1677–1690 (2013)

    Article  Google Scholar 

  • Carroll, E.L., Brooks, L., Baker, C.S., Burns, D., Garrigue, C., Hauser, N., Jackson, J.A., Poole, M.M., Fewster, R.M.: Assessing the design and power of capture–recapture studies to estimate demographic parameters for the endangered Oceania humpback whale population. Endanger. Species Res. 28, 147–162 (2015)

    Article  Google Scholar 

  • Chapuis, M.-P., Estoup, A.: Microsatellite null alleles and estimation of population differentiation. Mol. Biol. Evol. 24, 621–631 (2007)

    Article  Google Scholar 

  • Evanno, G., Regnaut, S., Goudet, J.: Detecting the number of clusters of individuals using the software STRUCTURE: a simulation study. Mol. Ecol. 14, 2611–2620 (2005)

    Article  Google Scholar 

  • Evett, I., Weir, B.: Interpreting DNA Evidence: Statistical Genetics for Forensic Scientists. Sinauer, Sunderland (1998)

    Google Scholar 

  • Fewster, R.M., Miller, S.D., Ritchie, J.: DNA profiling—a management tool for rat eradication. In: Veitch, C.R., Clout, M.N., Towns, D.R. (eds.) Island Invasives: Eradication and Management, pp. 430–435. IUCN, Gland (2011)

    Google Scholar 

  • Frankham, R.: Effective population size/adult population size ratios in wildlife: a review. Genet. Res. 66, 95–107 (1995)

    Article  Google Scholar 

  • Huelsenbeck, J.P., Andolfatto, P.: Inference of population structure under a Dirichlet process model. Genetics 175, 1787–1802 (2007)

    Article  Google Scholar 

  • Jacob, H.J., Brown, D.M., Bunker, R.K., Daly, M.J., Dzau, V.J., Goodman, A., Koike, G., Kren, V., Kurtz, T., Lernmark, Å., Levan, G., Mao, Y.-P., Pettersson, A., Pravenec, M., Simon, J.S., Szpirer, C., Szpirer, J., Trolliet, M.R., Winer, E.S., Lander, E.S.: A genetic linkage map of the laboratory rat, Rattus norvegicus. Nat. Genet. 9, 63–69 (1995)

    Article  Google Scholar 

  • Kalinowski, S.T., Manlove, K.R., Taper, M.L.: ONCOR: a computer program for genetic stock identification, v.2. Department of Ecology, Montana State University, Bozeman, USA (2008)

  • Link, W.A., Yoshizaki, J., Bailey, L.L., Pollock, K.H.: Uncovering a latent multinomial: analysis of mark-recapture data with misidentification. Biometrics 66, 178–185 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Luikart, G., Ryman, N., Tallmon, D.A., Schwartz, M.K., Allendorf, F.W.: Estimation of census and effective population sizes: the increasing usefulness of DNA-based approaches. Conserv. Genet. 11, 355–373 (2010)

    Article  Google Scholar 

  • Lukacs, P.M., Burnham, K.P.: Estimating population size from DNA-based closed capture–recapture data incorporating genotyping error. J. Wildl. Manag. 69, 396–403 (2005)

    Article  Google Scholar 

  • McClintock, B.T., Bailey, L.L., Dreher, B.P., Link, W.A.: Probit models for capture–recapture data subject to imperfect detection, individual heterogeneity and misidentification. Ann. Appl. Stat. 8, 2461–2484 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • McMillan, L.F., Fewster, R.M.: Visualizing genetic distributions for assignment tests using the saddlepoint approximation method (in review)

  • Miller, S.D., MacInnes, H.E., Fewster, R.M.: Detecting invisible migrants: an application of genetic methods to estimate migration rates. In: Thomson, D.L., Cooch, E.G., Conroy, M.J. (eds.) Modeling Demographic Processes in Marked Populations. Environmental and Ecological Statistics Series, vol. 3, pp. 417–437. Springer, Berlin (2009)

    Chapter  Google Scholar 

  • Otis, D.L., Burnham, K.P., White, G.C., Anderson, D.R.: Statistical inference from capture data on closed animal populations. Wildl. Monogr. 62, 3–135 (1978)

    MATH  Google Scholar 

  • Paetkau, D., Strobeck, C.: Microsatellite analysis of genetic variation in black bear populations. Mol. Ecol. 3, 489–495 (1994)

    Article  Google Scholar 

  • Paetkau, D., Slade, R., Burden, M., Estoup, A.: Genetic assignment methods for the direct, real-time estimation of migration rate: a simulation-based exploration of accuracy and power. Mol. Ecol. 13, 55–65 (2004)

    Article  Google Scholar 

  • Pella, J., Masuda, M.: The Gibbs and split-merge sampler for population mixture analysis from genetic data with incomplete baselines. Can. J. Fish. Aquat. Sci. 63, 576–596 (2006)

    Article  Google Scholar 

  • Piry, S., Alapetite, A., Cornuet, J.-M., Paetkau, D., Baudouin, L., Estoup, A.: GeneClass2: a software for genetic assignment and first-generation migrant detection. J. Hered. 95, 536–539 (2004)

    Article  Google Scholar 

  • Pompanon, F., Bonin, A., Bellemain, E., Taberlet, P.: Genotyping errors: causes, consequences and solutions. Nat. Rev. Genet. 6, 847–859 (2005)

    Article  Google Scholar 

  • Pritchard, J.K., Stephens, M., Donnelly, P.: Inference of population structure using multilocus genotype data. Genetics 155, 945–959 (2000)

    Google Scholar 

  • Rannala, B., Mountain, J.L.: Detecting immigration by using multilocus genotypes. Proc. Natl. Acad. Sci. USA 94, 9197–9201 (1997)

    Article  Google Scholar 

  • Robins, J.H., Miller, S.D., Russell, J.C., Harper, G.A., Fewster, R.M.: Where did the rats of Big South Cape Island come from? N. Z. J. Ecol. 40, 229–234 (2016)

    Article  Google Scholar 

  • Rousset, F.: Genepop’007: a complete reimplementation of the Genepop software for Windows and Linux. Mol. Ecol. Resour. 8, 103–106 (2008)

    Article  Google Scholar 

  • Russell, J.C., Fewster, R.M.: Evaluation of the linkage disequilibrium method for estimating effective population size. In: Thomson, D.L., Cooch, E.G., Conroy, M.J. (eds.) Modeling Demographic Processes in Marked Populations, pp. 291–320. Environmental and Ecological Statistics Series, vol. 3. Springer, Berlin (2009)

  • Russell, J.C., Towns, D.R., Anderson, S.H., Clout, M.N.: Intercepting the first rat ashore. Nature 437, 1107 (2005)

    Article  Google Scholar 

  • Russell, J.C., Miller, S.D., Harper, G.A., MacInnes, H.E., Wylie, M.J., Fewster, R.M.: Survivors or reinvaders? Using genetic assignment to identify invasive pests following eradication. Biol. Invasions 12, 1747–1757 (2010)

    Article  Google Scholar 

  • Schofield, M.R., Bonner, S.J.: Connecting the latent multinomial. Biometrics 71, 1070–1080 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Taberlet, P., Luikart, G.: Non-invasive genetic sampling and individual identification. Biol. J. Linn. Soc. 68, 41–55 (1999)

    Article  Google Scholar 

  • Vale, R.T.R., Fewster, R.M., Carroll, E.L., Patenaude, N.J.: Maximum likelihood estimation for model \(M_{t,\alpha }\) for capture–recapture data with misidentification. Biometrics 70, 962–971 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Veale, A.J., Edge, K.-A., McMurtrie, P., Fewster, R.M., Clout, M.N., Gleeson, D.M.: Using genetic techniques to quantify reinvasion, survival and in-situ breeding rates during control operations. Mol. Ecol. 22, 5071–5083 (2013)

    Article  Google Scholar 

  • Weir, B.S.: Genetic Data Analysis II. Sinauer, Sunderland (1996)

    Google Scholar 

  • Wikipedia: List of countries by sex ratio. Wikipedia (2015).

  • Wright, J.A., Barker, R.J., Schofield, M.R., Frantz, A.C., Byrom, A.E., Gleeson, D.M.: Incorporating genotype uncertainty into mark-recapture-type models for estimating abundance using DNA samples. Biometrics 65, 833–840 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Yoshizaki, J., Brownie, C., Pollock, K.H., Link, W.A.: Modeling misidentification errors that result from use of genetic tags in capture–recapture studies. Environ. Ecol. Stat. 18, 27–55 (2011)

    Article  MathSciNet  Google Scholar 

Download references


This work was funded by the Royal Society of New Zealand through Marsden grant 03-UOA-117. Thanks to Emma Carroll who checked the manuscript for biological soundness, and to Mark Bravington for comments on close-kin mark–recapture and other genetic methods. Many thanks to the associate editor and referees whose insightful comments greatly improved the paper.

Author information

Authors and Affiliations


Corresponding author

Correspondence to R. M. Fewster.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fewster, R.M. Some applications of genetics in statistical ecology. AStA Adv Stat Anal 101, 349–379 (2017).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: