AStA Advances in Statistical Analysis

, Volume 101, Issue 4, pp 439–460 | Cite as

Variance estimation for integrated population models

  • Panagiotis BesbeasEmail author
  • Byron J. T. Morgan
Original Paper


State-space models are widely used in ecology. However, it is well known that in practice it can be difficult to estimate both the process and observation variances that occur in such models. We consider this issue for integrated population models, which incorporate state-space models for population dynamics. To some extent, the mechanism of integrated population models protects against this problem, but it can still arise, and two illustrations are provided, in each of which the observation variance is estimated as zero. In the context of an extended case study involving data on British Grey herons, we consider alternative approaches for dealing with the problem when it occurs. In particular, we consider penalised likelihood, a method based on fitting splines and a method of pseudo-replication, which is undertaken via a simple bootstrap procedure. For the case study of the paper, it is shown that when it occurs, an estimate of zero observation variance is unimportant for inference relating to the model parameters of primary interest. This unexpected finding is supported by a simulation study.


Bootstrap Cross-validation Cubic splines Grey heron Mark–recovery–recapture data Overfitting Penalised likelihood Plug-in method Process/observation error estimation State-space models Time-dependent parameters 



We thank the Associate Editor, Roland Langrock, two anonymous referees, Stephen Freeman, Mark Maunder, Leo Polanski and Martin Ridout for their very helpful comments.

Supplementary material

10182_2017_304_MOESM1_ESM.pdf (177 kb)
Supplementary material 1 (pdf 176 KB)


  1. Abadi, F., Gimenez, O., Arlettaz, R., Schaub, M.: An assessment of integrated population models: bias, accuracy, and the violation of the assumption of independence. Ecology 91, 7–14 (2010)CrossRefGoogle Scholar
  2. Barry, S.C., Brooks, S.P., Catchpole, E.A., Morgan, B.J.T.: The analysis of ring-recovery data using random effects. Biometrics 59, 54–65 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  3. Bengtsson, T., Cavanaugh, J.E.: An improved Akaike information criterion for state-space model selection. Comput. Stat. Data Anal. 50, 2635–2654 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  4. Besbeas, P., Morgan, B.J.T.: Kalman filter initialisation for integrated population modelling. Appl. Stat. 61, 151–162 (2011)Google Scholar
  5. Besbeas, P., Morgan, B.J.T.: A threshold model for heron productivity. J. B. Agric. Environ. Stat. 17, 128–141 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  6. Besbeas, P., Morgan, B.J.T.: Goodness of fit of integrated population models using calibrated simulation. Methods Ecol. Evol. 5, 1373–1382 (2014)CrossRefGoogle Scholar
  7. Besbeas, P., Freeman, S.N., Morgan, B.J.T., Catchpole, E.A.: Integrating mark–recapture–recovery and census data to estimate animal abundance and demographic parameters. Biometrics 58, 540–547 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  8. Besbeas, P., Lebreton, J.-D., Morgan, B.J.T.: The efficient integration of abundance and demographic data. Appl. Stat. 52, 95–102 (2003)MathSciNetzbMATHGoogle Scholar
  9. Besbeas, P., Borysiewicz, R.S., Morgan, B.J.T.: Completing the ecological jigsaw. In: D.L. Thomson, E.G. Cooch, and M. J. Conroy (Eds.) Modelling Demographic Processes in Marked Populations. Springer Series: Environmental and Ecological Statistics, vol. 3, pp. 513–540. Springer, Berlin (2009)Google Scholar
  10. Besbeas, P., McCrea, R.S., Morgan, B.J.T.: Integrated population model selection in ecology. University of Kent Technical Report. (2015)
  11. Brooks, S.P., King, R., Morgan, B.J.T.: A Bayesian approach to combining animal abundance and demographic data. Anim. Biodivers. Conserv. 27, 515–529 (2004)Google Scholar
  12. Burnham, K.P., Rexstad, E.A.: Modeling heterogeneity in survival rates of banded waterfowl. Biometrics 49, 1194–1208 (1993)CrossRefzbMATHGoogle Scholar
  13. Chandler, R., Clark, J.: Spatially explicit integrated population models. Methods Ecol. Evol. 5, 1351–1360 (2014)CrossRefGoogle Scholar
  14. Dennis, B., Ponciano, J.M., Lele, S.R., Taper, M.L., Staples, D.F.: Estimating density dependence, process noise and observation error. Ecol. Monogr. 76, 323–341 (2006)CrossRefGoogle Scholar
  15. Dennis, B., Ponciano, J.M., Taper, M.L.: Replicated sampling increases efficiency in monitoring biological populations. Ecology 91, 610–620 (2010)CrossRefGoogle Scholar
  16. de Valpine, P., Hastings, A.: Fitting population models incorporating process noise and observation error. Ecol. Monogr. 72, 57–76 (2002)CrossRefGoogle Scholar
  17. de Valpine, P., Hilborn, R.: State-space likelihoods for nonlinear fisheries time-series. Can. J. Fish. Aquat. Sci. 62, 1937–1952 (2005)CrossRefGoogle Scholar
  18. Durbin, J., Koopman, S.J.: Time Series Analysis by State Space Methods. Oxford University Press, Oxford (2001)zbMATHGoogle Scholar
  19. Francis, R.I.C.C.: Data weighting in statistical fisheries stock assessment models. Can. J. Fish. Aquat. Sci. 68, 1124–1138 (2011)CrossRefGoogle Scholar
  20. Freckleton, R.P., Watkinson, A.R., Green, R.E., Sutherland, W.J.: Census error and the detection of density dependence. J. Anim. Ecol. 75, 837–851 (2006)CrossRefGoogle Scholar
  21. Gonçalves, S., Politis, D.: Discussion: Bootstrap methods for dependent data: a review. J. Korean Stat. Soc. 40, 383–386 (2011)CrossRefzbMATHGoogle Scholar
  22. Green, P., Silverman, B.: Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. Chapman & Hall/CRC Press, Boca Raton (1994)CrossRefzbMATHGoogle Scholar
  23. Kéry, M., Schaub, M.: Bayesian Population Analysis using WinBUGS: A Hierarchical Perspective. Academic Press, Cambridge (2012)Google Scholar
  24. King, R.: A review of Bayesian state-space modelling of capture–recapture–recovery data. Interface Focus 2, 190–204 (2012)CrossRefGoogle Scholar
  25. King, R.: Statistical ecology. Ann. Rev. Stat. Appl. 1, 401–426 (2014)CrossRefGoogle Scholar
  26. Knape, J.: Estimability of density dependence in models of time series data. Ecology 89, 2994–3000 (2008)CrossRefGoogle Scholar
  27. Knape, J., Korner-Nievergelt, F.: Estimates from non-replicated population surveys rely on critical assumptions. Methods Ecol. Evol. (2015). doi: 10.1111/2041-210X.12329 Google Scholar
  28. Knape, J., Besbeas, P., de Valpine, P.: Using uncertainty estimates in analyses of population time series. Ecology 94, 2097–2107 (2013)CrossRefGoogle Scholar
  29. McCrea, R.S., Morgan, B.J.T.: Analysis of Capture–Recapture Data. CRC Chapman & Hall, Boca Raton (2014)zbMATHGoogle Scholar
  30. McCrea, R.S., Morgan, B.J.T., Gimenez, O., Besbeas, P., Bregnballe, T., Lebreton, J.-D.: Multi-site integrated population modelling. J. Biol. Agric. Environ. Stat. 15, 539–561 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  31. Maunder, M.N., Deriso, R.B., Hanson, C.H.: Use of state-space population dynamics models in hypothesis testing: advantages over simple log-linear regressions for modeling survival, illustrated with application to longfin smelt (Spirinchus thaleichthys). Fish. Res. 164, 102–111 (2015)CrossRefGoogle Scholar
  32. Mazzettta, C., Morgan, B.J.T., Coulson, T.: A state-space modelling approach to population size estimation. Technical report, University of Kent Technical Report: UKC/SMSAS/10/025 (2010)Google Scholar
  33. Newman, K.B., Buckland, S.T., Morgan, B.J.T., King, R., Borchers, D.L., Cole, D.J., Besbeas, P.T., Gimenez, O., Thomas, L.: Modelling Population Dynamics: Model Formulation, Fitting and Assessment using State-Space Methods. Springer, New York (2014)CrossRefGoogle Scholar
  34. Patterson, T.A., Parton, A., Langrock, R., Blackwell, P.G., Thomas, L., King. R.: Statistical modelling of individual animal movement: an overview of key methods and a discussion of practical challenges. arXiv:1603.07511v3 [stat.AP] (2017)
  35. Pollock, K.H., Raveling, D.G.: Assumptions of modern band-recovery models, with emphasis on heterogeneous survival rates. J. Wildl. Manag. 46, 88–98 (1982)CrossRefGoogle Scholar
  36. Rice, J.: Bandwidth choice for nonparametric regression. Ann. Stat. 12, 1215–1230 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  37. Schaub, M., Abadi, F.: Integrated population models: a novel analysis framework for deeper insights into population dynamics. J. Ornithol. 152, 227–237 (2011)CrossRefGoogle Scholar
  38. Searle, S.R.: Matrix Algebra Useful for Statistics. Wiley, New York (1982)zbMATHGoogle Scholar
  39. Tavecchia, G., Besbeas, P., Coulson, T., Morgan, B.J.T., Clutton-Brock, T.H.: Estimating population size and hidden demographic parameters with state-space modelling. Am Nat. 173, 722–733 (2009)CrossRefGoogle Scholar
  40. Wang, J.-P., Lindsay, B.G.: A penalized nonparametric maximum likelihood approach to species richness estimation. J. Am. Stat. Assoc. 100, 942–959 (2005)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Department of StatisticsAthens University of Economics and BusinessAthensGreece
  2. 2.National Centre for Statistical Ecology, School of Mathematics, Statistics and Actuarial ScienceUniversity of KentCanterbury, KentEngland, UK

Personalised recommendations