Evaluating multi-regional population projections with Taylor’s law of mean–variance scaling and its generalisation

Abstract

Organisations that develop demographic projections usually propose several variants with different demographic assumptions. Existing criteria for selecting a preferred projection are mostly based on retrospective comparisons with observations, and a prospective approach is needed. In this work, we use the mean–variance scaling (spatial variance function) of human population densities to select among alternative demographic projections. We test against observed and projected Norwegian county population density using two spatial variance functions, Taylor’s law (TL) and its quadratic generalisation, and compare each function’s parameters between the historical data and six demographic projections, at two different time scales (long term: 1978–2010 vs. 2011–2040; and short term: 2006–2010 vs. 2011–2015). We find that short-term projections selected by TL agree more accurately than the other projections with the recent county density data and reflect the current high rate of international migration to and from Norway. The variance function method implemented here provides an empirical test of an ex ante approach to evaluating short-term human population projections.

Appendix

Overview of MAPE-R

Rescaled mean absolute percentage error (MAPE-R) is a modified post hoc measure of population forecast accuracy based on mean absolute percentage error (MAPE) developed by Swanson et al. (2000). Since absolute percentage errors (APEs) between observed and projected population sizes are often right-skewed, the central tendency measure, MAPE, is heavily influenced by outliers and tends to overestimate the true error (Rayer 2007). To describe the percentage errors more robustly, taking account of their large outliers, Swanson et al. (2000) first calculated a modified Box-Cox transformation of APEs. The parameter of the modified Box-Cox transformation (Box and Cox 1964) was selected to maximize the likelihood that the transformed APEs were symmetrical, so that the mean became appropriate. Symmetry of APE was evaluated using D’Agostino’s test of skewness (D’Agostino 1970), where a rejection of the null hypothesis of zero skewness indicated that APE was not symmetrically distributed and the Box-Cox transformation was necessary. Then Swanson et al. (2000) calculated the inverse of the mean transformed APEs (MAPE-R) to return them to the original data scale.

Details of the theory and empirical test of MAPE-R can be found in Coleman and Swanson (2007) and Swanson et al. (2000, 2011).

Implementation of MAPE-R

We followed the procedure outlined in Swanson et al. (2011) to implement MAPE-R. To measure the accuracy of Norwegian county population projections between 2011 and 2015 for each of the six demographic projections and each year from 2011 to 2015, we first examined the distribution of APE between the projected and observed county population density and tested its skewness using the right-sided D’Agostino’s test of skewness with significance level 0.1 (Table S1). Here a 0.1 significance level was used instead of a lower value to avoid ‘‘a greater cost in terms of a downwardly biased measure of accuracy in not transforming a potentially skewed distribution’’ (Swanson et al. 2011). If the hypothesis that the skewness of APEs was zero was rejected (P < 0.1), then for each projection from each year, we performed the Box-Cox transformation for APE of each county, estimated the transformation parameter (λ from −2 to 2) that maximized the likelihood function for symmetry, averaged the transformed APEs over counties, and calculated MAPE-R using the inverse of Box-Cox transformation (Box and Cox 1964). Finally, the five MAPE-Rs (one from each year from 2011 to 2015) of each projection were compared among the six projections using the one-way ANOVA for the equality of their inter-annual average (see Materials and methods). We used MAPE for comparison if the hypothesis that the APEs had zero skewness was not rejected. Skewness and the corresponding P of APE and transformed APE (APE-T) were listed in Table S1.

D’Agostino’s test of skewness was performed using the function ‘‘agostino.test’’ from R package ‘‘moments’’ (Komsta and Novomestky 2015). Parameter λ at which the likelihood function achieved maxima was found using R function ‘‘optimize’’ (R Core Team 2015). R code for the calculation of MAPE-R was given below.