The surface maps of the previous chapter showed that random fluctuations can be quite large. The present chapter on smoothing mortality data explains how we smoothed the observed mortality with P-splines. We illustrate our smoothing results with the same set of countries as in the previous chapter for unsmoothed data.
- Random Fluctuations
- Smoothed Death Rates
- Bold Solid Black Line
- Mortality Surface
- Mortality Trajectories
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
5.1 From Raw Death Rates to Smoothed Death Rates
We have seen in the previous chapter, that “raw” death rates can suffer from considerable random fluctuations. Assuming that data quality is not an issue, this noise can be caused by (1) very few numbers of deaths (numerator), by (2) very few persons exposed to the risk of dying (denominator) or by (3) small populations in general. Problem (1) typically occurs at young ages. We selected age 15 in France in Panel (a) of Fig. 5.1. Despite a large population in general, deaths occur—thankfully—relatively rarely at that age. (2) The opposite is true at advanced ages as shown in the middle panel of the same figure. Very few people are still alive at age 95 in Italy, although it is a large population having relatively high life expectancy. Problems (1) and (2) occur in countries with tens of millions of people only at young and old ages. The smaller the population size, the more ages are affected. Panel (c) illustrates issue (3) using Danish data. The mortality trajectory in highly developed countries is rather smooth around age 80. In countries with just a few millions of people, considerable random fluctuations can be even observed there. Please note that more than five million people live in Denmark. Hence, the challenge becomes even bigger in smaller countries such as the Baltic states, Luxembourg or, especially, in Iceland.
We decided therefore to smooth the data. Myriads of methods exist to smooth data. While the pattern over age can be appropriately captured by parametric models, the trajectory over time differs considerably between ages and countries. Our decision was therefore to use a non-parametric smoothing approach. We selected the so-called P-spline approach, originally developed by Eilers and Marx (1996), adapted to the analysis of mortality by Currie et al. (2004) and further refined by Camarda (2008). The author, Carlo Giovanni Camarda, also provides the R extension package “MortalitySmooth” (Camarda 2012), which makes it easy and straightforward to apply the method. At its core, the model assumes Poisson distributed death counts with the (log-)exposures as an offset to account for changing population sizes over time and/or age. The method uses B-splines as regression bases. Whereas the number and position of the basis functions is crucial for standard smoothing with B-splines, the P-spline approach uses “too many” bases, which would normally result in overfitting. The P in the name of the method refers to the penalization of adjacent regression coefficients that differ too much from each other. Further technical details about the basis functions, the order of the differences, the penalty term λ, etc. are extensively discussed in the aforementioned references. The bold solid black lines in each panel of Fig. 5.1 depict the data smoothed with P-splines for the three given ages over time. One can easily recognize that the selected smoothing method is flexible enough to model irregular developments but is not prone to overfit the data.
The univariate time series of Fig. 5.1 is synthetic. Only cartoon characters such as Bart Simpson or Eric Cartman can retain their age over time. In reality, each individual is 1 year later 1 year older. Therefore we smoothed the data simultaneously over age and time using the function Mort2Dsmooth of Camarda’s package “MortalitySmooth” (2012).
Raw death rates for Estonian women aged 60–80 years from 1980 to 2000 are illustrated in the left panel of Fig. 5.2 as a three-dimensional mortality surface. The general shape of increasing mortality over age can easily be observed. The right panel, featuring smoothed data, also shows the decline in mortality at higher ages over time, which is difficult to track down in the presence of noise in the data. The selected three-dimensional perspective plot appears appealing at first sight. The choice of angle and elevation is somehow arbitrary, though, and allows to accentuate certain features and suppress others. Since we often want to use the mortality surface for exploratory purposes, we have to give equal exposure to each unit. Therefore, we projected the three-dimensional data on the two-dimensional Lexis-plane, denoting the level of mortality by different colors (see Fig. 5.3 as an example).
Comparable to topographic maps, we added contour lines to depict the same levels of mortality. The general upward tendency of the contour lines indicate that the same level of mortality is shifting to higher and higher ages. Thus, for a given age mortality is decreasing, resulting in an increase in life expectancy.
Figures 5.4, 5.5, 5.6, 5.7, 5.8, 5.9, 5.10, and 5.11 depict the same set of countries as Figs. 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, and 4.8 in Chap. 4 for a proper comparison between “raw” rates and smoothed rates.Footnote 1 The smoothed surface maps make the major trends in the data more pronounced such as almost parallel straight upward lines in Australia, Spain, and Switzerland or the sudden survival improvements in survival among young Spanish men, starting in about 1990. Also large random fluctuations due to very few deaths as we have seen in the plot of raw death rates among children in Switzerland (Figs. 4.5 and 4.6) are removed by the smoothing procedure. While smoothing intrinsically involves some dampening of sudden changes in trends, the automatic procedure to find the optimal penalizing λs still
feature, for instance, the mortality crises among Russian men during the 1980s and 1990s. We do not want to go into further detail here as these smoothed surface maps serve as the major building blocks for the surface maps of rates of mortality improvement, which are the focus of our book and are presented in the next chapter.
The appendix contains therefore also maps of smoothed death rates for France, England & Wales, and Norway. They can be found in Figs. A.9, A.10, A.11, A.12, A.13, and A.14.
Camarda, C. G. (2008). Smoothing methods for the analysis of mortality development. PhD thesis, Universidad Carlos III de Madrid.
Camarda, C. G. (2012). MortalitySmooth: An R package for smoothing Poisson counts with P-splines. Journal of Statistical Software, 50(1), 1–24.
Currie, I. D., Durban, M., & Eilers, P. H. (2004). Smoothing and forecasting mortality rates. Statistical Modelling, 4, 279–298.
Eilers, P. H. C., & Marx, B. D. (1996). Flexible Smoothing with B-splines and Penalties. Statistical Science, 11(2), 89–102.
Rights and permissions
This chapter is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
© 2018 The Author(s)
About this chapter
Cite this chapter
Rau, R., Bohk-Ewald, C., Muszyńska, M.M., Vaupel, J.W. (2018). Surface Plots of Smoothed Mortality Data. In: Visualizing Mortality Dynamics in the Lexis Diagram. The Springer Series on Demographic Methods and Population Analysis, vol 44. Springer, Cham. https://doi.org/10.1007/978-3-319-64820-0_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-64818-7
Online ISBN: 978-3-319-64820-0
eBook Packages: Social SciencesSocial Sciences (R0)