A trend is a change in the statistical properties of the background state of a system (Chandler and Scott 2011). The simplest case is a linear trend, in which, when applicable, we need to specify only the trend coefficient and its uncertainty. Natural systems evolve continuously over time, and it is not always appropriate to approximate the background evolution with a constant trend. Furthermore, the time series can include multiple time dependent cycles, and they are typically non-stationary, i.e., their distributional properties change over time.
In this work, we apply dynamic regression analysis by using dynamic linear model (DLM) approach to time series analysis of Finnish temperatures. DLM is used to statistically describe the underlying processes that generate variability in the observations. The method will effectively decompose the series into basic components, such as level, trend, seasonality, and noise. The components can be allowed to change over time, and the magnitude of this change can be modeled and estimated. The part of the variability that is not explained by the chosen model is assumed to be uncorrelated noise and we can evaluate the validity of this assumption by statistical model residual diagnostics.
Our model is, of course, just one possibility to describe the evolution of the observed temperatures. We see it as a very natural extension to non-dynamic multiple linear regression model. The method allows us to estimate both the model states (e.g. time-varying trends) and the model parameters (e.g. variances related to temporal variability), and we can assess the uncertainties and statistical significance of the underlying features. In this study, we are not trying to use the model to predict future temperatures, but to detect trends by finding a description that is consistent with the observed temperature variability. To study the adequacy of our chosen model, we examine the model residuals to see if the modeling assumptions are fulfilled.
With a properly set-up and estimated DLM model, we can detect significant changes in the background state and estimate the trends. The magnitude of the trend is not prescribed by the modeling formulation, and the method does not favor finding a “statistically significant” trend. The statistical model provides a method to detect and quantify trends, but it does not directly provide explanations for the observed changes, i.e., whether for example natural variability or solar effects could explain the changes in the background level. Model diagnostics and the increase in the observational data will eventually falsify incorrect models and other poorly selected prior specifications (see e.g. Tarantola 2006).
Dynamic linear models are linear regression models whose regression coefficients can depend on time. This dynamic approach is well known and documented in time series literature (Chatfield 1989; Harvey 1991; Hamilton 1994; Migon et al. 2005). These models are sometimes called structural time series models or hidden Markov models. The latter comes from the fact that dynamic regression is best described by the state space approach where the hidden state variables describe the time evolution of the components of the system. Modern computationally oriented references of the state space approach include Petris et al. (2009) and Durbin and Koopman (2012). The first describes a software package dlm for R statistical language that can be used to do the calculations described in this paper. We have used the Matlab software and computer code described in Laine et al. (2014). In this work, we use a DLM to explain variability in the temperature time series using components for a smooth varying locally linear mean level, for a seasonal effect, and for noise that is allowed to have autoregressive correlation. The autoregressive stochastic error term is used to account for long-range dependencies, irregular cycles, and the effects of different forcing mechanisms that a model with only second order random walk for mean and stochastic seasonality does not suffice to explain.
A DLM can be formulated as a general linear state space model with Gaussian errors and written with an observation equation and a state evolution equation as
$$ y_{t} = F_{t} x_{t} + v_{t} ,\,\,\,v_{t} \sim N\left( {0,V_{t} } \right), $$
(1)
$$ x_{t} = G_{t} x_{t - 1} + w_{t} ,\,\,\,\,\,w_{t} \sim N(0,W_{t} ), $$
(2)
where \( y_{t} \) are the observations and \( x_{t} \) is a vector of unobserved states of the system at time t. Matrix \( F_{t} \) is the observation operator that maps the hidden states to the observations and matrix \( G_{t} \) is the model evolution operator that provides the dynamics of the hidden states. We assume that the uncertainties, represented by observation uncertainty \( v_{t} \) and model error \( w_{t} \) are Gaussian, with observation uncertainty covariance \( V_{t} \) and model error covariance \( W_{t} \). The time index \( t \) will go from 1 to n, the length of the time series to be analyzed. In this work, we analyze univariate temperature time series, but the framework would also allow the modeling of multivariate series. We use notation common to many time series textbooks, e.g., Petris et al. (2009).
Trend will be defined as a change in the mean state of the system after all known systematic effects, such as seasonality, have been accounted for. To build a DLM for the trend we start with a simple local level and trend model that has two hidden states \( x_{t} = \left[ {\begin{array}{*{20}c} {\mu_{t} } & {\alpha_{t} } \\ \end{array} } \right]^{T} \), where \( \mu_{t} \) is the mean level and \( \alpha_{t} \) is the change in the level from time t-1 to time t. This system can be written by the equations
$$ y_{t} = \mu_{t} + \varepsilon_{\text{obs}} ,\,\,\,\varepsilon_{\text{obs}} \sim N(0,\sigma_{t}^{2} ), $$
(3)
$$ \mu_{t} = \mu_{t - 1} + \alpha_{t} + \varepsilon_{\text{level}} ,\,\,\,\,\varepsilon_{\text{level}} \sim N(0,\sigma_{\text{level}}^{2} ), $$
(4)
$$ \alpha_{t} = \alpha_{t - 1} + \varepsilon_{\text{trend}} ,\,\,\,\varepsilon_{\text{trend}} \sim N(0,\sigma_{\text{trend}}^{2} ). $$
(5)
The Gaussian stochastic “ε” terms are used for the observation uncertainty and for random dynamics of the level and the trend. In terms of the state space Eqs. (1) and (2) this model is written as
$$ x_{t} = \left[ {\mu_{t } \alpha_{t} } \right], G_{\text{trend}} = \left[ {\begin{array}{*{20}c} 1 & 1 \\ 0 & 1 \\ \end{array} } \right], F_{\text{trend}} = \left[ {\begin{array}{*{20}c} 1 & 0 \\ \end{array} } \right],W_{\text{trend}} = \left[ {\begin{array}{*{20}c} {\sigma_{\text{level}}^{2} } & 0 \\ 0 & {\sigma_{\text{trend}}^{2} } \\ \end{array} } \right]{\text{and}} V_{t} = \left[ {\sigma_{t}^{2} } \right]. $$
(6)
Note that only the state vector \( x_{t} \) and the observation uncertainty covariance (a \( 1 \times 1 \) matrix) depend on time t. Depending on the choice of the variances \( \sigma_{\text{level}}^{2} \) and \( \sigma_{\text{trend}}^{2} \), the mean state \( \mu_{t} \) will define a smoothly varying background level of the time series. In our analyses, we will set \( \sigma_{\text{level}}^{2} = 0 \) and estimate \( \sigma_{\text{trend}}^{2} \) from the observations. As noted by Durbin and Koopman (2012), this will result in an integrated random walk model for the mean level \( \mu_{t} \), which can be interpreted as a cubic spline smoother, with well-based statistical descriptions of the stochastic components.
Temperature time series exhibit strong seasonal variability. In our DLM, the monthly seasonality is modeled with 11 state variables, which carry information of the seasonal effects of individual months. In general, the number of states is one less than the number of observations for each seasonal cycle when the model already has the mean level term. The corresponding matrices \( G_{\text{seas}} \), \( F_{\text{seas}} \) (\( 11 \times 11 \) and \( 1 \times 11 \) matrices) and the error covariance matrix \( W_{\text{seas}} \) (\( 11 \times 11 \)) for the time-wise variability in the seasonal components are modeled as (Durbin & Koopman, 2012):
$$ G_{\text{seas}} = \left[ {\begin{array}{*{20}c} 1 & { - 1} & { - 1} & \cdots & { - 1} \\ 1 & 0 & 0 & {} & 0 \\ 0 & 1 & 0 & {} & 0 \\ \vdots & {} & \ddots & {} & \vdots \\ 0 & \cdots & 0 & 1 & 0 \\ \end{array} } \right], F_{\text{seas}} = \left[ {\begin{array}{*{20}c} 1 & 0 & \ldots & 0 \\ \end{array} } \right], W_{\text{seas}} = \left[ {\begin{array}{*{20}c} {\sigma_{\text{seas}}^{2} } & 0 & \cdots & 0 \\ 0 & 0 & {} & {} \\ \vdots & {} & \ddots & {} \\ 0 & {} & {} & 0 \\ \end{array} } \right]. $$
(7)
We allow autocorrelation in the residuals using a first order autoregressive model (AR(1)). In DLM settings, we can estimate the autocorrelation coefficient and the extra variance term \( \sigma_{\text{seas}}^{2} \) together with the other model parameters. For a first order autoregressive component with a coefficient ρ and an innovation variance, \( \sigma_{AR}^{2} \), we simply define
$$ G_{AR} = \left[ \rho \right], F_{AR} = \left[ 1 \right], W_{AR} = \left[ {\sigma_{AR}^{2} } \right], $$
(8)
and both ρ and \( \sigma_{AR}^{2} \) can be estimated from the observations.
The next step in the DLM model construction is the combination of the selected individual model components into larger model evolution and observation equations by
$$ G = \left[ {\begin{array}{*{20}c} {G_{\text{trend}} } & 0 & 0 \\ 0 & {G_{\text{seas}} } & 0 \\ 0 & 0 & {G_{AR} } \\ \end{array} } \right], F = \left[ {\begin{array}{*{20}c} {F_{\text{trend}} } & {F_{\text{seas}} } & {F_{AR} } \\ \end{array} } \right], \,W = \left[ {\begin{array}{*{20}c} {W_{\text{trend}} } & 0 & 0 \\ 0 & {W_{\text{seas}} } & 0 \\ 0 & 0 & {W_{AR} } \\ \end{array} } \right], $$
(9)
and the analysis then proceeds to the estimation of the variance parameters and other parameters in model formulation (e.g. the AR coefficient ρ in the matrix \( G_{\text{AR}} \)), and to the estimation of the model states by state space Kalman filter methods.
To get more intuitive meaning of the model and the stochastic error terms involved, we write the observation equation for our model as
$$ y_{t} = \mu_{t} + \gamma_{t} + \eta_{t} + \varepsilon_{t} , t = 1, \ldots , n, $$
(10)
where \( y_{t} \) is the monthly temperature at time t, \( \mu_{t} \) is the mean temperature level, \( \gamma_{t} \) is the seasonal component for monthly data, \( \eta_{t} \) is an autoregressive error component, and \( \varepsilon_{t} \) is the error term for the uncertainty in the observed temperature values. The simplification \( \sigma_{\text{level}}^{2} = 0 \) in Eq. (4) allows us to write a second difference process for the mean level \( \mu_{t} \) as
$$ {{\Updelta}}^{2} \mu_{t} = \mu_{t - 2} - 2\mu_{t - 1} + \mu_{t} + \varepsilon_{\text{trend}},\,\,\,{\text{with}}\,\,\varepsilon_{\text{trend}} \sim N(0,\sigma_{\text{trend}}^{2} ), $$
(11)
see e.g. Durbin and Koopman (2012) Sect. 2.3.1. For the seasonal component \( \gamma_{t} \), we have a condition that the 12 consecutive monthly effects sum to zero on the average, so for each t:
$$ \mathop \sum \limits_{i = 0}^{11} \gamma_{t - i} = \varepsilon_{seas} ,\,\,{\text{with}}\,\,\varepsilon_{\text{seas}} \sim N(0,\sigma_{\text{seas}}^{2} ). $$
(12)
The term \( \eta_{t} \) follows a first order autoregressive process, AR(1), with coefficient ρ:
$$ \eta_{t + 1} = \rho \eta_{t} + \varepsilon_{AR} ,{\text{with}}\,\,\varepsilon_{AR} \sim N(0,\sigma_{AR}^{2} ). $$
(13)
Finally, the observation uncertainty term \( \varepsilon_{t} \) is assumed to be zero mean Gaussian as
$$ \varepsilon_{t} \sim N\left( {0,\sigma_{t}^{2} } \right), $$
(14)
where the observation standard deviations \( \sigma_{t} \) are assumed to be known and correspond to the uncertainties from the spatial representativeness of the observations and from the averaging and the homogenization processes (Tietäväinen et al. 2010). The additional error terms \( \sigma_{\text{trend}}^{2} \), \( \sigma_{\text{seas}}^{2} \), and \( \sigma_{AR}^{2} \) account for the modeling error in the components of the model and are estimated from the data.
In the model construction above, we have four unknown model parameters: the three variances for stochastic model evolution, \( \sigma_{\text{trend}}^{2} \), \( \sigma_{\text{seas}}^{2} \), \( \sigma_{AR}^{2} \) and the autoregressive coefficient ρ. If the values of these parameters are known, the state space representation and the implied Markov properties of the processes allow estimation of the marginal distributions of the states given the observations and parameter by the Kalman filter and Kalman smoother formulas (Durbin & Koopman, 2012). The Kalman smoother gives efficient recursive formulas to calculate the marginal distribution of model states at each time t given the whole set of observations \( y_{t} , t = 1, \ldots ,n \). In a DLM these distributions are Gaussian, so defined by a mean vector and a covariance matrix. In addition, the auxiliary parameter vector θ = [\( \sigma_{\text{trend}}^{2} \), \( \sigma_{\text{seas}}^{2} \), \( \sigma_{AR}^{2} \), ρ] can be estimated using a marginal likelihood function that is provided as a side product of the Kalman filter recursion. This likelihood can be used to estimate the parameter θ using maximum likelihood method and the obtained estimates can be plugged back to the equations. We use Bayesian approach and Markov chain Monte Carlo (MCMC) simulation to estimate the posterior distribution of θ and to account for its uncertainty in the trend analysis.
The level component \( \mu_{t} \) models the evolution of the mean temperature after the seasonal and irregular noise components have been filtered out. It allows us to study the temporal changes in the temperature. The trends can be studied visually, or by calculating trend related statistics from the estimated mean level component \( \mu_{t} \). Statistical uncertainty statements can be given by simulating realizations of the level component using MCMC and the Kalman simulation smoother (Durbin and Koopman 2012, Laine et al., 2014).
The strength of the DLM method is its ability to estimate all model components, such as trends and seasonality, in one estimation step and to provide a conceptually simple decomposition of the observed variability. Furthermore, the analysis does not require assumptions about the stationarity of the series in the sense required, e.g., in classical ARIMA time series analyses and ARIMA analyses can be seen as special cases of the DLM analyses. For example, the simple local level and trend DLM of Eqs. (3–5) is equivalent to the ARIMA (0,2,2) model. In addition, the state space methods can easily handle missing observations; they are extendible to non-linear state space models, to hierarchical parameterizations, and to non-Gaussian errors (e.g. Durbin and Koopman 2012 and Gonçalves and Costa 2013). Details of the construction procedure of a DLM model and estimations of model states and parameters can be found in Gamerman (2006) and in Petris et al. (2009). We use an efficient adaptive MCMC algorithm by Haario et al. (2006) and the Kalman filter likelihood to estimate the four parameters in θ. The details of the estimation procedure can be found in Laine et al. (2014) who use similar DLM model to study trends in stratospheric ozone concentrations. We also conducted our analyses with dlm-package in R-software (Petris 2010) to verify the computations.