Estimation of Innovations State Space Models

For any innovations state space model, the initial (seed) states and the parameters are usually unknown, and therefore must be estimated. This can be done using maximum likelihood estimation, based on the innovations representation of the probability density function.

In Chap. 3 we outlined transformations (referred to as “general exponential smoothing”) that convert a linear time series of mutually dependent random variables into an innovations series of independent and identically distributed random variables. In the heteroscedastic and nonlinear cases, such a representation remains a viable approximation in most circumstances, an issue to which we return in Chap. 15. These innovations can be used to compute the likelihood, which is then optimized with respect to the seed states and the parameters. We introduce the basic methodology in Sect. 5.1. The estimation procedures discussed in this chapter assume a finite start-up; consideration of the infinite start-up case is deferred until Chap. 12.

Any numerical optimization procedure used for this task typically requires starting values for the quantities that are to be estimated. An appropriate choice of starting values is important. The likelihood function may not be unimodal, so a poor choice of starting values can result in sub-optimal estimates. Good starting values (i.e., values that are as close as possible to the optimal estimates) not only increase the chances of finding the true optimum, but typically reduce the computational loads required during the search for the optimum solution. In Sect. 5.2 we will discuss plausible heuristics for determining the starting values.