Keywords

Introduction

A long-standing challenge for meteorology and climate science has been to develop techniques capable of producing predictions and simulations of the weather and climate across a range of timescales. Although these timescales should be viewed as a continuum, it is nevertheless helpful to identify a number of discrete timescales, ranging from very near term ‘nowcasting’ to climate change projections over centuries and millennia, as shown in Fig. 6.1.

Fig. 6.1
figure 1

Weather and climate timescales, forecasting tools and datasets

Full size image

This chapter briefly reviews how the nature of predictability differs across prediction timescales, identifies the major strategies used to create predictions and discusses some of the general challenges with using predictions in quantitative applications.

The Nature of Predictability

To understand weather and climate forecasting, it is important to understand the character of the physical system one is seeking to predict. The atmosphere (and the climate system more broadly) can be viewed as an example of a chaotic system (e.g., Lorenz 1963), associated with two distinct ‘types’ of predictability (Lorenz 1975, see also Schneider and Griffies 1999). These types—and their relationships to different timescales of weather and climate—are discussed herein.

Due to the complexity of the atmosphere (or climate) system, it is helpful to discuss predictability with reference to an analogous but simpler chaotic system. The Lorenz model contains three inter-dependent variables (U, X and Y) evolving deterministically over time. Each of the three dimensions can be understood as representing a meteorological quantity in analogy (e.g., eastward wind, northward wind and temperature). A typical atmosphere-only climate model will, however, have in excess of ~106 dimensions: one each for six key meteorological properties (wind in the horizontal and vertical, temperature, water vapour and surface pressure) at each point on a 3-dimensional grid (perhaps 192 × 120 × 30). In some cases, the time evolution of the equations may also include a stochastic (random) component rather than being purely deterministic. The equations of the Lorenz model may be written in finite difference form:

$$ {X}_t={X}_{t-1}+\left[\alpha \left({Y}_{t-1}-{X}_{t-1}\right)\right]\ \delta t $$
$$ {Y}_t={Y}_{t-1}+\left[{X}_{t-1}\left(\beta -{U}_{t-1}\right)-{Y}_{t-1}+\epsilon \right]\ \delta t $$
$$ {U}_t={U}_{t-1}+\left[{X}_{t-1}{Y}_{t-1}-\gamma {U}_{t-1}\right]\ \delta t $$

where α, β and γ are constant parameters, subscripts denote time-steps, and δt is the interval between adjacent time-steps. Following Palmer (1999), ϵ is used to denote a small external forcing (for the initial discussion it is assumed that ϵ = 0).

The time evolution of the Lorenz system can be represented as a trajectory (or path) in phase space.Footnote 1 Figure 6.2a shows a short section of a trajectory as an example: from an initial state near (U, X, Y) = (33, 15, 18), the model evolves to a state (24,-12,-18) over a ‘time’ interval ∑δt = 0.7. If the model is allowed to evolve for a longer period to produce a more extended trajectory (referred to as an attractor), a fuller view of the system’s properties emerges (Fig. 6.2b). The attractor clearly shows two lobes, with the system preferring to occupy states in one or the other of the lobes.

Fig. 6.2
figure 2

The Lorenz model and initial condition problems, using α = 10, β = 28, γ = 8/3 and ϵ = 0. See text for discussion

Full size image

Figure 6.2 can be used to illustrate an example of an initial condition problem or ‘predictability of the first kind’. Consider at some time t 1 that observations of the system are taken, and found to be U 1 = 30 and X 1 = 4. Assume that each of these measurements is subject to an observational error, ∆U 1 = ∆X 1 = 1, and Y 1 is unobservable in practice.Footnote 2 The systems current position in phase space is, therefore, not known exactly, but can be constrained to a relatively small region of phase space indicated by the black box in Fig. 6.2b. A prediction of the system’s state at some future snapshot in time, t 2 = t 1 + ∆t is sought—that is, we wish to predict the exact values of U2 and X2. This is analogous to weather forecasting: we ‘know’ the weather today and wish to predict the weather tomorrow.

There are many possible phase space trajectories that are consistent with the available knowledge of the initial conditions at time t 1. A selection of these are shown in Fig. 6.2c: some lead to the left lobe (black lines), whereas others remain in the right lobe B (grey lines). In consequence, a relatively small error in estimating the starting state (∆U 1 = ∆X 1 = 1) grows rapidly to a large error in the prediction (∆U 2 ~ ∆X 2 ~ 30). The rate of error growth is, however, very dependent on the initial state and the forecast time horizon considered; in this example there is very low predictability but, if the initial conditions correspond to some other regions of the attractor, there may be much more predictability (i.e., smaller errors), at least over short time horizons (i.e., small ∆t).

The evolution, shape and position of a trajectory are also sensitive to the model’s parameters (α, β and γ), typically referred to as boundary conditions. Figure 6.3 shows the original attractor from the previous figure (in black) and a new attractor (in grey)—the only difference is a small change in one of the boundary conditions, ϵ. Clearly, one can detect a change in the probability distribution of the observable quantity, U, as indicated by the relative frequency distributions in the bottom panel in Fig. 6.3. This is an example of ‘predictability of the second kind’, which concerns the ability to predict changes in the attractor in response to changes in external boundary conditions.Footnote 3 Clearly, if the response is large, then it can be more readily detected against the ‘internal’ variability corresponding to trajectories moving within a single attractor.

Fig. 6.3
figure 3

The Lorenz model and the long-term equilibrium climate change problem. The black and grey curves show two simulations with different boundary conditions (parameters as in Fig. 6.2, but with ϵ = 10 for the grey curve). See text for discussion

Full size image

Traditional climate change simulations studying the equilibrium climate under a future greenhouse gas concentration scenario can be viewed as an example of second kind predictability: it is a boundary condition prediction problem where one seeks to understand how the statistics of climate differ between two different sets of boundary conditions (e.g., Meehl et al. 2007).

If numerical weather prediction (NWP) (days-ahead) and long-term climate change simulations (decades-ahead) can be considered as examples of initial condition and boundary condition problems, then it is clear that much lies between these two extremes. It is therefore helpful to consider the timescales involved in the system one is seeking to predict.

The climate system in general contains many different components, varying on a wide range of timescales (Fig. 6.4). At forecast lead times of 1–2 days, it is typically sufficient to focus on the evolution of the faster components alone (e.g., troposphere and land-surface temperature) as the slower components (e.g., ocean temperature, ice sheets) change little during the lifetime of the prediction. Indeed, at very short lead times (minutes to hours) many aspects of the large-scale flow in the troposphere may even be considered fixed. Conversely, at longer forecast lead times, the evolution of slower components become significant (e.g., ocean circulation, land-surface moisture, ice sheets, sea ice and snow cover). For a prediction of tomorrow’s weather in London, it may, therefore, be sufficient to neglect small changes in the temperature of the North Atlantic ocean’s surface, but the same cannot be said for predicting the seasonal-average temperature over Europe several months in advance.

Fig. 6.4
figure 4

Indicative timescales of selected components in the climate system

Full size image

Weather and climate predictions (as outlined in Fig. 6.1)—particularly those in the range of several days to a few decades—are therefore a mixture of initial and boundary condition problems, and have been the subject of much research in recent years. This can be illustrated by considering, for example, a seasonal forecast. In such a forecast, the state of the troposphere and land-surface temperature change much more quickly (~days) than the forecast horizon (~months). From the perspective of these components, the problem is therefore boundary condition prediction and predictability of the second kind (i.e., the intention is to predict the statistical properties of the troposphere rather than estimate its state at a specific snapshot in time). However, for the ocean surface, soil moisture, snow cover and stratosphere—which only change slowly over the timescale of the forecast—the challenge is to determine the specific evolution, therefore concerns predictability of the first kind where initial conditions play a key role.

Prediction Strategies

There are two broad categories of predictive models used in weather and climate forecasting: statistical and dynamical. The primary characteristics of each are outlined below.

Statistical Models

Statistical predictions come in many forms and are in widespread use throughout academia and industry for many problems. Conceptually, these models are simple: historical observations are interrogated to find relationships between a predictand and a set of potential predictors. These historical relationships are then assumed to remain fixed into the future and are used to create a prediction. Common examples include single- or multi-variate autoregressive models (or more sophisticated versions such as ARMA and GARCH), artificial neural networks, support vector machines and ‘analogue’-based techniques.

Statistical models are an undoubtedly powerful prediction tool. It must, however, be recognised that there are limitations and dangers associated with this approach. The process of statistical modelling is essentially ad-hoc: the predictors to be used are not necessarily known a priori (unless informed by some prior physical or dynamical process understanding), and must, therefore, be established afresh for each new predictand. The ability to identify statistical relationships between predictors and predictand is also constrained by the quantity and quality of the available historic data: the records must be sufficiently long and homogeneous to robustly establish statistical relationships between variables. Finally, statistical models trained on historic data may ultimately be a rather poor guide for a climate system subject to changing boundary conditions (e.g., greenhouse gas concentrations). This is perhaps particularly the case for longer range forecasts where many plausible future states will simply not have been recorded in historic observations

Dynamical Models

In contrast to statistical models, dynamical prediction models numerically simulate the behaviour of the system itself and are the basis of the weather forecasts provided by most operational weather services. In NWP, the atmosphere is represented by fundamental physical equations based on the laws of motion, thermodynamics and conservation of mass. The equations are discretised in space and time (i.e., the atmosphere is divided into grid boxes) and solved iteratively in each grid box, advancing time-step by time-step. Additional physical and dynamical processes such as clouds, precipitation and radiation are represented through ‘parameterization schemes’ at the grid box level.

As discussed in section ‘The Nature of Predictability’, initial condition errors in NWP can grow rapidly over a short period (Fig. 6.2c) and ‘ensemble forecasting’ is widely used. An NWP ensemble consists of a large set of N individual ‘member’ weather forecast simulations, each starting from a slightly different set of initial or boundary conditions (all of which are consistent with the observations, subject to observational error). Typically, the set of initial conditions chosen will seek to maximise the difference between the ensemble members at the targeted forecast time—that is, to capture the widest possible range of uncertainty associated with errors in the initial conditions. Such forecasts, although still ‘predictions of the first kind’, provide N different potential realisations of the future weather state and must be interpreted probabilistically (typically N ~ 50 in many present NWP systems) with the ‘spread’ of outcomes in a good forecast system providing an indication of the predictability available from the particular set of initial conditions used.

Beyond a few days to a couple of weeks, additional climate system components must also be included in addition to the atmosphere (see Fig. 6.4 and section ‘The Nature of Predictability’), such as the stratosphere, oceans and sea ice. The resulting dynamical models draw strongly on the heritage of General Circulation Models (GCMs). First developed in the 1960s–1980s—see, for example, Smagorinsky et al. 1965 for a very early example—and continually developed since, GCMs are identical in concept to NWP models—insofar as they represent a physical model of the system one is attempting to predict or understand—but as they include more physical processes and must be run over longer timescales, they typically use much coarser resolution grid boxes than NWP. As in NWP, the use of ensembles in climate model simulations is common. In this case, however, the ensemble is typically used to sample several different sources of uncertaintyFootnote 4: natural climate variability (Deser et al. 2012), initial condition uncertainty (Scaife et al. 2014), parametric uncertainty (Stainforth et al. 2005) or model structural uncertainty (Taylor et al. 2012).

The power of dynamical models to simulate weather and climate is considerable. The skill of NWP models at lead times of several days ahead has increased continuously over recent decades, and GCMs are now used to produce very sophisticated realisations of physical phenomena affecting a wide range of industrial sectors. There are, however, limitations. NWP and GCM models are computationally expensive compared to statistical models, leading to a three-way trade-off between resolution (grid size), physical complexity (number of processes modelled) and computational feasibility (number of model years simulated and ensemble size). All dynamical models are subject to biases and/or growth of prediction error from many sources, such as deficiencies in model formulation (numerical approximations, missing processes), parametric uncertainty (ill-constrained properties in parameterisation schemes) and initial conditions. Before predictions are used, care should be taken to establish whether dynamical models produce a reliable representation of any particular phenomena of interest

Summary and Discussion

There are good reasons to believe that predictability exists in weather and climate forecasting across a range of timescales from hours to decades and beyond. This predictability may take one of two forms: either a prediction of the specific evolution of the weather (an initial condition problem) or else a prediction of the statistical properties of the climate (a boundary condition problem). Weather and climate forecasts in the intermediate range (several days to decades) typically incorporate some aspects of both forms of predictability, and a probabilistic approach to the resulting forecast is essential.

Both the statistical and dynamical approaches discussed above have great power in terms of achieving predictive skill. It is, however, emphasised that the two approaches should be seen as being complementary toolkits rather than competing philosophical strategies. Statistical methods are often used to ‘calibrate’ dynamical model output (reduce bias when compared against point observations) and configure dynamical models forecasts (e.g., by statistically identifying key boundary conditions such as sea-surface temperature patterns). Conversely, dynamical models enable deeper process understanding (helping to identify robust predictors for statistical models) and—with care—can be used to extend datasets by providing plausible artificially generated climate data (statistical robustness and rare events, including effects of a changing climate).