Probabilistic clustering of extratropical cyclones using regression mixture models

Abstract

A probabilistic clustering technique is developed for classification of wintertime extratropical cyclone (ETC) tracks over the North Atlantic. We use a regression mixture model to describe the longitude-time and latitude-time propagation of the ETCs. A simple tracking algorithm is applied to 6-hourly mean sea-level pressure fields to obtain the tracks from either a general circulation model (GCM) or a reanalysis data set. Quadratic curves are found to provide the best description of the data. We select a three-cluster classification for both data sets, based on a mix of objective and subjective criteria. The track orientations in each of the clusters are broadly similar for the GCM and reanalyzed data; they are characterized by predominantly south-to-north (S–N), west-to-east (W–E), and southwest-to-northeast (SW–NE) tracking cyclones, respectively. The reanalysis cyclone tracks, however, are found to be much more tightly clustered geographically than those of the GCM. For the reanalysis data, a link is found between the occurrence of cyclones belonging to different clusters of trajectory-shape, and the phase of the North Atlantic Oscillation (NAO). The positive phase of the NAO is associated with the SW–NE oriented cluster, whose tracks are relatively straight and smooth (with cyclones that are typically faster, more intense, and of longer duration). The negative NAO phase is associated with more-erratic W–E tracks, with typically weaker and slower-moving cyclones. The S–N cluster is accompanied by a more transient geopotential trough over the western North Atlantic. No clear associations are found in the case of the GCM composites. The GCM is able to capture cyclone tracks of quite realistic orientation, as well as subtle associated features of cyclone intensity, speed and lifetimes. The clustering clearly highlights, though, the presence of serious systematic errors in the GCM’s simulation of ETC behavior.

Appendix A: Expectation maximization algorithm

The EM algorithm is an iterative maximum likelihood (ML) procedure that provides a general and efficient framework for parameter estimation. At a base level, EM is an approximate root-finding procedure used to seek the root of the likelihood equation by iteratively searching for a set of parameters that maximize the probability of the observed data. EM is primarily used for finding ML parameter estimates in missing- or hidden-data problems. Parameter estimation in hidden-data problems is difficult because the likelihood equation takes on a complex form, often involving an integral or a sum over the hidden data itself.

For example, Eq. (5) in Sect. 3.3 gives the likelihood of ϕ given both Z and T (repeated here):

$$ L(\phi|{\bf Z},{\bf T}) = p({\bf Z}|{\bf T},\phi) = \prod_i^n \sum_k^K \alpha_k\, f_k({{\bf z}}_i|{{\bf T}}_i{\varvec{\beta}}_k,\Sigma_k). $$

(A1)

Notice that the hidden data in this case are the unknown cluster memberships which must be summed-out of the likelihood to arrive at L(ϕ|Z,T). It is understood in hidden-data problems that this operation cannot be easily carried out. The EM algorithm is an iterative two-step procedure used to circumvent this integration (or sum) by (1) indirectly estimating values for the unobserved data, and (2) finding the ML parameter estimates that correspond to the now completely observed data. The new ML estimates from step (2) are then used to re-estimate the hidden data in step (1), and these iterations are continued until some stopping criterion is reached (typically this involves stopping when the change in log-likelihood falls below a particular threshold, and thus the iterations have stabilized).

In the first step, the E-step, we estimate the hidden cluster memberships by forming the ratio of the likelihood of trajectory i under cluster k, to the sum-total likelihood of trajectory i under all clusters:

$$ w_{ik} = \frac{\alpha_k f_k({{\bf z}}_i|{{\bf T}}_i{\varvec{\beta}}_k,\Sigma_k)} {\sum_j^K \alpha_j f_j({{\bf z}}_i|{{\bf T}}_i{\varvec{\beta}}_j,\Sigma_j)}. $$

(A2)

These w _ik give the probabilities that the ith trajectory was generated from cluster k. They represent a posterior expectation for the value of the actual binary cluster memberships (i.e., the ith trajectory was either generated by the kth cluster or it was not).

In the second step, the M-step, the expected cluster memberships from the E-step are used to form the weighted log-likelihood function:

$$ {\mathcal{L}}(\phi|{\bf Z},{\bf T}) = \sum_i^n \sum_k^K w_{ik} \log \alpha_k\, f_k({{\bf z}}_i|{{\bf T}}_i{\varvec{\beta}}_k,\Sigma_k). $$

(A3)

The membership probabilities weight the contribution that the kth density component adds to the overall likelihood. In the case where the w _ik are binary, and thus cluster membership is perfectly known, this reduces to the usual fully-observed log-likelihood. This weighted log-likelihood is then maximized with respect to the parameter set ϕ.

For the sake of completeness, we give each of the re-estimation equations below. Let ${{{\bf w}}_{ik} = w_{ik}{{\bf I}}_{n_i},}$ where ${{{\bf I}}_{n_i}}$ is an n _i -vector of ones, and let W _k =  diag(w′_1k , ..., w′ _nk ) be an N ×  N diagonal matrix. Then, in the M-step we use W _k to calculate the mixture parameters

$$ \hat{{\varvec{\beta}}}_k = ({{\bf T}}'{{\bf W}}_k{{\bf T}})^{-1}{{\bf T}}'{{\bf W}}_k{{\bf Z}}, $$

(A4)

$$ \hat{\Sigma_k} = \frac{({{\bf Z}}-{{\bf T}}\hat{\beta}_k)'{{\bf W}}_k ({{\bf Z}}-{{\bf T}}\hat{\beta}_k)}{\sum_i^n w_{ik}}, $$

(A5)

and the mixture weights

$$ \hat{\alpha_k} = {\frac{1}{n}} \sum_i^n w_{ik} $$

(A6)

for k = 1,...,K. These update equations are equivalent to the well-known weighted least-squares solution in regression (Draper and Smith 1981). The diagonal elements of W _k represent the weights to be applied to Z and T during the weighted regression.

Because most of the difficult work is carried out in estimating the cluster memberships, the maximization carried out in the M-step is straightforward. This is a common attribute of the EM algorithm. Dempster et al. (1977b) showed that under fairly general conditions, the likelihood will never decrease during the E- and M-step iterations. Due to the presence of local maxima on the likelihood surface, the solution is not guaranteed to correspond to a global maximum. However, we can increase the chances of finding the global maximum by running the EM algorithm multiple times from different starting points in parameter space and selecting the parameters that result in the highest overall likelihood.