Lagrangian analysis by clustering

Abstract

We propose a new method for obtaining average velocities and eddy diffusivities from Lagrangian data. Rather than grouping the drifter-derived velocities in geographical bins, we group them by nearest-neighbor distance using a clustering algorithm. This yields sets with approximately the same number of observations, covering unequal areas. A major advantage is that, because the number of observations is the same for the clusters, the statistical accuracy is more uniform than with geographical bins. We illustrate the technique using synthetic data from a stochastic model, employing a realistic mean flow. The latter represents the surface currents in the Nordic Seas and is strongly inhomogeneous in space. We use the clustering algorithm to extract the mean velocities and diffusivities and compare the results with the corresponding quantities from the stochastic model. We perform a similar comparison with the means and diffusivities obtained with geographical bins. Clustering is more successful at capturing the mean flow and improves convergence in the eddy diffusivity estimates. We discuss both the advantages and shortcomings of the new method.

Appendix: The clustering algorithm

We base our clustering procedure on a generalized version of the Llloyd’s (1982) algorithm for the problem described by Eq. 5. However, contrary to conventional applications of k-means (MacKay 2003), in our problem, the number of clusters k does not need to be guessed at, but it is deduced from the total amount of data to match the desired number of cluster members m. Hence, we have developed here a procedure to partition the data into clusters with the number of members being as close as possible to a prescribed value m. This heuristic numerical solution is possibly not an optimal one, but it performed well for the purpose of this study. The implementation is done with the MATLAB k-means toolbox, modified accordingly. The steps of the algorithm are as follows:

• Choose the desired number of members in a cluster, m

• Given the total number of independent observations n and m, compute the target number of clusters, k=n/m

• Start k-means procedure (“batch phase”)

  • A random set of k clusters is randomly seeded

  • Assign each point to the nearest cluster center minimizing the squared Euclidean distance in geographical coordinates (Eq. 5)

  • Recompute the new cluster centers

  • The two previous steps continues until the convergence criterion is met (the assignment has not changed or maximum number of iterations is reached, set to be 200 here)

  • The four previous steps are repeated 100 times (for 100 initial seedings, or “replicates”) and the “best solution” (global minimum, that is, the lowest value of the sum of within-cluster distances, summed over all clusters) is the output

• End k-means procedure

• Clusters with the desired number of members are removed from consideration and stored, while the entire clustering procedure is repeated on the smaller data set. The process continues until all the data are grouped in clusters which satisfy m ∈ (m − 5, m + 5), or until maximum number of iterations, 400, is reached. The requirement was not met in some subsets, which considered typically clusters peripheral to the data-covered area. These were still included in the further analysis making the distribution curves in Fig. 4b differ from delta-functions.

Large number of iterations and the requirement of uniform splitting of the data makes the analysis computationally intensive. For that reason, we do not perform a check for a “local minimum” (in terms of Eq. 5) by a series of reassignments of the points between clusters. Nevertheless, we found that repeated runs of the entire procedure described above led merely to a slightly different arrangement of clusters, while the reported results from the Z-test (Fig. 5) changed only within ±2%.

The running time of the entire procedure was ca. 6 h on x86_64 GNU/Linux machine with 32 GB RAM.