Algorithmic Patterns for $\mathcal {H}$-Matrices on Many-Core Processors

Abstract

In this work, we consider the reformulation of hierarchical ($\mathcal {H}$) matrix algorithms for many-core processors with a model implementation on graphics processing units (GPUs). $\mathcal {H}$ matrices approximate specific dense matrices, e.g., from discretized integral equations or kernel ridge regression, leading to log-linear time complexity in dense matrix–vector products. The parallelization of $\mathcal {H}$ matrix operations on many-core processors is difficult due to the complex nature of the underlying algorithms. While previous algorithmic advances for many-core hardware focused on accelerating existing $\mathcal {H}$ matrix CPU implementations by many-core processors, we here aim at totally relying on that processor type. As main contribution, we introduce the necessary parallel algorithmic patterns allowing to map the full $\mathcal {H}$ matrix construction and the fast matrix–vector product to many-core hardware. Here, crucial ingredients are space filling curves, parallel tree traversal and batching of linear algebra operations. The resulting model GPU implementation hmglib is the, to the best of the authors knowledge, first entirely GPU-based Open Source $\mathcal {H}$ matrix library of this kind. We investigate application examples as present in kernel ridge regression, Gaussian Process Regression and kernel-based interpolation. In this context, an in-depth performance analysis and a comparative performance study against a standard multi-core CPU $\mathcal {H}$ matrix library highlights profound speedups of our many-core parallel approach.

A Batched Bounding Box Computation

As part of the traversal of the block cluster tree, we have to evaluate the admissibility condition (3) for index blocks $\tau \times \sigma$ involving the bounding boxes of $\tau$ and $\sigma$ in each node. In the following, we will discuss an algorithm to concurrently compute the bounding boxes for clusters $\tau$, $\sigma$ in all nodes on a given level l of the block cluster tree. The algorithm is based on batching, cf. Sect. 4.3.1.

We collect the set of nodes on a level l of the block cluster tree, i.e. $V_{I\times I}(l)$, in the array node_data of length $|V_{I\times I}(l)|$ composed of structs work_item and have the input points $\mathcal {Y}$ in an instance of struct point_set, cf. Sect. 4.1. As simplification, we only consider the concurrent computation of the bounding boxes for one cluster set, e.g. $\tau$, in each node/block cluster.

Note that standard implementations of $\mathcal {H}$ matrix algorithms first build a cluster tree and then construct the block cluster tree. In that case, it is natural to compute the bounding boxes of each cluster in the cluster tree construction. We here, however, have chosen a different approach. That is, we construct the block cluster tree without first building a cluster tree. This is efficient with respect to the clustering, since the Morton ordering of the point sets makes clustering cheap. Moreover, we prefer to do just one tree traversal (for the block cluster tree) instead of several tree traversals (for two cluster trees and the block cluster tree) to maximize the utilization of the many-core processor within a single tree traversal. However, this comes at the price that the bounding box computation becomes part of the block cluster tree traversal, which is quite unusual. To avoid to compute bounding boxes for a single cluster many times, we divide the bounding box calculation in a pre-processing step and the bounding box assignment. In the pre-processing step, we extract from all block clusters the list of clusters. This list potentially contains multiple copies of clusters. We then unify this list to create a unique list of clusters. For each unique cluster in that unified list, we compute the bounding boxes and store them in a lookup table. This corresponds to computing the bounding boxes on the cluster tree. In the bounding box assignment step, we use a lookup table to assign the pre-computed bounding boxes to the clusters in the block cluster tree. Thereby, we avoid recomputing bounding boxes for identical clusters.

Fig. 17

The boundary box computation is sped up by pre-computing bounding boxes for each cluster, once. They are stored in bb_lookup_table and accessed via a map between work items and the lookup table

Since the algorithmic demanding step is the pre-processing step, we stick to the description of this step. Here, we first identify the set of unique clusters, as defined before. We then create a lookup table bb_lookup_table storing for each unique cluster the bounding box information. In addition, we need a map from a node in node_data to the entry in the lookup table. Figure 17 exemplifies this idea.

Algorithm 7 describes our approach to compute the entries of the lookup tablebb_lookup_table. Function compute_bounding_box_lookup_table gets as input the coordinate array coords of the input point set $\mathcal {Y}$, the nodes $V_{I\times I}(l)$ on level l in node_data, and further size information. First, the lower index bounds $i_{l,1}$ and upper index bounds $i_{u,1}$ are extracted from each node and stored in arrays lower_index_bounds and upper_index_bounds. By construction, the (block) cluster tree traversal based on Z-order curves only creates clusters that do not overlap in the point set array and that, for a given lower index bound, will always have the same upper bound. Therefore, we can use parallel sorting and unification methods to identify the set of unique clusters. The unique clusters are collected (by their lower and upper index bounds) in unique_lower_index_bounds and unique_upper_index_bounds. The final step is to compute the coordinate minima and maxima in each cluster. This step follows the idea of batching, cf. Sect. 4.3.1. The batched array is the array of coordinates. The bounds for the batches are given by the unique lower and upper index bounds and the keys for the batches are the sequence of numbers $\{1,2,\ldots \}$. Results in the batched computation that are associated to points in $\mathcal {Y}$ and not being part of any cluster are finally removed by removing all batched compute results associated to the key 0.

Fig. 18

Creating the map between a work item and an entry in the lookup table requires sorting, a compute kernel for bounds assignment, a scan operation and a permutation operation

Our approach to compute the map between the nodes in node_data and the lookup table is summarized in Algorithm 8. Again, we first get the lower and upper index bounds. Then, without loss of generality, we sort the lower bounds of the clusters and keep the applied permutation in permutation. Next, we create a global array map of length $|V_{I\times I}(l)|$ and initialize it to “0”. The parallel kernel set_bounds_for_map of $|V_{I\times I}(l)|$ then sets a “1” in map wherever there are two different subsequent entries in the sorted lower_bounds. By an inclusive scan on map, we create growing indices in map marking identical entries in lower_bounds. The result is exemplified in Fig. 18. We finally permute back map by kernel permutation with $|V_{I\times I}(l)|$ threads leading to the required map.