Precision-aware deterministic and probabilistic error bounds for floating point summation

Abstract

We analyze the forward error in the floating point summation of real numbers, for computations in low precision or extreme-scale problem dimensions that push the limits of the precision. We present a systematic recurrence for a martingale on a computational tree, which leads to explicit and interpretable bounds with nonlinear terms controlled explicitly rather than by big-O terms. Two probability parameters strengthen the precision-awareness of our bounds: one parameter controls the first order terms in the summation error, while the second one is designed for controlling higher order terms in low precision or extreme-scale problem dimensions. Our systematic approach yields new deterministic and probabilistic error bounds for three classes of mono-precision algorithms: general summation, shifted general summation, and compensated (sequential) summation. Extension of our systematic error analysis to mixed-precision summation algorithms that allow any number of precisions yields the first probabilistic bounds for the mixed-precision FABsum algorithm. Numerical experiments illustrate that the probabilistic bounds are accurate, and that among the three classes of mono-precision algorithms, compensated summation is generally the most accurate. As for mixed precision algorithms, our recommendation is to minimize the magnitude of intermediate partial sums relative to the precision in which they are computed.

A Proof of Lemma 8

Define $\beta \equiv u(1+u)^2$ and

$$\begin{aligned} \omega _k \equiv |s_k| + |x_k| + S_k, \qquad 2\le k \le n-1. \end{aligned}$$

(A.1)

By assumption, $\beta < 1$. Lemma 7 implies

$$\begin{aligned} Z_k&= u\omega _k +(1+u)Y_k = u\omega _k + (1+u)^2C_{k-1},\qquad 3\le k\le n-1 \end{aligned}$$

(A.2)

$$\begin{aligned} C_k&= u\omega _k + uZ_k = u(1+u)\omega _k + \beta C_{k-1}, \end{aligned}$$

(A.3)

where $Z_2\le u\omega _2$ and $C_2\le u(1+u)\omega _2$. For $3\le k \le n$, define the vectors

$$\begin{aligned} {{\textbf {c}}}_k \equiv \begin{bmatrix} C_{k-1}&\cdots&C_2\end{bmatrix}^T, \quad {{\textbf {z}}}_k \equiv \begin{bmatrix} Z_{k-1}&\ldots&Z_2 \end{bmatrix}^T, \quad {{\textbf {w}}}_k \equiv \begin{bmatrix}\omega _{k-1}&\ldots&\omega _2\end{bmatrix}^T. \end{aligned}$$

From (A.3) follows the componentwise inequality

$$\begin{aligned} {\textbf {c}}_k \le u(1+u){\textbf {w}}_k + \beta {\textbf {U}}{} {\textbf {c}}_k, \end{aligned}$$

where ${\textbf {U}}$ is an upper shift matrix. Solving for ${\textbf {c}}_k$ gives another componentwise inequality with a unit upper triangular matrix ${\textbf {I}}-\beta {\textbf {U}}$,

$$\begin{aligned} {\textbf {c}}_k \le u(1+u)({\textbf {I}}-\beta {\textbf {U}})^{-1}{} {\textbf {w}}_k, \end{aligned}$$

and a bound

$$\begin{aligned} \Vert {\textbf {c}}_k\Vert _2 \le u(1+u)\Vert ({\textbf {I}}-\beta {\textbf {U}})^{-1}{} {\textbf {w}}_k\Vert _2 \le \tfrac{u(1+u)}{1-\beta }\Vert {\textbf {w}}_k\Vert _2. \end{aligned}$$

The bound for $\Vert {\textbf {z}}_k\Vert _2$ follows from (A.2) and the definition of $\beta$,

$$\begin{aligned} \Vert {\textbf {z}}_k\Vert _2 \le u\Vert {\textbf {w}}_k\Vert _2 + (1+u)^2\Vert {\textbf {c}}_k\Vert _2 \le \tfrac{u(2+2u+u^2)}{1-\beta }\Vert {\textbf {w}}_k\Vert _2. \end{aligned}$$

Finally, from $Y_k = (1+u)C_{k-1}$ follows the Frobenius norm bound

$$\begin{aligned} \left( \sum _{j=3}^k\left( Y_j^2 + C_{j-1}^2 + Z_{j-1}^2\right) \right) ^{1/2} = \left\| \begin{bmatrix}(1+u){\textbf {c}}_k&{\textbf {c}}_k&{\textbf {z}}_k \end{bmatrix}\right\| _F \le \alpha u\Vert {\textbf {w}}_k\Vert _2, \end{aligned}$$

where the higher order terms in $\alpha$ follow from the Taylor series expansion $(1-\beta )^{-2}=1 +2u +\mathcal {O}(u^2)$,

$$\begin{aligned} \alpha ^2 = \frac{1+3(1+u)^2 + 2(1+u)^4}{(1-\beta )^2} = 6 + 26u + \mathcal {O}(u^2). \end{aligned}$$