Before describing the inner architecture of Daisy, we review necessary background on finite-precision arithmetic and static analysis of their roundoff errors.
Floating-Point Arithmetic. One of the most commonly used finite-precision representations is floating-point arithmetic, which is standardized by IEEE754 [24]. The standard defines several precisions as well as rounding operators; here we will consider the most commonly used ones, i.e. single and double precision with operations in rounding-to-nearest mode. Then, arithmetic operations satisfy the following abstraction:
$$\begin{aligned} x \circ _{fl} y = (x \circ y)(1 + e) + d \;\text {, }\; |e |\le \epsilon _m, |d |\le \delta _m \end{aligned}$$
(1)
where \(\circ \in {+, -, *, /}\) and \(\circ _{fl}\) denotes the respective floating-point version. Square root follows similarly, and unary minus does not introduce roundoff errors. The machine epsilon \(\epsilon _m\) bounds the maximum relative error for so-called normal values. Roundoff errors of subnormal values, which provide gradual underflow, are expressed as an absolute error, bounded by \(\delta _m\). \(\epsilon _m = 2^{-24}, \delta _m = 2^{-150}\) and \(\epsilon _m = 2^{-53}, \delta _m = 2^{-1075}\) for single and double precision, respectively.
Higher precisions are usually implemented in software libraries on top of standard double floating-point precision [2]. Daisy supports quad and quad-double precision, where we assume \(\epsilon _m = 2^{-113}\) and \(\epsilon _m = 2^{-211}\), respectively. Depending on the library, \(\delta _m\) may or may not be defined, and Daisy can be adjusted accordingly.
Static analyses usually use this abstraction of floating-point arithmetic, as bit-precise reasoning does not scale, and furthermore is unsuitable for computing roundoff errors w.r.t. continuous real-valued semantics (note that Eq. 1 is also real-valued). The abstraction furthermore only holds in the absence of not-a-number special values (NaN) and infinities. Daisy’s static analysis detects such cases automatically and reports them as errors.
Fixed-Point Arithmetic. Floating-point arithmetic requires dedicated support, either in hardware or software, and depending on the application this support may be too costly. An alternative is fixed-point arithmetic which can be implemented with integers only, but which in return requires that the radix point alignments are precomputed at compile time. While no standard exists, fixed-point values are usually represented by bit vectors with an integer and a fractional part, separated by an implicit radix point. At runtime, the alignments are then performed by bit-shift operations. These shift operations can also be handled by special language extensions for fixed-point arithmetic [25]. For more details see [1], whose fixed-point semantics we follow. We use truncation as the rounding mode for arithmetic operations. The absolute roundoff error at each operation is determined by the fixed-point format, i.e. the (implicit) number of fractional bits available, which in turn can be computed from the range of possible values at that operation.
Range Arithmetic. The magnitude of floating-point and fixed-point roundoff errors depends on the magnitudes of possible values. Thus, in order to accurately bound roundoff errors, any static analysis first needs to be able to bound the ranges of all (intermediate) expressions accurately, i.e. tightly. Different range arithmetics have been developed and each has a different accuracy/efficiency tradeoff. Daisy supports interval [33] and affine arithmetic [18] as well as a more accurate, but also more expensive, combination of interval arithmetic and SMT [14].
Interval arithmetic (IA) [33] is an efficient choice for range estimation, which computes a bounding interval for each basic operation \(\circ \in \lbrace +, -, *, / \rbrace \) as
$$\begin{aligned}{}[x_0, x_1] \circ [y_0, y_1] = [ \min (x \circ y), \max ( x \circ y)], \text { where }x \in [x_0, x_1], y \in [y_0, y_1] \end{aligned}$$
and analogously for square root. Interval arithmetic cannot track correlations between variables (e.g. \(x - x \ne [0, 0]\)), and thus can introduce significant over-approximations of the true ranges, especially when the computations are longer.
Affine arithmetic (AA) [18] tracks linear correlations by representing possible values of variables as affine forms:
$$\begin{aligned} \hat{x} = x_0 + \sum _{i=1}^n x_i \epsilon _i, \quad \text { where } \epsilon _i \in [-1, 1] \end{aligned}$$
where \(x_0\) denotes the central value (of the represented interval) and each noise term \(x_i\epsilon _i\) denotes a deviation from this central value. The range represented by an affine form is computed as \([\hat{x}] = [x_0 - rad(\hat{x}), x_0\,+\,rad(\hat{x})]\), \(rad(\hat{x}) = \sum _{i=1}^n |x_i|\). Linear operations are performed term-wise and are computed exactly, whereas nonlinear ones need to be approximated and thus introduce over-approximations. Overall, AA can produce tighter ranges in practice (though not universally). In particular, AA is often beneficial when the individual noise terms (\(x_i\)’s) are small, e.g. when they track roundoff errors.
The over-approximation due to nonlinear arithmetic can be mitigated [14] by refining ranges computed by IA with a binary search in combination with a SMT solver which supports nonlinear arithmetic such as Z3 [19] or dReal [21].
Static Analysis for Roundoff Error Estimation. The worst-case absolute roundoff error that most static analyses approximate is:
$$\begin{aligned} \max _{x \in [a, b]}\;\;|f(x) - \tilde{f}(\tilde{x}) |\end{aligned}$$
(2)
where [a, b] is the range for x given in the precondition, and f and x are a mathematical real-valued arithmetic expression and variable, respectively, and \(\tilde{f}\) and \(\tilde{x}\) their finite-precision counterparts. This definition extends to multivariate f component-wise.
An automated and general estimation of relative errors (\(\frac{|f(x) - \tilde{f}(\tilde{x}) |}{|f(x)|}\)), though it may be more desirable, presents a significant challenge today. For instance, when the range of f(x) includes zero, relative errors are not well defined and this is often the case in practice. For a more thorough discussion, we refer the reader to [26]; the technique is also implemented within Daisy.
For bounding absolute errors, two main approaches exist today, which we review in the following.
Dataflow Analysis. One may think that just evaluating a program in interval arithmetic and interpreting the width of the resulting interval as the error bound would be sufficient. While this is certainly a sound approach, it computes too pessimistic error bounds in general. This is especially true if we consider relatively large ranges on inputs; we cannot distinguish which part of the interval width is due to the input interval or due to accumulated roundoff errors.
Thus, dataflow analysis computes roundoff error bounds in two steps, recursively over the abstract syntax tree (AST) of the arithmetic expression:
-
1.
range analysis computes sound range bounds (for real semantics),
-
2.
error analysis propagates errors from subexpressions and computes the new worst-case roundoffs using the previously computed ranges.
In practice, these two steps can be performed in a single pass over the AST. A side effect of this separation is that it provides us with a modular approach: we can choose different range arithmetics with different accuracy/efficiency tradeoffs for ranges and errors (and possibly for different parts of a program).
The main challenge of dataflow analysis is to minimize over-approximations due to nonlinear arithmetic (linear arithmetic can be handled well with AA). Previous tools chose different strategies. For instance, Rosa [14] employs the combination of interval arithmetic with a non-linear SMT-solver, which we described earlier. Fluctuat [22], which uses AA for both bounding the ranges as well as the errors, uses interval subdivision. In Fluctuat, the user can designate up to two variables whose input ranges will be subdivided into intervals of equal width. The analysis is performed separately for each and the overall error is then the maximum error over all subintervals. Interval subdivision increases the runtime of the analysis, especially for multivariate functions, and the choice of which variables to subdivide and by how much is usually not straight-forward.
Optimization-based Analysis. FPTaylor [41], Real2Float [31] and PRECiSA [34], unlike Daisy, Rosa, Gappa and Fluctuat, formulate the roundoff error bounds computation as an optimization problem, where the absolute error expression from Eq. 2 is to be maximized, subject to interval constraints on its parameters. Due to the discrete nature of floating-point arithmetic, FPTaylor optimizes the continuous, real-valued abstraction from Eq. 1. However, this expression is still too complex and features too many variables for optimization procedures in practice.
FPTaylor introduces the Symbolic Taylor approach, where the objective function is simplified using a first order Taylor approximation with respect to e and d (the variables representing roundoff errors at each arithmetic operation). To solve the optimization problem, FPTaylor uses a rigorous branch-and-bound procedure.