Daisy - Framework for Analysis and Optimization of Numerical Programs (Tool Paper)
- 14 Citations
- 5.6k Downloads
Abstract
Automated techniques for analysis and optimization of finite-precision computations have recently garnered significant interest. Most of these were, however, developed independently. As a consequence, reuse and combination of the techniques is challenging and much of the underlying building blocks have been re-implemented several times, including in our own tools. This paper presents a new framework, called Daisy, which provides in a single tool the main building blocks for accuracy analysis of floating-point and fixed-point computations which have emerged from recent related work. Together with its modular structure and optimization methods, Daisy allows developers to easily recombine, explore and develop new techniques. Daisy’s input language, a subset of Scala, and its limited dependencies make it furthermore user-friendly and portable.
Keywords
Finite Precision Computations Roundoff Error Analysis Affine Arithmetic (AA) Fixed-point Arithmetic Interval Subdivision1 Introduction
Floating-point or fixed-point computations are an integral part of many embedded and scientific computing applications, as are the roundoff errors they introduce. They expose an interesting tradeoff between efficiency and accuracy: the more precision we choose, the closer the results will be to the ideal real arithmetic, but the more costly the computation becomes. Unfortunately, the unintuitive and complex nature of finite-precision arithmetic makes manual optimization infeasible such that automated tool support is indispensable.
This has been recognized previously and several tools for the analysis and optimization of finite-precision computations have been developed. For instance, the tools Fluctuat [22], Rosa [14], Gappa [17], FPTaylor [41], Real2Float [31] and PRECiSA [34] automatically provide sound error bounds on floating-point (and some also on fixed-point) roundoff errors. Such a static error analysis is a pre-requisite for any optimization technique providing rigorous results, such as recent ones which choose a mixed-precision assignment [10] or an error-minimizing rewriting of the non-associative finite-precision arithmetic [15, 37].
Many of these techniques are complementary. The static analysis techniques have different strengths, weaknesses, and accuracy/efficiency tradeoffs, and optimization techniques should ideally be combined for best results [16]. However, today’s techniques are mostly developed independently, resulting in re-implementations and making re-combination and re-use challenging and time-consuming.
In this paper, we present the framework Daisy for the analysis and optimization of finite-precision computations. In contrast to previous work, we have developed Daisy from the ground up to be modular, and thus easily extensible. Daisy is being actively developed and currently already provides many of today’s state-of-the-art techniques — all in one tool. In particular, it provides dataflow- as well as optimization-based sound roundoff error analysis, support for mixed-precision and transcendental functions, rewriting optimization, interfaces to several SMT solvers and code generation in Scala and C. Daisy furthermore supports both floating-point and fixed-point arithmetic (whenever the techniques do), making it generally applicable to both scientific computing and embedded applications.
Daisy is aimed at tool developers as well as non-expert users. To make it user-friendly, we adopt the input format of Rosa, which is a real-valued functional domain-specific language in Scala. Unlike other tools today, which have custom input formats [41] or use prefix notation [12], Daisy’s input is easily human readable1 and natural to use.
Daisy is itself written in the Scala programming language [35] and has limited and optional dependencies, making it portable and easy to install. Daisy’s main design goals are code readability and extensibility, and not necessarily performance. We demonstrate with our experiments that roundoff errors computed by Daisy are nonetheless competitive with state-of-the-art tools with reasonable running times.
Daisy has replaced Rosa for our own development, and we are happy to report that simple extensions (e.g. adding support for fused multiply-add operations) were integrated quickly by MSc students previously unfamiliar with the tool.
-
static dataflow analysis for finite-precision roundoff errors [14] with mixed-precision support and additional support for the dReal SMT solver [21],
-
FPTaylor’s optimization-based absolute error analysis [41],
-
transcendental function support, for dataflow analysis following [13],
-
interval subdivision, used by Fluctuat [22] to obtain tighter error bounds,
-
rewriting optimization based on genetic programming [15].
We show in Sect. 5 that results computed by Daisy are competitive. The code is available open-source at https://github.com/malyzajko/daisy.
We focus primarily on sound verification techniques. The goal of this effort is not to develop the next even more accurate technique, rather to consolidate existing ones and to provide a solid basis for further research. Other efforts related to Daisy, which have been described elsewhere and which we do not focus on here are the generation and checking of formal certificates [4], relative error computation [26], and mixed-precision tuning [16].
2 User’s Guide: An Overview of Daisy
We first introduce Daisy’s functionality from a user’s perspective, before reviewing background in roundoff error analysis (Sect. 3) and then describing the developer’s view and the internals of Daisy (Sect. 4).
Installation. Daisy is set up with the simple build tool (sbt) [30], which takes care of installing all Scala-related dependencies fully automatically. This basic setup was successfully tested on Linux, macOS and Windows. Some of Daisy’s functionality requires additional libraries, which are also straight-forward to install: the Z3 and dReal SMT-solvers [19, 21], and the MPFR arbitrary-precision library [20]. Z3 works on all platforms, we have tested MPFR on Linux and Mac, and dReal on Linux.

Example input program
Each input program consists of a number of functions which are handled by Daisy separately. In the function’s precondition (the Open image in new window clause), the user provides the ranges of all input variables2. In addition, Daisy allows to specify an initial error (beyond only roundoff) on input variables with the notation Open image in new window
as well as additional (non-interval) constraints, e.g. Open image in new window
.
The function body consists of a numerical expression with possibly local variable declarations. Daisy supports arithmetic (\(+, -, *, /, \sqrt{}\)), the standard transcendental functions (\(\sin , \cos , \tan , \log , \exp \)) as well as fused multiply-add (FMA). Daisy currently does not support conditionals and loops; we discuss the challenges and possible future avenues in Sect. 6. The (optional) postcondition in the Open image in new window clause specifies the result’s required accuracy in terms of worst-case absolute roundoff error. For our controller, this information may be for instance determined from the specification of the system’s sensors or the analysis of the controller’s stability [32].
Main Functionality. The main mode of interaction with Daisy is through a command-line interface. Here we review Daisy’s main features through the most commonly used command-line options. Brackets denote a choice and curly braces optional parameters. For more options and more fine-grained settings, run Open image in new window .



Finite-precision arithmetic is not associative, i.e. different rewritings, even though they are equivalent under a real-valued semantics, will exhibit different roundoff errors. The Open image in new window optimization [15] uses genetic search to find a rewriting for which it can show the smallest roundoff error.
Daisy prints the analysis result to the terminal. If a postcondition is specified, but the computed error does not satisfy it, Daisy also prints a warning. Optionally, the user can also choose to generate executable code ( Open image in new window ) in Scala or C, which is especially useful for fixed-point arithmetic, as Daisy’s code generator includes all necessary bit shifts.
Static analysis computes a sound over-approximation of roundoff errors, but an under-approximation can also be useful, e.g. to estimate how big the over-approximation of static analysis is. This is provided by the Open image in new window analysis in Daisy which runs a program in the finite precision of interest and a higher-precision version side-by-side. For this, the MPFR library is required.
Screenshot of Daisy’s online interface
3 Theoretical Foundations
Before describing the inner architecture of Daisy, we review necessary background on finite-precision arithmetic and static analysis of their roundoff errors.
Higher precisions are usually implemented in software libraries on top of standard double floating-point precision [2]. Daisy supports quad and quad-double precision, where we assume \(\epsilon _m = 2^{-113}\) and \(\epsilon _m = 2^{-211}\), respectively. Depending on the library, \(\delta _m\) may or may not be defined, and Daisy can be adjusted accordingly.
Static analyses usually use this abstraction of floating-point arithmetic, as bit-precise reasoning does not scale, and furthermore is unsuitable for computing roundoff errors w.r.t. continuous real-valued semantics (note that Eq. 1 is also real-valued). The abstraction furthermore only holds in the absence of not-a-number special values (NaN) and infinities. Daisy’s static analysis detects such cases automatically and reports them as errors.
Fixed-Point Arithmetic. Floating-point arithmetic requires dedicated support, either in hardware or software, and depending on the application this support may be too costly. An alternative is fixed-point arithmetic which can be implemented with integers only, but which in return requires that the radix point alignments are precomputed at compile time. While no standard exists, fixed-point values are usually represented by bit vectors with an integer and a fractional part, separated by an implicit radix point. At runtime, the alignments are then performed by bit-shift operations. These shift operations can also be handled by special language extensions for fixed-point arithmetic [25]. For more details see [1], whose fixed-point semantics we follow. We use truncation as the rounding mode for arithmetic operations. The absolute roundoff error at each operation is determined by the fixed-point format, i.e. the (implicit) number of fractional bits available, which in turn can be computed from the range of possible values at that operation.
Range Arithmetic. The magnitude of floating-point and fixed-point roundoff errors depends on the magnitudes of possible values. Thus, in order to accurately bound roundoff errors, any static analysis first needs to be able to bound the ranges of all (intermediate) expressions accurately, i.e. tightly. Different range arithmetics have been developed and each has a different accuracy/efficiency tradeoff. Daisy supports interval [33] and affine arithmetic [18] as well as a more accurate, but also more expensive, combination of interval arithmetic and SMT [14].
The over-approximation due to nonlinear arithmetic can be mitigated [14] by refining ranges computed by IA with a binary search in combination with a SMT solver which supports nonlinear arithmetic such as Z3 [19] or dReal [21].
An automated and general estimation of relative errors (\(\frac{|f(x) - \tilde{f}(\tilde{x}) |}{|f(x)|}\)), though it may be more desirable, presents a significant challenge today. For instance, when the range of f(x) includes zero, relative errors are not well defined and this is often the case in practice. For a more thorough discussion, we refer the reader to [26]; the technique is also implemented within Daisy.
For bounding absolute errors, two main approaches exist today, which we review in the following.
Dataflow Analysis. One may think that just evaluating a program in interval arithmetic and interpreting the width of the resulting interval as the error bound would be sufficient. While this is certainly a sound approach, it computes too pessimistic error bounds in general. This is especially true if we consider relatively large ranges on inputs; we cannot distinguish which part of the interval width is due to the input interval or due to accumulated roundoff errors.
- 1.
range analysis computes sound range bounds (for real semantics),
- 2.
error analysis propagates errors from subexpressions and computes the new worst-case roundoffs using the previously computed ranges.
In practice, these two steps can be performed in a single pass over the AST. A side effect of this separation is that it provides us with a modular approach: we can choose different range arithmetics with different accuracy/efficiency tradeoffs for ranges and errors (and possibly for different parts of a program).
The main challenge of dataflow analysis is to minimize over-approximations due to nonlinear arithmetic (linear arithmetic can be handled well with AA). Previous tools chose different strategies. For instance, Rosa [14] employs the combination of interval arithmetic with a non-linear SMT-solver, which we described earlier. Fluctuat [22], which uses AA for both bounding the ranges as well as the errors, uses interval subdivision. In Fluctuat, the user can designate up to two variables whose input ranges will be subdivided into intervals of equal width. The analysis is performed separately for each and the overall error is then the maximum error over all subintervals. Interval subdivision increases the runtime of the analysis, especially for multivariate functions, and the choice of which variables to subdivide and by how much is usually not straight-forward.
Optimization-based Analysis. FPTaylor [41], Real2Float [31] and PRECiSA [34], unlike Daisy, Rosa, Gappa and Fluctuat, formulate the roundoff error bounds computation as an optimization problem, where the absolute error expression from Eq. 2 is to be maximized, subject to interval constraints on its parameters. Due to the discrete nature of floating-point arithmetic, FPTaylor optimizes the continuous, real-valued abstraction from Eq. 1. However, this expression is still too complex and features too many variables for optimization procedures in practice.
FPTaylor introduces the Symbolic Taylor approach, where the objective function is simplified using a first order Taylor approximation with respect to e and d (the variables representing roundoff errors at each arithmetic operation). To solve the optimization problem, FPTaylor uses a rigorous branch-and-bound procedure.
4 Developer’s Guide: Daisy’s Internals
This section provides more details on Daisy’s architecture and explains some of our design decisions. Daisy is written in the Scala programming language which provides a strong type system as well as a large collection of (parallel) libraries. While Scala supports both imperative and functional programming styles, we have written Daisy functionally as much as possible, which we found to be beneficial to ensuring correctness and readability of code.
4.1 Input Language and Frontend
Daisy’s input language is implemented as a domain-specific language in Scala, and Daisy’s frontend calls the Scala compiler which performs parsing and type-checking. While designing our own simple input format and parser would be certainly more efficient in terms of Daisy’s running time (and could be done in the future), we have deliberately chosen not to do this. An existing programming language provides clear semantics and feels natural to users. Using the Scala compiler furthermore helps to ensure that Daisy parses the program correctly, for instance that it indeed conforms e.g. to Scala’s typing rules. Furthermore, extending the input language is usually straight-forward.
The other major design decision was to make the input program real-valued. This explicitly specifies the baseline against which roundoff errors should be computed, but it also makes it easy for the user to explore different options. For instance, changing the precision only requires changing a flag, whereas a finite-precision input program (like FPTaylor’s or Fluctuat’s) requires editing the source code.
Mixed-precision is also supported respecting Scala semantics and is thus transparent. The user may annotate variables, including local ones, with different precisions. To specify the precision of every individual operation, the program can be transformed into three-address form (Daisy can do this automatically), and then each arithmetic operation can be annotated via the corresponding variable.
Daisy currently does not support data structures such as arrays or lists in its input language, mainly because the static analysis of these is largely orthogonal to the analysis of the actual computation and we believe that standard strategies like unrolling computations over array elements or abstracting the array as a single variable can be employed.
4.2 Modular Architecture


Overview of Daisy’s phases. Phases in curly braces are optional.
-
Open image in new window
provides an implementation of rational numbers based on Java’s BigInteger library. Rationals are used throughout Daisy for computations in order to avoid internal roundoff errors which could affect soundness.
-
Open image in new window
is an interface to GNU’s MPFR arbitrary precision library [20].
-
Open image in new window
and Open image in new window
provide implementations of interval and affine arithmetic. Daisy uses no external libraries for these in order to facilitate extensions and integration.
-
Open image in new window
implements Rosa’s combination of interval arithmetic with an SMT solver [14] for improved range bounds. Daisy uses the scala-smtlib library3 to interface with the Z3 and dReal SMT solvers. Other solvers can be added with little effort, provided they support the SMT-LIB standard [3].
-
Open image in new window
implement dataflow roundoff error analysis. The analysis is parametric in the range method used, and due to its implementation as a library function can be easily used in different contexts.
-
Open image in new window
provides methods for computing and simplifying partial derivatives.
-
Open image in new window
provides a generic implementation of a (simple) genetic search, which is currently used for the rewriting optimization.
The fixed-point precision class in Daisy supports any bitlength (i.e. only the frontend has a limited selection) and floating-point types can be straight-forwardly added by specifying the corresponding machine epsilon and representable range.
4.3 Implementation Details
Here we provide details about Daisy’s implementation of previous techniques. The dataflow analysis approach, e.g. in Rosa, only considered arithmetic operations without transcendental functions. Daisy extends this support by implementing these operations in interval and affine arithmetic. The former is straight-forward, whereas for AA Daisy computes sound linear approximations of the functions, following [13] which used this approach in a dynamic analysis. Following most libraries of mathematical functions, we assume that transcendental functions are rounded correctly to one unit in the last place. Since internal computations are performed with rational types, the operations for transcendental functions are approximated with the corresponding outward or upwards rounding to ensure soundness. To support the combination of interval arithmetic and SMT, we integrate the dReal solver in Daisy, which provides support for transcendental functions. Although dReal is only \(\delta \)-complete, this does not affect Daisy’s soundness as the algorithm relies on UNSAT answers, which are always sound in dReal.
Interval subdivision can be an effective tool to reduce overapproximations in static analysis results, which is why Daisy offers it for all its analyses. Daisy subdivides every input variable range into a fixed number of subintervals (the number can be controlled by the user) and takes the cartesian product. The analysis is then performed separately for each set of subintervals. This clearly increases the running time, but is also trivially parallelizable.
Daisy also includes an initial implementation of FPTaylor’s optimization-based static analysis. The major difference is that Daisy does not use a branch-and-bound algorithm for solving the optimization problem, but relies on the already existing range analyses. We would like to include a proper optimization solver in the future; currently custom interfaces have been an obstacle.
5 Experimental Evaluation
We have experimentally evaluated Daisy’s roundoff error analysis on a number of finite-precision verification benchmarks taken from related work [15, 16, 31, 41]. Benchmarks marked with a superscript \(^T\) contain transcendental functions. The goal of this evaluation is twofold. First, Daisy should be able to compute reasonably tight error bounds in a reasonable amount of time to be useful. Secondly, exploiting the fact that Daisy implements several different analysis methods within a single tool allows us to provide a direct comparison of their tradeoffs.
Roundoff errors for uniform 64-bit double precision by dynamic analysis, FPTaylor and Daisy (subset of benchmarks).
Daisy | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Benchmark | Dynamic | FPTaylor | IA - AA | Z3 - AA | Z3 - IA | dReal - AA | AA-AA+sub | opt - Z3 | % | Z3-AA+rw |
bspline0 | 2.84e-17 | 1.07e-16 | 1.62e-16 | 1.62e-16 | 1.62e-16 | 1.62e-16 | 1.62e-16 | 1.19e-16 | 1.11 | 1.62e-16 |
bspline1 | 1.74e-16 | 3.59e-16 | 7.96e-16 | 7.03e-16 | 8.14e-16 | 7.03e-16 | 5.17e-16 | 6.51e-16 | 1.44 | 4.81e-16 |
doppler | 7.04e-14 | 1.22e-13 | 4.19e-13 | 4.19e-13 | 4.36e-13 | 4.19e-13 | 2.61e-13 | 1.72e-13 | 1.41 | 1.72e-13 |
himmilbeau | 5.27e-13 | 1.00e-12 | 2.33e-12 | 1.00e-12 | 1.00e-12 | 1.00e-12 | 1.00e-12 | 1.42e-12 | 1.00 | 1.01e-12 |
invertedPend. | 2.43e-14 | 3.21e-14 | 3.67e-14 | 3.67e-14 | 3.67e-14 | 3.67e-14 | 3.67e-14 | 4.44e-14 | 1.14 | 2.43e-14 |
kepler0 | 4.02e-14 | 5.85e-14 | 1.04e-13 | 9.06e-14 | 1.14e-13 | 9.20e-14 | 7.88e-14 | 1.15e-13 | 1.35 | 5.70e-14 |
kepler1 | 1.27e-13 | 1.96e-13 | 4.82e-13 | 3.97e-13 | 4.81e-13 | 3.97e-13 | 3.30e-13 | 4.92e-13 | 1.68 | 2.89e-13 |
kepler2 | 5.21e-13 | 1.47e-12 | 2.47e-12 | 2.25e-12 | 2.69e-12 | 2.25e-12 | 1.93e-12 | 2.28e-12 | 1.31 | 1.73e-12 |
rigidBody1 | 2.00e-13 | 2.95e-13 | 3.22e-13 | 3.22e-13 | 3.22e-13 | 3.22e-13 | 3.22e-13 | 5.08e-13 | 1.09 | 2.24e-13 |
rigidBody2 | 2.03e-11 | 3.61e-11 | 3.65e-11 | 3.65e-11 | 3.65e-11 | 3.65e-11 | 3.65e-11 | 6.48e-11 | 1.01 | 2.91e-11 |
sine | 2.76e-16 | 4.44e-16 | 1.13e-15 | 6.95e-16 | 7.41e-16 | 6.95e-16 | 6.49e-16 | 6.54e-16 | 1.46 | 5.91e-16 |
sineOrder3 | 3.38e-16 | 5.94e-16 | 1.45e-15 | 1.23e-15 | 1.34e-15 | 1.23e-15 | 1.02e-15 | 8.00e-16 | 1.35 | 1.22e-15 |
sqroot | 2.35e-13 | 2.81e-13 | 3.13e-13 | 3.09e-13 | 3.21e-13 | 3.09e-13 | 2.97e-13 | 3.97e-13 | 1.06 | 2.89e-13 |
train4 out1 | 1.33e-10 | 3.39e-10 | 4.28e-10 | 4.28e-10 | 4.28e-10 | 4.28e-10 | 3.99e-10 | 5.21e-10 | 1.18 | 3.34e-10 |
train4 state9 | 5.93e-15 | 8.12e-15 | 8.66e-15 | 8.66e-15 | 8.66e-15 | 8.66e-15 | 8.66e-15 | 1.20e-14 | 1.07 | 3.33e-15 |
turbine1 | 6.78e-15 | 1.67e-14 | 9.49e-14 | 8.87e-14 | 9.14e-14 | 8.87e-14 | 4.26e-14 | 2.80e-14 | 1.68 | 8.68e-14 |
turbine2 | 1.06e-14 | 2.00e-14 | 1.39e-13 | 1.23e-13 | 1.29e-13 | 1.23e-13 | 4.35e-14 | 3.67e-14 | 1.84 | 1.19e-13 |
turbine3 | 4.38e-15 | 9.57e-15 | 7.07e-14 | 6.27e-14 | 6.55e-14 | 6.27e-14 | 1.96e-14 | 1.65e-14 | 1.72 | 5.98e-14 |
jetEngine | 5.24e-12 | 8.75e-12 | - | 1.15e-08 | 1.16e-08 | 1.15e-08 | 3.64e-08 | 2.20e-11 | 2.51 | 1.12e-08 |
pendulum1\(^T\) | 3.31e-16 | 3.47e-16 | 4.61e-16 | 4.61e-16 | 4.61e-16 | 4.61e-16 | 4.61e-16 | 4.67e-16 | 1.33 | 4.61e-16 |
pendulum2\(^T\) | 8.88e-16 | 9.15e-16 | 9.42e-16 | 9.42e-16 | 9.42e-16 | 9.42e-16 | 9.37e-16 | 1.16e-15 | 1.02 | 9.41e-16 |
analysis1\(^T\) | 1.41e-16 | 1.95e-16 | 1.67e-15 | 1.67e-15 | 1.67e-15 | 1.30e-15 | 1.61e-15 | 1.92e-15 | 6.67 | 1.67e-15 |
analysis2\(^T\) | 4.40e-16 | 5.49e-16 | 6.08e-14 | 6.08e-14 | 6.28e-14 | 3.11e-15 | 3.84e-15 | 1.28e-13 | 5.66 | 6.08e-14 |
logExp\(^T\) | 1.29e-15 | 1.99e-15 | 3.33e-12 | 3.33e-12 | 3.33e-12 | 3.33e-12 | 2.18e-13 | 3.31e-12 | 109.55 | 3.33e-12 |
sphere\(^T\) | 4.47e-15 | 8.18e-15 | 1.20e-14 | 1.20e-14 | 1.20e-14 | 1.20e-14 | 3.22e-14 | 1.63e-14 | 1.47 | 1.17e-14 |
Execution times of FPTaylor and Daisy for different settings
Benchmark | FPTaylor | Z3 - AA | AA-AA (sub) | opt - Z3 |
---|---|---|---|---|
bspline | 2 s 884 ms | 4 s 450 ms | 2 s 190 ms | 3 s 320 ms |
doppler | 1 s 465 ms | 3 s 221 ms | 2 s 657 ms | 2 s 939 ms |
himmilbeau | 660 ms | 3 s 545 ms | 1 s 975 ms | 2 s 760 ms |
invertedPend. | 14 s 69 ms | 3 s 109 ms | 2 s 31 ms | 2 s 570 ms |
kepler | 18 s 629 ms | 40 s 627 ms | 3 s 160 ms | 21 s 893 ms |
rigidBody | 1 s 430 ms | 6 s 31 ms | 2 s 206 ms | 4 s 118 ms |
sine | 1 s 580 ms | 4 s 49 ms | 2 s 179 ms | 3 s 114 ms |
sqrt | 7 s 381 ms | 4 s 92 ms | 1 s 884 ms | 2 s 988 ms |
traincar | 27 s 846 ms | 22 s 670 ms | 7s 452 ms | 15 s 61 ms |
turbine | 2 s 452 ms | 7 s 93 ms | 3 s 951 ms | 5 s 522 ms |
jetEngine | 1 s 434 ms | 35 s 267 ms | 3 s 583 ms | 19 s 425 ms |
transcendental | 34 s 547 ms | 2 s 770 ms | 2s 959 ms | 2 s 865 ms |
Comparison with FPTaylor. We first compare roundoff errors computed by Daisy with different methods against errors computed by FPTaylor (version from 20 Sept 2017) in Table 1. All errors are computed for uniform double floating-point precision, assuming roundoff errors on inputs. We abbreviate the settings used in Daisy by e.g. IA - AA, where IA and AA specify the methods used for computing the ranges and errors, respectively. ‘sub’ means subdivision, ‘rw’ rewriting and ‘opt’ denotes the optimization-based approach. We underline the lowest roundoff errors computed among the different Daisy settings (without rewriting). The column marked ‘%’ denotes the factor by which the lowest error computed by Daisy differs from FPTaylor’s computed error.
FPTaylor supports different backend solvers; we have performed experiments with the internal branch-and-bound and the Gelpia solver, but observed only minor differences. We thus report results for the Gelpia solver. We furthermore chose the lowest verbosity level in both FPTaylor and Daisy to reduce the execution time. Table 1 also shows an underapproximation of roundoff errors computed using Daisy’s dynamic analysis which provides an idea of the tightness of roundoff errors.
Table 2 shows the corresponding execution times of the tools. Execution times are average real time measured by the bash time command. We have performed all experiments on a Linux desktop computer with an Intel Xeon 3.30 GHz processor and 32 GB RAM, with Scala version 2.11.11.
The focus when implementing Daisy was to provide a solid framework with modular and clear code, not to improve roundoff error bounds. Nonetheless, Daisy’s roundoff error bounds are mostly competitive with FPTaylor’s, with the notable exception of the jetEngine benchmark (additionally, interval arithmetic fails to bound the divisor away from zero).
Overall we observe that using an SMT solver for tightening ranges is helpful, but interval subdivision is preferable. Furthermore, using affine arithmetic for bounding errors is preferable over interval arithmetic. Finally, rewriting can often improve roundoff error bounds significantly.
Our optimization-based analysis is not yet quite as good as FPTaylor’s, but acceptable for a first re-implementation. We suspect the difference is mainly due to the fact that Daisy does not use a dedicated optimization procedure, which we hope to include in the future.
Execution times of FPTaylor and Daisy are comparable. It should be noted that the times are end-to-end, and in particular for Daisy this includes the Scala compiler frontend, which takes a constant 1.3 s (irrespective of input). Clearly, with a hand-written parser this could be improved, but we do not consider this as critical. Furthermore, Daisy performs overflow checks at every intermediate subexpression; it is unclear whether FPTaylor does this as well.
Table 1 seems to suggest that one should use FPTaylor’s optimization-based approach for bounding roundoff errors. We include dataflow analysis in Daisy nonetheless for several reasons. First, dataflow analysis computes overflow checks without extra cost. Secondly, the optimization-based approach is only applicable when errors can be specified as relative errors, which is not the case for instance for fixed-point arithmetic, which is important for many embedded applications.
Roundoff errors for 32-bit floating-point and fixed-point arithmetic.
Z3 - AA | Z3 - AA + rewriting | |||
---|---|---|---|---|
Benchmark | float 32 | fixed 32 | float 32 | fixed 32 |
bspline0 | 8.69e-8 | 2.28e-9 | 8.69e-8 | 2.28e-9 |
bspline1 | 3.77e-7 | 7.86e-9 | 2.58e-7 | 6.00e-9 |
doppler | 2.25e-4 | 3.52e-6 | 9.26e-5 | 1.45e-6 |
himmilbeau | 5.37e-4 | 8.84e-6 | 6.85e-4 | 1.13e-5 |
invertedPendulum | 1.97e-5 | 3.54e-7 | 1.30e-5 | 2.03e-7 |
kepler0 | 4.87e-5 | 7.60e-7 | 3.06e-5 | 4.78e-7 |
kepler1 | 2.13e-4 | 3.33e-6 | 1.76e-4 | 2.76e-6 |
kepler2 | 1.21e-3 | 1.88e-5 | 8.85e-4 | 1.38e-5 |
rigidBody1 | 1.73e-4 | 3.12e-6 | 1.20e-4 | 2.30e-6 |
rigidBody2 | 1.96e-2 | 3.13e-4 | 1.56e-2 | 2.51e-4 |
sine | 3.73e-7 | 7.14e-9 | 3.17e-7 | 6.68e-9 |
sineOrder3 | 6.58e-7 | 1.31e-8 | 6.54e-7 | 1.39e-8 |
sqroot | 1.66e-4 | 8.00e-6 | 1.60e-4 | 7.68e-6 |
train4 out1 | 2.30e-1 | 4.14e-3 | 1.79e-1 | 3.34e-3 |
train4 state9 | 4.65e-6 | 1.45e-7 | 1.79e-6 | 1.03e-7 |
turbine1 | 4.76e-5 | 1.05e-6 | 4.66e-5 | 1.04e-6 |
turbine2 | 6.61e-5 | 1.19e-6 | 6.40e-5 | 1.16e-6 |
turbine3 | 3.37e-5 | 7.42e-7 | 3.21e-5 | 7.17e-7 |
jetEngine | 6.22 | 1.00e-1 | 1.44e-1 | 2.46e-3 |
6 Related Work
We have already mentioned the directly related techniques and tools Gappa, Fluctuat, Rosa, FPTaylor, Real2Float and PRECiSA throughout the paper. Except for Fluctuat and Rosa, these tools also provide either a proof script or a certificate for the correctness (of certain parts) of the analysis, which can be independently checked in a theorem prover. Certificate generation and checking for Daisy has been described in a separate paper [4].
Daisy currently handles straight-line arithmetic expressions, i.e. it does not handle conditionals and loops. Abstract interpretation of floating-point programs handles conditionals by joins, however, for roundoff error analysis this approach is not sufficient. The real-valued and finite-precision computations can diverge and a simple join does not capture this ‘discontinuity error’. Programs with loops are challenging, because roundoff errors in general grow with each loop iteration and thus a nontrivial fixpoint does not exist in general (loop unrolling can however be applied). Widening operators compute non-trivial bounds only for very special cases where roundoff errors decrease with each loop iteration. These challenges have been (partially) addressed [16, 23], and we plan to include those techniques in Daisy in the future. Nonetheless, conditionals and loops remain open problems.
Sound techniques have also been applied for both the range and the error analysis for bitwidth optimization of fixed-point arithmetic, for instance in [28, 29, 36, 38] and Lee et. al. [29] provide a nice overview of static and dynamic techniques.
Dynamic analysis can be used to find inputs which cause large roundoff errors, e.g. running a higher-precision floating-point program alongside the original one [5] or with a guided search to find inputs which maximize errors [11]. In comparison, Daisy’s dynamic analysis is a straight-forward approach, and some more advanced techniques could be integrated as well.
Optimization techniques targeting accuracy of floating-point computations, like rewriting [37] or mixed-precision tuning [10] include some form of roundoff error analysis, and any of the above approaches, including Daisy’s, can be potentially used as a building block.
More broadly related are abstract interpretation-based static analyses, which are sound w.r.t. floating-point arithmetic [6, 9, 27]. These techniques can prove the absence of runtime errors, such as division-by-zero, but cannot quantify roundoff errors. Floating-point arithmetic has also been formalized in theorem provers and entire numerical programs have been proven correct and accurate within these [7, 39]. Most of these formal verification efforts are, however, to a large part manual. Floating-point arithmetic has also been formalized in an SMT-lib [40] theory and SMT solvers exist which include floating-point decision procedures [8, 19]. These are, however, not suitable for roundoff error quantification, as a combination with the theory of reals would be necessary which does not exist today.
7 Conclusion
We have presented the framework Daisy which integrates several state-of-the-art techniques for the analysis and optimization of finite-precision programs. It is actively being developed, improved and extended and we believe that it can serve as a useful building block in future optimization techniques.
Footnotes
References
- 1.Anta, A., Majumdar, R., Saha, I., Tabuada, P.: Automatic verification of control system implementations. In: EMSOFT (2010)Google Scholar
- 2.Bailey, D.H., Hida, Y., Li, X.S., Thompson, B.: C++/Fortran-90 double-double and quad-double package. Technical report (2015)Google Scholar
- 3.Barrett, C., Fontaine, P., Tinelli, C.: The SMT-LIB Standard: Version 2.6. Technical report, University of Iowa (2017). www.SMT-LIB.org
- 4.Becker, H., Darulova, E., Myreen, M.O.: A verified certificate checker for floating-point error bounds. Technical report (2017). arXiv:1707.02115
- 5.Benz, F., Hildebrandt, A., Hack, S.: A dynamic program analysis to find floating-point accuracy problems. In: PLDI (2012)Google Scholar
- 6.Blanchet, B., Cousot, P., Cousot, R., Feret, J., Mauborgne, L., Miné, A., Monniaux, D., Rival, X.: A static analyzer for large safety-critical software. In: PLDI (2003)CrossRefGoogle Scholar
- 7.Boldo, S., Clément, F., Filliâtre, J.-C., Mayero, M., Melquiond, G., Weis, P.: Wave equation numerical resolution: a comprehensive mechanized proof of a C program. J. Autom. Reason. 50(4), 423–456 (2013)MathSciNetCrossRefGoogle Scholar
- 8.Brain, M., D’Silva, V., Griggio, A., Haller, L., Kroening, D.: Deciding floating-point logic with abstract conflict driven clause learning. Form. Methods Syst. Des. 45(2), 213–245 (2013)CrossRefGoogle Scholar
- 9.Chen, L., Miné, A., Cousot, P.: A sound floating-point polyhedra abstract domain. In: Ramalingam, G. (ed.) APLAS 2008. LNCS, vol. 5356, pp. 3–18. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89330-1_2CrossRefGoogle Scholar
- 10.Chiang, W.-F., Gopalakrishnan, G., Rakamaric, Z., Briggs, I., Baranowski, M.S., Solovyev, A.: Rigorous floating-point mixed precision tuning. In: POPL (2017)Google Scholar
- 11.Chiang, W.-F., Gopalakrishnan, G., Rakamaric, Z., Solovyev, A.: Efficient search for inputs causing high floating-point errors. In: PPoPP (2014)Google Scholar
- 12.Damouche, N., Martel, M., Panchekha, P., Qiu, C., Sanchez-Stern, A., Tatlock, Z.: Toward a standard benchmark format and suite for floating-point analysis. In: Bogomolov, S., Martel, M., Prabhakar, P. (eds.) NSV 2016. LNCS, vol. 10152, pp. 63–77. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54292-8_6CrossRefGoogle Scholar
- 13.Darulova, E., Kuncak, V.: Trustworthy numerical computation in scala. In: OOPSLA (2011)Google Scholar
- 14.Darulova, E., Kuncak, V.: Sound compilation of reals. In: POPL (2014)Google Scholar
- 15.Darulova, E., Kuncak, V., Majumdar, R., Saha, I.: Synthesis of fixed-point programs. In: EMSOFT (2013)Google Scholar
- 16.Darulova, E., Sharma, S., Horn, E.: Sound mixed-precision optimization with rewriting. Technical report (2017). arXiv:1707.02118
- 17.Daumas, M., Melquiond, G.: Certification of bounds on expressions involving rounded operators. ACM Trans. Math. Softw. 37(1), 2:1–2:20 (2010)MathSciNetCrossRefGoogle Scholar
- 18.de Figueiredo, L.H., Stolfi, J.: Affine arithmetic: concepts and applications. Numer. Algorithms 37(1), 147–158 (2004)MathSciNetCrossRefGoogle Scholar
- 19.de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78800-3_24CrossRefGoogle Scholar
- 20.Fousse, L., Hanrot, G., Lefèvre, V., Pélissier, P., Zimmermann, P.: MPFR: a multiple-precision binary floating-point library with correct rounding. ACM Trans. Math. Softw. 33(2) (2007)CrossRefGoogle Scholar
- 21.Gao, S., Kong, S., Clarke, E.M.: dReal: an SMT solver for nonlinear theories over the reals. In: Bonacina, M.P. (ed.) CADE 2013. LNCS (LNAI), vol. 7898, pp. 208–214. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38574-2_14CrossRefGoogle Scholar
- 22.Goubault, E., Putot, S.: Static analysis of finite precision computations. In: Jhala, R., Schmidt, D. (eds.) VMCAI 2011. LNCS, vol. 6538, pp. 232–247. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18275-4_17CrossRefGoogle Scholar
- 23.Goubault, E., Putot, S.: Robustness analysis of finite precision implementations. In: Shan, C. (ed.) APLAS 2013. LNCS, vol. 8301, pp. 50–57. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03542-0_4CrossRefGoogle Scholar
- 24.Computer Society IEEE. IEEE Standard for Floating-Point Arithmetic. IEEE Std 754–2008 (2008)Google Scholar
- 25.ISO/IEC. Programming languages — C — Extensions to support embedded processors. Technical report ISO/IEC TR 18037 (2008)Google Scholar
- 26.Izycheva, A., Darulova, E.: On sound relative error bounds for floating-point arithmetic. In: FMCAD (2017)Google Scholar
- 27.Jeannet, B., Miné, A.: Apron: a library of numerical abstract domains for static analysis. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 661–667. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02658-4_52CrossRefGoogle Scholar
- 28.Kinsman, A.B., Nicolici, N.: Finite precision bit-width allocation using SAT-modulo theory. In: DATE (2009)Google Scholar
- 29.Lee, D.U., Gaffar, A.A., Cheung, R.C.C., Mencer, O., Luk, W., Constantinides, G.A.: Accuracy-guaranteed bit-width optimization. Trans. Comp.-Aided Des. Integ. Cir. Sys. 25(10), 1990–2000 (2006)CrossRefGoogle Scholar
- 30.Lightbend. sbt - The interactive build tool (2017). http://www.scala-sbt.org/
- 31.Magron, V., Constantinides, G., Donaldson, A.: Certified roundoff error bounds using semidefinite programming. ACM Trans. Math. Softw. 43(4) (2017)MathSciNetCrossRefGoogle Scholar
- 32.Majumdar, R., Saha, I., Zamani, M.: Synthesis of minimal-error control software. In: EMSOFT (2012)Google Scholar
- 33.Moore, R.E.: Interval Analysis. Prentice-Hall, Englewood Cliffs (1966)zbMATHGoogle Scholar
- 34.Moscato, M., Titolo, L., Dutle, A., Muñoz, C.A.: Automatic estimation of verified floating-point round-off errors via static analysis. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2017. LNCS, vol. 10488, pp. 213–229. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66266-4_14CrossRefGoogle Scholar
- 35.Odersky, M., Spoon, L., Venners, B.: Programming in Scala: A Comprehensive Step-by-Step Guide. Artima Incorporation (2008)Google Scholar
- 36.Osborne, W.G., Cheung, R.C.C., Coutinho, J., Luk, W., Mencer, O.: Automatic accuracy-guaranteed bit-width optimization for fixed and floating-point systems. In: Field Programmable Logic and Applications, pp. 617–620 (2007)Google Scholar
- 37.Panchekha, P., Sanchez-Stern, A., Wilcox, J.R., Tatlock, Z.: Automatically improving accuracy for floating point expressions. In: PLDI (2015)Google Scholar
- 38.Pang, Y., Radecka, K., Zilic, Z.: An efficient hybrid engine to perform range analysis and allocate integer bit-widths for arithmetic circuits. In: ASPDAC (2011)Google Scholar
- 39.Ramananandro, T., Mountcastle, P., Meister, B., Lethin, R.: A unified Coq framework for verifying C programs with floating-point computations. In: CPP (2016)Google Scholar
- 40.Rümmer, P., Wahl, T.: An SMT-LIB theory of binary floating-point arithmetic. In: SMT (2010)Google Scholar
- 41.Solovyev, A., Jacobsen, C., Rakamarić, Z., Gopalakrishnan, G.: Rigorous estimation of floating-point round-off errors with symbolic Taylor expansions. In: Bjørner, N., de Boer, F. (eds.) FM 2015. LNCS, vol. 9109, pp. 532–550. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19249-9_33CrossRefGoogle Scholar
Copyright information
<SimplePara><Emphasis Type="Bold">Open Access</Emphasis> This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.</SimplePara> <SimplePara>The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.</SimplePara>