Optimization Modulo the Theories of Signed Bit-Vectors and Floating-Point Numbers

Optimization modulo theories (OMT) is an important extension of SMT which allows for finding models that optimize given objective functions, typically consisting in linear-arithmetic or Pseudo-Boolean terms. However, many SMT and OMT applications, in particular from SW and HW verification, require handling bit-precise representations of numbers, which in SMT are handled by means of the theory of bit-vectors (BV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathcal {B}}}{{\mathcal {V}}}$$\end{document}) for the integers and that of floating-point numbers (FP\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {FP}$$\end{document}) for the reals respectively. Whereas an approach for OMT with (unsigned) BV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathcal {B}}}{{\mathcal {V}}}$$\end{document} objectives has been proposed by Nadel & Ryvchin, unfortunately we are not aware of any existing approach for OMT with FP\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {FP}$$\end{document} objectives. In this paper we fill this gap, and we address for the first time OMT\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {OMT}$$\end{document} with FP\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {FP}$$\end{document} objectives. We present a novel OMT approach, based on the novel concept of attractor and dynamic attractor, which extends the work of Nadel and Ryvchin to work with signed-BV\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathcal {B}}}{{\mathcal {V}}}$$\end{document} objectives and, most importantly, with FP\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {FP}$$\end{document} objectives. We have implemented some novel OMT\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {OMT}$$\end{document} procedures on top of OptiMathSAT and tested them on modified problems from the SMT-LIB repository. The empirical results support the validity and feasibility of our novel approach.

Nevertheless, many SMT and OMT applications, in particular from SW and HW verification, require handling bit-precise representations of numbers, which in SMT are handled by means of the theory of bit-vectors (BV) for the integers and that of Floating-Point Numbers (FP) for the reals respectively and their combination (BV ∪ FP). For instance, during the verification process of a piece of software, one may look for the minimum/maximum value of some int or double parameter causing an SMT(BV ∪ FP) call to return sat-which typically corresponds to the presence of some bug-so that to guarantee a safe range for such parameter; also, one may want to find the maximum relative difference in the double values returned by two implementations of the same function.
Example 1 Consider some C/C++ library implementation of some mathematical function f : Double N −→ Double. Suppose one wants to substitute it with a new implementation f (...) of the same function. Given the ranges [l, u] for the values of x, one may want to find the maximum relative difference between the value returned by the two functions. This can be done, e.g., by finding the maximum value of s.t. the SMT(BV ∪ FP) formula ..) and ( f (x) = ...) are the SMT(BV ∪ FP) 1 encodings of the implementations of the functions f and f respectively. 2 Notice that here it is strictly necessary to use bit-precise representation of numbers provided by BV ∪ FP -rather than standard non-linear arithmetic-in order to reproduce the truncating and rounding errors and their propagation. (E.g., two C functions computing iteratively a 0 + a 1 * x + ... + a n * x n and a 0 + x * (a 1 + x * (... + x * (a n ))..)) by floating-point arithmetic may return different values on the same input value x, although they are mathematically equivalent.) OMT for the theory of (unsigned) bit-vectors was proposed by Nadel and Ryvchin [32], although a reduction of the problem to MaxSAT was already implemented in the SMT/OMT solver Z3 [10]. The work in [32] was based on the observation that OMT on unsigned BV can be seen as lexicographic optimization over the bits in the bitwise representation of the objective, ordered from the most-significant bit (MSB) to the least-significant bit (LSB). Notice that, in this domain, this corresponds to a binary search over the space of the values of the objective.
In this paper (as in [44]) we address-for the first time to the best of our knowledge-OMT for objectives in the theory of signed Bit-Vectors and, most importantly, in the theory of Floating-Point Arithmetic, by exploiting some properties of the two's complement encoding for signed BV and of the IEEE 754-2008 encoding for FP respectively. (We consider the former as a straightforward extension of [32], and the latter as our main contribution. ) We start from introducing the notion of attractor, which represents (the bitwise encoding of) the target value for the objective which the optimization process aims at. This allows us to easily leverage the procedure of [32] to work with both signed and unsigned bit-vectors, by minimizing lexicographically the bitwise distance between the objective and the attractor, that is, by minimizing lexicographically the bitwise-xor between the objective and the attractor.
Unfortunately there is no such notion of (fixed) attractor for FP numbers, because the target value changes as long as the bits of the objective are updated from the MSB to the LSB, and the optimization process may have to change dynamically its aim, even in the opposite direction. (For instance, as soon as the minimization process realizes there is no solution with a negative value for the objective and thus sets its MSB to 0, the target value is switched from −∞ to 0 + , and the search switches direction, from the maximization of the exponent and the significand to their minimization.).
To cope with this fact, we introduce the notions of dynamic attractor and attractor trajectory, representing the dynamics of the moving target value, which are progressively updated as soon as the bits of the objective are updated from the MSB to the LSB. Based on these ideas, we present novel OMT procedures for FP objectives, which require at most n + 2, incremental calls to an BV ∪FP solver, n being the number of bits in the representation of the objective. Notice that these procedures do not depend on the underlying BV ∪ FP procedure used, provided the latter allows for accessing and setting the single bits of the objective.
Notice that, unlike with the BV domain, this does not simply perform binary search over the space of the values of the objective. Rather, it first performs (a lexicographic bitwise search corresponding to) binary search of the exponent values, which very-rapidly converges to the right order of magnitude, followed by binary search on the significand values, which fine-tunes the final result.
We have implemented these OMT procedures on top of the OptiMathSAT OMT solver [43]. We have run an experimental evaluation of the procedures on modified SMT problems from the SMT-LIB library. The empirical results support the validity and feasibility of the novel approach.
The rest of the paper is organized as follows. In Sect. 2 we provide the necessary background on BV and FP theories and reasoning. In Sect. 3 we provide the novel theoretical definitions and results. In Sect. 4 we describe our novel OMT procedures. In Sect. 5 we present the empirical evaluation. In Sect. 6 we conclude, hinting some future directions.

Background
We assume some basic knowledge on SAT and SMT and briefly introduce the reader to the Bit-Vector and Floating-Point theories.

Bit-Vectors
A bit is a Boolean variable that can be interpreted as 0 or 1. A Bit-Vector (BV) variable v [n] is a vector of n bits, where v[0] is the Most Significant Bit (MSB) and v[n − 1] is the Least Significant Bit (LSB). 3 A BV constant of width n is an interpreted vector of n values in {0, 1}. We overline a bit value or a BV value to denote its complement (e.g., [11010010] is [00101101]). A BV variable/constant of width n can be unsigned, in which case its domain is [0, 2 n − 1], or signed, which we assume to comply with the Two's complement representation, so that its domain is [−2 (n−1) , 2 (n−1) − 1]. Therefore, the vector [11111111] can be interpreted either as the unsigned BV constant 255 [8] or as the signed BV constant −1 [8] . Following the SMT-LIBv2 standard [4], we may also represent a BV constant in binary form (e.g. 28 [8] is written #b00011100). A BV term is built from BV constants, variables and interpreted BV functions which represents standard Register-Transfer Level (RTL) operators: word concatenation (e.g. 3 [8] • x [8] ), sub-word selection (e.g. (3 [8] [6 : 3]) [4] ), modulo-n sum and multiplication (e.g. x [8] + 8 y [8] and x [8] · 8 y [8] ), bit-wise operators (like, e.g., and n , or n , xor n , nxor n , not n ), left and right shift << n , >> n . A BV atom can be built by combining BV terms with interpreted predicates (either signed or unsigned ones) like ≥ n , < n (e.g. 0 [8] ≥ 8 x [8] ) and equality. We refer the reader to [4,25] for further details on the syntax and semantics of Bit-Vector theory.
There are two main approaches for BV satisfiability, the "eager" and the "lazy" approach, which are substantially complementary to one another [26]. In the eager approach, BV terms and constraints are encoded into SAT via bit-blasting [17,18,24,25,33,34]. In the lazy approach, BV terms are not immediately expanded-so to avoid any scalability issue-and the BV solver is comprised by a layered set of techniques, each of which deals with a subportion of the BV theory [11,16,19,25].

Floating-Point
The theory of Floating-Point Numbers (FP), [4,14,37], is based on the IEEE standard 754-2008 [5] for floating-point arithmetic, restricted to the binary case. A FP sort is an indexed nullary sort identifier of the form (_ FP < ebits > < sbits >) s.t. both ebits and sbits are positive integers greater than one, ebits defines the number of bits in the exponent and sbits defines the number of bits in the significand, including the hidden bit. where t is either 0 or 1 and s is a BV which contains at least a 1. Setting aside special FP constants, the remaining FP values can be classified to be either normal or subnormal (a.k.a. denormal) [5]. A FP number is said to be subnormal when every bit in its exponent is equal to zero, and normal otherwise. The significand of a normal FP number is always interpreted as if the leading binary digit is equal 1, whereas for denormalized FP values the leading binary digit is always 0. This allows for the representation of numbers that are closer to zero, although with reduced precision. Notice that the absolute value of any subnormal FP number is smaller than the absolute value of any non-zero normal FP number, and that the value contribution of the significand bits is always less significant than that of the exponent bits.
The theory of FP provides a variety of built-in floating-point operations as defined in the IEEE standard 754-2008. This includes binary arithmetic operations (e.g. +, −, , ÷), basic unary operations (e.g. abs, −), binary comparison operations (e.g. ≤, <, =, =, >, ≥), the remainder operation, the square root operation and more. Importantly, arithmetic operations are performed as if with infinite precision, but the result is then rounded to the "nearest" representable FP number according to the specified rounding mode. Five rounding modes are made available, as in [5].
The most common approach for FP-satisfiability is to encode FP expressions into BV formulas based on the circuits used to implement floating-point operations, using appropriate under-and over-approximation schemes-or a mixture of both-to improve performance [15,[45][46][47]. Then, the BV-Solver is used to deal with the FP formula, using either the eager or the lazy BV approach. An alternative approach, based on abstract interpretation, is presented in [12,13,27]. With this technique, called Abstract CDCL (ACDCL), the set of feasible solutions is over-approximated with floating-point intervals, so that intervals-based conflict analysis is performed to decide FP-satisfiability.

Theoretical Framework
We first present our generalization of [32] to the case of signed Bit-Vector Optimization (Sect. 3.1), and then move on to deal with Floating-Point Optimization (Sect. 3.2).

Bit-Vector Optimization
Without any loss of generality, we assume that every objective function f (...) is replaced by a variable obj of the same type by conjoining "obj = f (...)" to the input formula. We use the symbol n to denote the bit-width of obj, and obj[i] to denote the i-th bit of obj, where obj[0] and obj[n − 1] are the Most Significant Bit (MSB) and the Least Significant Bit (LSB) of obj respectively. 4 We define the Bit-Vector Optimization problem as follows.
Definition 1 (OMT [BV] (BV ∪ T )) Let ϕ be a SMT(BV ∪ T ) formula for some (possibly empty) theory T and obj be a-signed or unsigned-BV variable occurring in ϕ. We call an Optimization Modulo BVproblem for BV ∪ T , OMT(BV ∪ T ), the problem of finding a BV ∪ T -model M for ϕ (if any) whose value of obj is a minimum wrt. the total order relation ≤ n for signed BVs if obj is signed, and the one for unsigned BVs otherwise. (The dual definition where we look for the maximum follows straightforwardly) Notice that the definition is independent on the extra theory T , provided that obj is a BV term. (In practice T may be empty, or contain FP or/and other theories like e.g. that of arrays.) Hereafter, unless otherwise specified and when it is not necessary to make T explicit, we will abbreviate "OMT [BV] (BV ∪ T )" into "OMT [BV] ".
We generalize the unsigned BV maximization procedures in [32] to the case of signed and unsigned BV optimization. To this extent, we introduce the novel notion of BV attractor.
Definition 2 (Attractor, attractor equalities). When minimizing [resp. maximizing], we call attractor for obj the smallest [resp. greatest] BV-value attr of the sort of obj. We call vector of attractor equalities the vector A s.t.
In essence, the attractor can be seen as the target value of the optimization search and therefore it can be used to determine the desired improvement direction and to guide the decisions taken by the optimization search. By construction, if a model M satisfies all equalities A[i], then the evaluation of obj in M is attr.
We use the symbol μ k to denote a generic (possibly partial) assignment which assigns at least the k most-significant bits of obj. We use the symbol τ k to denote an assignment to the k most-significant bits of obj. Definition 3 (lexicographic maximization) Consider an OMT [BV] instance ϕ, obj and the vector of attractor equalities A. We say that an assignment τ n to obj lexicographically maximizes A wrt. ϕ iff, for every k ∈ [0..n − 1], ). Given a model M for ϕ, we say that M lexicographically maximizes A wrt. ϕ iff its restriction to obj lexicographically maximizes A wrt. ϕ.
Starting from the MSB to the LSB, τ n [resp.M] in Definition 3 assigns to each obj[k] the value attr[k] unless it is inconsistent wrt. ϕ and the assignments to the previous obj Notice that this corresponds to the minimization of n−1 )]-where xor n is the bitwisexor operator and nxor n is its complement-because 2 n−1−i > n−1 k=i+1 2 n−1−k for every n > i ≥ 0. 5 The following fact derives from the above definitions and the properties of two's complement representation adopted by the SMT-LIBv2 standard for signed BV. 6 Theorem 1 An optimal solution of an OMT [BV] problem ϕ, obj is any model M of ϕ which lexicographically maximizes the vector of attractor equalities A.
Proof (We investigate the minimization case, since the maximization case is dual.) In the case of minimization with unsigned BV, attr is [00...00], so that the lexicographic maximization of A corresponds to minimize n−1 k=0 2 n−1−k · obj[k] which is the standard minimization for unsigned BV. In the case of minimization with signed BV, attr is [10...00], so that the lexicographic maximization of A corresponds to minimize 2 n−1 , which is the standard minimization for two's complement BV.
Definitions 2 and 3 with Theorem 1 suggest thus a direct extension to the minimization/maximization of signed BV of the algorithm for unsigned BV in [32]: apply the unsigned-BVmaximization [resp. minimization] algorithm of [32] to the objective obj def = (obj nxor n attr)[resp.obj def = (obj xor n attr)] instead of simply to obj [resp.obj].

Example 4 Let obj [3] be a signed 3-bit BV goal to be minimized and attr
be the corresponding vector of attractor equalities. Consider the three assignments Then τ 3 is lexicographically better than τ 3 , because τ 3 satisfies the attractor equality corresponding to the MSB whereas τ 3 does not; τ 3 is lexicographically worse than τ 3 because-all the rest being equal-τ 3 makes the attractor equality (obj[2] = 0) true. Indeed, τ 3 is nearer in value to the attractor than τ 3 and is farther in value than τ 3 .

Floating-Point Optimization
We define the Floating-Point Optimization problem as follows. [FP] (FP ∪ T )) Let ϕ be a SMT(FP ∪ T ) formula for some (possibly empty) theory T and obj be a FP variable occurring in ϕ. We call an Optimization Modulo FPproblem for FP ∪ T ,OMT [FP] (FP ∪ T ) the problem of finding a FP ∪ T -model M for ϕ (if any) whose value of obj, is either -minimum wrt. the usual total order relation ≤ for FP numbers, if ϕ is satisfied by at least one model M s.t. M (obj) is not NaN, -some binary representation of NaN, otherwise.

Definition 4 (OMT
(The dual definition where we look for the maximum follows straightforwardly.) As with BV, the definition is independent on the extra theory T , provided that obj is a FP term. In practice T may be empty, or contain BV or/and other theories like e.g. that of arrays. Hereafter, unless otherwise specified and when it is not necessary to make T explicit, we will abbreviate "OMT [FP] Definition 4 is necessarily convoluted because obj can be NaN. In fact, in the SMT-LIBv2 standard the comparisons {≤, <, ≥, >} between NaN and any other FP value are always evaluated false because NaN has multiple representations at the binary level (see Table 1). Also, requiring the optimal solution to be always different from NaN makes the resulting OMT [FP] problem ϕ ∧ ¬IsNaN(obj), obj unsatisfiable when ϕ is satisfied only by models M s.t. M(obj) is NaN. For these reasons, we admit NaN as the optimal solution value for obj if and only if ϕ is satisfied only by models M s.t. M(obj) is NaN.
In the rest of this section we assume that we have already checked, in sequence, that (i) the input formula ϕ is satisfiable-by invoking an SMT(FP) solver on ϕ. If the solver returns unsat, then there is no need to proceed; If the solver returns unsat, then we conclude that the minimum is NaN. Thus, we can safely focus our investigation on the restricted OMT [FP] problem ϕ noNaN , obj , where ϕ noNaN def = ϕ ∧ ¬IsNaN(obj), knowing it is satisfiable. In Sect. 3.1, we have introduced the concept of a BV attractor, showing how this value can be used to drive the optimization search towards the optimum value, when minimizing or maximizing a signed or unsigned BV goal. However, in the case of floating-point optimization, it is not possible to statically determine the attractor value in advance, before the search is even started. This is due to the more complex representation of FP variables, which uses three separate Bit-Vectors (i.e. sign, exponent and significand), and the presence of various classes of special values (i.e. zeros, infinity, NaN), which make Definition 2 ambiguous for FP optimization. We illustrate this problem with the following example.
Example 5 Let ϕ noNaN , obj be an OMT [FP] problem where obj is a FP objective, of sort (_ FP 3 5), to be minimized. To make our explanation easier to follow, we show in Table 1 a short list of sample values for an FP variable of the same sort as obj. Each FP value is represented as a triplet of bit-vectors sign, exp, sig -following the SMT-LIBv2 conventions described in Sect. 2-and also in decimal notation.
From Table 1, we immediately notice that the binary representation of both the exponent and the significant of a Floating-Point number grows in opposite directions in the positive and in the negative domains. In addition, by sorting the values according to their binary representation, we observe that −∞ [resp. +∞ ] is not the smallest [resp. greatest] representable FP value in the negative [resp. positive] domain. In fact, both extreme ends of the table are occupied by NaN, which has multiple binary representations.
In what follows, we temporarily disregard the effects of unit-propagation, which might assign some (or all) bits of obj as a result of some constraints in ϕ noNaN , and pick some values as candidate attractors for an FP goal to be minimized.
Assume that the optimal value of the FP goal is the sub-normal FP value (fp #b1 #b000 #b1111) (i.e. −15 64 ). Suppose that the attractor is chosen to be equal to the value −∞ listed at row 9 in Table 1, which is the smallest FP value wrt. total order relation ≤ for FP numbers. Then, it can be seen that after both the sign and the exponent bits have been decided to be equal #b1 and #b000 respectively, the remaining bits of the attractor pull the search in the wrong direction, that is, towards 0 − .
Selecting a different FP value as candidate attractor would not solve the problem; rather, it would result in a different set of issues. For instance, an attractor equal to the NaN value listed at row 10 in Table 1, which is the smallest representable FP value according to the binary ordering, would solve the problem for the previous case in which the optimum FP value is (fp #b1 #b000 #b1111). However, this attractor would remain an unsuitable choice for an OMT [FP] instance where obj is forced to be positive, because after the sign bit of the objective function has been decided to be equal #b0 the remaining bits of the attractor drive the search in the wrong direction, that is, towards +∞.
Since there is no statically-determined FP value that can be used as an attractor when dealing with floating-point optimization, we introduce the new concept of dynamic attractor.
Definition 5 (Dynamic attractor) Let ϕ noNaN , obj be a restricted OMT [FP] problem, where ϕ noNaN def = ϕ ∧¬IsNaN(obj) is a satisfiable SMT(FP) formula and obj is a FP objective to be minimized [resp. maximized]. Let k ∈ [0..n] and τ k be an assignment to the k most-significant bits of obj.
Then, we say that an FP-value attr τ k for obj is a dynamic attractor for objwrt. τ k iff it is the smallest [resp. largest] FP value different from NaN s.t. the k most-significant bits of attr τ k have the same value of the k most-significant bits of obj in τ k . We call vector of attractor equalities the vector The following fact derives from the above definitions and the properties of IEEE 754-2008 standard representation adopted by SMT-LIBv2 standard for FP.
Lemma 1 Let ϕ noNaN , obj be a restricted minimization [resp.maximi zation] OMT [FP] problem, let τ k be an assignment to obj Lemma 1 states that, given the current assignment τ k to the k most-significant-bits of obj, obj[k] = attr τ k [k] is always the best extension of τ k to the next bit (when consistent). A dynamic attractor attr τ k can thus be used by the optimization search to guide the assignment of the k + 1-th bit of obj towards the direction of maximum gain which is allowed by τ k , so that to obtain the "best" extension τ k+1 of τ k . Once the (new) assignment τ k+1 is found, the OMT solver can compute the dynamic attractor attr τ k+1 for obj wrt. τ k+1 and then use it to assign the k + 2-th bit of obj, and so on.
Let ϕ noNaN , obj be an OMT [FP] instance, s.t. obj is a FP variable of n bits, and τ 0 be an initially empty assignment. If at each step of the optimization search the assignment of the k-th bit of obj is guided by the dynamic attractor for obj wrt. τ k , then the corresponding sequence of n dynamic attractors (of increasing order k) is unique and depends exclusively on ϕ noNaN . Intuitively, this is the case because the (current) dynamic attractor always points in the direction of maximum gain. We illustrate this in the following example.

Example 6
Let ϕ noNaN , obj be an OMT [FP] problem where obj is a FP objective, of sort (_ FP 3 5), to be minimized, as in Example 5. At the beginning of the search, nothing is known about the structure of the solution. Therefore, τ 0 = ∅ and, since obj is being minimized, the dynamic attractor attr τ 0 for obj wrt. τ 0 is (fp #b1 #b111 #b0000) (i.e. −∞), which gives a preference to any feasible value of obj in the negative domain.
If we discover that the domain of the objective function can only be positive, so that the first bit of obj is permanently set to 0 in τ 1 , then the new dynamic attractor for obj wrt. τ 1 (i.e. attr τ 1 ) is equal to (fp #b0 #b000 #b0000) (i.e. 0 + ). Otherwise, attr τ i remains −∞ until, e.g., we discover there is no solution ≤ −8 so that the second bit in the exponent is forced to 0. Then attr τ 3 becomes (fp #b1 #b101 #b1111) (i.e., −31 4 ). ) Notice that all significand bits in the attractor pass from 0 to 1 because now we have a finite solution.
Finally, we make the following two observations. The first is that the sequence τ 0 , ..., τ n in Definition 6 can be iteratively constructed using its list of requirements, for instance, by means of a sequence of incremental calls to an SMT solver. The second, more important, observation is that τ n corresponds to the assignment of values which makes obj optimal in ϕ noNaN . Using the above definitions, we show that the following fact holds. ϕ noNaN , obj , τ 0 , ..., τ n , attr τ 0 , ...., attr τ n , A τ 0 , ..., A τ n , and ϕ be as in Definition 6. Then, any model M of ϕ noNaN which lexicographically maximizes the attractor trajectory ϕ is an optimal solution for the OMT [FP] problem ϕ noNaN , obj .

Theorem 2 Let
Proof (We prove the case of minimization, since the case of maximization is dual.) By Lemma 2 we have that τ n lexicographically maximize A ϕ . Let M be a model of ϕ noNaN which lexicographically maximizes A ϕ , and let μ be its restriction to obj. Since both τ n and M lexicographically maximize A ϕ , from the uniqueness of τ n , we immediately notice that μ = τ n , so that τ k =

OMT [FP] Procedures
In this paper, we consider two approaches for dealing with OMT [FP] : a baseline linear/binary search, based on the inline OMT schema for LAA objectives presented in [39], and Floating-Point Optimization with Binary Search (ofpbs), a brand-new engine inspired by the obv-bs algorithm for unsigned bit-vectors in [32] and by Theorem 2 and relative definitions in Sect. 3.2.

OMT-Based Approach
The OMT-based approach for OMT [FP] adapts the linear-and binary-search schemata for OMT with LAA objectives presented in [39] to deal with FP objectives.
In the basic linear-search schema, the optimization search is advanced by means of a sequence of linear cuts, each of which forces the OMT solver to look for a new model M which improves the value of obj wrt. the most recent model M. In the binary-search schema, instead, the OMT solver learns an incremental sequence of cuts which bisect the current domain of the objective function. For clarity, we recap here the essential elements of the binary-search schema presented in [38,39]. At the beginning of the optimization search and following each update of the lower-(lb) and upper-(ub) bounds of obj, the OMT solver computes a pivoting value pivot def = floor(ρ · ub + (1 − ρ) ·lb), for some value of ρ (e.g. 1 2 ). If pivot lies inside the range ]lb, ub], a cut of the form (obj < pivot) is learned. Otherwise, if-due to rounding side-effects of FP operations-pivot lies outside the range ]lb, ub], a cut of the form (obj < ub) is learned instead. If the cut is satisfiable, the upper-bound of obj is updated with a new model value of obj. Otherwise, the lower-bound is made equal to pivot [resp. ub]. The algorithm terminates when the search interval [lb, ub[ becomes empty. In general, it is reasonable to expect the binary-search schema to converge towards the optimal solution faster than the linear-search schema, because the feasible domain of a FP goal can be comprised by an exponentially large number of values (wrt. the bit-width of the cost function).
In either schema, whenever the optimization engine encounters for the first time a solution s.t. obj = NaN, the OMT solver learns a unit-clause of the form ¬(isNaN(obj)) so as to look for an optimal solution different from NaN (if any).
When dealing with FP objectives, differently from the case of LRA in [39], it is not necessary to implement a specialized optimization procedure within the FP-Solver in order to guarantee the termination of the optimization search. Indeed, such procedure is not available when Floating-Point terms are bit-blasted into bit-vectors eagerly, or when the acdcl FP-Solver is used, because by the time the optimization procedure is called the domain interval of any FP term contains a singleton value. Conversely, such a minimization procedure could be envisaged when the OMT solver uses a lazy FP-Solver as back-end, so as to speed-up the convergence towards the optimal solution 7 .

Floating-Point Optimization with Binary Search
The Floating-Point Optimization with Binary Search algorithm, ofp-bs, is a new engine for OMT [FP] (FP ∪T )-hereafter simply OMT [FP] -which is inspired by the obv-bs algorithm for OMT [BV] [32] and implements Definition 6 and Theorem 2. Here T may be empty, or contain BV and other theories (e.g. that of arrays). We assume that an SMT(BV ∪ FP ∪ T )solving procedure is available-hereafter simply "SMT"-even when BV is not part of T , because we need accessing explicitly to each bit in obj, which is not possible with plain FP.
The optimization search tries to lexicographically maximize the (implicit) attractor trajectory vector A ϕ , which is incrementally derived from the current value of the dynamic attractor. The raw value of the dynamic attractor's bits drive the optimization search towards the direction of maximum gain at any given point in time, without disrupting any decision that has been already made. The dynamic attractor is incrementally updated along the search, based on the outcome of the previous rounds of the optimization search. At each round, one bit of the objective function is assigned its final value. The first round decides the sign, the next batch of rounds decides the exponent, and the remaining rounds decide the fine-grained details of the significand.
The pseudo-code of ofp-bs is shown in Fig. 2. The arguments of the algorithm are the input formula ϕ and the FP objective obj, where obj is a FP variable with ebits bits in the exponent, sbits − 1 in the significand and n def = ebits + sbits bits overall. The procedure starts by checking whether the input formula ϕ is satisfiable and immediately terminates if this is not the case (rows 1-3). If M(obj) = NaN, then the procedure checks whether there exists a model M for ϕ ∧ ¬IsNaN(obj) (rows [4][5]. If this is not the case, the procedure terminates immediately and returns the pair sat, M (row 7). Otherwise, the model M is updated with the new model M (row 9). In every case, ϕ is permanently extended with the constraint ¬IsNaN(obj) (row 10).
At this point, the procedure initializes the value of the dynamic attractor by invoking an external function update_dynamic_attractor() with the empty assignment τ as parameter, so that the returned value is equal to −∞ when minimizing and +∞ when maximizing   [11][12]. Then, the execution moves to the section of code implementing the core part of the ofp-bs algorithm (rows 13-24), which consists of a loop over the bits of obj, starting from the MSB obj[0] down to the LSB obj[n − 1] (Fig. 3).
Inside this loop, ofp-bs first checks whether the value of obj[i] in M matches the i-th bit of the (current) dynamic attractor attr τ . If this is the case, then the i-th bit is already set to its "best" value in M. Thus, the assignment τ is extended so as to permanently set obj  In either case, the execution moves to the next iteration of loop.
After exactly n iterations of the loop, the optimization search terminates with the pair sat, M , where M is the optimum model of the given OMT [FP] instance. The ofp-bs algorithm requires at most n + 2 incremental calls to an underlying SMT(FP) solver. The test in rows 15 − 16 allows for saving lots of such SMT calls when the current model already assigns obj[i] to its corresponding value in the attractor.
The function update_dynamic_attractor() takes as input τ , a (partial) assignment over the k most-significant bits of obj, and i, the index of of the current loop iteration in ofp-bs. When obj is minimized (The implementation is dual when obj is maximized), the procedure essentially works as follows. If τ = ∅, then nothing is known about the solution of the problem, so −∞ is returned. Otherwise, the procedure must compute the smallest FP value different from NaN (if any) which extends τ . In this case, the procedure starts by flipping the value of attr τ [i], forcing obj[i] = attr τ [i] (row 3). This ensures that the value of the first i + 1 bits of obj in M, corresponding to the assignment τ , is the same as the first i +1 bits of the current dynamic attractor. The remaining n −i −1 bits of attr τ may also need to be updated to reflect this change. Since τ = ∅ then we know that the sign of the objective function has been permanently decided in τ . If obj[0] = 0 in τ , i.e. obj must be positive, the procedure must return the smallest positive FP value admitted by τ . Hence, we update attr τ with j=n−1 j=i+1 attr τ [ j] = 0 and return the corresponding FP value (rows 4-6). If obj[0] = 1 in τ , i.e. obj can be negative values, the procedure must return the largest negative FP value admitted by τ . When i ≤ ebits then at least one bit in the exponent of obj is assigned to 0 in τ (i.e. obj [i]). If that is the case, then we update attr τ with j=n−1 j=i+1 obj[ j] = 1 and return the corresponding FP value (rows 7-10). In practice, we notice that the block of code at rows 4-10 needs to be executed at most once because the decision of tracking the smallest positive value or the largest negative value (different from −∞) is permanent.

Example 8
Let ϕ noNaN , obj be a restricted OMT [FP] problem where obj is a FP objective, of sort (_ FP 3 5), to be minimized. We consider the case in which the input formula ϕ noNaN requires obj to be larger or equal −21 4 and it does not impose any other constraint on the value of obj. Given the sequence of (partial) assignments τ 0 , ..., τ 8 in Fig. 4, it can be seen that after determining the unsatisfiability of obj[2] = attr τ 2 [2], the dynamic attractor must start tracking the largest negative value different from −∞. Hence, the value of the last n − i − 1 bits of the dynamic attractor are set to be equal 1. Any subsequent call to update_dynamic_attractor() needs only to flip the value of attr τ [i], because the last n − i − 1 bits of the dynamic attractor are already set to be equal 1.
We stress the fact that, unlike with the LA [38,41] and BV [32] objective domains, ofpbs does not simply perform binary search over the space of the values of the objective. Rather, after deciding the sign, it first performs binary search of the exponent values, which very-rapidly converges to the right order of magnitude, followed by binary search on the significand values, which fine-tunes the final result.

Example 9
To understand the range-pruning power of binary search over the exponent, consider the case of a 32-bit FP obj with 8-bit exponent and 23-bit significand. After assigning, e.g., the sign bit to 0 (positive value) the range of possible values is [0 + , +∞] ([0 + , +3.4.10 38 ] if we exclude +∞); assigning then the first exponent bit to 0, the range reduces to [0 + , 2.0], reducing the range by more than a 10 38 factor; by further setting the second exponent bit to 0, [0 + , 1.1 · 10 −19 ], further reducing the range by more than a 10 19 factor, and so on.

Search Enhancements
Given a FP value attr and a FP goal obj, (a combination of) the following techniques can be used to adjust the behavior of the optimization search, similarly what has been proposed for the case of OMT [BV] by Nadel et al. in [32].
branching preference: the bits of the FP objective obj are marked, inside the OMT solver, as preferred variables for branching starting from the MSB down to the LSB. This ensures that conflicts involving the value of the objective function are handled as early as possible, possibly reducing the amount of work that needs to be redone after each back-jump. -polarity initialization: the phase-saving value of each obj[i] is initialized with the value of attr [i]. This encourages the OMT solver to assign the bits of obj so as to reassemble the bits of attr, thus possibly speeding-up the convergence towards the optimal value.
In the case of the basic OMT schema described in Sect. 4.1, the effectiveness of either technique depends on the initial choice for attr. In the lucky case, the value of attr pulls the optimization search in the right direction and speeds up the search. In the unlucky case, when attr pulls in the wrong direction, there is no visible effect or an overall slow down. For instance, in the case of the linear-search optimization schema, enabling both options with an unlucky choice of attr can cause the OMT solver to start the search from the furthest possible point from the optional solution, and thus enumerate an exponential number of intermediate solutions. Naturally, the OMT-based optimization search algorithm is still guaranteed to terminate even in the worst-case scenario, but the unpredictable performance makes using either technique a generally unsuitable option in practice.
In the case of the ofp-bs algorithm described in Sect. 4.2, we use the latest value of the dynamic attractor attr τ for both the branching preference (lines 11 and 18 of Fig. 2) and the polarity initialization (rows 12 and 19 of Fig. 2) techniques. We observe that the value of every bit in the dynamic attractor can change after the sign of the objective function has been decided. Furthermore, the value of all the significand's bits in the dynamic attractor can also change during the process of determining the optimal exponent value of the objective function (see, e.g., Example 5). As a consequence, if the OMT solver applies either enhancement before the correct improving direction is known, this may cause the underlying OMT engine to advance the search starting from a sub-optimal set of initial decisions. Enabling both enhancements at the same time could make things even worse. In order to mitigate this issue, we have designed a variant of our optimization-search approach which does not apply either enhancement on those bits of the objective function for which the best improving direction is not yet known. We have called this variant safe bits restriction.

Experimental Evaluation
We have implemented the procedures described in the previous sections on top of the Opti-MathSAT OMT solver (v. 1.6.2), and assessed its performance on a set of OMT [FP] formulas that have been automatically generated using the SMT(FP) benchmark-set of [4]. The formulas, the results and the scripts necessary to reproduce these results are made publicly available and can be downloaded from [1,2]. The experiments have been performed on an i7-6500U 2.50GHz Intel Quad-Core machine with 16G B of ram and running Ubuntu Linux 17.10. For each job pair we used a timeout of 600 seconds.
Experiment Setup. The OMT [FP] instances used in this experiment have been automatically generated starting from the satisfiable formulas included in the SMT(FP) benchmark-set of [4]. We did not consider any of the unsatisfiable instances that are present in the remote repository. For each of the original SMT(FP) formulas we applied the following transformations. First, we either relaxed or removed some of the constraints in the original problem, so as to broaden the set of feasible solutions. This step is necessary because the majority of the original SMT(FP) formulas admits only one solution. Second, for each FP variable v appearing inside a SMT(FP) problem we generated a pair of OMT [FP] instances, one for the minimization and another for the maximization of v. At the end of this step, we obtained 39536 OMT [FP] formulas. Third, we randomly selected up to 300 OMT [FP] instances from each of the five groups of problems in the OMT [FP] benchmark-set. This filtering step yielded a total of 1120 SMT-LIBv2 formulas.
The first two OMT-based baseline implementations we have considered are OptiMath-SAT(omt+lin) and OptiMathSAT(omt+bin), that run the linear-and the binary-search respectively. These configurations have been tested using both the eager and the lazy FP approaches. The third baseline implementation we have considered, named OptiMath-SAT(eager+obv-bs), is based on a reduction of the OMT [FP] problem to OMT [BV] and it uses OptiMathSAT's implementation of the obv-bs engine presented by Nadel et al. [32]. 8 For this test, we have generated an OMT [BV] benchmark-set using a BV encoding that mimics the essential aspects of the ofp-bs algorithm described Sect. 4.2. We compared these baseline approaches with a configuration using the ofp-bs algorithm and the eager FP approach, namely OptiMathSAT(eager+ofp-bs). We have separately tested the effect of enabling the branching preference (bp), the polarity initialization (pi) and the safe bits restriction (so) enhancements described in Sect. 3.2, whenever these options were supported by the given configuration. We have not included other tools in our experiment because we are not aware of any other OMT [FP] solver.
Last, in order to assess the significance of the optimization problems used in this experiment, we have collected the run-time statistics of OptiMathSAT on the SMT formulas   Table 2 and results in a drastic increase in the number of timeouts (Table 2). We justify these results as follows. First, when only polarity initialization is used, the phase-saving value that is being set by OptiMathSAT does not really matter because the optimization search is dominated by the structure of the formula itself rather than by the bits of the FP objective. Second, when polarity initialization is used on top of branching preference, there is an even more drastic decrease in performance due to the fact that the initial phase-saving value that is statically assigned by the OMT solver to the bits of the FP objective cannot be expected to be "good enough" for any situation. In fact, as illustrated in example 5, the initial phase-saving can be misleading and force the OMT solver-when running in linear-search-to explore an exponential number of intermediate satisfiable solutions.
In the case of the OMT-based binary-search optimization approach, we observe that it solves more formulas than linear-search and it generally appears to be faster (plot 3B in Fig. 6). Overall, polarity initialization does not seem to be beneficial, whereas enabling branching preference increases the number of formulas solved within the timeout. This behavior is different from the linear-search approach, and we conjecture that it is due to the fact that, with the OMT-based binary-search approach, branching over the bits of the objective function can reveal in advance any (partial) assignment to the bits of the objective function that it is inconsistent wrt. the pivoting cuts learned by the optimization engine.
Using the lazy FP engine results in fewer formulas being solved, although a significant number of these benchmarks is solved faster than with any other configuration (over 90 instances, for both configurations).
The OptiMathSAT(eager+obv-bs) configuration is able to solve 1013 formulas within the timeout, showing that OMT [FP] can be reduced to OMT [BV] effectively, and that-on the given benchmark-set-the performance of this approach are comparable with the best OMT [FP] configurations being tested. Overall, the best performance is obtained by using the ofp-bs engine, with up to 1019 benchmark-set instances solved in correspondence to the OptiMathSAT(eager+ofp-bs+pi) configuration. In plot 2B of Figs. 6 and 7, we show the pairwise comparison of the best ofp-bs configuration with the best OMT-based run. Similarly to the case of OMT-based optimization with linear-search, we observe that enabling branching preference generally makes the performance worse (plot 1A in Fig. 8). Instead, when polarity initialization is used we observe a general performance improvement that does not only result in an increase in the number of formulas being solved within the timeout, but also a noticeable reduction of the solving time as a whole. This is in contrast with the case of OMT-based optimization, and it can be explained by the fact that ofp-bs uses an internal heuristic function to dynamically determine and update the most appropriate phase-saving value for the bits of the objective function. An equally important role is played by the safe bits restriction, that limits the effects of branching preference and polarity initialization to only certain bits of the dynamic attractor. As illustrated by the plots in the second and third rows of Fig. 8 and by the data in Table 2, tThis feature is particularly effective when used in combination with branching preference.
The results of OptiMathSAT over the SMT-only version of the benchmark-set (no optimization) are reported in the last row of Table 2 and in the scatter-plot 3B in Fig. 7, and show that for a large number of instances the OMT problem is considerably harder than its SMT-only version. There are a few exceptions to this rule, that we ascribe to the fact that the removal of the objective function alters the internal stack of formulas, and this can have unpredictable consequences on the behavior of various internal heuristics that depend on it. A solution can be found in a shorter amount of time when the sequence of (heuristic) choices is compatible with its assignment and it requires little back-tracking effort.

Conclusions and Future Work
We have presented for the first time OMT procedures for (signed bit-vectors and) floatingpoint objectives, based on the novel notions of attractor and dynamic attractor, which we have implemented in OptiMathSAT and tested on modified problems from SMT-LIB.
Ongoing research involves implementing our ofp-bs procedure on top of the ACDCL SMT(FP) procedure-which is not immediate to do efficiently because the latter approach does not allow directly accessing and setting the single bits of the objective (since BV and FP are not signature-disjoint). Future research involves experimenting the new OMT procedure directly on problems coming from bit-precise SW and HW verification, produced, e.g., by the NuXmv model checker [3].
Funding Open access funding provided by Università degli Studi di Trento within the CRUI-CARE Agreement.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.