Abstract
We apply multivariate Lagrange interpolation to synthesizing polynomial quantitative loop invariants for probabilistic programs. We reduce the computation of a quantitative loop invariant to solving constraints over program variables and unknown coefficients. Lagrange interpolation allows us to find constraints with less unknown coefficients. Counterexampleguided refinement furthermore generates linear constraints that pinpoint the desired quantitative invariants. We evaluate our technique by several case studies with polynomial quantitative loop invariants in the experiments.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
1 Introduction
A probabilistic program may change its computation due to probabilistic choices. Consider, for instance, the MillerRabin algorithm for primality test [27]. Given a composite number, the algorithm reports incorrectly with probability at most 0.25. Since the outcome of the algorithm is not always correct, classical program correctness specifications [9, 14, 20] do not apply. For probabilistic programs, quantitative specifications are needed to reason about program correctness [8, 23, 24]. Instead of logic formulae, probabilistic programs are specified by numerical functions over program variables. Since a probabilistic program gives random outcomes, a numerical function may have different values on different executions. The expected value of a numerical function is then determined by the probability distribution induced by the executions of program.
Since probabilistic programs are specified by numerical functions, their correctness can be established by annotations with expectations. In particular, correctness of while loops can be proved by inferring special expectations called the quantitative loop invariants [24, 25]. Similar to classical programs, finding general quantitative loop invariants is hard. Techniques for generating linear quantitative loop invariants however are available [1, 15, 22, 25].
Interestingly, existing linear loop invariant generation techniques can be extended to synthesize polynomial invariants [1]. Observe that polynomial multivariate polynomials are linear combinations of monomials. For instance, any polynomial over x, y with degree 2 is a linear combination of the monomials \(1, x, y, x^2, y^2,\) and xy. It suffices to find coefficients of the monomials to represent any multivariate polynomial of a fixed degree. Linear loop invariant generation techniques can hence be applied to infer invariants of a fixed degree. The number of monomials however grows rapidly. Quadratic polynomials over 5 variables, for example, are linear combinations of 21 monomials. One then has to find as many coefficients. It is unclear whether the extended approach is still feasible.
In this paper, we develop a Lagrange interpolationbased technique to synthesize polynomial loop invariants for simple loops in probabilistic programs. Lagrange interpolation is a wellknown method to construct explicit expressions for polynomials by sampling. For example, suppose that the values of f(x) are known to be \(s_1\), \(s_3\), and \(s_4\) at the sampling points 1, 3, and 4, respectively. By Lagrange interpolation, we immediately have an explicit expression of \(f(x) = s_1 \cdot \frac{(x  3)(x  4)}{(1  3)(1  4)} + s_3 \cdot \frac{(x  1)(x  4)}{(3  1)(3  4)} + s_4 \cdot \frac{(x  1)(x  3)}{(4  1)(4  3)}.\) Our new technique employs multivariate Lagrange interpolation. Similar to previous techniques [15, 22], we use conditions of quantitative loop invariants as constraints. Lagrange interpolation moreover allows us to simplify the constraints and sometimes to determine several coefficients. In the example, suppose \(f(3) = 1\) is known. Then it suffices to determine \(s_1\) and \(s_4\) to construct an explicit expression of f(x). In contrast, if f(x) is represented as \(c_0 + c_1 x + c_2 x^2\), then \(f(3) = 1\) only gives \(c_0 + 3 c_1 + 9 c_2 = 1\) and determines none of the coefficients. Lagrange interpolation hence can reduce the number of unknown coefficients and make our technique more scalable.
Although there are less unknown coefficients, one still has to solve nonlinear constraints. We give heuristics to determine coefficients efficiently. Our heuristics first perform random experiments and obtain linear constraints about coefficients. An SMT solver is then used to find candidate coefficients from the constraints. If there is no candidate, then the desired loop invariant does not exist. Otherwise, quantifier elimination verifies whether the candidate coefficients give a loop invariant. If so, our technique has found a quantitative loop invariant. Otherwise, we add more linear constraints to exclude infeasible coefficients.
We apply our technique to find quantitative loop invariants for ten annotated loops from nontrivial probabilistic programs. Our case studies range from gambler’s ruin problem [13] to simulation of a fair coin with a biased coin [15]. Over 1000 random runs, our technique is able to synthesize polynomial quantitative loop invariants within 15 s on average. Besides, 97.5 % of the runs can finish within a 300 s timeout.
Related Work. Constraintbased techniques for automated loop invariants generation have been much progressed over the past years [4, 5, 17, 18, 21, 22, 29]. Gupta and Rybalchenko [18, 19] proposed a GEGAR framework, so that static and dynamic information of a program can be exploited incrementally to restrict the search space of qualitative loop invariants. Sankaranarayanan et al. [29] used Gröbner bases to reduce the generation of algebraic polynomial loop invariants to solving nonlinear constraints in the parametric linear form. These techniques however deal with classical programs and cannot be applied to probabilistic programs directly. McIver and Morgan [24] were among the first to consider quantitative loop invariants for probabilistic programs. Katoen et al. [22] studied the synthesis of quantitative loop invariants using a constraintsolving approach. The approach was further developed and implemented in the Prinsys tool [15], which synthesizes quantitative invariants by solving constraints over unknown template coefficients. The performance of the tool however is sensitive to manually supplied templates. Recently, a technique based on abstract interpretation is proposed in [1]. It formulates linear loop invariants with the collecting semantics and synthesizes coefficients via fixedpoint computation. Although the authors only report experiments on linear loop invariants, the technique can be extended to generate polynomial invariants by representing polynomials as linear combinations of monomials. The effectiveness of the extension however is unclear.
We have the following organization. After preliminaries, we review probabilistic programs in Sect. 3. Quantitative loop invariants are presented in Sect. 4. Section 5 introduces multivariate Lagrange interpolation. Our technical contribution is presented in Sect. 6. Applications are given in Sect. 7. We evaluate our technique in the following section. Section 9 concludes our presentation.
2 Preliminaries
Let \(\mathbf {x}_{m}\) be a sequence of variables \(x_1, x_2, \ldots , x_m\). We use \({\mathbb R}[\mathbf {x}_{m}^{n}]\) to denote the set of real coefficient polynomials over m variables of degree at most n. Observe that \({\mathbb R}[\mathbf {x}_{m}^{n}]\) can be seen as a vector space over \({\mathbb R}\) of dimension \(d = \genfrac(){0.0pt}1{m+n}{n}\). For instance, the set of d monomials \(\{ x_1^{d_1} x_2^{d_2} \cdots x_m^{d_m} : 0 \le d_1 + d_2 + \cdots + d_m \le n \}\) forms a basis of \({\mathbb R}[\mathbf {x}_{m}^{n}]\). Given \(f \in {\mathbb R}[\mathbf {x}_{m}^{n}]\) and expressions \(e_1, e_2, \ldots , e_m\), we use \(f (e_1, e_2, \ldots , e_m)\) to denote the polynomial obtained by replacing \(x_i\) with \(e_i\) for \(1 \le i \le m\) in f. Particularly, f(v) is the value of f at \({v} \in {\mathbb R}^m\).
A constraint is a quantifierfree logic formula with equality, linear order, addition, multiplication, division, and integer constants. A constraint is linear if it contains only linear expressions; otherwise, it is nonlinear. A quantified constraint is a constraint with quantifiers over its variables. A valuation over \(\mathbf {x}_{m}\) assigns a value to each variable in \(\mathbf {x}_{m}\). A model of a constraint is a valuation which evaluates the constraint to true.
Given a quantified constraint, quantifier elimination removes quantifiers and returns a logically equivalent constraint. Given a linear constraint, a Satisfiability Modulo Theory (SMT) solver returns a model of the constraint if it exists.
3 Probabilistic Programs
A probabilistic program in the probabilistic guarded command language is of the following form:
where E is an expression and G is a Boolean expression. For \(p \in (0, 1)\), the probabilistic choice command \(P_0 [p] P_1\) executes \(P_0\) with probability p and \(P_1\) with probability \(1  p\). For instance, \({x{\,}{:}{=}{\,}\,} {1}\,[0.75]\,{x{\,}{:}{=}{\,}0} \) sets x to 1 with probability 0.75 and to 0 with probability 0.25. A program state is a valuation over program variables. For simplicity, we assume program variables are in nonnegative integers, and use 0 and 1 for the truth values \(\mathsf {false}\) and \(\mathsf {true}\) respectively.
Example 1
Consider the following probabilistic program:
The program models a game where a player has x dollars at the beginning and keeps tossing a coin with head probability 0.5. The player wins one dollar for each head and loses one dollar for each tail. The game ends either when the player loses all his money, or when he wins \(yx\) dollars for a predetermined \(y>x\). The variable z counts the number of tosses made by the player during the game.
3.1 Expectations
From an initial program state, a probabilistic program can have different final program states due to probabilistic choice commands. Particularly, a function over program variables gives different values on different final program states. Note that a probabilistic program induces a probability distribution on final program states. One therefore can discuss the expected value of any function over program variables with respect to that probability distribution. More precisely, one can take an expectation transformer [24] approach to characterize a probabilistic program by annotating the program with expectations.
Formally, an expectation is a function mapping program states to a nonnegative real number. An expectation is called a postexpectation when it is to be evaluated on final program states. Similarly, an expectation is called a preexpectation if it is to be evaluated on initial program states. Let preE and postE be expectations, and prog a probabilistic program. We say a quantitative Hoare triple \(\langle preE\rangle \; prog\; \langle postE\rangle \) holds if the expected value of postE is no less than that of preE before executing prog. Note that the expected values of postE and preE are functions over states and hence are compared pointwisely.
For any Boolean expression G, define the indicator function \({[}{G}{]} = 1\) if G is true and \({[}{G}{]} = 0\) otherwise. Consider an qualitative Hoare triple \(\{ P \}\; prog\; \{ Q \}\) with a precondition P, a postcondition Q, and a classical program prog. Observe that \(\{ P \}\; prog\; \{ Q \}\) holds if and only if \(\langle {[}{P}{]}\rangle \; prog\; \langle {[}{Q}{]}\rangle \) holds. Expectations are therefore the quantitative analogue to predicates for classical programs.
3.2 Expectation Transformer for Probabilistic Programs
Let P and Q be probabilistic programs, g a postexpectation, x a program variable, E an expression, G a Boolean expression, and \(p \in (0, 1)\). Define the expectation transformer \(wp({\,}\cdot {\,}, g)\) as follows [24].
Here g[x / E] denotes the formula obtained from g by replacing free occurrences of x by E. The least fixed point operator \(\mu \) is defined over the domain of expectations [16]. It can be shown that \(\langle f\rangle \; P\; \langle g\rangle \) if and only if \(f \le wp(P, g)\). That is, wp(P, g) is the greatest lower bound of preexpectation of P with respect to g. We say wp(P, g) is the weakest preexpectation of P with respect to g.
Example 2
The weakest preexpectation of command \({x{\,}{:}{=}{\,}x+1} \,[p]\,{x{\,}{:}{=}{\,}x1} \) with respect to x is computed below:
It follows that \(\langle x + 2p  1\rangle \; {x{\,}{:}{=}{\,}x+1} \,[p]\,{x{\,}{:}{=}{\,}x1} \; \langle x\rangle \) holds.
4 Quantitative Loop Invariants
Given a preexpectation preE, a postexpectation postE, a Boolean expression G, and a loopfree probabilistic program body, we would like to verify whether
holds or not. One way to solve this problem is to compute the weakest preexpectation \(wp (\mathsf{while}\,(G)\,\{ body \}, postE)\) and check if it is not less than preE pointwisely. However, the weakest preexpectation of a \(\mathsf{while}\)command requires fixed point computation. To avoid the expensive computation, we can solve the problem by finding quantitative loop invariants.
Theorem 1
([15, 24]). Let preE be a preexpectation, postE a postexpectation, G a Boolean expression, and body a loopfree probabilistic program. To show
it suffices to find a loop invariant I which is an expectation such that

1.
(boundary) \(preE\le I\) and \(I \cdot {[}{\lnot G}{]} \le postE\);

2.
(invariant) \(I \cdot {[}{G}{]} \le wp(body, I)\);

3.
(soundness) the loop terminates from any state in G with probability 1, and

(a)
the number of iterations is finite;

(b)
I is bounded above by some fixed constant; or

(c)
the expected value of \(I \cdot {[}{G}{]}\) tends to zero as the loop continues to iterate.

(a)
In this paper, we only focus on checking the boundary and invariant conditions in Theorem 1. One however can show that any polynomial expectation is sound for all examples we consider. In fact, one can establish the soundness of a large class of loop invariants before any specific invariant is found. For instance, it can be shown that any polynomial expectation satisfies the third soundness condition, as long as the probability of exiting the loop is bounded below by a nonzero constant in each iteration. We refer the reader to [24] for more details of sufficient conditions for soundness.
5 Multivariate Lagrange Interpolation
Fix a degree n of quantitative loop invariants and number of variables m. Let \(d = \genfrac(){0.0pt}1{m+n}{n}\). Multivariate Lagrange interpolation is a method to construct an explicit expression for any polynomial in \({\mathbb R}[\mathbf {x}_{m}^{n}]\) by sampling, see e.g., [6, 26, 30]. Given d sampling points \(\mathbf {s}_{1}, \mathbf {s}_{2}, \dots ,\mathbf {s}_{d} \in \mathbb {R}^{m}\), we can compute a Lagrange basis as follows [28]. Let \(\{ b_{1}, b_{2}, \dots , b_{d} \} = \{ x_1^{d_1}x_2^{d_2}\cdots x_m^{d_m} : d_1 + d_2 + \cdots + d_m \le n \}\) be the set of monomials in \({\mathbb R}[\mathbf {x}_{m}^{n}]\). For \(1 \le i \le d\), define
Observe that \(M_i \in {\mathbb R}[\mathbf {x}_{m}^{n}]\) for \(1 \le i \le d\). Moreover, \(M_{i}(\mathbf {s}_{j}) = 0\) for \(i \ne j\) and \(M_{1}(\mathbf {s}_{1}) = M_2 (\mathbf {s}_{2}) = \cdots = M_{d} (\mathbf {s}_{d}) = r\) for some \(r \in {\mathbb R}\). If \(r=0\), then there is a geometrical dependency among the sampling points \(\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}\) [2], and thus no Lagrange basis could be determined from these points. If \(r \ne 0\), define \(B_{i} = M_{i} / r\) for \(1 \le i \le d\). Then \(\mathcal {B} (\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d})=\{B_{i} : 1 \le i \le d \} \subseteq {\mathbb R}[\mathbf {x}_{m}^{n}]\) is called a Lagrange basis of \({\mathbb R}[\mathbf {x}_{m}^{n}]\).
Observe that \(B_{i}(\mathbf {s}_{j}) = {[}{i=j}{]}\) for \(1 \le i, j \le d\). Thus \(\sum _{i=1}^d f (\mathbf {s}_{i})B_i (\mathbf {s}_{j}) = f (\mathbf {s}_{j})\) for \(1 \le j \le d\). Moreover, given any \(f \in {\mathbb R}[\mathbf {x}_{m}^{n}]\), we can write \(f = \sum _{i=1}^d f (\mathbf {s}_{i}) B_i\). Define the Lagrange functional \(\mathcal {L}[\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] : {\mathbb R}^d \rightarrow {\mathbb R}[\mathbf {x}_{m}^{n}]\) by
Then \(f = \mathcal {L} [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (f (\mathbf {s}_{1}), f (\mathbf {s}_{2}), \ldots , f (\mathbf {s}_{d}))\) for any \(f \in {\mathbb R}[\mathbf {x}_{m}^{n}]\). We shall call \(f (\mathbf {s}_{1}),\) \(f(\mathbf {s}_{2}),\) \(\ldots ,\) \(f (\mathbf {s}_{d}) \in {\mathbb R}\) the coefficients for f on basis \(\mathcal {B} (\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d})\).
6 Interpolation of Loop Invariants
Suppose we would like to find a quantitative loop invariant \(I \in {\mathbb R}[\mathbf {x}_{m}^{n}]\) for
where preE is a preexpectation, postE is a postexpectation, G is a Boolean expression, and body is a loopfree probabilistic program. Assume the soundness of I can be verified. We shall use Lagrange interpolation to find I.
Let \(\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d} \in {\mathbb R}^m\) be sampling points that determine a Lagrange basis. If the coefficients \(I (\mathbf {s}_{1}), I (\mathbf {s}_{2}), \ldots , I (\mathbf {s}_{d}) \in {\mathbb R}\) are known, then
by Lagrange interpolation. Our idea therefore is to find the coefficients via constraintsolving. By the boundary and invariant conditions in Theorem 1, we have the following requirements about any loop invariant I:
Example 3
Consider
The following must hold for any loop invariant I
Observe that \(wp ({x{\,}{:}{=}{\,}x+1} \,[0.5]\,{x{\,}{:}{=}{\,}x1};\;{z{\,}{:}{=}{\,}z+1}, I (x, y, z)) = wp ({x{\,}{:}{=}{\,}x+1} \,[0.5]\,{x{\,}{:}{=}{\,}x1}, I (x, y, z + 1)) = 0.5 \cdot wp ({x{\,}{:}{=}{\,}x+1}, I (x, y, z+1)) + 0.5 \cdot wp ({x{\,}{:}{=}{\,}x1}, I (x, y, z+1)) = 0.5 \cdot I (x+1, y, z+1) + 0.5 \cdot I (x1, y, z+1).\)
Requirements (1) can have indicators on both sides of inequality, which is beyond the capability of the solvers we use. We would like to obtain a constraint by removing indicators in two steps. First, we rewrite the expectations to a normal form. An expectation is in disjoint normal form (DNF) if it is of the form \(f = {[}{P_{1}}{]} \cdot f_{1} + \cdots + {[}{P_{k}}{]} \cdot f_{k}\), where \(P_1, P_2, \dots , P_k\) are disjoint, that is, at most one of \(P_1, P_2, \ldots , P_k\) evaluates to true on any valuation.
Theorem 2
([22]). Given an expectation of the form \(f = {[}{P_{1}}{]} \cdot f_{1} + \cdots + {[}{P_{k}}{]} \cdot f_{k}\), f is equivalent to the following expectation in DNF:
where \(K=\{1, 2, \dots , k\}\).
We then transform inequalities between expectations in DNF to constraints.
Theorem 3
([15]). Suppose \(f = {[}{P_1}{]} \cdot f_{1} + \cdots + {[}{P_{k}}{]} \cdot f_{k} \) and \(g = {[}{Q_{1}}{]} \cdot g_{1} + \cdots + {[}{Q_{h}}{]} \cdot g_{h} \) are expectations over \(\mathbf {x}_{m}\) in DNF. \(f \le g\) iff for every \(\mathbf {x}_{m}\)
where \(K=\{1, 2, \dots , k \}\) and \(H=\{1, 2, \dots , h\}\).
Example 4
By Theorems 2 and 3, requirements in Example 3 are equivalent to
for every x, y, z.
We define the loop invariant constraint \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}](c_1, c_2, \ldots , c_d)\) as the constraint transformed from the requirements (1), where the quantitative loop invariant I is replaced by Lagrange functional \(\mathcal {L} [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (c_1, c_2, \ldots , c_d)\).
Example 5
We have the following loop invariant constraint from Example 4.
With loop invariant constraints, it is easy to state our goal. Observe that \(\exists \mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}\, \exists c_1, c_2, \ldots , c_d\, \forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}](c_1, c_2, \ldots , c_d)\) implies the existence of a quantitative loop invariant satisfying the boundary and invariant conditions in Theorem 1. Our strategy hence is to choose sampling points \(\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}\) such that \(\exists c_1, c_2, \ldots , c_d \,\forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}](c_1, c_2, \ldots , c_d)\) holds.
We will choose sampling points to simplify the loop invariant constraint. Recall that sampling points are not unique in Lagrange interpolation. For a loop invariant constraint, we select sampling points so that several coefficients among \(c_1, c_2, \ldots , c_d\) are determined. This helps us to evaluate the quantified loop invariant constraint \(\exists c_1, c_2, \ldots , c_d\, \forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}](c_1, c_2, \ldots , c_d)\).
To evaluate the quantified loop invariant constraint, observe that the Lagrange functional \(\mathcal {L} [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (c_1, c_2, \ldots , c_d)\) is a multivariate polynomial over \(c_1, c_2, \ldots , c_d\) and \(\mathbf {x}_{m}\). A loop invariant constraint is hence nonlinear. However, \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (c_1, c_2, \ldots , c_d) (\mathbf {e})\) is a linear constraint over coefficients for every experiment \(\mathbf {e} \in {\mathbb Z}^m\), i.e., valuation over \(\mathbf {x}_{m}\). We therefore use experiments to construct a series of linear constraints and find coefficients by an SMT solver.
Algorithm 1 shows our toplevel algorithm. The algorithm starts by choosing sampling points (Sect. 6.1). The sampling points are then used to construct the initial linear constraint over coefficients (Sect. 6.2). The while loop evaluates the quantified loop invariant constraint \(\exists c_1, c_2, \ldots , c_d\, \forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}](c_1, c_2,\) \(\ldots , c_d)\). In each iteration, the algorithm selects coefficients \(\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d}\) by a model of the linear constraint obtained from an SMT solver. It then checks whether \(\forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}](\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d})\) is true. The algorithm does this by first trying a number of random experiments (Sect. 6.3). Only after the random experiments are passed, will the algorithm performs universal quantifier elimination to evaluate the quantified constraint (Sect. 6.4). If the random experiments fail, or quantifier elimination does not evaluate to true, our algorithm refines the linear constraint by a counterexample and reiterates (Sect. 6.5).
6.1 Choosing Sampling Points
In Lagrange interpolation, sampling points need be chosen in the first place. We would like to choose sampling points so that as many coefficients are determined as possible. To this end, observe that \(\mathcal {L} [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (c_1, c_2,\) \(\ldots ,\) \(c_d) (\mathbf {s}_{i}) = c_i\) for \(1 \le i \le d\). In other words, \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (c_1, c_2, \ldots , c_d) (\mathbf {s}_{i})\) can be significantly simplified if a sampling point \(\mathbf {s}_{i}\) is used as an experiment. Consider, for instance, the boundary condition in our running example:
If \(\mathbf {s}_{j} = (0, 3, 0)\) is a sampling point, then the condition can be simplified to \(0 \le c_j\) and \(c_j \le 0\). Thus \(c_j\) is determined by choosing (0, 3, 0) as both a sampling point and an experiment.
Ideally, one would choose sampling points so that all coefficients are determined. Unfortunately, such points tend to be geometrically dependent in practice. Thus we cannot expect to establish a Lagrange basis from these points exclusively. Instead, we try to find sampling points which yield a Lagrange basis and determine as many coefficients as possible. We adopt a weighted random search for this purpose. That is, we pick sampling points randomly according to their weights, so that points determining more coefficients are more likely to be picked. If the randomly selected sampling points fail to yield a Lagrange basis, we discard them and select other sampling points randomly again. In our experiments, this heuristic finds pretty good sampling points in reasonable time.
6.2 Initial Constraint
After sampling points are chosen, we compute the initial linear constraint over coefficients. Recall that \(\mathcal {L} [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (c_1, c_2, \ldots , c_d) (\mathbf {s}_{i}) = c_i\) for \(1 \le i \le d\). By taking sampling points as experiments, the loop invariant constraint \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2},\) \(\ldots , \mathbf {s}_{d}] (c_1, c_2, \ldots , c_d)\) is simplified to a linear constraint over \(c_1, c_2, \ldots , c_d\).
Example 6
Consider the loop invariant constraint in Example 5. We first choose 10 sampling points \(\mathbf {s}_{1},\dots ,\mathbf {s}_{10}\) (see table below) to establish a Lagrange basis. We then compute the initial constraints by simplifying the loop invariant constraint with the sampling points. For example, we obtain constraint \(c_2=0\) from point \(\mathbf {s}_{2} = (2, 2, 0)\) as follows:
We list all initial constraints in the following table, where \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2},\ldots , \mathbf {s}_{10}] (c_1,\) \(c_2, \ldots , c_{10})(\mathbf {s}_{i})\) is denoted by \(\psi (\mathbf {s}_{i})\) for simplicity.
i  \(\mathbf {s}_{i}\)  \(\psi (\mathbf {s}_{i})\)  i  \(\mathbf {s}_{i}\)  \(\psi (\mathbf {s}_{i})\)  i  \(\mathbf {s}_{i}\)  \(\psi (\mathbf {s}_{i})\) 

1  0, 3, 0  \(c_{1}=0\)  2  2, 2, 0  \(c_{2}=0\)  3  0, 3, 1  \(0\le c_{3}\le 1\) 
4  1, 1, 0  \(c_{4}=0\)  5  1, 1, 2  \(0\le c_{5}\le 2\)  6  2, 2, 1  \(0\le c_{6}\le 1\) 
7  3, 3, 0  \(c_{7}=0\)  8  0, 0, 1  \(0\le c_{8}\le 1\)  9  0, 1, 0  \(c_{9}=0\) 
10  2, 3, 3  \(6 \le 3c_{10}\le 4c_{1}36c_{2}+5c_{3}+ 30c_{4}+12c_{5}6c_{6}+16c_{7}14c_{8}\) 
Note that our choice of sampling points helps the initial constraints determine 5 coefficients. If a standard monomial basis were used, none of the coefficients could be determined by the initial constraints.
6.3 Random Experiments
From a linear constraint of coefficients, we obtain a model \(\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d}\) of the linear constraint from an SMT solver. Recall that we would like to check if \(\forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d})\) is true. Before using expensive quantifier elimination immediately, we first perform a number of random tests. If \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}]\) \((\hat{c}_{1},\) \(\hat{c}_{2}, \ldots , \hat{c}_{d}) (\mathbf {e})\) evaluates to true for all random experiments \(\mathbf {e} \in {\mathbb Z}^m\), the coefficients \(\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d}\) may induce a loop invariant. Otherwise, a witness experiment \(\mathbf {e}\) is used to refine the linear constraint over coefficients.
When the coefficients do not induce a loop invariant, the random experiments make it possible to avoid expensive quantifier elimination and to obtain a witness experiment without resorting to an SMT solver. This possibility is important, because the solver we use does not always find a valid witness experiment.
6.4 Universal Quantifier Elimination
After random tests, we perform quantifier elimination check if \(\forall \mathbf {x}_{m}.\) \(\phi [\mathbf {s}_{1},\) \(\mathbf {s}_{2}, \ldots ,\) \(\mathbf {s}_{d}] (\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d})\) is true. If so, the polynomial \(\mathcal {L} [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}]\) \((\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d})\) is a quantitative loop invariant satisfying the boundary and invariant conditions. Otherwise, we obtain a witness experiment to refine our linear constraint.
Universal quantifier elimination is carried out in two steps. We first eliminate the quantifiers in the ordered field theory [3, 11]. Intuitively, the ordered field theory formalizes real numbers \({\mathbb R}\). Since quantifier elimination tools such as \(\textsc {Redlog}\) [10] employ algebra and real algebraic geometry, eliminating quantifiers over real numbers is more efficient than over integers. If \(\forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}\), \(\mathbf {s}_{2}, \ldots , \mathbf {s}_{d}]\) \((\hat{c}_{1}\), \(\hat{c}_{2}, \ldots , \hat{c}_{d})\) is true over \({\mathbb R}\), it is also true over \({\mathbb Z}\). Thus \(\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d}\) induces a quantitative loop invariant. Otherwise, we perform quantifier elimination over \({\mathbb Z}\).
If \(\forall \mathbf {x}_{m}.\, \phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] (\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d})\) evaluates to true over \({\mathbb Z}\), we are done. Otherwise, quantifier elimination gives a constraint equivalent to the quantified query. We then use an SMT solver to obtain a witness experiment. We abort the procedure if the solver timesout or fails to yield a valid witness experiment.
6.5 Constraint Refinement
Let \(\mathbf {e} = (\hat{x}_{1}, \hat{x}_{2}, \ldots , \hat{x}_{m}) \in {\mathbb Z}^m\) be a witness experiment such that \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}]\) \((\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d}) (\mathbf {e})\) evaluates to false. Recall that we would like to find coefficients \(c_1, c_2, \ldots , c_d\) such that \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] ({c}_{1}, {c}_{2}, \ldots , {c}_{d})\) is true for every valuations over \(\mathbf {x}_{m}\). Particularly, \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] ({c}_{1}, {c}_{2}, \ldots , {c}_{d}) (\hat{x}_{1}, \hat{x}_{2}, \ldots , \hat{x}_{m})\) must also be true for such coefficients. Note that \(\phi [\mathbf {s}_{1}, \mathbf {s}_{2}, \ldots , \mathbf {s}_{d}] ({c}_{1}, {c}_{2}, \ldots , {c}_{d}) (\hat{x}_{1}, \hat{x}_{2}, \ldots , \hat{x}_{m})\) is a linear constraint on coefficients \(c_1, c_2, \ldots , c_d\) that excludes the incorrect coefficients \(\hat{c}_{1}, \hat{c}_{2}, \ldots , \hat{c}_{d}\). By adding the linear constraint to the current set of constraints, our algorithm will find different coefficients in the next iteration.
7 Applications
We have implemented a prototype in JavaScript to test our techniques. For each simple loop, we manually perform the weakest preexpectation computation and the DNF transformation to translate the requirements (1) into loop invariant constraints. We then use our prototype to find a quantitative loop invariant based on the constraints. Our prototype uses \({\textsc {GNU\,Octave}}\) [12] to compute Lagrange interpolation, \(\textsc {Z3}\) [7] to solve linear constraints, and \(\textsc {Redlog}\) [10] to perform quantifier elimination. The experiments are done on an Intel Xeon 3.07 GHz Linux workstation with 16 GB RAM.
We consider six types of applications: gambler’s ruin problem, geometric distribution, binomial distribution, sum of random series, product of random variables, and simulation of a fair coin. We also consider variants of geometric and binomial distributions. For the faircoin simulation, we find three quantitative loop invariants to prove the correctness and the expected execution time of the simulation. In each probabilistic program, we annotate the while loop with a preexpectation and a postexpectation. Note that the annotated preexpectation is by construction a precise estimate of the annotated postexpectation at the entrance of the loop.
Our results are summarized in Table 1. We use a fixed random seed for all experiments and compute the averages over 100 random runs with a 300 s timeout. The prototype may synthesize different loop invariants in different runs of the same experiment. We now discuss the applications in more details.
Gambler’s Ruin Problem. In Example 1, we consider a game where a player has x dollars initially and plays until he loses all his money or wins up to \(yx\) dollars for some \(y > x\). The expected number of rounds before the game ends is \(E[z]=x\cdot (yx)\). Our prototype proves this result within 3.6 s on average.
Geometric Distribution. The geometric distribution describes the number of tails before the first head in a sequence of cointossing. When the probability of head is 0.25, we expect to see \(\frac{10.25}{0.25} = 3\) tails before the first head. The following program computes a geometrically distributed random variable x:
Our prototype finds a quantitative loop invariant for the preexpectation \(E[x]=3y\) within 3 s on average.
We moreover consider the following variant of the game. A player keeps flipping a coin until the head turns up. He wins k dollars if the tail turns up at the kth flip. The variant is modeled as follows.
The expected amount of money a player can win is \(E[x]=\frac{1}{2}\left( 0.25^{2}1\right) = \frac{15}{2}\). Our prototype proves this result within 8 s on average.
Binomial Distribution. The binomials distribution describes the number of heads that appear in a fixed number of cointossing. If the probability of head is 0.25 and the number of tosses is n, then the expected number of heads is 0.25n. The following program computes a binomially distributed random variable x:
Our prototype proves \(E[x]=0.25ny\) within 4.5 s on average. We moreover consider the following variant. A player flips a coin for n times. At the kth flip, he wins k dollars if the head turns up and wins y dollars otherwise. This game can be modeled as follows.
The expected amount of money a player can win is \(E[x]=0.25\cdot \frac{1}{2}n(n+1)+(10.25)\cdot ny = \frac{1}{8}n^2  \frac{1}{8}n + \frac{3}{4}ny\). Our prototype proves this result within 77.5 s on average.
Sum of Random Series. Consider a game where a player flips a coin for n times. The player wins k dollars if the head turns up at the kth flip. The following program models this game when the head probability of the coin is 0.5:
The expected amount of money the player can win from this game is \(E[x]= 0.5\cdot \sum _{i=1}^n i=0.5\cdot \frac{1}{2}n(n+1)\) dollars. Our prototype proves this result within 2.5 s on average.
Product of Dependent Random Variables. We consider a game where two players flip a coin for n times. The first player wins one dollar for each head and the second player wins one dollars for each tail. When the head probability of the coin is 0.5, this game can be modeled by the following program where variables x, y represent the amount of money won by the respective players:
It can be shown that \(E[xy] = \frac{1}{4}(n^2 n)\). Our prototype proves this result within 15.7 s on average.
Simulation of a Fair Coin. We consider an algorithm that simulates a fair coin flip using biased coins [15]:
The algorithm uses two biased coins x and y with head probability 0.25. The main loop flips the two coins at each iteration and terminates when the coins show different outcomes. The value of x is then taken as the final outcome, with 1 representing the head and 0 representing the tail.
To see that the algorithm indeed simulates a fair coin flip, we prove
where loop denotes the whileloop in the program. Since \(x=y=0\) before the loop starts and \(xy=0\) after the loop stops, we see that \(0.5\le E[1x]\) and \(0.5 \le E[x]\) on termination. Since \(x\in \{0,1\}\), it follows that \(\Pr \{x=1\}=\Pr \{x=0\}=0.5\) on termination and thus the correctness of the algorithm is concluded.
Observe moreover that the number of iterations until the two coins show different outcomes is the geometric distribution with head probability \(0.25 \cdot 2(10.25) = 0.375\). Hence, the expected number of iterations is \(E[n] = \frac{10.375}{0.375}+1=\frac{8}{3}\). This result is verified by our prototype within 18.5 s on average.
8 Evaluation
Our technique is closely related to the \(\textsc {Prinsys}\) tool [15], which implements the constraintbased quantitative invariant synthesis approach developed in [22]. Prinsys receives a probabilistic program and a template with unknown coefficients. It derives loop invariant constraints from the template and exploits SMTsolvers to perform quantifier elimination and simplification for the constraints. The tool generates a formula, which is in effect a conjunction of nonlinear inequalities, describing all coefficients that make the supplied template an inductive loop invariant. A concrete quantitative invariant has to be derived manually by extracting solutions from the formula.
For our prototype, the input is a quantitative Hoare triple and there are three possible outputs: “unknown” (due to timeout or invalid counterexamples), “disproved” with a witness (a valuation of program variables), and “proved” with a proof (a quantitative loop invariant). For \(\textsc {Prinsys}\), it receives a program and a template, and outputs a constraint describing all inductive loop invariants in form of the template. To verify a specific Hoare triple with \(\textsc {Prinsys}\), one has to encode the interested pre and postexpectations as well as the form of possible invariants into the same template. Designing a template for Prinsys is a tricky task that needs to be done on a casebycase basis. In contrast, our technique does not require manually supplied templates, though the degree of loop invariants has to be fixed a priori.
One could use templates to represent nonlinear loop invariants. We nevertheless failed to verify any of our nonlinear examples with Prinsys. In particular, we could not generate formulae that subsume the quantitative loop invariants computed by our prototype. This however does not imply that our examples are beyond the capability of Prinsys, since we could not arguably try all templates manually. The designers of Prinsys also examined their tool on some nonlinear examples, e.g., the gambler’s ruin problem, and reported negative results in [15]. Generally, when the supplied template is nonlinear, it becomes intractable to derive a loop invariant, or even to decide the existence of a loop invariant, from the formula yielded by \(\textsc {Prinsys}\). Maybe a counterexamplerefinement approach is helpful here, but this requires further research and experiments.
9 Conclusion
We propose an automated technique to generate polynomial quantitative invariants for probabilistic programs by Lagrange interpolation. Fixing the degree of loop invariants, our technique can infer polynomial quantitative loop invariants for simple loops. By choosing sampling points carefully, constraints are simplified so that coefficients of loop invariants can be determined. We also develop a counterexampleguided refining heuristics to find coefficients of quantitative loop invariants. We report applications in several case studies.
Our technique does not yet support parameters such as probability in probabilistic choice commands. Such parameters would induce nonlinear constraints over coefficients and parameters. SMT solvers however could not find candidate coefficients and parameters as easily. Also, nondeterminism is not implemented in our prototype. We plan to address both issues in our future work.
References
Chakarov, A., Sankaranarayanan, S.: Expectation invariants for probabilistic program loops as fixed points. In: MüllerOlm, M., Seidl, H. (eds.) Static Analysis. LNCS, vol. 8723, pp. 85–100. Springer, Heidelberg (2014)
Charles, K., HangChin, L.: Vandermonde determinant and Lagrange interpolation in \(\mathbb{R}^s\). In: Nonlinear and Convex Analysis: Proceedings in Honor of Ky Fan, vol. 107, p. 23. CRC Press (1987)
Collins, G.E.: Quantifier elimination for real closed fields by cylindrical algebraic decompostion. In: Automata Theory and Formal Languages 2nd GI Conference Kaiserslautern, 20–23 May 1975, pp. 134–183. Springer (1975)
Colón, M.A., Sankaranarayanan, S., Sipma, H.B.: Linear invariant generation using nonlinear constraint solving. In: Hunt Jr., W.A., Somenzi, F. (eds.) CAV 2003. LNCS, vol. 2725, pp. 420–432. Springer, Heidelberg (2003)
Cousot, P.: Proving program invariance and termination by parametric abstraction, lagrangian relaxation and semidefinite programming. In: Cousot, R. (ed.) VMCAI 2005. LNCS, vol. 3385, pp. 1–24. Springer, Heidelberg (2005)
De Boor, C., Ron, A.: On multivariate polynomial interpolation. Constructive Approximation 6(3), 287–302 (1990)
de Moura, L., Bjørner, N.S.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)
Den Hartog, J., de Vink, E.P.: Verifying probabilistic programs using a hoare like logic. Int. J. Found. Comput. Sci. 13(03), 315–340 (2002)
Dijkstra, E.W.: A Discipline of Programming. Prenticehall, Englewood Cliffs (1976)
Dolzmann, A., Sturm, T.: Redlog: computer algebra meets computer logic. Acm Sigsam Bulletin 31(2), 2–9 (1997)
Dolzmann, A., Sturm, T., Weispfenning, V.: Real quantifier elimination in practice. In: Matzat, B.H., Greuel, G.M., Hiss, G. (eds.) Algorithmic Algebra and Number Theory, pp. 221–247. Springer, Heidelberg (1999)
Eaton, J.W., Bateman, D., Hauberg, S.: GNU Octave. Network thoery, London (1997)
Uslar, M., Specht, M., Rohjans, S., Trefke, J., Gonzalez, J.M.V.: Introduction. In: Uslar, M., Specht, M., Rohjans, S., Trefke, J., Gonzalez, J.M.V. (eds.) The Common Information Model CIM. POWSYS, vol. 2, pp. 3–48. Springer, Heidelberg (2012)
Floyd, R.W.: Assigning meanings to programs. Math. Aspects Comput. Sci. 19(1), 19–32 (1967)
Gretz, F., Katoen, J.P., McIver, A.: Prinsys—on a quest for probabilistic loop invariants. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 193–208. Springer, Heidelberg (2013)
Gretz, F., Katoen, J.P., McIver, A.: Operational versus weakest preexpectation semantics for the probabilistic guarded command language. Perform. Eval. 73, 110–132 (2014)
Gulwani, S., Srivastava, S., Venkatesan, R.: Program analysis as constraint solving. In: ACM SIGPLAN Notices, vol. 43, pp. 281–292. ACM (2008)
Gupta, A., Majumdar, R., Rybalchenko, A.: From tests to proofs. In: Kowalewski, S., Philippou, A. (eds.) TACAS 2009. LNCS, vol. 5505, pp. 262–276. Springer, Heidelberg (2009)
Gupta, A., Rybalchenko, A.: InvGen: an efficient invariant generator. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 634–640. Springer, Heidelberg (2009)
Hoare, C.A.R.: An axiomatic basis for computer programming. CACM 12(10), 576–580 (1969)
Kapur, D.: Automatically generating loop invariants using quantifier eliminationpreliminary report. In: IMACS International Conference on Applications of Computer Algebra. Citeseer (2004)
Katoen, J.P., McIver, A.K., Meinicke, L.A., Morgan, C.C.: Linearinvariant generation for probabilistic programs: In: Cousot, R., Martel, M. (eds.) SAS 2010. LNCS, vol. 6337, pp. 390–406. Springer, Heidelberg (2010)
Kozen, D.: Semantics of probabilistic programs. JCSS 22(3), 328–350 (1981)
McIver, A., Morgan, C.C.: Abstraction, Refinement and Proof for Probabilistic Systems. Springer, Heidelberg (2006)
Morgan, C.: Proof rules for probabilistic loops. In: Proceedings of the BCSFACS 7th Refinement Workshop, Workshops in Computing. Springer Verlag. (1996)
Olver, P.J.: On multivariate interpolation. Stud. Appl. Math. 116(2), 201–240 (2006)
Rabin, M.O.: Probabilistic algorithm for testing primality. J. Number Theory 12(1), 128–138 (1980)
Saniee, K.: A simple expression for multivariate Lagrange interpolation. SIAM Undergraduate Research Online 1(1) (2008)
Sankaranarayanan, S., Sipma, H.B., Manna, Z.: Nonlinear loop invariant generation using Gröbner bases. ACM SIGPLAN Notices 39(1), 318–329 (2004)
Sauer, T., Xu, Y.: On multivariate Lagrange interpolation. Math. Comput. 64(211), 1147–1170 (1995)
Acknowledgments
This work was supported by the Ministry of Science and Technology of Taiwan (1032221E001 019 MY3, 1032221E001 020 MY3) and the Natural Science Foundation of China (NSFC) under grant No. 61472473, 61428208, 61361136002, the CAS/SAFEA International Partnership Program for Creative Research Teams.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Chen, YF., Hong, CD., Wang, BY., Zhang, L. (2015). CounterexampleGuided Polynomial Loop Invariant Generation by Lagrange Interpolation. In: Kroening, D., Păsăreanu, C. (eds) Computer Aided Verification. CAV 2015. Lecture Notes in Computer Science(), vol 9206. Springer, Cham. https://doi.org/10.1007/9783319216904_44
Download citation
DOI: https://doi.org/10.1007/9783319216904_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 9783319216898
Online ISBN: 9783319216904
eBook Packages: Computer ScienceComputer Science (R0)