BitVector Model Counting Using Statistical Estimation
Abstract
Approximate model counting for bitvector SMT formulas (generalizing #SAT) has many applications such as probabilistic inference and quantitative informationflow security, but it is computationally difficult. Adding random parity constraints (XOR streamlining) and then checking satisfiability is an effective approximation technique, but it requires a prior hypothesis about the model count to produce useful results. We propose an approach inspired by statistical estimation to continually refine a probabilistic estimate of the model count for a formula, so that each XORstreamlined query yields as much information as possible. We implement this approach, with an approximate probability model, as a wrapper around an offtheshelf SMT solver or SAT solver. Experimental results show that the implementation is faster than the most similar previous approaches which used simpler refinement strategies. The technique also lets us model count formulas over floatingpoint constraints, which we demonstrate with an application to a vulnerability in differential privacy mechanisms.
Keywords
Model counting Bitvectors Floating point #SAT Randomized algorithms1 Introduction
Model counting is the task of determining the number of satisfying assignments of a given formula. Model counting for Boolean formulas, #SAT, is a standard modelcounting problem, and it is a complete problem for the complexity class #P in the same way that SAT is complete for NP. #P is believed to be a much harder complexity class than NP, and exact #SAT solving is also practically much less scalable than SAT solving. #SAT solving can be implemented as a generalization of the DPLL algorithm [17]. and a number of systems such as Relsat [4], CDP [6], Cachet [38], sharpSAT [41], DSHARP [35] and countAntom [9] have demonstrated various optimization techniques. However, not surprisingly given the problem’s theoretical hardness, such systems often perform poorly when formulas are large and/or have complex constraints.
Since many applications do not depend on the model count being exact, it is natural to consider approximation algorithms that can give an estimate of a model count with a probabilistic range and confidence. Some approximate model counters include ApproxCount [43], SampleCount [24], MiniCount [31], ApproxMC [12], ApproxMCp [30] and ApproxMC2 [13]. In this paper we build on the approximation technique of XOR streamlining [25], which reduces the number of solutions of a formula by adding randomlychosen XOR (parity) constraints. In expectation, adding one constraint reduces the model count by a factor of 2, and k independent constraints reduce the model count by \(2^k\). If a formula with extra constraints has \(n > 0\) solutions, the original formula likely had about \(n\cdot 2^k\). If the model count after constraints is small, it can be found with a few satisfiability queries, so XOR streamlining reduces approximate model counting to satisfiability. However to have an automated system, we need an approach to choose a value of k when the model count is not known in advance.
One application of approximate model counting is measuring the amount of information revealed by computer programs. For a deterministic computation, we say that the influence [36] is the basetwo log of the number of distinct outputs that can be produced by varying the inputs, a measure of the information flow from inputs to outputs. Influence computation is related to model counting, but formulas arising from software are more naturally expressed as SMT (satisfiability modulo theories) formulas over bitvectors than as plain CNF, and one wants to count values only of output variables instead of all variables. The theory of arithmetic and other common operations on boundedsize bitvectors has the same theoretical expressiveness as SAT, since richer operations can be expanded (“bitblasted”) into circuits. But bitvector SMT is much more convenient for expressing the computations performed by software, and SMT solvers incorporate additional optimizations. We build a system for this generalized version of the problem which takes as input an SMT formula with one bitvector variable designated as the output, and a specification of the desired precision.
Our algorithm takes a statistical estimation approach. It maintains a probability distribution that reflects an estimate of possible influence values, using a particle filter consisting of weighted samples from the distribution. Intuitively the mean of the distribution corresponds to our tool’s best estimate, while the standard deviation becomes smaller as its confidence grows. At each step, we refine this estimate by adding k XOR constraints to the input formula, and then enumerating solutions under those constraints, up to a maximum of c solutions (we call this enumeration process an exhaustuptoc query [36]). At a particular step, we choose k and c based on our previous estimate (prior), and then use the query result to update the estimate for the next step (posterior). The update from the query reweights the particle filter points according to a probability model of how many values are excluded by XOR constraints. We use a simple binomialdistribution model which would be exact if each XOR constraint were fully independent. Because this model is not exact, a technique based only on it does not provide probabilistic soundness, even though it performs well practically. So we also give a variant of our technique which does produce a sound bound, at the expense of requiring more queries to meet a given precision goal.
We implement our algorithm in a tool SearchMC that wraps either a bitvector SMT solver compatible with the SMTLIB 2 standard or a SAT solver, and report experimental results. SearchMC can be used to count solutions with respect to a subset of the variables in a formula, such as the outputs of a computation, the capability that Klebanov et al. call projected model counting [30], and Val et al. call subset model counting [42]. In our case the variables not counted need not be of bitvector type. For instance this makes SearchMC to our knowledge the first tool that can be used to count models of constraints over floatingpoint numbers (counting the floatingpoint bit patterns individually, as contrasted with computing the measure of a subset of \(\mathbb {R}^n\) [7, 15]). We demonstrate the use of this capability with an application to a security problem that arises in differential privacy mechanisms because of the limited precision of floatingpoint values.
Compared to ApproxMC2 [13] and ApproxMCp [30], concurrentlydeveloped approximate #SAT tools also based on XOR streamlining, our technique gives results more quickly for the same requested confidence levels.

Our approximate counting approach gives a twosided bound with userspecified confidence.

Our tool inherits the expressiveness and optimizations of SMT solvers.

Our tool gives a probabilistically sound estimate if requested, or can give a result more quickly if empirical precision is sufficient.
2 Background
XOR Streamlining. The main idea of XOR streamlining [25] is to add randomly chosen XOR constraints to a given input formula and feed the augmented formula to a satisfiability solver. One random XOR constraint will reduce the expected number of solutions in half. Consequently, if the formula is still satisfiable after the addition of s XOR constraints, the original formula likely has at least \(2^s\) models. If not, the formula likely has at most \(2^s\) models. Thus we can obtain a lower bound or an upper bound with this approach. There are some crucial parameters to determine the bounds and the probability of the bounds and they need to be carefully chosen in order to obtain good bounds. However, early systems [25] did not provide an algorithm to choose the parameters.
Influence. Newsome et al. [36] introduced the terminology of “influence” for a specific application of model counting in quantitative informationflow measurement. This idea can capture the control of input variables over an output variable and distinguish true attacks and false positives in a scenario of malicious input to a network service. The influence of input variables over an output variable is the \(\log _2\) of the number of possible output values.
Exhaustuptoc Query. Newsome et al. also introduced the terminology of an “exhaustuptoc query”, which repeats a satisfiability query up to some number c of solutions, or until there are no satisfying values left. This is a good approach to find a model count if the number of solution is small.
 1.
Sample a number of particles from a prior distribution.
 2.
Evaluate the importance weights for each particle and normalize the weights.
 3.
Resample particles (with replacement) according to the weights.
 4.
The posterior distribution represented by the resampled particles becomes the prior distribution to next round and go to step 2.
3 Design
This section describes the approach and algorithms used by SearchMC. It is implemented as a wrapper around an offtheself bitvector satisfiability solver that supports the SMTLIB2 format [3]. It takes as input an SMTLIB2 formula in a quantifierfree theory that includes bitvectors (QF_BV, or an extension like QF_AUFBV or QF_FPBV) in which one bitvector is designated as the output, i.e. the bits over which solutions should be counted. (For ease of comparison with #SAT solvers, SearchMC also has a mode that takes a Boolean formula in CNF, with a list of CNF variables designated as the output.) SearchMC repeatedly queries the SMT solver with variations of the supplied input which add XOR constraints and/or “blocking” constraints that exclude previouslyfound solutions; based on the results of these queries, it estimates the total number of values of the output bitvector for which the formula has a satisfying assignment.
SearchMC chooses fruitful queries by keeping a running estimate of possible values of the model count. We model the influence (\(\log _2\) of model count) as if it were a continuous quantity, and represent the estimate as a probability distribution over possible influence values. In each iteration we use the current estimate to choose a query, and then update the estimate based on the query’s results. (At a given update, the most recent previous distribution is called the prior, and the new updated one is called the posterior.) As the algorithm runs, the confidence in the estimate goes up, and the best estimate changes less from query to query as it converges on the correct result. Each counting query SearchMC makes is parameterized by k, the number of random XOR constraints to add, and c, the maximum number of solutions to count. The result of the query is a number of satisfying assignments between 0 and c inclusive, where a result that stops at c means the real total is at least c. Generally a low result leads to the next estimate being lower than the current one and a high result leads to the estimate increasing. We will describe the process of updating the probability distribution, and then give the details of the algorithms that use it.
This probability model lets us simulate the probability of various query results as a function of the unknown formula influence. We use this model as a weighting function for each particle and resample particles based on each particle’s weight value. Then, we estimate a posterior distribution from sampled particles that have all equal weights. For instance, given a prior distribution over the influence sampled at 0.1 bit intervals, we can compute a sampled posterior distribution by counting and renormalizing just the probability weights that correspond to a given query result value n. From the estimated posterior distribution, the mean \(\mu \) and the standard deviation \(\sigma \) are computed. Hence, the \(\mu \) is our best possible answer as our algorithm iterates and \(\sigma \) shows how much we are close to the true answer. Sequentially, the posterior distribution will be the next round’s prior distribution and for use in the very first step of the algorithm we also implement a case of the prior distribution as uniform over influence.
Next we compute a confidence interval (lower bound and upper bound) symmetrically from the mean of the posterior distribution even though the distribution is not likely to be symmetrical. There are several ways to compute the confidence interval but the difference of the results is negligible as the posterior distribution gets narrower. Therefore, we used a simple way to compute the confidence interval: a half interval from the left side of the mean and another half from the right side.
Variables. There are several variables: prior, post, width, k, c, nSat, UB, LB and \(\delta \). prior represents a prior distribution by sampled particles with corresponding weights. In one iteration, we obtain the updated posterior distribution post with resampled particles based on our probabilistic model as described above. The posterior becomes the prior distribution for the next iteration. While our algorithm is in the loop, it keeps updating post. width is the width of the output bitvector of an input formula f, which is an initial upper bound for the influence since the influence cannot be more than the width of the output bitvector. k is a number of random XOR constraints and c specifies the maximum number of solutions for the exhaustuptoc query. We obtain c and k using the ComputeCandK function shown as Algorithm 2 and discussed below. nSat is a number of solutions from the exhaustuptoc query. MBoundExhaustUpToC runs until it finds the model count exactly or c solutions from formula f with k random XOR constraints. UB and LB are variables to store an upper bound and a lower bound of the current model count approximation with a given confidence level as we describe above. \(\delta \) is the distance between the upper bound and lower bound. This parameter determines whether our algorithm terminates or not. If \(\delta \) is less than or equal to our input value thres, our algorithm terminates. If not, it runs again with updated post until \(\delta \) reaches the desired range size thres. An extreme case \(k=0\) denotes that our guess is equivalent to the true model count. In this case, we print out the exact count and terminate the algorithm.
Functions. To motivate the definition of the function ComputeCandK, we view an exhaustuptoc query as analogous to measuring influence with a boundedlength “ruler.” Suppose that we reduce the expected value of the model count by adding k XOR constraints to f. Then, we can use the “length\((\log _2c)\) ruler” to measure the influence starting at k and this measurement corresponds to the result of an exhaustuptoc query: the length\((\log _2c)\) ruler has c markings spaced logarithmically as illustrated in Fig. 1. Each iteration of the algorithm chooses a location (k) and length (c) for the ruler, and gets a noisy reading on the influence as one mark on the ruler. Over time, we want to converge on the true influence value, but we also wish to lengthen the rule so that the finer marks give more precise readings. Based on this idea, we have the ComputeCandK function to choose the length of and starting point of the ruler from a prior distribution. Then, we run an exhaustuptoc query and call Update to update the distribution based on the result of the query.
The k value denotes where to put the ruler. We want to place the ruler where the expected value of the prior distribution lies near the middle of the ruler hence our expected value is in the range of the ruler with high probability. Therefore, we subtract the half length of the ruler (\(\frac{1}{2}\log _2c\)) from the expected value \(\mu \) and then use the floor function to the value because k has to be an nonnegative integer value. The expected value always lies in the righthalf side of the ruler by using the floor function. However, it is not essential which rounding function is used. Note that there might be a case where k becomes negative. If this happens, we set \(k=0\) and \(c=\infty \), because our expected value is so small that we can run the solver exhaustively to give the exact model count. The formula for c is motivated by the intuition that the spacing between two marks near the middle of the ruler should be proportional to the standard deviation of the probability distribution, to ensure that a few different results of the query are possible with relatively high probability; the spacing between the two marks closest to \(\frac{1}{2}\log _2c = \log _2{\sqrt{c}}\) will be about \(\log _2(\sqrt{c}+\frac{1}{2})\log _2(\sqrt{c}\frac{1}{2})\). Setting this equal to \(\sigma \), solving for c, and taking the ceiling gives line 3 of Algorithm 2.
Probabilistic Sound Bounds. The binomial model performs well for choosing a series of queries, and it yields an estimate of the remaining uncertainty in the tool’s results, but because the binomial model differs in an hardtoquantify way from the true probability distributions, the bounds derived from it do not have any associated formal guarantee. In this section we explain how to use our tool’s same query results, together with a sound bounding formula, to compute a probabilistically sound lower and upper bound on the true influence. As a tradeoff, these bounds are usually not as tight as our tool’s primary results.
The idea is based on Theorem 2 from Chakraborty et al.’s work [12], which in turn is a variant on Theorem 5 by Schmidt et al. [39]. For convenience we substitute our own terminology.
Lemma 1
Chakraborty et al. use pivot for what we call c from an exhaustuptoc query and \(\textit{pivot}= 2 \lceil 3e^{1/2}(1+\frac{1}{\epsilon })^2 \rceil \). Since \(0<\epsilon <1\), c (pivot) should be always greater than 40 to make the lemma true with a probability of at least 0.6. (The constant 0.6 comes from \((1  e^{3/2})^2 \approx 0.6035\).)
In SearchMC’s iterations, given c and k, we can compute \(\epsilon \) value to estimate the bounds. Therefore, when c is greater than 40 from our tool’s iteration, we can compute a lower and upper bound such that the true influence is within the bounds with a probability of at least 0.6.
4 Experimental Results
We run our algorithm with a set of DQMR (Deterministic Quick Medical Reference) benchmarks [1] and ISCAS89 benchmarks [8] converted to CNF files by TGPro [14] and compare the results of the benchmarks with ApproxMC2 [13] and ApproxMCp [30]. ApproxMC2 and ApproxMCp are stateoftheart approximate #SAT solvers which we describe in more detail in Sect. 5. We used Cryptominisat2 as the backend solver with all the tools for fair comparison. For the parameters for the tools, we set a 60% confidence level, a confidence level adjustment \(\alpha = 0.25\) and a desired interval length of 1.7. As described above, SearchMCsound gives correct bounds with a probability of at least 0.6. Since the desired confidence level for ApproxMC2 is \(1\delta \), it can achieve a 60% confidence level by setting a parameter \(\delta =0.4\) which corresponds to our parameter \(CL=0.6\). Using the same confidence level for ApproxMCp avoids an apparent mistake in the calculation of its base confidence pointed out by Biondi et al. [5]. The length of the interval for ApproxMC2 is computed as \(\log _2(f\times (1+\epsilon ))\log _2(f\times (1/(1+\epsilon )))=1.7\) hence we can obtain the interval length 1.7 by setting a parameter \(\epsilon =0.8\), corresponding to our parameter \(thres=1.7\). Computing the interval for ApproxMCp is a little different. The length of the interval for ApproxMCp is \(\log _2(f\times (1+\epsilon ))\log _2(f\times (1\epsilon ))=1.7\) hence we can obtain the interval length 1.7 by setting a parameter \(\epsilon =0.53\). Note that SearchMC increases the c value of an exhaustuptoc query as it iterates while the corresponding ApproxMC2 and ApproxMCp parameters are fixed as a function of \(\epsilon \) (72 and 46, respectively) in this experiment. Also, we set an initial prior to be a uniform distribution over 0 to 64 bits for SearchMC. We tested 122 benchmarks (83 DQMRs and 39 ISCAS89s). All the tools were able to solve a set of 106 benchmarks (83 DQMRs and 23 ISCAS89s) within 2 h.
Figure 2a compares the quality of lower bounds and upper bounds computed by SearchMCsound, ApproxMCp and ApproxMC2*^{1}. Note that the benchmarks are arranged in increasing order of the true influence in Fig. 2a and d. The influence bounds are the computed bounds minus the true influence. Filled markers and empty markers represent reported lower bounds and upper bounds, respectively. SearchMCsound, ApproxMCp and ApproxMC2* outperform the requested 60% confidence level. The incorrect bounds are visible as empty markers below the dotted line and filled markers above the line.
SearchMCsound tends to give tighter bounds than the ApproxMC algorithms since it stops when the interval length becomes less than thres, while the interval lengths for the ApproxMCs are fixed by a parameter \(\epsilon \). We do not include the result of SearchMC in this figure to limit clutter, but the full results are available in the longer version of our paper [27]. In brief, SearchMC reported 65 correct bounds out of 106 benchmarks, which is slightly higher than the requested 60% confidence level.
Figure 2b shows another perspective on the tradeoff between performance and error. We selected a single benchmark and varied the parameter settings of each algorithm, measuring the absolute difference between the returned answer and the known exact result. We include results from running ApproxMC2* with parameter settings outside the range of its soundness proofs (shown as “disallowed” in the plot), since these settings are still empirically useful, and SearchMC makes no such distinction. From this perspective the tools are complementary depending on one’s desired performanceerror tradeoff. The results from all the tools improve with configurations that use more queries, but SearchMC performs best at getting more precise results from a small number of queries.
We also compare the runningtime performance with ApproxMCs and show the runningtime performance comparison on our 106 benchmarks in Fig. 2c. In this figure the benchmarks are sorted separately by running time for each tool, which makes each curve nondecreasing; but points with the same x position are not the same benchmark. Since ApproxMCp refined the formulas of ApproxMC, it used a smaller number of queries than ApproxMC2. SearchMC can solve all the benchmarks faster than ApproxMCs with 60% confidence level. SearchMCsound performs faster than ApproxMCp even SearchMCsound computes its confidence interval similarly to ApproxMCp. The SearchMC’s and SearchMCsound’s average running times are 24.59 and 108.24 s, compared to an average of 125.48 for ApproxMCp. ApproxMC2* requires an average of 298.11 s just for the subset of benchmarks all the tools can complete. We also compare the number of SAT queries on the benchmarks for all the tools in Fig. 2d. For this figure the benchmarks are sorted consistently by increasing true model count for all tools. The average number of SAT queries for SearchMC, SearchMCsound ApproxMCp and ApproxMC2* is about 14.7, 83.73, 1256.96 and 733.81 queries, respectively.
Results and performance of model counting (\(\log _2\) shown) of naive Laplacian noise in IEEE floating point
Problem size  All noise  Intersection  

Expected  SearchMC  Time  SearchMC  Time  
15e7, \(2^8\)  7.994  [7.374, 8.069]  164 s  1.000  312 s 
16e7, \(2^9\)  8.997  [8.566, 9.073]  470 s  3.322  1585 s 
16e8, \(2^{10}\)  9.999  [10.076, 10.844]  279 s  4.754  5279 s 
18e8, \(2^{10}\)  9.999  [9.675, 10.099]  583 s  1.000  1137 s 
19e8, \(2^{11}\)  10.999  [10.825, 11.404]  757 s  3.585  9848 s 
To measure this danger using model counting, we translated the standard approach for generating Laplacian noise, including an implementation of the natural logarithm, into SMTLIB 2 floating point and bitvector constraints. (We followed the log function originally by SunSoft taken from the musl C library, which uses integer operations to reduce the argument to \([\sqrt{2}/2, \sqrt{2})\), followed by a polynomial approximation.) A typical implementation might use doubleprecision floats with an 11bit exponent and 53bit fraction, and 32 bits of randomness, which we abbreviate “\(53e11, 2^{32}\)”, but we tried a range of increasing sizes. We measured the total number of distinct values taken by \(10 + \text {noise}\) as well as the size of the intersection of this set with the \(11 + \text {noise}\) set.
The results and running time are shown in Table 1. (For space reasons we omit results for some smaller formats, which can be found in the extended version of the paper [27].) We ran SearchMC with a confidence level of 80%, a confidence level adjustment of 0.5 and a threshold of 1.0; the SMT solver was MathSAT 5.3.13 with settings recommended for floatingpoint constraints by its authors. We use one random bit to choose the sign of the noise, and the rest to choose its magnitude. The sign is irrelevant when the magnitude is 0, so the expected influence for n bits of randomness is \(\log _2(2^n  1)\). SearchMC’s 80% confidence interval included the correct result in 4 out of 5 cases. The size of the intersections is small enough that SearchMC usually reports an exact result (always here). The size of the intersection is also always much less than the total set of noise values, confirming that using this algorithm and parameter setting for privacy protection would be illadvised. The running time increases steeply as the problem size increases, which matches the conventional wisdom that reasoning about floatingpoint is challenging. But because floatingpoint SMT solving is a young area, there is future solvers may significantly improve the technique’s performance.
Description of Archival Artifact. To facilitate reproduction of our experiments and future research, we have created an artifact archive containing code and data for performing the experiments described in this paper. This archive is a zip file containing data, instructions, source code, and binaries precompiled for Ubuntu 14.04 x8664, which we have tested for compatibility with the virtual machine used during the artifact evaluation process [26]. The archive includes SearchMC itself and the modified version of STP it uses for bitblasting, as well as scripts specific to the differentialprivacy experiment, and the benchmark input files we used for performance evaluation. Information about accessing this artifact is found at the end of the paper.
5 Related Work
Exact Model Counting. Some of the earliest Boolean model counters used the DPLL algorithm [17] for counting the exact number of solutions. Birnbaum et al. [6] formalized this idea and introduced an algorithm for counting models of propositional formulas. Based on this idea, Relsat [4], Cachet [38] sharpSAT [43] and DSHARP [35] showed improvements by using several optimizations. The major contribution of countAntom [9] is techniques for parallelization, but it provides stateoftheart performance even in singlethreaded mode.
Phan et al. [37] encode a full binary search for feasible outputs in a bounded model checker. This approach is precise, but requires more than one call to the underlying solver for each feasible output. Klebanov et al. [29] perform exact model counting for quantitative informationflow measurement, with an approach that converts C code to a CNF formula with bounded model checking and then uses exact #SAT solving. Val et al. [42] integrate a symbolic execution tool more closely with a SAT solver by using techniques from SAT solving to prune the symbolic execution search space, and then perform exact model counting restricted to an output variable.
Randomized Approximate Model Counting. Randomized approximate model counting techniques perform well on many kinds of a formula for which finding single solutions is efficient. Wei and Selman [43] introduced ApproxCount which uses nearuniform sampling to estimate the true model count but it can significantly overestimate or underestimate if the sampling is biased. SampleCount [24] improves this sampling idea and gives a lower bound with high probability by using a heuristic sampler. MiniCount [31] computes an upper bound under statistical assumptions by counting branching decisions during SAT solving. MBound [25] is an approximate model counting tool that gives probabilistic bounds on the model counts by adding randomlychosen parity constraints as XOR streamlining. Chakraborty et al. [12] introduced ApproxMC, an approximate model counter for CNF formulas, which automated the choice of XOR streamlining parameters. The ApproxMC algorithm, in our terminology, starts by fixing c and a total number of iterations based on the desired precision and confidence of the results. In each iteration ApproxMC searches for an appropriate k value, adds k XOR random constraints, and then performs an exhaustuptoc query on the streamlined formula and multiplies the result by \(2^k\). It stores all the individual estimates as a multiset and computes its final estimate as the median of the values. The original ApproxMC sequentially increases k in each iteration until it finds an appropriate k value. An improved algorithm ApproxMC2 [13] uses galloping binary search and saves a starting k value between iterations to make the selection of k more efficient. Other recent systems that build on ApproxMC include SMTApproxMC [11] and ApproxMCp [30]. ApproxMCp implements projection (counting over only a subset of variables), which we also require.
ApproxMC2, whose initial development was concurrent with our first work on SearchMC, is the system most similar to SearchMC: its binary search for k plays a similar role to our converging \(\mu \) value. However SearchMC also updates the c parameter over the course of the search, leading to fewer total queries. ApproxMC, ApproxMC2, and related systems choose the parameters of the search at the outset, and make each iteration either fully independent (ApproxMC) or dependent in a very simple way (ApproxMC2) on previous ones. These choices make it easier to prove the tool’s probabilistic results are sound, but they require a conservative choice of parameters. SearchMC’s approach of maintaining a probabilistic estimate at runtime means that its iterations are not at all independent: instead our approach is to extract the maximum guidance for future iterations from previous ones, to allow the search to converge more aggressively.
The runtime performance of SearchMC, like that of ApproxMC(2), is highly dependent on the behavior of SAT solvers on CNFXOR formulas. Some roots of the difficulty of this problem have been investigated by Dudek et al. [20, 21].
Nonrandomized Approximate Model Counting. Nonrandomized approximate model counting using techniques similar to static program analysis is generally faster than randomized approximate model counting techniques, and such systems can give good approximations for some problem classes. However, they cannot provide a precision guarantee for arbitrary problems, and it is not possible to give more effort to have more refined results.
Castro et al. [10] compute an upper bound on the number of bits about an input that are revealed by an error report. Meng and Smith [33] use twobitpattern SMT entailment queries to calculate a propositional overapproximation and count its instances with a model counter from the computer algebra system Mathematica. Luu et al. [32] propose a model counting technique over an expressive string constraint language.
Applications: Security and Privacy. Various applications of model counting have been proposed for security and privacy purposes. Castro et al. [10] use model counting and symbolic execution approaches to measure leaking private information from bug reports. They compute an upper bound on the amount of private information leaked by a bug report and allow users to decide on whether to submit the report or not. Newsome et al. [36] show how an untrusted input affects a program and introduce a family of techniques for measuring influence which can be applicable to x86 binaries. Biondi et al. [5] use CBMC and ApproxMC2 to quantify information flow on a set of benchmarks and evaluate the leakage incurred by a small instance of the Heartbleed OpenSSL bug.
6 Future Work and Conclusion
Closing the gap between the performance of SearchMC and SearchMCsound is one natural direction for future research. On one hand, we would like to explore techniques for asserting sound probabilistic bounds which can take advantage of the results of all of SearchMC’s queries. At the same time, we would like to find a model of the number of solutions remaining after XOR streamlining that is more accurate than our current binomial model, which should improve the performance of SearchMC. Another future direction made possible by the particle filter implementation is to explore different prior distributions, including unbounded ones. For instance, using a negative exponential distribution over influence as a prior would avoid the any need to estimate a maximum influence in advance, while still starting the search process with lowk queries which are typically faster to solve.
In sum, we have presented a new model counting approach SearchMC using XOR streamlining for SMT formulas with bitvectors and other theories. We demonstrate our algorithm that adaptively maintains a probabilistic model count estimate based on the results of queries. Our tool computes a lower bound and an upper bound with a requested confidence level, and yields results more quickly than previous systems.
Footnotes
 1.
ApproxMC2* refers to our own reimplementation of the ApproxMC2 algorithm. With the latest version of ApproxMC2 we encountered problems (which we are still investigating) in which the SAT solver would sometimes fail to perform Gaussian elimination, which unfairly hurt the tool’s performance. Our implementation also makes it easy to control the random seed for experiment repeatability.
Notes
Data Availability Statement and Acknowledgements
An archival snapshot of the tools and datasets analyzed in this work is available in the conference figshare repository at https://doi.org/10.6084/m9.figshare.5928604.v1 [28]. Updates will also be available via the project’s GitHub page at https://github.com/seonmokim/SearchMC. We would like to thank the anonymous conference and artifact reviewers for suggestions which have helped us to improve our system and the paper’s presentation. This research is supported by the National Science Foundation under grant no. 1526319.
References
 1.Bayesianinference as modelcounting benchmarks. http://www.cs.rochester.edu/users/faculty/kautz/Cachet/Model_Counting_Benchmarks/index.htm
 2.
 3.Barrett, C., Stump, A., Tinelli, C.: The SMTLIB standard: Version 2.0. Technical report (2010)Google Scholar
 4.Bayardo Jr., R.J., Schrag, R.C.: Using CSP lookback techniques to solve realworld SAT instances. In: Proceedings of AAAI, pp. 203–208 (1997)Google Scholar
 5.Biondi, F., Enescu, M.A., Heuser, A., Legay, A., Meel, K.S., Quilbeuf, J.: Scalable approximation of quantitative information flow in programs. Verification, Model Checking, and Abstract Interpretation. LNCS, vol. 10747, pp. 71–93. Springer, Cham (2018). https://doi.org/10.1007/9783319737218_4CrossRefGoogle Scholar
 6.Birnbaum, E., Lozinskii, E.L.: The good old DavisPutnam procedure helps counting models. J. Artif. Intell. Res. (JAIR) 10, 457–477 (1999)MathSciNetMATHGoogle Scholar
 7.Borges, M., Filieri, A., d’Amorim, M., Pasareanu, C.S., Visser, W.: Compositional solution space quantification for probabilistic software analysis. In: Proceedings of PLDI, pp. 123–132 (2014)CrossRefGoogle Scholar
 8.Brglez, F., Bryan, D., Kozminski, K.: Combinational profiles of sequential benchmark circuits. In: Proceedings of ISCAS, vol. 3, pp. 1929–1934 (1989)Google Scholar
 9.Burchard, J., Schubert, T., Becker, B.: LaissezFaire caching for parallel #SAT solving. In: Heule, M., Weaver, S. (eds.) SAT 2015. LNCS, vol. 9340, pp. 46–61. Springer, Cham (2015). https://doi.org/10.1007/9783319243184_5CrossRefMATHGoogle Scholar
 10.Castro, M., Costa, M., Martin, J.P.: Better bug reporting with better privacy. In: Proceedings of ASPLOS, pp. 319–328 (2008)CrossRefGoogle Scholar
 11.Chakraborty, S., Meel, K.S., Mistry, R., Vardi, M.Y.: Approximate probabilistic inference via wordlevel counting. In: Proceedings of AAAI, pp. 3218–3224 (2016)Google Scholar
 12.Chakraborty, S., Meel, K.S., Vardi, M.Y.: A scalable approximate model counter. In: Schulte, C. (ed.) CP 2013. LNCS, vol. 8124, pp. 200–216. Springer, Heidelberg (2013). https://doi.org/10.1007/9783642406270_18CrossRefGoogle Scholar
 13.Chakraborty, S., Meel, K.S., Vardi, M.Y.: Improving approximate counting for probabilistic inference: from linear to logarithmic SAT solver calls. In: Proceedings of IJCAI, pp. 3569–3576 (2016)Google Scholar
 14.Chen, H., MarquesSilva, J.: TGPro: a SATbased ATPG system. J. Satisf. Boolean Model. Comput. 8(1–2), 83–88 (2012)MathSciNetMATHGoogle Scholar
 15.Chistikov, D., Dimitrova, R., Majumdar, R.: Approximate counting in SMT and value estimation for probabilistic programs. In: Proceedings of TACAS, pp. 320–334 (2015)Google Scholar
 16.Cimatti, A., Griggio, A., Schaafsma, B.J., Sebastiani, R.: The MathSAT5 SMT solver. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013. LNCS, vol. 7795, pp. 93–107. Springer, Heidelberg (2013). https://doi.org/10.1007/9783642367427_7CrossRefMATHGoogle Scholar
 17.Davis, M., Logemann, G., Loveland, D.: A machine program for theoremproving. Commun. ACM 5(7), 394–397 (1962)MathSciNetCrossRefGoogle Scholar
 18.de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/9783540788003_24CrossRefGoogle Scholar
 19.Del Moral, P.: Nonlinear filtering: interacting particle solution. Markov Process. Relat. Fields 2(4), 555–580 (1996)MathSciNetMATHGoogle Scholar
 20.Dudek, J., Meel, K.S., Vardi, M.Y.: Combining the kCNF and XOR phasetransitions. In: Proceedings of IJCAI, pp. 727–734 (2016)Google Scholar
 21.Dudek, J., Meel, K.S., Vardi, M.Y.: The hard problems are almost everywhere for random CNFXOR formulas. In: Proceedings of IJCAI, pp. 600–606 (2017)Google Scholar
 22.Dwork, C.: Differential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006). https://doi.org/10.1007/11787006_1CrossRefGoogle Scholar
 23.Ganesh, V., Dill, D.L.: A decision procedure for bitvectors and arrays. In: Damm, W., Hermanns, H. (eds.) CAV 2007. LNCS, vol. 4590, pp. 519–531. Springer, Heidelberg (2007). https://doi.org/10.1007/9783540733683_52CrossRefGoogle Scholar
 24.Gomes, C.P., Hoffmann, J., Sabharwal, A., Selman, B.: From sampling to model counting. In: Proceedings of IJCAI, pp. 2293–2299 (2007)Google Scholar
 25.Gomes, C.P., Sabharwal, A., Selman, B.: Model counting: a new strategy for obtaining good bounds. In: Proceedings of AAAI, pp. 54–61 (2006)Google Scholar
 26.Hartmanns, A., Wendler, P.: figshare (2018). https://doi.org/10.6084/m9.figshare.5896615
 27.Kim, S., McCamant, S.: Bitvector model counting using statistical estimation. CoRR, abs/1712.07770 (2017)Google Scholar
 28.Kim, S., McCamant, S.: SearchMC: an approximate model counter using XOR streamlining techniques. figshare (2018). https://doi.org/10.6084/m9.figshare.5928604.v1
 29.Klebanov, V., Manthey, N., Muise, C.: SATbased analysis and quantification of information flow in programs. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 177–192. Springer, Heidelberg (2013). https://doi.org/10.1007/9783642401961_16CrossRefGoogle Scholar
 30.Klebanov, V., Weigl, A., Weisbarth, J.: Sound probabilistic #SAT with projection. In: Workshop on QAPL (2016)MathSciNetCrossRefGoogle Scholar
 31.Kroc, L., Sabharwal, A., Selman, B.: Leveraging belief propagation, backtrack search, and statistics for model counting. In: Perron, L., Trick, M.A. (eds.) CPAIOR 2008. LNCS, vol. 5015, pp. 127–141. Springer, Heidelberg (2008). https://doi.org/10.1007/9783540681557_12CrossRefMATHGoogle Scholar
 32.Luu, L., Shinde, S., Saxena, P., Demsky, B.: A model counter for constraints over unbounded strings. In: Proceedings of PLDI, pp. 565–576 (2014)CrossRefGoogle Scholar
 33.Meng, Z., Smith, G.: Calculating bounds on information leakage using twobit patterns. In: Proceedings of PLAS, pp. 1:1–1:12 (2011)Google Scholar
 34.Mironov, I.: On significance of the least significant bits for differential privacy. In: Proceedings of CCS, pp. 650–661 (2012)Google Scholar
 35.Muise, C., McIlraith, S.A., Beck, J.C., Hsu, E.I.: Dsharp: fast dDNNF compilation with sharpSAT. In: Kosseim, L., Inkpen, D. (eds.) AI 2012. LNCS (LNAI), vol. 7310, pp. 356–361. Springer, Heidelberg (2012). https://doi.org/10.1007/9783642303531_36CrossRefGoogle Scholar
 36.Newsome, J., McCamant, S., Song, D.: Measuring channel capacity to distinguish undue influence. In: Proceedings of PLAS, pp. 73–85 (2009)Google Scholar
 37.Phan, Q., Malacaria, P., Tkachuk, O., Păsăreanu, C.S.: Symbolic quantitative information flow. SIGSOFT Softw. Eng. Notes 37(6), 1–5 (2012)CrossRefGoogle Scholar
 38.Sang, T., Bacchus, F., Beame, P., Kautz, H.A., Pitassi, T.: Combining component caching and clause learning for effective model counting. In: Proceedings of SAT (2004)Google Scholar
 39.Schmidt, J.P., Siegel, A., Srinivasan, A.: Chernoffhoeffding bounds for applications with limited independence. SIAM J. Discret. Math. 8(2), 223–250 (1995)MathSciNetCrossRefGoogle Scholar
 40.Soos, M., Nohl, K., Castelluccia, C.: Extending SAT solvers to cryptographic problems. In: Kullmann, O. (ed.) SAT 2009. LNCS, vol. 5584, pp. 244–257. Springer, Heidelberg (2009). https://doi.org/10.1007/9783642027772_24CrossRefGoogle Scholar
 41.Thurley, M.: sharpSAT – counting models with advanced component caching and implicit BCP. In: Biere, A., Gomes, C.P. (eds.) SAT 2006. LNCS, vol. 4121, pp. 424–429. Springer, Heidelberg (2006). https://doi.org/10.1007/11814948_38CrossRefGoogle Scholar
 42.Val, C.G., Enescu, M.A., Bayless, S., Aiello, W., Hu, A.J.: Precisely measuring quantitative information flow: 10K lines of code and beyond. In: Proceeding of Euro S&P, pp. 31–46 (2016)Google Scholar
 43.Wei, W., Selman, B.: A new approach to model counting. In: Bacchus, F., Walsh, T. (eds.) SAT 2005. LNCS, vol. 3569, pp. 324–339. Springer, Heidelberg (2005). https://doi.org/10.1007/11499107_24CrossRefGoogle Scholar
Copyright information
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.