An SMT Solver for Regular Expressions and Linear Arithmetic over String Length

We present a novel length-aware solving algorithm for the quantifier-free first-order theory over regex membership predicate and linear arithmetic over string length. We implement and evaluate this algorithm and related heuristics in the Z3 theorem prover. A crucial insight that underpins our algorithm is that real-world instances contain a wealth of information about upper and lower bounds on lengths of strings under constraints, and such information can be used very effectively to simplify operations on automata representing regular expressions. Additionally, we present a number of novel general heuristics, such as the prefix/suffix method, that can be used in conjunction with a variety of regex solving algorithms, making them more efficient. We showcase the power of our algorithm and heuristics via an extensive empirical evaluation over a large and diverse benchmark of 57256 regex-heavy instances, almost 75% of which are derived from industrial applications or contributed by other solver developers. Our solver outperforms five other state-of-the-art string solvers, namely, CVC4, OSTRICH, Z3seq, Z3str3, and Z3-Trau, over this benchmark, in particular achieving a 2.4x speedup over CVC4, 4.4x speedup over Z3seq, 6.4x speedup over Z3-Trau, 9.1x speedup over Z3str3, and 13x speedup over OSTRICH.


Introduction
Satisfiability Modulo Theories (SMT) solvers that support theories over regular expression (regex) membership predicate and linear arithmetic over length of strings, such as CVC4 [22], Z3str3 [7], Norn [3], S3P [36], and HAMPI [19], have enabled many important applications in the context of analysis of stringintensive programs.Examples include symbolic execution and path analysis [9,29], as well as security analyzers that make use of string and regex constraints for input sanitization and validation [5,30,32].Regular expression libraries in programming languages provide very intuitive and popular ways for developers to express input validation, sanitization, or pattern matching constraints.Common to all these applications is the requirement for a rich quantifier-free (QF) firstorder theory over strings, regexes, and integer arithmetic.Unfortunately, the QF first-order theory of strings containing regex constraints, linear integer arithmetic on string length, string-number conversion, and string concatenation (but no string equations 6 ) is undecidable (see Appendix).It can also be shown that many non-trivial fragments of this theory are hard to decide (e.g., they have exponential-space lower bounds or are PSPACE-complete).Therefore, the task of creating efficient solvers to handle practical string constraints that belong to fragments of this theory remains a very difficult challenge.
Many modern solvers typically handle regex constraints via an automatabased approach [4].Automata-based methods are powerful and intuitive, but solvers must handle two key practical challenges in this setting.The first challenge is that many automata operations, such as intersection, are computationally expensive, yet handling these operations is required in order to solve constraints that are relevant to real-world applications.The second challenge relates to the integration of length information with regex constraints.Length constraints derived from automata may imply a disjunction of linear constraints, which is often more challenging for solvers to handle than a conjunction.
As we demonstrate in this paper, the challenges of using automata-based methods can be addressed via prudent use of lazy extraction of implied length constraints and lazy regex heuristics in order to avoid performing expensive automata operations when possible.Inspired by this observation, we introduce a length-aware automata-based algorithm, Z3str3RE (and its implementation as part of the Z3 theorem prover [16]), for solving regex constraints and linear integer arithmetic over length of string terms.Z3str3RE takes advantage of the compactness of automata in representing regular expressions, while at the same time mitigating the effects of expensive automata operations such as intersection by leveraging length information and lazy heuristics.Contributions: We make the following contributions in this paper.Z3str3RE: An SMT Solver for Regular Expressions and Linear Integer Arithmetic over String Length.In Section 3, we present a novel decision procedure for the QF first-order theory over regex membership predicate and linear integer arithmetic over string length.We also describe its implementation, Z3str3RE, as part of the Z3 theorem prover [7,16].The basic idea of our algorithm is that formulas obtained from practical applications have many implicit and explicit length constraints that can be used to reason efficiently about automata representing regexes.In Section 4 we present four heuristics that aid in solving regular expression constraints and that can be leveraged in general settings.Specifically, we present a heuristic to derive explicit length information directly from regexes, a heuristic to perform expensive automata operations lazily, a heuristic to refine lower and upper bounds on lengths of string terms with respect to regex constraints, and a prefix/suffix over-approximation heuristic to find empty intersections without constructing automata.All heuristics are designed to guide the search and avoid expensive automata operations whenever possible.Our solver, Z3str3RE, handles the above theory as well as extensions (e.g.word equations and substring function) via the existing support in Z3str3.We focus on the core algorithm as it is the centerpiece of our regex solver.We also carefully distinguish the novelty of our method from previous work.Empirical Evaluation and Comparison of Z3str3RE7 against CVC4, OSTRICH, Z3seq, Z3str3, and Z3-Trau: To validate the practical efficacy of our algorithm, we present a thorough evaluation of Z3str3RE in Section 5 and compare it against CVC4 [21], OSTRICH [13], Z3's sequence solver [16], Z3str3 [39], and Z3-Trau [1] on 57256 instances across four regex-heavy benchmarks with connections to industrial security applications, including instances from Amazon Web Services and AutomatArk [14].Z3str3RE significantly outperforms other state-of-the-art tools on the benchmarks considered, having more correctly solved instances in total, lower running time, and fewer combined timeouts/unknowns than other tools, and no soundness errors or crashes.We note that almost 75% of the benchmarks were obtained from industrial applications or other solver developers.Over all the benchmarks, we demonstrate a 2.4x speedup over CVC4, 4.4x speedup over Z3seq, 6.4x speedup over Z3-Trau, 9.1x speedup over Z3str3, and 13x speedup over OSTRICH.

Preliminaries
This section contains some basic definitions as well as a brief overview of the theoretical results which shape the landscape in which we state our contribution.

Basic Definitions
We first describe the syntax and semantics of the input language supported by our solver Z3str3RE (Algorithm 1).Syntax: The core algorithm we present in Section 3 accepts formulas of the quantifier-free many-sorted first-order theory of regex membership predicates over strings and linear integer arithmetic over string length function.The syntax of this theory is shown in Figure 1.
We denote the set of all string variables and all integer variables as Var str and Var int respectively, and the set of all string constants and all integer constants as Con str and Con int respectively.String constants are any sequence of zero or more characters over a finite alphabet (e.g., ASCII).
Atomic formulas are regular expression membership constraints and linear integer (in)equalities.Regex terms are denoted recursively over regex concatenation, union, Kleene star, and complement, and for a string constant w, the regex term "w" represents the regular language containing w only.All regex terms must be grounded (i.e.cannot contain variables).Linear integer arithmetic terms include integer constants and variables, addition, and string length.Multiplication by a constant is expanded to repeated addition.String terms are either string variables or string constants.The length of a string S is denoted by len(S), the number of characters in S. The empty string has length 0.
Our implementation Z3str3RE supports the theory in Figure 1 extended with more expressive functions and predicates, including word equations (equality between arbitrary string terms) and functions such as indexof and substr that are needed for program analysis.Z3str3RE handles these terms via existing support in Z3str3.We focus on the above input language in the presentation of our algorithm in this paper and theoretical content.

Semantics:
We refer the reader to [39] for a detailed description of the semantics of standard terms in this theory.We focus here on the semantics of terms which are less commonly known.The regex membership predicate S ∈ R, where S is a string term and R is a regex term, is defined by structural recursion as follows: S ∈ "w" iff S = w (where w is a string constant) iff either S = ǫ or there exists a positive integer n such that

Theoretical Landscape
To put our contributions in context, we briefly discuss a series of (un)decidability and complexity results developed around the fragments and extensions of the theory supported by Z3str3RE.
In particular, we consider extensions which may have a string-number conversion predicate numstr 8 and/or string concatenation.Both extensions are important to real-world program analysis.The predicate numstr has the syntax numstr(t int , t str ) and the following semantics: numstr(n, s) is true for a given integer n and string s iff s is a valid binary representation of the number n (possibly with leading zeros) and n is a non-negative integer.That is, s only contains the characters 0 and 1, and len(s)−1 i=0 s ′ [i]2 len(s)−i−1 = n, where s ′ [i] is 0 if the ith character in s is '0' and 1 if that character is '1'.String concatenation has the syntax t str ::= t str • t str and the usual semantics defined by SMT-LIB [8].
In the following, T LRE,n,c is the quantifier-free many-sorted first-order theory of linear integer arithmetic over string length function (L), regex (RE) membership predicates, string-number conversion (n), and string concatenation (c) 9 .The following quantifier-free fragments of T LRE,n,c are of interest: T LRE,c , T LRE , T RE,n,c , T RE,n , and T RE .The fragment T LRE,c (respectively, T LRE ) has all functions and predicates of T LRE,n,c except the string-number conversion predicate (and, respectively, except the string concatenation function).
The theory T RE,n,c (respectively, T RE,n and T RE ) has all functions and predicates of T LRE,n,c except the length function (and, respectively, the string concatenation function, and, in the case of T RE , the string-number conversion predicate).Note that while all these theories allow equalities between terms of sort Int, they do not allow equalities between terms of sort Str, and therefore can not express general word equations.
The theoretical landscape is laid out as follows.Firstly, following the results and techniques introduced in [3], we obtain that T LRE,c and, in particular, T LRE is decidable.A procedure deciding a formula from T LRE,c would first construct for each variable (string or integer), based on the regular expression constraints and length constraints which involve it, a finite automaton, then reduce the problem of checking the satisfiability of the formula to checking whether the constructed automata accept at least one string.A similar approach shows that T RE,n is decidable.
We observe that the presence of complements in regular expressions is an inherent source of complexity for these procedures.Indeed, we can easily encode the universality problem for regular expressions as a formula in the theory T RE .Moreover, given a regex R of length n over an alphabet Σ, deciding whether L(R) = Σ * is equivalent to deciding the satisfiability of the formula ϕ of T RE consisting of the atoms x ∈ R and x ∈ Σ * .Accordingly, by the results from [34], if the choice for R is restricted to regular expressions with at least k stacked complements, then there exists a positive rational number c such that the con- In other words, the depth of the stack of complements of the formula translates to the height of the tower of exponents in the complexity of deciding that formula ϕ.On the other hand, if we only consider regular expressions without stacked complements, then the decision problems for the considered theories are PSPACE-complete.Indeed, the automata-based approach described above can be implemented to work in nondeterministic polynomial space; strongly related complexity results are obtained in [24,23].
At the opposite end of the spectrum is the theory T LRE,n,c , which is undecidable.Indeed, one can show that the more specific theory T RE,n,c (i.e.disallowing arithmetic over length) has equivalent expressive power to the theory of word equations with regular constraints, a predicate allowing the comparison of the length of string terms, and the numstr predicate.Therefore, using the techniques from [15], one can show that the theory T LRE,n,c , in which we additionally allow arithmetic over length, is undecidable.(See Theorem 1 in the Appendix.) The inherent extremely high complexity of the satisfiability problems from T RE and of the theories extending it, as well as the undecidability of the respective problem for T LRE,n,c , suggest that the usage of heuristics will decisively influence any algorithmic approaches to solving these problems in practice.

Length-Aware Regular Expression Algorithm
This section outlines the high-level algorithm used by Z3str3RE to solve the satisfiability problem for T LRE , and its extension based on length-aware heuristics.

High-Level Algorithm
The pseudocode presented in Algorithm 1 is shown at a high level that captures the essence of the procedure being performed.Implementation-specific details are omitted for clarity.Z3str3RE incorporates a version of this algorithm as part of a DPLL(T)-style interaction with a core solver for Boolean combinations of atoms and other theory solvers able to handle arithmetic constraints and other terms.The tool handles string concatenation, string equality, and other string terms and predicates besides regex membership and string length via existing support in Z3str3, and leverages Z3's integer arithmetic solver for arithmetic reasoning and model construction.This high-level presentation is expanded in Section 4, where we describe several heuristics used in our implementation as part of the Z3str3RE tool.
The algorithm takes as input a conjunction φ of regex membership constraints and a conjunction ψ of linear integer arithmetic constraints over the lengths of string variables appearing in φ.Without loss of generality, it is assumed that all constraints in φ are positive; negative constraints S ∈ RE can be replaced with the positive complement S ∈ RE.The algorithm returns SAT if there is a satisfying assignment to all string variables consistent with the regex constraints φ and length constraints ψ.It is assumed that the algorithm has access to a procedure for checking the consistency of linear integer arithmetic constraints and for obtaining satisfying assignments to these constraints (in our implementation, this is fulfilled by Z3's arithmetic solver).
Lines 1-8 check whether the length information implied by φ is consistent with ψ.The function ComputeLengthAbstraction takes as input either a string term S or a regex RE and computes a system of length constraints corresponding to either an abstraction of derived length information from string constraints or an abstraction of length information derived from the regex RE.For example, given the regex (abc) * as input, ComputeLengthAbstraction would construct the length abstraction S ∈ (abc) * → len(S) = 3n, n ≥ 0 for a fresh integer variable n.If the length abstractions are inconsistent with the given length constraints, there can be no solution which satisfies both the length and regex constraints, and hence the algorithm returns UNSAT.Otherwise, line 7 refines the length abstraction L S with respect to the regex RE.This improves the efficiency of finding solutions to the augmented system of length constraints later in the algorithm.In our implementation, the lower and upper bounds of the length of S are checked against the lengths of accepting paths in the automaton for RE.
For instance, if L S implies that len(S) ≥ 5, but the shortest accepting path in the automaton has length 7, the lower bound is refined to len(S) ≥ 7.
Lines 9-15 check that the intersection of all automata constraining each string variable is non-empty.Although intersecting automata is relatively expensive (as it runs in quadratic time w.r.t. the size of the intersected automata), it is still more efficient to do this before enumerating length assignments, and taking the intersection here is necessary to maintain soundness.(The heuristics in Section 4 illustrate some methods by which this computation can be made more efficient or even avoided.)At this point in the algorithm, the length constraints ψ (as well as our length abstractions) are consistent, and the regex constraints φ are consistent, so we can check the joint consistency of φ ∧ ψ.The algorithm enumerates each possible model M of our augmented system of length constraints (line 18), obtaining a candidate assignment for the length of each string in φ.It is then necessary to check whether solutions of this length actually exist for each regex constraint in φ.The algorithm does this by enumerating all accepting paths in the corresponding automata having that respective length.Line 23 checks the characterconsistency of each accepting path of length len(S) in each automaton A. Here, by "character-consistent", we mean that some accepting path in each automaton follows transitions corresponding to the same characters of S. This can be checked in various ways, as the total number of paths, as well as their size, is always finite.For example, in our implementation of this algorithm in Z3str3RE, the solver converts each path to a disjunction of bit-vector character constraints and checks the satisfiability of the resulting system.If each string S is mapped to a path through the automata corresponding to its regex constraints, of a length that is consistent with the arithmetic constraints, then the system is satisfiable and the algorithm returns SAT.Otherwise, this process repeats for the next length assignment M .Line 33 is reached if and only if no solution of the combined system of length constraints has a satisfying assignment over string variables with that length.If this happens, the constraints φ and ψ are not jointly satisfiable and the algorithm returns UNSAT.
We demonstrate soundness, completeness, and termination of Algorithm 1 as follows.On line 4 we check whether ψ ∪ L S ∪ L RE is satisfiable.If not, we return UNSAT on line 5. Therefore, if control reaches line 18, ψ ∪ L S ∪ L RE must be satisfiable.By construction of L RE , every solution to φ is also a solution to L RE and vice versa (all solutions are preserved).It follows that satisfiability of ψ∪L S ∪L RE at line 18 implies the existence of a smallest valid string solution that satisfies all length constraints.The loop body (lines 19-31) finds and outputs this smallest valid string solution.Therefore, Algorithm 1 is a decision procedure for the QF first-order theory of regex constraints, string length, and linear integer arithmetic.
The conditionals inside the loop body (lines [19][20][21][22][23][24][25][26][27][28][29][30][31] suggest that we may discover unsatisfiability.As previously mentioned, Z3str3RE supports other highlevel operations that are not part of this theory via existing support in Z3str3.These conditionals provide support for including these operations, which may render the theory undecidable.These terms are not in Algorithm 1 because their inclusion would make the algorithm incomplete (see Section 2.2).Algorithm 1 describes the part of the implementation which is novel and complete.

Length-Aware and Prefix/Suffix Heuristics in Z3str3RE
In this section, we describe the length-aware heuristics that are used in Z3str3RE to improve the efficiency of regular expression reasoning.We present an empirical evaluation of the power of these heuristics in Section 5.6.

Computing Length Information from Regexes
The first length-aware heuristic we use is when constructing the length abstraction on line 3.If the regex can be easily converted to a system of equations describing the lengths of all possible solutions (for instance, in the case when it does not contain any complements or intersections), this system can be returned as the abstraction without constructing the automaton for RE yet.As previously illustrated, for example, given the regex (abc) * as input, ComputeLengthAbstraction would construct the length abstraction S ∈ (abc) * → len(S) = 3n, n ≥ 0 for a fresh integer variable n.Note that this can be done from the syntax of the regex without converting it to an automaton.Deriving length information from the automaton would be simple by, for example, constructing a corresponding unary automaton and converting to Chrobak normal form.However, performing automata construction lazily means we cannot rely on having an automaton in all cases; this technique also provides length information even when constructing an automaton would be expensive.
In cases where we cannot directly infer the length abstraction, the heuristic will fix a lower bound on the length of words in RE, and possibly an upper bound if it exists.Reasoning about the length abstraction early in the procedure gives our algorithm the opportunity to detect inconsistencies before expensive automaton operations are performed.This gives the arithmetic solver more opportunities to propagate facts discovered by refinement and potentially more chances to find inconsistencies or learn further derived facts.

Optimizing Automata Operations via Length Information
Similarly, computing the intersection I in line 11 is done lazily in the implementation of Z3str3RE and over several iterations of the algorithm.The most expensive intersection operations can be performed at the end of the search, after as much other information as possible has been learned.We use the following heuristics recursively to estimate the "cost" of each operation without actually constructing any automata: -For a string constant, the estimated cost is the length of the string.
-For a concatenation or a union of two regex terms X and Y , the estimated cost is the sum of the estimates for X and Y .-For a regex term X * , the estimated cost is twice the estimate for X.
-For a regex term X under complement, the estimated cost is the product of the estimates obtained from subterms of X.
In essence, the constructions which "blow up" the least are expected to be the least expensive and are performed first.In the best-case scenario, this could mean avoiding the most expensive operations completely if an intersection of smaller automata ends up being empty.In the worst case, all intersections are computed eventually, as this is necessary to maintain the soundness of our approach.

Leveraging Length Information to Optimize Search
Our implementation communicates integer assignments and lower/upper bounds with the external arithmetic solver in order to prune the search space.The search for length assignments on line 18 is done in practice as an abstractionrefinement loop involving Z3's arithmetic solver.The arithmetic solver proposes a single candidate model for the system of arithmetic constraints; the regex algorithm checks whether that model has a corresponding solution over the regex constraints.If it does not, it asserts a conflict clause blocking that combination of length assignments and regex constraints from being considered again.This is necessary in a DPLL(T)-style solver such as Z3 in order to handle Boolean structure in the input formula.

Constructing Over-Approximated Prefixes/Suffixes to Find Empty Intersections
As previously mentioned, computing automata intersections is expensive, but in many cases it is necessary in order to prove that a set of intersecting regex constraints has no solution.In some cases, this can be done "by inspection" from the syntax of the regex terms without constructing or intersecting any automata.
From the structure of a regular expression, it is easy to determine the first letter of all possible accepted strings that it matches.If several regexes would be intersected over the same string term, this is used to check whether these regexes have a prefix of length one in common.If they do not, their intersection cannot contain any strings other than the empty string (and we can also check whether the empty string could be accepted by a similar syntactic approach).A similar construction for suffixes of length 1 is also used.In this way, the heuristic can infer that the intersection of several regex constraints is either empty, resulting in a conflict clause, or can only contain the empty string, resulting in a new fact and a simplification of the formula -without actually constructing the intersection or, in fact, constructing any automata for these regexes.For example, consider the following regex constraints on a variable X: The prefix/suffix heuristic would proceed as follows.In the first constraint, the pattern abc is matched zero or more times, and could be empty; therefore, either X is empty or it must start with a and end with c.In the second constraint, each pattern is matched at least once, and cannot be empty; therefore X must 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000 55,000 60,000 0 5,000 start with a or b, end with a or b, and cannot be the empty string.Observe that according to the prefix heuristic, these constraints are consistent, since a is a valid prefix of both regexes; however, according to the suffix heuristic, they are inconsistent, as the possible suffixes a and b of the second regex do not include c, and the empty string is not a solution to both constraints.Hence we conclude that these constraints are not jointly satisfiable, and assert a conflict clause.
As demonstrated, all of these facts are derived from the syntax of the regular expression; the heuristic does not need to construct an automaton for either constraint in order to reason about them.By constructing an over-approximation of the possible solutions of X allowed by regex constraints, the heuristic can determine that their intersection is empty (or can only contain the empty string) without computing it precisely (which, as previously mentioned, is expensive to do and also requires constructing automata first).We limit this heuristic to the first letter as each additional letter causes the space required to keep track of these prefixes to increase exponentially.

Empirical Results
In this section, we describe the empirical evaluation of Z3str3RE, our implementation of the length-aware regular expression algorithm presented in Section 3, to validate the effectiveness of the techniques presented.We evaluate the correctness and efficiency of our tool against other solvers, as well as against different configurations of the tool in order to demonstrate the efficacy of the heuristics we describe.

Empirical Setup and Solvers Used
We compare Z3str3RE against five other leading string solvers available today.CVC4 [21]  regular expressions algebraically.Z3str3 [7] is the latest solver in the Z3-str family, and uses a reduction to word equations to reason about regular expressions.Z3str3RE is based on Z3str3 except for the length-aware algorithm and heuristics described in Sections 3 and 4. Z3seq [33] is the Z3 sequence solver, implemented by Nikolaj Bjørner and others at Microsoft Research, as part of the Z3 theorem prover.Z3seq uses a new theory of derivatives for solving extended regular expressions.Z3-Trau [1] is also based on Z3 and uses an automata-based approach known as "flat automata" with both under-and over-approximations.OSTRICH [13] uses a reduction from string functions (including word equations) to a model-checking problem that is solved using the SLOTH tool and an implementation of IC3.We used CVC4's binary version 1.8, commit 59e9c87 of Z3str3, the sequence solver included in Z3's binary version 4.8.9,Z3-Trau commit 1628747, and OSTRICH version 1.0.1.All of these tools support the full SMT-LIB standard for strings.We did not compare against the Z3str2 [39] or Norn [3] solvers as neither tool supports the str.to int or str.from int terms which represent string-number conversion, which are used in some sanitizer benchmarks.Additionally, Norn does not support many of the other highlevel string terms such as indexof or substr which are used in the benchmarks.The ABC [4] solver handles string and length constraints by conversion to automata.However, their method over-approximates the solution set of the input formula which may be unsound.Thus, we excluded ABC from our evaluation.We also were unable to evaluate against Trau [2] as the provided source code did not compile.All evaluations were performed on a server running Ubuntu 18.04.4LTS with two AMD EPYC 7742 processors and 2TB of memory using the ZaligVinder [20] benchmarking framework.A 20 second timeout was used.We cross-verified the models generated by each solver for satisfiable instances against all competing solvers.

Benchmarks
The comparison was performed on four suites of regex-based benchmarks with a total of 57256 instances.In total, almost 75% of the instances in our evaluation came from previously published industrial benchmarks or other solver developers.Over all instances, 10% contain extended regular expressions (having either complement or intersection, or both).We briefly describe each benchmark's origin and composition.AutomatArk is a set of 19979 benchmarks based on a collection of real-world regex queries collected by Loris D'Antoni from the University of Wisconsin, Madison, USA.We translated the provided regexes [14] into SMT-LIB syntax resulting in two sets of instances: a "simple" set with a single regex membership predicate per instance, and a "complex" set with 2-5 regex membership predi-cates (possibly negated) over a single variable per instance.The instances in this benchmark are evenly divided between simple and complex problems.
RegEx-Collected is a set of 22425 instances taken from existing benchmarks with the purpose of evaluating the performance of solvers against real-world regex instances.This benchmark includes all instances from the AppScan [38], BanditFuzz,10 JOACO [35], Kaluza [30], Norn [3], Sloth [18], Stranger [37], and Z3str3-regression [7] benchmarks in which at least one regex membership constraint appears.11additional restrictions are placed on which instances were chosen besides the presence of at least one regex membership predicate.We chose to evaluate against this benchmark in order to test the performance of solvers against instances that are already known to be challenging to solve and that have appeared in previously published and widely distributed benchmark suites.Additionally, these instances may contain regex terms in any context and with any other supported string operators.As a result, the benchmark is also exemplary of how string solvers perform in the presence of operations and predicates that are relevant to program analysis.
StringFuzz-regex-generated is a set of 4170 problems generated by the String-Fuzz string instance fuzzing tool [10].These instances only contain regular expression and linear arithmetic constraints.The motivation in choosing this benchmark is to isolate and evaluate the regex performance of a string solver in the context of mixed regex and arithmetic constraints.Tools with better regex and arithmetic solvers should perform better.Fuzz testing, as performed in the StringFuzz-regex-generated benchmark, has been shown to be extremely productive in discovering bugs and performance issues in SMT solvers.We chose to include these instances because they enable us to isolate the performance of the solver on regex-heavy constraints in a way that the industrial benchmarks or instances obtained from other solver developers cannot.
StringFuzz-regex-transformed is a set of 10682 instances which were produced by transforming existing industrial instances with StringFuzz.To create the StringFuzz-regex-transformed benchmark, we applied StringFuzz's transformers to instances supplied by Amazon Web Services related to security policy validation, handcrafted instances which are inspired by real-world input validation vulnerabilities, and the regex test cases included in Z3str3's regression test suite.The instances in this suite include regex constraints, arithmetic constraints on string length, string-number conversion (numstr), string concatenation, word equations, and other high-level string operations such as charAt, indexof, and substr.As is typical for fuzzing in software testing, the goal is to create a suite of tests from a given input that are similar in structure but that explore interesting behaviour not captured by a "typical" industrial instance.These transformed instances are often harder than the original industrial ones.3. Detailed results for the StringFuzz-regex-generated benchmark.Z3str3RE has the biggest lead with a score of 1.25.

Comparison and Scoring Methods
We compare solvers directly against the total number of correctly solved cases, total time with and without timeouts, and total number of soundness errors and program crashes.We also computed the biggest lead winner and largest contribution ranking following the scoring system used by the SMT Competition [6].Briefly, the biggest lead measures the proportion of correct answers of the leading tool to correct answers of the next ranking tool, and the contribution score measures what proportion of instances were solved the fastest by that solver.In accordance with the SMT Competition guidelines, a solver receives no contribution score (denoted as -) if it produces any incorrect answers on a given benchmark.In both cases, higher scores are better.0 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000 10,000

Analysis of Empirical Results
The cactus plot in Figure 2 shows the cumulative time taken by each solver on all cases in increasing order of runtime.Solvers that are further to the right and closer to the bottom of the plot have better performance.
Overall Z3str3RE solves more instances and performs better than all competing solvers.Across all benchmarks, Z3str3RE is over 2.4x faster than CVC4, 4.4x faster than Z3seq, 6.4x faster than Z3-Trau, 9.1x faster than Z3str3, and 13x faster than OSTRICH (including timeouts).Additionally, Z3str3RE has fewer combined timeouts and unknowns than other tools considered, and no soundness errors or crashes.We summarize these results in Table 1.Notably, both Z3-Trau [1] and OSTRICH [13]   Z3-Trau produced 5325 soundness errors and 2477 crashes on our benchmarks (13% of all instances), which is significantly higher than other tools used.OS-TRICH produced 10901 "unknown" responses on the benchmarks (19% of all instances), due to both unsupported features and crashes, and also produced 28 soundness errors.Over all benchmarks, Z3str3RE produced 291 unknowns.There are several potential reasons for this; the solver may have encountered a resource limit and returned UNKNOWN, or Z3str3 may have detected nontermination and returned UNKNOWN instead of looping forever.According to SMT Competition scoring, Z3str3RE won the division across all benchmarks with a lead of 1.02, and had the largest contribution to the division with a score of 145.07.CVC4 had a contribution score of 95.99, and Z3seq had a score of 19.87.OSTRICH, Z3-Trau, and Z3str3 received no contribution score as they each returned at least one incorrect answer.The presented results are typical of the performance of the evaluated tools over multiple runs.Results were crossvalidated within runs and between multiple runs.For a random single instance, the sample variance in execution time for 100 runs is 0.001 (0.07% of average execution time).Over 57256 instances, this is negligible.
The empirical results make clear the efficacy of length-aware automata-based techniques for regular expression constraints when accompanied with length constraints (which is typical for industrial instances).The effectiveness of our technique is demonstrated particularly by comparing Z3str3RE with Z3str3, as the only differences between these tools are the length-aware regex algorithm and heuristics implemented in Z3str3RE and bug fixes.By improving the regex algorithm and applying our heuristics, we achieved a speedup of over 9x and solved over 10000 more cases than Z3str3.

Detailed Experimental Results
Figure 3 and Table 2 show the detailed results for the AutomatArk benchmark.In this benchmark, Z3str3RE solves more instances than all other solvers, has the fewest timeouts/unknowns, and has the fastest overall running time.Including timeouts, Z3str3RE is 2.2x faster than CVC4, 4.7x faster than Z3seq, 40.4x faster than OSTRICH, 20.4x faster than Z3-Trau, and 32.3x faster than Z3str3.Figure 4 and Table 3 show the detailed results for the StringFuzz-regexgenerated benchmark.Z3str3RE solves more instances than all other solvers, has over 90% fewer timeouts than other solvers, no unknowns, and has the fastest overall running time.Including timeouts, Z3str3RE is 6.1x faster than CVC4, 6.9x faster than Z3seq, 10x faster than OSTRICH, 7.3x faster than Z3-Trau, and 4.3x faster than Z3str3.
Figure 5 and Table 4 show the detailed results for the StringFuzz-regextransformed benchmark.Z3str3RE solves more instances in total than all other solvers and has the lowest total running time without timeouts.Including timeouts, Z3str3RE is 2.7x faster than CVC4, 1.9x faster than Z3seq, 21x faster than OSTRICH, and 27x faster than Z3str3.Although Z3-Trau is 1.5x faster than Z3str3RE on this benchmark, including timeouts, Z3-Trau also produces 1241 answers with soundness errors and crashes on 718 other cases.Z3str3RE produces no wrong answers or soundness errors on the benchmark.Z3-Trau also solves 1923 fewer cases correctly in total than Z3str3RE.Figure 6 and Table 5 show the detailed results for the RegEx-Collected benchmark.Z3str3RE outperforms Z3seq, Z3str3, OSTRICH, and Z3-Trau on this benchmark and is competitive with CVC4 both in terms of total number of instances correctly solved and total running time.CVC4 solves 609 more instances than Z3str3RE on this benchmark, but Z3str3RE is 1.1x faster overall (including timeouts).Z3str3RE is 3.6x faster than Z3seq, 5.4x faster than OS-TRICH, 2.4x faster than Z3-Trau, and 2.6x faster than Z3str3.

Analysis of Individual Heuristics and Results
To demonstrate the effectiveness of individual heuristics described in Section 4 and implemented in Z3str3RE, we evaluated different configurations of the tool in which one or more heuristics were disabled.Figure 7 and Table 6 show the results.The plot line "Z3str3RE" shows the baseline performance of the tool with all heuristics enabled.The plot line "All heuristics off" shows the performance with all heuristics disabled.Each of the other plot lines shows the performance with the named heuristic disabled and all others kept enabled.From the plots and table, it is clear that Z3str3RE performs best with all heuristics enabled.Z3str3RE is 4.4x faster using all our heuristics than using none.Every other configuration of the tool performs significantly worse relative to the one with all heuristics enabled.Also, the length-aware and prefix/suffix heuristics provide significant boost over lazy intersections and the baseline.These results demonstrate empirically that each heuristic we introduce provides significant benefit in both total number of solved instances and total solver runtime, and that all of the heuristics can be used simultaneously for maximum efficacy.

Related Work
Comparison with Z3str3: Z3str3 [7] supports regex constraints via (incomplete) reduction to word equations.We have replaced this word-based technique with our automata-based approach introduced in this paper.As demonstrated by our evaluation, the length-aware automata-based approach used in Z3str3RE is more efficient at solving these constraints, and is sound and complete for the QF theory T LRE .
Comparison with Z3's sequence solver: Z3's sequence solver [16] supports a more general theory of "sequences" over arbitrary datatypes, which allows it to be used as a string solver.Z3seq uses regular expression derivatives to reduce regex constraints without constructing automata.The experiments show Z3str3RE performs better than Z3seq overall.
Comparison with CVC4: The CVC4 solver [21] uses an algebraic approach to solving regex constraints.As shown in the experiments, Z3str3RE performs better than CVC4, widely considered as one of the best SMT solvers for strings as well as many other theories.
Comparison with Z3-Trau: The Z3-Trau [1] solver builds on Trau [2], reimplemented in Z3, and enriched with new ideas e.g. a more efficient handling of string-number conversion.The evaluation of Z3-Trau exposed 5325 soundness errors and 2477 crashes on our benchmarks.
Comparison with OSTRICH: The OSTRICH solver [13] implements a reduction from straight-line and acyclic fragments of an input formula to the emptiness problem of alternating finite automata.OSTRICH produced 10901 "unknown" responses and 4575 timeouts on our benchmarks, as well as 28 soundness errors.Related Algorithms and Theoretical Results: The theory of word equations and various extensions have been studied extensively for many decades.
In 1977, Makanin proved that satisfiability for the QF theory of word equations is decidable [25]; in 1999, Plandowski showed that this is in PSPACE [27,28].Schulz [31] extended Makanin's algorithm to word equations with regex constraints.The satisfiability problem for the theory of word equations with length constraints still remains open [25,28,17,26], although the status of many other extensions of this theory was clarified [15].Automata-based approaches were used to reason about string constraints enhanced with a ReplaceAll function [12] or transducers [18].Liang et al. [22] present a formal calculus for a theory that extends T LRE with string concatenation (but not word equations).However, in that paper the authors do not present experimental results regarding implementation of the string calculus proposed.We have implemented an algorithm based on fundamentals of the theory and standard automata-based constructions, and presented a thorough experimental evaluation of our implementation, concluding with the presentation of a string solver using it.
Abdulla et al. [3] present an automata-based solver called Norn built upon results involving construction of length constraints from regex constraints.This approach differs significantly from our method.In particular, Norn only uses automata in inferring length constraints implied by regular expressions, then uses an algebraic approach to solve the remainder of the formula.By contrast, our tool uses a hybrid approach that includes both algebraic solving and automata-based reasoning in a symbiotic loop.In addition, we present several novel heuristics using length information to guide the search and, in some cases, avoid constructing automata or computing intersections.
The prefix/suffix over-approximation heuristic is inspired partly by the work of Brzozowski on regex derivatives [11].The heuristic we introduce is conceptually different as we examine possible prefixes (and suffixes) of strings that could be accepted by a regex in order to demonstrate unsatisfiability, rather than examining the set of all possible suffixes given a fixed prefix in order to demonstrate satisfiability.Our heuristic computes suffixes as well, whereas Brzozowski derivatives are traditionally computed with respect to prefixes of a string.Newer versions of Z3seq, including the one we evaluated, use an algorithm based on symbolic derivatives to reason about regular expressions [33].

Conclusions and Future Work
In this paper, we have shown the power of length-aware and prefix/suffix reasoning for regex constraints with our algorithm and its implementation in Z3str3RE via an extensive empirical comparison against five other state-of-the-art solvers (namely, CVC4, Z3seq, Z3str3, Z3-Trau, and OSTRICH) over a large and diverse benchmark of 57256 instances.Our length-aware method is very general and has wide applicability in the broad context of string solving.In the future, we plan to explore further length-aware heuristics which include more expressive functions and predicates, including indexof, substr, and string-number conversion.

FFig. 1 .
Fig. 1.Syntax of the input language accepted by Algorithm 1. Z3str3RE accepts an extension of this syntax supporting word equations and other string terms.

Algorithm 1 : 3 LRE 5 return UNSAT 6 end 7 refine 11 I 25 else 26 stop processing and try the next solution M 27 end 28 end 29 if
The length-aware algorithm for the theory T LRE of regex and integer constraints Input : Conjunction φ of constraints of the form S ∈ RE, and conjunction ψ of linear integer arithmetic constraints Output : SAT or UNSAT 1 forall constraints S ∈ RE in φ do 2 LS ← ComputeLengthAbstraction(S) ; ← ComputeLengthAbstraction(RE) ; 4 if ψ ∪ LS ∪ LRE inconsistent then LS as tightly as possible with respect to LRE ; 8 end 9 forall strings Si occurring in φ do 10 let R be the set of all regexes RE in all terms Si ∈ RE ; ← intersection of all regular expressions in R ; 12 if I is empty then 13 return UNSAT 14 end 15 end 16 LS ← the union of all length abstractions LS; 17 LRE ← the union of all length abstractions LRE; 18 forall solutions M of the system ψ ∪ LS ∪ LRE do 19 forall strings S occurring in φ do20 len(S) ← M[len(S)] ; 21 let A be the set of all automata for all regexes RE in all terms Si ∈ RE ; 22 determinize all accepting paths of length len(S) for each A in A ; 23 if each automaton has an accepting path of length len(S) and all paths are character-consistent then 24 continue all strings S were processed successfully with respect to this M then

Table 1 .
is a general-purpose SMT solver which reasons about strings and Combined results of string solvers on all benchmarks.Z3str3RE has the best overall performance on all benchmarks compared to CVC4, OSTRICH, Z3seq, Z3str3, and Z3-trau and the biggest lead with a score of 1.02.

Table 2 .
Detailed results for the Automatark benchmark.Z3str3RE has the biggest lead with a score of 1.01.

Table 4 .
Detailed results for the StringFuzz-regex-transformed benchmark.Z3str3RE has the biggest lead with a score of 1.0.
had significant runtime issues in our experiments.

Table 5 .
Detailed results for the RegEx-Collected benchmark.CVC4 has the biggest lead with a score of 1.03.

Table 6 .
Comparison of different heuristics in Z3str3RE on all benchmarks.