A #SAT Algorithm for Small Constant-Depth Circuits with PTF gates

Abstract

We show that there is a better-than-brute-force algorithm that, when given a small constant-depth Boolean circuit C made up of gates that compute constant-degree Polynomial Threshold functions or PTFs (i.e., Boolean functions that compute signs of constant-degree polynomials), counts the number of satisfying assignments to C in significantly better than brute-force time. Formally, for any constants d, k, there is an $\varepsilon > 0$ such that the zero-error randomized algorithm counts the number of satisfying assignments to a given depth-d circuit C made up of k-PTF gates such that C has at most $n^{1+\varepsilon }$ many wires. The algorithm runs in time $2^{n-n^{\Omega (\varepsilon )}}.$ Before our result, no algorithm for beating brute-force search was known for counting the number of satisfying assignments even for a single degree-k PTF (which is a depth-1 circuit with linearly many wires).We give two different algorithms for the case of a single PTF. The first uses a learning algorithm for learning degree-1 PTFs (or Linear Threshold Functions) using comparison queries due to Kane, Lovett and Moran (STOC 2018), and the second uses a proof of Hofmeister (COCOON 1996) for converting a degree-1 PTF to a depth-two threshold circuit with small weights. We show that both these ideas fit nicely into a memoization approach that yields the #SAT algorithms.

Appendices

A Proof of Theorem 12

We need the following definition from [20].

Definition 31

(Inference from comparison queries [20].) For a finite set $S\subseteq \{-1,1\}^n$ and a vector $w\in \mathbb {R}^n$, let $\text {infer}(S,w)$ denote the set of all points $a\in \{-1,1\}^n$ such that the sign of $\langle a,w \rangle$ can be inferred from label and comparison queries on S i.e. all queries of the form $\mathrm {sgn}(\langle s,w\rangle )$ and $\mathrm {sgn}(\langle s-t,w\rangle )$ respectively, where $s,t\in S$. Formally, this means that $a\in \text {infer}(S,w)$ if and only if for any $w'\in \mathbb {R}^n$ such that $\mathrm {sgn}(\langle s,w\rangle ) = \mathrm {sgn}(\langle s,w'\rangle )$ and $\mathrm {sgn}(\langle s-t,w\rangle ) = \mathrm {sgn}(\langle s-t,w'\rangle )$ for all $s,t\in S$, it also holds that $\mathrm {sgn}(\langle a,w\rangle ) = \mathrm {sgn}(\langle a,w'\rangle ).$ (Here, we also allow $\langle s,w\rangle$ to be 0 in which case we use the standard definition $\mathrm {sgn}(\langle s,w\rangle ) = 0.$)

Lemma 5.2 from [19] will be instrumental for us, which we restate (in a slightly weaker form) as follows.

Lemma 32

(Weaker version of Lemma 5.2 from [19]) Suppose $T\subseteq \{-1,1\}^r$. Then there exists a subset $S\subseteq T$ of size $|S| = \ell = O(r\log r)$ (which we shall term a ‘universal’ set for T) such that for every $w\in \mathbb {R}^r$, $|\text {infer}(S,w)\cap T| \ge \frac{|T|}{8}.$

We now begin the proof of Theorem 12.

The proof of existence of the decision tree in [19, Theorem 1.8] now proceeds as follows: the tree consists of levels such that at each level, we have a subset T containing those elements t of H for which $\mathrm {sgn}(\langle t,w\rangle )$ has not yet been inferred. But by Lemma 32, there is a subset $S\subseteq T$ such that by performing label and comparison queries on S, we can infer the signs of $\langle t,w\rangle$ for at least a constant fraction of |T|. Assume $S = \{s_1,\ldots ,s_\ell \}.$ The linear decision tree queries $\langle s_i, w\rangle \ge 0$ to determine $\mathrm {sgn}(\langle s_i,w\rangle )$ for each $i\in [\ell ],$ and then performs a comparison-based sorting algorithm to determine the signs of each $\langle s_i-s_j, w\rangle$ for each $i,j\in [\ell ]$. Note that this can be done with $O(\ell \log \ell )$ many queries of the form $\langle s_i-s_j,w\rangle \ge 0$ and hence, the queries of the linear decision tree have coefficients from $\{-2,-1,0,1,2\}$ as claimed. By the definition of S, we can now infer $\mathrm {sgn}(\langle t,w\rangle )$ for at least |T|/8 many $t\in T$. We then recurse in this manner on the smaller set of elements whose signs are not inferred, which constitutes the next level. After $O(\log |H|)$ many such levels, we have inferred $\mathrm {sgn}(\langle t,w\rangle )$ for each $t\in H$, which means we can output the truth table of the LTF defined by w on inputs from H. This gives us a linear decision tree of depth $O(\ell \log \ell \cdot \log |H|) = O(r\log ^2 r\cdot \log |H|)$ as claimed.

The above description of the linear decision tree is constructive given the universal sets S at each level. Therefore, to give an algorithm for constructing the linear decision tree, it suffices to give an algorithm to find these ‘universal’ sets S at each level. We next define a subroutine ${\mathcal {F}}$ that on input $T\subseteq \{-1,1\}^r$, outputs a subset S satisfying the conclusion of lemma 32. The basic idea of the subroutine is to cycle through all sets S of size $\ell$ and for each S, check if it is universal for the set T; by an observation of [20],¹³ this latter check can be performed efficiently (in the sense we need) by solving a linear program.

${\mathcal {F}}(T)$:

• 1. Let $\ell = O(r\log r)$ as in Lemma 32. If $|T|<\ell$, simply output T. Otherwise, proceed as follows.

• 2. For all $S\subseteq T$ of size $\ell$,

  • (a) Let $S = \{s_1,\ldots ,s_\ell \}.$

  • (b) For each permutation $\pi :[\ell ]\rightarrow [\ell ],$ $b\in \{0,1\}^{\ell -1}$ and $j\in [\ell +1]$, do the following.

    • i Solve a linear program to find a $w\in \mathbb {R}^r$ such that:

• $\langle s_{\pi (1)},w\rangle \ge \langle s_{\pi (2)},w\rangle \ge \cdots \ge \langle s_{\pi (\ell )},w\rangle ,$

• $\langle s_{\pi (i)},w\rangle = \langle s_{\pi (i+1)},w\rangle$ if and only if $b_i = 1,$ and

• $\langle s_{\pi (i)}, w \rangle \ge 0$ if and only if $i < j.$

We call such a w (if it exists) $(S,\pi ,b,j)$-feasible. Further, we say that $(\pi ,b,j)$ is S-feasible if such a w exists and S-infeasible otherwise. If $(\pi ,b,j)$ is S-infeasible, proceed to the next iteration of this loop.

• ii Set $I(S,\pi ,b,j) = S.$

• iii For each $t\in T{\setminus } S$, let $\beta = \beta (S,\pi ,b,j,t) = \mathrm {sgn}(\langle t,w\rangle ) \in \{-1,0,1\}$. For each $\gamma \in \{-1,0,1\}{\setminus } \{\beta \},$ solve a linear program to check if there is a $w^\gamma$ such that $w^\gamma$ is $(S,\pi ,b,j)$-feasible and furthermore satisfies $\mathrm {sgn}(\langle t,w^\gamma \rangle ) = \gamma .$ If there is no such $w^\gamma$ (for either value of $\gamma \in \{-1,0,1\}{\setminus } \{\beta \}$), then add t to the set $I(S,\pi ,b,j)$.

  • 3. Output a set S for which $|I(S,\pi ,b,j)|\ge |\frac{|T|}{8}$ for each S-feasible $(\pi ,b,j)$.

Claim 33

The subroutine $\mathcal {F}$ outputs a set S that is universal for T.

Proof

The main observation is the following. Fix any $S\subseteq T$ of size $\ell$ and $(\pi ,b)$ as chosen in Step 2(b) above. For any $w,w'\in \mathbb {R}^r$ that are $(S,\pi ,b,j)$-feasible, we have $\text {infer}(S,w) = \text {infer}(S,w')$ (this follows from the definition of $\text {infer}(S,w)$ above). In particular, the set $I(S,\pi ,b,j)$ computed by the algorithm in Step 2(b) is equal to $\text {infer}(S,w)$ for any w that is $(S,\pi ,b,j)$-feasible (and not just the w chosen in Step 2(b)i.).

Now, assume that $S \subseteq T$ of size $\ell$ is not universal for T. We argue that $\mathcal {F}$ cannot output S. We know there is a $w\in \mathbb {R}^n$ such that $\text {infer}(S,w) < |T|/8.$ Fix $(\pi ,b)$ such that w is $(S,\pi ,b,j)$-feasible. When the algorithm considers this $(\pi ,b,j)$ in Step 2(b), it will find that $I(S,\pi ,b,j) = \text {infer}(S,w)$ has size less than |T|/8. Hence, this set S will not be output by $\mathcal {F}.$

By a similar argument, it follows that if S is universal, then the algorithm will find that $|I(S,\pi ,b)|>|T|/8$ for any $(\pi ,b,j)$ that is S-feasible. Since Lemma 32 guarantees the existence of a universal subset S of size $\ell$, the algorithm outputs such a set. $\square$

Armed with the subroutine ${\mathcal {F}}$, we can now describe the algorithm to construct the linear decision tree for F exactly as in [19]. We give the details here for completeness. The algorithm takes as input $H\subseteq \{-1,1\}^r$ and outputs a linear decision tree ${\mathcal {T}}$ each of whose leaves is labelled by a function $v:H\rightarrow \{-1,1\}$.

${\mathcal {C}}(H)$:

• 1. Run $\mathcal {F}(H)$ and let $S = \{s_1,\ldots ,s_\ell \}$ be the universal set found by $\mathcal {F}$. If $S = H,$ the output tree ${\mathcal {T}}$ is simply a bruteforce algorithm that queries $\langle t, w \rangle \ge 0$ for each $t\in H$. Otherwise, proceed as follows.

• 2. Compute a linear decision tree ${\mathcal {T}}'$ which makes $O(\ell \log \ell )$ queries of the form $\langle s_i, w \rangle \ge 0$ and $\langle s_i-s_j, w \rangle \ge 0$, and uses a comparison-based sorting algorithm to compute $(\pi ,b,j)\in S_\ell \times \{0,1\}^{\ell -1}\times [\ell +1]$ such that w is $(S,\pi ,b,j)$-feasible.

• 3. For each leaf $\lambda '$ of ${\mathcal {T}}'$, which corresponds to some $(\pi ,b,j)$, do the following.

  • (a) Compute $I(S,\pi ,b,j)$ as in Step 2(b) of algorithm $\mathcal {F}$ above. Define a function $v_{\lambda '}:I(S,\pi ,b,j)\rightarrow \{-1,1\}$ by $v_{\lambda '}(t) = \beta (S,\pi ,b,j,t)$ where $\beta (S,\pi ,b,j,t)$ is as computed in Step 2(b) of algorithm $\mathcal {F}$ above.

  • (b) Recursively call ${\mathcal {C}}(H{\setminus } I(S,\pi ,b,j))$ to obtain a linear decision tree ${\mathcal {T}}_{\pi ,b,j}$ that at each leaf $\lambda ''$ outputs a $v_{\lambda ''}:H{\setminus } I(S,\pi ,b,j)\rightarrow \{-1,1\}.$

  • (c) Append ${\mathcal {T}}_{\pi ,b,j}$ to ${\mathcal {T}}$ at leaf $\lambda '$ and relabel each leaf $\lambda ''$ of ${\mathcal {T}}_{\pi ,b,j}$ by the function $v_{\lambda '}\cup v_{\lambda '}:H\rightarrow \{-1,1\}.$

• 4. Output the tree ${\mathcal {T}}$ thus constructed.

Correctness We start by arguing that the output function $v:H\rightarrow \{-1,1\}$ correctly computes the LTF defined by w at each point $t\in H$. The proof is by induction on the size of H. The base case, when $|H| < \ell$ is trivial, since $\mathcal {F}(H) = H$ in this case. We now argue the inductive case. In the proof of Claim 33 above, we observed that for any w that is $(S,\pi ,b,j)$-feasible, the set $I(S,\pi ,b,j)$ computed in $\mathcal {F}$ is exactly the set $\text {infer}(S,w).$ The same argument also shows that for each $t\in I(S,\pi ,b,j),$ $\beta (S,\pi ,b,j,t) = \mathrm {sgn}(\langle t, w \rangle )$ for each w that is $(S,\pi ,b,j)$-feasible. It follows from this that the partial function $v_{\lambda '}$ defined in Step 3(a) of $\mathcal {C}$ computes $\mathrm {sgn}(\langle t, w \rangle )$ correctly for each $t\in I(S,\pi ,b,j)$. By induction, the recursive algorithm outputs a linear decision tree that computes $\mathrm {sgn}(\langle t, w \rangle )$ correctly for each $t\in H{\setminus } I(S,\pi ,b,j)$. Putting these together, we see that the output tree ${\mathcal {T}}$ computes $\mathrm {sgn}(\langle t, w \rangle )$ correctly for all $t\in H.$

The bound claimed in Theorem 12 on the depth $\Delta$ of ${\mathcal {T}}$ follows easily, using the fact that S is universal and hence $|I(S,\pi ,b,j)|\ge |T|/8$ for each possible $(\pi ,b,j)$ corresponding to a leaf of ${\mathcal {T}}'$.

Running Time First let us analyze the running time of the subroutine ${\mathcal {F}}$ on a set $T\subseteq \{-1,1\}^r$ of size N. There are at most $O(N^\ell )$ subsets of T of size $\ell$ and at most $O(\ell ^\ell )$ triples $(\pi ,b,j)\in S_\ell \times \{0,1\}^{\ell -1}\times [\ell +1]$. Further, for fixed $S,\pi ,b,j$, we run a linear program that runs in time $\mathop {\mathrm {poly}}(\ell )$ for each $t\in T$. Therefore, we can bound the total running time of the subroutine by ${\mathsf {T}}_\mathcal {F}(N):= O(N^{\ell +1}\cdot \ell ^\ell \cdot \mathop {\mathrm {poly}}(\ell ))$.

The running time of $\mathcal {C}(H)$ can now be bounded by noting that $\mathcal {C}$ calls itself at most $2^{O( \Delta )}$ times and the running time per recursive call can be bounded by $\ell ^{O(\ell )}\cdot \mathop {\mathrm {poly}}({\mathsf {T}}_\mathcal {F}(|H|)) = (|H|\ell )^{O(\ell )}.$ This yields a running time of $2^{O(\Delta )}\cdot (|H|\ell )^{O(\ell )} = 2^{O(\Delta )}.$

B Proof of Lemma 15

Say f is represented by integer polynomial $P$ with parameters (n, M). First, note that $P$ can be thought of as a linear function L evaluated at the point $(y_1, \ldots , y_m)$ where each $y_i$ represents a monomial over $x_i$s, of degree at most $k$ and thus, $m = \sum _{i=0}^k \left( {\begin{array}{c}n\\ i\end{array}}\right) \le n^k$. Let this linear function be $L = \sum _{i=1}^{m} w_iy_i + w_0$. Let $\sum _i \left| w_i\right| + \left| w_0\right| = W$. Note that $\log _2(W)\le M$.

Now, for any $r \in \mathbb {R}$, define the following operation:

$$\begin{aligned} \mathrm {trunc}(r) = {\left\{ \begin{array}{ll} \left\lfloor r/2 \right\rfloor &{} r \ge 0 \\ \left\lceil r/2 \right\rceil &{} r < 0.\end{array}\right. } \end{aligned}$$

Also define the linear function $\mathrm {half}(L)$ in the following way: $\mathrm {half}(L) = \sum _{i=1}^{m} \mathrm {trunc}(w_i) y_i$. Using this, define the following sequence of linear functions.

$$\begin{aligned} L^{(0)} = L \text { and } L^{(\ell +1)} = \mathrm {half}(L^{(\ell )}) \end{aligned}$$

Note that $L^{(M+1)} \equiv 0$ since all its coefficients are 0. The rounding procedure used for truncating introduces errors. Define the error as follows.

$$\begin{aligned} \mathrm {err}(L) = \left\lceil \max _{y \in \{-1, 1\}^m} \max _{\ell \ge 0} \left| \frac{L^{(\ell )}(y)}{2} - L^{(\ell +1)}(y)\right| \right\rceil \end{aligned}$$

Trivially, we have $\mathrm {err}(L) \le m$. Lemma 1 from [11] characterizes the behaviour of the series of linear functions just defined.

Lemma 34

(Lemma 1 from [11]) Define the interval $T = [-2 \cdot \mathrm {err}(L), 2 \cdot \mathrm {err}(L)]$. Then the following holds for all $y\in \{-1,1\}^m$.

• 1. $L^{(\ell )}(y) \in T \implies L^{(\ell +1)}(y) \in T$.

• 2. $L^{(\ell )}(y) \not \in T \implies \mathrm {sgn}(L^{(\ell +1)}(y)) = \mathrm {sgn}(L^{(\ell )}(y))$.

Note that if $L^{(\ell +1)}(y) \in T$ then $L^{(\ell )}(y) \ge -6 \cdot \mathrm {err}(L)$. Thus, for any $y\in \{-1,1\}^m$, we can write

$$\begin{aligned} \mathrm {sgn}(L(y))= & {} \bigvee _{l=0}^{M} \left( L^{(\ell +1)}(y) \in [-2 \cdot \mathrm {err}(P), -1] \wedge L^{(\ell )}(y) \in [-6 \cdot \mathrm {err}(P), -2 \cdot \mathrm {err}(P) - 1]\right) \\= & {} \bigvee _{l=0}^{M} \bigvee _{i=-2 \cdot \mathrm {err}(P)}^{-1} \bigvee _{j =-6 \cdot \mathrm {err}(P)}^{-2 \cdot \mathrm {err}(P) - 1}\left( L^{(\ell +1)}(y) = i \wedge L^{(\ell )}(y) = j \right) \end{aligned}$$

The above represents the LTF $\mathrm {sgn}(L(y))$ as an OR of ANDs of ETFs. Moreover, the OR is a disjoint OR: this follows from Lemma 34 which implies that there can be at most one $\ell \le M$ such that $L^{(\ell +1)}(y)$ lies in the interval $[-2\mathrm {err}(L), 2\mathrm {err}(L)]$ and $L^{(\ell )}(y)$ does not. Note that this size of the disjoint OR is at most $(M+1)\cdot O(m^2) = O(Mn^{2k}).$

To finish the proof, it suffices to show that $(L^{(\ell +1)}(y) = i) \wedge (L^{(\ell )}(y) = j)$ can be represented as a single ETF. Define $G(y) = L^{(\ell +1)}(y) - i$ and $H(y) = L^{(\ell )}(y) - j$. The total sum of absolute values of coefficients of both $G$ and $H$ is at most $W' = W + m + 1$, using the trivial bound for $\mathrm {err}(L)$. Consider the ETF $(W' + 1)G(y) + H(y) = 0$. This is clearly equivalent to $G(y) = 0 \wedge H(y) = 0$. Also, the bit complexity of this ETF is at most $2\log _2 W' \le O(M + k \log n)$. Now, each ETF is over the variables $y_i$, which themselves were monomials over $x_i$. Thus each ETF can be thought of as a $k$-EPTF over $x_i$. This completes the proof.