Computing Majority by Constant Depth Majority Circuits with Low Fan-in Gates

Abstract

We study the following computational problem: for which values of k, the majority of n bits MAJ_n can be computed with a depth two formula whose each gate computes a majority function of at most k bits? The corresponding computational model is denoted by MAJk ∘ MAJk. We observe that the minimum value of k for which there exists a MAJk ∘ MAJk circuit that has high correlation with the majority of n bits is equal to Θ(n^1/2). We then show that for a randomized MAJk ∘ MAJk circuit computing the majority of n input bits with high probability for every input, the minimum value of k is equal to n^{2/3 + o(1)}. We show a worst case lower bound: if a MAJk ∘ MAJk circuit computes the majority of n bits correctly on all inputs, then k ≥ n^{13/19 + o(1)}.

Appendix: Proofs of Probabilistic Lemmas

Proof of Lemma 3

To bound the probability we will use Stirling’s approximation, the following simple form will be enough:

$$n! \sim \left( \frac{n}{e}\right)^{n} \sqrt{n}. $$

It is well known (see e.g. [7]) that the probability in (2) is maximal for l = pm, so it is enough to upper bound the probability for this l.

We have

$$\operatorname{Prob}\left[Y =mp\right] = \binom{m}{pm} p^{pm}(1-p)^{(1-p)m} = \frac{m!}{pm!(1-p)m!} p^{pm}(1-p)^{(1-p)m} . $$

Let us first consider binomial coefficients separately:

$$\begin{array}{@{}rcl@{}} \frac{m!}{pm!(1-p)m!} &\sim& \frac{\left( \frac{m}{e}\right)^{m} \sqrt{m}}{\left( \frac{pm}{e}\right)^{pm} \sqrt{pm}\left( \frac{(1-p)m}{e}\right)^{(1-p)m} \sqrt{(1-p)m}}\\ & =& \frac{1}{p^{pm}(1-p)^{(1-p)m}} \cdot\frac{1}{\sqrt{p(1-p)}\sqrt{m}}. \end{array} $$

Thus,

$$\operatorname{Prob}\left[Y =mp\right] = O\left( \frac{1}{\sqrt{m}}\right) . $$

□

Proof of Lemma 4

By Lemma 3 there is a constant α such that

$$ \operatorname{Prob}\left[\left|Y-\frac m2\right| < \alpha \sqrt{m}\right] < \frac{1}{100}. $$

(15)

Denote

$$\begin{array}{@{}rcl@{}} A &=& \operatorname{Prob}\left[Y \leq \frac{m}{2} - \alpha \sqrt{m} \right],\\ B &=& \operatorname{Prob}\left[\frac{m}{2}- \alpha \sqrt{m} < Y < \frac{m}{2} \right],\\ C &=& \operatorname{Prob}\left[\frac{m}{2} < Y < \frac{m}{2} + \alpha \sqrt{m}\right],\\ D &=&\operatorname{Prob}\left[Y \geq \frac{m}{2} + \alpha \sqrt{m} \right]. \end{array} $$

Clearly,

$$A+B+C+D = 1$$

and (15) can be rewritten as

$$B+C < \frac{1}{100}. $$

Let us denote X = B + C. Then A + D = 1 − X and $1-X \geq \frac {99}{100}$

To prove the lemma we need to show that $C+D \geq \frac 12 + {\Omega }\left (\varepsilon \sqrt {m}\right )$.

Consider $k = \frac m2 + t$ for some t > 0. Note that Prob [Y = k] ≥ Prob [Y = m − k], so

$$C \geq B. $$

Now consider the case of $t \geq \alpha \sqrt {m}$. We have

$$\operatorname{Prob}\left[Y = k\right] = \binom{m}{k}\left( \frac 12 + \varepsilon\right)^{k} \left( \frac 12 - \varepsilon\right)^{m-k} . $$

On the other hand

$$\operatorname{Prob}\left[Y = m-k\right] = \binom{m}{k}\left( \frac 12 + \varepsilon\right)^{m-k} \left( \frac 12 - \varepsilon\right)^{k}. $$

Then we have

$$\begin{array}{@{}rcl@{}} \frac{\operatorname{Prob}\left[Y = k\right]}{\operatorname{Prob}\left[Y = m-k\right]} &=& \frac{(\frac 12 + \varepsilon)^{2k-m}}{(\frac 12 - \varepsilon)^{2k-m}} =\left( \frac{\frac 12 + \varepsilon}{\frac 12 - \varepsilon} \right)^{2t}\\ &\geq&\left( 1 + \frac{2\varepsilon}{\frac 12 - \varepsilon} \right)^{2 \alpha \sqrt{m}}\geq 1 + c \varepsilon \sqrt{m} . \end{array} $$

for some positive constant c = c(α). The last inequality in this sequence is Bernoulli’s inequality.

Since this inequality holds for all $t \geq \alpha \sqrt {m}$, we have

$$D \geq A \cdot (1 + c \varepsilon \sqrt{m}) . $$

Thus we have

$$1- X = A+D \leq \frac{D}{1+c \varepsilon \sqrt{m}} + D. $$

We can rewrite

$$D \geq \frac{1+c \varepsilon \sqrt{m}}{2+c \varepsilon \sqrt{m}} (1-X) = \frac 12 (1-X) + \frac{c \varepsilon \sqrt{m}}{4 + 2c \varepsilon \sqrt{m}} (1-X). $$

Since $\varepsilon \sqrt {m}$ is positive and upper-bounded by 1, we have

$$\frac{c \varepsilon \sqrt{m}}{4 + 2c \varepsilon \sqrt{m}} = {\Theta}(\varepsilon \sqrt{m}). $$

It follows

$$A+D \geq \frac{1}{2} + {\Omega}(\varepsilon \sqrt{m}) (1-X) = \frac{1}{2} + {\Omega}\left( \varepsilon \sqrt{m}\right), $$

where in the last inequality we use that $1-X \geq \frac {99}{100}$.

□

Proof of Lemma 5

One simple way to prove this lemma is by a direct observation of probabilities.

In the setting with a random S^′ the probability that |T ∩ S^′| = l is

$$\operatorname{Prob}\left[|T\cap S^{\prime}|=l\right] = \frac{ \binom{k}{l}\binom{m-k}{t-l}}{\binom{m}{t}}. $$

In the setting with a random T the probability of the same event is

$$\operatorname{Prob}\left[|T\cap S^{\prime}|=l\right] = \frac{ \binom{t}{l}\binom{m-t}{k-l}}{\binom{m}{k}}. $$

It is straightforward to see that these probabilities are equal. □

Proof of Lemma 6

The probability under consideration is equal to

$$\operatorname{Prob}\left[|T\cap S^{\prime}|=l\right] = \frac{ \binom{k}{l}\binom{m-k}{t-l}}{\binom{m}{t}}. $$

It is convenient to introduce notation $c = \frac tm$. Note that then ε < c < 1 − ε. The probability above then can be rewritten as

$$\operatorname{Prob}\left[|T\cap S^{\prime}|=l\right] = \frac{ \binom{k}{l}\binom{m-k}{cm-l}}{\binom{m}{cm}}. $$

It is not hard to see that the maximum is achieved for l equal to ck (the probability is increasing for l < ck as a function of l and is decreasing for l > ck).

So we need to upper bound

$$ \frac{\binom{k}{ck}\binom{m-k}{c(m-k)}}{\binom{m}{cm}} = \frac{\frac{k!}{ck!(1-c)k!}\frac{(m-k)!}{c(m-k)!(1-c)(m-k)!}}{\frac{m!}{cm!(1-c)m!}}. $$

(16)

To bound the probability we again will use Stirling’s approximation:

$$n! \sim \left( \frac{n}{e}\right)^{n} \sqrt{n}. $$

As in the proof of Lemma 2 let us first consider binomial coefficients separately:

$$\begin{array}{@{}rcl@{}} \frac{m!}{cm!(1-c)m!} &\sim& \frac{\left( \frac{m}{e}\right)^{m} \sqrt{m}}{\left( \frac{cm}{e}\right)^{cm} \sqrt{cm}\left( \frac{(1-c)m}{e}\right)^{(1-c)m} \sqrt{(1-c)m}}\\ & =& \frac{1}{(c^{c}(1-c)^{1-c})^{m}} \cdot\frac{1}{\sqrt{c(1-c)}\sqrt{m}} \\ & =& d^{m} \cdot \frac{1}{\sqrt{c(1-c)}\sqrt{m}}, \end{array} $$

where by d we denote $\frac {1}{c^{c}(1-c)^{1-c}}$.

Now for (16) we have

$$\frac{d^{k} \cdot \frac{1}{\sqrt{c(1-c)}\sqrt{k}}\cdot d^{m-k} \cdot \frac{1}{\sqrt{c(1-c)}\sqrt{m-k}}}{d^{m} \cdot \frac{1}{\sqrt{c(1-c)}\sqrt{m}}} = \frac{\sqrt{m}}{\sqrt{c(1-c)}\sqrt{k}\sqrt{m-k}} \sim \frac{1}{\sqrt{k}}, $$

where the last equivalence follows since $\sqrt {m-k} = {\Theta }(\sqrt {m})$.

So, we have shown the first part of the lemma and the second part for l = ck. To ensure the second part for |l − ck| < d we can compare probabilities for l and l + 1:

$$\begin{array}{@{}rcl@{}} \frac{ \binom{k}{l}\binom{m-k}{cm-l}}{\binom{m}{cm}} = \frac{ \binom{k}{l + 1}\binom{m-k}{cm-l-1}}{\binom{m}{cm}}\cdot \frac{l + 1}{k-l}\cdot \frac{m-k-(cm-l)+ 1}{cm-l}. \end{array} $$

Note that if |l − ck| < d the probabilities differ by a constant factor. Thus the asymptotic of the probability is the same for all l satisfying |l − ck| < d. This finishes the proof of lemma. □

Proof of Lemma 7

We introduce the same notation as in the previous proof: $c = \frac tm$. The probability is bounded by

$$\begin{array}{@{}rcl@{}} \frac{ {\sum}_{r \in A}\binom{m-k}{cm-|r|}}{\binom{m}{cm}} &=& \sum\limits_{r \in A}\left( \frac{1}{\binom{k}{|r|}}\frac{ \binom{k}{|r|}\binom{m-k}{cm-|r|}}{\binom{m}{cm}} \right)\\ &\leq& \max_{|r|} \left( \frac{ \binom{k}{|r|}\binom{m-k}{cm-|r|}}{\binom{m}{cm}} \right) \sum\limits_{r \in A}\frac{1}{\binom{k}{|r|}} \leq \max_{|r|} \left( \frac{ \binom{k}{|r|}\binom{m-k}{cm-|r|}}{\binom{m}{cm}} \right), \end{array} $$

where the last inequality is LYM inequality (see e.g. [12], Theorem 8.6).

Now we can bound the probability by the same argument as in Lemma 6. □

Proof of Lemma 9

The lemma can be shown by a simple direct calculation:

$$\begin{array}{@{}rcl@{}} \operatorname{Prob}\{|T \cap S^{\prime}| \ge l\} &\le& \frac{\binom{k}{l}\binom{m-l}{t-l}}{\binom{m}{t}}\\ &\le& k^{l} \cdot \frac{t}{m} \cdot \frac{t-1}{m-1} \cdot \dotsm \cdot \frac{t-l + 1}{m-l + 1} \le k^{l} \cdot \left( \frac{t}{m}\right)^{l} = \left( \frac{kt}{m}\right)^{l}, \end{array} $$

where in the first inequality in the numerator we pick l elements in the set of k and then pick the rest t − l elements from all remaining m − l elements and in the second inequality we use a simple bound $\binom {k}{l}\leq k^{l}$. □

Proof of Lemma 10

By Lemma 6, there is a constant α such that

$$\operatorname{Prob}\left[\left|Y-\frac k2\right| < \alpha \sqrt{k}\right] < \frac{1}{100}. $$

Denote

$$\begin{array}{@{}rcl@{}} A &=& \operatorname{Prob}\left[Y \leq \frac{k}{2} - \alpha \sqrt{k} \right],\\ B &=& \operatorname{Prob}\left[\frac{k}{2}- \alpha \sqrt{k} < Y < \frac{k}{2} \right],\\ C &=& \operatorname{Prob}\left[\frac{k}{2} < Y < \frac{k}{2} + \alpha \sqrt{k}\right],\\ D &=&\operatorname{Prob}\left[Y \geq \frac{k}{2} + \alpha \sqrt{k} \right]. \end{array} $$

Clearly,

$$A+B+C+D = 1$$

and (15) can be rewritten as

$$B+C < \frac{1}{100}. $$

Let us denote X = B + C. Then A + D = 1 − X and $1-X \geq \frac {99}{100}$

To prove the lemma we need to show that $C+D \geq \frac 12 + {\Omega }\left (\varepsilon \sqrt {m}\right )$.

Consider $k = \frac m2 + t$ for some t > 0. Note that Prob [Y = k] ≥ Prob [Y = m − k], so

$$C \geq B. $$

Consider any $r \geq \alpha \sqrt {k}$ and denote $l = \frac k2 + r$. Then

$$\operatorname{Prob}\left[Y = l\right] = \frac{\binom{k}{l}\binom{m-k}{t-l}}{\binom{m}{t}} . $$

On the other hand

$$\operatorname{Prob}\left[Y = k-l\right] = \frac{\binom{k}{k-l}\binom{m-k}{t-(k-l)}}{\binom{m}{t}} . $$

Then we have

$$\frac{\operatorname{Prob}\left[Y = l\right]}{\operatorname{Prob}\left[Y = k-l\right]} = \frac{\binom{m-k}{t-l}}{\binom{m-k}{t-k+l}}= \frac{\binom{m-k}{\frac m2 +\varepsilon m -\frac k2 - r}}{\binom{m-k}{\frac m2 +\varepsilon m -\frac k2 + r}} . $$

Denote N = m − k. Then

$$\begin{array}{@{}rcl@{}} \frac{\operatorname{Prob}\left[Y = l\right]}{\operatorname{Prob}\left[Y = k-l\right]} &=& \frac{\binom{N}{\frac N2 +\varepsilon m - r}}{\binom{N}{\frac N2 +\varepsilon m + r}}= \frac{(\frac N2 +\varepsilon m + r)!(\frac N2 -\varepsilon m - r)!}{(\frac N2 +\varepsilon m - r)!(\frac N2 -\varepsilon m + r)!}\\ &=&\frac{(\frac N2 +\varepsilon m - r + 1)(\frac N2 +\varepsilon m - r + 2)\ldots(\frac N2 + \varepsilon m + r)}{(\frac N2 -\varepsilon m - r + 1)(\frac N2 -\varepsilon m - r + 2)\ldots(\frac N2 - \varepsilon m + r)}\\ &\geq&\left( \frac{\frac N2 +\varepsilon m + r}{\frac N2 -\varepsilon m + r}\right)^{2r}= \left( 1 + \frac{2\varepsilon m}{\frac N2 -\varepsilon m + r}\right)^{2r} \\ &\geq&1 + \frac{4 r\varepsilon m}{\frac N2 -\varepsilon m + r} \geq 1 + c\varepsilon r \geq 1 + c^{\prime}\varepsilon \sqrt{k}, \end{array} $$

for some positive constants c and c^′.

Since this inequality holds for all $r \geq \alpha \sqrt {k}$, we have

$$D \geq A \cdot (1 + c^{\prime} \varepsilon \sqrt{k}) . $$

Thus we have

$$1- X = A+D \leq \frac{D}{1+c^{\prime} \varepsilon \sqrt{k}} + D. $$

We can rewrite

$$D \geq \frac{1+c^{\prime} \varepsilon \sqrt{k}}{2+c^{\prime} \varepsilon \sqrt{k}} (1-X) = \frac 12 (1-X) + \frac{c^{\prime} \varepsilon \sqrt{k}}{4 + 2c^{\prime} \varepsilon \sqrt{k}} (1-X). $$

Since $\varepsilon \sqrt {k}$ is positive and upper-bounded by 1, we have

$$\frac{c^{\prime} \varepsilon \sqrt{k}}{4 + 2c^{\prime} \varepsilon \sqrt{k}} = {\Theta}(\varepsilon \sqrt{k}). $$

It follows

$$A+D \geq \frac{1}{2} + {\Omega}(\varepsilon \sqrt{k}) (1-X) = \frac{1}{2} + {\Omega}\left( \varepsilon \sqrt{k}\right), $$

where in the last inequality we use that $1-X \geq \frac {99}{100}$. □