Spin Systems on Bethe Lattices

Abstract

In an extremely influential paper Mézard and Parisi put forward an analytic but non-rigorous approach called the cavity method for studying spin systems on the Bethe lattice, i.e., the random d-regular graph Mézard and Parisi (Eur Phys J B 20:217–233, 2001). Their technique was based on certain hypotheses; most importantly, that the phase space decomposes into a number of Bethe states that are free from long-range correlations and whose marginals are given by a recurrence called Belief Propagation. In this paper we establish this decomposition rigorously for a very general family of spin systems. In addition, we show that the free energy can be computed from this decomposition. We also derive a variational formula for the free energy. The general results have interesting ramifications on several special cases.

Appendices

Appendix A. Proof of Lemmas 3.6 and 7.12

The proof of Lemma 3.6 requires the regularity lemma for measures from [24].³ Let $\lambda$ denote the Lebesgue measure. For $\mu \in \mathscr {K}$ and measurable $S,X\subset [0,1]$ we write

$$\begin{aligned} \mu _{S,X}=\frac{1}{\lambda (S)\lambda (X)}\int _S\int _X\mu _{s,x}{\mathrm d}x{\mathrm d}s\in \mathscr {P}(\Omega ), \end{aligned}$$

with the convention that $\mu _{S,X}$ is uniform if $\lambda (S)\lambda (X)=0$. Further, let $\varvec{X}=(X_1,\ldots ,X_K),\varvec{S}=(S_1,\ldots ,S_L)$ be a partitions of [0, 1) into pairwise disjoint measurable sets. We write $\#\varvec{X},\#\varvec{S}$ for the number K, L of classes, respectively. Then $\mu$ is $\varepsilon$-regular with respect to $(\varvec{X},\varvec{S})$ if there exists $R\subset [\#\varvec{X}]\times [\#\varvec{S}]$ such that the following conditions hold.

REG1::

$\lambda (X_i)>0$ and $\lambda (S_j)>0$ for all $(i,j)\in R$.

REG2::

$\sum _{(i,j)\in R}\lambda (X_i)\lambda (S_j)>1-\varepsilon$.

REG3::

for all $(i,j)\in R$ and almost all $s,s'\in S_j$ we have $\Vert {\int _{X_i}\mu _{s,x}-\mu '_{s',x}{\mathrm d}x}\Vert _{\mathrm {TV}}<\varepsilon \lambda (X_i)$.

REG4::

if $(i,j)\in R$, then for every $U\subset X_i$ with $\lambda (U)\ge \varepsilon \lambda (X_i)$ and every $T\subset S_j$ with $\lambda (T)\ge \varepsilon \lambda (S_j)$ we have

$$\begin{aligned} \left\| {\mu _{S,X_i}-\mu _{T,U}}\right\| _{\mathrm {TV}}<\varepsilon . \end{aligned}$$

A refinement of a partition $(\varvec{X},\varvec{S})$ is a partition $(\varvec{X}',\varvec{S}')$ such that for every pair $(i',j')\in [\#\varvec{X}']\times [\varvec{S}']$ there is a pair $(i,j)\in [\#\varvec{X}]\times [\varvec{S}]$ such that $(X_{i'}',S_{j'}')\subset (X_i,S_j)$.

Theorem A.1

([24]). For any $\varepsilon >0$ there exists $N=N(\varepsilon ,\Omega )$ such that for every $\mu \in \mathscr {K}$ the following is true. Every partition $(\varvec{X}_0,\varvec{S}_0)$ with $\#\varvec{X}_0+\#\varvec{S}_0\le 1/\varepsilon$ has a refinement $(\varvec{X},\varvec{S})$ such that $\#\varvec{X}+\#\varvec{S}\le N$ with respect to which $\mu$ is $\varepsilon$-regular.

Additionally, we need the strong cut metric, defined by

$$\begin{aligned} D_{\Box }(\mu ,\nu )&=\sup _{S,X,\omega }\left| {\int _S\int _X\mu _{s,x}(\omega )-\nu _{s,x}(\omega ){\mathrm d}x{\mathrm d}s}\right| \qquad (\mu ,\nu \in \mathscr {K}), \end{aligned}$$

where S, X range over measurable subsets of the unit interval and $\omega \in \Omega$. It is well known that $D_{\Box }(\,\cdot \,,\,\cdot \,)$ induces a metric on $\mathscr {K}$.

For $\mu ,\nu \in \mathscr {K}$ we define $\mu \oplus \nu :[0,1]^3\rightarrow \mathscr {P}(\Omega ^2)$ by $\mu \oplus \nu _{s,x_1,x_2}=\mu _{s,x_1}\otimes \mu _{s,x_2}$. Since $[0,1]^2$ with the Lebesgue measure is isomorphic as a measure space to [0, 1] with the Lebesgue measure, we can view $\mu \oplus \nu$ as a strong $\mathscr {P}(\Omega ^2)$-valued kernel. In particular, it makes sense to apply the strong cut metric to these kernels.

Proposition A.2

The map $(\mu ,\nu )\mapsto \mu \oplus \nu$ is continuous with respect to the strong cut metric.

Proof

Given $\varepsilon >0$ pick a small enough $\delta >0$ and assume that $D_{\Box }(\mu ,\mu ')<\delta$. Due to the triangle inequality it suffices to prove that $D_{\Box }(\mu \oplus \nu ,\mu '\oplus \nu )<\varepsilon$ for every $\nu$. Thus, we need to show that for any $X\subset [0,1]^2$, $S\subset [0,1]$ and $\sigma ,\tau \in \Omega$,

$$\begin{aligned} \left| {\int _X\int _S\left( {\mu _{s,x_1}(\sigma )-\mu '_{s,x_1}(\sigma )}\right) \nu _{s,x_2}(\tau ){\mathrm d}s{\mathrm d}x_1{\mathrm d}x_2}\right| <\varepsilon . \end{aligned}$$

(A.1)

To this end, we may assume that $\lambda (S)>\varepsilon ^2$ and that $\int _S\nu _{s,x_2}(\tau ){\mathrm d}s>\varepsilon ^2$ for all $(x_1,x_2)\in X$. Further, with $z=\int _0^1\nu _{s,x_2}(\tau ){\mathrm d}s$ consider the variable transformation

$$\begin{aligned} {\mathrm d}t=\frac{\nu _{s,x_2}(\tau ){\mathrm d}s}{z}. \end{aligned}$$

(A.2)

Let T be the inverse image of S under the transformation (A.2). Then we obtain for any $X_1\subset [0,1]$,

$$\begin{aligned} \int _{X_1}\int _S\left( {\mu _{s,x_1}(\sigma )-\mu '_{s,x_1}(\sigma )}\right) \nu _{s,x_2}(\tau ){\mathrm d}s{\mathrm d}x_1&=z\int _{X_1}\int _T\mu _{t,x_1}(\sigma )-\mu '_{t,x_1}(\sigma ){\mathrm d}t{\mathrm d}x_1. \end{aligned}$$

(A.3)

But the assumption $D_{\Box }(\mu ,\mu ')<\delta$ implies that the double integral on the r.h.s. of (A.3) is bounded by $\varepsilon ^4$ in absolute value (providing $\delta$ is small enough). Thus, (A.1) follows.

$\square$

Proof of Lemma 3.6

We may assume without loss that $f(\tau )=\varvec{1}\{\tau =\sigma \}$ for some $\sigma \in \Omega ^k$. Let $\varepsilon >0$, pick $\alpha =\alpha (\varepsilon )$, $\xi =\xi (\alpha )>0$ small enough and assume that $\mu ,\nu \in \mathscr {K}$ are such that $D_{\Box }(\mu ,\nu )<\delta$ for a small enough $\delta =\delta (\xi )>0$. Applying Theorem A.1 twice, we obtain $(\varvec{X},\varvec{S})$ with respect to which both $\mu ,\nu$ are $\xi$-regular, and $L=\#\varvec{X}+\#\varvec{S}$ is bounded in terms of $\xi$ only. Let $R'$ be the set of all pairs for which REG1–REG4 are satisfied for both $\mu ,\nu$ and that satisfy $\lambda (\varvec{X}_i,\varvec{S}_j)>\xi ^8/L$. Assuming that $\delta$ is sufficiently small, we obtain

$$\begin{aligned} |\mu _{S_i,X_j}-\nu _{S_i,X_j}|<\xi ^8\qquad \text{ for } \text{ all } (i,j)\in R'\text{. } \end{aligned}$$

(A.4)

Furthermore, consider the random variables

$$\begin{aligned} z_i&=\prod _{h=1}^k\mu _{S_i,\varvec{x}_h}(\sigma _h),&z&=\sum _{i\le \#\varvec{S}}z_i,\\ z_i'&=\prod _{h=1}^k\nu _{S_i,\varvec{x}_h}(\sigma _h),&z'&=\sum _{i\le \#\varvec{S}}z_i' \end{aligned}$$

and define $\mu ',\nu '\in \mathscr {K}$ as follows. To construct $\mu '$, partition the interval [0, 1] into pairwise disjoint sets $T_i$, $i\in [\#\varvec{S}]$, of measure $z_i/z$ and fill the strip $T_i\times [0,1]$ with a suitably scaled copy of $(\mu _{s,x})_{s\in S_i,x\in [0,1]}$. Construct $\nu '$ analogously from the $z_i'$. Then $\mathscr {D}_{\Box }(\mu ',f*\mu )=\mathscr {D}_{\Box }(\nu ',f*\nu )=0$. Furthermore, Proposition A.2 shows that with probability at least $1-\alpha$ we have

$$\begin{aligned} \sum _{i=1}^{\#\varvec{S}}\lambda (S_i)|z_i-z_i'|<\alpha ^2, \end{aligned}$$

provided that $\xi ,\delta$ are chosen small enough. Since also $z\ge \alpha$ because the function f is strictly positive, we conclude that with probability at least $1-\alpha$ we have $\mathscr {D}_{\Box }(\mu ',\nu ')<\alpha$. We thus obtain a coupling of the random variables $f*\mu ,f*\nu$ under which the expected cut distance is bounded by $\varepsilon$, as desired. $\quad \square$

Proof of Lemma 7.12

We proceed precisely as in the proof of Lemma 3.6, up until the point where the positivity of f is used. In the setup of Lemma 7.12, the function f may take the value 0 on kernels that take the value 1 with positive probability; however, since we are assuming that the values of the kernels are bounded by $\lambda /(1+\lambda )$. Therefore, the function f always attains values that are bounded away from 0. $\quad \square$

Appendix B. Proof of Lemma 3.3

The proof of Lemma 3.3 requires the following operation. For functions $f:\Omega ^{M\times N}\rightarrow \mathbb {R}$, $g:\Omega ^{L\times N}\rightarrow \mathbb {R}$ we define

$$\begin{aligned} f\otimes g&:\Omega ^{(M+L)\times N}\rightarrow \mathbb {R},&\sigma&\mapsto f\left( {(\sigma _{i,j})_{i\in [M],j\in [N]}}\right) \cdot g\left( {(\sigma _{i+M,j+N})_{i\in [L],j\in [N]}}\right) . \end{aligned}$$

Thus, the first M rows of $\sigma$ go into f, the last L rows go into g and we multiply the results.

We define a corresponding operation on kernels. Namely, for $\mu ,\nu \in \mathscr {K}$ we define $\mu \otimes \nu :[0,1]^3\rightarrow \mathscr {P}(\Omega ^2)$ by $\mu \oplus \nu _{s,t,x}=\mu _{s,x}\otimes \nu _{t,x}$. Since $([0,1]^2,\lambda \otimes \lambda )$ is isomorphic $([0,1],\lambda )$, we can view $\mu \otimes \nu$ as a $\mathscr {P}(\Omega ^2)$-valued kernel, and the cut metric extends to these kernels. Since the cut metric is invariant under swapping the axes, Proposition A.2 readily yields the following.

Proposition B.1

The map $(\mu ,\nu )\mapsto \mu \otimes \nu$ is continuous with respect to the cut metric.

As a final preparation toward the proof of Lemma 3.3 we need the following fact.

Lemma B.2

For any $f:\Omega \rightarrow \mathbb {R}$ the map $\mu \in \mathfrak {K}\mapsto \mathbb {E}\left\langle {{f},{\mu }}\right\rangle$ is continuous.

Proof

We may assume without loss that $f(\tau )=\varvec{1}\{\sigma =\tau \}$ for some $\sigma \in \Omega$. Then

$$\begin{aligned} \mathbb {E}\left\langle {{f},{\mu }}\right\rangle =\int _0^1\int _0^1\mu _{s,x}(\sigma ){\mathrm d}x{\mathrm d}s, \end{aligned}$$

and it is immediate from the definition of the cut metric that the integral on the right hand side is a continuous function of $\mu$. $\quad \square$

Proof of Lemma 3.3

Let $f:\Omega ^{m\times n}\rightarrow \mathbb {R}$ and let $\mu \in \mathfrak {K}$. Define $\nu =(\mu ^{\oplus n})^{\otimes m}$. Then $\nu$ is a kernel with values in $\Omega ^{mn}$ and the definition of $\left\langle {{\,\cdot \,},{\,\cdot \,}}\right\rangle$ ensures that $\mathbb {E}\left\langle {{f},{\mu }}\right\rangle =\mathbb {E}\left\langle {{f},{\nu }}\right\rangle$. This already shows that the map $\mu \mapsto \mathbb {E}\left\langle {{f},{\mu }}\right\rangle$ is continuous, because the map $\mu \mapsto \nu$ is continuous by Proposition A.2 and B.1 and the map $\nu \mapsto \mathbb {E}\left\langle {{f},{\nu }}\right\rangle$ is continuous by Lemma B.2. Now fix an integer $\ell \ge 2$ and let $\eta =\nu ^{\otimes \ell }$. Then

$$\begin{aligned} \mathbb {E}\left[{\left\langle {{f},{\mu }}\right\rangle ^\ell }\right]=\mathbb {E}\left[{\left\langle {{f},{\eta }}\right\rangle }\right] \end{aligned}$$

and thus the continuity of the map $\mu \mapsto \mathbb {E}\left[{\left\langle {{f},{\mu }}\right\rangle ^\ell }\right]$ follows from Proposition B.1 and Lemma B.2. $\quad \square$