On the Reliability Function for a BSC with Noiseless Feedback at Zero Rate

Abstract

We consider the transmission of nonexponentially many messages through a binary symmetric channel with noiseless feedback. We obtain an upper bound for the best decoding error exponent. Combined with the corresponding known lower bound, this allows us to find the reliability function for this channel at zero rate.

Appendix

Proof of equation (14).

Consider an $n$-simplex code $(\boldsymbol{x}_1,\boldsymbol{x}_2,\boldsymbol{x}_3)$ with $\boldsymbol{x}_1$ having first $n/3$ ones and then $2n/3$ zeros; $\boldsymbol{x}_2$ having first $n/3$ zeros, then $n/3$ ones, and then $n/3$ zeros; and $\boldsymbol{x}_3$ having first $2n/3$ zeros and then $n/3$ ones. Then $w(\boldsymbol{x}_1)=w(\boldsymbol{x}_2)=w(\boldsymbol{x}_3)=n/3$ and $d_{12}=d_{13}=d_{23}=2n/3$. Assume that an output $\boldsymbol{y}$ has $u_1n/3$ ones in the first $n/3$ positions, $u_2n/3$ ones in the next $n/3$ positions, and $u_3n/3$ ones in the last $n/3$ positions. Then

$$\begin{gathered}d(\boldsymbol{x}_1,\boldsymbol{y})/n=(1-u_1+u_2+u_3)/3,\\ d(\boldsymbol{x}_2,\boldsymbol{y})/n=(1+u_1-u_2+u_3)/3,\\ d(\boldsymbol{x}_3,\boldsymbol{y})/n=(1+u_1+u_2-u_3)/3.\end{gathered}$$

Since $d(\boldsymbol{x}_1,\boldsymbol{y})=d(\boldsymbol{x}_2,\boldsymbol{y})=d(\boldsymbol{x}_3,\boldsymbol{y})$, we obtain $u_1=u_2=u_3$ and

$$p(\boldsymbol{y}^n\,|\, \boldsymbol{x}_1)=p^{d(\boldsymbol{x}_1,\boldsymbol{y})} q^{n-d(\boldsymbol{x}_1,\boldsymbol{y})}=q^n z^{d(\boldsymbol{x}_1,\boldsymbol{y})}= q^nz^{(1+u)n/3},\quad z=p/q<1.$$

Therefore,

$$ \begin{aligned} \operatorname{\mathbf P}\nolimits\bigl\{p(\boldsymbol{y}^n\,|\, \boldsymbol{x}_1) \approx p(\boldsymbol{y}^n\,|\, \boldsymbol{x}_2) \approx p(\boldsymbol{y}^n\,|\, \boldsymbol{x}_3)\bigr\} &\sim \max_{0\le u\le 1}\operatorname{\mathbf P}\nolimits\Bigl\{p(\boldsymbol{y}^n\,|\, \boldsymbol{x}_1) \approx q^nz^{(1+u)n/3}\Bigr\}\\ &\sim \max_{0\le u\le 1} \biggl\{\binom{n}{un}p^{(1+u)n/3}q^{(2-u)n/3}\biggr\}\\ &\sim q^n\max_{0\le u\le 1}\biggl\{\binom{n}{un}z^{(1+u)n/3}\biggr\}, \end{aligned}$$

and

$$\frac{1}{n}\max_{0\le u\le 1}\ln\operatorname{\mathbf P}\nolimits\{p(\boldsymbol{y}^n\,|\, \boldsymbol{x}_1)\}=\ln q+\max_{0\le u\le 1}g(u),$$

(59)

where

$$g(u)=h(u)+(1+u)\ln(z^{1/3}),\qquad g'(u)=\ln\frac{1-u}{u}+\ln(z^{1/3}),\qquad g''(u)<0.$$

For a maximizing $u_0$, we obtain

$$ u_0 =\frac{1}{1+z^{-1/3}}= \frac{p^{1/3}}{p^{1/3}+ q^{1/3}},$$

and after a simple algebra,

$$\ln q+g(u_0)=\ln\bigl(p^{1/3}q^{2/3}+p^{2/3}q^{1/3}\bigr).$$

(60)

Now (59) and (60) imply equations (14) and (15). △