Quantum Limits on the Entropy of Bandlimited Radiation

Abstract

Physical limits on the amount of information carried by bandlimited waveforms radiated in one and three dimensions are considered. It is shown that the entropy of radiation can achieve the Bekenstein bound using a “burst” of energy, whose density vanishes as the radiating system expands. In comparison, black body radiation of infinite bandwidth achieves the same entropy scaling, that is proportional to the volume of the space, but requires an energy density that remains constant as the system expands. Rather than following the standard statistical physics approach of counting the number of eigenstates of the Hamiltonian of the quantum wave field, our derivation first considers an optimal subspace approximation, and then determines the number of bits that are required to represent any waveform in the space spanned by this representation with a minimum quantized energy error. This favors a geometric interpretation where the complexity of state counting is replaced by the one of determining the minimum cardinality covering of the signal space by high-dimensional balls, or boxes, whose size is lower bounded by quantum constraints. All derivations are given for both deterministic and stochastic settings.

Appendix

The computation of the entropy of a continuous random variable quantized at fixed resolution is standard [29]. Here, we adapt this computation to a quantized resolution that grows sub-linearly, as indicated in Fig. 2. Consider a positive random variable ${{\mathrm{\mathsf {A}}}}$ distributed according to a continuous density g(a). By continuity, for all k we let a(k) be a value such that

$$\begin{aligned} g[a(k)] (\sqrt{k+1}-\sqrt{k}) \sigma = \int _{\sqrt{k} \sigma }^{\sqrt{k+1} \sigma } g(a) da. \end{aligned}$$

(80)

The quantized random variable ${{\mathrm{\mathsf {A}}}}^{\sigma }$ is defined as

$$\begin{aligned} {{\mathrm{\mathsf {A}}}}^{\sigma } = a(k) \;\; \text{ if } \;\; \sqrt{k} \sigma \le {{\mathrm{\mathsf {A}}}}< \sqrt{k+1} \sigma . \end{aligned}$$

(81)

We compute the Shannon entropy

$$\begin{aligned} H_{{{\mathrm{\mathsf {A}}}}^\sigma }&= - \sum _{k=0}^{\infty } g[a(k)] (\sqrt{k+1}-\sqrt{k}) \sigma \log [ g(a(k)) (\sqrt{k+1}-\sqrt{k}) \sigma ] \nonumber \\&= - \sum _{k=0}^{\infty } g[a(k)] (\sqrt{k+1}-\sqrt{k}) \sigma \log [g(a(k)) (\sqrt{k+1}-\sqrt{k}) ] \nonumber \\&\quad - \sum _{k=0}^{\infty } g[a(k)] (\sqrt{k+1}-\sqrt{k}) \sigma \log \sigma \nonumber \\&= - \sum _{k=0}^{\infty } g[a(k)] (\sqrt{k+1}-\sqrt{k}) \sigma \log [g(a(k)) (\sqrt{k+1}-\sqrt{k})] \nonumber \\&\quad - \log \sigma \sum _{k=0}^{\infty } \int _{\sqrt{k} \sigma }^{\sqrt{k+1} \sigma } g(a) da \nonumber \\&= - \sum _{k=0}^{\infty } g[a(k)] (\sqrt{k+1}-\sqrt{k}) \sigma \log [g(a(k)) (\sqrt{k+1}-\sqrt{k})] - \log \sigma . \end{aligned}$$

(82)

For small values of $\sigma$, the first term approaches a Riemann integral that coincides with the differential entropy of $\mathsf {A}$, and we have

$$\begin{aligned} \lim _{\sigma \rightarrow 0} (H_{{{\mathrm{\mathsf {A}}}}^\sigma }+ \log \sigma ) = h_{{{\mathrm{\mathsf {A}}}}}. \end{aligned}$$

(83)

It now follows that by drawing $N_0$ independent zero mean Gaussian random variables of variance P, the entropy of the quantized sequence is, for small values of $\sigma$, approximated by

$$\begin{aligned} H&= N_0 H_{{{\mathrm{\mathsf {A}}}}^\sigma } \nonumber \\&= N_0 (h_{{{\mathrm{\mathsf {A}}}}} - \log \sigma ) \nonumber \\&= N_0 \log \left( \frac{\sqrt{2 \pi e P}}{\sigma } \right) . \end{aligned}$$

(84)