Skip to main content

Distributed Noise-Shaping Quantization: I. Beta Duals of Finite Frames and Near-Optimal Quantization of Random Measurements

Abstract

This paper introduces a new algorithm for the so-called “analysis problem” in quantization of finite frame representations that provides a near-optimal solution in the case of random measurements. The main contributions include the development of a general quantization framework called distributed noise shaping and, in particular, beta duals of frames, as well as the performance analysis of beta duals in both deterministic and probabilistic settings. It is shown that for random frames, using beta duals results in near-optimally accurate reconstructions with respect to both the frame redundancy and the number of levels at which the frame coefficients are quantized. More specifically, for any frame E of m vectors in \(\mathbb {R}^k\) except possibly from a subset of Gaussian measure exponentially small in m and for any number \(L \ge 2\) of quantization levels per measurement to be used to encode the unit ball in \(\mathbb {R}^k\), there is an algorithmic quantization scheme and a dual frame together that guarantee a reconstruction error of at most \(\sqrt{k}L^{-(1-\eta )m/k}\), where \(\eta \) can be arbitrarily small for sufficiently large problems. Additional features of the proposed algorithm include low computational cost and parallel implementability.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    As is common, our convention for \(\varepsilon \)-nets is synonymous to \(\varepsilon \)-coverings: \(\mathcal N\) is an \(\varepsilon \)-net for \(\mathscr {X}\) if for all \(x \in \mathscr {X}\) there exists \(y \in \mathcal {N}\) with \(\Vert x-y\Vert _2\le \varepsilon \).

  2. 2.

    Similar exponential error bounds that have been obtained previously in the case of conventional sigma-delta modulation or for other quantization schemes are not compatible with the results of this paper: The results of [17] and [11] are for a fixed frame-like system, but using a different norm in infinite dimensions, and the dependence on L is unavailable. The results in [12], obtained in yet another functional setting for sigma-delta modulation, come close to being optimal; however, these results were obtained under modeling assumptions on the quantization noise and circuit stability. The exponential near-entropic error decay in the bitrate obtained in [19] combine sigma-delta modulation with further (lossy) bit encoding. Finally, the exponential error decay reported in [1] is obtained with adaptive hyperplane partitions and does not correspond to linear reconstruction.

  3. 3.

    The following example shows that the minimum value can be strictly smaller: let \(H = \left[ \begin{matrix}1&{}\quad 0\\ -1&{}\quad 1\end{matrix}\right] \), \(E =\left[ \begin{matrix}1\\ 1\end{matrix}\right] \) for which \(\sqrt{m}/\sigma _{\mathrm{min}}(H^{-1}E) = \sqrt{2/5}\). Meanwhile, \(V = \left[ \begin{matrix}1&\quad 1 \end{matrix}\right] \) yields \(\sqrt{p}\Vert VH\Vert _{\infty \rightarrow \infty }/\sigma _{\mathrm{min}}(VE) = 1/2\).

References

  1. 1.

    Baraniuk, R.G., Foucart, S., Needell, D., Plan, Y., Wootters, M.: Exponential decay of reconstruction error from binary measurements of sparse signals. CoRR. arXiv:1407.8246 (2014)

  2. 2.

    Benedetto, J.J., Powell, A.M., Yılmaz, Ö.: Second-order sigma-delta \((\Sigma \Delta )\) quantization of finite frame expansions. Appl. Comput. Harmon. Anal. 20(1), 126–148 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Benedetto, J.J., Powell, A.M., Yılmaz, Ö.: Sigma-delta quantization and finite frames. IEEE Trans. Inf. Theory 52(5), 1990–2005 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Blum, J., Lammers, M., Powell, A.M., Yılmaz, Ö.: Sobolev duals in frame theory and sigma-delta quantization. J. Fourier Anal. Appl. 16(3), 365–381 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Bodmann, B.G., Paulsen, V.I.: Frame paths and error bounds for sigma-delta quantization. Appl. Comput. Harmon. Anal. 22(2), 176–197 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Chazelle, B.: The Discrepancy Method: Randomness and Complexity. Cambridge University Press, Cambridge (2002)

    MATH  Google Scholar 

  7. 7.

    Dajani, K., Kraaikamp, C.: Ergodic Theory of Numbers, Volume 29 of Carus Mathematical Monographs. Mathematical Association of America, Washington (2002)

    MATH  Google Scholar 

  8. 8.

    Daubechies, I., DeVore, R.A., Güntürk, C.S., Vaishampayan, V.A.: A/D conversion with imperfect quantizers. IEEE Trans. Inf. Theory 52(3), 874–885 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Davidson, K.R., Szarek, S.J.: Local operator theory, random matrices and Banach spaces. In: Johnson, W.B., Lindenstrauss, J. (eds.) Handbook of the Geometry of Banach Spaces, vol. 1, pp. 317–366. North-Holland, Amsterdam (2001)

  10. 10.

    Davidson, K.R., Szarek, S.J.: Addenda and corrigenda to: local operator theory, random matrices and Banach spaces. In: Johnson, W.B., Lindenstrauss, J. (eds.) Handbook of the Geometry of Banach Spaces, vol. 2, pp. 1819–1820. North-Holland, Amsterdam (2003)

  11. 11.

    Deift, P., Güntürk, C.S., Krahmer, F.: An optimal family of exponentially accurate one-bit sigma-delta quantization schemes. Commun. Pure Appl. Math. 64(7), 883–919 (2011)

  12. 12.

    Derpich, M.S., Silva, E.I., Quevedo, D.E., Goodwin, G.C.: On optimal perfect reconstruction feedback quantizers. IEEE Trans. Signal Process. 56(8, part 2), 3871–3890 (2008)

    MathSciNet  Article  Google Scholar 

  13. 13.

    Edelman, A.: Eigenvalues and condition numbers of random matrices. SIAM J. Matrix Anal. Appl. 9(4), 543–560 (1988)

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Galton, I., Jensen, H.T.: Oversampling parallel delta-sigma modulator A/D conversion. IEEE Trans. Circuits Syst. II Analog Digit. Signal Process. 43(12), 801–810 (1996)

    Article  Google Scholar 

  15. 15.

    Goyal, V.K., Vetterli, M., Thao, N.T.: Quantized overcomplete expansions in \(\mathbb{R}^N\): analysis, synthesis, and algorithms. IEEE Trans. Inf. Theory 44(1), 16–31 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Güntürk, C.S., Lammers, M., Powell, A.M., Saab, R., Yılmaz, Ö.: Sobolev duals for random frames and \(\Sigma \Delta \) quantization of compressed sensing measurements. Found. Comput. Math. 13(1), 1–36 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Güntürk, C.S.: One-bit sigma-delta quantization with exponential accuracy. Commun. Pure Appl. Math. 56(11), 1608–1630 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Güntürk, C.S.: Mathematics of analog-to-digital conversion. Commun. Pure Appl. Math. 65(12), 1671–1696 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  19. 19.

    Iwen, M., Saab, R.: Near-optimal encoding for sigma-delta quantization of finite frame expansions. J. Fourier Anal. Appl. 19(6), 1255–1273 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    Krahmer, F., Saab, R., Yılmaz, Ö.: Sigma-delta quantization of sub-gaussian frame expansions and its application to compressed sensing. Inf. Inference 3(1), 40–58 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Lammers, M., Powell, A.M., Yılmaz, Ö.: Alternative dual frames for digital-to-analog conversion in sigma-delta quantization. Adv. Comput. Math. 32(1), 73–102 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Lammers, M.C., Powell, A.M., Yılmaz, Ö.: On quantization of finite frame expansions: sigma-delta schemes of arbitrary order. In: Proceedings of SPIE 6701, Wavelets XII, 670108, vol. 6701, pp. 670108–670108-9 (2007)

  23. 23.

    Matoušek, J.: Geometric Discrepancy, Volume 18 of Algorithms and Combinatorics. Springer, Berlin (1999). (an illustrated guide)

    Google Scholar 

  24. 24.

    Molino, V.: Approximation by Quantized Sums. Ph.D. thesis, New York University (2012)

  25. 25.

    Parry, W.: On the \(\beta \)-expansions of real numbers. Acta Math. Acad. Sci. Hung. 11, 401–416 (1960)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Powell, A.M., Saab, R., Yılmaz, Ö.: Quantization and finite frames. In: Casazza, P.G., Kutyniok, G. (eds.) Finite Frames, pp. 267–302. Springer, New York (2013)

  27. 27.

    Rudelson, M., Vershynin, R.: Smallest singular value of a random rectangular matrix. Commun. Pure Appl. Math. 62(12), 1707–1739 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  28. 28.

    Rudelson, M., Vershynin, R.: Non-asymptotic theory of random matrices: extreme singular values. In: Bhatia, R., Pal, A., Rangarajan, G., Srinivas, V., Vanninathan, M. (eds.) Proceedings of the International Congress of Mathematicians, Vol. III, pp. 1576–1602. Hindustan Book Agency, New Delhi (2010)

  29. 29.

    Thao, N.T., Vetterli, M.: Lower bound on the mean-squared error in oversampled quantization of periodic signals using vector quantization analysis. IEEE Trans. Inf. Theory 42(2), 469–479 (1996)

    Article  MATH  Google Scholar 

  30. 30.

    Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. In: Eldar, Y.C., Kutyniok, G. (eds.) Compressed Sensing, pp. 210–268. Cambridge University Press, Cambridge (2012)

Download references

Acknowledgments

The authors would like thank Thao Nguyen, Rayan Saab and Özgür Yılmaz for the useful conversations on the topic of this paper and the anonymous referees for the valuable comments and the references they have brought to our attention.

Author information

Affiliations

Authors

Corresponding author

Correspondence to C. Sinan Güntürk.

Additional information

Communicated by Joel A. Tropp.

A Appendix

A Appendix

Lemma A.1

Let \(\xi \sim \mathcal {N}(0,I_l)\). For any \(0 < \varepsilon \le 1\), we have

$$\begin{aligned} \mathbb {P}\left( \Vert \xi \Vert _2 \le \varepsilon \sqrt{l} \right) \le \varepsilon ^{l} \mathrm{e}^{(1-\varepsilon ^2)l/2}. \end{aligned}$$

Proof

For any \(t \ge 0\), we have

$$\begin{aligned} \mathbb {P}\left( \Vert \xi \Vert ^2_2 \le \varepsilon ^2 l \right)\le & {} \int _{\mathbb {R}^l} \hbox {e}^{(\varepsilon ^2 l - \Vert x\Vert _2^2)t/2} \frac{\hbox {e}^{-\Vert x\Vert _2^2/2}}{(2\pi )^{l/2}} \,\mathrm {d}x \\= & {} \hbox {e}^{\varepsilon ^2 t l/2} \int _{\mathbb {R}^l} \frac{\hbox {e}^{-(1+t)\Vert x\Vert _2^2/2}}{(2\pi )^{l/2}} \,\mathrm {d}x = \left( \frac{\hbox {e}^{\varepsilon ^2 t}}{1+t} \right) ^{l/2}. \end{aligned}$$

Choosing \(t = \varepsilon ^{-2}-1\) yields the desired bound. \(\square \)

Proof of Theorem 4.3

For an arbitrary \(\tau > 1\), let \(\mathcal {E}_1\) be the event \(\{ \Vert \Omega \Vert _{2\rightarrow 2} \le 2\tau \sqrt{l}\}\). Proposition 4.1 with \(m=l\), \(E=\Omega \), \(t=2(\tau -1)\sqrt{l}\) implies

$$\begin{aligned} \mathbb {P}(\mathcal {E}_1^c) \le \hbox {e}^{-2(\tau -1)^2 l}. \end{aligned}$$

Next, consider a \(\rho \)-net Q of the unit sphere of \(\mathbb {R}^k\) with \(|Q| \le 2k(1+2/\rho )^{k-1}\) (see [27, Proposition 2.1]), where we set \(\rho = \varepsilon /(4\tau )\). Let \(\mathcal {E}_2\) be the event \(\{ \Vert \Omega z\Vert _2 \ge \varepsilon \sqrt{l},\ \forall z \in Q\}\). For each fixed \(z\in \mathbb {R}^k\) with unit norm, \(\Omega z\) has entries that are i.i.d. \(\mathcal {N}(0,1)\). By Lemma A.1, we have

$$\begin{aligned} \mathbb {P}(\mathcal {E}_2^c) \le |Q| \varepsilon ^{l} \hbox {e}^{(1-\varepsilon ^2)l/2} \le 2k(\varepsilon + 8 \tau )^{k-1} \varepsilon ^{l-k+1} \hbox {e}^{(1-\varepsilon ^2)l/2}. \end{aligned}$$

Suppose the event \(\mathcal {E}_1 \cap \mathcal {E}_2\) occurs. For any unit norm \(x \in \mathbb {R}^k\), there exists a \(z \in Q\) such that \(\Vert x-z\Vert _2 \le \rho \). Then \(\Vert \Omega (x-z)\Vert _2 \le 2\tau \rho \sqrt{l} = \varepsilon \sqrt{l}/2\) and \(\Vert \Omega z\Vert _2 \ge \varepsilon \sqrt{l}\), so that

$$\begin{aligned} \Vert \Omega x\Vert _2 \ge \Vert \Omega z\Vert _2 - \Vert \Omega (x-z)\Vert _2 \ge \varepsilon \sqrt{l}/2, \end{aligned}$$

hence \(\sigma _{\mathrm{min}}(\Omega ) \ge \varepsilon \sqrt{l}/2\). It follows that \(\mathcal {F}:=\left\{ \sigma _{\mathrm{min}}(\Omega ) \le \varepsilon \sqrt{l}/2 \right\} \subset \mathcal {E}_1^c \cup \mathcal {E}_2^c\), and therefore

$$\begin{aligned} \mathbb {P}\left( \mathcal {F}\right) \le \hbox {e}^{-2(\tau -1)^2 l} + 2k(\varepsilon + 8 \tau )^{k-1} \varepsilon ^{l-k+1} \hbox {e}^{l/2}. \end{aligned}$$

We still have the freedom to choose \(\tau > 1\) as a function of \(\varepsilon \), l, and k. For simplicity, we choose \(\tau = 1+\sqrt{\log \varepsilon ^{-1}}\), so that \(\hbox {e}^{-2(\tau -1)^2 l} = \varepsilon ^{2l}\). Noting that \(1 + k(1+8\tau )^{k-1} < (2+8\tau )^k\), we obtain

$$\begin{aligned} \mathbb {P}\left( \mathcal {F}\right)< \varepsilon ^{l-k+1} \left( 1 + 2k(1 + 8\tau )^{k-1} \hbox {e}^{l/2}\right) < 2 \left( 10+8\sqrt{\log \varepsilon ^{-1}}\right) ^k \hbox {e}^{l/2} \varepsilon ^{l-k+1}. \end{aligned}$$

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chou, E., Güntürk, C.S. Distributed Noise-Shaping Quantization: I. Beta Duals of Finite Frames and Near-Optimal Quantization of Random Measurements. Constr Approx 44, 1–22 (2016). https://doi.org/10.1007/s00365-016-9344-4

Download citation

Keywords

  • Finite frames
  • Quantization
  • Random matrices
  • Noise shaping
  • Beta encoding

Mathematics Subject Classification

  • 41A29
  • 94A29
  • 94A20
  • 42C15
  • 15B52