Skip to main content
SpringerLink
Log in

Overparameterized Maximum Likelihood Tests for Detection of Sparse Vectors

  • METHODS OF SIGNAL PROCESSING
  • Published:
Problems of Information Transmission Aims and scope Submit manuscript

Abstract

We address the problem of detecting a sparse high-dimensional vector against white Gaussian noise. An unknown vector is assumed to have only \(p\) nonzero components, whose positions and sizes are unknown, the number \(p\) being on one hand large but on the other hand small as compared to the dimension. The maximum likelihood (ML) test in this problem has a simple form and, certainly, depends of \(p\). We study statistical properties of overparametrized ML tests, i.e., those constructed based on the assumption that the number of nonzero components of the vector is \(q\) (\(q>p\)) in a situation where the vector actually has only \(p\) nonzero components. We show that in some cases overparametrized tests can be better than standard ML tests.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.

References

  1. Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O., Understanding Deep Learning (Still) Requires Rethinking Generalization, Commun. ACM, 2021, vol. 64, no. 3, pp. 107–115. https://doi.org/10.1145/3446776

    Article  Google Scholar 

  2. Belkin, M., Fit without Fear: Remarkable Mathematical Phenomena of Deep Learning through the Prism of Interpolation, Acta Numer., 2021, vol. 30, pp. 203–248. https://doi.org/10.1017/S0962492921000039

    Article  MathSciNet  MATH  Google Scholar 

  3. Belkin, M., Hsu, D., and Xu, J., Two Models of Double Descent for Weak Features, SIAM J. Math. Data Sci., 2020, vol. 2, no. 4, pp. 1167–1180. https://doi.org/10.1137/20M1336072

    Article  MathSciNet  MATH  Google Scholar 

  4. Dar, Y., Muthukumar, V., and Baraniuk, R.G., A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning, https://arxiv.org/abs/2109.02355 [stat.ML], 2021.

  5. Dobrushin, R.L., A Statistical Problem Arising in the Theory of Detection of Signals in the Presence of Noise in a Multi-Channel System and Leading to Stable Distribution, Teor. Veroyatnost. i Primenen., 1958, vol. 3, no. 2, pp. 173–185 [Theory Probab. Appl. (Engl. Transl.), 1958, vol. 3, no. 2, pp. 161–173]. https://doi.org/10.1137/1103015

    MathSciNet  MATH  Google Scholar 

  6. Burnashev, M.V. and Begmatov, I.A., On a Problem of Signal Detection Leading to Stable Distributions, Teor. Veroyatnost. i Primenen., 1990, vol. 35, no. 3, pp. 557–560 [Theory Probab. Appl. (Engl. Transl.), 1990, vol. 35, no. 3, pp. 556–560]. https://doi.org/10.1137/1135076

    MathSciNet  MATH  Google Scholar 

  7. Ingster, Yu.I. and Suslina, I.A., Nonparametric Goodness-of-Fit Testing Under Gaussian Models, Lect. Notes Statist., vol. 169. New York: Springer-Verlag, 2003. https://doi.org/10.1007/978-0-387-21580-8

  8. Bonferroni, C.E., Teoria statistica delle classi e calcolo delle probabilità, Pubbl. del R. Ist. Super. di Sci. Econ. e Commer. di Firenze, vol. 8, Firenze: Seeber, 1936.

    MATH  Google Scholar 

  9. Benjamini, Y. and Hochberg, Y., Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing, J. Roy. Statist. Soc. Ser. B, 1995, vol. 57, no. 1, pp. 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

    MathSciNet  MATH  Google Scholar 

  10. Benjamini, Y., Simultaneous and Selective Inference: Current Successes and Future Challenges, Biom. J., 2010, vol. 52, no. 6, pp. 708–721. https://doi.org/10.1002/bimj.200900299

    Article  MathSciNet  MATH  Google Scholar 

  11. Donoho, D. and Jin, J., Higher Criticism Thresholding: Optimal Feature Selection When Useful Features are Rare and Weak, Proc. Natl. Acad. Sci. U.S.A., 2008, vol. 105, no. 39, pp. 14790–14795. https://doi.org/10.1073/pnas.0807471105

    Article  MATH  Google Scholar 

  12. Anderson, T.W., The Integral of a Symmetric Unimodal Function over a Symmetric Convex Set and Some Probability Inequalities, Proc. Amer. Math. Soc., 1955, vol. 6, no. 2, pp. 170–176. https://doi.org/10.1090/S0002-9939-1955-0069229-1

    Article  MathSciNet  MATH  Google Scholar 

  13. Ibragimov, I.A. and Khas’minskii, R.Z., Asimptoticheskaya teoriya otsenivaniya, Moscow: Nauka, 1979. Translated under the title Statistical Estimation. Asymptotic Theory, New York: Springer, 1981.

    MATH  Google Scholar 

  14. Pyke, R., Spacings, J. Roy. Statist. Soc. Ser. B, 1965, vol. 27, no. 3, pp. 395–436; 437–449 (discussion). https://doi.org/10.1111/j.2517-6161.1965.tb00602.x; https://doi.org/10.1111/j.2517-6161.1965.tb00603.x

    MathSciNet  MATH  Google Scholar 

  15. Fedoryuk, M.V., Asimptotika: Integraly i ryady (Asymptotics: Integrals and Series), Moscow: Nauka, 1987.

    MATH  Google Scholar 

Download references

Acknowledgments

The author is grateful to an anonymous reviewer for his comments, which helped to improve the paper.

Author information

Authors and Affiliations

  1. Kharkevich Institute for Information Transmission Problems of the Russian Academy of Sciences, Moscow, Russia

    G. K. Golubev

Authors
  1. G. K. Golubev
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Translated from Problemy Peredachi Informatsii, 2023, Vol. 59, No. 1, pp. 46–63. https://doi.org/10.31857/S0555292323010047

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Golubev, G.K. Overparameterized Maximum Likelihood Tests for Detection of Sparse Vectors. Probl Inf Transm 59, 41–56 (2023). https://doi.org/10.1134/S0032946023010040

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0032946023010040

Keywords

Search

Navigation