Skip to main content
SpringerLink
Log in

Detection in High Dimensions

  • Chapter
  • First Online:
Cognitive Networked Sensing and Big Data

  • 2230 Accesses

Abstract

This chapter is the core of Part II: Applications.

Detection in high dimensions is fundamentally different from the traditional detection theory. Concentration of measure plays a central role due to the high dimensions. We exploit the bless of dimensions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This is not guaranteed. The probability that the random matrix X is singular is studied in the literature [241, 242].

  2. 2.

    We use [n] to denote the set =\(\left \{1,\ldots,n\right \}\).

Bibliography

  1. R. Qiu, Z. Hu, H. Li, and M. Wicks, Cognitiv Communications and Networking: Theory and Practice. John Wiley and Sons, 2012.

    Google Scholar 

  2. F. Zhang, Matrix Theory. Springer Ver, 1999.

    Google Scholar 

  3. N. J. Higham, Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, 2008.

    Google Scholar 

  4. R. Bhatia, Matrix analysis. Springer, 1997.

    Google Scholar 

  5. S. Boyd and L. Vandenberghe, Convex optimization. Cambridge Univ Pr, 2004.

    Google Scholar 

  6. J. Tropp, “User-friendly tail bounds for sums of random matrices,” Foundations of Computational Mathematics, vol. 12, no. 4, pp. 389–434, 2011.

    Article  MathSciNet  Google Scholar 

  7. P. Billingsley, Probability and Measure. Wiley, 2008.

    Google Scholar 

  8. R. Vershynin, “Introduction to the non-asymptotic analysis of random matrices,” Arxiv preprint arXiv:1011.3027v5, July 2011.

    Google Scholar 

  9. L. Chen, L. Goldstein, and Q. Shao, Normal Approximation by Stein’s Method. Springer, 2010.

    Google Scholar 

  10. B. Laurent and P. Massart, “Adaptive estimation of a quadratic functional by model selection,” The annals of Statistics, vol. 28, no. 5, pp. 1302–1338, 2000.

    Article  MathSciNet  MATH  Google Scholar 

  11. G. Raskutti, M. Wainwright, and B. Yu, “Minimax rates of estimation for high-dimensional linear regression over¡ formula formulatype=,” Information Theory, IEEE Transactions on, vol. 57, no. 10, pp. 6976–6994, 2011.

    Article  MathSciNet  Google Scholar 

  12. P. Zhang and R. Qiu, “Glrt-based spectrum sensing with blindly learned feature under rank-1 assumption,” IEEE Trans. Communications. to appear.

    Google Scholar 

  13. P. Zhang, R. Qiu, and N. Guo, “Demonstration of Spectrum Sensing with Blindly Learned Feature,” IEEE Communications Letters, vol. 15, pp. 548–550, May 2011.

    Article  Google Scholar 

  14. S. Hou, R. Qiu, J. P. Browning, and Wick, “Spectrum sensing in cognitive radio with robust principal component analysis,” in IEEE Waveform Diversity and Design Conference 2012, (Kauai, Hawaii), January 2012.

    Google Scholar 

  15. S. Hou, R. Qiu, J. P. Browning, and M. C. Wicks, “Spectrum sensing in cognitive radio with subspace matching,” in IEEE Waveform Diversity and Design Conference 2012, (Kauai, Hawaii), January 2012.

    Google Scholar 

  16. A. Guionnet and O. Zeitouni, “Concentration of the spectral measure for large matrices,” Electron. Comm. Probab, vol. 5, pp. 119–136, 2000.

    MathSciNet  MATH  Google Scholar 

  17. Z. Bai, “Methodologies in spectral analysis of large-dimensional random matrices, a review,” Statist. Sinica, vol. 9, no. 3, pp. 611–677, 1999.

    MathSciNet  MATH  Google Scholar 

  18. I. Johnstone, “High dimensional statistical inference and random matrices,” Arxiv preprint math/0611589, 2006.

    Google Scholar 

  19. G. W. Anderson, A. Guionnet, and O. Zeitouni, An Introduction to Random Matrices. Cambridge University Press, 2010.

    Google Scholar 

  20. M. Lopes, L. Jacob, and M. Wainwright, “A more powerful two-sample test in high dimensions using random projection,” arXiv preprint arXiv:1108.2401, 2011.

    Google Scholar 

  21. T. Tao and V. Vu, “Random matrices: The distribution of the smallest singular values,” Geometric And Functional Analysis, vol. 20, no. 1, pp. 260–297, 2010.

    Article  MathSciNet  MATH  Google Scholar 

  22. T. Tao and V. Vu, “On random ± 1 matrices: singularity and determinant,” Random Structures & Algorithms, vol. 28, no. 1, pp. 1–23, 2006.

    Article  MathSciNet  MATH  Google Scholar 

  23. Y. Yin, Z. Bai, and P. Krishnaiah, “On the limit of the largest eigenvalue of the large dimensional sample covariance matrix,” Probability Theory and Related Fields, vol. 78, no. 4, pp. 509–521, 1988.

    Article  MathSciNet  MATH  Google Scholar 

  24. I. Johnstone, “On the distribution of the largest eigenvalue in principal components analysis,” The Annals of statistics, vol. 29, no. 2, pp. 295–327, 2001.

    Article  MathSciNet  MATH  Google Scholar 

  25. S. Hou, R. C. Qiu, J. P. Browning, and M. C. Wicks, “Spectrum Sensing in Cognitive Radio with Subspace Matching,” in IEEE Waveform Diversity and Design Conference, January 2012.

    Google Scholar 

  26. S. Hou and R. C. Qiu, “Spectrum sensing for cognitive radio using kernel-based learning,” arXiv preprint arXiv:1105.2978, 2011.

    Google Scholar 

  27. R. Couillet and M. Debbah, Random Matrix Methods for Wireless Communications. Cambridge University Press, 2011.

    Google Scholar 

  28. A. Amini, “High-dimensional principal component analysis,” 2011.

    Google Scholar 

  29. L. Vandenberghe and S. Boyd, “Semidefinite programming,” SIAM review, vol. 38, no. 1, pp. 49–95, 1996.

    Article  MathSciNet  MATH  Google Scholar 

  30. D. Paul and I. M. Johnstone, “Augmented sparse principal component analysis for high dimensional data,” arXiv preprint arXiv:1202.1242, 2012.

    Google Scholar 

  31. Q. Berthet and P. Rigollet, “Optimal detection of sparse principal components in high dimension,” Arxiv preprint arXiv:1202.5070, 2012.

    Google Scholar 

  32. A. d’Aspremont, F. Bach, and L. Ghaoui, “Approximation bounds for sparse principal component analysis,” Arxiv preprint arXiv:1205.0121, 2012.

    Google Scholar 

  33. A. d’Aspremont, L. El Ghaoui, M. Jordan, and G. Lanckriet, A direct formulation for sparse PCA using semidefinite programming, vol. 49. SIAM Review, 2007.

    Google Scholar 

  34. H. Zou, T. Hastie, and R. Tibshirani, “Sparse principal component analysis,” Journal of computational and graphical statistics, vol. 15, no. 2, pp. 265–286, 2006.

    Article  MathSciNet  Google Scholar 

  35. P. Bickel and E. Levina, “Covariance regularization by thresholding,” The Annals of Statistics, vol. 36, no. 6, pp. 2577–2604, 2008.

    Article  MathSciNet  MATH  Google Scholar 

  36. N. El Karoui, “Spectrum estimation for large dimensional covariance matrices using random matrix theory,” The Annals of Statistics, vol. 36, no. 6, pp. 2757–2790, 2008.

    Article  MathSciNet  MATH  Google Scholar 

  37. S. Geman, “The spectral radius of large random matrices,” The Annals of Probability, vol. 14, no. 4, pp. 1318–1328, 1986.

    Article  MathSciNet  MATH  Google Scholar 

  38. J. Baik, G. Ben Arous, and S. Péché, “Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices,” The Annals of Probability, vol. 33, no. 5, pp. 1643–1697, 2005.

    Article  MathSciNet  MATH  Google Scholar 

  39. T. Tao, “Outliers in the spectrum of iid matrices with bounded rank perturbations,” Probability Theory and Related Fields, pp. 1–33, 2011.

    Google Scholar 

  40. F. Benaych-Georges, A. Guionnet, M. Maida, et al., “Fluctuations of the extreme eigenvalues of finite rank deformations of random matrices,” Electronic Journal of Probability, vol. 16, pp. 1621–1662, 2010.

    MathSciNet  Google Scholar 

  41. F. Bach, S. Ahipasaoglu, and A. d’Aspremont, “Convex relaxations for subset selection,” Arxiv preprint ArXiv:1006.3601, 2010.

    Google Scholar 

  42. E. J. Candès and M. A. Davenport, “How well can we estimate a sparse vector?,” Applied and Computational Harmonic Analysis, 2012.

    Google Scholar 

  43. E. Candes and T. Tao, “The Dantzig Selector: Statistical Estimation When p is much larger than n,” Annals of Statistics, vol. 35, no. 6, pp. 2313–2351, 2007.

    Article  MathSciNet  MATH  Google Scholar 

  44. F. Ye and C.-H. Zhang, “Rate minimaxity of the lasso and dantzig selector for the lq loss in lr balls,” The Journal of Machine Learning Research, vol. 9999, pp. 3519–3540, 2010.

    MathSciNet  Google Scholar 

  45. E. Arias-Castro, “Detecting a vector based on linear measurements,” Electronic Journal of Statistics, vol. 6, pp. 547–558, 2012.

    Article  MathSciNet  MATH  Google Scholar 

  46. E. Arias-Castro, E. Candes, and M. Davenport, “On the fundamental limits of adaptive sensing,” arXiv preprint arXiv:1111.4646, 2011.

    Google Scholar 

  47. E. Arias-Castro, S. Bubeck, and G. Lugosi, “Detection of correlations,” The Annals of Statistics, vol. 40, no. 1, pp. 412–435, 2012.

    Article  MathSciNet  MATH  Google Scholar 

  48. E. Arias-Castro, S. Bubeck, and G. Lugosi, “Detecting positive correlations in a multivariate sample,” 2012.

    Google Scholar 

  49. A. B. Tsybakov, Introduction to nonparametric estimation. Springer, 2008.

    Google Scholar 

  50. L. Balzano, B. Recht, and R. Nowak, “High-dimensional matched subspace detection when data are missing,” in Information Theory Proceedings (ISIT), 2010 IEEE International Symposium on, pp. 1638–1642, IEEE, 2010.

    Google Scholar 

  51. L. K. Balzano, Handling missing data in high-dimensional subspace modeling. PhD thesis, UNIVERSITY OF WISCONSIN, 2012.

    Google Scholar 

  52. L. L. Scharf, Statistical Signal Processing. Addison-Wesley, 1991.

    Google Scholar 

  53. L. L. Scharf and B. Friedlander, “Matched subspace detectors,” Signal Processing, IEEE Transactions on, vol. 42, no. 8, pp. 2146–2157, 1994.

    Article  Google Scholar 

  54. S. Hou, R. C. Qiu, M. Bryant, and M. C. Wicks, “Spectrum Sensing in Cognitive Radio with Robust Principal Component Analysis,” in IEEE Waveform Diversity and Design Conference, January 2012.

    Google Scholar 

  55. C. McDiarmid, “On the method of bounded differences,” Surveys in combinatorics, vol. 141, no. 1, pp. 148–188, 1989.

    MathSciNet  Google Scholar 

  56. M. Azizyan and A. Singh, “Subspace detection of high-dimensional vectors using compressive sampling,” in IEEE SSP, 2012.

    Google Scholar 

  57. M. Davenport, M. Wakin, and R. Baraniuk, “Detection and estimation with compressive measurements,” tech. rep., Tech. Rep. TREE0610, Rice University ECE Department, 2006, 2006.

    Google Scholar 

  58. J. Haupt and R. Nowak, “Compressive sampling for signal detection,” in Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, vol. 3, pp. III–1509, IEEE, 2007.

    Google Scholar 

  59. A. James, “Distributions of matrix variates and latent roots derived from normal samples,” The Annals of Mathematical Statistics, pp. 475–501, 1964.

    Google Scholar 

  60. S. Balakrishnan, M. Kolar, A. Rinaldo, and A. Singh, “Recovering block-structured activations using compressive measurements,” arXiv preprint arXiv:1209.3431, 2012.

    Google Scholar 

  61. R. Muirhead, Aspects of Mutivariate Statistical Theory. Wiley, 2005.

    Google Scholar 

  62. S. Vempala, The random projection method, vol. 65. Amer Mathematical Society, 2005.

    Google Scholar 

  63. C. Helstrom, “Quantum detection and estimation theory,” Journal of Statistical Physics, vol. 1, no. 2, pp. 231–252, 1969.

    Article  MathSciNet  Google Scholar 

  64. C. Helstrom, “Detection theory and quantum mechanics,” Information and Control, vol. 10, no. 3, pp. 254–291, 1967.

    Article  MathSciNet  Google Scholar 

  65. J. Sharpnack, A. Rinaldo, and A. Singh, “Changepoint detection over graphs with the spectral scan statistic,” arXiv preprint arXiv:1206.0773, 2012.

    Google Scholar 

  66. D. Ramírez, J. Vía, I. Santamaría, and L. Scharf, “Locally most powerful invariant tests for correlation and sphericity of gaussian vectors,” arXiv preprint arXiv:1204.5635, 2012.

    Google Scholar 

  67. A. Onatski, M. Moreira, and M. Hallin, “Signal detection in high dimension: The multispiked case,” arXiv preprint arXiv:1210.5663, 2012.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tennessee Technological University, Cookeville, Tennessee, USA

    Robert Qiu

  2. Utica, NY, USA

    Michael Wicks

Authors
  1. Robert Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Wicks
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Qiu, R., Wicks, M. (2014). Detection in High Dimensions. In: Cognitive Networked Sensing and Big Data. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4544-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-4544-9_10

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-4543-2

  • Online ISBN: 978-1-4614-4544-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation