Research in Data Science pp 211-237 | Cite as

# Compressed Anomaly Detection with Multiple Mixed Observations

## Abstract

We consider a collection of independent random variables that are identically distributed, except for a small subset which follows a different, anomalous distribution. We study the problem of detecting which random variables in the collection are governed by the anomalous distribution. Recent work proposes to solve this problem by conducting hypothesis tests based on mixed observations (e.g., linear combinations) of the random variables. Recognizing the connection between taking mixed observations and compressed sensing, we view the problem as recovering the “support” (index set) of the anomalous random variables from multiple measurement vectors (MMVs). Many algorithms have been developed for recovering jointly sparse signals and their support from MMVs. We establish the theoretical and empirical effectiveness of these algorithms in detecting anomalies. We also extend the LASSO algorithm to an MMV version for our purpose. Further, we perform experiments on synthetic data, consisting of samples from the random variables, to explore the trade-off between the number of mixed observations per sample and the number of samples required to detect anomalies.

## Notes

### Acknowledgements

The initial research for this effort was conducted at the Research Collaboration Workshop for Women in Data Science and Mathematics, July 17–21 held at ICERM. Funding for the workshop was provided by ICERM, AWM, and DIMACS (NSF grant CCF-1144502). SL was supported by NSF CAREER grant CCF− 1149225. DN was partially supported by the Alfred P. Sloan Foundation, NSF CAREER *#*1348721, and NSF BIGDATA *#*1740325. JQ was supported by NSF DMS-1818374.

## References

- 1.D. Angelosante, G.B. Giannakis, E. Grossi, Compressed sensing of time-varying signals, in
*16th International Conference on Digital Signal Processing*, Santorini-Hellas, Greece (2009), pp. 1–8Google Scholar - 2.D. Baron, M.B. Wakin, M.F. Duarte, S. Sarvotham, R.G. Baraniuk, Distributed compressed sensing (2005). Preprint available at https://www.ece.rice.edu/~shri/pub/DCS_TR.pdf
- 3.D. Baron, S. Sarvotham, R.G. Baraniuk, Bayesian compressive sensing via belief propagation. IEEE Trans. Signal Process.
**58**(1), 269–280 (2010)MathSciNetCrossRefGoogle Scholar - 4.M. Basseville, I.V. Nikiforov,
*Detection of Abrupt Changes: Theory and Application*(Prentice-Hall, Upper Saddle River, 1993)zbMATHGoogle Scholar - 5.T. Blumensath, M.E. Davies, Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal.
**27**(3), 265–274 (2009)MathSciNetCrossRefGoogle Scholar - 6.E.J. Candès, Compressive sampling, in
*Proceedings of the International Congress of Mathematicians*, vol. 3, Madrid, Spain (2006), pp. 1433–1452zbMATHGoogle Scholar - 7.E.J. Candès, T. Tao, Decoding by linear programming. IEEE Trans. Inf. Theory
**51**(12), 4203–4215 (2005)MathSciNetCrossRefGoogle Scholar - 8.E.J. Candès, T. Tao, Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory
**52**(12), 5406–5425 (2006)MathSciNetCrossRefGoogle Scholar - 9.E.J. Candès, J. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math.
**59**(8), 1207–1223 (2006)MathSciNetCrossRefGoogle Scholar - 10.J. Chen, X. Huo, Theoretical results on sparse representations of multiple-measurement vectors. IEEE Trans. Signal Process.
**54**(12), 4634–4643 (2006)CrossRefGoogle Scholar - 11.S.S. Chen, D.L. Donoho, M.A. Saunders, Atomic decomposition by basis pursuit. SIAM Rev.
**43**(1), 129–159 (2001)MathSciNetCrossRefGoogle Scholar - 12.M. Cho, W. Xu, L. Lai, Compressed hypothesis testing: to mix or not to mix? (2016). arXiv preprint arXiv:1609.07528Google Scholar
- 13.S.F. Cotter, B.D. Rao, K. Engan, K. Kreutz-Delgado, Sparse solutions to linear inverse problems with multiple measurement vectors. IEEE Trans. Signal Process.
**53**(7), 2477–2488 (2005)MathSciNetCrossRefGoogle Scholar - 14.D.L. Donoho, Compressed sensing. IEEE Trans. Inf. Theory
**52**(4), 1289–1306 (2006)MathSciNetCrossRefGoogle Scholar - 15.D.L. Donoho, X. Huo, Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory
**47**(7), 2845–2862 (2001)MathSciNetCrossRefGoogle Scholar - 16.M.R. Duarte, M.B. Wakin, D. Baron, R.G. Baraniuk, Universal distributed sensing via random projections, in
*5th International Conference on Information Processing in Sensor Networks*, Nashville, TN (2006), pp. 177–185Google Scholar - 17.M.F. Duarte, M.B. Wakin, D. Baron, S. Sarvotham, R.G. Baraniuk, Measurement bounds for sparse signal ensembles via graphical models. IEEE Trans. Inf. Theory
**59**(7), 4280–4289 (2013)MathSciNetCrossRefGoogle Scholar - 18.B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression. Ann. Stat.
**32**(2), 407–499 (2004)MathSciNetCrossRefGoogle Scholar - 19.Y.C. Eldar, G. Kutyniok,
*Compressed Sensing: Theory and Applications*(Cambridge University Press, Cambridge, 2012)CrossRefGoogle Scholar - 20.J. Filos, E. Karseras, W. Dai, S. Yan, Tracking dynamic sparse signals with hierarchical Kalman filters: a case study, in
*18th International Conference on Digital Signal Processing*, Fira, Greece (2013), pp. 1–6Google Scholar - 21.S. Foucart, H. Rauhut,
*A Mathematical Introduction to Compressive Sensing*(Birkhäuser, Basel, 2013)Google Scholar - 22.J. Haupt, R. Nowak, Signal reconstruction from noisy random projections. IEEE Trans. Inf. Theory
**52**(9), 4036–4048 (2006)MathSciNetCrossRefGoogle Scholar - 23.S. Ji, Y. Xue, L. Carin, Bayesian compressive sensing. IEEE Trans. Signal Process.
**56**(6), 2346–2356 (2008)MathSciNetCrossRefGoogle Scholar - 24.S.S. Keerthi, S. Shevade, A fast tracking algorithm for generalized LARS/LASSO. IEEE Trans. Neural Netw.
**18**(6), 1826–1830 (2007)CrossRefGoogle Scholar - 25.L. Lai, Y. Fan, H.V. Poor, Quickest detection in cognitive radio: a sequential change detection framework, in
*IEEE Global Telecommunications Conference*, New Orleans, LO (2008), pp. 1–5Google Scholar - 26.S.G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process.
**41**(12), 3397–3415 (1993)CrossRefGoogle Scholar - 27.M. Malloy, R. Nowak, On the limits of sequential testing in high dimensions, in
*45th Asilomar Conference on Signals, Systems and Computers*, Pacific Grove, CA (2011), pp. 1245–1249Google Scholar - 28.M. Malloy, R. Nowak, Sequential analysis in high-dimensional multiple testing and sparse recovery, in
*IEEE International Symposium on Information Theory*, St. Petersburg, Russia (2011), pp. 2661–2665Google Scholar - 29.M.L. Malloy, G. Tang, R.D. Nowak, Quickest search for a rare distribution, in
*46th Annual Conference on Information Sciences and Systems*, Princeton, NJ (2012), pp. 1–6Google Scholar - 30.M. Mishali, Y.C. Eldar, Reduce and boost: recovering arbitrary sets of jointly sparse vectors. IEEE Trans. Signal Process.
**56**(10), 4692–4702 (2008)MathSciNetCrossRefGoogle Scholar - 31.D. Needell, R. Vershynin, Signal recovery from incomplete and inaccurate measurements via regularized orthogonal matching pursuit. IEEE J. Sel. Topics Signal Process.
**4**(2), 310–316 (2010)CrossRefGoogle Scholar - 32.Y.C. Pati, R. Rezaiifar, P.S. Krishnaprasad, Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition, in
*27th Asilomar Conference on Signals, Systems and Computers*, vol. 1, Pacific Grove, CA (1993), pp. 40–44Google Scholar - 33.S. Patterson, Y.C. Eldar, I. Keidar, Distributed compressed sensing for static and time-varying networks. IEEE Trans. Signal Process.
**62**(19), 4931–4946 (2014)MathSciNetCrossRefGoogle Scholar - 34.H.V. Poor, O. Hadjiliadis,
*Quickest Detection*(Cambridge University Press, Cambridge, 2008)CrossRefGoogle Scholar - 35.J.A. Tropp, Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory
**52**(3), 1030–1051 (2006)MathSciNetCrossRefGoogle Scholar - 36.J.A. Tropp, A.C. Gilbert, Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory
**53**(12), 4655–4666 (2007)MathSciNetCrossRefGoogle Scholar - 37.J.A. Tropp, A.C. Gilbert, M.J. Strauss, Simultaneous sparse approximation via greedy pursuit, in
*IEEE International Conference on Acoustics, Speech, and Signal Processing*, vol. 5, Philadelphia, PA (2005), pp. 721–724Google Scholar - 38.E. van den Berg, M.P. Friedlander, Theoretical and empirical results for recovery from multiple measurements. IEEE Trans. Inf. Theory
**56**(5), 2516–2527 (2010)MathSciNetCrossRefGoogle Scholar - 39.Y. Xia, D. Tse, Inference of link delay in communication networks. IEEE J. Sel. Areas Commun.
**24**(12), 2235–2248 (2006)CrossRefGoogle Scholar - 40.G. Yu, G. Sapiro, Statistical compressed sensing of Gaussian mixture models. IEEE Trans. Signal Process.
**59**(12), 5842–5858 (2011)MathSciNetCrossRefGoogle Scholar