Functional Monitoring without Monotonicity
The notion of distributed functional monitoring was recently introduced by Cormode, Muthukrishnan and Yi to initiate a formal study of the communication cost of certain fundamental problems arising in distributed systems, especially sensor networks. In this model, each of k sites reads a stream of tokens and is in communication with a central coordinator, who wishes to continuously monitor some function f of σ, the union of the k streams. The goal is to minimize the number of bits communicated by a protocol that correctly monitors f(σ), to within some small error. As in previous work, we focus on a threshold version of the problem, where the coordinator’s task is simply to maintain a single output bit, which is 0 whenever f(σ) ≤ τ(1 − ε) and 1 whenever f(σ) ≥ τ. Following Cormode et al., we term this the (k,f,τ,ε) functional monitoring problem.
In previous work, some upper and lower bounds were obtained for this problem, with f being a frequency moment function, e.g., F0, F1, F2. Importantly, these functions are monotone. Here, we further advance the study of such problems, proving three new classes of results. First, we provide nontrivial monitoring protocols when f is either H, the empirical Shannon entropy of a stream, or any of a related class of entropy functions (Tsallis entropies). These are the first nontrivial algorithms for distributed monitoring of non-monotone functions. Second, we study the effect of non-monotonicity of f on our ability to give nontrivial monitoring protocols, by considering f = Fp with deletions allowed, as well as f = H. Third, we prove new lower bounds on this problem when f = Fp, for several values of p.
KeywordsCommunication complexity distributed algorithms data streams sensor networks
Unable to display preview. Download preview PDF.
- 1.Alon, N., Matias, Y., Szegedy, M.: The space complexity of approximating the frequency moments. J. Comput. Syst. Sci. 58(1), 137–147 (1999); Preliminary version In: Proc. 28th Annu. ACM Symp. Theory Comput., pp. 20–29 (1996)Google Scholar
- 2.Babcock, B., Olston, C.: Distributed top-k monitoring. In: Proc. Annual ACM SIGMOD Conference, pp. 28–39 (2003)Google Scholar
- 3.Bhuvanagiri, L., Ganguly, S.: Estimating entropy over data streams. In: Proc. 14th Annual European Symposium on Algorithms, pp. 148–159 (2006)Google Scholar
- 4.Cormode, G., Muthukrishnan, S., Yi, K.: Algorithms for distributed functional monitoring. In: Proc. 19th Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1076–1085 (2008)Google Scholar
- 5.Cormode, G., Muthukrishnan, S., Zhuang, W.: What’s different: Distributed, continuous monitoring of duplicate-resilient aggregates on data streams. In: Proc. 22nd International Conference on Data Engineering, p. 57 (2006)Google Scholar
- 6.Das, A., Ganguly, S., Garofalakis, M.N., Rastogi, R.: Distributed set expression cardinality estimation. In: Proc. 30th International Conference on Very Large Data Bases, pp. 312–323 (2004)Google Scholar
- 7.Estrin, D., Govindan, R., Heidemann, J.S., Kumar, S.: Next century challenges: Scalable coordination in sensor networks. In: MOBICOM, pp. 263–270 (1999)Google Scholar
- 8.Harvey, N.J.A., Nelson, J., Onak, K.: Sketching and streaming entropy via approximation theory. In: Proc. 49th Annual IEEE Symposium on Foundations of Computer Science, pp. 489–498 (2008)Google Scholar
- 9.Muthukrishnan, S.: Data streams: Algorithms and applications. In: Proc. 14th Annual ACM-SIAM Symposium on Discrete Algorithms, p. 413 (2003)Google Scholar
- 10.Muthukrishnan, S.: Some algorithmic problems and results in compressed sensing. In: Proc. 44th Annual Allerton Conference (2006)Google Scholar
- 12.Sharfman, I., Schuster, A., Keren, D.: A geometric approach to monitoring threshold functions over distributed data streams. ACM Trans. Database Syst. 32(4) (2007)Google Scholar
- 15.Woodruff, D.P.: Efficient and Private Distance Approximation in the Communication and Streaming Models. PhD thesis, MIT (2007)Google Scholar
- 16.Yi, K., Zhang, Q.: Multi-dimensional online tracking. In: Proc. 19th Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1098–1107 (2009)Google Scholar