Precision vs Confidence Tradeoffs for ℓ₂-Based Frequency Estimation in Data Streams

Abstract

We consider the data stream model where an n-dimensional vector x is updated coordinate-wise by a stream of updates. The frequency estimation problem is to process the stream in a single pass and using small memory such that an estimate for x _i for any i can be retrieved. We present the first algorithms for ℓ₂-based frequency estimation that exhibit a tradeoff between the precision (additive error) of its estimate and the confidence on that estimate, for a range of parameter values. We show that our algorithms are optimal for a range of parameters for the class of matrix algorithms, namely, those whose state corresponding to a vector x can be represented as Ax for some m ×n matrix A. All known algorithms for ℓ₂-based frequency estimation are matrix algorithms.