Recent Advances in Histogram Construction Algorithms

Abstract

Obtaining fast and accurate approximations to data distributions is a problem of central interest to database management. A variety of database applications including, approximate querying, similarity searching and data mining in most application domains, rely on such accurate approximations. A very popular way in database theory and practice to approximate a data distribution is by means of a histogram.

Histograms approximate a data distribution using a fixed amount of space, and under certain assumptions strive to minimize the overall error incurred by the approximation. The problem of histogram construction is of profound importance and has attracted a lot of research attention. A fundamental requirement for any good histogram construction algorithm is to approximate the underlying data distribution in a provably good way and be efficient in terms of running time. The typical assumption for constructing histograms is that the data set to be approximated is finite and of known size, or that the size can be easily derived by performing a single pass over the finite data set. The problem of histogram construction on datasets of known size has received a lot of research attention and the optimal as well as a lot of heuristic solutions exist. Wavelets synopses is in fact very simple histograms, and this approach allows us to minimize some objective fit criterion, than storing the k highest coefficients of a wavelet decomposition. In all these problems, given a sequence of data values x ₁,...,x _n , the task is to construct a suitable summary of the data which can be stored in small space (e.g. a small fixed number, say B, of the n coefficients in the Wavelet transform, or a histogram involving B buckets).

In this talk, we first present optimal and approximate histogram construction algorithms that have been recently proposed. We next discuss the algorithms to approximate a data stream in an online and incremental way using histograms. We also describe wavelet synopses algorithms that include deterministic and probabilistic thresholding schemes to select the wavelet coefficients. We finally point out the drawbacks of such existing schemes and introduce future research directions.