Aggregation-Aware Compression of Probabilistic Streaming Time Series

Abstract

In recent years, there has been a growing interest for probabilistic data management. We focus on probabilistic time series where a main characteristic is the high volumes of data, calling for efficient compression techniques. To date, most work on probabilistic data reduction has provided synopses that minimize the error of representation w.r.t. the original data. However, in most cases, the compressed data will be meaningless for usual queries involving aggregation operators such as SUM or AVG. We propose PHA (Probabilistic Histogram Aggregation), a compression technique whose objective is to minimize the error of such queries over compressed probabilistic data. We incorporate the aggregation operator given by the end-user directly in the compression technique, and obtain much lower error in the long term. We also adopt a global error aware strategy in order to manage large sets of probabilistic time series, where the available memory is carefully balanced between the series, according to their individual variability.

Appendix

AVG Operator. To extend PHA to the average (AVG) operator, we need a method for computing the average of two probabilistic histograms. Given two probabilistic histograms $H_1$ and $H_2$, aggregating them by an AVG operator means to make a histogram H such that the probability of each value k is equal to the cumulative probability of cases where the average of two values $x_1$ from $H_1$ and $x_2$ from $H_2$ is equal to k, i.e. $k=(x_1 + x_2) / 2$. In the following lemma, we present a formula for aggregating two histograms using the AVG operator.

Lemma 1

Let $H_1$ and $H_2$ be two probabilistic histograms. Then, the probability of each value k in the probabilistic histogram obtained from the average of $H_1$ and $H_2$, denoted as $AVG(H_1, H_2)[k]$, is computed as:

$AVG(H_1, H_2)[k] = \sum _{i=0}^k \! ( (H_1[i] \times H_2[2 \times k - i] + H_1[2 \times k - i] \times H_2[i])$

Proof. The probability of having a value k in $AVG(H_1, H_2)$ is equal to the cumulative probability of all cases where the average of two values i in $H_1$ and j in $H_2$ is equal to k. In other words, j should be equal to $2 \times k - i$. This is done by the sigma in the above equation.