Load-Aware Shedding in Stream Processing Systems

Abstract

Distributed stream processing systems are today gaining momentum as a tool to perform analytics on continuous data streams. Load shedding is a technique used to handle unpredictable spikes in the input load whenever available computing resources are not adequately provisioned. In this paper, we propose Load-Aware Shedding (LAS), a novel load shedding solution that, unlike previous works, does not rely neither on a pre-defined cost model nor on any assumption on the tuple execution duration. Leveraging sketches, LAS efficiently estimates the execution duration of each tuple with small error bounds and uses this knowledge to proactively shed input streams at any operator to limiting queuing latencies while dropping as few tuples as possible. We provide a theoretical analysis proving that LAS is an $({\varepsilon }, \delta )$-approximation of the optimal online load shedder. Furthermore, through an extensive practical evaluation based on simulations and a prototype, we evaluate its impact on stream processing applications.

This work has been partially funded by the MIUR SCN-00064 project RoMA and by Sapienza University of Rome through the project RM11916B75A3293D.

A preliminary short version of this work appeared in the Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems.

N. Rivetti—Independent researcher.

A Theoretical Analysis

Data streaming algorithms strongly rely on pseudo-random functions that map elements of the stream to uniformly distributed image values to keep the essential information of the input stream, regardless of the stream elements frequency distribution.

This appendix extends with the proofs the theoretical analysis of the quality of the shedding performed by LAS in two steps provided in Sect. 4 as well as the complexities presented in Sect. 3.

First we study the correctness and optimality of the shedding algorithm, under full knowledge assumption (i.e., the shedding strategy is aware of the exact execution duration $w_t$ for each tuple t). Then, in Appendix A.3, we provide a probabilistic analysis of the mechanism that LAS uses to estimate the tuple execution durations.

1.1 A.1 Time, Space and Communication Complexities

In this section we provide the proofs of the time, space and communication complexities presented in Sect. 3.

Theorem 1 [Time complexity of LAS]. For each tuple read from the input stream, the time complexity of LAS for the operator and the load shedder is $\mathcal {O}(\log 1/\delta )$.

Proof

By Listing 3.1, for each tuple read from the input stream, the algorithm increments an entry per row of both the ${\mathcal {F}}$ and ${\mathcal {W}}$ matrices. Since each has $\log 1/\delta$ rows, the resulting update time complexity is $\mathcal {O}(\log 1/\delta )$. By Listing 3.2, for each submitted tuple, the scheduler has to retrieve the estimated execution duration for the submitted tuple. This operation requires to read entry per row of both the ${\mathcal {F}}$ and ${\mathcal {W}}$ matrices. Since each has $\log 1/\delta$ rows, the resulting query time complexity is $\mathcal {O}(\log 1/\delta )$.

Theorem 2 [Space Complexity of LAS]. The space complexity of LAS for the operator and load shedder is $\mathcal {O} \left( \frac{1}{\varepsilon }\log \frac{1}{\delta }(\log m + \log n)\right)$ bits.

Proof

The operator stores two matrices of size $\log (\frac{1}{\delta }) \times \frac{e}{\varepsilon }$ of counters of size $\log m$. In addition, it also stores a hash function with a domain of size n. Then the space complexity of LAS on the operator is $\mathcal {O} \left( \frac{1}{\varepsilon }\log \frac{1}{\delta }(\log m + \log n)\right)$ bits. The load shedder stores the same matrices, as well as a scalar. Then the space complexity of LAS on the load shedder is also $\mathcal {O} \left( \frac{1}{\varepsilon }\log \frac{1}{\delta }(\log m + \log n)\right)$ bits.

Theorem 3 [Communication complexity of LAS]. The communication complexity of LAS is of $\mathcal {O} \left( \frac{m}{N} \right)$ messages and $\mathcal {O} \left( \frac{m}{N} \left( \frac{1}{\varepsilon }\log \frac{1}{\delta }(\log m + \log n) + \log m \right) \right)$ bits.

Proof

After executing N tuples, the operator may send the ${\mathcal {F}}$ and ${\mathcal {W}}$ matrices to the load shedder.

This generates a communication cost of $\mathcal {O} \left( \frac{m}{N} \frac{1}{\varepsilon }\log \frac{1}{\delta }(\log m + \log n)\right)$ bits via $\mathcal {O} \left( \frac{m}{N} \right)$ messages. When the load shedder receives these matrices, the synchronization mechanism kicks in and triggers a round trip communication (half of which is piggybacked by the tuples) with the operator. The communication cost of the synchronization mechanism is $\mathcal {O} \left( \frac{m}{N} \right)$ messages and $\mathcal {O} \left( \frac{m}{N} \log m \right)$ bits.

Note that the communication cost is low with respect to the stream size since the window size N should be chosen such that $N \gg 1$ (e.g., in our tests we have $N=1024$).

1.2 A.2 Correctness of LAS

We suppose that tuples cannot be preempted, that is they must be processed uninterruptedly on the available operator instance. As mentioned before, in this analysis we assume that the execution duration w(t) is known for each tuple t. Finally, given our system model, we consider the problem of minimizing ${d}$, the number of dropped tuples, while guaranteeing that the average queuing latency ${\overline{Q}}(t)$ will be upper-bounded by $\tau$, $\forall t \in \sigma$. The solution must work online, thus the decision of enqueueing or dropping a tuple has to be made only resorting to knowledge about tuples received so far in the stream.

Let OPT be the online algorithm that provides the optimal solution to Problem 1. We denote with ${\mathcal {D}}^{\sigma }_{\text{ O }PT}$ (resp. ${d}^{\sigma }_{\text{ O }PT}$) the set of dropped tuple indices (resp. the number of dropped tuples) produced by the OPT algorithm fed by stream $\sigma$ (cf., Sect. 2). We also denote with ${d}^{\sigma }_{\text {LAS}}$ the number of dropped tuples produced by LAS introduced in Sect. 3.3 fed with the same stream $\sigma$.

Theorem 4 [Correctness and Optimality of LAS]. For any $\sigma$, we have ${d}^{\sigma }_{\text {LAS}} = {d}^{\sigma }_{\text{ O }PT}$ and $\forall t\in \sigma , {\overline{Q}}^{\sigma }_{\text {LAS}}(t) \le \tau$.

Proof

Given a stream $\sigma$, consider the sets of indices of tuples dropped by respectively OPT and LAS, namely ${\mathcal {D}}^{\sigma }_{\text{ O }PT}$ and ${\mathcal {D}}^{\sigma }_{\text {LAS}}$. Below, we prove by contradiction that ${d}^{\sigma }_{\text {LAS}} = {d}^{\sigma }_{\text{ O }PT}$.

Assume that ${d}^{\sigma }_{\text {LAS}} > {d}^{\sigma }_{\text{ O }PT}$. Without loss of generality, we denote $i_1, \ldots , i_{{d}^{\sigma }_{\text{ L }AS}}$ the ordered indices in ${\mathcal {D}}^{\sigma }_{\text{ L }AS}$, and $j_1, \ldots , j_{{d}^{\sigma }_{\text{ O }PT}}$ the ordered indices in ${\mathcal {D}}^{\sigma }_{\text{ O }PT}$. Let us define a as the largest natural integer such that $\forall \ell \le a, i_\ell = j_\ell$ (i.e., $i_1 = j_1, \ldots , i_a = j_a$). Thus, we have $i_{a+1} \ne j_{a+1}$.

• Assume that $i_{a+1} < j_{a+1}$. Then, according to Sect. 3.3, the $i_{a+1}$-th tuple of $\sigma$ has been dropped by LAS as the method Check returned true. Thus, as $i_{a+1}\notin {\mathcal {D}}^{\sigma }_{\text{ O }PT}$, the OPT run has enqueued this tuple violating the constraint $\tau$. But this is in contradiction with the definition of OPT.

• Assume now that $i_{a+1} > j_{a+1}$. The fact that LAS does not drop the $j_{a+1}$ tuple means that Check returns false, thus that tuple does not violate the constraint on $\tau$. However, as OPT is optimal, it may drop some tuples for which Check is false, just because this allows it to drop an overall lower number of tuples. Therefore, if it drops this $j_{a+1}$ tuple, it means that OPT knows the future evolution of the stream and takes a decision on this knowledge. But, by assumption, OPT is an online algorithm, and the contradiction follows.

Then, we have that $i_{a+1} = j_{a+1}$. By induction, we iterate this reasoning for all the remaining indices from $a+1$ to ${d}^{\sigma }_{\text{ O }PT}$. We then obtain that ${\mathcal {D}}^{\sigma }_{\text{ O }PT}\subseteq {\mathcal {D}}^{\sigma }_{\text{ L }AS}$.

As by assumption ${d}^{\sigma }_{\text{ O }PT}<{d}^{\sigma }_{\text{ L }AS}$, we have that $\exists \ell \in {\mathcal {D}}^{\sigma }_{\text{ L }AS}\setminus {\mathcal {D}}^{\sigma }_{\text{ O }PT}$ such that $\ell$ has been dropped by LAS. This means that, with the same tuple index prefix shared by OPT and LAS, the method Check returned true when evaluated on $\ell$, and OPT would violate the condition on $\tau$ by enqueuing it. That leads to a contradiction. Then, ${\mathcal {D}}^{\sigma }_{\text{ L }AS}\setminus {\mathcal {D}}^{\sigma }_{\text{ O }PT}=\emptyset$, and ${d}^{\sigma }_{\text{ O }PT} = {d}^{\sigma }_{\text{ L }AS}$.

Furthermore, by construction, LAS never enqueues a tuple that violates the condition on $\tau$ because Check would return true.

Consequently, $\forall t\in \sigma , {\overline{Q}}^{\sigma }_{\text{ L }AS}(t) \le \tau$, which concludes the proof.

1.3 A.3 Execution Time Estimation

In this section, we analyze the approximation made on execution duration w(t) for each tuple t when the assumption of full knowledge is removed. LAS uses two matrices, ${\mathcal {F}}$ and ${\mathcal {W}}$, to estimate the execution time w(t) of each tuple submitted to the operator. By the Count Min  sketch algorithm (cf., Sect. 3.2) and Listing 3.1, we have that for any $t \in [n]$ and for each row $i\in [r]$,

$$\begin{aligned} {\mathcal {F}}[i][h_i(t)](m)&= \sum _{u=1}^n f_u 1_{\{h_i(u)=h_i(t)\}} \\&= f_t + \sum _{u=1,u\ne t}^n f_u 1_{\{h_i(u)=h_i(t)\}}. \end{aligned}$$

and

$${\mathcal {W}}[i][h_i(t)](m) = f_t w_t + \sum _{u=1,u\ne t}^n f_u w_u 1_{\{h_i(u)=h_i(t)\}},$$

Let us denote respectively by $w_{\min }$ and $w_{\max }$ the minimum and the maximum execution time of the items. We have trivially

$$w_{\min } \le \frac{{\mathcal {W}}[i][h_i(t)]}{{\mathcal {F}}[i][h_i(t)]} \le w_{\max }.$$

We define $S = \sum _{\ell =1}^n w_\ell$. We then have

Theorem 5

$$\begin{aligned}&\mathbb {E}\{{\mathcal {W}}[i][h_i(t)]/{\mathcal {F}}[i][h_i(t)]\} \\&=\frac{S-w_t}{n-1} - \frac{k(S-nw_t)}{n(n-1)} \left( 1 - \left( 1 -\frac{1}{k}\right) ^{n}\right) . \end{aligned}$$

It important to note that this result does not depend on m.

Proof

For any $t=1,\ldots ,n$, $\ell =0,\ldots ,n-1$ and $A \in U_\ell (t)$, we introduce the event $B(t,\ell ,A)$ defined by

$$\begin{aligned} B(t,\ell ,&A) = \{h_i(u)=h_i(t), \; \forall u \in A \text{ and } \\&h_i(u) \ne h_i(t), \; \forall u \in \{1,\ldots ,n\} \setminus (A \cup \{t\})\}. \end{aligned}$$

From the independence of the hash function $h_i$, we have

$$\mathbb {P}\{B(t,\ell ,A)\} = \left( \frac{1}{k}\right) ^\ell \left( 1 -\frac{1}{k}\right) ^{n-1-\ell }.$$

Let us consider the ratio

$${\mathcal {V}}_{i,t}={\mathcal {W}}[i][h_i(t)]/{\mathcal {F}}[i][h_i(t)].$$

For any $i=0,\ldots ,n$, we define

$$R_\ell (t) = \left\{ \frac{f_tw_t + \sum _{u \in A} f_uw_u}{f_t + \sum _{u \in A} f_u}, \; A \in U_\ell (t)\right\} .$$

We have $R_0(t) = \{w_t\}$. We introduce the set R(t) defined by

$$R(t) = \bigcup _{\ell =0}^{n-1} R_\ell (t).$$

Thus with probability 1,

$${\mathcal {W}}[i][h_i(t)]/{\mathcal {F}}[i][h_i(t)] \in R(t).$$

Let $x \in R(t)$. We have

$$\begin{aligned}&\mathbb {P}\{{\mathcal {V}}_{i,t} = x\} \\&= \sum _{\ell =0}^{n-1} \sum _{A \in U_\ell (t)} \mathbb {P}\{{\mathcal {V}}_{i,t} = x \mid B(t,\ell ,A)\} \mathbb {P}\{B(t,\ell ,A)\} \\&= \sum _{\ell =0}^{n-1} \left( \frac{1}{k}\right) ^\ell \left( 1 -\frac{1}{k}\right) ^{n-1-\ell } \sum _{A \in U_\ell (t)} 1_{\{x =X(t,A)\}}. \end{aligned}$$

where X(t, A) is the fraction:

$$X(t,A) = \frac{f_tw_t + \sum _{u \in A} f_uw_u}{f_t + \sum _{u \in A} f_u}$$

Thus

$$\begin{aligned}&\mathbb {E}\{{\mathcal {V}}_{i,t}\}\\&= \sum _{\ell =0}^{n-1} \left( \frac{1}{k}\right) ^\ell \left( 1 -\frac{1}{k}\right) ^{n-1-\ell } \sum _{A \in U_\ell (t)} \sum _{x \in R(t)} x1_{\{x = X(t,A)\}} \\&= \sum _{\ell =0}^{n-1} \left( \frac{1}{k}\right) ^\ell \left( 1 -\frac{1}{k}\right) ^{n-1-\ell } \sum _{A \in U_\ell (t)} X(t,A). \end{aligned}$$

Let us assume that all the $f_u$ are equal, that is for each u, we have $f_u = m/n$. The experimental evaluation tends to show that the worst case scenario of input streams is exhibited when all the items show the same number of occurrences in the input stream. We get

$$\begin{aligned}&\mathbb {P}\{{\mathcal {V}}_{i,t} = x\} \\&= \sum _{\ell =0}^{n-1} \left( \frac{1}{k}\right) ^\ell \left( 1 -\frac{1}{k}\right) ^{n-1-\ell } \sum _{A \in U_\ell (t)} 1_{\{x = \frac{w_t + \sum _{u \in A} w_u}{\ell + 1}\}} \end{aligned}$$

that concludes the proof.    $\square$