Controlling Packet Drops to Improve Freshness of Information

Abstract

Many systems require frequent and regular updates of certain information. These updates have to be transferred regularly from the source(s) to a common destination. We consider scenarios in which an old packet (entire information unit) becomes completely obsolete, in the presence of a new packet. We consider transmission channels with unit storage capacity; upon arrival of a new packet, if another packet is being transmitted then one of the packets is lost. We consider the control problem that consists of deciding which packet to discard so as to maximise the average age of information (AAoI). We derive drop policies that optimize the AAoI. We show that the state independent (static) policies like dropping always the old packets or dropping always the new packets are optimal in many scenarios, among an appropriate set of stationary Markov policies.

E. Altman—This work was financed by the ANR “Maestro 5G”.

Appendices

Appendix A: Proofs

Proof of Lemma 1: By conditioning on $\xi _{k,1}$, $T_{k, 0}$:

$$\begin{aligned} E[\boldsymbol{\varGamma }_k]= & {} E \left[ \boldsymbol{\varGamma }_k \ \boldsymbol{;} \ \xi _{k,1}> T_{k, 0 } \right] + E \left[ \boldsymbol{\varGamma }_k \ \boldsymbol{;} \ \xi _{k,1} \le T_{k, 0} \right] \\= & {} E \left[ T_{k, 0} \ \boldsymbol{;} \ \xi _{k,1}> T_{k, 0 } \right] + E \left[ \xi _{k, 1} + \tilde{\boldsymbol{\varGamma }} \ \boldsymbol{;} \ \xi _{k,1} \le T_{k, 0} \right] \\= & {} E \left[ T_{k, 0} ; \xi _{k,1} > T_{k, 0 } + \xi _{k, 1} \ \boldsymbol{;} \ \xi _{k,1} \le T_{k, 0} \right] \ + E\left[ \tilde{\boldsymbol{\varGamma }} \ \boldsymbol{;} \ \xi _{k,1} \le T_{k, 0} \right] , \end{aligned}$$

where is an IID copy of $\boldsymbol{\varGamma }_k$, which is independent of $T_{k, 0}$ and $\xi _{k, 1}$. By independence, and thus by further conditioning on T we have the following:

$$\begin{aligned} E(\boldsymbol{\varGamma }) = \frac{ E \left[ T ; \xi > T + \xi \ \boldsymbol{;} \ \xi \le T \right] }{ P( T \le \xi ) } = \frac{ E[Te^{-\lambda T} ] + ( 1- E[e^{-\lambda T} ] ) / \lambda - E[T e^{-\lambda T} ] }{ P( T \le \xi ) } = \frac{ 1- \gamma }{\lambda \gamma }. \end{aligned}$$

(25)

Using exactly similar logic:

Using (25),

$$\begin{aligned} E[\boldsymbol{\varGamma }^2]= & {} \frac{E[\min \lbrace T_0 , \xi _1 \rbrace ^2] + 2E[\boldsymbol{\varGamma }]E[ \xi _{k, 1} \ \boldsymbol{;} \ T_{k, 0} > \xi _{k, 1} ] }{\gamma } \nonumber \\= & {} \frac{ 2 (1- \gamma ) }{\lambda ^2 \gamma } - \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma } \nonumber +\frac{ 2 E[\boldsymbol{\varGamma }] }{\gamma } \left( \frac{1-\gamma }{\lambda } - E[Te^{-\lambda T} ] \right) \nonumber \\= & {} \frac{ 2 (1- \gamma ) }{\lambda ^2 \gamma } - \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma ^2 } + \frac{2 (1-\gamma )^2}{\lambda ^2 \gamma ^2} \ = \frac{ 2 (1- \gamma ) }{\lambda ^2 \gamma ^2 } - \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma ^2 } . \end{aligned}$$

(26)

Using (25) and (26), the first two moments of the renewal cycle are:

$$\begin{aligned} E[R_c]= & {} \frac{1}{\lambda } + \frac{1-\gamma }{\lambda \gamma } = \frac{1}{\lambda \gamma } \text { and } \\ E[R_c^2 ]= & {} E \left[ \xi _{k, 0}^2 + 2 \xi _{k, 0} \boldsymbol{\varGamma }_k + \boldsymbol{\varGamma }_k^2 \right] \nonumber { \ = \ } \frac{2}{\lambda ^2 \gamma ^2} - \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma ^2 }. \end{aligned}$$

   $\blacksquare$

Proof of Theorem 1: As a first step, one can easily observe that the coefficients of the lower bound function $f_o$ depend upon $\theta$ only via the stationary distribution $\pi _\theta$, in particular only via $\pi _\theta (0)$, i.e., $f_o (\theta ) = f_o (\pi _\theta (0) )$. Further the function $\theta \mapsto \pi _\theta (0)$ is ONTO (see (15)) and hence one can equivalently optimize $f_o$ using $\pi :=\pi _\theta (0)$:

$$ f_o (\theta ) = f_o (\pi ) = \frac{ d_o \bigg ( (b_n - b_o )\pi + b_o \bigg ) + 0.5( c_n -c_o) \pi + 0.5 c_o }{(d_n -d_o)\pi + d_o }. $$

The first derivative for the lower bound function is:

$$\begin{aligned} f_o^\prime (\pi )= & {} \frac{ 0.5 (c_n d_o - c_od_n) +d_o ( b_n d_o - b_o d_n ) }{ (\pi (d_n-d_o)+d_o ) ^2} . \end{aligned}$$

(27)

From (28) of Appendix B:

$$\begin{aligned} c_n d_o - c_o d_n= & {} \frac{ E[T^2]}{ \lambda \gamma } + \left( \frac{1}{\lambda } + E[T] \right) \left( E[T e^{-\lambda T}] - \frac{ (1- \gamma ) }{\lambda } \right) \frac{2}{\lambda \gamma ^2}. \end{aligned}$$

Thus the numerator of the derivative (27) is proportional to,

$$\begin{aligned} c_n d_o - c_o d_n + 2 d_o (b_n d_o - b_o d_n)= & {} \frac{ E[T^2]}{ \lambda \gamma } + \left( \frac{1}{\lambda } + E[T] \right) E[T e^{-\lambda T}] \left( \frac{2}{\lambda \gamma ^2} - 2 \frac{1}{\lambda \gamma ^2 }\right) \\&- \frac{ 1 }{\lambda \gamma } \left( \left( \frac{1}{\lambda } + E[T] \right) \frac{2(1-\gamma )}{\lambda \gamma } - E[T] \frac{2}{\lambda \gamma } \right) \\= & {} \frac{ E[T^2]}{ \lambda \gamma } + \frac{2}{\lambda ^2 \gamma } (E[T] - E[\boldsymbol{\varGamma }] ) > 0, \text { when }d_n \ge d_o. \end{aligned}$$

Thus the derivative $f_o' (\theta ) > 0$ for all $\theta$, hence the lower bound $f_o$ is increasing with $\pi$, and thus the unique minimizer of $f_o$ is at $\pi ^* = 0$. This implies the DOP scheme (see (15)) is optimal for AAoI ${\bar{a}}(\centerdot )$.    $\blacksquare$

Proof of Theorem 2: As before it suffices to show that the numerator of derivative of $f_n$ (with respect to $\pi$) is negative. Recall the following:

$$\begin{aligned} c_n d_o - c_o d_n= & {} \frac{ E[T^2]}{ \lambda \gamma } + \left( \frac{1}{\lambda } + E[T] \right) \left( E[T e^{-\lambda T}] - \frac{ (1- \gamma ) }{\lambda } \right) \frac{2}{\lambda \gamma ^2} , \\ d_o= & {} \frac{1}{\lambda \gamma } , \ \ b_o = \frac{E[ T e^{-\lambda T}]}{\gamma }, \ \ d_n = \frac{1}{\lambda } + E[T] \end{aligned}$$

The numerator of derivative of $f_n$ is proportional to,

$$\begin{aligned}&c_n d_o - c_o d_n + 2 d_n (b_n d_o - b_o d_n) \\&\qquad \quad = \frac{ E[T^2]}{ \lambda \gamma } + \left( \frac{1}{\lambda } + E[T] \right) E[T e^{-\lambda T}] \left( \frac{2}{\lambda \gamma ^2} - \frac{ 2}{\gamma } \left( \frac{1}{\lambda } + E[T] \right) \right) \\&\qquad \quad - \frac{ 2 }{\lambda \gamma } \left( \frac{1}{\lambda } + E[T] \right) \left( \frac{ 1-\gamma }{\lambda \gamma } - E[T] \right) \\&\qquad \quad = \frac{ E[T^2]}{ \lambda \gamma } - \frac{ 2}{\lambda \gamma } \left( \frac{1}{\lambda } + E[T] \right) \left( 1- \lambda E[T e^{-\lambda T}] \right) (d_o - d_n ). \end{aligned}$$

Thus the theorem follows from hypothesis.    $\blacksquare$

Appendix B: Some Useful Terms Used in the Proofs

The estimate of the term $c_n d_o - d_n c_o$:

$$\begin{aligned}&c_n d_o - d_n c_o = \left( \frac{2}{\lambda ^2 } + E[T^2] + \frac{2 E[T] }{\lambda } \right) \left( \frac{1}{\lambda } + \frac{1-\gamma }{\lambda \gamma } \right) \nonumber \\&- \left( \frac{1}{\lambda } + E[T] \right) \left( \frac{2}{\lambda ^2 } + \frac{ 2 (1- \gamma ) }{\lambda ^2 \gamma } - \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma ^2} + 2 \frac{ (1- \gamma )^2 }{\lambda ^2 \gamma ^2 } +\frac{2 (1- \gamma ) }{\lambda ^2 \gamma } \right) \nonumber \\ \nonumber \\&=\left( \frac{2}{\lambda ^2 } + E[T^2] + \frac{2 E[T] }{\lambda } \right) \left( \frac{1}{\lambda \gamma } \right) \nonumber \\&- \left( \frac{1}{\lambda } + E[T] \right) \left( \frac{2}{\lambda ^2 \gamma } \nonumber - \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma ^2} + 2 \frac{ (1- \gamma )^2 }{\lambda ^2 \gamma ^2 } +\frac{2 (1- \gamma ) }{\lambda ^2 \gamma } \right) \\ \nonumber \\&= E[T^2]\left( \frac{1}{\lambda \gamma } \right) \nonumber - \left( \frac{1}{\lambda } + E[T] \right) \left( - \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma ^2} + 2 \frac{ (1- \gamma )^2 }{\lambda ^2 \gamma ^2 } +\frac{2 (1- \gamma ) }{\lambda ^2 \gamma } \right) \\ \nonumber \\&= \frac{1}{\gamma } \left( \frac{ E[T^2]}{ \lambda } + \left( \frac{1}{\lambda } + E[T] \right) \left( \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma } \right) \right) \nonumber \\&- \frac{1}{\gamma } \left( \left( \frac{1}{\lambda } + E[T] \right) \left( \frac{2 (1- \gamma ) }{\lambda ^2 \gamma } \left( (1- \gamma )+ \gamma \right) \right) \right) \nonumber \\&= \frac{1}{\gamma } \left( \frac{ E[T^2]}{ \lambda } + \left( \frac{1}{\lambda } + E[T] \right) \left( \frac{2 E[T e^{-\lambda T}]}{ \lambda \gamma } \right) - \left( \frac{1}{\lambda } + E[T] \right) \frac{2 (1- \gamma ) }{\lambda ^2 \gamma } \right) \nonumber \\ \nonumber \\&= \frac{ E[T^2]}{ \lambda \gamma } + \left( \frac{1}{\lambda } + E[T] \right) \left( E[T e^{-\lambda T}] - \frac{ (1- \gamma ) }{\lambda } \right) \frac{2}{\lambda \gamma ^2} . \end{aligned}$$

(28)