A Learning Automata Based Stable and Energy-Efficient Routing Algorithm for Discrete Energy Harvesting Mobile Wireless Sensor Network

Abstract

Wireless sensor networks (WSN) have been widely used in urban network system and networked monitoring system, which provide easy connectivity and high physical data rate. Considering the battery-limited property of sensor nodes, recently, energy harvesting (EH) technology is introduced into WSN, which can alleviate traditional WSN problems (energy consumption, energy equilibrium, transmission efficiency, etc.). Current EH-WSN routing algorithms generally use the continuous energy harvesting mode, therefore, how to design an efficient routing algorithm for discrete energy harvesting mode and ensure the overall energy balance and conservation is still a great challenge and needs to be solved. Especially, under the mobile environment, the impact of route stability needs to be considered, which makes the design more complicated. To address the above problems, we propose a learning automata (LA) theory based stable and energy-efficient routing algorithm for discrete EH-mobile WSN (DEH-LA-SERA, for short). Firstly, we construct a multi-factors measurement model for sensor nodes, which contains node stability model, energy ratio function, expected harvesting energy model (using Markov decision process method) and direction judgement model. On this basis, we derive the node weighted value, i.e., selecting probability, which can be used to determine whether a node can be chosen as relay node. Secondly, with the help of LA theory, we construct a feedback mechanism to adjust the optimal path. With this solution, we can ensure the overall energy balance and conservation while holding the stability of selected path. As demonstrated in simulation experiments, our algorithm, DEH-LA-SERA, achieved the best performance in route survival time, energy consumption, energy balance and acceptable performance in end-to-end delay and packets delivery ratio.

Appendix: Convergence Analysis for Using LA Theory

In appendix, we analyse the convergence results of our LA based routing algorithm in detail. To find the conclusion, we first derive the drift function of our LA based routing algorithm; then, we use the the interpolation method to represent the action probability as a interpolation sequence. With this, we can approximate our algorithm (the normalized selecting probability, in this analysis, it can be also called as action probability) to an ordinary differential equation (ODE). Finally, we can find that the this ODE converges to the solution of the optimization problem.

In our paper, each node has a learning automaton, the candidate next hop nodes can be regarded as the the chosen actions of LA. Let $\{N_i\}$ represents the set of node i’s candidate next hop nodes, there is a tight correspondence between the selecting probability of candidate next hop nodes and the chosen action probability p. We can use the following expression to represent it

$$\begin{aligned} p_j=\frac{Nw_{j}}{\sum ^{N_i}_{j=1}Nw_j} \end{aligned}$$

(32)

Thus, we can use the theory of stochastic process to represent the genericity one step update formula of the chosen action probability (it is called the drift function, which means the increment in the conditional expectation,)

$$\begin{aligned} p'_{j}(t+1)=E[p_{j}(t+1)|p_{j}(t)]-p_{j}(t) |j\in N_i \end{aligned}$$

(33)

Owing to the fact that p(t) is a Markov process and the dynamics of it depend on the update factor ($\varLambda$), $\beta (t)$ directly depends on p(t) (not on the iteration time). Therefore, $p'_{t}$ can be derived by the function of $p_{t}$. Combining the formula (25)–(26), $p'_{j}(t)$ for our algorithm can be obtained as following

$$\begin{aligned} \begin{aligned} p_{j}'(t)&= \varLambda \cdot p_{j}(t)\left[ 1-p_{j}(t)E[\beta _{j}(t)]- \varLambda \cdot \sum _{q\ne j}p_{q}(t)p_{j}(t)E[\beta _{q}(t)]\right] \\&= \varLambda \cdot p_{j}(t)\sum _{q\ne j}p_{q}(t)[E[\beta _{ij}(t)]-E[\beta _{iq}(t)]]\quad |j,q\in N_i \end{aligned} \end{aligned}$$

(34)

q represents the ID of other candidate actions for node i, which are not selected. Noting that $\beta$ has been defined in Sect. 2.

Defining a function f(.), where

$$\begin{aligned} \begin{aligned}&{\text {maxmize:}}\,f(p)=E[\beta |p] \\&{\text {subject to:}}\,p_{j}\ge 0; i\in [1,]N; j\in N_i\\&\sum ^{N_i}_{j=1}p_{j}=1 \end{aligned} \end{aligned}$$

(35)

We can get the conclusion as following

$$\begin{aligned} f(p)=\sum _{q\ne j}p_q(t)E[\beta (t)] \end{aligned}$$

(36)

Thus, we can reach the following conclusion by differentiating both sides of the formula (36)

$$\begin{aligned} \frac{\vartheta f}{\vartheta p_j}=E[\beta _j(t)] \end{aligned}$$

(37)

Thus, the drift function $p_j(t)$ for our LA based routing algorithm can be represented as [substituting formula (37) in formula (34).]

$$\begin{aligned} p_j'(t) = \varLambda \cdot p_{j}(t)\cdot \sum _{q\ne j}p_{q} \left[ \frac{\vartheta f}{\vartheta p_j}-\frac{\vartheta f}{\vartheta p_q}\right] | j,q\in N_i \end{aligned}$$

(38)

Based on the definition in LA theory, $\varLambda$ represents the genericity maximum update rate. Through the above analysis, we find $p_j'(t)$ for our algorithm can be represented as the function of $p_j(t)$. Hence, we define a function $F_j()$ as following

$$\begin{aligned} \begin{aligned}&F_j(p_j)=p_j\cdot \sum _{q\ne j}p_{q} \left[ \frac{\vartheta f}{\vartheta p_j}-\frac{\vartheta f}{\vartheta p_q}\right] \\&\text {where}\\&p_j'(t)=\varLambda \cdot F_j(p_j) \end{aligned} \end{aligned}$$

(39)

Using the interpolation method to rewrite the action probability p(t) for our routing algorithm

$$\begin{aligned} p(t)=p^{\varLambda }(\tau ) \end{aligned}$$

(40)

where $\tau \in [t\varLambda ,(t+1)\varLambda ]$,$P^{\varLambda }(\tau )$ is a piecewise constant interpolation function. Now we just only care about the convergence of the interpolation sequence $P^{\varLambda }()$.

Generally, the genericity maximum update rate $\varLambda$ is close to 0. Hence based on the weak convergence theory [38], we can make an assertion that $p^{\varLambda }(\tau )$ can weakly converge to the solution set of an ordinary differential equation (ODE), which can be represented as

$$\begin{aligned} {\left\{ \begin{array}{ll} \frac{dx_{j}}{d\tau } =F_{j}(x) \\ x(0)=p^{\varLambda }(0)=p(0) \end{array}\right. } \end{aligned}$$

(41)

It should be explained why this conclusion is true. Firstly, based on the LA theory, $p(t+1)$, $\beta (t)$, $\alpha (t)$ constitute a Markov process(from the long term point of view, we do not prove this fact). Secondly, the outputs of the learning automata are finite (owing to the fact that the neighbors of node are finite). Thirdly, the feedback results of p(t) still take values between 0 and 1 (it means the probability space can be guaranteed). Fourthly, the feedback function defined in our algorithm $\varphi (t)$ is bounded, continuous and independent of $\varLambda$. Finally, this ODE has the unique solution for each initial x(0). Hence, we get the conclusion that when $\varLambda \rightarrow 0$, the sequence $p^\varLambda ()$ weakly converges to the solution of this ODE. Clearly, in our algorithm, $\varLambda$ meets this condition ($\varLambda$ in our algorithm is represented as a and b, and they all close to 0($a=b=0.10$)).

Remark 5

It needs to note that this ODE is a particular case of the weak convergence theory [38].