Joint Fronthaul Optimization and SDN Controller Placement in Dynamic 5G Networks

Abstract

To address the limitations of current Radio Access Networks (RANs), centralized-RANs adopting the concept of flexible splits of the BBU functions between Radio Units (RUs) and the central unit (CU) have been proposed. This concept can be implemented combining both the Mobile Edge Computing model and relatively large-scale centralized Data Centers. This architecture requires high bandwidth/low latency optical transport networks interconnecting RUs and compute resources adopting SDN control. This paper proposes a novel mathematical model based on Evolutionary Game Theory that allows to dynamically identify the optimal split option with the objective to unilaterally minimize the infrastructure operational costs in terms of power consumption. Optimal placement of the SDN controllers is determined by a heuristic algorithm in such a way that guarantees the stability of the whole system.

1 Introduction

The immense increase of network-connected devices and internet users, services and applications, creates huge bandwidth, mobility and speed demands, that cannot be fulfilled by existing technologies, mainly due to capital and operational costs [1]. Thus, a transition from current closed, static and unagile networks to open infrastructures that focus on flexible and optimized service delivery is needed. However, to achieve this, multiple requirements (data rate, latency, energy efficiency, bandwidth and network capacity etc.) have to be met, as different applications may require different network capabilities, features and performance.

In order to address the growth of traffic, network densification seems to be a promising solution. The main idea is to create very dense cells (Ultra Dense Networks - UDNs) by installing a large number of antennas with reduced range, achieving high bandwidth, and short delays [2]. Of course, this results in increased network capacity through the densification of the infrastructure. Although, we can solve the issue of increased traffic in data transmission by applying the classical technique of network densification, the economical and environmental impact of the investment in such an infrastructure should be taken into consideration. Beyond the increase in the associated capital expenditure significant increase in the overall energy consumption of the infrastructure is expected as additional base stations required to support the dense antenna deployment would be also needed. This will have a direct impact not only in the CO2 footprint of these solutions but also in the infrastructure operating costs directly associated with the energy consumption of the infrastructure.

Cloud Radio Access Networks (C-RANs) propose to overcome these limitations, by decoupling the BaseBand Units (BBUs) from the Base Stations (BSs) and place them in the Cloud, thus achieving a centralized manner of signal processing and management. The connection of the RU, and the Central Unit (CU), where the baseband processing is performed, is supported through an optical transport network. The combination of Cloud Computing with the centralized RAN architecture is ideal for planning shared network radio access, handling interference between nearby cells, and quickly and easily upgrading the network. Nevertheless, C-RAN suffers several limitations, the most important of which is the need for high capacity transport links to support fronthaul (FH) services, i.e. high bandwidth and very low latency connectivity between the RU and the CU [1]. Existing backhaul (BH) solutions are unable to offer the required capacity for the converged FH/BH transport network of future communication environments. In this manner, along with the adoption of advanced wireless and optical technologies (e.g. ub-6 GHz and 60 GHz bands, Wavelength Division Multiplexing (WDM) optical networks [1]), the concept of baseband processing split that allows some functions of the 5G protocol stack to be processed at the RUs, while the remaining ones to be processed at the CU has been proposed [1]. Flexible Functional Splits (FFS) specified by both 3GPPP and eCPRI [10] can relax FH requirements in terms of transport network specifications, but they may be both economically and environmentally inefficient. This is due to that, this architecture still requires the presence of computation and storage components at each RU in order to process the subset of BBU functions that will be dynamically decided to be performed locally at the RUs at different time instances.

To address this architectural inefficiency, Mobile Edge Computing (MEC) can be combined with the notion of flexible functional splits for making the system as cost and energy efficient as possible. In such a system, a set of low or medium processing power servers is placed in the wireless access domain or close to the edge of the network [3]. The processing capabilities that are required for the adoption of the FFS approach, can be removed from each RU and be placed in the MEC to which they are connected. Hence, the FFS technique can be addressed by adopting an architecture able to assign BBU processing functions dynamically between MEC servers and large-scale DCs placed at the optical access and metro domains that are hosting general purpose servers.

In general, 5G aims at incorporating many technologies, under the same infrastructure (FH/BH network). Efficient management and operation of such a heterogeneous infrastructure can be achieved applying novel network designs that are aligned with the Software Defined Networking (SDN) open reference architecture [4]. SDN refers to the migration of the control level out of the switches and its placement externally in a logical entity called a controller. The controller is in charge of populating the forwarding table of the switch. The communication between the two entities is carried out through a secure channel. This centralized structure makes the controller able to perform network management functions, while allowing easy modification of the network behavior through the centralized control layer. However, in such infrastructures the end to end latency is augmented. Considering that latency is critical to many network applications, a subject of current research is the size of the SDN network (controller placement problem) in order to be able to cope with the timing requirements of network services and applications [5, 6].

In this paper, we propose a next generation network solution that includes: (a) the concept of FFS between a set of servers that can offer a range of processing capabilities and can be geographically distributed across the network infrastructure, (b) the employment of the MEC architecture in the form of specific purpose low processing power servers embedded in the wireless access network (also known as cloudlets) and (c) a FH/BH transport network with SDN control, connecting the MEC domains with medium to large-scale DCs hosting general purpose servers placed at the optical access and metro domains. In this environment, the controller placement problem is investigated, under the scope of the stability of the whole system. To address this issue, we propose a novel mathematical model based on Evolutionary Game Theory (EGT) that allows network operators to dynamically adjust their FH split options with the objective to minimize their total operational expenditures. The stability of the proposed scheme depends on network latency, thus a metric for sizing the SDN FH/BH network is proposed.

The rest of the paper is organized as follows. After a brief overview of EGT in Sect. 2 the problem under consideration is analyzed in Sect. 3. Then, its application to the proposed network model is presented in Sect. 4, where the optimal split is identified applying EGT and the controller placement problem is addressed. Finally, conclusions are drawn in Sect. 5.

2 Evolutionary Game Theory: Basic Concepts

Evolutionary Game Theory (EGT) studies the interactions of non-cooperative players that play repeatedly strategic games [7]. Contrary to classic Game Theory that examines the behavior of rational players, EGT focuses on how the strategies can “survive” through evolution and how they help the players who choose them to “strengthen” and better meet their needs.

Evolutionary processes are described by three main components: the population, the game and the dynamical model that describe the processes. The most common dynamics is called the Replicator Equation (RE) and can be expressed as:

$$ \dot{x}_{i} \left( t \right) = x_{i} \left( t \right)\left( {F_{i} \left( {\varvec{x}\left( t \right)} \right) - \overline{F} \left( {\varvec{x}\left( t \right)} \right)} \right), \;i \in S $$

(1)

where $S$ is the set of strategies that are available to the population, $\varvec{x}\left( t \right) = \left[ { x_{1} \left( t \right)\; x_{2} \left( t \right)\; \ldots \;x_{i} \left( t \right)\; \ldots } \right] ^{T}$ is the population state at time t with $x_{i} \left( t \right)$ symbolizing the proportion of the population that uses strategy i at time $t,$ and $F_{i} \left( {\varvec{x}\left( t \right)} \right)$, $\overline{F} \left( {\varvec{x}\left( t \right)} \right)$ are the expected payoff of strategy i and the mean payoff respectively [7]. According to this equation the percentage growth rate $\dot{x}_{i} /x_{i}$ of the strategies that are currently used is equal to the excess of the current payoff versus the average population’s payoff. This means that strategies employed at present will be spread or eliminated depending on whether their payoff is better or worse than the average.

In the above, the interaction between individuals is assumed to be instant and their results immediate. However, this is not the case in most realistic scenarios. In communication networks especially, the impact of an action may be belated, due to network latency. Thus, it is more realistic to consider a system where the strategies evolve considering the payoff values perceived in a past moment. The adjusted RE is given below [8, 9]:

$$ \dot{x}\left( t \right) = x_{\varvec{i}} \left( t \right) \cdot \left( {f_{i} \left( {\varvec{x}\left( {t - \tau } \right)} \right) - \sum\nolimits_{i \in S} {x_{j} \left( t \right) \cdot f_{j} \left( {\varvec{x}\left( {t - \tau } \right)} \right)} } \right) $$

(2)

3 Application to 5G Networks

We consider the 5G network topology shown in Fig. 1. In this scenario, the RUs are installed, managed and operated by coexisting Mobile Network Operators (MNOs). The RUs share a set of computational resources that are located both at the edge of the access network (in a MEC server) and at the metro/core network (in the Cloud). The interconnection between the MEC server and the central cloud servers is carried out by an SDN- controlled optical FH/BH transport network.

Fig. 1.

Network architecture. In the MEC, a decision about which functions should be processed locally is made for each RU. The remaining set of functions for each RU are transferred through a common network infrastructure with centralized control to a DC for further processing.

The MNOs can decide where to perform the processing of the low layer functions of the LTE protocol stack. According to the eCPRI specification, three possible functional splits can be identified [10]. In split E (split 1 for simplicity) MEC is responsible for the RF processing of the received signals and the Cloud performs the entire baseband processing. In split IU (split 2), MEC handles the per cell processing (RF processing, cyclic prefix (CP) elimination, frequency domain transformation (FFT) and resource demapping), while the remaining functions are performed at the Cloud (Equalization, IDFT, QAM, multi-antenna processing, Forward Error Correction (FEC), higher level operations (MAC, RLC, PDCP). Finally, in split D (split 3) the entire lower layer function chain is performed at the MEC server, and the higher lever functions in the Cloud. One can conclude that as the split is placed lower in the 5G protocol stack, the required transport capacity increases [11].

Each RU periodically selects one of the three possible functional splits with probability $x_{i}$, $i = 1,..,3$. The decisions are sent to the SDN controller, who is responsible for the application of the policies. We consider the scenario in which all the necessary resources are available. When the policies have been applied, the payoffs are calculated and the RUs are reviewing their split option strategy. Specifically, if a better (lower) payoff is observed, then the probability of an RU to select the specific split option increases (decreases). The new policies are sent to the controller and the same procedure is repeated. The time between each repetition is referred to as revision time. To address this scenario, EGT can provide a suitable optimization framework that can be used to support energy-aware FH service provisioning over a common infrastructure.

Denote as $\varvec{x}\left( t \right) = \left[ { x_{1} \left( t \right)\;x_{2} \left( t \right)\;x_{3} \left( t \right)} \right]^{T}$ the state vector of the RU, where $x_{i} \left( t \right)$ refers to the RU’s probability of choosing split i. If the RU revises its strategy with a time rate $r_{i} \left( \varvec{x} \right)$, the change of the proportion of the probabilities is described by the following dynamical equation:

$$ \dot{x}_{i} \left( t \right) = \sum\nolimits_{j \in S} {x_{j} \left( t \right)r_{j} \left( {\varvec{x}\left( t \right)} \right)p_{j}^{i} \left( {\varvec{x}\left( t \right)} \right)} - \sum\nolimits_{j \in S} {x_{i} \left( t \right)r_{i} \left( {\varvec{x}\left( t \right)} \right)p_{i}^{j} \left( {\varvec{x}\left( t \right)} \right)} $$

(3)

where $S$ is the set of strategies that consists of the three possible splits and $p_{i}^{j} \left( \varvec{x} \right)$ is the rule of change in the probability of choosing split i when the RU changes from split i to split $j$ and can be expressed as:

$$ p_{i}^{j} \left( {\varvec{x}\left( t \right)} \right) = \left\{ {\begin{array}{*{20}c} {x_{j} \left( t \right)\left( {u\left( {j,t} \right) - u\left( {i,t} \right)} \right) \;j \ne i} \\ {1 - \sum\nolimits_{j \ne i} {x_{j} \left( t \right)\left( {u\left( {j,t} \right) - u\left( {i,t} \right)} \right)} \; otherwise} \\ \end{array} } \right. $$

(4)

with $u\left( {i,t} \right)$ symbolizing the payoff of split i at time $t$. Constituting Eqs. (4) to (3) and making the assumption that all time rates are constantly equal to one $\left( {r_{i} \left( \varvec{x} \right) \equiv 1} \right)$, the following differential equation comes up:

$$ \dot{x}_{i} \left( t \right) = x_{i} \left( t \right)\left[ {u\left( {i,t} \right) - \sum\nolimits_{j \in S} {x_{j} \left( t \right)u(j,t)} } \right] $$

(5)

which satisfies the replicator dynamics model introduced in Sect. 2.

3.1 Payoff Function

The objective of the MNOs is to minimize their own service power consumption requirements and, hence, the service operational costs. Thus, the payoff function per operator is formed by summing up the power consumption of the network and compute elements required to support FH services. Table 1 summarizes the network and processing demands of each functional split.

Table 1. Network and processing demands of each functional split

For this problem setting, the payoff of an RU operated by an MNO that chooses split i against another RU operated by a different MNO who chooses split $j$ is described by the payoff matrix $\varvec{A}$, with elements:

$$ a_{ij} = - \left( {P_{{PROCESSING_{ij} }} + P_{{NET_{ij} }} } \right) + b, \quad i,j \in S $$

(6)

where $P_{PROCESSING}$ and $P_{{NET_{ij} }}$ refer to the total compute and network energy consumption respectively, when split i competes with split $j$ and $b$ is a positive constant that guarantees the robustness of the system. Technical parameters like the oversampling factor $(N_{o} )$, the sampling frequency $(f_{s} )$, the quantization bits per I/Q $\left( {N_{Q} } \right)$, the number of receiving antennas $(N_{R} )$, the number of subcarriers used $\left( {N_{sc} } \right)$, the percentage of used resource elements $\left( \eta \right)$, and the spectral efficiency $\left( S \right)$ affect the required capacity and the power consumption of each processing function [11, 12].

Due to the nature of the SDN transport network, the payoff values are provided to the MNOs through the SDN controller. It is evident, that this kind of procedure indicates that the strategies will evolve based on information related to a past moment. This will be reflected to the expected payoff of the strategies.

Network delay is mainly composed of propagation, serialization, switching/routing and queuing delay. Although propagation and switching/routing delays are constant, the rest are highly affected by the network traffic. Due to this, we expect that network delay is a random variable that is characterized by a probability density function. Specifically, if the payoff is received not instantly, but after a random delay $\tau$, with probability distribution $P\left( t \right)$ the expected payoff of an RU using strategy $i$ as well as the average payoff are determined by [13]:

$$ u\left( {i,t} \right) = \int_{0}^{\infty } {P\left( \tau \right)\left( {\varvec{{\it A}x}\left( {t - \tau } \right)} \right)_{i} } \quad \& \quad \overline{u} = \sum\nolimits_{j \in S} {x_{j} \left( t \right)u(j,t)} $$

(7)

3.2 Stability Analysis

Substituting Eqs. (7) in (5) we get a nonlinear system of differential equations. Since this system cannot be easily solved by analytical methods it is important to examine its qualitative behavior without actually solving it. We concentrate on finding the stability of a solution exploiting the Lyapunov stability theorem. This method is based on the expansion of the right part of the dynamical system as a Taylor series about an equilibrium point $\varvec{x}^{0}$. If the initial condition $\varvec{x}\left( 0 \right) = \varvec{x}_{0}$ is close enough to $\varvec{x}^{0}$, then $\varvec{x}$ will be a small perturbation for some time interval extending from zero. Thus, it is acceptable to neglect the higher-order terms, and approximate the nonlinear system by the linear system [9]:

$$ \dot{\varvec{x}}\left( t \right) = \varvec{J}_{\varvec{o}} \varvec{x}\left( t \right) + \varvec{J}_{1} \varvec{x}\left( {t - \tau } \right) $$

(8)

where $\varvec{J}_{o} \in {\mathbb{R}}^{2x2}$ and $\varvec{J}_{1} \in {\mathbb{R}}^{2x2}$ are respectively, the Jacobian matrix, and the delayed Jacobian matrix evaluated at equilibrium at $\varvec{x}^{0}$.

The stability of the system requires that all roots of its characteristic equation have a negative real part. The characteristic equation can be expressed as:

$$ \det \left( {\varvec{I}\lambda - \varvec{J}_{o} - \varvec{J}_{1} Q} \right) = 0 \Rightarrow \lambda^{2} + D\lambda + E\lambda Q + FQ^{2} + GQ + H = 0 $$

(9)

where $\lambda \epsilon {\mathbb{C}},\text{ }\varvec{\it I}$ is the $N{\text{x}}N$ identity matrix, $Q = \int_{0}^{\infty } {P\left( \tau \right)e^{ - \lambda \tau } }$ corresponds to the Laplace transform of the delayed term in Eq. (8) and the parameters depend on the Jacobian matrices’ elements.

The system admits to seven equilibrium points: three corner points, one interior and three corner side points. The linearization about each of the three corner critical points produces an ordinary differential equation that is independent of the delayed variables as in the non-delayed three strategies game.

At the interior critical point all the payoffs are equal. The differential system that emerges depends only on the delayed variables, thus one should anticipate that the distributed delay will affect its stability. The characteristic equation is formed as:

$$ u^{2} + E \cdot u + F = 0, \; u = \frac{\lambda }{Q} $$

(10)

The last three critical points are equilibriums where only two of the three strategies survive (corner side points). Their characteristic equation can be written as:

$$ \left( {\lambda - l_{1} } \right) \cdot \left( {\lambda - l_{2} Q} \right) = 0 $$

(11)

where the parameters l_1 and l_2 depend on the corner side equilibrium point.

As we can conclude from the above, our analysis can be restricted for finding the solution of the equation:

$$ \lambda - C\int_{0}^{\infty } {P\left( \tau \right)e^{ - \lambda \tau } = 0} $$

(12)

The above equation is the characteristic equation of the linear differential equation:

$$ \dot{x}\left( t \right) = C\int_{0}^{\infty } {x\left( {t - \tau } \right)f\left( t \right)d\tau } $$

(13)

Thus, the conclusions derived for the stability of Eq. (13) can be expanded to our case. From [14] we derive the following necessary and sufficient condition for the asymptotic stability of the equilibriums:

Proposition:

If $C < 0$ and the expected value $\left( E \right)$ of the delay’s probability density satisfies the condition:

$$ E\left( \tau \right) < \frac{\pi }{\gamma \cdot \left| C \right|} $$

(14)

where $\gamma = 2$ when the pdf is symmetrical, or else $\gamma = { \sup }\{ \gamma |\cos \,w = 1 - \frac{\gamma w}{\pi }, w\; > \;0 \}$, then the equilibrium point is stable (the proof can be found in [14]). Last but not least, as far as the variance of the distribution is concerned, the stability of the system increases as the variance grows [14].

3.3 SDN Controller Placement

As it was mentioned earlier, the SDN controller is responsible for collecting and providing to the MNOs of the RUs the required information from all controlled devices. The maximum delay corresponds to the delay of the most distant node to the controller path plus the delay of the controller-MEC path. Thus, assuming that each controlled device may host a MEC, the stability of the system is achieved only when the round-trip time (RRT) of the controller’s path to the most remote device is less than the limit imposed by Eq. (14). Based on this limit, we propose a heuristic algorithm that tries to identify the minimum number and associated position of SDN controllers with the aim to guarantee the stability of the 5G infrastructure. This is performed with low computational complexity.

At first, the heuristic algorithm finds the maximum network radius, that is the number of hops of the longest end-to-end path. Then, for each node it calculates the maximum RRT to all the other nodes inside the network radius. If the result of all nodes is a number higher than the limit imposed in Eq. (14), the network radius is reduced by one, and the same procedure is repeated, until a case is found where the RRT from a node to all other nodes within the network radius meets the condition of Eq. (14). The nodes that meet this requirement, are marked as possible controller candidates. From this set, the algorithm chooses as the first controller the one that is connected to the largest number of devices within the network radius. These devices and the first controller are removed from the network, and the whole procedure is repeated for the downscaled network. The algorithm ends when the downscaled network has no network nodes.

4 Results and Discussion

In order to see the effectiveness of our model we considered the system described in Fig. 1 with the system parameters shown in Table 2. The cost ratio (remote/central processing) was assumed to be equal to two. Furthermore, the relationship of the transport network’s energy consumption with the required capacity for the support of the FH services was assumed to be nonlinear, since the non-linear model is best to describe the technology advancements in terms of energy efficiency of network devices [12, 15].

Table 2. Parameters of the system configuration

The stability analysis of system (5) indicates that the equilibrium point in such a scenario is $x_{1}^{*} = 0.2957, \;x_{2}^{*} = 0.7043, \;x_{3}^{*} = 0$. This means that in the non-delayed system the optimal split choice is split 2. However, as it was stated previously the SDN transport network introduces additional delay to the system. This delay can be divided to two main components, namely the processing delay of the SDN controller and the transport delay.

The SDN controller chosen for the implementation is the Opendaylight controller (ODL), that is a scalable controller infrastructure that supports SDN implementations in modern heterogeneous networks of different vendors [16]. For measuring the processing delay of the ODL controller, we developed an application that communicates externally with the controller. For evaluations, a linear network topology with Out of Band control plane was emulated in Mininet, a tool that can emulate and perform the functions of network devices in a single physical host or virtual machine (VM) [17]. Both Mininet and Opendaylight controller were implemented on the same machine (Intel® Core™ i5-7400U CPU @ 3.00 GHZ (4 cores)) to overcome the Ethernet interface speed limitations. 7.7 GiB of memory was available. The system was running Ubuntu 16.04 LTS-64 bit. The application implements at first step a mechanism for collecting data on the network topology and at second step a mechanism for sending echo messages to all switches simultaneously, and measuring the maximum time elapsed for receiving a reply. The time response of ODL is measured by averaging the results of a hundred number of tests, in order to achieve higher accuracy. The results showed an exponential relation between the controller’s processing delay and the network devices.

As far as the transport delay is concerned, we used monthly delay measurements extracted from GRNET [18], in order to find the dependence of the end to end transport delay on the end to end hops. Our analysis concluded that this relationship can be well approximated with a linear function. Furthermore, the best pdf that fitted the end to end delay was the generalized t-student distribution [19].

Taking these into consideration, we expect that the total induced SDN network’s delay will be a random variable that is characterized by the generalized t-student distribution, with expected value that depends on the size of the transport network and the hops between two network nodes. Thus, the upper delay limit for our example is given by Eq. (14) as: $E_{max} = 1.6449$ time units.

The assumed FH/BH transport network’s topology for our example is depicted in Fig. 2. The figure also shows the possible controller placements after implementing the heuristic algorithm described in the previous section. In order to test the validity of the heuristic, we investigate the evolution of strategies in two cases: (1) when the controller is placed in one of the proposed locations by the heuristic, (2) when the controller is placed in the location identified by the average propagation latency optimization technique described in [5]. Figure 3 illustrates the evolution of split option selection probability under the proposed EGT based approach and the average latency minimization scheme described above. As can be seen in the former case (Fig. 3a) after few sampling periods the scheme converges to a mixed solution where all antennas operate under a single split option mode that will be either split 1 or split 2. However, in the second case, the placement of the SDN controller at a node that does not satisfy the stability threshold imposed by Eq. (14) leading to an unstable operational mode for the 5G network. The reason behind this is that the increased control plane delay in this case introduces inaccurate information of the network status at the controller. Therefore, decision making is performed with outdated information that leads to an oscillation around the optimal operating point preventing it from converging to a stable solution.

Fig. 2.

Assumed FH/BH transport network for the system described in Fig. 1. The red circle represents the position of the SDN controller, after the implementation of the heuristic algorithm described in Sect. 3.3. The red square represents the optimal position estimated according to the average propagation latency-case described in [5]. (Color figure online)

Fig. 3.

Evolution of the probabilities of the three split options, with the parameters described in Table 2, when: (a) the controller is placed in the proposed location (red circle in Fig. 2) by the heuristic, (b) the controller is placed in the proposed location (red square in Fig. 2) of the average propagation latency-case described in [5]. (Color figure online)

5 Conclusion

To address the limitations of current RANs, centralized-RANs adopting the concept of flexible splits of the BBU functions between RUs and the CU have been proposed. To achieve further efficiency gains in terms of cost and energy consumption we proposed the implementation of this architectural model exploiting compute resources, required for the BBU function processing, located both at the MEC and relatively large-scale centralized DCs. This architecture adopts high bandwidth/low latency SDN controlled optical transport networks. In this scenario, and with the aim to dynamically identify the optimal split option that minimize infrastructure operational costs in terms of power consumption we have proposed a novel mathematical model based on EGT. In addition, optimal placement of the transport network SDN controllers is determined by a heuristic algorithm with the objective to guarantee the stability of the whole system.