Dynamic VNF Resource Scaling and Migration: A Machine Learning Approach

Abstract

In this chapter, VNF scalability via joint resource scaling and migration is studied in a local network segment, to meet a probabilistic delay requirement in the presence of non-stationary traffic with changing traffic statistics. A change point detection scheme determines boundaries between stationary traffic segments in an online manner as new traffic samples arrive, and provides a triggering signal for VNF scalability. Under the fBm traffic model assumption, the resource demand for a probabilistic delay guarantee is predicted for each newly detected stationary traffic segment, based on traffic parameter learning with Gaussian process regression and fBm resource provisioning model. Given the predicted resource demand, a dynamic VNF migration problem is formulated as a Markov decision process (MDP) with variable-length decision epochs, to maximize the long-term reward integrating load balancing, migration cost, and resource overloading penalty. The MDP is solved by a reinforcement leaning (RL) approach. Specifically, a penalty-aware deep $\mathcal {Q}$-learning algorithm demonstrating advantages in reducing training loss and increasing cumulative reward is employed to learn the adaptive VNF migration actions under the dynamics in change points, resource demand, and background traffic.