A Markov Decision Process based flow assignment framework for heterogeneous network access
We consider a scenario where devices with multiple networking capabilities access networks with heterogeneous characteristics. In such a setting, we address the problem of efficient utilization of multiple access networks by devices via optimal assignment of traffic flows with given utilities to different networks. We develop and analyze a device middleware functionality that monitors network characteristics and employs a Markov Decision Process (MDP) based control scheme that in conjunction with stochastic characterization of the available bit rate and delay of the networks generates an optimal policy for allocation of flows to different networks. The optimal policy maximizes, under available bit rate and delay constraints on the access networks, a discounted reward which is a function of the flow utilities. The flow assignment policy is periodically updated and is consulted by the flows to dynamically perform network selection during their lifetimes. We perform measurement tests to collect traces of available bit rate and delay characteristics on Ethernet and WLAN networks on a work day in a corporate work environment. We implement our flow assignment framework in ns-2 and simulate the system performance for a set of elastic video-like flows using the collected traces. We demonstrate that the MDP based flow assignment policy leads to significant enhancement in the QoS provisioning (higher rate allocation, lower packet delays and packet loss rates) for the flows and better access network utilization, as compared to policies that allocate flows to different networks using greedy approaches or heuristics like average available bit rate on the networks.
KeywordsHeterogeneous access networks Flow assignment Markov Decision Process Simulative performance evaluation
- 2.IEEE 802.21. http://www.ieee802.org/21/.
- 3.Cuevas, A., Moreno, J. I., Vidales, P., & Einsiedler, H. (2006). The IMS platform: A solution for next generation network operators to be more than bit pipes. In IEEE Communications Magazine, Issue on Advances of Service Platform Technologies.Google Scholar
- 5.The Network Simulator (ns-2). http://www.isi.edu/ns/.
- 8.Alpcan, T., & Başar, T. (2003). Global stability analysis of an end-to-end congestion control scheme for general topology networks with delay. In Proc. of the 42nd IEEE Conference on Decision and Control (pp. 1092–1097). Maui, HI.Google Scholar
- 9.Chebrolu, K., & Rao, R. (2002). Communication using multiple wireless interfaces. In Proc. IEEE Wireless Communications and Networking Conference (WCNC 2002) (Vol. 1, pp. 327–331).Google Scholar
- 10.Fodor, G., Furuksar, A., & Lundsjo, J. (2004). On access selection techniques in always best connected networks. In Proc. ITC Specialist Seminar on Performance Evaluation of Wireless and Mobile Systems.Google Scholar
- 11.Zemlianov, A., & de Veciana, G. (2005). Cooperation and decision-making in a Wireless multi-provider setting. In Proc. IEEE INFOCOM (pp. 1–14).Google Scholar
- 12.Shakkottai, S., Altman, E., & Kumar, A. (2005). The case for non-cooperative multihoming of users to access points in IEEE 802.11 WLANs. In Proc. IEEE INFOCOM.Google Scholar
- 14.Mariz, D., Cananea, I., Sadok, D., & Fodor, G. (2006). Simulative analysis of access selection algorithms for multi-access networks. In Proc. IEEE Internation Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM 2006).Google Scholar
- 15.Thompson, N., He, G., & Luo, H. (2006). Flow scheduling for end-host multihoming. In Proc. IEEE INFOCOM.Google Scholar
- 16.Kumar, D., Altman, E., & Kelif, J.-M. (2007). Globally optimal user-network association in an 802.11 WLAN and 3G UMTS hybrid cell. In Proc. 20th International Telegraph Congress. Ottawa, Canada.Google Scholar
- 17.Alpcan, T., Singh, J. P., & Başar, T. (2007). A robust flow control framework for heterogeneous network access. In Proc. 5th International Symposium on Modeling and Optimization in Mobile, Ad-hoc, and Wireless Networks (WiOpt).Google Scholar
- 18.Zhu, X., Agrawal, P., Singh, J. P., Alpcan, T., & Girod, B. (2007). Rate allocation for multi-user video streaming over heterogeneous access networks. In Proc. ACM Multimedia.Google Scholar
- 19.Singh, J. P., Alpcan, T., Zhu, X., & Agrawal, P. (2007). Towards heterogeneous network convergence: Policies and middleware architecture for efficient flow assignment, rate allocation and rate control for multimedia applications. In Workshop on Middleware for Next-generation Converged Networks and Application (MNCNA), ACM/IFIP/USENIX 8th International Middleware Conference.Google Scholar
- 20.Guo, F., Chen, J., Li, W., & Chiueh, T. (2004). Experiences in building a multihoming load balancing system. In: Proc. IEEE INFOCOM.Google Scholar
- 21.Konrad, A., Zhao, B., Joseph, A., & Ludwig, R. (2003). A markov-based channel model algorithm for wireless networks. Wireless Networks (pp. 189–199).Google Scholar
- 22.Salmatian, K., & Vaton, S. (2001). Hidden Markov Modeling for network communication channels. In Proc. ACM SIGMETRICS.Google Scholar
- 23.Froyland, G. (2001). Extracting dynamical behaviour via Markov models. In Nonlinear Dynamics and Statistics: Proceedings, Newton Institute, Cambridge, 1998 (pp. 283–324). Birkhauser.Google Scholar
- 25.Navratil, J., & Cottrell, R. L., Abing. http://www-iepm.slac.stanford.edu/tools/abing/.
- 26.Laboratory, U. N. R., CapProbe. http://www.cs.ucla.edu/NRL/CapProbe/.