Mobile Networks and Applications

, Volume 15, Issue 5, pp 710–728 | Cite as

Opportunistic Fair Scheduling in Wireless Networks: An Approximate Dynamic Programming Approach

  • Zhi Zhang
  • Sudhir Moola
  • Edwin K. P. ChongEmail author


We consider the problem of temporal fair scheduling of queued data transmissions in wireless heterogeneous networks. We deal with both the throughput maximization problem and the delay minimization problem. Taking fairness constraints and the data arrival queues into consideration, we formulate the transmission scheduling problem as a Markov decision process (MDP) with fairness constraints. We study two categories of fairness constraints, namely temporal fairness and utilitarian fairness. We consider two criteria: infinite horizon expected total discounted reward and expected average reward. Applying the dynamic programming approach, we derive and prove explicit optimality equations for the above constrained MDPs, and give corresponding optimal fair scheduling policies based on those equations. A practical stochastic-approximation-type algorithm is applied to calculate the control parameters online in the policies. Furthermore, we develop a novel approximation method—temporal fair rollout—to achieve a tractable computation. Numerical results show that the proposed scheme achieves significant performance improvement for both throughput maximization and delay minimization problems compared with other existing schemes.


approximate dynamic programming fairness Markov decision process resource allocation scheduling stochastic approximation wireless networks 


  1. 1.
    Zhang Z, Moola S, Chong EKP (2008) Approximate stochastic dynamic programming for opportunistic fair scheduling in wireless networks. In: Proc. 47th IEEE conference on decision and control, Cancun, 9–11 December 2008, pp 1404–1409Google Scholar
  2. 2.
    Knopp R, Humblet P (1995) Information capacity and power control in single cell multiuser communications. In: Proc. IEEE int. conference on communications 1995, vol 1, pp 331–335Google Scholar
  3. 3.
    Andrews M, Kumaran K, Ramanan K, Stolyar A, Whiting P, Vijayakumar R (2001) Providing quality of service over a shared wireless link. IEEE Commun Mag 39(2):150–153CrossRefGoogle Scholar
  4. 4.
    Bender P, Black P, Grob M, Padovani R, Sindhushyana N, Viterbi A (2000) Cdma/hdr: a bandwidth-efficient high-speed wireless data service for nomadic users. IEEE Commun Mag 38(7):70–77CrossRefGoogle Scholar
  5. 5.
    Andrews M (2005) A survey of scheduling theory in wireless data networks. In: Proc. 2005 IMA summer workshop on wireless communicationsGoogle Scholar
  6. 6.
    Parkvall S, Dahlman E, Frenger P, Beming P, Persson M (2001) The high speed packet data evolution of wcdma. In: Proc. IEEE VTC 2001, vol 3, pp 2287–2291Google Scholar
  7. 7.
    Liu X, Chong EKP, Shroff NB (2001) Opportunistic transmission scheduling with resource-sharing constraints in wireless networks. IEEE J Sel Areas Commun 19(10):2053–2064CrossRefGoogle Scholar
  8. 8.
    Liu X, Chong EKP, Shroff NB (2003) A framework for opportunistic scheduling in wireless networks. Comput Netw 41(4):451–474zbMATHCrossRefGoogle Scholar
  9. 9.
    Liu X, Chong EKP, Shroff NB (2004) Opportunistic scheduling: an illustration of cross-layer design. Telecommun Rev 14(6):947–959Google Scholar
  10. 10.
    Wang HS, Moayeri N (1995) Finite-state markov channel—a useful model for radio communication channels. IEEE Trans Veh Technol 43:163–171CrossRefGoogle Scholar
  11. 11.
    Kelly F (1997) Charging and rate control for elastic traffic. Eur Trans Telecommun 8:33–37CrossRefGoogle Scholar
  12. 12.
    Puterman ML (1994) Markov decision processes. Wiley, New YorkzbMATHCrossRefGoogle Scholar
  13. 13.
    Bertsekas DP (2001) Dynamic programming and optimal control, 2nd ed. Athena, BelmontzbMATHGoogle Scholar
  14. 14.
    Ross SM (1970) Applied probability models with optimization applications. Dover, New YorkzbMATHGoogle Scholar
  15. 15.
    Derman C (1970) Finite sate Markovian decision processes. Academic, New YorkGoogle Scholar
  16. 16.
    Altman E (1998) Constrained Markov decision processes. Chapman and Hall/CRC, LondonGoogle Scholar
  17. 17.
    Piunovskiy AB (1997) Optimal control of random sequences in problems with constraints. Kluwer, DordrechtzbMATHGoogle Scholar
  18. 18.
    Feinberg EA, Shwartz A (eds) (2002) Handbook of Markov decision processes: methods and applications. Kluwer, BostonzbMATHGoogle Scholar
  19. 19.
    Ross KW (1989) Randomized and past-dependent policies for Markov decision processes with multiple constraints. Oper Res 37(3):474–477zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Piunovskiy AB, Mao X (2000) Constrained markovian decision processes: the dynamic programming approach. Oper Res Lett 27:119–126zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Chen RC, Blankenship GL (2004) Dynamic programming equations for discounted constrained stochastic control. IEEE Trans Automat Contr 49(5):699–709CrossRefMathSciNetGoogle Scholar
  22. 22.
    Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena, BelmontzbMATHGoogle Scholar
  23. 23.
    Bertsekas DP, Tsitsiklis JN, Wu C (1997) Rollout algorithms for combinatorial optimization. J Heuristics 3:245–262zbMATHCrossRefGoogle Scholar
  24. 24.
    Bertsekas DP, Castanon DA (1999) Rollout algorithms for stochastic scheduling problems. J Heuristics 5(1):89–108zbMATHCrossRefGoogle Scholar
  25. 25.
    Gesbert D, Slim-Alouini M (2004) How much feedback is multi-user diversity really worth? In: Proc. IEEE international conference on communications, pp 234–238Google Scholar
  26. 26.
    Floren F, Edfors O, Molin BA (2003) The effect of feedback quantization on the throughput of a multiuser diversity scheme. In: Proc. IEEE global telecommunication conference (GLOBECOM’03), pp 497–501Google Scholar
  27. 27.
    Svedman P, Wilson S, Cimini LJ Jr., Ottersten B (2004) A simplified opportunistic feedback and scheduling scheme for ofdm. In: Proc. IEEE vehicular technology conference 2004Google Scholar
  28. 28.
    Al-Harthi Y, Tewfik A, Alouini MS (2007) Multiuser diversity with quantized feedback. IEEE Trans Wirel Commun 6(1):330–337CrossRefGoogle Scholar
  29. 29.
    Parekh AK, Gallager RG (1993) A generalized processor sharing approach to flow control in integrated services networks: the single-node case. IEEE/ACM Trans Netw 1(3):344–357CrossRefGoogle Scholar
  30. 30.
    Kushner HJ, Yin GG (2003) Stochastic approximation and recursive algorithms and applications, 2nd ed. Springer, New YorkzbMATHGoogle Scholar
  31. 31.
    Chen HF (2002) Stochastic approximation and its applications. Kluwer, DordrechtzbMATHGoogle Scholar
  32. 32.
    Gilbert E (1960) Capacity of a burst-noise channel. Bell Syst Technol J 39:1253–1265Google Scholar
  33. 33.
    Elliott EO (1963) Estimates of error rates for codes on burst-noise channels. Bell Syst Technol 42:1977–1997Google Scholar
  34. 34.
    Swarts F, Ferreira HC (1993) Markov characterization of channels with soft decision outputs. IEEE Trans Commun 41:678–682CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringColorado State UniversityFt. CollinsUSA

Personalised recommendations