Control Problems in Telecommunications: The Heavy Traffic Approach

  • Harold J. Kushner
Part of the The Springer International Series in Engineering and Computer Science book series (SECS, volume 629)


The goal of this chapter is to demonstrate the usefulness of analytical and numerical methods of stochastic control theory in the design, analysis and control of telecommunication networks. The emphasis will be concentrated on the heavy traffic approach for queueing type systems in which there is little idle time and the queue length processes can be approximated by reflected diffusion processes under suitable scaling. Three principal problems are considered: the multiplexer system, controlled admission in multiserver systems such as ISDN, and the polling or scheduling problem.


Control Problem Idle Time Wiener Process Heavy Traffic Good Effort 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    E. Altman. Applications of Markov decision processes in telecommunications: a survey. In E. Feinberg and A. Shwartz, editors, Markov Decision Processes: Models, Methods, Directions and Open Problems. Kluwer, Amsterdam, 2000.Google Scholar
  2. [2]
    E. Altman and H.J. Kushner. Admission control for combined guaranteed performance and best effort communications systems under heavy traffic. SIAM J. Control and Optimization, 37:1780–1807, 1999.MathSciNetMATHCrossRefGoogle Scholar
  3. [3]
    E. Altman and H.J. Kushner. Control of polling in presence of vacations in heavy traffic with applications to satellite and mobile radio systems. Submitted to SIAM J. on Control and Optimization, 1999.Google Scholar
  4. [4]
    V. Anantharam and J. Walrand. Special issue on control methods for communications networks. Automatica, 1999.Google Scholar
  5. [5]
    D. Anick, D. Mitra, and M.M. Sondhi. Stochastic theory of a data handling system with multiple sources. Bell Systems Technical J., 61:1971–1894, 1982.MathSciNetGoogle Scholar
  6. [6]
    S.L. Bell and R.J. Williams. Dynamic scheduling of a system with two parallel servers in heavy traffic with complete resource pooling; asymptotic optimality of a continuous review threshold policy. Preprint, Math. Dept., University of California, La Jolla, 1999.Google Scholar
  7. [7]
    P. Billingsley. Convergence of Probability Measures. Wiley, New York, 1968.MATHGoogle Scholar
  8. [8]
    A.A. Borovkov. Some limit theorems in the theory of mass service II. Theory of Probability and its Applications, 10:375–400, 1965.CrossRefGoogle Scholar
  9. [9]
    A.A. Borovkov. Asymptotic Methods in Queueing Theory. Wiley, New York, 1984.MATHGoogle Scholar
  10. [10]
    O.J. Boxma and H. Tagaki. Editors: Special issue on polling systems. Queueing Systems, 11, 1992.Google Scholar
  11. [11]
    M. Bramson. State space collapse with application to heavy traffic limits for multiclass queueing networks. Queueing Systems, 30:89–148, 1998.MathSciNetMATHCrossRefGoogle Scholar
  12. [12]
    P. Brémaud. Point Processes and Queues. Springer-Verlag, New York and Berlin, 1981.MATHCrossRefGoogle Scholar
  13. [13]
    J. Dai. Steady state analysis of reflected Brownian motions: characterization, numerical methods and queueing applications. PhD thesis, Operations Research Dept., Stanford University, 1990.Google Scholar
  14. [14]
    J. G. Dai and J. M. Harrison. Steady state analysis of RBM in a rectangle: numerical methods and a queueing application. Ann. Appl. Probab., 1:16–35, 1991.MathSciNetMATHCrossRefGoogle Scholar
  15. [15]
    J. G. Dai and J. M. Harrison. Reflected Brownian motion in an orthant: numerical methods for steady state analysis. Ann. Appl. Probab., 2:65–86, 1992.MathSciNetMATHCrossRefGoogle Scholar
  16. [16]
    J. G. Dai and J. M. Harrison. The QNET method for two moment analysis of closed manufacturing systems. Ann. Appl. Probab., 3:968–1012, 1993.MathSciNetMATHCrossRefGoogle Scholar
  17. [17]
    J.G. Dai, D.H. Yeh, and C. Zhou. The QNET method for reentrant queueing networks with priority disciplines. Operations Research, 45:610–623, 1997.MATHCrossRefGoogle Scholar
  18. [18]
    P. Dupuis and H. Ishii. On oblique derivative problems for fully nonlinear second order elliptic partial differential equations on nonsmooth domains. Nonlinear Anal., 15:1123–1138, 1990.MathSciNetMATHCrossRefGoogle Scholar
  19. [19]
    P. Dupuis and H. Ishii. On Lipschitz continuity of the solution mapping to the Skorokhod problem, with applications. Stochastics Stochastics Rep., 35:31–62, 1991.MathSciNetMATHGoogle Scholar
  20. [20]
    R. Elliott. Stochastic Calculus and Applications. Springer-Verlag, Berlin and New York, 1982.MATHGoogle Scholar
  21. [21]
    A.I. Elwalid and D. Mitra. Fluid models for the analysis and design of statistical multiplexing with loss priorities on multiple classes of bursty traffic. In Proc. IEEE INFOCOM′92, pages 415–425, New York, 1992. IEEE Press.Google Scholar
  22. [22]
    A. Ephremides and S. Verdu. Control and optimization methods in communication network problems. IEEE Trans, on Automatic Control, 34:930–942, 1989.MathSciNetMATHCrossRefGoogle Scholar
  23. [23]
    S.N. Ethier and T.G. Kurtz. Markov Processes: Characterization and Convergence. Wiley, New York, 1986.MATHGoogle Scholar
  24. [24]
    G.J. Foschini. On heavy traffic diffusion analysis and dynamic routing in packet switched networks. In K.M. Chandy and M. Reiser, editors, Computer Performance. North Holland, Amsterdam, 1977.Google Scholar
  25. [25]
    G.J. Foschini. Equilibria for diffusion models for pairs of communicating computers—symmetric case. IEEE Trans. Inf. Theory, 28:273–284, 1982.MathSciNetMATHCrossRefGoogle Scholar
  26. [26]
    G.J. Foschini and J. Salz. A basic dynamic routing problem and diffusion approximation. IEEE Transactions on Communications, 26:320–327, 1978.MATHCrossRefGoogle Scholar
  27. [27]
    J. M. Harrison and L.M. Wein. Scheduling networks of queues: heavy traffic analysis of a simple open network. Queueing Systems, 5:265–280, 1989.MathSciNetMATHCrossRefGoogle Scholar
  28. [28]
    J.M. Harrison. Brownian models of queueing networks with heterogeneous customer populations. In Proc. of the IMA, volume 10, pages 147–186, Berlin and New York, 1988. Springer-Verlag. Edited by P.-L. Lions and W. Fleming.Google Scholar
  29. [29]
    J.M. Harrison. The BIGSTEP approach to flow management in stochastic processing networks. In F.P. Kelly, S. Zachary, and I. Ziedins, editors, Stochastic Networks: Theory and Applications. Oxford University Press, Oxford, 1996.Google Scholar
  30. [30]
    J.M. Harrison. Heavy traffic analysis of a system with parallel servers: asymptotic optimality of discrete review policies. Ann. Appl. Probab., 8: 822–848, 1998.MathSciNetMATHCrossRefGoogle Scholar
  31. [31]
    J.M. Harrison and M.J. Lopez. Heavy traffic resource pooling in parallel server systems. To appear: Queueing Systems, 1999.Google Scholar
  32. [32]
    J.M. Harrison and Vien Nguyen. The QNET method for two moment analysis of open queueing networks. Queueing Systems, 6:1-32, 1990.MathSciNetMATHCrossRefGoogle Scholar
  33. [33]
    J.M. Harrison and M.I. Reiman. Reflected Brownian motion on an orthant. Ann. Probab., 9:302–308, 1981.MathSciNetMATHCrossRefGoogle Scholar
  34. [34]
    J.M. Harrison and J. Van Mieghem. Dynamic control of Brownian networks: state space collapse and equivalent workload formulations. Ann. Appl. Probab., 7:747–771, 1997.MathSciNetMATHCrossRefGoogle Scholar
  35. [35]
    D.L. Iglehart and W. Whitt. Multiple channel queues in heavy traffic I. Adv. Appl. Probab., 2:150–177, 1970.MathSciNetMATHGoogle Scholar
  36. [36]
    D.L. Iglehart and W. Whitt. Multiple channel queues in heavy traffic II: sequences, networks and batches. Adv. Appl. Probab., 2:355–369, 1970.MathSciNetMATHCrossRefGoogle Scholar
  37. [37]
    D.L. Iglehart and W. Whitt. Multiple channel queues in heavy traffic III: random server selection. Adv. Appl. Probab., 2:370–375, 1970.CrossRefGoogle Scholar
  38. [38]
    J. Jacod and A.N. Shiryaev. Limit Theorems for Stochastic Processes. Springer Verlag, Berlin and New York, 1987.MATHGoogle Scholar
  39. [39]
    F.P. Kelly and C.N. Laws. Dynamic routing in open queueing networks: Brownian models, cut constraints and resource pooling. Queueing Systems, 13:47–86, 1993.MathSciNetMATHCrossRefGoogle Scholar
  40. [40]
    C. Knessl. On the diffusion approximation to two parallel queues with processor sharing. IEEE Trans, on Automatic Control, 36:1356–1367, 1991.MathSciNetMATHCrossRefGoogle Scholar
  41. [41]
    C. Knessl and J. A. Morrison. Heavy traffic analysis of a data handling system with multiple sources. SIAM J. Appl. Math., 51:187–213,1991.MathSciNetMATHCrossRefGoogle Scholar
  42. [42]
    T.G. Kurtz. Approximation of Population Processes, volume 36 of CBMS-NSF Regional Conf. Series in Appl. Math. SIAM, Philadelphia, 1981.CrossRefGoogle Scholar
  43. [43]
    H.J. Kushner. Probability Methods for Approximations in Stochastic Control and for Elliptic Equations. Academic Press, New York, 1977.MATHGoogle Scholar
  44. [44]
    H.J. Kushner. Optimality conditions for the average cost per unit time problem with a diffusion model. SIAM J. Control Optim., 16:330–346, 1978.MathSciNetMATHCrossRefGoogle Scholar
  45. [45]
    H.J. Kushner. Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory. MIT Press, Cambridge, Mass., 1984.MATHGoogle Scholar
  46. [46]
    H.J. Kushner. Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim., 28:999–1048, 1990.MathSciNetMATHCrossRefGoogle Scholar
  47. [47]
    H.J. Kushner. Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems, volume 3 of Systems and Control. Birkhäuser, Boston, 1990.CrossRefGoogle Scholar
  48. [48]
    H.J. Kushner. Control of trunk line systems in heavy traffic. SIAM J. Control Optim., 33:765–803, 1995.MathSciNetMATHCrossRefGoogle Scholar
  49. [49]
    H.J. Kushner. Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Springer Verlag, Berlin and New York, 2001.Google Scholar
  50. [50]
    H.J. Kushner and Y.N. Chen. Optimal control of assignment of jobs to processors under heavy traffic. to appear: Stochastics and Stochastic Reports, 1999.Google Scholar
  51. [51]
    H.J. Kushner and P. Dupuis. Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag, Berlin and New York, 1992.MATHCrossRefGoogle Scholar
  52. [52]
    H.J. Kushner, D. Jarvis, and J. Yang. Controlled and optimally controlled multiplexing systems: A numerical exploration. Queueing Systems, 20:255–291, 1995.MathSciNetMATHCrossRefGoogle Scholar
  53. [53]
    H.J. Kushner and L.F. Martins. Numerical methods for stochastic singular control problems. SIAM J. Control Optim., 29:1443–1475, 1991.MathSciNetMATHCrossRefGoogle Scholar
  54. [54]
    H.J. Kushner and L.F. Martins. Heavy traffic analysis of a data transmission system with many independent sources. SIAM J. Appl. Math., 53:1095–1122, 1993.MathSciNetMATHCrossRefGoogle Scholar
  55. [55]
    H.J. Kushner and L.F. Martins. Numerical methods for controlled and uncontrolled multiplexing and queueing systems. Queueing Systems, 16:241–285, 1994.MathSciNetMATHCrossRefGoogle Scholar
  56. [56]
    H.J. Kushner and L.F. Martins. Heavy traffic analysis of a controlled multi class queueing network via weak convergence theory. SIAM J. on Control and Optimization, 34:1781–1797, 1996.MathSciNetMATHCrossRefGoogle Scholar
  57. [57]
    H.J. Kushner and K.M. Ramachandran. Optimal and approximately optimal control policies for queues in heavy traffic. SIAM J. Control Optim., 27:1293–1318, 1989.MathSciNetMATHCrossRefGoogle Scholar
  58. [58]
    H.J. Kushner and J. Yang. Numerical methods for controlled routing in large trunk line systems via stochastic control theory. ORSA J. Computing, 6:300–316, 1994.MATHCrossRefGoogle Scholar
  59. [59]
    H.J. Kushner and J. Yang. An effective numerical method for controlling routing in large trunk line networks. Math. Computation Simulation, 38:225–239, 1995.MathSciNetMATHCrossRefGoogle Scholar
  60. [60]
    C.N. Laws. Resource pooling in queueing networks with dynamic routing. Adv. in Appl. Probab., 24:699–726, 1992.MathSciNetMATHCrossRefGoogle Scholar
  61. [61]
    J.P. Lehoczky and S.E. Shreve. Absolutely continuous and singular stochastic control. Stochastics Stochastics Rep., 17:91–110, 1986.MathSciNetMATHGoogle Scholar
  62. [62]
    A.J. Lemoine. Networks of queues: A survey of weak convergence results. Management Sci., 24:1175–1193, 1978.MathSciNetMATHCrossRefGoogle Scholar
  63. [63]
    H. Levy and M. Sidi. Polling models: applications, modelling and optimization. IEEE Trans, on Communications, 38:1750–1760, 1991.CrossRefGoogle Scholar
  64. [64]
    R. Liptser and A.N. Shiryaev. Statistics of Random Processes. Springer-Verlag, Berlin and New York, 1977.MATHGoogle Scholar
  65. [65]
    A. Mandlebaum and G. Pats. State-dependent stochastic networks, part i: Approximations and applications with continuous diffusion limits. Annals of Applied Probability, 8:569–646, 1998.MathSciNetCrossRefGoogle Scholar
  66. [66]
    D.M. Markowitz, M.I. Reiman, and L.M. Wein. The stochastic economic lot size scheduling problem: heavy traffic analysis of dynamic cyclic policies. Report, MIT, School of Management, 1996.Google Scholar
  67. [67]
    L.F. Martins and H.J. Kushner. Routing and singular control for queueing networks in heavy traffic. SIAM J. Control Optim., 28:1209–1233, 1990.MathSciNetMATHCrossRefGoogle Scholar
  68. [68]
    L.F. Martins, S.E. Shreve, and H.M. Soner. Heavy traffic convergence of a controlled, multi-class queueing system. SIAM J. on Control and Optimization, 34:2133–2171, 1996.MathSciNetMATHCrossRefGoogle Scholar
  69. [69]
    D. Mitra. Stochastic theory of a fluid model of producers and consumers coupled by buffer. Adv. Appl. Probab., 20:646–676, 1988.MATHCrossRefGoogle Scholar
  70. [70]
    M.I. Reiman and L.M. Wein. Dynamic scheduling of a two class queue with setups. Operations Research, 46:532–547, 1998.MathSciNetMATHCrossRefGoogle Scholar
  71. [71]
    M.R. Reiman. Open queueing networks in heavy traffic. Math. Oper. Res., 9:441–458, 1984.MathSciNetMATHCrossRefGoogle Scholar
  72. [72]
    M.R. Reiman. Asymptotically optimal trunk reservation for large trunk groups. In Proceedings of the 28-th Conference on Decision and Control, New York, 1989. IEEE Press.Google Scholar
  73. [73] M.R. Reiman. Optimal trunk reservations for a critically loaded line. In A. Jensen and V.B. Jensen, editors, Teletraffic and Datatraffic in a Period of Change. North-Holland-Elsevier, 1991.Google Scholar
  74. [74]
    R. Srikant. Diffusion approximations for models of congestion control in high speed networks. In Proceedings of the 38-th Conference on Decision and Control, New York, 1998. IEEE Press.Google Scholar
  75. [75]
    R. Srikant. Control of communication networks. In T. Samed, editor, Perspectives in Control Engineering: Technologies, Applications, New Directions. IEEE Press, New York, 1999.Google Scholar
  76. [76]
    D.W. Stroock and S.R.S. Varadhan. Multidimensional Diffusion Processes. Springer-Verlag, New York, 1979.MATHGoogle Scholar
  77. [77]
    H. Takagi. Queueing analysis of polling models, an update. In J.H. Dshalalow, editor, Frontiers in Queueing, Models, Methods and Problems. CRC press, Boca Raton, Fla, 1994.Google Scholar
  78. [78]
    M. Taksar. Average optimal singular control and a related stopping problem. Math. Oper. Res., 10:63–81, 1985.MathSciNetMATHCrossRefGoogle Scholar
  79. [79]
    J.A. van Mieghem. Dynamic scheduling with convex delay costs: The generalized rule. Ann. Appl. Probab., 5:809–833, 1995.MathSciNetMATHCrossRefGoogle Scholar
  80. [80]
    J. Walrand. An Introduction to Queuing Networks. Prentice—Hall, Englewood Cliffs, 1988.Google Scholar
  81. [81]
    L.M. Wein. Optimal control of a two station Brownian network. Math. Oper. Res., 15:215–242, 1990.MathSciNetMATHCrossRefGoogle Scholar
  82. [82]
    W. Whitt. Mathematical methods in queueing theory: A survey. In Lecture Notes in Economics and Mathematical Systems, Vol 98, pages 307–350. Springer-Verlag, Berlin and New York, 1974.Google Scholar
  83. [83]
    R.J. Williams. Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse. Queueing Systems, 30:27–88, 1998.CrossRefGoogle Scholar
  84. [84]
    R.J. Williams. On dynamic scheduling of a parallel server system with complete resource pooling. Submitted to the Fields Institute Workshop on Analysis and Simulation of Communication Networks., 1998.Google Scholar
  85. [85]
    R.J. Williams and S.R.S. Varadhan. Brownian motion in a wedge with oblique reflection. Comm. Pure Appl. Math., 24:147–225, 1985.MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2001

Authors and Affiliations

  • Harold J. Kushner

There are no affiliations available

Personalised recommendations