Telecommunication Systems

, Volume 29, Issue 1, pp 9–31 | Cite as

Optimal Threshold Policies for Admission Control in Communication Networks via Discrete Parameter Stochastic Approximation



The problem of admission control of packets in communication networks is studied in the continuous time queueing framework under different classes of service and delayed information feedback. We develop and use a variant of a simulation based two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm for finding an optimal feedback policy within the class of threshold type policies. Even though SPSA has originally been designed for continuous parameter optimization, its variant for the discrete parameter case is seen to work well. We give a proof of the hypothesis needed to show convergence of the algorithm on our setting along with a sketch of the convergence analysis. Extensive numerical experiments with the algorithm are illustrated for different parameter specifications. In particular, we study the effect of feedback delays on the system performance.


admission control communication networks regularized semi-Markov modulated Poisson process two timescale stochastic approximation simultaneous perturbation stochastic approximation threshold type policies 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    D.P. Bertsekas, Dynamic Programming and Optimal Control, second edition (Athena Scientific, Belmont, MA, 2001).Google Scholar
  2. [2]
    D.P. Bertsekas and R. Gallager Data Networks (Prentice Hall, Englewood Cliffs, NJ, 1992).Google Scholar
  3. [3]
    D.P. Bertsekas and J.N. Tsitsiklis, Neuro-Dynamic Programming (Athena Scientific, Belmont, MA, 1996).Google Scholar
  4. [4]
    S. Bhatnagar and V.S. Borkar, Multiscale stochastic approximation for parametric optimization of hidden Markov models, Probability in the Engineering and Informational Sciences 11 (1997) 509–522.Google Scholar
  5. [5]
    S. Bhatnagar and V.S. Borkar, A two time scale stochastic approximation scheme for simulation based parametric optimization, Probability in the Engineering and Informational Sciences 12 (1998) 519–531.Google Scholar
  6. [6]
    S. Bhatnagar and V.S. Borkar, Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization, Simulation: Transactions of the Society for Modeling and Simulation International 79(10) (2003) 568–580.Google Scholar
  7. [7]
    S. Bhatnagar, M.C. Fu, S.I. Marcus and S. Bhatnagar, Two timescale algorithms for simulation optimization of hidden Markov models, HE Transactions 33(3) (2001) 245–258.Google Scholar
  8. [8]
    S. Bhatnagar, M.C. Fu, S.I. Marcus and P.J. Fard, “Optimal structured feedback policies for ABR flow control using two-timescale SPSA, IEEE/ACM Transactions on Networking 9(4) (2001) 479–491.Google Scholar
  9. [9]
    S. Bhatnagar, M.C. Fu, S.I. Marcus and I.-J. Wang, Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences, ACM Transactions on Modelling and Computer Simulation 13(2) (2003) 180–209.Google Scholar
  10. [10]
    O. Brandiere, Some pathological traps for stochastic approximation, SIAM J. Contr. and Optim. 36 (1998) 1293–1314.Google Scholar
  11. [11]
    H.F. Chen and T.E. Duncan and B. P.-Duncan, A Kiefer-Wolfowitz algorithm with randomized differences, IEEE Trans. Autom. Cont. 44(3) (1999) 442–453.Google Scholar
  12. [12]
    R.-G. Cheng, C.-J. Chang and L.-F. Lin, A QoS provisioning neural fuzzy connection admission controller for multimedia high speed networks, IEEE/ACM Trans, on Network. 7(1) (1999) 111–121.Google Scholar
  13. [13]
    E.K.P. Chong and P.J. Ramadge, Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times, SIAM J. Contr. and Optim. 31(3) (1993) 698–732.Google Scholar
  14. [14]
    E.K.P. Chong and P.J. Ramadge, Stochastic optimization of regenerative systems using infinitesimal perturbation analysis, IEEE Trans, on Autom. Contr. 39(7) (1994) 1400–1410.Google Scholar
  15. [15]
    M.C. Fu, Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis, J. Optim. Theo. Appl. 65 (1990) 149–160.Google Scholar
  16. [16]
    L. Gerencsér, S.D. Hill and Z. Vágó, Optimization over discrete sets via SPSA, in Proceedings of the IEEE Conference on Decision and Control (1999) pp. 1791–1795.Google Scholar
  17. [17]
    M. Grossglauser, S. Keshav and D.N.C. Tse, RCBR: A simple and efficient service for multiple time-scale traffic, IEEE Trans, on Network. 5(6) (1997) 741–755.Google Scholar
  18. [18]
    M.W. Hirsch, Convergent activation dynamics in continuous time networks, Neural Networks 2 (1989) 331–349.Google Scholar
  19. [19]
    Y.-C. Ho and X.-R. Cao Perturbation Analysis of Discrete Event Dynamical Systems (Kluwer, Boston, 1991).Google Scholar
  20. [20]
    F.P. Kelly, P.B. Key and S. Zachary, Distributed admission control, IEEE Journal on Selected Areas in Communications 18 (2000) 2617–2628.Google Scholar
  21. [21]
    S. Keshav An Engineering Approach to Computer Networking (Addison-Wesley, New York, 1997).Google Scholar
  22. [22]
    T.-H. Lee, K.-C. Lai and S.-T. Duann, Design of a real-time admission controller for ATM Networks, IEEE/ACM Trans, on Network 4(5) (1996) 758–765.Google Scholar
  23. [23]
    J. Liebeherr, D.E. Wrege and D. Ferrari Exact admission control for networks with a bounded delay service, IEEE/ACM Trans, on Network 4(6) (1996) 885–901.Google Scholar
  24. [24]
    P. Marbach, O. Mihatsch and J.N. Tsitsiklis, Call admission control and routing in integrated service networks using neuro-dynamic programming, IEEE Journal on Selected Areas in Communications 18(2) (2000) 197–208.Google Scholar
  25. [25]
    R. Pemantle, Nonconvergence to unstable points in urn models and stochastic approximations, Annals of Prob. 18 (1990) 698–712.Google Scholar
  26. [26]
    M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley, New York, 1994).Google Scholar
  27. [27]
    P.J. Schweitzer, Perturbation theory and finite Markov chains, J. Appl. Prob. 5 (1968) 401–413.Google Scholar
  28. [28]
    J.C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Autom. Contr. 37(3) (1992) 332–341.Google Scholar
  29. [29]
    F.J. Vazquez-Abad and H.J. Kushner, Estimation of the derivative of a stationary measure with respect to a control parameter, J. Appl. Prob. 29 (1992) 343–352.Google Scholar
  30. [30]
    J. Walrand and P. Varaiya, High-Performance Computer Networks (Morgan Kauffman, San Mateo, CA, 2000).Google Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Department of Computer Science and AutomationIndian Institute of ScienceBangaloreIndia

Personalised recommendations