Optimal Threshold Policies for Admission Control in Communication Networks via Discrete Parameter Stochastic Approximation
Received: 01 February 2003 Revised: 01 July 2004 Accepted: 01 October 2004 DOI:
10.1007/s11235-005-6629-5 Cite this article as: Bhatnagar, S. & Reddy, I.B.B. Telecommun Syst (2005) 29: 9. doi:10.1007/s11235-005-6629-5 Abstract
The problem of admission control of packets in communication networks is studied in the continuous time queueing framework under different classes of service and delayed information feedback. We develop and use a variant of a simulation based two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm for finding an optimal feedback policy within the class of threshold type policies. Even though SPSA has originally been designed for continuous parameter optimization, its variant for the discrete parameter case is seen to work well. We give a proof of the hypothesis needed to show convergence of the algorithm on our setting along with a sketch of the convergence analysis. Extensive numerical experiments with the algorithm are illustrated for different parameter specifications. In particular, we study the effect of feedback delays on the system performance.
Keywords admission control communication networks regularized semi-Markov modulated Poisson process two timescale stochastic approximation simultaneous perturbation stochastic approximation threshold type policies References
Dynamic Programming and Optimal Control
, second edition (Athena Scientific, Belmont, MA, 2001).
D.P. Bertsekas and R. Gallager
(Prentice Hall, Englewood Cliffs, NJ, 1992).
D.P. Bertsekas and J.N. Tsitsiklis,
(Athena Scientific, Belmont, MA, 1996).
S. Bhatnagar and V.S. Borkar, Multiscale stochastic approximation for parametric optimization of hidden Markov models, Probability in the Engineering and Informational Sciences 11 (1997) 509–522.
S. Bhatnagar and V.S. Borkar, A two time scale stochastic approximation scheme for simulation based parametric optimization, Probability in the Engineering and Informational Sciences 12 (1998) 519–531.
S. Bhatnagar and V.S. Borkar, Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization, Simulation: Transactions of the Society for Modeling and Simulation International 79(10) (2003) 568–580.
S. Bhatnagar, M.C. Fu, S.I. Marcus and S. Bhatnagar, Two timescale algorithms for simulation optimization of hidden Markov models, HE Transactions 33(3) (2001) 245–258.
S. Bhatnagar, M.C. Fu, S.I. Marcus and P.J. Fard, “Optimal structured feedback policies for ABR flow control using two-timescale SPSA, IEEE/ACM Transactions on Networking 9(4) (2001) 479–491.
S. Bhatnagar, M.C. Fu, S.I. Marcus and I.-J. Wang, Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences, ACM Transactions on Modelling and Computer Simulation 13(2) (2003) 180–209.
O. Brandiere, Some pathological traps for stochastic approximation, SIAM J. Contr. and Optim. 36 (1998) 1293–1314.
H.F. Chen and T.E. Duncan and B. P.-Duncan, A Kiefer-Wolfowitz algorithm with randomized differences, IEEE Trans. Autom. Cont.
44(3) (1999) 442–453.
R.-G. Cheng, C.-J. Chang and L.-F. Lin, A QoS provisioning neural fuzzy connection admission controller for multimedia high speed networks, IEEE/ACM Trans, on Network. 7(1) (1999) 111–121.
E.K.P. Chong and P.J. Ramadge, Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times, SIAM J. Contr. and Optim. 31(3) (1993) 698–732.
E.K.P. Chong and P.J. Ramadge, Stochastic optimization of regenerative systems using infinitesimal perturbation analysis, IEEE Trans, on Autom. Contr. 39(7) (1994) 1400–1410.
M.C. Fu, Convergence of a stochastic approximation algorithm for the
queue using infinitesimal perturbation analysis, J. Optim. Theo. Appl. 65 (1990) 149–160.
L. Gerencsér, S.D. Hill and Z. Vágó, Optimization over discrete sets via SPSA, in
Proceedings of the IEEE Conference on Decision and Control (1999) pp. 1791–1795.
M. Grossglauser, S. Keshav and D.N.C. Tse, RCBR: A simple and efficient service for multiple time-scale traffic,
IEEE Trans, on Network
. 5(6) (1997) 741–755.
M.W. Hirsch, Convergent activation dynamics in continuous time networks, Neural Networks 2 (1989) 331–349.
Y.-C. Ho and X.-R. Cao
Perturbation Analysis of Discrete Event Dynamical Systems
(Kluwer, Boston, 1991).
F.P. Kelly, P.B. Key and S. Zachary, Distributed admission control, IEEE Journal on Selected Areas in Communications 18 (2000) 2617–2628.
An Engineering Approach to Computer Networking
(Addison-Wesley, New York, 1997).
T.-H. Lee, K.-C. Lai and S.-T. Duann, Design of a real-time admission controller for ATM Networks, IEEE/ACM Trans, on Network 4(5) (1996) 758–765.
J. Liebeherr, D.E. Wrege and D. Ferrari Exact admission control for networks with a bounded delay service, IEEE/ACM Trans, on Network 4(6) (1996) 885–901.
P. Marbach, O. Mihatsch and J.N. Tsitsiklis, Call admission control and routing in integrated service networks using neuro-dynamic programming, IEEE Journal on Selected Areas in Communications 18(2) (2000) 197–208.
R. Pemantle, Nonconvergence to unstable points in urn models and stochastic approximations, Annals of Prob. 18 (1990) 698–712.
Markov Decision Processes: Discrete Stochastic Dynamic Programming
(John Wiley, New York, 1994).
P.J. Schweitzer, Perturbation theory and finite Markov chains, J. Appl. Prob. 5 (1968) 401–413.
J.C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Autom. Contr. 37(3) (1992) 332–341.
F.J. Vazquez-Abad and H.J. Kushner, Estimation of the derivative of a stationary measure with respect to a control parameter, J. Appl. Prob. 29 (1992) 343–352.
J. Walrand and P. Varaiya,
High-Performance Computer Networks
(Morgan Kauffman, San Mateo, CA, 2000).
Google Scholar Copyright information
© Springer Science + Business Media, Inc. 2005