[1]

D.P. Bertsekas, *Dynamic Programming and Optimal Control*, second edition (Athena Scientific, Belmont, MA, 2001).

[2]

D.P. Bertsekas and R. Gallager *Data Networks* (Prentice Hall, Englewood Cliffs, NJ, 1992).

[3]

D.P. Bertsekas and J.N. Tsitsiklis, *Neuro-Dynamic Programming* (Athena Scientific, Belmont, MA, 1996).

[4]

S. Bhatnagar and V.S. Borkar, Multiscale stochastic approximation for parametric optimization of hidden Markov models, Probability in the Engineering and Informational Sciences 11 (1997) 509–522.

[5]

S. Bhatnagar and V.S. Borkar, A two time scale stochastic approximation scheme for simulation based parametric optimization, Probability in the Engineering and Informational Sciences 12 (1998) 519–531.

[6]

S. Bhatnagar and V.S. Borkar, Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization, Simulation: Transactions of the Society for Modeling and Simulation International 79(10) (2003) 568–580.

[7]

S. Bhatnagar, M.C. Fu, S.I. Marcus and S. Bhatnagar, Two timescale algorithms for simulation optimization of hidden Markov models, HE Transactions 33(3) (2001) 245–258.

[8]

S. Bhatnagar, M.C. Fu, S.I. Marcus and P.J. Fard, “Optimal structured feedback policies for ABR flow control using two-timescale SPSA, IEEE/ACM Transactions on Networking 9(4) (2001) 479–491.

[9]

S. Bhatnagar, M.C. Fu, S.I. Marcus and I.-J. Wang, Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences, ACM Transactions on Modelling and Computer Simulation 13(2) (2003) 180–209.

[10]

O. Brandiere, Some pathological traps for stochastic approximation, SIAM J. Contr. and Optim. 36 (1998) 1293–1314.

[11]

H.F. Chen and T.E. Duncan and B. P.-Duncan, A Kiefer-Wolfowitz algorithm with randomized differences, IEEE Trans. Autom. Cont.
44(3) (1999) 442–453.

[12]

R.-G. Cheng, C.-J. Chang and L.-F. Lin, A QoS provisioning neural fuzzy connection admission controller for multimedia high speed networks, IEEE/ACM Trans, on Network. 7(1) (1999) 111–121.

[13]

E.K.P. Chong and P.J. Ramadge, Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times, SIAM J. Contr. and Optim. 31(3) (1993) 698–732.

[14]

E.K.P. Chong and P.J. Ramadge, Stochastic optimization of regenerative systems using infinitesimal perturbation analysis, IEEE Trans, on Autom. Contr. 39(7) (1994) 1400–1410.

[15]

M.C. Fu, Convergence of a stochastic approximation algorithm for the *GI/G/1* queue using infinitesimal perturbation analysis, J. Optim. Theo. Appl. 65 (1990) 149–160.

[16]

L. Gerencsér, S.D. Hill and Z. Vágó, Optimization over discrete sets via SPSA, in *Proceedings of the IEEE Conference on Decision and Control* (1999) pp. 1791–1795.

[17]

M. Grossglauser, S. Keshav and D.N.C. Tse, RCBR: A simple and efficient service for multiple time-scale traffic, *IEEE Trans, on Network*. 5(6) (1997) 741–755.

[18]

M.W. Hirsch, Convergent activation dynamics in continuous time networks, Neural Networks 2 (1989) 331–349.

[19]

Y.-C. Ho and X.-R. Cao *Perturbation Analysis of Discrete Event Dynamical Systems* (Kluwer, Boston, 1991).

[20]

F.P. Kelly, P.B. Key and S. Zachary, Distributed admission control, IEEE Journal on Selected Areas in Communications 18 (2000) 2617–2628.

[21]

S. Keshav *An Engineering Approach to Computer Networking* (Addison-Wesley, New York, 1997).

[22]

T.-H. Lee, K.-C. Lai and S.-T. Duann, Design of a real-time admission controller for ATM Networks, IEEE/ACM Trans, on Network 4(5) (1996) 758–765.

[23]

J. Liebeherr, D.E. Wrege and D. Ferrari Exact admission control for networks with a bounded delay service, IEEE/ACM Trans, on Network 4(6) (1996) 885–901.

[24]

P. Marbach, O. Mihatsch and J.N. Tsitsiklis, Call admission control and routing in integrated service networks using neuro-dynamic programming, IEEE Journal on Selected Areas in Communications 18(2) (2000) 197–208.

[25]

R. Pemantle, Nonconvergence to unstable points in urn models and stochastic approximations, Annals of Prob. 18 (1990) 698–712.

[26]

M.L. Puterman, *Markov Decision Processes: Discrete Stochastic Dynamic Programming* (John Wiley, New York, 1994).

[27]

P.J. Schweitzer, Perturbation theory and finite Markov chains, J. Appl. Prob. 5 (1968) 401–413.

[28]

J.C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Autom. Contr. 37(3) (1992) 332–341.

[29]

F.J. Vazquez-Abad and H.J. Kushner, Estimation of the derivative of a stationary measure with respect to a control parameter, J. Appl. Prob. 29 (1992) 343–352.

[30]

J. Walrand and P. Varaiya, *High-Performance Computer Networks* (Morgan Kauffman, San Mateo, CA, 2000).