D.P. Bertsekas, Dynamic Programming and Optimal Control, second edition (Athena Scientific, Belmont, MA, 2001).
D.P. Bertsekas and R. Gallager Data Networks (Prentice Hall, Englewood Cliffs, NJ, 1992).
D.P. Bertsekas and J.N. Tsitsiklis, Neuro-Dynamic Programming (Athena Scientific, Belmont, MA, 1996).
S. Bhatnagar and V.S. Borkar, Multiscale stochastic approximation for parametric optimization of hidden Markov models, Probability in the Engineering and Informational Sciences 11 (1997) 509–522.
S. Bhatnagar and V.S. Borkar, A two time scale stochastic approximation scheme for simulation based parametric optimization, Probability in the Engineering and Informational Sciences 12 (1998) 519–531.
S. Bhatnagar and V.S. Borkar, Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization, Simulation: Transactions of the Society for Modeling and Simulation International 79(10) (2003) 568–580.
S. Bhatnagar, M.C. Fu, S.I. Marcus and S. Bhatnagar, Two timescale algorithms for simulation optimization of hidden Markov models, HE Transactions 33(3) (2001) 245–258.
S. Bhatnagar, M.C. Fu, S.I. Marcus and P.J. Fard, “Optimal structured feedback policies for ABR flow control using two-timescale SPSA, IEEE/ACM Transactions on Networking 9(4) (2001) 479–491.
S. Bhatnagar, M.C. Fu, S.I. Marcus and I.-J. Wang, Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences, ACM Transactions on Modelling and Computer Simulation 13(2) (2003) 180–209.
O. Brandiere, Some pathological traps for stochastic approximation, SIAM J. Contr. and Optim. 36 (1998) 1293–1314.
H.F. Chen and T.E. Duncan and B. P.-Duncan, A Kiefer-Wolfowitz algorithm with randomized differences, IEEE Trans. Autom. Cont.
44(3) (1999) 442–453.
R.-G. Cheng, C.-J. Chang and L.-F. Lin, A QoS provisioning neural fuzzy connection admission controller for multimedia high speed networks, IEEE/ACM Trans, on Network. 7(1) (1999) 111–121.
E.K.P. Chong and P.J. Ramadge, Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times, SIAM J. Contr. and Optim. 31(3) (1993) 698–732.
E.K.P. Chong and P.J. Ramadge, Stochastic optimization of regenerative systems using infinitesimal perturbation analysis, IEEE Trans, on Autom. Contr. 39(7) (1994) 1400–1410.
M.C. Fu, Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis, J. Optim. Theo. Appl. 65 (1990) 149–160.
L. Gerencsér, S.D. Hill and Z. Vágó, Optimization over discrete sets via SPSA, in Proceedings of the IEEE Conference on Decision and Control (1999) pp. 1791–1795.
M. Grossglauser, S. Keshav and D.N.C. Tse, RCBR: A simple and efficient service for multiple time-scale traffic, IEEE Trans, on Network. 5(6) (1997) 741–755.
M.W. Hirsch, Convergent activation dynamics in continuous time networks, Neural Networks 2 (1989) 331–349.
Y.-C. Ho and X.-R. Cao Perturbation Analysis of Discrete Event Dynamical Systems (Kluwer, Boston, 1991).
F.P. Kelly, P.B. Key and S. Zachary, Distributed admission control, IEEE Journal on Selected Areas in Communications 18 (2000) 2617–2628.
S. Keshav An Engineering Approach to Computer Networking (Addison-Wesley, New York, 1997).
T.-H. Lee, K.-C. Lai and S.-T. Duann, Design of a real-time admission controller for ATM Networks, IEEE/ACM Trans, on Network 4(5) (1996) 758–765.
J. Liebeherr, D.E. Wrege and D. Ferrari Exact admission control for networks with a bounded delay service, IEEE/ACM Trans, on Network 4(6) (1996) 885–901.
P. Marbach, O. Mihatsch and J.N. Tsitsiklis, Call admission control and routing in integrated service networks using neuro-dynamic programming, IEEE Journal on Selected Areas in Communications 18(2) (2000) 197–208.
R. Pemantle, Nonconvergence to unstable points in urn models and stochastic approximations, Annals of Prob. 18 (1990) 698–712.
M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley, New York, 1994).
P.J. Schweitzer, Perturbation theory and finite Markov chains, J. Appl. Prob. 5 (1968) 401–413.
J.C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Autom. Contr. 37(3) (1992) 332–341.
F.J. Vazquez-Abad and H.J. Kushner, Estimation of the derivative of a stationary measure with respect to a control parameter, J. Appl. Prob. 29 (1992) 343–352.
J. Walrand and P. Varaiya, High-Performance Computer Networks (Morgan Kauffman, San Mateo, CA, 2000).