Journal of Heuristics

, Volume 5, Issue 1, pp 89–108 | Cite as

Rollout Algorithms for Stochastic Scheduling Problems

  • Dimitri P. Bertsekas
  • David A. Castanon


Stochastic scheduling problems are difficult stochastic control problems with combinatorial decision spaces. In this paper we focus on a class of stochastic scheduling problems, the quiz problem and its variations. We discuss the use of heuristics for their solution, and we propose rollout algorithms based on these heuristics which approximate the stochastic dynamic programming algorithm. We show how the rollout algorithms can be implemented efficiently, with considerable savings in computation over optimal algorithms. We delineate circumstances under which the rollout algorithms are guaranteed to perform better than the heuristics on which they are based. We also show computational results which suggest that the performance of the rollout policies is near-optimal, and is substantially better than the performance of their underlying heuristics.

rollout algorithms scheduling neuro-dynamic programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Barto, A. G., S. J. Bradtke, and S. P. Singh. (1995). “Learning to Act Using Real-Time Dynamic Programming,” Artificial Intelligence 72, 81–138.Google Scholar
  2. Bertsekas, D. P., J. N. Tsitsiklis, and C. Wu. (1997). “Rollout Algorithms for Combinatorial Optimization,” Heuristics 3, 245–262.Google Scholar
  3. Bertsekas, D. P., and J. N. Tsitsiklis. (1996). Neuro-Dynamic Programming. Belmont, MA: Athena Scientific.Google Scholar
  4. Ross, S. M. (1983). Introduction to Stochastic Dynamic Programming. N.Y.: Academic Press.Google Scholar
  5. Sutton, R., and A. G. Barto. (1998). Reinforcement Learning. Cambridge, MA: MIT Press.Google Scholar
  6. Tesauro, G., and G. R. Galperin. (1996). “On-Line Policy Improvement Using Monte Carlo Search,” presented at the 1996 Neural Information Processing Systems Conference, Denver, CO.Google Scholar
  7. Whittle, P. (1982). Optimization over Time, vol. I. N.Y.: Wiley.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Dimitri P. Bertsekas
    • 1
  • David A. Castanon
    • 2
  1. 1.Department of Electrical Engineering and Computer Science, M.I.T.Cambridge
  2. 2.Department of Electrical EngineeringBoston University, and ALPHATECH, Inc.Burlington

Personalised recommendations