Learning in Stochastic Machine Scheduling

  • Sebastián Marbán
  • Cyriel Rutten
  • Tjark Vredeveld
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7164)


We consider a scheduling problem in which two classes of independent jobs have to be processed non-preemptively by a single machine. The processing times of the jobs are assumed to be exponentially distributed with parameters depending on the class of each job. The objective is to minimize the sum of expected completion times. We adopt a Bayesian framework in which both job class parameters are assumed to be unknown. However, by processing jobs from the corresponding class, the scheduler can gradually learn about the value of these parameters, thereby enhancing the decision making in the future.

For the traditional stochastic scheduling variant, in which the parameters are known, the policy that always processes a job with Shortest Expected Processing Time (SEPT) is an optimal policy. In this paper, we show that in the Bayesian framework the performance of SEPT is at most a factor 2 away from the performance of an optimal policy. Furthermore, we introduce a second policy learning-SEPT (ℓ-SEPT), which is an adaptive variant of SEPT. We show that ℓ-SEPT is no worse than SEPT and empirically outperforms SEPT. However, both policies have the same worst-case performance, that is, the bound of 2 is tight for both policies.


Schedule Problem Completion Time Optimal Policy Schedule Policy Performance Guarantee 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Araman, V.F., Caldentey, R.: Dynamic pricing for nonperishable products with demand learning. Operations Research 57(5), 1169–1188 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Burnetas, A.N., Katehakis, M.N.: On sequencing two types of tasks on a single processor under incomplete information. Probability in the Engineering and Informational Sciences 7(1), 85–119 (1993)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Chen, L., Plambeck, E.L.: Dynamic inventory management with learning about the demand distribution and substitution probability. Manufacturing & Service Operations Management 10(2), 236–256 (2008)CrossRefGoogle Scholar
  4. 4.
    Dean, B.C.: Approximation Algorithms for Stochastic Scheduling Problems. PhD thesis, Massachusetts Institute of Technology (2005)Google Scholar
  5. 5.
    DeGroot, M.H.: Optimal Statistical Decisions. McGraw-Hill, New York (1970)zbMATHGoogle Scholar
  6. 6.
    Farias, F.F., Van Roy, B.: Dynamic pricing with a prior on market response. Operations Research 58(1), 16–29 (2010)zbMATHCrossRefGoogle Scholar
  7. 7.
    Gittins, J.C.: Multi-armed bandit allocation indices. Wiley, N.Y. (1989)zbMATHGoogle Scholar
  8. 8.
    Gittins, J.C., Glazebrook, K.D.: On Bayesian models in stochastic scheduling. Journal of Applied Probability 14(3), 556–565 (1977)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Gittins, J.C., Jones, D.M.: A dynamic allocation index for the sequential design of experiments. In: Progress in Statistics, pp. 241–266 (1974)Google Scholar
  10. 10.
    Glazebrook, K.D., Owen, R.W.: On the value of adaptive solutions to stochastic scheduling problems. Mathematics of Operations Research 20(1), 65–89 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Hamada, T., Glazebrook, K.D.: A Bayesian sequential single machine scheduling problem to minimize the expected weighted sum of flowtimes of jobs with exponential processing times. Operations Research 41(5), 924–934 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Hamada, T., Tamaki, M.: Some results on a Bayesian sequential scheduling on two identical parallel processors. Journal of the Operations Research Society of Japan 42(14), 316–329 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Lariviere, M.A., Porteus, E.L.: Stalking information: Bayesian inventory management with unobserved lost sales. Management Science 45(3), 346–363 (1999)zbMATHCrossRefGoogle Scholar
  14. 14.
    Lin, K.Y.: Dynamic pricing with real-time demand learning. Operations Research 174(1), 522–538 (2003)Google Scholar
  15. 15.
    Megow, N., Uetz, M., Vredeveld, T.: Models and algorithms for stochastic online scheduling. Mathematics of Operations Research 31(3), 513–525 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Megow, N., Vredeveld, T.: Approximation in Preemptive Stochastic Online Scheduling. In: Azar, Y., Erlebach, T. (eds.) ESA 2006. LNCS, vol. 4168, pp. 516–527. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Möhring, R.H., Radermacher, F.J., Weiss, G.: Stochastic scheduling problems I: General strategies. ZOR – Zeitschrift für Operations Research 28, 193–260 (1984)zbMATHGoogle Scholar
  18. 18.
    Möhring, R.H., Schulz, A.S., Uetz, M.: Approximation in stochastic scheduling:the power of LP-based priority policies. Journal of ACM 46(6), 924–942 (1999)zbMATHCrossRefGoogle Scholar
  19. 19.
    Rothkopf, M.H.: Scheduling with random service times. Management Science 12(9), 703–713 (1966)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Scarf, H.: Bayes solutions of the statistical inventory problem. The Annals of Mathematical Statistics 30(2), 490–508 (1959)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Schulz, A.S.: New old algorithms for stochastic scheduling. In: Algorithms for Optimization with Incomplete Information. Dagstuhl Seminar Proceedings, vol. 05031 (2005)Google Scholar
  22. 22.
    Smith, W.E.: Various optimizers for single stage production. Naval Research Logistics Quaterly 3, 59–66 (1956)CrossRefGoogle Scholar
  23. 23.
    Weiss, G.: Approximation results in parallel machines stochastic scheduling. Annals of Operations Research 26(1), 195–242 (1990)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Weiss, G.: Turnpike optimality of Smith’s rule in parallel machines stochastic scheduling. Mathematics of Operations Research 17(2), 255–270 (1992)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Sebastián Marbán
    • 1
  • Cyriel Rutten
    • 1
  • Tjark Vredeveld
    • 1
  1. 1.Department of Quantitative EconomicsMaastricht UniversityMaastrichtThe Netherlands

Personalised recommendations