Skip to main content
SpringerLink
Log in

Almost optimal policies for stochastic systemswhich almost satisfy conservation laws

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

When controlled stochastic systems have performances which satisfy generalisedconservation laws (GCL), an objective which is linear in the performance is optimised by aGittins index policy. We develop measures of the extent to which a system fails to satisfyGCL and derive suboptimality bounds for suitable index policies in terms of such measures.These bounds are used, inter alia, to explore the robustness in performance of cm‐typerules for a multiclass G/G/1 queueing system to departures from an assumption of exponentialservice times. We also study Gittins index policies for parallel processor versions of theclassical undiscounted and discounted multi‐armed bandit problems. In the undiscountedcase, the cost of an index policy comes within a constant of the optimal cost ‐ thisconstant being independent of the number of projects submitted for scheduling. In thediscounted case, under fairly mild conditions, Gittins index policies come within an O(1) quantity ofoptimality and are hence average reward optimal when the discount rate is small enough.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D. Bertsimas and J. Niño-Mora, Conservation laws, extended polymatroids and multi-armed bandit problems; a unified approach to indexable systems, Mathematics of Operations Research 21(1996) 257-306.

    Google Scholar 

  2. D. Blackwell, Discrete dynamic programming, Annals of Mathematical Statistics 33(1962)719-726.

    Google Scholar 

  3. A. Federgruen and H. Groenevelt, Characterization and optimization of achievable performance in general queueing systems, Operations Research 36(1988)733-741.

    Google Scholar 

  4. A. Federgruen and H. Groenevelt, M/G/c queueing systems with multiple customer classes. Characterization and control of achievable performance under non-preemptive priority rules, Management Science 34(1988)1121-1138.

    Google Scholar 

  5. E. Gelenbe and I. Mitrani, Analysis and Synthesis of Computer Systems, Academic Press, New York, 1980.

    Google Scholar 

  6. J.C. Gittins, Bandit processes and dynamic allocation indices, Journal of the Royal Statistical Society B41(1979)148-177.

    Google Scholar 

  7. J.C. Gittins, Bandit Processes and Dynamic Allocation Indices, Wiley, New York, 1989.

    Google Scholar 

  8. J.C. Gittins and D.M. Jones, A dynamic allocation index for the sequential design of experiments, in: Progress in Statistics, eds. J. Gani et al., North-Holland, Amsterdam, 1974, pp. 241-266.

    Google Scholar 

  9. K.D. Glazebrook, On the undiscounted tax problem with precedence constraints, Advances in Applied Probability 28(1996)1123-1144.

    Google Scholar 

  10. K.D. Glazebrook and R. Garbe, Reflections on a new approach to Gittins indexation, Journal of the Operational Research Society 47(1996)1301-1309.

    Google Scholar 

  11. K.D. Glazebrook and R. Garbe, Almost optimal policies for stochastic systems which almost satisfy conservation laws, Technical Report, Newcastle University.

  12. G.P. Klimov, Time sharing service systems I, Theory of Probability and its Applications 19(1974) 532-551.

    Google Scholar 

  13. J.G. Shanthikumar and D.D. Yao, Multiclass queueing systems: Polymatroidal structure and optimal scheduling control, Operations Research 40,Supplement 2 (1992)S293-299.

    Google Scholar 

  14. H.C. Tijms, Stochastic Models — An Algorithmic Approach, Wiley, Chichester, 1994.

    Google Scholar 

  15. J.C. Walrand, Introduction to Queueing Networks, Prentice-Hall, Englewood Cliffs, 1988.

    Google Scholar 

  16. R.R. Weber, On the Gittins index for multi-armed bandits, Annals of Applied Probability 2(1992) 1024-1033.

    Google Scholar 

  17. R.R. Weber, Scheduling jobs with stochastic processing requirements on parallel machines to minimize makespan or flowtime, Journal of Applied Probability 19(1982)167-182.

    Google Scholar 

  18. R.R. Weber, P. Varaiya and J. Walrand, Scheduling jobs with stochastically ordered processing times on parallel machines to minimize expected flowtime, Journal of Applied Probability 23(1986) 841-847.

    Google Scholar 

  19. G. Weiss, Approximation results in parallel machines stochastic scheduling, Annals of Operations Research 26(1990)195-242.

    Google Scholar 

  20. G. Weiss, Turnpike optimality of Smith's rule in parallel machines stochastic scheduling, Mathematics of Operations Research 17(1992)255-270.

    Google Scholar 

  21. G. Weiss, On almost optimal priority rules for preemptive scheduling of stochastic jobs on parallel machines, Advances in Applied Probability 27(1995)821-839.

    Google Scholar 

  22. P. Whittle, Multi-armed bandits and the Gittins index, Journal of the Royal Statistical Society B42(1980)143-149.

    Google Scholar 

  23. P. Whittle, Restless bandits, Journal of Applied Probability 25A(1988)287-298.

    Google Scholar 

Download references

Authors
  1. K.D. Glazebrook
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Garbe
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Glazebrook, K., Garbe, R. Almost optimal policies for stochastic systemswhich almost satisfy conservation laws. Annals of Operations Research 92, 19–43 (1999). https://doi.org/10.1023/A:1018934714800

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018934714800

Keywords

Search

Navigation