Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
  1. Home
  2. Probability Theory and Related Fields
  3. Article
Discrete multi-armed bandits and multi-parameter processes
Download PDF
Download PDF
  • Published: January 1986

Discrete multi-armed bandits and multi-parameter processes

  • Avi Mandelbaum1 

Probability Theory and Related Fields volume 71, pages 129–147 (1986)Cite this article

  • 261 Accesses

  • 48 Citations

  • Metrics details

Summary

The general multi-armed bandit problem is reformulated and solved as a control problem over a partially ordered set. The approach taken provides a technically convenient framework for bandit-like problems. It also adds insight to the structure of strategies over partially ordered sets.

Download to read the full article text

Working on a manuscript?

Avoid the common mistakes

References

  1. Bather, J.A.: Optimal Stopping of a Brownian Motion: A Comparison Technique. In: H. Chernoffs 60'th Birthday Festschrift. D. Siegmund et al. (eds.). New York: Academic Press 1983

    Google Scholar 

  2. Cairoli, R., Walsh, J.B.: Stochastic integrals in the plane. Acta. Math. 134, 111–183 (1975)

    Google Scholar 

  3. Gittins, J.C.: Bandit processes and dynamic allocation indices. J. R. Stat. Soc., Ser. B 41, 148–177 (1979)

    Google Scholar 

  4. Herkenrath, V., Kalin, D., Vogel, W. (eds.): Mathematical Learning Models-Theory and Algorithms: Proceedings of a Conference. Lect. Notes Stat. Berlin-Heidelberg-New York: Springer 1983

    Google Scholar 

  5. Karatzas, I.: Gittins indices in the dynamic allocation problem for diffusion processes. Ann. Probab. 12, 173–192 (1984)

    Google Scholar 

  6. Krengel, V., Sucheston, L.: Stopping rules and tactics for processes indexed by a directed set. J. Multivariate Anal. 11, 199–229 (1981)

    Google Scholar 

  7. Lawler, G.F., Vanderbei, R.J.: Markov strategies for optimal control problems indexed by a partially ordered set. Ann. Probab. 11, 642–647 (1983)

    Google Scholar 

  8. Mandelbaum, A., Vanderbei, R.J.: Optimal stopping and supermartingales over partially ordered sets. Z. Wahrscheinlichkeitstheor. Verw. Geb. 57, 253–264 (1981)

    Google Scholar 

  9. Mazziotto, G., Szpirglas, J.: Arrêt optimal sur le plan. Preprint (1981)

  10. Neveu, J.: Discrete-Parameter Martingales. Amsterdam: North Holland 1975

    Google Scholar 

  11. Snell, J.L.: Applications of Martingales system theorems. Trans. Am. Math. Soc. 73, 293–312 (1952)

    Google Scholar 

  12. Varaiya, P., Walrand, J., Buyukkoc, C.: Extensions of the multi-armed bandit problem. The discounted case. To be published in IEEE Trans. Autom. Control (1984)

  13. Walsh, J.B.: Martingales with a multi-dimensional parameter and stochastic integrals in the plane. Cours de eème Cycle. Laboratoire de Calcul de Probabilités, Université Paris VI, Année 76–77

  14. Washburn, R.B., Willsky, A.S.: Optional sampling of supermartingales indexed by partially ordered sets. Ann. Probab. 9, 957–970 (1981)

    Google Scholar 

  15. Whittle, P.: Optimization over Time: Dynamic Programming and Stochastic Control, Vol. I. New York: Wiley 1982

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Business, Stanford University, 94305, Stanford, CA, USA

    Avi Mandelbaum

Authors
  1. Avi Mandelbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mandelbaum, A. Discrete multi-armed bandits and multi-parameter processes. Probab. Th. Rel. Fields 71, 129–147 (1986). https://doi.org/10.1007/BF00366276

Download citation

  • Received: 03 September 1984

  • Issue Date: January 1986

  • DOI: https://doi.org/10.1007/BF00366276

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Stochastic Process
  • Control Problem
  • Probability Theory
  • Mathematical Biology
  • Bandit Problem
Download PDF

Working on a manuscript?

Avoid the common mistakes

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature