Skip to main content
SpringerLink
Log in

Stationary distributions and convergence for M/M/1 queues in interactive random environment

  • Published:
Queueing Systems Aims and scope Submit manuscript

Abstract

A Markovian single-server queue is studied in an interactive random environment. The arrival and service rates of the queue depend on the environment, while the transition dynamics of the random environment depend on the queue length. We consider in detail two types of Markov random environments: a pure jump process and a reflected jump diffusion. In both cases, the joint dynamics are constructed so that the stationary distribution can be explicitly found in a simple form (weighted geometric). We also derive an explicit estimate for the exponential rate of convergence to the stationary distribution via coupling.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arapostathis, A., Pang, G., Sandrić, N.: Ergodicity of Lévy-driven SDEs arising from multiclass many-server queues. Ann. Appl. Probab. 29(2), 1070–1126 (2019)

    Google Scholar 

  2. Arapostathis, A., Hmedi, H., Pang, G., Sandrić, N.: Uniform polynomial rates of convergence for a class of Lévy-driven controlled SDEs arising in multiclass many-server queues. In: Yin, G., Zhang, Q. (eds.) Modeling, Stochastic Control, Optimization, and Applications. The IMA Volumes in Mathematics and its Applications, vol. 164, pp. 1–20. Springer, New York (2019)

    Google Scholar 

  3. Ata, B., Peng, X.: An optimal callback policy for general arrival processes: a pathwise analysis. Working paper (2018)

  4. Bassamboo, A., Harrison, J.M., Zeevi, A.: Dynamic routing and admission control in high-volume service systems: asymptotic analysis via multi-scale fluid limits. Queueing Syst. 51(3–4), 249–285 (2005)

    Google Scholar 

  5. Bassamboo, A., Harrison, J.M., Zeevi, A.: Design and control of a large call center: asymptotic analysis of an LP-based method. Oper. Res. 54(3), 419–435 (2006)

    Google Scholar 

  6. Belopolskaya, Y., Suhov, Y.: Models of Markov processes with a random transition mechanism (2015). arXiv:1508.05598

  7. Blanchet, J., Chen, X.: Rates of convergence to stationarity for multidimensional RBM (2016). arXiv:1601.04111

  8. Borodin, A., Salminen, P.: Handbook of Brownian Motion. Facts and Formulae, 2nd edn. Birkhauser, Basel (2002)

    Google Scholar 

  9. Burdzy, K., Kendall, W.S.: Efficient Markovian couplings: examples and counterexamples. Ann. Appl. Probab. 10(2), 362–409 (2000)

    Google Scholar 

  10. Chen, H., Yao, D.D.: Fundamentals of Queueing Networks. Stochastic Modeling and Applied Probability, vol. 46. Springer, Berlin (2001)

    Google Scholar 

  11. Chen, M.-F., Li, S.-F.: Coupling methods for multidimensional diffusion processes. Ann. Probab. 17(1), 151–177 (1989)

    Google Scholar 

  12. Cogburn, R.: Markov chains in random environments: the case of Markovian environments. Ann. Appl. Probab. 8(5), 908–916 (1980)

    Google Scholar 

  13. Cogburn, R., Torrez, W.C.: Birth and death processes with random environments in continuous time. J. Appl. Probab. 18(1), 19–30 (1981)

    Google Scholar 

  14. Cornez, R.: Birth and death processes in random environments with feedback. J. Appl. Probab. 24(1), 25–34 (1987)

    Google Scholar 

  15. Douc, R., Fort, G., Guillin, A.: Subgeometric rates of convergence of \(f\)-ergodic strong Markov processes. Stoch. Proc. Appl. 119(3), 897–923 (2009)

    Google Scholar 

  16. Economou, A.: Generalized product-form stationary distributions for Markov chains in random environments with queueing applications. Adv. Appl. Probab. 37(1), 185–211 (2005)

    Google Scholar 

  17. Falin, G.: A heterogeneous blocking system in a random environment. J. Appl. Probab. 33(1), 211–216 (1996)

    Google Scholar 

  18. Gannon, M., Pechersky, E., Suhov, Y., Yambartsev, A.: A random walk in a queueing network environment. J. Appl. Probab. 53(2), 448–462 (2016)

    Google Scholar 

  19. Hunter, J.J.: Coupling and mixing times in a Markov chain. Lin. Algebra Appl. 430(10), 2607–2621 (2009)

    Google Scholar 

  20. Ichiba, T., Sarantsev, A.: Stationary distributions and convergence for Walsh diffusions. Bernoulli 25(4A), 2439–2478 (2019)

    Google Scholar 

  21. Karatzas, I., Shreve, S.E.: Brownian Motion and Stochastic Calculus. Graduate Texts in Mathematics, vol. 113. Springer, Berlin (1991)

    Google Scholar 

  22. Kelly, F.: Reversibility and Stochastic Networks. Wiley, Chichester (1979)

    Google Scholar 

  23. Kelly, F., Yudovina, E.: Stochastic Networks. Cambridge University Press, Cambridge (2014)

    Google Scholar 

  24. Krenzler, R., Daduna, H., Otton, S.: Jackson networks in nonautonomous random environments. Adv. Appl. Probab. 48(2), 315–331 (2016)

    Google Scholar 

  25. Krishnamoorthy, A., Pramod, P.K., Chakravarthy, S.R.: Queues with interruptions: a survey. TOP 22(1), 290–320 (2014)

    Google Scholar 

  26. Kumar, R., Lewis, M.E., Topaloglu, H.: Dynamic service rate control for a single-server queue with Markov-modulated arrivals. Naval Res. Logist. 60(8), 661–677 (2013)

    Google Scholar 

  27. Kurtz, T.G., Stockbridge, R.H.: Stationary solutions and forward equations for controlled and singular martingale problems. Electron. J. Probab. 6(17), 1–52 (2001)

    Google Scholar 

  28. Lions, P.-L., Sznitman, A.-S.: Stochastic differential equations with reflecting boundary conditions. Commun. Pure Appl. Math. 37(4), 511–537 (1984)

    Google Scholar 

  29. Liu, Y., Zhang, H., Zhao, Y.: Subgeometric ergodicity for continuous-time Markov chains. J. Math. Anal. Appl. 368(1), 178–189 (2010)

    Google Scholar 

  30. Levin, D.A., Perez, Y., Wilmer, E.L.: Markov Chains and Mixing Times, 2nd edn. American Mathematical Society, Providence (2017)

    Google Scholar 

  31. Lund, R., Meyn, S.P., Tweedie, R.L.: Computable exponential convergence rates for stochastically ordered Markov processes. Ann. Appl. Probab. 6(1), 218–237 (1996)

    Google Scholar 

  32. Meyn, S.P., Tweedie, R.L.: Stability of Markovian processes II. Continuous-time processes and sampled chains. Adv. Appl. Probab. 25(3), 487–517 (1993)

    Google Scholar 

  33. Neuts, R.F.: Matrix-Geometric Solutions in Stochastic Models: An Algorithmic Approach. Dover, Mineola (1995)

    Google Scholar 

  34. Norris, J.R.: Markov chains. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge (1997)

    Google Scholar 

  35. Regterschot, G.J.K., de Smit, J.H.A.: The queue \(M/G/1\) with Markov modulated arrivals and services. Math. Oper. Res. 11(3), 465–483 (1986)

    Google Scholar 

  36. Robert, P.: Stochastic Networks and Queues. Stochastic Modeling and Applied Probability, vol. 52. Springer, Berlin (2003)

    Google Scholar 

  37. Sarantsev, A.: Explicit rates of exponential convergence for reflected jump-diffusions on the positive half-line. ALEA Lat. Am. J. Probab. Math. Stat. 13(2), 1069–1093 (2016)

    Google Scholar 

  38. Sarantsev, A.: Reflected Brownian motion in a convex polyhedral cone: tail estimates for the stationary distribution. J. Theor. Probab. 32(3), 545–585 (2019)

    Google Scholar 

  39. Sawyer, S.A.: A formula for semigroups, with an application to branching diffusion processes. Trans. Am. Math. Soc. 152(1), 1–38 (1970)

    Google Scholar 

  40. Suhov, Y., Kelbert, M.: Probability and statistics by example. Markov chains: a primer in random processes and their applications. Cambridge University Press, Cambridge (2008)

    Google Scholar 

  41. Takine, T.: Single-server queues with Markov-modulated arrival and service speed. Queueing Syst. 49(1), 7–22 (2005)

    Google Scholar 

  42. Williams, R.J.: Reflected Brownian motion with skew symmetric data in a polyhedral domain. Probab. Theor. Relat. Fields 75, 459–485 (1987)

    Google Scholar 

  43. Williams, R.J.: Semimartingale reflecting Brownian motions in the orthant: a survey. IMA Vol. Math. Appl. 71, 125–137 (1995)

    Google Scholar 

  44. Yechiali, U.: A queueing-type birth-and-death process defined in a continuous-time Markov chain. Oper. Res. 21(2), 604–609 (1973)

    Google Scholar 

Download references

Acknowledgements

G. Pang was supported in part by NSF grants CMMI-1635410, and DMS/CMMI-1715875 and in part by an Army Research Office Grant W911NF-17-1-0019. Y. Belopolskaya was supported in part by RSF 17-11-01136. Y. Suhov thanks Department of Mathematics at Pennsylvania State University for hospitality and support.

Author information

Authors and Affiliations

  1. Harold and Inge Marcus Department of Industrial and Manufacturing Engineering, The Pennsylvania State University, University Park, PA, 16802, USA

    Guodong Pang

  2. Department of Mathematics and Statistics, University of Nevada in Reno, Reno, USA

    Andrey Sarantsev

  3. Petersburg Department of Mathematical Institute of Russian Academy of Science, Saint Petersburg State University of Architecture and Civil Engineering, Saint Petersburg, Russia

    Yana Belopolskaya

  4. Statistical Laboratory, University of Cambridge, Cambridge, UK

    Yuri Suhov

  5. Department of Mathematics, The Pennsylvania State University, University Park, PA, 16802, USA

    Yuri Suhov

Authors
  1. Guodong Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrey Sarantsev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yana Belopolskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuri Suhov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guodong Pang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pang, G., Sarantsev, A., Belopolskaya, Y. et al. Stationary distributions and convergence for M/M/1 queues in interactive random environment . Queueing Syst 94, 357–392 (2020). https://doi.org/10.1007/s11134-019-09644-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11134-019-09644-9

Keywords

Mathematics Subject Classification

Search

Navigation