Acta Informatica

, Volume 17, Issue 1, pp 43–61 | Cite as

Information theoretic approximations for M/G/1 and G/G/1 queuing systems

  • John E. Shore


This paper presents new results concerning the use of information theoretic inference techniques in system modeling and concerning the widespread applicability of certain simple queuing theory formulas. For the case when an M/G/1 queue provides a reasonable system model but when information about the service time probability density is limited to knowledge of a few moments, entropy maximization and cross-entropy minimization are used to derive information theoretic approximations for various performance distributions such as queue length, waiting time, residence time, busy period, etc. Some of these approximations are shown to reduce to exact M/M/1 results when G = M. For the case when a G/G/1 queue provides a reasonable system model, but when information about the arrival and service distributions is limited to the average arrival and service rates, it is shown that various well known M/M/1 formulas are information theoretic approximations. These results not only provide a new method for approximating the performance distributions, but they help to explain the widespread applicability of the M/M/1 formulas.


Probability Density Computational Mathematic Service Time Entropy Maximization Queue Length 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Muntz, R.R.: Analytic Modeling of Interactive Systems. Proc. IEEE 63, 946–953 (1975)Google Scholar
  2. 2.
    Chen, P.P.: Queueing Network Model of Interactive Computing Systems. Proc. IEEE 63, 954–957 (1975)Google Scholar
  3. 3.
    Kleinrock, L.: Queueing Systems, Vol. II: Computer Applications. New York: John Wiley 1976Google Scholar
  4. 4.
    Denning, P. J., Buzen, F.P.: The Operational Analysis of Queueing Network Models. Computing Surveys 10, 225–261 (1978)Google Scholar
  5. 5.
    Shore, J. E., Johnson, R.W.: Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy. IEEE Trans. Information Theory IT-26, 26–37 (1980)Google Scholar
  6. 6.
    Ferdinand, A.E.: A Statistical Mechanics Approach to Systems Analysis. IBM J. Res. Dev. 539–547 (1970)Google Scholar
  7. 7.
    Beneš, V.E.: Mathematical Theory of Connecting Networks and Telephone Traffic. New York: Academic Press 1965Google Scholar
  8. 8.
    Shore, J.E.: Derivation of Equilibrium and Time-Dependent Solutions ot M/M/∞//N and M/M/∞ Queueing Systems Using Entropy Maximization. Proceedings 1978 National Computer Conference, AFIPS, 1978, pp. 483–487Google Scholar
  9. 9.
    Bard, Y.: A Model of Shared DASD and Multipathing. CACM 23, 564–572 (1980)Google Scholar
  10. 10.
    Jaynes, E.T.: Information Theory and Statistical Mechanics I. Phys. Rev. 106, 171–190 (1957)Google Scholar
  11. 11.
    Elsasser, W.M.: On Quantum Measurements and the Role of the Uncertainty Relations in Statistical Mechanics. Phys. Rev. 52, 987–999 (1937)Google Scholar
  12. 12.
    Kullback, S.: Information Theory and Statistics. New York: Wiley 1959Google Scholar
  13. 13.
    Csiszàr, I.: I-Divergence Geometry of Probability Distributions and Minimization Problems. Ann. Prob. 3, 146–58 (1975)Google Scholar
  14. 14.
    Shore, J.E., Johnson, R.W.: Properties of Cross-Entropy Minimization. IEEE Trans. Inf. Theory IT-27, 472–482 (1980)Google Scholar
  15. 15.
    Johnson, R.W.: Determining Probability Distributions by Maximum Entropy and Minimum Cross-Entropy. Proceedings APL79, (ACM 0-89791-005), May 1979, pp. 24–29Google Scholar
  16. 16.
    Good, I.J.: Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables. Annals Math. Stat. 34, 911–934 (1963)Google Scholar
  17. 17.
    Jaynes, E.T.: Prior Probabilities. IEEE Trans. on Systems Science and Cybernetics SSC-4, 227–241 (1968)Google Scholar
  18. 18.
    Hobson, A., Cheng, B.: A Comparison of the Shannon and Kullback Information Measures. J. Stat. Phys. 7, 301–310 (1973)Google Scholar
  19. 19.
    Johnson, R.W.: Axiomatic Characterization of the Directed Divergences and Their Linear Combinations. IEEE Trans. Information Theory IT-25, 709–716 (1979)Google Scholar
  20. 20.
    Good, I.J.: Probability and the Weighing of Evidence. London: Charles Griffen 1950Google Scholar
  21. 21.
    Pinsker, M. S.: Information and Information Stability of Random Variables and Processes. Holden-Day: San Francisco 1964 (1981)Google Scholar
  22. 22.
    Shore, J. E.: Minimum Cross-Entropy Spectral Analysis. IEEE Trans. Acoustics, Speech, & Signal Proc. ASSP-29, 230–237 (1981)Google Scholar
  23. 23.
    Gray, R.M., Gray, A.H., Jr., Rebolledo, G., Shore, J.E.: Rate-Distortion Speech Coding With a Minimum Discrimination Information Distortion Measure. IEEE Trans. Inf. Theory IT-27, November 1981Google Scholar
  24. 24.
    Shore, J.E., Gray, R.M.: Minimum Cross-Entropy Pattern Classification and Cluster Analysis. IEEE Trans. Pattern Anal. & Mach. Intell., January 1982Google Scholar
  25. 25.
    Hobson, A.: A New Theorem of Information Theory. J. Stat. Phys. 1, 383–391 (1969)Google Scholar
  26. 26.
    Geronimus, L.: Orthogonal Polynomials. New York: Consultants bureau 1961Google Scholar
  27. 27.
    Gray, R.M., Buzo, A., Gray, A.H., Jr., Matsuyama, Y.: Distortion Measures for Speech Processing. IEEE Trans. Acoustics, Speech, and Signal Processing ASSP-28, 367–376 (1980)Google Scholar
  28. 28.
    Markel, J.D., Gray, A.H., Jr.: Linear Prediction of Speech. New York: Springer 1976Google Scholar
  29. 29.
    Cohen, J.W.: The Single Server Queue. Amsterdam: North-Holland 1969Google Scholar
  30. 30.
    Cox, D.R., Smith, W.L.: Queues. London: Chapman and Hall 1961Google Scholar
  31. 31.
    Kleinrock, L.: Queueing Systems, Vol I: Theory. New York: John Wiley 1975Google Scholar
  32. 32.
    Shore, J.E.: Information Theoretic Approximations for M/G/1 and G/G/1 Queueing Systems. Naval Research Laboratory Memorandum Report 4047, July 1979Google Scholar
  33. 33.
    Shore, J.E.: Accuracy of an Information-Theoretic, Light-Load Approximation for the M/M/1 Busy Period Probability Density. Naval Research Laboratory Memorandum Report 4037, July 1979Google Scholar
  34. 34.
    Ephremides, A.E.: private communicationGoogle Scholar

Copyright information

© Springer-Verlag 1982

Authors and Affiliations

  • John E. Shore
    • 1
  1. 1.Naval Research LaboratoryWashington, D.C.USA

Personalised recommendations