Queueing Systems

, Volume 10, Issue 1–2, pp 5–87 | Cite as

The Fourier-series method for inverting transforms of probability distributions

  • Joseph Abate
  • Ward Whitt
Article

Abstract

This paper reviews the Fourier-series method for calculating cumulative distribution functions (cdf's) and probability mass functions (pmf's) by numerically inverting characteristic functions, Laplace transforms and generating functions. Some variants of the Fourier-series method are remarkably easy to use, requiring programs of less than fifty lines. The Fourier-series method can be interpreted as numerically integrating a standard inversion integral by means of the trapezoidal rule. The same formula is obtained by using the Fourier series of an associated periodic function constructed by aliasing; this explains the name of the method. This Fourier analysis applies to the inversion problem because the Fourier coefficients are just values of the transform. The mathematical centerpiece of the Fourier-series method is the Poisson summation formula, which identifies the discretization error associated with the trapezoidal rule and thus helps bound it. The greatest difficulty is approximately calculating the infinite series obtained from the inversion integral. Within this framework, lattice cdf's can be calculated from generating functions by finite sums without truncation. For other cdf's, an appropriate truncation of the infinite series can be determined from the transform based on estimates or bounds. For Laplace transforms, the numerical integration can be made to produce a nearly alternating series, so that the convergence can be accelerated by techniques such as Euler summation. Alternatively, the cdf can be perturbed slightly by convolution smoothing or windowing to produce a truncation error bound independent of the original cdf. Although error bounds can be determined, an effective approach is to use two different methods without elaborate error analysis. For this purpose, we also describe two methods for inverting Laplace transforms based on the Post-Widder inversion formula. The overall procedure is illustrated by several queueing examples.

Keywords

Computational probability numerical inversion of transforms characteristic functions Laplace transforms generating functions Fourier transforms cumulative distribution functions calculating tail probabilities numerical integration Fourier series Poisson summation formula the Fourier-series method the Gaver-Stehfest method 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    J. Abate and H. Dubner, A new method for generating power series expansions of functions, SIAM J. Numer. Anal. 5 (1968) 102–112.Google Scholar
  2. [2]
    J. Abate and W. Whitt, Transient behavior of regulated Brownian motion I: starting at the origin, Adv. Appl. Prob. 19 (1987) 560–598.Google Scholar
  3. [3]
    J. Abate and W. Whitt, Transient behavior of theM/M/1 queue via Laplace transforms, Adv. Appl. Prob. 20 (1988) 145–178.Google Scholar
  4. [4]
    J. Abate and W. Whitt, Approximations for theM/M/1 busy-period distribution, in:Queueing Theory and its Applications, Liber Amicorum for J.W. Cohen, eds. O.J. Boxma and R. Syski (North-Holland, Amsterdam, 1988) pp. 149–191.Google Scholar
  5. [5]
    J. Abate and W. Whitt, Simple spectral representations for the M/M/1 queue, Queueing Systems 3 (1988) 321–346.Google Scholar
  6. [6]
    J. Abate and W. Whitt, Numerical inversion of Laplace transforms of probability distributions, AT&T Bell Laboratories, Murray Hill, NJ (1991).Google Scholar
  7. [7]
    J. Abate and W. Whitt, Numerical inversion of probability generating functions, AT&T Bell Laboratories, Murray Hill, NJ (1991).Google Scholar
  8. [8]
    M. Abramowitz and I.A. Stegun,Handbook of Mathematical Functions (National Bureau of Standards, Washington, DC, 1972).Google Scholar
  9. [9]
    N.C. Beaulieu, An infinite series for the computation of the complementary probability distribution function of a sum of independent random variables and its application to the sum of Rayleigh random variables, IEEE Trans. Commun. COM-38 (1990) 1463–1474.Google Scholar
  10. [10]
    R. Bellman, R.E. Kalaba and J. Lockett,Numerical Inversion of the Laplace Transform. Application to Biology, Economics, Engineering and Physics (American Elsevier, New York, 1966).Google Scholar
  11. [11]
    B.C. Berndt,Ramanujan's Notebooks, Part II (Springer, New York, 1989).Google Scholar
  12. [12]
    D. Bertsimas and D. Nakazato, Transient and busy period analysis for theGI/G/1 queue; the method of stages, Queueing Systems 10 (1992) 153–184.Google Scholar
  13. [13]
    H. Bohman, A method to calculate the distribution function when the characteristic function is known, Ark. Mat. 4 (1960) 99–157.Google Scholar
  14. [14]
    H. Bohman, A method to calculate the distribution function when the characteristic function is known, BIT 10 (1970) 237–242.Google Scholar
  15. [15]
    H. Bohman, From characteristic function to distribution function via Fourier analysis, BIT 12 (1972) 279–283.Google Scholar
  16. [16]
    E.O. Brigham and R.E. Conley, Evaluation of cumulative probability distribution functions: improved numerical methods, IEEE Proc. 58 (1970) 1367–1368.Google Scholar
  17. [17]
    A.S. Carasso, Infinitely divisible pulses, continuous deconvolution, and the characterization of linear time invariant systems, SIAM J. Appl. Math. 47 (1987) 892–927.Google Scholar
  18. [18]
    H.S. Carslaw,Introduction to the Theory of Fourier's Series and Integrals, 3rd ed. (Dover, New York, 1930).Google Scholar
  19. [19]
    J.K. Cavers, On the fast Fourier transform inversion of probability generating functions, J. Inst. Math. Appl. 22 (1978) 275–282.Google Scholar
  20. [20]
    D.C. Champeney,A Handbook of Fourier Theorems (Cambridge University Press, New York, 1987).Google Scholar
  21. [21]
    K.L. Chung,A Course in Probability Theory, 2nd ed. (Academic Press, New York, 1974).Google Scholar
  22. [22]
    J.W. Cooley, P.A.W. Lewis and P.D. Welch, Application of the fast Fourier transform to the computation of Fourier integrals, Fourier series, and convolution integrals, IEEE Trans. AU-15 (1967) 79–84.Google Scholar
  23. [23]
    J.W. Cooley, P.A.W. Lewis and P.D. Welch, Historical notes on the fast Fourier transform, Proc. IEEE 55 (1967) 1675–1677.Google Scholar
  24. [24]
    J.W. Cooley, P.A.W. Lewis and P.D. Welch, The fast Fourier transform algorithm: programming considerations of sine, cosine and Laplace transforms, J. Sound Vib. 12 (1970) 315–337.Google Scholar
  25. [25]
    J.W. Cooley and J.W. Tukey, An algorithm for the machine computation of complex Fourier series, Math. Comp. 19 (1965) 297–301.Google Scholar
  26. [26]
    D.R. Cox,Renewal Theory (Methuen, London, 1962).Google Scholar
  27. [27]
    K.S. Crump, Numerical inversion of Laplace transforms using a Fourier-series approximation, J. ACM 23 (1976) 89–96.Google Scholar
  28. [28]
    J.N. Daigle, Queue length distributions from probability generating functions via discrete Fourier transforms, Oper. Res. Lett. 8 (1989) 229–236.Google Scholar
  29. [29]
    B. Davies and B.L. Martin, Numerical inversion of Laplace transforms: a critical evaluation and review of methods, J. Comp. Phys. 33 (1970) 1–32.Google Scholar
  30. [30]
    R.B. Davies, Numerical inversion of a characteristic function, Biometrika 60 (1973) 415–417.Google Scholar
  31. [31]
    R.B. Davies, The distribution of a linear combination ofX 2 random variables, Appl. Stat. 29 (1980) 323–333.Google Scholar
  32. [32]
    P.J. Davis and P. Rabinowitz,Methods of Numerical Integration, 2nd ed. (Academic Press, New York, 1984).Google Scholar
  33. [33]
    M.A.B. Deakin, Euler's version of the Laplace transform, Amer. Math. Monthly 87 (1980) 264–269.Google Scholar
  34. [34]
    G. de Balbine and J. Franklin, The calculation of Fourier integrals, Math. Comp. 20 (1966) 570–589.Google Scholar
  35. [35]
    F.R. de Hoog, J.H. Knight and A.N. Stokes, An improved method for numerical inversion of Laplace transforms, SIAM J. Sci. Stat. Comput. 3 (1982) 357–366.Google Scholar
  36. [36]
    G. Doetsch,Introduction to the Theory and Application of the Laplace Transformation (Springer, New York, 1974).Google Scholar
  37. [37]
    B.T. Doshi, Analysis of clocked schedules — high priority tasks, AT&T Tech. J. 64 (1985) 633–659.Google Scholar
  38. [38]
    B.T. Doshi and J. Kaufman, Sojourn times in anM/G/1 queue with Bernoulli feedback, in:Queueing Theory and Its Applications, Liber Amicorum for J.W. Cohen, eds. O.J. Boxma and R. Syski (North-Holland, Amsterdam, 1988).Google Scholar
  39. [39]
    H. Dubner, Partitions approximated by finite cosine series, Math. Computation, to appear.Google Scholar
  40. [40]
    H. Dubner and J. Abate, Numerical inversion of Laplace transforms by relating them to the finite Fourier cosine transform, J. ACM 15 (1968) 115–123.Google Scholar
  41. [41]
    F. Durbin, Numerical inversion of Laplace transforms: An efficient improvement to Dubner and Abate's method, Comput. J. 17 (1974) 371–376.Google Scholar
  42. [42]
    W. Feller,An Introduction to Probability Theory and its Applications, Vol. I, 3rd ed. (Wiley, New York, 1968).Google Scholar
  43. [43]
    W. Feller,An Introduction to Probability Theory and its Applications, Vol. II, 2nd ed. (Wiley, New York, 1971).Google Scholar
  44. [44]
    H.E. Fettis, Numerical calculation of certain definite integrals by Poisson's summation formula, Math. Tables Other Aids Comput. 9 (1955) 85–92.Google Scholar
  45. [45]
    B. Fornberg, Numerical differentiation of analytic functions, ACM Trans. Math. Software 7 (1981) 512–526.Google Scholar
  46. [46]
    J. Foster and F.B. Richards, The Gibbs phenomenon for piecewise-linear approximations, Amer. Math. Monthly 98 (1991) 47–49.Google Scholar
  47. [47]
    B.S. Garbow, G. Giunta, J.N. Lyness and A. Murli, Algorithm 662, A FORTRAN software package for the numerical inversion of the Laplace transform based on Weeks' method, ACM Trans. Meth. Software 14 (1988) 171–176.Google Scholar
  48. [48]
    W. Gautschi, On the condition of a matrix arising in the numerical inversion of the Laplace transform, Math. Comput. 23 (1969) 109–118.Google Scholar
  49. [49]
    D.P. Gaver, Jr., Observing stochastic processes and approximate transform inversion, Oper. Res. 14 (1966) 444–459.Google Scholar
  50. [50]
    D.P. Gaver, Jr., Diffusion approximations and models for certain congestion problems, J. Appl. Prob. 5 (1968) 607–623.Google Scholar
  51. [51]
    J. Gil-Palaez, Note on the inversion theorem, Biometrika 38 (1951) 481–482.Google Scholar
  52. [52]
    I.J. Good, Analogs of Poisson's sum formula, Amer. Math. Monthly 69 (1962) 259–266.Google Scholar
  53. [53]
    F.J. Harris, On the use of windows for harmonic analysis with the discrete Fourier transform, Proc. IEEE 66 (1978) 51–83.Google Scholar
  54. [54]
    P.G. Harrison, Laplace transform inversion and passage-time distributions in Markov processes, J. Appl. Prob. 27 (1990) 74–87.Google Scholar
  55. [55]
    H. Heffes and D.M. Lucantoni, A Markov modulated characterization of packetized voice and data traffic and related statistical multiplexer performance, IEEE J. Sel. Areas Commun. SAC-4 (1986) 856–868.Google Scholar
  56. [56]
    D.P. Heyman, Mathematical models of database degradation, ACM Trans. Database Sys. 7 (1982) 615–631.Google Scholar
  57. [57]
    G. Honig and U. Hirdes, A method for the numerical inversion of Laplace transforms, J. Comput. Appl. Math. 10 (1984) 113–129.Google Scholar
  58. [58]
    T. Hosono, Numerical inversion of Laplace transform, J. Inst. Elec. Eng. Jpn. 54–A64 (1979) 494 (in Japanese).Google Scholar
  59. [59]
    T. Hosono, Numerical inversion of Laplace transform and some applications to wave optics, Radio Sci. 16 (1981) 1015–1019.Google Scholar
  60. [60]
    T. Hosono,Fast Inversion of Laplace Transform by BASIC (Kyoritsu Publishers, Japan, 1984; in Japanese).Google Scholar
  61. [61]
    T. Hosono, Numerical algorithm for Taylor series expansion, Electronics and Communications in Japan 69 (1986) 10–18.Google Scholar
  62. [62]
    T. Hosono, K. Yuda and A. Itoh, Analysis of transient response of electromagnetic waves scattered by a perfectly conducting sphere. The case of back- and forward-scattering, Electronics and Communications in Japan 71 (1988) 74–86.Google Scholar
  63. [63]
    J.T. Hsu and J.S. Dranoff, Numerical inversion of certain Laplace transforms by the direct application of fast Fourier transform (FFT) algorithm, Comput. Chem. Engng. 11 (1987) 101–110.Google Scholar
  64. [64]
    S. Ichikawa and A. Kishima, Application of Fourier-series technique to inverse Laplace transform (Part I), Mem. Fac. Eng. Kyoto U. 34 (1972) 53–67.Google Scholar
  65. [65]
    S. Ichikawa and A. Kishima, Application of Fourier-series technique to inverse Laplace transform (Part II), Mem. Fac. Eng. Kyoto U. 35 (1973) 393–400.Google Scholar
  66. [66]
    D.L. Jagerman, An inversion technique for the Laplace transform with applications, Bell Sys. Tech. J. 57 (1978) 669–710.Google Scholar
  67. [67]
    D.L. Jagerman, An inversion technique for the Laplace transform, Bell Sys. Tech. J. 61 (1982) 1995–2002.Google Scholar
  68. [68]
    D.L. Jagerman, MATHCALC, AT&T Bell Laboratories, Holmdel, NJ (1987).Google Scholar
  69. [69]
    D.L. Jagerman, The approximation sequence of the Laplace transform, AT&T Bell Laboratories, Holmdel, NJ (1989).Google Scholar
  70. [70]
    R. Johnsonbaugh, Summing an alternating series, Amer. Math. Monthly 86 (1979) 637–648.Google Scholar
  71. [71]
    J. Keilson, Exponential spectra as a tool for the study of single-server systems, J. Appl. Prob. 15 (1978) 162–170.Google Scholar
  72. [72]
    D.G. Kendall, A summation formula for finite trigonometric integrals, Quart. J. Math. 13 (1942) 172–184.Google Scholar
  73. [73]
    J.E. Kiefer and G.H. Weiss, A comparison of two methods for accelerating the convergence of Fourier-series, Comput Math. Appl. 7 (1981) 527–535.Google Scholar
  74. [74]
    Y. Kida, UBASIC Version 8.12, Faculty of Science, Kanazawa University, 1-1 Marunouchi, Kanazawa 920, Japan (1990).Google Scholar
  75. [75]
    L. Kleinrock,Queueing Systems, Vol. 1: Theory (Wiley, New York, 1975).Google Scholar
  76. [76]
    H. Kobayashi,Modeling and Analysis (Addison-Wesley, Reading, MA, 1978).Google Scholar
  77. [77]
    S. Koizumi, A new method of evaluation of the Heaviside operational expression by Fourier series, Phil. Mag. 19 (1935) 1061–1076.Google Scholar
  78. [78]
    V.I. Krylov and N.S. Skoblya,A Handbook of Methods of Approximate Fourier Transformation and Inversion of the Laplace Transformation (Mir Publ., Moscow, 1977).Google Scholar
  79. [79]
    Y.K. Kwok and D. Barthez, An algorithm for the numerical inversion of Laplace transforms, Inverse Problems 5 (1989) 1089–1095.Google Scholar
  80. [80]
    E. Lukacs,Characteristic Functions, 2nd ed. (Hafner, New York, 1970).Google Scholar
  81. [81]
    Y.L. Luke, Simple formulas for the evaluation of some higher transcendental functions, J. Math. Phys. 34 (1955) 298–307.Google Scholar
  82. [82]
    J.N. Lyness, Differentiation formulas for analytic functions, Math. Comp. 22 (1968) 352–356.Google Scholar
  83. [83]
    J.N. Lyness and G. Giunta, A modification of the Weeks method for numerical inversion of the Laplace transform, Math. Comp. 47 (1986) 313–322.Google Scholar
  84. [84]
    J.N. Lyness and C.B. Moler, Numerical differentation of analytic functions, SIAM J. Numer. Anal. 4 (1967) 202–210.Google Scholar
  85. [85]
    W.F. Magnus, F. Oberhettinger and R.P. Soni,Formulas and Theorems for the Special Functions of Mathematical Physics (Springer, New York, 1966).Google Scholar
  86. [86]
    M.R. Middleton, Transient effects inM/G/1 queues, Ph.D. dissertation, Stanford University (1979).Google Scholar
  87. [87]
    P.L. Mills, Numerical inversion of z-transforms with application to polymer kinetics, Comp. Chem. 2 (1987) 137–151.Google Scholar
  88. [88]
    A. Murli and M. Rizzardi, Algorithm 682, Talbot's method for the Laplacc inversion problem, ACM Trans. Math. Software 16 (1990) 158–168.Google Scholar
  89. [89]
    N. Mullineux and J.R. Reed, Numerical inversion of integral transforms, Comput. Math. Appl. 3 (1977) 299–306.Google Scholar
  90. [90]
    R.E. Nancc, U.N. Bhat and B.G. Claybrook, Busy period analysis of a time sharing system: transform inversion, J. ACM 19 (1972) 453–463.Google Scholar
  91. [91]
    I.P. Natanson,Constructive Function Theory, Vol. I, Uniform Approximation (F. Ungar, New York, 1964).Google Scholar
  92. [92]
    W.D. Neumann, UBASIC: A public-domain BASIC for mathematics, Notices Amer. Math. Soc. 36 (1989) 557–559.Google Scholar
  93. [93]
    A.H. Nuttall, Numerical evaluation of cumulative probability distribution functions directly from characteristic functions, IEEE Proc. 57 (1969) 2071–2072.Google Scholar
  94. [94]
    F. Oberhettinger,Fourier Transforms of Distributions and Their Inverses (Academic, New York, 1973).Google Scholar
  95. [95]
    W.C. Obi, LAPLACE — A performance analysis library (PAL) module, AT&T Bell Laboratories, Holmdel, NJ (1987).Google Scholar
  96. [96]
    R. Piessens, A bibliography on numerical inversion of the Laplace transform and its applications, J. Comput. Appl. Math. 1 (1975) 115–128.Google Scholar
  97. [97]
    R. Piessens, and N.D.P. Dang, A bibliography on numerical inversion of the Laplace transform and its applications: A supplement, J. Comput. Appl. Math. 2 (1976) 225–228.Google Scholar
  98. [98]
    R. Piessens and R. Huysmans, Algorithm 619. Automatic numerical inversion of the Laplace transforms, ACM Trans. Math. Softw. 10 (1984) 348–353.Google Scholar
  99. [99]
    L.K. Platzman, J.C. Ammons and J.J. Bartholdi, III, A simple and efficient algorithm to compute tail probabilities from transforms, Oper. Res. 36 (1988) 137–144.Google Scholar
  100. [100]
    S.D. Poisson, Mémoire sur le Calcul Numérique des Integrales Défines, Mem. Acad. Sci. Inst. France 6 (1823) 571–602.Google Scholar
  101. [101]
    E.L. Post, Generalized differentiation, Trans. Amer. Math. Soc. 32 (1930) 723–781.Google Scholar
  102. [102]
    L.R. Rabiner and B. Gold,Theory and Application of Digital Signal Processing (Prentice-Hall, Englewood Cliffs, NJ, 1975).Google Scholar
  103. [103]
    A.A.G. Requicha, Direct computation of distribution functions from characteristic functions using the fast Fourier transform, IEEE Proc. 58 (1970) 1154–1155.Google Scholar
  104. [104]
    J. Riordan,Stochastic Service Systems (Wiley, New York, 1962).Google Scholar
  105. [105]
    S.O. Rice, Efficient evaluation of integrals of analytic functions by the trapezoidal rule, Bell Sys. Tech. J. 52 (1973) 707–722.Google Scholar
  106. [106]
    S.O. Rice, Numerical evaluation of integrals with infinite limits and oscillating integrands, Bell. Sys. Tech. J. 54 (1975) 155–164.Google Scholar
  107. [107]
    S. Ross,Stochastic Processes (Wiley, New York, 1983).Google Scholar
  108. [108]
    B. Schorr, Numerical inversion of a class of characteristic functions, BIT 15 (1975) 94–102.Google Scholar
  109. [109]
    M. Silverberg, Improving the efficiency of Laplace-transform Inversion for network analysis, Electronics Lett. 6 (1970) 105–106.Google Scholar
  110. [110]
    R.M. Simon, M.T. Stroot and G.H. Weiss, Numerical inversion of Laplace transforms with applications to percentage labeled experiments, Comput. Biomed. Res. 6 (1972) 596–607.Google Scholar
  111. [111]
    W.L. Smith, On the distribution of queueing times, Proc. Camb. Phil. Soc. 49 (1953) 449–461.Google Scholar
  112. [112]
    W. Squire, The numerical treatment of Laplace transforms: the Koizumi inversion method, Int. J. Num. Meth. Eng. 20 (1984) 1697–1702.Google Scholar
  113. [113]
    H. Stehfest, Algorithm 368. Numerical inversion of Laplace transforms, Commun. ACM 13 (1970) 479–49 (erratum 13, 624).Google Scholar
  114. [114]
    F. Stenger, (1981) Numerical methods based on Whittaker cardinal, or sine functions, SIAM Rev. 23 (1981) 165–224.Google Scholar
  115. [115]
    D. Stoyan,Comparison Methods for Queues and Other Stochastic Models (Wiley, Chichester, 1983).Google Scholar
  116. [116]
    A. Talbot, The accurate numerical inversion of Laplace transforms, J. Inst. Math. Appl. 23 (1979) 97–120.Google Scholar
  117. [117]
    D. ter Haar, An easy approximate method of determining the relaxation spectrum of a viscoelastic material, J. Polymer Sci. 6 (1951) 247–250.Google Scholar
  118. [118]
    H.C. Tijms,Stochastic Modelling and Analysis: A Computational Approach (Wiley, Chichester, 1986).Google Scholar
  119. [119]
    G.P. Tolstov,Fourier Series (Dover, New York, 1976).Google Scholar
  120. [120]
    B. Van Der Pol and H. Bremmer,Operational Calculus (Cambridge Press, 1955; reprinted, Chelsea, New York, 1987).Google Scholar
  121. [121]
    J.M. Varah, Pitfalls in the numerical solution of linear ill-posed problems, SIAM J. Sci. Stat. Comput. 4 (1983) 164–176.Google Scholar
  122. [122]
    F. Veillon, Une nouvelle méthode de calcul de la transformée inverse d'une fonction au sens de Laplace et de la deconvolution de deux fonctions, R.A.I.R.O. 6 (1972) 91–98.Google Scholar
  123. [123]
    F. Veillon, Algorithm 486. Numerical inversion of Laplace transform, Commun. AGM 17 (1974) 587–589.Google Scholar
  124. [124]
    W.T. Weeks, Numerical inversion of Laplace transforms using Laguerre functions, J. ACM 13 (1966) 419–426.Google Scholar
  125. [125]
    D.V. Widder, The inversion of the Laplace integral and the related moment problem, Trans. Amer. Math. Soc. 36 (1934) 107–200.Google Scholar

Copyright information

© J.C. Baltzer A.G. Scientific Publishing Company 1992

Authors and Affiliations

  • Joseph Abate
    • 1
  • Ward Whitt
    • 2
  1. 1.RidgewoodUSA
  2. 2.AT&T Bell LaboratoriesMurray HillUSA

Personalised recommendations