Abstract
Cyclic reduction is an algorithm invented by G. H. Golub and R. W. Hockney in the mid 1960s for solving linear systems related to the finite differences discretization of the Poisson equation over a rectangle. Among the algorithms of Gene Golub, it is one of the most versatile and powerful ever created. Recently, it has been applied to solve different problems from different applicative areas. In this paper we survey the main features of cyclic reduction, relate it to properties of analytic functions, recall its extension to solving more general finite and infinite linear systems, and different kinds of nonlinear matrix equations, including algebraic Riccati equations, with applications to Markov chains, queueing models and transport theory. Some new results concerning the convergence properties of cyclic reduction and its applicability are proved under very weak assumptions. New formulae for overcoming breakdown are provided.
Similar content being viewed by others
References
Amodio, P.: Optimized cyclic reduction for the solution of linear tridiagonal systems on parallel computers. Comput. Math. Appl. 26(3), 45–53 (1993)
Amodio, P., Brugnano, L., Politi, T.: Parallel factorizations for tridiagonal matrices. SIAM J. Numer. Anal. 30(3), 813–823 (1993)
Amodio, P., Mastronardi, N.: A parallel version of the cyclic reduction algorithm on a hypercube. Parallel Comput. 19(11), 1273–1281 (1993)
Amodio, P., Mazzia, F.: Backward error analysis of cyclic reduction for the solution of tridiagonal systems. Math. Comp. 62(206), 601–617 (1994)
Amodio, P., Paprzycki, M.: A cyclic reduction approach to the numerical solution of boundary value ODEs. SIAM J. Sci. Comput. 18(1), 56–68 (1997) (Dedicated to C. William Gear on the occasion of his 60th birthday)
Anderson, B.D.O.: Second-order convergent algorithms for the steady-state Riccati equation. Int. J. Control 28(2), 295–306 (1978)
Bialecki, B.: Cyclic reduction and FACR methods for piecewise Hermite bicubic orthogonal spline collocation. Numer. Algorithms 8(2–4), 167–184 (1994)
Bini, D., Meini, B.: On the solution of a nonlinear matrix equation arising in queueing problems. SIAM J. Matrix Anal. Appl. 17(4), 906–926 (1996)
Bini, D.A., Böttcher, A.: Polynomial factorization through Toeplitz matrix computations. Linear Algebra Appl. 366, 25–37 (2003) (Special issue on structured matrices: analysis, algorithms and applications (Cortona, 2000))
Bini, D.A., Fiorentino, G., Gemignani, L., Meini, B.: Effective fast algorithms for polynomial spectral factorization. Numer. Algorithms 34(2–4), 217–227 (2003) (International Conference on Numerical Algorithms, Vol. II (Marrakesh, 2001))
Bini, D.A., Gemignani, L., Meini, B.: Factorization of analytic functions by means of Koenig’s theorem and Toeplitz computations. Numer. Math. 89(1), 49–82 (2001)
Bini, D.A., Gemignani, L., Meini, B.: Computations with infinite Toeplitz matrices and polynomials. Linear Algebra Appl. 343/344, 21–61 (2002) (Special issue on structured and infinite systems of linear equations)
Bini, D.A., Gemignani, L., Meini, B.: Solving certain matrix equations by means of Toeplitz computations: algorithms and applications. In: Fast Algorithms for Structured Matrices: Theory and Applications (South Hadley, MA, 2001), vol. 323 of Contemp. Math., pp. 151–167. American Mathematical Society, Providence (2003)
Bini, D.A., Higham, N.J., Meini, B.: Algorithms for the matrix pth root. Numer. Algorithms 39(4), 349–378 (2005)
Bini, D.A., Iannazzo, B., Latouche, G., Meini, B.: On the solution of algebraic Riccati equations arising in fluid queues. Linear Algebra Appl. 413(2–3), 474–494 (2006)
Bini, D.A., Iannazzo, B., Meini, B., Poloni, F.: Nonsymmetric algebraic Riccati equations associated with an M-matrix: recent advances and algorithms. In: Bini, D., Meini, B., Ramaswami, V., Remiche, M.-A., Taylor, P. (eds.) Numerical Methods for Structured Markov Chains, number 07461 in Dagstuhl Seminar Proceedings, Dagstuhl, Germany, (2008). Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Schloss Dagstuhl
Bini, D.A., Latouche, G., Meini, B.: Solving matrix polynomial equations arising in queueing problems. Linear Algebra Appl. 340, 225–244 (2002)
Bini, D.A., Latouche, G., Meini, B.: Numerical Methods for Structured Markov Chains. Numerical Mathematics and Scientific Computation. Oxford University Press, New York (2005) (Oxford Science Publications)
Bini, D.A., Meini, B.: On cyclic reduction applied to a class of Toeplitz-like matrices arising in queueing problems. In: Stewart, W.J. (ed.) Computations with Markov Chains, pp. 21–38. Kluwer, Dordrecht (1995)
Bini, D.A., Meini, B.: Improved cyclic reduction for solving queueing problems. Numer. Algorithms 15(1), 57–74 (1997)
Bini, D.A., Meini, B.: Effective methods for solving banded Toeplitz systems. SIAM J. Matrix Anal. Appl. 20(3), 700–719 (1999) (electronic)
Bini, D.A., Meini, B.: Approximate displacement rank and applications. In: Structured Matrices in Mathematics, Computer Science, and Engineering, II (Boulder, CO, 1999), vol. 281 of Contemp. Math., pp. 215–232. American Mathematical Society, Providence (2001)
Bini, D.A., Meini, B.: Non-skip-free M/G/1-type Markov chains and Laurent matrix power series. Linear Algebra Appl. 386, 187–206 (2004)
Bini, D.A., Meini B., Poloni, F.: Fast solution of a certain Riccati equation through Cauchy-like matrices. Electron. Trans. Numer. Anal. (2009). http://etna.mcs.kent.edu/
Bini, D.A., Meini, B., Poloni, F.: From algebraic Riccati equations to unilateral quadratic matrix equations: old and new algorithms. In: Bini, D., Meini, B., Ramaswami, V., Remiche, M.-A., Taylor, P. (eds.) Numerical Methods for Structured Markov Chains, number 07461 in Dagstuhl Seminar Proceedings, Dagstuhl, Germany, (2008). Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Schloss Dagstuhl
Bini, D.A., Meini, B., Ramaswami, V.: Analyzing M/G/1 paradigms through QBDs: the role of the block structure in computing the matrix G. In: Latouche, G., Taylor, P. (eds.) Advances in Algorithmic Methods for Stochastic Models, pp. 73–86. Notable Publications, Neshanic Station (2000) (Proceedings of the Third Conference on Matrix Analytic Methods)
Bini, D.A., Meini, B., Ramaswami, V.: A probabilistic interpretation of Cyclic Reduction and its relationships with Logarithmic Reduction. Calcolo 45, 207–216 (2008)
Bini, D.A., Meini, B., Spitkovsky, I.M.: Shift techniques and canonical factorizations in the solution of M/G/1-type Markov chains. Stoch. Models 21(2–3), 279–302 (2005)
Bini, D.A., Meini, B., Steffé, S., Van Houdt, B.: Structured Markov chains solver: algorithms. In: Proceedings of ValueTools, Pisa. ACM, New York (2006)
Bini, D.A., Meini, B., Steffé, S., Van Houdt, B.: Structured Markov chains solver: software tools. In: Proceedings of ValueTools, Pisa. ACM, New York (2006)
Boisvert, R.F.: Algorithms for special tridiagonal systems. SIAM J. Sci. Statist. Comput. 12(2), 423–442 (1991)
Bondeli, S., Gander, W.: Cyclic reduction for special tridiagonal systems. SIAM J. Matrix Anal. Appl. 15(1), 321–330 (1994)
Briggs, W.L., Turnbull, T.: Fast Poisson solvers for MIMD computers. Parallel Comput. 6(3), 265–274 (1988)
Buneman, O.: A compact non-iterative Poisson solver. Technical Report SUIPR Rep. 294, Institue for Plasma Research, Stanford University, Paolo Alto, California, (1969)
Buzbee, B.L., Golub, G.H., Nielson, C.W.: On direct methods for solving Poisson’s equations. SIAM J. Numer. Anal. 7, 627–656 (1970)
Cottle, R.W.: Manifestations of the Schur complement. Linear Algebra Appl. 8, 189–211 (1974)
Diamond, M.A., Ferreira, D.L.V.: On a cyclic reduction method for the solution of Poisson’s equations. SIAM J. Numer. Anal. 13(1), 54–70 (1976)
Dodson, D.S., Levin, S.A.: A tricyclic tridiagonal equation solver. SIAM J. Matrix Anal. Appl. 13(4), 1246–1254 (1992)
Engwerda, J.C., Ran, A.C.M., Rijkeboer, A.L.: Necessary and sufficient conditions for the existence of a positive definite solution of the matrix equation X + A * X − 1 A = Q. Linear Algebra Appl. 186, 255–275 (1993)
Ferrante, A., Levy, B.C.: Hermitian solutions of the equation X = Q + NX − 1 N *. Linear Algebra Appl. 247, 359–373 (1996)
Gallopoulos, E., Saad, Y.: A parallel block cyclic reduction algorithm for the fast solution of elliptic equations. In: Supercomputing (Athens, 1987), vol. 297 of Lecture Notes in Comput. Sci., pp. 563–575. Springer, Berlin (1988)
Gallopoulos, E., Saad, Y.: A parallel block cyclic reduction algorithm for the fast solution of elliptic equations. Parallel Comput. 10(2), 143–159 (1989)
Gander, W., Golub, G.H.: Cyclic reduction—history and applications. In: Scientific computing (Hong Kong, 1997), pp. 73–85. Springer, Singapore (1997)
Gautschi, W.: Orthogonal Polynomials: Computation and Approximation. Numerical Mathematics and Scientific Computation. Oxford University Press, New York (2004) (Oxford Science Publications)
Guo, C.-H.: Convergence analysis of the Latouche-Ramaswami algorithm for null recurrent quasi-birth-death processes. SIAM J. Matrix Anal. Appl. 23(3), 744–760 (2001/2002) (electronic)
Guo, C.-H.: Comments on a shifted cyclic reduction algorithm for quasi-birth-death problems. SIAM J. Matrix Anal. Appl. 24(4), 1161–1166 (2003) (electronic)
Guo, C.-H.: Efficient methods for solving a nonsymmetric algebraic Riccati equation arising in stochastic fluid models. J. Comput. Appl. Math. 192(2), 353–373 (2006)
Guo, C.-H., Higham, N.J.: Iterative solution of a nonsymmetric algebraic Riccati equation. SIAM J. Matrix Anal. Appl. 29(2), 396–412 (2007)
Guo, C.-H., Higham, N.J., Tisseur, F.: Detecting and solving hyperbolic quadratic eigenvalue problems. Technical Report MIMS EPrint: 2007.117, Manchester Institute for Mathematical Sciences, School of Mathematics (2007)
Guo, C.-H., Iannazzo, B., Meini, B.: On the doubling algorithm for a (shifted) nonsymmetric algebraic Riccati equation. SIAM J. Matrix Anal. Appl. 29(4), 1083–1100 (2007)
Guo, C.-H., Lancaster, P.: Algorithms for hyperbolic quadratic eigenvalue problems. Math. Comp. 74(252), 1777–1791 (2005) (electronic)
Guo, X.-X., Lin, W.-W., Xu, S.-F.: A structure-preserving doubling algorithm for nonsymmetric algebraic Riccati equation. Numer. Math. 103(3), 393–412 (2006)
He, C., Meini, B., Rhee, N.H., Sohraby, K.: A quadratically convergent Bernoulli-like algorithm for solving matrix polynomial equations in Markov chains. Electron. Trans. Numer. Anal. 17, 151–167 (2004) (electronic)
Heller, D.: Some aspects of the cyclic reduction algorithm for block tridiagonal linear systems. SIAM J. Numer. Anal. 13(4), 484–496 (1976)
Henrici, P.: Applied and Computational Complex Analysis, vol. 1. Wiley Classics Library. Wiley, New York (1988) (Power series—integration—conformal mapping—location of zeros, Reprint of the 1974 original, A Wiley-Interscience Publication)
Higham, N.J.: Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, Philadelphia (2008)
Ho, C.-T., Johnsson, S.L.: Optimizing tridiagonal solvers for alternating direction methods on Boolean cube multiprocessors. SIAM J. Sci. Statist. Comput. 11(3), 563–592 (1990)
Hockney, R.W.: A fast direct solution of Poisson’s equation using Fourier analysis. J. Assoc. Comput. Mach. 12, 95–113 (1965)
Hockney, R.W., Jesshope, C.R.: Parallel Computers: Architecture, Programming and algorithms. Hilger, Bristol (1981)
Iannazzo, B.: A note on computing the matrix square root. Calcolo 40(4), 273–283 (2003)
Juang, J.: Global existence and stability of solutions of matrix Riccati equations. J. Math. Anal. Appl. 258(1), 1–12 (2001)
Juang, J., Lin, W.-W.: Nonsymmetric algebraic Riccati equations and Hamiltonian-like matrices. SIAM J. Matrix Anal. Appl. 20(1), 228–243 (1999) (electronic)
Kailath, T., Sayed, A.H. (eds.) Fast Reliable Algorithms for Matrices with Structure. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (1999)
Latouche, G., Ramaswami, V.: A logarithmic reduction algorithm for quasi-birth-death processes. J. Appl. Probab. 30(3), 650–674 (1993)
Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM, Philadelphia (1999)
Lotti, G.: A note on the solution of not balanced banded Toeplitz systems. Numer. Linear Algebra Appl. 14(8), 645–657 (2007)
Meini, B.: Efficient computation of the extreme solutions of \(X+A^* X^{-1}A=Q\) and \(X-A^* X^{-1}A=Q\). Math. Comp. 71(239), 1189–1204 (2002) (electronic)
Meini, B.: The matrix square root from a new functional perspective: theoretical results and computational issues. SIAM J. Matrix Anal. Appl. 26(2), 362–376 (2004/2005) (electronic)
Meini, B.: Nonlinear matrix equations and structured linear algebra. Linear Algebra Appl. 413(2–3), 440–457 (2006)
Neuts, M.F.: Matrix-geometric Solutions in Stochastic Models, vol. 2 of Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore (1981) (An algorithmic approach)
Neuts, M.F.: Structured Stochastic Matrices of M/G/1 Type and Their Applications, vol. 5 of Probability: Pure and Applied. Marcel Dekker, New York (1989)
Ostrowski, A.: Recherches sur la méthode de Graeffe et les zéros des polynomes et des séries de Laurent. Chapitres III et IV. Acta Math. 72, 157–257 (1940)
Ostrowski, A.: Addition à notre mémoire: “Recherches sur la méthode de Graeffe et les zéros des polynômes et des séries de Laurent.” Acta Math. 75, 183–186 (1943)
Pavlov, V., Yalamov, P.: Stabilization by perturbation of ill-conditioned cyclic reduction. Int. J. Comput. Math. 68(3–4), 273–283 (1998)
Ramaswami, V.: The generality of Quasi Birth-and-Death processes. In: Alfa, A.S., Chakravarthy, S.R. (eds.) Advances in Matrix Analytic Methods for Stochastic Models, pp. 93–113. Notable Publications, Neshanic Station (1998)
Ramaswami, V.: Matrix analytic methods for stochastic fluid flows. In: Smith, D., Hey, P. (eds.) Teletraffic Engineering in a Competitive World, Proceedings of the 16th International Teletraffic Congress, pp. 1019–1030. Elsevier Science B.V., Edimburgh (1999)
Reichel, L.: The ordering of tridiagonal matrices in the cyclic reduction method for Poisson’s equation. Numer. Math. 56(2–3), 215–227 (1989)
Rodrigue, G., Wolitzer, D.: Preconditioning by incomplete block cyclic reduction. Math. Comp. 42(166), 549–565 (1984)
Rogers, L.C.G.: Fluid models in queueing theory and Wiener-Hopf factorization of Markov chains. Ann. Appl. Probab. 4(2), 390–413 (1994)
Rosmond, T.E., Faulkner, F.D.: Direct solution of elliptic equations by block cyclic reduction and factorization. Mon. Weather Rev. 104, 641–649 (1976)
Rossi, T., Toivanen, J.: A nonstandard cyclic reduction method, its variants and stability. SIAM J. Matrix Anal. Appl. 20(3), 628–645 (1999) (electronic)
Rossi, T., Toivanen, J.: A parallel fast direct solver for block tridiagonal systems with separable matrices of arbitrary dimension. SIAM J. Sci. Comput. 20(5), 1778–1796 (1999) (electronic)
Schwandt, H.: Cyclic reduction for tridiagonal systems of equations with interval coefficients on vector computers. SIAM J. Numer. Anal. 26(3), 661–680 (1989)
Schwandt, H.: Truncated interval arithmetic block cyclic reduction. Appl. Numer. Math. 5(6), 495–527 (1989)
Sun, W.: Cyclic reduction algorithm for solving collocation systems. Int. J. Comput. Math. 61(3–4), 293–305 (1996)
Swarztrauber, P.N.: A direct method for the discrete solution of separable elliptic equations. SIAM J. Numer. Anal. 11, 1136–1150 (1974)
Swarztrauber, P.N.: The methods of cyclic reduction, Fourier analysis and the FACR algorithm for the discrete solution of Poisson’s equation on a rectangle. SIAM Rev. 19(3), 490–501 (1977)
Swarztrauber, P.N.: Fast Poisson solvers. In: Studies in numerical analysis, vol. 24 of MAA Stud. Math., pp. 319–370. Mathematical Association of America, Washington, DC (1984)
Swarztrauber, P.N.: Approximate cyclic reduction for solving Poisson’s equation. SIAM J. Sci. Statist. Comput. 8(3), 199–209 (1987)
Swarztrauber, P.N., Sweet, R.A.: Algorithm 541—Efficient Fortran subprograms for the solution of separable elliptic partial differential equations. ACM Trans. Math. Softw. 5, 325–364 (1979)
Swarztrauber, P.N., Sweet, R.A.: Vector and parallel methods for the direct solution of Poisson’s equation. J. Comput. Appl. Math. 27(1–2), 241–263 (1989) (Reprinted in Parallel algorithms for numerical linear algebra, 241–263, North-Holland, Amsterdam, 1990)
Sweet, R.A.: A generalized cyclic reduction algorithm. SIAM J. Numer. Anal. 11, 506–520 (1974)
Sweet, R.A.: A cyclic reduction algorithm for solving block tridiagonal systems of arbitrary dimension. SIAM J. Numer. Anal. 14(4), 706–720 (1977)
Sweet, R.A.: A parallel and vector variant of the cyclic reduction algorithm. SIAM J. Sci. Statist. Comput. 9(4), 761–765 (1988)
Temperton, C.: Direct methods for the solution of the discrete Poisson equation: some comparisons. J. Comput. Phys. 31(1), 1–20 (1979)
Temperton, C.: On the FACR(l) algorithm for the discrete Poisson equation. J. Comput. Phys. 34(3), 314–329 (1980)
Vandebril, R., Van Barel, M., Mastronardi, N.: Matrix computations and semiseparable matrices, vol. 1. Johns Hopkins University Press, Baltimore (2008) (Linear systems)
Williams, D.: A “potential-theoretic” note on the quadratic Wiener-Hopf equation for Q-matrices. In: Seminar on Probability, XVI, vol. 920 of Lecture Notes in Math., pp. 91–94. Springer, Berlin (1982)
Yalamov, P., Pavlov, V.: Stability of the block cyclic reduction. Linear Algebra Appl. 249, 341–358 (1996)
Yalamov, P.Y.: On the stability of the cyclic reduction without back substitution for tridiagonal systems. BIT 35(3), 428–447 (1995)
Ye, J., Li, S.-q.: Analysis of multi-media traffic queues with finite buffer and overload control—part 1: Algorithm, pp. 1464–1474. INFOCOM, Paris (1991)
Ye, J., Li, S.-q.: Analysis of multi-media traffic queues with finite buffer and overload control—part ii: Applications, pp. 848–859. INFOCOM, Paris (1992)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bini, D.A., Meini, B. The cyclic reduction algorithm: from Poisson equation to stochastic processes and beyond. Numer Algor 51, 23–60 (2009). https://doi.org/10.1007/s11075-008-9253-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-008-9253-0