Computing and Visualization in Science

, Volume 14, Issue 2, pp 51–65 | Cite as

Iterant recombination with one-norm minimization for multilevel Markov chain algorithms via the ellipsoid method

  • Hans De Sterck
  • Killian MillerEmail author
  • Geoffrey Sanders


Recently, it was shown how the convergence of a class of multigrid methods for computing the stationary distribution of sparse, irreducible Markov chains can be accelerated by the addition of an outer iteration based on iterant recombination. The acceleration was performed by selecting a linear combination of previous fine-level iterates with probability constraints to minimize the two-norm of the residual using a quadratic programming method. In this paper we investigate the alternative of minimizing the one-norm of the residual. This gives rise to a nonlinear convex program which must be solved at each acceleration step. To solve this minimization problem we propose to use a deep-cuts ellipsoid method for nonlinear convex programs. The main purpose of this paper is to investigate whether an iterant recombination approach can be obtained in this way that is competitive in terms of execution time and robustness. We derive formulas for subgradients of the one-norm objective function and the constraint functions, and show how an initial ellipsoid can be constructed that is guaranteed to contain the exact solution and give conditions for its existence. We also investigate using the ellipsoid method to minimize the two-norm. Numerical tests show that the one-norm and two-norm acceleration procedures yield a similar reduction in the number of multigrid cycles. The tests also indicate that one-norm ellipsoid acceleration is competitive with two-norm quadratic programming acceleration in terms of running time with improved robustness.


Markov chain Iterant recombination Ellipsoid algorithm Multigrid Convex programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bause F., Kritzinger P.: Stochastic Petri Nets. Springer, Germany (1996)zbMATHGoogle Scholar
  2. 2.
    Bazaraa M.S., Sherali H.D., Shetty C.M.: Nonlinear Programming: Theory and Algorithms, 3rd edn. Wiley, New Jersey (2006)zbMATHCrossRefGoogle Scholar
  3. 3.
    Berman A., Plemmons R.J.: Nonnegative Matrices in the Mathematical Sciences. SIAM, Philadelphia, PA (1987)Google Scholar
  4. 4.
    Bertsimas D., Tsitisklis J.N.: Introduction to Linear Optimization. Athena Scientific, Belmont, MA (1997)Google Scholar
  5. 5.
    Bland R.G., Goldfarb D., Todd M.J.: The ellipsoid method: a survey. Oper. Res. 29(6), 1039–1091 (1981)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Brandt A., Mikulinsky V.: On recombining iterants in multigrid algorithms and problems with small islands. SIAM J. Sci. Comput. 16, 20–28 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Briggs W.L., Henson V.E., McCormick S.F.: A Multigrid Tutorial, 2nd edn. SIAM, Philadelphia, PA (2000)zbMATHCrossRefGoogle Scholar
  8. 8.
    Buchholz P.: Multilevel solutions for structured Markov chains. SIAM J. Matrix Anal. Appl. 22(2), 342–357 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Cao W.L., Stewart W.J.: Iterative aggregation/disaggregation techniques for nearly uncoupled Markov chains. JACM 32(3), 702–719 (1985)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Chatelin F., Miranker W.L.: Acceleration by aggregation of successive approximation methods. Linear Algebra Appl. 43, 17–47 (1982)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Sterck H., Manteuffel T., McCormick S.F., Miller K., Pearson J., Ruge J., Sanders G.: Smoothed aggregation multigrid for Markov chains. SIAM J. Sci. Comput. 32, 40–61 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Sterck H., Manteuffel T., McCormick S.F., Miller K., Ruge J., Sanders G.: Algebraic multigrid for Markov chains. SIAM J. Sci. Comput. 32, 544–562 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Sterck H., Manteuffel T., McCormick S.F., Nguyen Q., Ruge J.: Multilevel adaptive aggregation for Markov chains, with application to web ranking. SIAM J. Sci. Comput. 30, 2235–2262 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    De Sterck H., Manteuffel T., Miller K., Sanders G.: Top-level acceleration of adaptive algebraic multilevel methods for steady-state solution to Markov chains. Adv. Comput. Math. 35, 375–403 (2010)CrossRefGoogle Scholar
  15. 15.
    Sterck H., Miller K., Sanders G., Winlaw M.: Recursively accelerated multilevel aggregation for Markov chains. SIAM J. Sci. Comput. 32(3), 1652–1671 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Dziuban S.T., Ecker J.G., Kupferschmid M.: Using deep cuts in an ellipsoid algorithm for nonlinear programming. Math. Program. Stud. 25, 93–107 (1985)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Ecker J.G., Kupferschmid M.: An ellipsoid algorithm for nonlinear programming. Math. Program. 27, 83–106 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Frenk J.B.G., Gromicho J., Zhang S.: A deep cut ellipsoid algorithm for convex programming: theory and applications. Math. Program. 63, 83–108 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Goldfarb D., Todd M.J.: Modifications and implementation of the ellipsoid algorithm for linear programming. Math. Program. 23, 1–19 (1982)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Grassmann W., Taksar M., Heyman D.: Regenerative analysis and steady-state distributions for Markov chains. Oper. Res. 33(5), 1107–1116 (1985)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Haviv M.: Aggregation/disaggregation methods for computing the stationary distribution of Markov chains. SIAM J. Numer. Anal. 24(4), 952–966 (1987)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Horn R.A., Johnson C.R.: Matrix Analysis. Cambridge University Press, New York, NY (1985)zbMATHGoogle Scholar
  23. 23.
    Horton, G., Leutenegger, S.T.: A multi-level solution algorithm for steady-state Markov chains. In: Proceedings of the 1994 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pp. 191–200 (1994)Google Scholar
  24. 24.
    Iudin D.B., Nemirovskii A.S.: Informational complexity and effective methods of solution for convex extremal problems. Matekon Transl. Russ. East Eur. Math. Econ. 13, 3–25 (1976)Google Scholar
  25. 25.
    Khachiyan L.G.: A polynomial algorithm in linear programming. Sov. Math. Doklady 20, 191–194 (1976)Google Scholar
  26. 26.
    Koury J.R., McAllister D.F., Stewart W.J.: Iterative methods for computing stationary distributions of nearly completely decomposable Markov chains. SIAM J. Alg. Disc. Meth. 5(2), 164–186 (1984)MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Krieger U.R.: Numerical solution of large finite Markov chains by algebraic multigrid techniques. In: Stewart, W. (eds) Numerical Solution of Markov Chains, pp. 403–424. Kluwer, Dordrecht (1995)Google Scholar
  28. 28.
    Krieger U.R.: On a two-level multigrid solution method for Markov chains. Linear Algebra Appl. 223–224, 415–438 (1995)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Leutenegger, S.T., Horton, G.: On the utility of the multi-level algorithm for the solution of nearly completely decomposable Markov chains. Tech. Rep. 94-44, ICASE (1994)Google Scholar
  30. 30.
    Lüthi H.-J.: On the solution of variational inequalities by the ellipsoid method. Math. Oper. Res. 10(3), 515–522 (1985)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Mandel J., Sekerka B.: A local convergence proof for the iterative aggregation method. Linear Algebra Appl. 51, 163–172 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Marek I., Mayer P.: Convergence analysis of an iterative aggregation/disaggregation method for computing stationary probability vectors of stochastic matrices. Numer. Linear Algebra Appl. 5, 253–274 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Marek I., Mayer P.: Convergence theory of some classes of iterative aggregation/disaggregation methods for computing stationary probability vectors of stochastic matrices. Linear Algebra Appl. 363, 177–200 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Molloy M.K.: IEEE Trans. Comput. C-31, 913–917 (1982)CrossRefGoogle Scholar
  35. 35.
    Philippe B., Saad Y., Stewart W.J.: Numerical methods in Markov chain modeling. Oper. Res. 40(6), 1156–1179 (1992)zbMATHCrossRefGoogle Scholar
  36. 36.
    Rockafellar R.T.: Convex Analysis. Princeton University Press, New Jersey (1970)zbMATHGoogle Scholar
  37. 37.
    Shor N.Z.: Cut-off method with space extension in convex programming problems. Cybernetics 13, 94–96 (1977)Google Scholar
  38. 38.
    Simon H.A., Ando A.: Aggregation of variables in dynamic systems. Econometrica 29, 111–138 (1961)zbMATHCrossRefGoogle Scholar
  39. 39.
    Stewart W.J.: An Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton, NJ (1994)Google Scholar
  40. 40.
    Takahashi, Y.: A lumping method for numerical calculations of stationary distributions of Markov chains. Tech. Rep. B-18, Department of Information Sciences, Tokyo Institute of Technology (1975)Google Scholar
  41. 41.
    Treister E., Yavneh I.: On-the-fly adaptive smoothed aggregation for Markov chains. SISC 33, 2927–2949 (2011)Google Scholar
  42. 42.
    Treister E., Yavneh I.: Square and stretch multigrid for stochastic matrix eigenproblems. Numer. Linear Algebr. 17, 229–251 (2010)MathSciNetzbMATHGoogle Scholar
  43. 43.
    Trottenberg U., Oosterlee C.W., Schüller A.: Multigrid. Elsevier Academic Press, San Diego, California (2001)zbMATHGoogle Scholar
  44. 44.
    Washio T., Oosterlee C.W.: Krylov subspace acceleration for nonlinear multigrid schemes. Electron. Trans. Numer. Anal. 6, 271–290 (1997)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Hans De Sterck
    • 1
  • Killian Miller
    • 1
    Email author
  • Geoffrey Sanders
    • 2
  1. 1.Department of Applied MathematicsUniversity of WaterlooWaterlooCanada
  2. 2.Center for Applied Scientific ComputingLawrence Livermore National LaboratoryLivermoreUSA

Personalised recommendations