Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Polyhedral Omega: a New Algorithm for Solving Linear Diophantine Systems


Polyhedral Omega is a new algorithm for solving linear Diophantine systems (LDS), i.e., for computing a multivariate rational function representation of the set of all non-negative integer solutions to a system of linear equations and inequalities. Polyhedral Omegacombines methods from partition analysis with methods from polyhedral geometry. In particular, we combine MacMahon’s iterative approach based on the Omega operator and explicit formulas for its evaluation with geometric tools such as Brion decompositions and Barvinok’s short rational function representations. In this way, we connect two recent branches of research that have so far remained separate, unified by the concept of symbolic cones which we introduce. The resulting LDS solver Polyhedral Omegais significantly faster than previous solvers based on partition analysis and it is competitive with state-of-the-art LDS solvers based on geometric methods. Most importantly, this synthesis of ideas makes Polyhedral Omegathe simplest algorithm for solving linear Diophantine systems available to date. Moreover, we provide an illustrated geometric interpretation of partition analysis, with the aim of making ideas from both areas accessible to readers from a wide range of backgrounds.


  1. 1.

    Andrews, G.E., Paule, P.: MacMahon’s partition analysis IV: hypergeometric multisums. Sém. Lothar. Combin. B42i, 1–24 (1999)

  2. 2.

    Andrews G.E., Paule P.: MacMahon’s partition analysis XII: plane partitions. J. London Math. Soc. 76(3), 647–666 (2007)

  3. 3.

    Andrews G.E., Paule P., Riese A.: MacMahon’s partition analysis IX: k-gon partitions. Bull. Aust. Math. Soc. 64(2), 321–329 (2001)

  4. 4.

    Andrews G.E., Paule P., Riese A.: MacMahon’s partition analysis: the Omega package. European J. Combin. 22(7), 887–904 (2001)

  5. 5.

    Andrews G.E., Paule P., Riese A.: MacMahon’s partition analysis VI: a new reduction algorithm. Ann. Combin. 5(3-4), 251–270 (2001)

  6. 6.

    Andrews, G.E., Paule, P., Riese, A.: MacMahon’s partition analysis VII: constrained compositions. In: Berndt, B.C., Ono, K. (eds.) q-Series with Applications to Combinatorics, Number Theory, and Physics, pp. 11–27. Amer. Math. Soc., Providence, RI (2001)

  7. 7.

    Andrews G.E., Paule P., Riese A.: MacMahon’s partition analysis VIII: plane partition diamonds. Adv. Appl. Math. 27(2-3), 231–242 (2001)

  8. 8.

    Andrews G.E., Paule P., Riese A.: MacMahon’s partition analysis X: plane partitions with diagonals. South East Asian J. Math. Math. Sci. 3(1), 3–14 (2004)

  9. 9.

    Andrews, G.E., Paule, P., Riese, A.: MacMahon’s partition analysis XI: hexagonal plane partitions. SFB-Report, 2004-4. Johannes Kepler University Linz (2004)

  10. 10.

    Andrews, G.E., Paule, P., Riese, A., Strehl, V.: MacMahon’s partition analysis V: bijections, recursions, and magic squares. In: Betten, A., Kohnert, A., Laue, R., Wassermann, A. (eds.) Algebraic Combinatorics and Applications, pp. 1–39. Springer, Berlin (2001)

  11. 11.

    Aparicio Monforte, A., Ainhoa, Kauers, M.: Formal laurent series in several variables. Expo. Math. 31(4), 350–367 (2013)

  12. 12.

    Baldoni V., Berline N., De Loera J.A., Köppe M., Vergne M.: How to integrate a polynomial over a simplex. Math. Comp. 80, 297–325 (2011)

  13. 13.

    Barvinok, A.: Integer Points in Polyhedra. European Mathematical Society (EMS), Zürich (2008)

  14. 14.

    Barvinok, A., Pommersheim, J.: An algorithmic theory of lattice points in polyhedra. In: Billera, L.J., Björner, A., Greene, C., Simion, R.E., Stanley, R.P. (eds.) New Perspectives in Algebraic Combinatorics, pp. 91–147. Cambridge Univ. Press, Cambridge (1999)

  15. 15.

    Barvinok A., Woods K.: Short rational generating functions for lattice point problems. J. Amer. Math. Soc. 16, 957–979 (2003)

  16. 16.

    Barvinok A.I.: Computing the volume, counting integral points and exponential sums. Discrete Comput. Geom. 141, 123–141 (1993)

  17. 17.

    Barvinok A.I.: A polynomial time algorithm for counting integral points in polyhedra when the dimension is fixed. Math. Oper. Res. 19(4), 769–779 (1994)

  18. 18.

    Beck M., Braun B., Köeppe M., Savage C., Zafeirakopoulos Z.: s-Lecture hall partitions, self-reciprocal polynomials, and Gorenstein cones. Ramanujan J. 36(1-2), 123–147 (2015)

  19. 19.

    Beck M., Haase C., Matthews A.R: Dedekind-Carlitz polynomials as lattice-point enumerators in rational polyhedra. Math. Ann. 341(4), 945–961 (2008)

  20. 20.

    Beck M., Robins S.: Computing the Continuous Discretely. Springer, New York (2007)

  21. 21.

    Bousquet-Mélou M., Eriksson K.: Lecture hall partitions. Ramanujan J. 1(11), 101–111 (1997)

  22. 22.

    Breuer F., von Heymann F.: Staircases in \({\mathbb{Z}^2}\). Integers 10(6), 807–847 (2010)

  23. 23.

    Breuer, F., Zafeirakopoulos, Z.: Polyhedral Omega software package. (2015)

  24. 24.

    Brion M.: Points entiers dans les polyèdres convexes. Ann. Sci. École Norm. Sup. 21(4), 653–663 (1988)

  25. 25.

    Brion M., Vergne M.: Residue formulae, vector partition functions and lattice points in rational polytopes. J. Amer. Math. Soc. 10(4), 797–833 (1997)

  26. 26.

    Bruns W., Ichim B., Söger C.: The power of pyramid decomposition in Normaliz. J. Symbolic Comput. 74, 513–536 (2016)

  27. 27.

    Cayley A.: On an algebraical operation. Quart. J. Pure Appl. Math. xiii, 369–375 (1875)

  28. 28.

    Chen, Y., Dinwoodie, I., Dobra, A., Huber, M.: Lattice points, contingency tables, and sampling. In: Beck, M., Haase, C., Reznick, B., Vergne, M., Welker, V., Yoshida, R. (eds.) Integer Points in Polyhedra — Geometry, Number Theory, Representation Theory, Algebra, Optimization, Statistics, pp. 65–78. Amer. Math. Soc., Providence, RI (2005)

  29. 29.

    De Loera J.A., Haws D., Hemmecke R., Huggins P., Sturmfels B., Yoshida R.: Short rational functions for toric algebra and applications. J. Symbolic Comput. 38(2), 959–973 (2004)

  30. 30.

    De Loera J.A., Hemmecke R., Köppe M.: Pareto optima of multicriteria integer linear programs. INFORMS J. Comput. 21(1), 39–48 (2009)

  31. 31.

    De Loera, J.A., Hemmecke, R., Köppe, M.: Algebraic and Geometric Ideas in the Theory of Discrete Optimization. SIAM, Philadelphia, PA (2013)

  32. 32.

    De Loera J., Hemmecke R., Köppe M., Weismantel R.: Integer polynomial optimization in fixed dimension. Math. Oper. Res. 31(1), 147–153 (2006)

  33. 33.

    De Loera J.A., Hemmecke R., Tauzer J., Yoshida R.: Effective lattice point counting in rational convex polytopes. J. Symbolic Comput. 38(4), 1273–1302 (2004)

  34. 34.

    De Loera J.A., Rambau J., Santos F.: Triangulations: Structures for Algorithms and Applications. Springer-Verlag, Berlin (2010)

  35. 35.

    Duchon P., Flajolet P., Louchard G., Schaeffer G.: Boltzmann samplers for the random generation of combinatorial structures. Combin. Probab. Comput. 13(4-5), 577–625 (2004)

  36. 36.

    Ehrhart E.: Sur les polyèdres rationnels homothétiques à n dimensions. C. R. Acad. Sci. Paris 254, 616–618 (1962)

  37. 37.

    Ehrhart E.: Sur un problème de géométrie diophantienne linéaire I: polyèdres et réseaux. J. Reine Angew. Math. 226, 1–29 (1967)

  38. 38.

    Ehrhart E.: Sur un problème de géométrie diophantienne linéaire II: systèmes diophantiens linéaires. J. Reine Angew. Math. 227, 25–49 (1967)

  39. 39.

    Elliott E.B.: On linear homogeneous Diophantine equations. Quart. J. Pure Appl. Math. 136, 348–377 (1903)

  40. 40.

    Fukuda, K., Prodon, A.: Double description method revisited. In: Deza, M., Euler, R., Manoussakis, I. (eds.) Combinatorics and Computer Science, pp. 91–111. Springer, Berlin Heidelberg (1996)

  41. 41.

    Fukuda K., Rosta V.: Combinatorial face enumeration in convex polytopes. Comput. Geom. 4(4), 191–198 (1994)

  42. 42.

    Köppe, M., Verdoolaege, S.: Computing parametric rational generating functions with a primal Barvinok algorithm. Electron. J. Combin. 15, #R16 (2008)

  43. 43.

    Lasserre J.B., Zeron E.S.: On counting integral points in a convex rational polytope. Math. Oper. Res. 28, 853–870 (2003)

  44. 44.

    Lasserre J.B., Zeron E.S.: An alternative algorithm for counting lattice points in a convex polytope. Math. Oper. Res. 30(3), 597–614 (2005)

  45. 45.

    Lawrence J.: Valuations and polarity. Discrete Comput. Geom. 3, 307–324 (1988)

  46. 46.

    Lee, C.W.: Subdivisons and triangulations of polytopes. In: Goodman, J.E., O’Rourke, J. (eds.) Handbook of Discrete and Computational Geometry, pp. 383–406. Chapman & Hall/CRC, Boca Raton (2004)

  47. 47.

    Lenstra H.W.: Integer programming with a fixed number of variables. Math. Oper. Res. 8(4), 538–548 (1983)

  48. 48.

    Liu F.: Perturbation of transportation polytopes. J. Combin. Theory Ser. A 120(7), 1539–1561 (2013)

  49. 49.

    MacMahon P.A.: Memoir on the theory of the partitions of numbers. Part II. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. 192, 351–401 (1899)

  50. 50.

    MacMahon, P.A.: Combinatory Analysis. Vol. I, II. Dover Publications Inc., Mineola, NY (2004)

  51. 51.

    McMullen P.: The maximum numbers of faces of a convex polytope. Mathematika 17(2), 179–184 (1970)

  52. 52.

    Pak, I.: On sampling integer points in polyhedra. In: Cucker, F., Rojas, J.M. (eds.) Foundations of Computational Mathematics (Proceedings of SMALEFEST 2000), pp. 319–324. World Sci. Publ., River Edge, NJ (2002)

  53. 53.

    Pfeifle, J., Rambau, J.: Computing triangulations using oriented matroids. In: Joswig, M., Takayama, N. (eds.) Algrebra, Geometry and Software Systems, pp. 49–75. Springer, Berlin (2003)

  54. 54.

    Schrijver, A.: Theory of Linear and Integer Programming. JohnWiley & Sons, Inc. (1986)

  55. 55.

    Schrijver A.: Combinatorial Optimization: Polyhedra and Efficiency. Springer-Verlag, Berlin (2003)

  56. 56.

    Stanley R.P.: Linear homogeneous Diophantine equations and magic labelings of graphs. Duke Math. J. 40, 607–632 (1973)

  57. 57.

    Stein, W.A. et al.: Sage Mathematics Software (Version 6.4.1). Organization = The Sage Development Team, (2014)

  58. 58.

    Storjohann, A.: Near optimal algorithms for computing Smith normal forms of integer matrices. In: Lakshman, Y.N. (ed.) Proceedings of the International Symposium on Symbolic and Algebraic Computation: ISSAC’96, pp. 267–274. ACM Press, New York (1996)

  59. 59.

    Storjohann, A.: Algorithms for matrix canonical forms. PhD Thesis. ETH, Zürich (2000)

  60. 60.

    Varchenko A.N.: Combinatorics and topology of the disposition of affine hyperplanes in real space. Funct. Anal. Appl. 21(1), 9–19 (1987)

  61. 61.

    Verdoolaege, S.: barvinok: User Guide. (2011)

  62. 62.

    Verdoolaege S., Seghir R., Beyls K., Loechner V., Bruynooghe M.: Counting integer points in parametric polytopes using Barvinok’s rational functions. Algorithmica 48(1), 37–66 (2007)

  63. 63.

    Xin, G.: A fast algorithm for MacMahon’s partition analysis. Electron. J. Combin. 11(1), #R58 (2004)

  64. 64.

    Xin, G.: The ring of Malcev-Neumann series and the residue theorem. PhD Thesis. Brandeis University, ProQuest LLC, Ann Arbor, MI (2004)

  65. 65.

    Xin G.: A Euclid style algorithm for MacMahon’s partition analysis. J. Combin. Theory Ser. A 131, 32–60 (2015)

  66. 66.

    Ziegler G.M.: Lectures on Polytopes. Springer-Verlag, New York (1995)

Download references


Open access funding provided by Austrian Science Fund (FWF). The authors are grateful to Matthias Beck and Peter Paule for bringing them together in this research project, as well as for their assistance. Moreover, the authorswould like to thank Matthias Köppe and Fu Liu for helpful discussions.

Author information

Correspondence to Felix Breuer.

Additional information

Felix Breuer was partially supported by German Research Foundation (DFG) grant BR 4251/1-1 and by Austrian Science Fund (FWF) special research group Algorithmic and Enumerative Combinatorics SFB F50-06.

Zafeirakis Zafeirakopoulos was partially supported by the Austrian Science Fund (FWF) special research group Algorithmic and Enumerative Combinatorics SFB F50-06 and W1214-N15 project DK06, the Austrian Marshall Plan Foundation, by the European Union (European Social Fund - ESF) and Greek national funds through the Operational Program "Education and Lifelong Learning" of the National Strategic Reference Framework (NSRF) - Research Funding Program: THALIS - UOA (MIS 375891) and by the project 113F293 under the program 1001 of the Scientific and Technological Research Council of Turkey.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Breuer, F., Zafeirakopoulos, Z. Polyhedral Omega: a New Algorithm for Solving Linear Diophantine Systems. Ann. Comb. 21, 211–280 (2017).

Download citation


  • linear Diophantine system
  • linear inequality system
  • integer solutions
  • partition analysis
  • partition theory
  • polyhedral geometry
  • rational function
  • symbolic cone
  • generating function
  • implementation
  • Omega operator

Mathematics Subject Classification

  • 11Y50
  • 05A17