Discrete & Computational Geometry

, Volume 52, Issue 4, pp 806–823

# On the Theory of Coconvex Bodies

Article

## Abstract

If the complement of a closed convex set in a closed convex cone is bounded, then this complement minus the apex of the cone is called a coconvex set. Coconvex sets appear in singularity theory (they are closely related to Newton diagrams) and in commutative algebra. Such invariants of coconvex sets as volumes, mixed volumes, number of integer points, etc., play an important role. This paper aims at extending various results from the theory of convex bodies to the coconvex setting. These include the Aleksandrov–Fenchel inequality and the Ehrhart duality.

## Keywords

Coconvex bodies Aleksandrov-Fenchel inequalities Volume Valuations on polytopes Virtual convex polytopes

## 1 Introduction

The geometric study of coconvex bodies is motivated by singularity theory. The connections between coconvex geometry and singularity theory are similar to the connections between convex geometry and algebraic geometry. Many local phenomena studied by singularity theory are local manifestations of global algebraic geometry phenomena. Thus it would be natural to expect that many properties of coconvex bodies are manifestations of properties of convex bodies. In this paper, we prove a number of results of this spirit.

In the first subsection of the introduction, we briefly overview the connections of convex geometry with algebraic geometry, of algebraic geometry with singularity theory and, finally, of singularity theory with coconvex geometry. These were the main motivations of the authors, however, neither algebraic geometry, nor singularity theory appear later in the text. Thus the following subsection can be omitted.

### 1.1 Overview

The theory of Newton polytopes founded in 1970s revealed unexpected connections between algebraic geometry and convex geometry. These connections turned out to be useful for both fields. According to a theorem of Kouchnirenko and Bernstein [4, 15], the number of solutions of a polynomial system $$P_1=\cdots =P_d=0$$ in $$(\mathbb {C}{\setminus }\{0\})^d$$ equals $$d!$$ times the mixed volume of the corresponding Newton polytopes $$\Delta _1$$, $$\ldots$$ , $$\Delta _d$$. Recall that, for a complex polynomial $$P=\sum _{\alpha \in \mathbb {Z}^d} c_\alpha z^\alpha$$, the Newton polytope of $$P$$ is the convex hull in $$\mathbb R^d$$ of all points $$\alpha \in \mathbb {Z}^d$$ with $$c_\alpha \ne 0$$. The relationship between Algebra and Geometry contained in the Kouchnirenko–Bernstein theorem allowed to prove the Aleksandrov–Fenchel inequalities using transparent and intuitive algebraic geometry considerations [10, 20], to find previously unknown analogs of the Aleksandrov–Fenchel inequalities in algebraic geometry [8], to find convex-geometric versions of the Hodge–Riemann relations that generalize the polytopal Aleksandrov–Fenchel inequalities [18, 21]. The number of integer points in polytopes is a classical object of study in geometry and combinatorics. The relationship with algebraic geometry has enriched this area with explicit formulas of Riemann–Roch type [12], has allowed to connect the Ehrhart duality with the Serre duality (from topology of algebraic varieties). The integration with respect to the Euler characteristic, inspired by the connections with algebraic geometry, has allowed to find a better viewpoint on classical results of McMullen, simplify and considerably generalize them [11, 12]. Connections with algebraic geometry led to important results in combinatorics of simple (and non-simple) convex polytopes, and had many more follow-ups.

The theory of Newton polytopes has a local version, which studies singularities of sufficiently generic polynomials with given Newton diagrams at the origin. This theory connects singularity theory with somewhat unusual geometric objects, namely, Newton diagrams. A Newton diagram is the union of all compact faces of an unbounded convex polyhedron lying in a convex cone (which in this case coincides with the positive coordinate orthant) and coinciding with the cone sufficiently far from the origin. The complement in the cone of the given unbounded convex polyhedron is, in our terminology, a coconvex body (except that it is also convenient to remove the apex of the cone from the coconvex body for reasons that will become clear later). Computations of local invariants in algebraic geometry and singularity theory have persistently led to volumes and mixed volumes of coconvex bodies, the number of integer points in coconvex bodies, etc. Computation of local invariants is often reduced to computation of global invariants. Let us illustrate this effect on the following toy problem: compute the multiplicity of the zero root of a polynomial $$P(z)=a_kz^k+\cdots +a_nz^n$$ with $$a_k\ne 0$$ and $$a_n\ne 0$$. The multiplicity of the zero root of the polynomial $$P$$ equals the number of nonzero roots of a polynomial $$P_\varepsilon =P+\varepsilon$$ that vanish (i.e., tend to $$0$$) as $$\varepsilon \rightarrow 0$$. The Newton polytopes of the polynomials $$P_\varepsilon$$ ($$\varepsilon \ne 0$$) and $$P$$ are, respectively, the intervals $$[0,n]$$ and $$[k,n]$$. The lengths $$n$$ and $$n-k$$ of these intervals are equal to the number of nonzero roots of the polynomials $$P_\varepsilon$$ and $$P$$ (the answers in global problems). It follows that exactly $$k=n-(n-k)$$ solutions vanish as $$\varepsilon \rightarrow 0$$. Thus, in this simplest case, the multiplicity $$\mu =k$$ of the root 0 (the local invariant) equals the difference of two global invariants, namely, the lengths $$n$$ and $$n-k$$ of the intervals $$[0,n]$$ and $$[k,n]$$.

Similarly to this simple example, many questions of singularity theory (local questions) reduce to questions of algebraic geometry (global questions). Computing various local invariants for generic collections of functions with given Newton diagrams reduces to computing global algebro-geometric invariants for collections of generic polynomials with given Newton polytopes.

A systematic development of the coconvex bodies theory became a pressing need when, several years ago, relationships between convex and coconvex geometry on one side, algebraic geometry and singularity theory on the other side, were found that are far more general than the relationships based on Newton diagrams and Newton polytopes. For example, these relationships have allowed Kaveh and Khovanskii [9] to deduce non-trivial commutative algebra inequalities from a version of the Brunn–Minkowski inequality for coconvex bodies (this version follows from Theorem A). Computing Hilbert polynomials of algebraic varieties and their local versions for algebraic singularities leads to problems of counting integer points in lattice convex and coconvex polytopes.

### 1.2 Terminology and Notation

We start by recalling some terminology from convex geometry, see e.g. [19] for a detailed exposition. The Minkowski sum of two convex sets $$A$$, $$B\subset \mathbb R^d$$ is defined as $$A+B=\{a+b\,|\, a\in A,\ b\in B\}$$. For a positive real number $$\lambda$$, we let $$\lambda A$$ denote the set $$\{\lambda a\,|\, a\in A\}$$. By definition, a convex body is a compact convex set, whose interior is nonempty.

Let $$C\subset \mathbb R^d$$ be a convex cone with the apex at $$0$$ and a nonempty interior. Consider a closed convex subset $$\Delta \subset C$$ such that $$C{\setminus }\Delta$$ is bounded and nonempty. Note that sets $$\Delta$$ with specified properties exist only if $$C$$ is a salient cone, i.e., if $$C$$ contains no affine subspace of dimension 1. Then the set $$A=C{\setminus }(\Delta \cup \{0\})$$ is called a coconvex body. When we talk about volumes, we may replace $$A$$ with its closure $$\overline{A}$$. However, for the discussion of integer points in coconvex bodies, the distinction between $$A$$ and $$\overline{A}$$ becomes important. If $$A$$ and $$B$$ are coconvex sets with respect to the same cone $$C$$, then we can define $$A\oplus B$$ as $$C{\setminus }((\Delta _A+\Delta _B)\cup \{0\})$$, where $$\Delta _A$$ and $$\Delta _B$$ are the unbounded components of $$C{\setminus }A$$ and $$C{\setminus }B$$, respectively. It is clear that any $$C$$-coconvex set can be represented as a set-theoretic difference of two convex bounded sets. This representation allows to carry over a number of results concerning the convex bodies with the operation $$+$$ to the coconvex bodies with the operation $$\oplus$$.

In this paper, we describe several results of this type. Although the reduction from the “convex world” to the “coconvex world” is always simple and sometimes straightforward, the results obtained with the help of it are interesting because, firstly, they are related (through Newton diagrams) with singularity theory [2, 14] and commutative algebra [9], and, secondly, they are intrinsic, i.e., do not depend on a particular representation of a coconvex set as a difference of two convex sets. Coconvex Aleksandrov–Fenchel inequalities also appeared in [7] in the context of Fuchsian groups. This paper extends the earlier very short preprint [13] of the authors, in which just the coconvex Aleksandrov–Fenchel inequality has been discussed.

### 1.3 Aleksandrov–Fenchel Inequalities

A linear family of convex bodies is a collection of the following objects: a real vector space $$\mathcal {V}$$, an open subset $$\Omega \subset \mathcal {V}$$, a map $$f$$ from $$\Omega$$ to the set of all convex bodies in $$\mathbb R^d$$ such that
\begin{aligned} f(\lambda _1 v_1+\cdots +\lambda _n v_n)=\lambda _1 f(v_1)+\cdots +\lambda _n f(v_n) \end{aligned}
whenever all $$v_i\in \Omega$$, all $$\lambda _i$$ are positive, and $$\lambda _1 v_1+\cdots + \lambda _n v_n\in \Omega$$. A linear family of convex bodies with $$m$$ marked points is a linear family $$(\mathcal {V},\Omega ,f)$$ of convex bodies, in which some $$m$$ elements of $$\Omega$$ are marked.
With every linear family $$\alpha =(\mathcal {V},\Omega ,f)$$ of convex bodies, we associate the volume polynomial $$\mathrm {Vol}_\alpha$$ on $$\mathcal {V}$$ as follows. For $$v\in \Omega$$, we define $$\mathrm {Vol}_\alpha (v)$$ as the usual $$d$$-dimensional volume of the convex body $$f(v)$$. It is well known that the function $$\mathrm {Vol}_\alpha$$ thus defined extends to a unique polynomial on $$\mathcal {V}$$ that is homogeneous of degree $$d$$. For $$v\in \mathcal {V}$$, we let $$L_v$$ denote the usual directional (Lie) derivative along $$v$$. Thus $$L_v$$ is a differential operator that acts on functions, in particular, degree $$k$$ polynomials on $$\mathcal {V}$$ are mapped by this operator to degree $$k-1$$ polynomials. If $$\alpha =(\mathcal {V},\Omega ,f)$$ is a linear family of convex bodies with $$(d-2)$$ marked points $$v_1$$, $$\ldots$$ , $$v_{d-2}\in \Omega$$, then we define the Aleksandrov–Fenchel symmetric bilinear form $$B_\alpha$$ on $$\mathcal {V}$$ by the formula
\begin{aligned} B_\alpha (u_1,u_2)=\frac{1}{d!}L_{u_1}L_{u_2}L_{v_1}\cdots L_{v_{d-2}}(\mathrm {Vol}_\alpha ). \end{aligned}
Note that the expression in the right-hand side is a real number (called the mixed volume of the convex bodies $$f(u_1)$$, $$f(u_2)$$, $$f(v_1), \ldots , f(v_{d-2})$$ provided that $$u_1$$, $$u_2\in \Omega$$). Indeed, this is the result of the action of a homogeneous degree $$d$$ differential operator with constant coefficients on a homogeneous degree $$d$$ polynomial. The corresponding quadratic form $$Q_\alpha (u)=B_\alpha (u,u)$$ is given by the formula $$Q_\alpha =\frac{2}{d!}L_{v_1}\cdots L_{v_{d-2}}(\mathrm {Vol}_\alpha )$$. The expression in the right-hand side is the result of the action of a degree $$d-2$$ homogeneous differential operator with constant coefficients on a homogeneous degree $$d$$ polynomial, i.e., a quadratic form. The Aleksandrov–Fenchel inequality [1] states that, for all $$u_1\in \mathcal {V}$$ and $$u_2\in \Omega$$, we have
\begin{aligned} B_\alpha (u_1,u_2)^2\geqslant B_\alpha (u_1,u_1) B_\alpha (u_2,u_2). \end{aligned}
The Aleksandrov–Fenchel inequality is a far-reaching generalization of the classical isoperimetric inequality. See [18, 21] for generalizations of the Aleksandrov–Fenchel inequality for convex polytopes.
The following are standard corollaries of the Aleksandrov–Fenchel inequality:
• Brunn–Minkowski inequality: the function $$\mathrm {Vol}_\alpha ^{\frac{1}{d}}$$ is concave, i.e.
\begin{aligned} (\mathrm {Vol}_\alpha (tu+(1-t)v))^\frac{1}{d}\geqslant t\mathrm {Vol}_\alpha (u)^\frac{1}{d}+(1-t)\mathrm {Vol}_\alpha (v)^\frac{1}{d},\quad t\in [0,1]. \end{aligned}
• Generalized Brunn–Minkowski inequality: the function $$\big (L_{v_1}\ldots L_{v_k}\mathrm {Vol}_\alpha \big )^\frac{1}{d-k}$$ is concave.

• First Minkowski inequality:
\begin{aligned} \big (\frac{1}{d!}L_{u}L_{v}^{d-1}(\mathrm {Vol}_\alpha )\big )^d\geqslant \mathrm {Vol}_\alpha (u)\mathrm {Vol}_\alpha (v)^{d-1}. \end{aligned}
• Second Minkowski inequality: if all marked points coincide with $$u$$, then
\begin{aligned} B_\alpha (u,v)^2\geqslant \mathrm {Vol}_\alpha (u)\, B_\alpha (v,v) \end{aligned}

### 1.4 Coconvex Aleksandrov–Fenchel Inequalities

Define a linear family of $$C$$-coconvex bodies as a collection of the following objects: a vector space $$\mathcal {V}$$, an open subset $$\Omega \subset \mathcal {V}$$, a map $$g$$ from $$\Omega$$ to the set of all $$C$$-coconvex bodies such that
\begin{aligned} g(\lambda _1 v_1+\cdots +\lambda _n v_n)=\lambda _1 g(v_1)\oplus \cdots \oplus \lambda _n g(v_n) \end{aligned}
whenever all $$v_i\in \Omega$$, all $$\lambda _i$$ are positive, and $$\lambda _1 v_1+\cdots +\lambda _n v_n\in \Omega$$. A linear family of $$C$$-coconvex bodies with $$m$$ marked points is a linear family $$(\mathcal {V},\Omega ,g)$$ of $$C$$-coconvex bodies, in which some $$m$$ elements of $$\Omega$$ are marked. With every linear family $$\beta$$ of $$C$$-coconvex bodies, we associate the volume function $$\mathrm {Vol}_\beta$$ in the same way as with a linear family of convex bodies. The function $$\mathrm {Vol}_\beta$$ thus defined is also a homogeneous degree $$d$$ polynomial (we will prove this below). Given a linear family $$\beta$$ of $$C$$-coconvex bodies with $$d-2$$ marked points $$v_1$$, $$\ldots$$ , $$v_{d-2}$$, we define the coconvex Aleksandrov–Fenchel symmetric bilinear form as
\begin{aligned} B_\beta ^C(u_1,u_2)=\frac{1}{d!}L_{u_1}L_{u_2}L_{v_1}\cdots L_{v_{d-2}}(\mathrm {Vol}_\beta ). \end{aligned}
We will also consider the corresponding quadratic form $$Q^C_\beta =\frac{2}{d!}L_{v_1}\cdots L_{v_{d-2}}(\mathrm {Vol}_\beta )$$. The following is one of our main results.

### Theorem A

The form $$Q_\beta ^C$$ is non-negative, i.e., $$Q_\beta ^C(u)\geqslant 0$$ for all $$u\in \mathcal {V}$$. In particular, the corresponding symmetric bilinear form satisfies the Cauchy–Schwarz inequality
\begin{aligned} B_\beta ^C(u_1,u_2)^2\leqslant B_\beta ^C(u_1,u_1) B_\beta ^C(u_2,u_2). \end{aligned}

The inequality stated in Theorem A is called the coconvex Aleksandrov–Fenchel inequality. In recent paper [7], Theorem A is proved under the assumption that $$C$$ is a fundamental cone of some discrete group $$\Gamma$$ acting by linear isometries of a pseudo-Euclidean metric, and $$C{\setminus }(g(v)\cup \{0\})$$ is the intersection of some convex $$\Gamma$$-invariant set with $$C$$, for every $$v\in \Omega$$. Theorem A is motivated by an Aleksandrov–Fenchel type inequality for (mixed) intersection multiplicities of ideals [9].

The following inequalities can be derived from Theorem A in the same way as similar inequalities for convex bodies follow from the classical Alexandrov–Fenchel inequality (cf. [7]):
• Reversed Brunn–Minkowski inequality: the function $$\mathrm {Vol}_\beta ^{\frac{1}{d}}$$ is convex, i.e.
\begin{aligned} (\mathrm {Vol}_\beta (tu+(1-t)v))^\frac{1}{d}\leqslant t\,\mathrm {Vol}_\beta (u)^\frac{1}{d}+(1-t)\mathrm {Vol}_\beta (v)^\frac{1}{d},\quad t\in [0,1]. \end{aligned}
• Generalized reversed Brunn–Minkowski inequality: the function $$\displaystyle \big (L_{v_1}\cdots L_{v_k}\mathrm {Vol}_\beta \big )^\frac{1}{d-k}$$ is convex.

• First reversed Minkowski inequality:
\begin{aligned} \big (\frac{1}{d!}L_{u}L_{v}^{d-1}(\mathrm {Vol}_\beta )\big )^d\leqslant \mathrm {Vol}_\beta (u)\mathrm {Vol}_\beta (v)^{d-1}. \end{aligned}
• Second reversed Minkowski inequality: if all marked points coincide with $$u$$, then
\begin{aligned} B_\beta ^C(u,v)^2\leqslant \mathrm {Vol}_\beta (u)\, B_\beta ^C(v,v) \end{aligned}

### 1.5 Coconvex Polytopes as Virtual Convex Polytopes

The set of convex polytopes is closed under Minkowski addition but not closed under “Minkowski subtraction”. Virtual convex polytopes are geometric objects introduced in [11] that can be identified with formal Minkowski differences of convex polytopes. We will now briefly recall the notion of a virtual convex polytope.

Consider the smallest ring $$\mathcal {R}(\mathbb R^d)$$ of sets containing all closed half-planes in $$\mathbb R^d$$. Clearly, all convex polytopes belong to $$\mathcal {R}(\mathbb R^d)$$. There is a unique finitely additive measure $$\chi$$ on $$\mathcal {R}(\mathbb R^d)$$ such that, for every closed bounded set $$A\in \mathcal {R}(\mathbb R^d)$$, the number $$\chi (A)$$ is equal to the Euler characteristic of $$A$$. Let $$Z(\mathbb R^d)$$ be the $$\mathbb {Z}$$-algebra of all measurable functions with respect to $$\mathcal {R}(\mathbb R^d)$$ with values in $$\mathbb {Z}$$, and let $$Z_c(\mathbb R^d)$$ be its subalgebra consisting of functions with compact support. With every element $$A\in \mathcal {R}(\mathbb R^d)$$, we associate its indicator function $$\mathbb {I}_A$$ that is equal to 1 on $$A$$ and to 0 elsewhere. The additive group of $$Z_c(\mathbb R^d)$$ is spanned by the indicator functions of convex polytopes. If $$\alpha$$, $$\beta \in Z_c(\mathbb R^d)$$, we define the Minkowski product $$\alpha *\beta$$ as the convolution
\begin{aligned} \alpha *\beta (x)=\int \alpha (y)\beta (x-y) d\chi (y). \end{aligned}
(The integration is performed over $$\mathbb R^d$$ with respect to the measure $$\chi$$.) We recall one simple lemma from [11]:

### Lemma 1.1

Suppose that $$A$$ and $$B$$ are convex polytopes in $$\mathbb R^d$$. Then $$\mathbb {I}_{A+B}=\mathbb {I}_A*\mathbb {I}_B$$.

Thus the Minkowski sum of convex polytopes corresponds to the Minkowski product in $$Z_c(\mathbb R^d)$$.

### Proof

Consider the function
\begin{aligned} \varphi (x)=\int \mathbb {I}_A(y)\mathbb {I}_B(x-y) d\chi (y). \end{aligned}
If $$x\in A+B$$, then the set $$F(y)$$ of all $$y\in A$$ such that $$x-y\in B$$ is a nonempty convex polytope (clearly, both conditions $$y\in A$$ and $$x-y\in B$$ define systems of linear inequalities on $$y$$). Therefore, $$\varphi (x)=1$$. If $$x\not \in A+B$$, then the set $$F(y)$$ is empty; therefore, $$\varphi (x)=0$$. We see that $$\varphi$$ coincides with $$\mathbb {I}_{A+B}$$. $$\square$$

It follows from Lemma 1.1 that $$\mathbb {I}_{\{0\}}$$ is the identity element of the ring $$Z_c(\mathbb R^d)$$. It is proved in [11] that, for every convex polytope $$A$$ in $$\mathbb R^d$$, the indicator function $$\mathbb {I}_A$$ is an invertible element of the ring $$Z_c(\mathbb R^d)$$, i.e., there exists an element $$\varphi \in Z_c(\mathbb R^d)$$ with the property $$\varphi *\mathbb {I}_A=\mathbb {I}_{\{0\}}$$ (we write $$\varphi =\mathbb {I}_A^{-1}$$). The function $$\varphi :\mathbb R^d\rightarrow \mathbb {Z}$$ admits a simple explicit description: it is equal to $$(-1)^{\dim (A)}$$ on the relative interior of the set $$\{-x\,|\, x\in A\}$$ and to 0 elsewhere. Virtual (convex) polytopes are defined as elements of $$Z_c(\mathbb R^d)$$ of the form $$\mathbb {I}_A*\mathbb {I}_B^{-1}$$, where $$A$$ and $$B$$ are convex polytopes. If we identify convex polytopes with their indicator functions, then virtual polytopes are identified with formal Minkowski differences of convex polytopes. Virtual polytopes form a commutative group under Minkowski multiplication. Note that we do not deal with more general “virtual convex bodies” than virtual polytopes; the corresponding theory is more involved and is still developing, see e.g. [16].

Fix a linear function $$\xi :\mathbb R^d\rightarrow \mathbb R$$ such that $$\xi \geqslant 0$$ on $$C$$ and $$\xi ^{-1}(0)\cap C=\{0\}$$; for every subset $$X\subset C$$, we set $$X_t=X\cap \{\xi \leqslant t\}$$. The following theorem is a general principle that allows to reduce various facts about coconvex polytopes to the corresponding facts about convex polytopes.

### Theorem B

Let $$C\subset \mathbb R^d$$ be a salient convex polyhedral cone with the apex at $$0$$ and a nonempty interior. Suppose that $$\Delta \subset C$$ is a convex unbounded polyhedron different from $$C$$ such that $$C{\setminus }\Delta$$ is bounded, and $$A=C{\setminus }(\Delta \cup \{0\})$$ is the corresponding $$C$$-coconvex polytope.
1. (1)
The function $$-\mathbb {I}_A$$ is a virtual polytope. Moreover, we have
\begin{aligned} -\mathbb {I}_A=\mathbb {I}_{\Delta _t}*\mathbb {I}_{C_t}^{-1} \end{aligned}
for all sufficiently large $$t\in \mathbb R$$.

2. (2)
If $$A$$ and $$B$$ are $$C$$-coconvex polytopes, then
\begin{aligned} -\mathbb {I}_{A\oplus B}=(-\mathbb {I}_A)*(-\mathbb {I}_B). \end{aligned}

Theorem B explains our definition of a coconvex body, in particular, the choice of the boundary points that need to be included into it. Theorem A can be deduced from Theorem B and the convex Aleksandrov–Fenchel inequalities. There are many other consequences of Theorem B that deal with $$C$$-coconvex integer polytopes. Some of them are stated below.

Define a convex integer polytope as a convex polytope with integer vertices. Let $$\mathcal {R}_c(\mathbb {Z}^d)$$ be the minimal subring of the ring of sets $$\mathcal {R}(\mathbb R^d)$$ containing all convex integer polytopes. Similarly, let $$Z_c(\mathbb {Z}^d)$$ be the minimal subalgebra of $$Z_c(\mathbb R^d)$$ containing the indicator functions of all convex integer polytopes. Clearly, any function in $$Z_c(\mathbb {Z}^d)$$ is measurable with respect to $$\mathcal {R}_c(\mathbb {Z}^d)$$. A valuation on integer polytopes is by definition a finitely additive measure on $$\mathcal {R}_c(\mathbb {Z}^d)$$. For a valuation $$\mu$$ on integer polytopes and an element $$\varphi$$ of the ring $$Z_c(\mathbb R^d)$$, we define $$\mu (\varphi )$$ as the integral of the function $$\varphi$$ with respect to the measure $$\mu$$. Note that $$\mu (A)$$, the measure $$\mu$$ evaluated at a measurable set $$A$$, is the same as $$\mu (\mathbb {I}_A)$$. Similarly to valuations on integer polytopes, we define more general valuations on polytopes as finitely additive measures on $$\mathcal {R}_c(\mathbb R^d)$$.

Recall that a function $$P$$ on a commutative multiplicative group $$G$$ is said to be polynomial of degree $$\leqslant d$$ if, for every fixed $$g\in G$$, the function $$P(gx)-P(x)$$ is polynomial of degree $$\leqslant d-1$$. Polynomial functions of degree 0 are by definition constant functions. Define the group of virtual integer polytopes as the subgroup of the group of virtual polytopes generated by the indicator functions of all convex integer polytopes. Recall the following theorem of [11]: If a valuation $$\mu$$ on integer polytopes is polynomial of degree $$\leqslant k$$, i.e., for every convex integer polytope $$A$$, the function $$x\mapsto \mu (A+x)$$ is a polynomial on $$\mathbb {Z}^d$$ of degree at most $$k$$, then the function $$\varphi \mapsto \mu (\varphi )$$ is a polynomial function of degree $$\leqslant d+k$$ on the group of virtual integer polytopes with $$*$$ as the group operation. An important example of a valuation on integer polytopes is the valuation $$\mu$$ that assigns the number of integer points in $$X$$ to every $$X\in \mathcal {R}_c(\mathbb {Z}^d)$$. This valuation can be evaluated on all virtual integer polytopes. In particular, for every integer convex polytope $$A$$, the number $$\mu (\mathbb {I}_A^{*n})$$ depends polynomially on $$n$$ (cf. [11, 17]). This polynomial function is called the Ehrhart polynomial of $$A$$.

Let $$C$$, $$\Delta$$ and $$A$$ be as in Theorem B. Suppose that $$C$$ is an integer polyhedral cone, i.e., there exist elements $$v_1$$, $$\ldots$$ , $$v_n\in \mathbb {Z}^d$$ such that
\begin{aligned} C=\{t_1v_1+\cdots +t_nv_n\,|\, t_1,\ldots ,t_n\geqslant 0\}. \end{aligned}
Suppose also that all vertices of $$\Delta$$ belong to $$\mathbb {Z}^d$$ (in this case, we will say that $$A$$ is a $$C$$-coconvex integer polytope). The following statements are corollaries of Theorem B.

### Corollary 1.2

Let $$\mu$$ be a polynomial valuation of degree $$\leqslant k$$ on integer convex polytopes, and let $$A_1$$, $$\ldots$$ , $$A_n$$ be $$C$$-coconvex integer polytopes. The number
\begin{aligned} E(m_1,\ldots ,m_n)=\mu (m_1A_1\oplus m_2A_2\oplus \cdots \oplus m_nA_n) \end{aligned}
is defined for any choice of positive integers $$m_1$$, $$\ldots$$ , $$m_n$$, and depends polynomially $$($$of degree $$\leqslant k+d)$$ on $$m_1$$, $$\ldots$$ , $$m_n$$.

We will use the following notation. For a $$C$$-coconvex body $$A$$, we write $$\mathrm {Int}(A)$$ for the interior of $$A$$. We set $$\overline{A}$$ to be the closure of $$A$$, and define $$A^\bullet$$ as $$\overline{A}\cap \mathrm {Int}(C)$$.

### Corollary 1.3

The polynomial function $$E$$ defined in Corollary 1.2 can be evaluated at $$m_1=\cdots =m_n=-1$$, and we have
\begin{aligned} E(-1,\ldots ,-1)=(-1)^d\mu (-A_1^\bullet -\cdots -A_n^\bullet )-\mu (\{0\}). \end{aligned}

### 1.6 Generating Functions for Integer Points

Let us now consider the following valuation $$G$$ on integer convex polytopes with values in rational functions of some formal variables $$x_1$$, $$\ldots$$ , $$x_d$$:
\begin{aligned} G(X)=\sum _{a\in X\cap \mathbb {Z}^d} x^a. \end{aligned}
Here $$X$$ is any element of $$\mathcal {R}_c(\mathbb {Z}^d)$$, and $$x^a$$ is the monomial $$x_1^{a_1},\ldots ,x_d^{a_d}$$. The rational function $$G(X)$$ is called the generating function for the integer points in $$X$$. The following theorem is due to Brion [5], see also [3, 12]: the valuation $$G$$ extends to the minimal ring of sets $$\mathcal {R}(\mathbb {Z}^d)$$ containing all integer cones and their parallel translations by integer vectors; moreover, for any integer convex polytope $$A$$, we have
\begin{aligned} G(A)=\sum _{a\in \mathrm {Vert}(A)} x^a G(C_a), \end{aligned}
where $$\mathrm {Vert}(A)$$ is the set of all vertices of $$A$$, and $$C_a$$ is the cone spanned by $$A-a$$, the translate of $$A$$ by the vector $$-a$$.

Note that $$G(C_a)$$ can be computed explicitly, by subdividing $$C_a$$ into cones, each of which is spanned by a basis of $$\mathbb {Z}^d$$. If a cone $$C$$ is spanned by a basis of $$\mathbb {Z}^d$$, then the computation of $$G(C)$$ reduces to the summation of a geometric series. In this paper, we will prove the following theorem that generalizes Brion’s theorem to coconvex polytopes.

### Theorem C

Let $$C$$ be an integer polyhedral cone, and let $$A$$ be a $$C$$-coconvex integer polytope. Then
\begin{aligned} G(A)=-\sum _{a\in \mathrm {Vert}(A){\setminus }\{0\}} x^a G(C'_a)+G(C)-1, \end{aligned}
where $$C'_a$$ is the cone with the apex at $$0$$ such that a small neighborhood of $$0$$ in $$C'_a$$ coincides with a small neighborhood of $$0$$ in $$\overline{C{\setminus } A}-a$$.

Observe that usual exponential sums over the integer points of $$A$$ are obtained from the rational function $$G(A)$$ by substituting the exponentials $$e^{p_1}$$, $$\ldots$$ , $$e^{p_d}$$ for the variables $$x_1$$, $$\ldots$$ , $$x_d$$. Sums of quasi-polynomials (in particular, sums of polynomials) can be obtained from exponential sums by differentiation with respect to parameters $$p_1$$, $$\ldots$$ , $$p_d$$. A similar theory exists for integrals of exponentials, quasi-polynomials, etc., over convex or coconvex polytopes.

### Remark

In order to compute the number of integer points in $$A$$, one is tempted to substitute $$x_i=1$$ into the expression for the rational function $$G(A)$$ through the generating functions of cones. However, this is problematic as the denominator of this expression vanishes at the point $$(1,1,\ldots ,1)$$. To obtain a numeric value, one can, e.g., choose a generic line passing through the point $$(1,1,\ldots ,1)$$, consider the Laurent series expansion of $$G(A)$$ along this line, and then take the free coefficient of this Laurent series. The same procedure is applicable to computing a quasi-polynomial sum over $$A$$ for exceptional values of $$p=(p_1,\ldots ,p_d)$$, for which the rational function of $$e^{p_1}$$, $$\ldots$$ , $$e^{p_d}$$, equal to this quasi-polynomial sum at generic points, has a pole.

## 2 Proof of Theorem A

Recall that every quadratic form $$Q$$ on a finite dimensional real vector space $$\mathcal {V}$$ can be represented in the form
\begin{aligned} x_1^2+\cdots +x_k^2-x_{k+1}^2-\cdots -x_{k+\ell }^2 \end{aligned}
for a suitable linear coordinate system $$(x_1,\ldots ,x_m)$$, $$m\geqslant k+\ell$$. The pair $$(k,\ell )$$ is called the signature of $$Q$$. It is well known (see, e.g., [6]) that $$Q$$ has signature $$(1,\ell )$$ for some $$\ell$$ if and only if both of the following conditions are fulfilled:
1. (1)

there exists a vector $$v_0\in \mathcal {V}$$ with $$Q(v_0)>0$$;

2. (2)

the corresponding symmetric bilinear form $$B$$ (such that $$B(u,u)=Q(u)$$) satisfies the reversed Cauchy–Schwarz inequality: $$B(u,v)^2\geqslant Q(u)Q(v)$$ for all $$u\in \mathcal {V}$$ and $$v\in \mathcal {V}$$ such that $$Q(v)>0$$.

For the convenience of the reader, we provide a proof of this statement. Suppose that $$Q$$ has signature $$(1,\ell )$$. Then $$Q=x_0^2-x_1^2-\cdots -x_\ell ^2$$ with respect to some linear system of coordinates $$x_0$$, $$\ldots$$ , $$x_\ell$$, $$\ldots$$ (note that the dimension of $$\mathcal {V}$$ may be bigger than $$1+\ell$$). It suffices to take any nonzero vector along the $$x_0$$-axis as $$v_0$$. If $$Q(v)>0$$ and $$u\in \mathcal {V}$$ is not proportional to $$v$$, then the quadratic polynomial $$\lambda \mapsto Q(v+\lambda u)$$ cannot be everywhere positive as there is a vector of the form $$v+\lambda u$$ orthogonal to $$v_0$$, and $$Q(v+\lambda u)\leqslant 0$$ for such a vector. The discriminant of the polynomial $$\lambda \mapsto Q(v+\lambda u)$$ being nonnegative translates into the reversed Cauchy–Schwarz inequality. Conversely, the reversed Cauchy–Schwarz inequality implies that no restriction of $$Q$$ on a two-dimensional subspace is positive definite. Therefore, the signature of $$Q$$ is $$(k,\ell )$$ with $$k\leqslant 1$$. On the other hand, the existence of a vector $$v_0\in \mathcal {V}$$ such that $$Q(v_0)>0$$ implies $$k\geqslant 1$$.

Thus, the Aleksandrov–Fenchel inequality is equivalent to the fact that $$Q_\alpha$$ has signature $$(1,\ell (\alpha ))$$ for every finite-dimensional linear family $$\alpha$$ of convex $$d$$-dimensional bodies with $$d-2$$ marked points.

### 2.1 An Illustration in the Case $$d=2$$

To illustrate Theorem A in a simple case, we set $$d=2$$.

Introduce an affine coordinate system $$(x,y)$$ in $$\mathbb R^2$$. We may assume that $$C$$ lies in the upper half-plane $$y\geqslant 0$$, and that one of the two boundary rays of $$C$$ coincides with the positive $$x$$-semiaxis. Suppose that a $$C$$-coconvex body $$A$$ is a polygon with edges $$E_1(A)$$, $$\ldots$$ , $$E_n(A)$$, ordered so that adjacent edges have adjacent indices, and so that $$E_1(A)$$ touches the $$x$$-axis. Let $$\theta _k$$ denote the angle between the line containing the edge $$E_k(A)$$ and the $$x$$-axis, as indicated in Fig. 1. Here and in the sequel, $$k$$ runs through 1, $$\ldots$$ , $$n$$. Fix a Euclidean metric on $$\mathbb R^2$$. We will write $$h_k(A)$$ for the distance from the origin to the line containing the edge $$E_k(A)$$.
If we now have two co-convex polygons $$A$$ and $$B$$ with the same number of edges and with the same angles $$\theta _k$$, then we have
\begin{aligned} h_k(A\oplus B)=h_k(A)+h_k(B). \end{aligned}
Thus, if the angles $$\theta _k$$ are fixed, co-convex polygons can be identified with vectors $$(h_1(A),\ldots ,h_n(A))$$, and the operation $$\oplus$$ on $$C$$-coconvex polygons corresponds to the usual vector addition. The numbers $$h_k(A)$$ are called support numbers of $$A$$. We can now consider the following linear family $$\beta =(\mathcal {V},\Omega ,g)$$ of $$C$$-coconvex polygons. The space $$\mathcal {V}$$ is $$\mathbb R^n$$ (the dimension of the space being equal to the number of support numbers), $$\Omega$$ is a small neighborhood of $$(h_1(A),\ldots ,h_n(A))$$, and $$g(h_1,\ldots ,h_n)$$ is the $$C$$-coconvex polygon with support numbers $$h_1$$, $$\ldots$$ , $$h_n$$.
A $$C$$-coconvex polygon $$g(v)$$ can be naturally represented as a union of triangles $$T_1(v)$$, $$\ldots$$ , $$T_n(v)$$, as shown in Fig. 1. The angles of these triangles depend only on $$\theta _1$$, $$\ldots$$ , $$\theta _n$$. Namely, $$T_k(v)$$ has angles $$\theta _k$$, $$\pi -\theta _{k+1}$$, $$\theta _{k+1}-\theta _k$$. It follows that $$T_k(v)$$ and $$T_k(w)$$ are similar. We conclude that the area of $$T_k(v)$$ is the square of some linear functional $$\varphi _k$$ on $$\mathcal {V}$$. Indeed, if the angles of a triangle are fixed, then its area is equal, up to a constant positive factor depending only on the angles, to the square of the length of any chosen edge. The length $$\lambda _k$$ of the horizontal edge of $$T_k(v)$$ can be expressed as a linear functional on $$\mathcal {V}$$:
\begin{aligned} \lambda _k=\frac{h_k}{\sin \theta _k}-\frac{h_{k+1}}{\sin \theta _{k+1}} \end{aligned}
(if $$k=n$$, then the second term in the right-hand side should be omitted).
We can now write the following formula for the area of $$f(v)$$:
\begin{aligned} \mathrm {Vol}_\beta (v)=\varphi _1(v)^2+\cdots +\varphi _n(v)^2. \end{aligned}
It is obvious from this representation that $$\mathrm {Vol}_\beta (v)$$ is a positive definite quadratic form on $$\mathcal {V}$$. The Cauchy–Schwarz inequality written for this quadratic form yields the inequality
\begin{aligned} B^C_\beta (u,v)^2\leqslant \mathrm {Vol}_\beta (u)\mathrm {Vol}_\beta (v). \end{aligned}
This is the reversed Aleksandrov–Fenchel inequality for $$d=2$$ (for a special choice of a linear family $$\beta$$, but in fact the general case easily reduces to this special case). The argument presented above imitates some classical proofs of the isoperimetric inequality.

### 2.2 Reduction of Theorem A to the Aleksandrov–Fenchel Inequality

In this subsection, we prove Theorem A. The proof is a reduction to the convex Aleksandrov–Fenchel inequality. Fix a salient convex closed cone $$C$$ with the apex at $$0$$ and a linear family $$\beta =(\mathcal {V},\Omega ,g)$$ of $$C$$-coconvex bodies. We may assume that $$g(\Omega )$$ is bounded in the sense that there is a large ball in $$\mathbb R^d$$ that contains all coconvex bodies $$g(v)$$, $$v\in \Omega$$. Since $$g(\Omega )$$ is bounded, there exists a real number $$t_0>0$$ such that $$g(v)=g(v)_{t_0}$$ for all $$v\in \Omega$$.

Choose any $$t_1>t_0$$. We will now define a linear family $$\alpha =(\mathcal {V}\times \mathbb R,\Omega \times (t_0,t_1),f)$$ of convex bodies as follows. For $$v\in \Omega$$ and $$t\in (t_0,t_1)$$, we set $$f(v,t)$$ to be the convex body $$C_t{\setminus }(g(v)\cup \{0\})$$. The proof of the coconvex Aleksandrov–Fenchel inequality is based on the comparison between the linear families $$\alpha$$ and $$\beta$$.

We have the following relation between the polynomials $$\mathrm {Vol}_\alpha$$ and $$\mathrm {Vol}_\beta$$:
\begin{aligned} \qquad \qquad \!\mathrm {Vol}_\alpha (v,t)=\mathrm {Vol}(C_t)-\mathrm {Vol}_\beta (v), \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (V) \end{aligned}
which is clear from the additivity of the volume. The first term in the right-hand side has the form $$ct^d$$, where $$c$$ is some positive constant. The second term in the right-hand side does not depend on $$t$$. It follows from $$(V)$$ that $$\mathrm {Vol}_\beta$$ is a homogeneous degree $$d$$ polynomial.
Let us mark some points $$(v_1,s_1)$$, $$\ldots$$ , $$(v_{d-2},s_{d-2})$$ in $$\Omega \times (t_0,t_1)$$. Apply the differential operator $$\frac{2}{d!}L_{(v_1,s_1)}\ldots L_{(v_{d-2},s_{d-2})}$$ to both sides of $$(V)$$. We obtain that
\begin{aligned} \qquad \qquad \qquad \,Q_\alpha (v,t)=c't^2-Q^C_\beta (v),\qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (Q) \end{aligned}
where $$c'$$ is some positive constant (equal to $$c s_1\cdots s_{d-2}$$). In the right-hand side of $$(Q)$$, we have the difference of two quadratic forms, moreover, these two forms depend on disjoint sets of variables.

If $$q_1$$, $$q_2$$ are quadratic forms depending on disjoint sets of variables, and $$(k_1,\ell _1)$$, $$(k_2,\ell _2)$$, respectively, are signatures of these forms, then $$q_1+q_2$$ is a quadratic form of signature $$(k_1+k_2,\ell _1+\ell _2)$$. We now apply this observation to identity $$(Q)$$. The first term of the right-hand side, $$c't^2$$, has signature $$(1,0)$$. The signature of the left-hand side is equal to $$(1,\ell )$$ for some $$\ell \geqslant 0$$, by the classical Alexandrov–Fenchel inequality. It follows that the signature of $$Q^C_{\beta }$$ is $$(\ell ,0)$$, i.e. the form $$Q^C_{\beta }$$ is non-negative.

## 3 Proof of Theorem B and its Corollaries

In this section, we prove Theorem B and derive a number of corollaries from it.

### 3.1 Proof of Theorem B

Consider a salient closed convex cone $$C\subset \mathbb R^d$$ with the apex at $$0$$ and a nonempty interior. A closed convex subset $$\Delta \subset C$$ is said to be $$C$$-convex if $$\Delta +C=\Delta$$. The following lemma gives the most important example of $$C$$-convex sets.

### Lemma 3.1

If $$\Delta \subset C$$ is a convex subset such that $$C{\setminus }\Delta$$ is bounded, then $$\Delta$$ is $$C$$-convex.

### Proof

Since $$0\in C$$, we have $$\Delta +C\supset \Delta$$. It remains to prove that $$\Delta +C\subset \Delta$$. Assume the contrary: there are points $$x\in \Delta$$ and $$y\in C$$ such that $$x+y\not \in \Delta$$. Consider the line $$L$$ passing through the points $$x$$ and $$x+y$$. Since $$C{\setminus }\Delta$$ is bounded, there are points of $$\Delta$$ in $$L$$ far enough in the direction from $$x$$ to $$x+y$$. Thus $$x+y$$ separates two points of $$L\cap \Delta$$ in $$L$$. A contradiction with the convexity of $$\Delta$$. $$\square$$

### Lemma 3.2

Let $$\Delta \subset C$$ be a convex subset such that $$C{\setminus }\Delta$$ is bounded. For all sufficiently large $$t>0$$ and all $$s>0$$, we have $$\Delta _t+C_s=\Delta _{t+s}$$.

### Proof

If $$x\in \Delta _t$$ and $$y\in C_s$$, then $$x+y\in \Delta$$ by $$C$$-convexity (Lemma 3.1) and $$\xi (x+y)\leqslant t+s$$ since $$\xi (x)\leqslant t$$ and $$\xi (y)\leqslant s$$. On the other hand, take $$z\in \Delta _{t+s}$$ and consider two cases.

Case 1: we have $$\xi (z)\leqslant t$$. Then $$z\in \Delta _t$$, and, setting $$x=z$$, $$y=0$$, we obtain that $$z=x+y$$, $$x\in \Delta _t$$ and $$y\in C_s$$.

Case 2: we have $$\xi (z)>t$$. If $$\lambda =t/\xi (z)$$, then $$\xi (\lambda z)=t$$. We now set $$x=\lambda z$$, $$y=(1-\lambda )z$$. The number $$t$$ is sufficiently large; thus we may assume that all values of $$\xi$$ on $$C{\setminus }\Delta$$ are less than $$t$$. Then $$x\in \Delta _t$$ and $$y\in C_s$$. $$\square$$

### Proposition 3.3

Under the assumptions of Theorem B and for all sufficiently large $$t$$, we have
\begin{aligned} \mathbb {I}_{\Delta _t}*\mathbb {I}_{C_t}^{-1}=\mathbb {I}_{\Delta _t}-\mathbb {I}_{C_t}+\mathbb {I}_{\{0\}}. \end{aligned}

### Proof

It suffices to compute $$\mathbb {I}_{C_t}*(\mathbb {I}_{\Delta _t}-\mathbb {I}_{C_t}+\mathbb {I}_{\{0\}})$$ and verify that it is equal to $$\mathbb {I}_{\Delta _t}$$. Opening the parentheses in the former expression and using Lemma 3.2, we obtain
\begin{aligned} \mathbb {I}_{C_t+\Delta _t}-\mathbb {I}_{C_t+C_t}+\mathbb {I}_{C_t}=\mathbb {I}_{\Delta _{2t}}-\mathbb {I}_{C_{2t}}+\mathbb {I}_{C_t}. \end{aligned}
Since $$t$$ is sufficiently large, we have $$\mathbb {I}_{\Delta _{2t}}-\mathbb {I}_{C_{2t}}=\mathbb {I}_{\Delta _t}-\mathbb {I}_{C_t}$$ (both sides are equal to the minus indicator function of $$C{\setminus }\Delta$$), and we obtain the desired. $$\square$$
Let $$\Delta$$ be a closed convex subset of $$C$$ such that $$C{\setminus }\Delta$$ is bounded. Consider the corresponding $$C$$-coconvex body $$A=C{\setminus }(\Delta \cup \{0\})$$. Then, for all sufficiently large $$t$$, we have
\begin{aligned} -\mathbb {I}_{A}=\mathbb {I}_{\Delta _t}-\mathbb {I}_{C_t}+\mathbb {I}_{\{0\}}, \end{aligned}
in particular, the right-hand side is independent of $$t$$, provided that $$t$$ is large enough. It follows from Proposition 3.3 that $$-\mathbb {I}_A$$ is a virtual convex polytope $$\mathbb {I}_{\Delta _t}*\mathbb {I}_{C_t}^{-1}$$. Moreover, the same proposition implies that $$-\mathbb {I}_{(A\oplus B)}=(-\mathbb {I}_{A})*(-\mathbb {I}_{B})$$. Thus $$A\mapsto -\mathbb {I}_{A}$$ is a homomorphism from the semi-group of $$C$$-coconvex bodies to the multiplicative group of virtual polytopes. This completes the proof of Theorem B.

### 3.2 Proof of Corollaries 1.2 and 1.3

Corollary 1.2 follows directly from Theorem B and the theorem of [11] that a polynomial valuation on integer convex polytopes defines a polynomial function on the group of virtual integer polytopes. Corollary 1.3 follows from Theorem B and from Proposition 3.4 stated below.

### Proposition 3.4

The inverse of the virtual polytope $$-\mathbb {I}_{A}$$ in the multiplicative group of virtual polytopes is equal to
\begin{aligned} (-1)^{d-1}\mathbb {I}_{A^\bullet }\circ \sigma +\mathbb {I}_{\{0\}}. \end{aligned}
Here $$\sigma$$ is the antipodal map taking $$v$$ to $$-v$$.

### Proof

For a subset $$X\subset \mathbb R^d$$, we let $$\mathrm {Int}(X)$$ denote the interior of $$X$$. According to [11], the inverse of a virtual convex polytope $$\varphi$$ is equal to $$\star (\varphi )\circ \sigma$$, where the additive group homomorphism $$\star :Z_c(\mathbb {Z}^d)\rightarrow Z_c(\mathbb {Z}^d)$$ is defined uniquely by the property that $$\star (\mathbb {I}_B)=(-1)^d\mathbb {I}_{\mathrm {Int}(B)}$$ for every integer convex polytope $$B$$ with nonempty interior. Setting $$\varphi =-\mathbb {I}_{A}$$, we obtain that
\begin{aligned} \star (\varphi )=\star (\mathbb {I}_{\Delta _t}-\mathbb {I}_{C_t}+\mathbb {I}_{\{0\}})= (-1)^{d}(\mathbb {I}_{\mathrm {Int}(\Delta ^t)}-\mathbb {I}_{\mathrm {Int}(C_t)})+\mathbb {I}_{\{0\}}= (-1)^{d-1}\mathbb {I}_{A^\bullet }+\mathbb {I}_{\{0\}}. \end{aligned}
The desired claim follows. $$\square$$

### 3.3 Proof of Theorem C

Let $$C$$, $$A$$, $$\Delta$$ and $$\xi$$ be as above. We assume that $$C$$ is an integer polyhedral cone and that $$\Delta$$ has integer vertices. Let $$t>0$$ be sufficiently large, so that $$A$$ is contained in $$C_{t'}$$ for some $$t'<t$$.

Clearly, there is a homomorphism $$\Phi$$ from the additive group $$Z(\mathbb {Z}^d)$$ spanned by the indicator functions of integer cones to the additive group of rational functions of $$x_1$$, $$\ldots$$ , $$x_d$$ such that $$\Phi (\mathbb {I}_Q)=G(Q)$$ for every integer cone $$Q$$. Using the expression $$\mathbb {I}_{A}=-\mathbb {I}_{\Delta _t}+\mathbb {I}_{C_t}-\mathbb {I}_{\{0\}}$$, we obtain that
\begin{aligned} G(A)=-G(\Delta _t)+G(C_t)-1. \end{aligned}
Apply Brion’s theorem to $$G(\Delta _t)$$ and $$G(C_t)$$. Near all vertices of $$C_t$$, except $$0$$, the polytopes $$\Delta _t$$ and $$C_t$$ coincide. Therefore, the terms associated with these vertices cancel each other. What remains are the terms associated with the vertices of $$\Delta _t$$ that are simultaneously vertices of $$A$$ and the term with the vertex $$0$$ of $$C_t$$ (which is equal to $$G(C)$$). Thus we have
\begin{aligned} G(A)=-\sum _{a\in \mathrm {Vert}(A){\setminus }\{0\}} x^a G(C'_a)+G(C)-1, \end{aligned}
as desired.

### 3.4 A Viewpoint on Theorem A Through Virtual Polytopes

We will now interpret Theorem A in more conceptual terms involving Theorem B. The function $$\mathrm {Vol}$$ assigning to every convex polytope its volume extends to an additive group homomorphism $$\mu :Z_c(\mathbb R^d)\rightarrow \mathbb R$$ such that $$\mu (\mathbb {I}_{\Delta })=\mathrm {Vol}(\Delta )$$ for every convex polytope $$\Delta$$. The homomorphism $$\mu$$ can be viewed as a valuation on polytopes. By [12], the restriction of $$\mu$$ to the multiplicative group $$Z^*_c(\mathbb R^d)$$ of virtual convex polytopes is a homogeneous degree $$d$$ polynomial. It follows that there is a unique symmetric map
\begin{aligned} \mu :Z^*_c(\mathbb R^d)\times \cdots \times Z^*_c(\mathbb R^d)\rightarrow \mathbb R\end{aligned}
(slightly abusing the notation, we use the same letter $$\mu$$ to denote this map) with the property that $$\mu (\alpha ,\ldots ,\alpha )=\mu (\alpha )$$ for every $$\alpha \in Z_c^*(\mathbb R^d)$$ and that
\begin{aligned} \mu (\alpha _1*\beta ,\alpha _2,\ldots ,\alpha _d)=\mu (\alpha _1,\alpha _2,\ldots ,\alpha _d)+ \mu (\beta ,\alpha _2,\ldots ,\alpha _d) \end{aligned}
for every $$\alpha _1$$, $$\ldots$$ , $$\alpha _d$$, $$\beta \in Z^*_c(\mathbb R^d)$$. The number $$\mu (\alpha _1,\ldots ,\alpha _d)$$ is called the mixed volume of virtual convex polytopes $$\alpha _1$$, $$\ldots$$ , $$\alpha _d$$. This is justified by the observation that if $$\alpha _i=\mathbb {I}_{\Delta ^{(i)}}$$ for some convex polytopes $$\Delta ^{(i)}$$, then the mixed volume of $$\alpha _i$$s is equal to the mixed volume of $$\Delta ^{(i)}$$s. It is well known that the function $$\mu$$ is continuous with respect to a natural topology on $$Z^*_c(\mathbb R^d)$$, in which $$\mathbb {I}_{\Delta ^{(n)}}\rightarrow \mathbb {I}_{\Delta }$$ if convex polytopes $$\Delta ^{(n)}$$ converge to a convex polytope $$\Delta$$ in the Hausdorff metric.
Let $$C$$ be a salient closed convex cone with the apex at $$0$$ and a nonempty interior. Take a linear functional $$\xi$$ such that $$\xi \geqslant 0$$ on $$C$$ and $$\xi ^{-1}(0)\cap C=\{0\}$$. Suppose that a convex subset $$\Delta \subset C$$ is such that $$C{\setminus }\Delta$$ lies in the half-plane $$\xi < t_0$$. We have
for arbitrary $$T$$. Indeed, on the one hand, the left-hand side equals the limit of the right-hand side as $$T\rightarrow 0$$. On the other hand, the right-hand side is independent of $$T$$ by the multi-linearity of the mixed volume. We now let $$T$$ diverge to infinity, and observe that the convex polytope $$\frac{1}{T}\Delta _{t_0(T+1)}$$ converges to $$C_{t_0}$$. Thus we obtain that
It follows that the number given by both sides of this equality does not depend on the geometry of $$\Delta$$, it only depends on $$C$$ and on $$t_0$$ (indeed, the left-hand side depends only on the section $$\{\xi =t\}\cap \Delta$$). In particular, the $$d$$ copies of $$\Delta$$ can be replaced with $$d$$ different closed convex sets $$\Delta ^{(1)}$$, $$\ldots$$ , $$\Delta ^{(d)}$$ such that the complements $$C{\setminus }\Delta ^{(i)}$$ are contained in the set $$\{\xi <t_0\}$$:
\begin{aligned} \mu (\mathbb {I}_{C_{t_0}},\mathbb {I}_{\Delta ^{(2)}_{t_0}},\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}})= \mu (\mathbb {I}_{C_{t_0}},\mathbb {I}_{\Delta _{t_0}},\ldots ,\mathbb {I}_{\Delta _{t_0}}). \end{aligned}
(3.1)
As the convex set $$\Delta ^{(2)}_{t_0}$$ degenerates to the cone $$C_{t_0}$$, e.g., through the family $$\frac{1}{T}\Delta ^{(2)}_{Tt_0}$$, where $$T\rightarrow \infty$$, we obtain that
\begin{aligned} \mu (\mathbb {I}_{C_{t_0}},\mathbb {I}_{\Delta ^{(2)}_{t_0}},\mathbb {I}_{\Delta ^{(3)}_{t_0}},\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}})= \mu (\mathbb {I}_{C_{t_0}},\mathbb {I}_{C_{t_0}},\mathbb {I}_{\Delta ^{(3)}_{t_0}},\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}}) \end{aligned}
or, subtracting the right-hand side from the left-hand side,
\begin{aligned} \mu (\mathbb {I}_{C_{t_0}},\mathbb {I}_{\Delta ^{(2)}_{t_0}}*\mathbb {I}_{C_{t_0}}^{-1}, \mathbb {I}_{\Delta ^{(3)}_{t_0}},\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}})=0. \end{aligned}
(3.2)
Set $$A^{(i)}=C{\setminus }(\Delta ^{(i)}\cup \{0\})$$ to be the $$C$$-coconvex polytopes corresponding to $$\Delta ^{(i)}$$. Recalling that $$-\mathbb {I}_{A^{(i)}}=\mathbb {I}_{\Delta ^{(i)}_{t_0}}*\mathbb {I}_{C_{t_0}}^{-1}$$, we can rewrite equation (3.2) as
\begin{aligned} \mu (\mathbb {I}_{C_{t_0}},-\mathbb {I}_{A^{(i)}},\ldots )=0. \end{aligned}
(3.3)
Here dots replace any sequence of $$d-2$$ virtual convex polytopes from the group generated by $$\mathbb {I}_{\Delta ^{(i)}_{t_0}}$$.
Taking $$\Delta$$ as above and setting $$A=C{\setminus }(\Delta \cup \{0\})$$, we obtain that
\begin{aligned} \mu (-\mathbb {I}_A)=-\mu (\mathbb {I}_A)=-\mu (\mathbb {I}_{C_{t_0}})+\mu (\mathbb {I}_{\Delta _{t_0}}) \end{aligned}
from the additivity of the volume. Passing to the mixed volumes, we obtain that
\begin{aligned} \mu (-\mathbb {I}_{A^{(1)}},\ldots ,-\mathbb {I}_{A^{(d)}})=-\mu (\mathbb {I}_{C_{t_0}})+ \mu (\mathbb {I}_{\Delta ^{(1)}_{t_0}},\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}}). \end{aligned}
(3.4)
Setting $$\Delta =C$$ in (3.1) and substituting into (3.4), we obtain that
\begin{aligned} \mu (-\mathbb {I}_{A^{(1)}},-\mathbb {I}_{A^{(2)}},\ldots ,-\mathbb {I}_{A^{(d)}})= \mu (-\mathbb {I}_{A^{(1)}},\mathbb {I}_{\Delta ^{(2)}_{t_0}}\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}}). \end{aligned}
(3.5)
From the right-hand side, we can subtract $$\mu (-\mathbb {I}_{A^{(1)}},\mathbb {I}_{C_{t_0}},\mathbb {I}_{\Delta ^{(3)}_{t_0}},\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}})$$ (which is equal to zero by (3.3)) to obtain that
\begin{aligned} \mu (-\mathbb {I}_{A^{(1)}},-\mathbb {I}_{A^{(2)}},-\mathbb {I}_{A^{(3)}},\ldots ,-\mathbb {I}_{A^{(d)}})= \mu (-\mathbb {I}_{A^{(1)}},-\mathbb {I}_{A^{(2)}},\mathbb {I}_{\Delta ^{(3)}_{t_0}},\ldots ,\mathbb {I}_{\Delta ^{(d)}_{t_0}}). \end{aligned}
(3.6)
Fix the sets $$\Delta ^{(i)}$$ with $$i\geqslant 3$$. Then the left-hand side of (3.6) can be viewed (up to a sign) as the coconvex Aleksandrov–Fenchel from. In fact, it is equal to the minus mixed volume of $$C$$-coconvex bodies $$A^{(i)}$$. On the other hand, the right-hand side is a usual convex Aleksandrov–Fenchel form associated with convex polytopes $$\Delta ^{(i)}_{t_0}$$, $$i\geqslant 3$$, evaluated at virtual polytopes $$-\mathbb {I}_{A^{(1)}}$$ and $$-\mathbb {I}_{A^{(2)}}$$. Let $$B$$ denote this convex Aleksandrov–Fenchel form. It follows from the convex Aleksandrov–Fenchel inequality that $$B(\alpha ,\alpha )\leqslant 0$$ provided that $$\alpha \in Z_c^*(\mathbb R^d)$$ is orthogonal to some convex polytope $$\Delta$$ in the sense that $$B(\alpha ,\mathbb {I}_{\Delta })=0$$. But $$-\mathbb {I}_{A^{(1)}}$$ is orthogonal to $$C_{t_0}$$ by (3.3)! It follows that $$B(-\mathbb {I}_{A^{(1)}},-\mathbb {I}_{A^{(1)}})\leqslant 0$$. This implies the coconvex Aleksandrov–Fenchel inequality.

Thus we obtained another proof of Theorem A. Although this proof is no simpler than the one given in Sect. 2.2, it reveals the role of virtual convex polytopes and the fact that the coconvex Aleksandrov–Fenchel form is no different from the convex Aleksandrov–Fenchel form evaluated at certain virtual polytopes.

### Remark

We now sketch an analogy, which can be easily formalized and which may shed some light to the argument presented above. Consider complex algebraic varieties $$X$$, $$Y$$ and a regular map $$f:X\rightarrow Y$$. Fix a point $$y_0\in Y$$, and assume that $$f:X{\setminus }f^{-1}(y_0)\rightarrow Y{\setminus }\{y_0\}$$ is an isomorphism. A Cartier divisor $$D$$ in $$X$$ is said to be sub-exceptional if the support of $$D$$ maps to $$y_0$$ under $$f$$. Sub-exceptional divisors in $$X$$ correspond to coconvex polytopes (under the analogy, which we are discussing). A Cartier divisor $$D$$ in $$X$$ is said to be off-exceptional if $$D=f^*(\widetilde{D})$$ for some Cartier divisor $$\widetilde{D}$$ in $$Y$$, whose support does not contain $$y_0$$. Off-exceptional divisors correspond to the cone $$C$$ (we are looking at the local geometry of $$Y$$ near $$y_0$$, thus we do not distinguish between different off-exceptional divisors). Let $$S$$ and $$O$$ be a sub-exceptional and an off-exceptional divisors, respectively. If $$[S]$$ and $$[O]$$ stand for the classes of these divisors in the Chow ring, then obviously $$[S]\cdot [O]=0$$, since the supports of $$S$$ and $$O$$ are disjoint. This fact is analogous to Eq. (3.3). As is shown in [10], the Hodge index theorem implies the Aleksandrov–Fenchel inequalities for the intersection of divisors in $$X$$ (under some natural assumptions on $$X$$, e.g., when $$X$$ is projective and smooth). These inequalities can be used to provide an analog of Theorem A for the intersection form on sub-exceptional divisors. To make the described analogy into a precise correspondence, one takes $$X$$ and $$Y$$ to be toric varieties associated with integer polytopes $$\Delta _{t_0}$$ and $$C_{t_0}$$.

## Notes

### Acknowledgments

The first named author was partially supported by Canadian Grant N 156833-02. The second named author was partially supported by the Dynasty Foundation Grant, RFBR Grants 13-01-12449, 13-01-00969, and AG Laboratory NRU HSE, MESRF Grant ag. 11 11.G34.31.0023. The research comprised in Theorem A was funded by RScF, project 14-21-00053.

## References

1. 1.
Aleksandrov, A.D.: To the theory of mixed volumes of convex bodies II. New inequalities between mixed volumes and their applications. Mat. Sb. 2(6), 1205–1238 (1937) (in Russian)Google Scholar
2. 2.
Arnold, V., Varchenko, A., Gusein-Zade, S.: Singularities of Differentiable Maps. Springer, New York (1985)
3. 3.
Beck, M., Robins, S.: Computing the Continuous Discretely. Springer, New York (2009)Google Scholar
4. 4.
Bernstein, D.: The number of roots of a system of equations. Funct. Anal. Appl. 9(3), 183–185 (1975)
5. 5.
Brion, M.: Points entiers dans les polyèdres convexes. Ann. Sci. École Norm. Sup. (4) 21(4), 653–663 (1988)
6. 6.
Dubrovin, B.A., Novikov, S.P., Fomenko, A.T.: Modern Geometry: Methods and Applications, Part I. The Geometry of Surfaces, Transformation Groups, and Fields. Graduate Texts in Math., Vol. 93. Springer, New York (1991)Google Scholar
7. 7.
Fillastre, F.: Fuchsian convex bodies: Basics of Brunn–Minkowski theory. Geom. Funct. Anal. 23(1), 295–333 (2013)Google Scholar
8. 8.
Kaveh, K., Khovanskiĭ, A.G.: Newton–Okounkov bodies, semigroups of integral points, graded algebras and intersection theory. Ann. Math. 176(2), 925–978 (2012)
9. 9.
Kaveh, K., Khovanskiĭ, A.G.: Convex bodies and multiplicities of ideals. In: Proceedings of the Steklov Institute of Mathematics. Springer, New York (2013) (to appear)Google Scholar
10. 10.
Khovanskii, A.: Algebra and mixed volumes. In: Burago, Yu.D., Zalgaller, V.A. (eds.) Geometric Inequalities. Springer, Berlin (2010)Google Scholar
11. 11.
Khovanskiĭ, A., Pukhlikov, A.: Finitely additive measures of virtual polytopes. St. Petersburg Math. J. 4(2), 337–356 (1993)
12. 12.
Khovanskiĭ, A., Pukhlikov, A.: The Riemann–Roch theorem for integrals and sums of quasipolynomials on virtual polytopes. St. Petersburg Math. J. 4(4), 789–812 (1993)
13. 13.
Khovanskiĭ, A., Timorin, V.: Aleksandrov–Fenchel inequality for coconvex bodies. http://arxiv.org/abs/1305.4484
14. 14.
Khovansky, A., Varchenko, A.: Asymptotics of integrals over vanishing cycles and the Newton polyhedron. Sov. Math. Dokl. 32, 122–127 (1985)Google Scholar
15. 15.
Kouchnirenko, A.: Newton polyhedron and Milnor numbers. Funct. Anal. Appl. 9(1), 71–72 (1975)
16. 16.
Martinez-Maure, Y.: Hedgehog theory via Euler calculus. Beitr. Algebra Geom. doi:
17. 17.
McMullen, P.: Valuations and Euler-type relations on certain classes of convex polytopes. Proc. Lond. Math. Soc. Ser. 3 35(1), 113–135 (1977)
18. 18.
McMullen, P.: On simple polytopes. Invent. Math. 113, 419–444 (1993)
19. 19.
Schneider, R.: Convex Bodies: The Brunn–Minkowski Theory. Cambridge University Press, Cambridge (1993)
20. 20.
Teissier, B.: Du théorème de l’index de Hodge aux inégalités isopérimétriques. C. R. Acad. Sci. Paris Sér. A 288(4), 287–289 (1979)
21. 21.
Timorin, V.: An analogue of the Hodge–Riemann relations for simple polytopes. Russ. Math. Surv. 54(2), 381–426 (1999)

## Authors and Affiliations

1. 1.Department of MathematicsUniversity of TorontoTorontoCanada
2. 2.Independent University of MoscowMoscowRussia
3. 3.Institute for Systems AnalysisRussian Academy of SciencesMoscowRussia
4. 4.Faculty of Mathematics and Laboratory of Algebraic GeometryNational Research University Higher School of EconomicsMoscowRussia