# Lattice point generating functions and symmetric cones

## Abstract

We show that a recent identity of Beck–Gessel–Lee–Savage on the generating function of symmetrically constrained compositions of integers generalizes naturally to a family of convex polyhedral cones that are invariant under the action of a finite reflection group. We obtain general expressions for the multivariate generating functions of such cones, and work out their general form more specifically for all symmetry groups of type A (previously known) and types B and D (new). We obtain several applications of these expressions in type B, including identities involving permutation statistics and lecture hall partitions.

## Keywords

Lattice point generating function Polyhedral cone Finite reflection group Coxeter group Symmetrically constrained composition Permutation statistics Lecture hall partition## 1 Introduction

*symmetrically constrained compositions*, i.e., compositions of an integer

*M*into

*n*nonnegative parts

*λ*:=(

*λ*

_{1},

*λ*

_{2},…,

*λ*

_{ n }) satisfies the symmetric system of linear inequalities

*a*

_{1},

*a*

_{2},…,

*a*

_{ n }, the generating functions

*λ*. One viewpoint of [3] is geometric: The compositions (

*λ*

_{1},

*λ*

_{2},…,

*λ*

_{ n }) are interpreted as integer lattice points in the cone

*W*is the image of the permutation representation of

*S*

_{ n },

*a*=(

*a*

_{1},…,

*a*

_{ n }), and ( , ) is the standard inner product on

**R**

^{ n }. This viewpoint together with permutation statistics of

*S*

_{ n }gave rise to explicit (and in some instances surprising) generating function formulas.

Our goal is to generalize the results in [3] to cones of the form (1) where *W* is another reflection group. In addition to obtaining general multivariate generating function identities, we obtain several applications of these results for hyperoctahedral groups. These applications are similar in spirit to the applications in the symmetric-group case found in [3].

The outline of our paper is as follows. The general setup for our approach is discussed in the next section, which also contains our central result, Theorem 2.8. Section 3 illustrates our approach by re-deriving the main result in [3]. Sections 4 and 5 consider cones constrained by reflection groups of type *B* and *D*, respectively. Further, Sect. 4 contains applications obtained through specializing our generating functions in the type-*B* case.

## 2 General theory

Our goal in this section is to study integer points in cones that are constrained by the orbit of a single linear constraint under an appropriate group action on real space. This goal is realized in Theorem 2.8, where the multivariate generating function encoding the integer points in such a cone is expressed as a sum of simpler generating functions. Theorem 2.8 is an algebraic consequence of a geometric triangulation of the symmetric cone, which we obtain in Lemma 2.5. Proposition 2.6 makes the triangulation disjoint by using combinatorics of Coxeter groups as a tiebreaker for the walls separating the maximal cones in the triangulation. This is critical for our subsequent applications.

### 2.1 Almost irreducible finite reflection groups, Coxeter groups, and descents

In the following, we will consider finite reflection groups (i.e., finite subgroups of O(*V*) for some Euclidean space *V* that are generated by reflections—see, e.g., [10] for background) that act on the underlying Euclidean space in a restricted fashion. Namely, a finite reflection group *W*⊂O(*V*) acting on a Euclidean vector space *V* is called *almost irreducible* if *V* decomposes into *W*-invariant subspaces *V*=*V* _{1}⊕*V* _{2} such that *W* acts irreducibly and nontrivially on *V* _{1} and trivially on *V* _{2}, and that *V* _{2} is 1-dimensional.

### Example 2.1

*S* _{ n } acts almost irreducibly on **R** ^{ n } by permutation of the components. The irreducible summand consists of all vectors with component sum 0, and the trivial summand consists of all vectors with equal components. This is the case considered in [3].

### Example 2.2

Let *V* _{1} be a Euclidean vector space and *W*⊂O(*V* _{1}) a nontrivial irreducible reflection group, i.e., a nontrivial reflection group such that *V* _{1} does not contain any nontrivial proper *W*-invariant subspaces. Let *W* act trivially on **R** and set *V*=*V* _{1}⊕**R**. Then *W* acts almost irreducibly on *V*.

A *Coxeter group* of rank *r* is a group admitting a presentation with generators *s* _{1},…,*s* _{ r } and relations \((s_{j}s_{k})^{m_{jk}} = 1\) for *m* _{ jk }∈{1,2,3,…}∪{∞} subject to the conditions that *m* _{ jk }=*m* _{ kj } and *m* _{ jk }=1⇔*j*=*k*. Here, a value of *m* _{ jk }=∞ is to be understood as the absence of the corresponding relation. Such generators are called *simple generators*. For each Coxeter group considered, we will suppose that simple generators have been fixed once and for all. We refer the reader to [5] or [10] for further information regarding Coxeter groups and their relation to reflection groups.

*length*

*l*(

*σ*) of an element

*σ*∈

*W*of a Coxeter group

*W*is the smallest integer such that there is a decomposition \(\sigma= s_{j_{1}} \cdots s_{j_{l(\sigma)}}\) of

*σ*as a product of

*l*(

*σ*) not necessarily distinct simple generators. For any

*σ*∈

*W*, the

*right descent set*of

*σ*is

### Remark 2.3

Propositions 3.1, 4.1, and 5.1 review the connection between the definition of descent given here and definitions of descent for Coxeter groups of types *A*, *B*, and *D* in terms of the one-line notation.

Recall that if *W* is a finite reflection group, it is automatically a Coxeter group. Simple generators can be found as follows. Let \(\mathcal{H}\) be the union of all reflection hyperplanes for *W*; denote by *F* the closure of a connected component in \(V\backslash \mathcal{H}\). It is immediate that *F* is a convex polyhedral cone. Let *H* _{1},…,*H* _{ r } be the facet hyperplanes of *F* and let *s* _{ i } be the reflection at *H* _{ i }. Then *s* _{1},…,*s* _{ r } are simple generators of the Coxeter group *W*. See [6, V.3.2, Theorem 1] for the proof of these statements.

A subset *F*⊂*V* is a *fundamental domain* for *W* if *F* is the closure of an open set and each *W*-orbit intersects *F* in exactly one point. By [10, Sect. I.12], every such *F* is polyhedral, and is bounded by hyperplanes fixed by a set of simple reflections in *W*. Through the rest of this paper, when given a set of simple generators *s* _{1},…,*s* _{ r } of a reflection group *W*, we denote by *F* a fixed fundamental domain with bounding hyperplanes corresponding to *s* _{1},…,*s* _{ r }.

### 2.2 Triangulations of monoconditional cones

*φ*∈

*V*

^{∗}on a vector

*x*∈

*V*by 〈

*x*,

*φ*〉. Let

*W*⊂O(

*V*) be an almost irreducible reflection group. A

*symmetric cone*

*C*⊂

*V*is a convex polyhedral cone that is

*W*-invariant. A symmetric cone is called

*monoconditional*if there is a linear form

*φ*∈

*V*

^{∗}, such that \(V_{1}, V_{2} \not\subset\ker (\varphi )\) and

### Example 2.4

The positive orthant \(\mathbf{R}_{\geq0}^{n}\) is a monoconditional symmetric cone for the almost irreducible action of *S* _{ n } on **R** ^{ n } by permutation of the components. A possible linear form defining it is the projection on the first component.

Recall that a convex polyhedral cone is *pointed* if it does not contain a line, and it is *simplicial* if it is the set of linear combinations with nonnegative coefficients of a set of linearly independent vectors. A *triangulation* of a cone *C* is a finite collection *T* of simplicial cones such that *C* is the union of the elements of *T* and for any Δ_{1},Δ_{2}∈*T*, Δ_{1}∩Δ_{2} is a face common to both Δ_{1} and Δ_{2}.

### Lemma 2.5

*Let*

*W*⊂O(

*V*)

*be an almost irreducible reflection group*.

*Let*

*C*⊂

*V*

*be a monoconditional symmetric cone*.

*Then*

*C*

*is pointed*.

*Let*

*F*⊂

*V*

*be a fundamental domain for the action of*

*W*

*on*

*V*.

*Then the cone*

*C*

_{+}:=

*C*∩

*F*

*is simplicial*.

*In particular*,

*C*

*admits the triangulation*

### Proof

*φ*∈

*V*

^{∗}be a linear form defining

*C*as in (3); note that ( , ) is a

*W*-invariant inner product on

*V*. Let

*x*

_{ φ }∈

*V*such that 〈

*x*,

*φ*〉=−(

*x*,

*x*

_{ φ }) for all

*x*∈

*V*. Let

*x*

_{0}be the unique element of

*Wx*

_{ φ }∩

*F*. Then

*V*=

*V*

_{1}⊕

*V*

_{2}be the decomposition of

*V*into the irreducible and trivial component. Let \(x_{0} = x_{0}' + x_{0}''\) with \(x_{0}' \in V_{1}\) and \(x_{0}'' \in V_{2}\). By definition \(V_{2} \not\subset\ker(\varphi)\), and so

*x*

_{ φ }∉

*V*

_{1}, thus

*x*

_{0}∉

*V*

_{1}, and hence \(x_{0}'' \neq0\). As

*φ*is only determined up to multiplication by a positive scalar, suppose without loss of generality that \((x_{0}'', x_{0}'') = 1\). Note that every nonzero element of

*C*

_{+}has a nonzero

*V*

_{2}-component; otherwise, for some

*x*∈

*C*

_{+}it would hold that \((\sigma x, x_{0}') \leq(\sigma x,x_{0}'')=0\) for all

*σ*∈

*W*. This would imply that \((x,\sigma x_{0}')\leq0\) for all

*σ*∈

*W*, which leads to a contradiction since

*x*≠0 and \(\sum_{\sigma\in W}\sigma x_{0}' = 0\) by the irreducibility of the action of

*W*on

*V*

_{1}.

This implies that every nonzero element of *C* has a nonzero *V* _{2}-component, in particular, *C* is pointed.

*F*as simple generators of

*W*. Let

*l*denote the corresponding length function. Let

*x*∈

*F*∩

*V*

_{1}and

*σ*∈

*W*. Let

*H*be a facet hyperplane of

*σF*, such that

*l*(

*sσ*)>

*l*(

*σ*) for the reflection

*s*at

*H*. We claim that in this situation

*V*

_{1}=(

*H*∩

*V*

_{1})⊕

*H*

^{⊥}. According to this decomposition, write \(x_{0}' = v_{0} + w_{0}\) and

*σx*=

*v*

_{1}+

*w*

_{1}; then

*sσx*=

*v*

_{1}−

*w*

_{1}. We have \((\sigma x, x_{0}') = (v_{1}, v_{0}) + (w_{1}, w_{0})\) and \((s \sigma x, x_{0}') = (v_{1}, v_{0}) - (w_{1}, w_{0})\). Hence

*τ*∈

*W*, then

*l*(

*τ*) equals the number of reflection hyperplanes between

*F*and

*τF*. As

*l*(

*sσx*)>

*l*(

*σx*), this implies that

*x*

_{0}and

*σx*lie on the same side of

*H*. Hence (

*w*

_{1},

*w*

_{0})≥0, and so (5) implies the claim (4).

*l*(

*σ*), (4) implies that

*C*

_{+}is the cone over \(P - x_{0}''\), thus

*C*

_{+}={

*x*∈

*F*∣(

*x*,

*x*

_{0})≤0}. The cone

*F*∩

*V*

_{1}is a fundamental domain for the action of

*W*on

*V*

_{1}. Hence

*F*∩

*V*

_{1}is simplicial, as it is defined by the hyperplanes corresponding to the simple generators of

*W*, and there are dim(

*V*

_{1}) many such hyperplanes. We have

*F*=(

*F*∩

*V*

_{1})+

*V*

_{2}, and so dim(

*F*)=dim(

*F*∩

*V*

_{1})+1. The cone

*C*

_{+}is defined in

*F*by the single additional inequality (

*x*,

*x*

_{0})≤0, thus

*C*

_{+}is simplicial. □

*H*

_{1},…,

*H*

_{ n−1}of the facet hyperplanes of

*F*with the corresponding simple reflections

*s*

_{1},…,

*s*

_{ n−1}. For any subset

*J*⊂{1,…,

*n*−1}, let

For example, *C* _{∅}=*C* _{+}. If *J*≠∅ some of the facets of *C* _{+} are removed.

### Proposition 2.6

*In the situation of Lemma*2.5,

*C*

*decomposes as a disjoint union*

### Proof

*x*∈

*C*, let

*x*

_{+}∈

*Wx*∩

*C*

_{+}be the (unique)

*W*-conjugate of

*x*contained in the fundamental domain

*C*

_{+}. Let \(\tilde{W} := \{ \sigma\in W \mid\sigma x_{+} = x_{+} \}\) be the isotropy subgroup of

*x*

_{+}in

*W*. By Lemma 2.5,

*C*=⋃

_{ σ∈W }

*σC*

_{+}, and so the set

*W*(

*x*) is nonempty. Fix any

*σ*∈

*W*(

*x*).

We claim that \(W(x) = \sigma\tilde{W}\). Indeed, it is obvious that \(W(x) \supseteq\sigma\tilde{W}\). For the opposite inclusion, let \(\bar{\sigma}\in W(x)\). Then \(\bar{\sigma}^{-1} x \in C_{+}\), so \(\bar{\sigma}^{-1} = x_{+}\). By the same argument *σ* ^{−1} *x*=*x* _{+}, so \(\sigma^{-1} \bar{\sigma}\in\tilde{W}\). Hence \(\bar{\sigma}= \sigma\sigma^{-1} \bar{\sigma}\in\sigma\tilde{W}\), proving the claim.

By [10, Theorem 1.12(c)], the isotropy subgroup \(\tilde{W}\) is generated by the reflections it contains. So \(\tilde{W}\) is a parabolic subgroup of *W*. Hence \(W(x) = \sigma\tilde{W}\) contains a unique element of minimal length [10, §1.10], denoted by *σ* _{ x }.

Assume that \(\sigma_{x}^{-1}x \in H_{j}\) for some *j*∈*D* _{r}(*σ* _{ x }). Then \(s_{j} \sigma_{x}^{-1} x = \sigma_{x}^{-1}x\), and so \(x = \sigma_{x} s_{j} \sigma_{x}^{-1} x\). As \(\sigma_{x}^{-1} x \in C_{+}\), this implies that *σ* _{ x } *s* _{ j }∈*W*(*x*). On the other hand *l*(*σ* _{ x } *s* _{ j })<*l*(*σ* _{ x }), a contradiction. Hence \(\sigma_{x}^{-1} x \notin H_{j}\) for all *j*∈*D* _{r}(*σ* _{ x }). Hence \(\sigma_{x}^{-1} x \in C_{D_{\mathrm{r}}(\sigma_{x})}\), and so \(x \in\sigma_{x} C_{D_{\mathrm{r}}(\sigma_{x})}\). This proves that \(C = \bigcup_{\sigma\in W} \sigma C_{D_{\mathrm{r}}(\sigma)}\).

To prove disjointness, let \(x \in\sigma C_{D_{\mathrm{r}}(\sigma)}\) for some *σ*∈*W*. We have to show that *σ*=*σ* _{ x }. Clearly *σ*∈*W*(*x*). It remains to show that *σ* has minimal length in *W*(*x*). Assume that *σ* has not minimal length in *W*(*x*). Then there is *j*∈{1,…,*n*−1} such that *l*(*σs* _{ j })<*l*(*σ*) and *σs* _{ j }∈*W*(*x*) [10, §1.10]. From *σ*∈*W*(*x*) we conclude that *σ* ^{−1} *x*∈*C* _{+} and from *σs* _{ i }∈*W*(*x*) that *s* _{ j } *σ* ^{−1} *x*∈*C* _{+}. Hence *σ* ^{−1} *x*∈*H* _{ j }. Since *l*(*σs* _{ j })<*l*(*σ*) we have *j*∈*D* _{r}(*σ*). Hence \(\sigma^{-1}x \notin C_{D_{\mathrm{r}}(\sigma)}\), and so \(x \notin \sigma C_{D_{\mathrm{r}}(\sigma)}\), a contradiction. □

Our proposition above is reminiscent of the theory of *P*-partitions [12, 13]. For a given finite poset *P*, one can produce a cone of *P*-partitions. The standard approach to studying a *P*-partition cone, originating in the work of Stanley referenced above, is to recognize that each such cone is a disjoint union of various chambers of the type A braid arrangement where each chamber has some of its facets removed. The removal of facets in a given chamber is controlled by the descent statistic for the permutation indexing that chamber. Thus, each *P*-partition cone admits a unimodular triangulation of form similar to Proposition 2.6.

### 2.3 Generating functions for monoconditional cones

Let \(V^{*}_{\mathbf{C}} = V^{*} \otimes_{\mathbf{R}} \mathbf{C}\). Extend 〈 , 〉 to \(V \times V^{*}_{\mathbf{C}}\) by **C**-linearity in the second argument. Let *Γ*⊂*V* be a lattice and *S*⊂*V*. Suppose that there is a nonempty open subset \(B \subset V^{*}_{\mathbf{C}}\) such that the series ∑_{ x∈S∩Γ } *e* ^{−〈x,φ〉} converges for *φ*∈*B* and has a meromorphic continuation to \(V^{*}_{\mathbf{C}}\). We denote this continuation by *f* _{ S } and call it the *generating function* of *S* with respect to *Γ*.

### Example 2.7

*C*⊂

*V*is a cone, let

*C*is defined as

*C*⊂

*V*be a pointed cone, rational with respect to

*Γ*. Then ∑

_{ x∈C∩Γ }

*e*

^{−〈x,φ〉}converges on the interior of \(C^{\vee}_{\mathbf{C}}\) and has a meromorphic continuation

*f*

_{ C }to \(V^{*}_{\mathbf{C}}\); see, e.g., [2, Chap. 13].

From now on, suppose that *W* is crystallographic, i.e., that there is a *W*-invariant lattice *Γ* in *V*. A full-dimensional simplicial cone *C*⊂*V* is called *unimodular* (with respect to *Γ*) if it is generated by a basis of *Γ*. These generators are called *primitive*.

### Theorem 2.8

*In the situation of Lemma*2.5,

*suppose that*

*C*

_{+}

*is unimodular with respect to*

*Γ*.

*Let*

*b*

_{1},…,

*b*

_{ n }

*be the primitive generators of*

*C*

_{+},

*enumerated in the unique way such that*

*b*

_{ j }∉

*H*

_{ j }

*for*

*j*∈{1,…,

*n*−1}.

*Then the generating function of*

*C*

*is*

In practice, *Γ* is often endowed with a distinguished basis. In this case, it is often more convenient to work with the following formulation.

### Corollary 2.9

*In the situation of Theorem*2.8,

*let*

*e*

_{1},…,

*e*

_{ n }

*be a basis of*

*Γ*.

*Define coordinates*

*z*

_{ j }

*on*\(V^{*}_{\mathbf{C}}\)

*by*\(z_{j}(\varphi) := e^{- \langle e_{j}, \varphi\rangle}\).

*For*

*a*=

*a*

_{1}

*e*

_{1}+⋯+

*a*

_{ n }

*e*

_{ n }∈

*Γ*,

*let*\(z^{a} := z_{1}^{a_{1}} \cdots z_{n}^{a_{n}}\).

*Then*

### Proof of Theorem 2.8

*C*is pointed (by Lemma 2.5), its generating series converges on a nonempty domain and the generating series of all

*W*-conjugates of

*C*

_{+}converge there. As

*C*

_{+}is unimodular with primitive generators

*b*

_{1},…,

*b*

_{ n }, its generating function is

*C*

_{+}is only one lattice hyperplane away from the opposite facet. Hence

*j*∈{1,…,

*n*−1}. More generally,

*C*

_{ J }∩

*Γ*=(

*C*

_{+}∩

*Γ*)+∑

_{ j∈J }

*b*

_{ j }for any

*J*⊂{1,…,

*n*−1}. Applying this observation to

*J*=

*D*

_{r}(

*σ*) for a

*σ*∈

*W*and rephrasing it in terms of generating functions, one obtains

*σ*∈

*W*it follows that

## 3 Cones with the symmetry of a simplex

Theorem 2.8 specializes to more concrete identities once we fix a particular almost irreducible reflection group *W*. The case of *W* being the group of symmetries of a simplex has been treated in [3]. We include this case here to show how the result can be derived from Theorem 2.8.

*S*

_{ n }denote the group of permutations of the set {1,…,

*n*}. For

*π*∈

*S*

_{ n }, we define the

*descent set*of

*π*as

The group *S* _{ n } acts on **R** ^{ n } by permutation of the components. For *π*∈*S* _{ n }, let *σ* _{ π }∈O(**R** ^{ n }) denote the transformation by which *π* acts on **R** ^{ n }. Let *W*={*σ* _{ π }∣*π*∈*S* _{ n }}⊂O(**R** ^{ n }). Then *W* is the group of symmetries of the (*n*−1)-dimensional standard simplex. For *j*=1,…,*n*−1, let *s* _{ j }∈*W* be the transposition of the *j*th and (*j*+1)st component in **R** ^{ n }. Then *s* _{1},…,*s* _{ n−1} are simple generators of *W*.

The following shows that the definitions of descent given in (2) and (6) agree.

Our main result in this section is the following.

### Proposition 3.2

[3, Theorem 1]

*Fix integers*

*a*

_{1}≤⋯≤

*a*

_{ n }

*such that*

*a*

_{1}+⋯+

*a*

_{ n }=1.

*Let*

*Let*

*Σ*

_{ j }:=

*a*

_{1}+⋯+

*a*

_{ j }

*for*

*j*∈{1,…,

*n*−1}.

*The generating function of*

*C*

*with respect to*

**Z**

^{ n }

*is*

Note that the condition on the *a* _{ i } to be increasing is a normalization rather than a restriction.

### Proof

*C*is symmetric and monoconditional for

*W*. Let

*F*={

*x*∈

**R**

^{ n }∣

*x*

_{1}≥⋯≥

*x*

_{ n }}, a fundamental domain for

*W*. Then our chosen simple generators

*s*

_{1},…,

*s*

_{ n−1}of

*W*are the reflections at the facet hyperplanes of

*F*. Let

*x*

_{0}=(−

*a*

_{1},…,−

*a*

_{ n })∈

*F*. By the proof of Lemma 2.5, where The determinant of

*A*is

*a*

_{1}+⋯+

*a*

_{ n }=1, i.e.,

*A*is unimodular and so is

*C*

_{+}. Let

*b*

_{1},…,

*b*

_{ n }be the primitive generators of

*C*

_{+}, enumerated in the unique way such that

*b*

_{ j }∉

*H*

_{ j }for

*j*∈{1,…,

*n*−1}. Then by Corollary 2.9, the generating function of

*C*is

*D*

_{r}(

*σ*),

*b*

_{ j }, and the action of

*W*explicitly. The inverse of

*A*is where \(\varSigma_{j}' :=1 - \varSigma_{j}\). Then

*b*

_{ j }is the

*j*th column vector of

*B*. Let

*i*th component of

*b*

_{ j }, i.e., the (

*i*,

*j*)th component of

*B*. As defined above, with

*π*∈

*S*

_{ n }we associate

*σ*

_{ π }∈O(

*n*) by

*σ*

_{ π }

*e*

_{ i }=

*e*

_{ π(i)}. Then

*W*={

*σ*

_{ π }∣

*π*∈

*S*

_{ n }} and we have

*D*

_{r}(

*σ*

_{ π })=

*D*(

*π*). Hence □

## 4 Cones with hyperoctahedral symmetry

We now consider the case of cones which are symmetric under the action of a hyperoctahedral group. Let *W*⊂O(*n*) be the hyperoctahedral group on the first *n*−1 components of **R** ^{ n }. Let *s* _{1}∈*W* be the sign change in the first component and, for *j*=2,…,*n*−1, let *s* _{ j }∈*W* be the transposition of the (*j*−1)st and *j*th component in **R** ^{ n }. Then *s* _{1},…,*s* _{ n−1} are simple generators of *W*.

*π*∈

*S*

_{ n−1}and

*ε*∈{±1}

^{ n−1}, define

*σ*

_{ π,ε }∈O(

*n*) by

*π*(

*n*):=

*n*for

*π*∈

*S*

_{ n−1}and

*ε*

_{ n }:=1 for

*ε*∈{±1}

^{ n−1}. Then

*W*={

*σ*

_{ π,ε }∣

*π*∈

*S*

_{ n−1},

*ε*∈{±1}

^{ n−1}}. Let

*B*

_{ n−1}denote the set

*S*

_{ n−1}×{±1}

^{ n−1}, endowed with the group structure such that

*σ*:

*B*

_{ n−1}→

*W*becomes an isomorphism of groups.

*W*can be expressed more explicitly. For (

*π*,

*ε*)∈

*B*

_{ n−1}let

*ε*

_{0}

*π*(0):=0. Then the following holds.

### Proposition 4.1

[5, Proposition 8.1.2]

*For all*(

*π*,

*ε*)∈

*B*

_{ n−1},

*we have*

Note that the descent set defined in (8) is translated by +1 with respect to definitions found in the literature on signed permutations. This is because to have a consistent setup in Sect. 2, we always start the enumeration of the simple reflections with 1, whereas from a signed permutations perspective it is convenient to start this enumeration with 0.

*descent number*

*π*,

*ε*)∈

*B*

_{ n−1}. Similarly, the

*major index*is

*comajor index*is

*π*,

*ε*)∈

*B*

_{ n−1}. It follows that we have the relationship

### 4.1 The multivariate generating function

In this situation, Corollary 2.9 specializes as follows.

### Proposition 4.2

*The generating function of*

*C*

*with respect to*

**Z**

^{ n }

*is*

Note that the condition on the *a* _{ i } to be nonnegative and increasing is a normalization rather than a restriction.

### Proof

*C*is symmetric and monoconditional for

*W*. Let

*W*. The

*s*

_{1},…,

*s*

_{ n−1}defined previously are the simple generators of

*W*corresponding to

*F*. Let

*x*

_{0}:=(

*a*

_{1},…,

*a*

_{ n−1},−1)∈

*F*. By the proof of Lemma 2.5, where The matrix

*A*and hence

*C*

_{+}is unimodular. Let

*b*

_{1},…,

*b*

_{ n }be the primitive generators of

*C*

_{+}, enumerated in the unique way such that

*b*

_{ j }∉

*H*

_{ j }for

*j*<

*n*. Then by Corollary 2.9, the generating function of

*C*is

*A*is where

*Σ*

_{ j }:=

*a*

_{ j }+⋯+

*a*

_{ n−1}. Then

*b*

_{ j }is the

*j*th column vector of

*B*. Let

*i*th component of

*b*

_{ j }, i.e., the (

*i*,

*j*)th component of

*B*. By Proposition 4.1 and using our notation introduced at the beginning of this section, □

### 4.2 Hyperoctahedral Eulerian polynomials

In the remainder of Sect. 4, we provide applications of Proposition 4.2 with connections to permutation statistics and Ehrhart theory. Our first application is well known, going back to [8] and [14]; the polyhedral perspective of the following identity was first established in [14], also using Ehrhart theory.

### Corollary 4.3

*The hyperoctahedral Eulerian polynomials are given by*

### Proof

*n*−1)-dimensional hypercube. Our strategy to prove Corollary 4.3 is to compute the Ehrhart series

*P*in two different ways and to conclude by comparing the results.

*C*over

*P*,

*a*

_{1}=⋯=

*a*

_{ n−2}=0,

*a*

_{ n−1}=1, so by Proposition 4.2 its generating function is

_{ P }(

*t*) is obtained by evaluating

*f*

_{ C }at

*z*

_{1}=⋯=

*z*

_{ n−1}=1,

*z*

_{ n }=

*t*, we obtain On the other hand, by definition

### 4.3 The distribution of the comajor index

*k*∈

**N**and a variable

*t*, let

### Corollary 4.4

*The distribution of the comajor index on the hyperoctahedral group is given by*

### Proof

*n*−1)-dimensional cross-polytope. Our strategy to prove Corollary 4.4 is to compute the Ehrhart series

*P*in two different ways and to conclude by comparing the results.

*C*over

*P*,

*a*

_{1}=⋯=

*a*

_{ n−1}=1, so by Proposition 4.2 its generating function is

_{ P }(

*t*) is obtained by evaluating

*f*

_{ C }at

*z*

_{1}=⋯=

*z*

_{ n−1}=1,

*z*

_{ n }=

*t*, we obtain On the other hand, it is known [4, Theorem 2.7] that

### Remark 4.5

*B*

_{ n }arise from studying simple choices of the

*a*

_{ i }from the set of 0/1-vectors. It would be interesting to determine the structure of the multivariate generating functions (or their specializations) when other 0/1-vectors are used; due to the hyperoctahedral symmetry of our cones, this amounts to studying the case

### 4.4 Almost constant coefficients

*P*such that the cone over

*P*is of the form considered in Proposition 4.2 such that

*a*

_{1}through

*a*

_{ n−2}coincide. An example of such a polytope is shown in Fig. 2.

### Corollary 4.6

*Let*

*b*,

*c*≥0,

*not both*0.

*Let*

*Then the Ehrhart series of*

*P*

*is*

### Proof

In Proposition 4.2, we set *a* _{1}=⋯=*a* _{ n−2}=*c* and *a* _{ n−1}=*c*+*b*. Then Ehr_{ P }(*t*)=*f* _{ C }(*t*):=*f* _{ C }(1,…,1,*t*), the generating function of *C* evaluated at *z* _{1},…,*z* _{ n−1}=1, *z* _{ n }=*t*.

*b*>0, the generating function

*f*

_{ C }(

*t*) becomes For

*c*≥1, if

*b*=0 then

*f*

_{ C }(

*t*)=

*f*

_{ P }(

*t*

^{ c })[

*c*]

_{ t }where

*P*is the (

*n*−1)-dimensional cross-polytope discussed in Sect. 4.3; this can be seen by direct computation using Proposition 4.2, and is also a consequence of the fact that cross-polytopes are reflexive [4, Chap. 4]. Otherwise, to simplify, we make use of a result of Chow and Gessel [9, Eq. (26)] to compute the joint distribution of descent and comajor index over

*B*

_{ n }, namely

*π*,

*ε*)∈

*B*

_{ n }, using (9),

*f*

_{ C }(

*t*), we set

*n*=

*n*−1,

*x*=

*t*

^{ b }and

*q*=

*t*

^{ c }in the last line above and get □

### 4.5 Coefficients in arithmetic progression and lecture hall partitions

If we further generalize the results of the previous subsections to allow *a* _{ i } to be a linear function of *i*, then *f* _{ C }(*t*) can be expressed in terms of lecture hall partitions.

*P*

_{ n,2}by

*λ*∈

*L*

_{ n }, (

*π*,

*ε*)∈

*B*

_{ n }, and ⌈

*x*⌉=inf([

*x*,∞)∩

**Z**), set Setting

*k*=2,

*w*=1, and

*z*=

*y*in [11, Theorem 6], and using the bijection of [11, Theorem 3], the following holds. For a positive integer

*n*, using the notation

We can apply Proposition 4.2, with *z* _{1}=⋯=*z* _{ n−1}=1, *z* _{ n }=*t* and appropriate choices of *a* _{ i }, to establish a surprising connection between lecture hall partitions and type *B* symmetrically constrained cones.

### Corollary 4.7

*Let*

*d*≥0,

*c*≥−2

*d*,

*and*

*b*≥0,

*not all*0.

*Define*

*a*

_{1},…,

*a*

_{ n−1}

*by*

*Then*

### Proof

*i*≤

*n*−2, substituting the values of

*a*

_{ i }into Proposition 4.2 with

*z*

_{1}=⋯=

*z*

_{ n−1}=1,

*z*

_{ n }=

*t*, and then using (13), we obtain □

*n*=3,

*a*

_{1}=2 and

*a*

_{2}=4. Then

### Remark 4.8

It would be interesting to find a direct correspondence between lecture hall partitions and the points in the cone considered in Corollary 4.7.

## 5 Cones with symmetry of type *D*

In this section, we consider the case of monoconditional cones with symmetry given by a Coxeter group of type *D*. Unsurprisingly, much of the setup in this section is similar to the hyperoctahedral case; the most notable new feature is that we consider lattice point enumeration with respect to a sublattice of the standard integer lattice.

Throughout this section, let *W* be the finite reflection group of type *D* _{ n−1} on the first *n*−1 components of **R** ^{ n }. Specifically, let *s* _{1}∈O(*n*) be the reflection at the hyperplane {*x*∈**R** ^{ n }∣*x* _{1}+*x* _{2}=0}. For *j*=2,…,*n*−1, let *s* _{ j }∈O(*n*) be the transposition of the (*j*−1)st and *j*th component in **R** ^{ n }. Then *s* _{1},…,*s* _{ n−1} are the simple generators of *W*.

*D*

_{r}(

*σ*) and the action of

*W*explicitly. Let

*π*∈

*S*

_{ n−1}and

*ε*∈

*E*

_{ n−1}, define

*σ*

_{ π,ε }∈O(

*n*) by

*σ*

_{ π,ε }

*e*

_{ i }=

*ε*

_{ i }

*e*

_{ π(i)}for

*i*<

*n*and

*σ*

_{ π,ε }

*e*

_{ n }=

*e*

_{ n }. Then

*W*={

*σ*

_{ π,ε }∣

*π*∈

*S*

_{ n−1},

*ε*∈

*E*

_{ n−1}}. We use the convention that

*π*(

*n*):=

*n*for

*π*∈

*S*

_{ n−1}and

*ε*

_{ n }:=1 for

*ε*∈

*E*

_{ n−1}.

*π*∈

*S*

_{ n−1}and

*ε*∈

*E*

_{ n−1}let

*ε*

_{0}

*π*(0):=−

*ε*

_{2}

*π*(2).

### Proposition 5.1

[5, Proposition 8.2.2]

*For all*

*σ*

_{ π,ε }∈

*W*

*we have*

*P*, we use the symbol

### Proposition 5.2

*Fix integers*

*a*

_{1},…,

*a*

_{ n−1}

*such that*|

*a*

_{1}|≤

*a*

_{2}≤⋯≤

*a*

_{ n−1}≠0.

*Let*

*Let*

*The generating function of*

*C*

*with respect to*

*Γ*

*is*

*where*

*z*

_{1},…,

*z*

_{ n }

*are the coordinates corresponding to the standard lattice*

**Z**

^{ n }⊂

**R**

^{ n }.

Note that the conditions on the *a* _{ i } are normalizations rather than restrictions.

### Proof

*C*is symmetric and monoconditional for

*W*. Let

*W*. Our simple generators

*s*

_{1},…,

*s*

_{ n−1}defined at the beginning of this section are the simple generators of

*W*corresponding to the facets of

*F*. Let

*x*

_{0}:=(

*a*

_{1},…,

*a*

_{ n },−1)∈

*F*. By the proof of Lemma 2.5, where The inverse of

*A*is where

*Σ*

_{ j }:=

*a*

_{ j }+⋯+

*a*

_{ n−1}and \(\varSigma_{2}' := \varSigma_{2} - a_{1}\). Hence the

*Γ*-primitive generators of

*C*

_{+}are the column vectors

*b*

_{1},…,

*b*

_{ n }of the matrix As det(

*B*)=2

^{ n−2}=|

**Z**

^{ n }/

*Γ*| it follows that

*C*

_{+}is unimodular. Note that

*b*

_{1},…,

*b*

_{ n }are enumerated in the unique way such that

*b*

_{ j }∉

*H*

_{ j }for

*j*<

*n*, where

*H*

_{ j }is the reflection hyperplane for the reflection

*s*

_{ j }. Hence by Corollary 2.9 the generating function of

*C*is

*i*th component of

*b*

_{ j }, i.e., the (

*i*,

*j*)th component of

*B*. Thus □

## Notes

### Acknowledgements

We thank the anonymous referees for their thoughtful comments. M.B. is partially supported by the NSF (DMS-0810105 & DMS-1162638). T.B. has been supported by the Deutsche Forschungsgemeinschaft (SPP 1388). B.B. is partially supported by the NSF (DMS-0758321). We are grateful to the American Institute of Mathematics for supporting our SQuaRE working group “Polyhedral Geometry and Partition Theory” where our collaboration on this work began.

## References

- 1.Andrews, G.E., Paule, P., Riese, A.: MacMahon’s partition analysis VII: constrained compositions. In: Berndt, B.C., Ono, K. (eds.)
*q*-Series with Applications to Combinatorics, Number Theory, and Physics, pp. 11–27. American Mathematical Society, Providence (2001) CrossRefGoogle Scholar - 2.Barvinok, A.: Integer Points in Polyhedra. Zurich Lectures in Advanced Mathematics. European Mathematical Society, Zürich (2008). doi: 10.4171/052. MR 2455889 (2011a:52001) CrossRefzbMATHGoogle Scholar
- 3.Beck, M., Gessel, I.M., Lee, S., Savage, C.D.: Symmetrically constrained compositions. Ramanujan J.
**23**, 355–369 (2010). doi: 10.1007/s11139-010-9232-7 MathSciNetCrossRefzbMATHGoogle Scholar - 4.Beck, M., Robins, S.: Computing the Continuous Discretely: Integer-Point Enumeration in Polyhedra. Springer, Berlin (2007). doi: 10.1007/978-0-387-46112-0 Google Scholar
- 5.Bjorner, A., Brenti, F.: Combinatorics of Coxeter Groups. Springer, Berlin (2005). doi: 10.1007/3-540-27596-7 Google Scholar
- 6.Bourbaki, N.: Groupes et algèbres de Lie. Masson, Paris (1981). doi: 10.1007/978-3-540-34491-9. Chapitres 4, 5 et 6 zbMATHGoogle Scholar
- 7.Bousquet-Mélou, M., Eriksson, K.: Lecture hall partitions. Ramanujan J.
**1**(1), 101–111 (1997). doi: 10.1023/A:1009771306380 MathSciNetCrossRefzbMATHGoogle Scholar - 8.Brenti, F.:
*q*-Eulerian polynomials arising from Coxeter groups. Eur. J. Comb.**15**(5), 417–441 (1994). doi: 10.1006/eujc.1994.1046 MathSciNetCrossRefzbMATHGoogle Scholar - 9.Chow, C.-O., Gessel, I.M.: On the descent numbers and major indices for the hyperoctahedral group. Adv. Appl. Math.
**38**, 275–310 (2007). doi: 10.1016/j.aam.2006.07.003 MathSciNetCrossRefzbMATHGoogle Scholar - 10.Humphreys, J.E.: Reflection Groups and Coxeter Groups. Cambridge University Press, Cambridge (1990) CrossRefzbMATHGoogle Scholar
- 11.Pensyl, T.W., Savage, C.D.: Lecture hall partitions and the wreath products
*C*_{k}≀*S*_{n}, submitted (2012), available at http://www4.ncsu.edu/~savage/ - 12.Stanley, R.P.: Ordered Structures and Partitions. Memoirs of the American Mathematical Society, vol. 119. American Mathematical Society, Providence (1972). MR 0332509 (48 #10836) Google Scholar
- 13.Stanley, R.P.: Enumerative Combinatorics. Vol. 1. Cambridge Studies in Advanced Mathematics, vol. 49. Cambridge University Press, Cambridge (1997). With a foreword by Gian-Carlo Rota, Corrected reprint of the 1986 original, MR 98a:05001 CrossRefzbMATHGoogle Scholar
- 14.Steingrímsson, E.: Permutation statistics of indexed permutations. Eur. J. Comb.
**15**(2), 187–205 (1994). doi: 10.1006/eujc.1994.1021 CrossRefzbMATHGoogle Scholar