## Introduction

Ideals generated by binomials define schemes with much simpler geometry than arbitrary polynomial ideals [9], largely yielding to analysis by combinatorial methods [7, 18, 19]. Similarly, ideals homogeneous with respect to some grading or multigrading are simpler than general ideals. In principle, much of this simplicity persists after linear change of coordinates, or any other automorphism of the ambient affine space. Therefore, it seems natural to ask for an algorithm to decide whether a given ideal has any of these properties after applying an ambient automorphism.

Here, we present algorithms for this and related tasks. Let G be an affine algebraic group acting on affine space $$\mathbb A^n = {{\mathrm{Spec\,}}}\Bbbk [x_1,\dots ,x_n]$$ via a morphism $$\alpha : G \times \mathbb A^n \rightarrow \mathbb A^n$$. For a given ideal $$I \subseteq \Bbbk [\mathbf {x}] = \Bbbk [x_1,\dots ,x_n]$$, we provide algorithms for the following tasks, where $$I.\gamma$$ means the image of I under the corresponding (right) action of $$\gamma \in G$$ on $$\Bbbk [\mathbf {x}]$$.

1. (A1)

Find the elements $$\gamma \in G$$ such that $$I.\gamma$$ is generated by binomials (Algorithm 4.2).

2. (A2)

Find the elements $$\gamma \in G$$ such that $$I.\gamma$$ is unital, meaning generated by monomials and differences of monomials (Algorithm 4.5).

3. (A3)

Find the elements $$\gamma \in G$$ such that $$I.\gamma$$ admits a faithful $$\mathbb {Z}^r$$-grading with r as large as possible (Algorithm 3.12).

4. (A4)

Given a second algebraic group T which also acts on $$\mathbb A^n$$, find the elements $$\gamma \in G$$ such that $$I.\gamma$$ is stabilized by as large a subgroup of T as possible (Algorithm 3.10).

In each item, we can in particular determine whether there exists $$\gamma \in G$$ such that $$I.\gamma$$ has the respective property.

The case of $$G = GL _n$$ in (A1) originated with Eisenbud and Sturmfels [9, page 6], who raised the issue of determining when a given ideal is the image of a binomial ideal under an ambient linear automorphism.

It would be desirable to have algorithms for these questions with G being the entire automorphism group of affine space. However, as this group is currently a mystery [28, 30], it seems only fair to require that G be specified beforehand. In fact, the reader will lose none of the flavor or difficulty by assuming that $$G = GL _n$$ consists of all linear changes of coordinates.

The most important class of binomial ideals consists of those that are toric, meaning unital and prime. They define affine toric varieties, so (A2) can be used to find automorphisms in G under which the image of a given affine variety is (equivariantly embedded as) a toric variety. For varieties that are projective, we can even do better and detect whether they are toric without having to specify a group G beforehand:

1. (A5)

Given a normal, projective variety $$X \subseteq \mathbb {P}^n$$, decide if it is toric (Algorithm 5.4).

Even if it is not toric, taking the group T in (A4) to be the algebraic torus $$(\Bbbk ^*)^n$$, our method finds a large subtorus acting on $$V(I.\gamma )$$, turning the latter optimally into a T-variety. (If your definition of T-variety requires normality, then of course it can only work if V(I) is normal to begin with.)

Here are two examples where “hidden” toric structures turned out to be useful.

### Example 1.1

(Phylogenetics and group-based models) Group-based models are special statistical models: maps from the parameter space to the space of probability distributions [8]. In their original coordinates the maps are not monomial, so the Zariski closures of the images do not seem toric. However, a clever linear change of coordinates, known as the Discrete Fourier Transform, turns the varieties to equivariantly embedded toric varieties [14, 27]. This fact inspired numerous mathematicians both in statistics and in algebraic geometry [3, 4, 6, 23].

### Example 1.2

(Secant and tangential varieties of Segre–Veronese) Secant and tangential varieties are classical topics in algebraic geometry [32]. As an example of the difficulty of their geometric and algebraic properties, finding the defining equations of the secant variety of any Segre–Veronese variety was an open conjecture of Garcia, Stillman, and Sturmfels [12], solved only recently by Raicu [26]. Thus, it is surprising that both secant and tangential varieties of Segre–Veronese are covered by open toric varieties—complements of hyperplane sections [21, 22, 29]. Here, a nonlinear change of coordinates, inspired by computation of cumulants in statistics [33] played a crucial role.

### Methods

The principle that guides our algorithms concerns the comparison of families parametrized over a common base. When G acts on $$\mathbb A^n$$, the G-translates of I fiber over G (Definition 3.1 and Remark 3.2). If a second group T acts on $$\mathbb A^n$$ and it is desired to find a subgroup of T that stabilizes a G-translate of $$X \subseteq \mathbb A^n$$, then ask for $$\tau \in T$$ and $$\gamma \in G$$ such that $$\gamma .X$$ is stabilized by $$\tau$$. This problem fibers over $$T \times G$$ (Sect. 3.2), the point being to find the locus $$Y \subseteq T \times G$$ over which $$\tau .(\gamma .X) = \gamma .X$$. In the context of (A4), where the goal is to move X so as to make its stabilizing subgroup as large as possible, the algorithm then finds the locus of points in G over which the fiber of $$Y \rightarrow G$$ has maximal dimension. This locus is closed (Proposition 3.8). Note that Algorithm 3.12 for (A3) is the special case where $$T = (\Bbbk ^*)^n$$ is the algebraic torus acting diagonally on $$\mathbb A^n$$ (Sect. 3.3).

The upshot is that our computational engine consists of two algorithms for a family of ideals over an arbitrary affine base B: find the locus of points in B over which the fiber

1. (A6)

is contained in the fiber of a second given family of ideals over B (Algorithm 2.7) or

2. (A7)

defines a variety of dimension at least d (Algorithm 2.10).

These rely on geometry of constructible sets (Sect. 2.1) and comprehensive Gröbner bases (Sect. 2.2 and Theorem 2.3). Deterministic algorithms for (A7) are known [16], but Algorithm 2.10 is probabilistic and likely faster.

Having already the context of an arbitrary affine base B at our disposal, the algorithms for (A1) and (A2) work as well for a family of ideals over B.

(A1$$'$$):

Find the locus of points in B over which the fiber is binomial (Algorithm 4.2).

(A2$$'$$):

Find the locus of points in B over which the fiber is unital (Algorithm 4.5).

The reason is that the criterion for binomiality simply detects whether the reduced Gröbner basis is binomial (Sect. 4.1); it has nothing to do with a group action on the ambient affine space. Similarly, an ideal is unital precisely when its scheme is closed under coordinatewise multiplication (Proposition 4.4), while this employs the monoid structure on $$\Bbbk ^n$$, which is defined only once a basis of $$\Bbbk ^n$$ has been given, that monoid action is fixed from the outset, so it remains only to calculate which fibers respect it.

The final application, (A5), observes that high Veronese embeddings of normal projective varieties are projectively normal, after which an ambient automorphism must make the variety toric if anything can (Theorem 5.5). Thus our method applies (A2) with G being the general linear group acting on projective space. A speedup (Algorithm 5.1) for (A6) is available in this case because the fibers are known to be prime.

### Conventions

Everything throughout the paper is over a field $$\Bbbk$$. Implementation of the algorithms would require that $$\Bbbk$$ be “computable” in some appropriate sense, but most of our discussions are independent of this hypothesis. That said, we do assume, without further comment, that all algebras and schemes are of finite type over $$\Bbbk$$. Moreover, unless stated otherwise we assume all schemes to be affine. See Remark 6.4 for possible generalizations of our results to more general schemes.

Functions, variables, and spaces are denoted by English letters. Group elements or points in spaces are denoted by Greek letters. Thus $$f(\mathbf {x})$$ for $$\mathbf {x}= x_1,\dots ,x_n$$ is a function on a subscheme X of affine n-space $$\mathbb A^n$$ whose action on a $$\Bbbk$$-valued point $$\xi \in X$$ is $$\xi \mapsto f(\xi )$$. Hopefully this eliminates confusion regarding left vs. right actions on spaces vs. functions, the details of which are covered in Sect. 3.1. When we wish to think of points algebraically, as prime ideals in rings, then we use Fraktur letters. Thus $$\mathfrak {p}_\gamma \subseteq \Bbbk [G]$$ is the prime ideal corresponding to the point $$\gamma \in G$$. For any prime ideal $$\mathfrak {p}$$, its residue field is written $$\kappa _\mathfrak {p}$$. If $$\mathfrak {p}= \mathfrak {p}_\xi$$, say, then we also write $$\kappa _\xi = \kappa _{\mathfrak {p}_\xi }$$.

## Algorithms for Families of Schemes

Generally speaking, our algorithms are aimed at schemes over groups, thought of as families of schemes (or ideals) parametrized by the group. But many of our results hold over more arbitrary base schemes; we phrase those in terms of a commutative finitely generated $$\Bbbk$$-algebra S. The polynomial ring $$S[\mathbf {x}]$$, where $$\mathbf {x}= x_1,\dots ,x_n$$ denotes the sequence of variables, has spectrum $$\mathbb A^n_S = \mathbb A^n \times {{\mathrm{Spec\,}}}S$$, the affine space of dimension n over (the spectrum of) S. An ideal $$J \subseteq S[\mathbf {x}]$$ corresponds to a subscheme $$X \subseteq \mathbb A^n_S$$, usually thought of as a family of subschemes of $$\mathbb A^n = \mathbb A^n_\Bbbk$$ parametrized by (the $$\Bbbk$$-valued points of) $${{\mathrm{Spec\,}}}S$$, or as a family of ideals of $$\Bbbk [\mathbf {x}]$$. But if $$\mathfrak {p}$$ is any prime ideal of S, maximal or not, then the ideal defining the fiber $$X_\mathfrak {p}\subseteq \mathbb A^n_\mathfrak {p}$$ over $$\mathfrak {p}$$ is a specialization of J, namely the extension $$J\kappa _\mathfrak {p}[\mathbf {x}]$$ of J to the polynomial ring over the residue field $$\kappa _\mathfrak {p}= S_\mathfrak {p}/\mathfrak {p}S_\mathfrak {p}$$ of $$\mathfrak {p}$$.

### Constructible Sets

Let S be a finitely generated commutative $$\Bbbk$$-algebra. A subset of $${{\mathrm{Spec\,}}}S$$ is constructible if it is a finite union

\begin{aligned} \bigcup _{i = 1}^s U_i \cap C_i, \end{aligned}

where each $$U_i \subseteq {{\mathrm{Spec\,}}}S$$ is open and each $$C_i \subseteq {{\mathrm{Spec\,}}}S$$ is closed. We assume access to algorithms that

1. 1.

compute unions and intersections of constructible sets, and

2. 2.

determine whether a constructible set is empty.

See [2], for example.

For the sake of completeness, we recall an algebraic version of Baire’s theorem needed in the proof of Lemma 2.2, which is in turn used in the proof of Proposition 4.6.

### Theorem 2.1

(Baire’s Theorem) No irreducible scheme X of finite type over an uncountable field $$\Bbbk$$ is, as a topological space, a countable union of closed proper subsets.

### Proof

As only the topology is in play, there is no harm in assuming that all of the schemes involved are reduced. Intersecting each subset in such a union $$\bigcup _{i=1}^\infty Z_i$$ with the members of an affine open cover reduces to the case where X is affine. Hence X is a closed subscheme of $$\mathbb A^n$$, and its coordinate ring is a domain because it is reduced and irreducible. Noether normalization therefore reduces to the case $$X = \mathbb A^n$$, because it guarantees a finite surjective morphism to $$\mathbb A^{\dim X}$$ while preserving the fact that each $$Z_i$$ is a closed proper subset.

The goal is to show that $$\bigcup _{i=1}^\infty Z_i$$ is a proper subset of $$\mathbb A^n$$ given that each $$Z_i$$ is a proper closed subset. The proof is by induction on n, the case $$n = 1$$ following from the uncountability of $$\Bbbk$$. Let H be a hyperplane that is not contained in any of the (countably many) irreducible components of the $$Z_i$$; such an H exists—simplest is to choose it parallel to some given hyperplane—because $$\Bbbk$$ is uncountable and each $$Z_i$$ contains only finitely many hyperplanes parallel to any given one. The induction is concluded by noting that $$H \cap Z_i$$ is a proper closed subscheme of H for all i, so $$H \ne \bigcup _{i=1}^\infty H \cap Z_i$$. $$\square$$

### Lemma 2.2

If the field $$\Bbbk$$ is uncountable, then no countably infinite union of disjoint nonempty constructible sets is constructible.

### Proof

Let $$U = \bigcup _{i=1}^\infty Z_i$$ be the union of disjoint nonempty constructible sets $$Z_i$$. The proof is inductive on $$\dim {\overline{U}}$$, the case $$\dim {\overline{U}} = 0$$ being trivial.

For contradiction, suppose U is a finite union of intersections $$C_j \cap O_j$$ of closed irreducible sets $$C_j$$ and open sets $$O_j$$. One of the intersections, say $$C_1 \cap O_1$$, must intersect infinitely many $$Z_i$$ nontrivially. Let $$Z_i' := Z_i \cap C_1$$. Then $$C_1 = (C_1 \smallsetminus O_1) \cup \bigcup _{i=1}^\infty Z_i'$$. Theorem 2.1 implies that one of the sets $$\overline{Z_i'}$$ must equal $$C_1$$. By considering the constructible set $$C_1\setminus Z_i'$$ we conclude by induction on dimension. $$\square$$

### Comprehensive Gröbner Bases

The notion of comprehensive Gröbner basis [25, 31] adapts to our case as follows. Let S be a domain which is a finitely generated commutative $$\Bbbk$$-algebra and let $$I \subseteq S[\mathbf {x}]$$ be an ideal. Fix any term order. Buchberger’s algorithm for I over the generic point—that is, in the polynomial ring $$\kappa _0[\mathbf {x}]$$ over the fraction field $$\kappa _0$$ of S—finds a finite set of polynomials $$g_j \in I\kappa _0[\mathbf {x}]$$ that constitute a reduced Gröbner basis. Step i of the algorithm needs to assume that a leading coefficient $$f_i\in S$$ is nonzero. Consider the ideal $$J = \big \langle \prod _i f_i\big \rangle \subseteq S$$. Over any point $$\mathfrak {p}\in {{\mathrm{Spec\,}}}S \smallsetminus V(J)$$, the reduction of $$g_j$$ is the reduced Gröbner basis for the reduction of I. Indeed, over such a point the usual Buchberger algorithm makes exactly the same steps as the algorithm run over the generic point. Repeating the procedure for each irreducible component of V(J) yields:

• a partition of $${{\mathrm{Spec\,}}}S$$ into irreducible, relatively open sets $$U_i$$; and

• for each $$U_i$$ a finite set of polynomials $$g_{ij}$$ in I that specializes over any point of $$U_i$$ to a reduced Gröbner basis of the reduction of I.

This data structure is a relative reduced Gröbner basis. We further assume that

• $$U_i$$ are irreducible and

• the leading coefficients of $$(g_{ij})$$ do not vanish on $$U_i$$, and hence by inverting them that they are equal to 1.

### The Locus of Fiber Containment

For two schemes fibered over a fixed base scheme B, the methods in later sections rely on an ability to compute the locus of points in B where the fibers of the first scheme are contained in the fibers of the second one. In an affine setting, this locus is described by Theorem 2.3.

### Theorem 2.3

Let S be a finitely generated commutative $$\Bbbk$$-algebra. Given two ideals $$I_1, I_2 \subseteq S[\mathbf {x}]$$, the set of points $$\big \{\mathfrak {p}\in {{\mathrm{Spec\,}}}S \mid I_1\kappa _\mathfrak {p}[\mathbf {x}] \subseteq I_2\kappa _\mathfrak {p}[\mathbf {x}]\big \}$$ is constructible in $${{\mathrm{Spec\,}}}S$$. If $$I_2$$ is an extension of an ideal from $$\Bbbk [\mathbf {x}]$$, then this set is closed.

### Proof

Let $$f_1, \dots , f_r$$ generate $$I_1$$. Then $$I_1\kappa _\mathfrak {p}[\mathbf {x}] \subseteq I_2\kappa _\mathfrak {p}[\mathbf {x}]$$ if and only if $$I_1 S_\mathfrak {p}[\mathbf {x}] + \mathfrak {p}S_\mathfrak {p}[\mathbf {x}] \subseteq I_2 S_\mathfrak {p}[\mathbf {x}] + \mathfrak {p}S_\mathfrak {p}[\mathbf {x}]$$, and this is in turn equivalent to $$f_i \in I_2 S_\mathfrak {p}[\mathbf {x}] + \mathfrak {p}S_\mathfrak {p}[\mathbf {x}]$$ for all i. Therefore, it suffices to treat the case where $$I_1$$ is principal, say $$I_1 = \langle f\rangle$$.

Let $$(U_i,(g_{ij}))$$ be the relative reduced Gröbner basis for $$I_2$$ introduced in Sect. 2.2. To prove the first part of the theorem, fix an arbitrary stratum $$U_{i_0}$$. In analogy to the algorithm computing the comprehensive Gröbner basis, reduce f modulo $$(g_{i_0j})_j$$. This yields a presentation of the reduction of f as a combination of standard monomials with coefficients represented by elements of localizations of S. In $$U_{i_0}$$, the locus $$\mathfrak {p}\in {{\mathrm{Spec\,}}}S$$ over which $$f\in I_2 S_\mathfrak {p}[\mathbf {x}] + \mathfrak {p}S_\mathfrak {p}[\mathbf {x}]$$ is the closed set given by vanishing of all (numerators of) these coefficients of standard monomials. This finishes the proof of the first statement. Further, note that the locus of interest is closed in $$U_{i_0}$$.

If $$I_2$$ is an extension of an ideal from $$\Bbbk [\mathbf {x}]$$ then the relative reduced Gröbner basis is just the (extension of) the usual reduced Gröbner basis in $$\Bbbk [\mathbf {x}]$$. As before it suffices to assume that $$I_1$$ is principal. Here $$U_{i_0} = {{\mathrm{Spec\,}}}S$$ and the result is a closed set. $$\square$$

### Definition 2.4

The constructible set in Theorem 2.3 is the containment locus for $$I_1$$ in $$I_2$$ (or for $$X_2$$ in $$X_1$$, where $$I_i = I(X_i)$$). Its intersection with the containment locus for $$I_2$$ in $$I_1$$ is the coincidence locus of $$I_1$$ and $$I_2$$ (or of $$X_1$$ and $$X_2$$).

### Corollary 2.5

Let $$B = {{\mathrm{Spec\,}}}S$$ for a finitely generated commutative $$\Bbbk$$-algebra S. Set $$\mathbb A^n_B = \mathbb A^n_S$$. Let $$X \subseteq \mathbb A^n$$ be a subscheme and $$X_1 = X \times B \subset \mathbb A^n_B$$ be the constant family over B. Then the containment locus of $$X_1$$ in $$X_2$$ is closed for any family $$X_2 \subseteq \mathbb A^n_B$$ over B. More generally, if there exists an isomorphism $$\iota : \mathbb A^n_B \rightarrow \mathbb A^n_B$$ of schemes over B that transforms $$X_1$$ to the constant family, then the containment locus is closed.

### Proof

In the constant family case, the ideal of $$X_1$$ is simply (the extension of) the ideal of X. Therefore, this case follows immediately from Theorem 2.3. The last statement follows by first applying $$\iota$$ and then appealing to the case of the constant family. $$\square$$

### Example 2.6

The loci considered in Theorem 2.3 need not be closed. Indeed, let $$S = \Bbbk [s,t]$$ and $$I_1, I_2 \subseteq S[x]$$ be defined by $$I_1 = \langle x\rangle$$ and $$I_2 = \langle sx - t\rangle$$. Geometrically, $$V(I_1)$$ is just the st-plane, while $$V(I_2)$$ is the affine part of a blow-up of this plane at the origin. The fibers coincide exactly when $$t = 0$$ and $$s \ne 0$$.

We now present an algorithmic version of Theorem 2.3.

### Algorithm 2.7

Compute the containment locus for two families over same base

### Remark 2.8

In the situation of Theorem 2.3, the following stronger statement holds: Let $$\tilde{I}_2$$ denote the homogenization of $$I_2$$ with respect to a new variable $$x_0$$. If $$S[x_0, \mathbf {x}]/\tilde{I}_2$$ is flat over S, then the set $$\{\mathfrak {p}\in {{\mathrm{Spec\,}}}S \mid I_1\kappa _\mathfrak {p}[\mathbf {x}] \subseteq I_2\kappa _\mathfrak {p}[\mathbf {x}]\}$$ is closed in $${{\mathrm{Spec\,}}}S$$.

We only sketch the proof because the result is not used in later sections. Without loss of generality, assume that $$I_1$$ is principal, with generator $$f \in S[\mathbf {x}]$$. Denote by $$\tilde{f}$$ its homogenization. The first step is to show that there is a nonnegative integer $$N \in \mathbb {N}$$—a constant that depends on the ring S, the ideal $$I_2$$, and the generator f—such that for all $$\mathfrak {p}\in {{\mathrm{Spec\,}}}S$$, the equivalence $$x_0^N \tilde{f} \in \tilde{I}_2 S_\mathfrak {p}[x_0,\mathbf {x}] + \mathfrak {p}S_\mathfrak {p}[x_0,\mathbf {x}] \Leftrightarrow f \in I_2 S_\mathfrak {p}[\mathbf {x}] + \mathfrak {p}S_\mathfrak {p}[\mathbf {x}]$$ holds. Existence of this integer N can be proved by induction on $$\dim (S)$$ using a relative reduced Gröbner basis for $$\tilde{I}_2$$.

Next, observe that the question of whether $$x_0^N\tilde{f}$$ lies in $$\tilde{I}_2$$ is equivalent to solvability of a certain inhomogeneous system of linear equations with coefficients in S. Such a system is solvable if and only if the rank of the coefficient matrix equals the rank of the “extended matrix” that is obtained by appending the right-hand side of the system to the coefficient matrix. Since rank is determined by vanishing of minors, the locus where the system is solvable is constructible. (This yields another proof of the first part of Theorem 2.3.)

Finally, the rank of the coefficient matrix equals the value of the Hilbert function of $$S[x_0,\mathbf {x}]/\tilde{I}_2$$ in the degree of $$x_0^N \tilde{f}$$. The assumption that $$S[x_0, \mathbf {x}] / \tilde{I}_2$$ is flat implies that this Hilbert function is the same over every point $$\mathfrak {p}$$, so the rank of the matrix cannot drop, and the set of points where the system is solvable is therefore closed.

### The Locus of Large Fibers

Given a morphism of schemes $$X \rightarrow B$$, we need to compute the locus $$B^{\ge d} \subset B$$ of fibers of dimension at least d. A deterministic algorithm for this task was given by Kemper [16]. The probabilistic algorithm presented here is therefore not theoretically required for algorithms in the rest of paper, but it is much easier to implement and we expect it to run faster. In particular, Algorithm 2.10 has allowed us to compute explicit examples, some of which are presented here.

Note on conventions: Algorithm 2.10 and its proof assume (and implicitly use) that the field $$\Bbbk$$ is algebraically closed. The engine of the algorithm—and the only interesting part—is the following subroutine that takes an affine scheme affine over an affine base as input; it inherits the hypotheses from its parent algorithm where it is applied.

### Algorithm 2.9

Find locus of big fiber dimension, affine case

### Algorithm 2.10

Find locus of big fiber dimension, general case

### Proposition 2.11

Algorithm 2.10 is correct.

### Proof

It suffices to prove Algorithm 2.9 works on affine morphisms of affine schemes, because a fiber is large if and only if it is large in (at least) one member of an open cover of the source. So assume $$X \subseteq \mathbb A^n_B$$. Fix an irreducible component Z of $$B^{\ge d}$$.

### Claim

For a general point $$z\in Z$$ and a general affine subspace $$L \subseteq \mathbb A^n$$ of codimension d, the fiber $$f^{-1}(z)$$ intersects L.

### Proof of the claim

This is deduced by taking closures in projective space, where subvarieties of complementary dimension always intersect. Of course the projective closure of L is only guaranteed to intersect the projective closure of the fiber, but the difference between an affine algebraic subset and its projective closure has lower dimension.

The claim implies that $$(L' \times B) \cap X$$ in Algorithm 2.9 contains general points of Z, so $$B^{\ge d} \subseteq B_i$$. It remains to prove that the output $$B_i$$ cannot be strictly bigger. Consider a component Q of any $$B_i$$ (not necessarily the output one). Assume the fiber over a general point of Q has dimension smaller than d. To finish, we have to prove that $$B_{i+1} \ne B_i$$. Consider a fiber F over a general point $$q \in Q$$. Since $$\dim F < d$$, a general subspace of codimension d in $$\mathbb {P}^n$$ fails to meet the closure of F in $$\mathbb {P}^n$$, and this remains true in a neighborhood of q. Thus the general point of Q is not in the projection of $$L' \cap X$$. Hence Q is not a component of $$B_{i+1}$$. $$\square$$

### Remark 2.12

Most of the algorithms in this paper are too high-level to warrant specifying precise algebraic formats for their input or output. For example, the contents of Algorithms 2.9 and 2.10 do not care what form the input morphisms take, or precisely how the sets $$B_i$$ are presented as output. A morphism of schemes can in general be specified via open covers of the source and target by spectra of rings, with compatible ring homomorphisms. An affine morphism might be handed to Algorithm 2.9 this way, even if there exists a simpler presentation in which the spectra covering the source are in bijection with the spectra covering the target. Furthermore, an arbitrary scheme morphism might not explicitly be given by open covers at all; it might be specified by a homorphism of graded rings in projective cases, or multigraded rings in toric situations, or some combination of ring inclusions and homomorphisms in a context of cluster algebras, or some other method specific to the situation at hand. The point is that (for example) Algorithm 2.10 only requires that an open cover in the compute step can in fact be calculated; it does not provide instruction on how the geometric operations are actually carried out. It would obscure the mathematical and computational content to place restrictions on the input or output formats.

Alas, the high-level nature of our pseudocode makes complexity analysis difficult or impossible. But since the algorithms ultimately rely on Gröbner bases, their worst-case performance has hopelessly bad complexity anyway. Note that taken for granted are standard methods of computational algebra, like checking normality of rings, or computing homogenizations and radicals of ideals, implementations of which can be found in, for example, Macaulay2 [11]. The problem is exacerbated by the need to specify data structures for algebras over fields: Algorithm 4.2, for instance, does not care whether S has been given as a quotient ring, a subalgebra, or an image of a ring homomorphism.

### Remark 2.13

The algorithms in this paper do not assume that constructible sets are presented using radical ideals. In particular, the algorithm outputs might not be radical ideals. However, when comparing two constructible sets to discover containment or equality, it is sometimes simplest to take radicals. In the case of Algorithm 2.9, radicals could be used (this occurs explicitly in Example 5.3, which uses Algorithm 2.10); but for us it was faster to first run the subroutine a few times—with different linear forms—without computing radicals, and only later compare sets by computing radicals.

## Detecting Big Group Actions and Multigradings

### Group Actions and Families

Let G be an algebraic group (over $$\Bbbk$$, as always), acting on affine n-space $$\mathbb A^n$$ over $$\Bbbk$$. The action is a morphism $$\alpha : G \times \mathbb A^n \rightarrow \mathbb A^n$$, and it is assumed to be a left action, so $$\gamma .(\gamma '.\xi ) = (\gamma \gamma ').\xi$$ for points $$\gamma ,\gamma ' \in G$$ and $$\xi \in \mathbb A^n$$ over $$\Bbbk$$.

The geometric action on $$\mathbb A^n$$ is equivalent to an algebraic action on $$\Bbbk [\mathbf {x}]$$; for $$\gamma \in G$$ and $$f \in \Bbbk [\mathbf {x}]$$, the function $$f.\gamma$$ sends $$\xi \mapsto f(\gamma .\xi )$$ for all $$\xi \in \mathbb A^n$$. As the group G acts on the affine space one may assume that G is affine [1, Corollary 3.2.2]. Then, $$\alpha$$ induces a ring homomorphism $$\alpha ^*: \Bbbk [\mathbf {x}] \rightarrow \Bbbk [G] \otimes \Bbbk [\mathbf {x}]$$, where $$\Bbbk [G]$$ is the coordinate ring of G (not its group algebra over $$\Bbbk$$), satisfying the axioms dual to the group action axioms. For any point $$\gamma \in G$$ let $${{\mathrm{ev}}}_\gamma : \Bbbk [G] \rightarrow \kappa _\gamma$$ be the evaluation map, meaning the algebra morphism corresponding to the inclusion $$\{\gamma \} \hookrightarrow G$$. Then for a $$\Bbbk$$-valued group element $$\gamma$$, the composition $$({{\mathrm{ev}}}_\gamma \otimes {{\mathrm{id}}}) \circ \alpha ^*$$ gives a map $$\alpha ^\dagger : G \rightarrow {{\mathrm{End}}}(\Bbbk [\mathbf {x}],\Bbbk [\mathbf {x}])$$, and $$f.\gamma = \alpha ^\dagger (\gamma )(f)$$. For any ideal $$I \subseteq \Bbbk [\mathbf {x}]$$ and point $$\gamma \in G$$ over $$\Bbbk$$, set $$I.\gamma = \{f.\gamma \mid f \in I\}$$.

In the setting of schemes over G, the crucial families are the orbits. To define them, one more bit of general notation helps: for any two schemes X and B over $$\Bbbk$$, write $$X_B = B \times X$$ and consider it as a family over B. The reader will lose little by thinking always of $$B = G$$, as in the following definition.

### Definition 3.1

The orbit morphism is $$\omega := \pi _G \times \alpha : \mathbb A^n_G \rightarrow \mathbb A^n_G$$, where $$\pi _G$$ is the projection to G. If $$X \subseteq \mathbb A^n$$ is a subscheme, then its orbit is $$\mathcal {O}_G X = \omega (X_G)$$.

### Remark 3.2

On $$\Bbbk$$-valued points, the orbit morphism is $$(\gamma ,\xi ) \mapsto (\gamma ,\gamma .\xi )$$. Geometrically, the orbit of X is a family over G whose fibers are the translates of X by group elements. The terminology comes from the case where X is a point $$\xi$$, because the projection of $$\mathcal {O}_G \xi$$ to $$\mathbb A^n$$ is indeed the G-orbit of $$\xi$$.

More formally, if $$X \subseteq \mathbb A^n$$ is a subscheme, then $$\gamma .X$$ is the fiber of $$\mathcal {O}_G X$$ over $$\gamma \in G$$. Algebraically, X is defined by an ideal $$I \subseteq \Bbbk [\mathbf {x}]$$, and $$\mathcal {O}_G X$$ is defined by the ideal $$J \subseteq S[\mathbf {x}]$$, where $$S = \Bbbk [G]$$ and $$J = (\omega ^*)^{-1}(I)$$. The ideal $$I.\gamma ^{-1}$$ defining the subscheme $$\gamma .X$$ of the affine space $$\mathbb A^n_\gamma$$ over the residue field $$\kappa _\gamma$$ is a specialization of J, namely the extension $$J\kappa _\gamma [\mathbf {x}]$$ of J to the polynomial ring over the residue field of $$\gamma$$.

### Remark 3.3

Images of ideals are much easier to compute than preimages. Therefore, in order to compute the ideal J above in a concrete case, it might be a good idea to first compute the inverse map of $$\omega$$, which amounts to computing the map $$\gamma \mapsto \gamma ^{-1}$$ on G, and then obtain J as push-forward of I along that map.

### Remark 3.4

The locus where an orbit is contained in any given family is always closed, by Corollary 2.5.

### Finding Big Group Actions

In this section, T is an algebraic group acting on affine space via a map $$\beta : T \times \mathbb A^n \rightarrow \mathbb A^n$$. We are interested in finding those elements of G which make a given subscheme $$X \subseteq \mathbb A^n$$ invariant under T, or invariant under a subgroup of T whose dimension is as big as possible. For terminology, we say that a group acting on a space stabilizes a subspace if the subspace is preserved by the action (not necessarily pointwise); in contrast, we say that the group fixes a subspace if every point in the subspace is fixed by the group action.

### Example 3.5

Let $$T = (\Bbbk ^*)^n$$ be the n-dimensional torus acting diagonally on $$\mathbb A^n$$. If $$X \subseteq \mathbb A^n$$ is a nondegenerate subvariety (reduced irreducible subscheme not contained in a hyperplane) such that for some $$\gamma \in GL(n)$$ the subvariety $$\gamma .X \subseteq \mathbb A^n$$ is stabilized by a subtorus $$T' \subset T$$ with $$\dim T' = \dim X$$, then by some definitions $$\gamma .X$$ is already toric, and by others it becomes so after further rescaling the variables [9, Corollary 2.6]. In the latter case, $$\gamma '.X$$ is toric for some other $$\gamma ' \in GL(n)$$. (Yet other definitions require X to be normal; we do not.)

### Lemma 3.6

The stabilizer of any ideal $$I \subseteq \Bbbk [\mathbf {x}]$$ is a closed subgroup of T.

### Proof

For the scheme $$X \subseteq \mathbb A^n$$ defined by I, the containment loci both for $$X_T$$ in $$\mathcal {O}_T X$$ and for $$\mathcal {O}_T X$$ in $$X_T$$ are closed by Corollary 2.5, so their coincidence locus is closed. $$\square$$

### Remark 3.7

The two containment loci in the proof of Lemma 3.6 are in fact equal, because $$I.\tau \subseteq I \Rightarrow I.\tau = I$$ for $$\tau \in T$$. Indeed, $$I.\tau \subsetneq I$$ implies $$I.\tau ^i \subsetneq I.\tau ^{i-1}$$ for all $$i \in \mathbb {Z}$$ (including negative i), contradicting the noetherian property of $$\Bbbk [\mathbf {x}]$$.

If a subgroup $$T' \subseteq T$$ stabilizes a subscheme $$X \subseteq \mathbb A^n$$, then the action of T restricts to an action of $$T'$$ on X. We now consider the question for which $$\gamma \in G$$ there is a large subgroup of T stabilizing $$\gamma .X$$.

### Proposition 3.8

Let $$X \subseteq \mathbb A^n$$ be a closed subscheme. Let further $$G' \subseteq G$$ be the locus of those $$\gamma \in G$$ where the dimension of the stabilizer subgroup $$T(\gamma .X) \subseteq T$$ is maximal. Then $$G'$$ is closed.

### Proof

Let $$Y \subseteq T \times G$$ be the coincidence locus for $$\mathcal {O}_T(\mathcal {O}_G X)$$ and $$(\mathcal {O}_G X)_T$$, viewed as schemes over $$T \times G$$. The fiber $$Y_\gamma$$ of Y over $$\gamma \in G$$ is the stabilizer $$T(\gamma .X)$$ of $$\gamma .X$$ in T. The subset $$Y \subseteq T \times G$$ is closed by Corollary 2.5, and $$G'$$ is the locus of points $$\gamma \in G$$ such that $$Y_\gamma$$ has maximal dimension.

If $$d(\gamma ) = \dim Y_\gamma$$ then $$d(\gamma )$$ is the local dimension of $$Y_\gamma$$ at $$\tau$$ for all $$\tau \in Y_\gamma$$, in particular at $$\tau = 1_T$$, because $$Y_\gamma$$ is a subgroup of T. Upper semicontinuity of fiber dimension locally on the source [10, 13.1.3] implies that the subset $$Z \subseteq Y$$ with maximal local fiber dimension is closed in Y. Therefore $$1_T \times G' = Z \cap (1_T \times G)$$ is closed in the identity section $$1_T \times G$$. The result follows because $$G \rightarrow 1_T \times G$$ is an isomorphism. $$\square$$

### Example 3.9

Proposition 3.8 says that stabilizer dimensions for fibers of orbits $$\mathcal {O}_G X_G$$ are upper semicontinuous. In contrast, semicontinuity can fail for families that are not orbits. The simplest instance has $$T = \Bbbk ^*$$, $$S = \Bbbk [s]$$, and $$I = \langle sx, x(x-1)\rangle \subset S[x]$$. Then V(I) is the union of the s-axis with the point (0, 1). So for $$s \ne 0$$, V(I) is stable under T, while for $$s = 0$$ it is only stable under the trivial subgroup of T. Thus, the locus where the stabilizer has maximal dimension is not closed.

### Algorithm 3.10

Act to make stabilizing subgroup of maximal dimension

A $$\mathbb {Z}^r$$-multigrading on $$\Bbbk [\mathbf {x}]$$ is specified by n vectors $$\mathbf {a}_1, \dots , \mathbf {a}_n \in \mathbb {Z}^r$$ to serve as degrees of the variables: $$\deg x_i = \mathbf {a}_i$$. The multigrading is faithful if $$\mathbf {a}_1, \dots , \mathbf {a}_n$$ generate $$\mathbb {Z}^r$$.

### Remark 3.11

Details on multigraded algebra in general can be found in [24, Chapter 8]. A $$\mathbb {Z}^r$$-multigrading on $$\Bbbk [\mathbf {x}]$$ corresponds uniquely to the action on $$\Bbbk ^n$$ of the r-torus $$(\Bbbk ^*)^r$$. (References for this are hard to locate. An exposition appears in Appendix A.1 of the first arXiv version of [17], at http://arxiv.org/abs/math/0110058v1.) An abelian group homomorphism $$\mathbb {Z}^n \rightarrow \mathbb {Z}^r$$ sending the standard basis to $$\mathbf {a}_1, \dots , \mathbf {a}_n$$ corresponds (by applying the $${{\mathrm{Hom}}}_\mathbb {Z}(-,\Bbbk ^*)$$ functor) to an algebraic group homomorphism $$(\Bbbk ^*)^r \rightarrow (\Bbbk ^*)^n$$ that is injective precisely when the multigrading is faithful. The assertion that $$\deg x_i = \mathbf {a}_i$$ means that the $$j^thsps$$ generator $$\tau _j$$ of the r-torus acts on $$x_i$$ by $$x_i.\tau _j = \tau _j^{a_{ij}} x_i$$. Geometrically speaking, multigraded (i.e., homogeneous) ideals correspond to subschemes of $$\Bbbk ^n$$ that carry $$(\Bbbk ^*)^m$$-actions.

The following is the detailed algebraic phrasing of Algorithm 3.10 when T is the standard algebraic n-torus acting diagonally on $$\Bbbk ^n$$.

### Algorithm 3.12

Act to find a faithful multigrading of maximal rank

## Detecting Binomial and Unital Ideals

This section presents algorithms to decide whether $$I \subseteq \Bbbk [\mathbf {x}]$$ can be made binomial or unital by an automorphism of $$\Bbbk [\mathbf {x}]$$. Let us recall the relevant definitions.

### Definition 4.1

A binomial is a polynomial $$\mathbf {x}^\mathbb A- \lambda \mathbf {x}^\mathbf {b}$$ for some $$\lambda \in \Bbbk$$ and $$\mathbf {a},\mathbf {b}\in \mathbb {N}^n$$. An ideal in $$\Bbbk [\mathbf {x}]$$ is

1. 1.

binomial if it is generated by binomials, and

2. 2.

unital (cf. [18]) if it is generated by monomials and differences of monomials, meaning binomials $$\mathbf {x}^\mathbb A- \lambda \mathbf {x}^\mathbf {b}$$ with $$\lambda \in \{0,1\}$$.

3. 3.

toric if it is unital and prime.

### Algorithm to Locate Binomial Models

The following algorithm is based on comprehensive Gröbner bases (Sect. 2.2). At each stratum, it forces all coefficients except for the leading coefficient and (at most) one other to be 0; more precisely, it computes the locus where this is possible. The result is the binomial locus because an ideal is binomial if and only if some (equivalently, every) reduced Gröbner basis consists of binomials [9, Corollary 1.2].

### Algorithm 4.2

Find the locus of binomial fibers

### Algorithm to Locate Unital Models

It is possible for a group G to have the power to transform a given ideal I into binomial form without G being able to transform I into unital form, even though a larger group G could succeed in making I unital. Trivial examples exist: any principal nonunital binomial ideal with $$G = \{1\}$$ suffices. But it is even possible for G to contain the entire torus. (See also Example 4.8.)

### Example 4.3

Consider the binomial ideal $$I = \langle xz, z(z-1), z(y-2), x(x-1), x(y-1)\rangle$$. Its variety consists of the two points $$p_1 = (0,2,1)$$ and $$p_2 = (1,1,0)$$ and the line $$\{(0,\xi ,0) \mid \xi \in \mathbb {C}\}$$. The map that fixes x and z but sends $$y \mapsto y+z$$ makes I unital.

On the other hand, scaling by $$\tau \in (\mathbb {C}^*)^3$$ is unable to make I unital. Indeed, if $$\tau \in (\mathbb {C}^*)^3$$ then $$y(\tau .p_1) \ne 1$$ or $$y(\tau .p_2) \ne 1$$. As both cases are analogous, say $$y(\tau .p_1) \ne 1$$ and thus $$\tau .p_1 \ne (0,1,1)$$. Any monomial evaluated on (0, 1, 1) is either 0 or 1, and it is equal to 0 if and only if it is equal to 0 when evaluated on $$\tau .p_1$$. Thus if two monomials are equal when evaluated on $$\tau .p_1$$ then they are also equal when evaluated on (0, 1, 1). Hence, any unital binomial or monomial that vanishes on $$\tau .p_1$$ vanishes also on (0, 1, 1). If $$I.\tau$$ were unital, this would contradict that $$\tau .p_1$$ is the only point with $$z \ne 0$$.

The main idea of our algorithm is that unital ideals I can be characterized by the fact that the variety V(I) is closed under coordinatewise multiplication. It turns out to be more convenient to work with the algebraic counterpart of the multiplication, which is the diagonal map $$\Delta$$.

Precisely, let $$\Delta : \Bbbk [\mathbf {x}] \rightarrow \Bbbk [\mathbf {x}] \otimes _\Bbbk \Bbbk [\mathbf {x}]$$ be the algebra homomorphism defined by $$\Delta (x_i) = x_i \otimes x_i$$. This makes $$\Bbbk [\mathbf {x}]$$ into a bialgebra. The induced map $$\Delta ^*: \mathbb A^n \times \mathbb A^n \rightarrow \mathbb A^n$$ is the coordinatewise multiplication map.

### Proposition 4.4

The following are equivalent for an ideal $$I \subseteq \Bbbk [\mathbf {x}]$$.

1. 1.

I is unital.

2. 2.

I is a coideal with respect to $$\Delta$$; that is, $$\Delta (I) \subseteq \Bbbk [\mathbf {x}] \otimes _\Bbbk I + I \otimes _\Bbbk \Bbbk [\mathbf {x}]$$.

### Proof

The implication “1 $$\Rightarrow$$ 2” follows from the definitions, because

\begin{aligned} \Delta (\mathbf {x}^\mathbf {a})&=\mathbf {x}^\mathbf {a}\otimes \mathbf {x}^\mathbf {a}\in \Bbbk [\mathbf {x}] \otimes _\Bbbk I + I \otimes _\Bbbk \Bbbk [\mathbf {x}],\\ \Delta (\mathbf {x}^\mathbf {a}- \mathbf {x}^\mathbf {b})&=\mathbf {x}^\mathbf {a}\otimes (\mathbf {x}^\mathbf {a}- \mathbf {x}^\mathbf {b}) + (\mathbf {x}^\mathbf {a}- \mathbf {x}^\mathbf {b}) \otimes \mathbf {x}^\mathbf {b}\in \Bbbk [\mathbf {x}] \otimes _\Bbbk I + I \otimes _\Bbbk \Bbbk [\mathbf {x}]. \end{aligned}

The implication “2 $$\Rightarrow$$ 1” is essentially due to Artin, cf. [9, Remark p. 15]. Let us recall the argument.

If I is a coideal, then $$\Delta$$ induces $${\overline{\Delta }}: \Bbbk [\mathbf {x}]/I \rightarrow \Bbbk [\mathbf {x}]/I \otimes _k \Bbbk [\mathbf {x}]/I$$ compatible with the canonical projection $$\pi : \Bbbk [\mathbf {x}] \rightarrow \Bbbk [\mathbf {x}]/I$$. Now suppose that $$\pi (m_0) = \sum _{i=1}^\ell \lambda _i \pi (m_i)$$ is a relation among classes of monomials with $$\ell$$ minimal (and hence $$\lambda _i \ne 0$$ for $$i > 0$$).  Then

\begin{aligned} \sum _{i>0}\lambda _i\pi (m_i)\otimes \pi (m_i)&= {\overline{\Delta }}\Big (\sum _{i>0}\lambda _i\pi (m_i)\Big )\\&= {\overline{\Delta }}\big (\pi (m_0)\big ) = \pi (m_0) \otimes \pi (m_0)\\&= \Big (\sum _{i>0} \lambda _i \pi (m_i)\Big ) \otimes \Big (\sum _{i>0} \lambda _i \pi (m_i)\Big )\\&= \sum _{i,j>0} \lambda _i \lambda _j \pi (m_i)\otimes \pi (m_j). \end{aligned}

As $$\ell$$ is minimal, the $$\pi (m_i)$$ are independent for $$i > 0$$. Hence the $$\pi (m_i) \otimes \pi (m_j)$$ are also independent. But $$\lambda _i\lambda _j \ne 0$$ for $$i \ne j$$, so $$\ell = 1$$ and $$\lambda _1 = \lambda _1^2$$, whence $$\lambda _1 = 1$$. $$\square$$

### Algorithm 4.5

Find the locus of unital fibers

### Unital Loci from Group Actions

This subsection assumes $$\Bbbk$$ is uncountable.

Algorithm 4.5 works over an arbitrary affine base. The setup in Sect. 3, where the base is a group of ambient automorphisms, yields a further structural property of unital loci.

### Proposition 4.6

The set of points $$\gamma \in G$$ such that $$I.\gamma$$ is unital is a finite union

\begin{aligned} \bigcup _{\gamma \in U} G(I)\gamma \end{aligned}

of cosets of the stabilizer G(I). In particular, this set is closed.

### Proof

First, $$\Bbbk [\mathbf {x}]$$ has only countably many monomials. As every unital ideal is generated by finitely many monomials and differences of monomials, there are also only countably many unital ideals of the form $$I.\gamma$$. The set of points $$\gamma \in G$$ such that $$I.\gamma$$ is unital has the form

\begin{aligned} \bigcup _{\gamma \in U} G(I)\gamma \end{aligned}

and is constructible by Theorem 2.3 and Proposition 4.4. Lemma 2.2 implies that the union has to be finite. Finally, stabilizers are closed by Lemma 3.6, so their cosets are, as well, and so are finite unions thereof. $$\square$$

### Example 4.7

Under the componentwise action of the usual torus, any binomial prime ideal becomes unital—and hence toric—after rescaling the variables appropriately [9, Corollary 2.6]. The prime assumption here is essential, as Example 4.8 shows.

### Example 4.8

It need not be possible to find a group action—or any family with isomorphic fibers—taking a given ideal to a unital one, even if the original is a binomial ideal. Indeed, the ideal $$I = \langle u^5(u - v), v^5(u - 2v)\rangle$$ is binomial but $$\mathbb {C}[u,v]/I$$ is not (abstractly) isomorphic to a quotient of a polynomial ring R by a unital ideal J.

To see why, suppose such a J exists and consider R of smallest possible dimension. Since $$\mathbb {C}[u,v]/I$$ is supported at the origin (both $$u^{11}$$ and $$v^{11}$$ lie in I), the quotient R / J is supported at a single point. As J is unital, the variables can be numbered so that its support point is $$\xi = (1,\dots , 1,0,\dots , 0)$$. Extending J by all $$x_i$$ such that $$x_i(\xi ) = 0$$ then yields a reduced ideal [18, Theorem 9.12]. Hence $$x_1(\xi ) = 1 \Rightarrow x_1 - 1 \in J$$, which contradicts minimality of $$\dim R$$. Thus $$\xi = 0$$. Note that R / J has tangent space of dimension 2. Consequently, $$R = \mathbb {C}[x_1,x_2]$$. Indeed, if $$\dim R > 2$$ then J contains a binomial of the form $$x - g$$, where g is a monomial and x is a variable. If $$x \not \mid g$$ then $$x - g$$ eliminates x, contradicting minimality of $$\dim R$$. And if $$x \mid g$$ then repeatedly replace x in g to get binomials of the form $$x - g_i \in J$$ with $$\deg g_i \rightarrow \infty$$; the fact that R / J is Artinian proves that $$x \in J$$, again leading to a contradiction.

Since I is a complete intersection, $$J = \langle f_1,f_2\rangle$$. Since R / J is supported at the origin, it equals its localization at $$\langle x_1,x_2\rangle$$. As J is unital, Nakayama’s lemma produces unital binomials $$b_1, b_2$$ such that $$J = \langle b_1,b_2\rangle$$ as ideals of the localization $$\tilde{R}$$. More generally, minimal systems of generators of an ideal in a local ring have the same cardinality.

The local Hilbert function of $$\mathbb {C}[u,v]/I$$ implies that (i) neither $$b_1$$ nor $$b_2$$ has nonzero monomials of degree less than 6, and (ii) each of them contains a monomial of degree 6.

Now note that there are, up to scaling, precisely two distinct pairs of nonzero elements $$\ell _{1,i},\ell _{2,i} \in \langle u,v\rangle /\langle u,v\rangle ^2$$ for $$i = 1,2$$ such that $$\ell _{1,i}^a \ell _{2,i}^b = 0 \in \langle u,v\rangle ^6/\langle u,v\rangle ^7 \subset \mathbb {C}[u,v]/(I+\langle u,v\rangle ^7)$$ for $$a + b = 6$$. Further, the exponents satisfy $$a = 1$$ and $$b = 5$$ or $$a = 5$$ and $$b = 1$$. Indeed, suppose $$\ell _{1,i}, \ell _{2,i}$$ satisfy the above condition. If $$\ell _{1,i} \ne u$$ and $$\ell _{2,i} \ne u-v$$ (which provides one possible pair), it must be that

\begin{aligned} \ell _{1,i}^a \ell _{2,i}^b=\lambda u^5(u-v)+v^5(u-2v) \end{aligned}

for some $$\lambda \in \mathbb {C}$$. Dividing by $$u^6$$ and setting $$t = v/u$$, the equality above means that the polynomial $$P(t) = \lambda (1 - t) + t^5 - 2t^6$$ has two roots of multiplicity a and b, respectively. Swapping if necessary, assume $$a \ge 3$$. Vanishing of the second derivative forces the root to satisfy $$20t^3 - 60t^4 = 0$$, i.e., $$t = 0$$ or $$t = 1/3$$. If $$t = 0$$ is the root of P(t), then $$\lambda = 0$$ and a second pair of linear forms $$\{v, u - 2v\}$$ arises. When $$t = 1/3$$, nonvanishing of $$(d^4P(t)/dt^4)(1/3)$$ means it can be a root of multiplicity at most three, and a second root of multiplicity three would be required, but none exists.

The next goal is to exclude cases where $$b_1, b_2$$ are monomials or fail to be homogeneous. So assume $$b_1$$ is a monomial or inhomogeneous binomial. Then $$b_1$$ has image $$x_1^ax_2^{6-a}$$ in $$\tilde{R}/\langle x_1,x_2\rangle ^7$$; without loss of generality assume it is $$x_1^5x_2$$, providing the first pair $$\{\ell _{1,1}, \ell _{2,1}\}=\{x_1,x_2\}$$. This forces $$b_2$$ to be homogeneous, as it would otherwise provide a second monomial, necessarily equal to $$x_1x_2^5$$, contradicting the fact that the pairs $$\ell _{1,i},\ell _{2,i}$$ are distinct for $$i = 1,2$$. Hence, $$b_2$$ is a homogeneous degree 6 binomial.

Observe that for any nonzero element $$h \in \langle u,v\rangle /\langle u,v\rangle ^2 \subseteq \mathbb {C}[u,v]/I + \langle u,v\rangle ^2$$ there is at most a 1-dimensional family of elements of $$\langle u,v\rangle ^5/\langle u,v\rangle ^6$$ that annihilate it modulo $$I + \langle u,v\rangle ^7$$. Indeed, otherwise $$h P_1 = \alpha _1 u^5(u-v) + \beta _1 v^5(u - 2v)$$ and $$h P_2 = \alpha _2 u^5(u - v) + \beta _2 v^5(u - 2v)$$ for some linearly independent polynomials $$P_1, P_2$$ of degree 5. However, then h would be a common divisor of $$u^5(u - v)$$ and $$v^5(u - 2v)$$, which is not possible.

Hence, $$b_2$$ cannot be divisible by $$x_1$$ or $$x_2$$ and thus must equal $$x_1^6 - x_2^6$$. But then there is no second pair of linear forms $$\ell _{1,2}, \ell _{2,2} \in \mathbb {C}[x_1,x_2]/(J+\langle x_1,x_2\rangle ^7)$$ such that $$\ell _{1,2}^5 \ell _{2,2}^{\phantom {1}} = 0$$, because a polynomial of the form $$\lambda t + t^6 - 1$$ cannot have a root of multiplicity 3, so it must have more than two distinct roots. This concludes the proof that $$b_1$$ and $$b_2$$ are both homogeneous and that neither is a monomial. But this means that $$x_1 - x_2$$ divides both $$b_1$$ and $$b_2$$, which contradicts $$\dim \tilde{R}/J = 0$$.

## Detecting Toric Ideals and Varieties

This section presents an algorithm to check whether a given homogeneous prime ideal defines a variety that is abstractly isomorphic to a toric one (Sect. 5.2). While this could be done using our earlier algorithms, the hypothesis that I is prime allows significant simplifications (Sect. 5.1).

### Faster Algorithm for Fiber Containment of an Irreducible Family

The procedure in Sect. 5.2 is made faster by the following alternative to Algorithm 2.7 in the special case that the ideal that is requested to be larger (that is, $$I_2$$) has fibers that are known to be prime. The advantage is that we expect this algorithm to run much faster than Algorithm 2.7.

### Algorithm 5.1

Compute containment locus families when one has prime fibers

### Proposition 5.2

Algorithm 5.1 is correct.

### Proof

In any finitely generated commutative $$\Bbbk$$-algebra R (we apply it to $$\kappa _\mathfrak {p}[\mathbf {x}]$$), if one ideal I contains a prime ideal J, then $$I = J$$ if and only if $$\dim (R/I) = \dim (R/J)$$. $$\square$$

### Example 5.3

This is a Macaulay2 [11] demonstration of testing whether there exists $$\gamma \in G = (\mathbb {C},+)$$ such that $$I.\gamma$$ is toric, where $$I = \langle xy + 2y^2 - 1\rangle$$ and $$(\mathbb {C},+)$$ acts on $$\mathbb {C}^2$$ by $$\gamma .(x,y) = (x + \gamma y, y)$$. The code applies Algorithm 4.5, relying on the fast Algorithm 5.1 instead of Algorithm 2.7 to compute containment. Moreover, in view of Remark 3.3, it works with the inverse group action, so as to obtain the ideal I as the image of a map.

### Toric Varieties

This section shows how to detect whether a projective variety is toric without any prespecified group or other family of ambient automorphisms.

### Algorithm 5.4

Decide whether a normal projective variety is abstractly toric

### Theorem 5.5

Algorithm 5.4 is correct.

### Proof

The re-embedding can be done by attempting successively higher Veronese maps and checking whether each is projectively normal. The cited source guarantees that this procedure terminates.

It remains only to show that if X is toric, then there really exists $$\gamma \in GL _N$$ such that $$\gamma .X$$ is equivariantly embedded. The embedding of X distinguishes a very ample divisor L on X. If X is toric, then L is equivalent to a toric divisor $$L'$$ by [5, Theorem 4.2.1]. Therefore, the projectively normal embedding yields a surjection $$\Gamma (\mathbb {P}^N,\mathcal {O}(1)) \rightarrow \Gamma (X, L')$$. In particular, X is toric if and only if there exists an automorphism of $$\mathbb {P}^N$$ under which I(X) goes to a toric ideal. But all automorphisms of $$\mathbb {P}^N$$ are (projectively) linear, so the desired one is represented by some matrix $$\gamma \in GL _N$$. $$\square$$

### Remark 5.6

To check if a variety X is toric it is essential that X be projective. For example, it is an open problem to decide whether the affine variety defined by the ideal $$\langle x + x^2y + z^2 + t^3\rangle \subset \mathbb {C}[x,y,z,t,w]$$ is isomorphic to $$\mathbb A^4$$ [20, Remark 5.3].

## Conclusion

In retrospect, many of our algorithms apply not only to a group of automorphisms of an affine space but to an arbitrary family of transformations. To be precise, fix an arbitrary morphism $$\alpha : Y \times \mathbb A^n \rightarrow \mathbb A^n$$, thought of as a family of maps $$\mathbb A^n \rightarrow \mathbb A^n$$ parametrized by Y. For a $$\Bbbk$$-valued closed point $$\eta \in Y$$ denote by $$\alpha _\eta : \mathbb A^n \rightarrow \mathbb A^n$$ the morphism obtained by composition of the isomorphism $$\mathbb A^n \rightarrow \{\eta \} \times \mathbb A^n$$ and (the restriction of) $$\alpha$$. Given an affine variety $$X \subseteq \mathbb A^n$$ one may ask for the locus of points $$\eta \in Y$$ such that $$\alpha ^{-1}_\eta (X)$$ is defined by a unital ideal. In the group action setting, where $$Y = G$$ is a group and $$\alpha$$ is an action (Sect. 3.1), working with images and preimages are more or less equivalent: they amount to taking orbits for $$\gamma$$ or $$\gamma ^{-1}$$. But in this more general setting, working with preimages means computing inverse images of subschemes (images of ideals), which is trivial, instead of computing images of schemes (kernels of ring maps), which is a hard problem known as implicitization. Furthermore, the inverse image is closed, whereas for images of morphisms the global closure may be not compatible with closure fiberwise, which creates additional problems.

### Remark 6.1

Even using preimages of subschemes instead of images, extending our algorithms to this more general setting requires special attention. For example, although two families over Y can still be compared as in Algorithm 4.5, the dimension argument in Sect. 5.1 no longer necessarily applies.

### Remark 6.2

In contrast, the methods to test binomiality in Sect. 4.1 adapt verbatim to the case of arbitrary maps, as they only rely on comprehensive Gröbner bases.

### Remark 6.3

It is similarly easy to check if an ideal is generated by monomials in a similar way to Algorithm 4.2. Indeed, for each $$U_i$$ one only needs to check if it is possible for all coefficients of nonleading monomials to vanish. Alternatively, note that an ideal $$I \subseteq \Bbbk [x_1,\dots , x_n]$$ is monomial if and only if it is stable under the whole torus $$T = (\Bbbk ^*)^n$$ and apply Algorithm 3.10.

### Remark 6.4

Some of our algorithms easily generalize to work over a base that is not necessarily affine. Algorithms 2.7 and 5.1 can generalize by covering the base with affine open sets, as the problem is local on the base. On the other hand, an algebraic group acting faithfully on the affine space is automatically affine [1, Corollary 3.2.2], so there is no need to generalize the algorithms related to group actions. That said, Algorithm 3.10 should generalize to the case where neither the base nor the fiber is assumed to be affine. This indicates how some of the other algorithms might proceed in the case of a nonaffine group acting on a nonaffine scheme.

In view of our results, we find the following three problems of particular importance.

### Problem 1

Is the problem of determining if an affine variety is affine space decidable? Equivalently, is it decidable to test if a finitely generated $$\Bbbk$$-algebra is a polynomial ring?

### Problem 2

1. (a)

Is the problem of determining if a projective (nonnormal) variety admits a torus action with a dense orbit decidable?

2. (b)

Is the problem of determining if a projective (nonnormal) variety is (abstractly) isomorphic to the (closure of) the image of a monomial map decidable?

### Problem 3

Is the problem of determining if a given affine variety is toric decidable?

The last problem may be asked both for normal and arbitrary affine varieties.

Finally, a natural continuation of the results in Sect. 4 would be to try to find an algorithm which detects whether a given ideal contains at least one binomial after applying an automorphism of $$\Bbbk [\mathbf {x}]$$. (See [15] for an algorithm which detects binomials in ideals in fixed coordinates.) In general, this seems to be a difficult problem. The following example illustrates that the set of points $$\gamma \in G$$ where $$\gamma .I$$ contains a binomial can fail to be constructible.

### Example 6.5

Consider the additive group $$G = (\mathbb {C}^2, +)$$ acting on $$\mathbb A^3_\mathbb {C}$$ by the matrices

\begin{aligned} g_{\alpha \beta } = \begin{pmatrix} 1 &{}\quad 0 &{}\quad -\alpha \\ 0 &{}\quad 1 &{} \quad \beta \\ 0 &{}\quad 0 &{} \quad 1 \end{pmatrix}. \end{aligned}

The ideal $$I = \langle (x - y)^2, z\rangle \subseteq \mathbb {R}[x,y,z]$$ yields $$g_{\alpha \beta }.I = \langle (x-y)^2, z - \alpha x + \beta y\rangle$$. The claim is that $$g_{\alpha \beta }.I$$ contains a binomial if and only if $$\alpha$$ and $$\beta$$ are $$\mathbb {Q}$$-linearly dependent.

Start with the “only if” direction. Assume that $$g_{\alpha \beta }.I$$ contains a binomial $$\lambda \mathbf {x}^\mathbb A- \mu \mathbf {x}^\mathbf {b}$$ for some scalars $$\lambda , \mu \in \mathbb {C}$$ and exponents $$\mathbf {a},\mathbf {b}\in \mathbb {N}^3$$. Both generators of $$g_{\alpha \beta }.I$$ have a zero at $$(1,1,\alpha -\beta )$$, so this is also a zero of $$\lambda \mathbf {x}^\mathbb A- \mu \mathbf {x}^\mathbf {b}$$. This implies that

\begin{aligned} 0 = \lambda (\alpha {}-\beta )^{a_3} - \mu (\alpha {}-\beta )^{b_3}. \end{aligned}

By symmetry assume that $$a_3 \ge b_3$$. As $$\lambda \ne 0$$, there is no harm in assuming that $$\lambda = 1$$ and $$\mu = (\alpha -\beta )^{a_3 - b_3}$$. The differential operator $$D = \partial _x + \alpha \partial _z$$ has the property that $$D(f)(1, 1, \alpha -\beta ) = 0$$ for all $$f \in g_{\alpha \beta }.I$$ because this holds for the generators of the ideal. Now compute:

\begin{aligned} D(\mathbf {x}^\mathbf {a}- (\alpha -\beta )^{a_3 - b_3} \mathbf {x}^\mathbf {b})(1,1,\alpha -\beta ) = \bigl ((\alpha -\beta ) (a_1 - b_1) - \alpha (a_3 - b_3)\bigr )(\alpha -\beta )^{a_3-1}. \end{aligned}

It follows that either $$\alpha =\beta$$ or $$\alpha (a_1 - b_1 - a_3 + b_3) = \beta (a_1 - b_1)$$. In both cases, $$\alpha$$ and $$\beta$$ are $$\mathbb {Q}$$-linearly dependent.

Turning to the “if” direction, assume that $$\alpha$$ and $$\beta$$ are linearly dependent over $$\mathbb {Q}$$. Consider only the case where $$\alpha \ge \beta \ge 0$$, as the other cases are similar. In the given case, there exist $$p,q \in \mathbb {N}$$ with $$p \ge q > 0$$ and $$q \alpha - p \beta = 0$$. To see that $$g_{\alpha \beta }.I$$ contains the binomial $$(\alpha -\beta )^{p-q} x^p - y^q z^{p-q}$$, consider the lexicographic term order on $$\mathbb {C}[x,y,z]$$ with $$z \succ x \succ y$$. Computing the normal form of $$(\alpha -\beta )^{p-q} x^p$$ and $$y^q z^{p-q}$$ with respect to the given generators of $$g_{\alpha \beta }.I$$ and the given term order yields in both cases $$(\alpha -\beta )^{p-q} y^{p-1} \bigl (p x + (p-1) y \bigr )$$. Hence, their difference lies in $$g_{\alpha \beta }.I$$.