Constraining the Higgs trilinear coupling from an SU (2) quadruplet with bounded-from-below conditions

: Integrating out a heavy scalar can cause the Higgs trilinear coupling to deviate from its Standard Model value: a good example is provided by an SU (2) quadruplet. Constraints on the full theory, however, can limit the size of the deviation. We show that the bounded-from-below conditions for the Standard Model extended by an SU (2) quadruplet strongly constrain the Z 2 -breaking Higgs portal and can bound the Higgs trilinear coupling close to its Standard Model value. For TeV-scale quadruplet masses in models with custo-dial symmetry violation, these constraints can be a few times stronger than constraints from electroweak precision measurements. For the custodial quadruplet, these are the strongest theoretical constraints available.


Introduction
The discovery of the Higgs boson a decade ago was a spectacular confirmation of the validity of the Standard Model (SM) [1,2].No other new elementary particles have been found at the LHC.Heavy undiscovered particles, however, can indirectly influence the couplings of the Higgs boson, especially its self-coupling and the associated Higgs trilinear coupling measured at colliders.In particular, the presence of new scalars can change the trilinear coupling significantly (see e.g.[3]).At present, the Higgs trilinear coupling is very loosely bounded: its ratio to its SM value, κ λ , is constrained by −1.24 < κ λ < 6.49 at the CMS [4] and −0.4 < κ λ < 6.3 at ATLAS [5] at 95% confidence level.The High Luminosity LHC is expected to bring more precision with an estimated 0.1 < κ λ < 2.3 [6], while the ILC or the FCC-hh can constrain it to within 10% of its SM value [6][7][8].Before these higher precision measurements, our knowledge of the full scalar field content and the scalar potential remains scanty.
The minimal addition to the SM Higgs doublet is given by a single scalar multiplet ϕ.A representative example of such a new scalar is provided by an SU (2) quadruplet [9,10].Depending on its hypercharge, it can have different interactions with the Higgs doublet: the phenomenologically interesting possibilities are Y = 1/2 or Y = 3/2.The heavy quadruplet can be integrated out, leaving behind effective contributions to the SM [11].In particular, the Z 2 -breaking Higgs-quadruplet coupling λHϕ will induce the sixth order |H| 6 effective coupling to which the deviation in the Higgs trilinear coupling is proportional.The models with either Y = 1/2 or Y = 3/2 quadruplet alone violate custodial symmetry and are restricted by electroweak precision measurements, while a combination, the custodial quadruplet, has no such restrictions [11].
The full theory must obey additional constraints, such as perturbative unitarity and bounded-from-below conditions in the presence of the quadruplet.In the full theory, a sizeable value of the Z 2 -breaking λHϕ coupling can make the potential unbounded from below, unless balanced by other large, Z 2 -preserving, scalar couplings.
Our goal is to calculate the bounded-from-below conditions for the scalar potentials with Y = 1/2 or Y = 3/2 quadruplet and use them to constrain the λHϕ coupling.Additional constraints are provided by perturbative unitarity.The bounded-from-below conditions are derived using copositivity constraints [12] and positivity of quartic polynomials [13] on the gauge orbit space of gauge invariants [14][15][16][17].We use the P -matrix [14,16,18] and birdtrack [19,20] methods to find the orbit space.We show that the bounded-from-below conditions can constrain the Higgs couplings several times more strongly than electroweak precision measurements for quadruplet masses above a couple of TeV (see [11]) in models that break custodial symmetry.For the custodial quadruplet, the bounded-from-below conditions can be the strongest constraint, depending on the value of the Higgs effective quartic coupling.
We write the scalar potential of Y = 1/2 and Y = 3/2 quadruplets in section 2. The orbit space is determined in section 3. Bounded-from-below conditions for the scalar potential are derived in section 4 and perturbative unitarity constraints in section 5.The results are given in section 6 and conclusions in section 7. A birdtrack derivation of the P -matrix is given in Appendix A.

Scalar potential
We consider the most general scalar potential of the Higgs doublet H and an SU (2) quadruplet ϕ with hypercharge Y = 1/2 or Y = 3/2.We take the Higgs doublet in the unitary gauge.The SU (2) L quadruplet is given by a symmetric tensor ϕ ijk = ϕ (ijk) .In the case of Y = 3/2, for example, its four components are given in terms of the four canonically normalised propagating degrees of freedom as The scalar potential is the sum of the part invariant under the Z 2 symmetry ϕ → −ϕ, and a Z 2 -breaking part, which for Y = 1/2 is given by 1 and for Y = 3/2 by V We have taken λHϕ and λHϕ to be real without loss of generality.Note that other works may use different notation for the λHϕ coupling, such as λ or λ ϕ , possibly with different normalisation.With respect to the λ Φ of the Y = 3/2 quadruplet of Ref. [11], in particular, we have λHϕ = λ Φ/ √ 3, while for the Y = 1/2 quadruplet the normalisation is the same as ours.
When the quadruplet is integrated out, it will generate a deviation δ h 3 in the trilinear Higgs coupling, given by δ h 3 = κ λ − 1.At dimension d = 6, one has [11] where G F is the Fermi constant, m h is the SM Higgs boson mass, and The custodial quadruplet, transforming as a (4, 4) under the custodial SO(4 Hϕ and equal masses.In the custodial case, the contributions in Eq. (2.6) are therefore equal and sum up.
3 Orbit space 3.1 Orbit space and P -matrix formalism Under a gauge transformation T (θ) in the gauge group G, the components of a constant scalar field configuration Φ (e.g. a vacuum expectation value) will rotate among themselves through 1 Thanks to Luis Lavoura for pointing out the λHϕ interaction term that was missing in some of the literature.
However, this term does not contribute to the Higgs trilinear coupling.It is also easy to show that it can be made negative simultaneously with the λHϕ term.In order to to find the maximum allowed parameter space for the Higgs trilinear coupling, we set λHϕ = 0 in the rest of the paper.Finding the full orbit space and bounded-from-below conditions involving this term is beyond the scope of the current paper.
a gauge orbit as Φ → Φ θ = T (θ)Φ.Since the scalar potential V (Φ) is gauge-invariant, its value does not change under the transformation.In particular, for a unitary group all the states Φ θ have the same norm Φ * Φ.
For a compact group, all gauge-invariant polynomials of scalar fields can be given as combinations of elements of a finite polynomial basis (minimal integrity basis) p a with a = 1, . . ., q.The polynomial basis always includes the norms of fields.Therefore, we can write the scalar potential -which is gauge-invariant -as a function of the basis polynomials as V (Φ) = V (p a (Φ)).The potential has the same range as a function of the invariant polynomials, but is not affected by the degeneracy under group transformations.Since the polynomial basis is invariant under gauge transformations, a gauge orbit corresponds to a single point in the orbit space.Finding the bounded-from-below conditions and minimising the potential in the orbit space R n /G, with p a parameterising the orbits, can hence be advantageous [14][15][16][17].
Each subgroup of the full gauge group G is the isotropy subgroup G Φ of some field configuration Φ, and all the transformed states Φ θ have the same isotropy subgroup.The set or sub-manifold of gauge orbits that respects the same isotropy subgroup is called the stratum of the isotropy subgroup.It can be described by a finite number of polynomial equations or inequalities.The main stratum corresponds to a general field configuration and breaks the gauge symmetry completely.The field configurations that breaks the full gauge group G to G Φ lie in the stratum of G Φ .Thus the main stratum forms the interior of the orbit space, whereas the lower-dimensional strata that form the orbit space boundary correspond to more symmetrical field configurations invariant under different isotropy subgroups G Φ .The physical region of the invariant polynomials p a is called the orbit space.Therefore the orbit space consists of strata of different dimensions: vertices, edges, faces, . . ., up to the main stratum whose dimension is given by the number of invariants.
The boundary of the orbit space can be found with the help of the P -matrix method [14,16,18].The equations and inequalities that define the strata can be expressed in the form of positivity and rank conditions of a matrix P constructed from the gradients of the polynomial invariants p a .The P -matrix is the Hermitian square of the Jacobian matrix of gauge invariants, a q × q symmetric and positive semi-definite matrix, whose elements are constructed from gradients of basis invariants p a as where Φ i run over the field components (real or complex).It can be shown that the P -matrix elements are invariant under gauge transformations and can be given in terms of the minimal polynomial basis.The P -matrix is positive-definite only inside the physical orbit space.The boundary of the orbit space can be obtained by solving the polynomial equation det P = 0.The k-dimensional strata of the orbit space are given by the connected components of the set p a for which P is positive-semidefinite and rank P = k.For example, the full orbit space has a single vertex where the norms of fields vanish, given by vanishing order-1 minors of the P matrix.
The gauge invariants can be separated into the norms of fields and angular variables or orbit space variables.Henceforth, by orbit space we will mostly mean the reduced orbit space of these dimensionless variables.For the reduced orbit space of the dimensionless orbit variables, vertices are given by the minors at the lowest order at which the field norms do not vanish, edges at the next order, and so on.If the scalar potential depends on the orbit space variables linearly, then its minimum lies on the boundary of the orbit space (more precisely, on the intersection of the orbit space with its convex hull) [15,21,22].One does not have to minimise the potential over any flat or concave regions of the orbit space, since such a region is already accounted for in the convex hull by the boundary of the region.
The minimal integrity basis of polynomial invariants can involve higher-order invariants with dimension d > 4. It is straightforward to find such a basis.If the P -matrix for given invariants contains elements that cannot be expressed solely in terms of these invariants, we can extend the basis by these P -matrix elements and calculate a new P -matrix for the extended basis.The procedure can be repeated until all P -matrix elements can be expressed via the basis.It can be, however, very complicated to solve for the boundary of the orbit space and project out the higher-order invariants not present in the scalar potential.A more practical solution is to consider a partial P -matrix only for a partial basis of the invariants that appear in the scalar potential (a principal submatrix of the full P -matrix).The polynomial equation det P = 0 can then be solved directly in terms of field components.Because the orbit space boundary corresponds to more symmetric field configurations, such solutions can often be found by educated guesswork.

Orbit space for the Higgs doublet and a quadruplet
In order to ascertain the boundary of the orbit space, we need to calculate the P -matrix and solve the equation det P = 0. We calculate the P -matrix for the polynomial invariants present in the potential, namely the partial basis ) where the two definitions of p 5 are given for the Y = 1/2 and the Y = 3/2 quadruplet, respectively. 2Which one we use shall be clear from the context.We define, in terms of the gauge invariants in the potential, the dimensionless orbit space variables The main stratum of the (reduced) orbit space is thus three-dimensional and the boundary of the orbit space has two-dimensional faces whose borders are edges that end at the vertices of the orbit space.The field norms are given by where a coefficient of 1/2 is included in the definition of φ in order to simplify the potential.While the full orbit space contains both the orbit space variables and field norms, usually by orbit space we will mean the space of the dimensionless variables (3. 3).The scalar potential, in terms of orbit space variables and the field norms, is then given by Notice that the potential looks the same, in terms of the orbit space variables, for both Y = 1/2 and Y = 3/2 quadruplets.We can easily obtain partial knowledge about the orbit space.Defining a 2 × 2 Hermitian matrix M k l ≡ ϕ * ijk ϕ ijl , we see that p 2 = tr M , p 3 = tr M 2 and p 4 = H † M H.We then have In the basis with diagonal M , one can see that 1 2 ≤ ζ ≤ 1 by considering equal diagonal elements or setting one diagonal element zero.The Cauchy-Schwartz inequality implies 0 ≤ ξ ≤ 1. Considering also unit norms H † H = 1, tr M = 1, one can readily show that the boundary of the projection of the orbit space to the ζξ-plane is given by the ζ = 1 − 2ξ + 2ξ 2 curve. 3Furthermore, it is easy to see that −1 ≤ χ ≤ 1 for the Y = 3/2 quadruplet.
To fully determine the orbit space, we need to calculate the P -matrix.The symmetric P -matrix for the gauge invariants with d ≤ 4 is given by where in the case of the Y = 3/2 quadruplet, we obtain (ϕ * ijk ϕ jkl ϕ * lmn ϕ mnp ϕ * pqr ϕ qri + 2ϕ * ijk ϕ jkl ϕ * lmn ϕ imp ϕ * pqr ϕ qrn ), We computed the P -matrix from Eq. (3.1) with the Wolfram Mathematica computer algebra system, but the compact expressions in Eq. (3.8) were calculated in birdtrack notation [19,20] in Appendix A. The birdtrack calculation provides a cross-check for the code.
With the Higgs boson in the unitary gauge, the orbit space for the Higgs doublet and the Y = 3/2 quadruplet is given by Note that the Higgs field has canceled out of the orbit space variables which thus depend on the quadruplet only.This is due to our choice of unitary gauge and homogeneity of the invariant polynomials in h.In the limit of h → 0, l'Hôpital's rule gives the same result.The shape of the orbit space can now be found by considering the principal minors of the P -matrix.First-order principal minors give p 1 = p 2 = 0 (then, of course, all p i = 0).Second-order principal minors yield, among other equations, p 1 p 2 = 0. Taking p 2 = 0 implies p 1 = 0. Taking p 1 = 0 or h = 0 leaves a single equation which is not identically satisfied.Simplifying matters further, we consider the field configuration ϕ 112 = ϕ 122 = 0 invariant under the U (1) subgroup of SU (2), and consider only real fields.We obtain Inserting its solutions in Eqs.(3.10), (3.9) and (3.11), we obtain, for the orbit variables (ξ, ζ, χ), the solutions (1, 1, ±1), (0, 1, 0), (1/2, 1/2, ±1/ √ 2).These are vertices of the reduced orbit space (3.3) with nonzero field norms.
We next look at solutions with rank P = 3 with vanishing order-3 principal minors.We again consider only real fields and set ϕ 112 = ϕ 122 = 0. Going to unit p 2 = 1, we solve it e.g. for ϕ 222 .The orbit space is then a function of ϕ 111 only.The solutions then give either ϕ 111 = 0 or h = 0.The former choice gives again the above vertex solutions, while inserting the latter one in Eqs.(3.10), (3.9) and (3.11) and eliminating ϕ 111 from (3.3), we see that the orbit space has an edge given by bounded by vertices given by ξ = 1, ζ = 1, χ = ±1 which correspond to neutral field configurations.The other solutions to Eq. (3.12), listed above, in fact lie on the edge (3.13).
We can also check that the order-3 and higher principal minors indeed do vanish for the field configuration ϕ 112 = ϕ 122 = 0.This means that the faces bounded by the edge are determined by the edge and do not bulge out.Therefore, the convex hull of the orbit space is determined by the edge (3.13).
Allowing for non-zero ϕ 112 or ϕ 122 = 0 yields more solutions to the equation det P = 0, but they stay within the convex hull of the solution (3.13).We also checked that the convex hull for points from a random scan is within the convex hull of the edge (3.13).

Orbit space for the Higgs doublet and the Y = 1/2 quadruplet
For the Y = 1/2 quadruplet, the derivation is similar and we will not provide it in as much detail.The orbit variables ξ and ζ are given by Eqs.(3.9) and (3.10), respectively, while χ, related to p 5 , is given by Similarly, only the fifth column (and row) of the P -matrix changes.As expected, P 15 = 3p 5 and P 25 = p 5 just like in the Y = 3/2 case.The remaining elements are calculated in Appendix A and are given by ) The orbit space has two edges which meet at vertices given by ξ = 2/3, ζ = 5/9, χ = ±1/ √ 3 that correspond to real neutral field configurations.One of the edges, obtained by setting ϕ 112 = ϕ 122 = 0 and real field values, is similar to (3.13): The other edge, obtained by setting ϕ 111 = ϕ 112 = 0, is and its projection on the ξζ-plane is given by ζ = −3 + 8ξ − 4ξ 2 .The orbit space is symmetric with respect to the χ = 0 line, obtained by setting ϕ 112 = ϕ 222 = 0, and given by 4 Bounded-from-below conditions In order to make physical sense, the scalar potential of a theory must be bounded-frombelow to have a finite potential minimum.Since dimensionful terms in the potential can be neglected for large field values, it is enough to only consider constraints on the quartic part of the potential.Because we can always take the λHϕ term to be negative without loss of generality, it suffices to consider non-negative χ.
First of all, simple sufficient bounded-from-below conditions can easily be obtained by taking λ ϕ = λ ′ ϕ = 0, because then the quartic part of the potential (3.5) reduces to Because the last term in brackets is always negative, we can consider usual positivity, not copositivity, of its coefficient matrix.

Bounded-from-below conditions for the Y = 3/2 quadruplet
Since the orbit space for the Y = 3/2 quadruplet is simpler, we will consider this case first.The discriminant of the potential (4.1) with zero self-couplings is extremised for χ = 0 or χ = 1, yielding the sufficient bounded-from-below conditions For the full conditions, we have to minimise the quartic potential on the boundary of the orbit space, which in our case means its vertices and edges.Because we can consider only non-negative χ, we have to consider also the middle of the orbit space curved edge, at χ = 0, where we have ζ = 0, ξ = 1.The potential is biquadratic in field norms for χ = 0, so we can get the vacuum stability conditions from copositivity [12] at this point.At the χ = 1 vertex of the orbit space, we can consider positivity of the potential as a quartic polynomial in φ with h = 1 [13].We obtain the necessary vacuum stability conditions for the potential as Note that the last, most complex condition in (4.3)only depends on the combinations λ ϕ + λ ′ ϕ and λ Hϕ + λ ′ Hϕ .For h = 0, we obtain further necessary conditions of which the last one coincides with one of the conditions of Eq. (4.3).
The full conditions are obtained by considering vacuum stability conditions on the edge of the orbit space, where the potential on a unit circle, V + λ(1 − h 2 − φ 2 ), can be minimised with respect to the parameter χ on the edge (3.13), the fields h and φ, and the Lagrange multiplier λ enforcing the constraint.The minimisation equations, given by can only be solved numerically.The bounded-from-below conditions on the edge are given by the intersection of the condition for each real solution of the minimisation equations.The necessary and sufficient conditions are then given by Eqs.(4.3) and (4.6).In practice, it can be faster to consider the positivity of the potential as a quartic polynomial in φ with h = 1 and χ on the edge for a range of χ values, including the end points χ = 0 and χ = 1.
The parameter space boundary given by full necessary and sufficient vacuum stability conditions lies between the bounds given by Eq. (4.3) and Eq.(4.2).The difference is large only for when the quadruplet self-couplings are sizeable in comparison with its Higgs portal couplings.

Bounded-from-below conditions for the Y = 1/2 quadruplet
The sufficient conditions for the potential (4.1) with zero self-couplings are given by where we considered positivity of the potential (4.1) for the extremal values χ = 0 and χ = 1/ √ 3.
Considering copositivity at the end and middle points of the χ = 0 line (3.20), and positivity of the potential as a quartic polynomial in φ with h = 1 for the χ = 1/ √ 3 vertex of the orbit space, we find the necessary conditions The bounded-from-below conditions on the edges (3.18) and (3.19) are given by the intersection of for real minimum solutions of the potential on a unit circle V + λ(1 − h 2 − φ 2 ) with the orbit space parameters, respectively, on each edge.The minimum equations are similar to Eq. (4.5).On the central line (3.20), the potential is copositive and the bounded-from-below conditions have the same form as the conditions for the type II seesaw potential derived in Ref. [23]: (4.10) The full necessary and sufficient bounded-from-below conditions are given by Eqs.(4.8), (4.9) and (4.10).Just as for the Y = 3/2 case, the parameter space boundary given by full necessary and sufficient vacuum stability conditions lies between the bounds given by Eq. (4.7) and Eq.(4.8).The difference is large only for when the quadruplet self-couplings are sizeable in comparison with its Higgs portal couplings.

Perturbative unitarity
First of all, we require each quartic coupling to be less than 4π in absolute value in order for the theory to be perturbative.Further, perturbative unitarity constraints arise from the unitarity of the scattering matrix for the two-to-two scalar scattering amplitudes.At the order of the zeroth partial wave, the S-matrix is given by with a pair a of scalars scattering to another pair b with the matrix element M ba (cos θ).The angle θ is between the incoming (p a ) and outgoing (p b ) three-momenta in the centre-of-mass frame, and the Mandelstam variable s = (p 1 + p 2 ) 2 .The exponent δ a is unity if the particles in pair a are identical and zero otherwise; likewise for pair b and δ b .The eigenvalues a i 0 of the scattering matrix must satisfy (5.2) In the high energy limit with infinite scattering energy s → ∞ only quartic couplings contribute to scattering.We calculate the transition matrix for all possible two-to-two scatterings S 1 S 2 → S 3 S 4 of scalar bosons, including the Goldstones G 0 and G + .In this limit, the a ba 0 matrix element, where a ≡ S 1 S 2 and b ≡ S 3 S 4 , is given by We do not give an explicit list of possible initial and final states for the Y = 1/2 and Y = 3/2 cases nor the S-matrices, because they are very lengthy and it is straightforward to obtain them.
Most unitarity constraints, however, cannot be expressed analytically.Still, we can considerably simplify the unitarity constraints by considering them together with other conditions.We see that in practice the bounded-from-below conditions on the λHϕ coupling are much stronger than unitarity constraints which therefore practically do not depend on this coupling in the allowed region.In this approximation, the perturbative unitarity conditions are given by where we have omitted conditions that are already guaranteed by perturbativity.Note that because we set λHϕ = 0, the approximate constraints (5.4) apply both in the Y = 1/2 and Y = 3/2 cases.On the other hand, we can set all couplings save λHϕ to zero to get an upper bound, finding that (5.5)
Note that the Z 2 -breaking term (2.3) for the Y = 1/2 quadruplet has the same normalisation than Ref. [11], while for the Y = 3/2 quadruplet we have λHϕ = λ Φ/ √ 3.For this reason we display the bounds for √ 3 λHϕ for the Y = 3/2 quadruplet so as to be able to directly compare results.We see that the bounds from our conditions on λ Φ and λ Φ of Ref. [11] are roughly the same.Note that for very small quadruplet self-couplings it suffices to use the sufficient conditions (4.7) or (4.2).But even finite self-couplings play a role only for small Higgs portals, in which case the necessary conditions (4.3) are an excellent approximation to the full necessary and sufficient ones.
While larger self-couplings or Higgs portals do increase the region allowed by the boundedfrom-below conditions, the corresponding decrease in the region allowed by perturbative unitarity constraints is more significant and undoes any gain.In case of the Y = 1/2 quadruplet, the combination of bounded-from-below conditions and unitarity gives an upper bound of λHΦ ≈ 1.8 for λ H = λ SM H and λHΦ ≈ 2.8 for λ H = 2λ SM H (the same bounds apply to √ 3 λHΦ for the Y = 3/2 quadruplet).These bounds become stronger than the electroweak precision measurement bounds on models that violate custodial symmetry, elaborated on in Ref. [11], at quadruplet masses above about M ϕ ≈ 1.8 TeV and at M ϕ ≈ 2.4 TeV, respectively. The

Conclusions
We have derived bounded-from-below conditions and perturbative unitarity constraints (5.4) of the Higgs doublet H and an SU (2) quadruplet ϕ of hypercharge Y = 1/2 or Y = 3/2.The new conditions constrain the λHϕ coupling that induces a change in the trilinear Higgs coupling upon integrating out the quadruplet.The available parameter space is shown in figure 1.Smaller quadruplet self-couplings allow for larger values of λHϕ , for though the bounded-from-below conditions are relaxed for larger self-couplings, the perturbative unitarity constraints become severe much faster.
Although the λHϕ quartic is constrained by electroweak precision measurements for models that violate custodial symmetry, these constraints weaken with higher quadruplet masses.By comparison with the results of Ref. [11], we show that the combination of bounded-frombelow conditions and unitarity becomes stronger than electroweak precision constraints for

Figure 1 .
Figure 1.Parameter space for the Y = 1/2 and Y = 3/2 quadruplets allowed by bounded-frombelow constraints, where green stands for λ H = λ SM H = 0.129 and light green for λ H = 2λ SM H , together with the boundaries of the respective sufficient (dotted) and necessary (dashed) bounded-from-below constraint regions.Perturbative unitarity, which changes very little between the two values of λ H , is satisfied in the blue region.For Y = 3/2 we show √ 3| λHϕ | on the vertical axis to facilitate comparison with Ref. [11].