1 Introduction

Searches for new particles beyond the Standard Model (SM) at the LHC are difficult if light new particles have small production cross sections, and heavy new particles with larger production cross sections undergo dominantly cascade decays. This can be the case for supersymmetric extensions of the SM such as the Next-to-Minimal Supersymmetric Standard Model (NMSSM) [1, 2]. In the present article we focus on the Higgs sector of the NMSSM, where this phenomenon can take place. Accordingly wide regions of the parameter space of the Higgs sector of the NMSSM have not yet been explored, and it remains a challenging task for the future to cover them.

The ATLAS and CMS experiments at the LHC have started to search for final states resulting from one-step cascade decays in Higgs sectors beyond the SM. Given the existing phenomenological constraints from previous searches [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29], from properties of the mostly SM-like Higgs boson at 125 GeV [30,31,32], from direct detection experiments of dark matter [33,34,35,36] and more, it is a priori not clear in how far future such searches will explore new regions in the parameter space of the NMSSM. To this end it is useful to map out benchmark planes or lines satisfying existing phenomenological constraints, which is the task of the present article.

The Higgs sector of the NMSSM consists in two SU(2) doublets and a complex SU(2) singlet. In the CP-conserving NMSSM, the physical scalars can be decomposed into three neutral CP-even states, two neutral CP-odd states and one complex charged state. One of the three neutral CP-even states has to correspond to the SM-like Higgs boson. A priori the masses of the remaining states can assume a large range of values, depending on the five NMSSM-specific parameters \(\lambda \), \(\kappa \), \(A_\lambda \), \(A_\kappa \), \(\mu _\mathrm{eff}\) as well as on \(\tan \beta \) [1, 2].

In general, the three neutral CP-even states as well as the two neutral CP-odd states are mixtures of SU(2) doublets and a SU(2) singlet. Thereby all scalars obtain couplings to SM fermions and gauge bosons (originally reserved to the SU(2) doublets), and all possible CP-conserving trilinear couplings among CP-even and CP-odd scalars are non-zero. Still, in most of the parameter space one can denote each of the three CP-even scalars \(H_1\), \(H_2\) and \(H_3\) (ordered in mass) as either SM-like (h), or mostly “singlet-like” (\(H_S\), with small direct production cross sections) or mostly “MSSM-like” (H). Here MSSM-like refers to a nearly degenerate SU(2) doublet (if much heavier than the SM-like Higgs boson) consisting in a neutral CP-even, a neutral CP-odd and a charged complex state. Likewise one can denote each of the two CP-odd scalars \(A_1\), \(A_2\) as either mostly singlet-like (\(A_S\)) or mostly MSSM-like (A).

Given lower bounds on the mass of the MSSM-like charged Higgs boson both from direct searches and from \(b \rightarrow s+\gamma \) (although the charged Higgs contribution can partially be cancelled by supersymmetric contributions), the neutral CP-even and CP-odd members of the MSSM-like SU(2) doublet cannot be light; 350 GeV is a conservative lower bound on their masses. In contrast, any range is still allowed for the (independent) masses of the mostly singlet-like CP-even and CP-odd states. However, if lighter than  60 GeV, their couplings to the SM-like Higgs boson must be small enough in order to escape bounds from searches for decays of the SM-like Higgs boson into pairs of light scalars, and bounds from beyond-the-SM contributions to the total width of the SM-like Higgs boson.

Searches for heavier neutral scalars such as the neutral CP-even and CP-odd members of the MSSM-like SU(2) doublet can focus on their production via gluon fusion (ggF) and their decays into pairs of fermions or gauge bosons leading to events with little SM background once resonance-like excesses are looked for [3,4,5,6,7,8,9,10].

However, in models with extended Higgs sectors such as the NMSSM, heavy scalars can have sizeable branching fractions into two lighter scalars, or a Z boson and one scalar. For scalars with small direct production cross sections, such processes can be the only way to discover them. Corresponding searches have been performed in [11,12,13,14,15,16,17,18]. Finally heavy CP-even scalars can also be searched for in final states corresponding to resonant SM Higgs pair production, see [19,20,21,22,23,24,25,26,27,28]. For a recent review of boson pair production at the LHC see [29].

Benchmark points for Higgs-to-Higgs cascade processes in the NMSSM have been proposed in [37,38,39,40,41,42,43,44,45,46,47,48,49,50], see also the twiki web site of the LHC-HXSWG3-NMSSM working group [51]. However, earlier benchmark points are often outdated due to more recent limits from searches in the proposed (or other) channels.

The benchmark points presented here are chosen such that constraints from the existing searches above are satisfied. In addition we impose constraints from B-physics, constraints from properties of the SM-like Higgs boson (a mass within \(125\pm 2\) GeV allowing for theoretical uncertainties, and couplings in the \(\kappa \)-framework satisfying combined limits of ATLAS and CMS [30,31,32]), and constraints from stability of the electroweak vacuum. Constraints from the anomalous magnetic moment of the muon are left aside as these concern the smuon/gaugino sector which is irrelevant here.

We also require that the lightest supersymmetric particle is neutral (the lightest neutralino), since it is stable and contributes necessarily to the relic density of the universe. We do not require that it accounts for all of the observed dark matter relic density as there may exist additional contributions from physics far above the weak scale. (For this reason we do not require the absence of a Landau singularity below the GUT scale but confine ourselves to \(\lambda < 0.7\) in order to avoid a strong coupling regime close to the weak scale.) However, the stable lightest neutralino unavoidably contributes to dark matter direct detection experiments, and must satisfy corresponding constraints which are imposed on the benchmark points since the properties of the lightest neutralino (mass and annihilation rate typically via a CP-even or CP-odd scalar in the s-channel) depend on the same parameters as the NMSSM Higgs sector.

The above constraints are implemented in the code NMSSMTools_5.6.2 [52, 53] (for more details see the website [54]) coupled to MicrOmegas [55] for the calculation of the dark matter relic density and direct detection cross sections.

Usually the production cross section of heavy Higgs states via gluon fusion dominates and is considered here, although vector boson fusion can be relevant in some particular regions of the parameter space [56]. For the calculation of the cross sections \(ggF\rightarrow H/A\) (with \(M_{H/A} \ge 400\) GeV) we start with the BSM Higgs production cross sections at \(\sqrt{s} = 13\) TeV (update in CERN Report4 2016) from the twiki web page [57]. These are multiplied by the reduced couplings squared of H/A. Thereby we capture most of the radiative QCD corrections in the form of K-factors; the remaining theoretical uncertainties are at most of \({{\mathcal {O}}}(10\%) \).

In the next Sect. 2 we discuss masses and trilinear couplings (including \(H_i\)–\(A_j\)–Z) in the Higgs basis in the NMSSM confining ourselves to numerically dominant contributions. This allows to estimate which cascade decays are usually dominant. In Sect. 3 we present benchmark planes for various final states corresponding to \(H\rightarrow h+H_{S}\) in the space \(M_{H} - M_{H_{S}}\). For \(M_{H_{S}}\) we confine ourselves to the range \(M_{H_{S}}>60\) GeV: Otherwise the couplings of \(H_{S}\) must be small enough in order to satisfy constraints from \(h\rightarrow H_{S} + H_{S}\) leading to small allowed cross sections for its production via cascade decays, and its discovery seems more likely via decays of h. We also present a benchmark line with the largest possible cross sections for resonant SM-Higgs pair production \(H_2\rightarrow h+h\) as function of \(M_{H_2}\) (\(H_2\) is a mixture of \(H_S\) and H), cross sections for triple SM-Higgs production and for the yet unexplored process \(H\rightarrow h + (H_{S}\rightarrow t{\bar{t}})\).

In the NMSSM, singlet-like pseudoscalars \(A_S\) can have dominant branching fractions into \(\gamma \gamma \): tree level couplings to SM gauge bosons and fermions can vanish, but couplings to higgsino-like charginos \(\sim \lambda \) remain. While decays into chargino pairs are kinematically forbidden, chargino loops induce a coupling of \(A_S\) to photons making the diphoton channel the dominant decay mode. (The branching fraction into the loop induced \(Z+\gamma \) channel is about half as large unless kinematically suppressed, in which case the branching fractions into \(\gamma \gamma \) can become 99%.) The production of \(A_S\) can proceed via the production of the MSSM-like pseudoscalar A, and its decay \(A\rightarrow A_S + h\). Benchmark points for this process will be given as well.

Furthermore we show benchmark planes for final states corresponding to \(A\rightarrow H_{S}+Z\) in the space \(M_{A}-M_{H_{S}}\). These benchmark points are the same as for \(H\rightarrow h+H_{S}\) which allows to compare the cross sections, and hence to estimate the corresponding relative sensitivities. Cross sections for the yet unexplored processes \(A\rightarrow Z + (H_{S}\rightarrow h+h)\) and \(A\rightarrow Z + (H_{S}\rightarrow t{\bar{t}})\) are also given. Finally we present a benchmark line with the largest possible cross sections for \(A\rightarrow h+Z\) as function of \(M_{A}\). A summary is given in Sect. 4.

2 The Higgs sector and trilinear Higgs couplings in the NMSSM

The neutral Higgs sector of the NMSSM consists in three complex scalars \(H_u^0\), \(H_d^0\) and S where \(H_u^0\) and \(H_d^0\) are members of SU(2) doublets and S is a gauge singlet [1, 2]. Their self couplings originate from terms

$$\begin{aligned} W=\lambda {{\mathcal {H}}}_u {{\mathcal {H}}}_d {{\mathcal {S}}} + \frac{\kappa }{3} {{\mathcal {S}}}^3 + \dots \end{aligned}$$
(2.1)

in the superpotential W in terms of superfields \({{\mathcal {H}}}\) and \({{\mathcal {S}}}\), and from trilinear soft supersymmetry breaking terms

$$\begin{aligned} \left( \lambda A_\lambda H_u H_d S + \frac{\kappa }{3} A_\kappa S^3\right) + \text {h. c.}\; ; \end{aligned}$$
(2.2)

contributions from D-terms are relatively small. The self couplings have to be expressed in terms of physical states. To this end the weak eigenstates \(H_u^0\), \(H_d^0\) and S have to be expanded around their vacuum expectation values \(v_u\), \(v_d\) and s (where \(v^2=v_u^2+v_d^2 \simeq (174 \text {GeV})^2\), \(M_Z^2=g^2 v^2\) with \(g^2=\frac{g_1^2+g_2^2}{2}\)). The mass matrices have to be diagonalized, and in the CP-conserving case one obtains three neutral CP-even scalars and two neutral CP-odd scalars (after elimination of the Goldstone boson). General expressions for these mass matrices including the dominant radiative corrections are given in [2]. A first approximation to the physical states is obtained in the so-called Higgs basis where singlet-doublet mixing is neglected and the CP-even doublets are rotated by the same angle as the CP-odd sector. Defining \(\tan \beta =\frac{v_u}{v_d}\) and using hats for the Higgs basis (\({\widehat{H}}_{SM}\) is near to but not yet exactly equal to the physical SM Higgs boson h, and \({\widehat{H}}\) is near to but not yet exactly equal to the physical MSSM-like Higgs boson H) one has

$$\begin{aligned} H_d^0&=\cos \beta {\widehat{H}}_{SM} + \sin \beta {\widehat{H}}, \quad H_u^0 = \sin \beta {\widehat{H}}_{SM} - \cos \beta {\widehat{H}},\nonumber \\ A_d&= \sin \beta {\widehat{A}}, \!\!\quad A_u = \cos \beta {\widehat{A}}\; . \end{aligned}$$
(2.3)

To these the pure singlet states \({\widehat{H}}_S\) and \({\widehat{A}}_S\) have to be added. The tree level elements of the \(2\times 2\) mass matrix in the CP-odd sector in the basis \({\widehat{A}}, {\widehat{A}}_S\) are for the typical case \(s, A_\lambda \gg M_Z\)

$$\begin{aligned} M^2_{A,11}&= \frac{2\lambda s(A_\lambda +\kappa s)}{\sin 2\beta }\; ,\nonumber \\ M^2_{A,22}&= -3\kappa A_\kappa s + {{\mathcal {O}}}(M_Z^2)\; ,\nonumber \\ M^2_{A,12}&= \lambda v (A_\lambda -2\kappa s)\; . \end{aligned}$$
(2.4)

The tree level elements of the \(3\times 3\) mass matrix in the CP-even sector in the basis \({\widehat{H}}_{SM}, {\widehat{H}}, {\widehat{H}}_S\) are

$$\begin{aligned} M^2_{H,11}&= M_Z^2\left( \cos ^2 2\beta + \frac{\lambda ^2}{g^2}\sin ^2 2\beta \right) \; , \nonumber \\ M^2_{H,22}&= \frac{2\lambda s(A_\lambda +\kappa s)}{\sin 2\beta } + {{\mathcal {O}}}(M_Z^2)\; ,\nonumber \\ M^2_{H,33}&= 2\kappa s(A_\lambda +4\kappa s)\; , \nonumber \\ M^2_{H,12}&= {{\mathcal {O}}}(M_Z^2)\; ,\nonumber \\ M^2_{H,13}&= \lambda v (2\lambda s-(A_\lambda +2\kappa s)\sin 2\beta )\; ,\nonumber \\ M^2_{H,23}&= \lambda v(A_\lambda +2\kappa s)\cos 2\beta \; . \end{aligned}$$
(2.5)

Hence singlet-doublet mixing is of \(\mathcal{O}(\frac{v}{s},\frac{v}{A_\lambda })\) relative to the diagonal elements, but can still be large if the corresponding diagonal elements are close to each other.

Trilinear couplings are proportional to one of the vacuum expectation values \(v_u\), \(v_d\) and s, or to one of the trilinear soft supersymmetry breaking terms \(A_\lambda \) or \(A_\kappa \). The latter contributes only to the trilinear singlet Higgs couplings which play a negligible role for Higgs-to-Higgs decays since pure singlets have tiny production cross sections.Footnote 1 General expressions for the trilinear couplings can be found in [2], but it is instructive to compare the ones relevant for Higgs-to-Higgs decays for the typical case \(s, A_\lambda \gg v_u, v_d \approx M_Z\). In the Higgs basis these are (neglecting contributions of \({{\mathcal {O}}}(M_Z)\))

$$\begin{aligned} (a)&\sim {\widehat{H}} {\widehat{H}}_{SM} {\widehat{H}}_S:&-\frac{\lambda }{\sqrt{2}} (2\kappa s + A_\lambda ), \nonumber \\ (b)&\sim {\widehat{H}} {\widehat{H}}_{SM} {\widehat{H}}_{SM}:&0 , \nonumber \\ (c)&\sim {\widehat{H}}_{S} {\widehat{H}}_{SM} {\widehat{H}}_{SM}:&\frac{\lambda ^2}{\sqrt{2}} s - \frac{\lambda }{\sqrt{2}}\sin \beta \cos \beta (2\kappa s + A_\lambda ), \nonumber \\ (d)&\sim {\widehat{H}}_{S} {\widehat{H}} {\widehat{H}}:&\frac{\lambda ^2}{\sqrt{2}} s + \frac{\lambda }{\sqrt{2}}\sin \beta \cos \beta (2\kappa s + A_\lambda ), \nonumber \\ (e)&\sim {\widehat{A}} {\widehat{H}}_{SM} {\widehat{A}}_S:&\lambda {\sqrt{2}}\sin \beta \cos \beta (-2\kappa s + A_\lambda ), \nonumber \\ (f)&\sim {\widehat{A}} {\widehat{H}} {\widehat{A}}_S:&\frac{\lambda }{\sqrt{2}}(\sin ^2\beta - \cos ^2\beta ) (-2\kappa s + A_\lambda ). \end{aligned}$$
(2.6)

If the fields in the Higgs basis are good approximations to the physical fields, relevant processes for searches for \(ggF\rightarrow X \rightarrow Y + h\) are \(ggF\rightarrow {\widehat{H}} \rightarrow {\widehat{H}}_S + {\widehat{H}}_{SM}\) (using the trilinear coupling (a)) and \(ggF\rightarrow {\widehat{A}} \rightarrow {\widehat{A}}_S + {\widehat{H}}_{SM}\) (using the trilinear coupling (e)); singlet-like scalars have small production cross sections. The production cross sections for \({\widehat{H}}\) and \({\widehat{A}}\) are similar, but the trilinear couplings (a) are larger than the trilinear couplings (e) for \(2\sin \beta \cos \beta < 1\) and/or cancellations in \((-2\kappa s + A_\lambda )\) for \(\kappa s, A_\lambda > 0\) as considered here. This explains why (for similar masses of \({\widehat{H}}\) and \({\widehat{A}}\)) the process \(ggF\rightarrow {\widehat{H}} \rightarrow {\widehat{H}}_S + {\widehat{H}}_{SM}\) dominates over \(ggF\rightarrow {\widehat{A}} \rightarrow {\widehat{A}}_S + {\widehat{H}}_{SM}\).

An exception is the final state \(h+\gamma \gamma \) with \(\gamma \gamma \) from a BSM scalar or pseudo-scalar. As stated in the introduction, the mostly singlet-like pseudoscalar \(A_S\) can have a dominant branching fraction up to \(\sim 99\)% into diphotons if \(Z+\gamma \) is kinematically suppressed (up to \(\sim 66\%\) otherwise). For maximal cross sections, radiative corrections from supersymmetric particles to the pseudoscalar mass matrix play a relevant role. In the next section we consider a benchmark plane for this process as well. (A corresponding decoupling of \(H_S\) does not happen since in the scalar sector two mixing angles would have to vanish simultaneously, which would require \(\lambda s \rightarrow 0\) leading to massless higgsinos.)

At first sight the prospects for resonant SM Higgs pair production look dim: For \({\widehat{H}}\) with the largest production cross section via gluon fusion the dominant trilinear coupling (b) vanishes, whereas \({\widehat{H}}_S\) would not be produced via gluon fusion. However the scalar fields in the Higgs basis are not necessarily close to physical fields, and \({\widehat{H}}\) and \({\widehat{H}}_{S}\) can strongly mix. Indeed we found that the cross sections for resonant SM Higgs pair production can be quite large (see the next section) in this case.

Next we turn to decays \(H\rightarrow A+Z\) and \(A\rightarrow H+Z\). The relevant couplings are

$$\begin{aligned} H_i(p) A_j(p') Z_\mu :\ -i g\, C_i^H C_j^A (p-p')_\mu \end{aligned}$$
(2.7)

where \(C_i^H\) denote the \({\widehat{H}}\) components of the physical states \(H_i\), and \(C_j^A\) the \({\widehat{A}}\) components of the physical states \(A_j\).

Decays \(H_3 \rightarrow A_2 +Z\) and \(A_2 \rightarrow H_3 +Z\) are usually impossible for kinematic reasons if the physical states \(H_3\), \(A_2\) are well approximated by the Higgs basis \({\widehat{H}}\), \({\widehat{A}}\). Decays \(H_3 \rightarrow A_1 +Z\) with \(A_1 \sim A_{S}\) are proportional to the \({\widehat{A}}\) component of \( A_{S}\) induced by the off-diagonal element \(M^2_{A,12}\) in (2.4) which is usually quite small.

Decays \(A_{2} \rightarrow H_{1,2} +Z\) are proportional to the \({\widehat{H}}\) components of \(H_{1,2}\). These are typically larger for \(H_{S}\) compared to h since \(M^2_{H,23} > M^2_{H,12}\) in (2.5). As a consequence cross sections for searches for \(ggF\rightarrow H/A \rightarrow A/H +Z\) are usually dominated by \(ggF\rightarrow A \rightarrow H_{S}+Z\) in the NMSSM. On the other hand searches for \(ggF\rightarrow A \rightarrow h+Z\) are also frequently performed. In the NMSSM, the possible cross sections (for a given mass \(M_A\)) are also discussed in the next section.

3 Benchmark planes and lines

A significant excess in final states corresponding to \(H_3\rightarrow h+H_{2}\) would imply the simultaneous discovery of two new bosons beyond the Higgs sector of the Standard model, which may correspond to the CP-even scalars H and \(H_{S}\) in the NMSSM. (We recall that the physical states H and \(H_{S}\) are generally mixtures of the weak eigenstates.) Based on an integrated luminosity of up to 140 fb\(^{-1}\), corresponding searches have been performed by CMS in the channel \(H_3\rightarrow (h\rightarrow \tau \tau )+ (H_{S}\rightarrow bb)\) for mass ranges \(240\ \text {GeV}< M_{H_3} < 3000\) GeV and \(60\ \text {GeV}< M_{H_S} < 2800\) GeV in [11], and in the channel \(H_3\rightarrow (h\rightarrow bb)+ (H_{S}\rightarrow bb)\) for mass ranges \(900 \ \text {GeV}< M_{H_3} < 4000\) GeV and \(60\ \text {GeV}< M_{H_S} < 600\) GeV in [12].

We have prepared a plane of viable benchmark points for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow bb) + h\) covering the mass ranges \(400\ \text {GeV}< M_{H} < 2000\) GeV, and \(60\ \text {GeV}< M_{H_S} < 800\) GeV (or \(M_{H_S} <M_{H}-200\) GeV). Generally, masses of H and \(H_{S}\) are given with a precision of \(\pm 0.5\) GeV, except for \(M_{H_{S}}=60\) GeV which means \(60\le M_{H_{S}} \le 60.5\) GeV such that constraints from CMS in [11], valid for \(60\le M_{H_{S}}\), are satisfied.Footnote 2 Details of the NMSSM-specific parameters, masses, branching fractions and more for each point can be obtained in SLHA format from the authors.

Table 1 Possible cross sections (in pb) at 13 TeV for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow bb) + (h \rightarrow XX)\), \(XX = bb, \tau \tau , \gamma \gamma \) (columns 3-5), \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow h+h) + h\) (column 6), \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow tt) + h\) (column 7). Points indicated by \(^{(1)}\) in the second column have cross sections \(ggF\rightarrow H\rightarrow (H_{S} \rightarrow bb) + (h \rightarrow \tau \tau )\) at the boundary of the region excluded by CMS in [11]. Points indicated by \(^{(2)}\) in the second column have cross sections \(ggF\rightarrow H\rightarrow h + h\) at the boundary of the region excluded by ATLAS in [27]. Points indicated by \(^{(3)}\) in the second column have cross sections \(ggF\rightarrow A\rightarrow Z + (H_{S}\rightarrow bb)\) at the boundary of the region excluded by ATLAS in [14]
Full size table

For given values of \(M_{H}\) and \(M_{H_{S}}\), the remaining parameters are chosen such that the cross sections for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow bb) + h\) are relatively large, sometimes just below the upper limits from present constraints from the LHC (and from B-physics and dark matter direct detection), see below. The cross sections for decays of h such as \(h \rightarrow \tau \tau \) and \(h \rightarrow \gamma \gamma \) are closely related to the ones for \(h \rightarrow bb\) since the branching fractions of h satisfy the combined constraints from ATLAS and CMS [30,31,32]. Still, deviations of \(\sim 10\%\) from the Standard Model values are possible within these constraints, and sometimes realized within the NMSSM. Therefore we show in Table 1 the cross sections for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow bb) + (h \rightarrow XX)\) separately for \(XX = bb, \tau \tau , \gamma \gamma \) for all benchmark points. Points indicated by \(^{(1)}\) in the second column in Table 1 (and later in Table 4) have cross sections \(ggF\rightarrow H\rightarrow (H_{S} \rightarrow bb) + (h \rightarrow \tau \tau )\) at the boundary of the region excluded by CMS in [11]. (Points indicated by \(^{(2)}\) or \(^{(3)}\) have cross sections for resonant Higgs pair production or for \(ggF\rightarrow A\rightarrow Z + (H_{S}\rightarrow bb)\) just below the boundary of the region excluded by corresponding searches, see below.)

For illustration we show in Fig. 1 the allowed cross sections for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow bb) + (h \rightarrow bb)\) for \(M_{H_S}=200\) GeV as function of \(M_H\) from Table 1. For \(M_H \le 800\) GeV (red dotted line) these are limited by constraints from \(ggF\rightarrow A\rightarrow Z + (H_{S}\rightarrow bb)\) on the parameter space of the NMSSM which explains its irregular shape.

Fig. 1
figure 1

Allowed cross sections for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow bb) + (h \rightarrow bb)\) for \(M_{H_S}=200\) GeV as function of \(M_H\). For \(M_H \le 800\) GeV (red dotted line) these are limited by constraints from \(ggF\rightarrow A\rightarrow Z + (H_{S}\rightarrow bb)\) [14]

Full size image
Fig. 2
figure 2

Allowed cross sections for \(ggF\rightarrow A \rightarrow (h\rightarrow \tau \tau ) + (A_S\rightarrow \gamma \gamma )\) for \(M_A\) from \(\sim 410\) to 700 GeV

Full size image
Fig. 3
figure 3

Allowed cross sections for resonant Higgs pair production as function of \(M_{H_2}\). For \(M_{H_2} < 700\) GeV these correspond to the limits from ATLAS in [27]

Full size image

\(H_{S}\) has additional interesting decay modes other than \(H_{S} \rightarrow bb\). For instance, the branching fraction into \(\tau \tau \) is always smaller by a factor 0.1–0.14, depending on \(M_{H_{S}}\). This allows to estimate the cross sections for \(H_{S} \rightarrow \tau \tau \) from the ones into bb.

Table 2 Possible cross sections \(\sigma \) at 13 TeV for \(ggF\rightarrow A \rightarrow (h\rightarrow \tau \tau ) + (A_S\rightarrow \gamma \gamma )\)
Full size table
Table 3 Possible cross sections \(\sigma \) at 13 TeV for \(ggF\rightarrow H_2 \rightarrow h + h\)
Full size table

For \(M_{H_{S}}>250\) GeV, the branching fraction \(BR(H_{S}\rightarrow h+h)\) becomes sizeable (up to \(\sim 20\%\)), and the cascade \(H \rightarrow H_{S} + h\) leads to triple Higgs production. Furthermore, for \(M_{H_{S}}>350\) GeV, the branching fraction \(BR(H_{S}\rightarrow tt)\) becomes dominant. Since both processes are of interest, we added the cross sections for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow h+h) + h\) and for \(ggF\rightarrow H \rightarrow (H_{S} \rightarrow tt) + h\) (without branching fractions of h which are within \(\sim 10\%\) of the Standard Model values) in Table 1. The branching fractions for \(H_{S} \rightarrow h+h\) can become small for accidential cancellations within the corresponding trilinear coupling c) in (2.6); of potential interest are the cases where these cross sections are relatively large.

In principle the processes \(ggF\rightarrow A \rightarrow A_S + h\) can lead to identical signatures as the considered processes \(ggF\rightarrow H \rightarrow H_{S} + h\). However, we found in Sect. 2 that the considered processes have larger cross sections and are thus more promising for potential discoveries (or exclusions). As discussed in the introduction and in Sect. 2 the final state from \(A_S\rightarrow \gamma \gamma \) is an exception. In Table 2 we show possible cross sections for \(ggF\rightarrow A \rightarrow (h\rightarrow \tau \tau ) + (A_S\rightarrow \gamma \gamma )\) for \(M_A\) near 400, 500, 600 and 700 GeV and \(M_{A_S}=70, 100\) and 200 GeV. (For \(M_{A_S}=200\) GeV the channel \(A_S\rightarrow Z+\gamma \) is open, and the branching fraction for \(A_s\rightarrow \gamma \gamma \) shrinks from \(\sim 99\%\) to \(\sim 66\%\).) For illustration we show in Fig. 2 the allowed cross sections for \(ggF\rightarrow A \rightarrow (h\rightarrow \tau \tau ) + (A_S\rightarrow \gamma \gamma )\) for \(M_A\) from \(\sim 410\) to 700 GeV from Table 2.

Another interesting process within the NMSSM is resonant (SM) Higgs pair production. The role of the “resonance” can be played by H or by \(H_{S}\). The most constraining limits on SM Higgs pair production \(ggF\rightarrow H\rightarrow h+h\) originate from the combination of bbbb, \(bb\tau \tau \) and \(bb\gamma \gamma \) final states by ATLAS in [27] for \(M_H=250\)–3000 GeV. In fact, for \(M_{H}\le 650\) GeV the cross sections in the NMSSM could be larger than the limits obtained by ATLAS in [27], hence these limits constrain the parameter space of the NMSSM. For \(M_{H_{S}} \le 120\) GeV, these limits imply lower bounds on \(M_{H}\) (depending on \(M_{H_{S}}\)) slightly above 400 GeV. Among the selected benchmark points in Table 1 (and later in Table 4), points indicated by \(^{(2)}\) in the second column (for \(M_{H}\) near 400 GeV) have cross sections \(ggF\rightarrow H\rightarrow h + h\) just below the boundary of the region excluded by ATLAS in [27].

As discussed in Sect. 2, particularly large cross sections for resonant SM Higgs pair production can be found if H and \(H_{S}\) strongly mix. Then the notation \(H_2\), \(H_3\) is more appropriate, and the largest cross sections are found for \(ggF\rightarrow H_2 \rightarrow h+h\). In Table 3 we show possible cross sections in the NMSSM (for points which differ from the benchmark points in Table 1) for \(ggF\rightarrow H_2 \rightarrow h + h\) for \(M_{H_2} > 700\) GeV up to \(M_{H_2} =1200\) GeV; these are still below the limits from CMS in [28]. For illustration we show in Fig. 3 the allowed cross sections for resonant Higgs pair production as function of \(M_H\) from Table 3.

Table 4 Possible cross sections (in pb) at 13 TeV for \(ggF\rightarrow A \rightarrow Z + (H_{S} \rightarrow XX)\), \(XX = bb, \tau \tau , \gamma \gamma \) (columns 3-5) \(ggF\rightarrow A \rightarrow Z+ (H_{S} \rightarrow h+h)\) (column 6) \(ggF\rightarrow A \rightarrow Z+(H_{S} \rightarrow tt)\) (column 7) Points indicated by \(^{(1)}\) in the second column have cross sections \(ggF\rightarrow H_3\rightarrow (H_{S} \rightarrow bb) + (h \rightarrow \tau \tau )\) at the boundary of the region excluded by CMS in [11]. Points indicated by \(^{(2)}\) in the second column have cross sections \(ggF\rightarrow H_3\rightarrow h + h\) at the boundary of the region excluded by ATLAS in [27]. Points indicated by \(^{(3)}\) in the second column have cross sections \(ggF\rightarrow A_2\rightarrow Z + (H_{S}\rightarrow bb)\) at the boundary of the region excluded by ATLAS in [14]
Full size table
Table 5 Possible cross sections \(\sigma \) at 13 TeV for \(ggF\rightarrow A \rightarrow Z + h\)
Full size table

Finally we turn to cascade decays into a Z boson. As discussed in Sect. 2, the largest cross sections in the NMSSM correspond to the processes \(ggF\rightarrow A \rightarrow H_{S} +Z\); cross sections for \(ggF\rightarrow H \rightarrow A_{S} +Z\) are substantially smaller for equivalent masses of H/A and \(A_S/H_S\). Corresponding searches have been performed by CMS in [13] after \(\sim 36\) fb\(^{-1}\) for the mass ranges \(120\ \text {GeV}<M_{A}< 1000\) GeV and \(30 \ \text {GeV}<M_{H_S}< 780\) GeV, and by ATLAS in [14] after \(\sim 139\) fb\(^{-1}\) for \(230\ \text {GeV}<M_{A}< 800\) GeV and \(130 \ \text {GeV}<M_{H_S}< 700\) GeV. (We have adopted the notation to the interpretation within the NMSSM.)

The latter search excludes some regions in the parameter space of the NMSSM; the benchmark points shown here satisfy these constraints. Points indicated by \(^{(3)}\) in the second column of Tables 1 and 4 have cross sections \(ggF\rightarrow A\rightarrow Z + (H_{S}\rightarrow bb)\) at the boundary of the region excluded in [14]. In the Table 4 we show the cross sections for \(ggF\rightarrow A\rightarrow Z + (H_{S}\rightarrow XX)\) for various final states \(XX=bb,\tau \tau ,\gamma \gamma \) for the same benchmark points as in Table 1 which allows to compare the sensitivities in the various search channels. (For some points, the branching fraction for \(H_{S}\rightarrow \gamma \gamma \) can be particularly small due to cancellations among different loop contributions.) Also shown in Table 4 are cross sections for the yet unexplored processes \(ggF\rightarrow A \rightarrow Z+(H_{S} \rightarrow h+h)\) and \(ggF\rightarrow A \rightarrow Z+(H_{S} \rightarrow tt)\) which can possibly be within reach.

Searches for \(A \rightarrow h+Z\) have been performed by CMS in [15, 16] after \(\sim 36\) fb\(^{-1}\), and by ATLAS in [17] after \(\sim 139\) fb\(^{-1}\). In the NMSSM these cross sections are dominated by the production of the MSSM-like pseudo-scalar. In the Table 5 we show the largest possible cross sections in the NMSSM for \(400< M_{A} < 2000\) GeV (for points different from the previous benchmark points) which are well below the present limits obtained by CMS and ATLAS.

4 Summary

Searches for Higgs-to-Higgs and Higgs-to-Higgs+Z cascade decays at the LHC allow to explore extended Higgs sectors beyond the SM. In the present paper we have presented various benchmark planes and lines which show which cross sections are possible in which final states within the NMSSM, subject to present phenomenological and theoretical constraints. Some of the available searches by ATLAS and CMS already touch the parameter space of the NMSSM, and our tables allow to estimate which future searches can be promising not only using available data, but also after the upgrade of the LHC to High Luminosity after a suitable rescaling. The proposed search channels \(H\rightarrow h+(H_{S} \rightarrow t{\bar{t}})\), \(A \rightarrow Z+ (H_{S}\rightarrow h+h)\) and \(A \rightarrow Z+ (H_{S}\rightarrow t{\bar{t}})\) are new and have not been considered before.