1 Introduction

Despite the belief that there must be physics beyond the Standard Model (SM), the way it may manifest at collider experiments is yet to be understood. This circumstance reinforces the need to put forward new physics signals that searches at the Large Hadron Collider (LHC) could have been missing, as well as models in which those signals are produced. An example of such uncovered signal is multiboson production arising from the cascade decay of a new resonance [1, 2]. Such signal, namely a triboson resonance, was proposed as alternative interpretation of a \(3.4\sigma \) bump near 2 TeV in an ATLAS search for hadronically-decaying diboson resonances with Run 1 data [3]. Although searches for tri-W resonances have been performed by the CMS Collaboration [4], focusing on models with extra dimensions [5, 6], more general triboson as well as quadriboson resonance signals [7] are not experimentally covered.

The persistence of small bumps near 2 TeV in diboson searches using Run 2 data, with a small local significance of \(2\sigma \) both in the ATLAS [8] and CMS [9] experiments, motivated another uncovered signature, dubbed as ‘stealth boson’ [10]. This is a boosted particle (generically denoted as S) with a cascade decay

$$\begin{aligned} S\rightarrow A_1 A_2\rightarrow q{\overline{q}}q{\overline{q}}, \end{aligned}$$
(1)

where the intermediate particles \(A_{1,2}\) can be SM weak bosons (W and Z), the SM Higgs boson, or new scalars. When S is produced with a high boost from the decay of a much heavier resonance R, the products of its hadronic decay are reconstructed as a single jet with four-pronged structure. Multiboson signals involving cascade decays such as in Eq. (1) can give rise to merged four-pronged jets, and the complex jet structure makes the resulting signals quite more elusive for current searches.

Minimal extensions of the SM allowing hadronic cascade decays of stealth bosons were proposed in [11] and further explored in [12]. They were called minimal stealth boson models (MSBMs) and assume that the heavy resonance R is a colour-singlet neutral gauge boson \(Z'\). In order to break the corresponding \(\mathrm{U(1)}'\) symmetry and simultaneously account for the cascade decay in Eq. (1), MSBMs include two complex scalar singlets, together with extra matter that is required to cancel \(\mathrm{U(1)}'\) anomalies.

In this work we explore an alternative model that can give rise to these types of signatures. Our study is especially timely given the latest bump near 2 TeV reported by the CMS collaboration in the search for hadronically-decaying diboson resonances with the full Run 2 dataset, reaching a local significance of \(3.6\sigma \) [13]. The model we present in this work also introduces a heavy gauge boson \(Z'\), but replaces one of the scalar singlets of MSBMs by a complex scalar doublet. We label the model as UN2HDM, as it corresponds to a next-to-two-Higgs doublet model (N2HDM) [14,15,16,17,18,19,20] with an extra \(\text {U}(1)'\) symmetry. In comparison with MSBMs, the presence of a larger scalar sector in the UN2HDM leads to the presence of charged scalar particles. A detailed overview of the UN2HDM and the interactions of the new fields is presented in Sect. 2.

Our main goal is to show that, in the context of the UN2HDM, multiboson signals can be produced with sizeable cross sections in regions of the parameter space allowed by theoretical and experimental constraints. We generically denote the signals as multibosons, including also the case when the bosons are very boosted and their hadronic decay products merge into a single jet. In the notation of Eq. (1), S will be \(Z'\), and \(A_{1,2}\) two neutral or charged scalars, which subsequently undergo direct or cascade decays into boosted jets. After explaining in Sect. 3 how we scan the parameter space to find points allowed by constraints, we study in Sect. 4 three benchmark scenarios for multiboson signals. We discuss our results in Sect. 5.

2 The UN2HDM

Table 1 SM and U(1)\('\) hypercharges \(\psi \sim (Y,Y')\) of the fermions and scalars in the UN2HDM. Generation indices in SM fermions are omitted, as well as colour indices for quarks
Full size table

The UN2HDM extends the SM gauge group with an extra U(1)\('\) symmetry, thus featuring a new colour-singlet neutral gauge boson \(Z'\). Given the strong limits from \(Z'\) boson searches in their leptonic decays, \(Z'\rightarrow e^+e^-\) and \(Z'\rightarrow \mu ^+\mu ^-\), we consider the new \(Z'\) boson to be leptophobic, with the U(1)\('\) hypercharges of SM lepton doublets and singlets vanishing, i.e. \(Y'_\ell =Y'_e=0\). As already mentioned, the UN2HDM scalar sector contains two doublets \(\Phi _1\) and \(\Phi _2\), and one complex singlet \(\chi \). Cascade decays like the one in Eq. (1) cannot take place if both doublets have vanishing U(1)\('\) hypercharge. On the other hand, due to the requirements of U(1)\('\) gauge invariance of the Yukawa terms of SM leptons, and of a leptophobic \(Z'\), one of the Higgs doublets must have zero hypercharge under U(1)\('\). We therefore label the scalar doublets so that \(Y'_{\Phi _2} = 0\) and \(Y'_{\Phi _1} \ne 0\). The U(1)\('\) gauge invariance of the quark Yukawa terms with \(\Phi _2\) implies \(Y'_q = Y'_u= Y'_d\), generally non-vanishing. Therefore, we have the same Yukawa lagrangian found in a Type I two-Higgs doublet model (2HDM), namely

$$\begin{aligned} -{\mathcal {L}}_Y = Y_u{\overline{q}}_L{\tilde{\Phi }}_2u_R + Y_d{\overline{q}}_L\Phi _2 d_R + Y_e{\overline{\ell }}_L\Phi _2 e_R + \mathrm{h.c.}, \end{aligned}$$
(2)

where \(Y_X\,(X=u,d,e)\) are complex Yukawa matrices in generation space. The U(1)\('\) hypercharges of SM fermions are identical to those of MSBMs. Thus, all U(1)\('\) anomalies [21] cancel by adding the same extra matter as in MSBMs. Assuming the extra matter to be vector-like under the SM group, two simple solutions have been proposed in Ref. [11]: one with a set of vector-like quarks (Model 1), and another with a set of vector-like leptons (Model 2). In this work we will concentrate only on Model 2, for which the field content and hypercharge assignments are given in Table 1. The new vector-like leptons are the two SU(2) doublets \((N_1\,E_1)_{L,R}\) and the four singlets \({N_2}_{L,R}\) and \({E_2}_{L,R}\). \(N_i\) and \(E_i\) have electric charge 0 and − 1, respectively. Their hypercharges are fixed by anomaly cancellation, and all take values of \( \pm 9 Y'_q / 2\). Gauge invariance of the Yukawa interactions of the new leptons with the scalar singlet (which give rise to their masses) implies

$$\begin{aligned} Y_\chi '=9Y_q', \end{aligned}$$
(3)

leading to the following Yukawa interactions involving vector-like leptons:

$$\begin{aligned} {\mathcal {L}}_Y^\textrm{VLL}= & {} \overline{{\mathcal {E}}}_L(w_1^N{\tilde{\Phi }}_2{N_2}_R + w_1^E\Phi _2{E_2}_R) \nonumber \\{} & {} \quad + (w_2^N\overline{N_2}_R{\tilde{\Phi }}_2^\dagger + w_2^E\overline{E_2}_R\Phi _2^\dagger ){\mathcal {E}}_R \nonumber \\{} & {} \quad + y_1^N\overline{N_2}_L{N_2}_R\chi + y_1^E\overline{E_2}_L{E_2}_R\chi \nonumber \\{} & {} \quad + y_2\overline{{\mathcal {E}}}_L{\mathcal {E}}_R\chi ^\dagger +\mathrm{h.c.}, \end{aligned}$$
(4)

with \(w_i^{N,E}\) and \(y_i^{N,E}\) (\(i=1,2\)) being complex Yukawa couplings. We also assume a ‘dark lepton number’ that forbids Majorana mass terms for \(N_{2L}\) and \(N_{2R}\) [22]. Finally, as we will see later, a valid scalar mass spectrum can only be obtained if \(Y_{\Phi _1}'=Y_\chi '\).

With these \(\text {U}(1)'\) hypercharge assignments, the most general gauge-invariant scalar potential of the UN2HDM is

$$\begin{aligned} V&= m_{11}^2\Phi _1^\dagger \Phi _1 + m_{22}^2\Phi _2^\dagger \Phi _2 + \frac{m_0^2}{2}\chi ^\dagger \chi \nonumber \\&\quad + \frac{\lambda _1}{2}(\Phi _1^\dagger \Phi _1)^2 + \frac{\lambda _2}{2}(\Phi _2^\dagger \Phi _2)^2 + \lambda _3(\Phi _1^\dagger \Phi _1)(\Phi _2^\dagger \Phi _2) \nonumber \\&\quad + \lambda _4(\Phi _1^\dagger \Phi _2)(\Phi _2^\dagger \Phi _1) + \frac{\lambda _5}{2}(\chi ^\dagger \chi )^2 + \frac{\lambda _6}{2}(\Phi _1^\dagger \Phi _1)(\chi ^\dagger \chi ) \nonumber \\&\quad + \frac{\lambda _7}{2}(\Phi _2^\dagger \Phi _2)(\chi ^\dagger \chi ) + (\,\mu \chi \Phi _1^\dagger \Phi _2+ \mathrm{h.c.}), \end{aligned}$$
(5)

where \(\mu \) can be a complex parameter, while the remaining parameters are real. We define the scalar doublets \(\Phi _{1,2}\) and singlet \(\chi \) as

$$\begin{aligned} \begin{aligned} \Phi _k&= \frac{1}{\sqrt{2}}\begin{pmatrix} \sqrt{2}\phi _k^+ \\ v_ke^{i\varphi _k} + \rho _k + i\eta _k \end{pmatrix},\quad (k=1,2), \\ \chi&= \frac{1}{\sqrt{2}}(ue^{i\varphi _3} + \rho _3 + i\eta _3). \end{aligned} \end{aligned}$$
(6)

Without loss of generality, one can assume \(\varphi _1=0\), such that the vacuum expectation values (VEVs) are

$$\begin{aligned} \begin{aligned} \langle \Phi _1\rangle&= \frac{1}{\sqrt{2}}\begin{pmatrix} 0 \\ v_1 \end{pmatrix},\quad \langle \Phi _2\rangle = \frac{1}{\sqrt{2}}\begin{pmatrix} 0 \\ v_2e^{i\varphi _2} \end{pmatrix}\,\quad \\ \langle \chi \rangle&= \frac{1}{\sqrt{2}}ue^{i\varphi _3}. \end{aligned} \end{aligned}$$
(7)

Throughout this work, we will always assume nonzero VEVs \(v_1\), \(v_2\) and u. Note also that \(\varphi _2\) and \(\varphi _3\) can be rephased away through

$$\begin{aligned} \Phi _2\rightarrow \Phi _2'=e^{-i\varphi _2}\Phi _2,\quad \chi \rightarrow \chi '=e^{-i\varphi _3}\chi , \end{aligned}$$
(8)

which leaves V invariant, provided \(\mu \) is replaced by

$$\begin{aligned} \mu \rightarrow \mu '=\mu e^{-i(\varphi _2+\varphi _3)}, \end{aligned}$$
(9)

when V is expressed in terms of \(\Phi _2'\) and \(\chi '\). Since \(\mu \) is an arbitrary complex parameter, from now on we assume \(\varphi _2=\varphi _3=0\) without loss of generality, therefore having all VEVs real.

An useful feature of V is that the number of parameters is equal to the number of physical quantities (masses and mixing angles) needed to define the Higgs sector. This means that all eleven parameters shown in Eq. (5) can be written in terms of the three VEVs \(v_1\), \(v_2\) and u, as well as of the five scalar masses and three mixing angles that will be introduced later on.

For the cases we are interested in with \(v_1,v_2,u\ne 0\), the four minimisation conditions of the scalar potential are

$$\begin{aligned}&m_{11}^2 + \frac{v_1^2}{2}\lambda _1 + \frac{v_2^2}{2}(\lambda _3 + \lambda _4) + \frac{u^2}{4}\lambda _6 + \frac{u}{\sqrt{2}}\frac{v_2}{v_1}\Re (\mu )=0, \nonumber \\&m_{22}^2 + \frac{v_2^2}{2}\lambda _2 +\frac{v_1^2}{2}(\lambda _3 + \lambda _4) + \frac{u^2}{4}\lambda _7 + \frac{u}{\sqrt{2}}\frac{v_1}{v_2}\Re (\mu )=0, \nonumber \\&m_0^2 + u^2\lambda _5 + \frac{1}{2}v_1^2\lambda _6 + \frac{1}{2}v_2^2\lambda _7 + \sqrt{2}\frac{v_1v_2}{u}\Re (\mu )=0, \nonumber \\&\Im (\mu )=0. \end{aligned}$$
(10)

The first three equations allow to write \(m_{11}^2\), \(m_{22}^2\) and \(m_0^2\) in terms of the VEVs and the remaining parameters of the potential. The last equation in (10) implies that all parameters in the potential are real and, thus, there are no mixed \(\rho _i \eta _j\) mass terms. Then, the \(6\times 6\) neutral scalar mass matrix can be written in block-diagonal form as

$$\begin{aligned} {\mathcal {M}}^n = \begin{pmatrix} M^\rho &{} \quad 0 \\ 0 &{}\quad M^\eta \end{pmatrix}, \end{aligned}$$
(11)

where \(M^\rho \) and \(M^\eta \) are \(3\times 3\) real symmetric matrices defined in the \((\rho _1,\rho _2,\rho _3)\) and \((\eta _1,\eta _2,\eta _3)\) bases, respectively. Using henceforth the definitions

$$\begin{aligned} v = \sqrt{v_1^2 + v_2^2},\quad \tan \beta = v_2/v_1, \end{aligned}$$
(12)

and the notation \(s_\beta = \sin \beta \), \(c_\beta = \cos \beta \), the independent elements of \(M^\rho \) read

$$\begin{aligned}&M^\rho _{11} = v^2\lambda _1c_\beta ^2 - \frac{1}{\sqrt{2}}u\mu \tan \beta , \nonumber \\&M^\rho _{12} = v^2(\lambda _3+\lambda _4)s_\beta c_\beta + \frac{1}{\sqrt{2}}u\mu , \nonumber \\&M^\rho _{13} = \frac{v}{2}u\lambda _6c_\beta + \frac{1}{\sqrt{2}}v\mu s_\beta , \nonumber \\&M^\rho _{22} = v^2\lambda _2s_\beta ^2 - \frac{u\mu }{\sqrt{2}\tan \beta }, \nonumber \\&M^\rho _{23} = \frac{v}{2}u\lambda _7s_\beta + \frac{1}{\sqrt{2}}v\mu c_\beta , \nonumber \\&M^\rho _{33} = \lambda _5u^2 - \frac{1}{\sqrt{2}}\frac{v^2}{u}\mu c_\beta s_\beta , \end{aligned}$$
(13)

while for \(M_\eta \),

$$\begin{aligned}&M^\eta _{11} = -\dfrac{1}{\sqrt{2}}u\mu \dfrac{s_\beta }{c_\beta } \,\quad \quad M^\eta _{12} = \dfrac{1}{\sqrt{2}}u\mu , \nonumber \\&M^\eta _{13} = \dfrac{1}{\sqrt{2}}v\mu s_\beta ,\quad \quad M^\eta _{22} = -\frac{1}{\sqrt{2}}u\mu \dfrac{c_\beta }{s_\beta }, \nonumber \\&M^\eta _{23} = -\dfrac{1}{\sqrt{2}}v\mu c_\beta ,\quad \quad M^\eta _{33} = -\frac{1}{\sqrt{2}}\dfrac{v^2}{u}\mu c_\beta s_\beta . \end{aligned}$$
(14)

All terms in \(M^\eta \) are proportional to \(\mu \), which explains the need to set \(Y'_{\Phi _1} = Y'_\chi \), otherwise the \(\mu \) term in (5) would not be present and we would have a massless scalar.

The matrix \(M^\eta \) is diagonalised as \(R^T M^\eta R = (M^\eta )_\text {diag}\), using a rotation

$$\begin{aligned} R = \left( \begin{array}{ccc} -s_\beta &{}\quad c_\beta &{}\quad 0 \\ c_\beta &{}\quad s_\beta &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad 1 \end{array} \right) \left( \begin{array}{ccc} -s_\alpha &{}\quad 0 &{}\quad c_\alpha \\ 0 &{}\quad 1 &{}\quad 0 \\ c_\alpha &{}\quad 0 &{}\quad s_\alpha \end{array} \right) , \end{aligned}$$
(15)

being \(\alpha \) given by

$$\begin{aligned} \tan \alpha = -\frac{u}{vc_\beta s_\beta }. \end{aligned}$$
(16)

The only non-zero eigenvalue is

$$\begin{aligned} m_{A^0}^2 = - \frac{\mu }{\sqrt{2}} \left[ \frac{u}{s_\beta c_\beta } + \frac{s_\beta c_\beta v^2}{u} \right] , \end{aligned}$$
(17)

which corresponds to the squared mass of a CP-odd scalar \(A^0\). The matrix O that rotates the \(\rho _i\) fields to the mass basis is parameterised by three mixing angles, \(\alpha _1\), \(\alpha _2\), \(\alpha _3\), and it is given by

$$\begin{aligned} O = \begin{pmatrix} c_1c_2 &{}\quad s_1c_2 &{}\quad s_2 \\ -c_1s_2s_3 - s_1c_3 &{}\quad c_1c_3 - s_1s_2s_3 &{}\quad c_2s_3 \\ -c_1s_2c_3 + s_1s_3 &{}\quad -c_1s_3 - s_1s_2c_3 &{}\quad c_2c_3 \end{pmatrix}^T, \end{aligned}$$
(18)

where \(c_{1,2,3}=\cos \alpha _{1,2,3}\), \(s_{1,2,3}=\sin \alpha _{1,2,3}\) and \(-\pi /2 \le \alpha _{1,2,3} \le \pi /2\). Let us label as h the SM-like Higgs boson and \(H_{1,2}\) the new CP-even scalars, with \(H_1\) and \(H_2\) being always defined in such a way that \(m_{H_1}<m_{H_2}\). We can write \(M^\rho \) in terms of the masses of those three scalars and the three mixing angles introduced in Eq. (18). Namely,

$$\begin{aligned} M^\rho = O\begin{pmatrix} m_h^2 &{}\quad 0 &{}\quad 0 \\ 0 &{}\quad m_{H_1}^2 &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad m_{H_2}^2 \end{pmatrix}O^T. \end{aligned}$$
(19)

Finally, the charged-scalar mass matrix in the basis \((\phi _1^\pm ,\phi _2^\pm )\) is

$$\begin{aligned} {\mathcal {M}}^c = \left[ v^2\lambda _4 + \frac{\sqrt{2}u\mu }{s_\beta c_\beta }\right] \begin{pmatrix} -s_\beta ^2 &{}\quad s_\beta c_\beta \\ s_\beta c_\beta &{}\quad -c_\beta ^2 \end{pmatrix}, \end{aligned}$$
(20)

and its diagonalisation is performed as \(U^T {\mathcal {M}}^c U = ({\mathcal {M}}^c)_\text {diag}\), with

$$\begin{aligned} U = \left( \begin{array}{cc} -s_\beta &{}\quad c_\beta \\ c_\beta &{}\quad s_\beta \end{array} \right) , \end{aligned}$$
(21)

like in 2HDMs. The non-zero eigenvalue of \({\mathcal {M}}^c\) is

$$\begin{aligned} m_{H^\pm }^2 = -\lambda _4v^2 - \frac{\sqrt{2}u\mu }{c_\beta s_\beta }, \end{aligned}$$
(22)

corresponding to the squared mass of new charged scalars \(H^\pm \). In Appendix A we show how this result, together with Eqs. (17) and (19), can be used to write \(\mu \) and \(\lambda _{1-7}\) as functions of the physical parameters we have just presented.

Defining \({\tilde{H}}=(h,H_1,H_2)\), the couplings involving three CP-even scalars can be generically written as

$$\begin{aligned} {\mathcal {L}}_{3{\tilde{H}}} = -v\frac{\lambda _{ijk}}{S_{ijk}}{\tilde{H}}_i{\tilde{H}}_j{\tilde{H}}_k, \end{aligned}$$
(23)

where the coefficients \(\lambda _{ijk}\) are symmetric under index interchange. The symmetry factors \(S_{ijk}\) are equal to 1 if all indices are different, 2 if two indices are equal, or 6 if \(i=j=k\). The three-scalar interactions involving the pseudoscalar \(A^0\) and the charged scalars \(H^\pm \) are

$$\begin{aligned} {\mathcal {L}}_{{\tilde{H}}A^0A^0}&= -\frac{v}{2}\sum _{i=1}^3 g_{{\tilde{H}}_iA^0A^0}{\tilde{H}}_iA^0A^0,\end{aligned}$$
(24)
$$\begin{aligned} {\mathcal {L}}_{{\tilde{H}}H^+H^-}&= -v\sum _{i=1}^3 g_{{\tilde{H}}_iH^+H^-} {\tilde{H}}_iH^+H^-, \end{aligned}$$
(25)

respectively. The coefficients \(\lambda _{ijk}\), \(g_{{\tilde{H}}_iA^0A^0}\) and \(g_{{\tilde{H}}_iH^+H^-}\) are collected in Appendix B.

The gauge-boson masses and the gauge-scalar interactions can be obtained from the scalar kinetic terms

$$\begin{aligned} {\mathcal {L}} = |D_\mu \Phi _1|^2 + |D_\mu \Phi _2|^2 + |D_\mu \chi |^2, \end{aligned}$$
(26)

with the covariant derivatives defined as

$$\begin{aligned} D_\mu \Phi _1= & {} \left( \partial _\mu - igW_\mu ^a\frac{\tau ^a}{2} - \frac{1}{2}ig'B_\mu - ig_{Z'}Y'_{\Phi _1} B'_\mu \right) \Phi _1, \nonumber \\ D_\mu \Phi _2= & {} \left( \partial _\mu - ig W_\mu ^a\frac{\tau ^a}{2} - \frac{1}{2}ig'B_\mu \right) \Phi _2, \nonumber \\ D_\mu \chi= & {} \left( \partial _\mu - ig_{Z'}Y'_\chi B'_\mu \right) \chi . \end{aligned}$$
(27)

As usual, \(W_\mu ^a\) and \(B_\mu \) are the SM gauge fields, while \(B'_\mu \) is the one corresponding to the new U(1)\('\) symmetry. Notice that \(Y'_{\Phi _1}=Y'_\chi = 9 Y'_q\), as discussed above. As in the SM, the W-boson mass at the leading order is

$$\begin{aligned} m_W^2 = \frac{g^2}{4}v^2, \end{aligned}$$
(28)

whereas for the neutral gauge bosons we have, in the \((W_\mu ^3\,\,B_\mu \,\,B'_\mu )\) basis, the mass matrix

$$\begin{aligned} \left( \begin{array}{ccc} \frac{g^2}{4}v^2 &{}\quad -\frac{gg'}{4}v^2 &{}\quad -\frac{gg_{Z'}}{2}Y'_\chi v^2c_\beta ^2 \\ -\frac{gg'}{4}v^2 &{}\quad \frac{{g'}^2}{4}v^2 &{}\quad \frac{g'g_{Z'}}{2}Y'_\chi v^2c_\beta ^2 \\ -\frac{gg_{Z'}}{2}Y'_\chi v^2c_\beta ^2 &{}\quad \frac{g'g_{Z'}}{2}Y'_\chi v^2c_\beta ^2 &{}\quad g_{Z'}^2 Y^{\prime 2}_\chi \left( u^2 + v^2c_\beta ^2\right) \end{array} \right) . \end{aligned}$$
(29)

Similarly to the SM, one can write

$$\begin{aligned} \left( \begin{array}{c} W^3_\mu \\ B_\mu \end{array} \right) = \left( \begin{array}{cc} s_W &{}\quad c_W \\ c_W &{}\quad -s_W \end{array} \right) \left( \begin{array}{c} A_\mu \\ Z^0_\mu \end{array} \right) \end{aligned}$$
(30)

with \(c_W = \cos \theta _W\), \(s_W = \sin \theta _W\), \(\theta _W\) being the weak mixing angle. The \(A_\mu \) field is massless and corresponds to the physical photon. For the two remaining fields the mass matrix is

$$\begin{aligned} \left( \begin{array}{cc} m_{ZZ}^2 &{}\quad m_{ZZ'}^2 \\ m_{ZZ'}^2 &{}\quad m_{Z'Z'}^2 \end{array} \right) , \end{aligned}$$
(31)

where

$$\begin{aligned}&m_{ZZ}^2 = \frac{1}{4} g_1^2v^2 = \frac{m_W^2}{c_W^2}, \nonumber \\&m_{ZZ'}^2 = - \frac{1}{2} g_1 g_{Z'} Y_\chi ' v^2 c_\beta ^2, \nonumber \\&m_{Z'Z'}^2 = (g_{Z'}Y_\chi ')^2(u^2+v^2c_\beta ^2), \end{aligned}$$
(32)

with \(g_1 = g/c_W\). The diagonalisation of this matrix is

$$\begin{aligned} \left( \begin{array}{c} Z^0_\mu \\ B'_\mu \end{array} \right) = \left( \begin{array}{cc} c_Z &{}\quad -s_Z \\ s_Z &{}\quad c_Z \end{array} \right) \left( \begin{array}{c} Z_\mu \\ Z^\prime _\mu \end{array} \right) , \end{aligned}$$
(33)

with \(s_Z = \sin \theta _Z\), \(c_Z = \cos \theta _Z\), \(\theta _Z\) being the \(Z-Z'\) mixing angle. The tree-level masses of the SM Z boson and the new \(Z'\) correspond to the two non-zero eigenvalues,

$$\begin{aligned} m_{Z,Z'}^2= & {} \frac{1}{2}\bigg [m_{ZZ}^2 + m_{Z'Z'}^2 \nonumber \\{} & {} \pm \left( (m_{Z'Z'}^2 - m_{ZZ}^2)^2 + 4 m_{ZZ'}^4\right) ^{1/2}\bigg ], \end{aligned}$$
(34)

and the mixing angle is given by

$$\begin{aligned} \tan 2 \theta _Z = \frac{g_1 g_{Z'} Y'_\chi v^2 c_\beta ^2}{(g_{Z'} Y'_\chi )^2 (u^2 + v^2 c_\beta ^2) - g_1^2 v^2 / 4}. \end{aligned}$$
(35)

In the limit of small \(Z-Z'\) mixing, i.e. \(\theta _Z\ll 1\), the masses of Z and \(Z'\) can be approximated as:

$$\begin{aligned} m_{Z'}^2\simeq & {} (g_{Z'}Y'_\chi )^2(u^2 + v^2c_\beta ^2), \nonumber \\ m_{Z}^2\simeq & {} \frac{m_W^2}{c_W^2}\left[ 1 - \frac{(g_{Z'}Y'_\chi )^2 v^2c_\beta ^4}{m_{Z'}^2}\right] , \end{aligned}$$
(36)

while for \(\theta _Z\) one has

$$\begin{aligned} \theta _Z\simeq \frac{g_{Z'}Y'_\chi m_W vc_\beta ^2}{m_{Z'}^2c_W}. \end{aligned}$$
(37)

The interaction of the Z boson with fermions receives a small correction due to \(Z-Z'\) mixing,

$$\begin{aligned} {\mathcal {L}} = g_1 {\bar{\psi }} \left[ g_1 (T_3 - s_W^2 Q) c_Z + g_{Z'} Y' s_Z \right] \gamma ^\mu \psi Z_\mu , \end{aligned}$$
(38)

with \(T_3\) the third isospin component, Q the electric charge and \(Y'\) the \(\text {U}(1)'\) hypercharge of the field \(\psi \). The interaction of the \(Z'\) boson with quarks is

$$\begin{aligned} {\mathcal {L}} = g_{Z'} {\bar{q}} \gamma ^\mu (y_L P_L + y_R P_R) q Z'_\mu , \end{aligned}$$
(39)

where the left- and right-handed couplings are

$$\begin{aligned} y_L= & {} Y'_q c_Z - \frac{g_1}{g_{Z'}} (T_3 - Q s_W^2) s_Z, \nonumber \\ y_R= & {} Y'_q c_Z + \frac{g_1}{g_{Z'}} Q s_W^2 s_Z. \end{aligned}$$
(40)

The lagrangian terms involving two gauge bosons and one scalar can be written as

$$\begin{aligned} {\mathcal {L}}= & {} g_{ W^+W^- {\tilde{H}}_i} {W^-}^\mu W^+_\mu {\tilde{H}}_i + g_{ ZZ {\tilde{H}}_i} Z^\mu Z_\mu {\tilde{H}}_i \nonumber \\{} & {} + g_{ZZ' {\tilde{H}}_i}Z^\mu Z'_\mu {\tilde{H}}_i + (g_{ZW^\pm H^\pm } Z^\mu W^+_\mu H^- \nonumber \\{} & {} + g_{Z'W^\pm H^\pm } {Z'}^\mu W^+_\mu H^- + \mathrm{h.c.}) \end{aligned}$$
(41)

with

$$\begin{aligned} g_{W^+W^- {\tilde{H}}_i}= & {} \frac{g^2 v}{2}(O_{1i}c_\beta + O_{2i}s_\beta ), \nonumber \\ g_{ZZ {\tilde{H}}_i}= & {} \frac{g_1^2 v}{4}(O_{1i}c_\beta + O_{2i}s_\beta )c_Z^2 \nonumber \\{} & {} - g_1 g_{Z'} Y'_\chi v O_{1i} c_\beta c_Zs_Z\nonumber \\{} & {} + g_{Z'}^2 Y^{\prime 2}_\chi (O_{1i}vc_\beta + O_{3i}u)s_Z^2, \nonumber \\ g_{ZZ' {\tilde{H}}_i}= & {} -\frac{g_1^2 v}{2}(O_{1i}c_\beta + O_{2i}s_\beta )c_Zs_Z\nonumber \\{} & {} - g_1 g_{Z'} Y'_\chi v O_{1i}c_\beta (c_Z^2 - s_Z^2) \nonumber \\{} & {} +2 g_{Z'}^2 Y^{\prime 2}_\chi (O_{1i}vc_\beta + O_{3i}u)c_Zs_Z, \nonumber \\ g_{ZW^\pm H^\pm }= & {} -g g_{Z'} Y'_\chi v c_\beta s_\beta s_Z, \nonumber \\ g_{Z'W^\pm H^\pm }= & {} - g g_{Z'} Y'_\chi v c_\beta s_\beta c_Z. \end{aligned}$$
(42)

The Lagrangian terms with two scalars and one gauge boson are

$$\begin{aligned} {\mathcal {L}}= & {} g_{Z{\tilde{H}}_iA^0} {\tilde{H}}_i\overleftrightarrow {\partial _\mu } A^0 Z^\mu + g_{Z'{\tilde{H}}_iA^0}{\tilde{H}}_i\overleftrightarrow {\partial _\mu } A^0 {Z'}^\mu \nonumber \\{} & {} \!\!\! + g_{\gamma H^+H^-} {A}^\mu H^+ \overleftrightarrow {\partial _\mu } H^- + g_{ZH^+H^-} {Z}^\mu H^+ \overleftrightarrow {\partial _\mu } H^- \nonumber \\{} & {} \!\! \!+ g_{Z'H^+H^-} {Z'}^\mu H^+ \overleftrightarrow {\partial _\mu } H^- \!+\! (g_{ W^\pm {\tilde{H}}_iH^\pm }{W^+}^\mu H^-\overleftrightarrow {\partial _\mu } {\tilde{H}}_i \nonumber \\{} & {} \!\!\! + g_{W^\pm A^0 H^\pm }{W^+}^\mu H^-\overleftrightarrow {\partial _\mu } A^0 + \mathrm{h.c.}), \end{aligned}$$
(43)

with

$$\begin{aligned} g_{Z{\tilde{H}}_iA^0}= & {} \frac{g_1}{2}(O_{1i}s_\beta - O_{2i}c_\beta ) s_\alpha c_Z \nonumber \\{} & {} - g_{Z'}Y'_\chi (O_{1i}s_\beta s_\alpha + O_{3i}c_\alpha )s_Z, \nonumber \\ g_{Z'{\tilde{H}}_iA^0}= & {} \frac{g_1}{2}(O_{2i}c_\beta - O_{1i}s_\beta )s_\alpha s_Z\nonumber \\{} & {} - g_{Z'} Y'_\chi (O_{1i}s_\beta s_\alpha + O_{3i}c_\alpha )c_Z, \nonumber \\ g_{\gamma H^+H^-}= & {} i e, \nonumber \\ g_{Z H^+H^-}= & {} i \left[ g_1 \left( \frac{1}{2}-s_W^2\right) c_Z+ g_{Z'} Y'_\chi s_\beta ^2c_Z\right] , \nonumber \\ g_{Z'H^+H^-}= & {} i \left[ -g_1 \left( \frac{1}{2}-s_W^2\right) s_Z+ g_{Z'}Y'_\chi s_\beta ^2c_Z\right] , \nonumber \\ g_{W^\pm {\tilde{H}}_iH^\pm }= & {} i\frac{g}{2} (-O_{1i}s_\beta + O_{2i}c_\beta ), \nonumber \\ g_{W^\pm A^0 H^\pm }= & {} \frac{g}{2} s_\alpha . \end{aligned}$$
(44)

The above interactions allow to compute the different partial decay widths for the \(Z'\) boson and the scalars. These are collected in Appendix C.

3 Parameter space scan

We use the code ScannerS [23] to scan the parameter space of the UN2HDM and to check whether the points in parameter space are allowed or excluded at the 95\(\%\) confidence level (CL). In this analysis, the following constraints are taken into account:Footnote 1

  • Theoretical constraints imposed by perturbative unitarity, boundedness from below and vacuum stability conditions [15]. These are applied after calculating the parameters of the scalar potential using the equations collected in Appendix A;

  • Electroweak precision constraints, which use fit results for the oblique parameters S, T and U shown in [25]. These are compared with UN2HDM predictions for those parameters, implemented in ScannerS using the results of [26, 27];

  • Flavour constraints based on fit results of [25], which set limits in the \((m_{H^\pm },\tan \beta )\) plane;

  • Compatibility of the SM-like scalar with the properties of the experimentally discovered Higgs boson;

  • Bounds from direct searches for beyond SM scalars.

To incorporate the constraints mentioned in the last two points, ScannerS provides an interface to HiggsSignals [28, 29] and HiggsBounds [30,31,32,33,34]. Some of the inputs required by those two tools are the branching ratios of all scalars. These are computed by the library N2HDECAY [35], which is also included in ScannerSFootnote 2.

Table 2 List of common input parameters for the parameter space scan
Full size table

In order to increase the efficiency of the scan [23], we parameterise the mixing matrix of CP-even scalars using (i) the effective couplings of the SM-like Higgs boson h to top quarks, \(c(ht{\overline{t}})\); (ii) the effective coupling to SM gauge bosons c(hVV), with \(V=W,Z\); \(O_{32}\) and \(\textrm{sign}(O_{31})\). Using these four parameters we are able to compute the three mixing angles in O, c.f. (18) while simultaneously constraining the couplings of h to be SM-like. The type of 2HDM used is also an input, specifying the flavour constraints applied. Finally, the parameters \(m_{Z'}\) and \(g_{Z'}Y_q'\) are required to extract the VEV of the singlet and the \(Z-Z'\) mixing angle. We set a reference mass \(M_{Z'} = 2\) TeV. The \(Z'\) production cross section is determined by the product \(g_{Z'}Y'_q\), which we set to 0.1. The list of common parameters used for all chosen benchmarks is presented in Table 2, together with their varying ranges. (The ranges for the new scalar masses are different for the various benchmarks examined, as shown in the next section.) With the parameter values shown in Table 2, we get \(u\sim 2.2~\textrm{TeV}\) and \(\theta _Z<10^{-3}\).Footnote 3

For the parameter-space points allowed by the aforementioned constraints, we compute the \(Z'\) cross section into SM final states, especially for \(Z'\rightarrow W^+W^-\), \(Z'\rightarrow Zh\) and \(Z'\rightarrow t{\overline{t}}\), to require agreement with direct searches. This is done using MadGraph [37], where we consider \(Z'\) to be produced from proton-proton collisions with a centre-of-mass energy of 13 TeV. Searches of \(Z'\) decaying into dijets are also considered but they are less constraining [38].

4 Benchmarks

Multiboson signals are generated in the UN2HDM by cascade decay of the \(Z'\) boson into new scalars, which subsequently decay into W, Z bosons or other scalars. We focus on three scenarios that are representative of various types of multiboson signals:

  1. 1.

    \(Z'\rightarrow H^+H^-\), with \(H^\pm \rightarrow W^\pm h\);

  2. 2.

    \(Z'\rightarrow H_2A^0\), with \(H_2\rightarrow H_1H_1\) and \(A^0\rightarrow b{\overline{b}}\);

  3. 3.

    \(Z'\rightarrow H_1A^0\), with \(H_1\rightarrow W^+W^-\) and \(A^0\rightarrow Zh\);

The Feynman diagrams for these decays are shown in Fig. 1.

Fig. 1
figure 1

Feynman diagrams for the \(Z'\) cascade decays studied in each benchmark scenario

Full size image

In scenario 1, for \(m_{H^\pm }\) of few hundreds of GeV, the four bosons in the final state are resolved, yielding a quadriboson signal not experimentally covered. In scenario 2, with \(H_2\) and \(A_0\) relatively light, the decay \(H_2 \rightarrow H_1 H_1 \rightarrow 4b\) produces a four-pronged jet, while \(A_0 \rightarrow b {\bar{b}}\) produces a two-pronged jet. Finally, scenario 3 with relatively-light \(H_1\) and \(A^0\) produces two four-pronged jets (when \(h^0 \rightarrow b {\bar{b}}\)) with different mass and flavour content. The latter two are partly covered by a search for a heavy resonance decaying into two massive jets [42] that unfortunately does not consider jet substructure for the discrimination against SM dijet background. Table 3 collects the range for scalar masses used for the scan in each benchmark scenario. The scalar branching ratios are computed by N2HDECAY, while for the branching ratios of the \(Z'\) boson we use the partial widths collected in Appendix C. In all cases, we consider the new leptons to be heavy enough not to play any role in \(Z'\) decays.

Table 3 Mass ranges (in GeV) of the new scalars used in each benchmark
Full size table

4.1 Scenario 1

In this case, the branching ratio for \(Z' \rightarrow H^+ H^-\) can be up to 0.3, while fulfilling the direct limits on other \(Z'\) decay modes (and possible improvements with more data). Figure 2 (top) shows the branching ratio for \(Z' \rightarrow H^+ H^-\) versus \(Z' \rightarrow Z h\). The vertical line corresponds to the experimental upper limit at 95% CL derived for this mass from current searches [39], assuming \(g_{Z'} Y'_q = 0.1\). A similar plot can be obtained for \(Z' \rightarrow H^+ H^-\) versus \(Z' \rightarrow W^+ W^-\), but the allowed area has similar shape and the limit [40] is less constraining. It is remarkable that \(Z' \rightarrow H^+ H^-\) can be sizeable while \(Z' \rightarrow W^+W^-\) and \(Z' \rightarrow Zh\) vanish. The reason is that the interactions mediating the latter two modes are proportional to powers of the VEV \(v_1\) of the scalar doublet that has non-vanishing hypercharge, either explicitly from \(\cos \beta \) factors, or through the \(Z-Z'\) mixing. The limits from \(Z' \rightarrow t {\bar{t}}\) [41] do not constrain the parameter space allowed by ScannerS (bottom panel), but an improvement by more than a factor of two would exclude the value of \(g_{Z'} Y'_q\) used in this benchmark.

Fig. 2
figure 2

Branching ratio for \(Z' \rightarrow H^+ H^-\) versus \(Z' \rightarrow Z h\) (top) and \(Z' \rightarrow t {\bar{t}}\) (bottom) resulting from the parameter space scan in scenario 1. The vertical lines correspond to the experimental upper bounds. The colour grading is related to the \(m_{H^\pm }\) value as shown on the right

Full size image
Fig. 3
figure 3

Branching ratios for \(Z' \rightarrow H^+ H^-\) versus \(H^\pm \rightarrow W^\pm h\) resulting from the parameter space scan in scenario 1. The colour grading is related to the \(m_{H^\pm }\) value as shown on the right

Full size image

Requiring agreement with direct searches, the branching ratios for \(Z' \rightarrow H^+ H^-\) versus \(H^\pm \rightarrow W^\pm h\) are presented in Fig. 3. One can see that the \(H^\pm \) can mostly decay into \(W^\pm h\) while having \(\text {BR}(Z' \rightarrow H^+ H^-) \simeq 0.3\). As a result, the maximum branching ratio for \(Z' \rightarrow W^+ h W^- h\) reached is 0.27, leading to a cross section times branching ratio of 29 fb for \(g_{Z'} Y'_q = 0.1\).

4.2 Scenario 2

Fig. 4
figure 4

Branching ratio for \(Z' \rightarrow H_2 A^0\) versus \(Z' \rightarrow Z h\) (top) and \(Z' \rightarrow t {\bar{t}}\) (bottom) resulting from the parameter space scan in scenario 2. The vertical lines correspond to the experimental upper bound. The colour grading is related to the \(m_{H_2}\) value as shown on the right

Full size image
Fig. 5
figure 5

Branching ratios for \(A^0 \rightarrow b {\bar{b}}\) versus versus \(H_2 \rightarrow H_1 H_1\) resulting from the parameter space scan in scenario 2. The colour grading is related to the value of \(\textrm{BR}(Z' \rightarrow H_2 A^0)\) as shown on the right

Full size image

In this scenario, the branching ratio for \(Z' \rightarrow H_2 A^0\) can range up to 0.35, while fulfilling the direct limits on other \(Z'\) decay modes (and possible improvements with more data). Figure 4 shows the branching ratio for \(Z' \rightarrow H_2 A^0\) versus \(Z' \rightarrow Z h\). The analogous figure considering \(Z' \rightarrow W^+W^-\) has similar shape, but the current limit does not further constrain the parameter space, and is omitted for brevity. Once more, we observe that the decay into SM bosons \(Z' \rightarrow W^+W^-\), \(Z' \rightarrow Zh\) can have negligible rates while \(Z' \rightarrow H_2 A^0\) is sizeable. This is again because the latter decay is not suppressed when the VEV \(v_1\) is small. Direct limits from \(Z' \rightarrow t {\bar{t}}\) do not constrain further the parameter space allowed by ScannerS, as seen in the bottom panel. However, an improvement by more than a factor of two would make this constraint relevant. The results have little dependence on the masses of \(H_2\) and \(A^0\), which range within a narrow interval [90, 110] GeV in this scenario.

The size of the signal in this scenario is determined – besides the \(Z'\) cross section production that is fixed by its mass and \(g_{Z'} Y'_q\) – by the three different branching ratios for \(Z' \rightarrow H_2 A^0\), \(H_2 \rightarrow H_1 H_1\) and \(A^0 \rightarrow b {\bar{b}}\). (Since \(H_1\) is the lightest scalar, it decays to \(b {\bar{b}}\) nearly all the time.) We present in Fig. 5 the branching ratios for \(A^0 \rightarrow b {\bar{b}}\) versus \(H_2 \rightarrow H_1 H_1\), with the colour corresponding to \(\text {Br}(Z' \rightarrow H_2 A^0)\), for points fulfilling the limit from \(Z' \rightarrow Zh\). Clearly, all three branching ratios can be sizeable, with a maximum combined branching ratio for \(Z' \rightarrow H_1 H_1 b {\bar{b}}\) of 0.25, leading to a product of the cross section times branching ratio of 27 fb for \(g_{Z'} Y'_q = 0.1\).

Fig. 6
figure 6

Branching ratio for \(Z' \rightarrow H_1 A^0\) versus \(Z' \rightarrow Z h\) (top) and \(Z' \rightarrow t {\bar{t}}\) (bottom) resulting from the parameter space scan in scenario 3. The vertical lines correspond to the experimental upper bound. The colour grading is related to the \(m_{H_2}\) value as shown on the right

Full size image

4.3 Scenario 3

The \(Z'\) decay mode considered in this scenario is similar to the previous one, but considering instead the lightest new scalar \(H_1\), and larger masses for \(H_1\) and \(A^0\) to allow for other decay modes. The branching ratio for \(Z' \rightarrow H_1 A^0\) can also range up to 0.35, see Fig. 6 (top). The only direct limit that partially constrains the parameter space allowed by ScannerS is \(Z' \rightarrow Zh\). As discussed in the previous two benchmarks, future improvements of the limit on \(Z' \rightarrow t {\bar{t}}\) would become constraining (bottom panel).

Fig. 7
figure 7

Branching ratios for \(A^0 \rightarrow Zh\) versus versus \(H_1 \rightarrow WW\) resulting from the parameter space scan in scenario 3. The colour grading is related to the value of \(\textrm{BR}(Z' \rightarrow H_1 A^0)\) as shown on the right

Full size image

The size of the multiboson signal in this scenario is determined by the three different branching ratios for \(Z' \rightarrow H_1 A^0\), \(H_1 \rightarrow W^+ W^-\) and \(A^0 \rightarrow Z h\). We present in Fig. 7 the branching ratios for \(A^0 \rightarrow Z {\bar{h}}\) versus \(H_1 \rightarrow W^+ W^-\), with the colour grading corresponding to the value of \(\text {BR}(Z' \rightarrow H_1 A^0)\), for points fulfilling the direct limit from \(Z' \rightarrow Zh\). We point out that the high density of points around \(\text {BR}(H_1 \rightarrow W^+ W^-) \simeq 0.7\) is motivated by the fact that, when both channels are kinematically open, \(\text {BR}(H_1 \rightarrow W^+ W^-) \simeq 2 \times \text {BR}(H_1 \rightarrow ZZ)\). The latter is suppressed for \(H_1\) masses below the ZZ threshold, thus the rate into \(W^+ W^-\) can range up to unity for the \(H_1\) masses considered. Overall, the three branching ratios can be sizeable, with a maximum combined branching ratio for \(Z' \rightarrow W^+ W^- Z h\) of 0.29, leading to a cross section times branching ratio of 31 fb for \(g_{Z'} Y'_q = 0.1\). In Table 4 we show the values of scalar masses and \(\tan \beta \) of the points with maximum cross section times branching ratio for each scenario.

Table 4 Scalar masses and \(\tan \beta \) values in the point with highest cross section times branching ratio for each benchmark
Full size table

5 Discussion

The LHC has set very stringent limits on the production of new resonances decaying into SM particles, with around 140 \(\hbox {fb}^{-1}\) of Run 2 data collected at 13 TeV. Still, many final states remain to be explored. Conspicuously, these unexplored signals might be sizeable, yet compatible with existing limits. We have explored three scenarios for the production of multiboson signals from the decay of a \(Z'\) boson. With cross sections times branching ratio around 30 fb in the three scenarios, around 4000 multiboson events could be produced with the collected luminosity.

A wide variety of signal topologies is possible, depending on the decay modes of the W, Z and h bosons in the final state. Given the high performance achieved by jet taggers [43, 44], already the hadronic decays of SM bosons (which have the largest branching ratios) are expected to provide a good sensitivity to these types of signals. We note that semileptonic signals are also possible, for example from \(H_1 \rightarrow W^+ W^-\), when one of the W bosons decays hadronically and the other one leptonically. A detailed study is out of the scope of this work.

In order to be sensitive to the multiboson signals proposed in this work, dedicated analyses or anomaly-detection methods are required. The signal produced in scenario 2 has a dijet topology, with one jet having four-pronged structure and the other one two-pronged. In scenario 3, when at least one of the W bosons and the Z/h decay hadronically, the signal also has a dijet topology. It has been demonstrated that anomaly detection tools such as CWoLa [45, 46] and SOFIE [47] have the potential to uncover such signals. The signal of scenario 1 has four resolved bosons in the final state. This makes its discrimination from the background technically more demanding, as it features a 4-body resonance plus two intermediate two-body resonances. It is likely that the SOFIE or CATHODE [48] methods are efficient in its detection too.

In summary, the results in this paper show that there are complex new physics signals that could be at reach with already collected LHC data, which motivates the use of generic tools and anomaly-detection strategies to pursue the discovery of any type of physics beyond the SM. We stress that such signals appear naturally in the context of popular SM extensions for which studies are usually focused only on the most simple final-state topologies (mostly direct decays into SM particles).