1 Introduction

The scalar particle at \(\sim 125\,\, \mathrm {GeV}\) discovered by the ATLAS [1] and CMS [2] experiments has so far shown to be consistent with the Standard Model (SM) Higgs-boson prediction. The Higgs boson was the last missing piece in the description of electroweak symmetry breaking (EWSB) and the generation of masses of fundamental particles within the SM. The measurement of the mass of this new state already reached a remarkable precision [3]:Footnote 1

$$\begin{aligned} m_H = 125.09 \pm 0.21 (\text {stat.}) \pm 0.11 (\text {syst.}). \end{aligned}$$
(1.1)

However, other properties of the Higgs boson, while being in agreement with the SM predictions, are still measured with relatively large uncertainties [6, 7]. Thus, even though any theory beyond the SM necessarily needs to accommodate a state corresponding to a SM-like Higgs boson at \(\sim 125\,\, \mathrm {GeV}\), there is still ample room for interpretations of the Higgs-boson signal with sizable deviations w.r.t. the SM prediction.

Supersymmetry (SUSY) is one of the most studied beyond the Standard Model (BSM) extensions. SUSY combines bosonic and fermionic degrees of freedom of the fundamental fields and spacetime itself. In particular, SUSY models predict two scalar particles for each SM fermion and a fermion for each SM gauge boson. The simplest version of such models is the Minimal Supersymmetric Standard Model (MSSM) [8, 9]. Besides the doubling of the SM particle content due to SUSY, the MSSM contains a second Higgs doublet which in the \({\mathcal {CP}}\)-conserving case leads to a physical spectrum of two \({\mathcal {CP}}\)-even, one \({\mathcal {CP}}\)-odd and two charged Higgs bosons. Both the lighter or the heavier \({\mathcal {CP}}\)-even scalar can be interpreted as the SM-like Higgs boson at \(\sim 125\,\, \mathrm {GeV}\) [10,11,12]. Despite its simplicity, the MSSM is capable of fixing a few shortcomings of the SM. If the breaking of SUSY takes place not too far away from the electroweak scale, the hierarchy problem [13, 14] is solved by additional quantum corrections from the SUSY partners that cancel large corrections to the Higgs mass from the heavy SM fermions. Apart from that, the extended spectrum leads to the unification of the three gauge couplings in a singular point at very high energies [15]. Due to the conservation of R-parity, the lightest supersymmetric particle (LSP) is stable and can contribute to the dark matter relic abundance [16, 17].

However, the MSSM does not address all the open problems of the SM, and also introduces new issues, motivating non-minimal SUSY extensions of the SM. The most prominent example is the Next-to-Minimal supersymmetric standard model (NMSSM) [18, 19] which extends the particle content of the MSSM by a gauge-singlet superfield. The \(Z_3\)-symmetric NMSSM provides a solution to the \(\mu \)-problem by naturally associating an adequate scale to the \(\mu \)-parameter in the MSSM superpotential [20, 21]. In the NMSSM, the fermionic component of the singlet superfield (called singlino) extends the neutral fermion sector of the MSSM to a total of five neutralinos. Assuming \({\mathcal {CP}}\)-conservation, the complex scalar component of the singlet superfield will extend the \({\mathcal {CP}}\)-even and the \({\mathcal {CP}}\)-odd scalar sector by an additional particle state, respectively.

Neither the MSSM nor the NMSSM accommodate neutrino masses and lepton-flavor violation in the neutrino sector. Therefore, a well motivated extension of the SM is the \(\mu \)-from-\(\nu \) Supersymmetric Standard Model (\(\mu \nu \mathrm {SSM}\)) [22, 23]. In this model, the particle content of the MSSM is extended by right-handed neutrino superfields. Since they are gauge-singlets, the \(\mu \)-problem can be solved in total analogy to the NMSSM. Remarkably, in the \(\mu \nu \mathrm {SSM}\) it is possible to accommodate neutrino masses and mixings in agreement with experiments via an electroweak seesaw mechanism, dynamically generated during the EWSB [22, 24,25,26,27,28]. In addition to the Higgs doublet fields also the right- and left-handed scalar neutrinos acquire a vacuum expectation value (vev). Thus, the \(\mu \nu \mathrm {SSM}\) solves the \(\mu \)- and the \(\nu \)-problem (neutrino masses) simultaneously without the need to introduce additional energy scales beyond the SUSY-breaking scale. In contrast to the (N)MSSM, R-parity and lepton number are not conserved, leading to a completely different phenomenology, characterized by distinct prompt or displaced decays of the LSP [29,30,31,32]. Although the LSP is not stable anymore, the \(\mu \nu \mathrm {SSM}\) can provide a dark matter candidate with a gravitino with a lifetime longer than the age of the universe [33,34,35,36]. The breaking of R-parity is induced by a neutrino Yukawa term, with the size of the couplings \(Y^\nu _{ij} \le 10^{-6}\) determined by the electroweak seesaw. Because of the values of \(Y^\nu _{ij}\), mixings between SM particles and their supersymmetric partners are suppressed. Nevertheless, the additional sources of mixing effects induce a conceptually modified spectrum compared to the MSSM and the NMSSM. The spectrum will be described in detail in Sect. 2.

SUSY relates the quartic couplings of the neutral scalar potential to the gauge couplings of the underlying field theory. Therefore, within SUSY the scalar masses can be predicted in terms of other model parameters, and the precise value of the SM-like Higgs-boson mass is of particular significance. However, the SM-like Higgs-boson mass predictions strongly depend on quantum corrections which can be calculated only to certain order in perturbation theory. Missing higher-order contributions lead to a sizable amount of uncertainty which is usually of a few GeV (see below for details), hence an order of magnitude larger than the experimental uncertainty. This is why a lot of effort is made to predict the Higgs mass to the highest possible precision [37]. We briefly summarize the status of Higgs-mass predictions in the MSSM, the NMSSM, and the \(\mu \nu \mathrm {SSM}\) in the following.

In the MSSM the tree-level mass can be predicted by just two SUSY parameters, i.e., the ratio of the vevs of the Higgs doublets \(\tan \beta \), and either the mass of the \({\mathcal {CP}}\)-odd Higgs boson \(M_A\) or the mass of the charged Higgs boson \(M_{H^\pm }\), leading to an upper bound given by the Z-boson mass. Large loop corrections are needed to achieve a Higgs-boson mass of \(\sim 125\,\, \mathrm {GeV}\). Beyond the one-loop level, the dominant two-loop corrections of \({{\mathcal {O}}}(\alpha _t\alpha _s)\) [38,39,40,41,42,43], \({{\mathcal {O}}}(\alpha _t^2)\) [44, 45], \({{\mathcal {O}}}(\alpha _b\alpha _s)\) [46, 47] and \({{\mathcal {O}}}(\alpha _t\alpha _b)\) [46] are known (here we use \(\alpha _f = (Y^f)^2/(4\pi )\), with \(Y^f\) denoting the fermion Yukawa coupling). These corrections, together with a resummation of leading and subleading logarithms from the top/stop sector [48] (see also [49, 50] for more details on this type of approach), a resummation of leading contributions from the bottom/sbottom sector [46, 47, 51,52,53,54] (see also [55, 56]) and momentum-dependent two-loop contributions [57, 58] (see also [59]) are included in the public code FeynHiggs[40, 48, 60,61,62,63,64,65,66,67]. The most recent version of FeynHiggs contains an improved effective field theory calculation relevant for large SUSY scales [64, 66, 68]. The complete two-loop QCD contributions in the \({\mathcal {CP}}\)-violating MSSM were calculated in Ref. [69], but not yet included in FeynHiggs. A (nearly) full two-loop effective potential (EP) calculation, including even the leading three-loop corrections, has also been published [70, 71], which is, however, not publicly available as a computer code. Furthermore, another leading three-loop calculation of \({{\mathcal {O}}}(\alpha _t\alpha _s^2)\), depending on the various SUSY mass hierarchies, has been performed [72, 73], resulting in the code H3m and is now available as a stand-alone code [74]. It was proven that regularization by dimensional reduction preserves supersymmetry at the required three-loop order [75]. A new calculation of the three-loop contributions of the \({{\mathcal {O}}}(\alpha _t\alpha _s^2)\) extends the validity of these corrections to the whole parameter space of the \({\mathcal {CP}}\)-conserving MSSM [76]. Most recently, the leading logarithmic terms of the \({{\mathcal {O}}}(\alpha _t\alpha _s^3)\) have been obtained (see the updated version of the public code Himalaya) [77]. The theoretical uncertainty on the lightest \({\mathcal {CP}}\)-even Higgs-boson mass within the MSSM from unknown higher-order contributions is still at the level of about \(2{-}3~\,\, \mathrm {GeV}\) for scalar top masses at the TeV-scale, where the actual uncertainty depends on the considered parameter region [62, 78,79,80,81,82].

In the NMSSM the full one-loop calculation including the momentum dependence has been performed in the \(\smash {\overline{\mathrm {DR}}}\) renormalization scheme in Refs. [83, 84], or in a mixed on-shell (OS)-\(\smash {\overline{\mathrm {DR}}}\) scheme in Refs. [85,86,87]. Dominant two-loop contributions of \({{\mathcal {O}}}(\alpha _t\alpha _s,\alpha _t^2)\) have been calculated in the leading logarithmic approximation [88, 89], and of \({{\mathcal {O}}}(\alpha _t\alpha _s, \alpha _b\alpha _s)\) in the \(\smash {\overline{\mathrm {DR}}}\) scheme in the EP approach [83]. The two-loop corrections involving only superpotential couplings were given in Ref. [90]. A two-loop calculation of the \({{\mathcal {O}}}(\alpha _t\alpha _s)\) corrections with the top/stop sector renormalized in the OS scheme or in the \(\smash {\overline{\mathrm {DR}}}\) scheme was provided in Ref. [91], while the two-loop corrections of \({{\mathcal {O}}}(\alpha _t^2)\) in the \({\mathcal {CP}}\)-violating NMSSM were calculated in a mixed OS-\(\smash {\overline{\mathrm {DR}}}\) scheme [92]. These contributions are implemented in the public code NMSSMCalc. A consistent combination of a full one-loop calculation with all corrections beyond one-loop in the MSSM approximation was given in Ref. [87]. According to a comparison of the various two-loop contributions, at present the theoretical uncertainties from unknown higher-order corrections in the NMSSM are expected to be still larger than for the MSSM [92,93,94].

Beyond the MSSM and the NMSSM, only generic \(\smash {\overline{\mathrm {DR}}}\)-calculations of Higgs-boson mass corrections exist publicly available. An automated calculation of the full one-loop corrections, supplemented by partial two-loop corrections to neutral scalars [90] is implemented in the Mathematica package SARAH [95, 96], which can be used to produce a spectrum generator based on the public code SPheno [97]. A hybrid Higgs-boson mass calculation combining effective field theory and fixed-order calculations for a generic class of SUSY models is publicly available in the code FlexibleSUSY [98], also using the expression for the renormalization group equations and fixed-order self-energies as they are calculated by SARAH.

In a previous publication we presented the first calculation of radiative corrections to the neutral scalars in a mixed OS-\(\smash {\overline{\mathrm {DR}}}\) scheme for the \(\mu \nu \mathrm {SSM}\) with only one generation of right-handed neutrinos [99]. We described in detail the renormalization of the scalar potential, including the full one-loop quantum corrections. We consistently combined the full one-loop corrections with the leading MSSM-like two-loop contributions using FeynHiggs. We showed that the contributions from the (s)top- and the (s)bottom-sector are also dominant in the \(\mu \nu \mathrm {SSM}\), therefore proving that the combination of the one-loop result together with the two-loop contributions from FeynHiggs provides a calculation of the SM-like Higgs-boson mass at a similar accuracy as the NMSSM prediction. In this work, we go one step beyond and extend the calculation to the full \(\mu \nu \mathrm {SSM}\) with three generations of right-handed neutrinos. A striking difference between the one- and the three-generation case is that in the latter case the neutrino sector can be described in agreement with experimental results without having to rely on the radiative generation of neutrino masses. On account of this, we are able to present benchmark scenarios of the \(\mu \nu \mathrm {SSM}\) accurately accommodating a SM-like Higgs boson at \(\sim 125\,\, \mathrm {GeV}\), as well as correct neutrino mass differences and mixing angles. In addition, we show that it is possible to simultaneously explain two excesses measured at LEP and CMS at a mass of \(\sim 96\,\, \mathrm {GeV}\) at the \(1\sigma \) level. An earlier study in the \(\mu \nu \mathrm {SSM}\), before the discovery of the Higgs boson, discussing Higgs bounds and possible signals at the LHC, and suggesting the re-analysis of the LEP data in light of the excess, can be found in Ref. [100].

The paper is organized as follows. In Sect. 2 we describe the model and explain the particle mixings in each sector. In Sect. 3 we give details about the renormalization of the neutral scalar potential at the one-loop level, including the full set of free parameters of the \(\mu \nu \mathrm {SSM}\). We present the renormalization conditions applied to extract the parameter counterterms, either in the neutral scalar or the neutral fermion sector. In Sect. 4 we explain the extraction of the one-loop corrections to the \({\mathcal {CP}}\)-even scalar masses, based on the renormalization prescription introduced before. We also describe the incorporation of higher-order contributions from the MSSM. In Sect. 5 we discuss a set of benchmark scenarios with several light Higgs bosons. We conclude in Sect. 6.

2 The model: \(\mu \nu \mathrm {SSM}\) with three generations of right handed neutrinos

The superpotential of the \(\mu \nu \mathrm {SSM}\) with three generations of right-handed neutrinos is written as

$$\begin{aligned} W&= \; \epsilon _{ab} \left( Y^e_{ij} \, {\hat{H}}_d^a\, {\hat{L}}^b_i \, {\hat{e}}_j^c + Y^d_{ij} \, {\hat{H}}_d^a\, {\hat{Q}}^{b}_{i} \, {\hat{d}}_{j}^{c} + Y^u_{ij} \, {\hat{H}}_u^b\, {\hat{Q}}^{a}_{i} \, {\hat{u}}_{j}^{c} \right) \nonumber \\&\quad + \; \epsilon _{ab} \left( Y^{\nu }_{ij} \, {\hat{H}}_u^b\, {\hat{L}}^a_i \, {\hat{\nu }}^c_j - \lambda _{i} \, {\hat{\nu }}^c_i\, {\hat{H}}_u^b {\hat{H}}_d^a \right) + \frac{1}{3} \kappa {_{ijk}} {\hat{\nu }}^c_i{\hat{\nu }}^c_j{\hat{\nu }}^c_k \, \end{aligned}$$
(2.1)

where \({\hat{H}}_d^T=({\hat{H}}_d^0, {\hat{H}}_d^-)\) and \({\hat{H}}_u^T=({\hat{H}}_u^+, {\hat{H}}_u^0)\) are the MSSM-like doublet Higgs superfields, \({\hat{Q}}_i^T=({\hat{u}}_i, {\hat{d}}_i)\) and \({\hat{L}}_i^T=({\hat{\nu }}_i, {\hat{e}}_i)\) are the left-chiral quark and lepton superfield doublets, and \({\hat{u}}_{i}^{c}\), \({\hat{d}}_{i}^{c}\), \({\hat{e}}_i^c\) and \(\hat{\nu }_i^c\) are the right-chiral quark and lepton superfields. i and j are family indices running from one to three and \(a,b=1,2\) are indices of the fundamental representation of SU(2) with \(\epsilon _{ab}\) the totally antisymmetric tensor and \(\varepsilon _{12}=1\). The color indices are not written out. \(Y^u\), \(Y^d\) and \(Y^e\) are the usual Yukawa couplings also present in the MSSM. The trilinear singlet self couplings \(\kappa _{ijk}\) and the trilinear coupling with the Higgs doublets \(\lambda _i\) in the second row are analogues to the couplings of the singlet in the superpotential of the \(Z_3\)-symmetric NMSSM. The \(\mu \)-term is generated dynamically after the spontaneous EWSB, when the right-handed sneutrinos obtain a vev. The \(\kappa \)-term forbids a global U(1) symmetry avoiding the existence of a Goldstone boson in the \({\mathcal {CP}}\)-odd sector. The remarkable difference to the NMSSM is the additional Yukawa coupling \(Y^\nu \) which induces explicit breaking of R-parity through the \(\lambda \)- and \(\kappa \)-terms, and which justifies the interpretation of the singlet superfields as right-handed neutrino superfields. It should be pointed out that in this case lepton number is not conserved anymore, and also the flavor symmetry in the leptonic sector is broken. A more complete motivation of this superpotential can be found in Refs. [22, 24, 29].

Working in the framework of low-energy SUSY the corresponding soft SUSY-breaking Lagrangian is given by

$$\begin{aligned} -\mathcal {L}_{\text {soft}}&= \; \epsilon _{ab} \left( T^e_{ij} \, H_d^a \, {\widetilde{L}}^b_{iL} \, {\widetilde{e}}_{jR}^* + T^d_{ij} \, H_d^a\, {\widetilde{Q}}^b_{iL} \, {\widetilde{d}}_{jR}^{*} \right. \nonumber \\&\quad \left. +\, T^u_{ij} \, H_u^b {\widetilde{Q}}^a_{iL} {\widetilde{u}}_{jR}^* + \text {h.c.} \right) \nonumber \\&\quad + \; \epsilon _{ab} \left( T^{\nu }_{ij} \, H_u^b \, {\widetilde{L}}^a_{iL} \widetilde{\nu }_{jR}^* - T^{\lambda }_{i} \, \widetilde{\nu }_{iR}^* \, H_d^a H_u^b \right. \nonumber \\&\quad \left. + \frac{1}{3} T^{\kappa }_{ijk} \, \widetilde{\nu }_{iR}^* \widetilde{\nu }_{jR}^* \widetilde{\nu }_{kR}^* \ + \text {h.c.}\right) \nonumber \\&\quad + \; \left( m_{\widetilde{Q}}^2\right) _{ij} \widetilde{Q}_{iL}^{a*} \widetilde{Q}^a_{jL} +\left( m_{\widetilde{u}}^{2}\right) _{ij} \widetilde{u}_{iR}^* {\widetilde{u}}_{jR} \nonumber \\&\quad + \left( m_{\widetilde{d}}^2\right) _{ij} \widetilde{d}_{iR}^* {\widetilde{d}}_{jR} + \left( m_{\widetilde{L}}^2\right) _{ij} \widetilde{L}_{iL}^{a*} \widetilde{L}^a_{jL} \nonumber \\&\quad + \; \left( m_{\widetilde{\nu }}^2\right) _{ij} \widetilde{\nu }_{iR}^* \widetilde{\nu }_{jR} + \left( m_{\widetilde{e}}^2\right) _{ij} \widetilde{e}_{iR}^* {\widetilde{e}}_{jR} \nonumber \\&\quad +\, m_{H_d}^2 {H^a_d}^* H^a_d {+} m_{H_u}^2 {H^a_u}^* H^a_u {+} \left( m_{H_d\widetilde{L}}^2 \right) _{i}H_d^{a*} \widetilde{L}_{iL}^a \nonumber \\&\quad + \; \frac{1}{2}\, \left( M_3\, {{\widetilde{g}}}\, {{\widetilde{g}}} + M_2\, {\widetilde{W}}\, {\widetilde{W}} +M_1\, {{\widetilde{B}}}^0 \, {{\widetilde{B}}}^0 + \text {h.c.} \right) . \end{aligned}$$
(2.2)

In the first four lines the fields denote the scalar components of the corresponding superfields. In the last line the fields denote the fermionic superpartners of the gauge bosons. The scalar trilinear parameters \(T^{e,\nu ,d,u,\lambda ,\kappa }\) correspond to the trilinear couplings in the superpotential. The soft mass parameters \(m_{\widetilde{Q},\widetilde{u},\widetilde{d}, \widetilde{L},\widetilde{\nu },\widetilde{e}}^2\) are hermitian \(3\times 3\) matrices in family space. \(m_{H_d,H_u}^2\) are the soft masses of the doublet Higgs fields.

We will neglect flavor mixing in the squark and the quark sector, so the soft masses will be diagonal and we write \(m_{\widetilde{Q}_{i} }^2\), \(m_{\widetilde{u}_{i}}^2\) and \(m_{\widetilde{d}_{i}}^2\), as well as for the soft trilinears \(T^u_i=A^u_i Y^u_i\), \(T^d_i=A^d_i Y^d_i\), where the summation convention on repeated indices is not implied, and the quark Yukawas \(Y^u_{ii}=Y^u_i\) and \(Y^d_{ii}=Y^d_i\) are chosen to be diagonal. For the sleptons we define \(T^e_{ij}=A^e_{ij}Y^e_{ij}\) and \(T^\nu _{ij}=A^\nu _{ij} Y^\nu _{ij}\), again without summation over repeated indices.

\(m_{H_d\widetilde{L}}^2\) is a 3-dimensional vector in family space, which is always regarded to be absent in the tree-level Lagrangian of the \(\mu \nu \mathrm {SSM}\), because it spoils the electroweak seesaw mechanism. We include it here, because the operator is generated at the one-loop level and the parameters \(m_{H_d\widetilde{L}}^2\) are needed to renormalize the scalar potential. The same reasoning applies for the non-diagonal elements of the soft slepton masses \((m_{\widetilde{L}}^2)_{i \ne j}\) and \((m_{\widetilde{\nu }}^2)_{i \ne j}\).

Theoretically, the absence of soft mass parameters mixing different fields at tree level, \(( m_{H_d\widetilde{L}}^2 )_{i}\), \((m_{\widetilde{L}}^2)_{i \ne j}\), \((m_{\widetilde{Q}}^2)_{i \ne j}\), etc., can be justified by the diagonal structure of the Kähler metric in certain supergravity models, or when the dilaton field is the source of SUSY breaking in string constructions [29]. Notice also that when the down-type Higgs doublet superfield is interpreted as a fourth family of leptons the parameters \(m_{H_d\widetilde{L}}^2\) can be seen as off-diagonal elements of \(m_{\widetilde{L}}^2\) [101].

After the EWSB the neutral scalar fields will acquire a vev. This includes the left- and right-handed sneutrinos, because they are not protected by lepton-number conservation as in the MSSM and the NMSSM. We define the decomposition

$$\begin{aligned} H_d^0= & {} \frac{1}{\sqrt{2}} \left( H_{d}^{\mathcal {R}} + v_d + \text {i}\, H_{d}^{\mathcal {I}}\right) , \end{aligned}$$
(2.3)
$$\begin{aligned} H^0_u= & {} \frac{1}{\sqrt{2}} \left( H_{u}^{\mathcal {R}} + v_u + \text {i}\, H_{u}^{\mathcal {I}}\right) , \end{aligned}$$
(2.4)
$$\begin{aligned} \widetilde{\nu }_{iR}= & {} \frac{1}{\sqrt{2}} \left( \widetilde{\nu }^{\mathcal {R}}_{iR}+ v_{iR} + \text {i}\, \widetilde{\nu }^{\mathcal {I}}_{iR}\right) , \end{aligned}$$
(2.5)
$$\begin{aligned} \widetilde{\nu }_{iL}= & {} \frac{1}{\sqrt{2}} \left( \widetilde{\nu }_{iL}^{\mathcal {R}} + v_{iL} + \text {i}\, \widetilde{\nu }_{iL}^{\mathcal {I}}\right) , \end{aligned}$$
(2.6)

in such a way that after the EWSB they develop the real vevs

$$\begin{aligned} \langle H_d^0 \rangle= & {} \frac{v_d}{\sqrt{2}}\ , \, \quad \langle H_u^0 \rangle = \frac{v_u}{\sqrt{2}}, \, \nonumber \\ \langle \widetilde{\nu }_{iR}\rangle= & {} \frac{v_{iR}}{\sqrt{2}}\ , \, \quad \langle \widetilde{\nu }_{iL} \rangle = \frac{v_{iL}}{\sqrt{2}}, \end{aligned}$$
(2.7)

which is valid assuming \({\mathcal {CP}}\)-conservation, as we will do throughout this paper.

The neutral scalar potential of the \(\mu \nu \mathrm {SSM}\) is given at tree level with all parameters chosen to be real by the soft terms and the F- and D-term contributions of the superpotential. One finds

$$\begin{aligned} V^{(0)} = V_{\text {soft}} + V_F + V_D, \end{aligned}$$
(2.8)

with

$$\begin{aligned} V_{\text {soft}}&= \; \left( T^{\nu }_{ij} \, H_u^0\, \widetilde{\nu }_{iL} \, \widetilde{\nu }_{jR}^* - T^{\lambda }_{i} \, \widetilde{\nu }_{iR}^*\, H_d^0 H_u^0 \right. \nonumber \\&\quad \left. +\, \frac{1}{3} T^{\kappa }_{ijk} \, \widetilde{\nu }_{iR}^* \widetilde{\nu }_{jR}^* \widetilde{\nu }_{kR}^*\ + \text {h.c.} \right) \nonumber \\&\quad + \; \left( m_{\widetilde{L}}^2\right) _{ij} \widetilde{\nu }_{iL}^* \widetilde{\nu }_{jL} + \left( m_{\widetilde{\nu }}^2\right) _{ij} \widetilde{\nu }_{iR}^* \widetilde{\nu }_{jR} \nonumber \\&\quad + m_{H_d}^2 {H^0_d}^* H^0_d + m_{H_u}^2 {H^0_u}^* H^0_u, \end{aligned}$$
(2.9)
$$\begin{aligned} V_{F}&= \; \lambda _{j}\lambda _{j} H^0_{d}H_d^0{^{^*}}H^0_{u}H_u^0{^{^*}} + \lambda _{i}\lambda _{j} {\tilde{\nu }}^{*}_{iR}{\tilde{\nu }}_{jR}H^0_{d}H_d^0{^*} \nonumber \\&\quad + \lambda _{i}\lambda _{j} {\tilde{\nu }}^{*}_{iR}{\tilde{\nu }}_{jR} H^0_{u}H_u^0{^*} + \; \kappa _{ijk}\kappa _{ljm}{\tilde{\nu }}^*_{iR}{\tilde{\nu }}_{lR} {\tilde{\nu }}^*_{kR}{\tilde{\nu }}_{mR} \nonumber \\&\quad - \left( \kappa _{ijk}\lambda _{j}{\tilde{\nu }}^{*}_{iR}{\tilde{\nu }}^{*}_{kR} H_d^{0*}H_u^{0*} -Y^{\nu }_{ij}\kappa _{ljk}{\tilde{\nu }}_{iL}{\tilde{\nu }}_{lR}{\tilde{\nu }}_{kR}H^0_{u} \right. \nonumber \\&\quad + \; \left. Y^{\nu }_{ij}\lambda _{j}{\tilde{\nu }}_{iL} H_d^{0*}H_{u}^{0*}H^0_{u} {+}{Y^{\nu }_{ij}}\lambda _{k} {\tilde{\nu }}_{iL}^{*}{\tilde{\nu }}_{jR}{\tilde{\nu }}_{kR}^* H^0_{d} {+} \text {h.c.}\right) \nonumber \\&\quad + \; Y^{\nu }_{ij}{Y^{\nu }_{ik}} {\tilde{\nu }}^{*}_{jR} {\tilde{\nu }}_{kR}H^0_{u}H_u^0{^*} + Y^{\nu }_{ij}{Y^{\nu }_{lk}}{\tilde{\nu }}_{iL}{\tilde{\nu }}_{lL}^{*}{\tilde{\nu }}_{jR}^{*} {\tilde{\nu }}_{kR} \nonumber \\&\quad + Y^{\nu }_{ji}{Y^{\nu }_{ki}}{\tilde{\nu }}_{jL}{\tilde{\nu }}_{kL}^* H^0_{u}H_u^{0*}, \end{aligned}$$
(2.10)
$$\begin{aligned} V_D&= \; \frac{1}{8}\left( g_1^{2}+g_2^{2}\right) \left( \widetilde{\nu }_{iL} \widetilde{\nu }_{iL}^*+H^0_d {H^0_d}^* - H^0_u {H^0_u}^* \right) ^{2}. \end{aligned}$$
(2.11)

2.1 The neutral scalar sector

Using the decomposition from Eqs. (2.3)–(2.6) the linear and bilinear terms in the fields define the tadpoles \(T_{\varphi }\) and the scalar \({\mathcal {CP}}\)-even and \({\mathcal {CP}}\)-odd neutral mass matrices \(m_{\varphi }^2\) and \(m_{\sigma }^2\) after electroweak symmetry breaking,

$$\begin{aligned} V_H&=\cdots - {T_{\varphi }}_i\varphi _i + \frac{1}{2} \varphi ^T_i \left( m_{\varphi }^2 \right) _{ij} \varphi _j \nonumber \\&\quad + \frac{1}{2} \sigma ^T_i \left( m_{\sigma }^2 \right) _{ij} \sigma _j + \cdots . \end{aligned}$$
(2.12)

where we collectively denote with \(\varphi ^T=(H_d^{\mathcal {R}},H_u^{\mathcal {R}}, \widetilde{\nu }_{1R}^{\mathcal {R}}, \widetilde{\nu }_{2R}^{\mathcal {R}}, \widetilde{\nu }_{3R}^{\mathcal {R}} ,\widetilde{\nu }_{1L}^{\mathcal {R}} ,\widetilde{\nu }_{2L}^{\mathcal {R}} ,\widetilde{\nu }_{3L}^{\mathcal {R}})\) and \(\sigma ^T=(H_d^{\mathcal {I}},H_u^{\mathcal {I}}, \widetilde{\nu }_{1R}^{\mathcal {I}}, \widetilde{\nu }_{2R}^{\mathcal {I}}, \widetilde{\nu }_{3R}^{\mathcal {I}} ,\widetilde{\nu }_{1L}^{\mathcal {I}} ,\widetilde{\nu }_{2L}^{\mathcal {I}} ,\widetilde{\nu }_{3L}^{\mathcal {I}})\) the \({\mathcal {CP}}\)-even and \({\mathcal {CP}}\)-odd scalar fields, respectively. The linear terms are only allowed for \({\mathcal {CP}}\)-even fields and given by

$$\begin{aligned} T_{H_d^{\mathcal {R}}}&=-m_{H_d}^2v_d - \left( m_{H_d\widetilde{L}}^2 \right) _{i} v_{iL} \nonumber \\&\quad - \frac{1}{8}\left( g_1^2+g_2^2\right) v_d \left( v_d^2 + v_{iL}v_{iL} - v_u^2\right) \nonumber \\&\quad -\frac{1}{2} v_d v_u^2 \lambda _i \lambda _i + \frac{1}{\sqrt{2}} v_u v_{iR} T^\lambda _i \nonumber \\&\quad + \frac{1}{2} v_u^2 Y^\nu _{ji} \lambda _i v_{jL} - \frac{1}{2} v_d v_{iR} \lambda _i v_{jR} \lambda _j \nonumber \\&\quad +\frac{1}{2} v_u \kappa _{ikj} \lambda _i v_{jR} v_{kR} + \frac{1}{2} v_{iR} \lambda _i v_{jL} Y^\nu _{jk} v_{kR} \; , \end{aligned}$$
(2.13)
$$\begin{aligned} T_{H_u^{\mathcal {R}}}&=-\,m_{H_u}^2v_u+ \frac{1}{8}\left( g_1^2+g_2^2\right) v_u\left( v_d^2+v_{iL}v_{iL}-v_u^2\right) \nonumber \\&\quad -\frac{1}{2} v_d^2 v_u \lambda _i \lambda _i + \frac{1}{\sqrt{2}} v_d v_{iR} T^\lambda _i + v_d v_u Y^\nu _{ji} \lambda _i v_{jL} \nonumber \\&\quad - \frac{1}{\sqrt{2}} v_{iL} T^\nu _{ij} v_{jR} - \frac{1}{2} v_u v_{iR} \lambda _i v_{jR} \lambda _j \nonumber \\&\quad -\frac{1}{2} v_u Y^\nu _{ji} Y^\nu _{ki} v_{jL} v_{kL} - \frac{1}{2} v_u Y^\nu _{ij} Y^\nu _{ik} v_{jR} v_{kR} \nonumber \\&\quad + \frac{1}{2} v_d \kappa _{ijk} \lambda _i v_{jR} v_{kR} - \frac{1}{2} Y^\nu _{li} \kappa _{ikj} v_{jR} v_{kR} v_{lL}, \end{aligned}$$
(2.14)
$$\begin{aligned} T_{\widetilde{\nu }_{iR}^{\mathcal {R}}}&= -\,\left( m_{\widetilde{\nu }}^2\right) _{ij} v_{jR} - \frac{1}{\sqrt{2}} v_u v_{jL} T^\nu _{ji} - \frac{1}{2} v_u^2 Y^\nu _{ji} Y^\nu _{jk} v_{kR} \nonumber \\&\quad + v_d v_u \kappa _{ijk} \lambda _j v_{kR} - \frac{1}{\sqrt{2}} T^\kappa _{ijk} v_{jR} v_{kR} \nonumber \\&\quad +\frac{1}{2} v_d v_{jL} Y^\nu _{ji} v_{kR} \lambda _k - v_u Y^\nu _{lj} \kappa _{ijk} v_{kR} v_{lL} \nonumber \\&\quad - \frac{1}{2} v_{jL} Y^\nu _{ji} v_{kL} Y^\nu _{kl} v_{lR} - \kappa _{ijm} \kappa _{jlk} v_{kR} v_{lR} v_{mR} \nonumber \\&\quad -\frac{1}{2} \left( v_d^2 + v_u^2 \right) \lambda _i \lambda _j v_{jR} + \frac{1}{2} v_d v_{jL} Y^\nu _{jk} v_{kR} \lambda _i \nonumber \\&\quad +\frac{1}{\sqrt{2}} v_d v_u T^\lambda _i, \end{aligned}$$
(2.15)
$$\begin{aligned} T_{\widetilde{\nu }_{iL}^{\mathcal {R}}}&=-\, \left( m_{\widetilde{L}}^2\right) _{ij}v_{jL} - \left( m_{H_d\widetilde{L}}^2 \right) _{i}v_d \nonumber \\&\quad - \frac{1}{8}\left( g_1^2+g_2^2\right) v_{iL}\left( v_d^2+v_{jL}v_{jL}-v_u^2\right) \nonumber \\&\quad +\frac{1}{2} v_d v_u^2 Y^\nu _{ij} \lambda _{j} - \frac{1}{\sqrt{2}} v_u v_{jR} T^\nu _{ij} \nonumber \\&\quad - \frac{1}{2} v_u^2 Y^\nu _{ij} Y^\nu _{kj} v_{kL} + \frac{1}{2} v_d v_{jR} Y^\nu _{ij} v_{kR} \lambda _k \nonumber \\&\quad -\frac{1}{2} v_u Y^\nu _{ij} \kappa _{jkl} v_{kR} v_{lR} - \frac{1}{2} v_{jR} Y^\nu _{ij} v_{kL} Y^\nu _{kl} v_{lR}. \end{aligned}$$
(2.16)

The tadpoles vanish in the true vacuum of the model. We stress that the proportionality of \(T^\nu \) and \(Y^\nu \), as used above, assures that the condition \(v_{iL}\ll v_d, v_u, v_{iR}\), is fulfilled after solving the minimization conditions in Eq. (2.16). This is essential for the generation of the electroweak seesaw mechanism and the origin for the smallness of the left-handed neutrino masses, without introducing any further energy scale. During the renormalization procedure they will be treated as OS parameters, i.e., finite corrections will be canceled by their corresponding counterterms. This guarantees that the vacuum is stable w.r.t. quantum corrections.

The bilinear terms

$$\begin{aligned} m_{\varphi }^2= \left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} m_{H_{d}^{\mathcal {R}}H_{d}^{\mathcal {R}}}^{2}&{} m_{H_{d}^{\mathcal {R}}H_{u}^{\mathcal {R}}}^{2} &{} m_{H_{d}^{\mathcal {R}}\widetilde{\nu }_{jR}^{\mathcal {R}}}^{2} &{} m_{H_{d}^{\mathcal {R}}\widetilde{\nu }_{jL}^{\mathcal {R}}}^{2}\\ m_{H_{u}^{\mathcal {R}}H_{d}^{\mathcal {R}}}^{2} &{} m_{H_{u}^{\mathcal {R}}H_{u}^{\mathcal {R}}}^{2}&{} m_{H_{u}^{\mathcal {R}}\widetilde{\nu }_{jR}^{\mathcal {R}}}^{2} &{} m_{H_{u}^{\mathcal {R}}\widetilde{\nu }_{jL}^{\mathcal {R}}}^{2}\\ m_{\widetilde{{\nu }}^{\mathcal {R}}_{iR} H_{d}^{\mathcal {R}}}&{} m_{\widetilde{{\nu }}^{\mathcal {R}}_{iR} H_{u}^{\mathcal {R}}}&{} m_{\widetilde{{\nu }}^{\mathcal {R}}_{iR} \widetilde{{\nu }}^{\mathcal {R}}_{jR}}^{2} &{} m_{\widetilde{{\nu }}^{\mathcal {R}}_{iR} \widetilde{{\nu }}^{\mathcal {R}}_{jL}}^{2}\\ m_{\widetilde{\nu }_{iL}^{\mathcal {R}} H_{d}^{\mathcal {R}}}^{2} &{} m_{\widetilde{\nu }_{iL}^{\mathcal {R}} H_{u}^{\mathcal {R}}}^{2} &{} m_{\widetilde{\nu }_{iL}^{\mathcal {R}} \widetilde{\nu }^{\mathcal {R}}_{jR}}^{2} &{} m_{\widetilde{\nu }_{iL}^{\mathcal {R}} \widetilde{\nu }_{jL}^{\mathcal {R}}}^{2} \end{array} \right) , \end{aligned}$$
(2.17)

and

$$\begin{aligned} m^2_{\sigma }= \left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} m_{H_{d}^{\mathcal {I}}H_{d}^{\mathcal {I}}}^{2} &{} m_{H_{d}^{\mathcal {I}}H_{u}^{\mathcal {I}}}^{2} &{} m_{H_{d}^{\mathcal {I}}\widetilde{\nu }_{jR}^{\mathcal {I}}}^{2} &{} m_{H_{d}^{\mathcal {I}}\widetilde{\nu }_{jL}^{\mathcal {I}}}^{2}\\ m_{H_{u}^{\mathcal {I}}H_{d}^{\mathcal {I}}}^{2} &{} m_{H_{u}^{\mathcal {I}}H_{u}^{\mathcal {I}}}^{2}&{} m_{H_{u}^{\mathcal {I}}\widetilde{\nu }_{jR}^{\mathcal {I}}}^{2} &{} m_{H_{u}^{\mathcal {I}}\widetilde{\nu }_{jL}^{\mathcal {I}}}^{2}\\ m_{\widetilde{{\nu }}^{\mathcal {I}}_{iR} H_{d}^{\mathcal {I}}}^2&{} m_{\widetilde{{\nu }}^{\mathcal {I}}_{iR} H_{u}^{\mathcal {I}}}^2&{} m_{\widetilde{{\nu }}^{\mathcal {I}}_{iR} \widetilde{{\nu }}^{\mathcal {I}}_{jR}}^{2} &{} m_{\widetilde{{\nu }}^{\mathcal {I}}_{iR} \widetilde{{\nu }}^{\mathcal {I}}_{jL}}^{2}\\ m_{\widetilde{\nu }_{iL}^{\mathcal {I}} H_{d}^{\mathcal {I}}}^{2} &{} m_{\widetilde{\nu }_{iL}^{\mathcal {I}} H_{u}^{\mathcal {I}}}^{2} &{} m_{\widetilde{\nu }_{iL}^{\mathcal {I}} \widetilde{\nu }^{\mathcal {I}}_{jR}}^{2} &{} m_{\widetilde{\nu }_{iL}^{\mathcal {I}} \widetilde{\nu }_{jL}^{\mathcal {I}}}^{2} \end{array} \right) , \end{aligned}$$
(2.18)

are \(8\times 8\) matrices in family space whose rather lengthy entries are given in the Appendices A.1 and A.2 . We transform to the mass eigenstate basis of the \({\mathcal {CP}}\)-even scalars through a unitary transformation defined by the matrix \(U^H\) that diagonalizes the mass matrix \(m_{\varphi }^2\),

$$\begin{aligned} U^H m^2_{\varphi }\ {U^H}^{^T}= m^2_{h}, \end{aligned}$$
(2.19)

with

$$\begin{aligned} \varphi = {U^H}^{^T} h, \end{aligned}$$
(2.20)

where the \(h_i\) are the \({\mathcal {CP}}\)-even scalar fields in the mass eigenstate basis. Without \({\mathcal {CP}}\)-violation in the scalar sector the matrix \(U^H\) is real. Similarly, for the \({\mathcal {CP}}\)-odd scalars we define the rotation matrix \(U^A\) that diagonalizes the mass matrix \(m_{\sigma }^2\),

$$\begin{aligned} U^A m^2_{\sigma }\ {U^A}^{^T}= m^2_{A} \ , \qquad \text {with } \sigma ={U^A}^T A, \end{aligned}$$
(2.21)

which includes the neutral Goldstone boson \(A_1=G^0\). Because of the smallness of the neutrino Yukawa couplings \(Y^\nu _{ij}\), which also implies that the left-handed sneutrino vevs \(v_{iL}\) have to be small, so that the tadpole coefficients vanish at tree level [24], the mixing of the left-handed sneutrinos with the doublet fields and the singlets will be small.

It is a well known fact that the quantum corrections to the Higgs potential are highly significant in supersymmetric models, see e.g. Refs. [78, 102, 103] for reviews. As in the NMSSM [18], the upper bound on the lowest Higgs mass squared at tree level is relaxed through additional contributions from the right-handed sneutrinos [24];

$$\begin{aligned}&m_{h_1}^{(0)} \le M_Z^2 \left( \cos ^2 2\beta + \frac{2 \pmb {\lambda }^2}{g_1^2+g_2^2} \sin ^2 2\beta \right) , \nonumber \\&\quad \text {with} \quad \pmb {\lambda }^2 := \lambda _1^2+\lambda _2^2+\lambda _3^2. \end{aligned}$$
(2.22)

Nevertheless, quantum corrections were still shown to contribute significantly especially to the prediction of the SM-like Higgs-boson mass [85, 87, 94, 99, 104,105,106,107,108]. In Ref. [99] we already investigated how important the unique loop corrections of the \(\mu \nu \mathrm {SSM}\) beyond the NMSSM are in realistic scenarios, considering only one generation of right-handed neutrinos, finding only negligible differences compared to the NMSSM-like corrections. This is related to the small size of the neutrino Yukawas \(Y^\nu \) compared to the other couplings in the superpotential. In this paper we go beyond Ref. [99] and investigate the complete \(\mu \nu \mathrm {SSM}\) with three right-handed neutrinos. Thus, genuine effects from the \(\mu \nu \mathrm {SSM}\) are guaranteed to play a role in the prediction of the SM-like Higgs-boson mass just by the presence of additional singlets, whose couplings to the Higgs doublet fields are not suppressed by the size of \(Y^\nu \). Furthermore, the model can accommodate neutrino data at tree level, so we will be able to describe the phenomenology related to both the scalar and the fermionic sector (and their interplay) more precisely.

If the mixing of the \({\mathcal {CP}}\)-even and -odd sneutrinos with the doublet fields is small, which is always the case for the left-handed sneutrinos, one can obtain approximate analytical expressions for the tree-level masses of the sneutrinos. For the left-handed sneutrinos the dominant terms are proportional to the inverse of their vevs. In particular, assuming that only diagonal elements of \(Y^\nu _{ij}\) and \(\kappa _{ijk}\) are non-zero, one finds for the diagonal entries of the mass matrix \(m_\varphi ^2\) corresponding to the \({\mathcal {CP}}\)-even left-handed sneutrinos,

$$\begin{aligned} m_{\widetilde{\nu }_{iL}^{\mathcal {R}} \widetilde{\nu }_{iL}^{\mathcal {R}}}^{2}\sim & {} \frac{Y^\nu _{ii}}{2 v_{iL}} \left[ v_d v_u^2 \lambda _i + \sqrt{2} v_d v_{iR} \mu \right. \nonumber \\&\quad \left. - v_u v_{iR} \left( \sqrt{2} A^\nu _{ii} + \kappa _{iii} v_{iR} \right) \right] , \end{aligned}$$
(2.23)

where we defined the effective \(\mu \)-term as

$$\begin{aligned} \mu = \frac{1}{\sqrt{2}} \left( v_{1R} \lambda _1 + v_{2R} \lambda _2 + v_{3R} \lambda _3 \right) . \end{aligned}$$
(2.24)

Note that the first term in Eq. (2.23) can usually be neglected as long as \(v_{iR}\gg v_d,v_u\). Each \({\mathcal {CP}}\)-odd left-handed sneutrino is nearly degenerate with the corresponding \({\mathcal {CP}}\)-even one, though they are slightly lighter due to different contributions proportional to the gauge couplings,

$$\begin{aligned} m_{\widetilde{\nu }_{iL}^{\mathcal {I}} \widetilde{\nu }_{iL}^{\mathcal {I}}}^{2} - m_{\widetilde{\nu }_{iL}^{\mathcal {R}} \widetilde{\nu }_{iL}^{\mathcal {R}}}^{2} = - \frac{1}{4} \left( g_1^2 + g_2^2 \right) v_{iL}^2. \end{aligned}$$
(2.25)

For the \({\mathcal {CP}}\)-even right-handed sneutrinos we find, under the assumptions that non-diagonal elements of \(\kappa _{ijk}\) and \((m_{\widetilde{\nu }_R}^2)_{ij}\) vanish, for the \(3\times 3\) submatrix

$$\begin{aligned} m_{\widetilde{\nu }_{iR}^{\mathcal {R}} \widetilde{\nu }_{jR}^{\mathcal {R}}}^{2}\sim & {} \frac{1}{2} v^2 \lambda _i \lambda _j + \delta _{ij} \left[ \frac{1}{\sqrt{2}} v_{iR} \kappa _{iii} A^\kappa _{iii} + 2 \kappa _{iii}^2 v_{iR}^2 \right. \nonumber \\&\quad \left. -\frac{\lambda _i}{\sqrt{2} v_{iR}} \left( \mu v^2 - v_d v_u A^\lambda _i \right) \right] . \end{aligned}$$
(2.26)

Furthermore, in case of universal values \(\kappa _{iii}=\kappa \), \(A^\kappa _{iii}=A^\kappa \), \(v_{iR}=v_R\), \(\lambda _i=\lambda \) and \(A^\lambda _i=A^\lambda \), this matrix has the form

$$\begin{aligned} m_{\widetilde{{\nu }}^{\mathcal {R}}_{iR} \widetilde{{\nu }}^{\mathcal {R}}_{jR}}^{2} = \begin{pmatrix} a &{} \quad b &{} \quad b \\ b &{} \quad a &{} \quad b \\ b &{} \quad b &{} \quad a \end{pmatrix}. \end{aligned}$$
(2.27)

The eigenvalues of such a matrix are \(a-b\) , \(a-b\) and \(a+2b\), and only the mass eigenstate corresponding to the latter eigenvalue mixes with the SM-like Higgs boson [109]. Later we will make use of this fact to simplify the accommodation of SM-like Higgs boson properties, even when all three right-handed neutrinos have masses close to or even below \(125\,\, \mathrm {GeV}\), because two right-handed sneutrinos will be conveniently decoupled from the remaining scalars, and interact very weakly with SM particles. In extensions of the NMSSM with several gauge singlets, this decoupling can lead to practically stable particles. In the \(\mu \nu \mathrm {SSM}\) this is not possible, because the decoupling cannot be exact, even when \(\kappa _{ijk}\) and \((m_{\widetilde{\nu }_R}^2)_{ij}\) are diagonal. We stress that the universality of the \(\kappa _{iii}\) is stable with respect to the RGE running, if also the \(\lambda _i\) parameters are universal. In this case, one can deduce from the explicit form of the one-loop counterterms \(\delta \kappa _{ijk}\) shown in the Appendix B.2.2 that differences in the running are exclusively generated by terms proportional to \((Y^\nu )^2\), which are negligible in realistic scenarios. For \(v_{iR} \gg v_d,v_u\) Eq. (2.26) further simplifies to

$$\begin{aligned} m_{\widetilde{\nu }_{iR}^{\mathcal {R}} \widetilde{\nu }_{iR}^{\mathcal {R}}}^{2} \sim \frac{1}{\sqrt{2}} v_{iR} \kappa _{iii} A^\kappa _{iii} + 2 \kappa _{iii}^2 v_{iR}^2, \end{aligned}$$
(2.28)

while for the \({\mathcal {CP}}\)-odd right-handed sneutrinos one finds

$$\begin{aligned} m_{\widetilde{\nu }_{iR}^{\mathcal {I}} \widetilde{\nu }_{iR}^{\mathcal {I}}}^{2} \sim - \frac{3}{\sqrt{2}} A^\kappa _{iii} \kappa _{iii} v_{iR}. \end{aligned}$$
(2.29)

Thus, to avoid tachyons both in the \({\mathcal {CP}}\)-even and -odd scalar spectrum, we will follow the sign convention \(A^\kappa _{iii} < 0\), \(\kappa _{iii} > 0\) and \(v_{iR} > 0\).

Before we come to the one-loop renormalization of the neutral scalar potential we briefly describe the other relevant sectors of the \(\mu \nu \mathrm {SSM}\).

2.2 Squark sector

The numerically most important one-loop corrections to the scalar potential are expected from the stop/top-sector, analogous to the (N)MSSM [105,106,107, 110,111,112], due to the huge Yukawa coupling of the (scalar) top. The tree-level mass matrices of the squarks differ slightly from the ones in the MSSM. Neglecting flavor mixing in the squark sector, one finds for the up-type squark mass matrix \(M^{\widetilde{u}}_i\) of generation i,

$$\begin{aligned} \left( M^{\widetilde{u}}_i\right) _{11}&= \left( m_{\widetilde{Q}}^2\right) _i + \frac{1}{24}(3g_2^{2}-g_1^2)\left( v_d^2+v_{jL}v_{jL}-v_u^2 \right) \nonumber \\&\quad +\frac{1}{2}v_u^2{Y^{u}_i}^2, \end{aligned}$$
(2.30)
$$\begin{aligned} \left( M^{\widetilde{u}}_i\right) _{12}&=\frac{1}{2}\left( \sqrt{2} v_u A^u_i + v_{jL} Y^\nu _{jk} v_{kR} - v_d v_{jR} \lambda _j\right) Y^u_i, \end{aligned}$$
(2.31)
$$\begin{aligned} \left( M^{\widetilde{u}}_i\right) _{22}&= \left( m_{\widetilde{u}}^2\right) _i+\frac{1}{6}g_1^2(v_d^2+v_{jL}v_{jL}-v_u^2)+\frac{1}{2}v_u^2{Y^u_i}^2. \end{aligned}$$
(2.32)

It should be noted that in the non-diagonal element explicitly appear the neutrino Yukawa couplings. This term arises in the F-term contributions of the squark potential through the quartic coupling of up-type quarks and one left-handed and the right-handed sneutrinos after EWSB. The mass eigenstates \(\widetilde{u}_{i1}\) and \(\widetilde{u}_{i2}\) are obtained by the unitary transformation

$$\begin{aligned} \begin{pmatrix} \widetilde{u}_{i1} \\ \widetilde{u}_{i2} \end{pmatrix} =U^{\widetilde{u}}_i \begin{pmatrix} \widetilde{u}_{iL} \\ \widetilde{u}_{iR} \end{pmatrix}\; , \qquad U^{\widetilde{u}}_i{U^{\widetilde{u}}_i}^\dagger =\mathbb {1}. \end{aligned}$$
(2.33)

Similarly, for the down-type squarks it is

$$\begin{aligned} \left( M^{\widetilde{d}}_i\right) _{11}&= \left( m_{\widetilde{Q}}^2\right) _i - \frac{1}{24}(3g_2^{2}+g_1^2)(v_d^2+v_{jL}v_{jL}-v_u^2) \nonumber \\&\quad +\frac{1}{2}v_d^2{Y^{d}_i}^2, \end{aligned}$$
(2.34)
$$\begin{aligned} \left( M^{\widetilde{d}}_i\right) _{12}&= \frac{1}{2}\left( \sqrt{2} v_d A^d_i - v_u \lambda _j v_{jR}\right) Y^d_i, \end{aligned}$$
(2.35)
$$\begin{aligned} \left( M^{\widetilde{d}}_i\right) _{22}&= \left( m_{\widetilde{d}}^2\right) _i - \frac{1}{12}g_1^2(v_d^2+v_{jL}v_{jL}-v_u^2) + \frac{1}{2}v_d^2{Y^d_i}^2. \end{aligned}$$
(2.36)

The mass eigenstates \(\widetilde{d}_{i1}\) and \(\widetilde{d}_{i2}\) are obtained by the unitary transformation

$$\begin{aligned} \begin{pmatrix} \widetilde{d}_{i1} \\ \widetilde{d}_{i2} \end{pmatrix} =U^{\widetilde{d}}_i \begin{pmatrix} \widetilde{d}_{iL} \\ \widetilde{d}_{iR} \end{pmatrix}\; , \qquad U^{\widetilde{d}}_i{U^{\widetilde{d}}_i}^\dagger =\mathbb {1}. \end{aligned}$$
(2.37)

2.3 Charged scalar sector

Since R-parity, lepton-number conservation and lepton-flavor universality are broken, the six charged left- and right-handed sleptons mix with each other and with the two charged scalars from the Higgs doublets. In the basis \(C^T= ({H^-_d}^*,{H^+_u},\widetilde{e}_{iL}^*,\widetilde{e}_{jR}^*)\) we find the following mass terms in the Lagrangian:

$$\begin{aligned} {{\mathcal {L}}}_{C} \; = \; -{C^*}^T {m}^2_{H^+} C. \end{aligned}$$
(2.38)

Assuming \(\mathcal {CP}\) conservation \({m}^2_{H^+}\) is a symmetric matrix of dimension 8,

$$\begin{aligned} m_{H^+}^2= \left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} m_{H_{d}^-{H^{-}_d}^{*}}^{2} &{} m_{H_{d}^- H_{u}^+}^{2} &{} m_{{H_d^-} \widetilde{e}^*_{jL}}^2 &{} m_{{H_d^-} \widetilde{e}^*_{jR}}^2 \\ m_{{H_{u}^+}^* {H_d^-}^*}^{2} &{} m_{{H^{+}_u}^{*} H_{u}^{+}}^{2} &{} m_{{H_u^+}^*\widetilde{e}^*_{jL}}^{2} &{} m_{{H_u^+}^*\widetilde{e}^*_{jR}}^{2} \\ m_{\widetilde{e}_{iL} {{H^{-}_d}}^{*}}^2 &{} m_{\widetilde{e}_{iL} H_u^+}^2 &{} m_{\widetilde{e}_{iL} \widetilde{e}_{jL}^{*}}^{2} &{} m_{\widetilde{e}_{iL} \widetilde{e}_{jR}^{*}}^{2}\\ m_{\widetilde{e}_{iR} {{H^{-}_d}}^{*}}^2 &{} m_{\widetilde{e}_{iR} H_u^+}^2 &{} m_{\widetilde{e}_{iR} \widetilde{e}_{jL}^{*}}^{2} &{} m_{\widetilde{e}_{iR} \widetilde{e}_{jR}^{*}}^{2} \end{array} \right) . \end{aligned}$$
(2.39)

The entries are given in Appendix A.3. The mass matrix is diagonalized by an orthogonal matrix \(U^+\):

$$\begin{aligned} U^+ m^2_{H^+}\ {U^+}^{^T} = \left( m^2_{H^+}\right) ^{\text {diag}}, \end{aligned}$$
(2.40)

where the diagonal elements of \(\left( m^2_{H^+}\right) ^{\text {diag}}\) are the squared masses of the mass eigenstates

$$\begin{aligned} H^+=U^+\; C, \end{aligned}$$
(2.41)

which include the charged Goldstone boson \(H^+_1=G^\pm \).

2.4 Charged fermion sector

The charged leptons mix with the charged gauginos and the charged higgsinos. Following the notation of Ref. [29], we write the relevant part of the Lagrangian in terms of two-component spinors \((\chi ^-)^T = \left( {({e_{iL})}^{c}}^{^*}, \widetilde{W}^-, \widetilde{H}^-_d\right) \) and \((\chi ^+)^T = \left( ({e_{jR}})^c, \widetilde{W}^+, \widetilde{H}^+_u\right) \):

$$\begin{aligned} {{\mathcal {L}}}_{\chi ^\pm } \; = \; -({\chi ^{-}})^T {m}_{e} \chi ^+ + \mathrm {h.c.}. \end{aligned}$$
(2.42)

The \(5\times 5\) mass matrix \(m_e\) is defined by

$$\begin{aligned} m_e= \begin{pmatrix} \frac{v_d Y^e_{11}}{\sqrt{2}} &{} \frac{v_d Y^e_{12}}{\sqrt{2}} &{} \frac{v_d Y^e_{13}}{\sqrt{2}} &{} \frac{g_2 v_{1L}}{\sqrt{2}} &{} -\frac{v_{iR} Y^\nu _{i1}}{\sqrt{2}} \\ \frac{v_d Y^e_{21}}{\sqrt{2}} &{} \frac{v_d Y^e_{22}}{\sqrt{2}} &{} \frac{v_d Y^e_{23}}{\sqrt{2}} &{} \frac{g_2 v_{2L}}{\sqrt{2}} &{} -\frac{v_{iR} Y^\nu _{i2}}{\sqrt{2}} \\ \frac{v_d Y^e_{31}}{\sqrt{2}} &{} \frac{v_d Y^e_{32}}{\sqrt{2}} &{} \frac{v_d Y^e_{33}}{\sqrt{2}} &{} \frac{g_2 v_{3L}}{\sqrt{2}} &{} -\frac{v_{iR} Y^\nu _{i3}}{\sqrt{2}} \\ 0 &{} 0 &{} 0 &{} M_2 &{} \frac{g_2 v_{u}}{\sqrt{2}} \\ -\frac{v_{iL}Y^e_{1i}}{\sqrt{2}} &{} -\frac{v_{iL}Y^e_{2i}}{\sqrt{2}} &{} -\frac{v_{iL}Y^e_{3i}}{\sqrt{2}} &{} \frac{g_2 v_{d}}{\sqrt{2}} &{} \mu \end{pmatrix}. \end{aligned}$$
(2.43)

The mass matrix is diagonalized by two unitary matrices \(U^e_L\) and \(U^e_R\),

$$\begin{aligned} {U_R^e}^{^*} m_{e} {U_L^e}^{^\dagger } = m_{e}^{\text {diag}}, \end{aligned}$$
(2.44)

where \(m_{e}^{\text {diag}}\) contains the masses of the charged fermions in the mass eigenstate base

$$\begin{aligned} \chi ^{+}= & {} {U_L^e}^{^\dagger } \lambda ^+, \end{aligned}$$
(2.45)
$$\begin{aligned} \chi ^{-}= & {} {U_R^e}^{^\dagger } \lambda ^-. \end{aligned}$$
(2.46)

Note that terms mixing the SM leptons and the MSSM-like charginos in Eq. (2.43) are suppressed by the size of the left-handed vevs \(v_{iL}\) or the neutrino Yukawa couplings \(Y^\nu _{ij}\). The smallness of \(v_{iL}\) in comparison to the other vevs and \(M_2\) assures the decoupling of the three leptons from the Higgsino and the wino, prohibiting substantial lepton-flavor-universality and lepton-number violation in the charged fermion sector.

2.5 Neutral fermion sector

The three left-handed neutrinos and the right-handed neutrinos mix with the neutral Higgsinos and gauginos. Again, following Ref. [29], we write the relevant part of the Lagrangian in terms of two-component spinors \(({\chi ^{0}})^T=\left( {(\nu _{iL})^{c}}^{^*},{\widetilde{B}}^0, {\widetilde{W}}^{0},{\widetilde{H}}_{d}^0,{\widetilde{H}}_{u}^0,\nu _{jR}^*\right) \) as

$$\begin{aligned} {{\mathcal {L}}}_{\chi ^0} \; = \;-\frac{1}{2} ({\chi ^{0}})^T {m}_{\nu } \chi ^0 + \mathrm {h.c.}, \end{aligned}$$
(2.47)

where \({m}_{\nu }\) is the \(10\times 10\) symmetric mass matrix. The neutral fermion mass matrix is determined by

$$\begin{aligned}&{m}_{\nu }= \left( \begin{array}{ccccccc} 0 &{} 0 &{} 0 &{} -\frac{g_1 v_{1L}}{2} &{} \frac{g_2 v_{1L}}{2} &{} 0 &{} \frac{v_{iR} Y^\nu _{1i}}{\sqrt{2}} \\ 0 &{} 0 &{} 0 &{} -\frac{g_1 v_{2L}}{2} &{} \frac{g_2 v_{2L}}{2} &{} 0 &{} \frac{v_{iR} Y^\nu _{2i}}{\sqrt{2}} \\ 0 &{} 0 &{} 0 &{} -\frac{g_1 v_{3L}}{2} &{} \frac{g_2 v_{3L}}{2} &{} 0 &{} \frac{v_{iR} Y^\nu _{3i}}{\sqrt{2}} \\ -\frac{g_1 v_{1L}}{2} &{} -\frac{g_1 v_{2L}}{2} &{} -\frac{g_1 v_{3L}}{2} &{} M_1 &{} 0 &{} -\frac{g_1 v_{d}}{2} &{} \frac{g_1 v_{u}}{2} \\ \frac{g_2 v_{1L}}{2} &{} \frac{g_2 v_{2L}}{2} &{} \frac{g_2 v_{3L}}{2} &{} 0 &{} M_2 &{} \frac{g_2 v_{d}}{2} &{} -\frac{g_2 v_{u}}{2} \\ 0 &{} 0 &{} 0 &{} -\frac{g_1 v_{d}}{2} &{} \frac{g_2 v_{d}}{2} &{} 0 &{} -\mu \\ \frac{v_{iR} Y^\nu _{1i}}{\sqrt{2}} &{} \frac{v_{iR} Y^\nu _{2i}}{\sqrt{2}} &{} \frac{v_{iR} Y^\nu _{3i}}{\sqrt{2}} &{} \frac{g_1 v_u}{2} &{} -\frac{g_2 v_u}{2} &{} -\mu &{} 0 \\ \frac{v_u Y^\nu _{11}}{\sqrt{2}} &{} \frac{v_u Y^\nu _{21}}{\sqrt{2}} &{} \frac{v_u Y^\nu _{31}}{\sqrt{2}} &{} 0 &{} 0 &{} -\frac{v_u \lambda _1}{\sqrt{2}} &{} \frac{- v_d \lambda _1 + v_{iL} Y^\nu _{i1}}{\sqrt{2}} \\ \frac{v_u Y^\nu _{12}}{\sqrt{2}} &{} \frac{v_u Y^\nu _{22}}{\sqrt{2}} &{} \frac{v_u Y^\nu _{32}}{\sqrt{2}} &{} 0 &{} 0 &{} -\frac{v_u \lambda _2}{\sqrt{2}} &{} \frac{- v_d \lambda _2 + v_{iL} Y^\nu _{i2}}{\sqrt{2}} \\ \frac{v_u Y^\nu _{13}}{\sqrt{2}} &{} \frac{v_u Y^\nu _{23}}{\sqrt{2}} &{} \frac{v_u Y^\nu _{33}}{\sqrt{2}} &{} 0 &{} 0 &{} -\frac{v_u \lambda _3}{\sqrt{2}} &{} \frac{- v_d \lambda _3 + v_{iL} Y^\nu _{i3}}{\sqrt{2}} \end{array} \right. \cdots \nonumber \\&\quad \cdots \left. \begin{array}{ccc} \frac{v_{u}Y^\nu _{11}}{\sqrt{2}} &{} \frac{v_{u}Y^\nu _{12}}{\sqrt{2}} &{} \frac{v_{u}Y^\nu _{13}}{\sqrt{2}} \\ \frac{v_{u}Y^\nu _{21}}{\sqrt{2}} &{} \frac{v_{u}Y^\nu _{22}}{\sqrt{2}} &{} \frac{v_{u}Y^\nu _{23}}{\sqrt{2}} \\ \frac{v_{u}Y^\nu _{31}}{\sqrt{2}} &{} \frac{v_{u}Y^\nu _{32}}{\sqrt{2}} &{} \frac{v_{u}Y^\nu _{33}}{\sqrt{2}} \\ 0 &{} 0 &{} 0 \\ 0 &{} 0 &{} 0 \\ -\frac{v_{u} \lambda _1}{\sqrt{2}} &{} -\frac{v_{u} \lambda _2}{\sqrt{2}} &{} -\frac{v_{u} \lambda _3}{\sqrt{2}} \\ \frac{-v_d \lambda _1 + v_{iL}Y^\nu _{i1}}{\sqrt{2}} &{} \frac{-v_d \lambda _2 + v_{iL}Y^\nu _{i2}}{\sqrt{2}} &{} \frac{-v_d \lambda _3 + v_{iL}Y^\nu _{i3}}{\sqrt{2}} \\ \sqrt{2} v_{iR} \kappa _{11i} &{} \sqrt{2} v_{iR} \kappa _{12i} &{} \sqrt{2} v_{iR} \kappa _{13i} \\ \sqrt{2} v_{iR} \kappa _{12i} &{} \sqrt{2} v_{iR} \kappa _{22i} &{} \sqrt{2} v_{iR} \kappa _{23i} \\ \sqrt{2} v_{iR} \kappa _{13i} &{} \sqrt{2} v_{iR} \kappa _{23i} &{} \sqrt{2} v_{iR} \kappa _{33i} \end{array} \right) \end{aligned}$$
(2.48)

Because of the Majorana nature of the neutral fermions we can diagonalize \({m}_{\nu }\) with the help of just a single - but complex - unitary matrix \(U^V\),

$$\begin{aligned} {U^V}^{^*} m_{\nu }\ {U^V}^{^\dagger } = m_{\nu }^{\text {diag}}, \end{aligned}$$
(2.49)

with

$$\begin{aligned} \chi ^{0} = {U^V}^{^\dagger } \lambda ^0, \end{aligned}$$
(2.50)

where \(\lambda ^0\) are the two-component spinors in the mass basis. The eigenvalues of the diagonalized mass matrix \(m_{\nu }^{\text {diag}}\) are the masses of the neutral fermions in the mass eigenstate basis.

The mass matrix has a seesaw structure, assuring that the three lightest eigenvalues will be very small, so that the mass eigenstates \(\lambda _{1,2,3}^0\) can practically be identified with the SM left-handed neutrinos. Components from the MSSM-like neutralinos and the right-handed neutrinos are negligible for the three lightest states. Thus, the left-handed neutrino mixing can in very good approximation (using diagonal \(Y^e\)) be expressed in the usual PMNS formalism [113, 115] by the three mixing angles \(\theta _{12}\), \(\theta _{13}\) and \(\theta _{23}\),

$$\begin{aligned} \left( \begin{array}{c} \lambda ^0_{1} \\ \lambda ^0_{2} \\ \lambda ^0_{3} \end{array}\right)&= \left( \begin{array}{ccc} c_{12}c_{13} &{} -s_{12}c_{23}-c_{12}s_{23}s_{13} &{} s_{12}s_{23}-c_{12}c_{23}s_{13} \\ s_{12}c_{13} &{} c_{12}c_{23}-s_{12}s_{23}s_{13} &{} -c_{12}s_{23}-s_{12}c_{23}s_{13} \\ s_{13} &{} s_{23}c_{13} &{} c_{23}c_{13} \end{array}\right) \nonumber \\ \left( \begin{array}{c} \chi ^0_{1} \\ \chi ^0_{2} \\ \chi ^0_{3} \end{array}\right)&= U^{V}_{ij} \left( \begin{array}{c} \chi ^0_{1} \\ \chi ^0_{2} \\ \chi ^0_{3} \end{array}\right) , \end{aligned}$$
(2.51)

with \(i,j=\{1,2,3\}\), and we used the short-hand notation \(c_x=\cos \theta _x\) and \(s_x=\sin \theta _x\). In our numerical studies we fitted the experimentally well measured quantities

$$\begin{aligned} s_{13}^2 = |U^V_{31}|^2 \; , \qquad s_{12}^2 = \frac{|U^V_{21}|^2}{1-s_{13}^2} \; , \qquad s_{23}^2 = \frac{|U^V_{32}|^2}{1-s_{13}^2} \; , \end{aligned}$$
(2.52)
$$\begin{aligned} \delta m_{12}^2 = m_{\lambda _2^0}^2 - m_{\lambda _1^0}^2 \; , \qquad \Delta m_{13}^2 \sim \Delta m_{23}^2 = m_{\lambda _3^0}^2 - m_{\lambda _{1,2}^0}^2 \end{aligned}$$
(2.53)

We restricted ourselves in the neutrino sector to a tree-level analysis, because the one-loop corrections turn out to be moderate in size (in the normal hierarchy pattern) [28] and can always be compensated by a small shift in the neutrino Yukawa couplings \(Y^\nu _{ij}\) without affecting the conclusions drawn in the scalar sector, in particular for the observables related to the SM-like Higgs boson.

To reduce the parameter space in our analysis, we usually assume the couplings \(Y^\nu \) to be diagonal, as we do for the lepton Yukawa couplings \(Y^e\). We emphasize that non-diagonal \(Y^\nu _{ij}\) are not required to reproduce the correct neutrino mixing, because sizable flavor mixing is always present after the diagonalization of \(m_\nu \), generated by the mixing terms of the left-handed neutrino states with the gauginos, Higgsinos and right-handed neutrinos. Quantitatively, this can be illustrated assuming universal parameters \(\lambda := \lambda _i\), \(v_{R} := v_{iR}\), \(\kappa := \kappa _{iii}\) and \(Y^\nu _{i} : = Y^\nu _{ii}\) (\(\kappa _{ijk}=0\) and \(Y^\nu _{ij}=0\) otherwise), by the formula [27]

$$\begin{aligned}&(m^{\text {eff}}_{\nu })_{ij} \simeq \frac{Y^{\nu }_iY^{\nu }_jv_u^2}{6\sqrt{2} \kappa v_{R}} (1-3 \delta _{ij})-\frac{v_{iL} v_{jL}}{4M^{\text {eff}}} \nonumber \\&\quad -\frac{1}{4M^{\text {eff}}}\left[ \frac{v_d\left( Y^{\nu }_iv_{jL} +Y^{\nu }_jv_{iL}\right) }{3\lambda } +\frac{Y^{\nu }_iY^{\nu }_jv_d^2}{9\lambda ^2 }\right] , \end{aligned}$$
(2.54)

with

$$\begin{aligned} M^{\text {eff}}\equiv & {} \frac{M_1 M_2}{g_1^2 M_2 + g_2^2 M_1} -\frac{v^2}{2\sqrt{2} \left( \kappa v_R^2+\lambda v_u v_d\right) \ 3 \lambda v_R}\nonumber \\&\times \left( 2 \kappa v_R^{2} \frac{v_u v_d}{v^2} +\frac{\lambda v^2}{2}\right) . \end{aligned}$$
(2.55)

Equation (2.54) demonstrates that substantial flavor mixing is practically unavoidable in the \(\mu \nu \mathrm {SSM}\). The first two terms are of particular importance. The first term can be attributed to the mixing with the right-handed neutrinos and higgsinos, and the other terms also include the gaugino mixing. Note that for moderate values of \(\tan \beta \) and not too small values of \(\lambda \) the first two terms are the dominant contributions. They contain diagonal and non-diagonal contributions that can easily be adjusted by an appropriate choice of the parameters \(Y^\nu _i\), \(v_{iL}\) and the soft gaugino mass parameters \(M_1\) and \(M_2\). These parameters play only a minor role in the predictions for the SM-like Higgs boson mass and its mixing with the right-handed sneutrinos. Thus, the above mentioned parameters will be used to reproduce neutrino physics in agreement with experimental limits, without having to worry about spoiling the properties of the SM-like Higgs boson.

Table 1 Set of independent parameters initially entering the tree-level Higgs potential of the \(\mu \nu \mathrm {SSM}\) in the first row, and final choice of free parameters after the substitutions defined in the text

3 Renormalization of the Higgs potential at one-loop

At tree level the part of the Higgs potential relevant for the masses of the scalars is given by the tadpole coefficients in Eqs. (2.13)–(2.16) and the \(\mathcal {CP}\)-even and \(\mathcal {CP}\)-odd scalar mass matrix elements in Eqs. (2.17) and (2.18). We want to employ a renormalization procedure as close as possible to the ones used in the (N)MSSM. Therefore, we define in the following subsection certain replacements to obtain a new set of free parameters. The new set of free parameter will permit us to make use of a mixed On-Shell (OS)/\(\smash {\overline{\mathrm {DR}}}\) renormalization scheme. The precise definition of the counterterms of the free parameters will be given in Sect. 3.2. Finally, we describe the renormalization conditions applied on each parameter and the extraction of the counterterms in Sect. 3.3.

3.1 Parameter replacements

The vevs of the doublet Higgs fields \(v_u\) and \(v_d\) are substituted by the MSSM-like parameters \(\tan \beta \) and v according to

$$\begin{aligned} \tan \beta =\frac{v_u}{v_d} \qquad \text {and} \qquad v^2=v_d^2+v_u^2+v_{iL}v_{iL}. \end{aligned}$$
(3.1)

Note that the definition of \(v^2\) differs from the one in the MSSM by the term \(v_{iL}v_{iL}\). This allows to maintain the relations between \(v^2\) and the gauge-boson masses as they are in the MSSM. Numerically, the difference in the definition of \(v^2\) is negligible. Maintaining the functional form of \(\tan \beta \) as it is in the (N)MSSM is convenient to facilitate the comparison of the quantum corrections in the \(\mu \nu \mathrm {SSM}\) and the (N)MSSM, as the one-loop counterterm of \(\tan \beta \) is expressed without having to include the counterterms for the left-handed sneutrino vevs [99]. The gauge couplings \(g_1\) and \(g_2\) will be replaced by the gauge-boson masses \(M_W\) and \(M_Z\),

$$\begin{aligned} M_W^2=\frac{1}{4}g_2^2v^2 \qquad \text {and} \qquad M_Z^2=\frac{1}{4}\left( g_1^2+g_2^2\right) v^2. \end{aligned}$$
(3.2)

This is reasonable, because the gauge-boson masses are well measured physical observables, so we can define them as OS parameters. The explicit dependence of the quantum corrections on the mass counterterm for \(M_W^2\) drops out at the one-loop level, but it will contribute implicitly in the definition of the counterterm for \(v^2\). The scalar soft masses \(m_{H_d}^2\) and \(m_{H_u}^2\) and the diagonal elements of the soft slepton mass matrices \(m_{\widetilde{L}}^2\) and \(m_{\widetilde{\nu }}^2\) are replaced by the tadpole coefficient in which they appear. Using the tadpole coefficients as input parameters facilitates the absorption of quantum corrections that would spoil the true vacuum of the potential. Alternatively, one could also trade the vevs for the tadpole coefficients, and keep the soft masses as input parameters. However, it is computationally much more convenient to use the vevs as input and solve the tadpole equations for the squared soft masses, because they appear linearly, while solving the tadpole equations for the vevs, using the soft masses as input, is a complex non-linear problem with multiple solutions. The complete set of independent parameters is summarized in Table 1.

3.2 Counterterms

The entries of the neutral scalar mass matrices are functions of the independent parameters,

$$\begin{aligned} m_{\varphi }^2&=m_{\varphi }^2\left( M_Z^2, v^2, \tan \beta , \lambda _i, \dots \right) , \end{aligned}$$
(3.3)
$$\begin{aligned} m_{\sigma }^2&=m_{\sigma }^2\left( M_Z^2, v^2, \tan \beta , \lambda _i, \dots \right) , \end{aligned}$$
(3.4)

and we define their renormalization as

$$\begin{aligned} m_{\varphi }^2&\rightarrow m_{\varphi }^2+\delta m_{\varphi }^2, \end{aligned}$$
(3.5)
$$\begin{aligned} m_{\sigma }^2&\rightarrow m_{\sigma }^2+\delta m_{\sigma }^2. \end{aligned}$$
(3.6)

The mass counterterms \(\delta m_{\varphi }^2\) and \(\delta m_{\sigma }^2\) enter the renormalized one-loop scalar self-energies. They are given as a linear combination of the counterterms of the independent parameters, which we define as

$$\begin{aligned} \begin{aligned} T_{H_d^{\mathcal {R}}}&\rightarrow T_{H_d^{\mathcal {R}}}+\delta T_{H_d^{\mathcal {R}}} \; , \\ T_{H_u^{\mathcal {R}}}&\rightarrow T_{H_u^{\mathcal {R}}}+\delta T_{H_u^{\mathcal {R}}} \; , \\ T_{\widetilde{\nu }_{iR}^{\mathcal {R}}}&\rightarrow T_{\widetilde{\nu }_{iR}^{\mathcal {R}}}+\delta T_{\widetilde{\nu }_{iR}^{\mathcal {R}}}, \\ T_{\widetilde{\nu }_{iL}^{\mathcal {R}}}&\rightarrow T_{\widetilde{\nu }_{iL}^{\mathcal {R}}}+ \delta T_{\widetilde{\nu }_{iL}^{\mathcal {R}}}, \\ {m_{\widetilde{\nu }}^2}_{i\ne j}&\rightarrow {m_{\widetilde{\nu }}^2}_{i\ne j}+ \delta {m_{\widetilde{\nu }}^2}_{i\ne j}, \\ {m_{\widetilde{L}}^2}_{i\ne j}&\rightarrow {m_{\widetilde{L}}^2}_{i\ne j}+ \delta {m_{\widetilde{L}}^2}_{i\ne j}, \\ {m_{H_d\widetilde{L}}^2}_i&\rightarrow {m_{H_d\widetilde{L}}^2}_i+\delta {m_{H_d\widetilde{L}}^2}_i, \end{aligned} \qquad \begin{aligned} \tan \beta&\rightarrow \tan \beta +\delta \tan \beta , \\ v^2&\rightarrow v^2+\delta v^2, \\ v_{iR}^2&\rightarrow v_{iR}^2 + \delta v_{iR}^2, \\ v_{iL}^2&\rightarrow v_{iL}^2 + \delta v_{iL}^2, \\ M_W^2&\rightarrow M_W^2 +\delta M_W^2, \\ M_Z^2&\rightarrow M_Z^2 +\delta M_Z^2, \\ \end{aligned} \qquad \begin{aligned} \lambda _i&\rightarrow \lambda _i + \delta \lambda _i \; , \\ \kappa _{ijk}&\rightarrow \kappa _{ijk} + \delta \kappa _{ijk} \; , \\ Y^\nu _{ij}&\rightarrow Y^\nu _{ij} + \delta Y^\nu _{ij} \; , \\ T^\lambda _i&\rightarrow T^\lambda _i + \delta T^\lambda _i \; , \\ T^\kappa _{ijk}&\rightarrow T^\kappa _{ijk} + \delta T^\kappa _{ijk} \; , \\ T^\nu _{ij}&\rightarrow T^\nu _{ij} + \delta T^\nu _{ij} \; . \end{aligned} \end{aligned}$$
(3.7)

The divergent parts of the counterterms are fixed to cancel the UV divergences. The finite pieces, and thus the meaning of the parameters, have to be fixed by renormalization conditions. We will adopt a mixed renormalization scheme, where tadpoles and gauge boson masses are fixed OS, and the other parameters are fixed in the \(\smash {\overline{\mathrm {DR}}}\) scheme. The exact renormalization conditions will be given in Sect. 3.3. The dependence of the mass counterterms \(\delta m_{\varphi }^2\) and \(\delta m_{\sigma }^2\) on the counterterms of the free parameters is given at the one-loop level by the first order expansion w.r.t. the free parameters,

$$\begin{aligned} \delta m_{\varphi }^2= & {} \sum _{X\in \; \text {free param.}} \left( \frac{\partial }{\partial X} m_{\varphi }^2 \right) \delta X, \nonumber \\ \delta m_{\sigma }^2= & {} \sum _{X\in \; \text {free param.}} \left( \frac{\partial }{\partial X} m_{\sigma }^2 \right) \delta X. \end{aligned}$$
(3.8)

We define the mixing matrices \(U^H\) and \(U^A\) to diagonalize the renormalized mass matrices, so they do not have to be renormalized. The expressions for the counterterms of the scalar mass matrices in the mass eigenstate basis are then given by

$$\begin{aligned} \delta m_{h}^2=U^H \delta m_{\varphi }^2 {U^H}^T, \qquad \delta m_{A}^2=U^A \delta m_{\sigma }^2{U^A}^T, \end{aligned}$$
(3.9)

where we emphasize that \(\delta m_{h}^2\) and \(\delta m_{A}^2\) are not diagonal, as they would be in a purely OS renormalization procedure which is often used in theories with flavor mixing [116].

The field renormalization required to obtain finite scalar self-energies at arbitrary momentum, is defined by

$$\begin{aligned} \begin{pmatrix} H_d \\ H_u \\ \widetilde{\nu }_{iR} \\ \widetilde{\nu }_{iL} \end{pmatrix} \rightarrow \sqrt{Z} \begin{pmatrix} H_d \\ H_u \\ \widetilde{\nu }_{iR} \\ \widetilde{\nu }_{iL} \end{pmatrix}= \left( \mathbb {1} + \frac{1}{2} \delta Z \right) \begin{pmatrix} H_d \\ H_u \\ \widetilde{\nu }_{iR} \\ \widetilde{\nu }_{iL} \end{pmatrix}, \end{aligned}$$
(3.10)

where \(\sqrt{Z}\) and \(\delta Z\) are \(8 \times 8\) dimensional matrices and the equal sign is valid at the one-loop level. In contrast to the (N)MSSM, the field renormalization is not diagonal in the interaction basis. The reason is that the \(\mu \nu \mathrm {SSM}\) explicitly breaks lepton-number conservation and lepton-flavor universality, resulting in kinetic mixing terms at one-loop order.Footnote 2 For the \({\mathcal {CP}}\)-even and \({\mathcal {CP}}\)-odd neutral scalar fields the definition in Eq. (3.10) implies the following field renormalization in the mass eigenstate basis:

$$\begin{aligned} h \rightarrow \left( \mathbb {1} + \frac{1}{2} \delta Z^H \right) h \; , \qquad A \rightarrow \left( \mathbb {1} + \frac{1}{2} \delta Z^A \right) A, \end{aligned}$$
(3.11)

with

$$\begin{aligned} \delta Z^H = U^H \left( \delta Z \right) {U^H}^T \; \quad \text {and} \quad \delta Z^A = U^A \left( \delta Z \right) {U^A}^T.\nonumber \\ \end{aligned}$$
(3.12)

3.3 Renormalization conditions

In this section we briefly describe our choice for the renormalization conditions. We start with the OS conditions for the gauge boson mass parameters and the tadpole coefficients followed by our definitions for the \(\smash {\overline{\mathrm {DR}}}\) renormalized parameters, including the field renormalization. All counterterms are extracted diagrammatically by calculating one-loop corrections to linear, bilinear or trilinear terms of the Lagrangian, and identifying the part of the corrections that had to be absorbed individually by the counterterms of the parameters appearing in the tree-level expression of the term. We generated the Feynman diagrams using our FeynArts [118] model file, which was initially created with SARAH version 4.12.0 [95, 119]. We modified the model file by hand to be able to use FormCalc  [120] for further evaluations and to improve the analytical and numerical evaluation of the rather large expressions. Since the divergent parts of one-loop counterterms can in principle also be derived from the one-loop beta functions, for which generic analytical formulas exist [121,122,123,124,125,126], the diagrammatic calculation of the counterterms was an excellent test for the correctness of our FeynArts model file.

The determination of the counterterms for the set of independent parameters was done in a specific order, because in some cases the definition of the renormalization condition of one counterterm depends on other counterterms, that necessarily had to be determined before. In Fig. 1 we give an overview of the strategy for the extraction of the counterterms. We also highlight in color the sectors of the \(\mu \nu \mathrm {SSM}\) in which the corresponding counterterm was extracted (see caption). The exact definition of the counterterms and their final analytic expressions in terms of UV divergences for the \(\smash {\overline{\mathrm {DR}}}\) counterterms are listed in Appendix B.

Therein, divergent parts are expressed proporional to \(\Delta \),

$$\begin{aligned} \Delta = \frac{1}{\varepsilon }-\gamma _E+\ln {4\pi }, \end{aligned}$$
(3.13)

where loop integral are solved in \(4-2\varepsilon \) dimensions and \(\gamma _E=0.5772\dots \) is the Euler-Mascharoni constant. Since the field renormalization constants contribute only via divergent parts, they do not contribute to the finite result after canceling divergences in the self-energies. As regularization scheme we choose dimensional reduction [127, 128] which was shown to be SUSY conserving at the one-loop level [129]. In contrast to the OS renormalization scheme our field renormalization matrices are hermitian. This holds also true for the field renormalization in the mass eigenstate basis, because as already mentioned the rotations in Eqs. (2.19) and (2.21) diagonalize the renormalized tree-level scalar mass matrices, so Eq. (3.12) do not introduce non-hermitian parts into the field renormalization that would have to be canceled by a renormalization of the mixing matrices \(U^H\) and \(U^A\) themselves.

Fig. 1
figure 1

Strategy for extracting the counterterms needed for renormalizing the neutral scalar potential. The arrows indicate the order in which the counterterms were obtained, while the colors stand for the sector that was used to extract the counterterms. Red: Renormalization of electromagnetic coupling. Violet: Renormalization of \({\mathcal {CP}}\)-odd self-energies. Yellow: Renormalization of gauge boson self-energies. Blue: Renormalization of neutral fermion self-energies. Green: Renormalization of \(\mathcal {CP}\)-even scalar trilinear couplings. White: Completely fixed by the dependence on other counterterms. The counterterms in the dashed boxes do not belong to the set of independent parameters, but their counterterms were calculated as an intermediate step. The counterterms below one of the horizontal dashed lines could be extracted only after the counterterms above the same horizontal line were determined

3.3.1 OS conditions

The SM gauge boson masses are renormalized OS requiring

$$\begin{aligned} \text {Re}\left[ {\hat{\Sigma }_{ZZ}}^T\left( M_Z^2 \right) \right] =0 \qquad \text {and} \qquad \text {Re}\left[ {\hat{\Sigma }_{WW}}^T\left( M_W^2 \right) \right] =0,\nonumber \\ \end{aligned}$$
(3.14)

where \(\hat{\Sigma }^T\) stands for the transverse part of the renormalized gauge boson self-energy. For their mass counterterms these conditions yield

$$\begin{aligned}&\delta M_Z^2 = \text {Re}\left[ {\Sigma }_{ZZ}^T\left( M_Z^2 \right) \right] \qquad \text {and} \nonumber \\&\quad \delta M_W^2 = \text {Re}\left[ {\Sigma }_{WW}^T\left( M_W^2 \right) \right] . \end{aligned}$$
(3.15)

Here the \({\Sigma }^T\) (without the hat) denote the transverse part of the unrenormalized gauge boson self-energies.

For the tadpole coefficients \(T_{\varphi _i}\) the OS conditions read

$$\begin{aligned} T_{\varphi _i}^{(1)}+\delta T_{\varphi _i}=0, \end{aligned}$$
(3.16)

where \(T_{\varphi _i}^{(1)}\) are the one-loop contributions to the linear terms of the scalar potential, stemming from tadpole diagrams shown in Fig. 2. The tadpole diagrams are calculated in the mass eigenstate basis h. The one-loop tadpole contributions in the interaction basis \(\varphi \) are then obtained by the rotation

$$\begin{aligned} T_\varphi ^{(1)} = {U^H}^T T_h^{(1)}. \end{aligned}$$
(3.17)

Accordingly we find for the one-loop tadpole counterterms

$$\begin{aligned} \delta T_{\varphi _i}=-T_{\varphi _i}^{(1)}. \end{aligned}$$
(3.18)
Fig. 2
figure 2

Generic Feynman diagrams for the tadpoles \(T_{h_i}\)

3.3.2 \(\smash {\overline{\mathrm {DR}}}\) conditions

For practical purposes we decided to renormalize all remaining parameters in the \(\smash {\overline{\mathrm {DR}}}\) scheme, reflecting the fact that there are no physical observables yet that could be directly related to them. The counterterms of each parameter were obtained by calculating the divergent parts of one-loop corrections to different scalar and fermionic two- and three-point functions. We sketch the determination of the counterterms in the (possible) order in which they can be successively derived (see Fig. 1).

The general strategy for extracting the counterterms of the free parameters is the following. At first one finds a relatively simple tree-level expression, containing the parameter whose counterterm one wants to extract, and, apart from that, exclusively parameters whose countertems are already known. In our case we used bilinear and trilinear couplings in the neutral scalar and fermionic sector for this. Then one calculates the one-loop corrections to the term by evaluating the corresponding Feynman diagrams. As we only need the divergent parts of the loop corrections for \(\smash {\overline{\mathrm {DR}}}\) conditions, we are able to calculate the diagrams in the gauge eigenstate basis, where the number of diagrams is drastically reduced. Once the divergent contributions are known, the counterterm to be identified directly follows from the expression of the renormalized Green’s functions.

In this chapter we only state the general formulas for the renormalized two- and three-point Green’s functions. The exact conditions used to extract each countertem is listed in Appendix B.2. Therein, we also show the resulting analytic expressions for the counterterms renormalized in the \(\smash {\overline{\mathrm {DR}}}\) scheme.Footnote 3 A more detailed discussion in the case of the \(\mu \nu \mathrm {SSM}\) with one generation of right-handed neutrinos can be found in Ref. [99].

Neutral fermion sector

We derived most of the counterterms in the neutral fermion sector. The mass matrix elements of the neutral fermions \(\left( m_\nu \right) _{ij}\) get one-loop corrections via the neutral fermion self-energies \({\sum }_{\chi _{i}^0 \chi _{j}^0}\), that for Majorana fermions can be decomposed as

(3.19)

Defining for the renormalized mass matrix

$$\begin{aligned} \left( m_\nu \right) _{ij} \rightarrow \left( m_\nu \right) _{ij} + \delta \left( m_\nu \right) _{ij}, \end{aligned}$$
(3.20)

the renormalized scalar part of the self-energies at zero momentum is given by

$$\begin{aligned}&\hat{\Sigma }_{\chi _{i}^0 \chi _{j}^0}^S\left( 0\right) =\Sigma _{\chi _{i}^0 \chi _{j}^0}^S\left( 0\right) \nonumber \\&\quad -\frac{1}{2} \left( \delta Z_{ki}^\chi \left( m_\nu \right) _{kj} + \left( m_\nu \right) _{ik} \delta Z_{kj}^\chi \right) -\delta \left( m_\nu \right) _{ij}. \end{aligned}$$
(3.21)

The field renormalization constants can be obtained in the \(\smash {\overline{\mathrm {DR}}}\) scheme by calculating the divergent part of the fermionic piece,

$$\begin{aligned} \delta Z_{ij}^\chi =-\left. \Sigma _{\chi _{i}^0\chi _{j}^0}^F\right| ^{\mathrm{div}}. \end{aligned}$$
(3.22)

The divergent parts of the self-energies of the neutral fermions were calculated diagrammatically in the interaction basis, where diagrams with mass insertions have to be included. If \(\left( m_\nu \right) _{ij}\) containes just a single parameter whose counterterm is unknow, Eq. (3.21) provides a definition for the missing counterterm once the mass counterterm \(\delta \left( m_\nu \right) _{ij}\) is expressed in terms of the counterterms of the fundamental parameters. Unfortunately, the right-handed vevs \(v_{iR}\) always appear in sums over the family index and never isolated. Therefore we calculated loop corrections to the three elements \(\left( m_\nu \right) _{i,7}= - Y^\nu _{ij} v_{jR}/\sqrt{2}\), which provides us with a linear system of three independent equations;

$$\begin{aligned} Y^\nu _{ij} \delta v_{jR}&= \sqrt{2} \left. \Sigma _{\nu _{iL} \widetilde{H}^0_u}^S \right| ^{\mathrm{div}} - \frac{1}{2} \left( \delta Z_{ij}^\chi v_{kR} Y^\nu _{jk} - \delta Z_{i6}^\chi v_{jR} \lambda _j \right. \nonumber \\&\quad \left. + \delta Z_{77}^\chi v_{jR} Y^\nu _{ij} \right) - v_{jR} \delta Y^\nu _{ij}, \end{aligned}$$
(3.23)

that can be solved analytically for the three counterterms \(\delta v_{iR}^2 = 2 v_{iR} \; \delta v_{iR}\).

Neutral scalar trilinear couplings

General one-loop scalar three-point functions can be renormalized by wavefunction counterterms and the specific vertex counterterm as

$$\begin{aligned} \hat{\Gamma }_{\varphi _i \varphi _j \varphi _k}^{(1)}&= \Gamma _{\varphi _i \varphi _j \varphi _k}^{(0)} + \Gamma _{\varphi _i \varphi _j \varphi _k}^{(1)} - \frac{1}{2} \biggl ( \Gamma _{\varphi _l \varphi _j \varphi _k}^{(0)} \delta Z_{li} \nonumber \\&\quad + \Gamma _{\varphi _i \varphi _l \varphi _k}^{(0)} \delta Z_{lj} + \Gamma _{\varphi _i \varphi _j \varphi _l}^{(0)} \delta Z_{lk} \biggr ) - \delta \Gamma _{\varphi _i \varphi _j \varphi _k}^{(1)}, \end{aligned}$$
(3.24)

where \(\delta Z_{ij}\) are the scalar field renormailzation constants defined in Eq. (3.10), \(\Gamma _{\varphi _i \varphi _j \varphi _k}^{(0)}\) are the tree-level couplings, \(\Gamma _{\varphi _i \varphi _j \varphi _k}^{(1)}\) are the one-loop corrections obtained by evaluating the non-irreducible one-loop three-point diagrams, and \(\delta \Gamma _{\varphi _i \varphi _j \varphi _k}^{(1)}\) is the coupling counterterm given as a function of the counterterms of the independent parameters. The field renormalization constants are defined as \(\smash {\overline{\mathrm {DR}}}\)-parameters. We calculate the UV-divergent part of the derivative of the scalar \({\mathcal {CP}}\)-even self-energies in the interaction basis and define

$$\begin{aligned} \delta Z_{ij}=-\left. \frac{d}{dp^2}\Sigma _{\varphi _i \varphi _j} \right| ^{\mathrm{div}}. \end{aligned}$$
(3.25)

As before, if at tree level \(\Gamma _{\varphi _i \varphi _j \varphi _k}^{(0)}\) just contains a single parameter whose counterterm is still unknown, the counterterm can be extracted from the divergent part of the one-loop corrections \(\Gamma _{\varphi _i \varphi _j \varphi _k}^{(1)}\) demanding that the renormalized quantity is finite. Similarly to the vevs \(v_{iR}\), for the parameters \(\kappa _{iij}\) it is not possible to find a tree-level expression where each element appears isolated. However, using the renormalized expression in Eq. (3.24) for the vertex

$$\begin{aligned} \Gamma _{\widetilde{\nu }_{iR}^{\mathcal {R}} \widetilde{\nu }_{kR}^{\mathcal {R}} \widetilde{\nu }_{jL}^{\mathcal {R}}}^{(0)}= & {} \frac{1}{2} v_d \lambda _k Y^\nu _{ji} + \frac{1}{2} v_d \lambda _i Y^\nu _{jk} \nonumber \\&- \frac{1}{2} v_{lL} Y^\nu _{li} Y^\nu _{jk} - \frac{1}{2} v_{lL} Y^\nu _{lk} Y^\nu _{ji} - v_u \kappa _{ikl} Y^\nu _{jl},\nonumber \\ \end{aligned}$$
(3.26)

we can extract the counterterms for the three subsets \((\delta \kappa _{11j},\delta \kappa _{22j},\delta \kappa _{33j})\) by renormalizing the subset of vertices (\(\Gamma _{\widetilde{\nu }_{1R}^{\mathcal {R}} \widetilde{\nu }_{1R}^{\mathcal {R}} \widetilde{\nu }_{jL}^{\mathcal {R}}}\), \(\Gamma _{\widetilde{\nu }_{2R}^{\mathcal {R}} \widetilde{\nu }_{2R}^{\mathcal {R}} \widetilde{\nu }_{jL}^{\mathcal {R}}}\), \(\Gamma _{\widetilde{\nu }_{3R}^{\mathcal {R}} \widetilde{\nu }_{3R}^{\mathcal {R}} \widetilde{\nu }_{jL}^{\mathcal {R}}}\)). Thus, for each subset (\(\kappa _{iij}\, ; i=1,2,3\)) we get a linear system of three (\(j=1,2,3\)) equations to extract the counterterms \(\delta \kappa _{iij}\) from the condition that the renormalized one-loop three-point function is finite.

Fig. 3
figure 3

Generic diagrams for the \({\mathcal {CP}}\)-even (h) and \({\mathcal {CP}}\)-odd (A) scalar self-energies in the mass eigenstate basis

Neutral scalar masses

The soft scalar masses appear in the bilinear terms of the Higgs potential. They can be renormalized by calculating radiative corrections to scalar self-energies. Since our final aim is to obtain loop corrections for the \(\mathcal {CP}\)-even scalars, we used the \(\mathcal {CP}\)-odd scalar sector to extract the counterterms of the soft masses to have an independent crosscheck of both neutral scalar sectors.

The general form of the renormalized scalar self-energies at the one-loop level is

$$\begin{aligned}&\hat{\Sigma }_{X_i X_j}\left( p^2 \right) = \Sigma _{X_i X_j}\left( p^2 \right) + \frac{1}{2} p^2 \left( \delta Z_{ji} + \delta Z_{ij} \right) \nonumber \\&\quad -\frac{1}{2} \left( \delta Z_{ki} \left( m_{X}^2 \right) _{kj} + \left( m_{X}^2 \right) _{ik} \delta Z_{kj} \right) - \delta \left( m_{X}^2 \right) _{ij}, \end{aligned}$$
(3.27)

where \(X=(\varphi ,\sigma )\) represents either the \({\mathcal {CP}}\)-even or the \({\mathcal {CP}}\)-odd scalar fields and we made use of the fact that the field renormalization constants \(\delta Z\) and the mass matrix \(m_{X}^2\) are real. Demanding that the renormalized self-energies \(\hat{\Sigma }_{A_i A_j}\) are finite in the mass eigenstate basis we can define the divergent parts of the mass counterterms via

$$\begin{aligned}&\left. \delta \left( m_{A}^2 \right) _{ij}\right| ^{\mathrm{div}} = \left. \Sigma _{A_i A_j}\left( 0 \right) \right| ^{\mathrm{div}} \nonumber \\&\quad -\frac{1}{2}\left( \left( \delta Z^A\right) _{ji} m_{A_j}^2 + m_{A_i}^2 \left( \delta Z^A\right) _{ij} \right) \; , \end{aligned}$$
(3.28)

where the field counterterms in the mass eigenstate basis were defined in Eq. (3.12), and renormalized as \(\smash {\overline{\mathrm {DR}}}\) parameters like the ones for the \({\mathcal {CP}}\)-even scalars (see Eq. (3.25)), and the masses squared \(m_{A_i}^2\) are the eigenvalues of the diagonal \({\mathcal {CP}}\)-odd scalar mass matrix \(m_{A}^2\). In Fig. 3 we show the diagrams that have to be calculated to obtain the quantum corrections to scalar self-energies at the one-loop level in the mass eigenstate basis.

We calculated all diagrams in the ’t Hooft-Feynman gauge in which the Goldstone bosons \(A_1\) and \(H^\pm _1\) and the ghost fields \(u^\pm \) and \(u^Z\) have the same masses as the corresponding gauge bosons. Calculating the \({\mathcal {CP}}\)-odd self-energies \(\Sigma _{A_i A_i}\) diagrammatically, we obtain the mass counterterms in mass eigenstate basis through the Eq. (3.28). Now inverting the rotation in Eq. (3.9) yields the mass counterterms for the \({\mathcal {CP}}\)-odd self-energies in the interaction basis,

$$\begin{aligned} \left. \delta m_{\sigma }^2\right| ^{\mathrm{div}}= {U^A}^T \left. \delta m_{A}^2\right| ^{\mathrm{div}} U^A. \end{aligned}$$
(3.29)

Analytically, following the expansion in Eq. (3.8), some of the mass counterterms \(\delta m_\sigma ^2\) depend on the counterterms of the soft mass parameters. We use this dependences to extract the counterterms of \((m_{H_d \widetilde{L}}^2)_i\) and the counterterms of the non-diagonal elements of \((m_{\widetilde{L}}^2)_{ij}\) and \((m_{\widetilde{\nu }}^2)_{ij}\).

4 Loop corrected scalar masses

In the previous section we have defined an OS/\(\smash {\overline{\mathrm {DR}}}\) renormalization scheme for the \(\mu \nu \mathrm {SSM}\) neutral scalar sector. This can be applied (via the FeynArts model file, in which the counterterms are implemented) to any higher-order correction in the \(\mu \nu \mathrm {SSM}\). As a first application, we evaluate the full one-loop corrections to the \({\mathcal {CP}}\)-even scalar sector in the \(\mu \nu \mathrm {SSM}\). In the following we emphasize the differences w.r.t the analysis with just one right-handed neutrino from Ref. [99].

4.1 Evaluation at the one-loop level

The one-loop renormalized self-energies in the mass eigenstate basis are given by

$$\begin{aligned}&\hat{\Sigma }_{h_i h_j}^{(1)}\left( p^2 \right) = \Sigma _{h_i h_j}^{(1)}\left( p^2 \right) \nonumber \\&\quad + \delta Z^H_{ij} \left( p^2 - \frac{1}{2}\left( m_{h_i}^2 + m_{h_j}^2 \right) \right) - \left( \delta m_{h}^2\right) _{ij}, \end{aligned}$$
(4.1)

with the field renormalization constants \(\delta Z^H\) and the mass counter terms \(\delta m_{h}^2\) in the mass eigenstate basis defined by the rotations in Eqs. (3.12) and (3.9), respectively. \(\Sigma _{h_i h_j}\) is the unrenormalized self-energy obtained by calculating the diagrams shown in Fig. 3 with the \({\mathcal {CP}}\)-even states h on the external legs. The self-energies were calculated in the ’t Hooft-Feynman gauge, so that gauge-fixing terms do not yield counterterm contributions in the Higgs sector at the one-loop level. The loop integrals were regularized using dimensional reduction [127, 128] and numerically evaluated for arbitrary real momentum using LoopTools[120]. The contributions from complex values of \(p^2\) were approximated using a Taylor expansion with respect to the imaginary part of \(p^2\) up to first order.

In Eq. (4.1) we already made use of the fact that \(\delta Z^H\) is real and symmetric in our renormalization scheme. The mass counterterms are defined as functions of the counterterms of the free parameters following Eqs. (3.8) and (3.9). They contain finite contributions from the tadpole counterterms and from the counterterm for the gauge boson mass \(M_Z^2\). The matrix \(\delta m_{h}^2\) is real and symmetric.

The renormalized self-energies enter the inverse propagator matrix

$$\begin{aligned}&\hat{\Gamma }_{h}=\text {i}\left[ p^2\; \mathbb {1} - \left( m_{h}^2 - \hat{\Sigma }_{h}\left( p^2\right) \right) \right] \; , \nonumber \\&\quad \text {with } \left( \hat{\Sigma }_{h}\right) _{ij} = \hat{\Sigma }_{h_i h_j}. \end{aligned}$$
(4.2)

The loop-corrected scalar masses squared are the zeroes of the determinant of the inverse propagator matrix. The determination of corrected masses has to be done numerically when one wants to account for the momentum-dependence of the renormalized self-energies. This is done by an iterative method that has to be carried out for each of the six squared loop-corrected masses.Footnote 4

4.2 Inclusion of higher orders

In Eq. (4.2) we did not include the superscript \(^{(1)}\) in the self-energies. Restricting the numerical evaluation to a pure one-loop calculation would lead to very large theoretical uncertainties. These can be avoided by the inclusion of corrections beyond the one-loop level. Here we follow the approach of Refs. [87, 99] and supplement the \(\mu \nu \mathrm {SSM}\) one-loop results by higher-order corrections in the MSSM limit as provided by FeynHiggs (version 2.13.0) [40, 48, 60,61,62,63,64, 67].Footnote 5 In this way the leading and subleading two-loop corrections are included, as well as a resummation of large logarithmic terms, see the discussion in Sect. 1,

$$\begin{aligned} \hat{\Sigma }_{h}\left( p^2 \right) = \hat{\Sigma }_{h}^{(1)}\left( p^2 \right) + \hat{\Sigma }_{h}^{(2')} + \hat{\Sigma }_{h}^{\mathrm{resum}}. \end{aligned}$$
(4.3)

In the partial two-loop contributions \(\hat{\Sigma }_{h}^{(2')}\) we take over the corrections of \({{\mathcal {O}}}(\alpha _s\alpha _t,\alpha _s\alpha _b,\alpha _t^2,\alpha _t\alpha _b)\), assuming that the MSSM-like corrections approximate numerically well the corresponding \(\mu \nu \mathrm {SSM}\) corrections. This assumption is reasonable since the only difference between the squark sector of the \(\mu \nu \mathrm {SSM}\) in comparison to the MSSM are numerically suppressed terms proportional to \(v_{jL} Y^\nu _{jk} v_{kR}\) in the non-diagonal element of the up-type squark mass matrices (see Eq. (2.31)) and proportional to \(v_{iL}v_{iL}\) in the diagonal elements of the up- and down-type squark mass matrices (see Eqs. (2.30, 2.32, 2.34) and (2.36)). Furthermore, in Refs. [87, 108] the quality of the MSSM approximation was tested in the NMSSM, showing that the genuine NMSSM contributions are in most cases sub-leading. Our results in Ref. [99] confirm that the same holds true in the \(\mu \nu \mathrm {SSM}\) for the SM-like Higgs-boson mass. The same is expected for the contributions stemming from the resummation of large logarithmic terms given by \(\hat{\Sigma }_{h}^{\mathrm{resum}}\).

5 Numerical analysis

In the following we will present several benchmark points (BPs) that illustrate the phenomenology of the scalar sector of the \(\mu \nu \mathrm {SSM}\). We concentrate on scenarios in which a right-handed sneutrino is mixed with the SM-like Higgs boson. By setting \(\kappa _{ijk}=\kappa \;\delta _{ij}\delta _{jk}\), as explained in Sect. 2.1, we achieve that only a single right-handed sneutrino substantially mixes with the SM-like Higgs boson. Naturally, the mass scale of the right-handed sneutrinos will then be of the order of the SM-like Higgs boson. However, scenarios in which the decay of the SM-like Higgs boson to two right-handed sneutrinos is kinematically allowed, are experimentally very constrained [109].

In contrast to most of the previous studies of the \(\mu \nu \mathrm {SSM}\) with three generations of right-handed neutrinos [24, 25, 27, 28, 109], we will not always make the simplifying assumption that genuine low-energy \(\mu \nu \mathrm {SSM}\)-parameters have universal values independent of the family index. In Sect. 5.3 we elaborate on the effect of non-universal \(\lambda _i\) on the SM-like Higgs-boson mass, while keeping \(\pmb {\lambda }^2=\lambda _i\lambda _i\) constant. Since we know from Eq. (2.22) that the tree-level mass of the SM-like Higgs boson strongly depends on \(\pmb {\lambda }^2\), it will be discussed whether the loop corrections increase the dependence on the individual values \(\lambda _i\).

We consider the following experimental constraints on the scenarios presented:

Table 2 Low-energy values for the parameters, as defined in the text, of the scan over \(\lambda \). Dimensionful parameters are given in GeV. The parameters in the last row are fitted to neutrino oscillation data
  • We use the public code HiggsBounds v.5.2.0 [131,132,133,134,135] to determine whether a BP has been excluded by cross section limits from Higgs searches at LEP, LHC or Tevatron. These searches are mostly sensitive to the heavy Higgs and the right-handed sneutrinos, if these are substantially mixed with the SM-like Higgs boson. The production of the left-handed sneutrinos is much smaller at the LHC, and signals from their decay usually demand dedicated searches [30], especially if the left-handed sneutrino is the LSP [29, 31].

  • The properties of the SM-like Higgs boson, i.e., its mass and signal rates at LHC and Tevatron, are checked using the public code HiggsSignals v.2.2.1 [136,137,138]. Here we assume a theoretical mass uncertainty of \(3\,\, \mathrm {GeV}\). HiggsSignals provides us with a \(\chi ^2\)-analysis of \(n_{\text {obs}}=106\) observables in the 7+8 TeV data package and \(n_{\text {obs}}=101\) observables in the 13 TeV data package. In our plots we show the reduced \(\chi ^2_{\mathrm {red}}=\chi ^2/n_{\mathrm {obs}}\), where a value of \(\chi _{\mathrm {red}}^2=1\) means that on average the signal rates of the SM-like Higgs boson are at the level of the \(\pm 1\sigma \) range of the measurements.

  • The properties of the neutrino sector are in agreement with the measurements of the mass-squared differences and the mixing angles obtained from neutrino oscillation experiments. We check that our predictions are within the \(\pm 3\sigma \) bands published by the NuFit collaboration [139, 140],

    $$\begin{aligned} 6.80\,\, \mathrm {eV}^2&\le \delta m_{12}^2/10^{-5} \le 8.02\,\, \mathrm {eV}^2, \quad 2.399\,\, \mathrm {eV}^2 \nonumber \\&\le \Delta m_{13}^2/10^{-3} \le 2.593\,\, \mathrm {eV}^2, \end{aligned}$$
    (5.1)
    $$\begin{aligned} 0.0198&\le s_{13}^2 \le 0.0244, \quad 0.272 \le s_{12}^2 \nonumber \\&\le 0.346, \quad 0.418 \le s_{23}^2 \le 0.613, \end{aligned}$$
    (5.2)

    where we considered the normal mass ordering which is now favored by experiments [141]. A genetic algorithm was used to find parameter points that minimize the sum of squared deviations between theoretical prediction and experimental values specified above [142]. Even though the \(\mu \nu \mathrm {SSM}\) allows for flavor-violating decays of leptons, the existing experimental bounds (for instance on \(\mu \rightarrow e\gamma \)) are automatically fulfilled when the constrains on neutrino masses are taken into account [26].

For the necessary input of HiggsBounds and HiggsSignals we compute the decays of the scalars at leading order, but with the loop-corrected mixing matrix elements inserted in the expressions of the scalar couplings. In the limit of vanishing external momentum, which we used in the determination of the mixing matrix elements for the couplings, this method corresponds to include the finite wave-function renormalization factors (Z-factors) for each external scalar [63, 143]. For loop-induced decays and off-shell decays to vector bosons we implemented analytic results from the MSSM well known in the literature [144,145,146,147,148], and scaled the expressions with effective couplings defined by the mixing matrix elements and \(\tan \beta \) to obtain the result for the scalars in the \(\mu \nu \mathrm {SSM}\). For the coupling to b quarks we included the running bottom mass and for the decay to gluons the running of \(\alpha _s\) from \(M_Z\) to the mass of the decaying scalar, and finally add leading higher-order QCD corrections [146, 149].

As described in Sect. 2.5 we use the left-handed vevs \(v_{iL}\), the soft gaugino masses \(M_1\) and \(M_2\), and the neutrino Yukawa couplings \(Y^\nu _{ij}\) to fit the neutrino masses and mixings accurately, making use of the fact that they can be modified without spoiling the properties of the SM-like Higgs boson. Besides for the scenario presented in Sect. 5.4, it will be sufficient to just consider diagonal non-zero elements of \(Y^\nu _{ij}\). Because we concentrate here on the scalar sector of the \(\mu \nu \mathrm {SSM}\), and since the fitting has to be done numerically, we do the fitting in our scans just in one particular point for each analysis. By varying a parameter, the prediction for the neutrino properties can be outside the experimentally allowed range in some points. We indicate in our plots when this is the case. Since the neutral fermion mass matrix is of dimension 10, with large hierarchies between the neutrino sector and the remaining part, including one-loop corrections is time-consuming and numerically very challenging. Therefore we stick to a tree-level analysis for the neutrinos. However, we checked for several points that the one-loop corrections are sub-leading and can in principle be compensated by a slight change of the parameters.Footnote 6

5.1 Scan over \(\lambda \)

Fig. 4
figure 4

\({\mathcal {CP}}\)-even scalar spectrum in the scan over \(\lambda \) at tree level, one-loop level and partial two-loop level. We show in the brackets the dominant composition of the tree-level, one-loop and two-loop mass eigenstates \(h^{(0)}\), \(h^{(1)}\) and \(h^{(2)}\), in the experimentally allowed region of the plot. The desired SM-like Higgs-boson mass is indicated with the horizontal green band, assuming a theory uncertainty of \(3\,\, \mathrm {GeV}\). The red regions are excluded by direct searches for additional scalars. In the yellow region the prediction for the mixing angles of the neutrinos lies outside of the 3\(\sigma \) band of the experimental measurement. On top we show \(\chi _{\mathrm {red}}^2\) for various Higgs-boson signal strength measurements at LHC

The first scenario we are presenting is one with a light right-handed \(\mu \)-sneutrino that mixes substantially with the SM-like Higgs boson. We show the chosen parameters in Table 2. To simplify the notation we define \(\lambda =\lambda _i\), \(A^\lambda = A^\lambda _i\), \(A^\nu = A^\nu _{ii}\), \(\kappa =\kappa _{iii}\) and \(A^\kappa = A^\kappa _{iii}\) and vanishing otherwise. The soft parameters are given in terms of \(A^d=A^d_i\), \(A^e=A^e_{ii}\), \(m_{\widetilde{Q}}=m_{\widetilde{Q}_i}\), \(m_{\widetilde{u}}=m_{\widetilde{u}_i}\), \(m_{\widetilde{d}}=m_{\widetilde{d}_i}\), and \(m_{\widetilde{e}}=m_{\widetilde{e}_{ii}}\) and vanishing otherwise. We vary over the universal parameter \(\lambda \), while keeping the remaining parameters fixed. For the right-handed e- and \(\tau \)-sneutrino vevs we chose \(v_{1,3R}=1\,\, \mathrm {TeV}\), but set a smaller value of \(v_{2R}=765\,\, \mathrm {GeV}\) for the \(\mu \)-sneutrino vev to decrease the mass of the \({\mathcal {CP}}\)-even \(\mu \)-sneutrino to the range around the SM-like Higgs-boson mass. The choice to pick \(\widetilde{\nu }_{\mu R}\) as the light right-handed sneutrino is of no relevance. The large value of \(\kappa =0.5\) assures that the other two right-handed sneutrinos will have masses between 300 and \(400\,\, \mathrm {GeV}\), well above \(125\,\, \mathrm {GeV}\). Because the SM-like Higgs boson mass will get additional contributions from the mixing with \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\), \(\tan \beta \) can be chosen rather low.

As mentioned in the previous subsection, we fit the properties of the neutrinos in just one particular point of the parameter scan. In this scenario, this was done for \(\lambda = 0.168\), leading to the values of \(v_{iL}\), \(Y^\nu _{ii}\), \(M_1\) and \(M_2\) shown in Table 2. We emphasize that this effectively leaves just the trilinear parameters \(A^\nu _{ii}\) to adjust the masses of the left-handed sneutrinos. For the prediction of the masses of the right-handed sneutrinos and the SM-like Higgs boson, the fitted parameters only play a minor role.

In Fig. 4 we show the resulting spectrum of the light \({\mathcal {CP}}\)-even scalars. The remaining \({\mathcal {CP}}\)-even scalars not shown in the plot have masses above \(300\,\, \mathrm {GeV}\) and do not play a role in the following discussion. The dotted lines represent the tree-level masses, the dashed lines the masses including the full one-loop corrections, and the solid lines the one-loop + partial two-loop + resummed (referred to as two-loop in the following) corrected masses, as explained in Sect. 4.2. We mark four regions in the plot which are excluded either by HiggsBounds (red), or by not being in agreement with the neutrino oscillation data (yellow). We stress that region 2 is just excluded for the precise choice of parameters shown in Table 2. A new fit of the neutrino properties for each value of \(\lambda \) could easily accommodate predictions for the properties of the neutrinos in agreement with experiments. However, since this would exclusively affect the phenomenology of the heavier left-handed sneutrinos in the scalar sector, we do not apply the fit for each value of \(\lambda \).

Fig. 5
figure 5

Doublet components (\(H_d^{\mathcal {R}}\), \(H_u^{\mathcal {R}}\)) and the right-handed \(\mu \)-sneutrino component (\(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\)) of the lightest \({\mathcal {CP}}\)-even mass eigenstate \(h_1\), which are defined by \(|U^H_{1i}|^2\) with \(i=1,2\) and \(i=4\) respectively

This spectrum is characterized by the interplay between the light \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\) and the SM-like Higgs boson. For small \(\lambda \) the two lightest loop-corrected mass eigenstates \(h_1\) and \(h_2\) have roughly an equal amount of \(H_u^{\mathcal {R}}\)- and \(\widetilde{\nu }_{\nu R}^{\mathcal {R}}\)-admixture (see also Fig. 5). Consequently, region 1 is excluded by direct searches at the LHC, because the diphoton resonance search for a SM-like higgs boson excludes \(h_1\) via its decay to photons [150]. At \(\lambda \sim 0.14\) the point is reached where the mass of \(h_1\) drops well below \(125\,\, \mathrm {GeV}\). Thus, beyond that point \(h_1\) can be identified with \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\), as the doublet-component of \(h_1\) shrinks to values of roughly \(\sim 10\%\). \(h_2\), on the other hand, sheds its sneutrino admixture, so that it can be identified as the SM-like Higgs boson, and the large quantum corrections from the top/stop sector dominantly contribute to the mass of \(h_2\). This yields an increase of the SM-like Higgs boson mass of several GeV, so that beyond region 3 it agrees with the experimental value, assuming a theoretical uncertainty of \(3\,\, \mathrm {GeV}\).

An interesting observation is that in the allowed region of \(\lambda \) the large one-loop corrections change the order of \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\) and the SM-like Higgs boson. While the large shift of the SM-like Higgs-boson mass from \(\sim 83\,\, \mathrm {GeV}\) at tree level to \(\sim 125\,\, \mathrm {GeV}\) at two-loop level are familiar from the MSSM, the large one-loop corrections to \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\), with a tree-level mass of \(\sim 147 \,\, \mathrm {GeV}\) and a two-loop mass below \(100\,\, \mathrm {GeV}\), emphasize the importance of accurately taking into account the full parameter space of the \(\mu \nu \mathrm {SSM}\).

Fig. 6
figure 6

Effective couplings of the two light \({\mathcal {CP}}\)-even scalar mass eigenstates \(h_1^{(2)}\) (red) and \(h_2^{(2)}\) (blue) to up-type quarks (solid) and down-type quarks (dashed), normalized to the SM prediction

Table 3 The same as in Table 2 for the scan over \(v_R=v_{iR}\)

In the allowed region the doublet component of \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\) reaches values of approximately \(10\%\), which can be seen in Fig. 5, where we plot the down- and up-type doublet component \(H_d^{\mathcal {R}}\) and \(H_u^{\mathcal {R}}\), and the \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\)-component of the lightest \({\mathcal {CP}}\)-even scalar mass eigenstates \(h_1^{(2)}\). Naturally, this mixing will also affect the SM-like Higgs-boson properties. In this way, scenarios like the one shown here will be tested by experiments in two different and complementary ways, both caused by the mixing of \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\) and the SM-like Higgs boson: Firstly, direct searches for additional Higgs bosons can be applied to \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\), because it is directly coupled to SM particles. Secondly, precise measurements of the SM-like Higgs-boson couplings can detect (or exclude) possible variations from SM predictions. To illustrate the possible modifications, we show in Fig. 6 the effective coupling of the two light \({\mathcal {CP}}\)-even scalar mass eigenstates to up- and down-type quarks normalized to the SM-prediction which in good approximation can be expressed via the loop-corrected mixing matrix elements \(U^{H(2)}_{ij}\) and \(\beta \);

$$\begin{aligned} c_{h_i d d} = \frac{U_{i1}^{H(2)}}{\cos \beta }, \qquad c_{h_i u u} = \frac{U_{i2}^{H(2)}}{\sin \beta }. \end{aligned}$$
(5.3)

In the experimentally allowed region the effective coupling of the SM-like Higgs boson to up-type quarks shows deviations of roughly \(10\%\). This is of the order of precision expected by measurements of the SM Higgs boson couplings at the High-Luminosity LHC [151], and (depending on the center-of-mass energy deployed) an order of magnitude larger than the uncertainty expected for these kind of measurements at a possible future \(e^+e^-\) collider like the ILC [152,153,154]. Comparing to Fig. 4 we can see that the region where the effective couplings are closest to one, meaning equal to the SM prediction, does not coincide with the region where the \(\chi _{\mathrm {red}}^2\) from HiggsSignals is minimized. This is because the mass of the SM-like Higgs boson is slightly too small in this range of \(\lambda \), so even including a theoretical uncertainty of \(3\,\, \mathrm {GeV}\) some signal strength measurements implemented in HiggsSignals are not accounted for by \(h_2\) and \(\chi _{\mathrm {red}}^2\) becomes worse.

5.2 Scan over \(v_R\)

In Sect. 5.1 we showed that light right-handed sneutrinos with masses in the vicinity of the SM-like Higgs boson are theoretically possible and can induce measurable modifications of the SM-like Higgs-boson properties. Using data of direct searches and measurements of the couplings of the SM-like Higgs boson, the parameter space of these scenarios can be constrained effectively. In this section we present a scenario that is not excluded by current searches in which all three of the \({\mathcal {CP}}\)-even right-handed sneutrinos will have masses below \(125\,\, \mathrm {GeV}\). We chose the parameters appearing in the mass terms of the \(\widetilde{\nu }_{i R}^{\mathcal {R}}\) to be universal, i.e., \(\lambda := \lambda _i\), \(v_R := v_{iR}\), \(\kappa := \kappa _{iii}\), \(A^\lambda := A^\lambda _i\), \(A^\nu := A^\nu _{ii}\) and \(A^\kappa := A^\kappa _{iii}\). As explained at the beginning of Sect. 5, the universality of \(\kappa \) assures that only one of the \(\widetilde{\nu }_{i R}^{\mathcal {R}}\) mixes substantially with the SM-like Higgs boson, while the other two are practically decoupled. This makes it easier to control the total admixture of the doublet components of the \(\widetilde{\nu }_{i R}^{\mathcal {R}}\).

The complete set of free parameters is shown in Table 3. In this scenario we scan over \(v_R\), because they appear linearly in the Majorana-like mass terms of the \(\widetilde{\nu }_{i R}^{\mathcal {R}}\), so it is a convenient parameter to control their masses. Compared to the scan over \(\lambda \) in Sect. 5.1 the overall behavior of the SM-like Higgs boson is aligned more to the SM predictions by decreasing \(\lambda \). Consequently, because at tree level the additional contribution proportional to \(\pmb {\lambda }^2\) is smaller, \(\tan \beta \) is larger to increase the quantum corrections to the SM-like Higgs-boson mass. We also decrease \(\kappa \) to make the masses of the \(\widetilde{\nu }_{i R}^{\mathcal {R}}\) smaller. As before, the parameters in the last row of Table 3 were fitted to accurately predict the left-handed neutrino masses and mixings. The fit was done in the point \(v_R=1226\,\, \mathrm {GeV}\), but in this case the neutrino data is described accurately over the whole range of \(v_R\) at tree level.

In Fig. 7 we show the resulting light \({\mathcal {CP}}\)-even scalar spectrum. In the experimentally allowed region (\(1213\,\, \mathrm {GeV}\le v_R \le 1235\,\, \mathrm {GeV}\)) the lightest mass eigenstates are two almost degenerate right-handed sneutrinos. The third right-handed sneutrino is roughly \(20\,\, \mathrm {GeV}\) heavier, and it acquires substantial mixing with the SM-like Higgs boson. Naturally, the \(\widetilde{\nu }_{i R}^{\mathcal {R}}\) increase their masses when \(v_R\) becomes larger, but also the SM-like Higgs boson mass increases, because the mixing with the \(\widetilde{\nu }_{3 R}^{\mathcal {R}}\) gives additional contributions.

Fig. 7
figure 7

Light \({\mathcal {CP}}\)-even scalar spectrum in the scan over \(v_R\). Shown are the masses at tree level (dotted), at the one-loop level (dashed) and at the partial two-loop level (solid). We show in the brackets the dominant composition of the loop-corrected mass eigenstates \(h^{(2)}\) in the experimentally allowed region of \(v_R\). The desired SM-like Higgs-boson mass is indicated with the horizontal green band, assuming a theory uncertainty of \(3\,\, \mathrm {GeV}\). The red regions are excluded by direct searches for additional scalars. In the gray region the SM-like Higgs-boson mass is not predicted accurately. On top we show \(\chi _\mathrm{red}^2\) for various Higgs-boson signal strength measurements at LHC

The scenario is excluded experimentally for very small values of \(v_R\), because the two lightest mass eigenstates \(h_{1,2}\) become lighter than half the mass of the SM-like Higgs boson \(h_4\), so the decays of \(h_4\) into \(h_{1,2}\) opens up. Experimental searches for the decay of the SM-like Higgs boson into two lighter scalars that subsequently decay into two b-jets and a pair of \(\mu \)-leptons [155] exclude region 1 in Fig. 7. These additional decay channels of the SM-like Higgs boson are also the reason why the \(\chi ^2_{\mathrm {red}}\) rapidly increases in region 1, because it suppresses ordinary SM-like decays of \(h_4\).

When \(v_R\) increases above \(1235\,\, \mathrm {GeV}\) further constrains from direct searches for additional Higgs bosons and measurement of the properties of the SM-like Higgs boson become relevant. \(\chi _{\mathrm {red}}^2\) quickly increases above 2 at \(v_R \sim 1237\,\, \mathrm {GeV}\). Already at \(v_R \sim 1235\,\, \mathrm {GeV}\) the scenario is excluded by LEP searches [156]. Note, that in the red region 2 the mixing of \(\widetilde{\nu }_{3 R}^{\mathcal {R}}\) with the SM-like Higgs boson enlarges, while \(\widetilde{\nu }_{3 R}^{\mathcal {R}}\) is kinematically still in reach of being produced at LEP via the Higgsstrahlung-process. Consequently, the channel \(ee\rightarrow (h_3)Z \rightarrow (b\bar{b})Z\), where \(h_3\) is identified with \(\widetilde{\nu }_{3R}^{\mathcal {R}}\), excludes this interval. Interestingly, in the experimentally allowed region, where the mass of \(\widetilde{\nu }_{3 R}^{\mathcal {R}}\) is even smaller, LEP data cannot rule out this scenario. The reasons for this is not only the smaller mixing of \(\widetilde{\nu }_{3 R}^{\mathcal {R}}\) with the SM-like Higgs boson, but also that in the mass range below \(100\,\, \mathrm {GeV}\) LEP saw a slight excess over the SM background (see also Sect. 5.4) [156].

Beyond region 2 the current scenario is experimentally excluded by the measurement of the SM-like Higgs-boson mass in the gray region 3 and by the LHC cross section measurement of the process \(pp \rightarrow h_1 \rightarrow \gamma \gamma \) in region 4 [157]. This is because in region 3 the cross-over point is reached, in which the masses of the \(\widetilde{\nu }_{i R}^{\mathcal {R}}\) become larger than the SM-like Higgs-boson mass. Through the interference effects the SM-like Higgs-boson mass is pushed to lower values beyond that point. In region 4 the mass eigenstate corresponding to the SM-like Higgs boson is the lightest one at just about \(118\,\, \mathrm {GeV}\). Even though there are two scalars in the mass range of the experimentally measured Higgs-boson mass, there is no contribution to any signal-strength measurement at the LHC, reflected by the fact that the \(\chi _\mathrm{red}^2\) is huge in region 4. The reason is that these states correspond to the practically singlet like right-handed neutrino states. The third right-handed sneutrino carrying the doublet admixture taken from the SM-like Higgs boson has a mass of over \(140\,\, \mathrm {GeV}\). Hence, it also does not contribute to signal-strength measurements of the SM-like Higgs boson.

Table 4 The same as in Table 2 for the scan over \(\lambda _i\) while \(\pmb {\lambda }^2=\text {const}\)

On a side note we briefly discuss the remaining light scalar \(h_5\) in Fig. 7, which is the left-handed \(\tau \)-sneutrino at roughly 235–\(240\,\, \mathrm {GeV}\). The fit to the neutrino oscillation data generated a hierarchy between the vevs of the left-handed sneutrinos, with \(v_{3L}\) being the largest. As a result, since dominant tree-level contributions to the \(\widetilde{\nu }_{iL}^{\mathcal {R}}\)-masses scale with inverse of \(v_{iL}\) (see Eq. (2.23)), the \(\widetilde{\nu }_{\tau L}^{\mathcal {R}}\) is the lightest \({\mathcal {CP}}\)-even left-handed sneutrino. It is rather invisible to usual searches for additional Higgs bosons at colliders, but a dedicated analysis of LHC data was proposed to search for light left-handed sneutrinos in the framework of the \(\mu \nu \mathrm {SSM}\)   [30]. However, the analysis in Ref. [30] concentrated on \(\tau \)-sneutrinos as the LSP, whereas here there are even lighter SUSY particles in the spectrum. For a detailed discussion of distinct signatures at the LHC related to left-handed sneutrinos within the \(\mu \nu \mathrm {SSM}\) we refer to the literature [29,30,31].

5.3 Scan over \(\lambda _i\) while \(\pmb {\lambda }^2 =\lambda _i\lambda _i=\text {const.}\)

As already explained at the beginning of this section, there is no theoretical reason to choose the \(\mu \nu \mathrm {SSM}\)-like parameters universal w.r.t. the family index. While the tree-level upper bound on the lightest \({\mathcal {CP}}\)-even scalar mass approximately depends on the term \(\pmb {\lambda }^2 =\lambda _i\lambda _i\), and not on the individual \(\lambda _i\), this is not the case for the SM-like Higgs-boson mass, as soon as mixing-effects induced by the right-handed sneutrinos are considered. This effects can enter at tree level, or via radiative corrections proportional to \(\lambda _i\). These radiative corrections depend on the masses and the mixing of each of the right-handed sneutrino. Since it is not the case that all three \(\widetilde{\nu }_{iR}\) are degenerate, the radiative corrections are expected to depend strongly on the individual values of \(\lambda _i\). Also, when \(\pmb {\lambda }^2\) is fixed, the \(\mu \)-term which is dynamically generated after EWSB and linearly dependent on \(\lambda _i\) cannot be constant when the \(\lambda _i\) are varied. This can be another source of corrections to the SM-like Higgs-boson mass that explicitly depend on the individual values of the \(\lambda _i\).

However, the loop corrections proportional to \(\lambda _i\) are an order of magnitude smaller than the ones stemming from the (s)top-sector, partially because quantum contributions to the SM-like Higgs-boson mass at the one-loop level proportional to \(\lambda _i\) depend on the singlet-admixture of the SM-like Higgs boson which, in turn, cannot be too large to not spoil the measured signal strengths at the LHC. Nevertheless, we will give here a rough idea of how large the remnant effect of non-universal \(\lambda _i\) on the SM-like Higgs-boson mass can be while \(\pmb {\lambda }^2\) is kept constant. We performed a parameter scan over all possible values of the \(\lambda _i\) in a scenario in which \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\) has a mass between 92 and \(115\,\, \mathrm {GeV}\) and mixes substantially with the SM-like Higgs boson. The free parameters were set to the values shown in Table 4.

On can see that the scenario is very similar to the one in Sect. 5.1. When the \(\lambda _i\) are chosen uniformly we recover the BP at \(\lambda = 0.168\) in Fig. 4 which lies in the middle of the experimentally allowed parameter region. In Fig. 8 we illustrate the dependence of the SM-like Higgs-boson mass on the individual values of \(\lambda _i\). We show triangle plots [158] with the values of \(\lambda _i^2\) on the axes and with their sum \(\pmb {\lambda }^2=\text {const}\). The colors of the points indicate the mass of the SM-like Higgs boson at tree level (top left), at one-loop level (top right), the difference of the SM-like Higgs-boson mass at tree level and one-loop level (bottom left), and the one-loop mass of the right-handed \(\mu \)-sneutrino (bottom right) which is the lightest \({\mathcal {CP}}\)-even scalar in this scenario. We do not show the two-loop mass of the SM-like Higgs boson, because the two-loop corrections supplemented from FeynHiggs are purely MSSM-like corrections independent of \(\lambda _i\), thus not playing a role in the following discussion. However, the parameters are chosen such that the corrections beyond the one-loop level shift the SM-like Higgs-boson mass into the vicinity of \(\sim 125\,\, \mathrm {GeV}\) (see below). In the upper right plot one sees that the one-loop mass of the SM-like Higgs boson is the largest in the central point in which all \(\lambda _i\) are equal. The tree-level mass, on the other hand, shows the opposite behavior and is the largest in the corners of the upper left plot in which one of the \(\lambda _i\) is practically zero. The one-loop mass varies in the experimentally allowed region by more than \(1\,\, \mathrm {GeV}\). This demonstrates that for an accurate prediction of the SM-like Higgs-boson mass it is crucial to include the independent contributions of all three \(\lambda _i\) to the radiative corrections, when mixing effects between the right-handed sneutrinos and the SM-like Higgs boson are sizable.

Fig. 8
figure 8

SM-like Higgs-boson mass at tree level \(m_{h^{\mathrm {SM}}}^{(0)}=m_{h_1^{(0)}}\) (top left), and at one-loop level \(m_{h^{\mathrm {SM}}}^{(1)}=m_{h_2^{(1)}}\) (top right), and their difference \(\Delta m_{h^{\mathrm {SM}}}^{(1-0)} = m_{h^{\mathrm {SM}}}^{(1)} - m_{h^{\mathrm {SM}}}^{(0)}\) (bottom left), and the one-loop mass of \(\widetilde{\nu }_{\mu R}\) (bottom right) for fixed \(\pmb {\lambda }^2\) but varying \(\lambda _i\), indicated by the colors (and sizes for better visibility). Crossed points are excluded by either HiggsBounds, HiggsSignals with \(\chi ^2_\mathrm{red}\ge 150/101=1.485\) for the \(13\,\, \mathrm {TeV}\) data set, or because the SM-like Higgs boson mass including the two-loop corrections is smaller than 123 GeV or larger than 127 GeV

Note that the variation of the one-loop mass would be even larger if we neglect the experimental constrains. In this scenario, the main exclusion limit is the requirement to have the SM-like Higgs-boson mass above \(123\,\, \mathrm {GeV}\) at two-loop. In the corners of the plots, the mass of \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\), shown in the lower right plot, increases to values very close to the SM-like Higgs-boson mass. This increases the mixing between both scalars which, in turn, reduces the radiative one-loop corrections to the SM-like Higgs boson. Practically speaking, parts of the loop-corrections “are lost” to \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\). This is why the difference between the tree-level and the one-loop mass of the SM-like Higgs boson, shown in the lower left plot of Fig. 8, is the smallest when the mass of \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\) is the largest.

We also emphasize that the variation of the difference between tree-level and one-loop mass, as a measure for the size of genuine one-loop corrections, is more than twice as large as the variation of the one-loop mass of the SM-like Higgs boson. One can see a compensation of lower tree-level mass, but larger loop corrections in the center of the plots, leading to a more stable one-loop mass of the SM-like Higgs boson. This is due to the fact that the loop-corrected mass eigenstate of the SM-like Higgs boson is the physical state, whose properties are constrained by the experimental measurements. Therefore, the singlet-admixture cannot be very large at loop-level, and the parameter dependence of the SM-like Higgs-boson mass induced by this admixture necessarily cannot be too large. The tree-level state, on the other hand, is not physical and can have larger mixing with \(\widetilde{\nu }_{\mu R}^{\mathcal {R}}\), leading to a stronger dependence on parameters related to the sneutrinos, like \(\lambda _i\) in this case. Thus, even though the overall dependence of the SM-like Higgs-boson mass on parameters beyond the MSSM can be very large at tree level, the dependence will be diminished at loop-level in the parameter space that fulfills experimental constraints on the SM-like Higgs-boson properties.

5.4 CMS and LEP excess at \(\sim 96\,\, \mathrm {GeV}\)

Searches for the SM Higgs boson at LEP can nowadays be used to constrain the parameter space of models with additional scalar particles with masses below the SM-like Higgs-boson mass. Interestingly, an excess over the SM background was observed at a mass of 95-\(98\,\, \mathrm {GeV}\) in the Higgsstrahlung production channel with an associated decay of the Higgs boson to a pair of b-quarks [156]. Assuming that this excess can be explained by an additional Higgs boson, the signal strength, i.e, the cross section times branching ratio to b-quarks normalized to the SM prediction for a SM-like Higgs boson at the same mass, was extracted in Ref. [159] to be

$$\begin{aligned} \mu _{\mathrm {LEP}} = 0.117 \pm 0.057. \end{aligned}$$
(5.4)

At about the same mass, CMS observed an excess over the SM background in the pp production with associated decay to diphotons in the \(8\,\, \mathrm {TeV}\) and the \(13\,\, \mathrm {TeV}\) data [160, 161]. For the CMS excess, the signal strength was reported in Refs. [161, 162] to be

$$\begin{aligned} \mu _{\mathrm {CMS}} = 0.6 \pm 0.2. \end{aligned}$$
(5.5)

In a previous publication, we showed that it is possible to accommodate both excesses simultaneously at the \(1\sigma \) level in the \(\mu \nu \mathrm {SSM}\) with just one right-handed neutrino [99]. We described how a right-handed sneutrino at \(\sim 96\,\, \mathrm {GeV}\) can acquire substantial couplings to SM particles via its mixing with the SM-like Higgs boson.Footnote 7 However, we did not include an accurate prediction of the properties of the neutrinos in our analysis in the \(\mu \nu \mathrm {SSM}\) with one generation of right-handed neutrinos, because in that case at least one neutrino mass has to be generated via quantum corrections. On the contrary, in the \(\mu \nu \mathrm {SSM}\) with three generations of right-handed neutrinos, we can describe the mass differences and mixings of the neutrinos at tree level, which is of course much more feasible.

Table 5 Parameters of the scan to fit the LEP and the CMS excesses. Dimensionful parameters are given in GeV. If the family index is omitted the parameter has a universal value independent of the index

The values of the free parameters to fit the excesses are shown in Table 5. The scalar used to fit the excesses is the right-handed \(\tau \)-sneutrino \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\). This is assured by setting \(\kappa _{333} \ll \kappa _{111,222}\), so that \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) has a smaller mass than the other two right-handed sneutrinos. Then, \(v_{3R}\) and \(A^\kappa _{333}\) are used to tune the mass of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) to be at \(\sim 95\)\(98\,\, \mathrm {GeV}\). A sufficiently large mixing of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) with the SM-like Higgs boson is achieved with a large value of \(\lambda _3 \sim 0.54\), while \(\lambda _{1,2}=0.01\) are very small to avoid that the effective \(\mu \)-term becomes very large. Alternatively, one could have used smaller values for \(v_{1,2 R}\), but then the other two \({\mathcal {CP}}\)-even sneutrinos \(\widetilde{\nu }_{e,\mu \; R}^{\mathcal {R}}\) would have been very light as well, potentially carrying away some of the mixing between the SM-like Higgs boson and the right-handed sneutrinos.Footnote 8 Since \(\lambda _3\) is large, the SM-like Higgs boson receives additional contribution to the tree-level mass. This is why \(\tan \beta \) is set to a small value, and, besides \(A^u_3 = -\,650\,\, \mathrm {GeV}\), the soft trilinear parameters \(A^{u,d,e}\) can be set to zero. Apart from that, \(\tan \beta \) shall not be much larger than one to not suppress the coupling of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) to t-quarks, which scales with the inverse of \(\sin \beta \) (see Eq. (5.3)). Finally, we choose a range for \(A^\lambda _3\) in which the mixing between \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) and the SM-like Higgs boson is sufficiently large. We will show results for small ranges of the parameters \(\lambda _3\), \(v_{3R}\), \(A^\lambda _3\) and \(A^\kappa _{333}\). While \(v_{3R}\) and \(A^\kappa _{333}\) are mainly correlated to the mass of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\), \(\lambda _3\) and \(A^\lambda _3\) affect the doublet composition of \(\widetilde{\nu }_{3 R}\). This certainly is not an exhaustive parameter scan covering the complete parameter space, but the scan gives an idea of how the excesses can be accommodated within the \(\mu \nu \mathrm {SSM}\), and it resembles the solution we found in the \(\mu \nu \mathrm {SSM}\) with just one right-handed neutrino [99].

Fig. 9
figure 9

Values for \(\mu _{\mathrm {LEP}}\) (top) and for \(\mu _ {\mathrm {CMS}}\) (bottom) for each parameter point versus the mass of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\). The colors indicate the mass of the SM-like Higgs boson

Fig. 10
figure 10

Correlation of both signal strengths, with the colors encoding the mass of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\)

In Fig. 9 we show the results for the signal strength of the LEP excess \(\mu _{\mathrm {LEP}}\) (top) and of the CMS excess \(\mu _{\mathrm {CMS}}\) (bottom). In both plots the colors of the points indicate the SM-like Higgs-boson mass, while the mass of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) is shown on the horizontal axis. The signal strengths were calculated in the narrow-width approximation, and the branching ratio and cross section ratios w.r.t. the SM where calculated using the effective coupling approximation as explained in Ref. [99]. One can immediately see that it is rather easy to achieve the experimental value of \(\mu _{\mathrm {LEP}}\), whereas the largest values for \(\mu _{\mathrm {CMS}}\) reached in our scan are just below the lower limit of \(\mu _{\mathrm {CMS}}=0.4\), i.e., \(1\sigma \) below the central value. This is due to the fact that the main decay channel of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) is the decay to a pair of bottom quarks, and it is harder to achieve a substantial branching ratio to diphotons required for the CMS excess. Nevertheless, both excesses are fitted at the \(1\sigma \) level considering the experimental uncertainties, while fitting the neutrino data and being in agreement with the experimental constraints on the SM-like Higgs boson, which we again checked with HiggsSignals assuming a theoretical uncertainty of the SM-like Higgs-boson mass of \(3\,\, \mathrm {GeV}\).

In Fig. 10 we show the correlation of both signal strengths, with the colors encoding the mass of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\). The strong correlation one can see has its origin in the fact that both signal strengths increase with the amount of doublet-component of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\). In principle, one could achieve a further enhancement of \(\mu _{\mathrm {CMS}}\) and a suppression of \(\mu _{\mathrm {LEP}}\) by suppressing the down-type doublet component of \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\). Then, the branching ratio to bottom quarks becomes smaller, and the diphoton branching ratio increases because of the smaller total decay width due to the reduction of the decay width to bottom quarks. However, finding such points is difficult, because the dominant terms mixing the right-handed sneutrinos \(\widetilde{\nu }_{i R}^{\mathcal {R}}\) with the doublet fields \(H_d^{\mathcal {R}}\) and \(H_u^{\mathcal {R}}\) scale equally with \(\lambda _i\), \(A^\lambda _i\), \(\kappa _{ijk}\) and \(v_{iR}\) at tree level, as can be seen in Eqs. (A.4) and (A.5). The only difference are the factors \(v_d\) and \(v_u\) in each equation, respectively, which cannot be exploited too much, because, as mentioned before, \(\tan \beta \) should not be too far from one. This is why an extensive scan of the vast parameter space of the \(\mu \nu \mathrm {SSM}\) would be necessary to find parameter points in which \(\mu _{\mathrm {CMS}}\) is further enhanced without increasing \(\mu _{\mathrm {LEP}}\) too much, which however lies beyond the scope of this paper.

Instead, we will focus on the rest of the spectrum, which heavily depends on the values of the neutrino Yukawa couplings \(Y^\nu _{ij}\) and the vevs of the left-handed sneutrinos \(v_{iL}\), once the remaining parameters are fixed to the values listed in Table 5. As an example, we show in Table 6 two possible sets of parameters that accommodate accurate neutrino masses and mixings in the parameter scan of this section. In contrast to the other scenarios we presented before, here we will make use of non-zero values of the non-diagonal elements of \(Y^\nu _{ij}\) in one of the BPs. Naturally, this simplifies the accommodation of neutrino properties in agreement with experimental data, because there are six more free parameters that can be adjusted. The price to pay is that there is usually more than one set of parameters of \(Y^\nu _{ij}\) and \(v_{iL}\) that give accurate predictions for the neutrino sector. The reason for showing two distinct scenarios is that in the \(\mu \nu \mathrm {SSM}\) the scalar sector is deeply related to the neutrino sector. Thus, different sets of parameters predict fundamentally different scalar spectra, and since there is no theoretical argument that the neutrino Yukawa couplings have to be diagonal, we used the additional freedom to present a point in which, on top of the explanation of the LEP and the CMS excesses, there are a several other light particles possibly in reach of future colliders.

Table 6 Parameter sets BP1 and BP2 used to fit the neutrino oscillation data accurately in the scan reproducing the LEP and the CMS excesses. In the last four rows we list the masses of the six lightest non-SM particles (in addition to \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\) at \(\sim 96\,\, \mathrm {GeV}\)) for each BP. Dimensionful parameters are given in GeV

In both BPs the lightest BSM particle is the right-handed \(\tau \)-neutrino. This is because \(\kappa _{333}\) has to be small to decrease the mass of the corresponding sneutrino \(\widetilde{\nu }_{\tau R}^{\mathcal {R}}\). Consequently, also the Majorana mass term for the neutrino will be small and \(m_{\nu _{\tau R}}\) is small. However, its mass is still above half the SM-like Higgs-boson mass, so the decay of the SM-like Higgs boson into \(\nu _{\tau R}\) is forbidden. The striking difference between both BPs is the mass scale of the left-handed \(\tau \)-sneutrinos and sleptons. In BP2 the Yukawa coupling \(Y^\nu _{33}\) is the smallest diagonal element of \(Y^\nu \), and the vev corresponding to the third family of left-handed sneutrinos \(v_{3L}\) is the largest of the three. This reduces the masses of the left-handed \({\mathcal {CP}}\)-even and \({\mathcal {CP}}\)-odd \(\tau \)-sneutrino (see Eq. (2.23)) and the \(\tau \)-slepton to values below the SM-like Higgs-boson mass. In BP1, on the other hand, \(v_{2L}\) is the largest of the three left-handed vevs, and therefore the left-handed \(\mu \)-sneutrinos and \(\mu \)-slepton are the lightest left-handed sfermions, although still more than twice as heavy as the SM-like Higgs boson. In this way, the phenomenology of both BPs is distinct, even though the properties of the right-handed \(\tau \)-sneutrino, which is the particle used to fit the excesses, are not affected by the left-handed sector. This is because its branching ratios are dominantly given by the mixing-effects with the SM-like Higgs boson, which is not suppressed by the small neutrino Yukawa couplings \(Y^\nu _{ij}\).

Nonetheless, BP2 can give rise to additional interesting signal at colliders. A dedicated analysis of the collider phenomenology of light left-handed \(\tau \)-sneutrinos/sleptons at the LHC can be found in Refs. [29, 30], where it was shown that there are no direct bounds from LEP/LHC searches that can be used to set lower limits on the masses of these particles in the framework of the \(\mu \nu \mathrm {SSM}\). This analysis made use of the fact that the charged \(\widetilde{\tau }_L\) can be produced, and thus provide a source for the left-handed sneutrinos, since it decays into lighter \(\widetilde{\nu }_{\tau L}^{\mathcal {R}}\) or \(\widetilde{\nu }_{\tau L}^{\mathcal {I}}\). An important feature is that the subsequent decays of \(\widetilde{\nu }_{\tau L}^{\mathcal {R,I}}\) can be prompt or displaced. However, in Refs. [29, 30] it is assumed that the left-handed sneutrino is the LSP. This is not the case here, since the right-handed \(\tau \)-neutrino is even lighter. Signals at colliders from the lightest BSM particle \(\nu _{\tau R}\) are not expected, because it is a gauge singlet, thus cannot be produced directly. In principle, it can be produced indirectly via the decay of the sfermions. However, the spectrum is very compressed, such that a pair production of \(\nu _{\tau R}\) from the decays of \(\widetilde{\nu }_{\tau R}\) at \(\sim 96\,\, \mathrm {GeV}\) or the SM-like Higgs boson at \(\sim 125\,\, \mathrm {GeV}\) is kinematically forbidden, and the production of a pair of a right- and a left-handed \(\tau \)-neutrino is suppressed by the size of \(Y^\nu _{33}\).

6 Conclusion and outlook

The \(\mu \nu \mathrm {SSM}\) is a simple SUSY extension of the SM that is capable of describing neutrino physics in agreement with experimental data. As in other SUSY models, higher-order corrections are crucial to reach a theoretical uncertainty at the same level of the (anticipated) experimental accuracy. So far, higher-order corrections in the \(\mu \nu \mathrm {SSM}\) had been restricted to \(\smash {\overline{\mathrm {DR}}}\) calculations, which suffer from the disadvantage that they cannot be directly connected to (possibly future observed) new BSM particles. More recently we had evaluated the corrections to the neutral scalar masses and mixings, but restricting ourselves to one generation of heavy neutrinos [99].

In this paper we have presented the complete one-loop renormalization of the neutral scalar sector of the \(\mu \nu \mathrm {SSM}\) with three generation of right-handed neutrinos in a mixed on-shell/\(\smash {\overline{\mathrm {DR}}}\) scheme. In this way, for the first time, it is possible to evaluate the masses and mixings in the neutral scalar sector with high precision, while simultaneously describe correctly the experimental neutrino data, such as mass differences and mixing angles. An on-shell (OS) renormalization has been chosen for parameters that can be directly identified with (potentially) observable states, whereas a \(\smash {\overline{\mathrm {DR}}}\) renormalization has been chosen for the remaining parameters. We provide details on the full set renormalization parameters, which were implemented into a FeynArts model file (that can be provided by the authors upon request).

We have performed the calculation of the masses of the neutral scalars in the \(\mu \nu \mathrm {SSM}\) with three generations of heavy neutrinos, taking into account the full set of one-loop corrections as described above. These corrections have been supplemented by contributions to the neutral Higgs-boson sector of the MSSM at and beyond the two-loop level as provided by FeynHiggs. These corrections are crucial to obtain a reliable prediction of the mass of an SM-like Higgs boson around \(\sim 125 \,\, \mathrm {GeV}\).

The masses of the neutral scalar bosons have been evaluated in a set of scenarios that exemplify the relevant dependences on the underlying \(\mu \nu \mathrm {SSM}\) parameters. The scenarios are in agreement with all available searches for additional Higgs bosons (via the code HiggsBounds), as well with the properties of the SM-like Higgs boson as measured at the LHC (via the code HiggsSignals), while at the same time reproducing correctly the measured values of neutrino mass differences and mixing angles. In a first scenario we varied the assumed to be universal parameter \(\lambda \). We find that large one-loop corrections for the right-handed sneutrinos arise, that are crucial to accurately account for possible mixing effects between them and the SM-like Higgs boson. In a second scenario we have varied the assumed to be universal vev of the right-handed sneutrinos. Here we find that all three \({\mathcal {CP}}\)-even right-handed sneutrinos can have masses below \(\sim 125\,\, \mathrm {GeV}\), without being excluded by cross-section limits from direct searches for additional Higgs bosons. In our third scenario we deviate from the intergenerational universality assumptions. We take \(\pmb {\lambda }^2 := \lambda _i \lambda _i\) to be constant, but vary instead the individual \(\lambda _i\). We find that the non-universality of the \(\lambda _i\) has an important impact on the predictions of the neutral scalar masses. It has been shown that in the experimentally allowed parameter space the non-universality of the \(\lambda _i\) can account for deviations of the SM-like Higgs-boson mass of \(\sim 1 \,\, \mathrm {GeV}\), emphasizing the importance to consider the full set of \(\mu \nu \mathrm {SSM}\) parameters at the one-loop level.

As a final example we have discussed how the \(\mu \nu \mathrm {SSM}\) can describe two excesses in the searches for light Higgs bosons in the vicinity of \(\sim 96 \,\, \mathrm {GeV}\). These are a \(\sim 3\, \sigma \) excess in the diphoton final state as reported by CMS and a \(\sim 2\, \sigma \) excess in the \(b{\bar{b}}\) final state as published by LEP. We demonstrated that the \(\mu \nu \mathrm {SSM}\) can account at the \(\sim 1\,\sigma \) level for both excesses, while being in agreement with all available Higgs-boson searches and measurements, as well as the available neutrino data. We are eagerly awaiting updated experimental analyses from ATLAS and CMS to confirm or refute these excesses.

Further explorations of the scalar sector of the \(\mu \nu \mathrm {SSM}\) are necessary to cover the wealth of (possible) phenomenology that this model offers. This includes further studies dropping the (artificial) intergenerational universality assumptions, where in this paper we have taken only the first step. It furthermore includes the evaluation of the charged scalar sector at the one-loop level and beyond, which is expected to be important for the phenomenology at the LHC. Also studies going beyond the LHC searches may become relevant, such as analyses of the possibility of a first order phase transition, leading to gravitational waves created in the early universe. We leave these studies for future work.