On-shell neutral Higgs bosons in the NMSSM with complex parameters

Abstract

The Next-to-Minimal Supersymmetric Standard model (NMSSM) appears as an interesting candidate for the interpretation of the Higgs measurement at the LHC and as a rich framework embedding physics beyond the Standard Model. We consider the renormalization of the Higgs sector of this model in its $\mathcal{CP}$-violating version, and propose a renormalization scheme for the calculation of on-shell Higgs masses. Moreover, the connection between the physical states and the tree-level ones is no longer trivial at the radiative level: a proper description of the corresponding transition thus proves necessary in order to calculate Higgs production and decays at a consistent loop order. After discussing these formal aspects, we compare the results of our mass calculation to the output of existing tools. We also study the relevance of the on-shell transition matrix in the example of the $h_i\rightarrow \tau ^+\tau ^-$ width. We find deviations between our full prescription and popular approximations that can exceed 10%.

1 Introduction

Since the discovery of a Higgs-like particle with a mass around 125GeV by the ATLAS and CMS experiments [1, 2] at CERN, a lot of efforts have been made to reveal its nature as the particle responsible for electroweak symmetry breaking. While within the present experimental uncertainties the properties of the observed state are compatible with the predictions of the Standard Model (SM) [3] many other interpretations are possible as well, in particular as a Higgs boson of an extended Higgs sector.

One of the prime candidates for physics beyond the SM is softly broken supersymmetry (SUSY), which doubles the particle degrees of freedom by predicting two scalar partners for each SM fermion, as well as fermionic partners for all bosons—for reviews see [4, 5]. The Next-to-Minimal Supersymmetric Standard Model (NMSSM) [6, 7] is a well-motivated extension of the SM. In particular it provides a solution for the “$\mu$ problem” [8] of the Minimal Supersymmetric Standard Model (MSSM), by naturally relating the $\mu$ parameter to a dynamical scale of the Higgs potential [9, 10].

In contrast to the single Higgs doublet in the SM, the Higgs sector of the NMSSM contains two Higgs doublets (like the MSSM) and one Higgs singlet. After electroweak symmetry breaking the physical spectrum consists of five neutral Higgs bosons, $h_i$ ($i \in [1,5]$), and the charged Higgs boson pair $H^\pm$. Ever since the Higgs discovery, the possibility to interpret this signal in terms of an NMSSM (mostly) $\mathcal{CP}$-even Higgs boson has been emphasized in Refs. [11,12,13,14,15,16,17,18,19,20]. In particular, it has been argued that such a solution came with improved naturalness compared to the MSSM interpretation [21,22,23,24,25,26,27,28,29]. Moreover, several works have pointed out the possibility to accommodate deviations from a strict SM behavior in the diphoton rate, in Higgs-pair production or in associated production [30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47]. Admittedly, the viability of the extended NMSSM Higgs sector would be comforted by the detection of additional Higgs states. To this end, several search channels have been suggested, especially for states lighter than 125 GeV [48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65]. Another feature of the NMSSM phenomenology is the extended neutralino sector, due to the singlino.

In contrast to the situation in the MSSM, $\mathcal{CP}$-violation can already occur at the tree level in the NMSSM Higgs sector [10, 17, 66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82]. While low-energy observables place limits on such $\mathcal{CP}$-violating scenarios [83], especially on MSSM-like phases [84], $\mathcal{CP}$-violation beyond the SM appears as a well-motivated requirement for a successful baryogenesis [85]. Correspondingly, several computer tools have been proposed in the past few years to promote the study of the $\mathcal{CP}$-violating NMSSM: SPHENO [86,87,88,89] and FlexibleSUSY [90, 91]—which employ SARAH [92,93,94,95] in order to produce their modelfiles; FlexibleSUSY contains components from SoftSUSY [96, 97] and only the $\mathcal{CP}$-conserving case is explicitly mentioned for both—as well as NMSSMCALC [98, 99] and NMSSMTools [100,101,102,103].

In this work, we specialize in the $Z_3$-conserving version of the NMSSM, characterized by a scale-invariant superpotential. The main effort of our project consists in analyzing radiative corrections in the Higgs sector of the $\mathcal{CP}$-violating NMSSM. To serve this purpose, we elaborated a FeynArts [104, 105] model file and a set of Mathematica routines for the evaluation of the Higgs masses and wave-function normalization matrix at full one-loop order and beyond. These should serve as a basis for a future inclusion of the $\mathcal{CP}$-violating NMSSM in the FeynHiggs [106,107,108,109,110,111,112] package—originally designed for precise calculations of the masses, decays, and other properties of the Higgs bosons in the $\mathcal{CP}$-conserving or -violating MSSM. A first step in this direction is represented by Ref. [113], centering on the $\mathcal{CP}$-conserving NMSSM. In the current paper, we expand this project further. We follow the general methodology of FeynHiggs, relying on a Feynman-diagrammatic calculation of radiative corrections, which employs FeynArts [104, 105], FormCalc [114] and LoopTools [114]. Our chosen renormalization scheme differs somewhat from earlier proposals [88, 113, 115]. In particular, in our renormalization scheme, the electromagnetic coupling e—which is related to the fine-structure constant $\alpha = e^2/(4\pi )$—is defined in terms of the Fermi constant $G_F$ measured in muon decays.

In Sect. 2, we shall introduce relevant notations and describe the renormalization procedure underpinning our model file for the $\mathcal{CP}$-violating NMSSM. In this section we also describe our implementation of higher-order corrections in the Higgs sector. A numerical evaluation of our results follows in Sect. 3, where we will validate our calculation by a comparison with public codes. We will also insist on the relevance of the field-renormalization matrix for a consistent evaluation of the Higgs decays at the one-loop level, before a short conclusion in Sect. 4.

2 Higgs masses and mixing in the $\mathcal{CP}$-violating NMSSM

After a few general remarks concerning our notations and conventions, we present the renormalization conditions that we employ in our calculation. There, we focus on effects beyond the MSSM in the Higgs and higgsino sectors, since we otherwise align with the conventions of FeynHiggs, described in [109]. Then we discuss how to formally extract the loop-corrected Higgs masses and the wave-function normalization factors.

2.1 Conventions and relations at the tree level

In the following, we consider the $Z_3$-conserving version of the NMSSM and neglect flavor mixing. The superpotential of the NMSSM (showing only one generation of fermions/sfermions) reads

$$\begin{aligned} W= & {} Y_u\,\hat{u} \left( \hat{H}_2\cdot \hat{Q} \right) - Y_d\,\hat{d} \left( \hat{H}_1\cdot \hat{Q} \right) - Y_e\,\hat{e} \left( \hat{H}_1\cdot \hat{L} \right) \nonumber \\&+ \lambda \,\hat{S} \left( \hat{H}_2 \cdot \hat{H}_1 \right) + \frac{1}{3}\,\kappa \,\hat{S}^3, \end{aligned}$$

(2.1)

where $\hat{Q}$, $\hat{u}$, $\hat{d}$, $\hat{L}$, $\hat{e}$, $\hat{H}_1$, $\hat{H}_2$, $\hat{S}$ denote the quark, lepton and Higgs superfields. The dot $\cdot$ stands for the $SU(2)_\text {L}$-invariant product. The Yukawa couplings in Eq. (2.1) can be complex in general. However, their phases can be absorbed in a re-definition of the quark and lepton superfields. We may write the scalar fields in $\hat{H}_1$, $\hat{H}_2$ and $\hat{S}$ explicitly in terms of their (real and positive) vacuum expectation values (VEVs), $v_1$, $v_2$ and $v_s$, respectively, as well as their $\mathcal{CP}$-even, $\mathcal{CP}$-odd, and charged components, $\phi _i$, $\chi _i$, and $\phi ^\pm _i$,

$$\begin{aligned} \mathcal {H}_1&= \mathrm {e}^{{i}\,\xi _1} \begin{pmatrix} v_1 + \frac{\left( \phi _1 + {i}\chi _1\right) }{\sqrt{2}} \\ \phi _1^- \end{pmatrix},&\mathcal {H}_2&= \mathrm {e}^{{i}\,\xi _2} \begin{pmatrix} \phi _2^+ \\ v_2 + \frac{\left( \phi _2 + {i}\chi _2\right) }{\sqrt{2}} \end{pmatrix},\nonumber \\ \mathcal {S}&= \mathrm {e}^{{i}\,\xi _s} \left[ v_s + \tfrac{\left( \phi _s+i\chi _s\right) }{\sqrt{2}}\right] . \end{aligned}$$

(2.2)

Here $\xi _1$, $\xi _2$ and $\xi _s$ are the phases of the two Higgs doublets and the Higgs singlet, respectively. It is convenient to define the ratio $\tan {\beta }= v_2/v_1$, the geometric mean of the doublet VEVs $v = \sqrt{v_1^2 + v_2^2}$, as well as the sum $\xi = \xi _1 + \xi _2$ of the doublet phases. Since $\hat{S}$ transforms as a singlet under the SM-gauge transformations, the D-terms of the scalar potential are unchanged with respect to the MSSM. On the other hand, as compared to the MSSM, additional dimensionless, complex parameters $\lambda =|\lambda |\,\mathrm {e}^{{i}\,\phi _\lambda }$ and $\kappa =|\kappa |\,\mathrm {e}^{{i}\,\phi _\kappa }$ appear while the complex $\mu$-term is absent. The latter is dynamically generated as an effective $\mu$-term when the singlet field takes its VEV,

$$\begin{aligned} \mu _{\text {eff}}= |\mu _{\text {eff}}|\, \mathrm {e}^{{i}\,\phi _\mu } = |\lambda |\, v_s\,\mathrm {e}^{{i}(\phi _\lambda + \xi _s)}. \end{aligned}$$

(2.3)

In the NMSSM the phases $\xi$ and $\xi _s$ only appear in the combinations $\phi _{\lambda }+\xi _s+\xi$ and $\phi _{\kappa }+3\,\xi _s$, so that they could be absorbed in a re-definition of $\phi _{\lambda }$ and $\phi _{\kappa }$. Nevertheless, we will keep the dependence on all phases of the Higgs sector explicitly, in order to allow for more flexibility on the choice of input.

Soft SUSY-breaking in the NMSSM is parametrized by the complex trilinear soft-breaking parameters $A_\lambda = |A_\lambda |\,\mathrm {e}^{{i}\,\phi _{A_\lambda }}$, $A_\kappa = |A_\kappa |\,\mathrm {e}^{{i}\,\phi _{A_\kappa }}$, $A_u = |A_u|\,\mathrm {e}^{{i}\,\phi _{A_u}}$, $A_d = |A_d|\,\mathrm {e}^{{i}\,\phi _{A_d}}$, and $A_e = |A_e|\,\mathrm {e}^{{i}\,\phi _{A_e}}$, as well as the real soft-breaking mass terms $m_{1,2}^2$ and $m_S^2$ for the Higgs fields, and $m^2_{\tilde{Q}}$, $m^2_{\tilde{U}}$, $m^2_{\tilde{D}}$, $m^2_{\tilde{L}}$ and $m^2_{\tilde{E}}$ for the sfermions,

$$\begin{aligned} \mathcal {L}_{\text {soft}}= & {} - \,m_1^2\,|\mathcal {H}_{1}|^2 - m_2^2\,|\mathcal {H}_{2}|^2 - m_S^2\,|\mathcal {S}|^2\nonumber \\&-\, \left[ \lambda \, A_\lambda \, \mathcal {S} \left( \mathcal {H}_2 \cdot \mathcal {H}_1\right) + \tfrac{1}{3}\,\kappa \, A_\kappa \, \mathcal {S}^3 + \text {h.c.} \right] \nonumber \\&-\, m_{\tilde{Q}}^2\,|\tilde{Q}|^2 - m_{\tilde{U}}^2\,|\tilde{u}|^2 - m_{\tilde{D}}^2\,|\tilde{d}|^2\nonumber \\&-\, m_{\tilde{L}}^2\,|\tilde{L}|^2 - m_{\tilde{E}}^2\,|\tilde{e}|^2\nonumber \\&-\, \left[ -Y_u\, A_u\, \tilde{u} \left( \mathcal {H}_2 \cdot \tilde{Q}\right) + Y_d\, A_d\, \tilde{d} \left( \mathcal {H}_1 \cdot \tilde{Q}\right) \right. \nonumber \\&+\,\left. Y_e\, A_e\, \tilde{e} \left( \mathcal {H}_1 \cdot \tilde{L}\right) + \text {h.c.} \right] . \end{aligned}$$

(2.4)

Expanding the Higgs potential in terms of the charged Higgs fields $(\phi _1^+,\phi _2^+) = (\phi _1^-,\phi _2^-)^*$, and neutral Higgs fields $(\phi ,\chi )=(\phi _1, \phi _2, \phi _s, \chi _1, \chi _2, \chi _s)$ yields

$$\begin{aligned} V_{\text {H}}= & {} -\mathbf {T}\left( \phi ,\chi \right) ^{\text {T}} + \frac{1}{2}\left( \phi ,\chi \right) \mathbf {M}^2\left( \phi ,\chi \right) ^{\text {T}}\nonumber \\&+ \begin{pmatrix} \phi ^-_1,\phi ^-_2 \end{pmatrix} \mathbf {M}_{\phi ^\pm }^2 \begin{pmatrix} \phi ^+_1 \\ \phi ^+_2 \end{pmatrix} + \cdots \end{aligned}$$

(2.5)

Here $\mathbf {T} = (T_{\phi _1}, T_{\phi _2}, T_{\phi _s}, T_{\chi _1}, T_{\chi _2}, T_{\chi _s})$ denotes the tadpole coefficients of the neutral Higgs fields, and $\mathbf {M}^2$ and $\mathbf {M}_{\phi ^\pm }^2$ denote the mass matrices of the neutral and charged Higgs bosons, respectively.

Since $\mathbf {M}^2$ and $\mathbf {M}^2_{\phi ^\pm }$ are symmetric and hermitian matrices, respectively, we diagonalize them by an orthogonal $(6\times 6)$ matrix $\mathbf {U}_{n}$ and a unitary $(2\times 2)$ matrix $\mathbf {U}_{c}$, respectively,

$$\begin{aligned}&\mathbf {D}_{hG} = \text {diag}{\left( m_{h_1}^2,\,m_{h_2}^2,\,m_{h_3}^2,\,m_{h_4}^2,\,m_{h_5}^2,\,0\right) } = \mathbf {U}_{n}\,\mathbf {M}^2\,\mathbf {U}_{n}^{\text {T}}\,,\end{aligned}$$

(2.6a)

$$\begin{aligned}&\mathbf {D}_{h^\pm G^\pm } = \text {diag}{\left( M_{H^\pm }^2,\,0\right) } = \mathbf {U}_{c}\,\mathbf {M}^2_{\phi ^\pm }\,\mathbf {U}_{c}^\dagger . \end{aligned}$$

(2.6b)

These transformations define the five neutral Higgs boson mass eigenstates, $h_i$, ($i=1,\ldots ,5$), and the (would-be) Goldstone boson G, as well as the charged Higgs and (would-be) Goldstone states, $H^\pm$ and $G^\pm$, at the tree level,

$$\begin{aligned}&\left( h, G\right) ^{\text {T}} \equiv \left( h_1, h_2, h_3, h_4, h_5, G\right) ^{\text {T}} = \mathbf {U}_{n} \left( \phi ,\chi \right) ^{\text {T}}, \end{aligned}$$

(2.7a)

$$\begin{aligned}&\left( H^\pm , G^\pm \right) ^{\text {T}} = \mathbf {U}_{c} \left( \phi _1^\pm , \phi _2^\pm \right) ^{\text {T}}. \end{aligned}$$

(2.7b)

It is convenient to decompose $\mathbf {U}_{n}$ into two matrices $\mathbf {U}_{n}^G$ and $\mathbf {U}_{n}^5$, where $\mathbf {U}_{n}^G$ singularizes out the neutral Goldstone boson,

$$\begin{aligned} \mathbf {U}_{n} \left( \phi ,\chi \right) ^{\text {T}}&= \mathbf {U}_{n}^5 \mathbf {U}_{n}^G \left( \phi ,\chi \right) ^{\text {T}} = \mathbf {U}_{n}^5 \left( \phi _1, \phi _2, \phi _s, A, \chi _s, G\right) ^{\text {T}} \nonumber \\&= \left( h, G\right) ^{\text {T}}. \end{aligned}$$

(2.8)

In the $\mathcal{CP}$-violating NMSSM the five fields $h_i$ are in general superpositions of the $\mathcal{CP}$-even and -odd components $\phi _i$ and $\chi _j$. In the special case of

$$\begin{aligned} \sin {\left( \xi - 2\,\xi _s + \phi _\lambda - \phi _\kappa \right) }&= 0 \end{aligned}$$

(2.9)

$\mathcal{CP}$-conservation is restored in the Higgs sector at the tree level, and the neutral mass matrix $\mathbf {M}^2$ becomes block-diagonal with two $(3\times 3)$ sub-matrices for the $\mathcal{CP}$-even and -odd entries.

The five linearly independent tadpole coefficients are related to soft-breaking terms and combinations of phases as

$$\begin{aligned} m_1^2&= -|\mu _{\text {eff}}|^2 - \frac{1}{2}\,M_Z^2\,\cos {\left( 2\beta \right) } - \left( |\lambda |\,v\,\sin {\beta }\right) ^2 -\, \frac{T_{\phi _1}}{\sqrt{2}\,v\,\cos {\beta }}+\, |\mu _{\text {eff}}|\,\tan {\beta }\left( |A_\lambda |\,\cos {\zeta _2} + \frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\cos {\zeta _1}\right) , \end{aligned}$$

(2.10a)

$$\begin{aligned} m_2^2&= -|\mu _{\text {eff}}|^2 + \frac{1}{2}\,M_Z^2\,\cos {\left( 2\beta \right) } - \left( |\lambda |\,v\,\cos {\beta }\right) ^2-\, \frac{T_{\phi _2}}{\sqrt{2}\,v\,\sin {\beta }}+\, \frac{|\mu _{\text {eff}}|}{\tan {\beta }}\left( |A_\lambda |\,\cos {\zeta _2} + \frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\cos {\zeta _1}\right) , \end{aligned}$$

(2.10b)

$$\begin{aligned} m_s^2&= -|\lambda |^2\,v^2 - \frac{T_{\phi _s}\,|\lambda |}{\sqrt{2}\,|\mu _{\text {eff}}|} - \frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |} \left( |A_\kappa |\,\cos {\zeta _3} + 2 \frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\right) + \frac{1}{2}\frac{|\lambda |^2\,v^2}{|\mu _{\text {eff}}|}\sin {\left( 2\beta \right) }\left( |A_\lambda |\,\cos {\zeta _2} + 2\frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\cos {\zeta _1}\right) \end{aligned}$$

(2.10c)

$$\begin{aligned} \sin {\zeta _2}&= \frac{1}{|A_\lambda |}\left( -\frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\sin {\zeta _1} - \frac{T_{\chi _1}}{\sqrt{2}\,|\mu _{\text {eff}}|\,v\,\sin {\beta }}\right) \,, \end{aligned}$$

(2.10d)

$$\begin{aligned} \sin {\zeta _3}&= \frac{|\lambda |^2}{|\kappa |\,|\mu _{\text {eff}}|^2\,|A_\kappa |}\left( \frac{|\lambda |\,v^2}{2}\sin {\left( 2\beta \right) }\left( |A_\lambda |\,\sin {\zeta _2} - 2\frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\sin {\zeta _1}\right) + \frac{T_{\chi _s}}{\sqrt{2}} \right) \,, \end{aligned}$$

(2.10e)

where the masses of the W and Z bosons are denoted by $M_W$ and $M_Z$, respectively, and the phases combine to $\zeta _1 = \xi - 2\,\xi _s + \phi _\lambda - \phi _\kappa$, $\zeta _2 = \xi + \xi _s + \phi _{A_\lambda } + \phi _\lambda$ and $\zeta _3 = 3\,\xi _s + \phi _{A_\kappa } + \phi _\kappa$. The expressions of Eq. (2.10) make plain that the tadpole coefficients can substitute the five parameters $m_1^2$, $m_2^2$, $m_S^2$, $\phi _{A_\lambda }$ and $\phi _{A_\kappa }$, so that the latter will not be regarded as free parameters in the following. Finally, the tadpole coefficients in the (tree-level) mass basis, $\mathbf {T}_{h} = (T_{h_1},T_{h_2},T_{h_3},T_{h_4},T_{h_5},0)$, where the zero denotes the vanishing tadpole coefficient of the Goldstone mode, are obtained by $\mathbf {T}_h = \mathbf {U}_{n} \mathbf {T}$. The minimization of $V_{\text {H}}$ at the chosen Higgs VEVs is guaranteed through the condition that all tadpole coefficients $\mathbf {T}$ vanish at the tree level.

The trilinear parameter $\left| A_\lambda \right|$ can be expressed in terms of the charged Higgs mass $M_{H^\pm }$ as

$$\begin{aligned} |A_\lambda |\,\cos {\zeta _2}&= -\frac{|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\cos {\zeta _1} \nonumber \\&\quad + \left( M_{H^\pm }^2 - M_W^2 +\,|\lambda |^2\,v^2\right) \frac{\sin {\left( 2\beta \right) }}{2\,|\mu _{\text {eff}}|}. \end{aligned}$$

(2.11)

Our renormalization scheme will also involve the fermionic superpartners of the Higgs bosons, known as the higgsinos. We thus introduce here the Dirac spinors $\tilde{H}^{\pm }$ of the charged higgsino fields, as well as the Majorana spinor of the singlino $\tilde{S}$. In turn, these higgsino gauge eigenstates mix with the gauginos to form the mass states known as the neutralinos and charginos—see e.g. Eq. (11) in Ref. [113], where the NMSSM parameters $\lambda$, $\kappa$, $M_{1,2}$ and $\mu _{\text {eff}}$ should be promoted to complex values. Yet these mass states will play no role in the discussion below.

2.2 Renormalization of the Higgs potential

In the past, radiative corrections to the Higgs masses of the $\mathcal{CP}$-conserving NMSSM have been considered in the effective potential approach, see e.g. Refs. [116,117,118,119,120,121,122,123,124]. This topic has also been analyzed from the perspective of a diagrammatic expansion, including radiative corrections from part or the full set of the particle content of the NMSSM: see Refs. [113, 125, 126]. Both procedures have also been employed for the $\mathcal{CP}$-violating case: contributions to the effective potential have been discussed in Refs. [17, 67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82, 102], while contributions using the diagrammatic approach have been presented in Refs. [88, 115].

In the present work, the radiative corrections to the Higgs sector are calculated in the diagrammatic approach. To this end, we first establish a list of the independent parameters appearing in the linear and bilinear terms of the Higgs potential in Eq. (2.5):

$$\begin{aligned}&T_{h_{1,\ldots ,5}}, \ M_{H^\pm }^2, \ M_W^2, \ M_Z^2, \ e, \ \tan {\beta }, \ |\mu _{\text {eff}}|, \ |\lambda |, \ \phi _{\lambda }, \ |\kappa |, \ \phi _{\kappa }, \ \nonumber \\&\quad |A_\kappa |, \ \xi , \ \xi _s, \end{aligned}$$

(2.12)

where the electromagnetic coupling e is related to the fine structure constant $\alpha$ by $e=\sqrt{4\,\pi \,\alpha }$. Compared to Ref. [113], we use e as an independent parameter instead of the VEV v. This choice allows one to renormalize e to its value derived from the Fermi constant $G_F$ and does not require the reparametrization procedure employed in Ref. [113]. The difference between the renormalization employed here, and the renormalization and reparametrization procedure described in Ref. [113], is a sub-leading effect of two-loop order, however. Other proposals in the literature consist in fixing e from $\alpha \left( M_Z\right)$ [88, 115].

To all real and complex independent parameters, $g_r$ and $g_c$, respectively, that are given in Eq. (2.12), we apply the renormalization transformations

$$\begin{aligned} g_r \rightarrow g_r\left( 1 + \delta Z_r\right)= & {} g_r + \delta {g_r}\ , g_c \rightarrow g_c\left( 1 + \delta Z_c\right) \nonumber \\= & {} g_c + \delta {g_c} = g_c + \delta {\left| g_c\right| }\,\mathrm {e}^{{i}\,\phi _c} + {i}\,g_c\,\delta {\phi _c}.\nonumber \\ \end{aligned}$$

(2.13)

The renormalization transformations for the Higgs, singlino and charged higgsino fields read

$$\begin{aligned} \mathcal {H}_{1, 2}&\rightarrow \left( 1 + \tfrac{1}{2}\,\delta Z_{\mathcal {H}_{1,2}}\right) \mathcal {H}_{1,2}, ~\mathcal {S} \rightarrow \left( 1 + \tfrac{1}{2}\,\delta Z_{\mathcal {S}}\right) \mathcal {S},\end{aligned}$$

(2.14a)

$$\begin{aligned} \tilde{H}^\pm&\rightarrow \left( 1 + \tfrac{1}{2}\,\delta {Z_{\tilde{H}^\pm }^{\text {L}}}\,P_{\text {L}}+ \tfrac{1}{2}\,\delta {Z_{\tilde{H}^\pm }^{\text {R}}}\,P_{\text {R}}\right) \tilde{H}^\pm ,\nonumber \\ \tilde{S}&\rightarrow \left( 1 + \tfrac{1}{2}\,\delta {Z_{\tilde{S}}^{\text {L}}}\,P_{\text {L}}+\tfrac{1}{2}\,\delta {Z_{\tilde{S}}^{\text {R}}}\,P_{\text {R}}\right) \tilde{S}, \end{aligned}$$

(2.14b)

with $P_{\text {L}}$ and $P_{\text {R}}$ denoting the left- and right-handed projectors, respectively. Since the singlino is a Majorana field, the corresponding wave-function counterterms $\delta {Z_{\tilde{S}}^{\text {L}}}$ and $\delta {Z_{\tilde{S}}^{\text {R}}}$ are complex conjugates of one another.

2.3 Renormalization conditions at the one-loop order

For the parameters $M_{H^\pm }$, $M_W$, $M_Z$ and $\tan {\beta }$, which enter the one-loop calculation of the Higgs masses in the MSSM as well, we follow the renormalization prescription outlined in Ref. [109]: the on-shell renormalization scheme is employed for the gauge boson masses, $M_Z$ and $M_W$, and the charged Higgs mass $M_{H^\pm }$, while the parameter $\tan {\beta }$ is renormalized $\overline{\mathrm {DR}}$.

We apply the minimization conditions in order to fix the tadpole counterterms:

$$\begin{aligned} T^{(1)}_{h_i} + \delta {T_{h_i}}&= 0, \end{aligned}$$

(2.15)

where the $T^{(1)}_{h_i}$ correspond to the one-loop contributions to the tadpole parameters.

The counterterm of the electromagnetic coupling e is fixed by

$$\begin{aligned}&\delta {Z_e} = \delta {Z_e^{\text {Th}}} - \frac{1}{2}\,\Delta {r}^{\text {NMSSM}},\end{aligned}$$

(2.16a)

$$\begin{aligned}&\delta Z_e^{\text {Th}} = \frac{1}{2}\,\Pi ^{\gamma \gamma }(0) + \frac{s_\mathrm {w}}{c_\mathrm {w}} \frac{\Sigma _T^{\gamma Z}(0)}{M_Z^2}. \end{aligned}$$

(2.16b)

Here $\delta Z_e^{\text {Th}}$ is the counterterm of the charge renormalization within the NMSSM according to the static (Thomson) limit. The quantities $\Pi ^{\gamma \gamma }(0)$ and $\Sigma _T^{\gamma Z}(0)$ are, respectively, the derivative of the transverse part of the photon self-energy and the transverse part of the photon–Z self-energy at zero momentum transfer. For the quantity $\Delta r^{\text {NMSSM}}$, relating the elementary charge to the Fermi constant $G_F$ measured in muon decays, we use the result of Ref. [127] (see also Ref. [128]). The numerical value for the electromagnetic coupling e in this parametrization is obtained from the Fermi constant in the usual way as $e = 2\,M_W\,s_\mathrm {w}\,\sqrt{\sqrt{2}\,G_F}$. This choice differs from previous works, where either the charge renormalization condition was determined in terms of $\alpha (M_Z)$ [115], or instead v was renormalized $\overline{\mathrm {DR}}$ and the result was subsequently reparametrized to use the value of e derived from the Fermi constant [113].

The remaining independent parameters and the field-renormalization constants are renormalized $\overline{\mathrm {DR}}$. We present a detailed description of the $\overline{\mathrm {DR}}$ renormalization conditions that we apply. The actual cancellation of UV-divergences, that we recover at the diagrammatic level, represents a non-trivial check for the validity of the FeynArts model file employed for our calculation.

The $\overline{\mathrm {DR}}$ field renormalization constants for the Higgs fields are obtained:

$$\begin{aligned} \delta Z_{\mathcal {H}_1}&= -\mathfrak {R}\mathfrak {e}{\left[ \frac{\mathrm{{d}}\Sigma _{\phi _1\phi _1}^{(1)}}{\mathrm{{d}}p^2}\right] }_{\text {div}} , \delta Z_{\mathcal {H}_2} = -\mathfrak {R}\mathfrak {e}{\left[ \frac{\mathrm{{d}}\Sigma _{\phi _2\phi _2}^{(1)}}{\mathrm{{d}}p^2}\right] }_{\text {div}} ,\nonumber \\ \delta Z_{\mathcal {S}}&= -\mathfrak {R}\mathfrak {e}{\left[ \frac{\mathrm{{d}}\Sigma _{\phi _s\phi _s}^{(1)}}{\mathrm{{d}}p^2}\right] }_{\text {div}}, \end{aligned}$$

(2.17)

where $\Sigma _{ii}^{(1)}$ denotes the self-energy of field i at the one-loop order, and the subscript ’div’ denotes the UV-divergent piece (along with the universal finite pieces that are associated in the $\overline{\mathrm {DR}}$ scheme) of the quantity that it follows. The result does not depend on the momentum $p^2$.

The field-renormalization constants for the charged higgsino and the singlino fields are defined by the following conditions (the momentum $p^2$ again does not matter):

$$\begin{aligned} \delta Z_{\tilde{H}^\pm }^{\text {L}}&= -\left. \Sigma _{\tilde{H}^\pm \tilde{H}^\pm }^{\text {vec L}\,(1)}\right| _{\text {div}}\,, \delta Z_{\tilde{H}^\pm }^{\text {R}} = -\left. \Sigma _{\tilde{H}^\pm \tilde{H}^\pm }^{\text {vec R}\,(1)}\right| _{\text {div}}\,,\nonumber \\ \delta Z_{\tilde{S}}^{\text {L}}&= -\left. \Sigma _{\tilde{S}\tilde{S}}^{\text {vec L}\,(1)}\right| _{\text {div}}\,, \end{aligned}$$

(2.18)

where the self-energies of the fermion fields are decomposed into the left and right vector and the scalar contributions,

$$\begin{aligned} \Sigma _{ff}^{(1)}{(p^2)}&= \Sigma _{ff}^{\text {scal}\,(1)}{(p^2)} \nonumber \\&\quad + p_\mu \gamma ^\mu \left[ P_{\text {L}}\,\Sigma _{ff}^{\text {vec L}\,(1)}{(p^2)} {+} P_{\text {R}}\,\Sigma _{ff}^{\text {vec R}\,(1)}{(p^2)}\right] . \end{aligned}$$

(2.19)

The renormalization constants $\delta {|\lambda |}$, $\delta {|\kappa |}$, $\delta {\phi _{\lambda }}$, $\delta {\phi _{\kappa }}$, $\delta \xi$, $\delta \xi _s$ and $\delta {|A_\kappa |}$ are fixed by $\overline{\mathrm {DR}}$ conditions imposed on trilinear vertices involving scalar, $\mathcal{CP}$-even Higgs fields $\phi _{1,2,s}$, the singlino $\tilde{S}$, and the charged higgsino fields $\tilde{H}^\pm$, in the interaction basis in analogy to the procedure outlined in [129]. The renormalization condition imposed on the renormalized three-point function $\hat{\Gamma }_{ijk}$ for three arbitrary fields i, j and k reads

$$\begin{aligned} \hat{\Gamma }_{ijk} = \Gamma ^{(0)}_{ijk} + \Gamma ^{(1)}_{ijk} + \delta {\Gamma }_{ijk} \overset{!}{=} \text {finite}\, ,\Leftarrow \delta {\Gamma }_{ijk} {=} -\left. \Gamma ^{(1)}_{ijk}\right| _{\text {div}}\,, \end{aligned}$$

(2.20)

where $\Gamma _{ijk}^{(0)}$ and $\Gamma _{ijk}^{(1)}$ denote the vertex function at the tree-level and one-loop order, respectively, and $\delta {\Gamma _{ijk}}$ denotes the counterterm. The counterterm $\delta {\Gamma _{ijk}}$ is thus fixed by the divergent part of the vertex function. The renormalization constants of the independent parameters are subsequently fixed by linear relations to $\delta {\Gamma _{ijk}}$.

• For $\delta {\left| \lambda \right| }$ and $\delta {\phi _{\lambda }}$ we impose the $\overline{\mathrm {DR}}$ renormalization condition of Eq. (2.20) on the vertices ${\Gamma _{\tilde{S}\tilde{H}^-\phi _1^+}^{(0)} = \lambda }$ and ${\Gamma _{\tilde{S}\tilde{H}^+\phi _1^-}^{(0)} = \lambda ^*}$, which yields

  $$\begin{aligned} \frac{\delta \left| \lambda \right| }{\left| \lambda \right| }&= -\frac{1}{2}\left\{ \left. \frac{\Gamma _{\tilde{S}\tilde{H}^-\phi _1^+}^{(1)} }{ \Gamma _{\tilde{S}\tilde{H}^-\phi _1^+}^{(0)}}\right| _{\text {div}} + \frac{1}{2}\left( \delta Z_{\tilde{S}} + \delta Z_{\tilde{H}^\pm }^{\text {R}} + \delta Z_{\mathcal {H}_1} \right) \right\} \nonumber \\&\quad + \text {c.c.}\,, \end{aligned}$$

  (2.21a)
  $$\begin{aligned} \delta \phi _\lambda&= -\frac{1}{2{i}}\left\{ \left. \frac{\Gamma _{\tilde{S}\tilde{H}^-\phi _1^+}^{(1)} }{ \Gamma _{\tilde{S}\tilde{H}^-\phi _1^+}^{(0)}}\right| _{\text {div}} + \frac{1}{2}\left( \delta Z_{\tilde{S}} + \delta Z_{\tilde{H}^\pm }^{\text {R}} + \delta Z_{\mathcal {H}_1} \right) \right\} \nonumber \\&\quad + \text {c.c.} \end{aligned}$$

  (2.21b)

• For $\delta \xi$ we impose the renormalization condition of Eq. (2.20) on the vertices ${\Gamma _{\tilde{S}\tilde{H}^+\phi _2^-}^{(0)} = \lambda \,\mathrm {e}^{{i}\,\xi }}$ and ${\Gamma _{\tilde{S}\tilde{H}^-\phi _2^+}^{(0)} = \lambda ^*\,\mathrm {e}^{-{i}\,\xi }}$. The counterterm reads

  

  (2.22)

• We fix the renormalization constant $\delta \xi _s$ for the phase $\xi _s$ by applying Eq. (2.20) on the vertices $\Gamma _{\tilde{H}^-, \tilde{H}^+\phi _s}^{(0)} = \lambda \,\mathrm {e}^{{i}\,\xi _s}/\sqrt{2}$ and ${\Gamma _{\tilde{H}^+, \tilde{H}^-\phi _s}^{(0)} = \lambda ^*\,\mathrm {e}^{-{i}\, \xi _s}/\sqrt{2}}$, which yields

  $$\begin{aligned} \delta \xi _s&= \bigg \{-\frac{1}{2{i}}\bigg [ \frac{\Gamma _{\tilde{H}^-, \tilde{H}^+\phi _s}^{(1)} }{ \Gamma _{\tilde{H}^-, \tilde{H}^+\phi _s}^{(0)}}\bigg |_{\text {div}} + \frac{1}{2}\bigg ( \delta Z_{\tilde{H}^\pm }^{\text {L}}\nonumber \\&\quad + \delta Z_{\tilde{H}^\pm }^{\text {R}} + \delta Z_{\phi _s} \bigg )\bigg ] + \text {c.c.} \bigg \} - \delta \phi _\lambda \,. \end{aligned}$$

  (2.23)

• The absolute value and phase of $\kappa$ are renormalized by $\delta {\left| \kappa \right| }$ and $\delta {\phi _{\kappa }}$. We fix both renormalization constants by applying Eq. (2.20) on the vertex ${\Gamma _{\tilde{S}\tilde{S}\phi _s}^{(0)} = \sqrt{2}\,\kappa \,\mathrm {e}^{{i}\,\xi _s}}$, which yields

  $$\begin{aligned} \frac{\delta \left| \kappa \right| }{\left| \kappa \right| }&= -\frac{1}{2}\left\{ \left. \frac{\Gamma _{\tilde{S}\tilde{S}\phi _s}^{(1)} }{ \Gamma _{\tilde{S}\tilde{S}\phi _s}^{(0)}}\right| _{\text {div}} + \frac{1}{2}\left( 2\delta Z_{\tilde{S}} + \delta Z_{\phi _s} \right) \right\} + \text {c.c.}\,, \end{aligned}$$

  (2.24a)
  $$\begin{aligned} \delta \phi _\kappa&= \left\{ -\frac{1}{2{i}}\left[ \left. \frac{\Gamma _{\tilde{S}\tilde{S}\phi _s}^{(1)} }{ \Gamma _{\tilde{S}\tilde{S}\phi _s}^{(0)}}\right| _{\text {div}} + \frac{1}{2}\left( 2\delta Z_{\tilde{S}} + \delta Z_{\phi _s} \right) \right] + \text {c.c.} \right\} - \delta \xi _s. \end{aligned}$$

  (2.24b)

• The parameter $|\mu _{\text {eff}}|$ could be renormalized in the on-shell scheme for one of the charginos or neutralinos [130,131,132]. However, such schemes cannot be stabilized over the whole parameter space (due to mass-crossings). We thus prefer to apply the $\overline{\mathrm {DR}}$ condition

  $$\begin{aligned} \delta |\mu _{\text {eff}}|&= |\mu _{\text {eff}}|\left( \frac{\delta |\lambda |}{|\lambda |}+\frac{1}{2}\delta Z_S\right) \,. \end{aligned}$$

  (2.25)

• In order to fix $\delta {\left| A_{\kappa }\right| }$, we impose the renormalization condition of Eq. (2.20) on the vertex $\Gamma _{\phi _s\phi _s\phi _s}^{(0)} = -\sqrt{2}\,|\kappa |\left( \frac{6\,|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |} + |A_\kappa |\cos {\zeta _3}\right)$, [$\zeta _3$ was defined after Eq. (2.10)]. It reads

  $$\begin{aligned}&\delta |A_\kappa |= -|A_\kappa |\left( \frac{\delta {|\kappa |}}{|\kappa |} + \frac{\delta {\cos {\zeta _3}}}{\cos {\zeta _3}}\right) - \frac{6\,|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |\,\cos {\zeta _3}} \left( \frac{\delta |\mu _{\text {eff}}|}{|\mu _{\text {eff}}|} - \frac{\delta |\lambda |}{|\lambda |} + \frac{2\,\delta |\kappa |}{|\kappa |}\right) - \frac{\delta \Gamma _{\phi _s\phi _s\phi _s}}{\sqrt{2}\,|\kappa |\cos {\zeta _3}},\end{aligned}$$

  (2.26a)
  $$\begin{aligned}&\delta \zeta _3 = \left( \frac{\delta |\kappa |}{|\kappa |} + \frac{2\,\delta |\lambda |}{|\lambda |} - \frac{\delta |\mu _{\text {eff}}|}{\mu _{\text {eff}}} +\frac{\delta \sin {\left( 2\beta \right) }}{\sin {\left( 2\beta \right) }} {+} \frac{\delta \sin {\zeta _1}}{\sin {\zeta _1}} {+} \left. \frac{2\,\delta v}{v}\right| _{\text {div}}\right) \cos {\zeta _3}\sin {\zeta _3} {+} \left[ \frac{\delta \Gamma _{\phi _s\phi _s\phi _s}}{\sqrt{2}\,|\kappa |} {+} \frac{6\,|\kappa |\,|\mu _{\text {eff}}|}{|\lambda |}\left( \frac{\delta |\mu _{\text {eff}}|}{|\mu _{\text {eff}}|}\right. \right. \nonumber \\&\left. \left. - \frac{\delta |\lambda |}{|\lambda |} + \frac{2\,\delta |\kappa |}{|\kappa |}\right) \right] \frac{\sin {\zeta _3}}{|A_\kappa |} + \left( \left. \delta T_{\chi _s}\right| _{\text {div}} - \frac{|\lambda |\,v\,\cos {\beta }}{|\mu _{\text {eff}}|}\left. \delta T_{\chi _1}\right| _{\text {div}}\right) \frac{|\lambda |^2\,\cos {\zeta _3}}{ \sqrt{2}\,|\kappa |\,|A_\kappa |\,|\mu _{\text {eff}}|^2}, \end{aligned}$$

  (2.26b)

  where we used the one-loop relation $\delta {\cos {\zeta _3}} = -\sin {\zeta _3}\delta {\zeta _3}$, and $\delta v$ is not an independent counterterm, but a quantity depending on the counterterms to the electroweak parameters

  $$\begin{aligned} \delta v= & {} v \left( \frac{\delta M_W}{M_W} + \frac{\delta s_{\text {w}}}{s_{\text {w}}} - \delta Z_e\right) \,,\nonumber \\ \delta s_{\text {w}}= & {} \frac{c_{\text {w}}^2}{s_{\text {w}}}\left( \frac{\delta M_Z}{M_Z} - \frac{\delta M_W}{M_W}\right) . \end{aligned}$$

  (2.27)

We performed various consistency checks of our model file at the one-loop order:

• all the renormalized Higgs self-energies are UV-finite, for arbitrary values of the momentum,

• all the vertex-diagram amplitudes of a Higgs state decaying to SM-particles or a pair of charginos/neutralinos are UV-finite,

• the UV-divergences of the counterterms to gauge couplings, superpotential parameters or soft terms are consistent with the corresponding one-loop beta functions (see e.g. Refs. [6, 133]),

• in the $\mathcal{CP}$-conserving limit, our parameters and couplings are identical to the findings of a previously developed model file [113],

• in the MSSM limit, we have found agreement of the values of all our couplings with their counterparts in the complex MSSM, obtained with the model file of [134].

• we checked that $\phi _{\lambda }+\xi +\xi _s$ and $\phi _{\kappa }+3\,\xi _s$ were the only relevant combinations of the phases $\phi _{\lambda }$, $\phi _{\kappa }$, $\xi$ and $\xi _s$ at the level of amplitudes,

• finally, we checked explicitly, that the counterterms $\delta {\phi _\lambda }$, $\delta {\phi _\kappa }$, $\delta \xi$ and $\delta \xi _s$ vanish when all NMSSM contributions are included, as pointed out in Ref. [115]. This can also be placed in the perspective of the $\beta$-functions [133, 135,136,137,138,139]: the phases from the superpotential parameters have no scale-dependence (at least up to two-loop order); since $\xi$ and $\xi _s$ are spurious degrees of freedom, we could expect their counterterms to present the same vanishing behaviors as $\delta {\phi _\lambda }$ and $\delta {\phi _\kappa }$.

2.4 Quark Yukawa couplings

The Yukawa couplings of the top and bottom quarks, $Y_t$ and $Y_b$, have a sizable impact on radiative corrections to the Higgs masses. We present our prescriptions in this subsection.

The top Yukawa coupling $Y_t=\sqrt{2\sqrt{2}\,G_F}\,m_t/\sin \beta$ is defined by the on-shell top mass $m_t$.

For the bottom quark, we employ the running $\overline{\mathrm {DR}}$ bottom-mass of the SM (containing one-loop QCD corrections), $\overline{m}_b$, at the scale $m_t$ [140]. Additionally, we subtract the possibly large $\tan \beta$-enhanced one-loop contributions to $\overline{m}_b$—induced by gaugino–squark and higgsino–squark loops—from the numerical definition of $Y_b$ at the tree level: $Y_b=\sqrt{2\sqrt{2}\,G_F}\,\overline{m}_b/[\cos \beta \,|1+\Delta _b |]$, where $\Delta _b$ is discussed in e.g. Refs. [98, 140,141,142,143,144,145,146].

2.5 Higgs masses at higher orders

The masses of the Higgs bosons are obtained from the complex poles of the full propagator matrix. After rotating out the Goldstone mode¹ the inverse propagator matrix for the five Higgs fields $h_i$ is a $(5 \times 5)$ matrix that reads

$$\begin{aligned} \hat{\varvec{\Delta }}_{hh}^{-1}{(k^2)} = i \left[ k^2\mathbf {1} - \mathbf {D}_{hh} + \hat{\varvec{\Sigma }}_{hh}{(k^2)} \right] . \end{aligned}$$

(2.28)

Here $\mathbf {D}_{hh} = \text {diag}\{m^2_{h_1},\,m^2_{h_2},\,m^2_{h_3},\,m^2_{h_4},\,m^2_{h_5}\}$ denotes the diagonalized mass matrix of the Higgs fields without the Goldstone at the tree level, and $\hat{\varvec{\Sigma }}_{hh}$ denotes the matrix of the renormalized self-energy corrections of the neutral Higgs fields.

The five complex poles of the propagator are given by the values of the squared external momentum $k^2$ for which the determinant of the inverse propagator matrix vanishes,

$$\begin{aligned}&\det {\left[ \hat{\varvec{\Delta }}^{-1}_{hh}{(k^2)} \right] _{k^2 = \mathcal {M}^2_i}} \overset{!}{=} 0,\quad \mathcal {M}_i^2 \overset{!}{=} M_{h_i}^2 + {i}\,\Gamma _{h_i}\,M_{h_i}\,,\nonumber \\&i \in \{1,\ldots ,5\}, \end{aligned}$$

(2.29)

where we have explicitly stated the connection between the pole $\mathcal {M}_i^2$, the Higgs mass $M_{h_i}$ and the total width $\Gamma _{h_i}$ for each Higgs field $h_i$.

In order to account for the imaginary parts of the poles of the propagator matrix, we perform an expansion of the self-energies in terms of the imaginary part of the momentum, which is assumed to be small (also see section 4.3.5 of [147]),

$$\begin{aligned} \hat{\varvec{\Sigma }}_{hh}{(k^2)} \approx \hat{\varvec{\Sigma }}_{hh}{\left( \mathfrak {R}\mathfrak {e}{\left[ k^2\right] }\right) } + {i}\,\mathfrak {I}\mathfrak {m}{\left[ k^2\right] }\,\frac{\mathrm {d}}{{\mathrm {d}}k^2}\hat{\varvec{\Sigma }}_{hh}{\left( \mathfrak {R}\mathfrak {e}{\left[ k^2\right] }\right) } \end{aligned}$$

(2.30)

In this work, the renormalized self-energy $\hat{\varvec{\Sigma }}_{hh}$,

$$\begin{aligned} \hat{\varvec{\Sigma }}_{hh}{(k^2)} \approx \left. \hat{\varvec{\Sigma }}^{(\text {1L})}_{hh}{(k^2)} \right| ^{\text {NMSSM}} + \left. \hat{\varvec{\Sigma }}^{(\text {2L})}_{hh}{(k^2)} \right| _{k^2\;=\;0}^{\text {MSSM}}. \end{aligned}$$

(2.31)

is evaluated by taking into account the full contributions from the $\mathcal{CP}$-violating NMSSM at one-loop order and, as an approximation, the MSSM-like contributions at two-loop order of $\mathcal {O}{(\alpha _t\alpha _s)}$ [148] and $\mathcal {O}{(\alpha _{t}^2)}$ [149, 150] at vanishing external momentum as implemented in FeynHiggs.²

We note that the two-loop $O(\alpha _b\alpha _s)$ contributions to the Higgs self-energies are not included in our calculation. Still, as we employ the running bottom mass in the definition of $Y_b$ entering $\left. \hat{\varvec{\Sigma }}^{(\text {1L})}_{hh}{(k^2)} \right| ^{\text {NMSSM}}$, we expect that the missing two-loop piece is numerically sub-leading [151,152,153].

2.6 Wave-function normalization factors: the matrix $\mathbf {Z}^{\mathrm{mix}}$

In the Feynman-diagrammatic approach physical processes with external Higgs fields are defined in terms of the tree-level mass states $h_i$. When higher-order contributions are considered, however, the tree-level mass states are not physical states. Indeed, radiative corrections induce additional mass and kinematic mixing among the fields $h_i$, and the poles of the tree-level propagators do not coincide with $\mathcal {M}_i^2$. A relation between the amplitudes with an external tree-level Higgs mass state and those with an external physical Higgs state is necessary (though this relation is trivial if the fields are renormalized on-shell). For example, for a Higgs decaying into two fermions f this relation is given by the LSZ reduction formula,

$$\begin{aligned} \mathcal{A}[h_i^{\mathrm{phys}}\rightarrow f\bar{f}]=Z^{\mathrm{mix}}_{ij}\,\mathcal{A}[h_j\rightarrow f\bar{f}]. \end{aligned}$$

(2.32)

Here the superscript ‘phys’ denotes the amplitude with an external physical field. The coefficients $Z^{\mathrm{mix}}_{ij}$ can be expressed explicitly in terms of the full propagator matrix (see Refs. [154,155,156,157] and also section 5.3 of Ref. [158]), as

$$\begin{aligned} Z^{\mathrm{mix}}_{ij} = \left. \left[ {i}\,\frac{\mathrm {d}}{{\mathrm {d}}k^2}\left( \hat{\varvec{\Delta }}_{hh}^{-1}(k^2)\right) _{ii} \right] ^{\frac{1}{2}} \frac{ \left( \hat{\varvec{\Delta }}_{hh}(k^2)\right) _{ij} }{ \left( \hat{\varvec{\Delta }}_{hh}(k^2)\right) _{ii}} \right| _{k^2\rightarrow \mathcal {M}^2_i}. \end{aligned}$$

(2.33)

However, the analytical inversion becomes time-consuming in the case of a $(5\times 5)$ propagator matrix. Additionally, $\hat{\varvec{\Delta }}_{hh}$ needs to be evaluated at (or close to) its singular points $\mathcal {M}_i^2$, which can lead to numerical instabilities on the right-hand side of Eq. (2.33) (only the ratio of propagator matrix elements $(\hat{\varvec{\Delta }}_{hh})_{ij}$ is finite). In order to avoid these issues we employed an equivalent formulation of the coefficients $Z^{\mathrm{mix}}_{ij}$, which is outlined below.

We consider the following effective Lagrangian for the tree-level mass states $h_i^{\mathrm{(tree)}}$:

$$\begin{aligned} \mathcal{L}_{\mathrm{eff}} = \frac{1}{2{i}}\left[ \hat{\varvec{\Delta }}^{-1}_{hh}{(k^2)}\right] _{ij}h_i^{\mathrm{(tree)}}h_j^{\mathrm{(tree)}} \end{aligned}$$

(2.34)

and set $\mathbf {Z}^{\mathrm{mix}} = (Z^{\mathrm{mix}}_{ij})$ as the transition matrix to the physical Higgs fields: $h_i^{\mathrm{(loop)}}=Z^{\mathrm{mix}}_{ij}h_j^{\mathrm{(tree)}}$. The states $h_i^{\mathrm{(loop)}}$ are defined such that

$$\begin{aligned} \mathcal {L}_{\mathrm{eff}} = \frac{1}{2}\left[ k^2 - \mathcal {M}_{h_i}^2\right] (h_i^{\mathrm{(loop)}})^2 + \mathcal {O}{\left( \left[ k^2 - \mathcal {M}_{h_i}^2\right] ^2\right) }\,. \end{aligned}$$

(2.35)

In other terms, the $h_i^{\mathrm{(loop)}}$ should appear as on-shell fields with standard kinetic terms close to their mass-pole. Thus the coefficients $Z^{\mathrm{mix}}_{ij}$ should satisfy the ‘eigenvalue’ conditions³

$$\begin{aligned} \left[ \mathbf {D}_{hG} - \hat{\varvec{\Sigma }}_{hG}{\left( \mathcal {M}_{h_i}^2\right) }\right] _{kl}Z^{\mathrm{mix}}_{il}&= \mathcal {M}_{h_i}^2Z^{\mathrm{mix}}_{ik}. \end{aligned}$$

(2.36)

Once the roots of $\det {\left[ \hat{\varvec{\Delta }}^{-1}_{hh}{\left( k^2\right) }\right] }$ are known, the ith line of $\mathbf {Z}^{\mathrm{mix}}$ is thus determined as the eigenvector of $\mathbf {D}_{hG} - \hat{\varvec{\Sigma }}_{hG}$ for the eigenvalue $\mathcal {M}_{h_i}^2$.

Finally, the normalization of the ith line of $\mathbf {Z}^{\mathrm{mix}}$ is specified by the following condition on the kinetic term:

$$\begin{aligned} \left[ \frac{{\mathrm {d}} \hat{\varvec{\Delta }}^{-1}_{hh}}{{\mathrm {d}}k^2} {(\mathcal {M}_i^2)} \right] _{kl} Z^{\mathrm{mix}}_{ik}\,Z^{\mathrm{mix}}_{il} = \left[ \mathbb {I} + \frac{{\mathrm {d}}\hat{\varvec{\Sigma }}_{hG}}{{ \mathrm {d}}k^2}{(\mathcal {M}_i^2)}\right] _{kl} Z^{\mathrm{mix}}_{ik}\,Z^{\mathrm{mix}}_{il} = 1, \end{aligned}$$

(2.37)

such that the coefficients $Z^{\mathrm{mix}}_{ij}$ are uniquely specified (up to a sign without physical meaning) by Eq. (2.35). For the study of effects from the normalization of $\mathbf {Z}^\mathrm{{mix}}$, it is convenient to define the (squared) norm $\left| Z_i\right| ^2$ of its rows,

$$\begin{aligned} |Z_i|^2 = \sum _{j\;=\;1}^{5}{\left| Z^{\mathrm{mix}}_{ij}\right| ^2}\,, \end{aligned}$$

(2.38)

i.e. $\left| Z_i\right|$ correspond to the norm of the eigenvectors, that are associated to the complex pole $\mathcal {M}_{h_i}$, see Eqs. (2.35) and (2.37).

The determination of $\mathbf {Z}^\mathrm{{mix}}$ in terms of the eigenstates of $\hat{\varvec{\Delta }}^{-1}_{hh}{(k^2)}$ is numerically easier to handle than its determination via Eq. (2.33). Applying the two defining conditions Eqs. (2.36) and (2.37) to the expression of Eq. (2.33), one can verify that both definitions of $\mathbf {Z}^\mathrm{{mix}}$ are identical.

As we discussed, the components of the matrix $\mathbf {Z}^\mathrm{{mix}}$ establish the connection between the physical fields and the tree-level external legs. In the literature this matrix $\mathbf {Z}^\mathrm{{mix}}$ is often replaced by simplified versions neglecting the momentum dependence of the self-energies. With the aim of performing numerical comparisons in the following section, we introduce two such approximate definitions of the relation between loop-corrected and tree-level fields:

• The first approach consists in freezing the momentum to $k^2=0$ in the self-energy of Eq. (2.28). This assumption is known as the effective potential approximation. In this approach the inverse propagator matrix $\mathbf {\Delta }_{hh}^{-1}{(k^2=0)}$, as given by Eq. (2.28), is diagonalized by a simple orthogonal matrix $\mathbf {U}^0$, which approximates $\mathbf {Z}^\mathrm{{mix}}$.

• Another choice consists in replacing the momentum dependence of the self-energy in Eq. (2.28) by $[\hat{\varvec{\Sigma }}_{hh}{(k^2)}]_{ij}\rightarrow [\hat{\varvec{\Sigma }}_{hh}{((m_{h_i}^2+m_{h_j}^2)/2)}]_{ij}$ (given in the basis of the tree-level mass states). This procedure aims at more closely mimicking the actual values of the self-energies involved in the mass calculation. In this approach the inverse propagator is also diagonalized by an orthogonal matrix $\mathbf {U}^m$.

While these procedures capture the mixing effects induced by radiative corrections, at least partially, it is nevertheless obvious that they miss the normalization of the fields outlined in Eq. (2.37). This means in particular that the norm as defined in Eq. (2.38) will always be identical to 1 if the matrix $\mathbf {Z}^\mathrm{{mix}}$ is approximated by either $\mathbf {U}^0$ or $\mathbf {U}^m$. We will discuss the impact of these approximations in the following section.

Fig. 1

Masses of the light, SM-like state $h_1$ (left plot) and the two heavy states $h_2$ and $h_3$ (right plot) as a function of $\phi _{A_t}$. The solid lines denote the masses obtained with our calculation, while the squares denote the masses obtained with FeynHiggs with the options indicated in the text. The scenario is representative of the MSSM limit of the NMSSM and we employ the following parameters: $\lambda =\kappa =10^{-5}$, $\tan \beta =10$, $m_{H^{\pm }}=500$ GeV, $\mu _{\text {eff}}=250$ GeV, $A_{\kappa }=-100$ GeV, $m_{\tilde{F}}=1.5$ TeV, $|A_t|=A_b=2.5$ TeV, $2\,M_1=M_2=M_3/5=0.5$ TeV

3 Numerical analysis

In this section we present the results of our Higgs mass calculation and compare them with the output of public tools for several $\mathcal{CP}$-violating scenarios. We also investigate the relevance of the matrix $\mathbf {Z}^\mathrm{{mix}}$ for transition amplitudes in the example of the one-loop corrected decays of one Higgs field into a tau/anti-tau pair, $h_i\rightarrow \tau ^+\tau ^-$.

The choice for the top-quark mass is $m_t = 173.2$ GeV. Throughout this section all $\overline{\mathrm {DR}}$ parameters are defined at $m_t$, and all stop-parameters are on-shell parameters.

From the point of view of the Higgs phenomenology we test the scenarios presented in this section with the full set of experimental constraints and signals implemented in the public tools HiggsBounds-4.3.1 [159,160,161,162,163,164] and HiggsSignals-1.3.1 [164, 165].

3.1 Comparison with FeynHiggs in the MSSM limit

In the limit of vanishing $\lambda$ and $\kappa$, the singlet superfield decouples from the MSSM sector and one is left with an effective MSSM—the $\mu _{\text {eff}}$ term persists as long as $\kappa \sim \lambda$. We may then compare our results for the Higgs masses and the matrix $\mathbf {Z}^\mathrm{{mix}}$ in this limit to those of FeynHiggs-2.12.0. For this to be meaningful, we adjust the settings of FeynHiggs, so that they match the higher-order contributions and renormalization scheme of our NMSSM calculation. In particular, we impose that the one-loop field-renormalization constants and $\tan \beta$ are $\overline{\mathrm {DR}}$-renormalized and select full one-loop and leading two-loop MSSM contributions of $\mathcal {O}{(\alpha _t\alpha _s + \alpha _t^2)}$. We also require that FeynHiggs takes the $\tan \beta$-enhanced contributions to the down-type Yukawa couplings into account. The corresponding FeynHiggs input flags read FHSetFlags[4,0,0,3,0,2,0,0,1,1].

Fig. 2

Modules for the nine elements of the matrix $\mathbf {Z}^\mathrm{{mix}}$, obtained with our calculation (solid line) and FeynHiggs (squares). The colors follow the convention of Fig. 1: red for the coefficients defining the wave-function normalization of the light, SM-like state $h_1$, blue and green for the coefficients corresponding to the heavy states $h_2$ and $h_3$, respectively. The parameters are chosen as in Fig. 1

We consider a region in the parameter space of the NMSSM with the following characteristics: $\lambda =\kappa =10^{-5}$, $\tan \beta =10$, $m_{H^{\pm }}=500$ GeV, $\mu _{\text {eff}}=250$ GeV, $A_{\kappa }=-100$ GeV; the sfermion soft masses are set to the universal value of 1.5 TeV and the sfermion trilinear couplings to a value of 0.5 TeV, with the exception of the third generation parameters $|A_t|=A_b=2.5$ TeV; the gaugino masses are chosen as follows: $2M_1=M_2=M_3/5=0.5$ TeV. We then vary the phase $\phi _{A_t}$. A variation of $\phi _{A_t}$ (or of any MSSM-like phase) in such a naive direction is of limited phenomenological interest, since in this case limits from Electric Dipole Moments (EDMs) are violated almost as soon as $\mathcal{CP}$, see e.g. Ref. [84]. In the following we dismiss this issue, however, and allow $\phi _{A_t}$ to vary over its full range. Indeed, we are only interested in comparing our results with those of FeynHiggs. Due to the largely SM-like properties of the state with a mass close to 125 GeV, our scenario appears to retain characteristics that are compatible with the experimental data implemented in HiggsBounds and HiggsSignals, over the whole range of $\phi _{A_t}$. The results for the Higgs masses are displayed in Fig. 1. We observe a near perfect agreement between our results (solid curves) and those of FeynHiggs (squares) with differences of order MeV. This agreement is expected, since we closely follow the procedure for the renormalization and processing of the MSSM-like input of FeynHiggs. Moreover, due to the small values for $\lambda$ and $\kappa$, deviations induced by genuine NMSSM effects remain negligible. The results for the elements of the matrix $\mathbf {Z}^\mathrm{{mix}}$ are displayed in Fig. 2. Again, we find a very good agreement between our results and FeynHiggs with differences below 1 for the modules.

3.2 Comparison in the $\mathcal{CP}$-conserving limit

We now turn away from the MSSM limit. Our mass calculation can be confronted to the routines presented in [113] in the $\mathcal{CP}$-conserving case. Both approaches employ an identical renormalization scheme in this limit, with the exception of the electroweak VEV, which receives a $\overline{\mathrm {DR}}$ renormalization in [113] while we parametrize v in terms of $M_W$, $M_Z$ and e (see Eq. (2.27)). However, in [113] the input for v is obtained via a reparametrization from our scheme (the scheme using $\alpha (M_Z)$ as input is also considered), as explained in section 2.3 of that reference. Therefore, both mass predictions are directly comparable and the mismatch between them should be understood as an effect of two-loop electroweak order, due to the approximations in the reparametrization used by [113].

We consider the following region in the parameter space of the NMSSM: $\kappa =\lambda /2$, $\tan \beta =10$, $M_{H^{\pm }}=1$ TeV, $\mu _{\text {eff}}=125$ GeV, $A_{\kappa }=-70$ GeV; the soft masses are taken as in the previous subsection while the trilinear soft sfermion couplings are all set to 0.5 TeV, with the exception of $A_t=1.2$ TeV. We scan over $\lambda$ from $\sim$0 (the MSSM limit) to 0.5. The masses of the three lightest Higgs states are displayed in the plot on the left-hand side of Fig. 3. The dominantly SM-like state is the heaviest of the three (green curve). The two lighter states are dominantly singlet, $\mathcal{CP}$-even (red curve) or $\mathcal{CP}$-odd (blue curve). In the MSSM limit, these three states are significantly lighter than 125 GeV: this results in an unsatisfactory phenomenological situation in view of the LHC measurements. With increasing $\lambda$, the $\mathcal{CP}$-even singlet–doublet mixing uplifts the mass of the dominantly SM-like state, leading to phenomenologically viable characteristics—as tested with HiggsSignals—for $\lambda \sim 0.2$. There, we observe that $h_1$ possesses a sizable doublet component and a mass $M_{h_1}\simeq 100$ GeV, so that this state could offer an interpretation of the local excess observed at LEP in $e^+e^-\rightarrow Z+(H\rightarrow b\bar{b})$ searches [166].

Fig. 3

Masses of the three lightest Higgs states $h_1$ (red), $h_2$ (blue), $h_3$ (green) in the $\mathcal{CP}$-conserving limit for varying $\lambda =2\,\kappa$ and the following input: $\tan \beta =10$, $M_{H^{\pm }}=1$ TeV, $\mu _{\text {eff}}=125$ GeV, $A_{\kappa }=-70$ GeV, $m_{\tilde{F}}=1.5$ TeV, $A_t=2$ TeV, $A_{f\;\ne \;t}=0.5$ TeV, $2\,M_1=M_2=M_3/5=0.5$ TeV. On the left-hand side, we present our result (solid line) and that of [113] (squares). On the right-hand side, we plot the mass differences between these two codes, due to the scheme applied to the electric coupling

We then focus on the comparison with the masses predicted by [113]. The plot on the left-hand side of Fig. 3 illustrates a general agreement between our calculation (solid curves) and the results of [113] (squares). On the right-hand side of Fig. 3, we display the mass differences between the two procedures, which are due to differences of two-loop order induced by the reparametrization used by [113]. We observe vanishing effects in the MSSM limit, while the mass differences eventually reach $\mathcal {O}{\left( 40\,\text {MeV}\right) }$ for $\lambda \simeq 0.16$. This can be understood in the following fashion: the leading effect originates in the Higgs mass matrix at the tree level, where an explicit dependence on v appears only⁴ through terms of the form $\lambda \,v$ and $\kappa \,v$ (quadratically for the doublet and singlet mass entries, and linearly for the doublet–singlet mixing). These terms are processed differently in both approaches: in [113] v is regarded as an independent $\overline{\mathrm {DR}}$ parameter, while in our calculation v is a dependent quantity that is expressed in terms of the independent parameters $M_W$, $M_Z$ and e. While the reparametrization of [113] should restore the agreement between the two procedures, neglected effects of two-loop electroweak order in this reparametrization result in a small mismatch. Since the terms that convey this mismatch come with prefactors $\lambda$ or $\kappa$, the difference vanishes in the MSSM limit ($\lambda ,\kappa \rightarrow 0$). Moreover, in the regime under consideration, where $\tan \beta \gg 1$, it is possible to understand why the mass of the $\mathcal{CP}$-odd singlet (blue curve) is largely insensitive to the mismatch: terms $\propto (\lambda \,v)^2$ in the $\mathcal{CP}$-odd singlet mass entry are suppressed as $1/\tan \beta$. Additionally, leading one-loop radiative corrections of $\mathcal {O}{\left( \alpha _t\right) }$ induce further dependence on the processing of v. However, these corrections are suppressed for the points of Fig. 3, as the stops are relatively light.

On the whole, the numerical mismatch with the procedure of [113] is very minor, which places our current code in the direct continuity of this earlier work.

3.3 Comparison with NMSSMCALC

NMSSMCALC is particularly suitable for a comparison with our calculation, since its mixed $\overline{\mathrm {DR}}$/on-shell renormalization scheme is relatively close to the one that we use.⁵ Yet, we note several differences between the prescriptions implemented by NMSSMCALC and the procedure that we have outlined in Sect. 2 (defining our “default” calculation). First, NMSSMCALC applies a renormalization scheme for the electric charge employing $\alpha (M_Z)$ as input—whereas we decided to define $\alpha$ via its relation to $G_F$. Then the input parameters in the stop sector are defined in the $\overline{\mathrm {DR}}$ scheme in NMSSMCALC—while we employ on-shell definitions. Additionally, we resum large $\tan \beta$ effects from our definition of the bottom Yukawa, contrarily to the Higgs-mass calculation of NMSSMCALC. Finally, NMSSMCALC includes only $\mathcal {O}{\left( \alpha _t\alpha _s\right) }$ corrections at the two-loop order—where we consider $\mathcal {O}{\left( \alpha _t^2\right) }$ effects as well. However, the two-loop $\mathcal {O}{\left( \alpha _t\alpha _s\right) }$ contributions of NMSSMCALC are exhaustive in the NMSSM (including corrections for the self-energies with at least one external singlet field)—whereas ours are obtained in the MSSM approximation.

These observations mean that our mass calculation is not directly (at least, not quantitatively) comparable to the predictions of NMSSMCALC, since, of the items listed above, the first few produce a deviation relative to the scheme, while the later ones generate a mismatch of higher orders. Consequently, several adjustments need to be performed in order to make a comparison meaningful and control the sources of deviations. Thus, NMSSMCALC has been adjusted in view of accepting on-shell input in the stop sector.⁶ Moreover, we also establish a “modified” version of our routines that attempts to mimic the choices of NMSSMCALC—i.e. employing $\alpha (M_Z)$, discarding large-$\tan \beta$ effects for $Y_b$ and subtracting $\mathcal {O}{\left( \alpha _t^2\right) }$ corrections—although we cannot currently include $\mathcal {O}{\left( \alpha _t\alpha _s\right) }$ corrections beyond the MSSM, so that this effect should control the difference of our modified version with NMSSMCALC. Beyond this comparison with NMSSMCALC, we will also try to quantify the magnitude of the other higher-order effects that distinguish our “default” result from NMSSMCALC.

First, we consider the regime of the NMSSM with low $\tan \beta$ and large $\lambda$. This region in parameter space is well-known for maximizing the specific NMSSM tree-level contributions to the mass of the SM-like Higgs state as well as for stimulating singlet–doublet mixing effects and other genuine aspects of the NMSSM phenomenology. We employ the following parameters: $\lambda =0.7$, $|\kappa |=0.1$, $\tan \beta =2$, $M_{H^{\pm }}=1170$ GeV, $\mu _{\text {eff}}=500$ GeV, $A_{\kappa }=-70$ GeV; the soft masses are taken as in Fig. 1 with the exception of the squarks of the third generation, for which the soft masses and trilinear couplings are set to 500 and 100 GeV, respectively. In the regime under consideration genuine NMSSM effects are indeed sufficient to produce a SM-like state in the observed mass range without requiring large top/stop corrections. We vary the phase $\phi _{\kappa }$ (we restrict ourselves to a range where the tree-level squared Higgs masses remain positive). We note that, contrarily to MSSM-like phases, the phases from the singlet sector are allowed a wide range of variation without conflicting with the measured EDM [83, 169].

The results for the mass prediction are presented in Fig. 4. At vanishing $\phi _{\kappa }$ the mass of the SM-like state (in blue) is somewhat low, $m_{h_2} \sim 120$ GeV, so that this point in parameter space has a very marginal agreement with the observed characteristics of the Higgs state. For non-vanishing $\phi _{\kappa }$, however, a $\mathcal{CP}$-violating mixing with the lighter pseudoscalar singlet (in red) develops: this effect increases the mass of the light mostly $\mathcal{CP}$-even state $h_2$ but affects its otherwise SM-like properties only in a sub-leading way. Consequently, we recover an excellent agreement with the LHC results—as tested by HiggsSignals and HiggsBounds—for e.g. $\phi _{\kappa }\simeq -0.11$. Additionally, the dominantly $\mathcal{CP}$-odd singlet $h_1$ then has a mass close to 100 GeV. As it acquires a doublet $\mathcal{CP}$-even component via mixing, it could explain the LEP local excess in $b\bar{b}$ final states [166]. The mostly $\mathcal{CP}$-even singlet $h_3$ (in green), with mass at $\sim$210 GeV plays no significant role. The masses of the heavier doublet-like fields $h_4$ and $h_5$ are approximately constant and close to $M_{H^\pm }$.

In Fig. 4, we observe good agreement between our results (solid lines), computed as described in Sect. 2, and the predictions of NMSSMCALC (squares), although the corresponding masses are defined in different schemes and at different orders. For a more quantitative comparison, we turn to our “modified” scheme for the mass calculation. On the left-hand side of Fig. 5, we plot the deviation between the corresponding results and the predictions of NMSSMCALC for the three lightest Higgs states. We checked that the one-loop results are virtually identical, so that the differences between NMSSMCALC and our calculation are entirely controlled by two-loop effects. We observe typical deviations of order 0.5–1 GeV, which should be interpreted as the impact of $\mathcal {O}{\left( \alpha _t\alpha _s\right) }$ corrections beyond the MSSM-approximation. As could be expected, the masses of the mostly singlet states (red and green lines) tend to exhibit the largest effect, though the mass predictions for the SM-like state may still differ by $\sim$0.5 GeV (for $\phi _{\kappa }\simeq 0$). The plot on the right-hand side of Fig. 5 depicts the magnitude of $\mathcal {O}{\left( \alpha _t^2\right) }$ effects, which is quantified in our “default” scheme. Here again, the typical impact on the masses is of order 1 GeV. Expectedly, the masses of the almost pure singlet states (red curve at $\phi _{\kappa }\simeq 0$ or green curve) are insensitive to the corrections implemented in the MSSM-approximation. The mass of the mostly $\mathcal{CP}$-odd singlet (red curve) is only affected when the corresponding state acquires a non-vanishing doublet component ($\phi _{\kappa }\ne 0$).

In Fig. 6, we compare the $\mathbf {U}^0$ matrix elements that are delivered by NMSSMCALC (squares) with ours (solid line; NMSSMCALC does not provide $\mathbf {Z}^{\mathrm{mix}}$). The results show a satisfactory agreement also at this level.

Fig. 4

Masses of the three lighter Higgs fields as a function of $\phi _\kappa$ for the scenario $\lambda =0.7$, $|\kappa |=0.1$, $\tan \beta =2$, $M_{H^{\pm }}=1170$ GeV, $\mu _{\text {eff}}=500$ GeV, $A_{\kappa }=-70$ GeV,$m_{\tilde{Q}_3,\tilde{T},\tilde{B}}=0.5$ TeV, $A_t=A_b=0.1$ TeV, $2\,M_1=M_2=M_3/5=0.5$ TeV. The red color depicts the mass of the mostly $\mathcal{CP}$-odd, singlet-like state $h_1$; blue is associated to the essentially SM-like state $h_2$ and green corresponds to the mostly $\mathcal{CP}$-even singlet-like state $h_3$. We display our “default” result for the masses (solid curves) as well as the predictions of NMSSMCALC (squares)

Fig. 5

Impact of two-loop contributions in the scenario of Fig. 4. On the left-hand side, $\Delta {M_{h_i}} = M_{h_i} - M_{h_i}^{\texttt {NC}}$ correspond to the mass differences (for each of the three lightest Higgs states) between the predictions of our “modified” scheme and NMSSMCALC: $\mathcal {O}{\left( \alpha _t\alpha _s\right) }$ should dominate these deviations. On the right-hand side, $\Delta {M_{h_i}}$ corresponds to the mass-shifts associated to $\mathcal {O}{\left( \alpha _t^2\right) }$ contributions, which are calculated in our “default” scheme. The color of the curves match the convention of Fig. 4

Fig. 6

The matrix elements $|U^0_{ij}|$ in our calculation (full curves) and in NMSSMCALC (squares) for the scenario of Fig. 4

Fig. 7

Mass predictions as a function of $\phi _\kappa$ for the lightest, mostly SM-like Higgs state $h_1$ on the left-hand side, and the states $h_2$ (blue), $h_3$ (green) and $h_4$ (orange) on the right-hand side. The latter involve essentially the $\mathcal{CP}$-even singlet and the two $\mathcal{CP}$-even and $\mathcal{CP}$-odd heavy-doublet degrees of freedom that mix substantially. The full curves correspond to our default result; the squares are obtained with NMSSMCALC; the dotted line represents our modified result. The parameters are chosen as follows: $\lambda =0.2$, $|\kappa |=0.6$, $\tan \beta =25$, $m_{H^{\pm }}=1$ TeV, $\mu _{\text {eff}}=200$ GeV, $A_{\kappa }=-750$ GeV, $m_{\tilde{Q}_3,\tilde{T},\tilde{B}}=1.1$ TeV, $A_t=A_b=-2$ TeV, $2\,M_1=M_2=M_3/5=0.5$ TeV

Subsequently, we present our results in another region of the parameter space: $\lambda =0.2$, $|\kappa |=0.6$, $\tan \beta =25$, $m_{H^{\pm }}=1$ TeV, $\mu _{\text {eff}}=200$ GeV, $A_{\kappa }=-750$ GeV, the gaugino soft masses as well as the soft masses for the sfermions of first and second generations are chosen as before; for the third generation, the soft sfermion mass is set to 1.1 TeV; the trilinear soft terms are set to $-2$ TeV. With this choice of parameters, the singlet-like $\mathcal{CP}$-even state and the heavy $\mathcal{CP}$-even and $\mathcal{CP}$-odd doublet-like states receive comparable masses of the order of 1 TeV. This results in a sizable mixing for the corresponding fields $h_2$, $h_3$ and $h_4$, which includes both singlet-doublet admixture as well as $\mathcal{CP}$-violation (for non-vanishing $\phi _{\kappa }$). The SM-like Higgs state has a mass close to $\sim$124 GeV on the whole range of $\phi _{\kappa }$, which leads to good agreement with the Higgs properties measured at the LHC (as tested with HiggsSignals). The heaviest state $h_5$ has a mass of $\sim$1.2 TeV, which we will not comment further below.

On the left-hand side of Fig. 7, we show our prediction for the mass of the lightest (SM-like) Higgs state (full curve). The mass delivered by NMSSMCALC is represented by the squares at about  118 GeV, which is substantially smaller than ours (by $\approx$6 GeV). If we mimic the settings of NMSSMCALC (our “modified result”, dotted curve), this discrepancy is considerably reduced. In fact, the difference between our full result and NMSSMCALC’s is largely driven by the $\mathcal {O}{\left( \alpha _t^2\right) }$ two-loop contributions, missing in NMSSMCALC. Again, both results are virtually identical at the one-loop order.

On the right-hand side of Fig. 7, we turn to the heavier states $h_2$, $h_3$ and $h_4$ of this scenario. Our default results (full curves) are compatible with the predictions of NMSSMCALC (squares). The discrepancies are of order 1–3 GeV only, which should be considered from both the perspective of the different renormalization scheme of the electric coupling e and of the different two-loop contributions. Actually, the mass predictions match almost exactly when comparing NMSSMCALC with our modified scheme (dotted curves). The corresponding deviations are shown on the left-hand side of Fig. 8 and fall in the range of 100 MeV. In this precise case, the difference between our results and NMSSMCALC is essentially driven by the resummation of large-$\tan \beta$ effects in the b-quark Yukawa coupling. On the right-hand side of Fig. 8, we quantify the associated mass-shift and find an impact of a few GeV.

Finally, we turn to the $\mathbf {U}^0$ matrix elements for $h_2$, $h_3$ and $h_4$ in Fig. 9. There, we observe sizable deviations between our default result (solid curves) and NMSSMCALC (squares), which, however, have no deep-reason to agree in view of the diverging options. If we keep in mind that the main difference between our full scheme and NMSSMCALC is controlled by the large-$\tan \beta$ corrections to $Y_b$ in this precise scenario, it is not surprising to observe large shifts, as mixing angles are indeed very sensitive to small deviations in the mass-matrix for states that are very close in mass. These differences largely vanish when we identify the output of NMSSMCALC with our modified results (dotted lines), which is better equipped for comparisons with this code.

In summary, the results of our mass calculation are largely compatible with the predictions of NMSSMCALC. Deviations with a magnitude of $\mathcal {O}{\left( \text {GeV}\right) }$ are indeed within the expected range if we allow for the different renormalization scheme of e and higher-order contributions. Such discrepancies tend to be reduced sizable when we modify our routines to adopt the assumptions of NMSSMCALC. The impact of two-loop $\mathcal {O}{\left( \alpha _t\alpha _s\right) }$ contributions beyond the MSSM, two-loop $\mathcal {O}{\left( \alpha _t^2\right) }$ and the resummation of large-$\tan \beta$ effects in the Yukawa couplings can be clearly identified from this comparison. As a final remark, let us emphasize that the uncertainty on the Higgs-mass calculation from unknown higher-order contributions and parametric errors may reach several GeV [108, 167, 168].

Fig. 8

Impact of the higher-order effects in the mass predictions for $h_2$, $h_3$ and $h_4$ in the scenario of Fig. 7. On the left-hand side, we show the deviation in mass between NMSSMCALC and our modified scheme (controlled by $\mathcal {O}{\left( \alpha _t\alpha _s\right) }$ corrections beyond the MSSM-approximation). The plot on the right-hand side illustrates the impact of large-$\tan \beta$ effects in $Y_b$: the considered mass difference is that induced in our default scheme by the $\Delta _b$ term. The colors follow the conventions of Fig. 7. The discontinuities at $\phi _\kappa =\pm \frac{\pi }{2}$ originate from the mass calculation in NMSSMCALC

Fig. 9

The matrix elements $|U^0_{ij}|$ in our calculation (solid curves), in NMSSMCALC (squares), and in our modified calculation closer to the options of NMSSMCALC (dotted) for the scenario of Fig. 7

Fig. 10

On the left-hand side, the masses of the three lighter Higgs states are depicted as a function of $\phi _\kappa$, including all available two-loop contributions. On the right-hand side, the squared norms $\left| Z_i\right| ^2$ [as defined in Eq. (2.38)] of the five eigenvectors defining $\mathbf {Z}^{\mathrm{mix}}$ are shown. The scenario is characterized by $\lambda =0.7$, $|\kappa |=0.1$, $\tan \beta =2$, $m_{H^{\pm }}=1.2$ TeV, $\mu _{\text {eff}}=500$ GeV, $A_{\kappa }=-100$ GeV, $m_{\tilde{Q}_3,\tilde{T},\tilde{B}}=0.5$ TeV, $A_t=A_b=0.1$ TeV, $2\,M_1=M_2=M_3/5=0.5$ TeV

Fig. 11

In the left column, we show the decay widths $\Gamma (h_i\rightarrow \tau ^+\tau ^-)$ in the scenario of Fig. 10 for the five neutral Higgs states. The widths are computed at the one-loop level, and the mixing of the external, physical Higgs fields is expressed in terms of the matrix $\mathbf {Z}^{\mathrm{mix}}$(solid), or approximated by the matrices $\mathbf {U}^m$ (dashed) or $\mathbf {U}^0$ (dotted). In the right column, the differences $\Delta {\Gamma } = \Gamma - \Gamma _{\text {appr}}$ between the widths obtained with $\mathbf {Z}^{\mathrm{mix}}$and its approximate treatments $\mathbf {U}^m$ (dashed) and $\mathbf {U}^0$ (dotted) are depicted, normalized to the width $\Gamma$ obtained with $\mathbf {Z}^{\mathrm{mix}}$

3.4 The matrix $\mathbf {Z}^\mathrm{{mix}}$ and the $h_i\rightarrow \tau ^+\tau ^-$ decays

The matrix $\mathbf {Z}^\mathrm{{mix}}$ is not an observable quantity in itself. It is a renormalization-scheme dependent object relating the tree-level mass states of the Higgs sector to the physical Higgs fields. For on-shell renormalized fields $\mathbf {Z}^\mathrm{{mix}}$ is trivial. In any other renormalization scheme, however, it is mandatory to include this transition to the physical fields for a proper description of external legs in Feynman diagrams at higher orders.

A remarkable aspect of $\mathbf {Z}^\mathrm{{mix}}$ is that the eigenvectors that it contains do not preserve unitarity with respect to the tree-level fields. Instead, they satisfy the normalization condition given in Eq. (2.37). This is a feature that the approximations $\mathbf {U}^0$ and $\mathbf {U}^m$ are unable to capture (by construction). In a first step, we will show that the norms in $\mathbf {Z}^\mathrm{{mix}}$ can differ from 1 by a few percent in the scheme that we have described in Sect. 2. Beyond the normalization of the fields, $\mathbf {U}^0$ and $\mathbf {U}^m$ also differ from $\mathbf {Z}^\mathrm{{mix}}$ in that they diagonalize the mass-matrix away from the poles of the propagator.

However, as we wrote above, $\mathbf {Z}^{\mathrm{mix}}$ is a scheme-dependent object and we should not pay excessive attention to its actual structure. In order to characterize its role in an observable quantity, we will consider the $h_i\rightarrow \tau ^+\tau ^-$ decays at the one-loop level. We have chosen this particular channel as it is one of the main fermionic Higgs decays and proves technically simple to implement in a predictive way. Moreover, one-loop corrections are of purely electroweak nature—QCD contributions occur only at three-loop order and beyond—so that radiative corrections are expected to be moderate. This allows for a clean appreciation—free of large higher-order uncertainties—of the impact of the wave-function normalization matrix $\mathbf {Z}^\mathrm{{mix}}$. Radiative corrections⁷ are computed with our model file, except for the QED contributions, which are included according to the prescriptions of Refs. [170, 171]. There, $\mathbf {Z}^\mathrm{{mix}}$ intervenes in the decay amplitudes of the physical fields according to Eq. (2.32) (we drop the superscript ‘phys’ throughout this section). We will show that the substitution of $\mathbf {Z}^\mathrm{{mix}}$ by the approximations $\mathbf {U}^0$ and $\mathbf {U}^m$ may lead to sizable deviations in certain regions of the NMSSM parameter space. This result confirms the outcome of similar studies in the MSSM [109].

We turn to the following NMSSM input: the parameters are chosen as in Fig. 4, except for $M_{H^{\pm }}=1.2$ TeV and $A_{\kappa }=-100$ GeV. We have plotted the masses of the three lighter Higgs fields as a function of $\phi _\kappa$ in the plot on the left-hand side of Fig. 10. For vanishing $\phi _{\kappa }$ the lightest Higgs state $h_1$ is SM-like and we checked with HiggsBounds and HiggsSignals that this point is consistent with the experimental data. The dominantly $\mathcal{CP}$-odd, singlet-like state $h_2$ is only slightly heavier than the state $h_1$ in this case. For increasing values of $\phi _{\kappa }$ the mixing of the states $h_1$ and $h_2$ tends to lower the mass of the SM-like state $h_1$, which eventually becomes too light to accommodate the experimental data. The dominantly $\mathcal{CP}$-even, singlet-like state $h_3$ has a near constant mass of $\sim$210 GeV for all depicted values of $\phi _\kappa$. The two heavier, $\mathcal{CP}$-even and $\mathcal{CP}$-odd doublet-like states have masses close to $\sim$1.2 TeV.

The results for the squared norms $|Z_i|^2$ of the eigenvectors—see Eq. (2.38)—in this scenario are shown in the plot on the right-hand side of Fig. 10. We observe a departure from the value 1—which would correspond to a unitary transition, as modeled by the approximations $\mathbf {U}^0$ and $\mathbf {U}^m$—by a few percent. The local extrema at $\phi _{\kappa }\simeq 0$ for $|Z_1|^2$ (red curve) and $|Z_2|^2$ (blue curve) are associated to the sudden disappearance of the mixing between the light $\mathcal{CP}$-odd singlet and the SM-like states at $\phi _\kappa = 0$ ($\mathcal{CP}$-conserving limit). The discontinuities of $|Z_2|^2$ and $|Z_3|^2$ (green curve) at $\phi _{\kappa }\simeq \pm 0.5$ and $\pm 0.7$ correspond to the crossing of decay thresholds ($h_2\rightarrow W^+W^-$, $h_2\rightarrow 2\,Z$, $h_{3}\rightarrow h_1\,h_2$). These “spikes” are associated to the singularities of the first derivatives of the loop functions involved in the determination of $\mathbf {Z}^{\mathrm{mix}}$—the apparent singularities actually come with a finite height due to the imaginary parts of the poles. A proper description of these threshold regions would require that the interactions among the daughter particles (of the decays at threshold) are properly taken into account, which would result in e.g. interactions between the Higgs state and bound-states or s-waves of the daughter particles. This, however, goes beyond the scope of the present work.

We now turn to the decay widths $\Gamma {(h_i \rightarrow \tau ^+\tau ^-)}$ in the scenario of Fig. 10. The widths are displayed in the left column of Fig. 11 in the exhaustive description of the Higgs external leg (i.e. employing $\mathbf {Z}^{\mathrm{mix}}$; solid curves), in the $\mathbf {U}^m$ approximation (dashed lines) and in the $\mathbf {U}^0$ approximation (dotted lines), for the five Higgs-mass eigenstates. We observe a sharp variation close to $\phi _{\kappa }=0$ for the decays of $h_1$ and $h_2$, both in the full and approximate descriptions. It is associated to the mixing that develops between the SM-like state $h_1$ and the dominantly $\mathcal{CP}$-odd, singlet-like state $h_2$: this effect transfers part of the doublet component of $h_1$ to $h_2$, so that the second state acquires a non-vanishing coupling to SM fermions at the expense of the first. The sum of the decay widths for both these states remains approximately constant in the vicinity of $\phi _{\kappa }=0$. The width $\Gamma (h_3\rightarrow \tau ^+\tau ^-)$ appears to be an order of magnitude smaller than the corresponding widths for $h_1$ and $h_2$, an effect that is associated to the dominantly $\mathcal{CP}$-even, singlet-like nature of $h_3$. Still, $\Gamma (h_3\rightarrow \tau ^+\tau ^-)$ nearly doubles in the considered interval of $\phi _{\kappa }$, while the mass of $h_3$ is fairly stable: we can understand this fact in terms of the acquisition of a larger doublet component, which is channeled by the increased proximity of the masses of $h_2$ and $h_3$. The widths of $h_4$ and $h_5$ are essentially constant with only small relative changes. The general $\phi _\kappa$-dependency of the approximated and the full results are very similar. Yet, a systematic shift can be observed, especially in the case of $\mathbf {U}^0$. This is consistent with the findings of similar studies in the context of the MSSM [109].

On the right-hand side of Fig. 11, we show the difference between the full and the approximate results, $\Delta {\Gamma } = \Gamma - \Gamma _{\text {appr}}$, normalized to the more accurate one obtained with $\mathbf {Z}^{\mathrm{mix}}$. When $\mathbf {Z}^{\mathrm{mix}}$ is approximated by $\mathbf {U}^0$ (dotted lines), the typical discrepancy averages 4%, although the deviation reaches beyond 10% in the case of the decays of the two lightest Higgs states in the vicinity of, but not at, $\phi _{\kappa }\simeq 0$. We stress that this interval close to $\phi _{\kappa }=0$ corresponds to the phenomenologically relevant region from the perspective of the measured Higgs properties. The approximation of $\mathbf {Z}^{\mathrm{mix}}$ by $\mathbf {U}^m$ tends to provide better estimates of the full result, though deviations reach up to $\sim$7%. For both approximations the largest deviations from the more complete result employing $\mathbf {Z}^{\mathrm{mix}}$ are intimately related to the proximity in mass of the SM-like and dominantly $\mathcal{CP}$-odd, singlet-like states: as the approximations capture the dependence on the external momentum either partially ($\mathbf {U}^m$) or not at all ($\mathbf {U}^0$), the gap between the diagonal elements of the Higgs-mass matrix, hence the mixing between the two states, is not quantified properly. While this precise configuration might appear somewhat anecdotal, we wish to point out the popularity of NMSSM scenarios with a sizable singlet-doublet mixing. Dismissing this extreme case, the approximations of $\mathbf {Z}^{\mathrm{mix}}$ by $\mathbf {U}^0$, and to a lesser extent by $\mathbf {U}^m$, still generate errors of the order of a few percent at the level of the decay widths. In view of the precision of the measurements achievable at the LHC [3, 172,173,174,175], such discrepancies may appear of secondary importance. In the long run, however, if the Higgs couplings are studied more closely, e.g. at a linear collider [176,177,178,179], one would have to try and keep such sources of error to a minimum.

4 Conclusions

In this paper, we have discussed the renormalization of the NMSSM Higgs sector, including complex parameters. Radiative contributions to the Higgs self-energies have been included up to the leading two-loop MSSM-like effects of $\mathcal {O}{\left( \alpha _t\alpha _s+\alpha _t^2\right) }$. Beyond the calculation of on-shell Higgs masses in this scheme, we were interested in determining the transition matrix $\mathbf {Z}^{\mathrm{mix}}$ between the mass and tree-level states. The latter play an essential role in the proper description of external Higgs legs in physical processes at the radiative level.

Our predictions for the Higgs masses have been compared to the calculations of existing tools in several NMSSM scenarios. In the MSSM limit of the model, we have recovered an excellent agreement with FeynHiggs. For non-vanishing $\lambda$ and $\kappa$, we first compared our Higgs-mass prediction with the findings of a previous extension of FeynHiggs to the NMSSM in the case of real parameters, and found nearly identical results. Second, we compared our calculation in the case of complex parameters with NMSSMCalc and found values of the Higgs masses which are compatible, although small differences emerge as a result of different processing of the two-loop pieces, both for low and high $\tan \beta$.

Finally, we investigated the impact of the transition matrix $\mathbf {Z}^{\mathrm{mix}}$ on the $h_i\rightarrow \tau ^+\tau ^-$ width in a scenario with low $\tan \beta$ and large $\lambda$, where the SM-like Higgs state may have a sizable mixing with the $\mathcal{CP}$-odd singlet. We compared the full one-loop calculation of the width—i.e. including $\mathbf {Z}^{\mathrm{mix}}$—with the popular approximations $\mathbf {U}^0$ and $\mathbf {U}^m$—which are determined for fixed, unphysical momenta. We found typical deviations at the percent level, although larger effects can develop in the presence of almost-degenerate states, especially in the $\mathbf {U}^0$ approximation. Such precision effects will matter when the measurement of fermionic Higgs couplings reaches comparable accuracy.

In its current form, our mass-computing tool is contained within a Mathematica package. In time, our routines should be incorporated in an extension of FeynHiggs to the NMSSM.