1 Introduction

Despite its successes in describing the fundamental particles, most recently Higgs boson with 125 GeV [1, 2], the Standard Model of particles physics (SM) falls short in providing satisfactory explanations for various questions such as neutrinos mass [3], dark matter [4], B-meson anomalies [5, 6] and many others. One particular issue is the naturalness of the Higgs mass. From an experimental point of view, the mass of the Higgs boson (125 GeV) is in the same order as the electroweak scale. However, from a naturalness perspective, this mass is significantly larger than what would be expected at the electroweak scale. The reason behind this discrepancy lies in the substantial radiative corrections to the Higgs mass, resulting in an unnatural fine-tuning between the tree-level Higgs mass and the radiative corrections. These radiative corrections exhibit divergences and demonstrate a quadratic sensitivity to the highest scale in the theory [7]. And, to face this problem, solutions propose extending the SM so that the embedded new physics (NP) can compensate for these significant corrections to the Higgs boson mass. In this regard, Veltman [8] put forward some forty years ago, in the realm of the SM, the idea that the radiative corrections to the scalar mass should either vanish or be kept at a manageable level. Theoretically, however, such approach (often referred to by VC for Veltman condition) can mitigate the impact of radiative corrections and thus can be used as an unambiguous guide for physics beyond the SM.

In the literature, VC was the subject of many studies beyond standard model (BSM), where the SM Higgs sector is extended either by singlets [9,10,11], an extra doublet [12,13,14,15,16,17] or a triplet fields [18,19,20]. In such extensions, the VC approach is treated by either: i) dimensional regularization (DR) [21] which is convenient for handling divergences but does not fundamentally address the naturalness problem and can mask the need for fine-tuning, or via ii) cut-off regularization [22] which introduces a cut-off parameter \(\Lambda \) that limits the range of integration, thereby providing a clear physical interpretation of the regularization process. In a recent paper [23], the authors discussed dimensional regularization and its limitations in addressing the naturalness problem. They found that many proposed solutions often involve hidden adjustments when examined through the Wilsonian framework. Similarly, within the same context, conservative naturalness bounds on BSM extensions have been derived using a full two-loop RGE analysis [24]. Overall, the presence of VC leads to significant effects on corresponding parameter spaces and on Higgs phenomenologies in BSM.

In this work, we underline an extension wherein the 2HDM [25, 26] Lagrangian is extended by a real scalar singlet S. The resulting model, referred to as the N2HDM, has been extensively studied and is considered a potential benchmark in experimental analysis, notably with regard to dark matter (DM) purposes [27,28,29,30,31], where the singlet field is inert and constitutes a viable DM candidate. However, a non-vanishing vacuum expectation value (VEV) for the singlet field is equally important and remains fertile ground for further experimental discussions [32]. Indeed, this fact is emphasized throughout the mixing between the singlet field and 2HDM doublets, leading after the spontaneous electroweak symmetry breaking (EWSB), to three neutral CP-even Higgs bosons, namely: \(h_1\), \(h_2\) and \(h_3\), making the N2HDM even more interesting phenomenologically. As a result, the projected limits of 2HDM research are expected to be altered. Additionally, the mixing will enable the direct detection of the heavy Higgs bosons (which are mostly singlet), through, for example, \(gg \rightarrow h_3 \rightarrow ZZ\). Another notable aspect of N2HDM with a non-zero VEV for the singlet field is related to the couplings with SM particles, which can be sufficiently weak, allowing light Higgs bosons to evade exclusions by Higgs searches at LEP, Tevatron, and LHC in the low mass range. Additionally, like the 2HDM, the N2HDM presents four types of Yukawa interactions with no FCNC at tree level [33], depending on which type of fermions couples to which doublet \(H_{i}\). By convention, \(H_{2}\) is the doublet to which all fermions couple within type I, unlike type II where such doublet couples only to up-type quark whereas the remaining fermions couple to \(H_{1}\). Type III, also called flipped models, has down-type quarks coupling to the first doublet and up-type quarks and charged leptons coupling to the second doublet. Ultimately, in type IV, or the lepton-specific models, all quarks couple to the second doublet and charged leptons to the first doublet. Here, we intend to study the VC in the context of a Type-I and Type-II N2HDM throughout a conventional dimensional regularization, which is more consistent and particularly suitable for preserving the local gauge symmetry of the underlying Lagrangian. Moreover, we argue that the leptonic contribution is negligible when compared to the dominant top and bottom quark ones, and hence type I and type IV are identical, as are type II and type III. This provides an opportunity for focusing on types I and II and derive the corresponding VC for the three CP-even Higgs bosons \(h_i\,(i=1,2,3)\).

The paper is organized as follows. In the next section, we briefly review the main features of N2HDM and present the complete set of theoretical and experimental constraints that the model is subject to. Section 3 is devoted to deriving the modified VC in N2HDM. Finally before the conclusion in Sect. 5, we present and discuss our numerical result in Sect. 4.

2 N2HDM in a nutshell

In this section, we present a review of the N2HDM. We discuss the scalar potential and derive the spectrum and the parametrization of the model. We also present the Yukawa textures and discuss the natural flavor conservation of the model. Couplings of the Higgs bosons to gauge bosons are also shown and their sum rules are discussed.

2.1 The model

The scalar sector of N2HDM consists of two weak isospin doublets \(H_{i}\) (i = 1,2), with hypercharge \(Y= 1\) and a real singlet field with hypercharge \(Y= 0\) which are given by

$$\begin{aligned} H_{i}&= \left( \begin{array}{c} \phi _i^\pm \\ \frac{1}{\sqrt{2}}(v_i + \phi _i + i \chi _i) \\ \end{array} \right) ~~\textrm{and}~~S = v_s + \phi _s. \end{aligned}$$
(1)

The most general explicit form of the Lagrangian is given by

$$\begin{aligned} {\mathcal {L}}= & {} (D_\mu H_1)^\dagger (D^\mu H_1)+(D_\mu H_2)^\dagger (D^\mu H_2)\nonumber \\{} & {} +(\partial _\mu S)^\dagger (\partial ^\mu S)-V(H_1,H_2,S) \end{aligned}$$
(2)

where, by analogy to the 2HDM, a \({\mathbb {Z}}_2\) symmetry is imposed and corresponds to the invariance of the Lagrangian under a general gauge transformation, \(H_1\), \(H_2\) and S transform as \(H_1 \rightarrow H_1\), \(H_2 \rightarrow -H_2\) and \(S \rightarrow S\). Furthermore, a mandatory second discrete \({\mathbb {Z}}_2\) symmetry is also required, ensuring the singlet distinct role in the potential and corresponds to the invariance of the Lagrangian under the simultaneous transformations

$$\begin{aligned} H_1 \rightarrow H_1, \quad H_2 \rightarrow H_2, \quad S \rightarrow -S. \end{aligned}$$
(3)

Hence, one can then write the most general renormalizable scalar potential for the model that respect \(SU(2)_L\otimes U(1)_Y\) gauge symmetry as following [34, 35]:

$$\begin{aligned} V(H_1,H_2,S)= & {} m^2_{11}\, H_1^\dagger H_1 + m^2_{22}\, H_2^\dagger H_2 \nonumber \\{} & {} - \mu _{12}^2\, \left( H_1^\dagger H_2 + H_2^\dagger H_1\right) + \frac{1}{2}m^2_S S^2 \nonumber \\{} & {} + \frac{\lambda _1}{2} \left( H_1^\dagger H_1 \right) ^2 + \frac{\lambda _2}{2} \left( H_2^\dagger H_2 \right) ^2 \nonumber \\{} & {} + \lambda _3\, H_1^\dagger H_1\, H_2^\dagger H_2 + \lambda _4\, H_1^\dagger H_2\, H_2^\dagger H_1\nonumber \\{} & {} +\frac{\lambda _5}{2} \left[ \left( H_1^\dagger H_2 \right) ^2+ \left( H_2^\dagger H_1 \right) ^2 \right] + \frac{\lambda _6}{8} S^4 \nonumber \\{} & {} +\frac{1}{2}[\lambda _7 H_1^\dagger H_1 +\lambda _8 H_2^\dagger H_2 ] S^2 \end{aligned}$$
(4)

where \(m_{11}^2, m_{22}^2\) and \(m_{S}^2\) are mass terms. In the present study, we assume that all scalar couplings \(\lambda _{i}\,(i=1,2,3,4,5,6,7,8)\) are dimensionless real parameters and similarly \(\mu _{12}^2\) (which softly breaks the first \({\mathbb {Z}}_2\) symmetry mentioned above).

Given that, once the EWSB is taking place at some electrically neutral point in the field space, six physical Higgs states are generated: three CP-even scalars (namely \(h_1\), \(h_2\) and \(h_3\) with \(m_{h_1}< m_{h_2}< m_{h_3}\)), one CP-odd scalar (A) and a charged Higgs pair (\(H^\pm \)). The squared-masses of the CP-odd and charged Higgs states do not change with respect to the 2HDM, and lead together to [36],

$$\begin{aligned} m_{H^\pm }^2 = m_{A}^2 + \frac{1}{2} v^2 (\lambda _5 - \lambda _4), \end{aligned}$$
(5)

with the corresponding squared-mass matrices can be diagonalized by the same rotation matrix

$$\begin{aligned} {\mathcal {R}}_{\beta }=\begin{pmatrix} c_\beta &{}\quad s_\beta \\ -s_\beta &{}\quad c_\beta \end{pmatrix} \end{aligned}$$
(6)

whose entries \(c_\beta \equiv \cos (\beta )\) and \(s_\beta \equiv \sin (\beta )\) define the ratio \(t_\beta \equiv \tan \beta = s_\beta /c_\beta = v_2/v_1\).

On the other hand, the CP-even neutral sector is modified compared to the 2HDM and the corresponding mass matrix can be cast into a \(3\times 3\) one as following

$$\begin{aligned} {{\mathcal {M}}}_{E}^2= \left( \begin{array}{ccc} \mu ^2t_\beta + \lambda _1v^2 c^2_\beta \, &{} \quad -\mu ^2 + \frac{\lambda _L v^2 s_{2\beta }}{2} \, &{} \quad \lambda _7vv_S c_\beta \\ -\mu ^2 + \frac{\lambda _L v^2 s_{2\beta }}{2} \, &{} \quad \mu ^2 t_\beta ^{-1} + \lambda _2v^2 s^2_\beta \, &{} \quad \lambda _8vv_S s_\beta \\ \lambda _7vv_S c_\beta \, &{} \quad \lambda _8vv_S s_\beta \, &{} \quad \lambda _6v_S^2 \end{array} \right) \nonumber \\ \end{aligned}$$
(7)

with \(\lambda _L=\lambda _3+\lambda _4+\lambda _5\). Additionally, in this context, a switch from the gauge basis \((\phi _1,\ \phi _2,\ \phi _s)\) to the mass basis \((h_1,\ h_2,\ h_3)\) is done by means of a \(3\times 3\) rotation matrix given byFootnote 1,

$$\begin{aligned} {\mathcal {R}} = \left( \begin{array}{ccc} c_1 c_2 &{}\quad s_1 c_2 &{}\quad s_2 \\ -c_1 s_2 s_3 - s_1 c_3 &{}\quad c_1 c_3 - s_1 s_2 s_3 &{}\quad c_2 s_3 \\ -c_1 s_2 c_3 + s_1 s_3 &{}\quad -c_1 s_3 - s_1 s_2 c_3 &{}\quad c_2 c_3 \end{array} \right) , \end{aligned}$$
(8)

in such a way that

$$\begin{aligned}{} & {} h_i = {\mathcal {R}}_{ij} \phi _j \quad \big [i=1,2,3 \,\wedge \, j=1,2,s\big ] \quad \text {and} \nonumber \\{} & {} {{\mathcal {M}}}_{E}^2={\mathcal {R}}_{ij} ^{T} \left( \begin{array}{ccc} m_{h_1}^2 &{} \quad 0 &{} \quad 0 \\ 0 &{}\quad m_{h_3}^2 &{}\quad 0 \\ 0 &{}\quad 0 &{} \quad m_{h_3}^2 \end{array} \right) {\mathcal {R}}_{ij}. \end{aligned}$$
(9)

2.2 Theoretical constraints

As with any extension BSM, the N2HDM parameter space must fulfill extensive theoretical requirements, which are summarized below:

\(\circ \):

Boundedness from below (BFB) of the potential [34, 35].

\(\circ \):

Perturbative unitarity [35].

For more details, we refer the reader to the Appendix, in which the explicit form of the eigenvalues at the tree level is given.

\(\circ \):

The vacuum structure: while looking closer at the aforementioned requirements, it is quite evident that breaking any of them is likely to indicate that the potential electroweak vacuum is not a minimum. Typically, various extrema may be held aside from the electroweak minimum depending upon the parameter values of the scalar potential, which inevitably had implications on the constraints on the aforementioned parameters. Nonetheless, by restricting vacuum to be global minimum, the value of the scalar potential at such electroweak minimum, \(\langle V \rangle _{\text {EWSB}}\), reads:

$$\begin{aligned} \langle V \rangle _{\text {EWSB}}= & {} \frac{1}{32} \Big [-4 v^4 c_\beta ^4 \lambda _1 - 4 v^4 s_\beta ^4 \lambda _2\nonumber \\{} & {} + v^4 \big (-1 + c_{4\beta }\big ) \lambda _L \nonumber \\{} & {} - v_S^4 \lambda _6 - 4 v^2 v_S^2 \big (c_\beta ^2 \lambda _7 + s_\beta ^2 \lambda _8\big ) \Big ] \end{aligned}$$
(10)

and, thus, spontaneous electroweak symmetry breaking would be energetically disfavored if \(\langle V \rangle _{\text {EWSB}}>0\), leading to the following requirement

$$\begin{aligned}{} & {} \lambda _1\,c_\beta ^4 + \lambda _2\,s_\beta ^4 + 2\, \lambda _L\,s_\beta ^2\,c_\beta ^2 \nonumber \\{} & {} \quad + \frac{1}{4}\,\lambda _6\,\zeta ^4 + (\lambda _7\,c_\beta ^2 + \lambda _8\,s_\beta ^2)\,\zeta ^2 > 0, \end{aligned}$$
(11)

in which \(\zeta \) stands for the ratio \(\zeta \equiv v_S/v\).

3 Veltman conditions

We address in this section how the radiative corrections to the observed 125 GeV mass either vanish or are kept under control at a manageable level within the framework of the N2HDM. Such requirements, the so-called Veltman condition (VC) [8], have been studied over the years with the pioneering work of Stuckelberg [37]. Consequently, based on both Lorentz and gauge invariant dimensional regularization method as in [18], we seek to establish relations between the coupling constants of N2HDM (mass relations) by collecting and computing the quadratic divergences and setting all of them to zero to the extent possible.

Fig. 1
figure 1

Higgs self-energy diagrams. A and B can each be one of the scalar fields S, \(H_1\) or \(H_2\), resulting in 9 possible diagrams

Full size image
Fig. 2
figure 2

Higgs bosons \(h_i\,(i=1,2,3)\) tadpole diagrams showing the contribution at one loop of: fermions (f): straight line; vector bosons (\(V=W^\pm ,\,Z\)): wiggly line; ghosts (\(\eta =\eta ^{0},\,\eta ^{\pm }\)): long-dashed line and scalars (\(S=h_1,\,h_2,\,h_3,\,A,\,H^{\pm }\)): short dashed line

Full size image

Before getting into the nitty-gritty details of how to deal with the DR scheme, it may perhaps be useful to mention that the quadratic divergences of the Higgs self-energies could be derived in the symmetry unbroken phase, in terms of the original fields. Concretely, this approach has been the subject of many studies [38,39,40,41], the results of which helped shed a little light on how canceling quadratic divergences could restrict space parameters and physical observables BSM.

In the N2HDM, a prior study has explored the dark doublet phase (DDP), where only one \(SU(2)_{L}\) Higgs doublet and the singlet exhibit non-zero VEVs. For more details, we refer the reader to [42]. Therefore, in our study, we consider all VEVs as non-zero, which could provide a more comprehensive understanding of the VC in the N2HDM. Thus, the relevant two-point functions representing the Higgs self-energy are displayed in Fig. 1.

Thereafter, in furtherance of our analysis, we will focus on the phase where the \(SU(2) \times U(1)\) gauge symmetry is broken. So, by assuming that the vacuum is CP-even, one needs to calculate the quadratic divergences that show up in the tadpoles for the three CP-even neutral Higgs of our model. Also, it is noticeable that no QCD contribution appears at one loop level, hence only the electro-weak part of the N2HDM model is concerned in this procedure. Aside from the coupling constant, we just need to consider the propagator of the field in the loop as can be seen by the topologies in Fig. 2,

where the self-energy contribution to the free propagator can be written in a simple form in terms of the Passarino–Veltman function [43] as

$$\begin{aligned} A_0(m^2) =\frac{i}{16\pi ^2} \int \frac{d^nq}{q^2 - m^2} \longrightarrow \Delta _2, \end{aligned}$$
(12)

in which \(\Delta _2\) is a pure U.V. divergent number; stands for the pole term in the chosen \(n=2\) dimensional space-time.

Table 1 The various propagator and their contributions in the N2HDM
Full size table

By assuming a \(R_{\xi }\) Feynman–’t Hooft gauge-invariant generalization, we summarize in Table 1, all the scalar, vectorial, and fermionic propagator contributions. The \((G, G^{\pm })\) and \((\eta ^0, \eta ^{\pm })\) refer, respectively, to the Goldston bosons and Faddeev–Popov ghosts, while \(T^{\mu \nu }\) and \(L^{\mu \nu }\) are the transverse and the longitudinal projectors.

We begin by deriving the three independent relations necessary to eliminate the quadratic divergences from the N2HDM model. To achieve this, we sum up all the possible diagrams, taking into account the \(-1\) for the fermionic loops (including Faddeev–Popov ghosts), the symmetry factor \(s_i\) of the diagram i, and possibly the color factor for the quarks, which we omit here for conciseness. Consequently, for each Higgs particle \(h_k\) (\(k=1,2,3\)), one obtains:

$$\begin{aligned} T_{h_k}= & {} \sum \limits _{i=1}^{9} c_i^{h_k} s_i^{h_k} t_i^{h_k} - \sum \limits _{i=U}^{D} c_{i}^{h_k} s_{i}^{h_k} t_{i}^{h_k} -\sum \limits _{i=10}^{11} c_i^{h_k} s_i^{h_k} t_i^{h_k} \nonumber \\ \end{aligned}$$
(13)

where the couplings \(c_i^{h_k}\), the symmetry factors \(s_i^{h_k}\), and the propagator loops \(t_i^{h_k}\) for each CP-even neutral Higgs boson \(h_k\) are given in Appendix B. We should note here that only the dominate top and bottom quarks contributions are considered \((m_D \rightarrow m_b\), \(m_U \rightarrow m_t)\), multiplied by the color factor.

To satisfy the VCs, the quadratic divergences of the three tadpoles \(T_{h_1}\), \(T_{h_2}\), and \(T_{h_3}\) for the \(h_1\), \(h_2\) and \(h_3\) CP-even neutral scalar fields must vanish. However, this requirement may be reformulated differently, given that the generated expressions are rather long and complicated. Indeed, by using the inverse of the matrix \({\mathcal {R}}\) mentioned above, significant benefits arise from the linear combination of the fermionic coupling constants

$$\begin{aligned} {\mathcal {R}}^{-1}_{i1} c_{f {\bar{f}}}^{h_1} + {\mathcal {R}}^{-1}_{i2} c_{f {\bar{f}}}^{h_2} + {\mathcal {R}}^{-1}_{i3} c_{f {\bar{f}}}^{h_3} =0 \quad \forall \, i=1,2,3; \end{aligned}$$
(14)

further it turns out that

$$\begin{aligned} {\mathcal {R}}^{-1}_{i1} T_{h_1} + {\mathcal {R}}^{-1}_{i2} T_{h_2} + {\mathcal {R}}^{-1}_{i3} T_{h_3} =0 \quad \forall \, i=1,2,3 \end{aligned}$$
(15)

where its is straightforward to understand that all mixing angles disappear. Hence, the cancellation of the quadratic divergences at one-loop applied to N2HDM extension leads to a set of three conditions for type-I:

$$\begin{aligned}{} & {} (6m_W^2+3m_Z^2) + v^2\left( 3 \lambda _1 + 2 \lambda _3 + \lambda _4+\frac{\lambda _7}{4}\right) = 0 \end{aligned}$$
(16)
$$\begin{aligned}{} & {} (6m_W^2+3m_Z^2) + v^2\left( 3 \lambda _2 + 2 \lambda _3 + \lambda _4+\frac{\lambda _8}{4}\right) \nonumber \\{} & {} \quad =\frac{12\big (m_t^2+m_b^2\big )}{s_\beta ^2} \end{aligned}$$
(17)
$$\begin{aligned}{} & {} \left( \frac{3}{8} \lambda _6 + \lambda _7 + \lambda _8 \right) \,v\,v_S =0 \end{aligned}$$
(18)

whereas in type-II, one can get:

$$\begin{aligned}{} & {} (6m_W^2+3m_Z^2) + v^2\left( 3 \lambda _1 + 2 \lambda _3 + \lambda _4+\frac{\lambda _7}{4}\right) = \frac{12m_b^2}{c_\beta ^2} \end{aligned}$$
(19)
$$\begin{aligned}{} & {} (6m_W^2+3m_Z^2) + v^2\left( 3 \lambda _2 + 2 \lambda _3 + \lambda _4+\frac{\lambda _8}{4}\right) =\frac{12m_t^2}{s_\beta ^2} \end{aligned}$$
(20)
$$\begin{aligned}{} & {} \left( \frac{3}{8} \lambda _6 + \lambda _7 + \lambda _8\right) \,v\,v_S =0, \end{aligned}$$
(21)

where we have introduced the SM vev,.i.e. \(v=\sqrt{v_1^2+v_2^2}=246\) GeV and the Weinberg angle \(\theta _w\) such as \(c_w= \cos {\theta _{w}}\). Moreover, these equations can be re-written in compact form in terms of the electroweak scale as well as the singlet vev either in type-I or type-II. For example, the type-II N2HDM one-loop condition of the quadratic divergences can be expressed through the following:

$$\begin{aligned}{} & {} \text {Eq}.\ (19) \rightarrow \frac{\delta T_1}{v^2} \nonumber \\{} & {} \quad = \left[ -\frac{12m_b^2}{v^2 c_\beta ^2} +\left( 3 \lambda _1 + 2 \lambda _3 + \lambda _4+\frac{\lambda _7}{4}\right) \right. \nonumber \\{} & {} \qquad \left. +\frac{3m_W^2}{v^2}\left( 2+\frac{1}{c_W^2}\right) \right] \le \epsilon \end{aligned}$$
(22)
$$\begin{aligned}{} & {} \text {Eq}.\ (20) \rightarrow \frac{\delta T_2}{v^2} \nonumber \\{} & {} \quad = \left[ -\frac{12m_t^2}{v^2 s_\beta ^2} +\left( 3 \lambda _2 + 2 \lambda _3 + \lambda _4+\frac{\lambda _8}{4}\right) \right. \nonumber \\{} & {} \qquad \left. +\frac{3m_W^2}{v^2}\left( 2+\frac{1}{c_W^2}\right) \right] \le \epsilon \end{aligned}$$
(23)
$$\begin{aligned}{} & {} \text {Eq}.\ (21) \rightarrow \frac{\delta T_3}{vv_S} = \left[ \frac{3}{8} \lambda _6 + \lambda _7 + \lambda _8 \right] \le \epsilon \end{aligned}$$
(24)

Thus, as far as \(\delta T_1\), \(\delta T_2\), and \(\delta T_3\) are simultaneously close to zero, the VCs are satisfied. Usually, however, we generally assume that the ratios \(\delta T_i/v^2\) (i=1,2) and \(\delta T_3/v/v_S\) should not exceed an upper magnitude controlled by the dimensionless parameter \(\epsilon \).

Additionally, it is pertinent to emphasize that \(\lambda _1\), \(\lambda _2\), and \(\lambda _6\) repeat only once, respectively in Eqs. (19), (20) and (21), which is obvious since they are the coupling constants of the pure \(H_{1,2}\) doublets and S singlet quartic interactions. Moreover and perhaps most importantly, the VCs in the 2HDM type-II given in Ref. [17] are easily recovered, by setting to zero, the couplings \(\lambda _6\), \(\lambda _7\) and \(\lambda _8\) in the previous formula \(T_i\), so discarding any mixing between doublets and singlet. Also, if the Yukawa couplings are neglected, the VCs for \(H_1\) and \(H_2\) are the same for all N2HDM, which confirms the results of Ref [14].

Last and not least, since all graphs contributing to the quadratic divergences of the Higgs tadpoles \(T_{h_1}\), \(T_{h_2}\), and \(T_{h_3}\) have been evaluated, the considerations of this section lead to expressing the \(\lambda \)’s couplings in terms of physical masses and the \(\mu _{12}^2\) parameter. For the type-II 2HDM, where all our subsequent calculations have been done, one can read,

$$\begin{aligned} \left\{ \begin{matrix} (19) \\ (20) \\ (21) \end{matrix}\right. \Leftrightarrow \begin{bmatrix} \delta _1 \\ \delta _2 \\ \delta _3 \end{bmatrix}= \begin{bmatrix} A_{11} &{} A_{12} &{} A_{13} \\ A_{21} &{} A_{22} &{} A_{23} \\ A_{31} &{} A_{32} &{} A_{33} \end{bmatrix}\times \begin{bmatrix} m_{h_1}^2 \\ m_{h_2}^2 \\ m_{h_3}^2 \\ \end{bmatrix} \end{aligned}$$
(25)

where the square mass dimension \(\delta _i\) (i=1,2,3) parameters that be made zero, or at least controllably small, by some symmetry, are given by

$$\begin{aligned} \delta _1= & {} \frac{12 m_b^2}{\cos \beta ^2} + \frac{\mu _{12}^2}{\sin \beta \cos \beta }(1+3\tan \beta ^2) \nonumber \\{} & {} - m_A^2 - 2 m_{H^\pm }^2 - 6 m_W^2 - 3 m_Z^2 \end{aligned}$$
(26)
$$\begin{aligned} \delta _2= & {} \frac{12 m_t^2}{\sin \beta ^2} + \frac{\mu _{12}^2}{\sin \beta \cos \beta }(1+3\cot \beta ^2) \nonumber \\{} & {} - m_A^2 - 2 m_{H^\pm }^2 - 6 m_W^2 - 3 m_Z^2 \end{aligned}$$
(27)
$$\begin{aligned} \delta _3= & {} 0 \end{aligned}$$
(28)
Table 2 The modifier couplings \(\chi _t^{h_i}\), \(\chi _d^{h_i}\) and \(\chi _V^{h_i}\) of the N2HDM Higgs bosons \(h_i\), in type II
Full size table

and the matrix elements \(A_{ij}\) read as follows

$$\begin{aligned} A_{1i}= & {} \frac{ {\mathcal {R}}_{i1} \sec \beta \Big [v {\mathcal {R}}_{i3}+8 v_S \csc \beta {\mathcal {R}}_{i2} + 12 v_S \sec \beta {\mathcal {R}}_{i1}\Big ] }{4 v_S} \nonumber \\ A_{2i}= & {} \frac{ {\mathcal {R}}_{i2} \csc \beta \Big [v {\mathcal {R}}_{i3}+12 v_S \csc \beta {\mathcal {R}}_{i2} + 8 v_S \sec \beta {\mathcal {R}}_{i1}\Big ] }{4 v_S} \nonumber \\ A_{3i}= & {} \frac{ {\mathcal {R}}_{i3} \Big [3 v {\mathcal {R}}_{i3}+8 v_S \csc \beta {\mathcal {R}}_{i2} + 8 v_S \sec \beta {\mathcal {R}}_{i1}\Big ] }{8 v_S} \end{aligned}$$
(29)

with the \({\mathcal {R}}_{ij}\) are the matrix elements in Eq. (8) while \(\sec (x),\csc (x)\) are the trigonometric functions defined respectively by the inverse of \(\sin (x),\cos (x)\).

4 Results

In the following, we present our findings assuming that \(h_3\) is almost singlet, while tacitly distinguishing between two SM-like limits: (i) the SM-like \(h_1\) scenario, in which the neutral Higgs partners, \(h_2\) and \(h_3\), are heavier (with \(m_{h_2}<m_{h_3}\)), and (ii) the \(h_2\) SM-like limit, an intermediate state between two Higgs partners, with \(h_1\) being light and \(h_3\) heavier.

Before diving into the search for solutions to VC and the corresponding analysis, we would like to point out that in the rest of this study, we confine ourselves to the type-II N2HDM, however, the same methodology remains valid for type-I. In this regard, owing to their importance in the next calculations, we provide in Table 2 all the CP-even Higgs boson modifier couplings to quarks (\(\chi _t^{h_i}\), \(\chi _b^{h_i}\)) and gauge bosons (\(\chi _V^{h_i}\)), expressed in terms of the fields admixture of the mass eigenstates \(h_i\) and the mixing angle \(\beta \). We recall here that the 2HDM limit for \(h_{1,2}\) couplings to the gauge bosons [34, 35], in both scenarios, may be achieved by setting

$$\begin{aligned}&h_1\text {-scenario}:&\quad&\beta -\alpha _1=0 \quad \textrm{and} \quad \alpha _2=0, \end{aligned}$$
(30)
$$\begin{aligned}&h_2\text {-scenario}:&\quad&\beta -\alpha _1=\frac{\pi }{2} \quad \textrm{and} \quad \alpha _3=0, \end{aligned}$$
(31)

4.1 Approximate solution for VCs

To analytically handle the main purpose within N2HDM type-II, we should look for a region where the Eqs. (19), (20) and (21) are released automatically. Towards this end, we note that the difference of \(\lambda _1\) and \(\lambda _2\) can be extracted by two alternative methods, either by using the first two diagonal elements in \({{\mathcal {M}}}_{E}^2\) using both Eqs. (7) and (9) that express \(\lambda _1-\lambda _2\) in terms of \(\tan \beta \), \(\mu _{12}^2\) and the mass of the neutral partners depending on the scenario, or by evaluating: (19)-(20). In the latest case, the \(\lambda _7\) and \(\lambda _8\) couplings must be written in terms of the parameters of the physical basis as

$$\begin{aligned} \lambda _6= & {} \frac{1}{v_S^2} \sum _{i=1}^3 m_{h_i}^2 {\mathcal {R}}_{i3}^2 \end{aligned}$$
(32)
$$\begin{aligned} \lambda _7= & {} \frac{1}{v v_S \cos \beta } \sum _{i=1}^3 m_{h_i}^2 {\mathcal {R}}_{i1} {\mathcal {R}}_{i3} \end{aligned}$$
(33)
$$\begin{aligned} \lambda _8= & {} \frac{1}{v v_S \sin \beta } \sum _{i=1}^3 m_{h_i}^2 {\mathcal {R}}_{i2} {\mathcal {R}}_{i3} . \end{aligned}$$
(34)

Combining all the above together, and assuming both limits mentioned in Eqs. (30) and (31), we obtain the following expressions for partner masses squared of the SM-like \(h_1\) or \(h_2\) particle as,

$$\begin{aligned}&\underline{h_1-\textrm{scenario}}: \nonumber \\&m_{h_2}^2=\frac{384 v_S c_3 \Big [ 2 m_t^2 - 2 m_b^2 t_\beta ^2 + \mu _{12}^2 (1-t_\beta ^4)/(2 t_\beta ) \Big ]\Big [ v_S (1-t_\beta ^2) s_3 - v t_\beta c_3 \Big ]}{ s_{23} \Big [ (v^2+48v_S^2)((1-t_\beta ^2)^2-4 t_\beta ^2)- (v^2-48v_S^2)(1+t_\beta ^2)^2) \Big ] - 192 v v_S c_{23} t_\beta (1-t_\beta ^2)}\end{aligned}$$
(35)
$$\begin{aligned}&m_{h_3}^2=\frac{384 v_S c_3 \Big [ 2 m_t^2 - 2 m_b^2 t_\beta ^2 + \mu _{12}^2 (1-t_\beta ^4)/(2 t_\beta ) \Big ] s_3 \Big [ v_S (1-t_\beta ^2) - v t_\beta t_3 \Big ]}{ s_{23} \Big [ (v^2+48v_S^2)((1-t_\beta ^2)^2-4 t_\beta ^2)- (v^2-48v_S^2)(1+t_\beta ^2)^2) \Big ] - 192 v v_S c_{23} t_\beta (1-t_\beta ^2)} \end{aligned}$$
(36)

with \(m_{h_2}^2=m_{h_2}^2(t_\beta ,\alpha _3,\mu _{12}^2)\) and \(m_{h_3}^2=m_{h_3}^2(t_\beta ,\alpha _3,\mu _{12}^2)\)

$$\begin{aligned}&\underline{h_2-\textrm{scenario}}: \nonumber \\&m_{h_1}^2=\frac{192 v_S \Big [ 2 m_t^2 - 2 m_b^2 t_\beta ^2 + \mu _{12}^2 (1-t_\beta ^4)/(2 t_\beta ) \Big ]\Big [ v_S (1-t_\beta ^2) s_{22} + 2 v t_\beta c_2^2 \Big ]}{ s_{22} \Big [ (v^2+48v_S^2)((1-t_\beta ^2)^2-4 t_\beta ^2)- (v^2-48v_S^2)(1+t_\beta ^2)^2) \Big ] + 192 v v_S c_{22} t_\beta (1-t_\beta ^2)} \end{aligned}$$
(37)
$$\begin{aligned}&m_{h_3}^2=\frac{192 v_S \Big [ 2 m_t^2 - 2 m_b^2 t_\beta ^2 + \mu _{12}^2 (1-t_\beta ^4)/(2 t_\beta ) \Big ]\Big [ v_S (1-t_\beta ^2) s_{22} - 2 v t_\beta s_2^2 \Big ]}{ s_{22} \Big [ (v^2+48v_S^2)((1-t_\beta ^2)^2-4 t_\beta ^2)- (v^2-48v_S^2)(1+t_\beta ^2)^2) \Big ] + 192 v v_S c_{22} t_\beta (1-t_\beta ^2)} \end{aligned}$$
(38)

with \(m_{h_1}^2=m_{h_1}^2(t_\beta ,\alpha _2,\mu _{12}^2)\) and \(m_{h_3}^2=m_{h_3}^2(t_\beta ,\alpha _2,\mu _{12}^2)\), whereas \(s_{2i}\) and \(c_{2i}\) stand, respectively, for \(\cos 2\alpha _i\) and \(\sin 2\alpha _i\) respectively.

An interesting fact is that these masses, for each scenario, do not depend on the \(m_{125}\), and the limit \(\alpha _3\rightarrow 0\) (resp. \(\alpha _2\rightarrow 0\)) means that very small masses are analytically devoted to \(m_{h_3}\) for \(h_1\)-scenario (resp. \(h_2\)-scenario). To better illustrate this and other related points, we have plotted, regardless of any theoretical or experimental constraints, the Higgs partner masses versus \(\tan \beta \), as given by Eqs. (35), (36), (37) and (38), for various values of \(\mu _{12}^2\) and specific \(\alpha _{2,3}\) values. Hence, by drawing a clear line at 125.09 (GeV), we are referring in fact to two completely separate areas. The first one corresponds to \(h_1-\)scenario, in which the allowed domain for \(m_{h_2}, m_{h_3}\) should be located above this line. Conversely, in the \(h_2-\)scenario, only \(m_{h_3}\) is permitted to be above the line, whereas the \({h_1}\) Higgs boson, supposed to be the lightest, must be below in the grey area.

Fig. 3
figure 3

The mass of the partner of the SM-like \(h_1(h_2)\) Higgs particle versus \(\tan \beta \) based on Eq. (25) for two value of \(\mu _{12}^2\) and fixed value of \(\alpha _2\,(\alpha _3)\) within the N2HDM type-II. 2HDM-type II limit Ref. [17] is achieved in the left panel

Full size image

The first remark to note is that the 2HDM-type II limit [17] is matched as one can see in the left side of Fig. 3, whether for the \(h_1\)-scenario, or for the \(h_2\)-scenario. For example, in the case of exact \({\mathbb {Z}}_2\) symmetry, i.e. \(\mu _{12}^2=0\) GeV\(^2\), the partner \(h_2\) can survey only for low \(\tan \beta \) with a mass above the \(t\bar{t}\) threshold in the \(h_1-\)scenario. While for the \(h_2-\)scenario, the \(h_1\) shall take place only for \(\tan \beta \ge m_t/m_b \approx 43\), measuring about \(\approx 2 m_b\) in mass and less. Once \(\mu _{12}^2\) has non-zero value, the intermediate \(\tan \beta \) open up. So, for \(h_1-\)scenario, the Higgs partner mass is large for low \(\tan \beta \le 1\), e.g \(\gtrsim 2 m_t\) and is typically spreading for high \(\tan \beta \), while for the \(h_2-\)scenario there is almost no solution for the \(h_1\) Higgs partner mass except for a unique value of \(\tan \beta \approx 2.23\).

The picture is slightly more complicated when the singlet contribution occurs. Specifically, in the \(h_1\)-scenario with \(\mu _{12}^2=0\) GeV\(^2\), a straight and narrow solution for the \(h_2\) and \(h_3\) Higgs partner masses may take place for \(1 \le \tan \beta \le 4.5\), exhibiting an average splitting of 220 GeV. Whilst for \(\mu _{12}^2=10^4\) GeV\(^2\), this range is subsequently reduced by half, but solutions for such a scenario remain possible for high values of \(\tan \beta \). Conversely, in the opposite scenario, no solution exists for \(\tan \beta \gtrsim 1\) regardless of \(\mu _{12}^2\) value. However, for large and positive \(\mu _{12}^2\), possible non-zero masses for the Higgs partner \(h_1\) and \(h_3\) are intended to be all between the curves below the dashed line in the grey area.

4.2 Experimental constraints

In addition, and for the N2HDM to be consistent with data, several experimental constraints must be met, in particular,

\(\circ \):

Electroweak precision test observables (EWPT): related to the so-called oblique parameters, S, T and U [44, 45], and may provide a strong indirect probe of NP BSM. These parameters that quantify deviations from the SM in terms of radiative corrections to the W, Z, and the photon self energies, receive new contributions in the framework of N2HDM resulting from both neutral \(h_i\), A, and charged Higgs states. In this study we perform the \(\chi ^2\) test over the allowed parameter space of N2HDM taking into account the correlation between S and T [36, 46, 47]. Our \(\chi ^2_{S,T}\) is defined as:

$$\begin{aligned} \chi ^2_{S,T}= & {} \frac{1}{\hat{\sigma }^2_{1}(1-\rho ^2)}(S - \hat{S})^2 + \frac{1}{\hat{\sigma }^2_{2}(1-\rho ^2)}(T - \hat{T})^2\nonumber \\{} & {} -\frac{2\rho }{\hat{\sigma }_{1}\hat{\sigma }_{2}(1-\rho ^2)}(S - \hat{S})(T - \hat{T}) , \end{aligned}$$
(39)

where \(\hat{S}\) and \(\hat{T}\) are the measured values of S and T, \(\hat{\sigma }_{1,2}\) are their one-sigma errors and \(\rho \) their correlation. Fixing \(U=0\), they read [48],

$$\begin{aligned} S = 0.00 \pm 0.07, \quad T = 0.05\pm 0.06, \quad \rho _{S,T} = 0.92\nonumber \\ \end{aligned}$$
(40)
\(\circ \):

Higgs data and direct collider searches: for the sake of evaluating the SM-like Higgs boson, we use the updated HiggsTools library to appropriately test Higgs searches and check the Higgs signal rate constraints in the N2HDM taking into account various LEP, Tevatron and recent LHC 13 TeV search results. So, by scanning over the appropriate model parameters, we ensure that either the mass or the coupling to gauge bosons consistently falls within the experimental boundaries.

Additionally, by focusing on scenario where \(\tan \beta \) is not so large (.i.e \(\lesssim 12\)), the mass of the charged Higgs boson is restricted to be at least approximately 580 GeV [49], and we verify whether the obtained solutions align with the constraints imposed by the EWPO oblique parameters S, T, and U taking into account the extra Higgs boson contributions in the N2HDM.

Another important point to note here is that there are no solutions emerged when considering the \(h_2\)-scenario. So, throughout the rest of our discussion, we consider only the \(h_1\)-scenario and conduct a random scan of the N2HDM parameters, adhering to the ranges below:

$$\begin{aligned}{} & {} m_{h_1} = 125\,\text {GeV},\quad m_{h_{2,3}} \in [130,1000]\,\text {GeV},\nonumber \\{} & {} m_{H^\pm } \in [580,\,1000]\,\text {GeV}, \nonumber \\{} & {} m_{A} \in [200,\,1000]\,\text {GeV},\quad \mu _{12}^2\in [0,\,10^6]\,\text {GeV}^2,\nonumber \\{} & {} v_S \in [10^2,\,10^3]\,\,\text {GeV}\nonumber \\{} & {} \frac{-\pi }{2} \le \alpha _{1,2,3} \le \frac{\pi }{2},\nonumber \\{} & {} \text {and} \quad 0.5 \le \tan \beta \le 12, \end{aligned}$$
(41)

and setting \(m_t=172.44\) GeV, \(m_b=4.18\) GeV, \(m_W=80.38\) GeV, as well as \(m_Z=91.18\) GeV. The analytic solutions for Eq. (25), derived through \(\texttt{Mathematica}\), are independently implemented into a \(\texttt{Fortran}\) program during the scan. The latter is done in a succession of cuts as follows

\(\circ \):

Cut1. We begin by performing a random scan of the parameters, applying the positivity, unitarity, and vacuum constraints from Sect. 2.2. We impose \(m_{h_2}<m_{h_3}\) to avoid the degenerate scenario between the heavier CP-even states.

\(\circ \):

Cut2. In this step, the \(\chi ^2_{S,T} \) in Eq. (39) is considered at \(2\sigma \) confidence level, and HiggsTools is applied to ensure that the remaining points align with the requirements from the Higgs boson signal strengths and heavy Higgs searches.

\(\circ \):

Cut3. Finally, we keep only the points in the parameter space that fits within the definition of VC (as the parameter \(\epsilon \) must be controllably small, we consider the cases \(\epsilon =4,6,10\)).

4.3 Scanning results

Now, after scrutinizing the N2HDM space parameter by applying the BFB and the perturbative unitarity constraints and respecting the experimental ones, we investigate to what extent the Higgs spectrum of N2HDM, specifically \(h_2\), \(h_3\), A and \(H^\pm \), could be probed through such constraints. In addition; by providing our computational analysis of Eq. (25), we specifically highlight naturalness to demonstrate the VC effect on those Higgs masses.

Fig. 4
figure 4

Outcome of the N2HDM parameter scan, incorporating theoretical constraints, bounds from Higgs signal strengths, LHC searches for additional Higgs states, and satisfaction of the Veltman conditions. Points satisfying these constraints are displayed in the planes: \([m_{A},\,m_{H^\pm }]\) (upper side), \([m_{A},\,m_{h_3}]\) (middle side), and \([m_{A},\,m_{h_2}]\) (lower side). The parameter \(\epsilon \) is set to 10, 6, and 4 from left to right. The horizontal dotted line represents the lower limit for the \(H^\pm \) mass (\(m_{H^\pm } \gtrsim 650\) GeV), derived from flavor-physics observables as outlined in Ref. [50]

Full size image

In Fig. 4, we exhibit the correlation between the CP-odd Higgs boson mass, \(m_A\) and the other scalars masses, namely: \(m_{h_2}\), \(m_{h_3}\) and \(m_{H^\pm }\). We present the parameter space regions allowed by theoretical requirements in green, and those further validated by Higgsbounds and Higgssignals in blue. The orange samples reflect the remaining points after considering the Veltman conditions for three values of the \(\epsilon \) parameter. The dotted black horizontal line in the upper panel indicates the lower limit for the \(H^\pm \) mass; i.e. \(m_{H^\pm } \gtrsim 650\) GeV, derived from flavor-physics observables [50]. It is obvious that by decreasing the \(\epsilon \) parameter, the charged Higgs mass as well as \(m_A\) get drastically restricted - for instance, for \(\epsilon =4\) and taking into account limit from B-physics - most of the orange points falling within the range of \(m_A\) (resp. \(m_{H^\pm }\)), from 700 GeV to 858 GeV (resp. from 706 GeV to 824 GeV), as can be seen from upper left panel in Fig. 4.

The other two CP-even non-SM-like neutral Higgs bosons have similar situations and are relatively heavy and strongly constrained by naturalness as can be seen from the middle and lower panels in Fig. 4. For \(h_2\) (resp. \(h_3\)), the excluded Higgs mass region is significantly extended with lower bounds around 604 GeV (resp. 776 GeV) and upper bounds reaching 809 GeV (resp. 962 GeV).

Fig. 5
figure 5

The allowed regions in the \([\tan \beta , \mu _{12}]\) plane are depicted after considering the VC. The color coding and inputs are the same as in Fig. 4

Full size image

Moreover, a similar analysis is also performed in the \([\tan \beta , \mu _{12}]\) plane, and Fig. 5 makes it clear that if naturalness induced conditions are imposed, the allowed parameter space is reduced even more and only a few points remain viable, represented in orange, as the value of \(\epsilon \) parameter decreases. More precisely, we find that \(\mu _{12}\) and \(\tan \beta \) parameters are particularly sensitive to the naturalness conditions, mainly to \(\delta {m_{h_1}}\). As a result, solutions for Eq. (25) may only occur for \(\tan \beta \) slightly below 3.2, and for positive \(\mu _{12}\), in the range \(200\sim 510\) GeV.

To delve deeper into the results obtained, notably relating to mixing angles, we highlight in Fig. 6 a scatter plot in the plane \(\tan \beta \) and \(\text {sgn}[c(h_1VV)]\times \sin (\alpha _1-\pi /2)\), quantifying the singlet contribution in the \(h_1\) SM-like Higgs boson composition, which is defined by: \(\Sigma _{h_1}=\big |{\mathcal {R}}_{13}\big |^2\). Importantly, upon incorporating the latest results from LHC Run II using HiggsTools, it becomes apparent that the parameter space of the N2HDM is constrained, and the wrong-sign regime (where the couplings of the \(h_1=h_{125}\) to the fermions and massive gauge bosons are of opposite sign each other) seems to completely disappear. At this stage, significant singlet admixtures of up to \(20\%\) are still compatible with the LHC Higgs data as can be seen from the left panel in Fig. 6.

Fig. 6
figure 6

The correct sign limit in the N2HDM as a function of the singlet admixture

Full size image

Nevertheless, the situation about respect of naturalness considerations is more intriguing. In fact, including the VC will dramatically affect the so-called correct-sign regime that begins to shrink, and obviously, smaller values for the \(\epsilon \)-parameter result in a very limited area in the space parameter of N2HDM. The plots in Fig. 6 show the corresponding generated samples. A prime example of this is reflected for \(\epsilon =4\), where only \(\tan \beta <3.2\) and \(\sin \alpha _1\) delineated by the interval \([0.73;\,0.96]\) comply with all constraints. Furthermore, as can already be inferred from the middle and left plots, the singlet admixture is mostly restricted by the quasi-annulation of the VC, down from \(20\%\) to about \(14\%\) for \(\epsilon =4\). It is beyond doubt, however, that such a proportion might be affected by any future measurements either at the HL-LHC or an electron-positron collider.

Fig. 7
figure 7

The type-II N2HDM allowed sample meeting Veltman conditions for \(\epsilon =4\), marked by ‘\(\times \)’, is overlaid on the regions of the 2HDM Model II excluded by to the measured rates of Higgs boson production and decays by ATLAS [51]. The color coding indicates for \(\cos \alpha _2\) while the x-axis represents \(\sin (\alpha _1-\beta )\) (\(\cos (\beta -\alpha )\) in the 2HDM). The solid (short dashed) lines correspond to the observed (expected) limits at \(2\sigma \). while the long-dashed vertical line indicates the SM prediction

Full size image

4.4 Benchmark datasets

In this section, we investigate the convergence of the N2HDM towards the limit of a 2HDM decoupled from the singlet field. Such a situation may occur when \(\alpha _1\) and \(\alpha _{2,3}\) approach, \(\alpha + \frac{\pi }{2}\) and 0, respectively, where \(\alpha \) represents the mixing angle used for diagonalizing the CP-even Higgs sector in the 2HDM. Consequently, the reduced couplings of the light Higgs boson, assumed to be the observed 125 GeV Higgs boson, to either \(W^+W^-\) or ZZ are perfectly aligned between the 2HDM and N2HDM. These couplings are as follows:

$$\begin{aligned} g_{h_1VV}^{\text {N2HDM}} = c_{\alpha _2} c_{\beta -\alpha _1} \xrightarrow [\text {convergence}]{} g_{hVV}^{\text {2HDM}} = s_{\beta -\alpha }. \end{aligned}$$
(42)

Hence, as in the 2HDM alignment limit: \(c_{\beta -\alpha } =0 \iff s_{\beta -\alpha } =1\), one can proceed in exactly the same way within the N2HDM, and therefore:

$$\begin{aligned} g_{h_1VV}^{\text {N2HDM}} =1 \iff s_{\beta -\alpha _1}=0 \,\, \text {or} \,\, s_{\alpha _2} =0. \end{aligned}$$
(43)

In light of these postulates, we depict in Fig. 7 a projection of the surviving samples of the type-II N2HDM, while respecting the Veltman conditions for \(\epsilon =4\), in the \(\big [\sin (\beta -\alpha _1),\,\tan \beta \big ]\) plane. Such a projection is overlaid on the expected exclusion limits at \(2\sigma \) in the type-II 2HDM hypothesis, according to the previous convergence.

As can be seen, the model outcome falls squarely within the observed range at \(2\sigma \), fully reflecting the consistency of N2HDM with the experimental measurements, together with the VC. Nonetheless, it’s worth mentioning that for all the allowed samples, \(\chi _V^{h_1}\times \chi _b^{h_1}\) is defined as positive. Therefore, only the correct-sign regime is included within the tested area, while the wrong-sign regime is typically excluded due to the alignment limits we impose.

Based on this result, we have selected five Benchmark Points (BPs), whose corresponding model inputs are given in Table 3. Additionally, we outline in Table 4 the specifications of these benchmark points for the assumed 125-GeV Higgs boson, including the relative couplings to fermions and gauge bosons, i.e., \(\chi _t^{h_1}\), \(\chi _b^{h_1}\), \(\chi _V^{h_1}\), the measured signal strengths as well as the S, T and U parameters. Thus, small \(\tan \beta \) solutions, (below 3.5) exist, leading to a consistent agreement of \(R_{\gamma \gamma }^{h_1}\) and \(R_{Z\gamma }^{h_1}\) with existing data.

Table 3 Higgs bosons masses, \(\mu \)-parameter and singlet’s vev \(v_S\) (in GeV) are shown in the \(h_1\)-scenario for various values of angles \(\alpha \)’s and \(\beta \)
Full size table
Table 4 The \(h_1\) SM-like relative couplings, decays rates and S, T and U parameters. The experimental data for \(\chi _t^{h_1}\), \(\chi _b^{h_1}\), \(\chi _V^{h_1}\) [52], \(R^{h_1}_{\gamma \gamma }\) from [53], \(R^{h_1}_{Z\gamma } \) from [54], for \(\frac{\Gamma _{h_1}^{tot}}{\Gamma _{h_1}^{{tot}_{SM}}}\) from [52] and for oblique parameters from [48] are presented
Full size table

Regarding charged and neutral Higgs boson masses, the VC solution exhibits other predictable features. It can be argued, for instance, that the two partners \(h_2\) and \(h_3\) are well controlled and their values lie respectively in \(\sim 394-812\) GeV and \(\sim 691-966\) GeV, while for the CP-odd Higgs boson, A, quasi-cancellation of quadratic divergences imperatively demands that \(497 \lesssim m_A \lesssim 858\) GeV. The charged Higgs boson, meanwhile, exhibits a lower limit moderately higher than that imposed by the B-physics constraints, while its upper bounds have slightly decreased to 957 GeV.

Another interesting feature we can see in Fig. 7 relates to the placement of our BPs in relation to the experimental best-fit value. Indeed, due to the limited sensitivity to the \(\tan \beta \), the observed best-fit values for \(c_{\beta -\alpha }\) is found to be 0.002 in Type-II 2HDM, as reported by ATLAS [51]. Hence, considering the above convergence, it’s obvious that BP3 and BP5 are close to this value and correspond to heavy \(h_3\), A and \(H^{\pm }\) with a mass splitting ranging from 53 GeV to 110 GeV. Further to this, in view of the upcoming HL-LHC, all these benchmarks match well with the expected experimental results, particularly for the di-photon and Z+photon decays, signal strengths, whose signal strengths read [55]:

$$\begin{aligned} \mu _{\gamma \gamma }^{\text {HL-LHC}} = 1\pm 0.04, \quad \mu _{Z\gamma }^{\text {HL-LHC}} = 1\pm 0.23. \end{aligned}$$

Before ending, and given the significance of selected scenario, it is worth looking at the properties of the partners \(h_2\) and \(h_3\). Tables 5 and 6 outline key properties of these heavy neutral Higgs bosons that are congruent with the cancellation of quadratic divergencies. Here, the couplings of the partners to quarks and gauge bosons are given by the SM prediction rescaled by a common factor: \(\chi _t^{h_a}\), \(\chi _b^{h_a}\) and \(\chi _V^{h_a}\) (a=2,3), evaluated with respect to couplings the would-be SM Higgs boson with the same mass as \(h_a\).

Accordingly, for \(h_2\), Table 5 makes it clear that the corresponding coupling to down-quarks is negative, which is easily explained by looking at the \(h_2 d \bar{d}\) coupling former (see Table 2), especially that for all the survived points the mixing angle \(\alpha _2\) (resp. \(\alpha _3\)) values range from \(-\)0.38 (resp. \(-\)0.9) to \(-\)0.12 (resp. 0). Meanwhile, its coupling \(\chi _V^{h_2}\) has diminished for all BPs compared to the SM value. Furthermore, significant increase of \(R^{h_2}_{\gamma \gamma }\) was observed, ranging from 0.089 to 60 times larger the SM one, contrary to what \(R^{h_2}_{Z\gamma }\) which looks suppressed overall. The last column in Table 5 indicates that the total width \(\Gamma _{h_2}^{tot}\) remains far below its SM value; and as the decay into ZZ is a nearly zero the di-photon appears to be the most promising channel in searching for such scalar. Moreover, as \(m_{h_2} < m_{A} + m_{Z}\), \(m_{H^\pm } + m_{W^\pm }\), \(2 m_{A}\) for all survived points in Fig. 7, decays such as \(h_2 \rightarrow A Z\), \(H^\pm W^\mp \), and AA are suppressed. Only the \(h_2 \rightarrow h_1 h_1\) decay channel is found to be competitive with VV, especially \(W^+W^-\), and reaches \(43\%\) for \(m_{h_2} \sim 394\) GeV before the threshold where the decay into a pair of top quarks is kinematically open and dominates at high values of \(m_{h_2}\).

Table 5 The mass, relative couplings and decays rates for the partner \(h_2\) Higgs boson
Full size table
Table 6 The mass, relative couplings and decays rates for the partner \(h_3\) Higgs boson
Full size table

And lastly, the heavier partner (\(h_3\)) exhibits roughly the same features, as can be seen from Table 6. The signal strength \(R^{h_3}_{\gamma \gamma }\) shows a significant enhancement, amounting to 81 times its SM value, in contrast to \(R^{h_3}_{Z\gamma }\) which remains suppressed overall similar to \(h_2\). Meanwhile, the reduced total width \(\Gamma _{h_3}^{tot}\) may have further improvement – the corresponding ratio ranges from 0.01 to 1.96. On the other hand, despite its relative couplings to down-quarks being modest (\(-\)1.14 up to \(-\)0.63), the \(h_3\) decay into \(b\bar{b}\) is seen to be suppressed, with the \(t\bar{t}\)-channel being dominant. Thus, from a value of \(90\%\) (\(m_{h_3} \sim 600\) GeV), the branching ratio \(Br(h_3 \rightarrow t\bar{t})\) declines towards \(10\%\) for higher values of \(m_{h_3}\). This is because the \(h_3 \rightarrow h_1 h_2\) and \(h_3 \rightarrow A Z\) channels open up, and the involved couplings, i.e, \(c_{h_1 h_2 h_3}\) and \(c_{Z A h_3}\) are non zero and even enhanced as \(\sin (\alpha _1-\beta )\) gets closer to zero. Note that the Higgs-to-Higgs decay \(h_3 \rightarrow h_1 h_2\) dominates for higher mass, \(m_{h_3} \gtrsim 851\) GeV, and may be useful as soon as \(m_{h_2} > 250\) GeV, where the \(h_2 \rightarrow h_1 h_1\) channel becomes dominant, quickly reaching a rate of \(\sim 33\%\). As a result, an intermediate \(h_1 h_1 h_1\) state can in general be sizable and still pose a challenge for the HL-LHC, given its dominant direct decay modes to SM particles.

5 Conclusion

We have explored in this study to what extent the cancellation of quadratic divergences may theoretically restrict the parameter space of the N2HDM. In line with positivity and perturbativity requirements, we have considered the \(h_1\)-scenario where the lightest CP-even Higgs boson is assumed to be the observed 125 GeV. Furthermore, we have calculated the quadratic divergent correction for the Higgs boson self-energy, e.g., \(h_1\) and its partners \(h_2\) and \(h_3\), and demanded that those quantities be made zero or at least controllably small.

Then we have selected benchmark points of the type II N2HDM parameter space in the \(\sin (\alpha _1-\beta )\) and the \(\tan \beta \) plane, where our results showed that canceling out the one-loop quadratic divergences in the Higgs mass leaves a viable and significant parameter region, conducive to probing NP, while retaining consistency with the HL-LHC predictions, as well as the oblique parameters S, T and U, at the \(2\sigma \) level.measurements

Table 7 The couplings \(c_i^{h_1}\), symmetry factor \(s_i^{h_1}\) and the propagator loops \(t_i^{h_1}\) used for \(T_{h_1}\) calculation
Full size table

Signal strength of the SM-like Higgs boson \(h_1\) and its partners \(h_2\) and \(h_3\) indicate interesting prospects within the type II N2HDM framework. While certain decay channels may be inaccessible or suppressed, our findings underscore a potential challenges for exploring physics BSM.