Physical constraints derived from FCNC in the 3-3-1-1 model

We investigate several phenomena related to FCNCs in the $\text{3-3-1-1}$ model. The sources of FCNCs at the tree-level from both the gauge and Higgs sectors are clarified. Experiments on the oscillation of mesons most stringently constrain the tree-level FCNCs. The lower bound on the new physics scale is imposed more tightly than in the previous, $\text{M}_{\text{new}}>12 $ \text{TeV}. Under this bound, the tree-level FCNCs make a negligible contribution to the $\text{Br}(B_s \rightarrow \mu^+ \mu^-)$, $\text{Br}(B \rightarrow K^{*} \mu^+ \mu^-)$ and $\text{Br}(B^{+}\rightarrow K^{+}\mu^{+}\mu^{-})$. The branching ratio of radiative decay $b \rightarrow s \gamma$ is enhanced by the ratio $\frac{v}{u}$ via diagrams with the charged Higgs mediation. In contrast, the charged currents of new gauge bosons significantly contribute to the decay process $\mu \rightarrow e \gamma$.

flavor violation (LFV) processes l i → l j γ and b → sγ decay.
We organize our paper as follows. In Sec. II, we briefly overview the 3-3-1-1 model. In Sec III, we describe the tree-level FCNCs and study their effects on the mass difference of mesons. We predict the NP contributions to the rare decays of B s → µ + µ − , B → K * µ + µ − and B + → K + µ + µ − processes based on the constrained parameter space. Sec. IV studies the one-loop calculation of the relevant Feynman diagrams, which relate to the b → sγ and µ → eγ. The consequences of the parameters on the branching ratio of these decays are implied from the experimental data studied.
Our conclusions are given in Sec.V.

A. Symmetry and particle content
The gauge symmetry of the model is SU the color group, SU (3) L is an extension of the SU (2) L weak-isospin, and U (1) X , U (1) N define the electric charge Q and B − L operators [36] as follows where β, β are coefficients, and both are free from anomalies. The parameters β, β determine the Q and B − L charges of new particles. In this work, we consider the model with β = − 1 √ 3 . This is the simple 3-3-1-1 model for dark matter [31].
The electrically-neutral scalars can develop vacuum expectation values (VEVs) and break the symmetry of model via the following scheme where P is understood as the matter parity (W-parity) and takes the form: P = (−1) 3(B−L)+2s .
All SM particles have W-parity of +1 (called even W-particle) while new fermions have W-parity of −1 (called odd W-particle). With W-parity preserved, the lightest odd W-particle can not decay.
If the lightest particle has a neutral charge, it may account for dark matter (see [31]). The VEVs, u, v, break the electroweak symmetry and generate the mass for SM particles with the consistent condition: u 2 + v 2 = 246 2 GeV 2 . The VEVs, w, Λ, break SU (3) L , U (1) N groups and generate the mass for new particles. For consistency, we assume w, Λ u, v.
• One neutral CP-odd particle • Two charged fields that are given as follows For the odd W-particle spectrum, there exists a complex scalar particle For convenience, we list a few mass expressions for the physical fields that we will use for the calculations below

C. Fermion masses
The Yukawa interactions in the quark sector are written in [31] as follows After symmetry breaking, the up-quarks and down-quarks receive mass. Their mixing mass matrices have the following form In the general case, these matrices are not flavor-diagonal. They can be diagonalized by the unitary It means that the mass eigenstates relate to the flavor states by The CKM matrix is defined as The Yukawa interactions for leptons are written by The charged leptons have a Dirac mass [ The flavor states e a are related to the physical states e a by using two unitary matrices U l L,R as The neutrinos have both Dirac and Majorana mass terms. In the flavor states, n L = (ν L , ν c R ) T , the neutrino mass terms can be written as follows The mass eigenstates n L are related to the neutrino flavor states as n L = U ν † n L , where U ν is a 6 × 6 matrix and written in terms of The new neutral fermions N a are a Majorana field, and they obtain their mass via effective interactions [32,33]. We suppose that the flavor states N a relate to the mass eigenstates N a by using the unitary matrices U N L,R as

D. Gauge bosons
Let us review the characteristics of the gauge sector. In addition to the SM gauge bosons, the 3-3-1-1 model also predicts six new gauge bosons: X 0,0 * , Y ± , Z 2 , Z N . The gauge bosons are even W-parity except for the X, Y gauge bosons that carry odd W-parity. The masses of new gauge bosons have been given in [32], [33] as III.

A. Meson mixing at tree level
In previous works [32], [36], the authors have considered the FCNCs that couple to the new neutral gauge bosons Z 2 and Z N at tree-level. Due to the different arrangements between generations of quarks, the SM quarks couple to two Higgs triplets. Therefore, there exist FCNCs coupled to the new neutral Higgs bosons at tree-level. These interactions derive from the Yukawa Lagrangian (11). After rotating to the physical basis via using Eqs. (12), (13), (14), we obtain the following where t β = tan β = v u , and Γ u , Γ d are defined as: The first three terms of Eq. The Lagrangian of tree-level FCNCs mediated by Z 2 , Z N , which has been studied in [32], has the following form where ξ is a mixing angle that is determined by tan 2ξ = We now investigate the impact of FCNCs associated with both new gauge and scalar bosons on the oscillation of mesons. From FCNCs given in Eqs. (22)- (24), we obtain the effective Lagrangian that affects the meson mixing as with q denoting either u or d quark. This Lagrangian gives contributions to the mass difference of the meson systems as given We would like to remind the reader that the theoretical predictions of the meson mass differences account for both SM and all tree-level contributions. It hints that meson mass differences can be separated as where the SM contributions to the meson mass differences are given by [37], [38] ( The theoretical predictions, given in Eq. (28), are compared with the experimental values as given in [39], [40] (∆m K ) exp = 0.5293(9) × 10 −2 /ps, However, due to the long-distance effect in ∆m K , the uncertainties in this system are considerable.
Therefore, we require the theory to produce the data for the kaon mass difference within 30%, The SM predictions for B-meson mass difference are more accurate than those of kaon, and we have the following constraints by combining quadrature of the relative errors in the SM predictions and measurements [41] 0 or equivalently Let us do a numerical study from a set of all the input parameters that are taken by [40,[42][43][44][45] All mass parameters are in MeV. Besides, we assume t N = 1, g = √ 4πα/s W , where α = 1/128 and s 2 W = 0.231. The mixing matrix for right-handed quarks, V uR , is a unitary matrix, whereas V dR is parameterized by three mixing angles, θ R 12 , θ R 13 and θ R 23 , as where For instance, we can choose θ R 12 = π/6, θ R 13 = π/4 and θ R 23 = π/3. The NP scales require the following constraints w ∼ Λ ∼ −f u, v, due to the condition of diagonalization for the mixing mass matrices in [32]. We first study the role of FCNCs coupled to the scalar fields, H 1 , A, in meson mixing parameters.
To see its effect, we change the f -parameter, which only affects the masses of the H 1 , A (see in Eq. That is, the mixing parameters are affected slightly by FCNCs coupled to the scalar fields.
Next, we consider the contributions of FCNCs coupled to new gauge bosons to the meson mixing parameters. To estimate how important they are, we compare their contributions with those of the new scalar bosons. The ratio of these two contributions is presented in Fig. 2. The results show that the significant contribution comes from the FCNCs of new gauge bosons. It once again clarifies the small effect of the new scalar fields on the meson mixing systems.
Finally, we investigate the constraints on the VEVs from ∆m K,Bs,B d . In Fig.1, the allowed region of parameters that satisfies the constraints given in Eqs. (31), (33) is the green one. The electroweak symmetry breaking energy scale, u, is not constrained by conditions imposed on the meson mass mixing parameters. However, these conditions affect the NP scale w. From Fig. 1, we obtain a lower bound on the NP scale, w > 12 TeV. This lower bound is more stringent and is remarkably larger than that obtained previously [32]. This difference is because, in the previous study, the authors compared the NP contributions with experimental values and ignored the SM contributions to the theoretical predictions. Moreover, Eq. (131) in [32], the authors used .2871/ps, the upper limit for (∆m Bs ) NP is even greater than that of the experimental value given in Eq. (30). This is not reasonable because the theoretical prediction must consist of both SM and NP contributions. We must also consider the uncertainties of both SM and experimental predictions. Thus, the NP contributions have to be constrained by the conditions given in Eqs. (31,33).
Rare decays of B meson, in particular of the decay induced by the quark level transition, where M lD = Diag(m e , m µ , m τ ). It is worth noting that there is no neutral Higgs mediated FCNC in the lepton sector. The interactions of Z 2 and Z N with two charged leptons have been written where the form of coefficients [31].
Combining the quark FCNCs and the LFCNCs, we obtain the effective Hamiltonian for B s → µ + µ − , B → K * µ + µ − and B + → K + µ + µ − processes as follows where the operators are defined by The operators O 9,10,S,P are obtained from O 9,10,S,P by replacing P L ↔ P R . Their Wilson coefficients consist of the SM leading and tree-level NP contributions. For C 9,10 we split into the SM and NP contributions as: C 9,10 = C SM 9,10 + C NP 9,10 , where the central points of C SM 9,10 are given in [46], C SM 10 = −4.198, C SM 9 = 4.344, and the C NP 9,10,S,P are written by Noting that C SM S,P = C SM S,P = 0. Therefore, the C S,P , C S,P are obtained by NP contributions as follows where Γ l αα = ∆ l αa = u v m lα . From the effective Hamiltonian given in (38), we obtain the branching ratio of the where τ Bs is the total lifetime of the B s meson. If including the effect of oscillations in the B s −B s system, the theoretical and experimental results are related by [47] Br where y s = ∆Γ Bs 2Γ Bs = 0.0645(3) [39]. For B s → e + e − , the SM prediction [48] is and the experimental bound has been given in [49] as The SM contribution to the branching ratio of B s → e + e − is strongly suppressed to the current experimental upper bound. It may be an excellent place to look for NP. Completely contrary to B s → e + e − , the very recent measurement of the branching ratio (B s → µ + µ − ) is given by [7] Br(B s → µ + µ − ) exp = (3.09 +0.46 +0.15 −0.43 −0.11 ) × 10 −9 .
This experimental upper bound closes to the central value of the SM prediction (including the effect of B s −B s oscillations) that has been studied in [50] Br B s → µ + µ − SM = (3.66 ± 0.14) × 10 −9 .
It shows that experimental results are in slight tension with the SM prediction of Br(B s → µ + µ − ).
NP effects in B s → µ + µ − lead to new stringent constraints on NP scale. Let us concentrate on the numerical study of B s → µ + µ − . In the right panel of Fig. 3, we draw the NP contributions to each Wilson coefficient. Compared to the C NP 9,10 , the C S,P are further suppressed by a factor of 10 −4 ÷ 10 −5 . So, the main contribution of the NP to the Br(B s → µ + µ − ) comes from the C NP 10 . In the limit w > 12 TeV, the C NP 10 is positive. It causes the Br(B s → µ + µ − ) reduced about 5% , which brings the theoretical prediction and experimental values get closer together.
If the C NP 10 affects the decay process B s → µ + µ − , the C NP in the 3-3-1-1 model. In the limit, w > 12 TeV, we obtain its maximal prediction value C NP 9 −0.01. So, the NP coming from the 3-3-1-1 model can not explain the anomalies of B → K * µ + µ − process.
The measurements of the branching fraction of the decay B + → K + µ + µ − [23,24] have turned out to be slightly on the low side compared to SM expectations. Both the C 9 , C 10 contribute to the Br (B + → K + µ + µ − ). As predicted by the 3-3-1-1 model, the NP contribution to these parameters is minimal (see Fig. 3) because the NP scale satisfies the constraint w > 12 TeV. Both the C NP 9 and C NP 10 are too low and far from the values of global analysis, see in [51][52][53][54]. Thus, we believe that the NP effects in B + → K + µ + µ − remain small in the 3-3-1-1 model.

IV. RADIATIVE PROCESSES
The branching fraction and the photon energy spectrum of the radiative penguin b → sγ process have been firstly reported by CLEO experiment, Br(b → sγ) = (3.21 ± 0.43 ± 0.27 +0. 18 −0.10 ) × 10 −4 [8]. Recently, HFLAV group has obtained the average result by combining the measurements from CLEO, BaBar and Belle, Br(b → sγ) = (3.32 ± 0.15) × 10 −4 [39] for a photon-energy cut-off E γ > 1. 6 GeV. This result is in good agreement with the SM prediction up to Next-to-Next-to-Leading Order (NNLO) Br(b → sγ) = (3.36 ± 0.23) × 10 −4 [59], [60], with the same energy cut-off E γ . It suggests that the NP contributions to this process, if any, have to be small. Thus, studying the b → sγ decay can give a strong constraint on the NP scale. The radiative process b → sγ is most conveniently described in the framework of an effective theory that arises after decoupling of new particles. Excluding the charged currents associated with the W ± µ gauge boson, the 3-3-1-1 model contains new charged currents, which couple to the new charged gauge bosons Y ± µ , two charged Higgs bosons H ± 4 , H ± 5 , and the FCNCs coupled to the Z 2,N as given in Eq. (24). All of the above currents generate the b → sγ process.
Let us write down the charged scalar currents related to b → sγ. The H ± 4 only couples to the exotic quarks, so it does not create the flavor-changing charged currents (FCCCs) for SM quarks.
While H ± 5 couples to the SM quarks and creates the scalar FCCCs. The relevant Lagrangian is where , s 2β = sin 2β, t 2β = tan 2β. The charged currents associated with the W ± , Y ± , are described by the V-A currents as follows The effective Hamiltonian for the decay b → sγ is  split as the sum of the SM and 3-3-1-1 contributions Note that the Wilson coefficients C 7,8 will be ignored in our calculation since they are suppressed by the ratio m s /m b . The SM Wilson coefficients C SM 7,8 at the scale µ ∼ m W are first given by [61] C SM(0) 7 where the index 0 indicates that the Wilson coefficients are calculated without QCD correction.
The NP contributes to C NP 7,8 at the quantum level via the higher order charged current interactions in Eqs. (49), (50) and the FCNCs given in Eq. (24). They can be split into each contribution as follows where with all functions f γ,g and f γ,g are defined as shown below The C Z 2,N (0) 7 (m Z 2,N ) are obtained by the FCNCs coupled to the Z 2,N and have a form as given in with are given by For w = 10 TeV, we have m Y 3.2 TeV, and obtain C have the form as [63], The branching ratio Br(b → sγ) is given as where N (E γ ) = 3.6(6)×10 −3 is a non-perturbative contribution, ueν e ) = 0.580(16) [62] and branching ratio for semi-leptonic decay Br(b → ceν e ) = 0.1086(35) [40].
Other parameters are input as in Sec. III A.
The Br(b → sγ) behaves as a function of the new particle masses, such as m Y , m H 5 , m U . These masses are understood as free parameters. In the limit, u, v −f u 2 +v 2 uv ∼ w ∼ Λ, they can be rewritten as where, g = 4πα/s 2 .
In Fig. 4, we show the dependence of Br(b → sγ) on the NP scale w in the limit u, v −f u 2 +v 2 uv ∼ w ∼ Λ. Each panel corresponds to the scenarios of mass hierarchy and three different choices of t β . We see that the branching ratio strongly depends on the values of t β where the term containing t β comes from C H 5 7 . So we conclude that C H 5 7 plays an important role in the radiative decay process b → sγ. This is true for all three scenarios of the mass hierarchy. Besides, Fig.   4 indicates that the mass hierarchy does not affect Br(b → sγ) much. This result is understood as the main contribution coming from C H 5 7 , and it is stronger than other contributions by the coefficient t 2 β . In the large t β limit, the Br The lower bound on the NP scale depends on the value of the t β , specifically, w ≥ 1 TeV for t β = 1; w ≥ 4.1 TeV for t β = 10; w ≥ 7.7 TeV for t β = 20. These limits are weaker than the ones mentioned above.
To close this section, we consider the influence of NP on the Br(b → sγ) in the limit u, v −f ∼ w ∼ Λ. In Fig. 5, we see that the dependence of branching ratio on t β is not as strong as predicted in .

B. Charged lepton flavor violation
The charged lepton flavor violation (CLFV) processes are strongly suppressed in the SM with right-handed neutrinos, Br(l i → l j γ) 10 −55 . Meanwhile, the current experimental bounds limits are given as [40] Br(µ − → e − γ) < 4.2 × 10 −13 , It implies that the CLFV processes open a large window for studying the NP signals beyond the SM. Note that in the SM with right-handed neutrinos, the decay processes, l i → l j γ, come from the one-loop level with W ± mediated in the loop. The Br(l i → l j γ) is suppressed due to the mixing matrix elements of the neutrinos. The 3-3-1-1 model anticipates the existence of additional charged currents associated with the new charged particles, Y ± , H ± 4,5 . Consequently, the new oneloop diagrams in the model may contribute significantly to the Br(l i → l j γ). This branching ratio may reach the upper experimental bound given in Eq. (64). In order to study the CLFV processes, we first write down the relevant Lagrangian based on the physical states as follows The charged currents associated with the new gauge bosons are written in the physical states as follows L lepton Next, we write the effective Lagrangian relevant for the µ → eγ processes in the traditional form where the factors A L , A R are obtained by calculating all the one-loop diagrams. We use the 't Hooft-Feynman gauge and keep the external lepton masses for calculations. The obtained results are inspired by [66]. The factors A L,R are divided into individual contributions, as shown below where The functions f (x) and g(x) are defined by The notations m ν j , M ν j , m e , m µ are understood as the masses of light, heavy neutrinos, electron, and muon, respectively. From the effective Lagrangian (66), we finally got the branching ratio Br(µ → eγ) as follows where is the Fermi coupling constant, Br(µ → eν e ν µ ) = 100% as given in [40].
Before considering numerical calculations of the branching ratio Br(µ → eγ), let us make some assumptions. We assume that a diagonal matrix presents the Yukawa couplings h e ab in the flavor basis. Thus, the matrix U ν L is identified as the PMNS matrix U PMNS , which has been measured experimentally. Both the mixing matrices U ν R , V ν as well as U N L,R are new and not constrained by experiments. To simplify, we suppose that the Yukawa couplings of the right-handed neutrinos h ν are presented by a diagonal matrix. This indicates that the Majorana neutrino mass matrix has the form as M ν R = Diag(M ν 1 , M ν 2 , M ν 3 ) and thus the right-handed neutrino mixing mass matrix U ν R is a unit matrix. The mixing matrix V ν is also assumed to be diagonal. Finally, for the mixing matrix of the new leptons U N R , we can use three arbitrary angles θ N ij , (i, j = 1, 2, 3) and a Dirac CP phase δ N to parameterize.
With the above option, the Yukawa couplings h e , h ν can be translated into the charged lepton and sterile neutrino masses as follows The Yukawa couplings h ν , which determine the neutrino Dirac mass, are rewritten by using Casas-Ibarra parametrization as given in [65] h ν = where R is an orthogonal matrix which is presented via arbitrary angles as the following For the magnitudes of relevant masses and the VEVs, we also work on the limits u, v w ∼ Λ, To be consistent with the unitary bound [67], we need the constraint: where θ ij are the mixing angles of the neutrino mixing matrix.
In addition, the branching ratio Br(µ → eγ) also depends on the unknown parameters, such as six mixing angles (θ ij , θ N ij ), one CP phase δ N , the masses of new particles m N , M ν i . In the following, we are going to present the results of numerical calculations for the case where unknown parameters are chosen as θ N 12 = π/6, θ N 13 = π/3, θ N 23 = π/4, δ N = 0, The NP scale is strongly constrained by the experimental bounds on mixing mass parameters. We have obtained the lower bound on the new gauge boson mass M new > 12 TeV, which is more stringent than the constraint previously given in [32]. This change is because previous studies omitted the contributions of new Higgs, especially those of the SM. Our result is consistent with that of [68]. We also studied the tree-level FCNCs affecting the branching ratio of B s → µ + µ − , B → K * µ + µ − and B + → K + µ + µ − . In the parameter region consistent with the experimental constraints on the meson mass difference, the tree-level FCNCs give small contributions to these branching ratios, which is consistent with the measurement B s → µ + µ − [4-7] but can not explain the B → K * µ + µ − and B + → K + µ + µ − anomalies [16][17][18][19][20][21][22][23][24].
For the radiative decay processes, we concentrated on the flavor-changing b → sγ decay. The large contribution arises from the Wilson coefficient C H 5 7 yielded from one-loop diagrams with the new charged Higgs boson mediation. In spite of the enhanced contributions due to the factor t β = v/u, the predicted branching ratio Br(b → sγ) is consistent with the measurement [39], if M new is chosen as above mentioned. In contrast to the b → sγ decay, the branching ratio of the lepton flavor-violating µ → eγ decay obtains a large contribution from one-loop diagrams with new gauge bosons exchange. Due to the large mixing of new neutral leptons, the branching ratio Br(µ → eγ) can reach the experimental upper bound.