Rare top decay and CP violation in THDM

We discuss the formalism of the two Higgs doublet model of type III with CP violation from CP-even CP-odd mixing in the neutral Higgs bosons. The flavor-changing interactions among neutral Higgs bosons and fermions are presented at tree level in this type of model. These assumptions allow the study of rare top decays mediated by a neutral Higgs boson; particularly we are interested in t→cl+l-\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t\rightarrow c l^+l^-$$\end{document}. For this process we estimate the upper bounds of the branching ratios Br(t→cτ+τ-)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathrm{Br }(t\rightarrow c \tau ^+\tau ^-)$$\end{document} of the order of 10-9∼10-7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10^{-9}\sim 10^{-7}$$\end{document} for a neutral Higgs boson mass equal to 125 GeV and tanβ=1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tan \beta =1$$\end{document}, 1.5, 2, 2.5. For the case of t→cτ+τ-\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t\rightarrow c \tau ^+\tau ^-$$\end{document} the number of possible events is estimated to range from 1 to 10 events, which could be observed in future experiments at LHC with a luminosity of 300 fb-1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {fb}^{-1}$$\end{document} and 14 TeV for the energy of the center of mass. Also we estimate that the number of events for the process t→cl+l-\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t\rightarrow c l^+l^-$$\end{document} in different scenarios is of the order of 2,500.


Introduction
The latest results from LHC have confirmed the observation of one scalar particle with a mass of the electroweak scale. The ATLAS [1] and CMS [2] collaborations have reported the observation of a new particle with mass of around 125 GeV. The observation has an important significance of more than 5 standard deviations. Even with this research it is not yet possible for us to identify this particle as the Standard Model Higgs boson. However, if this result is confirmed by future analysis, it will be one of the greatest discoveries of mankind. On the other hand, the SM is often considered as an effective theory, valid up to an energy scale of O(GeV), which eventually will be replaced by a more fundamental theory, a e-mail: rgaitan@servidor.unam.mx; rgaitan@unam.mx b e-mail: remartinezm@unal.edu.co c e-mail: josehalim@comunidad.unam.mx d e-mail: suemi@unam.mx which will explain, among other things, the physics behind electroweak symmetry breaking and perhaps even the origin of flavor. Many examples of candidate theories, which range from supersymmetry [3,4] to strongly interacting models [5] as well as some extra dimensional scenarios [6], include a multi-scalar Higgs sector. In particular, models with two scalar doublets have been studied extensively [7], as they include a rich structure with interesting phenomenology.
The first versions of the two Higgs doublet model (THDM) are known as THDM-I [8,9] and THDM-II [10]. These versions involve natural flavor conservation and CP conservation in the potential through the introduction of a discrete symmetry. A general version which is named THDM-III allows the presence of flavor-changing scalar interactions (FCNSI) at tree level [11]. There are also some variants (known as top, lepton, neutrino), where one Higgs doublet couples predominantly to one type of fermion [7], while in other models it is even possible to identify a candidate for dark matter [12,13]. The definition of all these models depends on the Yukawa structure and symmetries of the Higgs sector, whose origin is still not known. The possible appearance of new sources of CP violation is another characteristic of these models [14].
Within THDM-I only one Higgs doublet generates all gauge and fermion masses, while the second doublet only knows about this through mixing, and thus the Higgs phenomenology will share some similarities with the SM, although the SM Higgs couplings will now be shared among the neutral scalar spectrum. The presence of a charged Higgs boson is clearly a signal of physics beyond the SM. Within THDM-II one also has natural flavor conservation [15], and its phenomenology will be similar to the THDM-I, although in this case the SM couplings are shared not only because of mixing, but also because of the Yukawa structure. The distinctive characteristic of THDM-III is the presence of FCNSI, which require a certain mechanism in order to suppress them, for instance one can impose a certain texture for the Yukawa couplings [16], which will then predict a pattern of FCNSI Higgs couplings [11]. Within all those models (THDM-I,  -II, -III) [17], the Higgs doublets couple, in principle, with all  fermion families, with a strength proportional to the fermion  masses, modulo other parameters. With higher energy, as planned, the LHC will also become an amazing top factory, allowing one to test the top properties, its couplings to SM channels, and rare decays [18]. One of the interesting rare decays for the top is t → cl + l − , which is a clear signal of new physics. In the literature this type of top decay is often known as rare top decay and it could be mediated at tree level by neutral gauge bosons in the context of physics beyond SM. For instance, models with additional gauge symmetries introduce a neutral gauge boson Z , which allows the rare top decay [19][20][21]. The results obtained for branching ratios with flavor-changing neutral currents are extremely suppressed due to the mass of the additional gauge boson Z , which must be of the order of TeV. However, in the framework of the THDM-III these rare top decay are possible at tree level through neutral Higgs bosons in the framework of general THDM with upper bounds of branching ratio t → cl + l − less suppressed.
In this work we discuss the flavor-changing neutral Higgs interactions due to Yukawa couplings and a CP violation source from the Higgs sector in the framework of THDM-III. Our analysis is devoted to the study of the t → cl + l − decay at tree level with the basic goal of identifying the effects of new physics. The organization of the paper goes as follows: Sect. 2 describes the CP violation source in the Higgs sector. The flavor-changing interaction between neutral Higgs bosons and fermions are introduced in Sect. 3. Section 4 contains the analysis of the branching ratio for rare top decay. Finally, in Sect. 5 we present our conclusion and discussion.

Neutral Higgs bosons spectrum
Let Φ 1 and Φ 2 denote two complex SU (2) L doublet scalar fields with hypercharge 1. The most general gauge invariant and renormalizable Higgs scalar potential in a covariant form with respect to a global U (2) transformation is given by [22] where Φ a = φ + a , φ 0 a T and a, b, c, d are labels with respect to two dimensional Higgs flavor space. The index convention means that replacing an unbarred index with a barred index is equivalent to complex conjugation and barred-unbarred indices denote a sum. In the usual notation for the SM, the μ and λ parameters are associated with the terms (Φ † Φ) and (Φ † Φ) 2 , respectively. In general, for the THDM there are six real parameters, μ 2 11 , μ 2 22 , λ 1,...,4 , and four complex parameters, μ 2 12 , λ 5,...,7 [7], which are rewritten as the parameters Y ab and Z abcd . It is noted that Z abcd = Z cdab and hermiticity of the potential implies Y ab = (Y ba ) * and Z abcd = (Z badc ) * . The most general U (1) E M -conserving vacuum expectation values are where After spontaneous symmetry breaking, an orthogonal transformation R is used to diagonalize the squared mass matrix for the neutral Higgs fields. The mass eigenstates of the neutral Higgs bosons are where i = 1, 2, 3 and R matrix can be written and The η 1,2 denote the real parts of the complex scalar field in a weak eigenstate, φ 0 whereas η 3 is written in terms of the imaginary parts and is orthogonal to the Goldstone boson, such as η 3 = −χ 1 sin β + χ 2 cos β. The neutral Higgs bosons h i are defined to satisfy the masses hierarchy given by the inequalities m h 1 ≤ m h 2 ≤ m h 3 [23,24].

Yukawa interactions with neutral scalar-pseudoscalar mixing
Now, we will describe the interactions between fermions and neutral Higgs bosons. The most general structure of the Yukawa interactions for fermions fields can be written as follows: where a = 1, 2 and i, j = 1, 2, 3 are summed over two Higgs doublets and fermions families, respectively. The Y f a , with f = u, d, e denoting the different fermions families, are the Yukawa matrices. The q L and l L are the left handed fermion doublets; meanwhile u R , d R , and e R correspond to the right handed singlets under SU (2) L . The 0 superscript in the fermion fields stands for weak eigenstates. After getting a correct spontaneous symmetry breaking by using (2), the lagrangian for the mass is obtained in the form The mass matrices in weak eigenstates are defined as The mass eigenstates are related with weak eigenstates through the unitary matrices As a result, the Yukawa matrices cannot be diagonalized separately, giving rise to flavor-changing neutral currents. However, the linear combination is diagonalizable and defines the fermion masses as where Therefore, THDM type III contains the models type I and II plus the FCNSI terms. We solve for Y 2 in order to obtain the THDM type II as follows [25]: In order to study the rare top decay we are interested in up-quarks and charged leptons fields. By using (3), the interactions between neutral Higgs bosons and fermions can be written in the form of the THDM type II with additional contributions which arise from Yukawa couplings Y 1 and contain flavor change. In order to simplify the notation we will omit the subscript 1 in the Yukawa couplings. Explicitly we write the interactions for up-type quarks and neutral Higgs bosons as meanwhile the interactions for charged leptons and neutral Higgs bosons are The fermion spinors are denoted (u 1 , u 2 , u 3 ) = (u, c, t) and (e 1 , e 2 , e 3 ) = (e, μ, τ ). The down-type quarks are analogous to the charged lepton sector. We note that (11) and (12) generalize expressions obtained by [23][24][25][26]. The CP conserving case is obtained if only two neutral Higgs bosons are mixed with well-defined CP states, for instance for α 2 = 0 and α 3 = π/2 is the usual limit.

Rare top decay through neutral Higgs bosons
We assume that the flavor-changing neutral Higgs interactions are responsible for rare top decay at tree level. The mass of the lightest physical Higgs boson h 1 is identified with the particle observed by ATLAS and CMS with a mass value of the order of 125 GeV, meanwhile the masses of h 2,3 are considered to be in the region of higher than 600 GeV. Then contributions of physical neutral Higgs bosons h 2,3 are neglected in the amplitude for the width of rare top decay and only the contributions of the lightest neutral Higgs boson are taken into account. Therefore, the width for rare top decay at tree level is given by where and In the expression for the width of the decay (13) we have used the usual notation for dimensionless parameters, x = 2E c /m t and y = 2E l /m t . We note that m 2 h 1 can be of the same order as the square of transferred momentum, then our result is computed without approximation in the propagator. By integrating the expression (13) we can estimate the branching 1.6 10 9 3.2 10 9 4.8 10 9 6.4 10 9 8.0 10 9 ratio for t → cl + l − . We use the experimental mean value for the full width of the top quark given by Γ t ≈ 1.6 GeV and the width of the Higgs field given by Γ H ≈ 1.6 GeV [27].
Suppression for FCNC can be achieved when a certain form of the Yukawa matrices, reproducing the observed fermion masses and mixing angles, is implemented in the model. This could be done by studying a certain ansatz for the Yukawa matrices [16]. The first proposal for the Higgs boson couplings, the so called Cheng-Sher ansatz [11], was based on the Fritzsch six-texture form of the mass matrices, namely Then, by assuming that each Yukawa matrix has the same hierarchy, one finds that A ≈ m 3 , B ≈ √ m 2 m 3 and C ≈ √ m 1 m 2 . Thus, if the structure is assumed to be based on the Cheng-Sher ansatz, then the Yukawa couplings obey the following pattern: Therefore, the resulting branching ratio only has dependence on α 1 and α 2 . The α 3 mixing angle is absent in the physical state for h 1 . The allowed regions for the α 1 -α 2 parameter space are obtained through the bounds of R γ γ , defined by For a charged Higgs boson with mass of the order of 100-300 GeV, Br(h 1 → γ γ ) contains an important contribution from the charged Higgs boson at one loop level, which affects the allowed regions for α 1 −α 2 . Thus, it is possible to find allowed values in the α 1 -α 2 parameter space if the parameters β and m H ± are fixed. A process used to set tan β and charged Higgs boson mass is, for instance, the flavor-changing process B → χ s γ [28], which receives a contribution from THDM through the charged Higgs boson. This contribution is comparable to the contribution of W ± from SM. For small values of tan β this process gives a bound to the charged Higgs boson mass of the order of 300 GeV [29,30]. Contributions from other processes such as B τ → τ ν τ , B → Dτ ν τ , Z →bb, B d,s → μ + μ− and B 0 -B 0 set bounds for the mass of H ± and tan β as m H ± < 400 GeV and tan β ≤ 10. Therefore, the allowed regions for the α 1,2 parameter space are obtained by experimental and theoretical constraints in the framework of the THDM type II with CP violation for fixed tan β and m H ± . For 0.5 ≤ R γ γ ≤ 2, m H ± = 300 GeV and tan β = 1, the α 1 -α 2 regions are [24] R 1 = {0.67 ≤ α 1 ≤ 0.8 and 0 ≤ α 2 ≤ 0.23} (18) and R 2 = {0.8 ≤ α 1 ≤ 1.14 and − 0.25 ≤ α 2 ≤ 0}.
For the same settings but with m H ± = 500 GeV, In order to reduce the α 1 -α 2 parameter space we consider these regions as an approximation. In addition, we will assume that τ + and τ − occur in the final state. Figure 1 shows the branching ratio of rare top decay for regions R 1 and R 2 ; meanwhile Fig. 2 is obtained for R 3 . For 1 ≤ R γ γ ≤ 2, m H ± = 350 GeV, and tan β = 1.5 the allowed parameter regions in the α 1 -α 2 plane in the framework of THDM with a potential but softly broken Z 2 discrete symmetry are [31] and R 5 = {0.93 ≤ α 1 ≤ 1.57 and −0.61 ≤ α 2 ≤ 0}.
For tan β = 2 the regions are Finally, for tan β = 2.5 the region is and Figures 3, 4, and 5 show the branching ratio for previous regions. We note that the branching ratio of rare top decay for tan β = 1 and m H ± = 500 GeV is bounded as Br(t → cτ + τ − ) ≤ 5 × 10 −7 for any α 1,2 . For a μ + and μ − pair in the final state we find that Br(t → cμ + μ − ) ≤ 1.9 × 10 −9 with the same tan β = 1. If the mixing angle β is fixed with values greater than tan β = 1, the branching ratio does not vary drastically over the whole α 1 -α 2 region; for instance if tan β = 45, then Br(t → cτ + τ − ) ≤ 2.8×10 −7 . Table 1 contains the upper bounds for the regions considered.

Discussion and conclusion
From 2015 to 2017 the experiment is expected to reach 100 fb −1 of data with an energy of the center of mass of 14 TeV. In the year 2021 one expects to reach a luminosity of the order of 300 fb −1 of data. Experiments with this luminosity could find evidence of new physics beyond SM. Then Run 3 in LHC could observe events for the neutral flavorchanging process such that t → ch → cl + l − , which can be explained in a naive form as Then we estimate the number of events using the upper bound for a branching ratio with σ ( pp → tt) ≈ 176 pb [32]. Table 1 contains this estimation for the considered regions. Finally, we compare our result with reported results in other frameworks, such as effective theories and THDM type I or II. Based on (11) we can write the branching ratio for t → ch 1 as where λ is the usual function. We find that Br(t → ch 1 ) ≤ 5 × 10 −3 with m h 1 = 125 GeV and tan β = 1. Despite the absence of flavor-changing neutral Higgs interactions in SM, t → ch SM decay can occur at one loop level. The reported result for the branching ratio is of the order of 10 −14 −10 −13 for m Z ≤ m SM ≤ 2m Z [33]. More recently, in the framework of the general THDM with CP-even (H 0 ) and CPodd (A 0 ) neutral Higgs bosons the branching ratios are estimated as Br(t → cH 0 ) = 2.2 × 10 −3 and Br(t → c A 0 ) = 1.2 × 10 −4 for m H 0 = 125 GeV and m A 0 = 150 GeV [34]. By using the effective operator formalism the flavor changing neutral Higgs interactions are introduced. An upper bound is estimated as Br(t → cH 0 ) = 2.7 % for a neutral Higgs mass of 125 GeV [35]. Top decay with effective theories is also studied, for the case of t → ch Br(t → cH 0 ) = 5 × 10 −3 for m h = 125 GeV is obtained [36]. In reference [37] has been estimated an upper bound of Br(t → cH) = 0.09 − 2.8 × 10 −3 for 114 ≤ m H ≤ 170 GeV through the one loop contributions of effective flavor-changing neutral couplings tcH on the electroweak precision observables in SM. For the Yukawa complex couplings and CP effects in THDM type III Br(t → cH 0 ) ≈ 10 −3 is predicted by [38]. From reference [31], Fig. 3, can be estimated the branching ratio of h 1 into τ s, which is of the order of BR(h 1 → τ τ ) ≈ 0.05 for any value of α 1 and α 2 . Using this BR and taking into account BR(t → ch 1 ) ≤ 10 −3 for different scenarios of the models, we obtain BR(t → ch 1 → cτ τ ) ≈ 5 × 10 −5 , which is two orders of magnitude larger than the value obtained by us for different regions of parameters; see Table 1. The number of events, in the best scenario, at LHC with 300 fb −1 of luminosity and 14 TeV for the energy of the center of mass is of the order of 2,500.