Enhancing thj production from top-Higgs FCNC couplings

In this paper, we study the single top and Higgs associated production pp → thj in the presence of top-Higgs FCNC couplings at the LHC. Under the current constraints, we find that the full cross section of pp → thj can be sizably enhanced in comparison with the SM predictions at 8 and 14 TeV LHC. We further explore the observability of top-Higgs FCNC couplings through pp → t(→ bℓ+νℓ)h(→ γγ)j and find that the branching ratios Br(t → qh), Br(t → uh) and Br(t → ch) can be respectively probed to 0.12%, 0.23% and 0.26% at 3σ sensitivity at 14 TeV LHC with ℒ=3000fb−1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \mathrm{\mathcal{L}}=3000\;{\mathrm{fb}}^{-1} $$\end{document}.


Introduction
The discovery of the Higgs boson at the LHC is a great triumph of the Standard Model (SM) and marks a new era in the particle physics [1,2]. Given the large uncertainties of the current Higgs data, there remains a plenty of room for new physics in Higgs sector [3,4]. 1 So the precise measurement of the Higgs boson's properties will be a dominant task at the LHC in the next decades.
Concerning the probe of new physics through the Higgs boson, the Yukawa couplings can play the important role since they are sensitive to new flavor dynamics beyond the SM. In particular, top quark, as the heaviest SM fermion, owns the strongest Yukawa coupling and has the preference to reveal the new interactions at the electroweak scale [5][6][7][8][9][10][11][12][13][14]. 2 One of the interesting things is that the top quark is just heavier than the observed Higgs boson, which makes the top quark flavor changing neutral current (FCNC) processes t → hq (q = u, c) be accessible in kinematics. In the SM, these top quark FCNC transitions are extremely suppressed by the G.I.M. mechanism [15][16][17][18]. But they can be greatly enhanced by the extended flavor structures in many new physics models, for example the minimal supersymmetric model (MSSM) with/without R-parity [19][20][21][22][23][24][25][26][27][28][29][30][31][32], two-Higgsdoublet model (2HDM) type-III [33][34][35][36][37][38][39][40][41][42][43][44][45], and the other miscellaneous models [46][47][48][49][50][51][52][53][54][55][56][57][58][59][60]. So the study of top-Higgs FCNC interactions is a common interest of the theory and experiment communities [61][62][63][64][65][66][67]. 3 However, up to now, the null results of the searches for t → qh at the LHC give the strong limits on the top-Higgs FCNC couplings. Among them, the most stringent constraint Br(t → hc) < 0.56% at 95% C.L. was reported by the CMS collaboration from a combination of the multilepton channel and the diphoton plus lepton channel [67]. Except for the widely studied t → qh decays, the importance of the single top+Higgs production pp → th in probing the top-Higgs FCNC couplings has been also emphasized in the recent theoretical studies [68][69][70][71][72][73]. 1 The recent reviews, see examples. 2 For top quark reviews, see, e.g. 3 For reviews on top FCNC processes in new physics models, see, e.g.  In this paper, we investigate the top-Higgs FCNC interactions through pp → thj with the sequent decays t → b + ν and h → γγ at the LHC. In the SM, the process pp → thj can only be induced by the weak charged current interaction and has a relative small cross section, which is about 18 (88) fb at 8 (14) TeV LHC. However, such a process is found to be very sensitive to modifications of the Higgs couplings [74][75][76][77][78][79][80][81]. In particular, the top-Higgs FCNC couplings tqh(q = u, c) can sizably enhance thj cross section due to the contributions from the strong interaction processes. To be specific, there are mainly three new kinds of processes that can contribute to the production of thj at the LHC: (1) gluon fusion gg → thj, it is the dominant contribution, as shown in figure 1, where hj can be produced not only by an on-shell top quark but also by an off-shell top quark via the new flavor changing couplings tqh; (2) qg fusion qg → thj, as shown in figure 2, which is the sub-leading contribution. However, such a process is affected by the initial Parton Distributions Functions (PDFs). So one can use this feature to disentangle the FCNC couplings of the top quark with light quarks; (3) qq annihilation qq → thj, which is similar to the s-channel of the process gg → thj but with qq instead of gg in the initial states. The contribution of this process is relatively smaller than (1) and (2) because of the suppression of color factor and PDFs; based on the above considerations, it is worthwhile to perform a complete calculation of pp → thj in the presence of the top-Higgs FCNC couplings by including the contributions (1)-(3), and explore its sensitivity to probe the top-Higgs FCNC couplings at the LHC. This paper is arranged as follows. In section 2, we set up the notations and briefly describe the top-Higgs FCNC interactions. In section 3, we discuss the observability of JHEP02(2015)061 the top-Higgs FCNC couplings through the process pp → thj at 14 TeV LHC. Finally, a summary is given in section 4.

Top-Higgs FCNC interactions
A general effective Lagrangian describing the top-Higgs FCNC interaction can be written as where h is the SM Higgs boson, and the real parameter κ L,R tqh denote the left-handed and right-handed FCNC couplings of the Higgs boson to the light up-type quarks q = u, c. We plot example Feynman diagrams in figure 1 and figure 2 for the partonic process gg → thq and qg → thg, respectively. Some diagrams of the process uū/dd → thq can be obtained by replacing the initial gluons with uū/dd in the s-channel in figure 1. By neglecting the light quark masses and assuming the dominant top decay width t → bW , the Leading Order (LO) branching ratio of t → qh can be approximately given by, .
is estimated as 10% according to the results of high order corrections to t → bW [82] and t → ch [83][84][85]. In some specific models, the left-handed coupling κ L tqh is not expected to be large because its relation with the CKM mixing parameter. Also, can be constrained by the low energy observables, such as B 0 − B 0 mixing [45,86]. However, we do not consider these indirect constraints in our study since they are model-dependent and their relevance strongly depends on the assumptions made for the generation of the quark flavor structures [87][88][89]. On the other hand, the CMS collaboration reported a model-independent bound κ L tqh 2 + κ R tqh 2 < 0.14 at 95% C.L. from the combined result of multilepton and diphoton in tt production [67], which indicates |κ L,R tqh | should be less than 0.14. In our work, we assume κ L tqh = κ R tqh = κ tqh and require κ tqh ≤ 0.1 to satisfy the direct constraint from the CMS result.

Numerical calculations and discussions
We implement the top-Higgs FCNC interactions by using the package FeynRules [90] and calcualte the LO cross section of pp → thj with MadGraph5 [91]. We use CTEQ6L as the parton distribution function (PDF) [92] and set the renormalization scale µ R and factorization scale µ F to be µ R = µ F = (m t + m h )/2. The SM input parameters are taken as follows [93]: In figure 3, we show the dependence of the cross sections σ thj on the top-Higgs FCNC couplings κ tqh at 8 and 14 TeV LHC respectively for three different cases: (I) κ tqh = κ tuh = κ tch , (II) κ tqh = κ tuh , κ tch = 0 and (III) κ tqh = κ tch , κ tuh = 0. For the three cases, the main contribution to pp → thj is from the resonant production pp → tt → thj. The non-resonant contributions are dominated by the process gq → thj. To be specific, we can have the following observations: • Case-(I ): when κ tqh = 0.1, the total cross section of pp → thj at 8 and 14 TeV LHC can be respectively enhanced up to nearly 176 and 145 times the SM predictions [76].
For the smaller values of κ tqh , the cross section will decrease and become comparable with the SM prediction when κ tqh ∼ 0.008. Here it should be mentioned that although the CMS collaboration has performed a search for thj event at √ s = 8 TeV and given a 95% upper limit on the thj cross section σ thj < 2.24 pb, this bound is not suitable for our case because a forward jet with |η| > 1.0 is required in the experimental analysis. We can also see that the full cross section of pp → thj is 1.08 (1.06) times larger than the one of pp → tt → thj at 8 (14) TeV LHC due to the contributions of the non-resonant productions of hj.
• Case-(II ) and (III ): for the same values of κ tuh and κ tch , the cross section of pp → thj in case-(II ) is much larger than that in case-(III ), since the up-quark has the larger JHEP02(2015)061 PDF than the charm-quark. This feature allows us to separately probe the couplings between κ tuh and κ tch at the LHC. So, in general, for a given collider energy and luminosity, we can expect the sensitivity to the coupling κ tuh will be better than κ tch . It should be also mentioned that the dominant contribution to pp → thj in case-(II ) and case-(III ) still come from gg-fusion process. The main difference between case-(II ) and case-(III ) lies in the contribution of qg → thj process. This makes the complete cross section of thj almost same as that of tt → thj in case-(III ) because of the small portion of c quark in the proton. To be specific, when √ s = 8 (14) TeV and κ tqh = 0.1, σ pp→thj is about 1.16(1.12) and 1.006(1.005) times larger than σ pp→tt→thj in case-(II ) and -(III ), respectively.
• Case-(I ) and (II ): we also find that the impact of qg → thj on increasing the cross section of thj production in case-(I ) is smaller than that in case-(II ). The reason is that the main production mode gg → thj in case-(I ) includes both of gg → thū and gg → thc, while in case-(II ) only the former process can contribute to gg → thj production. On the other hand, the cross section of qg → thj is almost same in case-(I ) and (II ) since it is dominated by the subprocess ug → thg.
In the following calculations, we perform the Monte Carlo simulation and explore the sensitivity of 14 TeV LHC to the top-Higgs FCNC couplings through the channel, which is characterized by two photons appearing as a narrow resonance centered around the Higgs boson mass. So the SM backgrounds to the eq. (3.2) include two parts: the resonant and the non-resonant backgrounds. For the former, they mainly come from the processes that have a Higgs boson decaying to diphoton in the final states, such as W hjj, Zhjj and tth productions. The additional jets in the W hjj/Zhjj events come from the initial or final state radiations. The cross sections of the resonant backgrounds are normalized to their NLO values; for the latter, the main background processes contain the diphoton events produced in association with top quarks, such as tjγγ and ttγγ. The W jjγγ production can also mimic the signal when one light jet is mistagged as a b jet. We generate signal and backgrounds events with MadGraph5 and perform the parton shower and the fast detector simulations with PYTHIA [94] and Delphes [95]. When generating the parton level events, we assume µ R = µ F to be the default event-by-event value. We cluster the jets by setting the anti-k t algorithm with a cone radius ∆R = 0.5 [96]. The b-jet tagging efficiency ( b ) is formulated as a function of the transverse momentum and rapidity of the jets [97]. The mis-tag of QCD jets is assumed to be the default value as in Delphes. In our simulation, we generate 100k events for the signals and backgrounds respectively.
In figure 4, we show the transverse momentum distributions of two photons in the signal with κ tqh = 0.1 and backgrounds at 14 TeV LHC. Since the two photons in the signal and the resonant backgrounds come from the Higgs boson, they have peaks around m h /2 and possess the harder p T spectrum than those in the non-resonant backgrounds. According to figure 4, we can impose the cuts p γ 1 T > 50 GeV and p γ 2 T > 25 GeV to suppress the non-resonant backgrounds. In figure 5, we present the normalized invariant mass distribution of two photons at 14 TeV LHC. Although the γγ decay channel has a small branching ratio, it has the advantage of the good resolution on the γγ resonance and is also free from the large QCD backgrounds. From figure 5, we can see that the spreading of the γγ invariant-mass peak at m h for the signal and the resonant backgrounds is relatively small. We will use a narrow invariant mass window |M γγ − M h | < 5 GeV to further reduce the non-resonant backgrounds.
In figure 6, we plot the normalized invariant mass distribution of the b jet and lepton at 14 TeV LHC, which is another effective cut to remove the backgrounds. From figure 6, we can see that the invariant mass M b 1 1 of the signal is always less than the top quark mass since the leading b jet and lepton in our signal come from the same top quark decay. The same feature also appears in the non-resonant background tjγγ. But other backgrounds can have higher invariant mass M b 1 1 than the signal. Very similar to M b 1 1 , the invariant mass distribution of the diphoton and leading light jet M γ 1 γ 2 j also has a peak around the top quark mass in the signal other than the backgrounds, which can be used to further remove the backgrounds. According to the above analysis, events are selected to satisfy the following criteria: • exact one isolated lepton with p T ( 1 ) > 20 GeV and |η 1 | < 2; • a hard jet with p T (j 1 ) > 25 GeV and |η j 1 | < 2.5 and one b-jet with p T (b 1 ) > 25 GeV and |η b 1 | < 2.5; • two photons with p γ 1 T > 50 GeV and p γ 2 T > 25 GeV and their invariant mass M γ 1 γ 2 in the range of M h ± 5 GeV; • the invariant mass of b-jet and lepton M b < 200 GeV; • the invariant mass of diphoton and leading jet M γ 1 γ 2 j 1 < 300 GeV.
In table 1, we give the cross sections of the signals in the Case-(I ), (II ) and (III ) and backgrounds after the cut flow at 14 TeV LHC, where κ tqh , κ tuh and κ tch are assumed to be 0.1 respectively. From table 1, we can see that all the non-resonant backgrounds after the cuts of the two photons are reduced by half while the signals and the resonant backgrounds are hurt slightly. Then, we impose the invariant mass cut M b 1 1 < 200 GeV to remove the backgrounds that do not involve the top quark. Since the photon final states have a good energy resolution in the detector, we require that M γ 1 γ 2 be in the range of 120 GeV < M γ 1 γ 2 < 130 GeV and M γ 1 γ 2 j 1 < 300 GeV, which can further suppress the backgrounds by half. So at the end of the cut flow, the largest background is tjγγ, which is followed by tth.
In figure 7, we plot the contours of statistical significance S/ √ B = 3σ of pp → thj at 14 TeV LHC for the Case-(I ), (II ) and (III ) in the plane of L-κ tqh . From figure 7, we can see that the flavor changing couplings κ tqh can be probed to 0.047, 0.063 and 0.065 at 3σ statistical sensitivity by fully calculating the production of thj for the case (I ), (II ) and (III ) respectively, which correspond to the branching ratios Br(t → qh) = 0.12%, Br(t → uh) = 0.23% and Br(t → ch) = 0.26% at 14 TeV LHC with L = 3000 fb −1 . Besides, the corresponding results of the resonant production pp → tt → thj for each case  Figure 7. Contour plots in L-κ tqh plane for statistical significance S/ √ B = 3σ of pp → thj at 14 TeV LHC. The conjugate processes have been included in the calculations. The cross section of tt is normalized to the approximately next-to-next-to-leading order value σ tt = 920 pb [98]. As a comparison, the corresponding results of the resonant production pp → tt → thj for each case are also displayed.  Table 1. Cut flow of the cross sections for the backgrounds and the signals in the Case-(I ), (II ) and (III ) at 14 TeV LHC, where κ tqh , κ tuh and κ tch are assumed to be 0.1 respectively and the symbol "-" stands for the events number less than one. As a comparison, the corresponding results of the resonant production pp → tt → thj for each case are also listed in the table.
are also displayed. We can see that the LHC sensitivity to the coupling κ tqh from the full calculation of thj production in the Case-(II ) can be improved by about 4% as a comparison with the resonant production pp → tt → thj, while for other two cases, the enhancement is negligible small. Here it should be mentioned that we normalize the leading order cross section of tt to the approximately next-to-next-to-leading order value σ tt = 920 pb [98]. But the contribution of qg → thj is calculated at the leading order due to lack of the high order correction. So if assuming that the k factor of the process qg → thj be the same as tt, we can expect the sensitivity to the coupling κ tqh from full calculation in Case-(I ) and (II ) will be further increased. Compared with other decay modes of the Higgs boson, our result

JHEP02(2015)061
is close to that of multi-leptons channel in tt → th(→ W W * , τ + τ − , ZZ * )j production [99], based native scaling in cross section and luminosity at 14 TeV LHC. Although the decay of h → bb has a larger branching ratio and seems more promising [73], the analysis was performed at the parton level without including the parton shower and detector effects. However, these effects are important for the Higgs mass reconstruction and can severely reduce the cut efficiency of Higgs mass window in bb channel.

Conclusion
In the work, we investigated the process pp → thj induced by the top-Higgs FCNC couplings at the LHC. We found that the cross section of pp → thj can be sizably enhanced in contrast with the SM predictions at 8 and 14 TeV LHC under the current constraints. We studied the observability of top-Higgs FCNC couplings through the process pp → t(→ b + ν )h(→ γγ)j by including the resonant and non-resonant hj production at 14 TeV LHC. Compared with the resonant production pp → tt → thj, such a full calculation can increase the LHC 3σ sensitivity to Br(t → qh) by 4% and Br(t → uh) by 10% at 14 TeV LHC with L = 3000 fb −1 because of the contribution of the non-resonant production qg → thj. Finally, the branching ratios Br(t → qh), Br(t → uh) and Br(t → ch) can be respectively probed to 0.12%, 0.23% and 0.26% at 3σ level at 14 TeV LHC with L = 3000 fb −1 .