Search for anomalous couplings via single top quark production in association with a photon at LHC

Considering the experimental constraints given by the CMS collaboration at s=8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sqrt{s} = 8$$\end{document} TeV on the strength of top quark flavour-changing neutral-current couplings tqγ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$tq\gamma $$\end{document} and tqg, we investigate the production of top quark in association with a photon and carry out a full simulation for the signals ℓνbγ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell \nu b\gamma $$\end{document} and jjbγ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$jjb\gamma $$\end{document} at 14 TeV LHC. In our numerical analysis, the contributions of single top production with a photon radiation off the top decay products are also included. The discovery potential for anomalous couplings tqγ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$tq\gamma $$\end{document} and tqg with an integrated luminosity of 100 fb-1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{-1}$$\end{document} is examined in detail.


Introduction
The top quark is generally considered as a sensitive probe of physics beyond the standard model (BSM) [1], with its large mass close to the electroweak symmetry breaking scale. Transitions between top quarks and other quark flavours mediated by neutral gauge bosons, the flavour-changing neutral-currents (FCNC), are forbidden at tree level in the standard model (SM) and suppressed at the level of quantum loop due to the Glashow-Iliopoulos-Maiani (GIM) mechanism [2]. The branching ratios for the processes t → qγ and t → qg are of the order of 10 −14 -10 −12 in the SM [3,4]. In contrast, several BSM scenarios, such as the two-Higgs doublet model, supersymmetry or technicolour, predict much larger rates [5,6], of the order of 10 −6 -10 −5 . It implies that observation of the large FCNC-induced couplings tqγ and tqg would indicate the existence of BSM.
The enhanced FCNC tqγ and tqg interactions are predicted by many extensions of the SM which include new exotic quarks [7], new scalars [8,9], supersymmetry [5,[10][11][12][13][14], or technicolour [6,15]. The BSM effects can be described a e-mail: lgguoyuchen@126.com b e-mail: cxyue@lnnu.edu.cn by a minimal set of the higher order effective operators independently from the underlying theory [16]. The effective operators not only simplify multiple free parameters of specific models in a model-independent way, but also they order them and allow us to consistently take into account higher order quantum corrections. This method appears in many studies to search top quark FCNC . It can facilitate the analysis of new physics effects in tqγ and tqg interactions. So we can give limits on the strength of anomalous top couplings in a model-independent way. The most general effective Lagrangian can be written as L eff = −eQ tū iσ μν q ν (κ L tuγ P L + κ R tuγ P R )t A μ − eQ tc iσ μν q ν (κ L tcγ P L + κ R tcγ P R )t A μ − g sū iσ μν q ν (κ L tug P L + κ R tug P R )T a tG aμ − g sc iσ μν q ν (κ L tcg P L +κ R tcg P R )T a tG aμ +h.c., (1) where Q t is the electric charge of the top quark, g s is the strong-coupling constant, T a = λ a /2 are colour matrices, q is the momentum of the gauge boson, and P L(R) denotes the left (right)-handed projection operators. is the new physics scale, which is related to the cutoff mass scale above which the effective theory breaks down. The terms with σ μν = 1 2 [γ μ , γ ν ] are suppressed by the GIM mechanism, and in consequence are absent at tree level in renormalizable theories, like the SM. Real dimensionless parameters κ L ,R tqV are the strength of anomalous couplings tqV with V = γ, g and q = u, c.
Among FCNC top quark decays, t → qg is very difficult to distinguish from generic multijet production via quantum chromodynamics (QCD). It has therefore been suggested to search for FCNC couplings in anomalous single top quark production. The existence of anomalous couplings tqγ and tqg would induce production of a top quark in association Fig. 1 Leading order Feynman diagrams for anomalous tγ production. Flavour violation (the dots) occurs either in the weak (a, b) or strong (c, d) sector with a photon, pp → tγ . Next-to-leading order QCD predictions for this process have been studied in [24,33]. This process has been probed at the CMS experiment, as yet, with no indication of any signal. The strong bounds on the strengths of anomalous couplings have been provided by the CMS experiment with √ s = 8 TeV [40,41]. We consider the CMS limits and give the discovery potential of 14 TeV LHC.
The aim of this paper is to investigate the limits on anomalous top couplings by considering tγ production. Unlike previous studies of tγ production, we focus on an analysis of signal and backgrounds based on the CMS detector simulation. In addition, we discuss the sensitivity of 14 TeV LHC to anomalous top couplings and detection potential bounds on the tqγ and tqg couplings.
The rest of the paper is organized as follows. In Sect. 2, we provide the cross sections of tγ production with √ s = 14 TeV. The simulation of signal and the expected backgrounds is discussed in detail. At the end of this section, we discuss the contribution of single top production with a photon radiation off the top decay products. In Sect. 3, we analyze the sensitivity of 14 TeV LHC to anomalous top couplings in detail. The limits on the branching ratios of top quarks into lighter quarks and photons or gluons are given correspondingly. Finally we summarize our results in Sect. 4.

Anomalous top couplings and tγ production at LHC
In this section we first study tγ production with the on-shell top quark in the final state, and then discuss the signal and background events for two channels j jbγ and νbγ depending on the W decay mode. A realistic and detailed analysis is presented, including object identification and event selection.

tγ production
From the general effective Lagrangian L eff as shown in Eq.
(1), one can see that anomalous couplings tqγ and tqg induce the process pp → tγ . The tγ production provides the opportunity to probe both anomalous tuV and tcV couplings. Less sensitivity to tcV couplings is expected due to the large enhancement of the charm quark parton distribution function (PDF). Therefore, we consider that each type of interaction, tuγ , tcγ and tug, tcg, should be treated independently. In our analysis, we take as the top quark mass, m t = 173.2 GeV, α s = 0.108, α = 1/128.92 and a simplified scenario with κ L tqγ = κ R tqγ = κ tqγ and κ L tqg = κ R tqg = κ tqg . The presence of the anomalous couplings tqγ and tqg leads to the single top production in association with an energetic photon by two main mechanisms related to the strong and weak sector. This process can take place through the s-and t-channel. The corresponding Feynman diagrams are shown in Fig. 1. The effective cross sections σ (s) can be evaluated fromσ (ŝ) by convoluting with f q 1 / p (x 1 ) and (2) whereŝ = x 1 x 2 s is the effective center-of-mass (c. m.) energy squared for the partonic process, and x min = m 2 t /s. For the quark distribution functions f q 1 / p (x 1 ) and f q 2 / p (x 2 ), we will use the form given by the leading order parton distribution function CT14 [42].
The effective Lagrangian is implemented in FeynRules [43] and subsequently passed to Madgraph5/aMC@NLO [44] framework by means of UFO module [45]. We assume that only one anomalous top coupling is nonzero. It is worth mentioning that if the photon is collinear to the initial quark, the cross section will have a divergence. To avoid this divergence, we set a minimum transverse momentum cut on emitted photons, p γ T > 50 GeV which is adopted by the CMS collaboration. In this case, the tγ cross section at 14 TeV are Obviously, the cross sections of tγ production only depend on the strengths of anomalous top couplings κ tqγ and κ tqg .
There are many alternatives for normalisation of coupling constants in L eff . The experimental results always use branching ratios. In order to compare with them, we will show our results by using branching ratios of top quark. In our analysis, the width of t → W b is assumed to be approximately top quark total width. The LO prediction for decay width of top quark decay to a bottom quark and a W boson is [46] The cross sections of anomalous tγ production versus the FCNC branching ratios The partial widths of the top FCNC decays t → qγ and t → qg are expressed as follows: We plot the cross section of tγ production originating from different anomalous couplings tqV versus the FCNC branching ratios in Fig. 2. From Fig. 2, we can see that the cross sections of anomalous tγ production increase with the FCNC branching ratios increasing. Experimental limits on the branching ratios of the rare top quark decays were established by experiments of top production at the LEP, HERA, Tevatron, and LHC accelerators [40,41,[47][48][49][50][51][52]. At present the most stringent upper limits at 95% confidence level (CL) from different sensitive channels are shown in Table 1. Using the upper limits on the branching ratios by the CMS experiment, we obtain the limits on the cross section in Fig. 2. Then we use Eq.

Signal and background simulation
Different decay channels of the gauge boson W give different experimental signals. There are two kinds of signals, νbγ and j jbγ . We have and The leptonic mode of tγ production is in general characterized by the presence of an isolated charged leptons (electrons or muons) together with a photon, missing transverse energy, and one b-jet. The νbγ final state is more attractive from the experimental point of view. On the one hand, it is relatively efficient to be searched by experiments. Everything in the final state could be the targeted objects. They can be reconstructed efficiently by subdetector systems of LHC detectors. The lepton reconstruction efficiency with P t > 5 GeV is more than 90% and particle identification can be made at detector level. The average efficiency of single photon reaches 91%. The b-tagging algorithm could be used with the efficiency of about 70%. On the other hand, the relatively clean signal is robust against the contamination of pileups and underlying events, since primary collision vertices of the signal events can be reconstructed. Moreover, σ ( + )/σ ( − ) can be used to determine whether tγ production comes from the up quark or charm quark initiated process [26,53]. It provides the opportunity to understand the underlying new physics.
We use Madgraph5/aMC@NLO to generate signal and backgrounds events in a collision energy √ s = 14 TeV. Higher order correction is taken into account for signal  by K factor (K = σ N L O /σ L O ), which is equal to 1.8 [24]. For numerical estimation, we take coupling constants κ tuγ = 0.01, κ tcγ = 0.02, κ tug = 0.01, and κ tcg = 0.03. Parton showering and fast detector simulations are subsequently performed by PYTHIA6 [54] and Delphes3 [55]. Jets are clustered by using the anti-k t algorithm with a cone radius R = 0.7 [56]. As demonstrated in [40,41], the signal of FCNC tγ production might suffer more realistic experimental issues of fake photon and mis-tagged b jet. The probability for jets to reconstruct a single photon candidate in the electromagnetic calorimeter is about 0.1% [57], and the misidentification probability of light quarks or gluons as b-jets is approximately 1.5% [58]. Thus the light jet could be misidentified as b-jet (or photon) candidate and therefore W γ +jets events will be one of backgrounds with a fake b-jet. Similarly, W +jets will contribute to backgrounds if two jets are misidentified as an isolated photon and a b-jet simultaneously, respectively. For the background events with more than one b-jet, we take into account tt and tt + γ and three single top processes with an additional photon. The measurement accuracy of the hadronic calorimeter is not enough to distinguish the W or Z boson. Thereby, Z γ +jets process also contributes to backgrounds. For the fully hadronic final state, the overwhelming QCD multijet backgrounds are large. We consider the main QCD backgrounds 4 j and bj j j for the signal j jbγ .
The background processes can be roughly categorized into the following three types: 1. The multijets background processes include W +jets, W γ +jets, Z γ +jets. To include the QCD effects, we generate multijet events with up to two jets (three for W +jets) that are matched to the parton shower using the MLMscheme [60] with merging scale x q = 15 GeV. 2. The top processes include tt, tt + γ and three single top production with a photon processes. The reconstructed top mass distribution of signal would be similar to this type of background. The multijets and top background processes contribute to both leptonic and hadronic mode. 3. The QCD processes include 4 j and bj j j. They only affect the j jbγ final state.
The cross sections of these processes are listed in Table 2. It is worthy of remarking that W +jets in the categories above is the dominant background for the leptonic mode, which can greatly affect the significance. In order to select the most relevant events, we introduce the following preselection cuts: • To trigger the signal events, every event is required to have one isolated photon and one b-jet. Additionally, N ( ) = 1 is applied by the leptonic mode of signal, and N ( j) < 4 for hadronic mode. The number of targeted objects in each events can help to suppress the background events effectively, especially to the events with fake particles. • One of the distinctive signatures of the signal is the presence of a highp T photon in the final state. The photon is expected to carry large momentum because of the recoil against the heavy top quark. Photon candidates with significant energy are required to have transverse momentum p T ≥ 50 GeV with |η| ≤ 2.5, using the CMS coordinate system presented [59]. Additionally, only leading jet with p T > 30 GeV for hadronic mode and p t ( ) > 20 GeV with / E >30 GeV for leptonic mode are considered in our analysis (Table 3).
• The particle flow isolation R = ( η) 2 + ( φ) 2 < 0.4 around the photon candidate is applied, where η is the rapidity gap and φ is the azimuthal angle gap between the particle pair. These cuts on photon ensure the events with exactly one photon candidate. In order to have well separated physical objects and remove radiated photons from high p T leptons or final state partons, it is required that R(jet,γ ) > 0.7 and R(lepton,γ ) > 0.7.   1 and N In Fig. 3, we plot the transverse mass windows with the preselection cuts by MadAnalysis 5 [61]. The left figure shows the distribution of the transverse mass of the reconstructed W , and the right figure for reconstructed top. The W transverse mass window can reduce Z γ +jets backgrounds efficiently while affecting the signal slightly. In the right figure, we see that the transverse mass of signal and top processes show narrower peaky distributions, while the multijet backgrounds distribute in a broad transverse mass region as expected. We could use the top transverse mass window to reduce the multijet backgrounds. From Fig. 3, we require transverse mass cuts as We calculate the statistical significance S/ √ (S + B) for the luminosity of 100 fb −1 , where S and B denote the number of the signal and background events, respectively. After taking into account transverse mass widow cuts to reject backgrounds, we can further suppress background and gain in the significance up to ∼5σ , respectively, as represented in Table  4.
Similar to leptonic mode, we display m( j j) and m( j jb) distributions of j jbγ with preselection cuts in Fig. 4. In the left figure, the sharp peak of signal corresponds to the W mass, while the right part with m( j j) > 100 GeV is caused by unidentified b quark. Other top background processes represent the same distributions as well. Due to the wide distribution in signal, we claim that the m( j j) cut will not be very effective in improving the significance of the signal. The right figure shows all backgrounds distribute in a broad invariant mass region, while a sharp peaky distribution close to top mass in signal events. As a result, we require the invariant mass m( j jb) to be around the top quark mass window, From Table 5, we can find that the top mass window cut can reduce about 4/5 background events (the rejected rates even reach 85-90% for 4 j and tγ + X ), whilst the signal only loses about 1/2 events. However, the observability of the j jbγ signal with an integrated luminosity of 100 fb −1 at 14 TeV LHC is unpromising, with less than 1σ level statistical significance.      For the signal of anomalous top couplings tqg, tγ associated production is not the only contribution to j jbγ and νbγ [20]. The additional Feynman diagrams for qg → ff bγ are depicted in Fig. 5. They correspond to direct top production and photon radiation of the b quark, W boson and W decay products, respectively.   We take account of preselection cuts to examine the contributions of the radiation process. By setting basic cuts and p γ T > 15 GeV, we present the cross sections of additional Feynman diagrams with the strengths of anomalous top couplings in Fig. 6. After setting our preselection cuts, we note that this part of contributions to total cross section have a rapid decrease with the photon transverse momentum increasing. When p γ T > 5 GeV, for j jbγ ( νbγ ), the total cross section which contains the contributions of radiation processes is doubling (tripling) of the tγ production with subsequent top decay. When p γ T > 40 GeV, the contributions of additional Feynman diagrams to the cross section quickly decrease to less than 10% of tγ production with subsequent top decay. Thus we conclude that the contributions from tγ production with subsequent top decay dominate in part of p γ T > 50 GeV. As a probe to research the top quark FCNC, radiation processes could help us to further understand whether it is induced by strong interactions. In this case, it is necessary to consider not only the final states of a top quark plus a photon but also the final state particles reconstructing a top quark. Thus we present the distributions for the m( ff b) and m( ff bγ ) in Fig. 7.
The contributions from these additional Feynman diagrams could enhance the signature to help us searching for top FCNC process. The radiation processes should be considered as a part of the top FCNC signals.

Sensitivity of anomalous top couplings at 14 TeV LHC
In this section, we study the sensitivity of anomalous couplings through the tγ production at 14 TeV LHC. For an adequate signal modelling, the photon radiation from top quark decay products is taken into account. The sensitivity of four anomalous top couplings at 14 TeV LHC which is defined as S/ √ S + B (SS) are presented in Fig. 8. The SS is obtained with the selection strategy talk about in section II. In the case of assuming a single non-vanishing coupling at a time, four choices of anomalous coupling parameters are subsequently fitted by polynomial functions so that 3σ and 5σ discovery ranges are extracted. Assuming that LHC at 14 TeV could collect an integrated luminosity of 100 fb −1 , we obtain the constraints of one-dimensional discovery limits of anomalous couplings as shown in Table 6. Obviously, the overwhelming QCD multijet backgrounds make the FCNC coupling constants in hadronic mode looser. The νbγ signal is more sensitive to search for anomalous couplings at 14 TeV LHC.
Actually, we allow for a set of non-vanishing couplings simultaneously, either in the weak sector (non-vanishing κ tuγ and κ tcγ ) or in the strong sector (non-vanishing κ tug and κ tcg ). In order to illustrate excluded detection potential regions of anomalous couplings to reach a given statistical significance, we plot the associated 3σ and 5σ discovery reaches in the κ tuγ -κ tcγ (κ tug -κ tcg ) planes for νbγ at 14 TeV LHC in Fig. 9. We observe a better sensitivity to flavour-changing interactions with an up quark than with a charm quark, as expected from parton densities, the charm content of the proton being suppressed with respect to its up content.
Since both tqγ and tqg operators contribute to the same final state, the interference effects should be considered. If κ tuγ = κ tcγ = κ tqγ , κ tug = κ tcg = κ tqg , for p γ T > 50 GeV, the total tγ cross section with contributions of tqγ and tqg operators is By applying the same selection strategy described as above, we presented excluded 14 TeV LHC detection poten-  Fig.  10. Compared to existing research as regards t Z production and same-sign top quark production [27,32], our results give more sensitive constraints on top anomalous couplings via tγ production.

Conclusions
Many of the extensions of the SM predict that the treelevel FCNC processes could exist. With Run-II of the LHC, more and more measurements in the top quark sector will be explored with an unprecedented precision. Measurements of single top production allow us to search for deviations from the SM predictions. While these deviations are often interpreted in terms of anomalous top couplings. The possible deviations can be described by the effects of effective operators, and experimental results can be used to determine useful constraints on each effective operator. The established deviations can then be evolved up to high scales, and matched to possible new physics scenarios.
In this paper, we have investigated FCNC couplings tqγ and tqg in a model-independent way. These interactions lead to possibly significant production rates for j jbγ and νbγ signals via tγ associated production with subsequent decay. We have considered the contribution of the single top production with photon radiation off the top decay products to this process. Once tγ production processes are eventually discovered, the single top decay with photon radiation processes could help us find out whether or not the origin of the FCNC interactions is in the strong sector. This contribution dominates when p γ T is small, and it quickly decreases with the photon transverse momentum increasing. For our cuts, the contributions of these processes could enhance the signature, so we include these processes in signal for our analysis of the sensitivity of anomalous couplings.
The sensitivities of 14 TeV LHC to anomalous FCNC couplings tqγ and tqg were calculated for both the νbγ and the j jbγ signals. Due to the overwhelming QCD multijet backgrounds, it is challenging to discover the FCNC tγ production via the hadronic mode explored at LHC. We found that it is most promising to observe anomalous top couplings via leptonic mode of tγ production at the LHC. We further discussed the interference effects on the cross section from contributions of tqγ and tqg operators. Then we presented excluded detection potential regions for the νbγ signal. With an integrated luminosity of 100 fb −1 and √ s = 14 TeV, for a 5σ discovery, the strengths needed of tqγ and tqg couplings that are in order are down to 0.001-0.01. We hope our results could help search for the signal of anomalous top couplings at 14 TeV LHC in operation.