Exploring the top-Higgs FCNC couplings at polarized linear colliders with top spin observables

We study the nature of flavor changing neutral couplings of the top quark with the Higgs boson and the up/charm quark in the tt¯\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ t\overline{t} $$\end{document} production at linear colliders. There are previous bounds on such tqH couplings at both, linear and hadronic colliders, with the assumption that the top couples equally to the left and the right handed fermions. In this paper we examine chirality of the tqH coupling and construct different observables which will be sensitive to it. The kinematics of the emitted q from t → qH in tt¯\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ t\overline{t} $$\end{document} production is discussed and it was found that the polar angle distribution of q is sensitive to the chiral nature of tqH couplings. The observables in the context of top-antitop spin correlations, which are sensitive to new physics in the top decay are considered using different spin-quantization bases. It was found that in particular the off-diagonal basis can be useful to distinguish among the chiral tqH couplings. The sensitivity of the unpolarized ILC in probing the couplings at the 3σ level at s=500\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \sqrt{s}=500 $$\end{document} GeV and ℒ = 500 fb−1 is also studied, resulting in predicted BR(t → qH) < 1.19 × 10−3. This limit is further improved to BR(t → qH) < 8.84 × 10−4 with the inclusion of initial beam polarization of left handed electrons and right handed positrons.


Introduction
The search for Flavor Changing Neutral Current (FCNC) processes, has been one of the leading tools to test the Standard Model (SM), in an attempt of either discovering or putting stringent limits on new physics scenarios. The discovery of the Higgs boson at the LHC has lead the way to a comprehensive program of measuring its properties and branching ratios, in order to look for deviations from the SM predicted Higgs. Within the SM, there are no FCNC transitions at tree level mediated by the Higgs boson, due to the the presence of only one Higgs doublet and at the one-loop level these FCNC interactions are extremely small. There are however many extensions of the SM where the suppression of the neutral flavor changing transitions due to the Glashow-Iliopoulos-Maiani (GIM) mechanism can be relaxed, with the presence of additional scalar doublets or through the additional contributions of new particles in the loop diagrams. In the presence of two or more scalar doublets, these FCNC interactions will be generated at tree level and can be very large unless some ad-hoc discrete symmetry is imposed.
Motivated by the nature of the standard Yukawa coupling scheme the authors of [1] observed that the new FCNC couplings in the general two-Higgs doublet model naturally JHEP01(2017)048 follow the hierarchical structure of the quark masses and therefore anyqq H coupling should experience the following structure g qq H ∼ √ m q m q , (1.1) indicating that the larger couplings can be expected in the FCNC interactions of a topquark with the Higgs field. The large production rate of the top quarks at the LHC allows one to look for a transition of the top quark to a quark of a different flavor but same charge, t → cH (and t → uH), as no symmetry prohibits this decay. The SM branching ratio of this process is extremely small, of the order BR(t → cH) SM ≈ 10 −15 [2,3], which is many orders of magnitude smaller than the value to be measured at the LHC at 14 TeV. Therefore an affirmative observation of the process t → qH, well above the SM rate, will be a conclusive indication of new physics beyond the SM. The probing of FCNC couplings in the quark sector, can be performed either at a high energy collider or indirect limits can be obtained from neutral meson oscillations (K 0 −K 0 , B 0 −B 0 and D 0 −D 0 ) [4][5][6]. The tqH coupling also affects the Z → cc decay at the loop level and is therefore constrained by the electroweak precision observables of the Z boson [7]. The ATLAS and the CMS collaborations have set upper limits on the flavor changing neutral currents in the top sector through the top pair production, with one top decaying to W b and the other top assumed to decay to qH. The leptonic decay mode of the W is considered and the different Higgs decay channels are analyzed, with the Higgs decaying either to two photons [8,9] or to bb [10,11]. Combining the analysis of the different Higgs decay channels, based at √ s = 8 TeV and an integrated luminosity of 20.3 (19.7) fb −1 , the 95% CL upper limits obtained by ATLAS (CMS) [10,12] are Br(t → cH) ≤ 4.6(4.0) × 10 −3 and Br(t → uH) ≤ 4.5(5.5) × 10 −3 . On the phenomenological side the sensitivity of LHC measurements to these non-standard flavor violating couplings in the top sector has been explored in great details, considering (a) the top quark pair production [13][14][15][16], (b) the single top + Higgs production [3,17] and (c) single top + W production [18]. The analysis of the tqH coupling has also been carried out in the context of the next generation e − e + linear colliders, the International Linear Collider (ILC) and the Compact Linear Collider (CLIC) [19,20]. These planned high energy e − e + colliders are expected to perform precision measurements of the top-quark and the Higgs boson. They will be able to scrutinize the couplings in the top-Higgs sector to extreme precision, making them suitable for the sensitive tests of physics beyond the SM. The baseline machine design for both colliders allows for up to ±80% electron polarization, while provisions have been made to allow positron polarization of ±30% as an upgrade option. Both these machines are designed to operate at centre of mass energies of 350, 500 and 1000 GeV, with the possibility of CLIC to be also adapted for the 3 TeV operation. Several studies have been carried out in the context of zero beam polarization at the ILC [21,22] in an attempt to constrain the tqH vertex.
The Higgs boson within the SM couples similarly toq L q R andq R q L , i.e. y LR = y RL . Most of the studies in the context of the FCNC in the Higgs sector takes into effect this consideration and assumes the similarity between the chiral couplings. In this work we have focussed on the chiral nature of the FCNC couplings and have shown how the inequality JHEP01(2017)048 of chiral couplings leads to distinct behaviour in the distributions of final states at linear colliders. We work in the context of initial beam polarization for both the electron and the positron, using the advantages of their adjustment for enhancing the sensitivities of the measured branching ratios and the asymmetries on the FCNC parameters. We also present the results in the case of transverse polarized beams.
It is a well known fact that by a detailed study of the top (antitop) decay products one can obtain valuable information about the top spin observables and then use them for the detailed exploration of the top quark pair production or decay dynamics to distinguish among different models of new physics ( [23] and references therein). In order to maximize the top spin effects it is advisable to choose a proper spin quantization axis. At the Tevatron, where the top quark pair production was dominated by the quark-antiquark annihilation a special off-diagonal axis was shown to exist [24], making top spins 100% correlated. On the other hand, at the LHC the top quark pair production is dominated by the gluon-gluon fusion and there is no such optimal axis for this process. 1 The tt production through the electron-positron annihilation at the linear colliders will be similar to the Tevatron production, therefore the top quark spins will also be maximally correlated in the off-diagonal basis. The t,t spin effects, can be analyzed in the lepton-lepton or lepton+jets final states through a number of angular distributions and correlations. The spin information is proportional to the spin analyzing power of the decay products of the top and will therefore differ from the SM one in the case of FCNC top-Higgs decay. We therefore also carry out a detailed study of the FCNC t → qH decay with different spin observables, and in different top-spin polarization basis, using both unpolarized and longitudinally polarized beams.
The outlay of the paper is as follows. We discuss in section 2, the most general FCNC lagrangian considered for our analysis. We give a brief review of the effects of initial beam polarizations in the tt production at the linear collider in section 3. The detailed analysis of the tqH final state is performed in section 4 and constraints are obtained from angular asymmetries. The top spin observables in the context of different spin bases are discussed in section 5. A thorough numerical study of the process e − e + → tt → qHW −b → qbbl −ν lb including top FCNC coupling is performed in section 6, and finally we conclude in section 7. The analytical form of the different production and decay matrices, along with the expressions for the top spin observables used for our analysis are listed in appendix A and B.

The flavor changing top quark coupling
We concentrate on the most general FCNC tqH Lagrangian of the form

JHEP01(2017)048
This Lagrangian gives rise to the tree-level FCNC decays t → Hq, (q = u, c) with the partial decay width given as The SM top-quark decay is dominated by t → bW + and it is given by We neglect the mass of the emitted quark m q , in our analysis. The branching ratios of the top decaying in the presence of these flavor violating Yukawa couplings is then given by where the NLO QCD corrections to the SM decay width [26] and the t → cH decay [27] are included in the factor α QCD = 1 + 0.97α s = 1.10 [17]. The total decay width of the top in the presence of these FCNC couplings is then GeV for m t = 173.3 GeV at NLO, while the experimentally observed value of the total top-quark width is, Γ t = 1.41 +0. 19 −0.15 GeV [28]. The additional FCNC decay processes give positive contributions to Γ t , proportional to (| g tq | 2 + | g qt | 2 ) and from the experimentally observed Γ t an upper bound on | g tq | 2 + | g qt | 2 can be obtained. These flavor changing couplings can also lead to the three body decay h → t * (→ W + b)q, where top is produced off-shell and q = u, c. Then total width of the Higgs gets modified and the couplings g tq , g qt can be independently constrained from the measurement of the Higgs decay width at the LHC [14].
3 Polarized beams in tt production at the e − e + linear collider The most general formula for the matrix element square |T e − e + | 2 for arbitrary polarized e − e + beams producing a tt pair is given in refs. [29,30]. However for the annihilation process with massless electron and positron, the helicity of the electron has to be opposite to that of the positron, and the final formula is reduced to the form, is the helicity amplitude for the process under consideration, and λ 1 , λ 2 are the helicities of the electron and the positron, respectively. P L e ∓ is the degree of the longitudinal polarization and P T e ∓ is the transversal polarization for the electrons and positrons.

JHEP01(2017)048
The α ∓ refers to the angle of polarization of the electron and the positron, respectively. The polarizations of the electron and the positron at the linear colliders are independent and can be arbitrarily changed. The proposed linear colliders (ILC and CLIC) assume that the following polarizations can be achieved 2 As it was shown in ref. [30], if one is interested in the φ t (azimuthal angle of the top quark) dependence of the cross section, instead of discussing φ t dependence directly, it is simpler to study α ∓ dependence, since the latter is explicit in above. It can be shown that from the rotational invariance with respect to the beam direction, i.e. the rotation of the final state by φ t is equivalent to the rotation of the initial state by −φ t . With this assumption eq. (3.1) becomes where η = α − + α + . The effects of various beam polarizations in above will be discussed in the following.
4 Analysis of the tqH final state at the e − e + linear collider We study the tt production in the context of the e − e + linear collider, where one of the top decays to W b, and the other decays to q(u, c)H and the leptonic decay mode of the W boson is considered: We first consider the leading order spin dependent differential cross-section of the top pair production in a generic basis. The total phase space is split into the product of the differential cross-section for the tt production, the three-particle decay of the antitop quark and the two-particle decay of the top quark, with the Higgs decaying to bb. We first do the analysis considering the decay of t to qH and the inclusive decay oft. In an attempt to make a comparative study, we also consider the tt production, with the SM decay of top to W + b, and the inclusive decay oft. This SM process will be a background for the tqH final state, with the H and the W decaying hadronically. Since the analysis is being similar for 2 It is important to note the role of the beam polarization in the tt production. For the −80% of the electron polarization and +30% of the positron polarization the initial state will be dominantly polarized as e − L e + R , giving in the SM a constructive interference of the γ and Z amplitudes for the production of tLtR pair, and a destructive interference for the production of tRtL, which then leads to a large positive forward-backward asymmetry.

JHEP01(2017)048
both, the considered signal and the SM background, we only discuss the calculation of the signal in details. The differential cross section in the centre of mass frame becomes dσ = 1 2s where √ s is the centre of mass energy and s 1 = (p q + p H ) 2 . The energies of the produced top and the antitop are given by E 1 , E 2 , whereas the energies of the decay products are denoted by E q and E H . For these decays, in the center of mass frame and in the narrow width approximation, we can express the elements of the phase space in (4.2) as The total matrix element squared |M 2 | in eq. (4.2), is defined as where M L,R λt is the production helicity amplitude of the top with a given helicity λ t . The helicities of the antitop are summed over. The production helicity amplitudes are listed in eqs. (A.1) of appendix A.1. The decay matrix of the top quark is defined as ρ D(t) λtλ t = M(λ t )M * (λ t ) and for t → qH the explicit expressions in the rest frame of the top, as well as in the centre of mass frame are given in appendix A.2. For the top decaying to W + b the spin density matrix ρ D(t) λtλ t , is given in appendix A.3, for both the top rest frame and the centre of mass frame.
We have performed our calculations, in the frame where the electron beam direction is in the positive z direction, with the top emitted at a polar angle θ t and the quark emitted in the top decay makes a polar θ q angle with the electron beam, as shown in figure 1. The four-vector in the rest frame of the top are related to the c.m. frame by the following boost and the rotation matrices (the boost matrix is along the z direction, whereas the rotation matrix is applied along the y axis): JHEP01(2017)048 where q top 1 is defined in the rest frame of the top. The momentum four-vectors in the c.m frame are given by The momentum of the emitted light quark | p q | is equal to its energy E q and in the c.m frame the following relations are obtained: , where cos θ tq is the angle between the top and the emitted light quark in the c.m. frame.
Combining the production and the density matrices in the narrow width approximation for t, we get the polar distribution of the emitted quark q, in the presence of the beam polarization after integrating over φ q , θ t , to be where |T | 2 is of the form given in eq. (3.4). We compute |T e − L e + R | 2 , |T e − R e + L | 2 for the considered process, and present them in the most general form: (4.11)

JHEP01(2017)048
The coefficients a i , b i can be deduced from the following expressions: and where A L,R and B L,R are combinations of the standard SM γ and Z couplings with the top and the leptons in the tt production given in the eq. (A.2). The Yukawa chiral couplings, as seen from eqs. (4.12), (4.13) are both proportional to the polar angle of the emitted light quark, cos θ q , cos 2 θ q , but have different dependencies. The coefficients of the coupling |g tq | 2 , which measures the coupling strength of t L with q R and the Higgs, are summed in eq. (4.12), whereas the coefficients of the other chiral coupling |g qt | 2 do not add up, but cancel each other partially. This is the case when the electron beam is left polarized and the positron is right polarized. This behaviour of |g tq | 2 and |g qt | 2 is reversed with the right polarized electrons and the left polarized positrons, as can be noticed from eq. (4.13), where the coefficients of |g qt | 2 add up. Therefore, it will be possible to control the influence of particular chiral couplings with a suitable choice of beam polarization. The case of transverse polarization is also considered, although both |g tq | 2 , |g qt | 2 involve same angular dependencies in eq. (4.14) and therefore cannot be used for the analysis of the chirality of the FCNC couplings. It is clear from eqs. (4.12), (4.13), (4.14), that |g tq | 2 and |g qt | 2 cannot be isolated separately, but their effects can be individually controlled with suitable choice of beam polarization. We next study different distributions in the presence of the chiral FCNC couplings and accordingly construct asymmetries to set limits on them.

Constraints on the chiral FCNC couplings by angular asymmetries
Next, we perform a detailed analysis of the signal FCNC process considered, along with the standard SM background (tt, t → W b, W decaying hadronically) and construct different asymmetries for obtaining limits on the couplings. The total cross section for both the signal and the background, in case of the longitudinal beam polarization is where again A L,R and B L,R are combinations of the SM γ and Z couplings with the quarks in the tt production given in appendix A.
We have performed our analysis considering |g tq | 2 + |g qt | 2 = 0.16, in accordance with the latest LHC bounds [17]. The background i.e. the SMtW b contribution is scaled down, to be compared with the signal. We are currently not applying any cuts on the final state, but a detailed analysis using all the experimental cuts will be performed in section 6.
It can be clearly seen from Fig: 2, that |g tq | 2 and |g qt | 2 are sensitive to the beam polarization. The different Cases behave similar in the unpolarized case, figure 2(a). Case 2 is most prominent when the electron beam is left polarized and the positron is right polarized, figure 2(b), whereas Case 3 is distinct for the scenario with right polarized electrons and left polarized positrons, figure 2(c). Therefore the manifestation of the dominance of one of the coupling, if present, will be prominent using the suitable initial beam polarization.
Using the above fact that the couplings are sensitive to the polar angle distributions of the quark, we next consider different asymmetries to give simultaneous limits to both of the couplings. The |g tq | 2 and |g qt | 2 terms are accompanied by cos θ q , cos(η − 2φ) and cos(η − 2φ) cos 2 θ q angular dependence. The asymmetries which will isolate these terms are the forward-backward asymmetry and the azimuthal asymmetry defined as where θ 0 is the experimental polar-angle cut [31,32] and Ω = d cos θ q dφ t . The forwardbackward asymmetry will isolate the terms proportional to cos θ q in eqs. (4.12) and (4.13). We plot in figure 3, the forward backward asymmetry as a function of the cut-off angle  Next, we plot the azimuthal asymmetry A φ (cos θ 0 ) as a function of cos θ 0 in figure 4. The terms proportional to cos(η − 2φ t ) in eq. (4.14) survive. We have considered η = 0 for our analysis and P T e − = 0.8 and P T e + = 0.3. The distribution is similar for the signal and the background, therefore this will not be an useful observable. 3 We compute the limits on the FCNC couplings from the measurement of the forwardbackward asymmetry, of e − e + → tt, t → bW + in the SM. The statistical fluctuation in the asymmetry (A), for a given luminosity L and fractional systematic error , is given as where σ and A are the values of the cross section and the asymmetry. The value of is set to zero for our analysis. We define the statistical significance of an asymmetry prediction for the new physics, A FCNC , as the number of standard deviations that it lies away from the SM result A SM ,   Region in blue will be probed at 5σ and the green+blue area will be explored at 3σ with unpolarized beams. The inclusion of the beam polarization probes yellow+green+blue area at 5σ and pink+yellow+green+blue at 3σ. The region which can not be probed by ILC with this choice of beam polarization is shown in grey.
where A FCNC is the asymmetry calculated for the process e − e + → t(→ cH)t. We show in figure 5 the |g tq | 2 − |g qt | 2 region, which can be probed at a statistical significance of 3σ and 5σ, with both unpolarized and polarized beams. The outside area surrounding solid lines can be probed with unpolarized beams and the outside area surrounding dashed lines can be probed with a beam polarization of P L e − = −0.8, P L e + = 0.3 (figure 5(a)), P L e − = 0.8, P L e + = −0.3 ( figure 5(b)). Obviously, the inclusion of the beam polarization can probe a greater region of the |g tq | 2 −|g qt | 2 parameter space. The cos θ q terms in eqs. (4.12)-(4.13) cancel each other in case of unpolarized beams. The region in grey is the one, which cannot be explored by ILC with this choice of the beam polarization.
Now we turn to the discussion of different top spin observables which can be used to study the FCNC couplings.

JHEP01(2017)048
5 Top spin observables at the ILC We investigate in this section the top spin polarization in the context of the linear collider, as the spin information of the decaying top is not diluted by hadronization. In an attempt to understand the top spin correlations, we work in the zero momentum frame (tt-ZMF) [33] of the tt quarks, which is The t and thet rest frames are then obtained, by boosting (no rotation is involved) into the tt-ZMF. This is different from the laboratory frame considered before in section 4, where the electron beam is chosen along the z axis, and the t and thet rest frames were constructed by boosting from the lab frame along with a suitable Wigner rotation. The top quark pair production at O(α em ) is given by a direct production with the γ and Z exchange: The spin four-vectors of the top, s t and the antitop, st satisfy the usual relations The leading order differential cross section for the tt production, in the presence of longitudinal polarization eq. (3.4), has the phase space factor eq. (4.4) and can be written in the spin density matrix representation as In the above equation, ρ = ρ P (tt) is the corresponding production spin density matrix describing the production of (on-shell) top quark pairs in a specific spin configuration, whilê s t (ŝ t ) is the unit polarization vector of the top (antitop) quark in its rest frame and σ = (σ 1 , σ 2 , σ 3 ) T is a vector of Pauli matrices. Conveniently, the most general decomposition of the spin density matrix ρ for the tt production is of the form where the functions A, B t i (B t i ) and C ij describe the spin-averaged production cross section, polarization of top (antitop) quark and the top-antitop spin-spin correlations, respectively. Using the spin four-vectors defined as

JHEP01(2017)048
the decomposition of the squared scattering amplitude |T | 2 can be written as and by comparing expressions (5.5) and (5.7) one can extract the functions A, B t i (B t i ) and C ij . The functions B t i (Bt i ) and C ij can be further decomposed as wherek denotes the unit vector, and we have kept only nonvanishing terms for our case. 4 The various top spin observables O i can then be calculated as where σ = dΦ tt Tr[ρ] is the unpolarized production cross-section, dΦ tt is the phase space differential and S t = σ/2⊗1 (S t = 1⊗σ/2) is the top (antitop) spin operator. We consider the following spin observables or H decay products like b orb in bb decay, or τ + (τ − ) in H → τ + τ − decay, or jets. On the other hand, the spin analyzer fort are W − orb, or a W − decay products l − ,ν or jets. We consider the q = c/u quark from the top and the l − from the antitop as spin analyzers in this work. The spin analyzers are calculated from the one-particle decay density matrices given as where α, α denote the t-quark spin orientations,p f andpf are the directions of flight of the final particles f andf in the rest frame of the top and the antitop quarks respectively. The values of various κ f (f ) for SM top (antitop) decays are presently known at NLO in QCD and can be found in [39,40]. The top quark polarization matrix can be also written as 14) and similarly for the antitop spin matrix ρt →f . The top spin analyzing power of q (κ q ) from the t → Hq decay can be calculated from eq. (A.4), in appendix A.2, at LO QCD (α s corrections are negligible [39,40]), with their flight directions being 100% correlated with the directions of the top spin. It is clear from eq. (5.15) that with |g qt | 2 |g tq | 2 , the spin information of the top is lost (κ q ≈ 0). However in the presence or dominance of only one of the coupling, the emitted quark acts as a perfect spin analyzer (κ q ≈ 1). The top (antitop)-quark polarization and spin-spin correlations can be measured using the double differential angular distribution of the top and antitop quark decay products: where θ f (θf ) is the angle between the direction of the top (antitop) spin analyzer f, (f ) in the t (t) rest frame and theâ (b) direction in the tt-ZMF, cf. [33]. Comparing eq. (5.18), with eq. (5.10), we have

JHEP01(2017)048
where O 2 andŌ 2 are related to the top, antitop spin polarization coefficients B t and Bt.
Since there is no CP violation in our case, we consider B ≡ B t = ∓Bt forâ = ±b . This limit is a good approximation for the charged leptons from W decays [39,40]. The spin observable O 3 is also related to the spin correlation function C ij in eq. (5.5), where the arrows refer to the up and down spin orientations of the top and the antitop quark with respect to theâ andb quantization axes, respectively. Also O 4 gets corrected by κ f κf depending on the final particles measured from the t and t decays.
The arbitrary unit vectorsâ andb specify different spin quantization axes which can be chosen to maximize/minimize the desired polarization and the correlation effects. We work with the following choices: ("helicity" basis) , a =b =p , ("beamline" basis) , a =b =d X , ("off − diagonal" basis (specific for some model X)) , a =b =ê X ("minimal" basis (specific for some model X)) (5.21) wherep is the direction of the incoming beam andq =q 1 is the direction of the outgoing top quark, both in the tt center of mass frame. The off-diagonal basis [24] is the one, where the top spins are 100% correlated and is given by quantizing the spins along the axisd SM determined asd where z =p ·q 1 = cos θ and γ = E t /m t = 1/ 1 − β 2 and which interpolates between the beamline basis at the threshold (γ → 1) and the helicity basis for ultrarelativistic energies (γ → ∞). We would like to point out here that this off-diagonal basisd SM is specific to the SM tt production, but a general procedure for finding such an off-diagonal basis is given in [41,42]. The idea is to determine the maximal eigenvalue of the matrix function C ij in eq. (5.5) and the corresponding eigenvector, which provides the off-diagonal quantization axisd X , for any model X [23].
Here we introduce the complementary basis to the "off-diagonal" one,ê SM , where the eigenvector corresponds to the minimal eigenvalue of C ij in the SM quark-antiquark production. The correlation of the top-antitop spins in this basis is minimal. This axis could be useful in the new model searches since the minimization of the top-antitop correlations in the SM can, in principle, enhance the non-SM physics. The 'minimal basis' is defined by the axisê .  Figure 6. The top quark spin vector s t in the tt production in t rest frame, with the direction of s t given by an angle ξ. The angle ξ is measured in the clockwise direction from thet momenta.
The 'off-diagonal' and the 'minimal' basis define the angle ξ between the top-quark spin vector and the antitop direction in the top-quark rest frame [25], shown in figure 6, or cos 2 ξ min + cos 2 ξ off(=max) = 1 , (5.25) as expected. As already stated, such axes which minimize or maximize spin correlations can be constructed for any model. The analytical form of the observables defined in eq. (5.10), is listed in appendix B for the SM tt production in the presence of longitudinal polarization. The observables O i , (i = 1,2,3,4) are then multiplied with the appropriate κ factors. The QCD radiative corrections for all the top spin observables considered here are calculated in [38] and it is shown to be small. Also recently it has been shown that the O(α S ) corrections to the maximal spin-spin correlations in the off-diagonal basis are negligible [37]. Therefore we neglect them all in our calculations.
Next, we present the results for spin correlations and top (antitop)-quark polarizations in the helicity basis (C hel , B hel ), beamline basis (C beam , B beam ), off-diagonal (C off , B off ) and the minimally polarized basis (C min , B min ), as defined by eqs. (5.21), (5.22) and (5.23) respectively, and check for their sensitivity to the initial beam polarization. These results are presented in the absence of cuts, realistic cuts severely distort the non-zero coefficients of eq. (5.11) and eq. (5.18). The observable O 1 as seen from eq. (B.1), is equal to 1 and is therefore independent of beam polarization. However, it is dependent on the value of κ f .
In table 1 we present the values of the different spin observables in the different spin basis considered here, in the presence of beam polarizations. We have considered the case, when the antitop is decaying to lepton (κf =1), κ f = κ q , eq. (5.15) for the FCNC top decay, and κ f = κ b , eq. (5.16) for the top decaying to W + b. We note that the top (antitop) spin polarizations are quite sensitive to the beam polarization, while this is not the case for the spin-spin correlations O 3 , O 4 where the influence of the beam polarizations gets diluted,

JHEP01(2017)048
Observables Basis P L e − = 0, P L e + = 0 P L e − = 0.8, Table 1. The value of the spin observables in different bases, with different choices of initial beam polarization. κ f = κ q for FCNC t-decays and see eqs. (B.6)-(B.10). Also note that all observables are proportional to κ f = κ q and will be equal to zero if g tq and g qt are equal.
6 Numerical analysis of the FCNC g tq , g qt couplings at the ILC In this section we perform a detailed numerical simulation of the FCNC interactions in the t → qH decay at the ILC. As before, the process we consider is the top pair production, with the top decaying to qH, the antitop decaying to W −b with the W − decaying leptonically and subsequently the Higgs decaying to a bb pair. The main background for the process under study comes from the tt pair production, with one of the top decaying hadronically and the other decaying to a lepton, ν and a b quark. We have performed our calculations, by first generating the Universal Feynrules Output (UFO) model file using FeynRules 2.3 [43], including the effective interaction, defined in eq. (2.1). The UFO file is then implemented in MadGraph 5 v2.4.2 [44,45], for Monte Carlo simulation. We also employ Pythia 8 [46] for parton showering and hadronization along with Fastjet-3.2.0 [47] for the jet formation. The cross section of the signal and the background, at √ s = 500 GeV, before the application of the event selection criteria is listed in table 2. We now describe in details the different cuts and conditions considered for our analysis. Since the top from the tqH final state decays to W b, the lepton from the W , tends to be energetic and isolated. Therefore firstly the events with one isolated lepton are selected, through the lepton isolation cut. An isolated lepton is identified, by demanding that the scalar sum of the energy of all the stable particles within the cone of ∆R = ∆η 2 + ∆φ 2 ≤ 0.2 about the lepton is less than 6(E l − 15) [48], where E l is the energy of the lepton. Furthermore, the transverse momenta of the leptons are assumed as p T >10 GeV. The events with more than one isolated lepton are discarded. The remaining stable visible particles of the event, are then clustered into four jets using the inbuilt k t algorithm in FastJet for e − e + collisions, which is similar to the Durham algorithm. The reconstructed JHEP01(2017)048 e − e + → tt P L e − = 0, P L e + = 0 P L e − = −0.8, P L e + = 0.3 P L e − = 0.8, P L e + = −0.3 signal: t → qbb,t → l −ν lb 73.4(|g tq | 2 + |g qt | 2 ) 120.5(|g tq | 2 + |g qt | 2 ) 62(|g tq | 2 + |g qt | 2 ) background: There will be three b tagged jets and a non b jet in the final state and therefore three possible combinations to reconstruct the Higgs mass from the b tagged jets. Additionally one of this pair of b jets reconstructing the Higgs mass, along with the the non b jet should give an invariant mass close to m t . We choose the combination of the jets, which minimizes the quantity |m b i b j − m H | 2 + |m b i b j Q − m t | 2 , with i, j taking values for various combinations of the b jets and Q is the non-b jet. The reconstructed Higgs mass is given by m b i b j , and the reconstructed top mass is denoted by m b i b j Q . In order to account for the detector resolution, we have smeared the leptons and the jets using the following parametrization. The jet energies are smeared [50] with the different contributions being added in quadrature, The momentum of the lepton is smeared as a function of the momentum and the angle cos θ of the emitted leptons [51] σ(P l ) The plots for the signal are constructed taking into account the current stringent LHC constraint on the FCNC couplings, |g tq | 2 + |g qt | 2 = 0.16. We have shown the results for Case 1, discussed in section 4.1, as the reconstructed mass will be the same for all three cases. We note that since we have not done a real detector analysis, the mass reconstruction of the W boson is poor in our case, due to the presence of missing energy. Therefore a loose cut on m W is applied for our analysis. It is clear from figure 7, that the cut imposed on the reconstructed m t and mt should be different. The reconstructed mass oft is broad, due to the presence of the missing energy from the W decay. We have applied the same kinematic cut to the mass of the top and the antitop for the sake of simplicity. The implementation of these cuts, eliminates the W bbjj and Zbbjj backgrounds. The kinematical cuts, which are imposed on the various reconstructed masses are summarized below: Additional cuts can be applied, on the energy of the emitted quark in the top rest frame [21], so as to increase the signal to background ratio. The energy of the emitted quark, as a result of the two body decay of the top is and is peaked around 42 GeV, for a Higgs mass of 125 GeV. The jet from the background, which will fake the q jet, will have a more spread out energy. We do not apply this cut, as the application of the above cuts already lead to a much reduced background. The energy distribution of both the signal and the background are shown in figure 8. Further on, we concentrate on the observables which will be sensitive to the chiral nature of the FCNC interactions. One of them is the polar angle distribution of the non-b jet, which was earlier shown in √ |g tq | 2 + |g qt | 2 = 0.16 Figure 9. The opening angle distribution, eq. (5.11) between the direction of the lepton (from t → l −νb ) and the non-b jet (from t → qH), in the t andt rest frame.
distributions which we consider here are all calculated in the tt−ZMF. The decay products, which act as spin analyzers for our case are the non-b jet (q) from the decay t → qH and the lepton (l − )from the decayt → l −νb . All the distribution plots are given with the number of surviving events, for L = 500 fb −1 . We plot the opening angle distribution 1/σ(dσ/d cos ϕ) (eq. (5.11)) in figure 9, which is sensitive to the top and the antitop spin analyzers. The distribution is flat for Case 1, when |g tq | 2 = |g qt | 2 , leading to κ q = 0. It peaks in the forward direction in the presence of |g tq | 2 , and in the backward direction for |g qt | 2 (clearly seen in the inset of figure 9). The top spin is considered in the normalized distribution 1/σ(dσ/d cos θ qst ), where θ qst is the angle between the direction of the top spin analyzer (non-b jet) in the top rest frame and the top spin quantization axis (s t ) in the tt-ZMF. The angle cos θ qst is the angle cos θ f defined in eq. This distribution is sensitive to the polarization of the top and we show in figure 10 the distribution calculated in the different bases. As expected, the 'beamline' basis and the 'off-diagonal' basis are most sensitive to the top polarization and therefore also to the decay dynamics of the top. The chiral nature of the FCNC coupling will be more clearly visible in these two basis, with a flat distribution in case of the equality of the two chiral coupling. The 'helicity' and the 'minimal' basis will not be effective in discriminating the chirality and they are shown just for the illustration. The effect is further enhanced with the beam polarizations of P L e − = −0.8 and P L e + = 0.3, in all the spin bases considered here. We show the distribution in the 'off-diagonal' basis in figure 11, as it is most sensitive to the beam polarization.  Figure 12. The normalized distribution of the product cos θ qst cos θ lst , (θ qst = ∠(p q ,â), θ qst = ∠(p l ,b)), using the off-diagonal basis, at √ s = 500 GeV, with polarized beams and L = 500 fb −1 .
The double differential angular distribution of the top and the antitop defined in eq. (5.18) provides a measurement of the spin-spin correlations. It was shown in ref. [52] that, for the experimental analysis, it is more suitable to use the one-dimensional distribution of the product of the cosines, O st,st = cos θ f cos θf , rather than analyzing eq. (5.18). We define cos θ f cos θf as cos θ qst cos θ lst for our analysis. The 1/σ(dσ/dO st,st ) distribution is shown in figure 12, using the 'off-diagonal' basis and a longitudinal beam polarization of P L e − = −0.8 and P L e + = 0.3 . The asymmetry of the plot around cos θ qst cos θ lst = 0, signals for the spin-spin correlation. The plot for Case 2 (|g qt | 2 = 0) shows more events for positive values for cos θ qst cos θ lst , whereas for Case 3 (|g tq | 2 = 0) one gets more events for negative values of cos θ qst cos θ lst .
We next estimate the sensitivity that can be obtained for the FCNC tqH couplings, given by the efficient signal identification and the significant background suppression which can be achieved at the linear collider. We adopt the following formula for the significance JHEP01(2017)048  Table 3. Upper bounds on |g tq | 2 + |g qt | 2 and the respective branching ratios, that can be obtained in the ILC, at √ s = 500 GeV, with a luminosity of 500 fb −1 . The results are presented for both, the polarized and the unpolarized case. measurement [53], 5) with N S and N B being the number of signal and background events. In figure 13 we present the contours of 3σ and 5σ significance for our process in the |g tq | 2 − |g qt | 2 plane. The sensitivity of the linear collider will increase with the implementation of beam polarization with left polarized electrons and right polarized positrons. Since the total cross section is proportional to |g tq | 2 + |g qt | 2 , the contours are symmetric in that plane. The sensitivity to the coupling |g tq | 2 + |g qt | 2 , as a function of the integrated luminosity for √ s = 500 GeV is shown figure 14. One can see that at 3σ statistical sensitivity and L = 500 fb −1 , |g tq | 2 + |g qt | 2 can be probed to 0.063 (0.056) with unpolarized (polarized) beams. The limits obtained from the asymmetries, specially A f b from section 4.1 will be more stronger and will not be symmetric in the |g tq | 2 − |g qt | 2 plane. We find the following upper bounds as listed in table 3 at the 2σ, 3σ and the 5σ level from the total cross section, in the case of the polarized and the unpolarized beams.

Conclusion
We have studied the flavor violating top-Higgs interactions, at the e − e + linear colliders using different beam polarizations. There are several works exhibiting the prospects of the LHC to constrain or discover these couplings, by considering several signatures of the flavor violating interactions. The LHC experiments have also looked into these couplings and have obtained bounds on the branching ratio of the process t → qH. These flavor violating interactions can have a chiral structure with the top coupling differently to the left handed and the right handed fermions. Since the branching ratio of the top to qH, as well as, the total production cross section is being proportional to |g tq | 2 + |g qt | 2 , the chiral nature won't be evident from these measurements. Therefore, we have looked in the context of the linear collider into various observables which will highlight this aspect of the couplings. The polar angle distribution of the quark emitted from the t → cH decay, will exhibit a behaviour sensitive to the nature of the coupling. This will change with the change of the beam polarization. The distribution will JHEP01(2017)048  be flat for all the polarization combinations if |g tq | 2 = |g qt | 2 . The presence of only one of the coupling (|g tq | 2 ) leads to a forward peak for e − L e + R polarization and will be unchanged for the e − R e + L polarization. The opposite behaviour is observed for |g qt | 2 . Next, the forwardbackward asymmetry A f b is used in order to constrain the |g tq | 2 − |g qt | 2 parameter space.
The spins of the tops are correlated in the top pair production and the decay products of the tops are correlated with the spins, therefore the decay products of the top and the antitop are correlated. The presence of new physics in the top decay will therefore, lead to a change in the correlation coefficient in the angular distribution of the top decay products. A right choice of spin basis of the top quark pair is also important in enhancing the correlation. We consider different observables in section 5, which are sensitive to the spin analyzing power (κ) of the top decay product. The quark emitted from the top FCNC JHEP01(2017)048 decay, will be a perfect spin analyzer (κ q = 1) in the presence of a single chiral coupling. The κ q of the emitted quark will be zero when |g tq | 2 = |g qt | 2 and the correlation will be lost. We have performed an analysis applying all the cuts at the linear collider in section 6, and have studied the spin observables in the context of different spin bases. We find that the off-diagonal basis along with the beamline basis are the most sensitive to the chirality of the couplings. The effect is even more enhanced by polarizing the initial beams of left handed electrons and right handed positrons.
Finally, we have obtained a limit on the couplings from the total cross section and find that BR(t → qH) can be probed to 5.59 × 10 −3 (8.84 × 10 −4 ) at 3σ level at the ILC, with √ s = 500 GeV, L = 500 fb −1 and a beam polarization of P L e − = 0(−0.8), P L e + = 0(0.3) , which hopefully will be observed at the future linear colliders.

JHEP01(2017)048
appear are defined below: where Q e = −1, Q t = 2/3, θ W is the Weinberg mixing angle and The squared matrix elements ρ for the top quark decay in its rest frame is given by where E top q , θ top q , φ top q are the energy and the polar and the azimuthal angle of the emitted quark q in the top rest frame, respectively. We obtain the relevant ρ D(t) λtλ t in the c.m. frame by making the following substitution in the above equations: (cos θ t sin θ q cos φ q − sin θ t cos θ q ± i sin θ q sin φ q ) , (A.5) where θ tq and E q are defined in eq. (4.9). The squared matrix elements is similar for antitop, with β replaced by −β. We have assumed m q = 0, for all our calculations.
A.3 The decays t → W b and t → l + νb The squared matrix elements ρ D(t) λtλ t for the top quark decaying to W + b, in the rest frame of the top is given by

JHEP01(2017)048
The squared matrix elements ρ D(t) λtλ t for the top quark decay to l + νb in its rest frame, is where E top f , θ top f , φ top f are the energy, and the polar and the azimuthal angle of the emitted fermion in the top rest frame, respectively. The emitted fermion is b in eq. (A.6) from the decay t → W + b and l from the decay t → l + νb in eq. (A.7). The squared matrix elements ρ Below we list the analytical expressions for the various spin observables calculated and considered in section 5.
where σ tt is the total cross section given by

JHEP01(2017)048
Open Access. This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.