Secondary lepton distributions as a probe of the top--higgs coupling at the LHC

The differential distributions in rapidity and angles of the secondary lepton in the associated production of the top quark pair and higgs boson in proton-proton collisions at the LHC are quite sensitive to the top-higgs coupling. However, the effects of anomalous couplings of the most general t anti-t h interaction with operators of dimension-six that are clearly visible in the signal of the associated production of the top quark pair and higgs boson are to large extent obscured by the background sub-processes with the same final state. This means that analyses of such effects, in addition to higher order corrections that are usually calculated for the on-shell top quarks and higgs boson, should include their decays and possibly complete off resonance background contributions to the corresponding exclusive reactions.


Introduction.
Associated production of the top quark pair and higgs boson was proposed as a sensitive probe of the top-higgs Yukawa coupling g tth at the e + e − linear collider (LC) [1], [2] more than 20 years ago [3]. A clean experimental environment of the LC seems to be the best place to study the higgs boson profile, including the measurement of g tth , but the project of LC is still at the rather early stage of TDR. Fortunately, the top quarks are copiously produced at the LHC that, among others, allows for more and more precise determination of the top quark pair production cross section and for measurements of the cross sections of tt + jets, see [4] for a review. The measurement of production of tt + bb [5] is particularly interesting, as it is relevant for observation of the associated production of the top quark pair and higgs boson, with the higgs decaying into bb. The latter should be the dominant decay mode, if the new boson at a mass of about 125 GeV observed at the LHC [6] is indeed the higgs.
The associated production of the top quark pair and higgs boson in the proton-proton collisions at the LHC is dominated by the gluon-gluon fusion mechanism. Taking into account decays: h → bb, t → bW + ,t →bW + and the subsequent decays of the W -bosons, one should consider hard scattering partonic processes as, e.g., corresponding to one of the W 's decaying hadronically and the other leptonically. Reaction A question arises whether the associated production of the higgs boson and top quark pair can be sensitive to possible modifications of the SM top-higgs Yukawa coupling or not. The question will be addressed in this work by showing how the distributions of the secondary lepton are changed in the presence of such modifications. The distributions computed with the signal diagrams only will be compared with those computed with the full set of the leading order Feynman diagrams that will demonstrate how the background contributions obscure relatively clear effects of the anomalous tth coupling in the signal cross section. Although the issue may seem somewhat premature from the experimental side, but in view of the excellent performance of the LHC, it may become relevant in quite a near future.

Calculation details
The calculation is performed in a fully automatic way with a new version [7] of carlomat [8], a general purpose program for Monte Carlo computation of lowest order cross sections. The most general Lagrangian of tth interaction including corrections from dimension-six operators that has been implemented in the program has the following form [9] where GeV, is the top-higgs Yukawa coupling. The couplings f and f ′ are assumed to be real. They describe, respectively, scalar and pseudoscalar departures from the purely scalar top-higgs Yukawa coupling of SM. The latter is reproduced for f = 1 and f ′ = 0. Other dimension-six gauge-invariant effective operators that may have affected the tth interaction are redundant, in a sense that they can be eliminated with the use of the equations of motion, both for the on-and off-shell particles [9]. Obviously, the process of associated production of the higgs boson and top quark pair will be affected by many other possible deviations from the SM couplings. They are not considered here, as the primary goal of the present work is to illustrate just the effects of the anomalous tth interaction on the distributions of the secondary lepton. However, some deviations, eg., the anomalous W tb coupling generated by the gauge-invariant dimension-six effective operators, which is present in the tth signal diagrams of Figs. 1(a)-(c) and in some off resonance background diagrams such as the one depicted in Fig. 1(d), can be easily included, as it has been already implemented in carlomat. See [10] for the illustration of some effects on the top quark pair production at the LHC that can be caused by the W tb coupling.
The couplings f and f ′ of Lagrangian (3) belong to least constraint couplings of the SM. For the higgs boson with a mass of 125 GeV, practically the only model independent way to constrain them is to measure the tth production [11]. First results of search for this process in pp collisions at the LHC are reported in [12]. Indirect constraints of the tth interaction vertex can be derived from measurements of the higgs boson production rate through the gluon-gluon fusion process, which is dominated by a top-quark loop, and of the higgs boson decay into 2 photons that, despite being dominated by the W boson loop, also receives a significant contribution from the top-quark loop. However, extraction of the tth coupling in this way relies on the assumption that the loops do not receive contributions from new massive fundamental particles beyond those of the SM. If two universal scale factors are assumed, one for the higgs boson Yukawa couplings to all the SM fermion species and the other for the higgs boson couplings to the EW gauge bosons, and if there is no new physical degrees of freedom, then the scalar coupling f of Eq. (3) can be constraint at 95% C.L. to be in the following regions: CMS [14].
It should be noted at this point that an opposite sign of the higgs boson coupling to fermions with respect to its coupling to the gauge bosons is required in the Lagrangian for the unitarity and renormalizability of the theory [16] and vacuum stability [17]. Therefore, the interval in the range of negative numbers in (4) is highly disfavoured. The relative sign of both couplings could probably be best determined in the reaction of associated production of the top quark and higgs boson in proton-proton collisions at the LHC through the underlying t-channel partonic process qb → tq ′ h [15].
In carlomat, the on-shell poles in propagators of unstable particles, both the s-and t-channel ones, are avoided by making the following substitutions: where the particle widths are assumed to be constant and the square root with positive real part is chosen, see [8] for details. In order to minimize unitarity violation effects at high energies caused by substitutions (6), which correspond to re-summation of one particle irreducible higher order contributions to s-channel propagators, the computation is performed in the complex mass scheme, where the electroweak (EW) couplings are parametrized in terms the complex EW mixing parameter sin 2 θ W = 1 − M 2 W /M 2 Z which preserves the lowest order Ward identities [18]. Note, that the electric charge e W can be defined as a real quantity in terms of as it enters all the EW couplings multiplicatively, which is our choice in the present work. The only effect of using the complex masses of (6) in Eq. (7) would be the overall change of normalization of the cross section. The top-higgs Yukawa coupling is defined in the complex mass scheme by i.e., it is a complex quantity, as it is parametrized in terms of the complex masses of (6) and complex EW mixing parameter sin θ W .

Results
In this section some results for the differential cross sections and distributions of reaction at √ s = 14 TeV are presented. For the sake of simplicity and easy reproducibility of the results, only one hard scattering process (2) that dominates at that energy is taken into account. It is folded with CTEQ6L parton distribution functions [19] at the scale Q = 2m t + m h .
The initial physical input parameters used in the computation are the following. The strong coupling between quarks and gluons is given by EW couplings are parametrized in terms of the electric charge of (7) that is kept real and the complex EW mixing parameter sin θ W , as described in Section 2, with the EW gauge boson Jets are identified with their original partons and the following cuts on the transverse momenta p T , pseudorapidities η, missing transverse energy / E T and separation ∆R ik in the pseudorapidity-azimuthal angle (ϕ) plane between the objects i and k are imposed: where the subscripts l and j stand for lepton and jet. Cuts (10) should allow to select events with separate jets, an isolated charged lepton and missing transverse momentum.
Moreover, 100% efficiency of b tagging is assumed and events of the associated production of top quark pair and higgs boson in reaction (9) are selected by imposing the following invariant mass cuts: on the invariant mass of two non b jets, b ∼b 1 and b ∼b 2 , on the transverse mass of the muon-neutrino system on the invariant mass of a b jet, b 1 , and the two non b jets on the transverse mass m T of a b quark, b 2 , muon and missing transverse energy and the invariant mass cut on two b jets, b 3 and b 4 , with either m cut bb = 20 GeV or, more optimistically, m cut bb = 10 GeV. In (14), m T is the transverse mass defined by with m being the invariant mass of the b-µ system given by m 2 = (p b 2 + p µ ) 2 . Cuts (11)- (14) should allow to identify the secondary W bosons, the top quarks and the higgs boson. They were used before in the context of the associated production of the top quark pair and higgs boson in e + e − collisions at the LC [20].
For the sake of illustration, the tth couplings of (3) are assigned the following values: f = 1, 0 and f ′ = 0, ±1 and the differential distributions of the final state muon, generally referred to as lepton, of reaction (9) are computed, first with the 56 signal Feynman diagrams of the associated production of the top quark pair and higgs boson and then with the complete set of 67 300 Feynman diagrams, as discussed in Section 1. The rapidity and angular differential cross sections and distributions of the lepton for which the effects of anomalous couplings are best visible will be shown in Figs. 3-6 and the distributions in the lepton transverse momentum or energy which are practically not affected by the couplings will not be presented.
The size of background contributions to the associated production of the higgs boson and top quark pair in pp collisions at √ s = 14 TeV is illustrated in Fig. 2, where the differential cross sections of (9) are plotted as functions of the muon rapidity, y l , cosine of the muon angle with respect to beam, cos θ lb and cosine of the muon angle with respect to the reconstructed higgs boson momentum, cos θ lh . The cross sections plotted in Fig. 2  In order to illustrate what a role invariant mass cuts (10)- (15) play, the distributions of the final state lepton of (9) at √ s = 14 TeV in y l and cos θ lb with different cuts are compared in Fig. 6. The left panels show the distributions with the cuts given by (10) and the right panels show the distributions with cuts (10)- (14) and (15) with m cut bb = 10 GeV for two different combinations of the scalar and pseudoscalar tth couplings of (3). Again the grey boxes show the SM results. Although the invariant mass cuts suppress the background contributions to some extent, the degree of suppression does not look very satisfactory. The signal significance µ = σ(f, f ′ )/σ SM and corresponding differences in expected numbers of events, ∆n = n(f, f ′ ) − n SM , for different combinations of the couplings (f, f ′ ), with the µ − in the forward (cos θ lh > 0) or backward (cos θ lh < 0) hemisphere with respect to the direction of the higgs boson, in reaction (9) at √ s = 14 TeV are shown in Table 1. The cuts are given by (10)- (15) with m cut bb = 20 GeV (columns 2-5) and m cut bb = 10 GeV (columns 6-9). The event numbers have been calculated assuming an integrated luminosity of 100 fb −1 and 100% detection efficiency. Therefore, they should be treated with care, in particular because of the fact that only the leading order contributions to reaction (9) are taken into account. However, the leading order predictions for signal significance µ are more reliable. In particular, µ ≈ 1.2 for f = 1 and |f ′ | = 1 indicates a potential of the reaction of associated production of the higgs boson and top quark pair in obtaining direct limits on the pseudoscalar coupling f ′ . If only the tth signal contributions to the cross section are taken into account, then the signal significance for this particular combination of couplings becomes even bigger, amounting to µ = 1.4 in the forward and µ = 1.6 in the backward hemisphere with respect to the direction of the higgs boson. This again shows how the off resonance background contributions obscure the signal of tth production.

Summary and conclusions
The differential cross sections and distributions of the final state lepton of (9) in rapidity, cosine of its angle with respect to the beam and cosine of its angle with respect to the recon- the tth signal diagrams only, which are substantially changed in the presence of anomalous tth couplings, have been compared with those computed with the full set of the leading order Feynman diagrams. The comparison have shown that the background contributions to large extent obscure the relatively clear effects of the anomalous tth coupling in the signal distributions. This means that analyses of such effects [21], in addition to higher order corrections [22] that are usually calculated for the on-shell top quarks and higgs boson, should include their decays and possibly complete off resonance background contributions to the corresponding exclusive reactions. The only reasonable way to make the effects of anomalous couplings better visible seems to be imposing more and more restrictive cuts.  (14) and (15) with m cut bb = 10 GeV (right panels) for different combinations of the scalar and pseudoscalar tth couplings.