Search for electroweak production of a vector-like T quark using fully hadronic final states

A search is performed for electroweak production of a vector-like top quark partner T of charge 2/3 in association with a top or bottom quark, using proton-proton collision data at s\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \sqrt{s} $$\end{document} = 13 TeV collected by the CMS experiment at the LHC in 2016. The data sample corresponds to an integrated luminosity of 35.9 fb−1. The search targets T quarks over a wide range of masses and fractional widths, decaying to a top quark and either a Higgs boson or a Z boson in fully hadronic final states. The search is performed using two experimentally distinct signatures that depend on whether or not each quark from the decays of the top quark, Higgs boson, or Z boson produces an individual resolved jet. Jet substructure, b tagging, and kinematic variables are used to identify the top quark and boson jets, and also to suppress the standard model backgrounds. The data are found to be consistent with the expected backgrounds. Upper limits at 95% confidence level are set on the cross sections for T quark-mediated production of tHQq, tZQq, and their sum, where Q is the associated top or bottom heavy quark and q is another associated quark. The limits are given for each search signature for various T quark widths up to 30% of the T quark mass, and are between 2 pb and 20 fb for T quark masses in the range 0.6–2.6 TeV. These results are significantly more sensitive than prior searches for electroweak single production of T → tH and represent the first constraints on T → tZ using hadronic decays of the Z boson with this production mode.


Introduction
We report on a search for electroweak production of a new heavy quark of charge 2/3 with nonchiral couplings, referred to as a vector-like quark. Unlike the standard model (SM) chiral fermions, such particles do not acquire their mass from a Yukawa coupling to the Higgs boson (H). Many proposed extensions of the SM contain vector-like quarks, which usually mix with the top quark (t). Such particles could have a role in stabilizing the Higgs boson mass, and thus offer a potential solution to the hierarchy problem. Vector-like quarks are discussed in detail in refs. [1][2][3][4] and have been the subject of phenomenological studies in various frameworks including those of refs. [5][6][7][8]. Much like the top quark, a vector-like top quark partner (T) could be produced either in pairs, dominantly through the strong interaction, or singly, in association with additional quarks through the electroweak interaction. The T quark could couple to bW, tZ, or tH; this leads to the corresponding T quark decays and to the associated electroweak production from processes such as those depicted in figure 1. The branching fractions and dominant electroweak production processes depend on the particular model; many models have substantial branching fractions to tZ or tH resulting in signatures that are of primary relevance to this paper. Neglecting the corrections due to decay particle masses, the branching fractions for the T singlet model of ref. [5] are 50% (bW), 25% (tZ), 25% (tH), while for the (TB) doublet model of ref. [5], the tZ and tH branching fractions tend to be approximately equal and depend on two mixing angles, θ u R and θ d R , with each branching fraction ranging from zero to 50%. Therefore specific models can have branching fractions as large as 50% for tZ and 50% for tH.
We perform a search targeting the electroweak production of a vector-like top quark partner T in fully hadronic final states in proton-proton (pp) collisions at √ s = 13 TeV with the CMS detector at the CERN LHC. We use two searches that target separately lower and higher mass values for the T quark. Both searches are designed to be sensitive to the decay to a top quark and a Higgs boson (T → tH), and to the decay to a top quark and a Z boson (T → tZ) with subsequent hadronic decays of X (X = H, Z). Both also consider a wide range of widths of the T quark, ranging from narrow, defined as small compared to the experimental mass resolution, to as much as 30% of the T quark mass. The event selections primarily require b tagging for the Higgs and Z boson candidates and so are most sensitive to X → bb. The experimental signature is a resonant peak in the tX invariant mass spectrum.
The searches are designed to seek evidence of T quarks produced in association with a bottom quark, dominated by the qg → Tbq process, and, separately, associated production with a top quark dominated by the qg → Ttq process, where the charge conjugate processes are also implied. These are electroweak production modes, with the production of -2 -JHEP01(2020)036 only a single T, that rely on a nonzero TbW coupling for the charged-current production, and a nonzero TtZ coupling for the neutral-current production. In order to be produced with an observable cross section, one needs a substantial partial width for the coupling to the initial state. As a consequence, currently accessible production cross sections in electroweak single production are associated with particle widths exceeding about 5%, which would affect the experimentally observable invariant mass distributions. The total width could also be enlarged if additional decay modes were present.
As a result of the lower requirement on the constituent center-of-mass energy and the larger available phase space, single production via the electroweak mechanism allows a search for vector-like top quarks with masses beyond those already tested with pair production. The qg → Tbq → tXbq process, with the top quark from the T quark decaying hadronically and X decaying to two b quarks, results in up to seven jets, four of which are b jets. The seven jets are associated with the production of seven fermions, namely qg → (qq b)(bb)bq . Similarly the qg → Ttq → tXtq process results in at least nine fermions. In each case the other associated quark (q or q) often results in a forward jet at high absolute pseudorapidity. The qg → Tbq process is expected to have a higher cross section than qg → Ttq from kinematic and coupling considerations.
Recent searches at the LHC for pair production of vector-like quarks have severely constrained the possible existence of lower-mass vector-like quarks that couple to heavy quarks [9][10][11][12][13][14][15][16][17][18]. These searches use several final states arising from the bW, tZ, and tH decay channels and usually model pair production under the assumption of a narrow width. In particular, for the T singlet model, the most stringent expected lower mass limit from pair production to date is 1.2 TeV [16] at 95% confidence level (CL). Pair production is based on the assumed universal strong coupling and so the quantum chromodynamics (QCD) pair production cross sections are known and model independent, and depend only on the T quark mass, m T . On the other hand, electroweak production depends on the strength of the T quark coupling at the production vertex, either TbW or TtZ, and therefore the production cross sections are model dependent. In some models, such as that of ref. [2], the couplings are constrained by precision observables to be quite small. In other models, such as that of ref. [7], cross sections two orders of magnitude higher than in ref. [2] may be feasible. The first direct experimental constraints on electroweak production of vector-like quarks were published in ref. [19]. Search results at √ s = 13 TeV include the search already performed by CMS for electroweak production of T with T → tH for both semileptonically decaying top quarks and hadronically decaying top quarks using the 2015 data set [20,21]. Other results at √ s = 13 TeV targeting electroweak production of T are described in refs. [22, 23] for T → bW, and for T → tZ with dielectron and dimuon decays of the Z in refs. [15,24,25], and using a missing energy signature for Z → νν decays in ref. [26]. The search reported here uses the 2016 data set to study fully hadronic final states with both merged and resolved jets resulting from electroweak production of a T quark with T → tH and T → tZ. This search represents a significant advance over prior electroweak production searches for T → tH, with expected 95% CL cross section upper limits typically 5-10 times lower than those reported in ref. [21], and is competitive with other searches for T → tZ in this production mode.

JHEP01(2020)036
For low values of the T quark mass, the quarks resulting from the top quark decay and from the Higgs or Z boson decay can be resolved as individual jets. However, as the T quark mass increases, the larger Lorentz boost from the decay will lead to the decay products of the top quark and the quarks from the Higgs or Z boson becoming progressively less and less resolved as separate jets. The jet multiplicity, correspondingly, is reduced. Furthermore, one can reconstruct both the top quark and the Higgs or Z boson by using large-area jet and substructure techniques, where area refers to the jet's extent in η-φ space. Consequently, two search signatures are defined as follows: • Low-mass search: reconstruction of a five-jet invariant mass signature for the T → tH and T → tZ decay modes. This search signature is based on multijet triggers with b tagging and is effective for low T masses (0.  where the individual jets from the decays can be resolved. • High-mass search: reconstruction of an invariant mass signature from two large-area jets for both the T → tH and T → tZ decay modes. This search signature is based on triggers using high transverse momentum jets and is effective for high T mass (>1.0 TeV). In this mass range, the final state particles from the decays of each of the two daughter particles resulting from the T quark decay (the t and the H or Z) produce a single large-area jet. This leads to events with two large-area jets.
Each search is designed to be sensitive to T quark production in association with either a bottom quark or a top quark. Besides the primary motivation of exploring the possibility of electroweak production of a vector-like quark, this analysis can be viewed more broadly as two independent searches for high mass signatures of physics beyond the SM at the LHC. As such they provide potential for discovery of new physics, independent of the specific models discussed here. The paper is organized as follows: this section has given the motivation to search for the singly produced T quark with two distinct signatures and two decay modes. The CMS detector and event reconstruction are described in section 2. The data set and the modeling of signal and background processes are described in section 3. Reconstruction methods common to the two searches are discussed in section 4. The event selection criteria, background estimation, and results are described for the low-mass search in section 5, and for the high-mass search in section 6. Systematic uncertainties for both signatures are discussed in section 7. The overall results are presented in section 8 and summarized in section 9.

The CMS detector and event reconstruction
The central feature of the CMS apparatus is a superconducting solenoid of 6 m internal diameter, providing a magnetic field of 3.8 T. Within the solenoid volume are a silicon pixel and strip tracker, a lead tungstate crystal electromagnetic calorimeter (ECAL), and a brass and scintillator hadron calorimeter (HCAL), each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity coverage provided by the -4 -JHEP01(2020)036 barrel and endcap detectors. Muons are detected in gas-ionization chambers embedded in the steel flux-return yoke outside the solenoid.
The ECAL consists of 75 848 crystals covering |η| < 3.00. The HCAL cells have widths of 0.087 in pseudorapidity and 0.087 in azimuth (φ) in the region |η| < 1.74. In the η-φ plane, and for |η| < 1.48, the HCAL cells map onto 5×5 arrays of ECAL crystals to form calorimeter towers projecting radially outwards from close to the nominal interaction point. For |η| > 1.74, the coverage of the towers increases progressively with |η| to a maximum of 0.174 in ∆η and ∆φ. The forward calorimeters extend the calorimetric coverage for hadronic jets to |η| = 5.0.
Events of interest are selected using a two-tiered trigger system [28]. The first level, composed of custom hardware processors, uses information from the calorimeters and muon detectors to select events at a rate of around 100 kHz within a time interval of less than 4 µs. The second level, known as the high-level trigger, consists of a farm of processors running a version of the full event reconstruction software optimized for fast processing, and reduces the event rate to around 1 kHz before data storage.
In the reconstruction, the vertex with the largest value of summed physics-object p 2 T is taken to be the primary pp interaction vertex. The physics objects are the jets, clustered using the anti-k T jet finding algorithm [29,30], with the tracks assigned to the vertex as inputs.
A particle-flow algorithm [31] aims to reconstruct and identify each individual particle in an event, with an optimized combination of information from the various elements of the CMS detector. The energy of photons is directly obtained from the ECAL measurement, corrected for zero-suppression effects. The energy of electrons is determined from a combination of the electron momentum at the primary interaction vertex as determined by the tracker, the energy of the corresponding ECAL cluster, and the energy sum of all bremsstrahlung photons spatially compatible with originating from the electron track. The momentum of muons is obtained from the fitted trajectory of the corresponding track reconstructed from the tracker and the muon detectors. The energy of charged hadrons is determined from a combination of their momentum measured in the tracker and the matching ECAL and HCAL energy deposits, corrected for zero-suppression effects and for the response function of the calorimeters to hadronic showers. Finally, the energy of neutral hadrons is obtained from the corresponding corrected ECAL and HCAL energies.
Jet momentum is determined as the vectorial sum of all particle momenta in the jet, and is found from simulation to be within 5 to 10% of the true momentum over the whole p T spectrum and detector acceptance. Additional pp interactions within the same or nearby bunch crossings (pileup) can contribute additional tracks and calorimetric energy depositions to the jet momentum. To mitigate this effect, tracks identified to be originating from pileup vertices are discarded, and an offset correction is applied to correct for remaining contributions. Jet energy corrections are derived from simulation -5 -JHEP01(2020)036 combination. The SM tHq process simulation included all decay modes for the top quark and Higgs boson. The ttH sample was generated with the decay H → bb with a Higgs boson mass of 125 GeV using powheg 2.0. The WW sample was generated with hadronic decays using powheg 2.0. The ZZ sample was generated with hadronic decays using Madspin [43] and applying the FxFx merging procedure [44] for matching jets from the matrix element calculation with those from the parton shower. The WZ sample was inclusive and generated with pythia 8. The ZH sample was generated with Z → bb with powheg 2.0.
The SM background events comprised uniquely of jets produced through the strong interaction, referred to as QCD multijet events, were also considered in the design of the analyses. Two sets of simulated samples were used: a sample generated using pythia 8 that is binned in the invariant p T associated with the hard process and an H T -binned sample using MadGraph5 amc@nlo at leading order with up to four partons in the matrix element calculations, using the MLM jet matching scheme [45] with pythia fragmentation and showering. All simulated event samples were generated using the NNPDF3.0 PDF set except that based on pythia 8 which used the NNPDF2.3 PDF set [46].

Reconstruction methods and primary selection
Particle-flow anti-k T jets are used. Small-area jets, denoted AK4 jets, are defined using a distance parameter of 0.4, whereas large-area jets are defined with a distance parameter of 0.8 and are denoted as AK8 jets.
For both searches, b tagging is used to identify jets that contain b-flavored hadrons (b jets). The b tagging is applied to AK4 jets and also subjets reconstructed as part of an AK8 jet. Depending on the search, b tagging involves several secondary vertexing algorithms: the online and offline CSVv2 discriminators and the DeepCSV discriminators [47]. Offline b tagging working points are defined with light-flavor jet mistag rates of approximately 0.1% (tight), 1% (medium), and 10% (loose). For AK4 jets, the tight CSVv2 b tagging working point has an efficiency of 41% while the medium b tagging working points have CSVv2 and DeepCSV efficiencies of 63 and 68%, respectively. In the case of AK8 jets, a grooming algorithm [48] looks for jet substructure and b tagging is applied to the resulting subjets. For AK8 jets with p T of around 400 GeV, the medium b tagging working point has an efficiency per subjet of about 51%, whereas the loose b tagging working point has an efficiency per subjet of about 75%. The mistag rates of jets originating from c-flavored hadrons for the tight, medium, and loose b tagging working points are about 2, 12, and 37%, respectively.
The event selection requires that the events have at least one satisfied trigger condition among a set of unprescaled high-level trigger algorithms. The set is specific to each search strategy. The trigger conditions for the low-mass search strategy rely on online jet information and, for some trigger conditions, also on b tagging information. The event selection is dominated by the trigger condition with the lowest jet multiplicity and the lowest p T threshold. This condition requires at least six jets with p T > 30 GeV with two of them passing the b tagging online criteria. The trigger conditions of the high-mass search strategy consist of a scalar p T sum trigger formed from all jets with a summed p T -7 -JHEP01(2020)036 threshold of 900 GeV and an inclusive large-area single jet trigger with a p T threshold of 450 GeV. The high-mass search uses three other trigger conditions that include some loose jet substructure requirements and lower thresholds on either the scalar p T sum (700 GeV), the inclusive single-jet p T (360 GeV), or on two jets, with the higher (lower) jet p T required to exceed 300 (200) GeV.
At least one primary vertex must be found within 24 cm longitudinally and within 2 cm radially of the center of the luminous region.

Low-mass search
The low-mass search strategy uses the invariant mass reconstructed from five AK4 jets as the main discriminating variable. The event selection requires at least six jets to conform with the trigger requirements. The selection criteria are based on the properties of the signal final state, in the cases of t → bW and H/Z → bb. The final state is composed of two jets coming from the W decay and three b jets (two coming from the H/Z and one from the top quark decay). The main background processes consist of QCD multijet production and top quark pair production. These backgrounds are not expected to result in a resonance in the five-jet invariant mass variable. A reduction of the QCD multijet background is achieved by imposing b tagging, and by requiring events to be consistent with the presence of all of the relevant states (W, H/Z, and top quark). The presence of only one top quark candidate from the selected jets is used to reduce the tt background.

Event selection
The following criteria define the first part of the selection: • Small-area jet multiplicity. The event should have at least six AK4 jets with p T > 40 GeV within |η| < 4.5.
• Leading jets. The jets with the highest p T (leading jets) have larger p T in the signal than in most of the backgrounds. Therefore, the leading jets must have p T > 170, 130, and 80 GeV, for the leading, second-leading, and third-leading jets, respectively.
• b -tagged jets. The considered T quark decay leads to three b quarks while most backgrounds have at most two b quarks. In the signal region labeled as 3T , at least three b-tagged jets using the tight DeepCSV working point are required for jets with |η| < 2.4. Other b tagging working points are used to estimate the background using control samples in data.
3) H/Z b-tagged jet pair as the H/Z candidate and then selects the other jets making up the W and top quark candidates by minimizing the total χ 2 . This procedure is found to improve the signal-to-background ratio by 30% compared to simply choosing the combination with the best total χ 2 . Finally, the total χ 2 must not exceed 15 in order to ensure good quality of the H/Z, W, and top quark candidates. It is found that the five jets are correctly identified about 73 and 64% of the time for the narrow width tHbq and tZbq cases, respectively.
• Second top quark mass. A large fraction of tt events survive the requirement on at least three b-tagged jets. These originate from incorrect b tagging of the jet arising from a charm quark from the W boson part of one of the top quark decays. In order to reduce this background, we define the second top quark mass as the invariant mass formed by the H/Z candidate and the remaining highest p T jet not used in the χ 2 calculation. For tt events, the second top quark mass has a peak around 172 GeV and there are nearly no signal events expected in that region. Therefore we require that the second top quark mass is greater than 250 GeV. This leads to about a factor of two reduction in the tt background.
• b b mass. Finally, the reconstructed boson from the T candidate must have a mass larger than 100 GeV, if looking for a H, and the mass must be smaller than 100 GeV, if looking for a Z. This ensures that there is no overlap between the two channels.
The second part of the selection uses the presence of a top quark and a Higgs or Z boson in the event. The variables are chosen to be as model independent as possible and -9 -JHEP01(2020)036 the selection criteria are optimized using the figure of merit described in [50]. The selection criteria are described below.
• Relative H T . The relative H T variable is defined as p T (H/Z cand )+p T (t cand ) /H T . In single T quark production, most of the momentum should be carried by the top quark and H/Z candidates, therefore the relative H T is an extremely good discriminator against tt and multijet events. The H/Z and t candidates from the T candidate decay must have a relative H T greater than 0.40.
• Max(χ 2 ). The maximum among the χ 2 values defined in eqs. (5.1)-(5.3) is examined and is required to be less than 3.0. This criterion is highly correlated to the χ 2 criteria but represents a tighter condition that ensures that each mass is identified with high quality. It is equivalent to requiring a mass window of at most ± √ 3σ for each candidate.
• ∆R of jets from H /Z decay. Because of the large mass of the T quark (above 0.6 TeV), the H/Z decay tends to be boosted (but the b jets not completely merged). A small spatial separation of ∆R(b H/Z , b H/Z ) < 1.1 between the two b-tagged jets is required, leading to a reduction of the background. The ∆R is defined as the inter-jet separation in η-φ space (∆R = (∆η) 2 + (∆φ) 2 ), where ∆η and ∆φ are the corresponding inter-jet separations in pseudorapidity and azimuth (in radians).
• H /Z χ 2 . As most of the backgrounds do not contain a genuine Higgs boson, χ 2 H/Z is a very discriminating criterion for the Higgs boson decay channel. We require χ 2 H < 1.5 for the H case and χ 2 Z < 1.0 for the Z case. It is equivalent to a mass window of ±16.5 GeV for the Higgs boson and ±11.4 GeV for the Z boson. A tighter χ 2 requirement is made for Z candidates to avoid background contamination from lower masses and to reduce overlap with H candidates.
• ∆R of jets from W decay. Given the Lorentz boost of the W in signal events, a requirement of ∆R(j W , j W ) < 1.75 reduces the QCD multijet background, while retaining most of the signal.
• ∆R of jets from top quark decay. The top quark decay products tend to be Lorentzboosted (but the jets do not completely merge) for the signal. A spatial separation between the b-tagged jet and the W candidate that is used to make the top quark candidate of ∆R(b t , W) < 1.2 is required. This further reduces the QCD multijet background.
The total number of events selected from the data sample in the 3T signal region is 615 (290 for the tZ selection and 325 for the tH selection). The number of expected signal events is 7.6 for a T quark mass of 0.7 TeV, Γ/m T = 0.01, left-handed chirality, and a T quark produced in association with a bottom quark with a product of cross section and branching fraction of 89 fb for each channel. For this signal process, the selection efficiency is presented in table 1 together with various simulated background processes for the Higgs and Z boson decay channels.
-10 -JHEP01(2020)036  Table 1. Cumulative efficiencies for the low-mass search after applying event selections for the signal and main backgrounds in the Higgs boson decay channel (upper half) and the Z boson decay channel (lower half). The first and last rows of each section give the expected numbers of events normalized to the integrated luminosity of 35.9 fb −1 . Uncertainties are statistical only. The signal values are for a mass of 0.7 TeV, Γ/m T = 0.01, left-handed chirality, and a T quark produced in association with a bottom quark with a product of cross section and branching fraction of 89 fb for each channel. The "Other backgrounds" column includes W+jets, Z+jets, single top quark, and tt H processes. It has been checked that the ttH process does not present a resonance in the tH channel. The number of expected ttH events is comparable to the expected T signal.

Background estimation and validation
None of the SM backgrounds are expected to result in a resonance in the five-jet invariant mass, therefore the spectrum of the background five-jet invariant mass distribution should have a monotonically decreasing shape. However, the second part of the selection criteria tends to shape the five-jet invariant mass distribution. In order to evaluate the shape of the five-jet invariant mass distribution for the background in data, two regions that are independent from the main 3T signal region are defined using looser b tagging criteria. In these two regions it is important to ensure that no bias with respect to the selection criteria is present, as all backgrounds are estimated from data. The extraction of signal is done by fitting the signal and background simultaneously in all three regions. The b tagging does not strongly influence the kinematic distributions of objects used to construct the five-jet invariant mass. Therefore we relax the b tagging criteria required for three of the five jets forming the T candidate. The first new region is called the 3M signal region; it requires three medium b-tagged jets but excludes events with three tight b-tagged -11 -JHEP01(2020)036 jets, bringing information on the background and possible signal shapes. The second new region is denoted as the 2M 1L signal region; in order to have significant numbers of events in this background-dominated region and to keep events with similar kinematics to the 3T signal region, the b tagging criteria are relaxed to two medium and one loose b-tagged jets but excluding three medium b-tagged jets.
Two additional samples are defined and used to validate the method; one is enriched in QCD multijet events, the other in tt events. In order to define the QCD multijet enriched control sample, the χ 2 criterion is relaxed to 50, the Max(χ 2 H/Z , χ 2 W , χ 2 t ) criterion is inverted, and χ 2 t > 1 is required to reduce the fraction of tt events. The QCD multijet sample is subdivided into a 3T region and a 2M 1L region (excluding 3M ) based on the b-tagged jet configurations. For the tt control sample, the χ 2 criterion is relaxed to 50, the Max(χ 2 H/Z , χ 2 W , χ 2 t ) and χ 2 H/Z criteria are inverted, the ∆R(b H/Z , b H/Z ) criterion is relaxed to 1.5, and χ 2 t < 1.5 is required. The tt sample is subdivided into a 2T 1L (two tight and one loose b-tagged jets) region and a 2M 1L region (excluding 2T 1L) based on the b-tagged jet configurations. For the 2T 1L (2M 1L) region, one of the tight (medium) b-tagged jets must be from the top quark candidate. A summary of the criteria changed to define each region is presented in table 2. The fraction of expected signal events is of the order of 3% in the QCD multijet region and 1% in the tt 2T 1L region (for a cross section times branching fraction of 600 fb).
Relaxing the b tagging requirement induces a change in the b tagging efficiency depending on the p T and η of the jet. As p T and η are two highly correlated variables, a reweighting procedure using η and momentum is used. Weights are derived jet-by-jet for each channel and for each b tagging working point using the QCD multijet control region. These weights reflect the differences in the efficiency between loose and medium and between medium and tight b-tagged jets. For each event, the product of the weights for all three b-tagged jets is applied to correct for the change in the b tagging efficiency going from 3T to 3M and from 3M to 2M 1L.
The validation of the method is done with the QCD multijet and tt control regions. The shape of the five-jet invariant mass distribution is compared between the QCD 3T region and the QCD 2M 1L region reweighted as 3T ; it is found to be satisfactory using the Kolmogorov-Smirnov test. Similarly, when comparing the shape of the five-jet invariant mass for the the tt 2T 1L region and the tt 2M 1L region reweighted as 3T , acceptable consistency is found.
The five-jet invariant mass distributions for the QCD multijet 3T region, the tt 2T 1L region, and the 3M signal region are presented in figure 2 together with a potential signal for m T = 0.7 TeV corresponding to a product of cross section times branching fraction of 600 fb. The 2M 1L region distribution is overlaid after applying the b tagging weight computed with respect to the 3T , 2T 1L, and 3M regions, respectively. An acceptable agreement is observed in each sample.

Low-mass search results
For each decay channel, three independent regions based on the b-tagged jet requirements are examined: 3T (largest signal over background ratio), 3M , and 2M 1L (background  Figure 2. The five-jet invariant mass distribution (black points with error bars) for the tHbq channel after the full selection in the QCD multijet 3T control region (upper), the tt 2T 1L control region (middle), and the 3M signal region (lower). The superimposed blue histogram, labeled "background", is the reweighted 2M 1L region distribution, used as an estimate of the background shape, with its normalization adjusted to match the number of entries observed in each region. A potential narrow-width signal (dashed red histogram) is added on top of the blue histogram for m T = 0.7 TeV and Γ/m T = 0.01, for a product of signal cross section and branching fraction of 600 fb. The light blue shaded area corresponds to the statistical uncertainties in the corresponding 2M 1L region. The last bin in each distribution also contains events with masses exceeding 1.3 TeV.

JHEP01(2020)036
3T region 3M region 2M 1L region 3T *3M but not 3T *2M 1L but not 3M Table 2. Criteria defining the various signal and control regions. The first line of each section gives the b tagging requirements. The criteria that differ are preceded by an asterisk "*". The bb mass requirements are different for the H (m bb > 100 GeV) and Z channels (m bb < 100 GeV). dominated). The overall background shape and normalization is driven by the observations in the 2M 1L region. The background shape is linked between the regions by two transfer functions; these are derived from the b tagging weights to correct for b tagging differences between the regions. One transfer function links the 3T region to the 3M region and the other links the 3M region to the 2M 1L region. The transfer functions, based on simple parametrizations of the dependence of the reweighting values on the five-jet invariant mass, are displayed in figure 3.  The signal is parametrized as a Gaussian shape following the fit of the T quark reconstructed mass for each of the simulation samples for each region. The variations of the Gaussian fit parameters (mean and standard deviation) with T quark mass are fitted for each region. The parametrizations for the tH and tZ channels are found to be compatible. The systematic uncertainties are discussed in section 7. Here we note simply that they are all taken as correlated between the channels, except the ones related to the transfer functions and the normalization between regions. For a given channel, the fit procedure adjusts the shape of the background bin by bin based on the data in each of the regions, taking into account the transfer function between regions. The overall fit uses 40 GeV wide bins; it includes three bin-independent fit parameters, namely the signal strength and two relative normalization factors between each region, and fit parameters for the background contribution in each bin of the 2M 1L region. and tH candidates (right) for each region fitted: 2M 1L (upper row), 3M (middle row), and 3T (lower row). The signal hypothesis shown is a T with a mass of 0.7 TeV, narrow width, and a product of the cross section and branching fraction of 600 fb for the tZbq and tHbq channels. The data are represented by the black points with error bars, the signal hypothesis is represented by the red dashed line, the blue histogram gives the fitted background, and the light blue band represents the uncertainty in the background fit. the tH channel. The local significance is 3.0 standard deviations for a T quark mass of 0.68 TeV. For the same T quark mass the local significance is 0.2 standard deviations in the tZ channel. In a search for a vector-like quark, one expects similar branching fractions for the tH and tZ channels. No overall excess is measured when considering the fit of all six distributions.

JHEP01(2020)036
The result of the median value for the limit in terms of the appropriate cross section is calculated using the asymptotic CL s framework [51][52][53].
Similarly, results are obtained in the cases where the T quark has 10, 20, and 30% widths and in the case of production in association with a top quark. In all cases, the observed results show no evidence for a signal. The studies have also been performed for the case of right-handed chirality. No effect of the chirality is observed, indicating that the low-mass search is insensitive to this property. The resulting limits are reported in section 8.

High-mass search
This search strategy focuses on reconstructing the invariant mass of T → tH and T → tZ candidates formed from two large-area jets in the fully hadronic channel. The large-area jets are associated with events in which the top quark and the Higgs or Z boson are each highly Lorentz-boosted, and correspondingly the search targets T masses of 1 TeV and above. The background consists mostly of top quark pair production and QCD multijet production.

Particle tagging
To identify boosted t → bW → bqq , H → bb, and Z → bb decays, jet substructure techniques [54] are used, which remove soft and collinear radiation from the clustered jet constituents. Clusters of the remaining constituents are identified with each of the quarks from the decay of the original particle. The soft-drop algorithm [48,55] is used to groom the jets, using the soft radiation fraction parameter z = 0.1 and the angular exponent parameter β = 0. The algorithm yields two soft-drop subjets. The jet mass after applying the soft-drop algorithm will be referred to as the soft-drop mass. The pruning grooming algorithm [56] is also used, leading to the pruned jet mass. For pruning, the minimum subjet p T as a fraction of the parent jet p T is required to exceed 0.1 and the separation angle in η-φ space between the two subjets must exceed 0.5. Furthermore, the N -subjettiness algorithm [57] is used to further select jets with three or two substructures for the top quark jets, and the H and Z boson jets, respectively. Flavor tagging is applied to identify b quark subjets using the CSVv2 multivariate discriminator, in order to further enhance the signal purity and suppress backgrounds from non-tt+jets multijet processes. The particle tagging criteria for boosted top quark, and H and Z boson jets are as follows: • H jet: an AK8 jet with p T > 300 GeV must have a pruned jet mass within the range 105-135 GeV. The ratio of the N -subjettiness variables τ 2 /τ 1 of the jet is required to be <0.6. At least one of the two soft-drop subjets must pass the medium b tag criterion and the other subjet should pass at least the loose b tag criterion.
• Z jet: an AK8 jet with p T > 200 GeV must have a pruned jet mass within the range 65-105 GeV. The requirements on the N -subjettiness ratio τ 2 /τ 1 and subjet b tagging are the same as those for the H jet.
-17 -JHEP01(2020)036 • t jet: an AK8 jet with p T > 400 GeV and a soft-drop mass within the range 105-220 GeV is required. The jet should have an N -subjettiness ratio τ 3 /τ 2 < 0.57, indicating that the large-area jet is likely to have three subjets. The soft-drop subjet with the highest CSVv2 discriminator value should pass the medium b tag criterion [58].
In addition, reversed-H-tagged, reversed-Z-tagged, and reversed-t-tagged jets are defined, with the same kinematic and N -subjettiness requirements as those for their tagged counterparts, but with complementary b tagging criteria, as follows: • Reversed-H -tagged jet: same criteria as for an H-tagged jet but with one subjet passing the medium b tag criterion and the other subjet failing the loose b tag criterion.
• Reversed-Z -tagged jet: same criteria as for a Z-tagged jet but with both subjets failing the loose b tag criterion.
• Reversed-t -tagged jet: same criteria as for a t-tagged jet but with the highest softdrop subjet b discriminant failing the medium b tag criterion.
The reversed-Z tag is defined differently from the reversed-H tag so that sensitivity to a potential tZ signal, including efficiency from Z → qq(q = b) decays, can be retained.

Event selection
Only events satisfying the following primary selection criteria are considered further, either as candidates for the signal or for the associated background control regions: • Small-area jet multiplicity: at least four AK4 jets with p T > 30 GeV and |η| < 5.
• Leading jet: the highest-p T AK8 jet should have p T > 400 GeV and a pruned mass greater than 50 GeV.
• Scalar p T sum: the scalar sum of the transverse momenta of the two highest-p T AK8 jets should exceed 850 GeV.
• Extra jet multiplicity: at least two of the AK4 jets should be separated by a distance ∆R > 1.2 from the two leading AK8 jets.
• Forward extra jet: at least one of the extra AK4 jets defined above should have |η| > 2.4.
The last two criteria are imposed in order to ensure evidence that the selected events contain a diquark (bq or tq) system that is produced in association with the T quark, where the quark (q) associated with the vector boson tends to be forward. Events passing the selection requirements above are by design expected to be almost fully efficient for the trigger requirements.

JHEP01(2020)036
Signal candidates passing the primary event selection are further categorized according to the following criteria. At this stage, the focus is on choosing the best pair of AK8 jets for constructing the T candidate mass.
• Double-tag: each of the two highest-p T selected AK8 jets must have either a t tag or an H/Z tag. Furthermore, one of these jets must have a t tag and the other an H/Z tag. In the ambiguous case, where both jets are t-tagged and H/Z-tagged, the higher-p T jet is assigned as the top quark candidate, and the lower-p T jet as the Higgs/Z boson candidate.
• H /Z tag isolation: motivated by reducing tt background, events are rejected if any AK4 jet is separated from the H/Z candidate jet by 0.55 < ∆R(j, H/Z) < 0.9.
Signal regions, S H and S Z , are defined using these criteria for the tH and tZ searches respectively. Related background control regions are defined and are described in section 6.3.
We denote the measured AK8 jet four-vectors corresponding to the top quark, Higgs boson and Z candidates as P t = (E t , p t ), P H = (E H , p H ) and P Z = (E Z , p Z ) and construct a corrected T mass-sensitive observable, m T . This observable takes advantage of the knowledge of the top quark and Higgs/Z boson masses to correct the masses of the AK8 jets; it is inspired by a similar variable used in ref. [59] that was based on a suggestion in ref. [60]. The reconstructed mass of the T candidate from the tX dijet system (m j tX ), with X = H/Z, is adjusted for deviations of the reconstructed top quark and Higgs/Z boson AK8 masses (m j t , m j X ) from the known t and H/Z masses [49], as follows: where the j superscripts denote jet-based measured mass quantities. This estimator is found to have better performance in terms of mass resolution by about 10% compared to the uncorrected mass estimator. It has also been verified that it is accompanied by a commensurate reduction in background acceptance. Example distributions of m T in the signal regions, S H and S Z , are shown in figure 5 for T masses of 1.2 and 1.8 TeV for both narrow and large widths. It can be seen that in the S Z region, in addition to the expected efficiency for the tZbq process, there is also substantial efficiency for the tHbq process; the reverse is not true for the S H region.

Background estimation
The m T distribution is used to determine the amount of signal potentially present in the data. A fit is performed that takes advantage of the relatively narrow signal shape in m T compared to the broader shape expected from the backgrounds. After the primary event selection criteria, the main backgrounds are tt and QCD multijet events. The tt background is estimated using simulated events and the QCD multijet background is estimated using control regions in data. Other smaller background sources, designated as "other" and consisting of W+jets, Z+jets, single top quark, tHq, ttH, ttW, ttZ, WW, ZZ, WZ, WH, For presentation purposes, the cross sections for tHbq, tZbq, tHtq and tZtq are set equal to 1 pb for all masses and fractional widths and normalized to the integrated luminosity of the data set. The left column illustrates potential signals with a range of masses and widths for the tHbq and tZbq channels. The right column illustrates potential signals for one mass and a large width for all four processes including also tHtq and tZtq. and ZH, are all estimated from simulation. Background templates are constructed using a smoothing procedure that fits the m T distributions with an empirical functional form. A simultaneous fit is then performed in eight regions: four regions designed to test for tH signal contributions and four regions designed to test for tZ signal contributions. The fit examines all eight regions and fits the m T distributions for the amounts of signal and QCD multijet background contributions using tt and other backgrounds predicted from simulation. Fits are performed under three hypothetical signal scenarios, tH only, tZ only, and tH+tZ. In the latter case, the small difference in cross section for tHbq relative to tZbq is taken from the singlet model calculation. For tHtq relative to tZtq, the difference from the (TB) doublet model calculation is used.
The criteria described in section 6.2 define the main signal regions (S H and S Z ) using the t-, H-, and Z-tagged jets. The additional six mutually exclusive regions are used as control regions in the fit and to predict the shapes and normalization of the QCD multijet background from data; these are denoted Q H , T H , R H for the tH signal and Q Z , L Z , R Z for the tZ signal. Regions Q H and Q Z are control regions for the QCD multijet background. Region T H is a tt enriched control region, while region L Z has sensitivity to non-bb Z -20 -JHEP01(2020)036  Table 3. Overview of the criteria used to define the mutually exclusive Q H , T H , R H , S H , Q Z , L Z , R Z , and S Z regions. These are based on the particle tagging criteria for t, H, and Z jets and for the reversed-t-tagged, reversed-H-tagged, and reversed-Z-tagged jets using the two highest p T AK8 jets.
decays. Regions R H and R Z serve as control regions that test the rejection of QCD multijet events by the H tag and Z tag criteria.
For the definition of the regions, only the two highest p T AK8 jets are examined and each jet must be either tagged or reversed-tagged. The signal region S H requires a t-tagged jet and an H-tagged jet. Region Q H requires a reversed-t-tagged jet and a reversed-H-tagged jet. Region T H requires a t-tagged jet and a reversed-H-tagged jet. Region R H requires a reversed-t-tagged jet and an H-tagged jet. Regions R H and S H include the isolation requirement on the H-tagged jet, while regions Q H and T H do not include an isolation requirement around the reversed-H-tagged jet. These choices define a T H region that is enhanced in tt events, thus providing a suitable control region. The order of assigning events starts with region S H , then proceeds with region T H , R H , and then Q H , where each subsequent region is not allowed to contain any of the events assigned to the previous region. In ambiguous cases, as is done for the signal region, the higher-p T jet is assigned as the t-tagged or reversed-t-tagged jet.
There are four analogous regions designated by Q Z , L Z , R Z , and S Z for events with tand Z-tagged jets for events that do not satisfy the Q H , T H , R H , or S H region definitions. The Q Z region has a reversed-t-tagged jet and a reversed-Z-tagged jet. The L Z region has a t-tagged jet and a reversed-Z-tagged jet. The R Z region has a reversed-t-tagged jet and a Z-tagged jet. These criteria make region L Z sensitive to hadronic decays of Z bosons other than bb from both the T → tZ signal and the background. Regions R Z and S Z include the isolation requirement on the Z-tagged jet, while regions Q Z and L Z do not include an isolation requirement around the reversed-Z-tagged jet. Events can only be assigned to one of the eight regions. These criteria lead to a well-defined m T value for each event corresponding to the mass assignments implicit in the tagging criteria. Table 3 summarizes the criteria for the eight regions.
A simultaneous fit is performed to the m T distributions in each of the eight regions to determine the amount of signal present. The signal templates are taken directly from the -21 -JHEP01(2020)036 simulated signal samples. The tt and other background contributions are found using the smoothed templates. The smoothed QCD multijet background shape is determined from the data in region Q H for the regions Q H , T H , R H , and S H and in region Q Z for the Q Z , L Z , R Z , and S Z regions. A binned likelihood fit is performed using the m T variable, with 50 bins of 50 GeV width over the range 0.6-3.1 TeV. All background components, except QCD multijet, are constrained within uncertainties using predictions from MC simulations. The numbers of QCD multijet events in regions S H and S Z are estimated using the control regions. The amount of QCD multijet background in each of the control regions Q H , T H , R H , Q Z , L Z , and R Z is found from the following. If the expected numbers of QCD multijet events in the four regions are N Q H , N T H , N R H , N S H then, if the Higgs boson and reversed-Higgs boson tagging are independent of the top quark tagging and reversed-top quark tagging criteria, one may write: Using eq. (6.2), one may then make a data-based prediction of the number of QCD multijet events in the signal region S H . Similarly, the number of QCD multijet background events in region S Z can be estimated from which are both consistent with the expected value of unity within their combined statistical and systematic uncertainties. The systematic uncertainties are discussed in section 7. The predictions of the QCD multijet contributions, which include the overall effect of the shape and the normalization, have been validated using a fit that uses a different set of eight control regions in data. These control regions are mutually exclusive to the previously defined eight regions, contain larger numbers of events, and are defined using loose b tagging criteria on the t tag and the reversed-t tag in a sample that excludes events with forward (|η| > 2.4) jets. The double ratio is taken to be fully correlated between the tH and tZ regions, with a central value of 1.0, and an assigned uncertainty of 0.6. This uncertainty is assessed based on the measured double ratios from the relatively low number of events in the QCD multijet simulation. The fits to data are observed to be insensitive to the exact uncertainty used in that the preferred value for the double ratio tends to be close to unity. Limits on the signal strength are extracted by fitting the signal and backgrounds to the data. The fit finds the amount of signal as well as the amounts of QCD multijet background in each of the eight regions. The QCD multijet event yields in the regions Q H , T H , and R H are allowed to float freely. The QCD multijet event yield in the region S H   Table 4. Post-fit numbers of events for the Q H , T H , R H , and S H regions for the data and specified background sources, for the overall eight-region background-only fit. The uncertainties include both the statistical and systematic components. The fitted background sums depend on the data. The expected event yields for various signal samples are also listed with statistical uncertainties only, along with the corresponding masses (TeV) and cross sections (fb). The fractional width considered is 30%. The percent efficiency in region S H is also noted in parentheses, alongside the event yield.
is constrained using eq. (6.2) with the double ratio being modeled with a Gaussian prior. The same procedure is used for regions Q Z , L Z , R Z , and S Z . All other uncertainties are treated either using log-normal priors (for those that change the event yields only), or as Gaussian priors (with shape variations corresponding to the ±1 standard deviation change in those uncertainties that affect the m T distributions as well as the yields). The fitting method is validated with a data sample based on simulation. The fit uses the modified frequentist approach for confidence levels, taking the profile likelihood ratio as the test statistic [51,52] and using the asymptotic approximation for limit setting [53].    Table 5. Post-fit numbers of events for the Q Z , L Z , R Z , and S Z regions for the data and specified background sources, for the overall eight-region background-only fit. The uncertainties include both the statistical and systematic components. The fitted background sums depend on the data. The expected event yields for various signal samples are also listed with statistical uncertainties only, along with the corresponding masses (TeV) and cross sections (fb). The fractional width considered is 30%. The percent efficiency in region S Z is also noted in parentheses, alongside the event yield.

High-mass search results
sections for the two production modes (pp → Tbq and pp → Ttq). These upper limits are reported in section 8 together with the limits from the low-mass search for four fractional width (Γ/m T ) values and the two decay modes (tH and tZ) as well as their sum (tH+tZ).

Systematic uncertainties
The systematic uncertainties can be classified into those that affect the overall yields of the signal and the background processes, and those that affect the invariant mass distributions m T (for the low-mass search) and m T (for the high-mass search). The sources of systematic uncertainties and their effects on the signal and the background are summarized in table 6. The trigger efficiency for the low-mass category is measured in data and is found to be about 97% with an assigned uncertainty of 3%. The trigger efficiency for the high-mass analysis is measured using hadronic triggers to be over 99.5%. There is a mild dependence on m T that is evaluated using a muon-based monitor trigger. The maximum variation is 3% and this is taken as the uncertainty in the overall event yields.
The jet energy scale uncertainties depend on the p T and η of the jets [61]. The jet energy resolution in data is found to be worse than that in simulation, and the discrepancy is corrected by applying an extra smearing to the energy of jets in simulated events. Both the jet energy scale and resolution uncertainties affect the overall scale and shapes of the invariant mass distributions. The uncertainties in the H and Z jet mass scale and resolution and the N -subjettiness selection [61] affect the high-mass search. The jet mass scale, resolution, and the Nsubjettiness selection efficiency were measured in a sample of tt+jets events with one top quark decaying leptonically and the other top quark decaying hadronically (semileptonic tt+jets events). The hadronically decaying top quark is boosted enough to produce a merged W → qq jet separated from a b jet. The jet mass scale was evaluated to be unity with an uncertainty of 2%. An additional uncertainty of about 10% was derived using simulations to account for the difference between the jet showering for a W → qq jet and an H → bb or Z → bb jet, using parton shower models from pythia 8 and herwig++ 2.7.1 [62]. The EE5C tune [63] is used for herwig++. The jet mass resolution was found to be 23% larger in the data than in the simulation, with an uncertainty of 18%. The ratio of the N -subjettiness selection efficiency in the data to the simulation (scale factor) was found to be 1.11 ± 0.08 with an additional jet p T -dependent uncertainty. The jet mass scale and N -subjettiness scale factor uncertainties affect only the event yields whereas the jet mass resolution affects both the yields and the m T distributions. The b tagging [47] efficiency scale factor uncertainties for AK4 jets are measured in multijets and tt+jets samples, separately for b quark jets and for light-quark and gluon jets. These uncertainties affect only the signal event yields for the low-mass analysis. For the high-mass analysis, the b tagging efficiency scale factor uncertainty for subjets is measured using multijet and tt+jets samples and is applied to the subjets for H and Z jets. These affect both the event yields and the m T distributions.
For the top quark jet tagging used in the high-mass search, the scale factor is measured using semileptonic tt+jets events, with a boosted top quark jet, using the method from ref. [58]. The scale factor with uncertainty is 1.07 +0.05 −0.03 for each top quark jet, and affects both the event yields and the m T distributions.
The PDF uncertainties were evaluated with the PDF4LHC procedure [64], by reweighting the simulated events using the eigenvectors of the NNPDF3.0 PDF set; it was found to change the overall event yields by 0. 5 Renorm./fact. scales -Shape Table 6. The systematic uncertainties in the signal and background yields for each search. The uncertainties marked "Shape" affect both the event yields and the distributions.
uncertainties were estimated by doubling and halving the nominal values used in the simulations. These were found to affect the high-mass search event yields for both the signal and the tt+jets background.
The uncertainty in the measurement of the integrated luminosity amounts to 2.5% [65] and affects the overall event yields of all simulated processes. The uncertainty associated with the mismodeling of pileup is evaluated based on a 4.6% variation on the pp total inelastic cross section [66] and affects the overall simulated event yields for both analyses.
The systematic uncertainties in the background estimation for the low-mass search are taken into account by inflating the uncertainty in the slope parameter of the transfer functions. This includes an uncertainty component justified from the difference observed when the weights are computed on light-flavor versus heavy-flavor jets and an uncertainty component coming from the uncertainties in the fit arising from the b tagging efficiency. Based on the observed differences, the slope parameter uncertainty is increased by a factor of four for the 2M 1L to 3M region transfer function and by a factor of three for the 3M to 3T region transfer function. For the high-mass search, the uncertainty in the background prediction includes the systematic uncertainty in the double ratio used to constrain the QCD multijet event yields in the different control and the signal regions (section 6.3). While the initial uncertainty in this double ratio is assigned to be 60%, the postfit uncertainty ranges from 23-25%, depending on signal model.

JHEP01(2020)036
The tt+jets cross section uncertainty is +6.1 −5.5 % at next-to-next-to-leading order [67], while that in the W+jets cross section is 3.8% [68,69]. These uncertainties affect only the high-mass search where both these backgrounds are taken from simulations. These uncertainties are propagated to the estimation of the QCD multijet background from the data, where the non-QCD multijet background components are subtracted from the data. The uncertainties also include m T -dependent modeling uncertainties for the various background components.

Search results
The low-mass search is sensitive to masses from 0.6 to 1.2 TeV, while the high-mass search is sensitive to masses from 0.7 to 2.6 TeV, but with its main sensitivity starting around 1 TeV. The limits presented here correspond to a confidence level of 95%. They are based on whichever of the two searches for each considered mass has the best estimated expected sensitivity for each decay channel (tH, tZ, and the sum), production mode, and fractional width. The limits from each search considered separately are presented for completeness in appendix A.
Upper limits are set on the cross sections for the two production modes (pp → Tbq and pp → Ttq) with the two decay modes (tH and tZ) as well as their sum (tH+tZ), for four fractional width (Γ/m T ) values. The individual limits for the tH decay mode neglect potential contributions from the tZ and bW decay modes and, similarly, the individual limits for the tZ decay mode neglect potential contributions from the tH and bW decay modes. The (tH+tZ) sum limits are computed assuming the relative cross sections for the two channels calculated in the respective model for a particular width based on ref. [70]. For pp → Tbq, this corresponds to the T singlet model with κ W = κ H = κ Z and for pp → Ttq, to the (TB) doublet model with κ H = κ Z and κ W = 0, where κ W , κ H , and κ Z denote the T coupling parameter to bW, tH, and tZ, respectively. Only potential contributions from bW are neglected for the (tH+tZ) experimental limits. The calculated cross sections for tH and tZ are approximately the same but unequal. This is due both to mass effects explicit in the decay width expressions of ref. [5] and to additional amplitude contributions associated with the 2 → 4 finite width calculation arising from T quarkmediated Feynman diagrams relevant to the large-width regime, such as the t-channel exchange of a T quark. The fractional width values include 10, 20, and 30%, and the narrow-width case. Given the estimated effective Gaussian mass resolution of about 5%, the experimental limits set using the narrow-width simulated signal samples are applicable to (Breit-Wigner) fractional widths of up to about 5%. The experimental upper limits on cross sections are generally more restrictive for smaller widths given the narrower line shapes. The computed cross sections are found to depend approximately linearly on the fractional width. Figure 8 shows the cross section upper limits for Tbq production for the tHbq and tZbq channels, and their sum. The figure includes results for a narrow fractional width, corresponding to Γ/m T ≤ 0.05, and for a fractional width of 10%. Results for the same quantities are shown in figure 9 for fractional widths of 20 and 30%. Similar results for -28 -

JHEP01(2020)036
Ttq production for the tHtq and tZtq channels, and their sum are shown in figure 10 for narrow fractional width and for 10% fractional width, and the corresponding results for fractional widths of 20 and 30% are shown in figure 11. Superimposed on these results are the expected cross sections for the T singlet model and for the (TB) doublet model. The tH and tZ branching fractions for a narrow width are both approximately 25% for the T singlet model and 50% for the (TB) doublet model.
The presented results are evaluated using left-handed chirality for the T in the pp → Tbq cases and right-handed chirality for the pp → Ttq cases. Studies with the opposite chirality for narrow width T → tH and T → tZ have shown similar sensitivity with the differences being small for the low-mass search and at most 10% for the high-mass search.
The results of this search show that the observed limits are consistent with the expected limits arising from the background-only hypothesis. Depending on the fractional width, it can be seen that this search has expected sensitivity for T masses within the T singlet model up to 1.28 TeV (for tHbq + tZbq with 30% fractional width). For T masses below 1 TeV, the models of the associated production with a bottom quark are strongly constrained by the observed limits from the low-mass search signature, which are generally more stringent than expected above 0.75 TeV; for the T singlet model masses in the range 0.70 to 1.00 TeV are excluded at 95% CL for some fractional widths between 5 and 30%. For the T quark masses above 1.00 TeV, the observed limits are above model predictions, and so no exclusion at 95% CL is possible with this data set in this mass range. The models with the associated production with a top quark have lower cross sections with a median expected sensitivity for T quark masses within the (TB) doublet model of 0.82 TeV for the largest fractional width of 30%. However, for this model, no range of masses is excluded at 95% CL for any of the masses and fractional widths considered here.
The presented experimental upper limits on the cross sections are of general interest in more model independent approaches and demonstrate the great potential of electroweak single production searches to test vector-like quark production at mass scales far beyond those accessible with pair production.

Summary
A search for a vector-like top quark partner T in the electroweak single production mode with fully hadronic final states has been performed using pp collision events at √ s = 13 TeV collected by the CMS experiment in 2016. The data sample corresponds to an integrated luminosity of 35.9 fb −1 . The T quarks are assumed to couple only to the standard model third-generation quarks. The decay channels exploited are T → tH and T → tZ with the hadronic decay of the top quark and primarily the bb decay of the Higgs and Z bosons. This search is designed to be sensitive to T quark fractional widths of up to 30% and a wide range of masses. The background is mostly due to standard model tt+jets and QCD multijet events with some contributions from W+jets processes. No significant excess of data above the standard model background is observed and upper limits at 95% confidence level are set on σ B(T → tH) and σ B(T → tZ), which vary between 2 pb and 20 fb for T masses ranging from 0.6 to 2.6 TeV in the Tbq and Ttq production channels. Results from -29 -JHEP01(2020)036  Figure 8. The observed and median expected upper limits at 95% CL on the cross sections for production associated with a bottom quark for the tHbq (upper row) and tZbq (middle row) channels, and their sum, tHbq + tZbq (lower row), for different assumed values of the T quark mass. The inner (green) band and the outer (yellow) band indicate the regions containing 68 and 95%, respectively, of the distribution of limits expected under the background-only hypothesis. The left column is for a narrow fractional width (Γ/m T ≤ 0.05) and the right column is for a fractional width of Γ/m T = 0.1. The vertical dashed lines are the crossover points in sensitivity that indicate the mass intervals used for presenting the low-mass and high-mass search results. The dashed red and dot-dashed blue curves are for the T singlet model. Given the specified width, the couplings are implicit in the model. Two curves corresponding to Γ/m T = 0.05 (dot-dashed blue) and Γ/m T = 0.01 (dashed red) are shown for the narrow fractional width.  Figure 9. The observed and median expected upper limits at 95% CL on the cross sections for production associated with a bottom quark for the tHbq (upper row) and tZbq (middle row) channels, and their sum, tHbq + tZbq (lower row), for different assumed values of the T quark mass. The inner (green) band and the outer (yellow) band indicate the regions containing 68 and 95%, respectively, of the distribution of limits expected under the background-only hypothesis. The left column is for a fractional width of 20% and the right column is for a fractional width of 30%. The vertical dashed lines are the crossover points in sensitivity that indicate the mass intervals used for presenting the low-mass and high-mass search results. The dashed red curves are for the T singlet model. Given the specified width, the couplings are implicit in the model.    Figure 11. The observed and median expected upper limits at 95% CL on the cross sections for production associated with a top quark for the tHtq (upper row) and tZtq (middle row) channels, and their sum, tHtq +tZtq (lower row), for different assumed values of the T quark mass. The inner (green) band and the outer (yellow) band indicate the regions containing 68 and 95%, respectively, of the distribution of limits expected under the background-only hypothesis. The left column is for a fractional width of 20% and the right column is for a fractional width of 30%. The vertical dashed lines are the crossover points in sensitivity that indicate the mass intervals used for presenting the low-mass and high-mass search results. The dashed red curves are for the (TB) doublet model. Given the specified width, the couplings are implicit in the model.

JHEP01(2020)036
combining the two decay channels assuming equal couplings are also reported. Compared with prior electroweak single production searches, this search is significantly more sensitive for T → tH. The search gives the first constraints using this production mode on T → tZ for hadronic decays of the Z boson. These results are competitive with those from searches for T → tZ using other Z decay modes. The combined T → tH and T → tZ results for associated production with a bottom quark lead to constraints on T quarks in the T singlet model for masses below 1.00 TeV. The expected sensitivity for this model extends to 1.28 TeV (for 30% fractional width), which is comparable to the mass reach of the most stringent pair production searches.

Acknowledgments
We congratulate our colleagues in the CERN accelerator departments for the excellent performance of the LHC and thank the technical and administrative staffs at CERN and at other CMS institutes for their contributions to the success of the CMS effort. In addition, we gratefully acknowledge the computing centers and personnel of the Worldwide LHC Computing Grid for delivering so effectively the computing infrastructure essential to our analyses. Finally, we acknowledge the enduring support for the construction and operation of the LHC and the CMS detector provided by the following funding agencies: BMBWF and FWF (

JHEP01(2020)036
Academy of Sciences, the New National Excellence ProgramÚNKP, the NKFIA research grants 123842, 123959, 124845, 124850, 125105, 128713, 128786, and 129058 (Hungary) Figure 12. The observed and median expected upper limits at 95% CL on the cross sections for production associated with a bottom quark for the tHbq (upper row) and tZbq (middle row) channels, and their sum, tHbq + tZbq (lower row), for different assumed values of the T quark mass. The inner (green) bands and the outer (yellow) bands indicate the regions containing 68 and 95%, respectively, of the distribution of limits expected under the background-only hypothesis. The left column is for a narrow fractional width (Γ/m T ≤ 0.05) and the right column is for a fractional width of Γ/m T = 0.1. The dashed red curves are for the T singlet model. Given the specified width, the couplings are implicit in the model.  Figure 13. The observed and median expected upper limits at 95% CL on the cross sections for production associated with a bottom quark for the tHbq (upper row) and tZbq (middle row) channels, and their sum, tHbq + tZbq (lower row), for different assumed values of the T quark mass. The inner (green) bands and the outer (yellow) bands indicate the regions containing 68 and 95%, respectively, of the distribution of limits expected under the background-only hypothesis. The left column is for a fractional width of 20% and the right column is for a fractional width of 30%. The dashed red curves are for the T singlet model. Given the specified width, the couplings are implicit in the model.  Figure 14. The observed and median expected upper limits at 95% CL on the cross sections for production associated with a top quark for the tHtq (upper row) and tZtq (middle row) channels, and their sum, tHtq + tZtq (lower row), for different assumed values of the T quark mass. The inner (green) bands and the outer (yellow) bands indicate the regions containing 68 and 95%, respectively, of the distribution of limits expected under the background-only hypothesis. The left column is for a narrow fractional width (Γ/m T ≤ 0.05) and the right column is for a fractional width of Γ/m T = 0.1. The dashed red curves are for the (TB) doublet model. Given the specified width, the couplings are implicit in the model.  Figure 15. The observed and median expected upper limits at 95% CL on the cross sections for production associated with a top quark for the tHtq (upper row) and tZtq (middle row) channels, and their sum, tHtq +tZtq (lower row), for different assumed values of the T quark mass. The inner (green) bands and the outer (yellow) bands indicate the regions containing 68 and 95%, respectively, of the distribution of limits expected under the background-only hypothesis. The left column is for a fractional width of 20% and the right column is for a fractional width of 30%. The dashed red curves are for the (TB) doublet model. Given the specified width, the couplings are implicit in the model.

JHEP01(2020)036
Open Access. This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.
[9] ATLAS collaboration, Search for pair production of vector-like top quarks in events with one lepton, jets and missing transverse momentum in √ s = 13 TeV pp collisions with the ATLAS detector, JHEP 08 (2017)  [13] ATLAS collaboration, Search for pair production of up-type vector-like quarks and for four-top-quark events in final states with multiple b-jets with the ATLAS detector, JHEP 07