Search for single production of vector-like quarks decaying to a top quark and a $\mathrm {W}$ boson in proton–proton collisions at $\sqrt{s} = 13 \,\text {TeV}$

Abstract

A search is presented for the single production of vector-like quarks in proton–proton collisions at $\sqrt{s}=13\,\text {TeV}$. The data, corresponding to an integrated luminosity of 35.9$\,\text {fb}^{-1}$, were recorded with the CMS experiment at the LHC. The analysis focuses on the vector-like quark decay into a top quark and a $\mathrm {W}$ boson, with one muon or electron in the final state. The mass of the vector-like quark candidate is reconstructed from hadronic jets, the lepton, and the missing transverse momentum. Methods for the identification of $\mathrm {b}$ quarks and of highly Lorentz boosted hadronically decaying top quarks and $\mathrm {W}$ bosons are exploited in this search. No significant deviation from the standard model background expectation is observed. Exclusion limits at 95% confidence level are set on the product of the production cross section and branching fraction as a function of the vector-like quark mass, which range from 0.3 to 0.03$\,\text {pb}$ for vector-like quark masses of 700 to 2000$\,\text {GeV}$. Mass exclusion limits up to 1660$\,\text {GeV}$ are obtained, depending on the vector-like quark type, coupling, and decay width. These represent the most stringent exclusion limits for the single production of vector-like quarks in this channel.

Introduction

The discovery of the Higgs boson ($\mathrm {H}$) [1, 2] with a mass of 125$\,\text {GeV}$ completes the particle content of the standard model (SM). Even though the SM yields numerous accurate predictions, there are several open questions, among them the origin of the $\mathrm {H}$ mass stability at the electroweak scale. Various models beyond the SM have been proposed that stabilise the $\mathrm {H}$ mass at the measured value; some examples are Little Higgs [3,4,5] or Composite Higgs models [6], in which additional top quark partners with masses at the TeV scale are predicted. Since the left- (LH) and right-handed (RH) chiral components of these particles transform in the same way under the SM electroweak symmetry group, they are often referred to as “vector-like quarks” (VLQs). In contrast to a fourth chiral quark generation, their impact on the $\mathrm {H}$ properties is small, such that VLQs have not been excluded by the measurements of $\mathrm {H}$ mediated cross sections [7,8,9].

Several searches for VLQs have been performed at the CERN LHC, setting lower exclusion limits on the VLQ mass $m_\mathrm {{VLQ}}$ [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. Many of these analyses study the pair production of VLQs via the strong interaction. In contrast, the analysis presented here searches for the single VLQ production via the weak interaction, where a hadronic jet is emitted at a low angle with respect to the beam direction. Furthermore, VLQs with enhanced couplings to the third generation quarks (i.e. VLQ $\mathrm {B}$ and $X_{5/3}$ quarks with an electric charge of 1 / 3 and 5 / 3 respectively) are produced in association with a bottom ($\mathrm {b}$) or top ($\mathrm {t}$) quark, leading to the $\mathrm {B}$ +$\mathrm {b}$, $\mathrm {B}$ +$\mathrm {t}$, and $X_{5/3}$+$\mathrm {t}$ production modes.

While a VLQ $\mathrm {B}$ quark could decay into the $\mathrm {H} \mathrm {b}$, $\mathrm {Z} \mathrm {b}$, or $\mathrm {t}\mathrm {W}$ final state, a VLQ $X_{5/3}$ quark could only decay into the $\mathrm {t}$ $\mathrm {W}$ final state. This search focuses on the $\mathrm {t}$ $\mathrm {W}$ final state. In Fig. 1, two leading-order (LO) Feynman diagrams are shown for the single production of $\mathrm {B}$ and $X_{5/3}$ quarks and their decay into $\mathrm {t}$ $\mathrm {W}$. This paper presents the first search of this signature in proton–proton ($\mathrm {p}\mathrm {p}$) collision data recorded at a centre-of-mass energy of 13 $\,\text {TeV}$. Results at $\sqrt{s} = 8 \,\text {TeV}$ have been obtained by the ATLAS collaboration [32].

Fig. 1

Leading order Feynman diagrams for the production of a single vector-like $\mathrm {B}$ or $X_{5/3}$ quark in association with a $\mathrm {b}$ (left) or $\mathrm {t}$ (right) and a light-flavour quark, and the subsequent decay of the VLQ to $\mathrm {t}\mathrm {W}$

In this analysis, final states with a single muon or electron, several hadronic jets, and missing transverse momenta $p_{\mathrm {T}} ^\text {miss}$ are studied. Because of the high mass of the VLQ, the $\mathrm {t}$ and $\mathrm {W}$ can have high Lorentz boosts, leading to highly collimated decays of the $\mathrm {W}$ boson, the top quark and non-isolated leptons. For signal events, the mass of the $\mathrm {B}$ and $X_{5/3}$ quarks can be reconstructed using hadronic jets, the lepton, and the $p_{\mathrm {T}} ^\text {miss}$. The associated $\mathrm {b}$ and $\mathrm {t}$, as well as the leptons originating from their decay, have much lower transverse momenta $p_{\mathrm {T}}$ and are not considered for the reconstruction or selection.

The dominant SM background processes are top quark pair (${\mathrm {t}\overline{\mathrm {t}}}$) production, $\mathrm {W}$+jets and $\mathrm {Z}$+jets production, single $\mathrm {t}$ production, and multijet production via the strong force. All SM backgrounds contributing to this search are predicted from dedicated control regions in data, defined through the absence of a forward jet.

This paper is organised as follows: Sect. 2 provides a description of the CMS detector. Section 3 introduces the data set and the simulated events. This is followed by the event selection in Sect. 4, as well as by the description of the reconstruction of the VLQ mass in Sect. 5. In Sect. 6, a method to estimate the background is discussed. Systematic uncertainties are detailed in Sect. 7. The final results of the analysis, as well as the statistical interpretation in terms of exclusion limits, are discussed in Sect. 8.

The CMS detector and physics objects

The central feature of the CMS apparatus is a superconducting solenoid of 6$\,\text {m}$ internal diameter, providing a magnetic field of 3.8$\,\text {T}$. Within the solenoid volume are a silicon pixel and strip tracker, a lead tungstate crystal electromagnetic calorimeter (ECAL), and a brass and scintillator hadron calorimeter (HCAL), each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity coverage provided by the barrel and endcap detectors. Muons are detected in gas-ionisation chambers embedded in the steel flux-return yoke outside the solenoid.

The particle-flow event algorithm [33] aims to reconstruct and identify each individual particle with an optimised combination of information from the various elements of the CMS detector. The energy of photons is directly obtained from the ECAL measurement, corrected for zero-suppression effects. The energy of electrons is determined from a combination of the electron momentum at the primary interaction vertex, the energy of the corresponding ECAL cluster, and the energy sum of all bremsstrahlung photons spatially compatible with originating from the electron track [34]. The energy of muons is obtained from the curvature of the corresponding track [35]. The energy of charged hadrons is determined from a combination of their momentum measured in the tracker and the matching ECAL and HCAL energy deposits, corrected for zero-suppression effects and for the response function of the calorimeters to hadronic showers. Finally, the energy of neutral hadrons is obtained from the corresponding corrected ECAL and HCAL energy.

The reconstructed vertex with the largest value of summed physics-object $p_{\mathrm {T}} ^2$ is taken to be the primary $\mathrm {p}\mathrm {p}$ interaction vertex. The physics objects used are the jets, clustered with the jet finding algorithm [36, 37] with the tracks assigned to the vertex as inputs, and the associated missing transverse momentum, taken as the negative vector sum of the $p_{\mathrm {T}}$ of those jets.

A more detailed description of the CMS detector, together with a definition of the coordinate system used and the relevant kinematic variables, can be found in Ref. [38].

Data and simulated samples

In this analysis, $\mathrm {p}\mathrm {p}$ collision data at a centre-of-mass energy of $13\,\text {TeV}$ taken in 2016 by the CMS experiment are analyzed. The data have been collected with muon and electron triggers [39]. For the muon trigger, a muon candidate with $p_{\mathrm {T}} > 50 \,\text {GeV}$ is required. Data events in the electron channel are collected using a logical combination of two triggers: the first requires an electron candidate with $p_{\mathrm {T}} > 45\,\text {GeV}$ and a hadronic jet candidate with $p_{\mathrm {T}} > 165\,\text {GeV}$, the second requires an electron candidate with $p_{\mathrm {T}} > 115\,\text {GeV}$. In the trigger selection, reconstructed leptons and jets must be in the central part of the detector, with a pseudorapidity of $|\eta | < 2.4$. No lepton isolation criteria are applied at the trigger level. The collected data correspond to an integrated luminosity of 35.9$\,\text {fb}^{-1}$ [40].

For the study of dominant SM background processes and for the validation of the background estimation, simulated samples using Monte Carlo (MC) techniques are used. The top quark pair production via the strong interaction and single top quark production in the t-channel, and the $\mathrm {t}\mathrm {W}$ process are generated with the next-to-leading-order (NLO) generator powheg [41,42,43] (version v2 is used for the first two and version v1 for the third). The event generator MadGraph 5_amc@nlo (v2.2.2) [44] at NLO is used for single top quark production in the s-channel. The $\mathrm {W}$+jets and $\mathrm {Z}$+jets processes are also simulated using MadGraph 5_amc@nlo (v2.2.2). The $\mathrm {W}$+jets events are generated at NLO, and the FXFX scheme [45] is used to match the parton shower emission. The $\mathrm {Z}$+jets events are produced at LO with the MLM parton matching scheme [46]. The production of quantum chromodynamics (QCD) multijet events has been simulated at LO using pythia [47]. All generated events are interfaced with pythia for the description of the parton shower and hadronisation. The parton distribution functions (PDFs) are taken from the NNPDF 3.0 [48] sets, with the precision matching that of the matrix element calculations. The underlying event tune is CUETP8M1 [49, 50], except for the simulation of top quark pairs and single top quark production in the t-channel, which use CUETP8M2T4 [51].

Signal events are generated at LO using MadGraph 5_amc@nlo for $\mathrm {B}$ and $X_{5/3}$ with VLQ decay widths relative to the VLQ mass of $(\Gamma /m)_{\mathrm {VLQ}} = 1$, 10, 20, and 30%. The samples with 1% relative VLQ width are simulated in steps of 100$\,\text {GeV}$ for masses between 700 and 2000$\,\text {GeV}$. Samples with 10, 20, and 30% relative VLQ widths are generated in steps of 200$\,\text {GeV}$ for masses ranging from 800 to 2000$\,\text {GeV}$, using a modified version of the model proposed in Refs. [52,53,54]. Separate signal samples are generated for the two main production modes, in which VLQs are produced in association either with a $\mathrm {b}$ quark or with a $\mathrm {t}$ quark, viz. $\mathrm {p}\mathrm {p}\rightarrow \mathrm {B} \mathrm {b}\mathrm {q}$ and $\mathrm {p}\mathrm {p}\rightarrow \mathrm {B} \mathrm {t}\mathrm {q}$. The theoretical cross sections for VLQ production are calculated using Refs. [55,56,57], where a simplified approach is used to provide a model-independent interpretation of experimental results for narrow and large mass width scenarios, as already used for the interpretation of singly produced vector-like $\mathrm {T}$ and $\mathrm {B}$ quarks [18, 19]. The $\textsc {MADSPIN}$ package [58, 59] is used to retain the correct spin correlations of the top quark and $\mathrm {W}$ boson decay products. Interference effects between signal and SM processes have been found to be negligible in this analysis.

All generated events are passed through a Geant4 [60] based detector simulation of the CMS detector. Additional $\mathrm {p}\mathrm {p}$ interactions originating from the same bunch crossing (in-time pileup), as well as from the following or previous bunch crossings (out-of-time pileup) are taken into account in the simulation.

Event selection

The physics objects used in this analysis are muons, electrons, hadronic jets, ${\vec {p}}_{\mathrm {T}}^{\text {miss}}$, and $S_{\mathrm {T},\text {lep}}$ (defined as the scalar sum of the lepton $p_{\mathrm {T}}$ and $p_{\mathrm {T}} ^\text {miss}$).

For each event, jets are clustered from reconstructed particles using the infrared and collinear safe anti-$k_{\mathrm {T}}$ algorithm [36] with a distance parameter $R=0.4$ (AK4 jet). Additionally, jets with $R=0.8$ (AK8 jet) are also clustered in every event with the anti-$k_{\mathrm {T}}$ algorithm, which are used for $\mathrm {t}$ and $\mathrm {W}$ tagging. The jet clustering is performed with the FastJet [37] package. Jet momentum is determined as the vectorial sum of all particle momenta in the jet, and is found from simulation to be within 5–10% of the true momentum over the whole $p_{\mathrm {T}}$ spectrum and detector acceptance. Additional $\mathrm {p}\mathrm {p}$ interactions within the same or nearby bunch crossings can contribute additional tracks and calorimetric energy depositions to the jet momentum. To mitigate this effect, tracks identified to be originating from pileup vertices are discarded, and an offset correction is applied to correct for remaining contributions. Jet energy corrections are derived from simulation studies so that the average measured response of jets becomes identical to that of particle level jets. In situ measurements of the momentum balance in dijet, photon+jet, $\mathrm {Z}$+jet, and multijet events are used to account for any residual differences in the jet energy scale in data and simulation. Additional selection criteria are applied to each jet to remove jets potentially dominated by anomalous contributions from various subdetector components or reconstruction failures [61].

From the corrected and reconstructed AK4 jet$\mathrm {s}$, those are considered that have $p_{\mathrm {T}} > 30 \,\text {GeV}$ and $|\eta | < 4$, while AK8 jet$\mathrm {s}$ must have $p_{\mathrm {T}} >170 \,\text {GeV}$ and $|\eta | < 2.4$.

Events selected in the analysis are required to have one reconstructed muon or electron with $p_{\mathrm {T}} >55 \,\text {GeV}$ and $|\eta | < 2.4$. Electrons and muons are selected using tight quality criteria with small misidentification probabilities of about 0.1% for muons and 1% for electrons [34, 62]. In the electron channel, a AK4 jet must have $p_{\mathrm {T}} >185 \,\text {GeV}$ and $|\eta | < 2.4$ if the electron has $p_{\mathrm {T}} <120 \,\text {GeV}$, reflecting the trigger selection. Events with more than one muon or electron passing the same tight identification criteria and having $p_{\mathrm {T}} > 40 \,\text {GeV}$ and $|\eta | < 2.4$ are discarded. Selected events contain two AK4 jet$\mathrm {s}$ with $p_{\mathrm {T}} > 50 \,\text {GeV}$, which are in the central part of the detector with $|\eta | < 2.4$. Additionally at least one AK8 jet is required. For the reconstruction AK4 jet$\mathrm {s}$ are used with $p_{\mathrm {T}} > 30 \,\text {GeV}$ and $|\eta | < 2.4$, while the AK4 jet$\mathrm {s}$ emitted close to the beam pipe and employed in the background estimation must fulfill $p_{\mathrm {T}} > 30 \,\text {GeV}$ and $2.4< |\eta | < 4$.

Because of the high Lorentz boosts of the top quarks and $\mathrm {W}$ bosons from the heavy VLQ decay, signal events can have leptons in close vicinity to the jets. For this reason, standard lepton isolation would reduce the selection efficiency considerably. Therefore, for the suppression of events originating from QCD mulitjet processes, either the perpendicular component of the lepton momentum relative to the geometrically closest AK4 jet $p_{\mathrm {T,rel}}$, is required to exceed $40 \,\text {GeV}$ or the angular distance of the lepton to the jet, ${\varDelta }R (\ell ,\mathrm {jet}) = \sqrt{\smash [b]{({\varDelta }\eta )^2 + ({\varDelta }\phi )^2}}$, must be larger than 0.4, where $\phi$ is the azimuthal angle in radians. Furthermore, for selecting an event, the magnitude of ${\vec {p}}_{\mathrm {T}}^{\text {miss}}$ has to be greater than 50$\,\text {GeV}$ in the muon channel and greater than 60$\,\text {GeV}$ in the electron channel. This requirement reduces the amount of background from multijet production. The final selection is based on the variable $S_{\mathrm {T},\text {lep}}$, which is required to be larger than 250$\,\text {GeV}$ in the muon channel and 290$\,\text {GeV}$ in the electron channel.

Events are separated into categories exploiting the tagging techniques for boosted top quarks and $\mathrm {W}$ bosons decaying hadronically, as well as for hadronic jets originating from $\mathrm {b}$ quarks. Jets with $R=0.8$ are used to identify the hadronic decays of highly boosted top quarks and $\mathrm {W}$ bosons [63, 64]. For top quark jets $p_{\mathrm {T}} >400 \,\text {GeV}$ is required, and for $\mathrm {W}$ boson jets the requirement is $p_{\mathrm {T}} >200 \,\text {GeV}$. The “soft drop” (SD) declustering and grooming algorithm [65, 66] with $z=0.1$ and $\beta =0$ is employed to identify subjets and to remove soft and wide-angle radiation. The groomed jet mass, $m_{\mathrm {SD}}$, is used to identify top quark and $\mathrm {W}$ boson candidates. Tagged top quark candidates ($\mathrm {t}$ tagged) are required to have $105< m_{\mathrm {SD}} < 220 \,\text {GeV}$ and one of the subjets must fulfill the loose $\mathrm {b}$ tagging criterion, based on the combined secondary vertex (CSVv2) [67] algorithm. The loose criterion is defined to give a 80% efficiency of correctly identifying $\mathrm {b}$ jets, with a 10% probability of incorrectly tagging a light quark jet. Additionally, the jet must have a N-subjettiness [68, 69] ratio $\tau _3/\tau _2 < 0.5$ and its angular distance to the lepton ${\varDelta }R{(\ell ,\mathrm {t}\ \mathrm {tag})}$ must be larger than 2. Identified $\mathrm {W}$ boson candidates ($\mathrm {W}$ tag) must have $65< m_{\mathrm {SD}} < 95 \,\text {GeV}$. The medium $\mathrm {b}$ tag criterion is used on AK4 jet$\mathrm {s}$, defined to give a $60\%$ efficiency of correctly identifying $\mathrm {b}$ jets, with a $1\%$ probability of incorrectly tagging a light quark jet.

Selected events are attributed to different mutually exclusive event categories. Events containing at least one $\mathrm {t}$ tag constitute the first category (“$\mathrm {t}$ tag”). If no $\mathrm {t}$ tag is found, all events with at least one $\mathrm {W}$ tag are grouped into a second category (“$\mathrm {W}$ tag”). The remaining events are attributed to three further categories based on the multiplicity of $\mathrm {b}$ tags found in the event. We distinguish events with at least two (“${\ge }2$ $\mathrm {b}$ tag”), exactly one (“1 $\mathrm {b}$ tag”), and no $\mathrm {b}$ tag (“0 $\mathrm {b}$ tag”). These five categories are built separately in the muon and in the electron channel leading to a total of ten categories.

Mass reconstruction

Hadronic jets, leptons, and ${\vec {p}}_{\mathrm {T}}^{\text {miss}}$ are used to reconstruct the mass of the VLQ, denoted $m_\mathrm {reco}$. In signal events, the lepton in the final state always originates from the decay of a $\mathrm {W}$ boson, either the $\mathrm {W}$ boson from the VLQ decay or the $\mathrm {W}$ boson from the top quark decay. The neutrino four-momentum can thus be reconstructed from the components of ${\vec {p}}_{\mathrm {T}}^{\text {miss}}$, the $\mathrm {W}$ mass constraint, and the assumption of massless neutrinos.

In the case when a hadronic jet with a $\mathrm {t}$ tag is found, $m_\mathrm {reco}$ is calculated from the four-momentum of the $\mathrm {t}$-tagged jet and the four-momentum of the leptonically decaying $\mathrm {W}$ boson. If several hadronic jets with $\mathrm {t}$ tags are present, the one with the largest angular distance to the reconstructed leptonic $\mathrm {W}$ boson decay is used. Once the $\mathrm {t}$-tagged jet has been selected, all overlapping AK4 jet jets in the event are removed in order to avoid double counting of energy. For the shown $m_\mathrm {reco}$ distributions these events form the $\mathrm {t}$ tag category. For events in the other categories the hadronic part of the VLQ decay is reconstructed from combinations of AK4 jet$\mathrm {s}$ with $|\eta | < 2.4$. Each possible jet assignment for the decays of the $\mathrm {W}$ boson and $\mathrm {t}$ quark is tested exploiting the following $\chi ^2$ quantity

$$\begin{aligned} \chi ^2&= \frac{\left( m_{\mathrm {t}}- \overline{m}_{\mathrm {t}} \right) ^2}{\sigma ^2_{\mathrm {t}}} + \frac{\left( m_{\mathrm {W} }- \overline{m}_{\mathrm {W} } \right) ^2}{\sigma ^2_{\mathrm {W} }} \nonumber \\&\quad + \frac{\left( {\varDelta }R(\mathrm {t},\mathrm {W} )-\pi \right) ^2}{\sigma ^2_{{\varDelta }R}}+ \frac{\left( p_{\mathrm {T},\mathrm {W} }/ p_{\mathrm {T},\mathrm {t}}- 1\right) ^2}{\sigma ^2_{p_{\mathrm {T}}}}. \end{aligned}$$

(1)

For each event, the jet assignment with the maximum $\chi ^2$ probability is selected. For the $\chi ^2$ quantity the $p_{\mathrm {T}}$ balance, $p_{\mathrm {T},\mathrm {W} }/ p_{\mathrm {T},\mathrm {t}}$, the angular distance, ${\varDelta }R(\mathrm {t},\mathrm {W} )$, and the reconstructed masses of the top quark candidate $m_{\mathrm {t}}$ and the $\mathrm {W}$ boson candidate $m_{\mathrm {W} }$ are used. The expected values $\overline{m}_{\mathrm {t}}$ and $\overline{m}_{\mathrm {W} }$, and their standard deviations $\sigma _{\mathrm {t}}$ and $\sigma _{\mathrm {W} }$ are obtained from simulation for correctly reconstructed events and it is verified that the values are independent of the VLQ mass. Here, correctly reconstructed events are defined by the assignment of jets to generated $\mathrm {t}$ quarks and $\mathrm {W}$ bosons, where the generated particles from the VLQ decay are unambiguously matched within a distance of ${\varDelta }R < 0.4$ to the reconstructed particles. It was also verified in simulation that the expected values of ${\varDelta }R(\mathrm {t},\mathrm {W} )$ and the $p_{\mathrm {T}}$ balance are $\pi$ and 1, with their standard deviations $\sigma _{{\varDelta }R}$ and $\sigma _{p_{\mathrm {T}}}$. In order to account for cases where the $\mathrm {W}$ boson from the VLQ decay decays into a lepton and neutrino, the $\chi ^2$ is calculated for each permutation with the second term omitted. Cases where the hadronic decay products of the $\mathrm {W}$ bosons or the top quark are reconstructed in a single AK4 jet are included by omitting the first or second term in the calculation of the $\chi ^2$.

Fig. 2

Distributions of $m_\mathrm {reco}$ for the $\mathrm {B}$ +$\mathrm {b}$ production mode, obtained for simulated events with a muon in the final state, reconstructed with a $\mathrm {t}$ tag (top) and with the $\chi ^2$ method (bottom) for right-handed VLQ couplings and various VLQ masses $m_{\mathrm {B}}$. Signal events are shown assuming a production cross section of 1$\,\text {pb}$ and a relative VLQ decay width of 1%

The distributions of $m_\mathrm {reco}$ in simulation for the $\mathrm {B}$ +$\mathrm {b}$ production mode with right-handed couplings are shown in Fig. 2 for events with a muon in the final state. The reconstruction of events with a $\mathrm {t}$ tag (top) is best suited for high VLQ masses where the decay products of the top quark are highly boosted, while the $\chi ^2$ method (bottom) yields a stable performance for all VLQ masses, where the decay products of the $\mathrm {W}$ boson and top quark are reconstructed from several jets. Additionally, the latter method enables the reconstruction of events with a lepton from the top quark decay chain. Mass resolutions between 10–15% are achieved for both reconstruction methods, with peak values of the $m_\mathrm {reco}$ distributions at the expected values. The VLQs with left-handed couplings (not shown) have a lower selection efficiency by 20–25% because of a smaller lepton $p_{\mathrm {T}}$, on average, but otherwise features a behaviour similar to VLQs with right-handed couplings. Distributions obtained for the final states with an electron are similar to those with a muon.

Background estimation

The data sample obtained after the selection is then divided into a signal region with a jet in the forward region of the detector with $2.4< |\eta | < 4.0$ and a control region without such a jet. The distribution of background processes in the signal region is estimated using the shape of the $m_\mathrm {reco}$ distribution in the control region. Residual differences in the shapes of the $m_\mathrm {reco}$ distributions between signal and control regions are investigated in each of the signal categories by using simulated SM events. Differences can arise from different background compositions in signal and control regions due to the presence of a forward jet. The observed differences are small, with average values of 10%, and are corrected for by multiplicative factors applied to the background predictions in the validation and signal regions. The largest differences are observed for $m_\mathrm {reco}$ values below 800$\,\text {GeV}$, with values no larger than about 20%.

In order to validate the VLQ mass reconstruction, data are compared to simulation in the control region. In Fig. 3 the distributions of $m_\mathrm {reco}$ are shown in the muon (upper) and electron (lower) channels for events with a $\mathrm {t}$ tag (left) and events reconstructed with the $\chi ^2$ method (right). The ${\mathrm {t}\overline{\mathrm {t}}}$ and $\mathrm {t}\mathrm {W}$ standard model processes provide irreducible backgrounds in the reconstructed VLQ mass distributions, showing good agreement between the data and simulation. The contribution of signal events in the control region is small and is taken into account by a simultaneous fit to signal and control regions in the statistical extraction of the results.

Fig. 3

Distributions of $m_\mathrm {reco}$ in data and simulation in the control region for the muon (upper) and electron (lower) channels for events reconstructed with a $\mathrm {t}$ tag (left) and with the $\chi ^2$ method (right). The VLQ signal is shown for the $\mathrm {B}$ +$\mathrm {b}$ production mode and right-handed VLQ couplings. The vertical bars illustrate the statistical uncertainties on the data, while the shaded area shows the total uncertainties for the background simulation. The lower panels show the ratio of data to the background prediction. The dark and light gray bands correspond to the statistical and total uncertainties, respectively

In order to validate the background estimation, a validation region is constructed from requiring events with reconstruction p-values smaller than 0.08. The p-values are calculated as the probability of obtaining the $\chi ^2$ as given by Eq. (1), where the number of degrees of freedom of the selected hypothesis are taken into account. For events with a $\mathrm {t}$ tag, the same $\chi ^2$ quantity is evaluated for the selected hypothesis. The validation region has an order of magnitude fewer events than the signal region and a negligible amount of signal contamination. The $m_\mathrm {reco}$ distributions for the two most sensitive categories are shown in Fig. 4 for the muon (upper) and electron (lower) channels. The observed number of events is found to be in good agreement with the predicted number of events from the background estimation in the validation region, with no statistically significant deviations. Similar observations are made for the other signal categories.

Fig. 4

Distributions of $m_\mathrm {reco}$ in the validation region of the two most sensitive categories in the muon channel (upper) and electron channel (lower). The lower panels show the difference of data and background expectations in units of the total (stat. and sys.) uncertainty on the background estimate

Systematic uncertainties

Systematic uncertainties can affect both the overall normalisation of background components and the shapes of the $m_\mathrm {reco}$ distributions for signal and background processes. The main uncertainty in the shape of the $m_\mathrm {reco}$ distribution from the background estimation based on a control region in data is related to the kinematic difference between the signal and control regions. Correction factors are applied to account for this difference, obtained from SM simulations. These uncertainties have a size of 10% on average, with maximum values of 20% at small values of $m_\mathrm {reco}$. Compared to these uncertainties, the effects from uncertainties in the SM simulations are negligible on the background estimation, as these cancel to a large degree when building the ratios between signal and control regions. The uncertainties in the overall normalisation of the background predictions are obtained from a fit to the data in the signal region.

Uncertainties in the MC simulation are applied to all simulated signal events. In the following, the systematic uncertainties are summarized.

• The uncertainty in the integrated luminosity measurement recorded with the CMS detector in the 2016 run at $\sqrt{s}=13\,\text {TeV}$ is 2.5% [40].

• The estimation of pileup effects is based on the total inelastic cross section. This cross section is determined to be 69.2$\,\text {mb}$. The uncertainty is taken into account by varying the total inelastic cross section by 4.6% [70].

• Simulated events are corrected for lepton identification, trigger, and isolation efficiencies. The corresponding corrections are applied as functions of $|\eta |$ and $p_{\mathrm {T}}$. The systematic uncertainties due to these corrections are taken into account by varying each correction factor within its uncertainty.

• The scale factors for the jet energy scale and resolution are determined as functions of $|\eta |$ and $p_{\mathrm {T}}$ [61]. The effect of the uncertainties in these scale factors are considered by varying the scale factors within their uncertainties. Jets with distance parameters of 0.4 and 0.8 are modified simultaneously. The results of variations for AK4 jet$\mathrm {s}$ are propagated to the measurement of ${\vec {p}}_{\mathrm {T}}^{\text {miss}}$.

• The uncertainties due to the PDFs are evaluated by considering 100 replicas of the NNPDF 3.0 set according to the procedure described in Ref. [71]. The associated PDF uncertainties in the signal acceptance are estimated following the prescription for the LHC [71].

• Uncertainties associated with variations of the factorisation $\mu _\mathrm {f}$ and renormalisation scales $\mu _\mathrm {r}$ are evaluated by varying the respective scales independently, by factors of 0.5 and 2.

• Corrections for the $\mathrm {b}$ tagging efficiencies and misidentification rates for AK4 jet$\mathrm {s}$, and subjets of AK8 jet$\mathrm {s}$ are applied. These are measured as a function of the jet $p_{\mathrm {T}}$ [67]. The corresponding uncertainties are taken into account by varying the corrections within their uncertainties for heavy- and light-flavour jets separately.

• An uncertainty on the $\mathrm {t}$ tagging efficiency of $+\,7$ and $-\,4\%$ is applied to signal events with a $\mathrm {t}$ tag [64]. The uncertainty on the $\mathrm {W}$ tagging efficiency is determined from jet mass resolution (JMR) and scale (JMS) uncertainties, which are added in quadrature. An additional JMR uncertainty is derived from the differences in the hadronisation and shower models of pythia and herwig++ [72]. The uncertainty depends on the $p_{\mathrm {T}}$ of the $\mathrm {W}$ boson; for VLQs with a mass of 700$\,\text {GeV}$ it is around 2% and for a mass of 1800$\,\text {GeV}$ it is around 6%. An uncertainty of 1% is assigned to the JMS, as obtained from studies of the jet mass in fully merged hadronic $\mathrm {W}$ boson decays.

In Table 1, a summary of the uncertainties considered for signal events is shown, where the largest uncertainties come from the jet energy scale and the jet tagging. For the uncertainties connected to the PDF, $\mu _\mathrm {f}$ and $\mu _\mathrm {r}$ only the signal acceptance and shape differences are propagated. The uncertainties with the largest impact on the analysis are the uncertainties associated with the data-driven background estimation, being more than two times larger than the jet energy scale uncertainties in the signal.

Table 1 Uncertainties considered for simulated signal events in the $\mathrm {B}$ +$\mathrm {b}$ production mode ($m_{\mathrm {B} {}} = 900 \,\text {GeV}$) for right-handed VLQ couplings for the $\mathrm {t}$ tag and $\mathrm {W}$ tag categories. The uncertainties in the $\mathrm {b}$ tag categories are of comparable size to those in the $\mathrm {W}$ tag category

Results

The $m_\mathrm {reco}$ distributions in the ten categories are measured in the signal and control region, which are defined by the presence or absence of a forward jet with $|\eta | > 2.4$. For the background estimate in the signal regions, a simultaneous binned maximum likelihood fit of both regions is performed using the Theta [73] package. In these fits, the signal cross section and the background normalisations in the different signal categories are free parameters. The shapes of the $m_\mathrm {reco}$ distributions for the SM background in the signal regions are taken from the corresponding control regions. Systematic uncertainties are taken into account as additional nuisance parameters. A common nuisance parameter is used for uncertainties in the muon and electron channels if a similar effect is expected on the shape or normalisation of the $m_\mathrm {reco}$ distribution in both channels similarly. The nuisance parameters for the shape uncertainties are taken to be Gaussian distributed. For the uncertainties on the normalisation log-normal prior distributions are assumed.

The measured distributions of $m_\mathrm {reco}$ for the signal categories are shown in Figs. 5 and 6 for the muon and electron channels, together with the background predictions obtained from the control regions. The signal $m_\mathrm {reco}$ distributions for a vector-like $\mathrm {B}$ quark with right-handed couplings produced in association with a $\mathrm {b}$ quark are shown for illustration, for two different VLQ masses with an assumed production cross section of 1$\,\text {pb}$ and a relative VLQ width of 1%. No significant deviation from the background expectation is observed in any of the categories.

Fig. 5

Distributions of $m_\mathrm {reco}$ measured in the signal region for events with a jet in the forward direction with $|\eta |>2.4$ in the muon channel. Shown are the sensitive categories: $\mathrm {t}$ tag (upper left), $\mathrm {W}$ tag (upper right), ${\ge }2$ $\mathrm {b}$ tag (middle left), 1 $\mathrm {b}$ tag (middle right) and 0 $\mathrm {b}$ tag (lower). The background prediction is obtained from control regions as detailed in the main text. The distributions from two example signal samples for the $\mathrm {B}$ +$\mathrm {b}$ production mode with right-handed VLQ couplings with a cross section of 1$\,\text {pb}$ and a relative width of 1% are shown for illustration

Fig. 6

Distributions of $m_\mathrm {reco}$ measured in the signal region for events with a jet in the forward direction with $|\eta |>2.4$ in the electron channel. Shown are the sensitive categories: $\mathrm {t}$ tag(upper left), $\mathrm {W}$ tag(upper right), ${\ge }2$ $\mathrm {b}$ tag (middle left), 1 $\mathrm {b}$ tag (middle right) and 0 $\mathrm {b}$ tag (lower). The background prediction is obtained from control regions as detailed in the main text. The distributions from two example signal samples for the $\mathrm {B}$ +$\mathrm {b}$ production mode with right-handed VLQ couplings with a cross section of 1$\,\text {pb}$ and a relative VLQ width of 1% are shown for illustration

Exclusion limits on the product of the VLQ production cross section and branching fraction are calculated at 95% confidence level ($\text {CL}$) for VLQ masses between 700 and 2000$\,\text {GeV}$ by using a Bayesian statistical method [73, 74]. Pseudo-experiments are performed to extract expected upper limits under the background-only hypothesis. For the signal cross section parameter an uniform prior distribution, and for the nuisance parameters log-normal prior distributions are used. The nuisance parameters are randomly varied within their ranges of validity to estimate the 68 and 95% $\text {CL}$ expected limits. Correlations between the systematic uncertainties across all channels are taken into account through a common nuisance parameter. The statistical uncertainties of the background predictions are treated as an additional Poisson nuisance parameter in each bin of the $m_\mathrm {reco}$ distribution.

Figure 7 shows the 95% $\text {CL}$ upper limits on the product of the cross section and branching fraction for the $\mathrm {B}$ +$\mathrm {b}$ production mode for left- and right-handed VLQ couplings and a relative VLQ width of 1% (upper left and upper right), for the left-handed VLQ couplings and a relative VLQ width of 10% (lower left), as well as a comparison of the observed exclusion limits for relative VLQ widths between 10 and 30% (lower right). In Fig. 8, the 95% $\text {CL}$ upper limits on the product of the cross section and branching fraction for the production modes $\mathrm {B}$ +$\mathrm {t}$ (upper left) and $X_{5/3}$+$\mathrm {t}$ (upper right) and right-handed VLQ couplings are shown. The figure also shows the $X_{5/3}$+$\mathrm {t}$ exclusion limits for left-handed VLQ couplings with a 10% relative VLQ width (lower left) and a comparison of the observed exclusion limits for VLQ widths between 10 and 30% for left-handed couplings (lower right). The predicted cross sections for variations of the relative VLQ mass width (dashed lines) are taken from Refs. [55,56,57]. For a set of VLQ masses the expected and observed 95% $\text {CL}$ upper limits for the $\mathrm {B}$ +$\mathrm {b}$ and the $X_{5/3}$+$\mathrm {t}$ production modes are also given in Table 2 for VLQs with widths of 1% and 10% and left-handed couplings, as well as for widths of 1% and right-handed couplings. The exclusion limits for the $\mathrm {B}$ +$\mathrm {t}$ production mode are similar to those for the $X_{5/3}$+$\mathrm {t}$ production mode.

The obtained exclusion limits range from 0.3 to 0.03$\,\text {pb}$ for VLQ masses between 700 and 2000$\,\text {GeV}$. For VLQs with a relative width of 1% and purely left-handed couplings an increase of about 25% of the 95% $\text {CL}$ upper limits is observed because of the reduced signal acceptance, in comparison to the right-handed couplings. The expected limits for VLQ with relative widths of 10–30% and left-handed couplings only show small differences. Although the predicted cross sections for the SM backgrounds are considerably larger at 13$\,\text {TeV}$, similar exclusion limits on the product of cross section and branching fraction are achieved compared to the results obtained at 8$\,\text {TeV}$ in the more restricted mass range considered in Ref. [32]. However, because of the increase of the VLQ signal cross section at 13$\,\text {TeV}$, with this analysis, the existence of VLQ $\mathrm {B}$ ($X_{5/3}$) quarks with left-handed couplings and a relative width of 10, 20, and 30% can be excluded for masses below 1490, 1590, and 1660$\,\text {GeV}$ (920, 1300, and 1450$\,\text {GeV}$) respectively. The results represent the most stringent exclusion limits for singly produced VLQ in this channel.

Fig. 7

Upper limits at 95% $\text {CL}$ on the product of the VLQ production cross section and branching fraction for the $\mathrm {B}$ +$\mathrm {b}$ production mode for a relative VLQ width of 1% and left- and right-handed VLQ couplings (upper left and right), for 10% relative VLQ width and left-handed VLQ couplings (lower left), and a comparison of the observed exclusion limits for relative VLQ widths of 10, 20, and 30% for left-handed couplings (lower right). The dashed lines show the theoretical predictions

Fig. 8

Upper limits at 95% $\text {CL}$ on the product of the VLQ production cross section and branching fraction for the $\mathrm {B}$ +$\mathrm {t}$ and $X_{5/3}$+$\mathrm {t}$ production modes for right-handed VLQ couplings assuming a relative VLQ width of 1% (upper left and right), for the $X_{5/3}$+$\mathrm {t}$ production mode with left-handed VLQ couplings and a 10% relative width (lower left) and a comparison of the observed exclusion limits for left-handed couplings for relative widths of 10, 20, and 30% (lower right). The dashed lines show the theoretical predictions

Table 2 Observed (expected) upper limits at 95% $\text {CL}$ on the product of the cross section and branching fraction for the $\mathrm {B}$ +$\mathrm {b}$ and $X_{5/3}$+$\mathrm {t}$ production modes, for a set of VLQ masses, for VLQs widths of 1% and 10%, and for left-handed and right-handed couplings. The exclusion limits for the $\mathrm {B}$ +$\mathrm {t}$ production mode (not shown) are very similar to those for the $X_{5/3}$+$\mathrm {t}$ mode

Summary

A search for singly produced vector-like quarks decaying into a top quark and a $\mathrm {W}$ boson has been performed using the 2016 data set recorded by the CMS experiment at the CERN LHC. The selection is optimised for high vector-like quark masses, with a single muon or electron, significant missing transverse momentum, and two jets with high $p_{\mathrm {T}}$ in the final state. Vector-like quarks in the single production mode can be produced in association with a $\mathrm {t}$ or a $\mathrm {b}$ quark and a forward jet. The latter feature is used to obtain the background prediction in the signal regions from data. The mass of the vector-like quark is reconstructed from the hadronic jets, the missing transverse momentum, and the lepton in the event. Different decay possibilities of the $\mathrm {t}$ and $\mathrm {W}$ are considered. The reach of the search is enhanced by $\mathrm {t}$, $\mathrm {W}$, and $\mathrm {b}$ tagging methods. No significant deviation from the standard model prediction is observed. Upper exclusion limits at 95% confidence level on the product of the production cross section and branching fraction range from around 0.3–0.03$\,\text {pb}$ for vector-like quark masses between 700 and 2000$\,\text {GeV}$. Depending on the vector-like quark type, coupling, and decay width to $\mathrm {t}$ $\mathrm {W}$, mass exclusion limits up to 1660$\,\text {GeV}$ are obtained. These represent the most stringent exclusion limits for the single production of vector-like quarks in this channel.