Search for pair production of third-generation leptoquarks decaying into a bottom quark and a $\tau$-lepton with the ATLAS detector

Abstract

A search for pair-produced scalar or vector leptoquarks decaying into a b-quark and a $\tau$-lepton is presented using the full LHC Run 2 (2015–2018) data sample of 139 fb$^{-1}$ collected with the ATLAS detector in proton–proton collisions at a centre-of-mass energy of $\sqrt{s} =13$ TeV. Events in which at least one $\tau$-lepton decays hadronically are considered, and multivariate discriminants are used to extract the signals. No significant deviations from the Standard Model expectation are observed and 95% confidence-level upper limits on the production cross-section are derived as a function of leptoquark mass and branching ratio $\mathcal {B}$ into a $\tau$-lepton and b-quark. For scalar leptoquarks, masses below 1460 GeV are excluded assuming $\mathcal {B}=100$%, while for vector leptoquarks the corresponding limit is 1650 GeV (1910 GeV) in the minimal-coupling (Yang–Mills) scenario.

1 Introduction

Many extensions of the Standard Model (SM) of particle physics predict particles known as leptoquarks (LQs) [1,2,3,4,5,6,7]. These particles provide a connection between the lepton and quark sectors, which are similar in structure in the SM. LQs can be scalar (spin-0) or vector (spin-1) bosons, and they carry colour and a fractional electric charge. They also have non-zero lepton and baryon numbers, and decay into quark–lepton pairs. They can mediate neutral and charge currents, and therefore can potentially provide an explanation for hints of violations of lepton universality observed in flavour experiments [8,9,10,11,12,13,14].

This analysis searches for the pair-production of LQs that couple strongly to the third generation of quarks and leptons in proton–proton (pp) collisions at the LHC. Within the Buchmüller–Rückl–Wyler (BRW) model [15], which is the benchmark for scalar LQs in this analysis, it is assumed that these LQs can only interact within the same family via a Yukawa interaction. This interaction is described by two parameters, a model parameter $\beta$ and a coupling parameter $\lambda$. In the BRW model, the pair-production cross-section is independent of $\lambda$. This analysis also considers pair-production of vector LQs [16, 17] corresponding to the $U_1$ state in the BRW classification [15]. The scenarios considered in this model differ by a dimensionless coupling constant k, which is zero for the minimal-coupling scenario and one for the Yang–Mills scenario. For both scalar and vector LQs, the parameter $\beta$ controls the coupling to charged leptons. For third generation LQs, results are generally given in terms of the mass of the LQ ($m_{\text {LQ}}$) and its branching ratio ($\mathcal {B}$), rather than $\beta$ as is the case for first- and second-generation LQs. This is because $\beta$ is not equal to $\mathcal {B}$ for third-generation LQs due to the sizable top-quark mass.

ATLAS and CMS have published searches for LQs coupling to the first, second and third generations [18,19,20,21,22,23,24,25,26,27]. Each generation of LQs is split into up-type and down-type LQs with different electric charges. For instance, for the third generation they are split into up-type LQs ($\text {LQ}^{\text {u}}_{3}$), which decay into $b\tau$ or $t\nu$, and down-type LQs ($\text {LQ}^{\text {d}}_{3}$), which decay into $t\tau$ or $b\nu$. Both types of LQs are currently excluded for masses below 1150 $\text {GeV}$ for the BRW model, for all values of $\mathcal {B}$.

This paper updates the ATLAS search for an up-type LQ pair decaying into $b\tau$ [19], shown in Fig. 1, using the full Run 2 data sample and an updated analysis strategy, prioritising high LQ masses that are not yet excluded in the benchmark models considered. Analysis improvements include updated analysis-optimisation and background-estimation methods, as well as updates to several object identification algorithms. The analysis signature is two jets, at least one of which must be identified as containing a b-hadron, and two $\tau$-leptons. For the $\tau$-leptons, the cases considered are where both decay hadronically or where one $\tau$-lepton decays into a light lepton (electron or muon, $\ell$) and neutrinos and the other decays hadronically. The mass range considered for the LQ is from 300 to 2000 $\text {GeV}$. The extraction of the signals is performed through a simultaneous likelihood fit to multivariate discriminants. For the results, both scalar and vector LQs are considered, with the limits on vector LQs interpreted in the context of two scenarios, the Yang–Mills scenario and the minimal-coupling scenario [28].

Fig. 1

Pair production of a leptoquark (LQ) and its subsequent decay into a b-quark and a $\tau$-lepton

The paper is structured as follows. After a brief description of the ATLAS detector, the data sample, simulated backgrounds and simulated signals are described. This is followed by a description of the event reconstruction, the object selection, the event selections for the signal regions, and the multivariate discriminants that are used in the final fit. The next sections include a description of the data-driven background estimation methods, the systematic uncertainties, and finally the statistical methods and results.

2 ATLAS detector

The ATLAS detector [29] at the LHC is a multipurpose particle detector with a forward–backward symmetric cylindrical geometry and a near $4\pi$ coverage in solid angle.¹ The inner tracking detector consists of pixel and microstrip silicon detectors covering the pseudorapidity region $|\eta | < 2.5$, surrounded by a transition radiation tracker to enhance electron identification in the range of $|\eta | < 2.0$. An additional innermost pixel layer, the insertable B-layer [30, 31], was added before Run 2 of the LHC. The inner detector (ID) is surrounded by a thin superconducting solenoid providing a 2 T axial magnetic field, and by a fine-granularity lead/liquid-argon (LAr) electromagnetic (EM) calorimeter covering $|\eta | < 3.2$. Hadronic calorimetry is provided by a steel/scintillator-tile calorimeter in the central pseudorapidity range ($|\eta | < 1.7$). The endcap and forward regions are instrumented with LAr calorimeters for both the EM and hadronic energy measurements up to $|\eta | = 4.9$. The muon spectrometer (MS) surrounds the calorimeters and is based on three large superconducting air-core toroidal magnets with eight coils each. Three layers of high-precision tracking chambers provide coverage in the range of $|\eta | < 2.7$, while dedicated fast chambers allow triggering in the region $|\eta | < 2.4$. A two-level trigger system [32], consisting of a hardware-based first-level trigger followed by a software-based high-level trigger (HLT), is used to select events. An extensive software suite [33] is used in data simulation, in the reconstruction and analysis of real and simulated data, in detector operations, and in the trigger and data acquisition systems of the experiment.

3 Data and simulation samples

The data used in this search correspond to an integrated luminosity of $139\,\hbox {fb}^{-1}$ of pp collision data collected by the ATLAS detector between 2015 and 2018 at a centre-of-mass energy $\sqrt{s}=13$ $\text {TeV}$. The uncertainty in the combined 2015–2018 integrated luminosity is 1.7% [34], obtained using the LUCID-2 detector [35] for the primary luminosity measurements. The presence of additional interactions in the same or neighbouring bunch crossing, referred to as pile-up, is characterised by the average number of such interactions, $\langle \mu \rangle$, which was 33.7 for the combined data sample. Only events recorded under stable beam conditions and for which all relevant detector subsystems were known to be in a good operating condition are used.

Dedicated Monte Carlo (MC) simulated samples are used to model SM processes and estimate the expected signal yields. All samples were passed through the full ATLAS detector simulation [36] based on Geant4  [37], except for the signal samples that use a parameterised fast simulation of the calorimeter response [38] and Geant4 for the other detector systems. The simulated events were reconstructed with the same algorithms as used for data, and contain a realistic modelling of pile-up interactions. The pile-up profiles match those of each data sample between 2015 and 2018, and are obtained by overlaying minimum-bias events simulated using the soft QCD processes of Pythia 8.186 [39] with the NNPDF2.3 leading-order (LO) [40] set of parton distribution functions (PDFs) and the A3 [41] set of tuned parameters (tune). The MC samples are corrected to account for the differences between simulation and data in terms of the pile-up, the energy and momentum scales, and the reconstruction and identification efficiencies of physics objects.

Simulated events with pair-produced up-type ($Q = +\frac{2}{3}$) scalar LQs were generated at next-to-leading order (NLO) in QCD with MadGraph5_aMC@NLO v2.6.0 [42], using the LQ model of Ref. [43], in which fixed-order NLO QCD calculations  [44, 45] are interfaced to Pythia 8.230 [46] for the parton shower (PS) and hadronisation. Parton luminosities were provided by the five-flavour scheme NNPDF3.0 NLO [47] PDF set with $\alpha _s = 0.118$ and the underlying event (UE) was modelled with the A14 tune [48, 49]. The coupling parameter $\lambda$ was set to 0.3, resulting in a relative LQ width of approximately 0.2% and ensuring the LQs decay promptly. In all cases, $\beta = 0.5$ such that the couplings to charged leptons and neutrinos were equal and the decay products were interfaced to MadSpin [50] to preserve spin correlations. Different values for $\mathcal {B}$ were then obtained by reweighting the simulated events according to the generator information about their decay following the procedure in Ref. [19]. Signal cross-sections were obtained from the calculation of the pair production of scalar coloured particles, such as the hypothesised supersymmetric partner of the top quark, as these particles have the same production modes and their pair-production cross-section depends only on their mass. These processes were computed at approximate next-to-next-to-leading order (NNLO) in QCD with resummation of next-to-next-to-leading-logarithmic (NNLL) soft gluon terms  [51,52,53,54]. The cross-sections do not include contributions from t-channel lepton exchange, which are neglected in Ref. [43] and may lead to corrections at the percent level [55]. The nominal cross-section and its uncertainty were derived using the PDF4LHC15_mc PDF set, following the recommendations of Ref. [56]. For LQ masses between 300 $\text {GeV}$ and 2000 $\text {GeV}$, the cross-sections range from 10 pb to 0.01 fb.

Simulated events with pair-produced up-type vector LQs were generated at LO in QCD with MadGraph5_aMC@NLO v2.6.0, using the LQ model of Ref. [17] and the NNPDF3.0 NLO PDF set with $\alpha _s = 0.118$. Decays of the LQs were performed with MadSpin, while PS and hadronisation were simulated using Pythia8.244 with the A14 tune. The full model includes two additional vector states that are necessary to obtain a realistic extension of the SM, a colour singlet $Z^{\prime }$ and a colour octet $G^{\prime }$. However, these are not present in the MadGraph model and hence do not contribute to the Feynman diagrams considered for pair production of vector leptoquarks. The samples were produced with a coupling strength $g_U = 3.0$, where $g_U$ represents the overall coupling between the LQ and the fermion, motivated by a suppression of the production cross-section for the additional mediators in the ultraviolet completion of the model, which might otherwise be in tension with existing LHC limits. This choice of coupling results in a relative LQ width of around 10%. In all cases, $\beta = 0.5$ and the same reweighting as in the scalar LQ case is then used to probe different $\mathcal {B}$ values. As mentioned, the model introduces two different coupling scenarios, the minimal-coupling scenario and the Yang–Mills scenario. In the latter case the LQ is a massive gauge boson and has additional couplings to the SM gauge bosons, resulting in enhanced cross-sections. $\mathcal {B}$ is assumed to be unaffected by these couplings since the corresponding decays are either forbidden or heavily suppressed. Since no higher-order cross-sections are available for this model, the LO MadGraph5_aMC@NLOcross-sections were used and vary between 94 pb (340 pb) and 0.05 fb (0.61 fb) for LQ masses between 300 $\text {GeV}$ and 2000 $\text {GeV}$ in the minimal-coupling (Yang–Mills) case. Above 500 $\text {GeV}$, kinematic differences between the two scenarios are negligible.

Scalar (vector) LQ samples were produced with LQ masses between 300 $\text {GeV}$ to 2000 $\text {GeV}$, with a mass interval of 50 $\text {GeV}$ in the range of 800–1600 $\text {GeV}$ (1400–1600 $\text {GeV}$) and 100 $\text {GeV}$ otherwise.

Table 1 The list of generators used for the simulation of the SM background processes. Information is given on the matrix element (ME) generator (including the perturbative QCD order), the PDF set, the parton shower (PS) and the underlying event (UE). The perturbative order (in QCD unless otherwise specified) of the cross-section used to normalise the different samples is also presented. $(\S )$ The $t\bar{t}-Wt$ interference was handled using the diagram removal scheme. $({\dagger })$ The cross-sections from Sherpa at NLO were used to normalise the WW, WZ, ZZ and $t\bar{t}W/Z$ events. $({\ddagger })$ The $qq\rightarrow ZH$ process was normalised to the NNLO (QCD) + NLO( EW) cross-section for the $pp \rightarrow ZH$ process [57,58,59,60,61,62], after subtracting the $gg\rightarrow ZH$ contribution

Background samples were simulated using different MC event generators depending on the process. All background processes are normalised to the most accurate available theoretical calculation of their respective cross-sections. The most relevant event generators, the accuracy of theoretical cross-sections, the UE parameter tunes, and the PDF sets used in simulating the SM background processes are summarised in Table 1. For all samples, except those generated using Sherpa, the EvtGen v1.2.0 [87] program was used to simulate the properties of the b- and c-hadron decays.

4 Event reconstruction and object definitions

The LQ signature of interest in this search gives rise to a set of reconstructed objects that consist primarily of $\tau$-leptons, which may decay into light leptons or hadronically, and jets from the hadronisation of quarks, specifically b-quarks. In addition, neutrinos produced in the decay of $\tau$-leptons and the semileptonic decay of b-hadrons contribute to the missing transverse momentum ${\pmb p_{\text {T}}^{\text {miss}}}$ of the event. To be considered for analysis, events are required to have at least one pp interaction vertex, reconstructed from two or more charged-particle tracks with transverse momentum $p_{\text {T}} > 500$ $\text {MeV}$; the one with the highest summed $p_{\text {T}} ^2$ of associated tracks is selected as the primary vertex.

Electron candidates are reconstructed by matching ID tracks to energy clusters in the EM calorimeter. They must satisfy $p_{\text {T}} > 7$ $\text {GeV}$ and lie in the range of $|\eta | < 2.47$, excluding the transition region between the barrel and endcap detectors ($1.37< | \eta | < 1.52$). Electrons are further identified using a likelihood-based method, based on the track quality, the profile of the shower measured in the EM calorimeter and the consistency between the track and the energy cluster [88]. Two identification criteria are used to select electrons in this analysis: ‘veto electrons’² are required to satisfy the ‘loose’ identification working point, while ‘signal electrons’ are required to satisfy the more stringent ‘tight’ working point.

Muon candidates are reconstructed from tracks in the MS, matched with compatible tracks in the ID where coverage allows; in regions where the MS is only partially instrumented ($|\eta | < 0.1$) an energy deposit in the calorimeter compatible with a minimum-ionising particle is combined with a compatible ID track instead. They must satisfy $p_{\text {T}} > 7$ $\text {GeV}$ and lie in the range of $|\eta | < 2.7$. Muons are further identified based on the number of hits in the various ID subdetectors and MS stations, the compatibility between the measurements in the two detectors and the properties of the resulting track fit. Two identification criteria [89] are used to select muons: ‘veto muons’ (See footnote 2) must satisfy a ‘loose’ identification requirement, while the ‘signal muons’ are required to satisfy the ‘medium’ (‘high-$p_{\text {T}}$ ’) working point if the $p_{\text {T}}$ is less than (greater than) 800 $\text {GeV}$. The more stringent high-$p_{\text {T}}$ requirements remove around $20\%$ of muons but improve the $p_{\text {T}}$ resolution by $\approx 30\%$ above 1.5 $\text {TeV}$, significantly suppressing potential backgrounds [90].

To suppress misidentified light leptons or those arising from hadron decays, all light-lepton candidates must satisfy an isolation criterion that limits the presence of tracks (calorimeter deposits) in a $p_{\text {T}}$-dependent (fixed) radius cone. The resulting efficiency is above 99% for both electrons and muons in the signal regions. Finally, signal leptons must satisfy stricter requirements on their $p_{\text {T}}$ depending on the data-taking period, as detailed in Sect. 5.

Jets are reconstructed from topological energy clusters and charged-particle tracks, resulting from a particle-flow algorithm [91], using the anti-$k_t$ algorithm with a radius parameter of $R=0.4$ [92, 93]. They are required to satisfy $p_{\text {T}} > 20$ $\text {GeV}$ and lie in the range of $|\eta | < 2.5$. To suppress jets from pile-up, jets with $p_{\text {T}} < 60$ $\text {GeV}$ and $|\eta | < 2.4$ are required to originate from the primary vertex using a multivariate ‘jet vertex tagger’ [94]. A multivariate algorithm based on a deep neural network, known as the ‘DL1r tagger’ [95,96,97], is used to identify jets containing b-hadrons (b-jets) based on the jet kinematics, the impact parameters of tracks associated with the jet and the reconstruction of displaced vertices. This analysis uses a working point with a 77% efficiency for true b-jets, as measured in simulated $t\bar{t}$ events, and corresponding rejection factors³ for light-flavour jets, charm jets and $\tau$-leptons of 170, 5 and 21, respectively [98, 99].

Hadronically decaying $\tau$-lepton candidates are seeded by jets, which are required to have one or three associated tracks (referred to hereafter as ‘one-prong’ or ‘three-prong’ candidates, respectively) with a total charge of $\pm 1$ [100]. The visible decay products ($\tau _{\text {had-vis}}$) must satisfy $p_{\text {T}} > 20$ $\text {GeV}$ and lie in the range of $|\eta | < 2.47$, excluding the transition region defined above. True $\tau _{\text {had-vis}}$ candidates are discriminated from quark- and gluon-initiated jets via a recurrent neural network (RNN) using calorimeter- and tracking-based variables as input and trained separately on one- and three-prong candidates [101]. The ‘loose’ working point used has an efficiency of approximately $85\%$ and $75\%$ for one- and three-prong $\tau _{\text {had-vis}}$ respectively. A further boosted decision tree (BDT) is used to reject one-prong $\tau _{\text {had-vis}}$ candidates originating from electrons with an efficiency of about $95\%$ [102]. For the estimation of the background from jets misidentified as $\tau _{\text {had-vis}}$ (described in Sect. 6), anti-$\tau _{\text {had-vis}}$ candidates are defined in the same way as above but are required to fail to satisfy the nominal loose RNN working point requirements and instead satisfy a looser requirement that has an efficiency of $99\%$ for selecting true $\tau _{\text {had-vis}}$ candidates.

Table 2 Summary of the event selections for the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ categories. Where two objects are required, the thresholds on the sub-leading object are given in parenthesis. Where the selection depends on data-taking period, the different possible threshold values are separated by commas

Fig. 2

The expected acceptance times efficiency (including object identification and reconstruction, triggering, and event selection) for the scalar and vector LQs, with both the minimal-coupling and the Yang–Mills scenarios, at $\beta = 0.5$ as a function of $m_{\text {LQ}}$ in the a $\tau _{\text {lep}}\tau _{\text {had}}$ and b $\tau _{\text {had}}\tau _{\text {had}}$ channels. The values include the leptonic and hadronic branching ratios of the tau lepton. The error bars, which are in general smaller than the markers, indicate the statistical uncertainty

The ${\pmb p_{\text {T}}^{\text {miss}}}$ (with magnitude $E_{\text {T}}^{\text {miss}}$) is computed from the negative vectorial sum of the selected and calibrated objects described above, along with an extra track-based ‘soft term’ to account for the energy of particles originating from the primary vertex but not associated to any of the reconstructed objects [103, 104].

To resolve ambiguities whereby the same detector signature may be reconstructed as more than one physics object, a sequential overlap-removal procedure is applied. First, electron candidates are discarded if they share a track with a more energetic electron or a muon identified in the MS; if the muon is identified in the calorimeter it is removed instead. Any $\tau _{\text {had-vis}}$ candidate within $\Delta R = 0.2$ of an electron or a muon (which must be reconstructed in the MS if the $\tau _{\text {had-vis}}$ $p_{\text {T}}$ is above 50 $\text {GeV}$) is then rejected. Jets are discarded if they lie within $\Delta R = 0.2$ of an electron or have fewer than three associated tracks and lie within the same distance of a muon. Electron or muon ($\tau _{\text {had-vis}}$) candidates within $\Delta R = 0.4$ ($\Delta R = 0.2$) of any remaining jet are then removed. Finally, ambiguities between anti-$\tau _{\text {had-vis}}$ candidates and jets within $\Delta R = 0.2$ are resolved in favour of the jet if it is b-tagged or the anti-$\tau _{\text {had-vis}}$ otherwise.

5 Event selection

The event selection targets a signature consisting of a pair of $\tau$-leptons and a pair of b-quarks. It splits the events into two orthogonal signal categories based on the $\tau$-lepton decay mode: the $\tau _{\text {lep}}\tau _{\text {had}}$ channel, which selects events with a light lepton, an oppositely charged $\tau _{\text {had-vis}}$ and one or two b-jets, and the $\tau _{\text {had}}\tau _{\text {had}}$ channel, which selects events with two opposite-charge $\tau _{\text {had-vis}}$ and one or two b-jets. Multivariate techniques are used to search for a LQ-pair signal in the two signal regions (SRs).

5.1 Signal regions

Candidate events were recorded using a combination of single-light-lepton [105, 106] and single-$\tau _{\text {had-vis}}$ triggers [107]. The single-lepton trigger used in the $\tau _{\text {lep}}\tau _{\text {had}}$ channel required a reconstructed light lepton at the HLT, with a minimum $E_{\text {T}}$ threshold ranging from 24 to 26 $\text {GeV}$ for electrons and a minimum $p_{\text {T}}$ threshold ranging from 20 to 25 $\text {GeV}$ for the muons, depending on the data-taking period. Offline leptons are required to be geometrically matched to the corresponding trigger object and have a $p_{\text {T}}$ threshold 1–2 $\text {GeV}$ above the HLT threshold so that the trigger was fully efficient. The single-$\tau _{\text {had-vis}}$ triggers used in the $\tau _{\text {had}}\tau _{\text {had}}$ channel required a reconstructed HLT $\tau _{\text {had-vis}}$ with a period-dependent minimum $p_{\text {T}}$ threshold ranging between 80 $\text {GeV}$ and 160 $\text {GeV}$. The corresponding $p_{\text {T}}$-threshold for the offline $\tau _{\text {had-vis}}$, which is again required to be geometrically matched to the trigger object, ranges between 100 $\text {GeV}$ and 180 $\text {GeV}$, while the non-trigger-matched $\tau _{\text {had-vis}}$ is required to have $p_{\text {T}} > 20$ $\text {GeV}$.

Following the trigger selection, the $\tau _{\text {lep}}\tau _{\text {had}}$ category requires exactly one ‘signal’ light lepton and an oppositely charged $\tau _{\text {had-vis}}$, while the $\tau _{\text {had}}\tau _{\text {had}}$ category requires exactly two opposite-charge $\tau _{\text {had-vis}}$ and no ‘veto’ light leptons. Both categories require at least two jets, one or two of which must be b-tagged, with $p_{\text {T}} > 45~(20)$ $\text {GeV}$ for the leading (sub-leading) jet.

The invariant mass of the two $\tau$-lepton decay products is an important variable with which to reject the Z+jets background. It is calculated using the missing mass calculator (MMC) [108], with the light lepton and the $\tau _{\text {had-vis}}$ (two $\tau _{\text {had-vis}}$) and the ${\pmb p_{\text {T}}^{\text {miss}}}$ as input in the $\tau _{\text {lep}}\tau _{\text {had}}$ ($\tau _{\text {had}}\tau _{\text {had}}$) category, and it is required to satisfy $m_{\tau \tau }^{\text {MMC}}\notin 40$–150 $\text {GeV}$. Two further selections are applied to target the characteristic LQ signature while reducing the large multi-jet background. The scalar sum of the transverse momenta ($s_{\text {T}}$), calculated taking into account the light lepton or $\tau _{\text {had-vis}}$, two leading jets and the $E_{\text {T}}^{\text {miss}}$, is a powerful discriminator. It is required to satisfy $s_{\text {T}}> 600$ $\text {GeV}$, while the $E_{\text {T}}^{\text {miss}}$ itself is required to exceed 100 $\text {GeV}$.

The full event selection is summarised in Table 2 and the resulting acceptance times efficiency is shown in Fig. 2 as a function of $m_{\text {LQ}}$. Since the analysis prioritises high mass LQs that have not yet been excluded in the benchmark models under consideration, it is not optimal for low LQ masses.

5.2 Multivariate signal extraction

Following the event selection, the LQ signal is extracted using a multivariate discriminant. To obtain near-optimal sensitivity and continuity over the full range of LQ masses considered, a parameterised neural network (PNN) [109], parameterised in terms of the generated LQ mass, is chosen. The PNN consists of three hidden layers, each with 32 nodes, implemented in Keras [110] with the Tensorflow  [111] backend.

The PNN inputs consist of a combination of multiplicity, kinematic and angular quantities that discriminate between the signal and the dominant background. In the case of the $b\tau$ invariant mass, the most likely combination of the $\tau$-lepton and a b-jet⁴ is chosen based on a mass-pairing strategy that minimises the mass difference between the two resulting LQ candidates. The variables, which are similar for both the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ categories, are summarised in Table 3 and defined as follows:

• $\tau _{\text {had-vis}}$ $p_{\text {T}} ^{0}$ is the transverse momentum of the highest-$p_{\text {T}}$ $\tau _{\text {had-vis}}$;

• $s_{\text {T}}$ is the scalar sum of the transverse momenta defined above;

• $N_{b-\text {jets}}$ is the number of b-jets;

• $m(\tau , \text {jet})_{0,1}$ are the larger (0) and smaller (1) of the two LQ masses obtained via the mass-pairing strategy ($\tau _{\text {had}}\tau _{\text {had}}$ channel only);

• $m(\ell , \text {jet})$ and $m(\tau _{\text {had}}, \text {jet})$ are the mass of the light-lepton or $\tau _{\text {had-vis}}$, respectively, combined with its mass-paired b-jet ($\tau _{\text {lep}}\tau _{\text {had}}$ channel only);

• $\Delta R(\ell , \text {jet})$ ($\Delta R(\tau _{\text {had}}, \text {jet})$) is the $\Delta R$ between the light lepton (leading $\tau _{\text {had-vis}}$) and the mass-paired jet in the $\tau _{\text {lep}}\tau _{\text {had}}$ ($\tau _{\text {had}}\tau _{\text {had}}$) category;

• $\Delta \phi (\ell , E_{\text {T}}^{\text {miss}})$ is the azimuthal opening angle between the lepton and the $E_{\text {T}}^{\text {miss}}$ ($\tau _{\text {lep}}\tau _{\text {had}}$ category only);

• $E_{\text {T}}^{\text {miss}}$ $\phi$ centrality quantifies the transverse direction of the ${\pmb p_{\text {T}}^{\text {miss}}}$ relative to the light lepton and $\tau _{\text {had-vis}}$ (two $\tau _{\text {had-vis}}$) in the $\tau _{\text {lep}}\tau _{\text {had}}$ ($\tau _{\text {had}}\tau _{\text {had}}$) category and is defined in Ref. [112].

A selection of representative input distributions, after the background corrections described in Sect. 6, are presented in Figures 3 and 4 for the $\tau _{\text {lep}}\tau _{\text {had}}$ SR and the $\tau _{\text {had}}\tau _{\text {had}}$ SR, respectively. While the relative importance of the variables varies with LQ mass, the $s_{\text {T}}$ and mass variables are generally the most performant.

Table 3 Summary of variables used as inputs to the PNN in the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ categories. The variables are defined in the text

Fig. 3

Signal (solid lines), post-fit background (filled histograms) and data (dots with statistical error bars) distributions of representative PNN input variables in the $\tau _{\text {lep}}\tau _{\text {had}}$ SR: a $\Delta R(\ell , \text {jet})$, b $m(\tau _{\text {had}}, \text {jet})$ and c $s_{\text {T}}$. The normalisation and shape of the backgrounds are determined from the background-only likelihood fit to data and the ratios of the data to the sum of the predicted backgrounds are shown in the lower panels. ‘Other’ refers to the sum of minor backgrounds (vector boson + jets, diboson and Higgs boson). The hatched band indicates the combined statistical and systematic uncertainty in the total background prediction. The expected signal for a 1.4 $\text {TeV}$ scalar LQ, scaled by the indicated factor for visibility, is overlaid. The last bin includes the overflow

Fig. 4

Signal (solid lines), post-fit background (filled histograms) and data (dots with statistical error bars) distributions of representative PNN input variables in the $\tau _{\text {had}}\tau _{\text {had}}$ SR: a $\Delta R(\tau _{\text {had}} ^0, \text {jet})$ where $\tau _{\text {had}} ^0$ is the leading $\tau$-lepton, b the larger of the two $\tau$-jet mass combinations $m(\tau _{\text {had}}, \text {jet})_0$ and c $s_{\text {T}}$. The normalisation and shape of the backgrounds are determined from the background-only likelihood fit to data and the ratios of the data to the sum of the predicted backgrounds are shown in the lower panels. ‘Other’ refers to the sum of minor backgrounds (vector boson + jets, diboson and Higgs boson). The hatched band indicates the combined statistical and systematic uncertainty in the total background prediction. The expected signal for a 1.4 $\text {TeV}$ scalar LQ, scaled by the indicated factor for visibility, is overlaid. The last bin includes the overflow

The PNNs are trained on all scalar LQ signal masses simultaneously against the main $t\bar{t}$ and single-top backgrounds, taking into account both the true and misidentified $\tau _{\text {had-vis}}$ components with the latter corrected as described in Sect. 6. The same PNN training is used for both vector LQ models since separate trainings are found to provide a negligible improvement in sensitivity. For the signals, the generated LQ mass is used as the parameterisation input in addition to the input variables described above, while in the case of the backgrounds a mock LQ mass is randomly assigned from the range of signal LQ masses such that the resulting training is independent of the mass. In all cases, the input variables are standardised by subtracting the median value and dividing by the interquartile range. Comparing the output between the training dataset and an independent testing dataset showed no sign of overtraining. The resulting PNN score distributions, which peak at higher values for LQ signals than for the background processes, are used as the final analysis discriminants.

6 Background modelling

The dominant background in the $\tau _{\text {had}}\tau _{\text {had}}$ and $\tau _{\text {lep}}\tau _{\text {had}}$ channels is top production, including $t\bar{t}$ and single-top-quark production. A subdominant background is Z boson production in association with heavy-flavour quarks (bb, bc, cc), termed $Z+\text {HF}$ hereafter. Both top production and $Z+\text {HF}$ are estimated from simulation to which data-driven corrections are applied. In the $\tau _{\text {had}}\tau _{\text {had}}$ channel, multi-jet events form a non-negligible background that is estimated by using data-driven techniques. Small contributions to the background from all other processes are estimated by using simulated events. This section describes the background estimation methods used for top-quark-pair and single-top backgrounds, multi-jet backgrounds, and the $Z+\text {HF}$ background. The background is validated for $\tau _{\text {had}}\tau _{\text {had}}$ and $\tau _{\text {lep}}\tau _{\text {had}}$ events in a region with an inverted $s_{\text {T}}$ selection, as well as a region with a low PNN score and the signal region selection. In addition, the $\tau _{\text {had}}\tau _{\text {had}}$ multi-jet estimate is validated in a region where the two $\tau _{\text {had-vis}}$ have the same electric charge. The potential signal contamination in all regions described in this section is negligible.

The process of estimating the backgrounds follows several steps. First, an overall shape correction is determined for the top background, as described in Sect. 6.1.1. Then, with this in place, a shape and normalisation correction is determined for the top backgrounds with jets misidentified as $\tau _{\text {had-vis}}$, as described in Sect. 6.1.2. After applying these corrections, a prediction for the shape and normalisation of multi-jet backgrounds is determined for the $\tau _{\text {had}}\tau _{\text {had}}$ channel in Sect. 6.2. Finally, with all relevant corrections in place, a normalisation factor is determined for the $Z+\text {HF}$ backgrounds, as described in Sect. 6.3. The resulting corrections are only weakly coupled due to the high purity of each control region, meaning that corrections for a specific background process do not significantly affect the overall background in control regions targeting other backgrounds. All of these corrections are applied in the final SR fit.

6.1 Top quark backgrounds

For top-quark-pair and single-top-quark production (top backgrounds), events are estimated separately based on whether the $\tau _{\text {had-vis}}$ candidate in the event is correctly identified (referred to as a true $\tau _{\text {had-vis}}$) or whether it is a quark- or gluon-initiated jet misidentified as a $\tau _{\text {had-vis}}$ (referred to as a fake $\tau _{\text {had-vis}}$). The small contributions from light leptons that are misidentified as $\tau _{\text {had-vis}}$ are considered together with the true $\tau _{\text {had-vis}}$ contribution. Events with a true $\tau _{\text {had-vis}}$ and a hadronic jet misidentified as a light lepton contribute negligibly to the $\tau _{\text {lep}}\tau _{\text {had}}$ channel and are not considered.

These backgrounds are estimated in a multi-step data-driven process that is applied to simulated events. First, all top backgrounds are scaled by an $s_{\text {T}}$-dependent reweighting factor (RF), and then simulated background events with misidentified $\tau _{\text {had-vis}}$ are further corrected by a scale factor (SF) that is binned in the $\tau _{\text {had-vis}}$ $p_{\text {T}}$.

6.1.1 Overall reweighting of top backgrounds

The motivation for scaling the $t\bar{t}$ and single-top backgrounds arises from mismodelling of the data by simulation observed in control regions (CRs). It is seen that this effect becomes more pronounced for events with higher momentum top quarks, which is where this analysis is primarily focused. This mismodelling has also been observed in ATLAS measurements of the $t\bar{t}$ differential cross-section, where it is seen that the number of events is overestimated at high top-quark $p_{\text {T}}$ [113,114,115].

For this reason, a CR is defined to determine a binned shape and normalisation correction of the simulated top quark events to data. Events in this CR are required to have two b-jets with $p_{\text {T}}$ greater than 45 and 20 $\text {GeV}$, exactly two light leptons (ee, $\mu \mu$ or $e\mu$) with opposite charge, $E_{\text {T}}^{\text {miss}} > 100$  $\text {GeV}$, and a dilepton mass ($m_{\ell \ell }$) $>110$  $\text {GeV}$. They are also required to have $m_{b\ell } > 250$ $\text {GeV}$, where $m_{b\ell } = \min (\max (m_{b_{0}\ell _{0}},m_{b_{1}\ell _{1}}),\max (m_{b_{0}\ell _{1}},m_{b_{1}\ell _{0}}))$, where the 0 and 1 indices refer to the leading and sub-leading b-tagged jets and leptons in order of transverse momentum. This region is orthogonal to the SRs and is over 99% pure in $t\bar{t}$ events.

The RFs are derived by subtracting all non-top backgrounds, as estimated using simulation, from data. A ratio of the remaining events to the prediction of $t\bar{t}$ and single-top events in simulation is then calculated. This factor is binned in $s_{\text {T}}$, with one bin up to 400 $\text {GeV}$, steps of 100 $\text {GeV}$ from 400 to 1400 $\text {GeV}$, and then one bin for values greater than 1400 $\text {GeV}$. The values of the RFs decrease from 0.97 at low $s_{\text {T}}$ to approximately 0.62 in the highest $s_{\text {T}}$ bin. Even in the highest $s_{\text {T}}$ bin, the signal contamination remains at the percent level. The largest relative contribution of single-top events is also at high $s_{\text {T}}$. This reweighting is applied in both the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ SRs for $t\bar{t}$ and single-top events with true and misidentified $\tau$-leptons, as well as in all CRs. The uncertainty in this RF is taken from the statistical uncertainty in the factor, bin-by-bin in $s_{\text {T}}$, and its impact on the shape and normalisation of the final PNN score distribution are considered. In addition, top background modelling uncertainties are propagated through the reweighting process, so that modified RFs are applied when evaluating such uncertainties in the final fit.

6.1.2 Top backgrounds with jets misidentified as $\tau _{\text {had-vis}}$

In addition to this overall RF, the estimation of top backgrounds with jets misidentified as $\tau _{\text {had-vis}}$ in the SRs is performed using simulated events with additional data-driven corrections. A fit is performed in a $\tau _{\text {lep}}\tau _{\text {had}}$-based CR to simultaneously correct the overall normalisation of true $\tau _{\text {had-vis}}$ and misidentified $\tau _{\text {had-vis}}$ events while deriving an SF to be applied to misidentified $\tau _{\text {had-vis}}$ events in the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ SRs. The RF for top backgrounds is applied to this CR before the fit. The SFs obtained are then applied in the SRs, in order to correct the $\tau _{\text {had-vis}}$ misidentification rate in simulation to that observed in data.

The CR has the same selection as the SR for the $\tau _{\text {lep}}\tau _{\text {had}}$ channel, except that the $\tau _{\text {had-vis}}$ $p_{\text {T}} > 100$ $\text {GeV}$ requirement is removed and $s_{\text {T}}$ is required to be in a range of 400–600 $\text {GeV}$. This region is 97% pure in $t\bar{t}$ events, with a mixture of both correctly identified and misidentified $\tau _{\text {had-vis}}$ that varies with $\tau _{\text {had-vis}}$ $p_{\text {T}}$.

The distribution used for this estimation is the transverse mass of the light lepton and missing transverse momentum, defined as $m_{\text {T}}(\ell , E_{\text {T}}^{\text {miss}}) =$$\sqrt{(E_{\text {T}}^{\text {miss}} + p_{\text {T},\ell })^2 - (E_{\text {T},x}^{\text {miss}} + p_{x,\ell })^2 - (E_{\text {T},y}^{\text {miss}} + p_{y,\ell })^2 }$. The expected shapes for top backgrounds with true and misidentified $\tau _{\text {had-vis}}$ in this distribution differ significantly, making it possible to constrain the two background sources. The normalisation of the true and misidentified $\tau _{\text {had-vis}}$ background is allowed to vary freely, and SFs for the misidentified $\tau _{\text {had-vis}}$ background are determined in bins of $\tau _{\text {had-vis}}$ $p_{\text {T}}$. All detector-related uncertainties and top background modelling uncertainties are included as nuisance parameters in the fit. An example fit in a single bin of $p_{\text {T}}$ is shown in Fig. 5 for the $\tau _{\text {had-vis}}$ CRs. Depending on $\tau _{\text {had-vis}}$ $p_{\text {T}}$, the SFs run from 0.90 in the lowest $p_{\text {T}}$ bin down to 0.56 in the highest $p_{\text {T}}$ bin.

Fig. 5

Post-fit plots for true and misidentified $\tau _{\text {had-vis}}$ in the $\tau _{\text {had-vis}}$ CR, in an example $p_{\text {T}}$ bin ($\tau _{\text {had-vis}}$ $p_{\text {T}}$ >100 $\text {GeV}$). ‘Other’ refers to the sum of minor backgrounds (vector boson + jets, diboson and Higgs boson). The lower panels show the ratios of the data to the sum of the predicted backgrounds. The hatched bands indicate the combined statistical and systematic uncertainty in the total background predictions. The dashed lines denote the total pre-fit backgrounds for comparison, while the last bins include the overflow

For the estimation of top backgrounds with misidentified $\tau _{\text {had-vis}}$, an uncertainty is considered that arises from the limited number of events and an additional uncertainty is defined by comparing the nominal SFs to SFs derived with a more inclusive $s_{\text {T}}$ selection ($s_{\text {T}}<600$ $\text {GeV}$). This last uncertainty is intended to address a possible $s_{\text {T}}$-dependence in the mismodelling of top backgrounds. The difference between the central values for SFs measured with these two $s_{\text {T}}$ selections is taken as the $s_{\text {T}}$-dependence uncertainty.

6.2 Multi-jet backgrounds with jets misidentified as $\tau _{\text {had-vis}}$

For the $\tau _{\text {had}}\tau _{\text {had}}$ channel, multi-jet processes can contribute to the SR at non-negligible levels. For this reason, the $\tau _{\text {had}}\tau _{\text {had}}$ channel uses a data-driven fake-factor (FF) method to estimate this background. These FFs are measured in a CR with the same selection as the $\tau _{\text {had}}\tau _{\text {had}}$ SR, except that the two $\tau _{\text {had-vis}}$ candidates have the same charge and the $E_{\text {T}}^{\text {miss}}$ requirement is loosened to 80 $\text {GeV}$. The FF is defined as the ratio of events where both $\tau _{\text {had-vis}}$ are loose to the number of events where one $\tau _{\text {had-vis}}$ is loose and the other is an anti-$\tau _{\text {had-vis}}$. These FFs are derived as a function of transverse momentum and the number of charged-particle tracks of the $\tau _{\text {had-vis}}$ candidate. The FFs are measured from data after subtracting all predicted non-multi-jet background contributions. The FFs range between approximately zero and 0.25.

The non-multi-jet background contributions that are subtracted, however, suffer from the same mismodelling issues described in the previous two sections. The top backgrounds are therefore corrected by the RFs and SFs derived as described in Sects. 6.1.1 and 6.1.2, respectively. Since the SFs are anticipated to be different for fake $\tau _{\text {had-vis}}$ passing the $\tau _{\text {had-vis}}$ and anti-$\tau _{\text {had-vis}}$ requirements, dedicated SFs are measured for this data-driven estimation. Specifically, the anti-$\tau _{\text {had-vis}}$ region uses SFs that are derived in a CR as described in Sect. 6.1.2, except that the $\tau _{\text {had-vis}}$ identification requirement is changed to that of an anti-$\tau _{\text {had-vis}}$. An example fit in a single bin of $p_{\text {T}}$ is shown in Fig. 6 for the anti-$\tau _{\text {had-vis}}$ CR. Depending on $\tau _{\text {had-vis}}$ $p_{\text {T}}$, these SFs vary between 0.77 and 0.95.

To construct the background estimate, FFs are applied to a region with the $\tau _{\text {had}}\tau _{\text {had}}$ SR selection, except that the $\tau _{\text {had-vis}}$ identification requirement is changed to that of an anti-$\tau _{\text {had-vis}}$. This provides both a shape and a normalisation for the multi-jet contribution in the PNN score distribution.

Fig. 6

Post-fit plot for true and misidentified $\tau _{\text {had-vis}}$ in the the anti-$\tau _{\text {had-vis}}$ CR, in an example $p_{\text {T}}$ bin ($\tau _{\text {had-vis}}$ $p_{\text {T}}$ >100 $\text {GeV}$). ‘Other’ refers to the sum of minor backgrounds (vector boson + jets, diboson and Higgs boson). The lower panels show the ratios of the data to the sum of the predicted backgrounds. The hatched bands indicate the combined statistical and systematic uncertainty in the total background predictions. The dashed lines denote the total pre-fit backgrounds for comparison, while the last bins include the overflow

For the estimation of multi-jet backgrounds in $\tau _{\text {had}}\tau _{\text {had}}$, uncertainties are considered due to the statistical uncertainty of the FFs, decorrelated in $p_{\text {T}}$ bins, and to the uncertainty in the subtraction of different backgrounds using simulation. Top events with a correctly identified $\tau _{\text {had-vis}}$, top events with a misidentified $\tau _{\text {had-vis}}$, and other small backgrounds are considered separately. The top events are varied by the overall uncertainty defined by the procedure to determine the modelling uncertainties, but evaluated in the anti-$\tau _{\text {had-vis}}$ region, resulting in an uncertainty of approximately 50%. The other backgrounds are varied by a conservative value of 30%; inflating the size of this uncertainty further was found to have negligible impact on the result. In addition, a 20% overall uncertainty in the estimate is applied based on checks of the method in the validation regions described above. The total uncertainty in the multi-jet background is $-64$% and $+61$%.

6.3 $Z+\text {HF}$ background

The normalisation of the $Z+\text {HF}$ background, which is a relatively small contribution in the SRs, is observed to be in disagreement with the NLO cross-section in Sherpa (e.g. Ref. [116]). It is therefore determined from data using a $Z+\text {HF}$ CR that targets events containing a Z boson decaying into a light-lepton pair and produced in association with two heavy-flavour jets. The composition of this control region is approximately 60% $Z+\text {HF}$ events and 40% $t\bar{t}$ events, with less than 1% arising from backgrounds with misidentified $\tau _{\text {had-vis}}$. Since the contribution from backgrounds with misidentified $\tau _{\text {had-vis}}$ is negligible, only the RF for the $t\bar{t}$ shape is included in this CR.

Data for the CR was recorded using a combination of the single-lepton triggers described above and additional dilepton triggers requiring pairs of same-flavour leptons. At the analysis level exactly two oppositely-charged same-flavour leptons, passing the ‘veto’ quality requirements and $p_{\text {T}}$ thresholds based on the corresponding trigger thresholds, are required. The invariant mass of the resulting lepton pair $m_{\ell \ell }$ is required to lie between 75 $\text {GeV}$ and 110 $\text {GeV}$. In addition, exactly two b-jets with $p_{\text {T}} > 20$ $\text {GeV}$ are required and their invariant mass $m_{bb}$ is required to be less than 40 $\text {GeV}$ or greater than 150 $\text {GeV}$ to avoid the Higgs boson mass peak. The RFs for top backgrounds derived in Sect. 6.1.1 are then applied.

A fit to the $m_{\ell \ell }$ distribution is performed to discriminate between the $Z+\text {HF}$ and top backgrounds, with the normalisation of both processes allowed to vary freely and all systematic uncertainties described in Sect. 7 included. The resulting $Z+\text {HF}$ normalisation factor is $1.36 \pm 0.11$ and is used to correct the $Z+\text {HF}$ background entering into the final fit (described in Sect. 8), which is allowed to vary within the associated uncertainty.

7 Systematic uncertainties

The systematic uncertainties considered include detector-related uncertainties, modelling and theoretical uncertainties, and uncertainties derived for the data-driven background estimates, the latter of which have already been described in Sect. 6. Uncertainties are evaluated by shifting the central value upward or downward by one standard deviation, and then propagating the differences to the PNN score distributions that are used in the final fit.

Detector-related uncertainties are defined as uncertainties relating to the detector response, object reconstruction and object identification. There are systematic uncertainties associated with each of the reconstructed objects considered, as well as the $E_{\text {T}}^{\text {miss}}$. For light leptons, $\tau _{\text {had-vis}}$, and jets, uncertainties are considered for energy scale and resolution, reconstruction and identification, while uncertainties in isolation are also considered for light leptons. For the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ channel, uncertainties associated with the lepton and $\tau _{\text {had-vis}}$ trigger efficiencies, respectively, are considered. For b-jets, additional uncertainties are considered for the efficiency of (mis)tagging b-jets, c-jets, and light-quark-initiated jets. Uncertainties related to energy scale and resolution, and the inclusion of soft terms, are considered for the $E_{\text {T}}^{\text {miss}}$. Finally, there is also an uncertainty associated with shape and normalisation components that arises from uncertainties in the simulation of pile-up collisions.

Theoretical and modelling uncertainties include uncertainties in the cross-section calculations, which have only a normalisation component, and uncertainties in the acceptance, for which normalisation and shape components are considered. For top backgrounds, relative acceptance uncertainties are also defined to take into account normalisation differences for $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ SRs.

For $t\bar{t}$ processes, shape and normalisation uncertainties are considered that arise from changing the matrix element and parton shower simulation software, and from varying the initial and final state radiation, PDF, and $\alpha _s$. The matrix element uncertainty is determined by comparing the Powheg+Pythia 8 sample with an aMC@NLO+Pythia 8 sample. The parton shower uncertainty is determined by comparing the Powheg+Pythia 8 sample with a Powheg+Herwig 7 [117, 118] sample. The other modelling uncertainties are evaluated using internal weights in the nominal $t\bar{t}$ sample.

For single-top processes, acceptance uncertainties with shape and normalisation components are considered. Uncertainties are considered from changing the matrix element, parton shower, and impacts of diagram interference. In addition, variations of initial- and final-state radiation and PDFs are considered. The matrix element uncertainty is determined by comparing the Powheg+Pythia 8 sample with an aMC@NLO+Pythia 8 sample. The parton shower uncertainty is determined by comparing the Powheg+Pythia 8 sample with a Powheg+Herwig 7 sample. The diagram interference uncertainty is evaluated by comparing the nominal single top samples, which use a diagram removal scheme, with alternative samples that utilise a diagram subtraction scheme [119]. The other modelling uncertainties are evaluated using internal weights in the nominal single-top samples.

All $t\bar{t}$ and single-top modelling uncertainties are also propagated through the top reweighting procedure, such that there is an uncertainty in the RF corresponding to each modelling uncertainty.

For all Z+jets processes, uncertainties due to the choice of generator are evaluated by comparing the nominal Sherpasimulated samples with alternative samples simulated by MadGraphwith LO-accurate matrix elements that contain up to four final-state partons, using Pythiafor parton showering. In addition, uncertainties are considered by taking an envelope of variations in the renormalisation and factorisation scales and PDF values using internal weights in the simulated Sherpasample. For this process specifically, uncertainties are also included based on varying the matrix element matching scale and the resummation scale for soft-gluon emission. Uncertainties are considered separately for the $Z+\text {HF}$ background and the remaining $Z+(bl,cl,ll)$ background, where l indicates a light-flavour jet, and only normalization is taken into account. All of these uncertainties are included in the $Z+\text {HF}$ fit described in Sect. 6.3, and their sum in quadrature, taking relative acceptance uncertainties into account, is considered as the uncertainty in the SRs for the final fit.

Table 4 Post-fit yields for background events, determined from a background-only fit, compared with the observed number of data events in the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ SRs. ‘Fake $\tau _{\text {had}}$ (top)’ refers to top backgrounds where a jet is misidentified as the $\tau _{\text {had-vis}}$ of the event. ‘Other’ refers to the sum of minor backgrounds (vector boson + jets, diboson and Higgs boson); it is primarily composed of Z$\rightarrow \tau \tau$ in association with light-flavour quarks and $W \rightarrow \ell \nu ~+~$jets events. The total background is not identical to the sum of the individual components since the latter are rounded for presentation, while the sum is calculated with the full precision before being subsequently rounded. Systematic uncertainties are included. Due to the large correlations, individual uncertainties can be significantly larger than the total uncertainty

For other minor backgrounds, the following uncertainties are considered. A conservative uncertainty of 50% on the normalisation of the diboson backgrounds is included. For W+jets processes, which form a background to the analysis due to a jet being misidentified as a light lepton or a $\tau _{\text {had}}$, a conservative 100% uncertainty is taken into account, decorrelated between the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ channels. Within the $\tau _{\text {lep}}\tau _{\text {had}}$ channel, the uncertainty is also decorrelated between the cases where it is the light lepton or the $\tau _{\text {had}}$ that is misidentified. In both cases, the choice of the size of these uncertainties has a negligible impact on the results.

Fig. 7

The PNN score distributions in the $\tau _{\text {lep}}\tau _{\text {had}}$ SR for a $m_{\text {LQ}}= 500$ $\text {GeV}$, b $m_{\text {LQ}}= 1.1$ $\text {TeV}$, c $m_{\text {LQ}}= 1.4$ $\text {TeV}$. The normalisation and shape of the backgrounds are determined from the background-only likelihood fit to data and the ratios of the data to the sum of the backgrounds are shown in the lower panels. ‘Other’ refers to the sum of minor backgrounds (vector boson + jets, diboson and Higgs boson). The hatched bands indicate the combined statistical and systematic uncertainty in the total background predictions. The expected signals for scalar LQs with the corresponding masses, scaled by the indicated factors for visibility, are overlaid. Since the PNN score itself is not a physical quantity, it is represented solely by the bin number

Fig. 8

The PNN score distributions in the $\tau _{\text {had}}\tau _{\text {had}}$ SR for a $m_{\text {LQ}}= 500$ $\text {GeV}$, b $m_{\text {LQ}}= 1.1$ $\text {TeV}$, c $m_{\text {LQ}}= 1.4$ $\text {TeV}$. The normalisation and shape of the backgrounds are determined from the background-only likelihood fit to data and the ratios of the data to the sum of the backgrounds are shown in the lower panels. ‘Other’ refers to the sum of minor backgrounds (vector boson + jets, diboson and Higgs boson). The hatched bands indicate the combined statistical and systematic uncertainty in the total background predictions. The expected signals for scalar LQs with the corresponding masses, scaled by the indicated factors for visibility, are overlaid. Since the PNN score itself is not a physical quantity, it is represented solely by the bin number

For signal samples, uncertainties arising from variations of scale, initial-state radiation, PDF, and $\alpha _s$ are considered, using alternative weights internal to the signal samples. Differences in shape are observed to be negligibly small in the PNN score distributions, so only variations in normalisation are included for the final fit.

The relative impact of the different sources of uncertainty on the analysis varies depending on the LQ model considered and the mass probed. Generally, the largest impact comes from the statistical uncertainties, which increase with $m_{\text {LQ}}$. In the scalar LQ case, for example, the statistical impact on the limit ranges from 60% at the lowest $m_{\text {LQ}}$ evaluated to 80% above 1000 GeV. The main systematic uncertainties come from the $t\bar{t}$ and single-top-quark modelling uncertainties, including their interference, and normalisation. There is also a significant effect from the signal acceptance uncertainties, which increases with $m_{\text {LQ}}$, particularly for the vector LQ models.

Fig. 9

The observed (solid line) and expected (dashed line) 95% CL upper limits on the LQ pair production cross-sections assuming $\mathcal {B}= 1$ as a function of $m_{\text {LQ}}$ for a the scalar LQ case, b the vector LQ case in the minimal-coupling scenario, c vector LQs in the Yang–Mills scenario. The surrounding shaded bands correspond to the $\pm 1$ and $\pm 2$ standard deviation ($\pm 1 \sigma , \pm 2 \sigma$) uncertainty in the expected limit. The theoretical prediction in each model, along with its uncertainty, is shown by the lines with the hatched bands

8 Statistical interpretation and results

The data are compared with the expectation, including the background modelling corrections outlined in Sect. 6, by performing simultaneous binned maximum-likelihood fits to the PNN score distributions, separately for each LQ hypothesis, in the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ SRs. For each hypothesis, the binning of the PNN score distributions is chosen separately to maximise the expected sensitivity, while ensuring sufficient background events in the signal-enhanced PNN bins and preserving the stability of the fit. In addition to the relative signal-strength modifier, $\mu$, the top normalisation is free to float in the fit and is constrained by the background-enhanced PNN bins.

The statistical and systematic uncertainties affecting the signal and background model, described in Sect. 7, are represented by deviations from the nominal model scaled by Gaussian- or Poisson-constrained nuisance parameters that are profiled in the fit. Common sources of systematic uncertainty are correlated across the SRs.

The resulting event yields in the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ SRs, based on a background-only fit to the data, are presented in Table 4. Corresponding post-fit PNN score distributions for representative LQ signals at masses of 500 $\text {GeV}$, 1.1 $\text {TeV}$ and 1.4 $\text {TeV}$ are shown in Fig. 7 (Fig. 8) for the $\tau _{\text {lep}}\tau _{\text {had}}$ ($\tau _{\text {had}}\tau _{\text {had}}$) SR. At high values of the PNN score, top backgrounds dominate in the $\tau _{\text {lep}}\tau _{\text {had}}$ channel, while the $\tau _{\text {had}}\tau _{\text {had}}$ background consists of a roughly even mixture of all background sources. Overall, good agreement with the SM background expectation is observed in all cases, although there is a slight deficit of data relative to the background prediction in the highest PNN score bin for the $\tau _{\text {had}}\tau _{\text {had}}$ channel.

Table 5 Observed and expected lower limits on the LQ mass at 95% CL for the three different LQ models, assuming $\mathcal {B}= 1$

Fig. 10

The observed (solid line) and expected (dashed line) 95% CL upper limits on the branching ratio into charged leptons as a function of $m_{\text {LQ}}$ for a the scalar LQ case, b the vector LQ case in the minimal-coupling scenario, c vector LQs in the Yang–Mills scenario. The observed exclusion region is above the solid line, with the theoretical uncertainty in the model indicated by the dotted lines around this. The expected limit is indicated by the dashed line and the surrounding shaded bands correspond to the $\pm 1$ and $\pm 2$ standard deviation ($\pm 1 \sigma , \pm 2 \sigma$) uncertainty in the expected limit. No limits are presented for $\mathcal {B}< 0.1$ due to the lack of expected signal events in this final state

Since no significant excess is observed, upper limits on the scalar and vector LQ pair production cross-sections for each mass hypothesis are computed based on the modified frequentist CL$_{\text {s}}$ method [120], using a profile likelihood test statistic [121] under the asymptotic approximation. The resulting observed and expected limits, assuming $\mathcal {B}= 1$, as a function of $m_{\text {LQ}}$ at 95% confidence level (CL) are shown in Fig. 9 for all LQ models. The expected contributions of the $\tau _{\text {lep}}\tau _{\text {had}}$ and $\tau _{\text {had}}\tau _{\text {had}}$ channels are approximately equal at high $m_{\text {LQ}}$, while the $\tau _{\text {had}}\tau _{\text {had}}$ is up to a factor of two more sensitive at low $m_{\text {LQ}}$. The improvement in the observed limit compared with the expectation is driven by the data deficit in the highest $\tau _{\text {had}}\tau _{\text {had}}$ PNN score bin mentioned above and is larger at high $m_{\text {LQ}}$ since the signal becomes more localised at high PNN score as $m_{\text {LQ}}$ increases. The theoretical prediction for the cross-section of scalar or vector LQ pair production is indicated by the solid line along with its uncertainties.

The corresponding expected and observed 95% CL lower limits on the LQ mass for the three different LQ models are summarised in Table 5, providing an improvement in mass reach for a scalar LQ of more than 450 $\text {GeV}$ compared with the previous $36\,\hbox {fb}^{-1}$ result in this channel [19]. They extend the full Run 2 ATLAS reach for third-generation up-type LQs by around 200 $\text {GeV}$ in all three models compared with the search in the $LQLQ \rightarrow t\nu t\nu$ decay mode [27].

The results are also expressed as upper limits on the branching ratio to charged leptons as a function of $m_{\text {LQ}}$ for each LQ model in Fig. 10. For all models investigated, constraints on the LQ mass are reduced by no more than 15% going from $\mathcal {B}=1$ to $\mathcal {B}= 0.5$, while scalar LQ masses up to around 850 $\text {GeV}$ are excluded for couplings into charged leptons as low as 0.1; the corresponding $\mathcal {B}=0.1$ exclusion for vector LQ is around 1100 $\text {GeV}$ (1300 $\text {GeV}$) in the minimal-coupling (Yang–Mills) scenario.

9 Conclusion

A search for pair-produced scalar or vector leptoquarks decaying into a b-quark and a $\tau$-lepton is presented. The analysis exploits the full data sample recorded with the ATLAS detector in Run 2 of the LHC, corresponding to 139 fb$^{-1}$ of proton-proton collisions at $\sqrt{s} = 13$ $\text {TeV}$. No significant deviations from the Standard Model expectation are observed and upper limits on the production cross-section are derived as a function of LQ mass and branching ratio into a charged lepton. Scalar LQs with masses below 1460 $\text {GeV}$ are excluded assuming a 100% branching ratio, while for vector LQs the corresponding limit is 1650 $\text {GeV}$ (1910 $\text {GeV}$) in the minimal-coupling (Yang–Mills) scenario. For branching ratios as low as 10%, scalar LQ masses below around 860 $\text {GeV}$ are excluded; the corresponding mass limits for vector LQs are 1120 $\text {GeV}$ (1360 $\text {GeV}$) in the minimal-coupling (Yang–Mills) scenario. These results significantly improve the sensitivity compared to previous ATLAS LQ searches, extending the mass reach for third-generation up-type LQs by more than 200 $\text {GeV}$ in all models and surpassing the previous ATLAS search in this final state by more than 450 $\text {GeV}$ for scalar LQs. In addition to the increased luminosity, this is due to upgraded hadronic $\tau$-lepton and b-jet identification, improved multivariate techniques and better background estimation methods.