1 Introduction

The standard model (SM) of particle physics has been exceptionally successful in accommodating a multitude of experimental measurements and observations, yet it falls short in a variety of aspects. These shortcomings motivate theoretical extensions of the SM that typically introduce new particles, which could be created in proton–proton (\(\text{ p }{}{}\) \(\text{ p }{}{}\)) collisions at the CERN LHC. In this analysis, we test theoretical models that predict new heavy resonances that decay to pairs of vector bosons (\(\text{ W }{}{}\) and \(\text{ Z }{}{}\) bosons, collectively referred to as \(\text{ V }{}{}\) bosons). These models usually aim to clarify open questions in the SM such as the large difference between the electroweak and the Planck scales. We consider the bulk scenario of the Randall–Sundrum (RS) model with warped extra dimensions [1,2,3,4,5], where the spin-2 bulk graviton has an enhanced branching fraction to massive particles, and the heavy vector triplet (HVT) framework [6], which serves as a template that reproduces a large class of explicit models predicting spin-1 resonances.

No significant deviations from the SM background expectation have been observed in previous searches by the CMS Collaboration for such particles in the \(\text{ V }{}{}\) \(\text{ V }{}{}\)  [7,8,9,10,11] and \(\text{ V }{}{}\) \(\text{ H }{}{}\)  [12,13,14,15,16,17] channels, where \(\text{ H }{}{}\) denotes the Higgs boson. Similar results were obtained independently by the ATLAS Collaboration in \(\text{ V }{}{}\) \(\text{ V }{}{}\)  [18,19,20,21,22,23] and \(\text{ V }{}{}\) \(\text{ H }{}{}\)  [24,25,26] resonance searches. In addition, statistical combinations of diboson and leptonic decay channels of the 2016 data set [27, 28] have been performed, which extend the exclusion regions of the individual analyses. Lower limits on the masses of these resonances have been set at the \(\text {Te}\text {V}\) scale. The search presented here focusses on resonances with masses above 1.2 \(\text {Te}\text {V}\), in the decays of which the vector bosons are produced at high Lorentz boost. Because of the large boost of the vector bosons, their decay products are merged into single, large-radius jets, leading to dijet final states. These jets are identified through dedicated jet substructure algorithms. Compared to previous analyses in this final state [7, 10, 21, 23, 25, 29], an improved background estimation and signal extraction procedure based on a three-dimensional (3D) maximum likelihood fit is employed, increasing the sensitivity of the analysis. The method can be applied to any search with final states expected to cause resonant behaviour in three observables, whereas previous methods used solely the invariant mass of the final decay products as the search variable. The improved sensitivity and scope has motivated a reanalysis of the \(\text{ p }{}{}\) \(\text{ p }{}{}\) collision data collected by the CMS experiment during the 2016 data taking period, as well as a first analysis of the 2017 data set, corresponding to a total integrated luminosity of 77.3 \(\,\text {fb}^{-1}\) at a centre-of-mass energy of 13 \(\text {Te}\text {V}\).

2 The CMS detector

The central feature of the CMS apparatus is a superconducting solenoid of 6 m internal diameter, providing a magnetic field of 3.8 T. Within the solenoid volume are a silicon pixel and strip tracker, a lead tungstate crystal electromagnetic calorimeter, and a brass and scintillator hadron calorimeter, each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity (\(\eta \)) coverage provided by the barrel and endcap detectors. Muons are detected in gas-ionization chambers embedded in the steel flux-return yoke outside the solenoid. A more detailed description of the CMS detector, together with a definition of the coordinate system used and the relevant kinematic variables, can be found in Ref. [30].

Events of interest are selected using a two-tiered trigger system [31]. The first level, composed of custom hardware processors, uses information from the calorimeters and muon detectors to select events at a rate of around 100 kHz within a time interval of less than 4\(\,\upmu \text {s}\). The second level, known as the high-level trigger, consists of a farm of processors running a version of the full event reconstruction software optimized for fast processing, and reduces the event rate to around 1 kHz before data storage.

3 Simulated events

The resonances associated with the considered phenomenologies are the bulk gravitons (\(\text{ G }{}{} _{\mathrm{bulk}}\)) generated for the bulk scenario [3,4,5] of the RS model of warped extra dimensions [1, 2], and the heavy new bosons (\({\text {W}}^{\prime }\) and \({\text {Z}}^{\prime }\)) that can be part of an heavy vector triplet [6] or can be mass degenerate as a vector singlet [32, 33].

The bulk graviton model is characterized by two free parameters: the mass of the first Kaluza–Klein (KK) excitation of a spin-2 boson (the KK bulk graviton), and the ratio \(\tilde{\kappa }=\kappa \sqrt{8\pi }/{M_\mathrm {Pl}} \), with \(\kappa \) being the unknown curvature scale of the extra dimension and \({M_\mathrm {Pl}}\) the Planck mass. A scenario with \(\tilde{\kappa }=0.5\) is considered in this analysis, as motivated in Ref. [34].

The HVT framework generically represents a large number of models predicting additional gauge bosons, such as the composite Higgs [35,36,37,38,39] and little Higgs [40, 41] models. The benchmark points are formulated in terms of a few parameters: two coefficients \(c_{\mathrm{F}}\) and \(c_{\text{ H }{}{} {}}\), that scale the couplings of the additional gauge bosons to fermions; to the Higgs boson and longitudinally polarized SM vector bosons, respectively, and \(g_{\mathrm{V}}\), representing the typical strength of the new vector boson interaction. For the analysis presented here, samples were simulated in the HVT model B, corresponding to \(g_{\mathrm{V}}=3\), \(c_{\text{ H }{}{} {}}=-0.98\), and \(c_{\mathrm{F}}=1.02\) [6]. For these parameters, the new resonances are narrow and have large branching fractions to vector boson pairs, while the fermionic couplings are suppressed.

All signals considered in the analysis satisfy the narrow-width approximation. The quoted results are therefore valid independent of the exact theoretical signal widths as long as the resonance widths remain smaller than the detector resolution. This makes our modelling of the detector effects on the signal shape independent of the actual model used for generating the events. All simulated samples are produced with a relative resonance width of 0.1%, in order to be within the validity of the narrow-width approximation. Monte Carlo (MC) simulated events of the bulk graviton and HVT signals are generated at leading-order (LO) in quantum chromodynamics (QCD) with MadGraph 5_amc@nlo versions 2.2.2 and 2.4.3 [42] and hadronization showering is simulated with pythia versions 8.205 and 8.230 [43], for 2016 and 2017 detector conditions, respectively. The NNPDF 3.0 [44] LO parton distribution functions (PDFs) are used together with the CUETP8M1 [45] and CP5 [46] underlying event tunes in pythia for 2016 and 2017 conditions, respectively.

Simulated samples of the SM background processes are used to optimize the analysis and create background templates, as described in Sect. 5. The QCD multijet production is simulated with four generator configurations: pythia only, the LO mode of MadGraph 5_amc@nlo  [47] matched and showered with pythia , powheg  [48,49,50,51] matched and showered with pythia , and herwig ++ 2.7.1 [52] with the CUETHS1 tune [45]. Top quark pair production is modelled at next-to-LO (NLO) with powheg  [53], showered with pythia . To calculate systematic uncertainties related to the vector boson tagging efficiency, two additional simulated samples of top quark production at LO are used: one generated with MadGraph 5_amc@nlo and interfaced with pythia , and the second one generated and showered with pythia . The production of \(\text{ W }{}{}\) +jets and \(\text{ Z }{}{}\) +jets (\(\text{ V }{}{}\) +jets) is simulated at LO with MadGraph 5_amc@nlo matched and showered with pythia . The same underlying event tune as for the signal samples is used for those of the background. Two corrections dependent on the transverse momentum (\(p_{\mathrm{T}}\) ) [54, 55] are applied to the \(\text{ V }{}{}\) +jets backgrounds to correct the \(p_{\mathrm{T}}\)-distribution of the vector bosons computed at LO in QCD to the one predicted at NLO in QCD, and to account for electroweak effects at high \(p_{\mathrm{T}}\). The NNPDF 3.1 [56] next-to-NLO (NNLO) PDFs are employed for simulated \(\text{ V }{}{}\) +jets events with the 2017 data taking conditions for both the 2016 and 2017 data analyses.

All samples are processed through a Geant4-based [57] simulation of the CMS detector. To simulate the effect of additional pp collisions within the same or adjacent bunch crossings (pileup), additional inelastic events are generated using pythia and superimposed on the hard-scattering events. The MC simulated events are weighted to reproduce the distribution of the number of reconstructed pileup vertices observed in the 2016 and 2017 data separately.

4 Reconstruction and selection of events

4.1 Jet reconstruction

Event reconstruction is based on the particle flow (PF) algorithm [58], which reconstructs and identifies individual particles with information from the various elements of the CMS detector. Jets are reconstructed from these particles, using the anti-\(k_{\mathrm {T}}\) jet clustering algorithm [59] with a distance parameter of \(R=0.8\) (AK8 jets) as implemented in the FastJet package [60]. In order to mitigate the effect of pileup, two different algorithms are used: for 2016 data and simulation, charged particles identified as originating from pileup vertices are excluded before jet clustering begins. For 2017, we take advantage of the pileup per particle identification (PUPPI) [61] algorithm. This method uses local shape information of charged pileup, event pileup properties, and tracking information in order to rescale the four-momentum of each neutral and charged PF candidate with a weight that describes the likelihood that each particle originates from a pileup interaction. All jets are further required to pass tight jet identification requirements [62]. Jets are corrected for nonlinearities in \(p_{\mathrm{T}}\) and \(\eta \) using jet energy corrections as described in Ref. [63]. Additionally, residual contributions from pileup are corrected using the approach outlined in Ref. [64].

Two variables are used to tag jets as potentially originating from vector boson decays to quarks for further event selection: the “groomed” mass of the jet obtained using a modified mass drop algorithm [65, 66] known as soft drop [67], and the N-subjettiness ratio \(\tau _{21} =\tau _{2}/\tau _{1}\) obtained with the N-subjettiness algorithm [68]. For both 2016 and 2017 data these observables are reconstructed from AK8 jets with PUPPI pileup mitigation applied, decreasing their dependence on pileup as shown in Ref. [62], while the overall jet four-momenta are calculated using the pileup mitigation algorithms as described above.

The groomed jet mass is calculated using the soft drop algorithm, with angular exponent \(\beta =1\), soft cutoff threshold \(z_{\mathrm{cut}}<0.1\), and characteristic radius \(R_{0}=0.8\) [67], which is applied to remove soft, wide-angle radiation from the jet. This is a generalization of the “modified mass” drop tagger algorithm  [65, 66], and the two are identical when the angular exponent \(\beta =0\). This algorithm is based on reclustering the constituents of the AK8 jets using the Cambridge–Aachen algorithm [69, 70]. The soft drop jet mass \(m_{\mathrm{jet}}\), calculated as the invariant mass of the four-momenta sum of the final remaining jet constituents, weighted according to the PUPPI algorithm, is utilized in the offline analysis and will be denoted as jet mass in the following. The mass is corrected for \(p_{\mathrm{T}}\)- and \(\eta \)-dependent nonuniformities due to detector effects, following the procedure described in Ref. [62].

This algorithm is used for the offline analysis, while the jet-trimming algorithm [71] is used at trigger level, see Sect. 4.2. The jet-trimming algorithm reclusters each AK8 jet starting from all its original constituents using the \(k_{\mathrm {T}}\) algorithm [72] to create subjets with a size parameter \(R_{\mathrm{subjet}}\) set to 0.2, discarding any subjet with \(p_{\mathrm{T}} ^{\mathrm{subjet}}/p_{\mathrm{T}} ^{\mathrm{jet}}<0.03\).

The N-subjettiness variable, \(\tau _{N}\), is defined as

$$\begin{aligned} \tau _N=\frac{1}{r_{0}}\sum _{k}p_{\mathrm{T},k} \min (\varDelta R_{1,k},\varDelta R_{2,k},\ldots ,\varDelta R_{N,k}), \end{aligned}$$
(1)

where the index k runs over the PF constituents of the jet, and the distances \(\varDelta R_{n,k}\) are calculated relative to the axis of the n-th subjet. The normalization factor \(r_{0}\) is calculated as \(r_{0}=R_{0}\sum _{k} p_{\mathrm{T},k} \), setting \(R_{0}\) to the distance parameter used in the clustering of the original jet. The variable \(\tau _{N}\) quantifies the compatibility of the jet clustering with the hypothesis that exactly N subjets are present, with small values of \(\tau _{N}\) indicating greater compatibility. The ratio between 2- and 1-subjettiness, \(\tau _{21} =\tau _{2}/\tau _{1}\), is found to be a powerful discriminant between jets originating from \(\text{ V }{}{}\) decays into quarks (V boson jets) and jets developed from prompt quarks and gluons (quark/gluon jets). Jets from \(\text{ W }{}{}\) or \(\text{ Z }{}{}\) decays in signal events are characterized by lower values of \(\tau _{21} \) relative to SM backgrounds. However, the \(\tau _{21} \) variable shows a dependence on the jet \(p_{\mathrm{T}}\)-scale as well as the jet mass. This particularly affects the monotonically falling behaviour of the nonresonant background distributions. Since this search probes a large range of jet masses and dijet invariant masses (\(m_{\mathrm{jj}} \)), we decorrelate \(\tau _{21} \) from the jet \(p_{\mathrm{T}}\)-scale and jet mass following the “designed decorrelated taggers (DDT)” methodology presented in Ref. [73]. We thereby reduce the \(\tau _{21}\) profile dependence on \(\rho '=\ln (m_{\mathrm{jet}} ^2/(\mathrm {p}_T\mu ))\), where \(\mu =1\,\text {Ge}\text {V} \). This leads to the following definition of \(\tau _{21}^{\mathrm{DDT}}\):

$$\begin{aligned} \tau _{21}^{\mathrm{DDT}} =\tau _{21}- M\,\rho ', \end{aligned}$$
(2)

where M is the extracted slope from a fit to the \(\tau _{21}\) profile versus \(\rho '\) in QCD multijet events simulated with pythia after applying the full analysis selections. It is evaluated to be \(M=-0.080\). The \(\tau _{21}\) (upper) and \(\tau _{21}^{\mathrm{DDT}}\) (lower) profile dependencies on \(\rho '\) are shown in Fig. 1. We observe a small residual difference between intervals of \(p_{\mathrm{T}}\), but this has a negligible impact on the analysis.

Fig. 1
figure 1

The \(\tau _{21}\) (upper) and \(\tau _{21}^{\mathrm{DDT}}\) (lower) profile dependencies on \(\rho '=\ln (m_{\mathrm{jet}} ^2/(p_{\mathrm{T}} \mu ))\) examined in QCD multijet events simulated with pythia . A fit to the linear part of the spectrum for \(p_{\mathrm{T}} >200\,\text {Ge}\text {V} \) yields the slope \(M=-0.080\), which is used to define the mass- and \(p_{\mathrm{T}}\)-decorrelated variable \(\tau _{21}^{\mathrm{DDT}} =\tau _{21}-M\,\rho '\)

We observe a significant gain in analysis sensitivity when using \(\tau _{21}^{\mathrm{DDT}}\). Since this variable is a function of both N-subjettiness and the ratio of jet mass and transverse momentum, it leads to a larger separation between signal and background as shown in the comparison of \(\tau _{21}^{\mathrm{DDT}}\) and \(\tau _{21}\) in Fig. 2 (upper). Furthermore, using the \(\tau _{21}^{\mathrm{DDT}}\) variable reduces the dependency of \(m_{\mathrm{jet}}\) on \(m_{\mathrm{jj}}\), leading to smoothly falling distributions in the jet mass as shown in Fig. 2 (lower).

Fig. 2
figure 2

Distribution of the N-subjettiness discriminants (\(\tau _{21}\) and \(\tau _{21}^{\mathrm{DDT}}\)) for \(\text{ W }{}{}\)-jets and quark/gluon jets from QCD multijet events (upper). Distributions in the jet mass of QCD multijet events for four \(m_{\mathrm{jj}}\) bins in the range 1126–5500\(\,\text {Ge}\text {V}\) after a cut on \(\tau _{21}\) and \(\tau _{21}^{\mathrm{DDT}}\) corresponding to the same mistag rate of about 2% (lower). For both discriminants, darker colours correspond to higher \(m_{\mathrm{jj}}\) ranges. The distributions are arbitrarily scaled for better readability. The analysis selections applied to derive these distributions are specified in the plots. For this analysis the working point of \(\tau _{21}^{\mathrm{DDT}} {\le }0.43\) is chosen

4.2 Trigger and preliminary offline selection

Events are selected online with a variety of different jet triggers based on the highest jet \(p_{\mathrm{T}}\) or the \(p_{\mathrm{T}}\) sum of all jets in the event (\(H_{\mathrm{T}}\) ). For some of these triggers additional requirements on the trimmed mass are applied in order to be able to lower the \(p_{\mathrm{T}}\) and \(H_{\mathrm{T}}\) thresholds. For example, for 2017 data taking, requiring the trimmed jet mass of the leading-\(p_{\mathrm{T}}\) jet to be above 30\(\,\text {Ge}\text {V}\) allows the lowering of the \(p_{\mathrm{T}}\) threshold from 500 to 360\(\,\text {Ge}\text {V}\) while maintaining a similar rate. In the case of the \(H_{\mathrm{T}}\) -triggers, the threshold can be lowered from 1050 to 750–800\(\,\text {Ge}\text {V}\) when requiring a trimmed jet mass greater 50\(\,\text {Ge}\text {V}\). The \(H_{\mathrm{T}}\) -triggers utilize a standard jet collection of anti-\(k_{\mathrm {T}}\) jets with a distance parameter \(R = 0.4\), while the triggers based on the trimmed jet mass operate on AK8 jets. The triggers used for the 2017 data set are conceptually similar to those used in 2016, and correspond to those used in Ref. [29]. The 2017 trigger thresholds were slightly greater than those in 2016 in order to maintain the same trigger rate despite a higher instantaneous luminosity.

The trigger efficiency as a function of the dijet invariant mass is measured in an orthogonal single muon data set, shown in Fig. 3, using a combination of all triggers (upper), and as a function of the jet mass for the triggers exploiting the trimmed jet mass (lower). For the trimmed jet mass triggers, the efficiency plateau as a function of the jet mass does not reach 100% for the full 2017 data set (full yellow circles), since these triggers were not used for the first 4.8\(\,\text {fb}^{-1}\) of data recorded. The trigger efficiency excluding this period is shown with open yellow circles. The combination of all triggers is \(>99\%\) efficient above dijet invariant masses of 990 and 1126\(\,\text {Ge}\text {V}\) for the full 2016 and 2017 data sets, respectively. For simplicity, the subsequent analysis requires the dijet invariant mass to be above 1126\(\,\text {Ge}\text {V}\) for both data sets. Given the \(m_{\mathrm{jj}}\)resolution of about 10%, the lowest resonance mass that is accepted with high efficiency by the analysis is 1.2 \(\text {Te}\text {V}\).

Fig. 3
figure 3

The trigger efficiency as a function of the dijet invariant mass for a combination of all triggers used in this analysis (upper) and the event efficiency for either of the selected jets to pass triggers requiring an online trimmed mass of at least 30\(\,\text {Ge}\text {V}\) as a function of the jet mass (lower). The solid yellow circles correspond to the trigger efficiencies for the full 2017 data set and do not reach 100% efficiency because the jet mass based triggers were unavailable for a period at the beginning of data taking (corresponding to 4.8\(\,\text {fb}^{-1}\)). The open yellow circles are the corresponding efficiencies excluding this period. The uncertainties shown are statistical only

All events are required to have at least one primary vertex reconstructed within a 24\(\,\text {cm}\) window along the beam axis, with a transverse distance from the average \(\text{ p }{}{}\) \(\text{ p }{}{}\) interaction region of less than 2\(\,\text {cm}\)  [74]. The reconstructed vertex with the largest value of summed physics-object \(p_{\mathrm{T}} ^2\) is taken to be the primary \(\text{ p }{}{}\) \(\text{ p }{}{}\) interaction vertex. The physics objects are the jets, clustered using the jet finding algorithm [59, 60] with the tracks assigned to the vertex as inputs, and the associated missing transverse momentum, taken as the negative vector sum of the \(p_{\mathrm{T}}\) of those jets.

4.3 Event selection

Fig. 4
figure 4

Jet mass (upper left) and \(\tau _{21}^{\mathrm{DDT}}\) (upper right) distributions for selected jets (one random jet per event), and dijet invariant mass distribution (lower), for events with a jet mass between 55 and 215\(\,\text {Ge}\text {V}\). For the QCD multijet simulation, several alternative predictions are shown, scaled to the data minus the other background processes, which are scaled to their SM expectation as described in the text. The different signal distributions are scaled to be visible. No selection on \(\tau _{21}^{\mathrm{DDT}}\) is applied. The ratio plots show the fraction of data over QCD multijet simulation for pythia 8 (black markers), herwig ++ (dotted line), and MadGraph +pythia 8 (dashed line)

Events are selected by requiring at least two jets with \(p_{\mathrm{T}} >200\,\text {Ge}\text {V} \) and \(|\eta |<2.5\). The two jets with the highest \(p_{\mathrm{T}}\) in the event are selected as potential vector boson candidates and are required to have a separation of \(|\varDelta \eta |<1.3\) in order to reduce the QCD multijet background. In addition to the requirement that the two jets invariant mass \(m_{\mathrm{jj}}>1126\,\text {Ge}\text {V} \), based on the trigger selection discussed above, it is further required that \(m_{\mathrm{jj}}<5500\,\text {Ge}\text {V} \). The upper cut on mjj is well above the highest dijet mass event observed in data. To simplify the modelling of the 3D shapes in the \(m_{\mathrm{jj}} \)\(m_{\mathrm{jet1}} \)\(m_{\mathrm{jet2}} \) space, the two jets are labelled at random so that the mass distributions of the first and second selected jet, \(m_{\mathrm{jet1}}\) and \(m_{\mathrm{jet2}}\), have the same shape. Jets originating from the misreconstruction of a high momentum lepton are rejected by requiring an angular separation \(\varDelta R>0.8\) to muons (electrons) with \(p_{\mathrm{T}} \) greater than 20 (35)\(\,\text {Ge}\text {V}\) and satisfying identification criteria optimized for high-momentum leptons [75, 76]. To reduce the QCD multijet background, we require the jet mass to be between 55 and 215\(\,\text {Ge}\text {V}\). The selected events are further grouped into two categories according to their likelihood to originate from a boson decay into quarks, as quantified by \(\tau _{21}^{\mathrm{DDT}}\) . The jet mass, \(\tau _{21}^{\mathrm{DDT}}\), and dijet invariant mass distributions for data and simulation are shown in Fig. 4.

In the high-purity (HPHP) category, both jets are required to have \(0<\tau _{21}^{\mathrm{DDT}} \le 0.43\), while in the low-purity (HPLP) category only one of the jets needs to fulfill this requirement, and the other must satisfy \(0.43<\tau _{21}^{\mathrm{DDT}} \le 0.79\). These conditions yield the highest expected signal significance over the whole mass range, while at the same time selecting at least 95% of the signal. The addition of the HPLP category improves the expected cross section upper limit by around 20% at high dijet invariant mass where the background is low. Finally, a loose requirement of \(\rho =\ln (m_{\mathrm{jet}} ^2/p_{\mathrm{T}} ^2) < -1.8\) is applied in order to veto events in which the jet mass is high, but the jet \(p_{\mathrm{T}}\) is low. In these cases the cone size of \(\varDelta R=0.8\) is too small to contain the full jet, affecting both the jet mass resolution and the \(\tau _{21}^{\mathrm{DDT}}\) tagging efficiency, which is not well modelled in simulation. This selection has a negligible effect on the signal, which typically has jets with masses around the \(\text{ W }{}{}\) or \(\text{ Z }{}{}\) boson mass and high \(p_{\mathrm{T}}\) .

4.4 Substructure variable corrections and validation

Figure 4 shows a notable deviation in the shape of the \(\tau _{21}^{\mathrm{DDT}}\) distribution between data and simulation. Such mismodelling introduces a bias in the jet tagging efficiency for the signal, and as a consequence in the measured signal rate. We therefore compute scale factors to correct the signal efficiency accordingly. For the background jets, this mismodelling requires no further correction, because of the data-driven approach adopted in this analysis, where the background shape and normalization are fitted to data with large pre-fit uncertainties as described in the following sections.

The \(\text{ W }{}{}\) boson tagging scale factors and jet mass scale and resolution uncertainties are estimated from data by isolating a control sample of merged \(\text{ W }{}{}\) bosons in a high-\(p_{\mathrm{T}}\)\(\hbox {t}\bar{\hbox {t}}\) sample. This is done by performing a simultaneous fit to the jet mass distributions for the two ranges of \(\tau _{21}^{\mathrm{DDT}}\), as detailed in Ref. [62]. To extract the efficiency from a clean sample of merged W bosons, the \(\hbox {t}\bar{\hbox {t}}\) sample is split into two components, depending on whether the quarks from the W boson decay at truth level are within \(\delta R = 0.8\) or not, i.e. on whether the hadronic boson decay is merged into a single jet or not. Only the merged component is considered in the efficiency calculation and the mass scale and resolution extraction.

The efficiencies and scale factors obtained are listed in Table 1 for 2016 and 2017 data, with the corresponding fits shown in Fig. 5. The \(\text{ W }{}{}\) boson tagging efficiency in the selected \(\hbox {t}\bar{\hbox {t}}\) events of around 7% is relatively low compared to the efficiency in signal events, since these events are dominated by \(\text{ W }{}{}\) boson jets with a \(p_{\mathrm{T}}\) of around 200\(\,\text {Ge}\text {V}\), just at the threshold where the decay products of the \(\text{ W }{}{}\) boson merge into a single jet. The signal jets, however, mostly have a \(p_{\mathrm{T}}\) above 600\(\,\text {Ge}\text {V}\), and a tagging efficiency around 35%. The signal efficiency for \(\tau _{21}^{\mathrm{DDT}}\) increases with the jet \(p_{\mathrm{T}}\) , whereas the background efficiency is constant, as shown in Ref. [62]. Two systematic uncertainties in the scale factors are added: one due to differences in MC generation and modelling of the parton shower and one due to NNLO corrections. The former is evaluated by comparing the resulting scale factors when using \(\hbox {t}\bar{\hbox {t}}\) simulation produced with different generators. The latter is evaluated by comparing the extracted efficiencies with and without reweighting according to the top quark \(p_{\mathrm{T}}\), where the reweighting is derived from data in order to better describe the observed \(p_{\mathrm{T}}\) distribution in \(\hbox {t}\bar{\hbox {t}}\) data [77]. The jet mass scale and resolution are estimated in the same fits and also listed in Table 1. The difference in jet mass scale between data and simulation is around 2%, and the jet mass resolution difference is roughly 8%. These are used to scale and smear the jet mass in simulation, and their uncertainties are additionally inserted as systematic uncertainties in the final fit.

Fig. 5
figure 5

The jet mass distribution for events that pass (left) and fail (right) the \(\tau _{21}^{\mathrm{DDT}} \le 0.43\) selection in the \(\hbox {t}\bar{\hbox {t}}\) control sample. The results of the separate fits to data and to simulation are shown by the dash-dotted blue and solid red lines, respectively. The background components of the fits are shown as dashed and dash-dotted lines. The fit to 2016 data is shown in the upper panels and the fit to 2017 data in the lower panels. The associated uncertainties are shown in Table 1 and discussed further in Sect. 4.4

Table 1 The \(\text{ W }{}{}\) jet mass peak position (m) and resolution (\(\sigma \)), and the \(\text{ W }{}{}\) -tagging efficiencies, as extracted from top quark enriched data and from simulation, together with the corresponding data-to-simulation scale factors. The uncertainties in the scale factors include systematic uncertainties estimated as described in Ref. [62]

5 The multi-dimensional fit

The background estimation technique used in previous versions of this analysis [7, 10, 29] relied on a one-dimensional (1D) fit of the dijet invariant mass after a tight jet mass selection (65–105\(\,\text {Ge}\text {V}\)) has been applied. In the analysis presented here, we take advantage of the fact that the signal peaks in three observables (the jet masses \(m_{\mathrm{jet1}} \) and \(m_{\mathrm{jet2}} \), and the dijet invariant mass \(m_{\mathrm{jj}} \)), and attempt to extract the signal by searching for peaks in the multi-dimensional \(m_{\mathrm{jj}} \)\(m_{\mathrm{jet1}} \)\(m_{\mathrm{jet2}} \) space. This method permits searches for generic resonances, decaying to two SM or non-SM bosons, anywhere in the jet mass and dijet invariant mass spectra in the future. Additionally, tight jet mass cuts as used in previous diboson resonance searches are no longer needed, as we fit the full jet mass line shape to extract the signal. Since such a cut around the vector boson mass leads to about 20% inefficiency for the \(\text{ W }{}{}\) and \(\text{ Z }{}{}\) boson signals, including all the events that would fall outside the mass window reduces the statistical uncertainties in the fitting procedure. Furthermore, the background \(m_{\mathrm{jj}}\) shape is better constrained at high dijet invariant masses than it is in the previous method.

Fitting the jet mass and resonance mass together also allows us to add nuisance parameters that simultaneously affect the jet masses and the resonance mass, accounting for their correlation. We build a three dimensional background model starting from simulation. As the number of simulated events is small, a forward-folding kernel approach is used to ensure a full and smooth model, as described in Sect. 5.2.1. Further, to account for discrepancies in the QCD multijet background simulation and data, we allow the background model to adapt to the data using physically motivated shape variations.

The random jet labelling adapted in the analysis results in essentially the same jet mass distributions for jet-1 and jet-2 in the modelling and removes any correlations between the two jet masses. Thus only the distribution for one of the jet masses are shown in the following figures.

5.1 Signal modelling

For each mass point \(m_{\mathrm{X}}\) and each purity category, the signal yield per pb of cross section is calculated as the integral of the histogram produced from the parameterization. The total signal yield for events passing all analysis selections divided by the number of generated events as a function of \(m_{\mathrm{X}}\) is shown in Fig. 6.

Fig. 6
figure 6

Total signal efficiency as a function of \(m_{\mathrm{X}}\) after all selections are applied, for signal models with a \({\text {Z}}^{\prime }\) decaying to \(\text{ W }{}{}\) \(\text{ W }{}{}\) , \(\text{ G }{}{} _{\mathrm{bulk}} \) decaying to \(\text{ W }{}{}\) \(\text{ W }{}{}\) , \({\text {W}}^{\prime } \) decaying to \(\text{ W }{}{}\) \(\text{ Z }{}{}\) , and \(\text{ G }{}{} _{\mathrm{bulk}} \) decaying to \(\text{ Z }{}{}\) \(\text{ Z }{}{}\) . The denominator is the number of generated events. The solid and dashed lines show the signal efficiencies for the HPHP and HPLP categories, respectively. The decrease in efficiency between 5.0 and 5.5 \(\text {Te}\text {V}\) is due to the requirement \(m_{\mathrm{jj}} <5500\,\text {Ge}\text {V} \)

The signal shape in three dimensions is defined as a product of the shape of the resonance mass and the jet masses:

$$\begin{aligned}&P_{\mathrm{sig}}(m_{\mathrm{jj}},m_{\mathrm{jet1}},m_{\mathrm{jet2}} |\overline{\theta }^{\mathrm{s}}(m_{\mathrm{X}}) ) = P_{\text{ V }{}{} {}\text{ V }{}{} {}}(m_{\mathrm{jj}} |\overline{\theta }_1^{\mathrm{s}}(m_{\mathrm{X}}))\\&\quad \times P_{\mathrm{j1}}(m_{\mathrm{jet1}} |\overline{\theta }_2^{\mathrm{s}}(m_{\mathrm{X}})) \, P_{\mathrm{j2}}(m_{\mathrm{jet2}} |\overline{\theta }_3^{\mathrm{s}}(m_{\mathrm{X}})). \end{aligned}$$

The shapes for \(m_{\mathrm{jj}}\), \(m_{\mathrm{jet1}}\), and \(m_{\mathrm{jet2}}\) are parameterized independently as a function of the hypothesized mass (\(m_{\mathrm{X}}\) ) of a new particle and a set of general probability density function (pdf) parameters \(\overline{\theta }^{\mathrm{s}}=(\overline{\theta }_1^{\mathrm{s}}, \overline{\theta }_2^{\mathrm{s}}, \overline{\theta }_3^{\mathrm{s}})\) that depend on \(m_{\mathrm{X}}\). The parameters \(\overline{\theta }^{\mathrm{s}}\) denote for instance the mean and width of the analytic function chosen to model the signal distributions. The \(m_{\mathrm{jet}}\) and \(m_{\mathrm{jj}}\) distributions can be treated as uncorrelated since correlations are found to be negligible for the signal. The signal is parameterized by fitting the simulated resonance mass and jet mass line shapes for each \(m_{\mathrm{X}}\), extracting the quantities, and then interpolating these to intermediate values of the resonance mass. \(P_{\mathrm{j}1}\) and \(P_{\mathrm{j}2}\) are fitted and parameterized separately from each other using different sets of \(\overline{\theta }^{\mathrm{s}}\), although they are effectively identical because of the random jet labelling. For the parameterization of the resonance mass \(m_{\mathrm{jj}}\) and the \(m_{\mathrm{jet}}\) masses, double-sided Crystal Ball (dCB) functions [78] are used for each \(m_{\mathrm{X}}\). Each parameter of the dCB is interpolated between different resonance masses using polynomials of a degree sufficient to ensure a smooth shape interpolation for all resonance masses. The resulting signal shapes for all signal models are shown in Fig. 7 for the dijet invariant mass (upper) and the mass of jet-1 in the HPHP category (lower). Because of the random jet labelling the distribution for jet-2 is effectively identical to that shown for jet-1. The jet mass scale and resolution as a function of the dijet invariant mass are extracted from the mean and width of the dCB function. The mass scale and resolution are shown in Fig. 8 after the full HPHP (HPLP) analysis selections have been applied. The jet mass resolution increases about 3% from the lowest to the highest resonance mass, while its scale is found to be stable. The mean of the dijet invariant mass distributions is consistent with the mass of the resonance \(m_{\mathrm{X}}\), as seen in Figure 7 upper.

Fig. 7
figure 7

The final \(m_{\mathrm{jj}}\) (upper) and \(m_{\mathrm{jet1}}\) (lower) signal shapes extracted from the parameterization of the dCB function. The same \(m_{\mathrm{jj}}\) shapes are used for both purity categories. The jet mass distributions are shown for a range of resonance masses between 1.2 and 5.2 \(\text {Te}\text {V}\) for one of the two jets in the events in the HPHP category. Because the jets are labelled randomly, the jet mass distributions for the second jet are essentially the same as the one shown. The distributions for a \(\text{ G }{}{} _{\mathrm{bulk}} \) decaying to \(\text{ W }{}{}\) \(\text{ W }{}{}\) have the same shapes as those for the \({\text {Z}}^{\prime }\) signal and are therefore not visible

Fig. 8
figure 8

The mass scale (upper) and resolution (lower) of the jet as a function of \(m_{\mathrm{X}}\), obtained from the mean and width of the dCB function used to fit the jet mass spectrum. The HPHP (solid lines) and HPLP (dotted lines) categories are shown for different signal models. The distributions are only shown for one of the two jets in the event, since the distributions for the second jet are essentially the same

5.2 Background modelling

5.2.1 Nonresonant background

As mentioned above, previous versions of this analysis estimate the QCD multijet background by a parametric fit to the data in the \(m_{\mathrm{jj}} \) signal region. The fit is well-constrained by highly populated bins with small statistical uncertainties at low \(m_{\mathrm{jj}} \), but is less constrained for high values of \(m_{\mathrm{jj}} \). This method allows the incorporation of additional information in the fit by modelling the correlations between the jet mass and the dijet invariant mass for SM background processes, which were not explicitly studied in the past. It is important to note that the correlations between \(m_{\mathrm{jet}} \) and \(m_{\mathrm{jj}} \) have to be modelled for the QCD multijet background, as opposed to the signals negligible correlations due to its localization in the three-dimensional space. In this analysis, we build a three dimensional background model starting from simulation. Since the size of the simulated samples is limited, we start from particle-level information and use a “forward-folding” kernel approach that is similar to the technique presented in Ref. [79] and used in Ref. [11]. Finally, we incorporate sufficient nuisance parameters into the fit to account for any discrepancies between data and simulation. In order to model the QCD multijet background in the 3D \(m_{\mathrm{jj}}\)\(m_{\mathrm{jet1}}\)\(m_{\mathrm{jet2}}\) hyperplane, we use the following conditional product:

$$\begin{aligned}&P_{\mathrm{QCD}}(m_{\mathrm{jj}},m_{\mathrm{jet1}},m_{\mathrm{jet2}} |\overline{\theta }^{\mathrm{QCD}}) = P_{\text{ V }{}{} {}\text{ V }{}{} {}}(m_{\mathrm{jj}} |\overline{\theta }_1^{\mathrm{QCD}})\\&\quad \times P_{\mathrm{cond,1}}(m_{\mathrm{jet1}} |m_{\mathrm{jj}},\overline{\theta }_2^{\mathrm{QCD}}) \, P_{\mathrm{cond,2}}(m_{\mathrm{jet2}} |m_{\mathrm{jj}},\overline{\theta }_3^{\mathrm{QCD}}). \end{aligned}$$

Since the jet mass is correlated with the jet \(p_{\mathrm{T}}\) for the QCD multijet background, its shape is required to be modelled conditionally as a function of \(m_{\mathrm{jj}} \) for both jets. Two two-dimensional (2D) templates (denoted as \(P_{\mathrm{cond,1}}\) and \(P_{\mathrm{cond,2}}\)) are modelled for the two jet masses separately, containing different jet mass shapes in bins of \(m_{\mathrm{jj}} \). The \(m_{\mathrm{jj}} \) distribution is computed as a 1D pdf. The parameter sets denoted by \(\overline{\theta }^{\mathrm{QCD}}\) represent the nuisance parameters in each pdf.

To build the 2D conditional templates, \(P_{\mathrm{cond,1}}\) and \(P_{\mathrm{cond,2}}\), each available particle-level event is smoothed with a 2D Gaussian kernel, where each 2D kernel links the particle-level event quantities to the reconstruction level. Thus each simulated event contributes a smoothed Gaussian shape to the total conditional pdf. The Gaussian kernel depends on the dijet invariant mass scale and resolution, as well as the jet mass scale and resolution. The \(m_{\mathrm{jet}} \) and \(m_{\mathrm{jj}} \) scale and resolution are extracted from a Gaussian fit to either \(m_{\mathrm{jet}} (\mathrm {reco})/m_{\mathrm{jet}} (\mathrm {gen})\) or \(m_{\mathrm{jj}} (\mathrm {reco})/m_{\mathrm{jj}} (\mathrm {gen})\), in bins of particle-level jet \(p_{\mathrm{T}}\).

The mass scale and resolution are then used to populate the conditional 2D histogram. Each generated event i is smeared with a 2D Gaussian kernel,

$$\begin{aligned} k(m_{\mathrm{jet}},m_{\mathrm{jj}})= & {} \frac{w_i}{2\pi r_{m_{\mathrm{jj}},i} r_{m_{\mathrm{jet}},i}}\\&\times \exp \left[ -\frac{1}{2} \left( \frac{m_{\mathrm{jj}}-s_{m_{\mathrm{jj}},i}}{r_{m_{\mathrm{jj}},i}} \right) ^2-\frac{1}{2} \left( \frac{m_{\mathrm{jet}}-s_{m_{\mathrm{jet}},i}}{r_{m_{\mathrm{jet}},i}} \right) ^2 \right] , \end{aligned}$$

where \(s_{i}\) and \(r_{i}\) are the scale and the resolution derived in the previous step, and \(w_i\) is a product of event weights accounting for the normalization effects such as the individual sample production cross sections. In this way, the jet mass in generated events is scaled and smeared according to the evaluated scale and resolution, and a 2D histogram is filled with smooth Gaussian shapes. According to this procedure the jet mass (\(m_{\mathrm{jj}}\)) resolution is about 7–10% (3–7%) of the mass of the generated jet, depending on its \(p_{\mathrm{T}}\). This procedure is performed separately for \(m_{\mathrm{jet1}}\) and \(m_{\mathrm{jet2}}\) however the two resulting templates \(P_{\mathrm{cond,1}}\) and \(P_{\mathrm{cond,2}}\) are essentially the same because of the random jet labels. Finally, we interpolate the 2D histogram in order to have valid values of the pdf in all \(m_{\mathrm{jj}} \) bins. Starting from the histogram, coarsely binned in \(m_{\mathrm{jj}} \), for each \(m_{\mathrm{jet}} \) bin a spline is fitted over all \(m_{\mathrm{jj}} \) bins. The spline is then used to interpolate values of the histogram for all final \(m_{\mathrm{jj}} \) bins, resulting in a 2D histogram with the desired binning.

To build the 1D template for the dijet invariant mass, \(P_{\text{ V }{}{} {}\text{ V }{}{} {}}\), a 1D Gaussian kernel is constructed starting from particle-level quantities where, for each MC event, a Gaussian probability distribution, rather than a single point representing the mean, contributes to the total 1D pdf using the same procedure as for the 2D templates.

Because of the low number of events in the HPHP category, the 3D kernel derived in the HPLP category, which has a similar shape, is used to model the HPHP background. This is done by adapting the templates derived in the HPLP category to the HPHP category in the QCD multijet simulation through a fit, and then by using the corresponding post-fit distribution as the nominal template for the HPHP category. The free parameters in the fit are the alternate shapes proportional to \(m_{\mathrm{jj}}\), \(m_{\mathrm{jet}}\), 1/\(m_{\mathrm{jj}}\), and 1/\(m_{\mathrm{jet}}\), as listed in Sect. 6. The projections on the three different axes of the final 3D pdf, in bins of \(m_{\mathrm{jj}} \) or \(m_{\mathrm{jet}} \), are shown in Fig. 9, compared to the spectra obtained using bare QCD multijet simulation events. Good agreement is observed, and any residual discrepancies are covered by the systematic shape uncertainties described in Sect. 6 and also shown in Fig. 9. Repeating the template building process and performing fits to a control region in data where both jets fail the high-purity requirement confirms the validity of this approach. In addition, the adaptability of the method was further confirmed by fitting a QCD multijet background generated at NLO with powheg .

5.2.2 Resonant background

The resonant background is defined as all SM processes containing at least one jet originating from a genuine \(\text{ W }{}{}\) or \(\text{ Z }{}{}\) boson decay. It is dominated by \(\text{ V }{}{}\) +jets events, with a minor contribution from \(\hbox {t}\bar{\hbox {t}}\) production and an inconsequential contribution from SM \(\text{ V }{}{} {}\text{ V }{}{} {}\) production, that is absorbed into the \(\text{ V }{}{}\) +jets modelling. As the labelling of each jet is arbitrary, each jet mass distribution contains two contributions: a resonant part consisting of genuine vector-boson jets, peaking around the \(\text{ W }{}{}\) or \(\text{ Z }{}{}\) boson mass; and a nonresonant part, composed of mistagged jets originating from a prompt quark or a gluon. These two contributions are modelled separately for each jet mass dimension. A 3D pdf for the resonant backgrounds, \(P_{\text{ V }{}{} {}+{\mathrm{jets}}}\), is built as a product of three 1D pdfs as follows:

$$\begin{aligned}&P_{\text{ V }{}{} {}+{\mathrm{jets}}} \left( m_{\mathrm{jet1}}, m_{\mathrm{jet2}}, m_{\mathrm{jj}} |\overline{\theta } \right) = 0.5 \, (P_{\text{ V }{}{} {}\text{ V }{}{} {}} ( m_{\mathrm{jj}} |\overline{\theta }_1 )\\&\quad \times P_{\mathrm{res}} ( m_{\mathrm{jet1}} |\overline{\theta }_2 ) \, P_{\mathrm{nonres}} (m_{\mathrm{jet2}} |\overline{\theta }_3 )) + 0.5 \, ( P_{\text{ V }{}{} {}\text{ V }{}{} {}} ( m_{\mathrm{jj}} |\overline{\theta }_1 )\\&\quad \times P_{\mathrm{res}} ( m_{\mathrm{jet2}} |\overline{\theta }_2 ) \, P_{\mathrm{nonres}} (m_{\mathrm{jet1}} |\overline{\theta }_3)). \end{aligned}$$

The resonant mass shape \(P_{\mathrm{res}}\) is derived by fitting a dCB function to the simulated jet mass spectrum, performed separately for \(m_{\mathrm{jet1}}\) and \(m_{\mathrm{jet2}}\). The resonant events are separated from the nonresonant ones when building the pdfs by requiring that there is a generated boson in a cone of \(\varDelta R=0.8\) around the reconstructed merged jet. The nonresonant component of the jet mass shape is fitted separately with a Gaussian function. The contributions of \(\text{ W }{}{}\) +jets and \(\hbox {t}\bar{\hbox {t}}\) production are considered as one combined background shape, because both have a resonant peak around the \(\text{ W }{}{}\) -boson mass, while the \(\text{ Z }{}{}\) +jets background contribution is considered separately. The top mass peak does not need to be modelled since the overall contribution of \(\hbox {t}\bar{\hbox {t}}\) events is less than 2%. The nonresonant dijet invariant mass shape of the \(\text{ V }{}{}\) +jets backgrounds, \(P_{\text{ V }{}{} {}\text{ V }{}{} {}}\), is modelled with a one dimensional kernel, in the same way as the dijet invariant mass shape of the QCD multijet background.

6 Systematic uncertainties

6.1 Systematic uncertainties in the background estimation

Uncertainties in the QCD multijet background shape are included in the fit using alternative pdfs derived with the template-building method described in Sect. 5.2.1. We define five nuisance parameters that vary the shape, each of the parameters corresponding to an upward and a downward variation of alternative shapes that simultaneously affect all three dimensions. The first nuisance parameter accounts for a variation of the underlying \(p_{\mathrm{T}}\) spectrum, and the two corresponding mirrored templates are obtained by applying up and down variations of the expected yields to each bin along the two jet masses and \(m_{\mathrm{jj}}\) by a quantity proportional to \(m_{\mathrm{jet}}\) and \(m_{\mathrm{jj}}\). The second nuisance parameter is a variation of the mass scale, and is taken into account through two mirrored alternative shapes obtained by applying up and down variations of each bin content along the two jet masses and \(m_{\mathrm{jj}}\) by a quantity proportional to 1/\(m_{\mathrm{jet}}\) and 1/\(m_{\mathrm{jj}}\). Two additional alternative shapes that simultaneously affect resonance mass and jet groomed mass are also added in order to take into account differences in MC generation and modelling of the parton shower. These alternative templates are derived using the herwig ++ and MadGraph +pythia 8 QCD multijet simulation. This allows us to include all known background variations into the fit. For events with a large \(m_{\mathrm{jet}}\) (\(>175\,\text {Ge}\text {V} \)) and low \(m_{\mathrm{jj}}\) (\(<1200\,\text {Ge}\text {V} \)), there is an expected turn-on due to the trigger thresholds. Therefore, an additional shape uncertainty parameterizing any discrepancy between the 3D template and the QCD multijet simulation is added to the fit. Note that this shape uncertainty only affects this particular region, which is far from where a diboson signal, as relevant for this analysis, is expected. The nuisance parameters associated with these alternative shapes are constrained using Gaussian pdfs in the fit, with the pre-fit values chosen in order to cover any differences between data and simulation observed in the control regions. The alternative shapes described above are shown in Fig. 9.

A similar procedure is used for the \(\text{ V }{}{}\) +jets background, adding two alternative shapes to the \(m_{\mathrm{jj}} \) templates derived by a variation proportional to \(m_{\mathrm{jj}} \) and \(1/m_{\mathrm{jj}} \). The resonant jet mass shapes for this background are subject to the same uncertainties as the signal. The normalizations of the \(\text{ V }{}{}\) +jets and QCD background are obtained directly from simulation and are allowed to vary within 20 and 50%, respectively. The same nuisance parameters are used for the fit to 2016 and 2017 data, which reduces the fit complexity while not impacting the result of the fit.

6.2 Systematic uncertainties in the signal prediction

The dominant uncertainty in the signal selection efficiency arises from uncertainties in the boson tagging efficiency. The effect of this uncertainty is evaluated per jet and assumed to be fully correlated between both jets in the event. The \(\text{ W }{}{}\) boson tagging efficiency scale factor is fully anticorrelated between the HPHP and HPLP categories (3–10%), and fully correlated between signal and \(\text{ V }{}{}\) +jets backgrounds. The \(p_{\mathrm{T}}\)-dependence uncertainty in the scale factor arises from the extrapolation to higher \(p_{\mathrm{T}}\) ’s of the \(\text{ W }{}{}\) boson tagging efficiency scale factors, which are measured in \(\hbox {t}\bar{\hbox {t}}\) events where the jet has a \(p_{\mathrm{T}}\) around 200\(\,\text {Ge}\text {V}\). This uncertainty is estimated in signal simulation, and is based on the difference in tagging efficiency between the samples matched and showered either with pythia or with herwig ++, as a function of \(p_{\mathrm{T}}\), relative to the difference at 200\(\,\text {Ge}\text {V}\). This is considered as correlated between the \(\tau _{21}^{\mathrm{DDT}} \) categories, and is given as \(6\,(7) \% \ln (p_{\mathrm{T}}/300\,(\text {Ge}\text {V} {}))\) for the HPHP (HPLP) categories. The shape uncertainties in the jet masses are considered fully correlated between signal and \(\text{ V }{}{}\) +jets backgrounds, allowing the data to constrain these parameters. These affect the mean and the width of the signal and \(\text{ V }{}{}\) +jets pdfs. All uncertainties entering the fit are listed in Table 2.

Table 2 Summary of the systematic uncertainties and their impact the affected quantities. Numbers in parentheses correspond to uncertainties for the 2016 analysis if these differ from those for 2017. Dashes indicate shape variations that cannot be described by a single parameter, and are discussed in the text
Fig. 9
figure 9

Nominal QCD multijet simulation using pythia 8 (data points) and three-dimensional pdfs derived using a forward-folding kernel approach (black solid line), shown together with the five alternate shapes that are added to the multi-dimensional fit as shape nuisance parameters. The shapes for the high-purity (left) and low-purity (right) categories obtained with the 2017 simulation are shown for the projection on \(m_{\mathrm{jet1}}\) (upper) and \(m_{\mathrm{jj}}\) (lower). The projection on \(m_{\mathrm{jet2}}\) is omitted since it is equivalent to the \(m_{\mathrm{jet1}}\) projection except for statistical fluctuations. The pdfs and the simulations shown are normalized to unity. The normalization uncertainty of 50% is not shown. The distributions for 2016 simulations are similar

7 Statistical interpretation

To test for the presence of narrow resonances decaying to two vector bosons we follow the \(\text {CL}_\mathrm{s}\) prescription, evaluated using asymptotic expressions described in Refs. [80,81,82]. The limits are computed using a shape analysis of the three dimensional \(m_{\mathrm{jj}}\)\(m_{\mathrm{jet1}}\)\(m_{\mathrm{jet2}}\) spectrum, where the 3D signal and background pdfs obtained above are fitted simultaneously to the data for each signal mass hypothesis and category. The signal and background yields are determined simultaneously in this fit. Systematic uncertainties are treated as nuisance parameters and profiled in the statistical interpretation using log-normal constraints, while Gaussian constraints are used for shape uncertainties.

Figures 10 and 11 show the \(m_{\mathrm{jet}}\) and \(m_{\mathrm{jj}}\) spectra in data for the high- and low-purity categories, respectively. The solid gray curves represent the results of the maximum likelihood fit to the data, allowing the signal yields to assume their best fit value, while the lower panels show the corresponding pull distributions, quantifying the agreement between the hypothesis of signal plus background and the data. The resonant background components are shown separately. A signal is superimposed onto all three projections corresponding to a signal yield as expected from the theoretical prediction and the analysis selection efficiency, and scaled by a factor of 5. The background yields in the signal region extracted from a background-only fit, together with their post-fit uncertainties, are summarised in Table 3 and compared with observations, separately for the two categories. The extracted \(\text{ V }{}{}\) +jets cross sections are found to be compatible with the SM expectations within one standard deviation of the post-fit uncertainties.

Fig. 10
figure 10

For the HPHP category: comparison between the signal+background fit and the data distributions of \(m_{\mathrm{jet1}}\) (upper left), \(m_{\mathrm{jet2}}\) (upper right), and \(m_{\mathrm{jj}}\) (lower). The background shape uncertainty is shown as a gray shaded band, and the statistical uncertainties of the data are shown as vertical bars. An example of a signal distribution is overlaid, where the number of expected events is scaled by a factor of 5. Shown below each mass plot is the corresponding pull distribution \((\hbox {Data-fit})/\sigma \), where \(\sigma =\sqrt{\sigma _{\mathrm{data}}^2-\sigma _{\mathrm{fit}}^2}\) for each bin to ensure a Gaussian pull-distribution, as defined in Ref. [83]

Fig. 11
figure 11

For the HPLP category: comparison between the signal+background fit and the data distributions of \(m_{\mathrm{jet1}}\) (upper left), \(m_{\mathrm{jet2}}\) (upper right), and \(m_{\mathrm{jj}}\) (lower). The background shape uncertainty is shown as a gray shaded band, and the statistical uncertainties of the data are shown as vertical bars. An example of a signal distribution is overlaid, where the number of expected events is scaled by a factor of 5. Shown below each mass plot is the corresponding pull distribution \((\hbox {Data-fit})/\sigma \), where \(\sigma =\sqrt{\sigma _{\mathrm{data}}^2-\sigma _{\mathrm{fit}}^2}\) for each bin to ensure a Gaussian pull-distribution, as defined in Ref. [83]

Table 3 Observed yield and background yields extracted from the background-only fit together with post-fit uncertainties, in the two purity categories

No significant excess over the background estimation is observed. Upper limits on the production cross section at 95% confidence level (\(\text {CL}\)) are set. Limits are set in the context of the bulk graviton model and the HVT model B scenario, using the narrow-width approximation. Figure 12 shows the resulting limits as a function of the resonance mass compared to theoretical predictions. The theoretical cross sections shown in Figure 12 are calculated to LO in QCD as detailed in Ref [6, 34]. For the HVT model B, we exclude at 95% \(\text {CL}\) \({\text {W}}^{\prime }\) and \({\text {Z}}^{\prime }\) spin-1 resonances with masses below 3.8 and 3.5  \(\text {Te}\text {V}\), respectively. In the narrow-width bulk graviton model, upper limits on the production cross sections for \(\text{ G }{}{} _{\mathrm{bulk}} \rightarrow \text{ W }{}{} {}\text{ W }{}{} (\text{ Z }{}{} {}\text{ Z }{}{} {})\) are set in the range from 20 (27 )\(\,\text {fb}\) for a resonance mass of 1.2  \(\text {Te}\text {V}\), down to 0.2 \(\,\text {fb}\) for a resonance mass of 5.2  \(\text {Te}\text {V}\).

Fig. 12
figure 12

Observed and expected 95% \(\text {CL}\) upper limits on the product of the production cross section (\(\sigma \)) and the branching fraction, obtained after combining categories of all purities with 77.3\(\,\text {fb}^{-1}\) of 13 \(\text {Te}\text {V}\) data, for \(\text{ G }{}{} _{\mathrm{bulk}} \rightarrow \text{ W }{}{} \text{ W }{}{} \) (upper left), \(\text{ G }{}{} _{\mathrm{bulk}} \rightarrow \text{ Z }{}{} \text{ Z }{}{} \) (upper right), \({\text {W}}^{\prime } \rightarrow \text{ W }{}{} \text{ Z }{}{} \) (lower left), and \({\text {Z}}^{\prime } \rightarrow \text{ W }{}{} \text{ W }{}{} \) (lower right) signals. For each signal scenario the theoretical prediction (red line) and its uncertainty associated with the choice of PDF set (red hashed band) is shown. The theory cross sections (red line) are calculated at LO in QCD [6, 34]

The expected upper limits obtained using the multi-dimensional fit method introduced here are compared to those obtained in a previous search [29] using the same data set in order to estimate the sensitivity gain by using the new method. Figure 13 shows the expected limits for one signal model based on an analyses of the data collected in 2016, using either the fit method presented here, or previous methods. We obtain a 20–30% improvement in sensitivity when using the multi-dimensional fit method, and about a 35–40% improvement when combining the data sets recorded in 2016 and 2017 relative to the individual results. The same conclusion holds for the other signal models investigated in this paper. The results obtained with the multi-dimensional fit are the best to date in the \(\text{ V }{}{}\) \(\text{ V }{}{}\) channel and reach a similar sensitivity at high masses (\(m_{\mathrm{X}}\)) as the combination of diboson and leptonic decay channels for the 36\(\,\text {fb}^{-1}\) recorded in 2016 [27, 28].

Fig. 13
figure 13

Expected 95% \(\text {CL}\) upper limits on the product of the production cross section (\(\sigma \)) and the branching fraction for a \(\text{ G }{}{} _{\mathrm{bulk}} \rightarrow \text{ W }{}{} \text{ W }{}{} \) signal using 35.9\(\,\text {fb}^{-1}\) of data collected in 2016 obtained using the multi-dimensional fit method presented here (red solid line), compared to the result obtained with previous methods (black dash-dotted line) [29]. The final limit obtained when combining data collected in 2016 and 2017 is also shown (blue dashed line)

8 Summary

A search is presented for resonances with masses above 1.2  \(\text {Te}\text {V}\) that decay to \(\text{ W }{}{}\) \(\text{ W }{}{}\), \(\text{ Z }{}{}\) \(\text{ Z }{}{}\), or \(\text{ W }{}{}\) \(\text{ Z }{}{}\) boson pairs. Each of the two bosons decays into one large-radius jet, yielding dijet final states. The search is conducted using a novel approach based on a three-dimensional maximum likelihood fit in the dijet invariant mass as well as the two jet masses, thus taking advantage of the fact that the expected signal is resonant in all three mass dimensions. This method yields an improvement in sensitivity of up to 30% relative to previous search methods. The new method places additional constraints on systematic uncertainties affecting the signal by measuring the standard model background from \(\text{ W }{}{}\) or \(\text{ Z }{}{}\) production with associated jets, accounting for 5% of the overall improvement in sensitivity. Decays of \(\text{ W }{}{} \) and \(\text{ Z }{}{} \) bosons are identified using jet substructure observables that reduce the background from quantum chromodynamics multijet production. No evidence is found for a signal, and upper limits on the resonance production cross section are set as a function of the resonance mass. The limits presented in this paper are the best to date in the dijet final state, and have a similar sensitivity as the combinations of diboson and leptonic decay channels using the 2016 data set. The results are interpreted within bulk graviton models, and as limits on the production of the \({\text {W}}^{\prime }\) and \({\text {Z}}^{\prime }\) bosons within the heavy vector triplet framework. For the heavy vector triplet model B, we exclude at 95% confidence level \({\text {W}}^{\prime }\) and \({\text {Z}}^{\prime }\) spin-1 resonances with masses below 3.8 and 3.5  \(\text {Te}\text {V}\), respectively. In the narrow-width bulk graviton model, upper limits on the production cross sections for \(\text{ G }{}{} _{\mathrm{bulk}} \rightarrow \text{ W }{}{} \text{ W }{}{} ~(\text{ Z }{}{} {}\text{ Z }{}{} {})\) are set in the range of 20 (27)\(\,\text {fb}\) to 0.2 \(\,\text {fb}\) for resonance masses between 1.2 and 5.2  \(\text {Te}\text {V}\).