1 Introduction

Despite its remarkable success in describing all known elementary particles and forces, the standard model (SM) of particle physics leaves several fundamental questions unanswered. Such open questions include the nature of dark matter, the asymmetry between matter and antimatter, the hierarchy of particle masses, and the stability of the Higgs field. Searches for new physics processes beyond the SM (BSM) that might shed light on these issues have so far failed to yield any discoveries. Experiments at the CERN LHC and elsewhere continue to perform new measurements and to cast an even wider net of searches for new phenomena. We do not have clear theoretical indications about the nature of these new phenomena, but they can manifest themselves beyond current theoretical models. This motivates a growing interest for model-nonspecific searches [1,2,3,4] to complement direct model-specific searches. More comprehensive model-independent frameworks such as effective field theories that probe BSM physics in generic terms [5,6,7,8,9] provide a complementary approach. The addition of new detectors, further extending the capability and coverage of the LHC experiments, also offers a new venue to explore final-state topologies and areas of phase space which were not previously covered.

This paper describes a generic search for a hypothetical massive particle X produced in association with one SM particle in central exclusive production processes at the LHC. In the interaction, the two colliding protons survive after exchanging two colourless particles, and can be recorded in the CMS-TOTEM precision proton spectrometer (CT-PPS hereafter). The detection and accurate measurement of both forward protons allow the full kinematic reconstruction of the event, including the determination of the four-momentum of X extracted from the balance between the four-momenta of the tagged SM particle(s) and the forward protons. This “missing mass” (\(m_{\text {miss}}\)) approach allows searching for BSM particles without assumptions about their decay properties, except that the decay width can be considered narrow enough to produce a resonant mass peak, thus allowing generic BSM searches. In this analysis we demonstrate the feasibility of exploiting such a technique by searching for a massive particle produced in association with a Z boson or a photon in the final state, as illustrated in Fig. 1. The analysis relies on the assumptions that the process is exclusive, i.e. that nothing is produced in addition to \({{\textrm{Z}}}/\upgamma +{{\textrm{X}}} \), and that there is no ambiguity in the selection of the charged leptons from the Z decay or the \({\varvec{\gamma }}\) with respect to a possible signature of X in the detector.

The \(m_{\text {miss}}\) technique is used to probe the mass of this particle. The precise (percent level) proton momentum reconstruction of CT-PPS allows one to search for \(m_{\text {miss}}\) signatures in the high invariant-mass range (600–1600 GeV) covered by the CT-PPS acceptance, with unprecedented resolution. In this high mass range, electroweak processes are generally enhanced relative to quantum chromodynamics (QCD)-induced processes [10] and thus we assume a photon–photon induced exclusive production process.

Fig. 1
figure 1

Schematic diagram of the photon–photon production of a Z boson or a photon with an additional, unknown particle X, giving rise to \(m_{\text {miss}} = m_{{{\textrm{X}}}} \). The production mechanism does not have to proceed through photon exchange. Other colourless exchange mechanisms (e.g. double pomeron [11]) are also allowed. For high-mass central exclusive production, electroweak processes are expected to dominate, and QCD-based colourless exchanges are expected to be suppressed

In 2017, a large sample of proton–proton (\({\textrm{pp}}\)) collision data at \(\sqrt{s}=13\, \textrm{TeV}\) was recorded with the CMS detector [12] and CT-PPS [13]. The trigger for the central CMS detector is provided by either an isolated photon or a pair of electrons or muons from \({{\textrm{Z}}} \rightarrow {\ell } ^+{\ell } ^-\) decays. The quantity \(m_{\text {miss}}\) is constructed using energy conservation and the four-momenta of the reconstructed boson in the central detector and the final-state protons in CT-PPS. Final states with a Z boson (\(\upgamma \)) are selected from \({\textrm{pp}}\) collision data corresponding to an integrated luminosity of 37.2 (2.3)\(\,\text {fb}^{-1}\). Events with a \(\upgamma \) in the final state were collected with a prescaled trigger.

The paper is organised as follows: the experimental setup is described in Sect. 2, and it is followed in Sect. 3 by the description of the simplified generic Monte Carlo (MC) simulation which is used to simulate the associated exclusive production of a massive particle of narrow width, together with a Z boson or photon and forward protons. The event selection is summarised in Sect. 4 and the strategy used to estimate the background, which is dominated by a random coincidence of Z boson or \(\upgamma \) production with protons from a different collision in the same bunch crossing (pileup, PU), is discussed in Sect. 5. The description of the statistical analysis used to compare the observed \(m_{\text {miss}}\) spectrum with the background plus signal model is presented in Sect. 6 and the results are discussed in Sect. 7, followed by a summary in Sect. 8. Tabulated results are provided in the HEPData record for this analysis [14].

2 The CMS and CT-PPS detectors

The central feature of the CMS apparatus is a superconducting solenoid of \(6\,\hbox {m}\) internal diameter, providing a magnetic field of \(3.8\,\hbox {T}\). Within the solenoid volume are located a silicon pixel and strip tracker with coverage in pseudorapidity up to \(|\eta |=2.5\), surrounded by a lead tungstate crystal electromagnetic calorimeter (ECAL), and a brass and scintillator hadron calorimeter directly outside the ECAL. The muon detection system consists of three types of gas-ionisation chambers embedded in the steel flux-return yoke outside the solenoid. A more detailed description of the CMS detector, together with a definition of the coordinate system used and the relevant kinematic variables, can be found in Ref. [12]. Events are selected online and stored at a maximal rate of about 1 kHz using a two-tier triggering system [15, 16].

The CT-PPS is an array of movable, near-beam “Roman pot” (RP) [17] devices containing tracking and timing detectors inserted horizontally at a distance from the beam corresponding to about 14 standard deviations of the transverse distribution of the LHC beam, i.e. in the range from 1.5 to \(3.0\,\hbox {mm}\). The detectors are used to reconstruct the flight path of protons coming from the interaction point (IP) through 210 m of LHC beamline.

The tracking stations provide a measurement of the proton trajectories with respect to the beam position. Knowledge of the magnetic fields traversed by the proton from the IP to the RPs allows the reconstruction of its fractional momentum loss, with respect to the momentum of the incident proton, \(\xi =\Delta p/p\sim D_x^{-1} x\), where x represents the horizontal displacement of the scattered proton at the RP location, and \(D_x\) the horizontal dispersion, a property of the accelerator optics. The techniques used for the alignment and calibration of the apparatus and proton reconstruction are detailed in Refs. [18,19,20,21].

In its 2017 configuration, CT-PPS is able to measure protons which have lost approximately 2-20 % of their initial momentum, and which remain inside the beam pipe until they hit the RPs. This corresponds to an invariant mass of the central system in the range 600–1600 GeV.

The performance of CT-PPS and its potential for high-mass exclusive measurements was validated by the observation, by the CMS and TOTEM Collaborations, of proton-tagged (semi-) exclusive dilepton events recorded in 2016 [22].

If both the forward protons and the central boson are measured, the kinematics of the hypothetical X particle can be fully reconstructed. Given the excellent resolution of the charged-lepton and photon reconstruction in the central CMS detector and of the protons in CT-PPS, we search for a resonance in the \(m_{\text {miss}}\) distribution.

The missing mass is defined as:

$$\begin{aligned} m_{\text {miss}} ^2=\left[ (P^\text {in}_{{{\textrm{p}}} _1}+P^\text {in}_{{{\textrm{p}}} _2}) - (P_{\textrm{V}} +P^\text {out}_{{{\textrm{p}}} _1}+P^\text {out}_{{{\textrm{p}}} _2})\right] ^2, \end{aligned}$$
(1)

where \(P_{\textrm{V}} \) is the four-momentum of the boson and \(P^\text {out,in}_{{{\textrm{p}}} _i}\) (\(i=1\), 2) are the four-momenta of the outgoing and incoming protons, respectively.

In this analysis, two RP tracking stations per side, or “arm”, of CMS are used. The stations in LHC sector 56 are located on the side that corresponds to positive z coordinates in the CMS coordinate system, while the stations on the other side are located in LHC sector 45, corresponding to negative z. In 2017, the inner stations, closest to the CMS detector, were instrumented with silicon strip detectors and the outer stations with 3D pixel detectors. The strip detectors were designed for low-luminosity and low-PU conditions. They cannot resolve multiple tracks and have lower radiation hardness. In the case of multiple protons crossing a strip detector, zero tracks are reconstructed. In spite of these limitations, the reconstruction of protons requiring tracks in both the pixel and strip detectors (multi-RP reconstruction) yields high-purity samples of protons with excellent momentum resolution, which more than compensates for the loss in the overall acceptance and efficiency. After comparing various reconstruction options, the multi-RP reconstruction was selected as the preferred method. However, to maximise the efficiency of this search, pixel-only reconstruction (single-RP) is used as a fallback in case the multi-RP reconstruction fails in an event.

3 Physics model and event simulation

3.1 Signal model

The signal is simulated using a simplified MC model that uses as main inputs the mass of the X particle, \(m_{{{\textrm{X}}}} \), and the \(p_z\) spectrum of the \({\textrm{VX}}\) system, where \({\textrm{V}} ={{\textrm{Z}}},\upgamma \). It is assumed that the vector boson V is produced isotropically in the \({\textrm{VX}}\) frame, and that the leptons from the \({{\textrm{Z}}} \) decay are also produced isotropically in the \({{\textrm{Z}}} \) reference frame. Samples are produced for a range of \(m_{{{\textrm{X}}}} \) values, with the requirement that the corresponding mass of the \({\textrm{VX}}\) system (\(m_{{\textrm{VX}}}\)) falls within the acceptance of CT-PPS. For each event, the \(m_{{\textrm{VX}}}\) is generated assuming an exponential spectrum, \(m_{{\textrm{VX}}}=m_{{{\textrm{X}}}} +\varepsilon +100\,\textrm{GeV}\), where \(\varepsilon \) is a randomly distributed variable following an exponential probability distribution function with decay constant \(\tau =-0.04\, \textrm{GeV}^{-1}\). The values chosen for these constants were found to provide a good coverage of the phase space within the CT-PPS acceptance; however, their precise values and the choice of the shape of the spectrum have negligible effect on the result of this analysis. The di-proton \(p_z\) values, \(p_z({\textrm{pp}})\), on the other hand, have a sizeable effect on the proton acceptance. They are, by momentum conservation, the same as those of the \({\textrm{VX}}\) system but have opposite sign; the \(p_z\) distribution is obtained using the equivalent photon approximation [23] resulting in a Gaussian distribution with a mass-dependent width, ranging from 856 to 1250 GeV for X masses varying from 600 to 1600 GeV.

The outgoing protons are transported from the IP to the CT-PPS RPs using the LHC optics, taking into account aperture limitations, which depend on the beam crossing angle. The simulation of the protons includes beam divergence and vertex smearing at the IP also depending on the beam crossing angle, as detailed in Ref. [19]. The hits in the RP detectors are generated taking into account sensor acceptance, efficiency, and resolution. Corrections for the acceptance and reconstruction efficiency of the photon or the leptons from Z boson decay in the central CMS detector are also applied. More details are given in Sect. 4.

Given the wide \(p_z\) spectrum of the \({\textrm{VX}}\) system with respect to the CT-PPS acceptance, we divide the generated events into two sets: inside and outside the fiducial volume. The definition of the fiducial volume is based on the kinematic properties of the generator-level particles from the hard process: the boson (or its decay products) and the outgoing protons are used in the definition as summarised in Table 1. The requirements on the outgoing protons are not symmetric. The acceptances of the \(+z\) and \(-z\) arms of CT-PPS are different, mostly as a result of differences in the LHC optics settings. The requirements for the \(\xi \) range at generator level are chosen accordingly for the proton in the +z arm (\(\xi _{+}^\text {gen}\)) and -z arm (\(\xi _{-}^\text {gen}\)). Four main LHC beam crossing angle configurations are used in 2017, each resulting in a different detector acceptance as a function of \(\xi \). The average of the lower edges of the \(\xi \) acceptance is defined as the start of the \(\xi \) range for the fiducial region, and the average of the upper edges as the end of the range [18]. This approach offers a compromise between maximal acceptance of the protons recorded in the data and minimal use of outer regions of phase space that were not fully efficient during the entire data taking due to varying beam configurations. The fiducial volume is used to define the reference normalisation of the signal.

Events falling outside of these fiducial requirements are considered as background in this analysis; the normalisation of the background is allowed to float freely in the fits described later, independently of the signal term. This definition reduces the dependence on model assumptions, and specifically the precise shape of the \(p_z\) spectrum of the generated signal.

Table 1 Combined CMS+CT-PPS fiducial volume selection criteria in the Z and \(\upgamma \) analyses. The leading- and subleading-\(p_{\textrm{T}}\) leptons, where \(p_{\textrm{T}}\) is transverse momentum, are labelled as \({\ell } _1\) and \({\ell } _2\), respectively. The Z boson mass is noted as \(m_{{{\textrm{Z}}}}\)

3.2 Simulation samples for background validation

Although the background in this analysis is fully modelled from data, standard full simulation MC samples, including the simulation of the CMS detector response with Geant4  [24], are used to optimise the event selection and to validate the efficiencies of the particle reconstruction and identification, as well as of the triggers. Each background process is generated in coincidence with additional minimum-bias events using pythia8 to simulate PU events, and with a frequency distribution matching that observed in the data. The PU protons are modelled with a dedicated event mixing technique, which is explained in more detail in Sect. 5.

The main process considered for the validation is Drell-Yan, in particular Z boson production (\({{\textrm{Z}}} +\) jets), simulated using MadGraph 5_amc@nlo v2.2.2, with FxFx merging [25], at next-to-leading order (NLO) precision in QCD. For the photon analysis the main process considered is the production of an isolated photon in association with jets (\(\upgamma +\)jets), modelled at the leading-order using MadGraph 5_amc@nlo with MLM merging [26]. Top quark production processes (single top \({\textrm{tW}}\) and \(\textrm{t}\bar{\textrm{t}}\)) are also simulated with powheg  [27, 28] as well as diboson production (\({\textrm{WW}}\), \({\textrm{WZ}}\), and \({\textrm{ZZ}}\)), the latter using pythia8 version 8.226 [29]. The pythia8 program is used as the parton shower generator for the processes simulated with a matrix element approach. Furthermore, single- and double-diffractive [30] Z boson samples are produced with pythia8 and Pomwig [31] to check that the contribution of these processes to the final sample is consistent with the background estimate based on the data.

4 Event reconstruction and selection

A particle-flow algorithm [32] is used for offline reconstruction of the events in the central CMS detector, exploiting an optimised combination of information from the various subdetectors. The energy of photons is obtained from the ECAL measurement, while the energy of electrons is determined from a combination of the electron momentum at the primary interaction vertex as determined by the tracker, the energy of the corresponding ECAL cluster, and the energy sum of all bremsstrahlung photons spatially compatible with originating from the electron track [33]. The energy of muons is obtained from the curvature of the corresponding tracks [34].

Protons coming from the IP are reconstructed by CT-PPS, either from a single RP station (single-RP reconstruction) or using two RP stations (multi-RP reconstruction). In the single-RP method, only pixel detectors are used for proton reconstruction since the strip detectors have significantly lower efficiency. Efficiency corrections are applied to the simulated protons and account for three components: the effect of radiation damage in the RP strip and pixel detectors, the inefficiency of the strip stations when they are hit by several protons, and the effect of matching the pixel and strip stations in a combined multi-RP proton fit. The radiation damage efficiency corrections are parametrised as functions of time and beam crossing angle, separately for the strip and pixel detectors in each arm of CT-PPS. All these efficiencies have been measured in data. The efficiency of matching the pixel and strip stations in the combined multi-RP proton reconstruction has also been measured in data. Further details can be found in Ref. [18].

The events are selected if they pass a single-photon trigger (transverse momentum \(p_{\textrm{T}} >90\,\textrm{GeV}\) in the ECAL barrel region), or a combination of double-lepton triggers with the leading lepton \(p_{\textrm{T}} >17\,(23)\,\textrm{GeV}\) and the subleading lepton \(p_{\textrm{T}} >8\,(12)\,\textrm{GeV}\) for muons (electrons) and a single-muon trigger (\(p_{\textrm{T}} >27\,\textrm{GeV}\)). Because of its larger rate, the photon trigger was prescaled throughout the data taking period analysed, resulting in a data set with lower effective integrated luminosity.

Offline, the events are preselected if they contain one isolated photon with \(p_{\textrm{T}} >95\,\textrm{GeV}\) in the barrel, or at least two leptons passing the electron or muon identification and isolation criteria. In all cases the isolation is based on the \(p_{\textrm{T}}\) of all particle-flow candidates found in a cone around the object being selected. An isolation cone with a radius of \(\sqrt{\Delta \eta ^2+\Delta \phi ^2}=0.3\), where \(\phi \) is the azimuthal angle in radians, is used for electrons and photons, and 0.4 for muons.

For the leptonic selection, the following requirements must be fulfilled:

  • leading lepton: \(p_{\textrm{T}} >30\,\textrm{GeV}\) and \(|\eta |<2.4\);

  • sub-leading lepton: \(p_{\textrm{T}} >20\,\textrm{GeV}\) and \(|\eta |<2.4\);

  • dilepton mass \(m({\ell } _1, {\ell } _2)>20\,\textrm{GeV}\), and the two leptons have opposite electric charge.

The events are then categorised as:

  • Same-flavour events (\({\textrm{ee}}\) or \(\upmu \upmu \)) – if \(m({\ell } _1, {\ell } _2)\) is consistent with the mass of a decayed Z boson, i.e. \(|m({\ell } _1,{\ell } _2)-m_{{\textrm{Z}}} |<10\,\textrm{GeV}\). These events are furthermore categorised in a control region with \(p_{\textrm{T}} ({{\textrm{Z}}})<10\,\textrm{GeV}\) and a search region with \(p_{\textrm{T}} ({{\textrm{Z}}})>40\,\textrm{GeV}\);

  • single-photon events (\(\upgamma \)) – if one photon is selected at the trigger and offline levels;

  • different-flavour events (\({\textrm{e}} \upmu \)) – used as a control sample for background modelling.

No veto requirements for additional objects in the event are imposed, thus allowing the presence of other activity in the event besides the leptons from the Z boson decay, or the photon. In the signal regions of the same-flavour and single-photon samples, in addition to the selection criteria described above, it is required that at least one proton be reconstructed in each arm of CT-PPS. The protons are obtained from the combined multi-RP reconstruction (default), or the pixel-only single-RP reconstruction when the former is not available in an event. Dedicated calibrations are used for the two main periods of the 2017 data taking, and for the 120, 130, 140, and 150 \(\mu \)rad LHC beam crossing angles. Events recorded during periods with other angles are excluded from this analysis. The reconstructed protons are pre-selected with \(\xi _{\pm }>0.035\). Additional requirements on the lower value of \(\xi \) remove the region where the acceptance varies quickly close to the edge of the sensors; likewise an upper \(\xi \) requirement is imposed to reflect the LHC collimator apertures. Further details can be found in Ref. [19].

The simulation of the signal samples is performed at the generator level and does not include the CMS detector response. The efficiency and acceptance corrections are applied to the generated leptons from the Z boson decay, or to the generated central photon, based on the expectations from CMS full simulation MC samples with similar dilepton/photon topologies. As a cross-check, for one of the mass points (\(m_{{{\textrm{X}}}} =950\,\textrm{GeV}\)), the full CMS detector simulation and PU conditions are applied and used for comparison. The results of the full and the simplified simulation are in good agreement.

Fig. 2
figure 2

Comparison of the \(m_{\text {miss}}\) shapes for the simulated \({\textrm{pp}}\rightarrow {\textrm{ppZX}}\) signal events within the fiducial region and those outside it, after including the effect of PU protons as described in the text, for a generated \(m_{{{\textrm{X}}}} \) mass of 1000 GeV. A fiducial cross section of 1 pb is used to normalise the simulation. From left to right and top to bottom, the distributions are shown for the different proton reconstruction categories: multi(\(+z\))-multi(\(-z\)), multi(\(+z\))-single(\(-z\)), single(\(+z\))-multi(\(-z\)) and single(\(+z\))-single(\(-z\))

Figure 2 shows the expected signal shape as a function of \(m_{\text {miss}}\) for events inside and outside the fiducial region for a generated \(m_{{{\textrm{X}}}} =1000\,\textrm{GeV}\) and for different categories of proton reconstruction: a multi-RP proton on both sides, one side, or pixel-only single-RP protons on both sides. One can observe that events outside the fiducial region, where at least one signal proton is not in the CT-PPS acceptance, are expected to have a non-resonant spectrum. In addition there is a clear difference in shape between the two components, and the resolution of the peak for events in the fiducial region depends strongly on the proton reconstruction used. The expected resolution in the reconstruction of the \(m_{\text {miss}}\) variable is \(\approx \) 2% (7%) with the multi- (single-)RP reconstruction algorithm. The so-called fallback categories, which make use of pixel-only reconstruction, have a worse resolution and also contain a smaller number of selected events with respect to the main category in this analysis. Nevertheless, these categories contribute to increasing the overall sensitivity of the search by \(\approx 20\%\). Events inside the fiducial region exhibit also a non-resonant component because of PU. The shape and relative contribution of this component is category-dependent because of the different reconstruction efficiency and acceptance of the detectors.

Figure 3 summarises the overall product of acceptance and selection efficiency for the Z and \(\upgamma \) analyses including both the central object(s) (photon or leptons) and the proton contributions, for events produced inside the fiducial volume. In both channels, the overall product of acceptance and efficiency is maximal for \(m_{{{\textrm{X}}}} \) in the range 800–1150 GeV. For about half of the events, the multi-RP reconstruction algorithm is used for both protons. An important contribution to the loss of the efficiency for multi-RP protons in the 2017 data is due to the inclusion of downtime of parts of the detector as an inefficiency component and to the inefficiency caused by multiple tracks from PU protons in the RP strip-detectors [18]. Approximately a factor of two in overall efficiency is recovered through the inclusion of pixel-only single-RP protons when multi-RP protons are not available.

Fig. 3
figure 3

Product of the acceptance and the combined reconstruction and identification efficiency, as a function of \(m_{{{\textrm{X}}}} \), for events generated inside the fiducial volume defined in Table 1. The curves shown in the left panel display the different final states, while the ones in the right panel show the contributions from the different proton reconstruction categories in the \({{\textrm{Z}}} \rightarrow \upmu \upmu \) analysis: multi(\(+z\))-multi(\(-z\)), multi(\(+z\))-single(\(-z\)), single(\(+z\))-multi(\(-z\)) and single(\(+z\))-single(\(-z\)). The definitions of the fiducial region and of the signal model used to estimate the acceptance are provided in the text

5 Background model

Several sources of backgrounds are considered:

  • inclusive SM processes (mainly \({{\textrm{Z}}} +\) jets or \(\upgamma +\) jets), with two protons from PU events (combinatorial background);

  • a single-diffractive process, with one additional proton from a PU event;

  • double-diffractive processes;

  • exclusive SM processes (mainly \(\upgamma \upgamma \rightarrow {\ell } {\ell } \));

  • signal-induced background (from signal events in which one or both of the protons escape detection).

Of these sources, by far the most important one is the combinatorial background, originating from the random superposition of proton candidates from PU collisions reconstructed in the RPs on both sides. We employ a background model fully based on data, by replacing the proton on one side of the CMS detector (single mixing) or on both sides (double mixing), with a proton (or protons) from a randomly chosen event, from the same data taking period and with the same beam conditions. It is expected that the background model correctly reproduces the combinatorial background by mimicking precisely the process of the superposition of uncorrelated pile-up protons, and that the single mixing simulates to good approximation the contribution from single-diffractive processes. The contributions from other fully exclusive or double-diffractive processes are estimated, from simulation, to be negligible. The signal-induced background is taken from the signal simulation and included separately in the final fit.

In order to avoid any residual statistical correlation with the hypothetical signal present in the signal region, the protons for the mixing procedure are taken from a control sample that is orthogonal by construction to the signal sample. This is done by using \({{\textrm{Z}}} \rightarrow \upmu \upmu \) events with \(p_{\textrm{T}} ({{\textrm{Z}}})<10\,\textrm{GeV}\). The background estimate obtained with protons from a different control region (\({\textrm{e}} \upmu \) events passing the full selection) is compared to the default background model, and the difference considered as a systematic uncertainty. An additional systematic uncertainty is considered from the difference found with respect to the background estimate obtained from single mixing, reflecting a possible difference between the modelling of purely combinatorial background and single-diffractive processes.

The mixing with random protons from other events is repeated 100 times per data event in order to increase the statistical precision of the background prediction. The same procedure is also applied to simulated events, taking care that the relative fractions of integrated luminosity for each of the data-taking periods is respected when selecting the protons to be mixed.

A twofold validation of the background estimation method is performed. A first check is made using MC simulation samples with embedded PU protons. Figure 4 illustrates the distributions of the reconstructed \(\xi \) and di-proton rapidity for simulated events, compared to these two distributions seen in data. In general, good agreement is observed, indicating that the samples are dominated by inclusive production of Z and \(\upgamma \) bosons with two forward protons from PU.

Fig. 4
figure 4

Distributions of the reconstructed proton \(\xi \) in the negative arm (left), positive arm (middle), and the corresponding di-proton rapidity (right) from the proton mixing procedure with simulated MC events are compared to data, in the upper panels in each plot. Processes other than the ones displayed in the figures are estimated to have negligible residual contributions. The lower panels display the ratio between the data and the background model, with the arrows indicating values lying outside the displayed range. The hatched band illustrates the statistical uncertainty of the background model. The \({\textrm{ee}}\), \(\upmu \upmu \), and photon final states are shown from top to bottom. The \({\textrm{ee}}\) and \(\upmu \upmu \) events are displayed without the Z boson \(p_{\textrm{T}}\) requirement. For illustration, the simulated signal distributions are superimposed for various choices of \(m_{{{\textrm{X}}}} \), normalised to a generated fiducial cross section of 100 \(\text {pb}\)

A second check is made using the \({\textrm{e}} \upmu \) data control sample. The \({\textrm{e}} \upmu \) data control sample is expected to be dominated by \(\textrm{t}\bar{\textrm{t}}\) events and to contain no signal events. Figure 5 illustrates some of the distributions obtained with this sample. Selected \({\textrm{e}} \upmu \) events are mixed multiple times with protons from random \({{\textrm{Z}}} \rightarrow \upmu \upmu \) events with \(p_{\textrm{T}} ({{\textrm{Z}}})<10\,\textrm{GeV}\) to simulate the combinatorial background shape, which is compared to the shape observed in unaltered \({\textrm{e}} \upmu \) events. The observed \(\xi \) distributions, those of the di-proton invariant mass, and of \(m_{\text {miss}}\) obtained with the combinatorial background model are in good agreement with the unaltered data. The event selection is representative of the final event selection in the signal region, with exactly one multi-RP proton candidate in each arm and a \(p_{\textrm{T}} ({\textrm{e}} \upmu )>40\,\textrm{GeV}\) requirement applied in the signal selection. The latter is used as a requirement analogous to that on \(p_{\textrm{T}} ({{\textrm{Z}}})\). The uncertainty band reflects the uncertainty from the the limited event sample size and the comparison with the background shapes (either from mixing with protons from \({\textrm{e}} \upmu \) events or from mixing protons separately in each arm from \({{\textrm{Z}}} \rightarrow \upmu \upmu \)).

Fig. 5
figure 5

Validation of the background modelling method, using the \({\textrm{e}} \upmu \) control sample. Selected \({\textrm{e}} \upmu \) events are mixed with protons from \({{\textrm{Z}}} \rightarrow \upmu \upmu \) events with \(p_{\textrm{T}} ({{\textrm{Z}}})<10\,\,\textrm{GeV}\) to simulate the combinatorial background shape, while the data points are unaltered \({\textrm{e}} \upmu \) events. The proton \(\xi \) distributions for both CT-PPS arms (upper row), those of the di-proton invariant mass (lower left), and of \(m_{\text {miss}}\) (lower right) are shown. The lower panel in each plot displays the ratio between the data and the background model, with the arrows indicating values lying outside the displayed range. The gray uncertainty band around the background prediction represents the contribution from the limited sample size. The red uncertainty band represents the effect of adding in quadrature the differences with the alternative mixing approaches described in the text

5.1 Proton mixing in simulated signal samples

To model protons originating from PU interactions in the signal MC samples, an approach is used similar to that for the background modelling in data. We add protons from events in the data control sample to simulated events, making sure that the protons are selected proportionally to the relative fractions of integrated luminosity for each of the data-taking periods. This procedure guarantees that the effects of PU protons on the missing mass reconstruction are correctly reproduced. As these protons are indistinguishable in the analysis from the signal protons, their presence can cause events outside the fiducial region to pass the event selection, or they may replace a signal proton in the \(m_{\text {miss}}\) reconstruction, thus leading to a dilution of the signal, as is visible in Fig. 2. The PU protons mixing procedure takes the strip inefficiency into account and propagates its effect to the multi-RP reconstruction efficiency. However, when mixing protons from different events, an additional correction needs to be applied in the case no proton was reconstruced in the strip detectors. This could either have been caused by the absence of a proton within detector acceptance, or the presence of multiple protons. In the mixing procedure in signal samples, both options are taken into account with relative probabilities derived as a function of the run conditions, based on dedicated measurements using truly empty events.

6 Statistical analysis and systematic uncertainties

Using the signal and background \(m_{\text {miss}}\) distributions, we perform a statistical analysis to search for central exclusive \({\textrm{VX}}\) production. The search is carried out for different categories (a total of 96 subsamples) of the data with different proton reconstruction algorithms, different beam crossing angles and different primary vertex multiplicities. Depending on the proton reconstruction algorithm used, different binnings within the range 0–2 TeV are used. Bins with a width of 25 GeV are used when both protons are reconstructed with the multi-RP algorithm, 50 GeV when one of the protons is reconstructed with the pixel detector only, and 75 GeV when both protons are reconstructed with the pixels only. The choice of these values is related to the resolution of the reconstruction algorithms used in these categories. While the bin width varies depending on the proton reconstruction algorithm used, it is independent of the mass in the region used for the search and reflects the CT-PPS resolution for \(m_{\text {miss}}\) reconstruction.

A likelihood model containing three components is built to describe the \(m_{\text {miss}}\) spectrum: a signal and two background contributions. The background is composed of the combinatorial and the signal-induced contribution outside the fiducial volume. The normalisations of each of these components are left to float freely in the fit, with the normalisation of the contribution outside the fiducial volume independent of that of the signal, to reduce any dependence on the exact shape of the \(p_z({\textrm{pp}})\) spectrum. The statistical analysis is based on RooStats [35].

The systematic uncertainties associated with the background and signal predictions are incorporated as nuisance parameters in the profile likelihood fit. They may affect the shapes of the expected \(m_{\text {miss}}\) distributions of the signal and backgrounds, as well as their normalisations. We consider the nuisance parameters uncorrelated between signal and background shapes, and between different categories, even if similar methods are used to estimate the uncertainties. The following systematic uncertainties are considered:

Pileup proton spectra::

The \(m_{\text {miss}}\) shape for the background models is determined after replacing the protons of the selected events with protons from low-\(p_{\textrm{T}}\)\({{\textrm{Z}}} \rightarrow \upmu \upmu \) events. For the signal the same source is used to model the contribution from PU protons. As an alternative source of PU protons we use \({\textrm{e}} \upmu \) events. The difference between the two \(m_{\text {miss}}\) shapes is determined point by point and symmetrically applied as a pre-fit estimate of the uncertainties associated with the PU proton spectra. This systematic effect mostly affects the background shape, and the impact is approximately 4% on average. There is a significant difference in statistical precision between the two samples; in some bins this generates large spurious variations (e.g. in the tails of distributions) that are mitigated with a smoothing procedure.

Single-diffractive backgrounds::

Given that we do not explicitly separate the background into its components (pure combinatorial, single diffractive, and double diffractive), we add an uncertainty to estimate the possible difference in shape by mixing protons from a single arm instead of two. The differences in shape between the positive-only (or negative-only) mixing and the double mixing are taken as an estimate of this uncertainty. The impact on the background shape is approximately 2% on average.

CT-PPS efficiency::

The main source of uncertainty is the inefficiency induced by radiation damage in the strip detectors. The efficiency itself is taken into account by reweighting the signal simulation. The weights are then varied according to their uncertainty, to generate alternative \(m_{\text {miss}}\) shapes. This uncertainty is in the range 2–5%, depending on the event category.

Time dependence::

The CT-PPS conditions were not uniform throughout the 2017 data taking. This is taken into account by simulating independently each crossing angle and the two main data taking configurations before and after the so-called second technical stop (TS2) of the LHC [36]. We assign a time-dependent uncertainty by varying the relative contribution of the pre- and post-TS2 simulations; this impacts the signal by approximately 1% on average. The weights are varied by 3%, which reflects the overall uncertainty on the estimated integrated luminosity for the 2017 data set [37].

\(p_{z}({\textrm{pp}})\) spectrum::

it is obtained with the equivalent photon approximation, as described in Sect. 3.1, and its width is parametrised as function of \(m_{{{\textrm{X}}}} \). We estimate the impact of the uncertainty of this parameterisation by varying the width at the generator level for each mass point (typically 10%). The difference with respect to the nominal prediction is used as an uncertainty and is estimated to alter the signal shape by <1%.

Selection efficiency::

The uncertainties in the trigger and selection efficiencies for each final state have a negligible effect on the signal shape and only cause an approximately 3% normalisation uncertainty.

Integrated luminosity::

An additional 2.3% normalisation uncertainty is assigned to the normalisation of the signal [37].

Limited event count::

Residual uncertainties due to limited sample sizes are also included in the fit using the Beeston–Barlow method [38]. They are typically small for the signal and background (\({\le }\,1\%\)).

7 Results

A total of 948,070 events with an \({\textrm{ee}}\) and 1,477,237 with a \(\upmu \upmu \) final state, as well as 85,024 with a photon in the final state are selected for the analysis. Events with dilepton final states (\({\textrm{ee}}\) and \(\upmu \upmu \)) originate almost exclusively (\(>99.7\%\)) from \({{\textrm{Z}}} \rightarrow {\ell } {\ell } \) production, while the majority of those with a \(\upgamma \) final state are estimated to come from single-photon production (\(>99.8\%\)). The MC-based expectations agree with the data within 1–10%, depending on the final state and the subsample. In the statistical analysis we use the total number of events selected in the data as an initial estimate for the normalisation of the combinatorial background, so these residual normalisation differences have no impact on the results.

The observed distributions of the \(m_{\text {miss}}\) variable are shown in Figs. 6, 7, and 8. The distributions are plotted after the fit is performed and therefore include the adjustments to the background shape and normalisation derived from the observed data. The normalisation of the out-fiducial component after the fit is typically close to 0. A reference signal is overlaid for illustration. Data and background expectation agree within 10% or better in all subsamples.

Fig. 6
figure 6

\(m_{\text {miss}}\) distributions in the \({{\textrm{Z}}} \rightarrow {\textrm{ee}}\) final state. The distributions are shown for protons reconstructed with (from left to right) the multi-multi, multi-single, single-multi and single-single methods, respectively. The background distributions are shown after the fit. The lower panels display the ratio between the data and the background model, with the arrows indicating values lying outside the displayed range. The expectations for a signal with \(m_{{{\textrm{X}}}} = 1000\,\textrm{GeV}\) are superimposed and normalised to 1 \(\text {pb}\)

Fig. 7
figure 7

\(m_{\text {miss}}\) distributions in the \({{\textrm{Z}}} \rightarrow \upmu \upmu \) final state. The distributions are shown for protons reconstructed with (from left to right) the multi-multi, multi-single, single-multi and single-single methods, respectively. The background distributions are shown after the fit. The lower panels display the ratio between the data and the background model, with the arrows indicating values lying outside the displayed range. The expectations for a signal with \(m_{{{\textrm{X}}}} = 1000\,\textrm{GeV}\) are superimposed and normalised to 1 \(\text {pb}\)

Fig. 8
figure 8

\(m_{\text {miss}}\) distributions in the \(\upgamma \) final state. The distributions are shown for protons reconstructed with (from left to right) the multi-multi, multi-single, single-multi and single-single methods, respectively. The background distributions are shown after the fit. The lower panels display the ratio between the data and the background model, with the arrows indicating values lying outside the displayed range. The expectations for a signal with \(m_{{{\textrm{X}}}} = 1000\,\textrm{GeV}\) are superimposed and normalised to 10 \(\text {pb}\)

We set limits on the cross section for the anomalous central exclusive reaction \({{\textrm{pp}}\rightarrow {\textrm{ppZ}}/\upgamma +{{\textrm{X}}}}\). The 95% confidence level (\(\text {CL}\)) upper limits are calculated using a modified frequentist approach with the \(\text {CL}_\text {s}\) criterion [39, 40]. An asymptotic approximation is used for the test statistic [41, 42]. The limits obtained are shown in Fig. 9. Overall the observed limits are within the 95% central interval of the expected limits, within the explored mass range. The fluctuations in the observed limit are compatible with expectations from effects due to the detector resolution. The limits translate into local p values [43] supporting the background-only hypothesis within two standard deviations.

Fig. 9
figure 9

Upper limits on the \({\textrm{pp}}\rightarrow {\textrm{ppZ}}/\upgamma +{{\textrm{X}}} \) cross section at 95% \(\text {CL}\), as a function of \(m_{{{\textrm{X}}}} \). The 68 and 95% central intervals of the expected limits are represented by the dark green and light yellow bands, respectively, while the observed limit is superimposed as a curve. The upper plots correspond to the \({{\textrm{Z}}} \rightarrow {\textrm{ee}}\) and \({{\textrm{Z}}} \rightarrow \upmu \upmu \) final states, while the lower plots correspond to the combined Z and \(\upgamma \) analyses

8 Summary

A search is presented for anomalous central exclusive \({{\textrm{Z}}}/\upgamma +{{\textrm{X}}} \) production using proton–proton (\({\textrm{pp}}\)) data samples corresponding to an integrated luminosity up to \(37.2{\,\text {fb}^{-1}} \) recorded in 2017 by the CMS detector and the CMS-TOTEM precision proton spectrometer (CT-PPS). A hypothetical X resonance is searched for in the mass region between 0.6 and 1.6 TeV, with selections optimised for the best expected significance. Benefitting from the excellent mass resolution of 2% offered by the combination of the CMS central detector and CT-PPS, for the first time at the CERN Large Hadron Collider (LHC), the missing mass distribution is used to perform a model-independent search. Upper limits on the visible cross section of the \({\textrm{pp}}\rightarrow {\textrm{ppZ}}/\upgamma +{{\textrm{X}}} \) process are set in a fiducial volume, using a generic model, in the absence of significant deviations in data with respect to the background predictions. Upper limits in the 0.025–0.089 \(\text {pb}\) range are obtained for \(\sigma ({\textrm{pp}}\rightarrow {\textrm{ppZX}})\) and 0.47–1.75 \(\text {pb}\) for \(\sigma ({\textrm{pp}}\rightarrow {\textrm{pp}}\upgamma {{\textrm{X}}})\). With these results we demonstrate the feasibility of the missing mass approach for searches at the LHC.