Search for lepton flavour violating decays of a neutral heavy Higgs boson to mu tau and e tau in proton-proton collisions at sqrt(s)=13 TeV

A search for lepton flavour violating decays of a neutral non-standard-model Higgs boson in the μτ and eτ decay modes is presented. The search is based on protonproton collisions at a center of mass energy √ s = 13 TeV collected with the CMS detector in 2016, corresponding to an integrated luminosity of 35.9 fb−1. The τ leptons are reconstructed in the leptonic and hadronic decay modes. No signal is observed in the mass range 200–900 GeV. At 95% confidence level, the observed (expected) upper limits on the production cross section multiplied by the branching fraction vary from 51.9 (57.4) fb to 1.6 (2.1) fb for the μτ and from 94.1 (91.6) fb to 2.3 (2.3) fb for the eτ decay modes. Submitted to the Journal of High Energy Physics c © 2019 CERN for the benefit of the CMS Collaboration. CC-BY-4.0 license ∗See Appendix A for the list of collaboration members ar X iv :1 91 1. 10 26 7v 1 [ he pex ] 2 2 N ov 2 01 9


Introduction
crystal electromagnetic calorimeter (ECAL), and a brass and scintillator hadron calorimeter (HCAL), each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity (η) coverage provided by the barrel and endcap detectors. Muons are detected in gas-ionization chambers embedded in the steel flux-return yoke outside the solenoid. Events of interest are selected using a two-tiered trigger system [48]. The first level, composed of custom hardware processors, uses information from the calorimeters and muon detectors to select events at a rate of around 100 kHz within a time interval of less than 4 µs. The second level, known as the high-level trigger, consists of a farm of processors running a version of the full event reconstruction software optimised for fast processing, and reduces the event rate to around 1 kHz before data storage.

Collision data and event simulation
The data used in this analysis have been collected in pp collisions at the LHC, at a centreof-mass energy of 13 TeV, with the CMS detector in 2016, and correspond to an integrated luminosity of 35.9 fb −1 [49]. A trigger requiring at least one muon is used to collect the data sample in the H → µτ h and H → µτ e channels. Triggers requiring at least one electron, or a combination of an electron and a muon are used for the H → eτ h and H → eτ µ channels respectively. Simulated samples of signal and background events are produced with different event generators. The H → µτ and H → eτ decay samples are generated with POWHEG 2.0 [50][51][52][53][54][55] at next-to-leading-order (NLO) in perturbative quantum chromodynamics. Only the gluon fusion (ggH) [56] production mode has been considered in this analysis. These scalar boson samples are generated assuming the narrow width approximation for a range of masses from 200 to 900 GeV. The Z+jets and W+jets processes are simulated using the MAD-GRAPH5 aMC@NLO 2.2.2 [57] generator at leading order with the MLM jet matching and merging [58]. The MADGRAPH5 aMC@NLO generator is also used for diboson production which is simulated at NLO with the FxFx jet matching and merging scheme [59]. The POWHEG 2.0 and 1.0 at NLO are used for top quark-antiquark (tt) and single top quark production, respectively. The POWHEG and MADGRAPH5 aMC@NLO generators are interfaced with PYTHIA 8.212 [60] for parton showering and fragmentation. The PYTHIA parameters for the underlying event description are set to the CUETP8M1 tune [61]. The set of parton distribution functions (PDFs) used is NNPDF30nloas0118 [62]. The CMS detector response is modelled using GEANT4 [63].
Because of the high instantaneous luminosities attained during data taking, events have multiple pp interactions per bunch crossing (pileup). This effect is taken into account in simulated samples, by generating concurrent minimum bias events, and overlapping them with simulated hard events. All simulated samples are weighted to match the pileup distribution observed in data, which has an average of approximately 23 interactions per bunch crossing.

Event reconstruction
The event reconstruction is performed using a particle-flow (PF) algorithm, which aims to reconstruct and identify each individual particle in an event (PF candidate), with an optimised combination of information from the various elements of the CMS detector [64]. In this process, the identification of the particle type for each PF candidate (photon, electron, muon, charged or neutral hadron) plays an important role in the determination of the particle direction and energy. The primary pp vertex of the event is identified as the reconstructed vertex with the largest value of summed physics-object p 2 T , where p T is the transverse momentum. The physics objects are the jets, clustered using the jet finding algorithm [65,66] with the tracks assigned to the vertex as inputs, and the associated missing transverse momentum, taken as the negative vector sum of the p T of those jets.
A muon is identified as a track in the silicon detectors, consistent with the primary pp vertex and with either a track or several hits in the muon system, associated with an energy deposit in the calorimeters compatible with the expectations for a muon [64,67]. Identification is based on the number of spatial points measured in the tracker and in the muon system, the track quality, and its consistency with the event vertex location. The identification working point chosen for this analysis reconstructs muons with an efficiency above 98%. The energy is obtained from the corresponding track momentum. An important aspect of muon reconstruction is the lepton isolation that is described later in this section.
An electron is identified as a charged-particle track from the primary pp vertex in combination with one or more ECAL energy clusters. These clusters are matched with the track extrapolation to the ECAL and with possible bremsstrahlung photons emitted when interacting with the material of the tracker [68]. Electron candidates are accepted in the pseudorapidity range |η| < 2.5, with the exception of the region 1.44 < |η| < 1.57 where service infrastructure for the detector is located. They are identified using a multivariate-analysis (MVA) discriminator that combines observables sensitive to the amount of bremsstrahlung along the electron trajectory, the geometric and momentum matching between the electron trajectory and associated clusters, as well as various shower shape observables in the calorimeters. Electrons from photon conversions are removed. The chosen working point for selecting the electrons assures an average identification efficiency of 80% with a misidentification probability of 5%. The energy of electrons is determined from a combination of the track momentum at the primary vertex, the corresponding ECAL cluster energy, and the energy sum of all bremsstrahlung photons associated with the track.
Charged hadrons are identified as charged-particle tracks neither identified as electrons, nor as muons. Finally, neutral hadrons are identified as HCAL energy clusters not linked to any charged hadron trajectory, or as a combined ECAL and HCAL energy excess with respect to the expected charged hadron energy deposit. All the PF candidates are clustered into hadronic jets using the infrared-and collinear-safe anti-k T algorithm [65], implemented in the FASTJET package [69], with a distance parameter of 0.4. Jet momentum is determined as the vectorial sum of all particle momenta in the jet, and is found from simulation to be, on average, within 5 to 10% of the true momentum over the whole p T spectrum and detector acceptance. Additional proton-proton interactions within the same or nearby bunch crossings can contribute additional tracks and calorimetric energy depositions, increasing the apparent jet momentum. To mitigate this effect, tracks identified to be originating from pileup vertices are discarded and an offset correction is applied to correct for remaining contributions. Jet energy corrections are derived from simulation studies so that the average measured response of jets becomes identical to that of particle level jets. In situ measurements of the momentum balance in dijet, photon+jet, Z +jets, and multijet events are used to determine any residual differences between the jet energy scale in data and in simulation, and appropriate corrections are made [70]. Additional selection criteria are applied to each jet to remove jets potentially dominated by instrumental effects or reconstruction failures.
Hadronically decaying τ leptons (τ h ) are reconstructed and identified using the hadrons-plusstrips algorithm [71,72]. The reconstruction starts from a jet and searches for the products of the main τ lepton decay modes: one charged hadron and up to two neutral pions, or three charged hadrons. To improve the reconstruction efficiency in the case of conversion of the photons from a neutral-pion decay, the algorithm considers the PF photons and electrons from and M vis is the visible mass of the τ − e or τ − µ system.

Event selection
The event selection is performed in two steps. An initial selection is followed by another, final, set of requirements on kinematic variables that exploit the distinct event topology of the signal. The event sample defined by the initial selection is used in the background estimation described in Section 6. The event selection begins by requiring two isolated leptons of opposite charge, different flavour, and separated by ∆R > 0.3. The isolation of the τ h candidates is included in the MVA discriminator described in Section 4. Events with additional µ, e, or τ h candidates respectively with p T > 10, 5, or 20 GeV are discarded. The kinematic requirements applied are dictated by the triggers or detector acceptance and are summarized in Table 1.
The events are then divided into two categories according to the number of jets in the event. The jets are required to have p T > 30 GeV and |η| < 4.7. Events with no jets form the 0-jet category while events with exactly one jet form the 1-jet category. The 1-jet category includes ggH production with initial state radiation. Events with more than one jet are discarded.
The final selection is given in Table 2. It begins by tightening the p T requirement of the prompt lepton from the Higgs boson decay, as it provides a powerful discriminant against the background. The τ lepton in the H decay is highly boosted, leading to a collimation of the decay products. This can be exploited by either limiting the azimuthal separation of the decay products including the p miss T , or imposing a requirement on the transverse mass m T (τ), which is strongly correlated with the azimuthal separation. These selection criteria are optimised for each decay mode in two m H ranges to obtain the most stringent expected upper limits. The low-and high-mass regions are defined to be 200 < m H < 450 GeV and 450 < m H < 900 GeV, respectively. A binned likelihood fit to the M col distributions is then used to extract signal and background contributions. The M col approximates the Higgs mass better than the widely used M vis , and therefore improves the separation of the signal from the background.

Background estimation
The most significant background in the µτ h and eτ h channels comes from the W+jets process and from events comprised uniquely of jets produced through the strong interaction, referred to as quantum chromodynamics (QCD) multijet events. In these processes, jets are misidentified as electrons, muons or τ leptons. This background is estimated with the collected data. The main background in the µτ e and eτ µ channels is tt production. It is estimated using simulations. Other smaller backgrounds include electroweak diboson (WW, WZ, and ZZ), Drell-Yan (DY)→ ( = e, µ) + jets, DY→ ττ + jets, SM Higgs boson (H → ττ, WW), Wγ ( * ) +jets, and single top quark production processes. These are estimated using simulations. Gluon fusion, vector boson fusion and associated production mechanisms are considered for the SM Higgs boson background. The background estimation techniques are described in detail below, and are validated with control regions that are enhanced with the dominant backgrounds. OA The DY→ , ττ background is estimated from simulation. A reweighting is applied to the generator-level Z boson p T and invariant mass, m ,τ τ , distributions to correct for a shape discrepancy between data and simulation. The reweighting factors, extracted from a control region enriched in Z → µµ events, are applied in bins of Z boson p T and m ,τ τ as explained in [44]. Additional corrections for e → τ h and µ → τ h misidentification rates are applied to the simulated DY sample when the reconstructed τ h candidate is matched to an electron for the H → eτ h channel or a muon for the H → µτ h channel, respectively, at the generator level. These corrections depend on the lepton η and are measured in Z → data events.
The tt background is also estimated using simulation. The overall normalisation of this estimate in the signal region is corrected with a rescaling factor derived from a control region enriched in tt events, defined by requiring the initial selection with the additional requirement that at least one of the jets is b tagged. Figure 1 (upper left) shows the data compared to the background estimate in the tt-enriched region in the H → µτ e channel.
Jets from W +jets and QCD multijet events that are misidentified as electrons, muons and, mainly, τ leptons, are leading source of background in the µτ h and eτ h channels. In W +jets events, one lepton candidate is expected to be a genuine lepton from the W decay and the other a jet misidentified as a lepton. In QCD multijet events, both lepton candidates are misidentified jets. A technique fully based on control samples in data is used to estimate the misidentified lepton background in the µτ h and eτ h channels, for which it is the dominant contribution. In the µτ e and eτ µ channels, this background is estimated using a combination of simulated samples and control regions in data. These methods have been used in Refs. [37] and [44], and a detailed description can be found in those publications. However, we are briefly describing the techniques in the following subsections.

Misidentified lepton background estimation from control samples in data
The misidentified-lepton background is estimated from data. The misidentification probabilities, f i , where i = µ, e or τ h , are evaluated with independent Z+jets data sets and then applied to a control sample. The control sample is obtained by relaxing the signal selection requirements, the µ, e or τ h isolation, and excluding events passing the signal selection. The f i are estimated using events with a Z boson candidate and one jet that can be misidentified as µ, e, or τ. The Z boson candidate is formed requiring two muons with p T > 26 GeV, |η| < 2.4, and I µ rel < 0.15. The muons are required to have opposite charges and the dimuon invariant mass, m µµ , must satisfy 76 < m µµ < 106 GeV. The contribution from diboson events, where the third lepton candidate corresponds to a genuine µ, e, or τ, is subtracted using simulation. Two Z+jets samples are defined: a signal-like one, in which the jet satisfies the same µ, e, or τ selection criteria used in the H → µτ or H → eτ selections, and a background-enriched Z+jets sample with relaxed identification on the jet misidentified as µ, e, or τ, but excluding events selected in the signal-like sample. The requirements on the third candidate, i.e. the misidentified  jet, depend on the lepton flavour. The two samples are used to estimate f i as where N i (Z+jets signal-like) is the number of events with a third candidate (µ, e, or τ) that passes the signal-like sample selection, and N i (Z+jets background-enriched) is the number of events in the background-enriched sample. The background-enriched selection used to estimate the misidentified µ and e contribution requires an isolation of 0.15 < I µ rel < 0.25 and 0.1 < I e rel < 0.5, respectively. In both cases the misidentification rate is computed and applied as a function of the lepton p T . The lepton selection for the τ h background-enriched sample requires that the τ h lepton candidates are identified using a loose τ h identification and isolation working point but are not identified by the tight working point used for the signal selection. The loose and tight working points have an efficiencies of 70 and 50% for genuine τ h candidates, respectively.
The τ h misidentification rates have a p T dependence that varies with the number of charged pions in the decay. They are estimated and applied as a function of p T and for either one or three charged pions in the decay. The misidentified background in the signal sample is obtained from control samples for each lepton flavour. The selection requirements for these samples are the same as for the signal sample except that the µ, e, or τ should pass the identification and isolation criteria used for the Z+jets background-enriched sample, but not those defining the Z+jets signal-like sample. To estimate the misidentified background in the signal sample, each event in this background enriched sample is weighted by a factor f i /(1 − f i ). The background from misidentified muons and electrons is estimated to be less than 5% of the misidentified τ h lepton background and is neglected.
The background estimate is validated in a like-sign sample by applying the misidentification rate f i to events selected by requiring the µ, e, or τ in the pairs having the same charge in both the background-enriched and the signal-like samples. This validation is performed after the initial selection described in Section 5. Figure 1 (upper right) shows the data compared to the background estimate in the like-sign control region for the H → µτ h channel. The like-sign selection enhances the misidentified-lepton background, and this sample is expected to be composed of a similar fraction of W +jets and QCD multijet events. The background estimate is also validated in a W boson enriched control sample. This data sample is obtained by applying the signal sample requirements and m T cuts, 50 < m T ( ) < 110 GeV ( = µ or e) and m T (τ) > 50 GeV. The misidentified background in the signal region and W boson enriched control sample are both dominated by W +jets events, with QCD multijet events forming a small fraction of the samples. Figure 1 (lower) shows the data compared to the background estimate in the W+jets enriched sample for the H → µτ h channel. The background expectation for the H → eτ h channel is also validated with the same samples and gives similar agreement.

W+jets and QCD background estimation in µτ e and eτ µ channels
The W +jets background contribution to the misidentified background is estimated with simulations. The QCD multijet contribution is estimated with like-sign data events that pass all the other signal requirements. The remaining non-QCD background is estimated using simulation. The resulting sample is then rescaled to account for the differences between the background composition in the like and opposite sign samples. The scaling factors are extracted from QCD multijet enriched control samples, composed of events where the lepton candidates satisfy inverted isolation requirements, as explained in Ref. [44].

Systematic uncertainties
Systematic uncertainties arise from both experimental and theoretical sources and can affect the normalisation and the shape of the collinear mass distribution. They are summarized in Table 3. Table 3: The systematic uncertainties for the four channels. All uncertainties are treated as correlated between the categories, except those with more values separated by the ⊕ symbol. In the case of two values, the first value is the correlated uncertainty and the second value is the uncorrelated uncertainty for each individual category. In the case of three values, the first and second values correspond to the uncertainties arising from factorisation and renormalisation scales and PDF variations and are correlated between categories, while the third value is the uncorrelated uncertainty for each individual category. Two values separated by the "-" sign represent the range of the uncertainties from the different sources and/or in the different jet categories. The uncertainties in the muon, electron and τ lepton selection including the trigger, identification (ID), and isolation efficiencies are estimated from collision data sets of Z bosons decaying to ee, µµ, τ µ τ h [67,68,72]. They result in changes of normalisation, with the exception of the uncertainty on high p T τ lepton efficiency that changes both yield and M col distribution shape. The b tagging efficiency is measured in collision data, and the simulation is adjusted accordingly to match with it. The uncertainty in this measurement is taken as the systematic error affecting the normalisation of the simulation [75].
The uncertainties in the estimate of the misidentified-lepton backgrounds (µ → τ h , e → τ h , jet → τ h , µ, e) are extracted from the validation tests in control samples, as described in Sec-tion 6; they affect both the normalisation and the shape of the M col distribution. The uncertainty in the QCD multijet background yield is 30%, and corresponds to the uncertainty in the extrapolation factor from the same-sign to the opposite-sign region. The uncertainties in the background contributions from Z, WW, ZZ, Wγ, tt and single-top quark arise predominantly from those in the measured cross sections of these processes and are applied as uncertainties in sample normalisation.
The uncertainties in the Higgs boson production cross sections due to the factorisation and the renormalisation scales, as well as the PDFs and the strong coupling constant (α S ), result in changes in normalisation. They are taken from Ref. [78] and summarized in Table 3 for the SM Higgs boson and Table 4 for heavy Higgs bosons. Only effects on the total rate are considered. Effects on the acceptance have been neglected. Shape and normalization uncertainties arising from the uncertainty in the jet energy scale are computed by propagating the effect of altering each source of jet energy scale uncertainty by ±1 standard deviation to the fit templates of each process. There are 27 independent sources of jet energy scale uncertainty, fully correlated between categories and τ lepton decay channels.
The uncertainty in the τ h energy scale is treated equally for the two independent channels: H → µτ h and H → eτ h . It is propagated to the collinear mass distributions. Also, the uncertainty in the energy scale of electrons and muons misidentified as τ h is propagated to the M col distributions and produces changes in the distribution shape and normalization. Systematic uncertainties in the electron energy scale and resolution include the effects of electron selection efficiency, pseudorapidity dependence and categorisation, summed in quadrature. They impact both the normalization and shape of the M col distribution. The systematic uncertainties in the energy resolution have negligible effect. The uncertainty in muon energy scale and resolution is also treated in the same manner. Scale uncertainties on the energy from jets with p T below 15 GeV and PF candidates not clustered inside jets (unclustered energy scale uncertainty) are also considered [76]. They are estimated independently for four particle categories: charged particles, photons, neutral hadrons, and very forward particles which are not contained in jets. The effect of shifting the energy of each particle by its uncertainty is propagated to p miss T and leads to both changes in shape of the distribution and in overall predicted yields. The different systematic uncertainties from the four particle categories, for the unclustered energy scale, are considered uncorrelated.
The bin-by-bin uncertainties [79] account for the statistical uncertainties in every bin of the template distributions of every process. They are uncorrelated between bins, processes, and categories.
Shape uncertainties related to the pileup have been considered by varying the weights applied to simulation. This weight variation is obtained changing by 5% the total inelastic cross section used in the estimate of the pileup events in data [80]. The new values are then applied, event by event, to produce alternative collinear mass distributions used as shape uncertainties in the fit. Other minimum bias event modelling and simulation uncertainties are estimated to be much smaller and are therefore neglected. The uncertainty on the integrated luminosity affects all processes with normalization taken directly from simulation.

Results
After all selection criteria have been applied, a binned maximum likelihood fit is performed on the M col distributions in the range 0-1400 GeV, looking for a signal-like excess on top of the estimated background. No fit on the control region is performed. The systematic uncertainties are represented by nuisance parameters, assuming a log normal prior for normalisation parameters, and Gaussian priors for M col shape uncertainties. The uncertainties that affect the shape of the M col distribution, mainly those corresponding to the energy scales, are represented by nuisance parameters whose variation results in a modification of the distribution [81,82]. A profile likelihood ratio is used as test statistic. The 95% CL upper limits on the H production cross section times branching fraction to LFV lepton channels, σ(gg → H)B(H → µτ) and σ(gg → H)B(H → eτ), are set using the CL s criterion [83,84] and the asymptotic approximation of the distributions of the LHC test-statistic [85], in a combined fit to the M col distributions. The limits are also computed per channel and category. The upper limits are derived in the analysed mass range in steps of 50 GeV. Where simulated samples are not available, shapes and yields are interpolated from the neighbouring samples with a moment morphing algorithm that derive the M col distribution from the two closest simulated mass points.

H → µτ results
The distributions of the collinear mass M col compared to the signal and background contributions in the H → µτ h and H → µτ e channels, in each jet category, are shown in Figs. 2 and 3. No excess over the background expectation is observed. The observed and median expected 95% CL upper limits on σ(gg → H)B(H → µτ) range from 51.9 (57.4) fb to 1.6 (2.1) fb, and are given for each category in Table 5.The limits are also summarized graphically in Fig. 4 for the individual categories, and in Fig. 5 for the combination of the two τ decay channels.

H → eτ results
The distributions of the collinear mass M col compared to the signal and background contributions in the H → eτ h and H → eτ µ channels, in each category, are shown in Figs. 6 and 7. No excess over the background expectation is observed. The observed and median expected 95% CL upper limits on σ(gg → H)B(H → eτ) range from 94.1 (91.6) fb to 2.3 (2.3) fb, and are given for each category in Table 6. The limits are also summarized graphically in Fig. 8 for the individual categories, and in Fig. 9 for the combination of both two τ decay channels.

Summary
The first direct search for lepton flavour violating decays of a neutral non-standard-model Higgs boson (H) in the µτ and eτ channels is presented in this paper. The analyzed data set corresponds to an integrated luminosity of 35.9 fb −1 of proton-proton collision data recorded at √ s = 13 TeV. The results are extracted from a fit to the collinear mass distributions. No

CMS
, combined τ e evidence is found for lepton flavour violating decays of H in the investigated mass range. The observed (expected) upper limits at 95% confidence level on the product of production cross section with branching fraction, for H mass in the range 200-900 GeV, decaying to µτ and eτ vary from 51.9 (57.4) fb to 1.6 (2.1) fb and from 94.1 (91.6) fb to 2.3 (2.3) fb, respectively.

Acknowledgments
We congratulate our colleagues in the CERN accelerator departments for the excellent performance of the LHC and thank the technical and administrative staffs at CERN and at other CMS institutes for their contributions to the success of the CMS effort. In addition, we gratefully acknowledge the computing centres and personnel of the Worldwide LHC Computing Grid for delivering so effectively the computing infrastructure essential to our analyses. Finally, we acknowledge the enduring support for the construction and operation of the LHC and the CMS detector provided by the following funding agencies: Fellowship, Chulalongkorn University and the Chulalongkorn Academic into Its 2nd Century Project Advancement Project (Thailand); the Nvidia Corporation; the Welch Foundation, contract C-1845; and the Weston Havens Foundation (USA).