Constraints on anomalous Higgs boson couplings from its production and decay using the WW channel in proton–proton collisions at $\sqrt{s} = 13~\text {TeV}$

Abstract

A study of the anomalous couplings of the Higgs boson to vector bosons, including ${\textit{CP}}$-violation effects, has been conducted using its production and decay in the WW channel. This analysis is performed on proton–proton collision data collected with the CMS detector at the CERN LHC during 2016–2018 at a center-of-mass energy of 13 TeV, and corresponds to an integrated luminosity of 138$\,\text {fb}^{-1}$. The different-flavor dilepton $({\textrm{e}} {{\upmu }})$ final state is analyzed, with dedicated categories targeting gluon fusion, electroweak vector boson fusion, and associated production with a W or Z boson. Kinematic information from associated jets is combined using matrix element techniques to increase the sensitivity to anomalous effects at the production vertex. A simultaneous measurement of four Higgs boson couplings to electroweak vector bosons is performed in the framework of a standard model effective field theory. All measurements are consistent with the expectations for the standard model Higgs boson and constraints are set on the fractional contribution of the anomalous couplings to the Higgs boson production cross section.

1 Introduction

After the discovery of the Higgs boson (H) by the ATLAS and CMS Collaborations in 2012 [1,2,3], the CMS [4,5,6,7,8,9,10,11] and ATLAS [12,13,14,15,16,17,18] experiments set constraints on the spin-parity properties of the Higgs boson and its couplings with gluons and electroweak (EW) gauge bosons, denoted here as Hgg and HVV, respectively. The Higgs boson quantum numbers are consistent with the standard model (SM) expectation $J^{PC} = 0^{++},$ but the possibility of small, anomalous couplings is not yet ruled out. In beyond-the-SM (BSM) theories, interactions with the Higgs boson may occur through several anomalous couplings, which lead to new tensor structures in the interaction terms that can be both ${\textit{CP}}$-even or ${\textit{CP}}$-odd. The ${\textit{CP}}$-odd anomalous couplings between the Higgs boson and BSM particles may generate ${\textit{CP}}$ violation in the interactions of the Higgs boson.

In this paper, we study the tensor structure of the Hgg and HVV couplings, and we search for several anomalous effects, including ${\textit{CP}}$ violation, using the different-flavor dilepton $({\textrm{e}} {{\upmu }})$ final state from $\textrm{H}\rightarrow \textrm{WW}$ decays. The Higgs boson production processes include gluon fusion (ggH), EW vector boson fusion (VBF), and associated production with a W or Z boson (VH). Higgs boson production and decay processes are sensitive to certain anomalous contributions, which can be described by higher-dimensional operators in an effective field theory (EFT) [19] that can modify the kinematic distributions of the Higgs boson decay products and the particles from associated production.

Each production process of the Higgs boson is identified using its kinematic features, and events are assigned to corresponding production categories. The matrix element likelihood approach (MELA) [20,21,22,23,24] is employed to construct observables that are optimal for the measurement of anomalous couplings, or EFT operators, at the production vertex. These and other decay-based variables are used to explore all kinematic features of the events, giving the analysis sensitivity to simultaneous anomalous effects at the Higgs boson production and decay vertices. Fully simulated signal samples that include anomalous couplings incorporate the detector response into the analysis.

The analysis is based on the proton–proton $(\textrm{pp})$ collision data collected at the CERN LHC from 2016 to 2018, at a center-of-mass energy of 13 TeV, corresponding to an integrated luminosity of 138$\,\text {fb}^{-1}$. This paper builds on a previous analysis conducted by the CMS Collaboration in the $\textrm{H}\rightarrow \textrm{WW}$ channel [25], which focused on measuring the Higgs boson production cross sections and coupling parameters in the so-called $\kappa$ framework [26]. We follow a formalism used in previous CMS analyses of anomalous couplings in Run 1 and Run 2 [4,5,6,7,8,9,10,11, 27, 28], focusing on the case where the Higgs boson is produced on-shell. The coupling parameters are extracted using the signal strength and the fractional contributions of the couplings to the cross section. A general study of the HVV interaction is performed with four anomalous couplings analyzed individually. Through SU(2) x U(1) symmetry considerations, the anomalous HVV couplings are reduced in number to three and analyzed simultaneously. The primary HVV coupling measurements are performed in terms of cross section fractions with additional interpretations in terms of EFT couplings included. A study of the Hgg interaction is also performed in terms of a ${\textit{CP}}$-odd anomalous coupling cross section fraction.

This paper is organized as follows. The phenomenology of anomalous couplings is discussed in Sect. 2. Section 3 gives a brief overview of the CMS apparatus. Data sets and Monte Carlo (MC) simulation samples are discussed in Sect. 4. The event reconstruction and selection are outlined in Sects. 5 and 6, respectively. Methods to estimate backgrounds are given in Sect. 7. In Sect. 8, we discuss the kinematic variables associated with Higgs boson production and decay. Sources of systematic uncertainties are presented in Sect. 9. The results are presented and discussed in Sect. 10. Finally, a summary is given in Sect. 11. Tabulated results are provided in the HEPData record for this analysis [29].

2 Phenomenology

In this analysis, we investigate anomalous coupling effects in gluon fusion or electroweak Higgs boson production, as well as in its decay to WW pairs. A detailed discussion of the theoretical considerations can be found in Refs. [22, 24, 28]. The interaction of the spin-zero Higgs boson with two spin-one gauge bosons such as WW, $\textrm{ZZ}$, ${\textrm{Z}} {{\upgamma }},$ ${{\upgamma }}{{\upgamma }},$ or gg, can be parametrized by the scattering amplitude

(1)

where and are the spin-one gauge boson four-momentum and polarization vectors, $m_{\textrm{V}1}$ is the pole mass of the boson, and ${\tilde{f}}^{(i)}_{\mu \nu } = \frac{1}{2} \epsilon _{\mu \nu \rho \sigma } f^{(i),\rho \sigma }$ (with $\epsilon _{\mu \nu \rho \sigma }$ the Levi-Civita symbol), is the scale of BSM physics, and v is the Higgs field vacuum expectation value.

The only leading tree-level contributions in the scattering amplitude are $a_{1}^\textrm{ZZ}\ne 0$ and $a_{1}^{\textrm{WW}} \ne 0;$ other $a_{1}$ coupling parameters $({\textrm{Z}} {{\upgamma }},$ ${{\upgamma }}{{\upgamma }},$ gg) do not contribute because the pole mass vanishes. Additional $\textrm{ZZ}$ and WW couplings are considered anomalous contributions. Anomalous terms arising in the SM via loop effects are typically small and are not yet accessible experimentally. The BSM contributions, however, could yield larger coupling parameters. Among the anomalous contributions, considerations of symmetry and gauge invariance require $\kappa _1^\textrm{ZZ} = \kappa _2^\textrm{ZZ},$ $\kappa _1^{{{\upgamma }}{{\upgamma }}} = \kappa _2^{{{\upgamma }}{{\upgamma }}} = 0,$ $\kappa _1^{\textrm{gg}} = \kappa _2^{\textrm{gg}} = 0,$ and $\kappa _1^{{\textrm{Z}} {{\upgamma }}} = 0$ [24]. The presence of ${\textit{CP}}$-odd couplings together with any of the other couplings (all of them ${\textit{CP}}$-even), will result in ${\textit{CP}}$ violation. We reduce the number of independent parameters by assuming that $a_2^{{{\upgamma }}{{\upgamma }}},$ $a_3^{{{\upgamma }}{{\upgamma }}},$ $a_2^{{\textrm{Z}} {{\upgamma }}}$ and $a_3^{{\textrm{Z}} {{\upgamma }}}$ are constrained in direct decays of ${\textrm{H}} \rightarrow {{\upgamma }}{{\upgamma }}$ and ${\textrm{Z}} {{\upgamma }},$ therefore fixing them to be zero. The $a^{\textrm{gg}}_{2}$ term results from loop effects in the SM.

The relationship between the $\textrm{ZZ}$ and WW couplings is mostly relevant for VBF production. There are no kinematic differences between the $\textrm{ZZ}$ and WW fusion processes; therefore, it is not possible to disentangle the couplings. One possibility is to set the $\textrm{ZZ}$ and WW couplings to be equal, $a_{i} = a_{i}^\textrm{ZZ} = a_{i}^{\textrm{WW}},$ leaving four HVV anomalous couplings to be measured: $a_2,$ $a_3,$ $\kappa _1/(\varLambda _1)^2,$ and $\kappa _2^{{\textrm{Z}} {{\upgamma }}}/(\varLambda _1^{{\textrm{Z}} {{\upgamma }}})^2.$ The $a_{1}^\textrm{ZZ} = a_{1}^{\textrm{WW}}$ relationship also appears under custodial symmetry. This approach provides a general test of the Higgs boson Lagrangian tensor structure and a search for ${\textit{CP}}$ violation in HVV interactions. In an alternative approach, the SU(2) $\times$ U(1) symmetry reduces the number of independent HVV anomalous couplings to three $(a_2,$ $a_3,$ and $\kappa _1/(\varLambda _1)^2)$ through the introduction of the following coupling parameter relationships [19] :

$$\begin{aligned} a_1^{\textrm{WW}}&= a_1^{\textrm{ZZ}}, \end{aligned}$$

(2)

$$\begin{aligned} a_2^{\textrm{WW}}&= c_\text {w} ^2 a_2^{\textrm{ZZ}}, \end{aligned}$$

(3)

$$\begin{aligned} a_3^{\textrm{WW}}&= c_\text {w} ^2 a_3^{\textrm{ZZ}}, \end{aligned}$$

(4)

$$\begin{aligned} \frac{\kappa _1^{\textrm{WW}}}{(\varLambda _1^{\textrm{WW}})^2}&= \frac{1}{c_\text {w} ^2-s_\text {w} ^2} \left( \frac{\kappa _1^{\textrm{ZZ}}}{(\varLambda _1^{\textrm{ZZ}})^2} -2 s_\text {w} ^2 \frac{a_2^{\textrm{ZZ}}}{m_{\textrm{Z}} ^2}\right) , \end{aligned}$$

(5)

$$\begin{aligned} \frac{\kappa _2^{{\textrm{Z}} {{\upgamma }}}}{(\varLambda _1^{{\textrm{Z}} {{\upgamma }}})^2}&= \frac{ 2 s_\text {w} c_\text {w}}{c_\text {w} ^2-s_\text {w} ^2} \left( \frac{\kappa _1^{\textrm{ZZ}}}{(\varLambda _1^{\textrm{ZZ}})^2} - \frac{a_2^{\textrm{ZZ}}}{m_{\textrm{Z}} ^2}\right) , \end{aligned}$$

(6)

where $c_\text {w}$ and $s_\text {w}$ are the cosine and sine of the weak mixing angle, respectively, and $m_{\textrm{Z}}$ is the ${\textrm{Z}}$ boson mass. With this approach, there is a linear relationship between the scattering amplitude couplings and the SM EFT (SMEFT) couplings in the Higgs basis [19]:

$$\begin{aligned} \delta c_{\text {z}}&= \frac{1}{2} a_1^{\textrm{ZZ}} - 1, \end{aligned}$$

(7)

$$\begin{aligned} c_{\text {zz}}&= -\frac{2 s_\text {w} ^2 c_\text {w} ^2}{e^2} a_2^{\textrm{ZZ}}, \end{aligned}$$

(8)

$$\begin{aligned} \tilde{c}_{\text {zz}}&= -\frac{2 s_\text {w} ^2 c_\text {w} ^2}{e^2} a_3^{\textrm{ZZ}}, \end{aligned}$$

(9)

$$\begin{aligned} c_{\text {z} \Box }&= \frac{m_{\textrm{Z}} ^2 s_\text {w} ^2}{e^2} \, \frac{\kappa _1^{\textrm{ZZ}}}{(\varLambda _1^{\textrm{ZZ}})^2}, \end{aligned}$$

(10)

where e is the electron charge. The amplitude couplings may also be related to the SMEFT Warsaw basis [19, 30] couplings through the following translation [28, 31] :

$$\begin{aligned} \delta a_1^{\textrm{ZZ}}&= \frac{v^2}{\varLambda ^2} \left( 2c_{\text {H}\Box } + \frac{6e^2}{s_\text {w} ^2}c_{\text {HWB}} + \left( \frac{3c_\text {w} ^2}{2s_\text {w} ^2} -\frac{1}{2}\right) c_{\text {HD}} \right) , \end{aligned}$$

(11)

$$\begin{aligned} \kappa _1^{\textrm{ZZ}}&= \frac{v^2}{\varLambda ^2} \left( -\frac{2e^2}{s_\text {w} ^{2}}c_{\text {HWB}} + \left( 1-\frac{1}{2s_\text {w} ^2}\right) c_{\text {HD}} \right) , \end{aligned}$$

(12)

$$\begin{aligned} a_2^{\textrm{ZZ}}&= -2\frac{v^2}{\varLambda ^2} \left( s_\text {w} ^2 c_{\text {HB}} + c_\text {w} ^2 c_{\text {HW}} + s_\text {w} c_\text {w} c_{\text {HWB}} \right) , \end{aligned}$$

(13)

$$\begin{aligned} a_3^{\textrm{ZZ}}&= -2\frac{v^2}{\varLambda ^2} \left( s_\text {w} ^2 c_{\text {H}\tilde{\text {B}}} + c_\text {w} ^2 c_{\text {H}\tilde{\text {W}}} + s_\text {w} c_\text {w} c_{\text {H}\tilde{\text {W}}\text {B}} \right) , \end{aligned}$$

(14)

where $\varLambda$ is the UV cutoff of the theory (set to 1 TeV), and $\delta a_1^{\textrm{ZZ}}$ is a correction to the SM value of $a_1^{\textrm{ZZ}}.$ Further discussion on the EFT operators corresponding to the couplings considered here may be found in Chapter 2.2 of Ref. [19]. The assumed constraints on $a_2^{{{\upgamma }}{{\upgamma }}},$ $a_3^{{{\upgamma }}{{\upgamma }}},$ $a_2^{{\textrm{Z}} {{\upgamma }}}$ and $a_3^{{\textrm{Z}} {{\upgamma }}}$ imply that only one of the three coupling parameters $c_{\text {HW}},$ $c_{\text {HWB}},$ and $c_{\text {HB}}$ is independent; the same is also true for their ${\textit{CP}}$-odd counterparts $c_{\text {H}\tilde{\text {W}}},$ $c_{\text {H}\tilde{\text {W}}\text {B}},$ and $c_{\text {H}\tilde{\text {B}}}.$ Therefore, we have four independent HVV couplings in both the Higgs and Warsaw basis. All the EFT couplings are expected to be zero in the SM.

We thus adopt two approaches to the HVV coupling study. In Approach 1, we use the $a_{i}^\textrm{ZZ} = a_{i}^{\textrm{WW}}$ relationship and individually analyze each of the four anomalous couplings. In Approach 2, we enforce the SU(2) x U(1) relationships from Eqs. (2–6) and analyze the three independent anomalous couplings both individually and simultaneously. Approach 1 may be considered to follow the relationships from Eqs. (2–5) in the limiting case $c_\text {w} = 1.$

It is convenient to measure the fractional contribution of the anomalous couplings to the Higgs boson cross section rather than the anomalous couplings themselves. For the anomalous HVV couplings, the effective fractional cross sections $f_{ai}$ are defined as

(15)

where $\sum _{j}$ sums over all the coupling parameters considered, including $a_1,$ and $\sigma _{i}$ is the cross section for the process corresponding to $a_{i} = 1$ and $a_{j \ne i} = 0.$ Many systematic uncertainties cancel out in the ratio, and the physical range is conveniently bounded between $-1$ and $+1.$ Our primary measurements are performed in terms of cross section fractions, with additional interpretations in terms of the SMEFT Higgs and Warsaw basis couplings also included. For consistency with previous CMS measurements, the $\sigma _i$ coefficients used to define the fractional cross sections correspond to the $\textrm{gg}\rightarrow {\textrm{H}} \rightarrow \textrm{VV} \rightarrow 2{\textrm{e}} 2{{\upmu }}$ process [28]. The numerical values are given in Table 1 as calculated using the JHUGen simulation [20,21,22,23]. Two sets of values are shown corresponding to the different coupling relationships adopted in Approach 1 and 2.

It has been shown that the angular correlations of the associated jets in the ggH + 2 jets process are sensitive to anomalous Hgg coupling effects at the production vertex [32]. The quark-quark initiated process, $\textrm{qq} \rightarrow \textrm{qqH},$ corresponds to the gluon scattering topology sensitive to anomalous effects. For the anomalous Hgg coupling, the effective fractional cross section can be defined as

(16)

The $\sigma _3^\text {gg}$ and $\sigma _{2}^{\textrm{gg}}$ cross sections correspond to $a_3^{\textrm{gg}} = 1, a_2^{\textrm{gg}} = 0$ and $a_2^{\textrm{gg}} = 1, a_3^{\textrm{gg}} = 0,$ respectively, and are equal. With this analysis it is not possible to distinguish the top quark, bottom quark, and heavy BSM particle contributions in the gluon fusion loop. As such, the Hgg coupling is treated as an effective coupling with heavy degrees of freedom integrated out.

Table 1 The cross sections $(\sigma _i)$ of the anomalous contributions $(a_i)$ relative to the SM value $(\sigma _1)$ used to define the fractional cross sections $f_{ai}$ for the Approach 1 and 2 coupling relationships. For the $\kappa _{1}$ and $\kappa _{2}^{{\textrm{Z}} {{\upgamma }}}$ couplings, the numerical values $\varLambda _1 = \varLambda _1^{{\textrm{Z}} {{\upgamma }}} = 100\,\text {Ge}\hspace{-.08em}\text {V}$ are chosen to keep all coefficients of similar order of magnitude

3 The CMS detector

The CMS apparatus [33] is a multipurpose, nearly hermetic detector, designed to identify electrons, muons, photons, and (charged and neutral) hadrons [34,35,36,37]. A global reconstruction “particle-flow” (PF) algorithm [38] combines the information provided by the all-silicon inner tracker and by the crystal electromagnetic and brass-scintillator hadron calorimeters, operating inside a 3.8$\,\text {T}$ superconducting solenoid, with data from gas-ionization muon detectors interleaved with the solenoid return yoke, to build $\uptau$leptons, jets, missing transverse momentum, and other physics objects [39,40,41].

Events of interest are selected using a two-tiered trigger system [42, 43]. The first level (L1), composed of custom hardware processors, uses information from the calorimeters and muon detectors to select events at a rate of around 100$\,\text {kHz}$ within a fixed latency of about 4$\,\upmu \text {s}$ [42]. The second level, known as the high-level trigger (HLT), consists of a farm of processors running a version of the full event reconstruction software optimized for fast processing, and reduces the event rate to around 1$\,\text {kHz}$ before data storage [43]. A more detailed description of the CMS detector, together with a definition of the coordinate system and kinematic variables, can be found in Ref. [33].

4 Data sets and simulation

The data sets included in this analysis were recorded with the CMS detector in 2016, 2017, and 2018, and correspond to integrated luminosities of 36.3, 41.5, and 59.7$\,\text {fb}^{-1}$, respectively [44,45,46]. The collision events must fulfill HLT selection criteria that require the presence of one or two leptons satisfying isolation and identification requirements. For the 2016 data set, the single-electron trigger has a transverse momentum ($p_{\textrm{T}}$) threshold of 25$\,\text {Ge}\hspace{-.08em}\text {V}$ for electrons with pseudorapidity $\left| {\eta }\right| < 2.1$ and 27$\,\text {Ge}\hspace{-.08em}\text {V}$ for $2.1< \left| {\eta }\right| < 2.5,$ whereas the single-muon trigger has a $p_{\textrm{T}}$ threshold of 24$\,\text {Ge}\hspace{-.08em}\text {V}$ for $\left| {\eta }\right| < 2.4.$ For the 2017 (2018) data set, the $p_{\textrm{T}}$ threshold is 35 (32)$\,\text {Ge}\hspace{-.08em}\text {V}$ for the single-electron trigger (covering $\left| {\eta }\right| < 2.5)$ and 27 (24)$\,\text {Ge}\hspace{-.08em}\text {V}$ for the single-muon trigger $(\left| {\eta }\right| <2.4)$. The dilepton ${\textrm{e}} {{\upmu }}$ trigger has $p_{\textrm{T}}$ thresholds of 23 and 12$\,\text {Ge}\hspace{-.08em}\text {V}$ for the leading and subleading leptons, respectively, with the same coverage in pseudorapidity for electrons and muons as above. During the first part of data taking in 2016, a lower $p_{\textrm{T}}$ threshold of 8$\,\text {Ge}\hspace{-.08em}\text {V}$ for the subleading muon was used.

Monte Carlo event generators are used to model the signal and background processes. For each process, three independent sets of simulated events, corresponding to the three years of data taking, are used. This approach includes year-dependent effects in the CMS detector, data taking, and event reconstruction. All simulated events corresponding to a given data set share the same set of parton distribution functions (PDFs), underlying event (UE) tune, and parton shower (PS) configuration. The PDF sets used are NNPDF 3.0 [47, 48] for 2016 and NNPDF 3.1 [49] for 2017 and 2018. The CUETP8M1 [50] tune is used to describe the UE in 2016 simulations, whereas the CP5 [51] tune is adopted in 2017 and 2018 simulated events. The MC samples are interfaced with pythia 8.226 [52] in 2016, and 8.230 in 2017 and 2018, for the modeling of UE, PS, and hadronization. Standard Model Higgs boson production through ggH, VBF, and VH is simulated at next-to-leading order (NLO) accuracy in quantum chromodynamics (QCD), including finite quark mass effects, using powheg v2 [53,54,55,56,57,58,59]. The minlo hvj [58] extension of powheg v2 is used for the simulation of $\textrm{WH}$ and quark-induced $\textrm{ZH}$ production, providing NLO accuracy for the $\textrm{VH}+0$- and 1-jet processes. For ggH production, the simulated events are weighted to match the NNLOPS [60, 61] prediction in the hadronic jet multiplicity ($N_{\text {jet}}$) and Higgs boson $p_{\textrm{T}}$ distributions. The weighting is based on $p_{\textrm{T}}$ and $N_{\text {jet}}$ as computed in the simplified template cross section scheme 1.0 [62]. The minlo hjj [63] generator, which provides NLO accuracy for $N_{\text {jet}} \ge 2,$ is also used for ggH production. The associated production processes with top quarks ($\textrm{t}{\bar{\textrm{t}}}\textrm{H}$) and bottom quarks (${\textrm{b}{\bar{\textrm{b}}}\textrm{H}}$) are simulated with powheg v2 and MadGraph 5_amc@nlo v2.2.2 [64], respectively, and have a negligible contribution in the analysis phase space. All SM Higgs boson samples are normalized to the cross sections recommended in Ref. [19]. The Higgs boson mass in the event generation is assumed to be 125$\,\text {Ge}\hspace{-.08em}\text {V}$, while a value of 125.38$\,\text {Ge}\hspace{-.08em}\text {V}$ [65] is used for the calculation of cross sections and branching fractions. The decay to a pair of W bosons and subsequently to leptons or hadrons is performed using the JHUGen v5.2.5 generator in 2016, and v7.1.4 in 2017 and 2018, for ggH, VBF, and quark-induced $\textrm{ZH}$ samples. The Higgs boson and W boson decays are performed using pythia 8.212 for the other signal simulations.

The ggH, VBF, and VH Higgs boson events with HVV anomalous couplings are generated with JHUGen at LO accuracy. With respect to the $\kappa _2^{{\textrm{Z}} {{\upgamma }}}/(\varLambda _1^{{\textrm{Z}} {{\upgamma }}})^2$ coupling parameter discussed in Sect. 2, the sign convention of the photon field is determined by the sign in front of the gauge fields in the covariant derivative. In this analysis, we define the covariant derivative $D_\mu = \partial _\mu -\textrm{i}e \sigma ^i W_\mu ^i/(2s_w) + \textrm{i}e B_\mu /(2 c_w)$ following the convention in JHUGen [31]. The JHUGen and powheg SM Higgs boson simulations were compared after parton showering and no significant differences in the distributions of kinematic observables were found. We adopt the JHUGen simulation to describe the kinematic features in all production modes with HVV anomalous couplings. The expected yields are scaled to match the SM theoretical predictions [19] for inclusive cross sections and the powheg SM prediction of relative event yields in the event categorization based on associated particles. Simulation of the ggH + 2 jets process with Hgg anomalous couplings is done using minlo X0jj [66] at NLO in QCD. A large number of signal samples with various anomalous couplings were generated. The MELA package [20,21,22,23,24] contains a library of matrix elements from JHUGen for different Higgs boson signal hypotheses. Matrix elements from different coupling signal hypotheses, but with the same production mechanism, are used to reweight the generated signal events. This procedure is used in the construction of the predictions for the different coupling components and their interference, allowing us to cover all points in the signal model phase space with sufficient statistical precision.

Background events are produced using several simulations. The quark-initiated nonresonant WW process is simulated with powheg v2 [67] at NLO accuracy for inclusive production. A reweighting is performed to match the diboson $p_{\textrm{T}}$ spectrum computed at NNLO+NNLL QCD accuracy [68, 69]. The mcfm v7.0 [70,71,72] generator is used to simulate gluon-induced WW production at LO accuracy, with the normalization chosen to match the NLO cross section [73]. Nonresonant EW production of WW pairs with two additional jets is simulated at LO accuracy with MadGraph 5_amc@nlo v2.4.2 using the MLM matching and merging scheme [74]. Top quark pair production ($\textrm{t}{\bar{\textrm{t}}}$) and single top quark processes, including $\textrm{tW}$, s- and t-channel contributions, are simulated with powheg v2 [75,76,77]. A reweighting of the top quark and antiquark $p_{\textrm{T}}$ spectrum at parton level is performed for the $\textrm{t}{\bar{\textrm{t}}}$simulation in order to match the NNLO and next-to-next-to-leading logarithm (NNLL) QCD predictions, including also the NLO EW contribution [78].

The Drell–Yan (DY) production of a charged-lepton pair is simulated with MadGraph 5_amc@nlo v2.4.2 at NLO accuracy with up to two additional partons, using the FxFx matching and merging scheme [79]. Production of a W boson associated with an initial state radiation photon (${\textrm{W}} {}{{\upgamma }}$) is simulated with MadGraph 5_amc@nlo v2.4.2 at NLO accuracy with up to 1 additional parton, using the FxFx jet merging. Diboson processes containing at least one Z boson or a virtual photon $({{\upgamma }}^{*})$ with a mass as low as 100$\,\text {Me}\hspace{-.08em}\text {V}$ are generated with powheg v2 [67] at NLO accuracy. Production of a W boson in association with a ${{\upgamma }}^{*}$ (${\textrm{W}} {}{{{\upgamma }}}^{*}$) for masses below 100$\,\text {Me}\hspace{-.08em}\text {V}$ is simulated by pythia 8.212 in the parton showering of ${\textrm{W}} {}{{\upgamma }}$ events. Triboson processes with inclusive decays are also simulated at NLO accuracy with MadGraph 5_amc@nlo v2.4.2.

For all processes, the detector response is simulated using a detailed description of the CMS detector, based on the Geant4 toolkit [80]. The distribution of additional $\textrm{pp}$ interactions within the same or nearby bunch crossings (pileup) in the simulation is reweighted to match that observed in data. The efficiency of the trigger system is evaluated in data on a per lepton basis using dilepton events consistent with the Z boson decay. The overall efficiencies of the trigger selections used in the analysis are obtained as the average of the per-lepton efficiencies weighted by their probability. The resulting efficiencies are applied directly on simulated events.

5 Event reconstruction

The identification and measurement of the properties of individual particles (PF candidates) in an event is achieved in the PF algorithm by combining information from various subdetectors. Electrons are identified and their momenta are measured in the pseudorapidity interval $\left| {\eta }\right| < 2.5$ by combining tracks in the silicon tracker with spatially compatible energy deposits in the electromagnetic calorimeter. Muons are identified and their momenta are measured in the pseudorapidity range $\left| {\eta }\right| < 2.4$ by matching tracks in the muon system and the silicon tracker. For better rejection of nonprompt leptons, increasing the sensitivity of the analysis, leptons are required to be isolated and well reconstructed using a set of criteria based on the quality of the track reconstruction, shape of calorimetric deposits, and energy flux in the vicinity of the particle’s trajectory [34, 35]. In addition, a selection based on a dedicated multivariate analysis (MVA) tagger developed for the CMS $\textrm{t}{\bar{\textrm{t}}}\textrm{H}$ analysis [81] is added in all channels for muon candidates.

Multiple $\textrm{pp}$ interaction vertices are identified from tracking information by use of the adaptive vertex fitting algorithm [82]. The primary $\textrm{pp}$ interaction vertex is taken to be the vertex corresponding to the hardest scattering in the event, evaluated using tracking information alone, as described in Section 9.4.1 of Ref. [83]. Leptons are required to be associated to the primary vertex using transverse and longitudinal impact parameter criteria [34, 35].

Hadronic jets are clustered from PF candidates using the infrared- and collinear-safe anti-$k_{\textrm{T}}$ algorithm with distance parameters of 0.4 (AK4) and 0.8 (AK8). The jet momentum is determined as the vectorial sum of all particle momenta in the jet. The AK8 jets considered are required to be reconstructed within the silicon tracker acceptance $(\left| {\eta }\right| < 2.4)$, whereas AK4 jets are reconstructed in the range $\left| {\eta }\right| < 4.7.$ For AK4 jets, contamination from pileup is suppressed using charged-hadron subtraction which removes charged PF candidates originating from vertices other than the primary interaction vertex. The residual contribution from neutral particles originating from pileup vertices is removed by means of an event-by-event jet-area-based correction to the jet four-momentum [84]. For AK8 jets, the pileup-per-particle identification algorithm (PUPPI) [85] is used to mitigate the effect of pileup at the reconstructed-particle level, making use of local shape information, event pileup properties, and tracking information. Additional selection criteria are applied to remove jets potentially dominated by instrumental effects or reconstruction failures [84].

The AK8 jets are used to reconstruct hadronic Vboson decays in a single merged jet when the decay products are highly collimated. This approach targets boosted W or Z bosons originating from the VH production mode. Such Lorentz-boosted Vdecays are identified using the ratio of the 2- to 1-subjettiness [86], $\tau _{2}/\tau _{1}$, and the groomed jet mass $m_{\text {J}}.$ The groomed mass is calculated after applying a modified mass drop algorithm [87, 88], known as the soft-drop algorithm [89], with parameters $\beta = 0,$ $z_\text {cut} = 0.1,$ and $R_0 = 0.8.$ The algorithm also identifies two hard subjets within the AK8 jet.

We refer to the identification of jets likely originating from bottom quarks as b tagging [90, 91]. For each AK4 jet in the event, a score is calculated through a multivariate combination of different jet properties, making use of boosted decision trees and deep neural networks. A jet is considered b-tagged if its associated score exceeds a threshold, tuned to achieve a certain tagging efficiency as measured in $\textrm{t}{\bar{\textrm{t}}}$events. The chosen working point corresponds to about 90% efficiency for bottom quark jets and to a mistagging rate of about 10% for light-flavor or gluon jets and of about 50% for charm quark jets.

The missing transverse momentum vector ${\vec p}_{\textrm{T}}^{\hspace{1.66656pt}\text {miss}}$ is computed as the negative vector sum of the transverse momenta of all the PF candidates in an event, and its magnitude is denoted as $p_{\textrm{T}} ^\text {miss}$ [41]. The PUPPI algorithm is applied to reduce the pileup dependence of the ${\vec p}_{\textrm{T}}^{\hspace{1.66656pt}\text {miss}}$ observable by computing the ${\vec p}_{\textrm{T}}^{\hspace{1.66656pt}\text {miss}}$ from the PF candidates weighted by their probability to originate from the primary interaction vertex [41].

6 Event selection

The analysis is performed using $\textrm{H}\rightarrow \textrm{WW}$ candidate events in the ${\textrm{e}} {{\upmu }}$ final state. For an event to be selected, the transverse momenta of the leading lepton $p_{\textrm{T}} ^{\ell 1}$ and the subleading lepton $p_{\textrm{T}} ^{\ell 2}$ must be greater than 25 and 13$\,\text {Ge}\hspace{-.08em}\text {V}$, respectively. The $p_{\textrm{T}} ^{\ell 2}$ threshold in the case of a muon is lowered to 10$\,\text {Ge}\hspace{-.08em}\text {V}$ for the 2016 data set because of the lower threshold in the corresponding HLT algorithm. Events containing additional leptons with $p_{\textrm{T}} > 10\,\text {Ge}\hspace{-.08em}\text {V}$ are discarded. The dilepton system is required to have an invariant mass $m_{\ell \ell }$ greater than 12$\,\text {Ge}\hspace{-.08em}\text {V}$ and transverse momentum $p_{\textrm{T}} ^{\ell \ell }$ above 30$\,\text {Ge}\hspace{-.08em}\text {V}$. A requirement on the missing transverse momentum of $p_{\textrm{T}} ^\text {miss} > 20\,\text {Ge}\hspace{-.08em}\text {V}$ is implemented. We define transverse mass discriminating variables $m_{\textrm{T}} ^{{\textrm{H}}}$ and $m_{\textrm{T}} ^{\ell 2}$ as

$$\begin{aligned}{} & {} m_{\textrm{T}} ^{{\textrm{H}}} = \sqrt{2p^{\ell \ell }_{\textrm{T}} p_{\textrm{T}} ^\text {miss} [1-\cos {\varDelta \varPhi (\vec {p}_{\text {T}}^{\ell \ell },\vec {p}_{\text {T}}^{\text {miss}})}]}, \end{aligned}$$

(17)

$$\begin{aligned}{} & {} m_{\textrm{T}} ^{\ell 2} = \sqrt{2p_{\textrm{T}} ^{\ell 2} p_{\textrm{T}} ^\text {miss} [1-\cos {\varDelta \varPhi (\vec {p}_{\text {T}}^{\ell 2},\vec {p}_{\text {T}}^{\text {miss}})}]}, \end{aligned}$$

(18)

and select events with $m_{\textrm{T}} ^{{\textrm{H}}} > 60\,\text {Ge}\hspace{-.08em}\text {V}$ and $m_{\textrm{T}} ^{\ell 2} > 30\,\text {Ge}\hspace{-.08em}\text {V}.$ The $m_{\textrm{T}} ^{{\textrm{H}}}$ requirement suppresses the $\text {DY}\rightarrow \uptau {}\uptau$ background process and avoids overlap with the $H\rightarrow \uptau \uptau$ analysis [11]. To ensure orthogonality with a future off-shell $\textrm{H}\rightarrow \textrm{WW}$ analysis we require $m_{\textrm{T}} ^{{\textrm{H}}} < 125\,\text {Ge}\hspace{-.08em}\text {V}.$ In addition, the region $76.2< m_{\ell \ell } < 106.2\,\text {Ge}\hspace{-.08em}\text {V}$ is excluded to avoid overlap with the off-shell $\textrm{H}\rightarrow \textrm{ZZ} \rightarrow 2\ell 2{{\upnu }}$ analysis [10]. These requirements will simplify a future combination of Higgs boson decay final states. Finally, events with any b-tagged jets with $p_{\textrm{T}} > 20\,\text {Ge}\hspace{-.08em}\text {V}$ are vetoed. These base selection criteria are summarized in Table 2.

Table 2 Summary of the base selection criteria

For the HVV coupling analysis, exclusive selection criteria, which are based on the associated jet activity in the event, are applied that target the ggH, VBF, and VH production processes. The AK4 (AK8) jets considered are required to have $p_{\textrm{T}} > 30~(200)\,\text {Ge}\hspace{-.08em}\text {V}.$ In the ggH channel, zero or one AK4 jet is required in the event. For the VBF and Resolved VH channels, we require two AK4 jets with dijet masses of $m_{\text {jj}} > 120\,\text {Ge}\hspace{-.08em}\text {V}$ and $60< m_{\text {jj}} < 120\,\text {Ge}\hspace{-.08em}\text {V},$ respectively. The Boosted VH channel requires the presence of a V-tagged AK8 jet (Vjet); such jets have a groomed mass in the region $65< m_{\text {J}} < 105\,\text {Ge}\hspace{-.08em}\text {V}$ and satisfy the requirement $\tau _{2}/\tau _{1} < 0.4.$ In the other channels, a Vjet veto is implemented to ensure orthogonality. These production channels for the HVV coupling study are summarized in Table 3.

Table 3 Summary of the ggH, VBF, and VH production channels used for the HVV coupling study

As the production vertex of the ggH + 2 jets process is sensitive to anomalous Hgg coupling effects, we use a 2-jet ggH channel that follows the VBF selection described above for the Hgg coupling analysis. The $\mathrm HWW$decay vertex is not sensitive to anomalous Hgg effects, and so decay-based variables are not studied in this channel. This permits a relatively tight selection of $m_{\ell \ell } < 55\,\text {Ge}\hspace{-.08em}\text {V}$ which is beneficial for background suppression. The 0- and 1-jet ggH channels are also included to constrain the ggH signal strength. All channels included for the Hgg coupling study are summarized in Table 4.

Table 4 Summary of ggH channel selections used for the Hgg coupling study

Control regions (CRs) are defined using the base selection criteria together with a set of alternative requirements summarized in Table 5. They are used to validate the background description and to estimate the number of background events in the signal region (SR). A dedicated $\uptau {}\uptau$ CR targets events from the DY process ${\textrm{Z}} \rightarrow \uptau {}\uptau$ with $\uptau$ leptons decaying leptonically to produce the ${\textrm{e}} {{\upmu }}$ final state. Also a top quark CR is defined to enhance events with one or more top quarks decaying to a W boson and bottom quark. Splitting events according to the number of associated jets, separate $\uptau {}\uptau$ and top quark CRs are defined for the 0-, 1- and 2-jet SRs. An additional CR with an enhanced contribution from the nonresonant WW background is used in the 2-jet SR. All CRs are used in the final data fit to constrain the DY, top quark, and WW background normalizations.

Table 5 Summary of the $\uptau {}\uptau ,$ top quark, and WW control region requirements

Additional $\uptau {}\uptau ,$ top quark, and WWCRs are defined requiring a Vjet. These CRs are used to validate the background description in the Boosted VH channel. However, they generally do not have a sufficient number of events to significantly constrain the background normalizations in the final fit to the data. As such, we rely on the 2-jet CRs to determine the normalizations to be used in the Boosted VH channel. Agreement between data and the background prediction in the Vjet CRs is observed when using normalizations determined in the 2-jet CRs.

7 Background estimation

The nonprompt-lepton backgrounds originating from leptonic decays of heavy quarks, hadrons misidentified as leptons, and electrons from photon conversions are suppressed by identification and isolation requirements imposed on electrons and muons. In this analysis, the nonprompt-lepton background primarily originates from ${\textrm{W}} +$jets events and is estimated from data, as described in detail in Ref. [92]. The procedure involves measuring the rate at which a nonprompt lepton passing a loose selection further passes a tight selection (misidentification rate) and the corresponding rate for a prompt lepton to pass this selection (prompt rate). The misidentification rate is measured in a data sample enriched in multijet events, whereas the prompt rate is measured using a tag-and-probe method [93] in a data sample enriched in DY events. The nonprompt-lepton background estimation is validated with data in a CR enriched with ${\textrm{W}} +$jets events, in which a pair of same-sign leptons is required.

The backgrounds from top quark processes and nonresonant WW production are estimated using a combination of MC simulations and the dedicated CRs described in the previous section. The normalisations of these backgrounds are left as free parameters in the fit, keeping different parameters for each jet multiplicity region. The top quark background normalization is measured from the observed data in the top quark enriched CRs. A separate normalization parameter is included for the quark-induced and gluon-induced WW backgrounds. For the 2-jet regions, the WW enriched CR is used to constrain the WW background normalisation parameters. In the 0- and 1-jet channels, these parameters are constrained directly in the signal regions, which span the high $m_{\ell \ell }$ phase space enriched in WW events.

The $\text {DY}\rightarrow \uptau {}\uptau$ background process is estimated with a data-embedding technique [94]. As for the top quark and WW backgrounds, the DY normalization is left unconstrained in the data fit. The $\text {DY}\rightarrow \uptau {}\uptau$ enriched CR described in Sect. 6 is used to constrain the free normalization parameters in the 0-, 1-, 2-jet regions. The data-embedded samples cover the events that pass the ${\textrm{e}} {{\upmu }}$ triggers, which represent the vast majority of the selected events. The remaining $\text {DY}\rightarrow \uptau {}\uptau$ events, which enter the analysis through the single-lepton triggers $({\approx }5\%$ of the total), are estimated using MC simulation.

Fig. 1

Topologies of the Higgs boson production and decay for vector boson fusion ${\textrm{q}} {{\textrm{q}} ^\prime }\rightarrow {\textrm{q}} {{\textrm{q}} ^\prime } {\textrm{H}}$ (left), ${\textrm{q}} \bar{{\textrm{q}}}^\prime \rightarrow \textrm{VH}$ (center), and gluon fusion with decay $\textrm{gg} \rightarrow {\textrm{H}} \rightarrow 2 \ell 2\nu$ (right). For the electroweak production topologies, the intermediate vector bosons and their decays are shown in green and the ${\textrm{H}} \rightarrow \textrm{WW}$ decay is marked in red. For the $\textrm{gg} \rightarrow {\textrm{H}} \rightarrow 2 \ell 2\nu$ topology, the ${\textrm{W}}$ boson leptonic decays are shown in green. In all cases, the incoming particles are depicted in brown and the angles characterizing kinematic distributions are marked in blue. Five angles fully characterize the orientation of the production and decay chain and are defined in the suitable rest frames

The $\textrm{WZ}$ and ${\textrm{W}} {}{{{\upgamma }}}^{*}$ background contributions are simulated as described in Sect. 4, and a data-to-simulation scale factor is derived in a three-lepton CR, as described in Ref. [92]. The contribution of the ${\textrm{W}} {}{{\upgamma }}$ process may also be a background because of photon conversions in the detector material. This process is estimated using MC simulation and validated using data in a CR requiring events with a leading $\upmu$ and a trailing e with same sign and a separation in $\varDelta R=\sqrt{\smash [b]{\varDelta \phi ^2 + \varDelta \eta ^2}}$ (where $\phi$ is the azimuthal angle in radians) smaller than 0.5. Triple vector boson production is a minor background in all channels and is estimated using MC simulation.

8 Observables and kinematic discriminants

In this paper, we search for anomalous HVV and Hgg coupling effects by studying:

• 1. the two quark jets from VBF and VH production (HVV coupling);

• 2. the $\textrm{H}\rightarrow \textrm{WW}$ decay products (HVV coupling); and

• 3. the two quark jets from ggH + 2 jets production (Hgg coupling).

The VBF, VH, and ggH production and decay topologies relevant for the HVV coupling are illustrated in Fig. 1.

When combined with the momentum transfer of the vector bosons, the five angles illustrated for VBF/VH production provide complete kinematic information for production and decay of the Higgs boson. The illustration for Higgs boson production via ggH in association with two jets is identical to the VBF diagram, except for replacing the intermediate vector bosons by gluons. Full production kinematic information is extracted for VBF, VH, and ggH + 2 jets candidate events using discriminants built from the matrix element calculations of the MELA package. The MELA approach is designed to reduce the number of observables to a minimum, while retaining all essential information. To form the production-based MELA kinematic discriminants, we use jets to reconstruct the four-momentum of the associated production particles. The presence of two neutrinos in the final state means it is not possible to reconstruct the four-momentum of all the Higgs boson decay products. Therefore, decay-based kinematic discriminants built from matrix elements are not used in this analysis. Instead, we rely on kinematic variables related to the measured final state of the Higgs boson decay. The strategies used for each of the topologies listed above are now discussed in more detail.

8.1 Kinematic features of two quark jets in VBF and VH channels

Kinematic distributions of associated particles in VBF and VH production are sensitive to the anomalous HVV couplings of the Higgs boson.

As illustrated in Fig. 1, a set of seven observables can be defined for the VBF and VH production topologies: $\varOmega = \{\theta _{1}^{(\prime )}, \theta _{2}^{(\prime )}, \theta ^*, \varPhi , \varPhi _1, q_1^{2}, q_2^{2} \},$ with $q_1^2$ and $q_2^2$ the squared four-momenta of the vector bosons [22]. Three types of discriminants are defined using the full kinematic description characterized by $\varOmega .$ The first type of discriminant is designed to separate signal and background Higgs boson production processes:

$$\begin{aligned} {\mathcal {D}}_{\text {sig}} = \frac{{\mathcal {P}}_{\text {sig}}(\varOmega ) }{{\mathcal {P}}_{\text {sig}}(\varOmega ) +{\mathcal {P}}_{\text {bkg}}(\varOmega )}, \end{aligned}$$

(19)

where the probability density ${\mathcal {P}}$ for a specific process is calculated from the matrix elements provided by the MELA package. The second type of discriminant separates the anomalous coupling BSM process from that of the SM:

$$\begin{aligned} {\mathcal {D}}_{\text {BSM}} = \frac{{\mathcal {P}}_{\text {BSM}}(\varOmega ) }{{\mathcal {P}}_{\text {BSM}}(\varOmega ) + {\mathcal {P}}_{\text {SM}}(\varOmega )}. \end{aligned}$$

(20)

Throughout this document the generic BSM label is generally replaced by the specific anomalous coupling state targeted. For the $a_{3}$ ${\textit{CP}}$-odd and $a_{2}$ ${\textit{CP}}$-even coupling parameters, we use, respectively, ${\mathcal {D}}_{{0-}}$ and ${\mathcal {D}}_{0+},$ whereas for the $\varLambda _1$ coupling parameters we use ${\mathcal {D}}_{\varLambda _1}$ and ${\mathcal {D}}_{\varLambda _1}^{{\textrm{Z}} \gamma }.$ The third type of discriminant isolates the interference contribution:

$$\begin{aligned} {\mathcal {D}}_{\text {int}} = \frac{ {\mathcal {P}}_{\text {SM-BSM}}^{\text {int}}(\varOmega )}{{\mathcal {P}}_{\text {SM}}(\varOmega ) +{\mathcal {P}}_{\text {BSM}}(\varOmega )}, \end{aligned}$$

(21)

where ${\mathcal {P}}_{\text {SM-BSM}}^{\text {int}}$ is the interference part of the probability distribution for a process with a mixture of the SM and BSM contributions. The ${\textit{CP}}$ label is generally used for the $a_3$ coupling parameter, as the BSM signal in this case is a pseudoscalar and the interference discriminant is a ${\textit{CP}}$-sensitive observable. The ${\mathcal {P}}$ values are normalized to give the same integrated cross sections in the relevant phase space of each process. Such normalization leads to a balanced distribution of events in the range between 0 and 1 for ${\mathcal {D}}_{\text {sig}}$ and ${\mathcal {D}}_{\text {BSM}},$ and between $-1$ and $+1$ for ${\mathcal {D}}_{\text {int}}.$

The selected events are split into three main production channels: VBF, Resolved VH, and Boosted VH. In the first two channels, the four-momenta of the two AK4 jets assigned as the associated particles are used in the MELA probability calculation. For the Boosted VH category, we use the four-momentum of the two subjets of the V-tagged AK8 jet. An estimate of the Higgs boson four-momentum is also required for the probability calculation. This can not be measured directly since the final state contains two neutrinos. As such, we construct a proxy Higgs boson four-momentum in the following manner. The $p_{\text {x}}$ and $p_{\text {y}}$ of the dineutrino system are estimated from the ${\vec p}_{\textrm{T}}^{\hspace{1.66656pt}\text {miss}}$ in a given event. The corresponding $p_{\text {z}}$ is then set to equal that of the dilepton system, which is based on the observed correlation between these variables at the generator level for simulated signals. Finally, the mass of the dineutrino system is set equal to the mean value of the generator-level dineutrino mass. The resulting four-momentum can then be combined with that of the measured dilepton system to create a proxy Higgs boson four-momentum. We note that the MELA probability calculation for the production vertices is largely based on the kinematic features of the associated particles, so the reconstruction of the proxy Higgs boson has a relatively small effect on the final discriminants. As an illustrative example of the MELA based discriminants used in this analysis, Fig. 2 shows the ${\mathcal {D}}_{{0-}}$ discriminant in the VBF and Resolved VH production channels for a number of different signal hypotheses. The discriminants are designed to target the dominant signal production process in a given channel.

Fig. 2

The ${\mathcal {D}}_{{0-}}$ discriminant in the VBF (upper) and Resolved VH (lower) production channels for a number of VBF (upper) and VH (lower) signal hypotheses. Pure $a_1$ $(f_{a3} = 0)$ and $a_3$ $(f_{a3} = 1)$ HVV signal hypotheses are shown along with two mixed coupling hypotheses $(f_{a3} = 0.005$ and $f_{a3} = 0.01)$. All distributions are normalized to unity

In the VBF channel, a ${\mathcal {D}}_{\text {VBF}}$ discriminant is constructed, following Eq. (19), where ${\mathcal {P}}_{\text {sig}}$ corresponds to the probability for the VBF production hypothesis, and ${\mathcal {P}}_{\text {bkg}}$ corresponds to that of gluon fusion production in association with two jets. The discriminant is also suitable for separating SM backgrounds from the VBF signal process. In the Resolved and Boosted VH channels, the corresponding discriminants do not give a significant level of separation with respect to ggH production or SM backgrounds. This is due to the relatively tight selection criteria, which limit the phase space to VH-like events. Hence, these discriminants are not included in the VH channels.

The ${\mathcal {D}}_{{\textit{CP}}}$ discriminant is sensitive to the sign of the interference between the ${\textit{CP}}$-even SM and ${\textit{CP}}$-odd BSM states. An asymmetry between the number of events detected with positive and negative ${\mathcal {D}}_{{\textit{CP}}}$ values is expected for mixed ${\textit{CP}}$ states. Therefore, a forward-backward categorization (forward defined as ${\mathcal {D}}_{{\textit{CP}}} > 0$ and backward as ${\mathcal {D}}_{{\textit{CP}}} < 0)$ is used to analyze the ${\textit{CP}}$-odd couplings. Similarly, ${\mathcal {D}}_{\text {int}}$ gives sensitivity to the sign of the interference between the SM and $a_2$ HVV BSM states. A forward-backward ${\mathcal {D}}_{\text {int}}$ categorization is also included. The value of ${\mathcal {D}}_{\text {int}}$ used to define the categories is chosen to symmetrize the SM Higgs boson expectation. In the case of the $\varLambda _1$ measurements, the interference discriminants were shown to be highly correlated with the ${\mathcal {D}}_{\text {BSM}}$ discriminants and so are not considered.

We now discuss the categorization and construction of the final multidimensional discriminants used for the two HVV coupling approaches defined in Sect. 2. The binning of the final discriminants was optimized to ensure sufficient statistical precision in the predictions of all bins, while retaining the kinematic information required to discriminate between the SM and anomalous coupling signal hypotheses.

8.1.1 VBF/ VH analysis strategy for Approach 1

In Approach 1, each of the four anomalous HVV coupling parameters $(a_2,$ $a_3,$ $\kappa _1/(\varLambda _1)^2,$ and $\kappa _2^{{\textrm{Z}} {{\upgamma }}}/(\varLambda _1^{{\textrm{Z}} {{\upgamma }}})^2)$ are analyzed separately. For this purpose, we construct a multidimensional discriminant for each of the four anomalous couplings in the VBF, Resolved VH, and Boosted VH channels.

In the VBF channel, we use two bins of the production discriminant ${\mathcal {D}}_{\text {VBF}},$ corresponding to low and high purity, using a bin boundary of 0.75. The $m_{\ell \ell }$ variable, which is sensitive to anomalous effects at the $\textrm{H}\rightarrow \textrm{WW}$ decay vertex, is included with two bins in the range 12–76.2$\,\text {Ge}\hspace{-.08em}\text {V}$. A bin boundary of 45$\,\text {Ge}\hspace{-.08em}\text {V}$ is chosen based on the expected signal shape changes induced by anomalous effects. Finally, one of the ${\mathcal {D}}_{\text {BSM}}$ discriminants is included with ten equally sized bins. Depending on the anomalous coupling under study this discriminant may be ${\mathcal {D}}_{0+},$ ${\mathcal {D}}_{{0-}},$ ${\mathcal {D}}_{\varLambda _1}$ or ${\mathcal {D}}_{\varLambda _1}^{{\textrm{Z}} \gamma }.$

For the VH channels, the $m_{\ell \ell }$ and ${\mathcal {D}}_{\text {BSM}}$ observables are used to build 2D kinematic discriminants. The $m_{\ell \ell }$ bins are the same as for the VBF channel. In the Resolved VH channel, we use four ${\mathcal {D}}_{\text {BSM}}$ bins of equal size. For the Boosted VH case, three variable bins with boundaries of 0.6 and 0.8 are used, a large first bin is chosen because relatively little signal is expected at low values of ${\mathcal {D}}_{\text {BSM}}.$ A distinct multidimensional discriminant is constructed for each anomalous coupling hypothesis in the VH channels.

For the $a_{3}$ coupling parameter, a forward-backward categorization of events based on ${\mathcal {D}}_{{\textit{CP}}}$ is implemented. In the case of the $a_{2}$ coupling parameter, ${\mathcal {D}}_{\text {int}}$ is largely correlated with ${\mathcal {D}}_{0+}$ in the VH channels. Therefore, a forward-backward ${\mathcal {D}}_{\text {int}}$ categorization is implemented only in the VBF channel. Figures 3, 4 and 5 show the discriminants used in the final fit to the data for the $a_{2},$ $a_{3},$ $\kappa _1/(\varLambda _1)^2,$ and $\kappa _2^{{\textrm{Z}} {{\upgamma }}}/(\varLambda _1^{{\textrm{Z}} {{\upgamma }}})^2$ Approach 1 coupling studies in the VBF and VH channels. A summary of the observables used in the HVV Approach 1 analysis may be found in Table 6.

8.1.2 VBF/ VH analysis strategy for Approach 2

In Approach 2, we use one categorization strategy and build one multidimensional discriminant in each channel to target all the HVV coupling parameters $(a_2,$ $a_3,$ $\kappa _1/(\varLambda _1)^2)$ simultaneously. In the VBF channel, the ${\mathcal {D}}_{{\textit{CP}}}$ and ${\mathcal {D}}_{\text {int}}$ discriminants are used to create four interference categories. Both ${\mathcal {D}}_{\text {VBF}}$ and $m_{\ell \ell }$ are used as for Approach 1. All three ${\mathcal {D}}_{\text {BSM}}$ discriminants that target the $a_{2}, a_{3}$ and $\kappa _1/(\varLambda _1)^2$ coupling parameters are included. However, the number of bins we implement is limited by the number of simulated events. Also the ${\mathcal {D}}_{\text {BSM}}$ discriminants are significantly correlated and so have similar performance for all couplings. Therefore, we use the ${\textit{CP}}$-odd discriminant ${\mathcal {D}}_{{0-}}$ and just one of the $C\!P$-even discriminants, ${\mathcal {D}}_{0+},$ both with three bins and bin boundaries of 0.1 and 0.9. A dedicated rebinning strategy is applied to the $[{\mathcal {D}}_{{0-}}, {\mathcal {D}}_{0+}]$ distribution merging bins dominated by the SM Higgs boson prediction or with low precision in the background prediction. In the VH channels, just two categories using ${\mathcal {D}}_{{\textit{CP}}}$ are defined and the discriminant is built using $m_{\ell \ell }$ as for Approach 1. Again, both ${\mathcal {D}}_{{0-}}$ and ${\mathcal {D}}_{0+}$ are chosen for the final discriminant. For the Resolved VH channel, we use three bins with boundaries of 0.25 and 0.75, whereas for the Boosted VH case we use two bins with a boundary of 0.8. The same rebinning strategy described for the VBF channel is applied to both Resolved and Boosted VH multidimensional discriminants. Table 6 includes a summary of the observables used in the HVV Approach 2 analysis.

Fig. 3

Observed and predicted distributions after fitting the data for $[{\mathcal {D}}_{\text {VBF}}, m_{\ell \ell }, {\mathcal {D}}_{0+}]$ in the VBF channel (upper), and for $[m_{\ell \ell }, {\mathcal {D}}_{0+}]$ in the Resolved VH (lower left) and Boosted VH (lower right) channels. For the VBF channel, the ${\mathcal {D}}_{\text {int}} < 0.4$ (left) and ${\mathcal {D}}_{\text {int}} > 0.4$ (right) categories are shown. The predicted Higgs boson signal is shown stacked on top of the background distributions. For the fit, the $a_{1}$ and $a_2$ HVV coupling contributions are included. The corresponding pure $a_{1}$ $(f_{a2} = 0)$ and $a_2$ $(f_{a2} = 1)$ signal hypotheses are also shown superimposed, their yields correspond to the predicted number of SM signal events scaled by an arbitrary factor to improve visibility. The uncertainty band corresponds to the total systematic uncertainty. The lower panel in each figure shows the ratio of the number of events observed to the total prediction

Fig. 4

Observed and predicted distributions after fitting the data for $[{\mathcal {D}}_{\text {VBF}}, m_{\ell \ell }, {\mathcal {D}}_{{0-}}]$ in the VBF channel (upper), and for $[m_{\ell \ell }, {\mathcal {D}}_{{0-}}]$ in the Resolved VH (middle) and Boosted VH (lower) channels. For each channel, the ${\mathcal {D}}_{{\textit{CP}}} < 0$ (left) and ${\mathcal {D}}_{{\textit{CP}}} > 0$ (right) categories are shown. For the fit, the $a_{1}$ and $a_3$ HVV coupling contributions are included. More details are given in the caption of Fig. 3

Fig. 5

Observed and predicted distributions after fitting the data for $[{\mathcal {D}}_{\text {VBF}}, m_{\ell \ell }, {\mathcal {D}}_{\varLambda 1}]$ (upper left) and $[{\mathcal {D}}_{\text {VBF}}, m_{\ell \ell }, {\mathcal {D}}_{\varLambda 1}^{{\textrm{Z}} \gamma }]$ (upper right) in the VBF channel, and for $[m_{\ell \ell }, {\mathcal {D}}_{\varLambda 1}]$ (left) and $[m_{\ell \ell }, {\mathcal {D}}_{\varLambda 1}^{{\textrm{Z}} \gamma }]$ (right) in the Resolved VH (middle) and Boosted VH (lower) channels. For the fits, the $a_{1}$ and $\kappa _1/(\varLambda _1)^2$ (left) or $a_{1}$ and $\kappa _2^{{\textrm{Z}} {{\upgamma }}}/(\varLambda _1^{{\textrm{Z}} {{\upgamma }}})^2$ (right) HVV coupling contributions are included. More details are given in the caption of Fig. 3

8.2 Kinematic features of $\textrm{H}\rightarrow \textrm{WW}$ decay products in 0- and 1-jet ggH channels

Similar to the SM $\textrm{H}\rightarrow \textrm{WW}$ analysis [25], we use $m_{\ell \ell }$ and $m_{\textrm{T}}$ to build 2D discriminants in the 0- and 1-jet ggH channels. The distributions have nine bins for $m_{\ell \ell }$ in the range 12–200$\,\text {Ge}\hspace{-.08em}\text {V}$ and six bins for $m_{\textrm{T}}$ in the range 60–125$\,\text {Ge}\hspace{-.08em}\text {V}$. The bin widths vary and are optimized to achieve good separation between the SM Higgs boson signal and backgrounds, as well as between the different anomalous coupling signal hypotheses. In particular, a finer binning with respect to the SM $\textrm{H}\rightarrow \textrm{WW}$ analysis is implemented in regions where anomalous effects are most significant. Figure 6 shows the $[m_{\textrm{T}}, m_{\ell \ell } ]$ distributions in the 0- and 1-jet ggH channels. The same $[m_{\textrm{T}}, m_{\ell \ell } ]$ discriminant is used to study all HVV anomalous couplings for both Approach 1 and 2.

8.3 Kinematic features of two quark jets in 2-jet ggH channel

For the Hgg coupling, we adopt a similar approach to the VBF $C\!P$ study, where the ${\textit{CP}}$-odd $a_{3}$ HVV coupling parameter is included. In this case, the optimal observables are ${\mathcal {D}}^{\textrm{ggH}}_{{0-}}$ and ${\mathcal {D}}^{\textrm{ggH}}_{{\textit{CP}}},$ targeting the ${\textit{CP}}$-odd $a_{3}$ Hgg coupling parameter. A forward-backward categorization is implemented using ${\mathcal {D}}^{\textrm{ggH}}_{{\textit{CP}}},$ and the ${\mathcal {D}}_{\text {VBF}}$ and ${\mathcal {D}}^{\textrm{ggH}}_{{0-}}$ observables are used to build 2D discriminants. The $m_{\ell \ell }$ variable is not considered in this case because it is not sensitive to anomalous Hgg effects. For ${\mathcal {D}}_{\text {VBF}},$ the bin boundary is relaxed to 0.5 to ensure sufficient ggH events are accepted in the more VBF-like bin. For ${\mathcal {D}}_{{0-}},$ eight (five) bins are used in the more (less) VBF-like bin with larger bin sizes at the extremes of the distribution to ensure sufficient precision in the background and signal predictions. The 0- and 1-jet channels discussed previously are also included in this study to constrain the ggH signal strength. The $[{\mathcal {D}}_{\text {VBF}},$ ${\mathcal {D}}^{\textrm{ggH}}_{{0-}}]$ distributions used to analyze the Hgg $a_{3}$ anomalous coupling in the 2-jet ggH channel are shown in Fig. 7. A summary of the observables used in the Hgg analysis is given in Table 6.

Table 6 The kinematic observables used for the interference based categorization and for the final discriminants used in the fits to data to study the HVV and Hgg couplings. For each of the anomalous HVV couplings in Approach 1, we have a dedicated analysis in the VBF and VH channels. In Approach 2, we use one analysis to target all anomalous HVV couplings simultaneously

Fig. 6

Observed and predicted distributions after fitting the data for $[m_{\textrm{T}}, m_{\ell \ell } ]$ in the 0- (upper) and 1-jet (lower) ggH channels. For the fit, the $a_{1}$ and $a_3$ HVV coupling contributions are included. More details are given in the caption of Fig. 3

Fig. 7

Observed and predicted distributions after fitting the data for $[{\mathcal {D}}_{\text {VBF}},$ ${\mathcal {D}}^{\textrm{ggH}}_{{0-}}]$ in the 2-jet ggH channel. Both the ${\mathcal {D}}^{\textrm{ggH}}_{\text {CP}} < 0$ (upper) and ${\mathcal {D}}^{\textrm{ggH}}_{\text {CP}} > 0$ (lower) categories are shown. In this case, the VBF and ggH signals are shown separately. For the fit, the $a_2^{\textrm{gg}}$ and $a_3^{\textrm{gg}}$ coupling contributions are included. The corresponding pure $a_2^{\textrm{gg}}$ $(f_{a3}^{\textrm{ggH}} = 0)$ and $a_3^{\textrm{gg}}$ $(f_{a3}^{\textrm{ggH}} = 1)$ signal hypotheses are also shown superimposed, their yields correspond to the predicted number of SM signal events. More details are given in the caption of Fig. 3

9 Systematic uncertainties

The signal extraction is performed using binned templates to describe the various signal and background processes. Systematic uncertainties that change the normalization or shape of the templates are included. All the uncertainties are modeled as nuisance parameters that are profiled in the maximum likelihood fit described in Sect. 10. The systematic uncertainties arise from both experimental or theoretical sources.

9.1 Experimental uncertainties

The following experimental systematic uncertainties are included in the final fit to data:

• The total uncertainty associated with the measurement of the integrated luminosity for 2016, 2017, and 2018 is $1.2\%$ [44], $2.3\%$ [45], and $2.5\%$ [46], respectively. This uncertainty is partially correlated among the three data sets, resulting in an overall uncertainty of 1.6%.

• The systematic uncertainty in the trigger efficiency is determined by varying the tag lepton selection criteria and the Z boson mass window used in the tag-and-probe method. It affects both the normalization and the shape of the signal and background distributions, and is kept uncorrelated among data sets. The total normalization uncertainty is less than 1%.

• The tag-and-probe method is also used to determine the lepton identification and isolation efficiency. Corrections are applied to account for any discrepancy in the efficiencies measured in data and simulation. The corresponding systematic uncertainty is about 1% for electrons and 2% for muons.

• The uncertainties in the determination of the lepton momentum scale mainly arise from the limited data sample used for their estimation. The impact on the normalization of the signal and background templates ranges between 0.6–1.0% for the electron momentum scale and is about 0.2% for the muon momentum scale. They are treated as uncorrelated among the three data-taking years.

• The jet energy scale uncertainty is modeled by implementing eleven independent nuisance parameters corresponding to different jet energy correction sources, six of which are correlated among the three data sets. Their effects vary in the range of 1–10%, mainly depending on the jet multiplicity in the analysis phase space. Another source of uncertainty arises from the jet energy resolution smearing applied to simulated samples to match the $p_{\textrm{T}}$ resolution measured in data. The effect varies in a range of 1–5%, depending on the jet multiplicity and is uncorrelated among the data sets. These uncertainties are included for both AK4 and AK8 jets. In addition, the $m_{\text {J}}$ scale and resolution, and Vtagging corrections with their corresponding uncertainties are included for V-tagged AK8 jets. These variables are calibrated in a top quark–antiquark sample enriched in hadronically decaying W bosons [95].

• The effects of the unclustered energy scale, jet energy scale, and lepton $p_{\textrm{T}}$ scales are included for the calculation of the missing transverse momentum. The resulting normalization systematic uncertainty is 1–10% and is treated as uncorrelated among the years.

• Both the normalization and shape of the signal and background templates are affected by the jet pileup identification uncertainty. The effect is below 1%.

• The uncertainty associated with the b tagging efficiency is modeled by seventeen nuisance parameters out of which five are of a theoretical origin and are correlated among the three data sets. The remaining set of four parameters per data set are treated as uncorrelated as they arise from the statistical accuracy of the efficiency measurement [90].

• Estimation of the nonprompt-lepton background is affected by the limited size of the data sets used for the misidentification rate measurements. It is also affected by the difference in the flavor composition of jets misidentified as leptons between the misidentification rate measurement region (enriched in multijet events) and the signal phase space. The effects on the nonprompt-lepton background estimation range between a few percent to about 10% depending on the SR and are treated as nuisance parameters uncorrelated between electrons and muons and among the three data sets. A normalization uncertainty of 30% [92] is assigned to fully cover for any discrepancies with respect to data in a ${\textrm{W}} +$jets CR and is treated as uncorrelated among data sets.

• The statistical uncertainties due to the limited number of simulated events are also included for all bins of the background distributions used to extract the results [96].

9.2 Theoretical uncertainties

Multiple theoretical uncertainties are considered and are correlated among data sets, unless stated otherwise:

• The uncertainties related to the choice of PDF and $\alpha _\textrm{S}$ have a minor effect on the shape of the distributions. Therefore, only normalization effects related to the event acceptance and to the cross section are included. However, these uncertainties are not considered for the backgrounds that have their normalization constrained through data in dedicated CRs. For the Higgs boson signal processes, these uncertainties are calculated by the LHC Higgs cross section working group [19].

• The theoretical uncertainties arising from missing higher-order corrections in the cross section calculations are also included. Background simulations are reweighted to the alternative scenarios corresponding to renormalization $\mu _{\text {R}}$ and factorization $\mu _{\text {F}}$ scales varied by factors 0.5 or 2 and the envelopes of the varied templates are taken as the one standard deviations. For background processes that have their normalization constrained through data in dedicated CRs, we consider only the shape effect of the uncertainties coming from the missing higher-order corrections. The WWnonresonant background has the uncertainties derived by varying $\mu _{\text {R}},$ $\mu _{\text {F}},$ and the resummation scale. For the ggH and VBF signal processes, the effects of the missing higher-order corrections on the overall cross section are decoupled into multiple sources according to the recipes described in Ref. [19].

• The uncertainty due to the pileup modeling was included for the main simulated background processes (DY, WW, top quark) as well as the ggH and VBF signals. The effect is determined by varying the total inelastic $\textrm{pp}$ cross section (69.2$\,\text {mb}$ [97, 98]) within the assigned 5% uncertainty.

• The PS modeling mainly affects the jet multiplicity, causing migration of events between categories that results in template shape changes. Associated uncertainties are evaluated by reweighting events with varied PS weights computed with pythia 8.212. The effect on the signal strength is found to be below 1%.

• Uncertainties associated with UE modeling are evaluated by varying the UE tune parameters used in the MC sample generation. Systematic uncertainties are correlated between the 2017 and 2018 data sets since they share the same UE tunes, whereas for 2016 the uncertainty is considered uncorrelated. The UE uncertainty has a minimal effect on the template shapes and affects the normalization by about 1.5%.

• A 15% uncertainty is applied to the relative fraction of the gg-induced component in nonresonant WW production [99]. The relative fraction between single top quark and $\textrm{t}{\bar{\textrm{t}}}$processes is assigned a systematic uncertainty of 8% [100]. Additional process-specific (DY, ) uncertainties, related to corrections to account for possible discrepancies between data and simulation, are assigned and are correlated among data sets.

10 Results

Fig. 8

Expected (dashed) and observed (solid) profiled likelihood on $f_{a2}$ (upper left), $f_{\varLambda 1}$ (upper right), $f_{a3}$ (lower left), and $f_{\varLambda 1}^{{\textrm{Z}} {{\upgamma }}}$ (lower right) using Approach 1. In each case, the signal strength modifiers are treated as free parameters. The dashed horizontal lines show the 68 and 95% $\text {CL}$ regions. Axis scales are varied for $f_{a2}$ and $f_{\varLambda 1}$ to improve the visibility of important features

Fig. 9

Expected (dashed) and observed (solid) profiled likelihood on $f_{a2}$ (upper left), $f_{\varLambda 1}$ (upper right) and $f_{a3}$ (bottom) using Approach 2. The other two anomalous coupling cross section fractions are either fixed to zero (blue) or left floating in the fit (red). In each case, the signal strength modifiers are treated as free parameters. The dashed horizontal lines show the 68 and 95% $\text {CL}$ regions. Axis scales are varied for $f_{a2}$ and $f_{\varLambda 1}$ to improve the visibility of important features

Table 7 Summary of constraints on the anomalous HVV and Hgg coupling parameters with the best fit values and allowed 68 and 95% $\text {CL}$ (in square brackets) intervals. For Approach 1, each $f_{ai}$ is studied independently. For Approach 2, each $f_{ai}$ is shown separately with the other two cross section fractions either fixed to zero or left floating in the fit. In each case, the signal strength modifiers are treated as free parameters

Fig. 10

The observed correlation coefficients between HVV anomalous coupling cross section fractions and signal strength modifiers (left) and between SMEFT Higgs basis coupling parameters (right)

Fig. 11

Expected (dashed) and observed (solid) profiled likelihood on the $\delta c_{\text {z}}$ (upper left), $c_{\text {z}\Box }$ (upper right), $c_{\text {zz}}$ (lower left), and $\tilde{c}_{\text {zz}}$ (lower right) couplings of the SMEFT Higgs basis. All four couplings are studied simultaneously. The dashed horizontal lines show the 68 and 95% $\text {CL}$ regions

The optimization and validation of the analysis were performed using simulation and data in CRs. The data in the SRs were examined once all details of the analysis were finalized. For the final results, we perform a binned maximum likelihood fit to the data combining all channels and data-taking periods. The statistical approach was developed by the ATLAS and CMS Collaborations in the context of the LHC Higgs Combination Group [101]. The likelihood function is defined for candidate events as:

$$\begin{aligned}{} & {} {\mathcal {L}}(\text {data} | \mu _{\textrm{ggH}}, \mu _{\text {EW}}, f_{ai}, \theta ) \nonumber \\{} & {} \quad =\prod _{\textrm{j}}\text {Poisson}(n_{\textrm{j}}|s_{\textrm{j}}(\mu _{\textrm{ggH}}, \mu _{\text {EW}}, f_{ai}, \theta )+b_{\textrm{j}}(\theta )) p(\tilde{\theta } | \theta ), \nonumber \\ \end{aligned}$$

(22)

where j runs over all bins and $n_{\textrm{j}}$ is the observed number of data events in each bin. Total signal and background expectations in each bin are represented by $s_{\textrm{j}}$ and $b_{\textrm{j}},$ respectively. The individual signal and background processes considered in each category are described using binned templates of multidimensional discriminants as described in Sect. 8. Each signal process is parametrized as a linear combination of terms originating from the SM, and anomalous couplings and their interference. The signal expectation depends on the parameters $\mu _{\textrm{ggH}},$ $\mu _{\text {EW}},$ and $f_{ai}$, and is constrained by the data fit. Both the signal and background expectations are functions of $\theta ,$ which represents the full set of nuisance parameters corresponding to the systematic uncertainties. The CRs described in Sect. 6 are included in the fit in the form of single bins, representing the number of events in each CR.

The $\mu _{\textrm{ggH}}$ and $\mu _{\text {EW}}$ parameters correspond to the Higgs boson signal strength modifiers for the ggH and VBF/VH signals, respectively. Signal yields for the VBF and VH processes are related to each other because the same HVV couplings enter both in production and decay of the Higgs boson. The ggH signal is initiated predominantly by the top fermion couplings and is unrelated to the VBF and VH production mechanisms. As the signal strength modifiers are free parameters in the fit, the overall signal event yield is not used to discriminate between alternative signal hypotheses. The $f_{ai}$ parameter corresponds to the anomalous coupling cross section fraction and determines the shape of the signal expectation. The cross section fraction for the SM coupling is simply taken as $1 - \left| {f_{ai}}\right| .$ In Approach 1, the SM and just one anomalous HVV coupling are included, and each $f_{ai}$ is thus studied independently. Depending on the particular anomalous coupling under investigation, $f_{ai}$ may represent $f_{a2}$, $f_{a3}$, $f_{\varLambda 1}$, or $f_{\varLambda 1}^{{\textrm{Z}} {{\upgamma }}}$. For Approach 2, the SM and three anomalous HVV couplings are included. In this case, $f_{ai}$ represents $f_{a2}$, $f_{a3}$ and $f_{\varLambda 1}$, which are studied simultaneously. It is explicitly required that $\left| {f_{a2}}\right| + \left| {f_{a3}}\right| + \left| {f_{\varLambda 1}}\right| \le 1$ to avoid probing an unphysical parameter space. Finally, there is just one anomalous coupling corresponding to $f_{a3}^{\textrm{ggH}}$ to consider for the Hgg vertex. For this study, we also include the effect of the ${\textit{CP}}$-odd HVV anomalous coupling on the VBF process. This is achieved by including $f_{a3}$ as a free parameter in the fit. The $p(\tilde{\theta }|\theta )$ are the probability density functions (PDFs) for the observed values of the nuisance parameters, $\tilde{\theta },$ obtained from calibration measurements. The systematic uncertainties that affect only the normalizations of the signal and background processes are treated as PDFs following a log-normal distribution, whereas shape-altering systematic uncertainties are treated as Gaussian PDFs [101].

Additional interpretations in terms of the SMEFT Higgs and Warsaw basis coupling parameters are also considered using Eqs. (7–10) and Eqs. (11–14), respectively. In each case, four independent couplings are studied simultaneously and the effect of the couplings on the total width of the Higgs boson is taken into account. For the $f_{ai}$ measurements, this effect is absorbed by the signal strength modifiers. A parameterization of the partial widths of the main Higgs boson decay modes as a function of the couplings is used to determine the effect on the Higgs boson width [24, 28].

The likelihood is maximized with respect to the signal modifier parameters and with respect to the nuisance parameters. Confidence level ($\text {CL}$) intervals are determined from profile likelihood scans of the respective parameters. The allowed 68% and 95% $\text {CL}$ intervals are defined using the set of parameter values at which the profile likelihood function $-2\varDelta \ln {{\mathcal {L}}} = 1.00$ and 3.84 [102], respectively, for which exact coverage is expected in the asymptotic limit [103]. The likelihood value at a given $f_{ai}$ is determined by the shape of the signal hypothesis and the relative signal event yields between categories. Expected results are obtained using the Asimov data set [104] constructed using the SM values of the signal modifier parameters.

For Approach 1, where we assume $a_{i}^\textrm{ZZ}=a_{i}^{\textrm{WW}},$ the expected and observed $f_{a2}$, $f_{a3}$, $f_{\varLambda 1}$, and $f_{\varLambda 1}^{{\textrm{Z}} {{\upgamma }}}$ likelihood scans are shown in Fig. 8. Significant interference effects for negative values of $f_{a2}$, around $-0.25,$ and positive values of $f_{\varLambda 1}$, around 0.5, are evident. Relatively large changes in the signal shape with respect to the SM are predicted at these values. Also evident are narrow minima around $f_{ai}$  = 0. The anomalous coupling terms in Eq. (1) have a $q_{i}^{2}$ dependence, which can be larger at the VBF/VH production vertex than at the Higgs decay vertex. This causes the cross section and the shape of the VBF/VH signal hypothesis to change rapidly with $f_{ai}$. For $f_{\varLambda 1}^{{\textrm{Z}} {{\upgamma }}}$, there are no anomalous effects at the Higgs decay vertex and so the only structure present is the narrow minimum related to the VBF/VH production vertex. The axis scales are varied to improve the visibility of important features for $f_{a2}$ and $f_{\varLambda 1}$. For Approach 2, where the SU(2) x U(1) coupling relationships from Eqs. (2–6) are adopted, the expected and observed $f_{a2}$, $f_{a3}$ and $f_{\varLambda 1}$ likelihood scans are shown in Fig. 9. The results are shown for each $f_{ai}$ separately with the other two $f_{ai}$ either fixed to zero or left floating in the fit. The measured values of the signal strength parameters correspond to $\mu _{\text {EW}}=0.9^{+0.19}_{-0.24}$ and $\mu _{\textrm{ggH}}=0.9^{+0.38}_{-0.20}$ when all parameters float simultaneously. It is notable that the observed $-2\varDelta \ln {{\mathcal {L}}}$ profile values are generally lower than expected. This is consistent with a downward statistical fluctuation in the number of VBF and VH events. The lowest $\mu _{\text {EW}}$ value measured is 0.82 for the Approach 1 $f_{a3}$ fit which can be compared with the highest value of 0.97 for the corresponding $f_{\varLambda 1}$ fit. In each case, the uncertainty in $\mu _{\text {EW}}$ is about 20% and as such all fitted values are consistent with both the SM and each other. More generally, all anomalous HVV coupling parameter measurements are consistent with the expectations for the SM Higgs boson. The p-value compatibility of the full Approach 2 fit, where all signal parameters float simultaneously, with the SM is 91%. A summary of constraints on the anomalous HVV coupling parameters with the best fit values and allowed 68% and 95% $\text {CL}$ intervals are shown in Table 7. The most stringent constraints on the HVV anomalous coupling cross section fractions are at the per mille level. Some constraints are less stringent than expected due to the fitted values of $\mu _{\text {EW}}$ being lower than the SM expectation. The observed correlation coefficients between HVV anomalous coupling cross section fractions and signal strength modifiers are displayed in Fig. 10.

For the SMEFT Higgs basis interpretation, the expected and observed constraints on the $\delta c_{\text {z}},$ $c_{\text {z}\Box },$ $c_{\text {zz}},$ and $\tilde{c}_{\text {zz}}$ coupling parameters are shown in Fig. 11. Table 8 presents a summary of the constraints on the couplings whereas Fig. 10 reports the observed correlation coefficients between them. For the Warsaw basis interpretation, the expected and observed constraints on the $c_{\text {H}\Box },$ $c_{\text {HD}},$ $c_{\text {HW}},$ $c_{\text {HWB}},$ $c_{\text {HB}},$ $c_{\text {H}\tilde{\text {W}}},$ $c_{\text {H}\tilde{\text {W}}\text {B}},$ and $c_{\text {H}\tilde{\text {B}}}$ coupling parameters are presented in Table 9. To cover all the Warsaw basis coupling parameters, three independent fits to the data were performed with a different choice of four independent couplings in each. A summary of the constraints on the SMEFT Higgs and Warsaw basis coupling parameters is presented in Fig. 12.

Table 8 Summary of constraints on the SMEFT Higgs basis coupling parameters with the best fit values and 68% $\text {CL}$ uncertainties. All four couplings are studied simultaneously

Finally, the expected and observed $f_{a3}^{\textrm{ggH}}$ likelihood scans are shown in Fig. 13. The result is consistent with the expectation for a SM Higgs boson. Excluding the effect of the ${\textit{CP}}$-odd HVV anomalous coupling, by fixing $f_{a3}$ to zero, has a negligible effect. For $\left| {f_{a3}^{\textrm{ggH}}}\right|$ approaching unity, the observed $-2\varDelta \ln {{\mathcal {L}}}$ profile values are larger than expected. This is consistent with downward statistical fluctuations in the data for a couple of bins where sensitivity to the $a_{3}$ Hgg coupling contribution is enhanced (Fig. 7 upper). The constraint on the anomalous Hgg coupling parameter with the best fit value and allowed 68% $\text {CL}$ interval is shown in Table 7.

Table 9 Summary of constraints on the SMEFT Warsaw basis coupling parameters with the best fit values and 68% $\text {CL}$ uncertainties. Only one of $c_{\text {HW}},$ $c_{\text {HWB}},$ and $c_{\text {HB}}$ is independent, the same is also true for $c_{\text {H}\tilde{\text {W}}},$ $c_{\text {H}\tilde{\text {W}}\text {B}},$ and $c_{\text {H}\tilde{\text {B}}}.$ Three independent fits to the data were performed with a different choice of four independent couplings in each

Fig. 12

Summary of constraints on the SMEFT Higgs (upper) and Warsaw (lower) basis coupling parameters with the best fit values and 68% $\text {CL}$ uncertainties. For the Warsaw basis, only one of $c_{\text {HW}}$ , $c_{\text {HWB}},$ and $c_{\text {HB}}$ is independent, the same is also true for $c_{\text {H}\tilde{\text {W}}},$ $c_{\text {H}\tilde{\text {W}}\text {B}},$ and $c_{\text {H}\tilde{\text {B}}}$

Fig. 13

Expected (dashed) and observed (solid) profiled likelihood on $f_{a3}^{\textrm{ggH}}$. The signal strength modifiers and the CP-odd HVV anomalous coupling cross section fraction are treated as free parameters. The crossing of the observed likelihood with the dashed horizontal line shows the observed 68% $\text {CL}$ region

11 Summary

This paper presents a study of the anomalous couplings of the Higgs boson (H) with vector bosons, including ${\textit{CP}}$ violating effects, using its associated production with hadronic jets in gluon fusion, electroweak vector boson fusion, and associated production with a W or Z boson, and its subsequent decay to a pair of W bosons. The results are based on the proton–proton collision data set collected by the CMS detector at the LHC during 2016–2018, corresponding to an integrated luminosity of 138$\,\text {fb}^{-1}$ at a center-of-mass energy of 13TeV. The analysis targets the different-flavor dilepton $({\textrm{e}} {{\upmu }})$ final state, with kinematic information from associated jets combined using matrix element techniques to increase sensitivity to anomalous effects at the production vertex. Dedicated Monte Carlo simulation and matrix element reweighting provide modeling of all kinematic features in the production and decay of the Higgs boson with full simulation of detector effects. A simultaneous measurement of four Higgs boson

couplings to electroweak vector bosons has been performed in the framework of a standard model effective field theory. All measurements are consistent with the expectations for the standard model Higgs boson and constraints are set on the fractional contribution of the anomalous couplings to the Higgs boson cross section. The most stringent constraints on the HVV anomalous coupling cross section fractions are at the per mille level. These results are in agreement with those obtained in the $\textrm{H}\rightarrow \textrm{ZZ}$ and $H\rightarrow \uptau \uptau$ channels, and also significantly surpass those of the previous $\textrm{H}\rightarrow \textrm{WW}$ anomalous coupling analysis from the CMS experiment in both scope and precision.