Studies of the resonance structure in $D^{0} \rightarrow K^\mp \pi ^\pm \pi ^\pm \pi ^\mp$ decays

Abstract

Amplitude models are constructed to describe the resonance structure of ${D^{0}\rightarrow K^{-}\pi ^{+}\pi ^{+}\pi ^{-}}$ and ${D^{0} \rightarrow K^{+}\pi ^{-}\pi ^{-}\pi ^{+}}$ decays using pp collision data collected at centre-of-mass energies of 7 and 8 TeV with the LHCb experiment, corresponding to an integrated luminosity of 3.0 $fb^{-1}$. The largest contributions to both decay amplitudes are found to come from axial resonances, with decay modes $D^{0} \rightarrow a_1(1260)^{+} K^{-}$ and $D^{0} \rightarrow K_1(1270/1400)^{+} \pi ^{-}$ being prominent in ${D^{0}\rightarrow K^{-}\pi ^{+}\pi ^{+}\pi ^{-}}$ and $D^{0}\rightarrow K^{+}\pi ^{-}\pi ^{-}\pi ^{+}$, respectively. Precise measurements of the lineshape parameters and couplings of the $a_1(1260)^{+}$, $K_1(1270)^{-}$ and $K(1460)^{-}$ resonances are made, and a quasi model-independent study of the $K(1460)^{-}$ resonance is performed. The coherence factor of the decays is calculated from the amplitude models to be $R_{K3\pi } = 0.459\pm 0.010\,(\mathrm {stat}) \pm 0.012\,(\mathrm {syst}) \pm 0.020\,(\mathrm {model})$, which is consistent with direct measurements. These models will be useful in future measurements of the unitary-triangle angle $\gamma$ and studies of charm mixing and $C\!P$ violation.

1 Introduction

The decays¹ ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ and ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^+} {{\pi } ^-}$ have an important role to play in improving knowledge of the Cabibbo–Kobayashi–Maskawa (CKM) unitarity-triangle angle $\gamma \equiv \arg (-{V_{{u} {d}}} {V_{{u} {b}}^*}/{V_{{c} {d}}} {V_{{c} {b}}^*})$. Sensitivity to this parameter can be obtained by measuring $C\!P$-violating and associated observables in the decay ${{{B} ^-}} \rightarrow D {{K} ^-}$, where D indicates a neutral charm meson reconstructed in final states common to both ${{D} ^0}$ and ${{\overline{D}{}} {}^0}$, of which ${{K} ^\mp } {{\pi } ^\pm } {{\pi } ^\pm } {{\pi } ^\mp }$ are significant examples [1, 2]. A straightforward approach to such an analysis is to reconstruct the four-body D-meson decays inclusively, which was performed by the LHCb collaboration in a recent measurement [3]. Alternatively, additional sensitivity can be sought by studying the variation of the observables across the phase space of the D decays, a strategy that requires knowledge of the variation of the decay amplitudes of the charm mesons.

Studies of charm mixing and searches for $C\!P$ violation in the ${D} ^0$–${\overline{D}{}} {}^0$ system, which for these final states have only been performed inclusively [4], will also benefit from an understanding of the variation of the decay amplitudes across their phase space. These decay modes are also a rich laboratory for examining the behaviour of the strong interaction at low energy, through studies of the intermediate resonances that contribute to the final states. All these considerations motivate an amplitude analysis of the two decays.

The decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ has a branching ratio of $(8.29 \pm 0.20)\%$ [5], which is the highest of all ${{D} ^0}$ decay modes involving only charged particles, and is predominantly mediated by Cabibbo-favoured (CF) transitions. The decay ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$ is dominated by doubly Cabibbo-suppressed (DCS) amplitudes, with small contributions from mixing-related effects, and occurs at a rate that is suppressed by a factor of $(3.22 \pm 0.05) \times 10^{-3}$ [4] compared to that of the favoured mode. The favoured and suppressed modes are here termed the ‘right-sign’ (RS) and ‘wrong-sign’ (WS) decay, respectively, on account of the charge correlation between the kaon and the particle used to tag the flavour of the parent meson.

In this paper, time-integrated amplitude models of both decay modes are constructed using pp collision data collected at centre-of-mass energies of 7 and 8 TeV with the LHCb experiment, corresponding to an integrated luminosity of 3.0 fb$^{-1}$. The RS sample size is around 700 times larger than the data set used by the Mark III collaboration to develop the first amplitude model of this decay [6]. An amplitude analysis has also been performed on the RS decay by the BES III collaboration [7] with around 1.6% of the sample size used in this analysis. This paper reports the first amplitude analysis of the WS decay.

The paper is organised as follows. In Sect. 2 the detector, data and simulation samples are described, and in Sect. 3 the signal selection is discussed. The amplitude-model formalism is presented in Sect. 4, and the fit method and model-building procedure in Sect. 5. Section 6 contains the fit results and conclusions are drawn in Sect. 7.

2 Detector and simulation

The LHCb detector [8] is a single-arm forward spectrometer covering the pseudorapidity range $2<\eta <5$, designed for the study of particles containing $b$ or $c$ quarks. The detector includes a high-precision tracking system consisting of a silicon-strip vertex detector surrounding the pp interaction region, a large-area silicon-strip detector located upstream of a dipole magnet with a bending power of about $4\,{\mathrm {Tm}}$, and three stations of silicon-strip detectors and straw drift tubes placed downstream of the magnet. The tracking system provides a measurement of momentum, $p$, of charged particles with a relative uncertainty that varies from 0.5% at low momentum to 1.0% at 200${\mathrm {\,GeV\!/}c}$. The minimum distance of a track to a primary vertex (PV), the impact parameter, is measured with a resolution of $(15+29/p_{\mathrm { T}}){\,\upmu \mathrm {m}}$, where $p_{\mathrm { T}}$ is the component of the momentum transverse to the beam, in ${\mathrm {\,GeV\!/}c}$. Different types of charged hadrons are distinguished using information from two ring-imaging Cherenkov (RICH) detectors. Photons, electrons and hadrons are identified by a calorimeter system consisting of scintillating-pad and preshower detectors, an electromagnetic calorimeter and a hadronic calorimeter. Muons are identified by a system composed of alternating layers of iron and multiwire proportional chambers.

The trigger [9] consists of a hardware stage, based on information from the calorimeter and muon systems, followed by a software stage, in which all charged particles with $p_{\mathrm { T}} >500\,(300){\mathrm {\,MeV\!/}c}$ are reconstructed for 2011 (2012) data. At the hardware trigger stage, events are required to have a muon with high $p_{\mathrm { T}}$ or a hadron, photon or electron with high transverse energy in the calorimeters. The software trigger requires a two-, three- or four-track secondary vertex with a significant displacement from the primary pp interaction vertices. At least one charged particle must have a transverse momentum $p_{\mathrm { T}} > 1.7(1.6){\mathrm {\,GeV\!/}c}$ and be inconsistent with originating from a PV. A multivariate algorithm [10] is used for the identification of secondary vertices consistent with the decay of a $b$ hadron.

In the simulation, pp collisions are generated using Pythia  [11] with a specific LHCb configuration [12]. Particle decays are described by EvtGen  [13]. The interaction of the generated particles with the detector, and its response, are implemented using the Geant4 toolkit [14, 15] as described in Ref. [16].

3 Signal selection and backgrounds

The decay chain ${\overline{B}{}} \rightarrow {{D} ^*} (2010)^{+} \mu ^{-} X$ with ${{D} ^*} (2010)^{+} \rightarrow {{D} ^0} \pi _{\mathrm {slow}}^{+}$ is reconstructed as a clean source of ${{D} ^0}$ mesons for analysis. The ${{D} ^0}$ mesons are reconstructed in the ${{K} ^\mp } {{\pi } ^\pm } {{\pi } ^\pm } {{\pi } ^\mp }$ final states. The charged pion, $\pi _{\mathrm {slow}}^{+}$, originating from the ${{D} ^*} (2010)^{+}$ is referred to as ‘slow’ due to the small Q-value of the decay. The charge of the muon and slow pion are used to infer the flavour of the neutral D meson. Candidates are only accepted if these charges lead to a consistent hypothesis for the flavour of the neutral D meson. All other aspects of the reconstruction and selection criteria are identical between the RS and WS samples.

The two-dimensional plane $m_{K \pi \pi \pi }$ vs. $\Delta m$, where $m_{K \pi \pi \pi }$ is the invariant mass of the ${{D} ^0}$ meson candidate, and $\Delta m = m_{K \pi \pi \pi \pi _{\mathrm {slow}}} - m_{K \pi \pi \pi }$ is mass difference between the ${D} ^*(2010)^{+}$ and ${{D} ^0}$ meson candidates, is used to define signal and sideband regions with which to perform the amplitude analysis and study sources of background contamination. The signal region is defined as $\pm 0.75{\mathrm {\,MeV\!/}c^2} (\pm 18{\mathrm {\,MeV\!/}c^2})$ of the signal peak in $\Delta m (m_{K \pi \pi \pi })$, which corresponds to about three times the width of the peak.

It is required that the hardware trigger decision is either due to the muon candidate or is independent of the particles constituting the reconstructed decay products of the $B$ candidate. For example, a high-$p_T$ particle from the other $B$ meson decay in the event firing the hadron trigger. The software trigger decision is required to either be due to the muon candidate or a two- three- or four-track secondary secondary vertex.

The WS sample is contaminated by a category of RS decays in which the kaon is mis-identified as a pion, and a pion as a kaon. To suppress this background, it is required that the kaon is well identified by the RICH detectors. The residual contamination from this background is removed by recalculating the mass of the ${D} ^0$ candidate with the mass hypotheses of a kaon and each oppositely charged pion swapped, then vetoing candidates that fall within 12${\mathrm {\,MeV\!/}c^2}$ of the nominal mass of the ${D} ^0$ meson. As the majority of particles from the PV are pions, the particle identification requirements on the kaon also reduces the background from random combinations of particles.

Remaining background from random combinations of particles can be divided into two categories. Candidates where the ${{D} ^0}$ is reconstructed from a random combination of tracks are referred to as combinatorial background. Candidates where the ${{D} ^0}$ is correctly reconstructed but paired with an unrelated $\pi _{\mathrm {slow}}^{+}$ are referred to as mistag background. This latter source of background is dominated by RS decays. Both of these backgrounds are suppressed using a multivariate classifier based on a Boosted Decision Tree (BDT) [17,18,19] algorithm. The BDT is trained on RS data candidates from the signal region and the sidebands of the WS data, and uses 15 variables related to the quality of the reconstruction of the PV, $B$ and ${{D} ^0}$ decay vertices, and the consistency of tracks in the signal candidate incoming from these vertices. Variables pertaining to the ${{D} ^0}$ kinematics and its decay products are avoided to minimise any bias of the phase-space acceptance.

The signal and background yields in the signal region for each sample is determined by simultaneously fitting the two-dimensional $\Delta m$ vs. $m_{K \pi \pi \pi }$ distribution for both samples. The ${{D} ^0}$, muon and slow pion candidates are constrained to originate from a common vertex in calculating the ${{D} ^0}$ and ${{D} ^{*+}}$ masses. This requirement improves the resolution of the $\Delta m$ distribution by approximately a factor of two. The signal is modelled with a product of two Cruijff [20] functions. The Cruijff shape parameters are shared between both samples. The combinatorial background is modelled by a first-order polynomial in $m_{K \pi \pi \pi }$, and by a threshold function in $\Delta m$,

$$\begin{aligned} \mathcal {P}(Q) \propto (1+pQ)\left( 1+Q+pQ^2\right) ^{a}, \end{aligned}$$

(1)

where $Q=\Delta m - m_{\pi }$ and the parameters p, a are determined by the fit. The background shape parameters, including those for the polynomial in $m_{K \pi \pi \pi }$, are allowed to differ between WS and RS samples. The mistag background component is a product of the signal shape in $m_{K \pi \pi \pi }$ and the combinatorial background shape in $\Delta m$. The optimal requirement on the output of the BDT classifier is selected by repeating the fit varying this requirement, and maximising the expected significance of the WS signal, which is defined as

$$\begin{aligned} S = \frac{\hat{N}_{\mathrm {sig}}}{\sqrt{\hat{N}_{\mathrm {sig}}+N_{\mathrm {bkg}}}}, \end{aligned}$$

(2)

where $N_{\mathrm {bkg}}$ is the background yield in the signal region. The expected number of WS candidates, $\hat{N}_{\mathrm {sig}}$, is estimated by scaling the number of RS signal candidates in the signal region by the ratio of branching fractions. The yields of the various contributions for both samples are listed in Table 1, and the $m_{K \pi \pi \pi }$ and $\Delta m$ distributions, with the fit projections superimposed, are shown in Fig. 1. The purities of the RS and WS samples after selection are found to be 99.6 and $82.4\%$, respectively, with $4\%$ of WS candidates arising from mistagged decays. Studies of simulated data indicate that the selected sample has a relatively uniform acceptance across the phase space, with approximately 30% reductions in acceptance near the edges of the kinematically allowed region. The samples also have a relatively uniform selection efficiency in decay time, being constant within $\pm \,10\%$ for lifetimes greater than one average lifetime of the $D$ meson.

Fig. 1

Invariant mass and mass difference distributions for RS (top) and WS (bottom) samples, shown with fit projections. The signal region is indicated by the filled grey area, and for each plot the mass window in the orthogonal projection is applied

Table 1 Signal and background yields for both samples in the signal region, presented separately for each year of data taking

For the amplitude analysis, a kinematic fit is performed constraining the ${D} ^0$ mass to its known value [21], which improves the resolution in the ${D} ^0$ phase space. This also forces all candidates to lie inside the kinematically allowed region. Candidates are only accepted if this kinematic fit converges.

4 Formalism of amplitude model

The amplitudes contributing to the decays ${{D} ^0} \rightarrow {{K} ^\mp } {{\pi } ^\pm } {{\pi } ^\pm } {{\pi } ^\mp }$ are described in terms of a sequence of two-body states. It is assumed that once these two-body states are produced, rescattering against other particles can be neglected. Two-body processes are often referred to as isobars and this approximation as the isobar model. Isobars can be described in terms of resonances, typically using the relativistic Breit-Wigner amplitude for narrow vector and tensor states. For scalar states, there typically are multiple broad overlapping resonances, in addition to significant nonresonant scattering amplitudes between the constituent particles of the state. Such states cannot be described in terms of Breit-Wigner amplitudes and instead the K-matrix formalism [22, 23] is adopted, and will be denoted by $\left[ {{\pi } ^+} {{\pi } ^-} \right] ^{L=0}$ and $\left[ {{K} ^\mp } {{\pi } ^\pm } \right] ^{L=0}$ throughout for ${{\pi } ^+} {{\pi } ^-}$ and ${{K} ^\mp } {{\pi } ^\pm }$ S-waves, respectively.

The following decay chains are considered:

• Cascade decays have the topology ${{D} ^0} \rightarrow X \left[ Y \left[ P_1P_2\right] P_3 \right] P_4$ – the ${{D} ^0}$ meson decays into a stable pseudoscalar state $P_4$ and an unstable state X. The unstable state then decays to three pseudoscalars $P_{1,2,3}$ via another intermediate unstable state (Y). There are three distinct possibilities for cascade decays. The resonance X can either have isospin $I=1/2$, and will therefore decay into the ${{K} ^\mp } {{\pi } ^\pm } {{\pi } ^\mp }$ final state, or have isospin $I=1$ and therefore will decay into the ${{\pi } ^+} {{\pi } ^-} {{\pi } ^\pm }$ final state. In the ${{K} ^\mp } {{\pi } ^\pm } {{\pi } ^\mp }$ case, the next state in the cascade Y can either be in ${{K} ^\mp } {{\pi } ^\pm }$ or ${{\pi } ^+} {{\pi } ^-}$, referred to as cases (1) and (2), respectively. In the ${{\pi } ^+} {{\pi } ^-} {{\pi } ^\pm }$ case, there is only the ${{\pi } ^+} {{\pi } ^-}$ state, referred to as case (3).

  • 1. $\left[ {{K} ^\mp } {{\pi } ^\pm } \right] {{\pi } ^\mp }$ Example: ${{D} ^0} \rightarrow K_{1}(1270) ^{-} \left[ {{\overline{K}{}} {}^*} (892)^{0} \left[ {{K} ^-} {{\pi } ^+} \right] {{\pi } ^-} \right] {{\pi } ^+}$.

  • 2. ${{K} ^\mp } \left[ {{\pi } ^+} {{\pi } ^-} \right]$ Example: ${{D} ^0} \rightarrow K_{1}(1270) ^{-} [ \rho (770)^{0} \left[ {{\pi } ^-} {{\pi } ^+} \right] {{K} ^-} ] {{\pi } ^+}$.

  • 3. ${{\pi } ^+} {{\pi } ^-} {{\pi } ^\pm }$ Example: ${{D} ^0} \rightarrow a_1(1260)^{+} \left[ \rho (770)^{0} \left[ {{\pi } ^-} {{\pi } ^+} \right] {{\pi } ^+} \right] {{K} ^-}$.

  Two complex parameters can be used to describe cascade decays: the coupling between the ${{D} ^0}$ meson and the first isobar, and then the coupling between the first isobar and the second intermediate state. One of the couplings between isobars can be fixed by convention, typically the dominant channel. For example, for the $a_{1}(1260)^{+}$ resonance, the couplings for subdominant decay chains such as $a_{1}(1260)^{+} \rightarrow \left[ {{\pi } ^+} {{\pi } ^-} \right] ^{L=0}{{\pi } ^+}$ are defined with respect to the dominant $a_{1}(1260)^{+} \rightarrow \rho (770)^{0} {{\pi } ^+}$ decay.

• Quasi two-body decays have the topology ${{D} ^0} \rightarrow X\left[ P_1P_2 \right] Y \left[ P_3P_4\right]$ – the ${{D} ^0}$ meson decays into a pair of unstable states, which in turn each decay to a pair of stable pseudoscalar mesons. The only possibility where X, Y form resonances of conventional quark content is $X \left[ {{K} ^-} {{\pi } ^+} \right] Y \left[ {{\pi } ^+} {{\pi } ^-} \right]$, with an example of a typical process being ${{D} ^0} \rightarrow {{\overline{K}{}} {}^*} (892)^{0}[{{K} ^-} {{\pi } ^+} ]\rho (770)^{0}\left[ {{\pi } ^+} {{\pi } ^-} \right]$. The parameters to be determined describe the coupling between the ${{D} ^0}$ initial state and the quasi two-body state. In the above example, there are three different possible orbital configurations of the vector–vector system, and hence this component has three complex parameters.

Decay chains are described using a product of dynamical functions for each isobar and a spin factor. The amplitude for each decay chain is explicitly made to respect Bose symmetry by summing over both possible permutations of same-sign pions. The total amplitude is then modelled as a coherent sum of these processes. Spin factors are modelled using the Rarita–Schwinger formalism following the prescription in Ref. [24]; the details of this formulation are included in Appendix A.

Resonances are modelled with the relativistic Breit-Wigner function unless otherwise stated, which as a function of the invariant-mass squared, s, takes the form

$$\begin{aligned} \mathcal {T}(s) = \frac{\sqrt{k} B_L(q,0) }{ m_0^2 - s -im_0\Gamma (s) } , \end{aligned}$$

(3)

where the mass of the resonance is $m_0$ and $\Gamma (s)$ is the energy-dependent width. The form factor for a decay in which the two decay products have relative orbital angular momentum L is given by the normalised Blatt–Weisskopf function [25] $B_L(q,0)$, where q is the three-momentum of either decay product in the rest frame of the resonance, and is normalised to unity at zero momentum transfer. The factor k normalises the lineshape integrated over all values of s if the Blatt–Weisskopf form-factor and energy dependence of the width are neglected, and is included to reduce correlations between the coupling to the channel and the mass and width of the resonance.

For a resonance that decays via a single channel to two stable particles, such as $\rho (770)^{0}\rightarrow {{\pi } ^+} {{\pi } ^-}$, the width is given by

$$\begin{aligned} \Gamma (s) = \frac{\Gamma _0 q m_0}{q_0 \sqrt{s}} \left( \frac{q}{q_0}\right) ^{2L} B_L(q,q_0)^2, \end{aligned}$$

(4)

where $\Gamma _0$ is the width at the resonance mass, and $q_0$ is the linear momentum of either decay product evaluated at the rest mass of the resonance. The energy-dependent width of a resonance that decays to a three-body final-state must account for the dynamics of the intermediate decay process, and follows that developed for the decay $\tau ^{+}\rightarrow a_1(1260)^{+}{{\overline{\nu }} _\tau }$ by the CLEO Collaboration in Ref. [26]. The width of a resonance R decaying into three bodies abc can be expressed in terms of the spin-averaged matrix element of the decay $\mathcal {M}_{R\rightarrow abc}$ integrated over the phase space of the three-particle final state,

$$\begin{aligned} \Gamma ( s ) \propto \frac{1}{s} \int ds_{ab} ds_{bc} \left| \mathcal {M}_{R\rightarrow abc} \right| ^2 , \end{aligned}$$

(5)

where the matrix element consists of a coherent sum over the intermediate states in the three-body system, described using the isobar model and using the fitted couplings between the resonance and the intermediate isobars. In the example of the decay of the $a_{1}(1260)^{+}$ resonance, these are predominately the couplings to the $\rho (770)^{0}{{\pi } ^+}$ and $\left[ {{\pi } ^+} {{\pi } ^-} \right] ^{L=0}{{\pi } ^+}$ intermediate states. The width is normalised such that $\Gamma (m_0^2) = \Gamma _0$. In the three-body case, exponential form-factors are used rather than normalised Blatt–Weisskopf functions,

$$\begin{aligned} F(q) = e^{-r^2 q^2 / 2 }, \end{aligned}$$

(6)

where r characterises the radius of the decaying resonance.

The K-matrix formalism [22] provides a convenient description of a two-particle scattering amplitude, which is particularly useful in parameterising S-wave systems. This formulation can then be used in the description of multibody decays on the assumption that rescattering against the other particles in the decay can be neglected. The K-matrix formalism is used in this analysis to describe the ${{\pi } ^+} {{\pi } ^-}$ and ${{K} ^\mp } {{\pi } ^\pm }$ S-waves due to its relative success in parameterising the scalar contributions to three-body decays [27, 28] of the $D$ meson.

The ${{\pi } ^+} {{\pi } ^-}$ S-wave (isoscalar) amplitude is modelled using the K matrix from Ref. [27, 29], which describes the amplitude in the mass range $280 {\mathrm {\,MeV\!/}c^2}< \sqrt{s} < 1900 {\mathrm {\,MeV\!/}c^2}$, considering the effects of five coupled channels, $\pi \pi$, $K K$, $\pi \pi \pi \pi$, $\eta \eta$, $\eta ^{\prime }\eta$, and five poles with masses which generate the resonances. The K matrix also includes polynomial terms that describe nonresonant scattering between hadrons. The coupling to each of these poles and the direct coupling to each of the five channels depend on the production mode, which is modelled using the production vector or P-vector approach, in which the amplitude is

$$\begin{aligned} \mathcal {A}(s) = \left( I-i\hat{\rho } \hat{K}\right) ^{-1} \hat{P}, \end{aligned}$$

(7)

where $\hat{\rho }$ is the two-body phase-space matrix. The complex-valued vector function, $\hat{P}$, has one component for each of the coupled channels, and describes the coupling between the initial state and either one of the poles or a direct coupling to one of these channels. The generic P-vector for the isoscalar K-matrix therefore has 10 complex parameters. An additional complexity in the four-body case is that there are several initial states that couple to the ${{\pi } ^+} {{\pi } ^-}$ S-wave, each of which has its own P vector. Several simplifying assumptions are therefore made to the P vector to avoid introducing an unreasonable number of degrees of freedom. The only direct production terms included in the P vector are to the $\pi \pi$ and $K K$ states, as the production of the ${{\pi } ^+} {{\pi } ^-}$ final state via a direct coupling to another channel all have similar structure below their corresponding production thresholds. The couplings to poles 3, 4 and 5 (where the numbering of the poles is defined in Ref. [29]) are also fixed to zero, as production of these poles only has a small effect within the phase space. This choice reduces the number of free parameters per S-wave production mechanism to four complex numbers. The couplings to the poles are described by $\beta _0$ and $\beta _1$, while the direct couplings to each channel by $f_{\pi \pi }$ and $f_{K K}$. The production vectors used here should therefore be considered as a minimal simplified model. For production of ${{\pi } ^+} {{\pi } ^-}$ S-wave states via resonances, such as the decay chain $a_{1}(1260)^{+} \rightarrow [{{\pi } ^+} {{\pi } ^-} ]^{L=0}{{\pi } ^+}$, improved sensitivity to the structure of the ${{\pi } ^+} {{\pi } ^-}$ state can be achieved by studying a decay mode that produces the $a_{1}(1260)^{+}$ with a larger phase space. In several cases, one or more of these couplings are found to be negligible for a given production mode, and therefore are fixed to zero.

The ${{K} ^\mp } {{\pi } ^\pm }$ S-wave is modelled using the K matrices from the analysis of ${{D} ^+} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+}$ by the FOCUS collaboration [28]. The $I=1/2$ K matrix considers two channels, $K \pi$ and $K \eta ^{\prime }$, and a single pole which is responsible for generating the $K^*(1430)^{0}$ resonance. Additionally, the K matrix includes polynomial terms that describe nonresonant scattering between the hadrons. The ${{K} ^\mp } {{\pi } ^\pm }$ S-wave also contains a $I=3/2$ component. No poles or inelasticity are expected with this isospin, and therefore the associated amplitude can be modelled using a K matrix consisting of a single scalar term.

The $I=1/2$ amplitudes are constructed in the Q-vector [23] approximation. The P vector has the same pole structure as the K matrix, and therefore the approximation

$$\begin{aligned} \hat{K}\hat{P} \approx \hat{\alpha }(s) \end{aligned}$$

(8)

can be made, where $\hat{\alpha }(s)$ is a slowly varying complex vector. This is sometimes referred to as the Q-vector [23] approximation, and allows the insertion of $\hat{K}^{-1} \hat{K}$ into Eq. (7), and the rephrasing of the $I=1/2$ decay amplitude, $\mathcal {A}_{1/2}$, in terms of the T-matrix elements from scattering:

$$\begin{aligned} \mathcal {A}_{1/2} = \alpha _{K \pi } \hat{T}_{11} + \alpha _{K \eta ^{\prime }} \hat{T}_{12}, \end{aligned}$$

(9)

where

$$\begin{aligned} \hat{T} = \left( 1 - i\hat{\rho } \hat{K} \right) ^{-1} \hat{K}, \end{aligned}$$

(10)

which is the transition matrix associated with the $I=1/2$ scattering process. Given the relatively small energy range available to the ${{K} ^\mp } {{\pi } ^\pm }$ system, it is reasonable to approximate $\hat{\alpha }(s)$ as a constant. Inclusion of polynomial terms in $\hat{\alpha }(s)$ is found not to improve the fit quality significantly. The coupling to the $K \eta ^{\prime }$ channel, $\alpha _{K \eta ^{\prime }}$, is defined with respect to the coupling to the $K \pi$ channel, $\alpha _{K \pi }$ in all production modes. If the phase of $\alpha _{K \eta ^{\prime }}$ is zero, the phase shift of the $I=1/2$ component matches that found in scattering experiments, which is the expected result if Watson’s theorem [30] holds for these decays. Similar to the ${{\pi } ^+} {{\pi } ^-}$ S-wave, the components of $\hat{\alpha }$ and the coupling to the $I=3/2$ channel are allowed to differ between production modes.

5 Fit formalism and model construction

Independent fits are performed on the ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ and ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$ data sets, using an unbinned maximum likelihood procedure to determine the amplitude parameters. The formalism of the fit is described in Sects. 5.1–5.3, and the method for systematically selecting plausible models is discussed in Sect. 5.4.

5.1 Likelihood

The probability density functions (PDFs) are functions of position in ${{D} ^0}$ decay phase-space, $\mathbf {x}$, and are composed of the signal amplitude model and the two sources of background described in Sect. 3:

$$\begin{aligned} P(\mathbf {x}) = \varepsilon (\mathbf {x}) \phi (\mathbf {x}) \left( \frac{ Y_s }{\mathcal {N}_s} \left| \mathcal {M}(\mathbf {x}) \right| ^2 + \frac{ Y_c }{\mathcal {N}_c} \mathcal {P}_c(\mathbf {x}) + \frac{ Y_m }{\mathcal {N}_m} |\overline{\mathcal {M}}(\mathbf {x})|^2 \right) . \end{aligned}$$

(11)

The signal PDF is described by the function $\left| \mathcal {M}(\mathbf {x})\right| ^2$, where $\mathcal {M}(\mathbf {x})$ is the total matrix element for the process, weighted by the four-body phase-space density $\phi (\mathbf {x})$, and the phase-space acceptance, $\varepsilon (\mathbf {x})$. The mistag component involving $\overline{\mathcal {M}}(\mathbf {x})$, is only present in the WS sample, and is modelled using the RS signal PDF. The combinatorial background is modelled by $\mathcal {P}_c(\mathbf {x})$, and is present in both samples. The normalisation of each component is given by the integral of the PDF over the phase space, $\mathcal {N}_{i}$, where $i=(c,s,m)$, weighted by the fractional yield, $Y_i$, determined in Sect. 3.

The PDF that describes the combinatorial background in the WS sample is fixed to the results of a fit to the two sidebands of the $m_{K \pi \pi \pi }$ distribution, below $1844.5{\mathrm {\,MeV\!/}c^2}$ and above $1888.5{\mathrm {\,MeV\!/}c^2}$. The components in this model are selected using the same algorithm to determine the resonant content of the signal modes, which is discussed in Sect. 5.4. In this case, the PDF incoherently sums the different contributions and assumes no angular correlations between tracks. The contamination from combinatorial background in the RS sample is very low, and hence this contribution can safely be assumed to be distributed according to phase space, that is $\mathcal {P}_c(\mathbf {x}) =1$.

The function to minimise is

$$\begin{aligned} \mathcal {L} = -2 \sum _{i \in \mathrm {data} } \log ( P(\mathbf {x}_i) ). \end{aligned}$$

(12)

As the efficiency variation across the phase space factorises in the PDF, these variations result in a constant shift in the likelihood everywhere except the normalisation integrals, and hence can be neglected in the minimisation procedure. Efficiency variations can then be included in the fit by performing all integrals using simulated events that have been propagated through the full LHCb detector simulation and selection. These events are referred to as the integration sample. The values of the normalisation integrals are independent of the generator distribution of the integration sample, however the uncertainties on the integrals are minimised when integration events approximate the function being integrated, which is known as importance sampling. Therefore, integration samples are generated using preliminary models that do not include efficiency effects.

5.2 Goodness of fit

The quality of fits is quantified by computing a $\chi ^2$ metric. Candidates are binned using an adaptive binning scheme. Five coordinates are selected, and the phase space is repeatedly divided in these coordinates such that each bin contains the same number of candidates, following the procedure described in Ref. [4]. The division is halted when each bin contains between 10 and 20 entries. This procedure results in 32,768 approximately equally populated bins for the RS sample, and 256 for the WS sample. Five two- and three-body invariant mass-squared combinations are used as coordinates for the binning procedure, $s_{{{\pi } ^+} {{\pi } ^-} {{\pi } ^+}}, s_{{{K} ^-} {{\pi } ^+}}, s_{{{K} ^-} {{\pi } ^-}}, s_{{{\pi } ^+} {{\pi } ^-}}$ and $s_{{{K} ^-} {{\pi } ^+} {{\pi } ^-}}$. The $\chi ^2$ is defined as

$$\begin{aligned} \chi ^2 = \sum _{i \in \mathrm {bins}} \frac{ \left( N_{i} - \langle {N}_{i} \rangle \right) ^2 }{ N_{i} + \bar{\sigma }_{i}^{2} } , \end{aligned}$$

(13)

where $N_i$ is the observed number of candidates in bin i and $\langle {N}_i \rangle$, the expected number of entries determined by reweighting the integration sample with the fitted PDF. The statistical uncertainty from the limited size of the integration sample, $\bar{\sigma }_i$, is included in the definition of the $\chi ^2$, and is estimated as

$$\begin{aligned} \bar{\sigma }_i^2 = \sum _{j \in \mathrm {bin}(i)} \omega _j^2 , \end{aligned}$$

(14)

where $\omega _j$ is the weight of integration event j. The $\chi ^2$ per degree of freedom is used as the metric to optimise the decay chains included in a model, using the model-building procedure described in Sect. 5.4.

5.3 Fit fractions

The values of coupling parameters depend strongly on various choices of convention in the formalism. Therefore, it is common to define the fractions in the data sample associated with each component of the amplitudes (fit fractions). In the limit of narrow resonances, the fit fractions are analogous to relative branching fractions. The fit fraction for component i is

$$\begin{aligned} \mathcal {F}_{i} = \frac{ \int d \mathbf {x}\left| \mathcal {M}_i (\mathbf {x}) \right| ^2 }{ \int d \mathbf {x}\left| \sum _{j} \mathcal {M}_j(\mathbf {x}) \right| ^2 }. \end{aligned}$$

(15)

For cascade processes, the different secondary isobars contribute coherently to the fit fractions. The partial fit fractions for each sub-process are then defined as the fit fraction with only the contributions from the parent isobar included in the denominator.

5.4 Model construction

The number of possible models that could be used to fit the amplitudes is extremely large due to the large number of possible decay chains ($\approx 100$). A full list of the components considered is included in Appendix B.

A model of “reasonable” complexity typically contains $\mathcal {O}(10)$ different decay chains. Therefore, the number of possible models is extremely large, and only an infinitesimal fraction of these models can be tested. An algorithmic approach to model building is adopted, which begins with an initial model and attempts to iteratively improve the description by adding decay chains. For ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ the initial model is that constructed by the Mark III collaboration [6], augmented by knowledge from other analyses, such as the additional decay channels of the $a_1(1260)^{+}$ found in the amplitude analysis of the decay ${{{D} ^0} \rightarrow {{\pi } ^+} {{\pi } ^-} {{\pi } ^+} {{\pi } ^-}}$ performed by the FOCUS collaboration [31]. The two-body nonresonant terms in the Mark III model are replaced with the relevant K matrices, and the four-body nonresonant term replaced with a quasi two-body scalar–scalar term $[{{K} ^-} {{\pi } ^+} ]^{L=0}[{{\pi } ^+} {{\pi } ^-} ]^{L=0}$, modelled using a product of K matrix amplitudes.

For ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$, where no previous study exists, the initial model is obtained by inspection of the invariant-mass distributions. There are clear contributions from the ${{K} ^*} (892)^{0}$ and $\rho (770)^{0}$ resonances, and therefore combined with the expectation that the vector–vector contributions should be similar between WS and RS, the quasi two-body mode ${{D} ^0} \rightarrow {{K} ^*} (892)^{0}\rho (770)^{0}$ is included in all three allowed orbital states $L=(0,1,2)$. The scalar–scalar contribution should also be comparable between WS and RS decay modes, and hence the quasi two-body term ${{D} ^0} \rightarrow [{{K} ^+} {{\pi } ^-} ]^{L=0} [{{\pi } ^+} {{\pi } ^-} ]^{L=0}$ is also included.

The steps of the model-building procedure are

• 1. Take a model and a set of possible additional decay chains, initially the complete set discussed in Appendix. B. Perform a fit to the data using this model adding one of these decay chains.

• 2. If adding this decay chain improves the $\chi ^2$ per degree of freedom by at least 0.02, then retain the model for further consideration.

• 3. On the first iteration, restrict the pool of decay chains that are added to the model to those 40 contributions that give the largest improvements to the fit.

• 4. Reiterate the model-building procedure, using the 15 models with the best fit quality from step 2 as starting points. Finish the procedure if no model has improved significantly.

The model-building procedure therefore results in an ensemble of parametrisations of comparable fit quality.

6 Fit results

This section presents fit results and systematic uncertainties, with the latter discussed first in Sect. 6.1. The model-building procedure outlined in Sect. 5.4 results in ensembles of parameterisations of comparable fit quality. The models discussed in this section, which are referred to as the baseline models, and are built to include all decay chains that are common to the majority of models that have a $\chi ^2$ per degree of freedom differing from the best-fitting models by less than 0.1. The results for these baseline models are shown and their features discussed in Sects. 6.2 and 6.3 for the RS decay and the WS decay, respectively. The general features of models in the ensembles are discussed in Sect. 6.4. In Sect. 6.5 the models are used to calculate the coherence factor of the decays, and an assessment is made of the stability of the predicted coherence factors, strong-phase differences and amplitude ratios with respect to the choice of WS model in regions of phase space.

6.1 Systematic uncertainties

Several sources of systematic uncertainty are considered. Experimental issues are discussed first, followed by uncertainties related to the model and the formalism.

All parameters in the fit have a systematic uncertainty originating from the limited size of the integration sample used in the likelihood minimisation. This effect is reduced by importance sampling. The remaining uncertainty is estimated using a resampling technique. Half of the integration sample is randomly selected, and the fit performed using only this subsample. This procedure is repeated many times, and the systematic uncertainty from the finite integration statistics is taken to be $1/\sqrt{2}$ of the spread in fit parameters.

There is an additional systematic uncertainty due to the imperfect simulation, which affects the efficiency corrections. The RS data are divided into bins in the ${{D} ^0}$ transverse momentum, in which the efficiency corrections may be expected to vary, and the fit is performed independently in each bin. The results of these fits are combined in an uncertainty-weighted average, including the correlations between the different parameters, and the absolute difference between the parameters measured by this procedure and the usual fitting procedure is assigned as the systematic uncertainty. Additionally, the data is divided by data-taking year and software trigger category and independent fits performed using these subsamples. The fit results are found to be compatible within the assigned uncertainties between these samples, hence no additional systematic uncertainty is assigned.

The uncertainty associated with the determination of the signal fraction and mistag fraction in each sample is measured by varying these fractions within the uncertainties found in the fit to the $m_{K \pi \pi \pi }$ vs. $\Delta {m}$ plane.

Parameters that are fixed in the fit, such as the $\rho (770)^{0}$ mass and width, are randomly varied according to the uncertainties given in Ref. [21], and the corresponding spreads in fit results are assigned as the uncertainties. It is assumed that input correlations between these parameters are negligible. When performing fits to the WS sample, several parameters, such as the mass, width and couplings of the $K_1(1270)^{\pm }$ resonance, are fixed to the values found in the RS fit. The uncertainty on these parameters is propagated to the WS fit by randomly varying these parameters by their uncertainties. The radii of several particles used in the Blatt–Weisskopf form factor are varied using the same procedure. The ${{D} ^0}$ radial parameter is varied by $\pm \,0.5\mathrm {\,GeV} ^{-1}c$.

The uncertainty due to the background model in the WS fit is estimated using pseudo-experiments. A combination of simulated signal events generated with the final model and candidates from outside of the ${D} ^0$ signal region is used to approximate the real data. The composite dataset is then fitted using the signal model, and differences between the true and fitted values are taken as the systematic uncertainties on the background parametrisation.

The choice of model is an additional source of systematic uncertainty. It is not meaningful to compare the coupling parameters between different parametrisations, as these are by definition the parameters of a given model. It is however useful to consider the impact the choice of parametrisation has on fit fractions and the fitted masses and widths. Therefore, the model choice is not included in the total systematic uncertainty, but considered separately in Sect. 6.4 and 6.5.

The total systematic uncertainty is obtained by summing the components in quadrature. The total systematic uncertainty is significantly larger than the statistical uncertainty on the RS fit, with the largest contributions coming from the form factors that account for the finite size of the decaying mesons. For the WS fit, the total systematic uncertainty is comparable to the statistical uncertainty, with the largest contribution coming from the parametrisation of the combinatorial background. A full breakdown of the different sources of systematic uncertainty for all parameters is given in Appendix C.

Fig. 2

Distributions for six invariant-mass observables in the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. Bands indicate the expectation from the model, with the width of the band indicating the total systematic uncertainty. The total background contribution, which is very low, is shown as a filled area. In figures that involve a single positively-charged pion, one of the two identical pions is selected randomly

Table 2 Fit fractions and coupling parameters for the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. For each parameter, the first uncertainty is statistical and the second systematic. Couplings g are defined with respect to the coupling to the channel ${{D} ^0} \rightarrow [{{\overline{K}{}} {}^*} (892)^{0}\rho (770)^{0}]^{L=2}$. Also given are the $\chi ^2$ and the number of degrees of freedom ($\nu$) from the fit and their ratio

6.2 Results for the RS decay

Invariant mass-squared projections for ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ are shown in Fig. 2 together with the expected distributions from the baseline model. The coupling parameters, fit fractions and other quantities for this model are shown in Table 2. The $\chi ^2$ per degree of freedom for this model is calculated to be $40483/32701 = 1.238$, which indicates that although this is formally a poor fit, the model is providing a reasonable description of the data given the very large sample size. Three cascade contributions, from $a_1(1260)^+$, $K _1(1270)^-$ and $K (1460)^-$ resonances, are modelled using the three-body running-width treatment described in Sect. 4. The masses and widths of these states are allowed to vary in the fit. The mass, width and coupling parameters for these resonances are presented in Tables 3, 4 and 5. The values of these parameters are model dependent, in particular on the parametrisation of the running width described by Eq. (5) and of the form factors described by Eq. (6), and thus there is not a straightforward comparison with the values obtained by other experiments.

The largest contribution is found to come from the axial vector $a_1(1260)^{+}$, which is a result that was also obtained in the Mark III analysis [6]. This decay proceeds via the colour-favoured external W-emission diagram that is expected to dominate this final state.

There are also large contributions from the different orbital angular momentum configurations of the quasi two-body processes ${{D} ^0} \rightarrow {{\overline{K}{}} {}^*} (892)^{0}\rho (770)^{0}$, with a total contribution of around $20\%$. The polarisation structure of this component is not consistent with naive expectations, with the D wave being the dominant contribution and the overall hierarchy being $\mathrm {D}>\mathrm {S}>\mathrm {P}$. This result may be compared with that obtained for the study ${{D} ^0} \rightarrow \rho (770)^{0}\rho (770)^{0}$ in Ref. [32], where the D-wave polarisation of the amplitude was also found to be dominant.

Fig. 3

Argand diagram for the model-independent partial-wave analysis (MIPWA) for the $K (1460)$ resonance. Points show the values of the amplitude that are determined by the fit, with only statistical uncertainties shown

A significant contribution is found from the pseudoscalar state $K (1460)^{-}$. This resonance is a $2^{1}S_0$ excitation of the kaon [33]. Evidence for this state has been reported in partial-wave analyses of the process ${{K} ^\pm } {p} \rightarrow {{K} ^\pm } {{\pi } ^+} {{\pi } ^-} {p}$ [34, 35], manifesting itself as a $0^{-}$ state with mass $\approx 1400{\mathrm {\,MeV\!/}c^2}$ and width $\approx 250{\mathrm {\,MeV\!/}c^2}$, coupling to the ${{\overline{K}{}} {}^*} (892)^{0}{{\pi } ^-}$ and $[{{\pi } ^-} {{\pi } ^+} ]^{L=0}{{K} ^-}$ channels. The intermediate decays of the $K (1460)^{-}$ meson are found to be roughly consistent with previous studies, with approximately equal partial widths to ${{\overline{K}{}} {}^*} (892)^{0}{{\pi } ^-}$ and $[{{\pi } ^+} {{\pi } ^-} ]^{L=0}{{K} ^-}$. The resonant nature of this state is confirmed using a model-independent partial-wave analysis (MIPWA), following the method first used by the E791 collaboration [36, 37]. The relativistic Breit-Wigner lineshape is replaced by a parametrisation that treats the real and imaginary parts of the amplitude at 15 discrete positions in $s_{{{K} ^-} {{\pi } ^+} {{\pi } ^-}}$ as independent pairs of free parameters to be determined by the fit. The amplitude is then modelled elsewhere by interpolating between these values using cubic splines [38]. The Argand diagram for this amplitude is shown in Fig. 3, with points indicating the values determined by the fit, and demonstrates the phase motion expected from a resonance.

Four-body weak decays contain amplitudes that are both even, such as $D \rightarrow [VV^{\prime }]^{L=0,2}$, where V and $V^{\prime }$ are vector resonances, and odd, such as $D \rightarrow [VV^{\prime }]^{L=1}$, under parity transformations. Interference between these amplitudes can give rise to parity asymmetries which are different in ${{D} ^0}$ and ${{\overline{D}{}} {}^0}$ decays. These asymmetries are the result of strong-phase differences, but can be mistaken for ${C\!P}$ asymmetries [39]. Both sources of asymmetry can be studied by examining the distribution of the angle between the decay planes of the two quasi two-body systems, $\phi$, which can be constructed from the three-momenta $\mathbf {p}$ of the decay products in the rest frame of the ${D} ^0$ meson as

$$\begin{aligned} \cos (\phi )&= \hat{\mathbf {n}}_{{{K} ^-} \pi ^{+}} \cdot \hat{\mathbf {n}}_{{{\pi } ^-} \pi ^{+}}\nonumber \\ \sin (\phi )&= \frac{\mathbf {p}_{\pi ^{+}} \cdot \hat{\mathbf {n}}_{{{K} ^-} \pi ^{+}}}{ \left| \mathbf {p}_{\pi ^{+}} \times \hat{\mathbf {p}}_{{{K} ^-} \pi ^{+}} \right| } , \end{aligned}$$

(16)

where $\hat{\mathbf {n}}_{ab}$ is the direction normal to the decay plane of a two-particle system ab,

$$\begin{aligned} \hat{\mathbf {n}}_{ab} = \frac{ \mathbf {p}_a \times \mathbf {p}_b }{\left| \mathbf {p}_a \times \mathbf {p}_b \right| }, \end{aligned}$$

(17)

and $\hat{\mathbf {p}}_{{{K} ^-} {{\pi } ^+}}$ is the direction of the combined momentum of the ${{K} ^-} {{\pi } ^+}$ system.

The interference between P-even and P-odd amplitudes averages to zero when integrated over the entire phase space. Therefore, the angle $\phi$ is studied in regions of phase space. The region of the ${{\overline{K}{}} {}^*} (892)^{0}$ and $\rho (770)^{0}$ resonances is studied as the largest P-odd amplitude is the decay ${{D} ^0} \rightarrow [{{\overline{K}{}} {}^*} (892)^{0}\rho (770)^{0}]^{L=1}$. Selecting this region allows the identical pions to be distinguished, by one being part of the ${{\overline{K}{}} {}^*} (892)^{0}$-like system and the other in the $\rho (770)^{0}$-like system. The data in this region are shown in Fig. 4, divided into quadrants of helicity angles, $\theta _A$ and $\theta _B$, defined as the angle between the ${{K} ^-}/{{\pi } ^-}$ and the ${{D} ^0}$ in the rest frame of the ${{K} ^-} {{\pi } ^+}/{{\pi } ^-} {{\pi } ^+}$ system. The distributions show clear asymmetries under reflection about $180^{\circ }$, indicating parity nonconservation. However, equal and opposite asymmetries are observed in the ${C\!P}$-conjugate mode ${{\overline{D}{}} {}^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$, indicating that these asymmetries originate from strong phases, rather than from ${C\!P}$-violating effects. Bands show the expected asymmetries based on the amplitude model, which has been constructed according to the ${C\!P}$-conserving hypothesis, and show reasonable agreement with the data.

Fig. 4

Parity violating distributions for the RS decay in the ${{\overline{K}{}} {}^*} (892)^{0}\rho (770)^{0}$ region defined by $\pm 35 \mathrm {\,MeV}$($\pm 100 \mathrm {\,MeV}$) mass windows about the nominal ${{\overline{K}{}} {}^*} (892)^{0}$ $(\rho (770)^{0})$ masses. Bands show the predictions of the fitted model including systematic uncertainties

Table 3 Table of fit fractions and coupling parameters for the component involving the $a_{1}(1260)^{+}$ meson, from the fit performed on the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. The coupling parameters are defined with respect to the $a_{1}(1260)^{+} \rightarrow \rho (770)^{0}{{\pi } ^+}$ coupling. For each parameter, the first uncertainty is statistical and the second systematic

Table 4 Table of fit fractions and coupling parameters for the component involving the $K_{1}(1270) ^{-}$ meson, from the fit performed on the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. The coupling parameters are defined with respect to the $K_{1}(1270) ^{-}\rightarrow \rho (770)^{0}{{K} ^-}$ coupling. For each parameter, the first uncertainty is statistical and the second systematic

Table 5 Table of fit fractions and coupling parameters for the component involving the $K(1460)^{-}$ meson, from the fit performed on the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. The coupling parameters are defined with respect to the $K(1460)^{-} \rightarrow {{\overline{K}{}} {}^*} (892)^{0}{{\pi } ^-}$ coupling. For each parameter, the first uncertainty is statistical and the second systematic

Fig. 5

Distributions for six invariant-mass observables in the WS decay ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$. Bands indicate the expectation from the model, with the width of the band indicating the total systematic uncertainty. The total background contribution is shown as a filled area, with the lower region indicating the expected contribution from mistagged ${{\overline{D}{}} {}^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$ decays. In figures that involve a single negatively-charged pion, one of the two identical pions is selected randomly

Table 6 Fit fractions and coupling parameters for the WS decay ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$. For each parameter, the first uncertainty is statistical and the second systematic. Couplings g are defined with respect to the coupling to the decay ${{D} ^0} \rightarrow [K ^{*}(892)^{0}\rho (770)^{0}]^{L=2}$. Also given are the $\chi ^2$ and the number of degrees of freedom ($\nu$) from the fit and their ratio

6.3 Results for the WS decay

Invariant mass-squared distributions for ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$ are shown in Fig. 5. Large contributions are clearly seen in $s_{{{K} ^+} {{\pi } ^-}}$ from the ${{K} ^*} (892)^{0}$ resonance. The fit fractions and amplitudes of the baseline model are given in Table 6. The $\chi ^2$ per degree of freedom for the fit to the WS data is $350/243=1.463$. If the true WS amplitude has a comparable structure to the RS amplitude, it contains several decay chains at the $\mathcal {O}(1\%)$ level that cannot be satisfactorily resolved given the small sample size, and hence the quality of the WS fit is degraded by the absence of these subdominant contributions.

Dominant contributions are found from the axial kaons, $K _1(1270)^{+}$ and $K _1(1400)^{+}$, which are related to the same colour-favoured W-emission diagram that dominates the RS decay, where it manifests itself in the $a_{1}(1260)^{+} {{K} ^-}$ component. The contribution from the $K _1(1400)^{+}$ resonance is larger than that from the $K _1(1270)^{+}$ resonance. It is instructive to consider this behaviour in terms of the quark states, $^1P_1$ and $^3P_1$, which mix almost equally to produce the mass eigenstates,

$$\begin{aligned} | K _1(1400) \rangle&= \cos (\theta _K) | ^3P_1\rangle - \sin (\theta _K) | ^1P_1 \rangle \nonumber \\ | K _1(1270) \rangle&= \sin (\theta _K) | ^3P_1\rangle + \cos (\theta _K) | ^1P_1 \rangle , \end{aligned}$$

(18)

where $\theta _K$ is a mixing angle. The mixing is somewhat less than maximal, with Ref. [40] reporting a preferred value of $\theta _K = (33^{+6}_{-2})^{\circ }$. In the WS decay, the axial kaons are produced via a weak current, which is decoupled from the $^1P_1$ state in the ${\mathrm {SU}(3)}$ flavour-symmetry limit. If the mixing were maximal, the mass eigenstates would be produced equally, but a smaller mixing angle results in a preference for the $K _1(1400)$, which is qualitatively consistent with the pattern seen in data. In the RS decay, the axial kaons are not produced by the external weak current, and hence there is no reason to expect either quark state to be preferred. The relatively small contribution from the $K _1(1400)$ is then understood as a consequence of approximately equal production of the quark states.

The coupling and shape parameters of the $K _1(1270)^{+}$ resonance are fixed to the values measured in the RS nominal fit. A fit is also performed with these coupling parameters free to vary, and the parameters are found to be consistent with those measured in the RS decay.

A large contribution is found from ${{D} ^0} \rightarrow \rho (1450)^{0}{{K} ^*} (892)^{0}$ decays in all models that describe the data well. This contribution resembles a quasi nonresonant component due to the large width of the $\rho (1450)^0$ resonance, and is likely to be an effective representation of several smaller decay chains involving the ${{K} ^*} (892)^0$ resonance that cannot be resolved with the current sample size.

6.4 Alternative parametrisations

Table 7 Decay chains taken into account in alternative parametrisations of the RS decay mode ${{{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}}$. For each chain, the fraction of models in the ensemble that contain this decay, together with the associated average fit fraction, $\langle \mathcal {F} \rangle$, are shown. Components are not tabulated if they contribute to all models in the ensemble, or if they contribute to less than 5% of the models

Table 8 Dependence of fit fractions (and partial fractions) on the choice of the RS model. This dependence is expressed as the mean value and the RMS of the values in the ensemble. Also shown is the fit fractions of the baseline model presented in Sect. 6.2

Table 9 Dependence of the fitted masses and widths on the final choice of the RS model. This dependence is expressed as the mean value and the RMS of the values in the ensemble. The values found for the baseline model presented in Sect. 6.2 are reported for comparison

The model-finding procedure outlined in Sect. 5.4 results in ensembles of parametrisations of comparable quality and complexity. The decay chains included in the models discussed above are included in the majority of models of acceptable quality, with further variations made by addition of further small components. The fraction of models in this ensemble containing a given decay mode are shown in Table 7 for the RS decay mode with the average fit fraction associated with each decay chain also tabulated. The ensemble of RS models consists of about 200 models with $\chi ^2$ per degree of freedom varying between 1.21 and 1.26. Many of the decay chains in the ensemble include resonances, such as the $K _1(1270)^{-}$, decaying via radially excited vector mesons, such as the $\rho (1450)^{0}$ and ${{K} ^*} (1410)^{0}$ mesons. In particular, the decay $K _1(1270)^{-}\rightarrow \rho (1450)^{0}{{K} ^-}$ is included in the models discussed in Sects. 6.2 and 6.3 and is found in the majority of the models in the ensemble. This decay channel of the $K _1(1270)^{-}$ meson has a strong impact at low dipion masses due to the very large width of the $\rho (1450)^{0}$ resonance, of about 400${\mathrm {\,MeV\!/}c^2}$. Models excluding this component are presented as alternative parametrisations in Appendix E as this decay mode has not been studied extensively in other production mechanisms of the $K _1(1270)^{-}$ resonance, and the ensemble contains models without this decay chain of similar fit quality to the baseline model. The situation can be clarified with independent measurements of the properties of these resonances. The $a_1(1640)^{+}$ resonance is also found in many models in the ensemble, and is likely to be present at some level despite only the low-mass tail of this resonance impacting the phase space. This resonance strongly interferes with the dominant $a_1(1260)^{+}$ component and, as the parameters of this resonance are poorly known, improved external inputs will be required to correctly constrain this component.

The coupling parameters cannot strictly be compared between different models, as in many cases these coupling parameters have a different interpretation depending on the choice of the model. However, it is instructive to consider how the fit fractions vary depending on the choice of model, which is shown in Table 8. It is also useful to consider how the choice of model impacts upon the fitted masses and widths, which is shown in Table 9. The values for the model described in Sect. 6.2 are also shown, which has compatible values with the ensemble. The variation with respect to the choice of model is characterised by the RMS of the parameters in the ensemble, and is of a comparable size to the combined systematic uncertainty from other sources on these parameters.

Table 10 Decay chains taken into account in alternative parametrisations of the WS decay mode ${{{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}}$. For each chain, the fraction of models in the ensemble that contain this decay, together with the associated average fit fraction, $\langle \mathcal {F} \rangle$, are shown. Components are not tabulated if they contribute to all models in the ensemble, or if they contribute to less than 5% of the models

The ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$ ensemble consists of 108 models, all of which have a $\chi ^2$ per degree of freedom of less than 1.45, with the best models in the ensemble having a $\chi ^2$ per degree of freedom of about 1.35. The fraction of models in this ensemble containing a given decay mode are shown in Table 10. In particular, there should be percent-level contributions from some of the decay chains present in the ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ mode, such as ${{D} ^0} \rightarrow a_1(1260)^{-}{{K} ^+}$ and ${{D} ^0} \rightarrow K ^{*}(892)^{0}\left[ {{\pi } ^+} {{\pi } ^-} \right] ^{L=0}$. In addition to the marginal decays of the $K _1(1270)^{+}$ present in the ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$ ensemble, the models suggest contributions from the $K ^*(1680)$, which resembles a nonresonant component due to its large width and position on the edge of the phase space. As is the case for the large ${{D} ^0} \rightarrow K ^*(892)^{0}\rho (1450)$ component, this contribution is likely to be mimicking several smaller decay channels that cannot be resolved with the current sample size.

6.5 Coherence factor

The coherence factor $R_{K 3\pi }$ and average strong-phase difference $\delta _{K 3\pi }$ are measures of the phase-space-averaged interference properties between suppressed and favoured amplitudes, and are defined as [41]

$$\begin{aligned} R_{K 3\pi } e^{-i \delta _{K 3\pi }} = \frac{ \int d\mathbf {x}\mathcal {A} \left( {{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+} \right) \mathcal {A}^{*}\left( {{\overline{D}{}} {}^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+} \right) }{ \sqrt{ \int d\mathbf {x}\left| \mathcal {A}\left( {{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+} \right) \right| ^2 \int d\mathbf {x}\left| \mathcal {A}\left( {{\overline{D}{}} {}^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+} \right) \right| ^2 } }, \end{aligned}$$

(19)

where $\mathcal {A}({{D} ^0} \rightarrow {{K} ^+} 3\pi )$ is the amplitude of the suppressed decay and $\mathcal {A}({{\overline{D}{}} {}^0} \rightarrow {{K} ^+} 3\pi )$ is the favoured amplitude for ${{\overline{D}{}} {}^0}$ decays. Additionally, it is useful to define the average ratio of amplitudes as

$$\begin{aligned} r_{K 3\pi } = \sqrt{ \frac{ \int d\mathbf {x}\left| \mathcal {A}\left( {{{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}}\right) \right| ^2 }{\int d\mathbf {x}\left| \mathcal {A}\left( {{{\overline{D}{}} {}^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}}\right) \right| ^2} }. \end{aligned}$$

(20)

Knowledge of these parameters is necessary when making use of this decay in ${{{B} ^-}} \rightarrow D {{K} ^-}$ transitions for measuring the $C\!P$-violating phase $\gamma$ [41], and can also be exploited for charm mixing studies. Observables with direct sensitivity to the coherence factor and related parameters have been measured in $e^{+}e^{-}$ collisions at the $\psi (3770)$ resonance with CLEO-c data [42], and through charm mixing at LHCb [4]. A global analysis of these results [42] yields

$$\begin{aligned} R_{K 3\pi }&= 0.43^{+0.17}_{-0.13} \nonumber \\ \delta _{K 3\pi }&= (128^{+28}_{-17})^{\circ } \nonumber \\ r_{K3\pi }&= (5.49\pm 0.06)\times 10^{-2} . \end{aligned}$$

The baseline models presented in Sect. 6 can be used to calculate the model-derived coherence factor

$$\begin{aligned} R_{K 3\pi }^{\mathrm {mod}} = 0.458 \pm 0.010 \pm 0.012 \pm 0.020, \end{aligned}$$

where the first uncertainty is statistical, the second systematic, and the third the uncertainty from the choice of WS model. This uncertainty is assigned by taking the spread in values from an ensemble of alternative models from the model-building algorithm, requiring that models have a $\chi ^2$ per degree of freedom of less than 1.5, and that all unconstrained components in the fit have a significance of $> 2\sigma$. This result is in good agreement with the direct measurement in Ref. [42]. This analysis has no sensitivity to $\delta _{K 3\pi }$ and $r_{K 3\pi }$ as each amplitude contains an arbitrary independent amplitude and phase.

The stability of the local phase description can also be verified by evaluating the model-derived coherence factor and associated parameters in different regions of phase space. This is equivalent to changing the definition of Eq. (19) such that integrals are performed over a limited region of phase space. In this case, it is also possible to determine the local values of $\delta _{K 3\pi }$ and $r_{K 3\pi }$ relative to the phase-space averaged values. Therefore, overall normalisation factors are fixed such that the central values of the direct measurement are correctly reproduced.

In order to define these regions, the phase space is divided into hypercubes using the algorithm described in Sect. 5.2. The division is done such that the hypercubes cannot be smaller in any dimension than $50{\mathrm {\,MeV\!/}c^2}$. The hypercubes are grouped into bins of average phase difference between the two amplitudes in the bin, using the amplitude models described in Sect. 6. The range $[-\,180^{\circ },180^{\circ }]$ in phase difference between the two decay modes is split into eight bins. The division of this range is done such that each bin is expected to have an approximately equal population of WS events within the bin. The coherence factors, average strong phases and amplitude ratios and their RMS spread arising from the choice of WS model are summarised in Table 11. Good stability with respect to the choice of model is observed, which is a consequence of the dominant features of the amplitude being common for all models, and gives confidence to using the models presented in this paper to define regions of interest for future binned measurements of $\gamma$ or studies of charm mixing. The relatively high coherence factor in some regions of phase-space demonstrates the potential improvements in sensitivity to measurements of ${C\!P}$-violating observables.

Table 11 Coherence factor and average strong-phase differences in regions of phase space. The spread of coherence factors, average strong-phase difference and ratio of amplitudes from choice of WS model characterised with the RMS of the distribution

7 Conclusions

The four-body decay modes ${{D} ^0} \rightarrow {{K} ^\mp } {{\pi } ^\pm } {{\pi } ^\pm } {{\pi } ^\mp }$ have been studied using high-purity time-integrated samples obtained from doubly tagged ${{\overline{B}{}}} \rightarrow {{D} ^{*+}} (2010)[{{D} ^0} {{\pi } ^+} ]\mu X$ decays. For the RS decay mode ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$, the analysis is performed with a sample around sixty times larger than that exploited in any previous analysis of this decay. For the WS mode ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$, the resonance substructure is studied for the first time with $\approx 3000$ signal candidates.

Both amplitude models are found to have large contributions from axial resonances, the decays ${{D} ^0} \rightarrow a_1(1260)^{+}{{K} ^-}$ and ${{D} ^0} \rightarrow K _1(1270/1400)^{+}{{\pi } ^-}$ for ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ and ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$, respectively. This is consistent with the general picture that W-emission topologies are crucial in describing these decays. Interference between the parity-even and parity-odd amplitudes causes large local parity violations, which are shown to be reasonably well modelled in the RS decay. A significant contribution from the pseudoscalar resonance $K (1460)^{-}$ is identified, which is validated using the model-independent partial waves method.

The coherence factor is calculated using the models, and is found to be consistent with direct measurements. It is found that the calculated value is relatively stable with respect to the parametrisation of subdominant amplitudes in the WS model. These models therefore provide a valuable input to future binned measurements of the ${C\!P}$-violating parameter $\gamma$ and charm-mixing studies.

Appendices

Appendices

A Spin formalism

The effects of spin and orbital angular momentum are calculated using the Rarita–Schwinger formalism, following a similar prescription to that described in Ref. [24]. Spin-matrix elements for quasi two-body processes are constructed in terms of a series of polarisation and pure orbital angular momentum tensors. Consider the decay of particle a that has integer spin J, into particles b and c, which have integer spin $s_b$, $s_c$, respectively. All three particles have an associated polarisation tensor, $\epsilon ^{(a,b,c)}$, which is of rank equal to the spin of the particle. The decay products b, c will also in general have a relative orbital angular momentum l, which is expressed in terms of the pure orbital angular momentum tensor, $L_{\mu \ldots \nu }$, which is of rank l. The matrix element for the decay is

$$\begin{aligned} \mathcal {M}_{a\rightarrow bc} = \epsilon _{\mu _a \ldots \nu _a}^{(a)*} \epsilon _{\mu _b \ldots \nu _b }^{(b)} \epsilon _{\mu _c \ldots \nu _c }^{(c)} L_{ \mu _l \ldots \nu _l }^{(l)} G^{ \mu _a \ldots \nu _a \mu _b \ldots \nu _b \mu _c \ldots \nu _c \mu _l \ldots \nu _l } , \end{aligned}$$

(21)

where the tensor $G^{\ldots }$ combines the polarisation and pure orbital angular momentum tensor to produce a scalar object. This tensor is constructed from combinations of the metric tensor $g_{\mu \nu }$ and the Levi-Civita tensor contracted with the four-momenta of the decaying particle, $\varepsilon _{\mu \nu \alpha \beta }P^{\mu }$. The second of these tensors is used only if $J-(l-s_b-s_c)$ is odd, and ensures that matrix elements have the correct properties under parity transformations. The matrix element can also be written by defining the current, $\mathcal {J}$, of the decaying particle:

$$\begin{aligned} \mathcal {M}_{a\rightarrow bc} = \epsilon _{\underline{\mu }}^{(a)*}\mathcal {J}^{(a)\underline{\mu }} , \end{aligned}$$

(22)

where the $\underline{\mu }$ represents a set of Lorentz indices $\mu \ldots \nu$, a shorthand which will be used throughout this section. The isobar model factorises an N-body decay into a sequence of two-body processes. Each of these quasi two-body decays can be described with a single spin matrix element, and hence the total matrix element is the product of $N-1$ matrix elements:

$$\begin{aligned} \mathcal {M} = \prod _{i=0}^{N-1} \mathcal {M}_{a_i \rightarrow b_ic_i}. \end{aligned}$$

(23)

For example, consider the quasi two-body decay $P\rightarrow X [ab] Y[cd]$. The matrix element for this decay is

$$\begin{aligned} \mathcal {M} = \sum _i \sum _j \mathcal {M}_{P \rightarrow X_iY_j} \mathcal {M}_{X_i\rightarrow ab } \mathcal {M}_{Y_j\rightarrow cd}, \end{aligned}$$

(24)

where the sums are over the possible polarisations of the intermediate states.

It is preferable to build a generic formulation of the total matrix element for arbitrary topologies, spins and angular momenta, rather than performing an explicit computation for each possible process. A generic approach to computing matrix elements is to introduce a generalised “current” associated with a decaying particle that has absorbed the matrix elements of its decay products. This current can be written in terms of the currents of its decay products as

$$\begin{aligned} \mathcal {J}^{\underline{\mu }} = L^{ (l) }_{\underline{\beta }} G^{ \underline{\mu \nu \alpha \beta } } \times \left( \mathcal {S}^{1}_{\underline{\nu \gamma }} \mathcal {J}_1^{\underline{\gamma }} \right) \times \left( \mathcal {S}^{2}_{\underline{\alpha \eta }} \mathcal {J}_2^{\underline{\eta }} \right) , \end{aligned}$$

(25)

where $\mathcal {S}^{1,2}_{\underline{\mu }}$ is the spin-projection operator of decay products (1,2), which has been used to sum intermediate polarisation tensors, using the definition

$$\begin{aligned} \sum _i \epsilon _{i\underline{a}} {\epsilon _{i\underline{b}}}^* = \mathcal {S}_{\underline{ab}} . \end{aligned}$$

(26)

The first few projection operations, which are sufficient for describing charm decays, are

$$\begin{aligned} \mathcal {S}_{\mu \nu }(P) \quad&= \quad -g_{\mu \nu } + \frac{ P_{\mu }P_{\nu } }{P^2} \nonumber \\ \mathcal {S}_{\mu \nu \alpha \beta }(P) \quad&= \quad \frac{1}{2}\left( \mathcal {S}_{\mu \alpha }S_{\nu \beta } + \mathcal {S}_{\mu \beta }\mathcal {S}_{\nu \alpha } \right) - \frac{1}{3}\mathcal {S}_{\mu \nu }\mathcal {S}_{\alpha \beta }. \end{aligned}$$

(27)

This operator projects out the component of a tensor that is orthogonal to the four-momentum of a particle, and has rank 2J for a particle of spin J. The orbital angular momentum tensors are also constructed from the spin projection operators and the relative momentum of the decay products, $Q_a$ [24], and are written as

$$\begin{aligned} L_\mu&= - \mathcal {S}_{\mu \nu }(P_a) Q_a^\nu \nonumber \\ L_{\mu \nu }&= \mathcal {S}_{\mu \nu \alpha \beta }(P_a) Q_a^\alpha Q_a^\beta . \end{aligned}$$

(28)

The matrix element for a generic cascade of particle decays can then be calculated recursively. In the case of the decay of a spinless particle, the matrix element for the total decay process is identical to the current of the decaying particle. The generalised current is therefore merely a convenient device for organising the computation of spin matrix elements, but is not in general associated with the propagation of angular momentum. It is also useful to define the spin-projected currents, $\mathcal {S}_{\underline{\mu \nu }} J^{\underline{\nu }}$, which will be written as $S,V^{\mu },T^{\mu \nu }$ for (pseudo)scalar, (pseudo)vector and (pseudo)tensor states, respectively.

The rules for how the different spin-projected currents are written in terms of each other is given in Table 12, where these relations are derived by considering the symmetries of Lorentz indices and the parity properties of the matrix element. All of the coupling structures necessary to describe $P\rightarrow 4 P$ are uniquely determined by these constraints, although this property does not hold in general. This allows complicated spin configurations to be calculated in terms of a simple and consistent set of rules. The rules are written with consistent dependencies to clarify their derivations, and in some cases simplified forms are also given. These simplifications typically rely on the symmetry properties of the Levi-Civita tensor and the relationship $\mathcal {S}^{\underline{ab}}\mathcal {S}_{\underline{bc}} = \mathcal {S}^{\underline{a}}_{\underline{c}}$, which is the defining characteristic of a projection operator.

Table 12 Rules for calculating the current associated with a given decay chain in terms of the currents of the decay products. Where relevant, the spin projection operator $\mathcal {S}$ and the orbital angular momentum operators L are those for the decaying particle

B List of decay chains

The list of possible decay chains is built from what is allowed by the relevant conservation laws. Approximately one hundred different decay chains modes are included as possible contributions to the model. Certain cascade decays already have well known sub-branching ratios. For example, although the $K _1(1400)$ decays almost exclusively via the ${{K} ^*} (892)$, the various decays of the $K _1(1400)$ are treated separately without assumption about their branching ratios. The different components can be split into the same groups as in Sect. 4:

• ${{D} ^0} \rightarrow Y_{\pi \pi }\left[ \pi \pi \right] Y_{K\pi }\left[ K\pi \right]$, where $Y_{\pi \pi }$ is one of the following states: $\rho (770)^{0}$, $\rho (1450)^{0}$, $f_2(1270)$ or $[{{\pi } ^+} {{\pi } ^-} ]^{L=0}$, and $Y_{K\pi }$ is one of the following: $K ^{*}(892)^{0}$, $K ^{*}(1410)^{0}$, $K ^{*}(1680)^{0}$, $K _{2}^{*}(1430)^{0}$ or $[{{K} ^\mp } {{\pi } ^\pm } ]^{L=0}$.

  The $[{{\pi } ^+} {{\pi } ^-} ]^{L=0}$ and $[{{K} ^\mp } {{\pi } ^\pm } ]^{L=0}$ contributions are modelled using K matrices. In cases with a scalar contribution and a radial recurrence of a vector state, such as $\rho (1450)^{0}[{{K} ^\mp } {{\pi } ^\pm } ]^{L=0}$, the K matrix is fixed to be the same as the first vector, i.e. the K-matrix parameters of $\rho (770)^{0}[{{K} ^\mp } {{\pi } ^\pm } ]^{L=0}$. For vector–vector and vector–tensor contributions, the different possible polarisation states are included together in the model building. The contributions from the radial excitations of the kaon are only included as a possibility when included with the ${{\pi } ^+} {{\pi } ^-}$ S-wave, as the other decay chains involving this resonance, for example the decay $K ^*(1410)^{0}\rho (770)^{0}$, tend to have large interference terms, which requires fine tuning with other amplitudes and hence are considered to be unphysical.

• ${{D} ^0} \rightarrow X_{\pi \pi \pi } \left[ Y_{\pi \pi } \left[ \pi \pi \right] \pi \right] K$, where $X_{\pi \pi \pi }$ is one of the following states: $a_{1}(1260)^{\pm }$, $a_1(1640)^\pm$, $\pi (1300)^\pm$ or $a_{2}(1320)^\pm$ .

• ${{D} ^0} \rightarrow X_{K\pi \pi } \left[ Y_{K\pi } \left[ K\pi \right] \pi \right] \pi$, ${{D} ^0} \rightarrow X_{K\pi \pi } \left[ Y_{\pi \pi }\left[ \pi \pi \right] K \right] \pi$, where $X_{K\pi \pi }$ is one of the following states: $K _{1}(1270)^\pm$, $K _{1}(1400)^\pm$, $K ^{*}(1410)^\pm$, $K ^{*}(1680)^\pm$, $K _{2}^{*}(1430)^\pm$ or $K (1460)^\pm$.

All of these states are considered under all possible orbital configurations that obey the respective conservation laws.

Table 13 Legend for systematic uncertainties, including whether this sources of uncertainty is considered on the RS/WS decay mode

C Systematic uncertainties

The various contributions assigned for different systematic uncertainties are summarised in this appendix by a series of tables. The legend for these is given in Table 13, including which sources of uncertainty are considered on each decay mode. The breakdown of systematic uncertainties for the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ for coupling parameters, fit fractions and other parameters are given in Tables 14 and 15 for the quasi two-body decay chains and cascade decay chains, respectively. The systematic uncertainties for the WS mode ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$ are given in Table 16 for both coupling parameters and the fit fractions.

Table 14 Systematic uncertainties on the RS decay coupling parameters and fit fractions for quasi two-body decay chains

Table 15 Systematic uncertainties on the RS decay coupling parameters, fit fractions and masses and widths of resonances for cascade topology decay chains

Table 16 Systematic uncertainties on the WS decay coupling parameters and fit fractions

D Interference fractions

The interference fraction between decay chains a and b is

$$\begin{aligned} I(a,b) = \frac{ \mathrm {Re}\left( \int d \mathbf {x}\mathcal {M}_a (\mathbf {x}) \mathcal {M}_b(\mathbf {x})^{*} \right) }{ \int d\mathbf {x}\left| \sum _{j}\mathcal {M}_j(\mathbf {x}) \right| ^2 }, \end{aligned}$$

(29)

where the sum over j is over all of the decay chains. For cascade processes, the different secondary isobars contribute coherently to the interference fractions. The interference fractions are presented in Tables 17 and 18 for RS and WS decay modes, respectively. For each decay mode, the largest interference fractions are between the axial vector decay chain, and the lowest orbital angular momentum vector–vector decay chain.

Table 17 Interference fractions for the RS mode ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$, only shown for fractions $>0.5\%$. For each fraction, the first uncertainty is statistical and the second systematic

Table 18 Interference fractions for the WS mode ${{D} ^0} \rightarrow {{K} ^+} {{\pi } ^-} {{\pi } ^-} {{\pi } ^+}$, only shown for fractions $>0.5\%$. For each fraction, the first uncertainty is statistical and the second systematic

Fig. 6

Distributions for six invariant-mass observables in the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. Bands indicate the expectation from a model which excludes the decay chain $K_1(1270)^{-}\rightarrow \rho (1450)^{0}{{K} ^-}$, with the width of the band indicating the total systematic uncertainty. The total background contribution, which is very low, is shown in green

E Models excluding $K_1(1270)^{-} \rightarrow \rho (1450)^{0}{{K} ^-}$

The results for the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$ are shown in this appendix for a model that excludes the amplitude $K _1(1270)^{-}\rightarrow \rho (1450)^{0}{{K} ^-}$. The fit projections are shown in Fig. 6. The $\chi ^2$ per degree of freedom of this fit is 1.28. The fit fractions and parameters are shown in Table 19, and the partial fractions and parameters for the components associated with the resonances $a_{1}(1260)^{+}$, $K _1(1270)^{-}$ and $K (1460)^{-}$ in Tables 20, 21, 22, respectively. This model would be preferred to that presented in Sect. 6 if the $K_1(1270)\rightarrow \rho (1450)^{0}{{K} ^-}$ decay chain is excluded by investigations of the $K_1(1270)$ resonance in other production modes.

Table 19 Table of fit fractions and coupling parameters and other quantities for the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$, for a model excluding the decay chain $K_1(1270)^{-}\rightarrow \rho (1450)^0 {{K} ^-}$. Also given is the $\chi ^2$ per degree of freedom ($\nu$) for the fit. The first uncertainty is statistical and the second systematic. Couplings are defined with respect to the coupling to the channel ${{D} ^0} \rightarrow [{{\overline{K}{}} {}^*} (892)^{0}\rho (770)^{0}]^{L=2}$

Table 20 Table of fit fractions and coupling parameters for the component involving the $a_{1}(1260)^{+}$ meson, from the fit performed on the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. The coupling parameters are defined with respect to the $a_{1}(1260)^{+} \rightarrow \rho (770)^{0}{{\pi } ^+}$ coupling. For each parameter, the first uncertainty is statistical and the second systematic

Table 21 Table of fit fractions and coupling parameters for the component involving the $K_{1}(1270) ^{-}$ meson, from the fit performed on the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. The coupling parameters are defined with respect to the $K_{1}(1270) ^{-}\rightarrow \rho (770)^{0}{{K} ^-}$ coupling. For each parameter, the first uncertainty is statistical and the second systematic

Table 22 Table of fit fractions and coupling parameters for the component involving the $K(1460)^{-}$ meson, from the fit performed on the RS decay ${{D} ^0} \rightarrow {{K} ^-} {{\pi } ^+} {{\pi } ^+} {{\pi } ^-}$. The coupling parameters are defined with respect to the $K(1460)^{-} \rightarrow {{\overline{K}{}} {}^*} (892)^{0}{{\pi } ^-}$ coupling. For each parameter, the first uncertainty is statistical and the second systematic