Effect of direct CP violation in charm on γ extraction from B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ Dalitz plot analysis

Abstract

A possible effect of direct CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay on the γ measurement from B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ Dalitz plot analysis is considered. Systematic uncertainty of γ coming from the current limits on direct CP violation in $D\to K^{0}_{S}\pi ^{+}\pi^{-}$ is estimated, and a modified model-independent procedure of B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ Dalitz plot analysis is proposed that gives an unbiased γ measurement even in presence of direct CP violation in charm decays. The technique is applicable to other three-body D decays such as $D^{0}\to K_{S}^{0}K^{+}K^{-}$, D ⁰→π ⁺ π ⁻ π ⁰, etc.

1 Introduction

The mechanism of CP violation in particle physics is of primary importance because of its impact on cosmological baryogenesis and possible antimatter existence in the universe. In the quark sector, CP violation is studied by measuring the elements of Cabibbo–Kobayashi–Maskawa (CKM) mixing matrix [1, 2] with the convenient representation given by the Unitarity Triangle (UT), the angles and sides of which are parameters observable in various decays of B mesons. Precision measurement of the UT angle γ (also denoted as ϕ ₃) is an essential ingredient in searches for New Physics phenomena in B decays. The value of γ acts as one of the Standard Model reference points against which other measurements of the UT parameters are compared. The irreducible theoretical uncertainty in the extraction of the angle γ from B ^±→DK ^± decays is due to electroweak corrections and is extremely small, of the order of 10⁻⁶ [3]. However, the experimental determination of γ value remains a challenge owing to the low probabilities of the decays involved.

The types of measurements that dominate γ sensitivity are based on B ^±→DK ^± decays where the neutral D meson decays into a CP eigenstate (commonly referred to as the GLW method [4, 5]), suppressed Kπ state (ADS method [6]), or self-conjugate three-body final state such as $K^{0}_{S}\pi^{+}\pi^{-}$ (GGSZ or Dalitz plot method [7, 8]). None of these methods are systematically limited at the current level of precision. However, obtaining a degree-level precision on γ will require some subtle effects to be taken into account. The effect of charm mixing has already been considered by several authors [9–11] and was found to be negligible in most cases. Recent evidence of direct CP violation in singly Cabibbo-suppressed two-body D decays reported by LHCb [12] has triggered discussions of the possible effect of CP violation in charm on γ measurements using the GLW technique [13–15]. Although the evidence of large CP violation in D→hh decays is not supported by the updated LHCb measurements [16, 17], this effect can play its role in precision measurements of γ.

In this paper, we consider a possible effect of direct CP violation in charm on the measurement of γ using the Dalitz plot analysis of B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decays. The decay $D\to K^{0}_{S}\pi^{+}\pi^{-}$ is dominated by Cabibbo-favored transitions, and thus direct CP asymmetry coming from the Standard Model effects is expected to be very small. However, future precision measurements of γ can reach the point where this contribution will become significant. On the other hand, if disagreement in the UT parameters due to New Physics will be found in future measurements, the method that can distinguish whether the New Physics contribution enters charm or B decays will be essential. In addition, “effective” CP violation in $D\to K^{0}_{S}\pi ^{+}\pi^{-}$ decay of the order of 10⁻³ should arise from the CP violation in the neutral kaon system if this effect it not explicitly accounted for.

The goal of this paper is twofold. First, we estimate the systematic uncertainty on γ coming from the current limits on direct CP violation in the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay. Second, we show that in the model-independent analysis using quantum-correlated $D\overline{D}$ data at charm threshold it is possible to account for the CP violation in charm and obtain an unbiased measurement of γ without significantly sacrificing the statistical precision.

Although the decay $D\to K^{0}_{S}\pi^{+}\pi^{-}$ is used throughout this paper, the same approach can be applied to other three-body final states of the neutral D decay, such as $D^{0}\to K_{S}^{0}K^{+}K^{-}$, D ⁰→π ⁺ π ⁻ π ⁰, etc.

2 Formalism of model-independent γ measurement with CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$

The procedure to extract γ from B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ in a model-independent way employed in the current analyses [18–20] assumes CP conservation in D decays. The technique uses binned Dalitz plot distributions, and in order to utilize the assumption of CP conservation the bins are chosen symmetrically to the exchange of Dalitz plot variables of the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay (invariant masses squared of $K^{0}_{S}\pi^{+}$ and $K^{0}_{S}\pi^{-}$ combinations): $m^{2}_{K_{S}^{0}\pi^{+}}\leftrightarrow m^{2}_{K_{S}^{0}\pi^{-}}$. The bins are denoted with the index i which runs from $-\mathcal{N}$ to $\mathcal{N}$ excluding zero; symmetric bins have the same |i| and the flip of the sign i↔−i corresponds to the reflection $m^{2}_{K_{S}^{0}\pi^{+}}\leftrightarrow m^{2}_{K_{S}^{0}\pi^{-}}$. Current analyses [18–20] are performed with $\mathcal{N}=8$: this number of bins, together with a special choice of the shape of the bins over the Dalitz plot, provides a statistical precision for the γ measurement that approaches the precision of the unbinned model-dependent technique with the limited B ^±→DK ^± data available today [21].

The procedure of model-independent Dalitz plot analysis is described in detail in Refs. [7, 21]; here we give only the final equations. The analyses use four categories of events involving $D\to K^{0}_{S}\pi^{+}\pi^{-}$: flavor-tagged $D\to K^{0}_{S}\pi^{+}\pi^{-}$, neutral D mesons tagged in CP eigenstate from $\psi(3770)\to D\overline{D}$ process (D _CP ), correlated pairs of neutral D mesons where both D are reconstructed in the $K^{0}_{S}\pi^{+}\pi^{-}$ state, and B ^±→DK ^± decays with $D\to K^{0}_{S}\pi^{+}\pi ^{-}$. The number of events in bin i of the flavor-tagged D decay is different for D ⁰ and $\overline {D}{}^{0}$; it is denoted as K _i and $\overline{K}_{i}$, respectively. However, in the CP-conserving case and with the symmetric binning described above $K_{i} = \overline{K}_{-i}$, so $\overline{K}_{i}$ are not independent. The numbers of events in bins are then related as

$$ M_i = h_{CP} \bigl[K_i + K_{-i} + 2 \sqrt{K_i K_{-i}} C_i \bigr] $$

(1)

for D decays into a CP eigenstate,

(2)

for correlated $D\overline{D}$ pairs both decaying into $K^{0}_{S}\pi^{+}\pi^{-}$ (here one has to deal with two correlated Dalitz plots and thus the number of events is described with two indices i and j), and

$$ N^{\pm}_i = h_{B^{\pm}} \bigl[K_{\pm i} + r_{\pm}^2 K_{\mp i} + 2\sqrt {K_i K_{-i}}(x_{\pm} C_i \pm y_{\pm} S_i) \bigr] \\ $$

(3)

for $D\to K^{0}_{S}\pi^{+}\pi^{-}$ from B ^±→DK ^±. Here x _±=r _B cos(δ _B ±γ), y _±=r _B sin(δ _B ±γ), $r^{2}_{\pm}=x^{2}_{\pm} + y^{2}_{\pm}$. The free parameters x _± and y _± hold the information about the phase γ and hadronic parameters in B ^±→DK ^± decay: the amplitude ratio r _B and the strong phase difference δ _B . The other free parameters are the normalization factors h _CP , $h_{\rm corr}$ and $h_{B^{\pm}}$, and phase terms C _i and S _i . The terms C _i and S _i describe the average sine and cosine of the strong phase difference between D ⁰ and $\overline {D}{}^{0}$ amplitudes over the bin i. In the case of CP conservation they satisfy C _i =C _−i , S _i =−S _−i . Thus, these parameters are independent only for i>0. The system of (1), (2), and (3) is overconstrained and can be solved with the maximum likelihood fit to obtain x _± and y _± and, thus, the value of γ.

Now we turn to the case when CP is not conserved in the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay. The relations between symmetric bins of the Dalitz plot do not hold anymore. We still use the notation for bin number $i=-\mathcal{N},\ldots,{-}1,1,\ldots,\mathcal{N}$, but now $\overline{K}_{i}\neq K_{-i}$, C _i ≠C _−i , and S _i ≠−S _−i in general. In principle, now the binning is not required to be symmetric, although in our studies we keep the same binning as in the CP-conserving case to allow for a direct comparison of the two approaches.

The equations relating the numbers of events in bins of the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ Dalitz plots are:

$$ M_i = h_{CP} \Bigl[K_i + \overline{K} _{i} + 2\sqrt{K_i\overline {K} _{i}} C_i \Bigr], $$

(4)

(5)

and

(6)

Note that the number of phase terms C _i , S _i is doubled compared to the CP-conserving case since their values for i<0 are now independent. The numbers of flavor-tagged events K _i and $\overline{K}_{i}$ also have to be obtained independently, but since the available samples of flavor-tagged D decays are large, this should not limit the accuracy of the measurement. There are $4\mathcal{N}+8$ free parameters for $4\mathcal{N}^{2}+6\mathcal{N}$ equations (1), (2), and (3), and thus the system of equations still remains solvable. As a result, arbitrarily large CP violation in the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay does not lead to a bias in the measurement of the x,y parameters, and, hence, of the value of γ when using this technique. We remind that there is a principal ambiguity in this measurement: it is not sensitive to the simultaneous change of sign of all S _i which causes the signs of y _± observables to flip. This ambiguity is resolved by the weak model assumption that the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ amplitude is described with a sum of Breit–Wigner amplitudes [21].

The decays of a neutral D in a CP eigenstate into $K^{0}_{S}\pi^{+}\pi^{-}$ are obtained from the process $\psi(3770)\to D\overline{D}$, where the other (tagging) D meson is reconstructed in the CP eigenstate of the opposite parity. Therefore, if CP is violated in the decay of the tagging D, Eq. (4) would not be valid. This effect is expected to be larger for CP-even tags using Cabibbo-suppressed decays (D→K ⁺ K ⁻,π ⁺ π ⁻) than for CP-odd tags which are mostly Cabibbo-favored (such as $D\to K_{S}^{0}\pi^{0}$). Without the CP-tagged D decay, the remaining equations (5) and (6), which do not include D decays other than $K^{0}_{S}\pi^{+}\pi ^{-}$, have two additional ambiguities. One is an additional discrete ambiguity: the simultaneous change of sign of all C _i followed by a flip of x _± signs. The choice between the two solutions can, though, be made using Eq. (4) with the good assumption that CP violation in the tagging D decay is small. The other, more important ambiguity is the rotation by the arbitrary phase δϕ:

(7)

with the simultaneous rotation of γ by the same value δϕ.¹ Thus, the single decay mode $D\to K^{0}_{S}\pi^{+}\pi^{-}$ cannot resolve the CP-violating phases originating from B and D decays and D _CP decay has to serve as a reference. Any CP-violating phase in this decay directly translates into the uncertainty on the angle γ. Generally, the analysis using only B→DK and $\psi(3770)\to D\overline{D}$ decays can be influenced by the common CP violating phase in charm which directly affects the γ measurements but is not observable otherwise.

The CP violating phase δϕ can be independently controlled in the B decay where the $D^{0}-\overline{D}{}^{0}$ admixture appears with known CP-violating phase other than γ. This is possible using the decay B ⁰→Dπ ⁰, $D\to K^{0}_{S}\pi^{+}\pi^{-}$. Neutral D in this decay is a coherent admixture of D ⁰ and $\overline{D}{}^{0}$ states determined by the CKM phase β [22]. Using the binned approach, the decay time distributions for B ⁰ and $\overline{B}{}^{0}$ mesons are

(8)

where t is the difference $t=t_{\rm sig}-t_{\rm tag}$ between the B decay time and the time at which it was tagged to be $\overline{B}{}^{0}$ or B ⁰, τ is the average neutral B lifetime, and Δm is the mass difference of the two B mass eigenstates. In the relations above, we neglected the Cabibbo-suppressed contribution to B ⁰→Dπ ⁰ which is of the order of $|V_{ub}V^{*}_{cd}/V_{cb}V^{*}_{ud}|\simeq0.02$. It introduces additional parameters similar to B→DK case (amplitude ratio $r_{D\pi^{0}}$ and strong phase $\delta_{D\pi^{0}}$) which, however, can be obtained from data in the time-dependent analysis [22].

The CP violating phase in the $D\to K^{0}_{S}\pi^{+}\pi ^{-}$ decay would enter the difference between the angles β observed in B ⁰→Dπ ⁰ and $B^{0}\to J/\psi K^{0}_{S}$ decays. The uncertainty in γ will then be limited by the theoretical uncertainties in β extraction from these decays (mostly from $B^{0}\to J/\psi K^{0}_{S}$ since B ⁰→Dπ ⁰ is tree-dominated), and by the experimental precision of β measurement in B ⁰→Dπ ⁰. The analyses performed by Belle [23] and BaBar [24] suggest that the precision that can be obtained with the Belle II experiment with the integrated luminosity 50 ab⁻¹ can be around 2^∘. Additional charmed B decays sensitive to β, e.g. B ⁰→Dπ ⁺ π ⁻ [25], can be used to improve this precision, not only with Belle II, but also with the LHCb experiment.

The technique described above can be applied not only to $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decays with CP violation, but also to other non-self-conjugate final states, such as $D\to K^{0}_{S} K^{-}\pi ^{+}$, D→K ⁺ π ⁻ π ⁰, etc.

3 Bias of γ measurement due to CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay

In this section, we investigate how the current limits on CP violation in the decay $D\to K^{0}_{S}\pi^{+}\pi^{-}$ affect the model-independent measurement of γ using the current approach in which CP conservation is assumed in charm decays.

The first study placing limits on direct CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay has been performed by the CLEO collaboration [26] and has recently been improved by CDF [27]. Both measurements use isobar formalism to parametrize the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay amplitude and assume that a CP asymmetry may appear in any of the quasi two-body amplitudes. Specifically, the amplitude is represented as the sum of resonance components for both the D ⁰ and $\overline{D}{}^{0}$ decays:

(9)

where $\mathcal{A}_{D}$ and $\overline {\mathcal{A}}_{D}$ are the amplitudes of D ⁰ and $\overline{D}{}^{0}$ decays, respectively, a ₀ and δ ₀ are the amplitude and the phase of the non-resonant component (assumed to be CP-conserving), $\mathcal{M}_{j}$ and $\overline{\mathcal{M}}_{j}$ are the quasi two-body resonant matrix elements (typically the relativistic Breit–Wigner amplitudes), and $A^{\pm}_{j}$ are the (complex) amplitudes of the resonant components. In the CP-violating case, $A^{+}_{j}\neq A^{-}_{j}$. In general, the presence of a CP-violating amplitude can result in the difference of both the magnitudes and the phases of the complex numbers $A^{+}_{j}$ and $A^{-}_{j}$. The parametrization adopted by CLEO and CDF is

$$ A^{\pm}_j = a_j e^{i(\delta_j\pm\phi_j)}(1\pm b_j/a_j), $$

(10)

where a _j and δ _j are CP-averaged parameters, while b _j /a _j and ϕ _j are small parameters related to the CP violation.

We perform Monte Carlo (MC) studies to estimate the systematic uncertainty in the γ measurement arising from the current limits on CP violation in the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay obtained by CDF. A parametrization of the form (10) is used. The $D\to K^{0}_{S}\pi^{+}\pi^{-}$ amplitude model used to generate event samples is based on the Belle measurement [28]. A CP asymmetry of 10 % is introduced one-by-one for the amplitude and phase in each partial amplitude (i.e. b _j /a _j =0.1 and ϕ _j =0.1 radian). Large number of flavor-tagged D, D _CP , correlated $D\overline{D}$, and B ^±→DK ^± samples are generated so that the statistical error of γ measurement does not exceed 0.2^∘. The values γ=70^∘, r _B =0.1, δ _B =130^∘ are used at the generation stage. The samples are then fitted to extract γ value and other related parameters without taking CP violation into account. The resulting values of γ for each variation of the D ⁰ decay amplitude are shown in Fig. 1.

Fig. 1

Bias of γ measurement in the fit without accounting for CP asymmetry, using the amplitude generated without CP asymmetry (filled square), as well as with CP asymmetry in the magnitude b/a=10 % (filled circles), and in phase ϕ=0.1 (open circles) in each amplitude component. Statistical errors are comparable with the size of markers

After the bias in γ for a 10 % CP asymmetry is obtained from MC, we recalculate the bias associated with current experimental limits by taking the central values of CP asymmetries and their errors from the CDF measurement [27] assuming a linear dependence of γ bias on CP-violating parameters b _j /a _j and ϕ _j . The resulting contributions from CP violation in each resonance are given in Table 1. Finally, we calculate the total error by summing up the central values of bias linearly for each contribution, and the errors quadratically. The errors of the individual components are assumed to be uncorrelated. As expected, the resulting γ bias is consistent with zero within its error since no evidence of CP violation has been found by CDF. Thus the γ uncertainty is taken as the error of the bias and amounts to around 3^∘.

Table 1 Contributions of CP violating amplitudes in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay measured by CDF [27] to the γ measurement bias for each contributing resonance, and the total γ bias

4 Model-independent analysis with CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$

Here we present results of the MC study performed to estimate how the fit procedure which allows for CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay described in Sect. 2 affects the statistical precision of the γ measurement compared to the CP-conserving case.

We have performed MC simulation with 10⁶ events of flavor-specific $D\to K^{0}_{S}\pi^{+}\pi^{-}$ of each flavor, 120000 correlated $\psi(3770)\to D\overline{D}$ decays, 120000 $D_{CP}\to K^{0}_{S}\pi^{+}\pi^{-}$ decays, and 60000 B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decays of each B sign. This B sample size corresponds roughly to the data sample expected in the upgraded phase of the LHCb experiment and at the Super B factory. The ratio of ψ(3770) and B samples was taken to be the same as in current analyses; we expect that the sufficient sample of ψ(3770) decays will be collected by BES-III experiment and future tau-charm factory. In addition to this sample denoted by the factor k=1, we repeat the simulation with four times smaller (k=1/4) and four times larger (k=4) samples to check how the error scales with the sample size. Current world-average values for the parameters of B→DK decays are taken: γ=70^∘, r _B =0.1, δ _B =130^∘. A total of 1000 pseudoexperiments are generated and fitted for the sample of each size.

Each MC sample is fitted with two techniques: (a) the one which assumes CP conservation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay, and (b) the one which allows for CP violation, as described in Sect. 2. Using the similar variations of the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ amplitude involving 10 % CP asymmetry in each resonance component as in Sect. 3, we show that the bias in γ measurement is consistent with zero (see Fig. 2). We also compare the statistical precision of the two approaches from the spread of fitted γ values between pseudoexperiments. As shown in Table 2, the reduction of statistical precision due to increased number of free parameters does not exceed 10 %.

Fig. 2

Bias of γ measurement in the fit with CP asymmetry accounted for, using the amplitude generated without CP asymmetry (filled square), as well as with CP-violation in the magnitude b/a=10 % (filled circles), and in phase ϕ=0.1 (open circles) in each amplitude component. Note different vertical scale compared to Fig. 1

Table 2 Comparison of the precision of the model-independent γ measurement for the fit procedures with and without accounting for the CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay. The size of the simulated sample defined by the factor k is described in the text

5 Conclusion

We have shown that the current best limits on CP violation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay coming from the measurement performed by CDF [27] translates to systematic uncertainty in the determination of the CKM phase γ from B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay of the order of 3^∘. While the current world-average precision of γ is 9−12^∘ [29, 30] and is not limited yet by this uncertainty, the data sample to be collected by LHCb experiment before its upgrade should allow measurement with a precision around 5^∘ in which the B ^±→DK ^±, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ Dalitz analysis will have significant weight [31]. It is thus useful to study the CP asymmetry in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ with a larger data sample (e.g. at B factories and LHCb) to reduce this uncertainty.

In addition, we have shown that even if the $D\to K^{0}_{S}\pi ^{+}\pi^{-}$ decay is found to exhibit CP violation, it is possible to account for it and perform the unbiased measurement of γ in B ^±→DK ^±, $D\to K^{0}_{S}\pi ^{+}\pi^{-}$ decay in a model-independent way. Compared to the model-independent technique which assumes CP conservation in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ [7, 21], this method has more free parameters which, however, leads to a reduction of the statistical precision not exceeding 10 %. This approach reduces the possibly large number of CP-violating degrees of freedom in $D\to K^{0}_{S}\pi^{+}\pi^{-}$ amplitude to a single CP-violating phase which directly affects the measurement of γ. This phase can be controlled using $D\overline{D}$ threshold data where the $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decay is tagged by the other D decaying to the CP-eigenstate through Cabibbo-favored transition (e.g. $K^{0}_{S}\pi^{0}$). However, this procedure should assume the absence of CP violation in the tagging decay and thus is model-dependent. Another possibility is to access this phase from the difference of measurements of the angle β in $B^{0}\to J/\psi K^{0}_{S}$ and B ⁰→Dπ ⁰, $D\to K^{0}_{S}\pi ^{+}\pi^{-}$ decays. The accuracy of this approach will be limited by the experimental precision of β determination from B ⁰→Dπ ⁰, $D\to K^{0}_{S}\pi^{+}\pi^{-}$ decays (about 2^∘ with Belle II), but can be improved further by using other modes with $D\to K^{0}_{S}\pi^{+}\pi^{-}$, such as B ⁰→D ⁰ π ⁺ π ⁻.