1 Introduction

Three-body non-leptonic B meson decays constitute a large portion of the branching fraction and therefore attract a lot of attentions for several phenomenological applications, such as deeply testing the standard model (SM), the study of CP violation and the exaction of the CKM angles [1]. The branching fractions and CP asymmetries of a large number of channels have been studied extensively by BaBar [2,3,4,5,6,7], Belle [8,9,10,11] and LHCb [12,13,14,15,16]. Stimulated by the abundant experimental measurements, the theoretical explorations have been carried out in the recent years. Many three-body nonleptonic decays of heavy mesons have been studied in detail in QCD factorization [17,18,19,20,21,22,23], PQCD approach [24,25,26,27,28,29,30,31,32,33] and other theoretical methods [34,35,36,37].

Different from the two-body B decays where the momenta of final states are fixed, The kinematics of the three-body decay is completely determined by two of the three kinematic invariants \(s_{12}\), \(s_{13}\) and \(s_{23}\), with the definition \(s_{ij}=(P_i + P_j)^2/M_B^2\). The physical kinematical region in the plane of two invariants (the Dalitz plot) is given by a triangle. The Dalitz plot analysis has been proved to be a powerful tool for studies of multi-body decays, and has been applied widely in the experimental analysis. The analysis of three-body B decays using this technique enables one to study the properties of various resonances. Indeed, most of the quasi-two-body B decays are extracted from the Dalitz-plot analysis of three-body ones. In addition, the Dalitz plot analysis of three-body B decays provides a nice methodology for extracting information on the unitarity triangle in SM. For example, the Dalitz analysis together with the isospin symmetry allows one to extract the angle \(\alpha \) from the \(B\rightarrow \pi \pi \pi \) decays with the vector resonance \(\rho \) [38].

On the theoretical side, three-body decays of heavy B mesons are rather more complicated as they receive resonant and nonresonant contributions and involve 3-body matrix elements. The interference between nonresonant and resonant amplitudes makes it difficult to disentangle these two distinct contributions clearly. The experimental measurements show that some three-body B decays are dominated by intermediate vector, scalar and tensor resonances, namely, they proceed via quasi-two-body decays containing a resonance state and a bachelor. While for some others, the nonresonant contributions govern the decay amplitudes [18, 39]. Summarizing the experimental measurements, we find that there is a large nonresonant fraction in the penguin-dominant modes. For example, the nonresonant fraction reaches about \(90\%\) in \(B\rightarrow KKK\) decays, \(17-40\%\) in \(B\rightarrow K\pi \pi \) decays, and only \(14\%\) in \(B\rightarrow \pi \pi \pi \) decays [40]. However, there is not any reliable theoretical description of the amplitudes of B meson three-body decays so far, and most studies is are still model dependent. The isobar model [41] and the K-matrix formalism [42] are popularly adopted in the experimental analysis, especially the former [12]. In these models, the resonant amplitudes are modeled and the non-resonant contributions are often described by an empirical distribution in order to reproduce the whole phase space [43].

Besides a large number of charmless three-body B decays, a lot of charmed three-body decays have also been measured in B factories [44] and LHCb [45,46,47,48]. For example, \(B_s\rightarrow \overline{D}^0K^-\pi ^+\) have been detailed analyzed in LHCb in Ref. [45], where the resonant structures of the \(\overline{D}^0K^-\) and \(K^-\pi ^+\) components have been studied using the Dalitz plot technique. For the \(\overline{D}^0K^-\) component, a structure is found at \(m_{\overline{D}^0K^-}\approx 2.86\) GeV, which is viewed as an admixture of spin-1 and spin-3 resonances. For the \(K^-\pi ^+\) component, \(K^*\) resonances and the corresponding fit fractions have been well measured and the branching fractions of the corresponding quasi-two-body decays were also reported, based on the fit fraction and the branching fraction of the \(B_s\rightarrow \overline{D}^0K^-\pi ^+\) from previous LHCb measurement [49]. Motivated by this, we shall investigate the CKM-favored \(B_{(s)}\rightarrow \overline{D}K \pi \) decays in PQCD approach, and focus on the S, P and D-wave resonant contributions with respect to the \(K\pi \) component.

We all know that the resonant three-body decays (quasi-two-body decays) correspond to the edge of the Dalitz plot, where the two particles move collinearly with large energy and the bachelor particle recoils back. The meson-pair and the rest one move fast and back-to-back in the B meson rest frame so that the interaction between the meson-pair and the bachelor meson is highly suppressed. If we regard the meson-pair as a whole, the quasi-two-body decay is very similar to a two-body decay and the factorization formula for two-body decay can be applied safely. In practice, we also need to introduce a new wave function to describe the meson-pair, including the angular momenta. The soft interactions between two mesons are also absorbed in this nonperturbative wave function.

In PQCD approach, the amplitude of quasi-two-body B decay can be factorized into different parts according to the “characteristic” scales. As we know, the physics above the mass of W boson \(m_{W}\) is weak interaction and can be calculated perturbatively. The physics between \(m_{W}\) scale and b quark mass \(m_b\) scale can be contained in the Wilson coefficient \(C(\mu )\), which can be calculated within the renormalization group equation and the Wilson coefficient at the scale \(m_{W}\). The physics between \(m_b\) scale and the factorization scale \(\Lambda _h\) is dominated by exchanging one hard gluon and can be calculated perturbatively. This part is also called hard kernel H. The physics below \(\Lambda _h\) scale is soft and nonperturbative, which can be described by the universal wave functions. Therefore, the decay amplitude of \(B \rightarrow D (R\rightarrow ) K \pi \) decay in PQCD can be decomposed into the convolution as [50]

$$\begin{aligned} {\mathscr {A}}\sim & {} \int dx_idb_i\Big [C(t)\otimes {H}(x_i,b_i,t)\otimes \Phi _B(x_1,b_1)\nonumber \\{} & {} \otimes \Phi _D(x_2,b_2)\otimes \Phi _{K\pi }(x_3,b_3)\otimes e^{-S(t)}\Big ], \end{aligned}$$
(1)

where \(x_i\) are the momentum fraction of the quarks in the initial and final mesons, and \(b_i\) are the conjugate variables of the intrinsic transverse momentum \(k_{Ti}\) of the quarks. \(\Phi _B\), \(\Phi _D\) are the wave functions of the B meson and the \(\overline{D}\) meson, respectively. \(\Phi _{K\pi }\) is the new introduced two-meson wave function. The exponential term \(e^{-S(t)}\) is the Sudakov form factor obtained from the resummation of the double logarithms arising from the retained intrinsic transverse momentum \(K_T\) of the inner quarks [51,52,53].

The layout of the present paper is as follows. In Sect. 2, we firstly show the Hamiltonian governing those quasi-two-body \(B \rightarrow \overline{D}K\pi \) decays considered in this paper. The wave functions of \(K-\pi \) pair are also discussed. In Sect. 3 we address the numerical results of the branching fractions of those quasi-two-body decays. In this section based on the obtained branching fractions we will also probe the corresponding two-body \(B \rightarrow \overline{D} R\) (R denoting the intermediate resonance) decays and compare them with the previous results and the experimental data. Finally, we conclude this work in Sect. 4

2 The decay formalism and wave function

For the quasi-two-body decays, the Dalitz plot analysis help us to interpret the decay amplitude in isobar model conveniently, which is popularly used to describe the complex amplitude of three-body decays by the experimentalists. In this model the total decay amplitude is represented by a coherent sum of amplitudes from N individual decay channels with different resonances,

$$\begin{aligned} {\mathscr {A}}=\sum _{i=1}^{N}a_i{\mathscr {A}}_i, \end{aligned}$$
(2)

where \(a_i\) is the complex coefficient showing the relevant magnitude and the phase of different decay channels. The phase is a new source of the CP asymmetry, namely, the CP asymmetry caused by the interference among the different channels. \({\mathscr {A}}_i\) is the amplitude of the quasi-two-body decay associated with the certain resonance, which can be perturbatively calculated in PQCD.

The weak Hamiltonian \({\mathscr {H}}_{eff}\) of \({\bar{b}}\rightarrow {\bar{c}}{u}\bar{q}\) governing the \(B_{(s)}\rightarrow \overline{D}K\pi \) decays can be expressed as [54]

$$\begin{aligned} {\mathscr {H}}_{eff}=\frac{G_F}{\sqrt{2}}\Big \{V_{uq}^*V_{cb}(C_1O_1+C_2O_2)\Big \}\left| _{q=d, s}\right. , \end{aligned}$$
(3)

where \(V_{cb}\) and \(V_{uq}\) are the CKM matrix elements. The \(C_{1}\) and \(C_2\) are the so-called Wilson coefficients corresponding to the four-quark operators \(O_1\) and \(O_2\) respectively, which are all tree operators. The explicit expressions of the operators are written as

$$\begin{aligned} O_1= & {} (\bar{b}_{\alpha }c_{\beta })_{V-A}(\bar{u}_{\beta }q_{\alpha })_{V-A};\;\;\;\nonumber \\ O_2= & {} (\bar{b}_{\alpha }c_{\alpha })_{V-A}(\bar{u}_{\beta }q_{\beta })_{V-A}. \end{aligned}$$
(4)

We all know that the direct CP asymmetry is related to the interference between the tree type contribution and that of the penguin operators. Therefore, there are no direct local CP asymmetries in decays induced by the \({\bar{b}}\rightarrow {\bar{c}}{u}{\bar{q}}\) transition in SM, because the penguin operators do not involved in these decays.

It can be found from the factorization formula as shown in Eq. (1) that the most important inputs are the wave functions of the initial and final mesons. For the initial B meson and the final \(\overline{D}\) meson, their wave functions have been well determined and adopted widely in the studies of two-body \(B_{(s)}\rightarrow PP,PV,VV\) [52, 55,56,57] and \(B_{(s)}\rightarrow DP,DV,DS,DT\) [58,59,60,61,62,63,64] decays and other decay processes [65,66,67], and we will not discuss them any more in this paper. In the quasi-two-body decays, the new introduced parameters are the two-meson wave functions corresponding to different resonances. In current work, according to the LHCb analysis [45], we will consider the resonant contributions from the scalar resonance \(K^*_0(1430)\), the vector resonances \(K^*(892)\) and \(K^*(1410)\), and the tensor resonance \(K^*_2(1430)\) to the \(B_{(s)}\rightarrow \overline{D} K\pi \) decays. So, three kinds of wave functions will be discussed in the following sections. It should be noted that at present the forms of the two-meson wave functions based on QCD-inspired approach are still absent, and many phenomenological attempts have been performed to determine these forms and to constrain the involved parameters based on the available experimental data.

We firstly discuss the S-wave \(K\pi \) pair wave function \(\Phi _{S,K\pi }\), which can be given as [68],

(5)

with P and \(\omega \) denoting the momentum and the invariant mass of the \(K\pi \) pair respectively, satisfying \(P^2=\omega ^2\). The light-like vectors \(n=(1,0,{\textbf{0}}_T)\) and \(v=(0,1,{\textbf{0}}_T)\) are the dimensionless vectors. \(\phi _S\) and \(\phi _S^{s,t}\) are the twist-2 and twist-3 light-cone distribution amplitudes (LCDAs), respectively. The inner parameter z is the momentum fraction of the spectator quark, and \(\xi \) is the momentum fraction of the K meson in the \(K\pi \) pair. Similar to the LCDA of the meson, the LCDAs of the \(K\pi \) pair can also be decomposed into the Gegenbauer polynomials together with the corresponding Gegenbauer moments, which are given as

$$\begin{aligned} \phi _S(z,\xi ,\omega )= & {} \frac{6}{2\sqrt{2N_C}}F_S(\omega )z(1-z)\nonumber \\{} & {} \times \!\left[ \frac{1}{\mu _S}\!+\!B_1C_1^{3/2}(1\!-\!2z)\!+\!B_3C_3^{3/2}(1\!-\!2z)\!\right] \!,\nonumber \\\end{aligned}$$
(6)
$$\begin{aligned} \phi _S^s(z,\xi ,\omega )= & {} \frac{1}{2\sqrt{2N_c}}F_S(\omega ),\end{aligned}$$
(7)
$$\begin{aligned} \phi _S^t(z,\xi ,\omega )= & {} \frac{1}{2\sqrt{2N_C}}F_S(\omega )(1-2z), \end{aligned}$$
(8)

with

$$\begin{aligned} \mu _S=\frac{\omega }{m_2-m_1}, \end{aligned}$$
(9)

where \(m_{1,2}\) are the masses of the running current quarks in the resonance \(K_0^*(1430)\). \(C_{1,3}^{3/2}(t)\) and \(B_{1,3}\) are the Gegenbauer polynomials and corresponding Gegenbauer moments respectively. For the twist-3 LCDAs, the asymptotic forms are adopted for simplicity. However, the values of the inner parameters \(B_{1,3}\) in twist-2 LCDA have not been determined by the QCD-inspired approach so far. We determined them to be \(B_1=-0.4\pm 0.2\) and \(B_3=0.7\pm 0.4\) by phenomenological method together with the available experimental measurements of the \(B_s\rightarrow \overline{D}^0K^-\pi ^+\) decay [45]. In Ref. [29], we had investigated the \(B_s\rightarrow K^0(\overline{K}^0)K^{\pm }\pi ^{\mp } \) and fitted \(B_1=-0.4\pm 0.2\) and \(B_3=-0.8\pm 0.4\). It is obvious that the values of \(B_1\) are almost same and the signs of \(B_3\) are different. We also note that, in previous studies of the charmless \(B_s\rightarrow K^0(\overline{K}^0)K^{\pm }\pi ^{\mp } \) decays the terms proportional to the fraction \((\frac{m_i}{M_{B}})^2\) have been neglected since the mass of the final K or \(\pi \) is small enough to be omitted, in comparison with the mass \(M_{B}\) of B meson. In this work, when we consider the charmed B meson decays with massive \(\overline{D}\) meson involved in final states, the terms proportional to the fraction \((\frac{M_D}{M_B})^2\) can not be neglected any more, which will change the behaviour of the propagator of the inner quarks. Compared with \(B_1\), \(B_3\) has a large fluctuation in the fitting procedure. As a result, the new fitted \(B_{3}\) differs from the previous result, which is also confirmed in Ref. [69]. In Ref. [70], these parameters were also taken as the same as those in wave function of the \(K_0^*(1430)\) meson with large uncertainties considered.

Unlike the distribution amplitudes of meson, the time-like form factor \(F_S(\omega )\) is involved. In particular, this form factor is parameterized by the relativistic Breit–Wigner (RBW) model which is adopted extensively in experimental analysis, and has been proved to be a valid model for describing the narrow resonance that can be well separated from the other resonant and non-resonant contributions. However, RBW model fails to describe the \(F_S(\omega )\) associated with the \(K_0^*(1430)\) resonance, because \(K_0^*(1430)\) resonance interferes strongly with a slowly varying non-resonant contribution [71]. In order to overcome this deficiency, the so-called LASS line shape [72, 73] is developed to describe the combined contributions including the resonant and non-resonant parts, which explicit expression is given as

$$\begin{aligned} F_S(\omega )=\frac{\omega }{\mid p_1\mid (\cot \delta -i)}+e^{2i\delta }\frac{m_0\Gamma _0\frac{m_0}{\mid p_0\mid }}{m_0^2-\omega ^2-im_0\Gamma _0\frac{\mid p_1\mid }{\omega }\frac{m_0}{p_0}}, \end{aligned}$$
(10)

where the phase \(\delta \) is determined by \(\cot \delta =\frac{1}{a\mid p_1\mid }+\frac{r\mid p_1\mid }{2}\) with the scattering length \(a=1.95\pm 0.09~\textrm{GeV}^{-1}\) and the effective range \(r=1.76\pm 0.36~\textrm{GeV}^{-1}\) [74]. The \(|p_1|\) is the magnitude of the momentum of one of daughter of the resonance \(K_0^*(1430)\) in the center-of-mass frame of the \(K\pi \) pair and the \(|p_0|\) is the value of the \(|p_1|\) when \(\omega =m_0\). Finally, \(m_0\) and \(\Gamma _0\) are the pole mass and width of the resonance \(K_0^*(1430)\). From Eq. (10), one can easily find that the first term represents the non-resonant contribution, while the second term is the resonant one. Usually, a cutoff at \(\omega =1.7~\textrm{GeV}\) is suggested for the nonresonant contributions, which implies that there is only resonant contribution when the invariant mass is larger than \(1.7~\mathrm GeV\). In this paper we will also adopt the LASS model to evaluate the S-wave contributions to the \(B_{(s)}\rightarrow \overline{D} K\pi \) decays, which has also been adopted by LHCb.

Phenomenologically, the wave function of the P-wave \(K\pi \) pair can be modeled from the wave function of the vector meson. Due to the law of conservation of angular momentum, only the longitudinal polarization component contributes to the decay amplitude of \(B \rightarrow {\overline{D}} (R\rightarrow ) K \pi \), so we only keep the longitudinal wave function

(11)

where the expressions of twist-2 and twist-3 LCDAs are

$$\begin{aligned} \phi _P(z,\xi ,\omega )= & {} \frac{3F_P^{\parallel }(\omega )}{\sqrt{2N_C}}z(1-z)\nonumber \\{} & {} \times \left[ 1+a_1C_1^{3/2}(t)+a_2C_2^{3/2}(t)\right] (2\xi -1),\nonumber \\ \end{aligned}$$
(12)
$$\begin{aligned} \phi _P^s(z,\xi ,\omega )= & {} \frac{3F_P^{\perp }(\omega )}{2\sqrt{2N_C}}t(2\xi -1),\end{aligned}$$
(13)
$$\begin{aligned} \phi _P^t(z,\xi ,\omega )= & {} \frac{3F_P^{\perp }(\omega )}{2\sqrt{2N_C}}t^2(2\xi -1), \end{aligned}$$
(14)

with \(t=1-2z\). For the twist-3 LCDAs \(\phi _P^{s,t}\), we also take the asymptotic forms. The Gegenbauer moments \(a_{1,2}\) in the twist-2 LCDA are taken as \(a_1=-0.4\) and \(a_2=0.46\) [69].

For the time-like form factor \(F_P^{\parallel }\) corresponding to the resonances \(K^*(892)\) and \(K^*(1410)\), it can be well modeled by the RBW model, and be given as [73]

$$\begin{aligned} F_{\textrm{RBW}}(\omega )=\frac{m_0^2}{m_0^2-\omega ^2-im_0\Gamma (\omega )}, \end{aligned}$$
(15)

with the nominal mass \(m_0\) of the resonance. The mass-dependent width \(\Gamma (\omega )\) corresponding to a resonance with spin L can be expressed as [3]

$$\begin{aligned} \Gamma (\omega )=\Gamma _0\left( \frac{|p_1|}{|p_0|}\right) ^{2L+1}\left( \frac{m_0}{\omega }\right) X_L^2(r|p_1|), \end{aligned}$$
(16)

with the same definitions as the Eq. (10) for the \(\Gamma _0\), \(|p_1|\) and \(|p_0|\). The \(X_L\) is the Blatt–Weillkopf barrier factor [75], which depends on the angular momentum of the meson pair, with the expression as

$$\begin{aligned}{} & {} L=0,X_0(a)=1,\end{aligned}$$
(17)
$$\begin{aligned}{} & {} L=1,X_1(a)=\sqrt{\frac{1+a_0^2}{1+a^2}},\end{aligned}$$
(18)
$$\begin{aligned}{} & {} L=2,X_2(a)=\sqrt{\frac{a_0^4+3a_0^2+9}{a^4+3a^2+9}}, \end{aligned}$$
(19)

\(a_0\) being the value of the a at the pole mass of the resonance. The inner parameter r in Eq. (16) reflects the effective barrier of the resonance, and is usually taken to be \(r=4~\textrm{GeV}^{-1}\approx 0.8\) fm [76] for each resonance, because it dose not affect the numerical results remarkably. Currently, \(X_1(a)\) is used for the P-wave longitudinal time-like form factor, that is

$$\begin{aligned} F_P^{\parallel }(\omega )=F_{\textrm{RBW}}(\omega )\mid _{L=1}. \end{aligned}$$
(20)

For the transverse time-like form factor, \(F_P^{\perp }(\omega )\) can be obtained using the relation [25]

$$\begin{aligned} \frac{F_P^{\perp }(\omega )}{F_P^{\parallel }(\omega )}\approx \frac{f_V^{\perp }}{f_V}, \end{aligned}$$
(21)

with the decay constants \(f_V^{(\perp )}\) of the vector resonance.

Fig. 1
figure 1

Typical Feynman diagrams for the quasi-two-body decay \(B^0\rightarrow {\overline{D}}^0(R\rightarrow )K\pi \) in PQCD, where the black squares stand for the weak vertices, large (purple) spots on the quark lines denote possible attachments of hard gluons, and the green ellipse represents \(K\pi \)-pair and the red one is the light bachelor \(D^0\) meson

Now, we turn to discussion of the wave function of the D-wave \(K\pi \) pair. Because the initial B meson and final state D meson are all pseudoscalars, the helicity \(\lambda =\pm 2\) components vanish due to the conservation of the angular momentum. Therefore, the behavior of the D-wave \(K\pi \) pair are very similar to the P-wave one [77]. The only differences are the LCDAs \(\phi _D\), \(\phi _D^s\), and \(\phi _D^t\), whose expressions are

$$\begin{aligned}{} & {} \phi _D(z,\xi ,\omega )\nonumber \\{} & {} \quad =\sqrt{\frac{2}{3}}\frac{6F_D^{\parallel }(\omega )}{2\sqrt{2N_C}}z(1-z) \times \left[ 3a_D(2z-1)\right] P_2(2\xi -1),\end{aligned}$$
(22)
$$\begin{aligned}{} & {} \phi _D^s(z,\xi ,\omega )\nonumber \\{} & {} \quad =\sqrt{\frac{2}{3}}\frac{-9F_D^{\perp }(\omega )}{4\sqrt{2N_C}} \times \left[ a_D(1-6z+6z^2)\right] P_2(2\xi -1),\end{aligned}$$
(23)
$$\begin{aligned}{} & {} \phi _D^t(z,\xi ,\omega )\nonumber \\{} & {} \quad =\sqrt{\frac{2}{3}}\frac{9F_D^{\perp }(\omega )}{4\sqrt{2N_C}} \times \left[ a_D(1-6z+6z^2)(2z-1)\right] P_2(2\xi -1),\nonumber \\ \end{aligned}$$
(24)

where the Gegenbauer moment \(a_D\) is determined to be 0.25 and function \(P_2(t)\) is the Legendre polynomial. \(F_D^{\parallel }(\omega )\) and \(F_D^{\perp }(\omega )\) are the D-wave longitudinal and transverse time-like form factors respectively, which can also be well parameterized by RBW model with the spin-2 resonance \(K_2^*(1430)\).

In light of the effective Hamiltonian Eq. (3), we plot the Feynman diagrams contributing to quasi-two-body \(B_{(s)}\rightarrow \overline{D}K\pi \) decays at the leading order in Fig. 1. Based on the factorization formula and the wave functions of initial and final states, we then calculate the total decay amplitudes with the CKM matrix elements and the wilson coefficients, which are presented as

$$\begin{aligned} {\mathscr {A}}(B^0\rightarrow \overline{D}^0(K^+\pi -))= & {} \frac{G_F}{\sqrt{2}}V_{cb}^*V_{us}\nonumber \\{} & {} \times \left[ (C_1+C_2/3){\mathscr {F}}_{K\pi }^{LL} +C_2{\mathscr {M}}_{K\pi }^{LL}\right] ,\end{aligned}$$
(25)
$$\begin{aligned} {\mathscr {A}}(B^0\rightarrow D^-(K^+\pi ^0))= & {} \frac{G_F}{\sqrt{2}}V_{cb}^*V_{us}\nonumber \\{} & {} \times \left[ (C_1/3+C_2){\mathscr {F}}_{D}^{LL}+C_1{\mathscr {M}}_{D}^{LL}\right] ,\end{aligned}$$
(26)
$$\begin{aligned} {\mathscr {A}}(B^0\rightarrow D_s^-(K^+\pi ^0))= & {} \frac{G_F}{\sqrt{2}}V_{cb}^*V_{ub}\nonumber \\{} & {} \times \left[ (C_1+C_2/3){\mathscr {A}}_{K\pi }^{LL}+C_2{\mathscr {W}}_{K\pi }^{LL}\right] ,\end{aligned}$$
(27)
$$\begin{aligned} {\mathscr {A}}(B^+\rightarrow \overline{D}^0(K^+\pi ^0))= & {} \frac{G_F}{\sqrt{2}}V_{cb}^*V_{us} \left[ (C_1/3+C_2){\mathscr {F}}_{D}^{LL}+C_1{\mathscr {M}}_{D}^{LL}\right. \nonumber \\{} & {} \left. +(C_1+C_2/3){\mathscr {F}}_{K\pi }^{LL}+C_2{\mathscr {M}}_{K\pi }^{LL}\right] ,\end{aligned}$$
(28)
$$\begin{aligned} {\mathscr {A}}(B_s\rightarrow \overline{D}^0(K^-\pi ^+))= & {} \frac{G_F}{\sqrt{2}}V_{cb}^*V_{ud}\nonumber \\{} & {} \times \left[ (C_1+C_2/3){\mathscr {F}}_{K\pi }^{LL}+C_2{\mathscr {M}}_{K\pi }^{LL}\right] ,\end{aligned}$$
(29)
$$\begin{aligned} {\mathscr {A}}(B_s\rightarrow D_s^-(K^+\pi ^0))= & {} \frac{G_F}{\sqrt{2}}V_{cb}^*V_{us}\nonumber \\{} & {} \times \left[ (C_1/3+C_2){\mathscr {F}}_{D}^{LL}+C_1{\mathscr {M}}_{D}^{LL}\right. \end{aligned}$$
(30)
$$\begin{aligned}{} & {} \left. +(C_1+C_2/3){\mathscr {A}}_{K\pi }^{LL}+C_2{\mathscr {W}}_{K\pi }^{LL}\right] . \end{aligned}$$
(31)

In above decay amplitudes, the symbols \({\mathscr {F}}_{K\pi (D)}^{LL}\), \({\mathscr {M}}_{K\pi (D)}^{LL}\), \({\mathscr {A}}_{K\pi (D)}^{LL}\) and \({\mathscr {W}}_{K\pi (D)}^{LL}\) denote the contributions from the factorizable emission type diagrams, the hard-rescattering emission diagrams, the W boson annihilation type diagrams, and the W boson exchange type diagrams, respectively. The subscript \(K\pi (D)\) represents the \(K\pi \) pair (D meson) is recoiled, and the superscript LL stands for the contributions coming from \((V-A)\bigotimes (V-A)\) current. For the sake of simplicity, the expressions of the \({\mathscr {F}}_{K\pi (D)}^{LL}\), \({\mathscr {M}}_{K\pi (D)}^{LL}\), \({\mathscr {A}}_{K\pi (D)}^{LL}\) and \({\mathscr {W}}_{K\pi (D)}^{LL}\) are not given any more, which can be obtained by making the following substitutions in the results of Ref. [28],

$$\begin{aligned} \phi _K^{a,p}\rightarrow \phi _D,\;\phi _K^t=0,\; r_K\rightarrow r_D=m_d/m_B. \end{aligned}$$
(32)

We also point out that when considering the resonant contribution from the D-wave resonance \(K_2^*(1430)\), the contribution \({\mathscr {F}}_{D}^{LL}\) disappears due to the fact that the tensor structure can not be produced through \((V-A)\) current.

3 Numerical results and discussions

In this section, we firstly list the parameters we used in the numerical calculations, including the QCD scale, the masses and the lifetimes of the \(B/B_s\) mesons, the mass of the \(\overline{D}\) meson, the masses and the widths of the intermediate resonances, and the CKM matrix elements,

$$\begin{aligned} \Lambda _{QCD}^{f=4}= & {} 0.25\pm 0.05~\textrm{GeV},\; m_{B}=5.279~ \textrm{GeV},\;\nonumber \\ m_{B_s}= & {} 5.366~ \textrm{GeV},\nonumber \\ \tau _{B^+}= & {} 1.638~ \textrm{ps},\;\tau _{B^0}=1.519~ \textrm{ps},\;\tau _{B_s}=1.520~ \textrm{ps},\nonumber \\ m_{D^0}= & {} 1.865~ \textrm{GeV},\;m_{D^+}=1.869~ \textrm{GeV},\;\nonumber \\ m_{D_s}= & {} 1.96~ \textrm{GeV},\nonumber \\ m_{K^*(892)}= & {} 0.892~ \textrm{GeV},\;m_{K^*(1410)}=1.414~ \textrm{GeV},\;\nonumber \\ m_{K_0^*(1430)}= & {} 1.425~ \textrm{GeV},\nonumber \\ m_{K^*_2(1430)}= & {} 1.427~ \textrm{GeV},\;\Gamma _{K^*(892)}=51.4~ \textrm{MeV},\;\nonumber \\ \Gamma _{K^*(1410)}= & {} 232~ \textrm{MeV},\nonumber \\ \Gamma _{K^*_0(1430)}= & {} 270~\textrm{MeV},\;\Gamma _{K_2^*(1430)}=100~ \textrm{MeV},\nonumber \\ V_{cb}= & {} 0.04182^{+0.00085}_{-0.00074},\;V_{ud}=0.97435\pm 0.00016,\;\nonumber \\ V_{us}= & {} 0.22500\pm 0.00067. \end{aligned}$$
(33)

In Table 1, we present the numerical results of quasi-two-body \(B_{(s)}\rightarrow \overline{D} K\pi \) decays, together with the experimental data from LHCb [45, 46]. In our calculations, we take three kinds of errors into accounts to evaluate the theoretical uncertainties. The first uncertainties arise from the nonperturbative parameters in the wave functions of the initial and final states, such as the decay constants \(f_B=(0.19\pm 0.02)~\textrm{GeV}\) and \(f_{B_s}=(0.23\pm 0.02)~\textrm{GeV}\) of B and \(B_s\) meson respectively; the shape parameter \(\omega _B=(0.4\pm 0.04)~\textrm{GeV}\) and \(\omega _{B_s}=(0.5\pm 0.05)~\textrm{GeV}\) for B meson and \(B_s\) meson; the Gegenbauer moments in LCDAs of \(K\pi \) pair and the \(\overline{D}\) meson. We emphasize that this kind of uncertainties is dominant, but these uncertainties could be reduced with the improvements of the experimental measurements and the developments of the nonperturbative approaches. The second uncertainties origin from the unknown contributions from QCD radiative corrections [78, 79] and the power corrections [80, 81], which are characterized by varying the \(\Lambda _{\textrm{QCD}}=0.25\pm 0.05 ~\textrm{GeV}\) and the factorization scale t from 0.8t to 1.2t. In recent years, there are some attempts to study these two kinds of corrections in two-body B decays [82,83,84], but the relate researches on three-body B decays have not been carried out yet. The last uncertainties are caused by the uncertainties of the CKM matrix elements \(V_{cb}\) and \(V_{ud(s)}\), which have marginal effects on the branching fractions.

Table 1 The flavour-averaged branching ratios (in \(10^{-5}\)) of \(B_{(s)}\rightarrow \overline{D}K\pi \) decays with resonances \(K^*(892)/(1410)\), \(K_0^*(1430)\) and \(K_2^*(1430)\) in PQCD approach, together with the experimental data [45, 46] from LHCb

From Table 1, it can be seen that for the \(B_s\rightarrow \overline{D}^0 (K^*\rightarrow ) K^- \pi ^+\) decays the branching fractions we calculated are in good agreement with the LHCb experimental measurements that are based on a data sample corresponding to an integrated luminosity of 3.0 \(fb^{-1}\) of pp collision data with respect to the \(K^*(892)\), \(K^*(1410)\), \(K_0^*(1430)\) and \(K^*_2(1430)\) resonances. Using the fitted wave functions of \(K\pi \) pair we also calculate the branching fractions of other \(B_{(s)}\rightarrow \overline{D} (K^*\rightarrow ) K\pi \) decays, and they are expected to be tested in experiments in future. Due to the enhancement by the large CKM elements, most branching fractions are at the range \(10^{-6}-10^{-4}\), which are measurable in LHCb and Belle-II experiments.

For the quasi-two-body \(B_s\rightarrow \overline{D}^0(R\rightarrow ) K^- \pi ^+\) decays corresponding to the S, P and D wave intermediate resonances \(K^*\), the branching fractions are larger than the other corresponding decays, because the quasi-two-body \(B_s\rightarrow \overline{D}^0 K^- \pi ^+\) decays are governed by the process \(\bar{b} \rightarrow \bar{c} u \bar{d}\), which is enhanced by \(\mid \frac{V_{ud}}{V_{us}}\mid ^2\) compared to the other decays such as quasi-two-body \(B^0\rightarrow \overline{D}^0K^+\pi ^-\) decays governed by \(\bar{b} \rightarrow \bar{c} u s\). As a result, the \(B_s\rightarrow \overline{D}^0K^-\pi ^+\) decays are firstly measured in the LHCb experiment. However, for the quasi-two-body \(B^0 \rightarrow D_s^- K^+\pi ^0 \) decays that are also governed by the \(\bar{b} \rightarrow \bar{c} u d\) transition and enhanced by the large CKM elements \(V_{cb}V_{ud}\), the branching fractions are far smaller than those of \(B_s\rightarrow \overline{D}^0K^-\pi ^+\) decays, because they are pure annihilation type decays. In particular, they are viewed as to be power suppressed in comparison with the emission diagrams. For the \(B_s\rightarrow \overline{D}^0K^-\pi ^+\) and \(B^0\rightarrow \overline{D}^0 K^+\pi ^-\) decays with the \(\overline{D}\) emitted, the branching fractions corresponding to the S and D wave resonances are also sizable or even similar to that of the P wave resonance. However, the S and D wave resonant branching fractions of the \(B^0\rightarrow D^-K^+\pi ^0\), \(B^+\rightarrow \overline{D}^0 K^+\pi ^0\) and \(B_s\rightarrow D_s^-K^+\pi ^0\) decays with \(K\pi \) pair emitted are much smaller than that of P wave resonant branching fractions, and it is because the amplitudes with S wave or D wave \(K-\pi \) pair emission are highly suppressed or forbidden.

Table 2 The branching ratios (in \(10^{-5}\)) of \(B_{(s)}\rightarrow \overline{D}K^*\) decays probed from the quasi-two-body \(B_{(s)}\rightarrow \overline{D}(K^*\rightarrow )K\pi \) decays based on the narrow-width-approximation(NWA), together with the experimental data [85] and the former PQCD predictions from Refs. [59, 61, 64]

In light of SU(3) symmetry, we define a ratio between the quasi-two-body \(B_s\rightarrow \overline{D}^0(K^*\rightarrow )K^-\pi ^+\) and \(B^0\rightarrow \overline{D}^0 (K^*\rightarrow )K^+\pi ^-\) decays, and the naive estimation is given as

$$\begin{aligned} R=\frac{Br(B_s\rightarrow \overline{D}^0(K^*\rightarrow )K^-\pi ^+)}{Br(B^0\rightarrow \overline{D}^0 (K^*\rightarrow )K^+\pi ^-)}\sim |\frac{V_{ud}}{V_{us}}|^2\frac{\tau _{B_s}}{\tau _{B^0}}\sim 18.\nonumber \\ \end{aligned}$$
(34)

Accordingly, we also evaluate these ratios from Table 1 as

$$\begin{aligned} R= & {} \frac{Br(B_s\rightarrow \overline{D}^0(\bar{K}^*(892)\rightarrow )K^-\pi ^+)}{Br(B^0\rightarrow \overline{D}^0 (K^*(892)\rightarrow )K^+\pi ^-)}\nonumber \\{} & {} \sim \frac{28.6^{+17.2}_{-14.4}}{1.71^{+1.05}_{-0.88}} \sim 16.7^{+14.3}_{-12.1}, \end{aligned}$$
(35)
$$\begin{aligned} R= & {} \frac{Br(B_s\rightarrow \overline{D}^0(\bar{K}^*(1410)\rightarrow )K^-\pi ^+)}{Br(B^0\rightarrow \overline{D}^0 (K^*(1410)\rightarrow )K^+\pi ^-)}\nonumber \\{} & {} \sim \frac{1.74^{+1.11}_{-0.93}}{0.10^{+0.06}_{-0.05}}\sim 17.4^{+15.2}_{-12.7},\end{aligned}$$
(36)
$$\begin{aligned} R= & {} \frac{Br(B_s\rightarrow \overline{D}^0(\bar{K}_0^*(1430)\rightarrow )K^-\pi ^+)}{Br(B^0\rightarrow \overline{D}^0 (K_0^*(1430)\rightarrow )K^+\pi ^-)}\nonumber \\{} & {} \sim \frac{19.7_{-10.9}^{+11.6}}{1.58_{-0.80}^{+0.87}}\sim 12.5^{+10.0}_{-9.4},\end{aligned}$$
(37)
$$\begin{aligned} R= & {} \frac{Br(B_s\rightarrow \overline{D}^0(\bar{K}_2^*(1430)\rightarrow )K^-\pi ^+)}{Br(B^0\rightarrow \overline{D}^0 (K_2^*(1430)\rightarrow )K^+\pi ^-)}\nonumber \\{} & {} \sim \frac{3.95_{-1.90}^{+2.70}}{0.25_{-0.11}^{+0.16}}\sim 16.0^{+14.7}_{-10.3}. \end{aligned}$$
(38)

It is obvious that there are large uncertainties in the results Eqs. (3538). The acceptable deviation between our calculations and naive estimations attributes to the complicate wave functions of the \(K\pi \) pair. As aforementioned, although there are some attempts to study the wave functions of meson pair, the wave functions based on the first principle are still absent. Therefore, we have to employ the phenomenological models that are introduced in Sect. 2. In this regard, the accuracy of wave functions of heavy mesons should be further improved in future.

Under the narrow-width-approximation (NWA), the quasi-two-body decay and corresponding two-body process satisfy the factorization relation

$$\begin{aligned}{} & {} BF(B\rightarrow M_1R\rightarrow M_1M_2M_3)\approx BF(B\rightarrow M_1R)\nonumber \\{} & {} \quad \times BF(R\rightarrow M_2M_3), \end{aligned}$$
(39)

where the R represents the resonance. Therefore, we could extract the branching fractions of the related two-body decays, by combining our numerical results and the corresponding branching fractions of the resonances decay to the \(K\pi \). In turn, the comparison with the experimental data of two-body decays can help us to further verify the reliability of the wave functions of meson pair. In PDG [85], the branching fractions of the resonances decays to \(K \pi \) are summarized as follows,

$$\begin{aligned} BF(K^*(892)\rightarrow K\pi )\sim & {} 1,\end{aligned}$$
(40)
$$\begin{aligned} BF(K^*(1410)\rightarrow K\pi )= & {} (6.6\pm 1.3)\%,\end{aligned}$$
(41)
$$\begin{aligned} BF(K^*_0(1430)\rightarrow K\pi )= & {} (93\pm 10)\%,\end{aligned}$$
(42)
$$\begin{aligned} BF(K_2^*(1430)\rightarrow K\pi )= & {} (49.9\pm 1.2)\%. \end{aligned}$$
(43)

Using above data and the results in Table 1, we roughly extract the branching fractions of the corresponding two-body \(B\rightarrow \overline{D}R\) decays and present the results in Table 2. It can be seen that most branching fractions determined from NWA agree well with the available experimental data [85] and the previous predictions [59, 61, 64] based on PQCD within the uncertainties. Of course, if we wonder to test PQCD approach with these decays, the uncertainties in both theoretical and experimental sides should be reduced in future. We also note that for the pure annihilation \(B^0\rightarrow D_s^-K^{*+}(892)\) decay channel, the branching fraction \((15.4^{+4.6}_{-4.2})\times 10^{-5}\) we obtained is much larger than the experimental data \((3.5\pm 1.0)\times 10^{-5}\), though it is still in agreement with the previous study \((18.2^{+6.3}_{-6.4}) \times 10^{-5}\) [59]. In fact, the \(B^0\rightarrow D_s^-K^{*+}(892)\) decay occurs only with the annihilation diagrams, which are power suppressed. This deviation may indicate that the contributions from high power corrections become important, and we left this study to be our future work.

4 Summary

In this work we have investigated the resonant contributions of the quasi-two-body \(B\rightarrow D K\pi \) decays with the vector resonances \(K^*(892)/(1410)\), the scalar resonance \(K_0^*(1430)\) and the tensor resonance \(K_2^*(1430)\). We first fitted the parameters in the wave functions of the \(K\pi \) pair with the experimental results from LHCb. With the wave functions we then calculated the other quasi-two-body \(B_{(s)}\rightarrow \overline{D}(K^*\rightarrow ) K\pi \) decays. Most branching fractions are at the range of \(10^{-6}\sim 10^{-4}\), which is very promising to be measured experimentally. With the narrow-width-approximation, we also explored the branching fractions of the corresponding two-body decays, and most results are in agreement with the available experimental data. We hope our results to be further tested in experiments, so as to test the wave functions of \(K\pi \) pair and PQCD approach. Because all decays are governed by only tree operators, there are no direct CP asymmetries in these decays. If large CP asymmetry is observed in experiments, it would be a signal of new physics beyond SM.