1 Introduction

Three-body hadronic B meson decays are a rich field for the experiments and theoretical studies. These decay processes offer one of the best tools for the analyses of direct CP violation and also provide a testing ground for the dynamical models of the strong interaction. Strong dynamics in a three-body B meson decay is much more complicated than that in a two-body case, the three-body processes receive nonresonant and resonant contributions, as well as the significant final-state interactions [1,2,3]. The nonresonant contributions have been studied with the method of heavy meson chiral perturbation theory (HMChPT) [4,5,6,7] valid in the soft meson limit in Ref. [8]. The exponential factor \(e^{-\alpha _{\mathrm{NR}} p_B\cdot (p_i+p_j)}\) is introduced so that the HMChPT results are recovered in the soft meson limit where \(p_i, p_j \rightarrow 0\). In addition to the nonresonant background, it is urgent to study the resonant contributions which are, in most cases, the dominant part of a three-body decay process. Analyses of the three-body decays utilizing the Dalitz plots [9, 10] enable us to investigate the properties of various scalar, vector and tensor resonant states with the isobar model [11, 12] in terms of the usual Breit–Wigner model [13].

Fig. 1
figure 1

a Differential branching ratios for the \(B^+\rightarrow K^+\bar{K}^{*0}\rightarrow K^+K\pi \) decays, and b differential distributions of \(\mathcal{A}_{cp}\) in \(\omega \) for the decay modes \(B^+\rightarrow K^+\bar{K}^{*0}\rightarrow K^+K\pi \) and \(B^+\rightarrow \bar{K}^0 K^{*+}\rightarrow \bar{K}^0K\pi \). Shaded bands show the estimated theoretical uncertainties

On the theoretical side, no proof of factorization has been done for the decays of the B meson into three final state mesons. As a first step, however, we can restrict ourselves to the specific kinematical configurations, in which two energetic final state mesons almost collimating to each other. For such topologies, the three-body interactions are expected to be suppressed strongly due to power counting rules [14]. In such quasi-two-body region of phase space, the obvious generalization of the factorization theorem for two-body decays applies, and in the region where all invariant masses are large, factorization has been explicitly shown at the leading non-trivial order in Refs. [15, 16]. It’s reasonable for us to assume the validity of the factorization for these quasi-two-body B decays. In the “quasi-two-body” mechanism, the two-body scattering and all possible interactions between the two involved particles are included but the interactions between the bachelor particle and the daughter mesons from the resonance are neglected. Substantial progress on three-body hadronic B meson decays by means of symmetry principles has been made for example in Refs. [17,18,19,20,21,22,23,24]. The QCD-improved factorization [15] has also been widely applied to the studies of the three-body hadronic B meson decays in Refs. [8, 14, 25,26,27,28,29,30,31,32,33]. The detailed factorization properties of the \(B^+\rightarrow \pi ^+\pi ^+\pi ^-\) in different regions of phase space were investigated in Ref. [25]. The CP violations and the contributions of the strong kaon-pion interactions have been investigated in the \(B \rightarrow K\pi \pi \) decays utilizing an approximate construction of relevant scalar and vector form factors in Ref. [14]. In Ref. [28], the authors studied the decays of \(B^\pm \rightarrow \pi ^\pm \pi ^\mp \pi ^\pm \) within a quasi-two-body QCD factorization approach and introducing the scalar and vector form factors for the S and P waves, as well as a relativistic Breit-Wigner (RBW) formula for the D wave to describe the meson-meson final state interactions.

The perturbative QCD (PQCD) factorization approach has been employed in Refs. [34,35,36,37,38], where the strong dynamics between the two final state hadrons in resonant regions are factorized into a new non-perturbative input, the two-hadron distribution amplitudes (DAs) \(\Phi _{h_1h_2}\) [39,40,41,42,43,44,45]. Both nonresonant and resonant contributions can be accommodated into this new input in PQCD factorization approach. In the PQCD approach, we have studied the S-wave resonance contributions to the decays of \(B^0_{(s)}\) mesons into a charmonium meson plus pion-pion (koan-pion) pair [46,47,48,49,50], the P-wave resonance contributions to the decays \(B \rightarrow P (\rho ,\rho (1450),\rho (1700)) \rightarrow P \pi \pi \) [51,52,53], \(B_{(c)} \rightarrow D (\rho ,\rho (1450),\rho (1700)) \rightarrow D \pi \pi \) [54,55,56] and \(B \rightarrow \eta _c(1S,2S) (\rho ,\rho (1450),\rho (1700)) \rightarrow \eta _c(1S,2S)\pi \pi \) [57], as well as the D-wave resonant contributions to the decays \(B \rightarrow P f_2(1270) \rightarrow P \pi \pi \) [58]. All these works indicate that the PQCD factorization approach is appropriate for exclusive hadronic three-body B meson decays.

The measurements for the branching ratios and CP violating asymmetries for \(B \rightarrow K \pi \pi \) and other decay modes have been reported by BABAR  [59,60,61,62,63,64], Belle [65,66,67,68] and LHCb Collaborations [69,70,71,72,73,74,75]. These three-body decays are known experimentally to be dominated by the low energy resonances on \(\pi \pi \), KK and \(K\pi \) channels on the Dalitz plots. In this work, we shall extend our recent works on the P-wave two-pion resonant contributions to the kaon-pion cases. Motivated by the recent detailed Dalitz plot analyses of \(K\pi \) invariant mass spectrum by BABAR [61, 64, 76,77,78,79], Belle [65, 66, 68, 80,81,82,83], CLEO [84,85,86] and LHCb [87, 88] Collaborations, we will calculate the decay modes \(B \rightarrow K\pi h\), where h is the light pseudoscalar pion or kaon, and study the \(K\pi \) pair originating from a vector quark-antiquark state, while other partial waves are beyond the scope of the present work.

The relevant Feynman diagrams are the same as Fig. 1 in the Ref. [52]. The P-wave contributions are parameterized into the time-like vector form factors involved in the kaon-pion DAs. We adopt the RBW line shape for the P-wave resonance \(K^*(892)\) to parameterize the time-like form factors [64]. Throughout the remainder of the paper, the symbol \(K^*\) is used to denote the \(K^*(892)\) resonance. By employing the kaon-pion DAs, the P-wave contributions to the related three-body B meson decays can be simplified into quasi-two-body processes \(B \rightarrow K^* h\rightarrow K\pi h\).

As is well known, the QCD-improved factorization (QCDF) [15, 89,90,91,92], the perturbative QCD (PQCD) factorization approach [93,94,95] and the soft-collinear-effective theory (SCET) [96,97,98,99,100] are the three popular factorization approaches to deal with the hadronic B meson decays. For most \(B \rightarrow h_1 h_2\) decay channels, the theoretical predictions obtained by using these different factorization approaches agree well with each other and also are well consistent with the data within errors. QCDF and SCET are based on the collinear factorization, in which B meson transition form factors contain the end-point singularity. That is why soft form factors need to be introduced in these approaches. PQCD is based on the \(k_T\) factorization, in which the implementation of the Sudakov resummation suppresses the small \(k_T\) region. It has been shown that \(k_T^2\) is of order \(m_b\Lambda _{QCD}\) in Ref. [101]. The end-point singularity is then smeared, and the form factors are factorizable. That is, the different power countings for the parton \(k_T\) lead to different factorization formalisms. As to the other modes, the hard-collinear modes correspond to the hard kernels in PQCD, and the collinear modes correspond to the DAs in PQCD. The calculation of the infrared logarithms in QCD and in \(k_T\)-dependent DAs for the \(B \rightarrow \pi \) form factors and their cancellation have been done at one-loop level in Ref. [102]. One can see that the \(k_T\)-dependent next-to-leading-order hard kernel for the \(B \rightarrow \pi \) transition form factors is infrared-finite. Therefore, the form factors in \(B \rightarrow \pi \) transition are factorizable. The all-order proof for the \(k_T\) factorization of the \(B \rightarrow \pi \) form factors has been done in Ref. [103].

With the introduction of a two-meson DAs, the LO diagrams for three-body hadronic B meson decays are reduced to those for quasi-two-body decays. The hard kernel H describes the dynamics of the strong and electroweak interactions in three-body hadronic decays in a similar way as the one for the two-body \(B\rightarrow h_1 h_2\) decays. The \(\Phi _B\) and \(\Phi _{h_3}\) are used to describe the wave functions for the B meson and the bachelor particle \(h_3\), which absorb the non-perturbative dynamics in the process. The \(\Phi _{h_1h_2}\) is the two-hadron (K plus \(\pi \) in this work) distribution amplitude, which describes the structure of the final-state K-\(\pi \) pair. As a result, one can describe the typical PQCD factorization formula for a \(B\rightarrow h_1h_2h_3\) decay amplitude as the form of [34, 35],

$$\begin{aligned} {\mathcal {A}}=\Phi _B\otimes H\otimes \Phi ^{P\text {-wave}}_{h_1h_2}\otimes \Phi _{h_3}. \end{aligned}$$
(1)

This paper is organized as follows. In Sect. 2, we give a brief introduction for the theoretical framework. The numerical results, some discussions and the conclusions will be given in last two sections. The factorization formulas for the relevant three-body decay amplitudes are collected in the Appendix.

2 Framework

In the light-cone coordinates, we let the kaon-pion pair and the final-state h move along the direction of \(n=(1,0,0_{\mathrm{T}})\) and \(v=(0,1,0_{\mathrm{T}})\), respectively, in the rest frame of the B meson. The kinematic variables of the decay \(B(p_B) \rightarrow (K\pi )(p)h(p_3)\) can be chosen as

$$\begin{aligned} p_{B}= & {} \frac{m_{B}}{\sqrt{2}}(1,1,0_{\mathrm{T}}),~\quad p=\frac{m_{B}}{\sqrt{2}}(1,\eta ,0_{\mathrm{T}}),\nonumber \\ p_3= & {} \frac{m_{B}}{\sqrt{2}}(0,1-\eta ,0_{\mathrm{T}}), \end{aligned}$$
(2)

where \(m_{B}\) is the mass of B meson, the variable \(\eta \) is defined as \(\eta =\omega ^2/m^2_{B}\), the invariant mass squared \(\omega ^2=p^2\) for the kaon-pion pair. If we choose \(\zeta =p^+_1/p^+\) as kaon momentum fraction, the kaon momentum \(p_1\) and pion momentum \(p_2\) can be written as

$$\begin{aligned} p_1= & {} \left( \zeta \frac{m_B}{\sqrt{2}}, (1-\zeta )\eta \frac{m_B}{\sqrt{2}}, p_{1\mathrm{T}}\right) ,\nonumber \\ p_2= & {} \left( (1-\zeta )\frac{m_B}{\sqrt{2}}, \zeta \eta \frac{m_B}{\sqrt{2}}, p_{2\mathrm{T}}\right) . \end{aligned}$$
(3)

We employ \(x_B, z, x_3\) to denote the momentum fraction of the positive quark in each meson, \(k_{BT}, k_{\mathrm{T}}, k_{3{\mathrm{T}}}\) is assigned to the transverse momentum of the positive quark, respectively. The momentum \(k_B\) of the spectator quark in the B meson, the momentum k for the resonant state \(K^*(892)\) and \(k_3\) for the final-state h are of the form of

$$\begin{aligned} k_{B}= & {} \left( 0,x_B \frac{m_{B}}{\sqrt{2}} ,k_{BT}\right) ,\quad k= \left( \frac{m_{B}}{\sqrt{2}}z,0,k_{\mathrm{T}}\right) ,\nonumber \\ k_3= & {} \left( 0,(1-\eta )x_3 \frac{m_B}{\sqrt{2}},k_{3{\mathrm{T}}}\right) , \end{aligned}$$
(4)

The momentum fractions \(x_{B}\), z and \(x_3\) run from zero to unity.

The P-wave kaon-pion DAs are introduced in analogy with the case of two-pion ones [51] if ignoring the masses of the kaon and pion mesons,

(5)

The expansions of the nonlocal matrix elements for the vector, scalar and tensor spin projectors up to twist-3 are listed below similar to our recent work [104],

$$\begin{aligned}&\langle K(p_1)\pi (p_2)|\bar{q}_1(y^-)\gamma _{\mu }q_2(0)|0\rangle \nonumber \\&\quad =(p_1-p_2)_{\mu } \int _0^1 dz e^{izP\cdot y}\phi _0(z,\omega ), \end{aligned}$$
(6)
$$\begin{aligned}&\langle K(p_1)\pi (p_2)|\bar{q}_1(y^-) I q_2(0)|0\rangle \nonumber \\&\quad =\omega \int _0^1 dze^{izP\cdot y}\phi _s(z,\omega ), \end{aligned}$$
(7)
$$\begin{aligned}&\langle K(p_1)\pi (p_2)|\bar{q}_1(y^-)\sigma _{\mu \nu }q_2(0)|0\rangle \nonumber \\&\quad =-i\frac{(p_{1\mu }p_{2\nu }-p_{1\nu }p_{2\mu })}{\omega } \int _0^1 dze^{izP\cdot y}\phi _t(z,\omega ), \end{aligned}$$
(8)

with the quark content \(q_1=s, q_2=u(d)\) or \(q_1=u(d), q_2=s\). The DA \(\phi _0\) is the twist-2 component, while the DAs \(\phi _s,\phi _t\) are the twist-3 ones. Following the steps of S-wave kaon-pion resonance [50, 105], it is worthwhile to stress that the P-wave kaon-pion system has similar DAs as the ones for a light vector meson, but we replace the vector decay constants with the time-like form factor:

$$\begin{aligned} \phi _0= & {} \frac{3F_{K\pi }(s)}{\sqrt{2N_c}} z (1-z)\left[ 1+a_{1K^*}^{||}3(2z-1)\right. \nonumber \\&\left. +\,a_{2K^*}^{||}\frac{3}{2}(5(2z-1)^2-1)\right] P_1(2\zeta -1), \end{aligned}$$
(9)
$$\begin{aligned} \phi _s= & {} \frac{3F_s(s)}{2\sqrt{2N_c}}(1-2z)P_1(2\zeta -1), \end{aligned}$$
(10)
$$\begin{aligned} \phi _t= & {} \frac{3F_t(s)}{2\sqrt{2N_c}}(2z-1)^2P_1(2\zeta -1), \end{aligned}$$
(11)

where the Legendre polynomial \(P_1(2\zeta -1)=2\zeta -1\) and the Gegenbauer moments \(a_{1K^*}^{||}\) and \(a_{2K^*}^{||}\) will be regarded as free parameters and determined in this work. The time-like form factors \(F_{K\pi }(s), F_s(s), F_t(s)\) define the normalization of the \(K\pi \) two-meson distribution amplitudes. Note that the hadronic matrix element in Eq. (7) vanishes for the local operator with \(y^-=0\), namely, as the DA \(\phi _s\) is integrated over the parton momentum fraction z. It implies that the scalar form factor \(F_s\) can be defined only via a nonlocal matrix element. In principle, \(F_s\) and \(F_t\) should be different. However, we are not able to distinguish them currently because of limited data. Thus, we supposed that they are equal. Following Ref. [51], we also assume that

$$\begin{aligned} F_s(s) = F_t(s) \approx (f_{K^*}^T/f_{K^*}) F_{K\pi }(s). \end{aligned}$$
(12)

with \(f_{K^*}=0.217 \pm 0.005 {\mathrm{GeV}}, f^T_{K^*}=0.185 \pm 0.010 {\mathrm{GeV}}\) [106].

For the narrow resonance \(K^*\), we adopt the RBW line shape for the P-wave resonance \(K^*\) to parameterize the time-like form factors \(F_{K\pi }(s)\), which is widely adopted in the experimental data analyses. The explicit expressions are in the following form [64],

$$\begin{aligned} F_{K\pi }(s)=\frac{m_{K^*}^2}{m^2_{K^*} -s-im_{K^*}\Gamma (s)}, \end{aligned}$$
(13)

with the kaon-pion invariant mass squared \(s=\omega ^2=m^2(K\pi )\).

Here, the mass-dependent width \(\Gamma (s)\) is defined by

$$\begin{aligned} \Gamma (s)=\Gamma _{K^*}\frac{m_{K^*}}{\sqrt{s}}\left( \frac{|\overrightarrow{p_1}|}{|\overrightarrow{p_0}|}\right) ^3, \end{aligned}$$
(14)

where \(m_{K^*}\) and \(\Gamma _{K^*}\) are the pole mass and width of resonance state \(K^*\) respectively. The \(|\overrightarrow{p_1}|\) is the momentum vector of the resonance decay product measured in the resonance rest frame, while \(|\overrightarrow{p_0}|\) is the value of \(|\overrightarrow{p_1}|\) when \(\sqrt{s}=m_{K^*}\). The explicit expression of kinematic variables \(|\overrightarrow{p_1}|\) is

$$\begin{aligned} |\overrightarrow{p_1}|=\frac{\sqrt{\lambda (\omega ^2,m_K^2,m_{\pi }^2)}}{2\omega }, \end{aligned}$$
(15)

with the kaon (pion) mass \(m_K\) (\(m_{\pi })\) and the \(\hbox {K}\ddot{a}\hbox {ll}\acute{e}\hbox {n}\) function \(\lambda (a,b,c)= a^2+b^2+c^2-2(ab+ac+bc)\).

Table 1 The CP averaged branching ratios and direct CP-violating asymmetries of \(B_{(s)}\rightarrow K (K^* \rightarrow ) K \pi \) decays calculated in PQCD approach together with experimental data [107, 108]. The theoretical errors corresponding to the uncertainties due to the next-to-leading-order effects (the hard scale t and the QCD scale \(\Lambda _{\mathrm{QCD}}\)), the shape parameters \(\omega _{B_{(s)}}\) in the wave function of \(B_{(s)}\) meson and the Gegenbauer moments (\(a_{1K^*}^{||}\) and \(a_{2K^*}^{||}\)), respectively
Table 2 The CP averaged branching ratios and direct CP-violating asymmetries of \(B_{(s)}\rightarrow \pi (K^* \rightarrow ) K \pi \) decays calculated in PQCD approach together with experimental data [107, 108]. The theoretical errors corresponding to the uncertainties due to the next-to-leading-order effects (the hard scale t and the QCD scale \(\Lambda _{\mathrm{QCD}}\)), the shape parameters \(\omega _{B_{(s)}}\) in the wave function of \(B_{(s)}\) meson and the Gegenbauer moments (\(a_{1K^*}^{||}\) and \(a_{2K^*}^{||}\)), respectively

3 Numerical results

The following input parameters (the masses, decay constants and QCD scale are in units of GeV) will be used [107] in numerical calculations,

$$\begin{aligned} \Lambda ^{(f=4)}_{ \overline{MS} }= & {} 0.25, \quad m_{B^0}=5.280, \quad m_{B_s}=5.367,\nonumber \\&m_{B^\pm }=5.279, \nonumber \\ m_{\pi ^{\pm }}= & {} 0.140, \quad m_{\pi ^0}=0.135, \quad \nonumber \\&m_{K^{\pm }}=0.494, \quad m_{K^0}=0.498,\nonumber \\ m_{K^{*0}}= & {} 0.89581, \quad m_{K^{*\pm }}=0.89166,\quad \nonumber \\&m_{b}(\text {pole})=4.8, \quad \bar{m_c}(\bar{m_c})=1.275, \nonumber \\ m_s(1{\mathrm{GeV}})= & {} 0.119, \quad \Gamma _{K^*}=0.50, \quad f_B= 0.19\pm 0.02, \quad \nonumber \\&f_{B_s}=0.236\pm 0.02,\nonumber \\ \tau _{B^0}= & {} 1.519\; ps, \quad \tau _{B_{s}}=1.512\; ps, \quad \nonumber \\&\tau _{B^\pm }=1.638\;ps. \end{aligned}$$
(16)

The b-quark mass is chosen as pole mass and the c-quark mass corresponds to the running mass in the modified minimal substraction scheme (\(\overline{MS}\) scheme), while the s-quark mass is the estimation of the so-called “current-quark masses” in a mass-independent subtraction scheme such as \(\overline{MS}\) at a scale \(\mu \approx 1 {\mathrm{GeV}}\). The values of the Wolfenstein parameters are adopted as given in the Ref. [107]: \(A=0.836\pm 0.015, \lambda =0.22453\pm 0.00044\), \(\bar{\rho } = 0.122^{+0.018}_{-0.017}\), \(\bar{\eta }= 0.355^{+0.012}_{-0.011}\). While the B meson and kaon (pion) DAs are the same as widely adopted in the PQCD approach [52].

For the decay \(B \rightarrow K^*(892) h\rightarrow K \pi h\), the differential branching ratio is described as [107],

$$\begin{aligned} \frac{d\mathcal{B}}{ds}=\tau _{B}\frac{|\overrightarrow{p_1}||\overrightarrow{p_3}|}{64\pi ^3m^3_{B}}|\mathcal{A}|^2, \end{aligned}$$
(17)

with \(\tau _{B}\) the mean lifetime of B-meson, and \(s=\omega ^2\) the invariant mass squared. The kinematic variables \(|\overrightarrow{p_1}|\) and \(|\overrightarrow{p_3}|\) denote the kaon momentum and h’s momentum in the center-of-mass frame of the K-\(\pi \) pair,

$$\begin{aligned} |\overrightarrow{p_1}|=\frac{\sqrt{\lambda (\omega ^2,m_K^2,m_{\pi }^2)}}{2\omega }, ~\quad |\overrightarrow{p_3}|=\frac{\sqrt{\lambda (m_B^2,m_P^2,\omega ^2)}}{2\omega }.\nonumber \\ \end{aligned}$$
(18)

By using the differential branching fraction in Eq. (17), and the decay amplitudes in the Appendix, we calculate the CP averaged branching ratios (\(\mathcal B\)) and the direct CP-violating asymmetries (\(\mathcal A_{CP}\)) for the concerned decays \(B \rightarrow K^*h\rightarrow K\pi h\), which are shown in the Tables 1 and 2 together with some currently available experimental measurements. The Gegenbauer moments \(a_{1K^*}^{||}=0.05\pm 0.02, a_{2K^*}^{||}=0.15\pm 0.05\) are determined to cater to the data [107, 108], which differ from those in the DAs for a longitudinally polarized \(K^*\) meson [106]. The first theoretical error from the variation of the hard scale t from 0.75t to 1.25t (without changing \(1/b_i\)) and the QCD scale \(\Lambda _{\mathrm{QCD}}=0.25\pm 0.05\) GeV, which characterizes the effect of the NLO QCD contributions. The second error comes from the variations of the shape parameter of the \(B_{(s)}\) meson distribution amplitude \(\omega _B=0.40\pm 0.04\) GeV or \(\omega _{B_s}=0.50\pm 0.05\) GeV [93, 94, 109, 110]. The last one is caused by the Gegenbauer moments \(a_{1K^*}^{||}=0.05\pm 0.02, a_{2K^*}^{||}=0.15\pm 0.05\). The first two errors are comparable and contribute the main uncertainties in our approach, while the last one is less than \(15\%\). The errors from \(\tau _{B^\pm }\), \(\tau _{B^0}\), \(\tau _{B_s}\) and the Wolfenstein parameters in [107] are very small and have been neglected.

From the numerical results as shown in those two tables, one can address some issues as follows:

  1. 1.

    The isospin conservation is assumed for the strong decays of an \(I=1/2\) resonance \(K^*\) to \(K\pi \) when we compute the branching fractions of the quasi-two-body process \(B \rightarrow K^*h\rightarrow K\pi h\), namely,

    $$\begin{aligned} \frac{\Gamma (K^{*0} \rightarrow K^+\pi ^-)}{\Gamma (K^{*0} \rightarrow K\pi )}=2/3, \frac{\Gamma (K^{*+} \rightarrow K^+\pi ^0)}{\Gamma (K^{*+} \rightarrow K\pi )}=1/3.\nonumber \\ \end{aligned}$$
    (19)

    Taking \(B^0 \rightarrow \pi ^0(K^{*0}\rightarrow )K \pi \) decay as an example, we can obtain the quasi-two-body branching fraction \(\mathcal{B}(B^0 \rightarrow \pi ^0(K^{*0}\rightarrow )K^+ \pi ^-)\) under the narrow width approximation relation

    $$\begin{aligned} {\mathcal {B}}(B^0 \rightarrow \pi ^0 K^{*0}\rightarrow \pi ^0 K \pi )= & {} {\mathcal {B}}( B^0 \rightarrow \pi ^0 K^{*0}) \nonumber \\&\cdot \, {{\mathcal {B}}}(K^{*0} \rightarrow K\pi ),\nonumber \\ {\mathcal {B}}(B^0 \rightarrow \pi ^0 K^{*0}\rightarrow \pi ^0 K^+ \pi ^- )= & {} {\mathcal {B}}( B^0 \rightarrow \pi ^0 K^{*0}) \nonumber \\&\cdot \,{{\mathcal {B}}}(K^{*0} \rightarrow K^+\pi ^-), \nonumber \\ \end{aligned}$$
    (20)

    where we assume the \(K^*\rightarrow K\pi \) branching fraction to be 100%.

  2. 2.

    It is worth of stressing that there already exist many well known results for \(B_{(s)} \rightarrow hK^*\) in the two-body framework both in the PQCD [106, 111,112,113] and QCDF [15, 114, 115] approaches. In the narrow width limit, the branching ratios of the two-body decays \(B_{(s)} \rightarrow hK^*\) are extracted from the corresponding quasi-two-body decay modes as listed in Tables 1 and 2. One can see that the branching ratios of the quasi-two-body decay modes are in good agreement with those two-body analyses as presented in Refs. [106, 111,112,113] in PQCD approach. The consistency between the theoretical predictions and the measured values for the branching ratios supports the PQCD factorization for exclusive hadronic B meson decays. The measured CP violation is just a number in two-body B decays, while in three-body decays, one can measure the distribution of CP asymmetry. The CP asymmetry in the three-body framework is moderated by the finite width of the \(K^*\) resonance appearing in the time-like form factor \(F_{K\pi }\). It may be more appropriate to treat \(B\rightarrow hK^*\) as three-body decays. By comparing with corresponding results in the QCDF approach [15, 114, 115], we find that the PQCD predictions for the branching ratios as listed in Tables 1 and  2 are similar to the QCDF results. Since the mechanism and the source of the CP asymmetries for the considered decay modes are very different in the PQCD approach and the QCDF approach, the QCDF results for the direct CP asymmetries are quite different from ours. Because currently available experimental measurements still have relatively large uncertainties, we have to wait for more time to test these different predictions.

  3. 3.

    We calculated the branching ratios and CP violations of the quasi-two-body decays \(B \rightarrow K K^* \rightarrow K K\pi \) as shown in Table 1. Moreover, there is no CP violation for the decays \(B^0 \rightarrow K^0(\bar{K}^{*0}\rightarrow )K \pi \), \(B^0 \rightarrow \bar{K}^0(K^{*0}\rightarrow )K \pi \), \(B_s^0 \rightarrow K^0(\bar{K}^{*0}\rightarrow )K \pi \) and \(B_s^0 \rightarrow \bar{K}^0(K^{*0}\rightarrow )K \pi \) within the standard model, since there is only one kind of penguin operator involved in the decay amplitudes of the considered decays, which can be seen from Eqs. (A7A10). The PQCD predictions of the sum of branching ratios of \(B_s^0 \rightarrow K^+(K^{*-}\rightarrow )K\pi \) and \(B_s^0 \rightarrow K^-(K^{*+}\rightarrow )K \pi \) decays, as well as the sum of branching ratios of \(B_s^0 \rightarrow K^0(\bar{K}^{*0}\rightarrow )K \pi \) and \(B_s^0 \rightarrow \bar{K}^0(K^{*0}\rightarrow )K \pi \) are in consistent with the LHCb measurements [87, 88] and support their first observations of \(B_s^0\) meson decays to \(K^{*\pm }K^\mp \) and \(K^0_SK^{*0}\). The LHCb Collaboration reported that there is no evidence for the decay \(B^0 \rightarrow K^0_SK^{*0}\) and an upper limit is set on the branching ratio. Our result for the \(\mathcal{B}(B^0 \rightarrow K^0(\bar{K}^{*0}\rightarrow )K \pi )\) plus \(\mathcal{B}(B^0 \rightarrow \bar{K}^0(K^{*0}\rightarrow )K \pi )\) is around \(0.54 \times 10^{-6}\), which can be examined in the forthcoming experiments.

  4. 4.

    For the considered decay channels \(B \rightarrow \pi (K^* \rightarrow ) K \pi \), there are already some experimental measurements for the branching ratios and CP asymmetries shown in the forth column in Table 2. Although the error bars of the CP violations are still large, one can find that our theoretical calculations have the same sign as these measured entries. For the four decay modes, \(\mathcal{B}( B^+ \rightarrow \pi ^+(K^{*0}\rightarrow )K \pi )=(7.12^{+2.77}_{-2.07})\times 10^{-6}\), \(\mathcal{B}(B^0 \rightarrow \pi ^-(K^{*+}\rightarrow )K \pi )=(6.51^{+2.33}_{-1.75})\times 10^{-6}\), \(\mathcal{B}(B^+ \rightarrow \pi ^0(K^{*+}\rightarrow )K \pi )=(5.00^{+1.78}_{-1.34})\times 10^{-6}\) and \(\mathcal{B}(B^0 \rightarrow \pi ^0(K^{*0}\rightarrow )K \pi ) =(2.07^{+0.83}_{-0.62})\times 10^{-6}\), the PQCD predictions are in agreement with the world averages within errors. When more data become available, we do recommend the LHCb and/or Belle-II experiments to remeasure the direct CP asymmetry in channels like \(B^+ \rightarrow \pi ^0(K^{*+}\rightarrow )K \pi \), \(B^0 \rightarrow \pi ^0(K^{*0}\rightarrow )K \pi \) and so on, because these decay modes may have large branching ratios and large direct CP asymmetries.

  5. 5.

    For the \(B_s^0 \rightarrow \pi ^+(K^{*-}\rightarrow )K\pi \) decay process, our prediction is \(\mathcal{B}=(8.08^{+2.88}_{-2.09}) \times 10^{-6}\) at leading-order in the quasi-two-body framework in this work, such a branching ratio is a bit larger than the value \((3.3\pm 1.2)\times 10^{-6}\) in [107]. For the corresponding two-body modes, the previous theoretical predictions as given in Refs. [15, 106, 115] are larger than the data as well. In Ref. [111], the authors considered the next-to-leading-order (NLO) corrections and found that the NLO contribution will result in a 37% reduction of the leading order PQCD prediction for the “tree” dominated decay \(B_s^0 \rightarrow \pi ^+K^{*-}\). The authors confirmed that the branching ratios of the quasi-two-body modes in the three-body and two-body frameworks are close to each other in Ref. [51], since the \(\mathcal{B}(\rho \rightarrow \pi \pi )\approx 100\%\). Compared with the previous calculations of the two-body decays \(B_{(s)} \rightarrow P K^*\) from PQCD [106, 111,112,113], we can obtain the consistency between the two-body and three-body modes. Maybe we can assume that the PQCD prediction of the branching ratio of the quasi-to-body decay \(B_s^0 \rightarrow \pi ^+(K^{*-}\rightarrow )K\pi \) will accommodate to data if we take the NLO contributions into consideration in the three-body framework. However, how to evaluate the NLO corrections to the three-body decays in the PQCD framework is a big task and will be left for the future studies.

In Fig. 1a, we show the \(\omega \)-dependence of differential decay rate \(d\mathcal{B}(B^+ \rightarrow K^+\bar{K}^{*0} \rightarrow K^+K\pi )/d\omega \). The \(\bar{K}^{*0}\) is visible as a narrow peak near 0.89 GeV. We find that the main portion of the branching ratios lies in the region around the pole mass of the \(K^*\) resonance as expected by examining the distribution of the branching ratios in the kaon-pion invariant mass \(\omega \). The central values of \(\mathcal{B}\) are \(0.25\times 10^{-7}\) and \(0.37\times 10^{-7}\) when the integration over \(\omega \) is limited in the range of \(\omega =[m_{K^*}-0.5\Gamma _{K^*}, m_{K^*}+0.5\Gamma _{K^*}]\) or \(\omega =[m_{K^*}-\Gamma _{K^*}, m_{K^*}+\Gamma _{K^*}]\) respectively, which amount to \(50\%\) and \(74\%\) of the total branching ratio \(\mathcal{B}=0.50\times 10^{-7}\) as listed in Table 1. In two-body B decays, the measured CP violation is just a number due to the fixed kinematics. While in three-body decays, the decay amplitudes depend on the \(K\pi \) invariant mass, which resulting in the differential distribution of direct CP asymmetries. In Fig. 1b, we display the differential distributions of \({\mathcal {A}}_{CP}\) for the two decay modes \(B^+\rightarrow K^+\bar{K}^{*0}\rightarrow K^+K\pi \) (black solid line) and \(B^+\rightarrow \bar{K}^0 K^{*+}\rightarrow \bar{K}^0K\pi \) (purple dotted line), respectively. One can find a falloff of \({\mathcal {A}}_{CP}\) with \(\omega \) for \(B^+\rightarrow K^+\bar{K}^{*0}\rightarrow K^+K\pi \). It implies that the direct CP asymmetries in the above three quasi-two-body decays, if calculated as the two-body decays with the \(K^*\) resonance mass being fixed to \(m_{K^*}\), may be overestimated. The ascent of the differential distribution of \({\mathcal {A}}_{CP}\) with \(\omega \) for \(B^+\rightarrow \bar{K}^0 K^{*+}\rightarrow \bar{K}^0K\pi \) suggests that its direct CP asymmetry, if calculated in the two-body formalism, may be underestimated.

4 Summary

In this paper, we calculated the quasi-two-body decays \(B_{(s)}\rightarrow K^*(892)h\rightarrow K\pi h\) by using the PQCD factorization approach. The relativistic Breit–Wigner formula for the P-wave narrow resonance \(K^*(892)\) was adopted to parameterize the time-like form factor \(F_{K\pi }\). The kaon-pion distribution amplitude \(\Phi _{K\pi }^\mathrm{P}\) with the P-wave time-like form factor \(F_{K\pi }\) was employed to describe the resonant state \(K^*\) and its interactions with the kaon-pion pair. We predicted the branching ratios and the direct CP asymmetries of the concerned decay channels, and compared their differential branching ratios with currently available data. General agreements between the PQCD predictions and the data can be achieved by tuning the Gegenbauer moments of the P-wave kaon-pion DAs. The direct CP asymmetry of the \(B_{(s)}\rightarrow K^*(892)h\rightarrow K\pi h\) modes are not numbers but depend on the kaon-pion invariant mass. Owing to the isospin conservation in \(K^*(892) \rightarrow K\pi \) decays, we can obtain the separate branching ratios of the corresponding quasi-two-body decays. More precise data from the LHCb and the future Belle II will test our predictions.