1 Introduction

Multi-body B meson decays offer one of the best tools for studying direct CP violation and provide an interesting testing ground for strong interaction dynamical models. Theoretically, \(B_{(s)} \rightarrow V_1V_2, S_1S_2, S_1V_2, V_1S_2\) decays (here \(V_{1,2}\) and \(S_{1,2}\) denote the vector and scalar mesons, respectively) are treated as two-body final states and have been studied in the two-body framework using QCD factorization (QCDF) [1,2,3,4,5,6], the perturbative QCD (PQCD) approaches [7,8,9,10,11,12,13,14,15,16,17,18,19], the soft-collinear-effective theory (SCET) [20,21,22,23,24,25] and the factorization-assisted topological amplitude approach (FAT) [26]. While they are at least four-body decays on the experimental side shown in Fig. 1, since the vector (scalar) mesons decay via the strong interaction with a nontrivial width. In recent years, four-body charmless hadronic B decays have been reported by BABAR [27,28,29,30,31,32], Belle [33,34,35,36] and LHCb [37,38,39,40,41,42,43,44] Collaborations, and the branching ratios for many partial waves have been measured, see Tables 1 and 2 for a summary of the experimental results. In addition to the branching ratios and the polarization fractions, other observables constructed from the helicity amplitudes are also interesting. The phenomenology of these decay modes provides rich opportunities for our understanding of the mechanism for hadronic weak decays and their CP asymmetry, and the search for physics beyond the standard model (SM).

Besides the direct CP asymmetries, there is another signal of CP violation in the angular distribution of \(B_{(s)} \rightarrow V_1V_2\) decays, which is called triple-product asymmetries (TPAs) [47,48,49,50,51,52,53,54,55]. These triple products are odd under the time reversal transformation (T), and also contribute potential signals of CP violation due to the CPT theorem. TPAs have already been measured by BABAR, Belle, CDF and LHCb [40, 43, 56,57,58,59,60]. It is known that a non-vanishing direct CP violation needs the interference of at least two amplitudes with a weak phase difference \(\Delta \phi \) and a strong phase difference \(\Delta \delta \). The direct CP violation is proportional to \(\sin \Delta \phi \sin \Delta \delta \), while TPAs go as \(\sin \Delta \phi \cos \Delta \delta \). If the strong phases are quite small, the magnitude of the direct CP violation is close to zero, but the TPA is maximal. Hence direct CP violation and TPAs complement each other. Even if the effect of CP violation is absent, T-odd triple products (also called “fake” TPAs), which are proportional to \(\cos \Delta \phi \sin \Delta \delta \), can provide useful complementary information on new physics [51]. TPAs are excellent probes of physics beyond the SM since most TPAs are expected to be tiny within the SM and are not suppressed by the small strong phases.

As is well known, the kinematics of two-body decays is fixed, while the multi-body decay amplitudes depend on at least two kinematic variables. Meanwhile, the multi-body decays not only receive the resonant and nonresonant contributions, but also involve the possible significant final-state interactions (FSIs) [61,62,63]. In this respect, multi-body decays are considerably more challenging than two-body decays, but provide a number of theoretical and phenomenological advantages. In two-body B decays, the measured CP violation is just a number while the CP asymmetry depends on the invariant mass of the two-meson pair and varies from region to region in the Dalitz plot [64, 65] in the three-body modes [66]. In addition, strong phases in multi-body decays arise nonperturbatively already at the leading power, through complex phases in matrix elements such as \(F_{\pi }\sim <0|j|\pi \pi>\) and so on. Since the \(B_{(s)} \rightarrow V_1V_2, V_1S_2, S_1V_2, S_1S_2\) decays are expected to proceed through \(V_1(S_1) \rightarrow P_1P_2\) and \(V_2(S_2) \rightarrow Q_1Q_2\) with \(P_{1,2}\) (\(Q_{1,2}\)) denoting pseudoscalar meson \(\pi \) or K, it is meaningful to study such decays in the four-body framework, which provide useful information for understanding the CP-violation mechanisms.

As addressed above, multi-body decays of heavy mesons involve more complicated dynamics than two-body decays. A factorization formalism that describes a multi-body decay in full phase space is not yet available at present. It has been proposed that the factorization theorem of three-body B decays is approximately valid when two particles move collinearly and the bachelor particle recoils back [67, 68]. More details can also be found in Refs. [69, 70]. This situation exists particularly in the low \(\pi \pi \) or \(K\pi \) invariant mass region (\(\lesssim \)2 GeV) of the Dalitz plot where most resonant structures are seen. The Dalitz plot is typically dominated by resonant quasi-two-body contributions along the edge. This proposal provides a theoretical framework for studies of resonant contributions based on the quasi-two-body-decay mechanism. Several theoretical approaches have been developed for describing the three-body hadronic decays of B mesons based on the symmetry principles [71,72,73,74,75,76], the QCDF [77,78,79,80,81,82,83,84,85,86,87] and the PQCD approaches [88,89,90,91,92,93,94,95,96,97,98,99,100,101,102]. Recently, the localized CP violation and branching fraction of the four-body decay \({\bar{B}}^0\rightarrow K^-\pi ^+\pi ^+\pi ^-\) have been calculated by employing a quasi-two-body QCDF approach in Refs. [103, 104]. Similar to three-body B meson decays, four-body \(B_{(s)} \rightarrow R_1R_2\rightarrow (P_1P_2)(Q_1Q_2)\) decay modes (\(R_{1,2}\) represents the vector or scalar intermediate resonance) are assumed to proceed dominantly with two intermediate resonances \(R_1\) or \(R_2\) each decaying to a pseudoscalar pair. As a first step, we can only restrict ourselves to the specific kinematical configurations in which each two particles move collinearly and two pairs of final state particles recoil back in the rest frame of the B meson, see Fig. 1. Naturally the dynamics associated with the pair of final state mesons can be factorized into a two-meson distribution amplitude (DA) \(\Phi _{h_1h_2}\) [105,106,107,108,109,110,111]. Thereby, the typical PQCD factorization formula for the considered four-body decay amplitude can be described as the form of,

$$\begin{aligned} {\mathcal {A}}=\Phi _B\otimes H\otimes \Phi _{P_1P_2}\otimes \Phi _{Q_1Q_2}, \end{aligned}$$
(1)

where \(\Phi _B\) is the universal wave function of the B meson and absorbs the non-perturbative dynamics in the process. The \(\Phi _{P_1P_2,Q_1Q_2}\) is the two-hadron DA, which involves the resonant and nonresonant interactions between the two moving collinearly mesons. The hard kernel H describes the dynamics of the strong and electroweak interactions in four-body hadronic decays in a similar way as the one for the corresponding two-body decays.

In this work, we study the four-body decays \(B_{(s)} \rightarrow (\pi \pi )(K\pi )\) in the PQCD approach based on \(k_T\) factorization with the relevant Feynman diagrams illustrated in Fig. 2. In the considered \((\pi \pi )\) invariant-mass range, the vector resonance \(\rho (770)\) is expected to contribute, together with the scalar resonances \(f_0(500)\) and \(f_0(980)\). The \(K\pi \) spectrum is dominated by the vector \(K^*(892)\) resonance and the scalar resonance \(K_0^*(1430)\). Throughout the remainder of the paper, the symbol \(\rho \) is used to denote the \(\rho (770)\) resonance and \(K^*\) is for \(K^*(892)\) resonance. For a comparison with the LHCb experiment [43], the invariant mass of the \(\pi \pi \) pair is restricted from 300 to 1100 MeV and the range of invariant mass for \(K\pi \) pair varies from 750 to 1200 MeV. We calculate the branching ratios and polarization fractions of each partial waves. Besides, a set of CP-violating observables is investigated using \(B_{(s)}\) meson decays reconstructed in the \((\pi \pi )(K\pi )\) quasi-two-body final state. Particular emphasis is placed on the TPAs of the considered decays. It should be mentioned that there is a possibility of existing two identical final state pions. Taking the \(B^+ \rightarrow (f_0(980) \rightarrow )\pi ^+\pi ^-(K^{*+}\rightarrow )K^0\pi ^+\) decay for an example, in the experimental side, they see four final states \(\pi ^+_{1}\pi ^-K^0\pi ^+_{2}\) first, and have to pair one of the positive charged pions with the \(\pi ^-\). They have to try two possible combinations. From the theoretical point of view, we deal with \(B^+ \rightarrow f_0(980) K^{*+}\rightarrow \pi ^+_{1}\pi ^-K^0\pi ^+_{2}\) in the quasi-two-body mechanism and see \(f_0(980)K^{*+}\) first. Then each quasi-two-body intermediate resonance decays into two pseudoscalars (\(f_0(980)\rightarrow \pi ^+_{1}\pi ^-\) and \(K^{*0}\rightarrow K^0\pi ^+_{2}\)), respectively. Each meson pair generates a smaller invariant mass and flies back-to-back as shown in Fig. 1. It is evident that the final states \(\pi ^+\) have been already specified unambiguously on the theoretical side. Certainly, it is experimentally difficult to identify which resonance a \(\pi ^+\) comes from in the nonresonant region on a Dalitz plot. From the theoretical viewpoint, this difficulty implies that the tail of our Breit-Wigner propagator may not describe the events in the nonresonant region correctly. However, it is known that the nonresonant region gives a minor contribution, so the misparing of a \(\pi ^+\) with other final states should not make a significant impact to our results.

The layout of the present paper is as follows. In Sect. 2, we give an introduction for the theoretical framework. The numerical values and some discussions will be given in Sect. 3. Section 4 contains our conclusions. The Appendix collects the explicit PQCD factorization formulas for all the decay amplitudes.

Fig. 1
figure 1

Graphical definitions of the helicity angles \(\theta _1\), \(\theta _2\) and \(\phi \). Taking the \(B^0 \rightarrow R_1R_2\) decay as an example, with each quasi-two-body intermediate resonance decaying to two pseudoscalars (\(R_1\rightarrow \pi ^+\pi ^-\) and \(R_2\rightarrow K^+\pi ^-\)), \(\theta _1(\theta _2)\) is denoted as the angle between the directions of motion of \(\pi ^+\) (\(\pi ^-\)) in the \(R_1(R_2)\) rest frame and \(R_1(R_2)\) in the \(B^0\) rest frame, and \(\phi \) is the angle between the plane defined by \(\pi ^+\pi ^-\) and the plane defined by \(K^+\pi ^-\) in the \(B^0\) rest frame

Table 1 Experimental branching fractions (in units of \(10^{-6}\)) of \(B \rightarrow (\pi \pi )(K\pi )\) decays [45, 46]
Table 2 Experimental CP asymmetries (in units of %) of \(B \rightarrow (\pi \pi )(K\pi )\) decays [45, 46]
Fig. 2
figure 2

Typical leading-order Feynman diagrams for the four-body decays \(B \rightarrow (R_1\rightarrow ) \pi \pi (R_2\rightarrow ) K\pi \) with \(q=(u,d,s)\), where the symbol \(\bullet \) denotes a weak interaction vertex. The diagrams (a)–(d) represent the \(B\rightarrow (R_1\rightarrow ) \pi \pi \) transition, as well as the diagrams (e)–(h) for annihilation contributions. If we exchange the position of \(R_1(\rightarrow \pi \pi )\) and \(R_2 (\rightarrow K\pi )\), we will find the diagrams (a)–(d) for the \(B\rightarrow (R_2\rightarrow ) K\pi \) transition

2 Framework

2.1 Kinematics

Table 3 Parameters used to describe intermediate states in our framework. GS and RBW refer to the Gounaris-Sakurai and relativistic Breit-Wigner line shapes, respectively

Consider a four-body \(B(p_B)\rightarrow R_1(p)R_2(q)\rightarrow P_1(p_1)P_2(p_2)Q_1(q_1)Q_2(q_2)\) decay, we will work in the B meson rest frame and employ the light-cone coordinates for momentum variables. The B meson momentum \(p_{B}\), the total momenta of the pion-pion and kaon-pion pairs, \(p=p_1+p_2\), \(q=q_1+q_2\), and the quark momentum \(k_i\) in each meson are chosen as

$$\begin{aligned} p_{B}= & {} \frac{m_{B}}{\sqrt{2}}(1,1,0_{\mathrm{T}}),\quad k_{B}=\left( 0,x_B \frac{m_{B}}{\sqrt{2}} ,k_{B \mathrm T}\right) ,\nonumber \\ p= & {} \frac{m_{B}}{\sqrt{2}}(g^+,g^-,0_{\mathrm{T}}),\quad k_p= \left( x_1 g^+\frac{m_{B}}{\sqrt{2}},0,k_{\mathrm{T}}\right) ,\nonumber \\ q= & {} \frac{m_{B}}{\sqrt{2}}(f^-,f^+,0_{\mathrm{T}}), \quad k_q=\left( 0,x_2f^+ \frac{m_B}{\sqrt{2}},k_{3\mathrm{T}}\right) , \end{aligned}$$
(2)

where \(m_{B}\) is the mass of B meson. The momentum fractions \(x_{B}\), \(x_1\) and \(x_2\) run from zero to unity.

The factors \(g^{\pm }\) and \(f^{\pm }\) can be obtained through the momentum conservation \(p_B=p+q\), and \(p^2=\omega _1^2\) and \(q^2=\omega _2^2\), respectively. The explicit expressions of \(g^{\pm }\) and \(f^{\pm }\) are written in the following form

$$\begin{aligned} g^{\pm }= & {} \frac{1}{2}\left[ 1+\eta _1-\eta _2\pm \sqrt{(1+\eta _1-\eta _2)^2-4\eta _1}\right] ,\nonumber \\ f^{\pm }= & {} \frac{1}{2}\left[ 1-\eta _1+\eta _2\pm \sqrt{(1+\eta _1-\eta _2)^2-4\eta _1}\right] , \end{aligned}$$
(3)

with the factor \(\eta _{1,2}=\frac{\omega _{1,2}^2}{m^2_{B}}\). If the meson pairs are in the P-wave configuration, the corresponding longitudinal polarization vectors are defined as

$$\begin{aligned} \epsilon _{p}=\frac{1}{\sqrt{2\eta _1}}(g^+,-g^-,\mathbf{0} _{T}),\quad \epsilon _{q}=\frac{1}{\sqrt{2\eta _2}}(-f^-,f^+,\mathbf{0} _{T}),\nonumber \\ \end{aligned}$$
(4)

which satisfy the normalization \(\epsilon _{p}^2=\epsilon _{q}^2=-1\) and the orthogonality \(\epsilon _{p}\cdot p=\epsilon _{q}\cdot q=0\).

As usual we also define the momentum \(p_{1,2}\) of pion-pion pair and \(q_{1,2}\) for the kaon-pion pair as

$$\begin{aligned} p_1= & {} \left( \frac{m_{B}}{\sqrt{2}}(\zeta _1+\frac{r_1-r_2}{2\eta _1})g^+,\frac{m_{B}}{\sqrt{2}}(1-\zeta _1+\frac{r_1-r_2}{2\eta _1})g^-,\mathbf{p} _{T}\right) ,\nonumber \\ p_2= & {} \left( \frac{m_{B}}{\sqrt{2}}(1-\zeta _1-\frac{r_1-r_2}{2\eta _1})g^+,\frac{m_{B}}{\sqrt{2}}(\zeta _1-\frac{r_1-r_2}{2\eta _1})g^-,-\mathbf{p} _{T}\right) ,\nonumber \\ q_1= & {} \left( \frac{m_{B}}{\sqrt{2}}(1-\zeta _2+\frac{r_3-r_4}{2\eta _2})f^-,\frac{m_{B}}{\sqrt{2}}(\zeta _2+\frac{r_3-r_4}{2\eta _2})f^+,\mathbf{q} _{T}\right) ,\nonumber \\ q_2= & {} \left( \frac{m_{B}}{\sqrt{2}}(\zeta _2-\frac{r_3-r_4}{2\eta _2})f^-,\frac{m_{B}}{\sqrt{2}}(1-\zeta _2-\frac{r_3-r_4}{2\eta _2})f^+,-\mathbf{q} _{T}\right) ,\nonumber \\ p_{\mathrm{T}}^2= & {} \zeta _1(1-\zeta _1)\omega _1^2+\frac{(m_1^2-m_2^2)^2}{4\omega _1^2}-\frac{m^2_1+m^2_2}{2},\nonumber \\ q_{\mathrm{T}}^2= & {} \zeta _2(1-\zeta _2)\omega _2^2+\frac{(m_3^2-m_4^2)^2}{4\omega _2^2}-\frac{m^2_3+m^2_4}{2}, \end{aligned}$$
(5)

with \(\zeta _1+\frac{r_1-r_2}{2\eta _1}=p_1^+/p^+\) and \(\zeta _2+\frac{r_3-r_4}{2\eta _2}=q_1^-/q^-\) characterizing the momentum fraction for one of pion-pion (kaon-pion) pair, and the mass ratios \(r_{1,2,3,4}=m_{1,2,3,4}^2/m^2_B\), \(m_{1,2,3,4}\) being the masses of the final state meson.

One can obtain the relation between \(\zeta _{1,2}\) and the polar angle \(\theta _{1,2}\) in the dimeson rest frame in Fig. 1,

$$\begin{aligned} 2\zeta _{1}-1= & {} \sqrt{1-2\frac{r_1+r_2}{\eta _1}+\frac{(r_1-r_2)^2}{\eta _1^2}}\cos \theta _1,\nonumber \\ 2\zeta _{2}-1= & {} \sqrt{1-2\frac{r_3+r_4}{\eta _2}+\frac{(r_3-r_4)^2}{\eta _2^2}}\cos \theta _2, \end{aligned}$$
(6)

with the upper and lower limits of \(\zeta _{1,2}\)

$$\begin{aligned} \zeta _{1\text {max,min}}= & {} \frac{1}{2}\left[ 1\pm \sqrt{1+4\alpha _1}\right] ,\nonumber \\ \zeta _{2\text {max,min}}= & {} \frac{1}{2}\left[ 1\pm \sqrt{1+4\alpha _2}\right] . \end{aligned}$$
(7)

For the sake of simplicity, we generally use the factor \(\alpha _1=-\frac{r_1+r_2}{2\eta _1}+\frac{(r_1-r_2)^2}{4\eta _1^2}\) and \(\alpha _2=-\frac{r_3+r_4}{2\eta _2}+\frac{(r_3-r_4)^2}{4\eta _2^2}\) in the following sections.

The differential branching fraction for the \(B_{(s)}\rightarrow (\pi \pi )(K\pi )\) in the \(B_{(s)}\) meson rest frame is expressed as

$$\begin{aligned} \frac{d^5{\mathcal {B}}}{d\Omega }=\frac{\tau _B k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)}{16(2\pi )^6m_B^2} |A|^2, \end{aligned}$$
(8)

where \(d \Omega \) with \(\Omega \equiv \{\theta _1, \theta _2,\phi ,\omega _1,\omega _2\}\) stands for the five-dimensional measure spanned by the three helicity angles and the two invariant masses, and

$$\begin{aligned} k(\omega _1,\omega _2)= & {} \frac{\sqrt{[m_B^2-(\omega _1+\omega _2)^2][m_B^2-(\omega _1-\omega _2)^2]}}{2m_B}, \end{aligned}$$
(9)

is the momentum of the \(\pi \pi \) pair in the \(B_{(s)}\) meson rest frame. The explicit expression of kinematic variables \(k(\omega )\) is defined in the \(h_1h_2\) center-of-mass frame

$$\begin{aligned} k(\omega )=\frac{\sqrt{\lambda (\omega ^2,m_{h_1}^2,m_{h_2}^2)}}{2\omega }, \end{aligned}$$
(10)

with the K\(\ddot{a}\)ll\(\acute{e}\)n function \(\lambda (a,b,c)= a^2+b^2+c^2-2(ab+ac+bc)\) and \(m_{h_1,h_2}\) being the final state mass.

The four-body phase space has been derived in the analyses of the \(K\rightarrow \pi \pi l\nu \) decay [112], the semileptonic \({\bar{B}}\rightarrow D(D^*)\pi l \nu \) decays [113], semileptonic baryonic decays [114, 115], and four-body baryonic decays [116]. One can confirm that Eq. (8) is equivalent to those in Refs. [114, 116] by appropriate variable changes. Replacing the helicity angle \(\theta \) by the meson momentum fraction \(\zeta \) via Eq. (6), the Eq. (8) is turned into

$$\begin{aligned}&\frac{d^5{\mathcal {B}}}{d\zeta _1d\zeta _2d \omega _1d \omega _2d\phi }\nonumber \\&\quad = \frac{\tau _B k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)}{4(2\pi )^6m_B^2\sqrt{1+4\alpha _1}\sqrt{1+4\alpha _2}}|A|^2. \end{aligned}$$
(11)

2.2 Helicity amplitudes

One can disentangle the helicities of \(R_1(\rightarrow \pi \pi )R_2(\rightarrow K\pi )\) final state via an angular analysis, depicted in Fig. 1. Taking the \(B^0 \rightarrow R_1R_2 \rightarrow (\pi ^+\pi ^-)(K^+\pi ^-)\) decay as an example, the \(\theta _1\) is the angle between the \(\pi ^+\) direction in the \((\pi ^+\pi ^-)\) rest frame and the \((\pi ^+\pi ^-)\) direction in the \(B^0\) rest frame, \(\theta _2\) is the angle between the \(K^+\) direction in the \((K^+\pi ^-)\) rest frame and the \((K^+\pi ^-)\) direction in the \(B^0\) rest frame, and \(\phi \) is the angle between the \((\pi ^+\pi ^-)\) and \((K^+\pi ^-)\) decay planes. A \(\pi \pi (K\pi )\) pair can be produced in the S or P-wave configuration in the selected invariant mass regions.

One decomposes the decay amplitudes into six helicity components: \(h=VV\) (3), VS, SV, and SS, each with a corresponding amplitude \(A_h\). The first three, commonly referred to as the P-wave amplitudes, are associated with the final states, where both \(\pi \pi \) and \(K\pi \) pairs come from intermediate vector mesons. In the transversity basis, a P-wave decay amplitude can be decomposed into three components: \(A_0\), for which the polarizations of the final-state vector mesons are longitudinal to their momenta, and \(A_{\parallel }\) (\(A_\perp \)), for which the polarizations are transverse to the momenta and parallel (perpendicular) to each other. As the S-wave \(\pi \pi (K\pi )\) pair arises from \(R_1\) (\(R_2\)) labelled in Fig. 2a, the corresponding single S-wave amplitude is denoted \(A_{SV}\) (\(A_{VS}\)). The double S-wave amplitude \(A_{SS}\) is associated with the final state, where both two-meson pairs are generated in the S wave. Specifically, these helicity amplitudes for the \(B_{(s)}\rightarrow (\pi \pi )(K\pi )\) decay denote

$$\begin{aligned}&A_{VV}: B_{(s)} \rightarrow \rho (\rightarrow \pi \pi ) K^*(\rightarrow K\pi ), \nonumber \\&A_{VS}: B_{(s)} \rightarrow \rho (\rightarrow \pi \pi )K_0^*(1430)(\rightarrow K\pi ) ,\nonumber \\&A_{SV}: B_{(s)} \rightarrow f_0(500,980)(\rightarrow \pi \pi ) K^*(\rightarrow K\pi ),\nonumber \\&A_{SS}: B_{(s)} \rightarrow f_0(500,980)(\rightarrow \pi \pi ) \nonumber \\&K_0^*(1430)(\rightarrow K\pi ). \end{aligned}$$
(12)

Relying on the Eq. (6), we get the full decay amplitude in Eq. (11) as a coherent sum of the P-, S-, and double S-wave components by including the \(\zeta _{1,2}\) dependencies instead of \(\theta _{1,2}\) and azimuth-angle dependencies,

$$\begin{aligned} A= & {} A_0\frac{2\zeta _1-1}{\sqrt{1+4\alpha _1}}\frac{2\zeta _2-1}{\sqrt{1+4\alpha _2}}\nonumber \\&+A_{\parallel }2\sqrt{2}\sqrt{\frac{\zeta _1(1-\zeta _1)+\alpha _1}{1+4\alpha _1}} \sqrt{\frac{\zeta _2(1-\zeta _2)+\alpha _2}{1+4\alpha _2}}\cos \phi \nonumber \\&+i A_{\perp }2\sqrt{2}\sqrt{\frac{\zeta _1(1-\zeta _1)+\alpha _1}{1+4\alpha _1}} \sqrt{\frac{\zeta _2(1-\zeta _2)+\alpha _2}{1+4\alpha _2}}\sin \phi \nonumber \\&+A_{VS}\frac{2\zeta _1-1}{\sqrt{1+4\alpha _1}}+A_{SV}\frac{2\zeta _2-1}{\sqrt{1+4\alpha _2}}+A_{SS}. \end{aligned}$$
(13)

On basis of Eq. (11), we can obtain the branching ratio for each helicity state,

$$\begin{aligned} {\mathcal {B}}_h= & {} \frac{\tau _B}{4(2\pi )^6m_B^2}\frac{2\pi }{9}C_h\nonumber \\&\times \int d\omega _1d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)|A_h|^2, \end{aligned}$$
(14)

where the invariant masses \(\omega _{1,2}\) are integrated over the chosen \(\pi \pi \) and \(K\pi \) mass window, respectively. The coefficients \(C_h\) are the results of the integrations over \(\zeta _1,\zeta _2,\phi \) in terms of Eq. (14) and listed as follows,

$$\begin{aligned} C_h=\left\{ \begin{aligned}&(1+4\alpha _1)(1+4\alpha _2)&\quad h=0,\parallel ,\perp \\&3(1+4\alpha _{1,2})&\quad h=VS,SV \\&9&\quad h=SS. \\ \end{aligned}\right. \end{aligned}$$
(15)

The CP-averaged branching ratio and the direct CP asymmetry in each component are defined as below,

$$\begin{aligned} {\mathcal {B}}_h^{avg}=\frac{1}{2}({\mathcal {B}}_h+\mathcal {{\bar{B}}}_h), \quad {\mathcal {A}}^{\text {dir}}_h=\frac{\mathcal {{\bar{B}}}_h-{\mathcal {B}}_h}{\mathcal {{\bar{B}}}_h+{\mathcal {B}}_h}, \end{aligned}$$
(16)

respectively, where \(\mathcal {{\bar{B}}}_h\) is the branching ratio of the corresponding CP-conjugate channel.

For the VV decays, the polarization fractions \(f_{\lambda }\) with \(\lambda =0\), \(\parallel \), and \(\perp \) are described as

$$\begin{aligned} f_{\lambda }=\frac{{\mathcal {B}}_{\lambda }}{{\mathcal {B}}_0+{\mathcal {B}}_{\parallel }+{\mathcal {B}}_{\perp }}, \end{aligned}$$
(17)

with the normalisation relation \(f_0+f_{\parallel }+f_{\perp }=1\).

2.3 Triple product asymmetries

In this work, we not only calculate the direct CP asymmetries, but also pay more attention to the TPAs. Consider a four-body decay \(B \rightarrow R_1 (\rightarrow P_1P_2) R_2(\rightarrow Q_1Q_2)\), in which one measures the four particles’ momenta in the B rest frame. We define \({\hat{n}}_{R_i} (i=1,2)\) is a unit vector perpendicular to the \(R_i\) decay plane and \({\hat{z}}\) is a unit vector in the direction of \(R_1\) in the B rest frame. Thus we have

$$\begin{aligned} {\hat{n}}_{R_1}\cdot {\hat{n}}_{R_2}=\cos \phi , \quad {\hat{n}}_{R_1}\times {\hat{n}}_{R_2}=\sin \phi {\hat{z}}, \end{aligned}$$
(18)

implying a T-odd scalar triple product

$$\begin{aligned} ({\hat{n}}_{R_1}\times {\hat{n}}_{R_2})\cdot {\hat{z}}= & {} \sin \phi ,\nonumber \\ 2({\hat{n}}_{R_1}\cdot {\hat{n}}_{R_2})({\hat{n}}_{R_1}\times {\hat{n}}_{R_2})\cdot {\hat{z}}= & {} \sin 2\phi . \end{aligned}$$
(19)

One can define a TPA as an asymmetry between the number of decays involving positive and negative values of \(\sin \phi \) or \(\sin 2\phi \),

$$\begin{aligned} {\mathcal {A}}_T^1= & {} \frac{\Gamma (\cos \theta _1\cos \theta _2\sin \phi>0)-\Gamma (\cos \theta _1\cos \theta _2\sin \phi<0)}{\Gamma (\cos \theta _1\cos \theta _2\sin \phi >0)+\Gamma (\cos \theta _1\cos \theta _2\sin \phi <0)}, \end{aligned}$$
(20)
$$\begin{aligned} {\mathcal {A}}_T^2= & {} \frac{\Gamma (\sin 2\phi>0)-\Gamma (\sin 2\phi<0)}{\Gamma (\sin 2\phi >0)+\Gamma (\sin 2\phi <0)}. \end{aligned}$$
(21)

According to Eq. (6), the TPAs associated with \(A_{\perp }\) for the considered four-body decays are derived from the partially integrated differential decay rates as [40, 50]

$$\begin{aligned} {\mathcal {A}}_T^1= & {} \frac{\Gamma ((2\zeta _1-1)(2\zeta _2-1)\sin \phi>0)-\Gamma ((2\zeta _1-1)(2\zeta _2-1)\sin \phi<0)}{\Gamma ((2\zeta _1-1)(2\zeta _2-1)\sin \phi >0)+\Gamma ((2\zeta _1-1)(2\zeta _2-1)\sin \phi <0)}\nonumber \\= & {} -\frac{2\sqrt{2}}{\pi {\mathcal {D}}}\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2) \text {Im}[A_{\perp }A_0^*], \end{aligned}$$
(22)
$$\begin{aligned} {\mathcal {A}}_T^2= & {} \frac{\Gamma (\sin 2\phi>0)-\Gamma (\sin 2\phi<0)}{\Gamma (\sin 2\phi >0)+\Gamma (\sin 2\phi <0)}\nonumber \\= & {} -\frac{4}{\pi {\mathcal {D}}}\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2) \text {Im}[A_{\perp }A_{\parallel }^*],\nonumber \\ \end{aligned}$$
(23)

with the denominator

$$\begin{aligned} {\mathcal {D}}= & {} \int d\omega _1 d\omega _2 k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\times (|A_0|^2+|A_{\parallel }|^2+|A_{\perp }|^2). \end{aligned}$$
(24)

It has been found that TPAs originate from the interference of the CP-odd amplitudes \(A_{\perp }\) with the other CP-even amplitudes \(A_0\), \(A_{\parallel }\). The above TPAs contain the integrands \(\text {Im}[A_{\perp }A_i^*]=|A_{\perp }||A_i^*|\sin (\Delta \phi +\Delta \delta )\) with \(i={0,\parallel }\), where \(\Delta \phi \) and \(\Delta \delta \) denote the weak and strong phase differences between the amplitudes \(A_{\perp }\) and \(A_i\), respectively. As already noted, \(\text {Im}[A_{\perp }A_i^*]\) can be nonzero even if the weak phases vanish. Thus, it is not quite accurate to identify a nonzero TPA as a signal of CP violation. To obtain a true CP violation signal, one has to compare the TPAs in the B and \({\bar{B}}\) meson decays. The helicity amplitude for the CP-conjugated process can be inferred from Eq. (13) through \(A_0\rightarrow {\bar{A}}_0\), \(A_{\parallel } \rightarrow {\bar{A}}_{\parallel }\) and \(A_{\perp } \rightarrow -{\bar{A}}_{\perp }\), in which the \({\bar{A}}_{\lambda }\) are obtained from the \(A_{\lambda }\) by changing the sign of the weak phases. Thus, the TPAs \(\bar{{\mathcal {A}}}_T^i\) for the charge-conjugate process are defined similarly, but with a multiplicative minus sign. We denote by \({\bar{A}}_0,{\bar{A}}_{\parallel },{\bar{A}}_{\perp }\) transversity amplitudes for the CP-conjugate decay \({\bar{B}}\rightarrow {\bar{R}}_1(\rightarrow {\bar{P}}_1{\bar{P}}_2){\bar{R}}_2(\rightarrow {\bar{Q}}_1{\bar{Q}}_2)\) and the corresponding three angles will be denoted by \({\bar{\theta }}_1\), \({\bar{\theta }}_2\), and \({\bar{\phi }}\). Obviously \(\theta _1={\bar{\theta }}_1\), \(\theta _2={\bar{\theta }}_2\) and \(\phi ={\bar{\phi }}\). We then have the TPAs for the CP-averaged decay rates [50]

$$\begin{aligned}&{\mathcal {A}}_T^{(1)\text {ave}}(\text {true})\nonumber \\&\quad \equiv \frac{[\Gamma (T>0)+{\bar{\Gamma }}({\bar{T}}>0)]-[\Gamma (T<0)+{\bar{\Gamma }}({\bar{T}}<0)]}{[\Gamma (T>0)+{\bar{\Gamma }}({\bar{T}}>0)]+[\Gamma (T<0)+{\bar{\Gamma }}({\bar{T}}<0)]}\nonumber \\&\quad =-\frac{2\sqrt{2}}{\pi ({\mathcal {D}}+\bar{{\mathcal {D}}})}\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\qquad \text {Im}[A_{\perp }A_0^*-{\bar{A}}_{\perp }{\bar{A}}_0^*], \end{aligned}$$
(25)
$$\begin{aligned}&{\mathcal {A}}_T^{(2)\text {ave}}(\text {true})\nonumber \\&\quad \equiv \frac{[\Gamma (\sin 2\phi>0)+{\bar{\Gamma }}(\sin 2{\bar{\phi }}>0)]-[\Gamma (\sin 2\phi<0)+{\bar{\Gamma }}(\sin 2{\bar{\phi }}<0)]}{[\Gamma (\sin 2\phi>0)+{\bar{\Gamma }}(\sin 2{\bar{\phi }}>0)]+[\Gamma (\sin 2\phi<0)+{\bar{\Gamma }}(\sin 2{\bar{\phi }}<0)]}\nonumber \\&\quad =-\frac{4}{\pi ({\mathcal {D}}+\bar{{\mathcal {D}}})}\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\qquad \times \text {Im}[A_{\perp }A_{\parallel }^*-{\bar{A}}_{\perp }{\bar{A}}_{\parallel }^*], \end{aligned}$$
(26)
$$\begin{aligned}&{\mathcal {A}}_T^{(1)\text {ave}}(\text {fake})\nonumber \\&\quad \equiv \frac{[\Gamma (T>0)-{\bar{\Gamma }}({\bar{T}}>0)]-[\Gamma (T<0)-{\bar{\Gamma }}({\bar{T}}<0)]}{[\Gamma (T>0)+{\bar{\Gamma }}({\bar{T}}>0)]+[\Gamma (T<0)+{\bar{\Gamma }}({\bar{T}}<0)]}\nonumber \\&\quad =-\frac{2\sqrt{2}}{\pi ({\mathcal {D}}+\bar{{\mathcal {D}}})}\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\qquad \times \text {Im}[A_{\perp }A_0^*+{\bar{A}}_{\perp }{\bar{A}}_0^*], \end{aligned}$$
(27)
$$\begin{aligned}&{\mathcal {A}}_T^{(2)\text {ave}}(\text {fake}) \equiv \frac{[\Gamma (\sin 2\phi>0)-{\bar{\Gamma }}(\sin 2{\bar{\phi }}>0)]-[\Gamma (\sin 2\phi<0)-{\bar{\Gamma }}(\sin 2{\bar{\phi }}<0)]}{[\Gamma (\sin 2\phi>0)+{\bar{\Gamma }}(\sin 2{\bar{\phi }}>0)]+[\Gamma (\sin 2\phi<0)+{\bar{\Gamma }}(\sin 2{\bar{\phi }}<0)]}\nonumber \\&\quad =-\frac{4}{\pi ({\mathcal {D}}+\bar{{\mathcal {D}}})}\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\qquad \times \text {Im}[A_{\perp }A_{\parallel }^*+{\bar{A}}_{\perp }{\bar{A}}_{\parallel }^*], \end{aligned}$$
(28)

with \({\bar{\Gamma }}\) being the decay rate of the CP-conjugate process, \(T=(2\zeta _1-1)(2\zeta _2-1)\sin \phi \) and \({\bar{T}}=(2\zeta _1-1)(2\zeta _2-1)\sin {\bar{\phi }}\) and the denominator is

$$\begin{aligned} \bar{{\mathcal {D}}}= & {} \int d\omega _1 d\omega _2 k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\times (|{\bar{A}}_0|^2+|{\bar{A}}_{\parallel }|^2+|{\bar{A}}_{\perp }|^2), \end{aligned}$$
(29)

for the CP-conjugate decay.

It is shown that the term \(\text {Im}[A_{\perp }A_{0,\parallel }^*-{\bar{A}}_{\perp }{\bar{A}}_{0,\parallel }^*]\) is proportional to \(\sin \Delta \phi \cos \Delta \delta \), which is nonzero only in the presence of the weak phase difference. Then TPAs provide an alternative measure of CP violation. Furthermore, compared with direct CP asymmetries, \({\mathcal {A}}_T^{(i)\text {ave}}(\text {true})\) does not suffer the suppression from the strong phase difference, and is maximal when the strong phase difference vanishes. While for the term \(\text {Im}[A_{\perp }A_{0,\parallel }^*+{\bar{A}}_{\perp }{\bar{A}}_{0,\parallel }^*]\propto \cos \Delta \phi \sin \Delta \delta \), the \({\mathcal {A}}_T^{(i)\text {ave}}(\text {fake})\) can be nonzero when the weak phase difference vanishes. Such a quantity is referred as a fake asymmetry (CP conserving), which reflects the effect of strong phases [50, 51], instead of CP violation.

For a more direct comparison with the measurements from LHCb [43], the so-called true and fake TPAs are then defined as

$$\begin{aligned} {\mathcal {A}}_T^1(\text {true})\equiv & {} \frac{1}{2}({\mathcal {A}}_T^1+\bar{{\mathcal {A}}}_T^1)\nonumber \\= & {} -\frac{\sqrt{2}}{\pi }\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\times \text {Im}\left[ \frac{A_{\perp }A_0^*}{{\mathcal {D}}}-\frac{{\bar{A}}_{\perp }{\bar{A}}_0^*}{\bar{{\mathcal {D}}}}\right] , \end{aligned}$$
(30)
$$\begin{aligned} {\mathcal {A}}_T^2(\text {true})\equiv & {} \frac{1}{2}({\mathcal {A}}_T^2+\bar{{\mathcal {A}}}_T^2)\nonumber \\= & {} -\frac{2}{\pi }\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\times \text {Im}\left[ \frac{A_{\perp }A_{\parallel }^*}{{\mathcal {D}}}-\frac{{\bar{A}}_{\perp }{\bar{A}}_{\parallel }^*}{\bar{{\mathcal {D}}}}\right] , \end{aligned}$$
(31)
$$\begin{aligned} {\mathcal {A}}_T^1(\text {fake})\equiv & {} \frac{1}{2}({\mathcal {A}}_T^1-\bar{{\mathcal {A}}}_T^1)\nonumber \\= & {} -\frac{\sqrt{2}}{\pi }\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\times \text {Im}\left[ \frac{A_{\perp }A_0^*}{{\mathcal {D}}}+\frac{{\bar{A}}_{\perp }{\bar{A}}_0^*}{\bar{{\mathcal {D}}}}\right] , \end{aligned}$$
(32)
$$\begin{aligned} {\mathcal {A}}_T^2(\text {fake})\equiv & {} \frac{1}{2}({\mathcal {A}}_T^2-\bar{{\mathcal {A}}}_T^2)\nonumber \\= & {} -\frac{2}{\pi }\int d\omega _1 d\omega _2k(\omega _1)k(\omega _2)k(\omega _1,\omega _2)\nonumber \\&\times \text {Im} \left[ \frac{A_{\perp }A_{\parallel }^*}{{\mathcal {D}}}+\frac{{\bar{A}}_{\perp }{\bar{A}}_{\parallel }^*}{\bar{{\mathcal {D}}}}\right] , \end{aligned}$$
(33)

where the true or fake labels refer to whether the asymmetry is due to a real CP asymmetry or due to effects from final-state interactions that are CP symmetric. It should be stressed that two asymmetries defined in Eqs. (25) and (30) are different in the most case, as well as two asymmetries in Eqs. (26) and (31). They become equal when no direct CP violation occurs in the total rate, namely \({\mathcal {D}}=\bar{{\mathcal {D}}}\).

2.4 Two-meson distribution amplitudes

2.4.1 S-wave two-meson DAs

Here we briefly introduce the S and P-wave two-meson DAs and the corresponding time-like form factors used in our framework. One can see that resonant contributions through two-body channels can be included by parameterizing the two-meson DAs. The S-wave two-meson DA is written in the following form [117],

(34)

In what follows the subscripts S and P are always associated with the corresponding partial waves.

For the scalar resonances \(f_0(500)\) and \(f_0(980)\), the asymptotic forms of the individual DAs in Eq. (34) have been parameterized as [105,106,107,108]

$$\begin{aligned} \phi ^0_S(z,\omega ^2)= & {} \frac{9F_S(\omega ^2)}{\sqrt{2N_c}}a_S z(1-z)(1-2z), \end{aligned}$$
(35)
$$\begin{aligned} \phi ^s_S(z,\omega ^2)= & {} \frac{F_S(\omega ^2)}{2\sqrt{2N_c}}, \end{aligned}$$
(36)
$$\begin{aligned} \phi ^t_S(z,\omega ^2)= & {} \frac{F_S(\omega ^2)}{2\sqrt{2N_c}}(1-2z), \end{aligned}$$
(37)

with the time-like scalar form factor \(F_S(\omega ^2)\) and the Gegenbauer coefficient \(a_S\). While for the scalar resonance \(K_0^*(1430)\), we will adopt similar formulas as those for a scalar meson [118, 119], bearing in mind large uncertainties that may be introduced by this approximation. The detailed expressions of DAs for various twists are as follows:

$$\begin{aligned} \phi ^0_S= & {} \frac{6}{2\sqrt{2N_c}}F_S(\omega ^2)z(1-z)\nonumber \\&\times \left[ \frac{1}{\mu _S}+B_1C_1^{3/2}(t)+B_3C_3^{3/2}(t) \right] , \end{aligned}$$
(38)
$$\begin{aligned} \phi ^s_S= & {} \frac{1}{2\sqrt{2N_c}}F_S(\omega ^2), \end{aligned}$$
(39)
$$\begin{aligned} \phi ^t_S= & {} \frac{1}{2\sqrt{2N_c}}F_S(\omega ^2)(1-2z), \end{aligned}$$
(40)

where the Gegenbauer polynomials \(C_{1}^{3/2}(t)=3t\), \(C_{3}^{3/2}(t)=\frac{5}{2}t(7t^2-3)\) with \(t=1-2z\) and \(\mu _S=\omega /(m_{02}-m_{01})\). \(\omega \) is the \(K\pi \) invariant mass and \(m_{01,02}\) are the running current quark masses. The Gegenbauer moments \(B_{1,3}\) at the 1 GeV scale from Scenario I in the QCD sum rule analysis and the related running current quark masses can be found in Refs. [2, 119, 120].

2.4.2 P-wave two-meson DAs

The P-wave two-meson DAs related to both longitudinal and transverse polarizations are organized in analogy with those in Ref. [92]. The explicit expressions of the P-wave pion-pion (kaon-pion) DAs associated with longitudinal and transverse polarization are described as follows,

(41)
(42)

The two-pion DAs for various twists are expanded in terms of the Gegenbauer polynomials:

$$\begin{aligned} \phi _P^0(z,\omega ^2)= & {} \frac{3F_P^{\parallel }(\omega ^2)}{\sqrt{2N_c}}z(1-z)\nonumber \\&\times \left[ 1 +a^0_{2\rho }\frac{3}{2}(5(1-2z)^2-1)\right] , \end{aligned}$$
(43)
$$\begin{aligned} \phi _P^s(z,\omega ^2)= & {} \frac{3F_P^{\perp }(\omega ^2)}{2\sqrt{2N_c}}(1-2z)\nonumber \\&\times \left[ 1 +a^s_{2\rho }(10z^2-10z+1)\right] , \end{aligned}$$
(44)
$$\begin{aligned} \phi _P^t(z,\omega ^2)= & {} \frac{3F_P^{\perp }(\omega ^2)}{2\sqrt{2N_c}}(1-2z)^2\nonumber \\&\times \left[ 1 +a^t_{2\rho }\frac{3}{2}(5(1-2z)^2-1)\right] , \end{aligned}$$
(45)
$$\begin{aligned} \phi _P^T(z,\omega ^2)= & {} \frac{3F_P^{\perp }(\omega ^2)}{\sqrt{2N_c}}z(1-z)\nonumber \\&\times \left[ 1+a^{T}_{2\rho }\frac{3}{2}(5(1-2z)^2-1)\right] , \end{aligned}$$
(46)
$$\begin{aligned} \phi _P^a(z,\omega ^2)= & {} \frac{3F_P^{\parallel }(\omega ^2)}{4\sqrt{2N_c}}(1-2z)\nonumber \\&\times \left[ 1+a_{2\rho }^a(10z^2-10z+1)\right] , \end{aligned}$$
(47)
$$\begin{aligned} \phi _P^v(z,\omega ^2)= & {} \frac{3F_P^{\parallel }(\omega ^2)}{8\sqrt{2N_c}}\nonumber \\&\times \bigg \{\left[ 1+(1-2z)^2\right] +a^v_{2\rho }\left[ 3(2z-1)^2-1\right] \bigg \},\nonumber \\ \end{aligned}$$
(48)

with the Gegenbauer coefficients \(a_{2\rho }^i\) and the two P-wave form factors \(F_P^{\parallel }(\omega ^2)\) and \(F_P^{\perp }(\omega ^2)\).

For the kaon-pion DAs, the various twist DAs \(\phi _P^i\) have similar forms as the corresponding ones for the \(K^*\) meson [121] by replacing the decay constants with the time-like form factors,

$$\begin{aligned} \phi _P^0(z,\omega ^2)= & {} \frac{3F_P^{\parallel }(\omega ^2)}{\sqrt{2N_c}} z(1-z)\nonumber \\&\times \left[ 1+a_{1K^*}^{||}3t+a_{2K^*}^{||}\frac{3}{2}(5t^2-1)\right] , \end{aligned}$$
(49)
$$\begin{aligned} \phi _P^s(z,\omega ^2)= & {} \frac{3F_P^{\perp }(\omega ^2)}{2\sqrt{2N_c}}t, \end{aligned}$$
(50)
$$\begin{aligned} \phi _P^t(z,\omega ^2)= & {} \frac{3F_P^{\perp }(\omega ^2)}{2\sqrt{2N_c}} t^2, \end{aligned}$$
(51)
$$\begin{aligned} \phi _P^T(z,\omega ^2)= & {} \frac{3F_P^{\perp }(\omega ^2)}{\sqrt{2N_c}}z(1-z)\nonumber \\&\times \left[ 1+a_{1K^*}^{\perp }3 t+ a_{2K^*}^{\perp }\frac{3}{2}(5 t^2-1)\right] , \end{aligned}$$
(52)
$$\begin{aligned} \phi _P^a(z,\omega ^2)= & {} \frac{3F_P^{\parallel }(\omega ^2)}{4\sqrt{2N_c}}t, \end{aligned}$$
(53)
$$\begin{aligned} \phi _P^v(z,\omega ^2)= & {} \frac{3F_P^{\parallel }(\omega ^2)}{8\sqrt{2N_c}}(1+ t^2), \end{aligned}$$
(54)

with \(t=(1-2z)\). The Gegenbauer moments associated with longitudinal polarization \(a_{1K^*}^{\parallel },a_{2K^*}^{\parallel }\) are determined in Ref. [102], while the Gegenbauer moments associated with transverse polarization \(a_{1K^*}^{\perp },a_{2K^*}^{\perp }\) are adopted the same values as those longitudinal ones.

2.4.3 Time-like form factor

The strong interactions between the resonance and the final-state meson pair, including elastic rescattering of the final-state meson pair, can be factorized into the time-like form factor \(F_{S,P}(\omega ^2)\), which is guaranteed by the Watson theorem [122]. We usually use the relativistic Breit-Wigner (BW) line shape for a narrow resonance to parameterize the time-like form factors \(F^{\parallel }(\omega ^2)\) [123]. The explicit formula is expressed as [124],

$$\begin{aligned} F^{\parallel }(\omega ^2)= & {} \frac{ m_i^2}{m^2_i -\omega ^2-im_i\Gamma _i(\omega ^2)}, \end{aligned}$$
(55)

where the \(m_i\) and \(\Gamma _i\) are the pole mass and width of the corresponding resonances shown in Table 3, respectively. The mass-dependent width \(\Gamma _i(\omega )\) is defined as

$$\begin{aligned} \Gamma _i(\omega ^2)= & {} \Gamma _i\left( \frac{m_i}{\omega }\right) \left( \frac{k(\omega )}{k(m_i)}\right) ^{(2L_R+1)}{.} \end{aligned}$$
(56)

The \(k(\omega )\) is the momentum vector of the resonance decay product measured in the resonance rest frame, while \(k(m_i)\) is the value of \(k(\omega )\) when \(\omega =m_i\). \(L_R\) is the orbital angular momentum in the \(\pi \pi \) (\(K\pi \)) system and \(L_R=0,1,\ldots \) corresponds to the \(S,P,\ldots \) partial-wave resonances. Due to the limited studies on the form factor \(F^{\perp }(\omega ^2)\), we use the two decay constants \(f_i^{(T)}\) of the intermediate particle to determine the ratio \(F^{\perp }(\omega ^2)/F^{\parallel }(\omega ^2)\approx (f_i^T/f_i)\).

The BW formula does not work well for \(f_0(980)\), since its pole mass is close to the \(K{\bar{K}}\) threshold. For scalar resonance \(f_0(980)\), we adopt the Flatté parametrization where the resulting line shape is above and below the threshold of the intermediate particle [128]. If the coupling of a resonance to the channel opening nearby is very strong, the Flatté parametrization shows a scaling invariance and does not allow for an extraction of individual partial decay widths. Thus, we employ the modified Flatté model suggested by Bugg [129] following the LHCb collaboration [125, 130],

$$\begin{aligned} F(\omega ^2)=\frac{m_{f_0(980)}^2}{m_{f_0(980)}^2-\omega ^2-im_{f_0(980)}(g_{\pi \pi }\rho _{\pi \pi }+g_{KK}\rho _{KK}F^2_{KK})}.\nonumber \\ \end{aligned}$$
(57)

The coupling constants \(g_{\pi \pi }=0.167\) GeV and \(g_{KK}=3.47g_{\pi \pi }\) [125, 130] describe the \(f_0(980)\) decay into the final states \(\pi ^+\pi ^-\) and \(K^+K^-\), respectively. The phase space factors \(\rho _{\pi \pi }\) and \(\rho _{KK}\) read as [125, 128, 131]

$$\begin{aligned} \rho _{\pi \pi }= & {} \frac{2}{3}\sqrt{1-\frac{4m^2_{\pi ^\pm }}{\omega ^2}} +\frac{1}{3}\sqrt{1-\frac{4m^2_{\pi ^0}}{\omega ^2}},\nonumber \\ \rho _{KK}= & {} \frac{1}{2}\sqrt{1-\frac{4m^2_{K^\pm }}{\omega ^2}} +\frac{1}{2}\sqrt{1-\frac{4m^2_{K^0}}{\omega ^2}}. \end{aligned}$$
(58)

If there are overlapping resonances or there is significant interference with a nonresonant component both in the same partial wave, the relativistic BW function leads to unitarity violation within the isobar model [132]. This is the case for the \(K\pi \) S-wave at low \(K\pi \) mass, where the \(K^*_0(1430)\) resonance interferes strongly with a slowly varying NR S-wave component. In this work, the time-like scalar form factor \(F_S(\omega ^2)\) for the S-wave \(K\pi \) system is parametrized by using a modified LASS line shape [133] for the S-wave resonance \(K^*_0(1430)\), which has been widely used in experimental analyses [124],

$$\begin{aligned} F_S(\omega ^2)= & {} \frac{\omega }{k(\omega )[\cot (\delta _B)-i]}\nonumber \\&+e^{2i\delta _B}\frac{m_0^2 \Gamma _0/k(m_0)}{m_0^2-\omega ^2-im_0^2 \frac{\Gamma _0}{\omega }\frac{k(\omega )}{k(m_0)}}, \end{aligned}$$
(59)

with

$$\begin{aligned} \cot (\delta _B)=\frac{1}{ak(\omega )}+\frac{rk(\omega )}{2}, \end{aligned}$$
(60)

where the first term in Eq. (59) is an empirical term from the elastic kaon-pion scattering and the second term is the resonant contribution with a phase factor to retain unitarity. Here \(m_0\) and \(\Gamma _0\) are the pole mass and width of the \(K^*_0(1430)\) state. The parameters \(a=(3.1\pm 1.0)~\mathrm{GeV^{-1}}\) and \(r=(7.0\pm 2.4)~\mathrm{GeV^{-1}}\) are the scattering length and effective range [124], respectively. The slowly varying part (the first term in the Eq. (59)) is not well modeled at high masses and it is set to zero for \(m(K\pi )\) values above 1.7 GeV [124].

In experimental investigations of three-body hadronic B meson decays, the wide \(\rho \) resonant contribution is usually parameterized as the Gounaris–Sakurai (GS) model [134] based on the BW function [123]. It is a way to interpret the observed structures beyond the \(\rho \) resonance in terms of higher mass isovector vector mesons. By taking the \(\rho -\omega \) interference and the excited states into account, the form factor \(F^{\parallel }(\omega ^2)\) can be written in the form of [127]

$$\begin{aligned} F^{\parallel }(\omega ^2)= & {} \left[ \mathrm{GS}_\rho (s,m_{\rho },\Gamma _{\rho }) \frac{1+c_{\omega } \mathrm{BW}_{\omega }(s,m_{\omega },\Gamma _{\omega })}{1+c_{\omega }}\right. \nonumber \\&\left. +\Sigma c_i \mathrm{GS}_i(s,m_i,\Gamma _i)\right] \left[ 1+\Sigma c_i\right] ^{-1}, \end{aligned}$$
(61)

where \(s=m^2(\pi \pi )\) is the two-pion invariant mass squared, \(i=(\rho ^{\prime }(1450), \rho ^{\prime \prime }(1700), \rho ^{\prime \prime \prime }(2254))\), \(\Gamma _{\rho ,\omega ,i}\) is the decay width for the relevant resonance, \(m_{\rho ,\omega ,i}\) are the masses of the corresponding mesons, respectively. The explicit expressions of the function \(\mathrm{GS}_\rho (s,m_{\rho },\Gamma _{\rho })\) are described as follows [123]

$$\begin{aligned}&\mathrm{GS}_\rho (s, m_\rho , \Gamma _\rho ) \nonumber \\&\quad = \frac{m_\rho ^2 [ 1 + d(m_\rho ) \Gamma _\rho /m_\rho ] }{m_\rho ^2 - s + f(s, m_\rho , \Gamma _\rho ) - i m_\rho \Gamma (s, m_\rho , \Gamma _\rho )}, \end{aligned}$$
(62)

with the functions

$$\begin{aligned} \Gamma (s, m_\rho , \Gamma _\rho )= & {} \Gamma _\rho \frac{s}{m_\rho ^2} \left( \frac{\beta _\pi (s) }{ \beta _\pi (m_\rho ^2) } \right) ^3~,\nonumber \\ d(m)= & {} \frac{3}{\pi } \frac{m_\pi ^2}{k^2(m^2)} \ln \left( \frac{m+2 k(m^2)}{2 m_\pi } \right) \nonumber \\&+ \frac{m}{2\pi k(m^2)} - \frac{m_\pi ^2 m}{\pi k^3(m^2)},\nonumber \\ f(s, m, \Gamma )= & {} \frac{\Gamma m^2}{k^3(m^2)} \left[ k^2(s) [ h(s)-h(m^2) ]\right. \nonumber \\&\left. + (m^2-s) k^2(m^2) h'(m^2)\right] ,\nonumber \\ k(s)= & {} \frac{1}{2} \sqrt{s} \beta _\pi (s),\nonumber \\ h(s)= & {} \frac{2}{\pi } \frac{k(s)}{\sqrt{s}} \ln \left( \frac{\sqrt{s}+2 k(s)}{2 m_\pi } \right) , \end{aligned}$$
(63)

where \(\beta _\pi (s) = \sqrt{1 - 4m_\pi ^2/s}\). Since the process \(\omega \rightarrow \pi ^+\pi ^-\) suffers the G-parity suppression, we find that the interference between the \(\rho \) and \(\omega \) does not significantly change the values in our calculations. Hence, it’s reasonable to set \(c_{\omega }=0\) in our latter numerical calculations.

In contrast to the vector resonances, the identification of the scalar mesons is a long-standing puzzle. Scalar resonances are difficult to resolve because some of them have large decay widths, which cause a strong overlap between resonances and background. For a comparison, we here parameterize the \(f_0(500)\) contribution in two different ways: the BW and the Bugg model [135], respectively. The form factor of \(f_0(500)\) with the Bugg resonant lineshape is written in the following form [135]

$$\begin{aligned} T_{11}(s)= & {} M \; \Gamma _1(s) \left[ M^2 - s - g^2_1 \frac{s-s_A}{M^2-s_A}\right. \nonumber \\&\times \left. \left[ j_1(s) - j_1(M^2) \right] - iM \sum _{i=1}^4 \Gamma _i(s) \right] ^{-1}, \end{aligned}$$
(64)

where \(s=\omega ^2 =m^2(\pi ^+\pi ^-)\), \(j_1(s) = \frac{1}{\pi }\left[ 2 + \rho _1 \ln \left( \frac{1 - \rho _1}{1 + \rho _1}\right) \right] \), the functions \(g^2_1(s)\), \(\Gamma _i(s)\) and other relevant functions in Eq. (64 ) are the following

$$\begin{aligned} g^2_1(s)= & {} M(b_1 + b_2s)\exp [-(s - M^2)/A],\nonumber \\ M\; \Gamma _1(s)= & {} g^2_1(s)\frac{s-s_A}{M^2-s_A}\rho _1(s), \nonumber \\ M\; \Gamma _2(s)= & {} 0.6g^2_1(s)(s/M^2)\exp (-\alpha |s-4m^2_K|)\rho _2(s),\nonumber \\ M\;\Gamma _3(s)= & {} 0.2g^2_1(s)(s/M^2)\exp (-\alpha |s-4m^2_\eta |)\rho _3(s), \nonumber \\ M\;\Gamma _4(s)= & {} M\; g_{4\pi }\; \rho _{4\pi }(s)/\rho _{4\pi }(M^2), \quad \mathrm {with}\nonumber \\ \rho _{4\pi }(s)= & {} 1.0/[1 + \exp (7.082 - 2.845s)]. \end{aligned}$$
(65)

For the parameters in Eqs. (64, 65), we use their values as given in the fourth column of Table I in Ref. [135]:

$$\begin{aligned} M= & {} 0.953 \mathrm{GeV}, \quad s_A = 0.41 \ m_{\pi }^2, \quad b_1 = 1.302 \mathrm{GeV}^2,\nonumber \\ b_2= & {} 0.340, \quad A = 2.426 \mathrm{GeV}^2, \quad g_{4\pi } = 0.011 \mathrm{GeV}. \end{aligned}$$
(66)

And the parameters \(\rho _{1,2,3}\) in Eq. (65) are the phase-space factors of the decay channels \(\pi \pi \), KK and \(\eta \eta \) respectively, and have been defined as [135]

$$\begin{aligned} \rho _i(s) = \sqrt{1 - 4\frac{m^2_i }{s} }, \end{aligned}$$
(67)

with \(m_1=m_\pi , m_2=m_K\) and \(m_3=m_\eta \).

3 Numerical results

In this section, we calculate the CP-averaged branching rations (\(\mathcal B\)), direct CP-violating asymmetries (\(\mathcal A^{\text {CP}}\)) and the polarization fractions \(f_{\lambda }\), as well as estimate the size of TPAs, respectively. The pole mass and decay width for the corresponding resonance have been summarized in Table 3. Before proceeding with the numerical analysis, the meson masses and the decay constants (in units of GeV), together with the B meson lifetimes (in units of ps) are collected below [13, 46, 89]:

$$\begin{aligned} m_{B}= & {} 5.280, \quad m_{B_s}=5.367, \nonumber \\ m_b= & {} 4.8, \quad m_{K^\pm }=0.494, \nonumber \\ m_{K^0}= & {} 0.498, \quad m_{\pi ^{\pm }}=0.140, \nonumber \\ m_{\pi ^0}= & {} 0.135, \quad f_B=0.21\pm 0.02, \nonumber \\ f_{B_s}= & {} 0.23\pm 0.02, \quad f_{\rho }=0.216 \pm 0.003, \nonumber \\ f^T_{\rho }= & {} 0.184,\quad f_{K^*}=0.217 \pm 0.005, \nonumber \\ f^T_{K^*}= & {} 0.185 \pm 0.010, \quad \tau _{B^0}=1.519,\nonumber \\ \tau _{B^{\pm }}= & {} 1.638, \quad \tau _{B_{s}}=1.512. \end{aligned}$$
(68)

The values of the Wolfenstein parameters are adopted as given in Ref. [136]: \(A=0.836\pm 0.015, \lambda =0.22453\pm 0.00044\), \({\bar{\rho }} = 0.122^{+0.018}_{-0.017}\), \({\bar{\eta }}= 0.355^{+0.012}_{-0.011}\).

The Gegenbauer moments are adopted the same values as those determined in Refs. [2, 92, 102, 137]

$$\begin{aligned} a_S= & {} 0.2\pm 0.2, \nonumber \\ B_1= & {} -0.57\pm 0.13, \nonumber \\ B_3= & {} -0.42\pm 0.22, \nonumber \\ a^0_{2\rho }= & {} 0.08\pm 0.13, \nonumber \\ a^s_{2\rho }= & {} -0.23\pm 0.24,\nonumber \\ a^t_{2\rho }= & {} -0.354\pm 0.062, \nonumber \\ a^{T}_{2\rho }= & {} 0.50\pm 0.50, \nonumber \\ a^a_{2\rho }= & {} 0.40\pm 0.40,\nonumber \\ a^v_{2\rho }= & {} -0.50\pm 0.50,\nonumber \\ a_{1K^*}^{||}= & {} a_{1K^*}^{\perp }=0.31 \pm 0.16, \nonumber \\ a_{2K^*}^{||}= & {} a_{2K^*}^{\perp }=1.188\pm 0.098. \end{aligned}$$
(69)

In our numerical calculations, the theoretical uncertainties quoted in the tables are estimated from three sources: the first theoretical uncertainty results from the parameters of the wave functions of the initial states, such as the shape parameter \(\omega _B=0.40\pm 0.04\) GeV or \(\omega _{B_s}=0.48\pm 0.048\) GeV in \(B_{(s)}\) meson wave function. The second one is due to the Gegenbauer moments in various twist DAs of \(\pi \pi \) and \(K\pi \) pair with different intermediate resonances. The last one is caused by the variation of the hard scale t from 0.75t to 1.25t (without changing \(1/b_i\)) and the QCD scale \(\Lambda _{\mathrm{QCD}}=0.25\pm 0.05\) GeV, which characterizes the effect of the next-to-leading-order QCD contributions. The possible errors due to the uncertainties of CKM matrix elements are very small and can be neglected safely. The major uncertainty comes from the Gegenbauer moments, which amounts to 30–50% in magnitude. These parameters need to be constrained more precisely in order to improve the accuracy of theoretical predictions for four-body B meson decays.

We perform an amplitude analysis of \(B_{(s)} \rightarrow (\pi \pi )(K\pi )\) decays in the two-body invariant mass regions 300\(<m(\pi \pi )<\)1100 MeV, accounting for the \(\rho \), \(f_0(500)\) and \(f_0(980)\) resonances, and 750\(<m(K\pi )<\)1200 MeV, which is dominated by the \(K^*(892)\) meson, but the S-wave resonance \(K_0^*(1430)\) is expected to contribute. From the numerical results, one can address some issues as follows.

Table 4 CP-averaged branching ratios and polarization fractions of the two-body \(B_{(s)} \rightarrow \rho K^*\) decays in the PQCD approach compared with the previous predictions in the PQCD approach [18], the updated predictions in the QCDF [3, 4], SCET [25] and FAT [26]. Experimental results for branching ratios are taken from Table 1 and for polarization fractions from [45]. The theoretical uncertainties are attributed to the variations of the shape parameter \(\omega _{B_{(s)}}\) in the \(B_{(s)}\) meson DA, of the Gegenbauer moments in various twist DAs of \(\pi \pi \) and \(K\pi \) pair, and of the hard scale t and the QCD scale \(\Lambda _{\mathrm{QCD}}\)

3.1 Branching ratios and polarization fractions of two-body \(B_{(s)} \rightarrow \rho K^*\) decays

The isospin conservation is assumed for the strong decays of an \(I=1/2\) resonance \(K^*\) to \(K\pi \), namely,

$$\begin{aligned} \frac{\Gamma (K^{*0} \rightarrow K^+\pi ^-)}{\Gamma (K^{*0} \rightarrow K\pi )}= & {} 2/3, \nonumber \\ \frac{\Gamma (K^{*+} \rightarrow K^+\pi ^0)}{\Gamma (K^{*+} \rightarrow K\pi )}= & {} 1/3, \end{aligned}$$
(70)

where we assume the \(K^*\rightarrow K\pi \) branching fraction to be 100%. According to the relation of the decay rates between the quasi-two-body and the corresponding two-body decay modes

$$\begin{aligned}&{\mathcal {B}}(B_{(s)} \rightarrow \rho (\rightarrow \pi \pi )K^*(\rightarrow K\pi ))\nonumber \\&\quad \approx {\mathcal {B}}(B_{(s)} \rightarrow \rho K^*)\times {\mathcal {B}}(\rho \rightarrow \pi \pi )\times {\mathcal {B}}(K^*\rightarrow K\pi ),\nonumber \\ \end{aligned}$$
(71)

with \({{\mathcal {B}}}(\rho \rightarrow \pi \pi )=100\%\), we extract the two-body \(B_{(s)} \rightarrow \rho K^*\) branching ratios and summarize them in Table 4. The polarization fractions of the two-body \(B_{(s)} \rightarrow \rho K^*\) decays calculated in this work are also listed in Table 4. For a comparison, we show the updated predictions in the QCDF [3, 4], the previous predictions in the PQCD approach [18], SCET [25] and FAT [26]. Experimental results for branching ratios are taken from Table 1 and for polarization fractions from [45]. One can see that, except for the colour-suppressed (“ C ”) decay \(B^0_s\rightarrow \rho ^0 {\bar{K}}^{*}(892)^{0}\), our predictions of the branching ratios are in good agreement with those two-body analyses as presented in the PQCD approach [18], and also similar to those predicted in the QCDF approach [3, 4], SCET [25] and FAT [26] within errors. However, the situation of the “ C ”-type decay \(B^0_s\rightarrow \rho ^0 {\bar{K}}^{*}(892)^{0}\) is more complicated: (a) as claimed in Ref. [88] that the widths of the resonant states and the interactions between the final state meson pairs will show their effects on the branching ratios, the new four-body prediction deviates from the previous calculations in the PQCD approach, but agrees well with the corresponding results in the QCDF approach, SCET and FAT within errors; (b) the transverse polarization contribution in the PQCD approach is comparable to the longitudinal one due to the chirally enhanced annihilation and the hard scattering diagrams, which is quite different from those predictions in the QCDF approach, SCET and FAT. More precise data from the future LHCb and Belle II experiments will help us differentiate these factorization approaches and understand the underlying mechanism of the multi-body B meson hadronic weak decays.

For the charmless \(B_{(s)} \rightarrow \rho (\rightarrow \pi \pi )K^*(\rightarrow K\pi )\) decays, it is naively expected that the helicity amplitudes satisfy the hierarchy pattern \(|A_0|\gg |A_+|\gg |A_-|\), which are related to the spin amplitudes \((A_{0}, A_{\parallel }, A_{\perp })\) in Appendix by

$$\begin{aligned} A_{\pm }=\frac{A_{\parallel }\pm A_{\perp }}{\sqrt{2}}, \end{aligned}$$
(72)

while \(A_{0}\) is common to both bases. The above hierarchy relation satisfies the expectation in the factorization assumption that the longitudinal polarization should dominate based on the quark helicity analysis [138, 139]. However, large transverse polarization of order 50% is observed in \(B\rightarrow K^*\phi \), \(B\rightarrow K^*\rho \) and \(B_s\rightarrow \phi \phi \) decays, which poses an interesting challenge for the theory. The interest in the polarization in penguin transition, such as \(b\rightarrow s\) decays \(B \rightarrow \rho K^*\), is motivated by their potential sensitivity to physics beyond the SM. Measurements of the longitudinal polarization fraction in \(B \rightarrow \rho K^*\) by BABAR [27, 31, 32] and Belle [33] reveal a large fraction of transverse polarization, indicating an anomaly of polarization. An angular analysis of the \(B^0 \rightarrow \rho ^0 K^{*0}\) decay by LHCb measurement found an unexpectedly low longitudinal polarization fraction \(f_0=0.164\pm 0.015\pm 0.022\) where the first uncertainty is statistical and the second systematic [43].

As shown in Table 4, the longitudinal polarization fraction \(f_0\) for the \(B_{(s)} \rightarrow \rho K^*\) decays from the PQCD approach (including the present work) are around 50–80%, which are mostly greater than the transverse one \(f_T=f_{\parallel }+f_{\perp }\) in contrast to observations. The QCDF [3, 4], SCET [25] and FAT [26] yield the similar pattern \(f_0\sim f_T\) in despite of large uncertainties. In the PQCD approach, the large transverse polarization fraction can be interpreted on the basis of the chirally enhanced annihilation diagrams, especially the \((S-P)(S+P)\) penguin annihilation, introduced by the QCD penguin operator \(O_6\) [140], which is originally introduced in Ref. [141]. A special feature of the \((S-P)(S+P)\) penguin annihilation operator is that the light quarks in the final states are not produced through chiral currents. So, there is no suppression to the transverse polarization caused by the helicity flip. Then the polarization fractions satisfy \(f_0\approx f_T\). However, these effects are not able to fully account for the above polarization anomaly. Our predictions for the longitudinal polarization fractions agree with the previous PQCD calculations [18]. It is worth mentioning that we have employed the same Gegenbauer moments for the transversely polarized \(K\pi \) DAs as those for the longitudinal polarized ones (see Eq. (69)) in this work, together with the same Gegenbauer moments for \(\pi \pi \) associated with the transverse polarizations from previous work [92]. The Gegenbauer moments from the twist-3 DAs of \(K\pi \) pair may make significant sense to the polarization fractions, which has been verified in Ref. [7]. To be honest, these Gegenbuaer moments should be fitted similarly as those in Ref. [102]. With more and more experimental measurements, we can determine the precise values of these Gegenbauer moments for transversely polarized DAs.

Table 5 CP-averaged branching ratios (in units of \(10^{-6}\)) for the four-body \(B \rightarrow [SS,SV,VS] \rightarrow (\pi \pi ) (K\pi )\) decays, with \(S=[f_0(500),f_0(980),K^*_0(1430)]\) and \(V=[\rho , K^*]\). The experimental data are taken from [46]. The sources of the theoretical errors are the same as in Table 4

3.2 Branching ratios of \(B_{(s)} \rightarrow [SS,SV,VS]\rightarrow (\pi \pi )(K\pi )\) decays

In contrast to the vector resonances, the identification of the scalar mesons is a long-standing puzzle. Scalar resonances are difficult to resolve because some of them have large decay widths, which cause a strong overlap between resonances and background. In fact, compared with the \(B_{(s)} \rightarrow VV \rightarrow (\pi \pi )(K\pi )\) decays, there are much less experimental data for the \(B_{(s)} \rightarrow [SS,SV,VS]\rightarrow (\pi \pi )(K\pi )\) decays. Furthermore, the underlying structure of scalar mesons is not theoretically well established (for a review, see Ref. [46]). We hope that the situation can be improved using nonperturbative QCD tools including lattice QCD simulations. The \(f_0(980)\) is strongly produced in \(D^+_s\) decay [142], which implies a large \({\bar{s}}s\) component, assuming Cabibbo-favored \(c \rightarrow s\) decay. Meanwhile, the prominent appearance of the \(f_0(980)\) points to a dominant (\({\bar{s}}s\)) component in the semileptonic \(D_s\) decays and decays of \(B_{(s)}\) mesons. However, there also exists some experimental evidences indicating that \(f_0(980)\) is not purely an \({\bar{s}}s\) state. Ratios of decay rates of B and/or \(B_s\) mesons into \(J/\psi \) plus \(f_0(980)\) or \(f_0(500)\) were proposed to allow for an extraction of the flavor mixing angle and to probe the tetraquark nature of those mesons within a certain model [143, 144]. The phenomenological fits of the LHCb do neither support a contribution of the \(f_0(980)\) in the \(B\rightarrow J/\psi \pi \pi \) [130] nor an \(f_0(500)\) in \(B_s \rightarrow J/\psi \pi \pi \) decays [125] by employing the isobar model. Hence the authors conclude that their data is incompatible with a model where \(f_0(980)\) is formed from two quarks and two antiquarks (tetraquarks) at the eight standard deviation level. In addition, they extract an upper limit for the mixing angle of \(17^\circ \) at \(90\%\) confidence level between the \(f_0(980)\) and the \(f_0(500)\) that would yield a substantial (\({\bar{s}}s\)) content in \(f_0(980)\) [130]. But in fact a substantial \(f_0(980)\) contribution is also found in the B-decays in a dispersive analysis of the same data that allows for a model-independent inclusion of the hadronic final state interactions in Ref. [145], which puts into question the conclusions of Ref. [130]. At this stage, the quark structure of scalar particles are still quite controversial. On the theory side, there are some studies on the \(f_0(980)\) by assuming the \(f_0(980)\) as a pure \({\bar{s}}s\) state. For example, the authors studied the \(B_s \rightarrow J/\psi f_0(980)\) with the light-cone QCD sum rule and factorization assumption in Ref. [146] and using generalized factorization and SU(3) flavor symmetry in Ref. [147]. In Ref. [148], the authors calculated the \({\bar{B}}_s \rightarrow f_0(980)\) form factor from the light-cone sum rules with B-meson DAs, and investigated the S-wave \({\bar{B}}_s \rightarrow KK\) form factors to study the width effect, where the \(f_0(980)\) is dominated by the \({\bar{s}}s\) configuration. As a first approximation, we take into account the scalar meson \(f_0(500),f_0(980),K_0^*(1430)\) in the \({\bar{q}}{q}\) density operator with \(q=(u,d,s)\). The S-wave time-like form factor \(F_S(\omega ^2)\) used to parameterize the S-wave two-pion and kaon-pion DAs have been determined in Refs. [117, 137].

We list the branching ratios of the four-body decays \(B_{(s)} \rightarrow [SS,SV,VS]\rightarrow (\pi \pi )(K\pi )\) with experimental data [46] in Table 5. So far, only five of them, say \(B^{+} \rightarrow (f_0(980) \rightarrow )\pi ^+\pi ^-(K^{*+}\rightarrow )K^{0}\pi ^{+}\), \(B^{0} \rightarrow (f_0(980) \rightarrow )\pi ^+\pi ^-(K^{*0}\rightarrow )K^{+}\pi ^{-}\), \(B^{0} \rightarrow (\rho ^{-} \rightarrow )\pi ^{-}\pi ^{0}(K_0^{*}(1430)^{+}\rightarrow )K^{0}\pi ^{+}\), \(B^{0} \rightarrow (\rho ^{0} \rightarrow )\pi ^{+}\pi ^{-}\)\((K_0^{*}(1430)^{0}\rightarrow )K^{+}\pi ^{-}\), and \(B^0 \rightarrow (f_0(980) \rightarrow )\pi ^+\pi ^-\)\((K_0^{*}(1430)^{0}\rightarrow )K^+\pi ^-\), have been reported by experiments. It is shown that, except for the colour suppressed (“ C ”) decay \(B^{0} \rightarrow (\rho ^{0} \rightarrow )\pi ^{+}\pi ^{-}(K_0^{*}(1430)^{0}\rightarrow )K^{+}\pi ^{-}\), our predictions of other four channels are consistent with the available experimental data within errors, with the remaining predictions awaiting for the examinations from future experimental measurements. However, the branching ratio \(\mathcal{B}(B^{0} \rightarrow (\rho ^{0} \rightarrow )\pi ^{+}\pi ^{-}(K_0^{*}(1430)^{0}\rightarrow )K^{+}\pi ^{-})=(7.9^{+4.9}_{-3.6})\times 10^{-6}\) estimated in this work is smaller than the experimental data \(\mathcal{B}=(18\pm 4)\times 10^{-6}\) [46] by a factor of \(\sim 2\). Since only leading order contributions are considered in this work, it indicates that this decay mode might be more sensitive to next-to-leading order corrections, and it is similar to the situation of other “ C ”-type decays, such as \(B^0\rightarrow \pi ^0\pi ^0,\rho ^0\rho ^0\). Besides, under the isospin limit, it is naively expected that

$$\begin{aligned} R=\frac{\mathcal{B}(B^0 \rightarrow (\rho ^0 \rightarrow )\pi ^+\pi ^-(K_0^{*}(1430)^{0}\rightarrow )K^+\pi ^-)}{\mathcal{B}(B^0 \rightarrow (\rho ^- \rightarrow )\pi ^-\pi ^0(K_0^{*}(1430)^{+}\rightarrow )K^0\pi ^+)}=\frac{1}{2},\nonumber \\ \end{aligned}$$
(73)

which is not borne out by experiment and needs to be further studied in the future. Among the three different kinds of theoretical errors considered in our work, one can see that the most important theoretical uncertainties for the branching ratio are caused by the nonperturbative input parameters of the wave functions for some decay modes. Taking the decay \(B^+ \rightarrow (f_0(500) \rightarrow )\pi ^+\pi ^-(K^{*+}\rightarrow )K^0\pi ^+\) as an example, which is dominated by the \(B \rightarrow (f_0(500) \rightarrow ) \pi \pi \) transition progress, its branching ratio is much more sensitive to the Gegenbauer moment \(a_S\) from the S-wave DAs. In the PQCD approach, wave functions are the most important input parameters and the improved knowledge of them is expected to yield improved estimates of the branching ratios and other observables, which may lead to better consistency with the data.

Since the \(f_0(500)\) is very broad, we use BW formula and Bugg model to parameterize the \(f_0(500)\) resonance and compare their results. It is found that the model-dependence of the decay rate is indeed not significant. The central values of PQCD predictions based on the Bugg model are consistent with the ones from the BW formula. Our prediction of \(B^0\rightarrow (f_0(980)\rightarrow )\pi ^+\pi ^-(K^{*0}\rightarrow )K^+\pi ^-\) is consistent with the current data, and also comparable with that from Table III in [104] within errors. In order to compare our predictions with other theoretical results for decays involving \(f_0(980)\), we use the \(\mathcal{B}(f_0(980)\rightarrow \pi ^+\pi ^-)=0.50\), which is taken from [6] and in agreement with the value of \(\mathcal{B}(f_0(980)\rightarrow \pi ^+\pi ^-)=0.46\) obtained in [131]. We can extract the branching ratios of the two-body decays \(B \rightarrow f_0(980)K^*\) from the corresponding four-body decays \(B \rightarrow f_0(980)(\rightarrow \pi \pi )K^*(\rightarrow K\pi )\) in Table 5 under the narrow width approximation. Taking the decay \(B^0\rightarrow f_0(980)K^{*0}\) as an example, we obtain its branching ratio \(\mathcal{B}(B^0\rightarrow f_0(980)K^{*0})=8.7\times 10^{-6}\), which is in good agreement with previous two-body results in the QCDF approach [6] and PQCD approach [15]. Strictly speaking, the narrow width approximation is not fully justified since such approximation has its scope of application. As mentioned above, the nonperturbative input parameters from the wave functions make important sense to the branching ratios. We can fit the related Gegenbauer moments with abundant data to match the experiment in the future. However, the fact that their rates can be accommodated in the two-quark picture for \(f_0(980)\) does not mean that \({\bar{q}}{q}\) composition should be supported. It is too difficult to make theoretical predictions on these decay modes based on the four-quark picture for scalar resonances. We just assume they are constituted by two quarks at this moment.

The decays \(B \rightarrow \rho K_0^{*}(1430)\) have already been studied systematically in the two-body framework within the PQCD approach [16]. Taking the two measured channels \(B^{0} \rightarrow \rho ^{-}K_0^{*}(1430)^{+} \rightarrow (\pi ^{-}\pi ^{0})(K^{0}\pi ^{+})\) and \(B^{0} \rightarrow \rho ^{0}K_0^{*}(1430)^{0} \rightarrow (\pi ^{+}\pi ^{-})(K^{+}\pi ^{-})\) as examples, we have:

$$\begin{aligned}&\mathcal{B }(B^0 \rightarrow \rho ^- K_0^{*}(1430)^{+} \rightarrow (\pi ^-\pi ^0)(K^0\pi ^+))\nonumber \\&\quad = \left\{ \begin{array}{ll} (11.8^{+8.1}_{-5.8})\times 10^{-6} &{} \mathrm{this \quad work},\\ (7.0^{+2.9}_{-1.7})\times 10^{-6} &{} \mathrm{PQCD}~[16],\\ (19\pm 8)\times 10^{-6} &{} \mathrm{Data} ~[46],\\ \end{array} \right. \end{aligned}$$
(74)
$$\begin{aligned}&\mathcal{B }(B^0 \rightarrow \rho ^0 K_0^{*}(1430)^{0}\rightarrow (\pi ^+\pi ^-)(K^+\pi ^-))\nonumber \\&\quad = \left\{ \begin{array}{ll} (7.9^{+4.9}_{-3.6})\times 10^{-6} &{} \mathrm{this \quad work},\\ (3.2^{+1.0}_{-0.7})\times 10^{-6} &{} \mathrm{PQCD}~[16],\\ (18\pm 4)\times 10^{-6} &{} \mathrm{Data} ~[46].\\ \end{array} \right. \end{aligned}$$
(75)

The results from the previous PQCD work [16] are obtained by multiplying the relevant two-body branching ratios according to Eqs. (70)–(71). Since the width of the resonant state and the interactions between the final state meson pair will show their effects on the branching ratios, the new four-body predictions are relatively larger than the converted values from previous PQCD calculations, but more close to the experimental data. Therefore, it seems more appropriate to treat these decay modes as four-body decays.

Table 6 Direct CP asymmetries (in units of \(\%\)) for the \(B_{(s)} \rightarrow VV \rightarrow (\pi \pi )(K\pi )\) decays compared with the previous predictions in the PQCD approach [18], the updated predictions in the QCDF [3, 4], SCET [25] and FAT [26]. Experimental results for branching ratios are taken from Table 2. The sources of the theoretical errors are the same as in Table 4
Table 7 Direct CP asymmetries (in units of \(\%\)) for the four-body \(B \rightarrow [SS,SV,VS] \rightarrow (\pi \pi ) (K\pi )\) decays, with \(S=[f_0(500),f_0(980),K^*_0(1430)]\) and \(V=[\rho , K^*]\). The sources of the theoretical errors are the same as in Table 4

3.3 Direct CP asymmetries

In Table 6, we show the direct CP asymmetries with each helicity state (\({\mathcal {A}}_{0,\parallel ,\perp }^{\text {CP}}\)) for the four-body \(B_{(s)} \rightarrow VV \rightarrow (\pi \pi ) (K\pi )\) decays together with those summed over all helicity states (\({\mathcal {A}}^{\text {CP}}\)). For comparison, the updated results of the QCDF [3, 4], SCET [25] and FAT [26] as well as the PQCD predictions in two-body framework [18] are also presented. Meanwhile, direct CP asymmetries for the four-body \(B_{(s)} \rightarrow [SS,SV,VS] \rightarrow (\pi \pi ) (K\pi )\) decays are displayed in Table 7. As we know, the kinematics of the two-body decays is fixed, the decay amplitudes of the quasi-two-body decays depend on the invariant mass of the final-state pairs, which result in the differential distribution of direct CP asymmetries. The CP asymmetry in the four-body framework is moderated by the finite width of the intermediate resonance appearing in the time-like form factor \(F(\omega ^2)\). Thus, it is reasonable to see the differences of direct CP asymmetries between the two-body and four-body frameworks in the PQCD approach. By comparing the numerical results as listed in Table 6, due to the different mechanism and origins of the strong phase, one can see that the QCDF and SCET results for the direct CP asymmetries are quite different from ours for some decay modes. As is well known, besides the weak phase from the CKM matrix elements, the direct CP asymmetry is proportional to the strong phase. In the SCET, the strong phase is only from the nonperturbative charming penguin at leading power and leading order, while in the QCDF and PQCD approaches, the strong phase comes from the hard spectator scattering and annihilation diagrams respectively. Besides, the power corrections such as penguin annihilation, which are essential to resolve the CP puzzles in the QCDF, are often plagued by the endpoint divergence that in turn break the factorization theorem [3]. In the PQCD approach, the endpoint singularity is cured by including the parton’s transverse momentum. Anyway, since current experimental measurements still have relatively large uncertainties, we have to wait for more time to test these different predictions.

In Tables 6 and 7 , a large CP asymmetry can be understood due to the sizable interference between the tree and penguin amplitudes, while a small value of CP asymmetry is attributed to the dominant tree or penguin amplitudes. For example, among the six considered \(B_{(s)}\rightarrow \rho K^*\rightarrow (\pi \pi )(\pi K)\) decays as presented in Table 6, the CP asymmetries \({\mathcal {A}}^{\text {CP}}\) for the two penguin-dominant processes \(B^0 \rightarrow (\rho ^0\rightarrow )\pi ^+\pi ^- (K^{*0}\rightarrow )K^+\pi ^-\) and \(B^+ \rightarrow (\rho ^+\rightarrow )\pi ^+\pi ^0 (K^{*0}\rightarrow )K^+\pi ^-\) are indeed quite small: less than \(1\%\). However, for the “ Color-suppressed ” decay \(B_s^0 \rightarrow (\rho ^0\rightarrow )\pi ^+\pi ^- ({{\bar{K}}}^{*0}\rightarrow )K^-\pi ^+\), due to the large penguin contributions from the chirally enhanced annihilation diagrams, the sizable interference between the tree and penguin contributions makes the direct CP asymmetries \({\mathcal {A}}^{\text {CP}}\) as large as \(\sim 30\%\). For four \(B^{+,0} \rightarrow \rho K^* \rightarrow (\pi \pi )(K\pi )\) decays, our predictions of CP asymmetries are in agreement with observations within uncertainties. Moreover, a helicity-specific analysis would provide interesting further insights. Very recently, LHCb [43] has reported the CP asymmetry associated with longitudinal polarization \({\mathcal {A}}^0_{\rho K^*}=-0.62\pm 0.09\pm 0.09\), where the first uncertainty is statistical and the second systematic. The data is much different from our prediction \({\mathcal {A}}_0^{\text {CP}}(B^0 \rightarrow \rho ^0(\rightarrow \pi ^+\pi ^-)K^{*0}(\rightarrow K^+\pi ^-))=3.5\%\). Considering the branching ratio of the \(B^0 \rightarrow \rho ^0(\rightarrow \pi ^+\pi ^-)K^{*0}(\rightarrow K^+\pi ^-)\) decay associated with the longitudinal polarization, the contributions of the penguin diagrams (\(\mathcal{B}=2.98\times 10^{-6}\)) are larger than the tree ones (\(\mathcal{B}=5.74 \times 10^{-8}\)) by roughly a factor of 52, which results in the smallness of direct CP asymmetries. The big gap between the theory and experiment should be resolved in the future.

In the limit of U-spin symmetry, some of \(B_s\) decays can be related to \(B^0\) ones. For \(B_{(s)} \rightarrow VV\) decays, it has been studied in [4, 18] and seems to hold well. Since we have calculated the B and \(B_s\) decays to VV in this work in the PQCD approach, we also check the U-spin symmetry in some decay modes studied in [4, 18]:

$$\begin{aligned}&{\mathcal {A}}^{\text {CP}}\left( B_s\rightarrow \rho ^+K^{*-}\right) \nonumber \\&\quad =-{\mathcal {A}}^{\text {CP}}(B^0\rightarrow \rho ^-K^{*+})\frac{{\mathcal {B}}(B^0\rightarrow \rho ^-K^{*+})}{{\mathcal {B}}(B_s\rightarrow \rho ^+K^{*-})}\frac{\tau _{B_{s}}}{\tau _{B^0}},\nonumber \\&{\mathcal {A}}^{\text {CP}}\left( B_s\rightarrow \rho ^0{\bar{K}}^{*0}\right) \nonumber \\&\quad =-{\mathcal {A}}^{\text {CP}}\left( B^0\rightarrow \rho ^0 K^{*0}\right) \frac{{\mathcal {B}}\left( B^0\rightarrow \rho ^0 K^{*0}\right) }{{\mathcal {B}}\left( B_s\rightarrow \rho ^0{\bar{K}}^{*0}\right) }\frac{\tau _{B_{s}}}{\tau _{B^0}}. \end{aligned}$$
(76)

On basis of these U-spin relations along with the branching ratios, the lifetimes of B and \(B_s\) mesons and the direct CP asymmetries in B decays, we can get the relevant direct CP asymmetries in \(B_s\) decays. This can be then compared with the corresponding predictions in the PQCD approach to check whether the U-spin symmetry works well or not. We show this comparison in Table 8, where the entries in the last two columns have to be compared with each other. It turns out that U-spin symmetry is in general acceptable within the calculational errors.

Table 8 The direct CP asymmetries (\(\%\)) in \(B_s \rightarrow \rho K^*\) decays via U-spin relation together with the direct PQCD prediction
Table 9 PQCD predictions for the TPAs (\(\%\)) of the four-body \(B_{(s)} \rightarrow (\rho \rightarrow ) \pi \pi (K^*\rightarrow ) K\pi \) decays. The sources of theoretical errors are same as in Table 4 but added in quadrature

3.4 Triple product asymmetries in \(B_{(s)} \rightarrow \rho (\rightarrow \pi \pi )K^*(\rightarrow K\pi )\) decays

The predicted TPAs for the \(B_{(s)} \rightarrow (\pi \pi )(K\pi )\) decays are displayed in Table 9. It is shown that our PQCD predictions of “true” CP-violating TPAs are very small in the SM, which makes the measurement of a large value for that TPA point clearly towards the presence of new physics. As “fake” TPAs are due to strong phases and require no CP violation, the large fake \({\mathcal {A}}_{\text {T-fake}}^{1,2}\) simply reflects the importance of the strong final-state phases.

Since the left-handedness of the weak interaction \(A_{-}\ll A_{+}\) is expected, it implies \(A_{\parallel }\approx A_{\perp }\). The \({\mathcal {A}}_T^2\) term requires both transversely polarized components \(A_{\parallel }\) and \(A_{\perp }\) and the decay amplitude associated with transverse polarization is smaller than that for longitudinal polarization in the naive expectation. Hence \({\mathcal {A}}_T^2\) is power suppressed relative to \({\mathcal {A}}_T^1\). Meanwhile, the smallness of \(\mathcal {A}_T^2\) is also attributed to the suppression from the strong phase difference between the perpendicular and parallel polarization amplitudes, which was found in the PQCD framework [18] and supported by the LHCb Collaboration [43]. An observation of \({\mathcal {A}}_T^2\) with large values can signify physics beyond the SM. As mentioned above, \(({\mathcal {A}}_T^{1,2}+\bar{{\mathcal {A}}}_T^{1,2})/2\ne {\mathcal {A}}_T^{(1,2)\text {ave}}(\text {true})\) when the decay channel has a nonzero CP asymmetries. We find that the greater difference between the \({\mathcal {A}}_T^{1,2}(\text {true})\) and \({\mathcal {A}}_T^{(1,2)\text {ave}}(\text {true})\) appears with the larger direct CP asymmetry.

Recently, the measurements of “true” and “fake” TPAs for \(B^0 \rightarrow \rho ^0 K^{*0} \rightarrow (\pi ^+ \pi ^-)(K^+\pi ^-)\) have been reported by LHCb Collaboration [43]. The PQCD prediction of \({\mathcal {A}}_{\text {T-true}}^1\) agrees well with the experiment \({\mathcal {A}}_{\text {T-true}}^{\rho K^*,1}=-0.0210\pm 0.0050\pm 0.0022\), where the first uncertainty is statistical and the second systematic. While for “fake” TPAs, our predictions are a little larger than the measurements but compatible within large uncertainties. It should be stressed that there are large uncertainties in both experimental measurements and the theoretical calculations for TPAs, so the discrepancy between the data and the theoretical results could be clarified with the high precision both in experimental and theoretical sides. Since “fake” TPAs strongly affected by the strong phases, we lack a perfect knowledge of all the possible signals of the strong phases, such as final-state interactions. For this reason, we just estimate the size of the corresponding TPAs. We hope the future experiments can test our predictions.

4 Conclusion

In this work, we have presented six helicity amplitudes of four-body decays \(B_{(s)} \rightarrow (\pi \pi )(K\pi )\), where \(\pi \pi \) invariant-mass spectrum is dominated by the vector \(\rho \) resonance and scalar \(f_0(500), f_0(980)\) resonances, and the vector \(K^*\) resonance and scalar resonance \(K_0^*(1430)\) are expected to contribute in the \(K\pi \) invariant-mass range. We have examined the branching ratios, polarization fractions, direct CP asymmetries, triple product asymmetries in \(B_{(s)} \rightarrow [VV,SS,SV,VS] \rightarrow (\pi \pi )(K\pi )\) decays. In our numerical study, there exist many theoretical uncertainties in the calculation. The uncertainties of the nonperturbative parameters of the two-meson DAs and the variation of the hard scale provide the dominant theoretical errors to the theoretical predictions for branching ratios and other physical observables. Therefore, the relevant Gegenbauer moments should be further constrained to improve the precision of theoretical predictions and meet with future data. In addition, one should make a great effort to evaluate the higher-order contributions to four-body B meson decays in order to reduce the sensitivity to the variation of the hard scales.

We have extracted the branching ratios of two-body \(B\rightarrow \rho K^*\) decays from the results for the corresponding four-body decays under the narrow-width approximation and shown the polarization fractions of the related decay channels. The obtained two-body branching ratios agree well with previous theoretical studies performed in the two-body framework within errors. The predicted hierarchy pattern for the longitudinal polarization fractions in the \(B_{(s)}\) meson decays into the P-wave \(\pi \pi \) and \(K\pi \) pairs is compatible with the data roughly. However, there is a big gap between our prediction of longitudinal polarization fraction for \(B^0 \rightarrow \rho ^0 K^{*0}\) and the recent LHCb measurement, which should be resolved. In addition, we have calculated the branching ratios of the four-body decays \(B_{(s)} \rightarrow [SS,SV,VS]\rightarrow (\pi \pi )(K\pi )\). For the decays associated with scalar resonance \(f_0(500)\), we have used the BW and Bugg models to parameterize the wide \(f_0(500)\) meson respectively but found that the model-dependence of the PQCD predictions is not significant. The branching ratios of \(B^0 \rightarrow (\rho ^- \rightarrow )\pi ^-\pi ^0(K_0^{*}(1430)^{+}\rightarrow )K^0\pi ^+\) and \(B^0 \rightarrow (\rho ^0 \rightarrow )\pi ^+\pi ^-(K_0^{*}(1430)^{0}\rightarrow )K^+\pi ^-\) decays, which are related to isospin limit, remain puzzling and need to be resolved.

We have calculated the direct CP asymmetries with each helicity state (\({\mathcal {A}}_{0,\parallel ,\perp }^{\text {CP}}\)) for the four-body \(B_{(s)} \rightarrow VV \rightarrow (\pi \pi ) (K\pi )\) decays, together with the direct CP asymmetries of \(B_{(s)} \rightarrow [SS,SV,VS] \rightarrow (\pi \pi )(K\pi )\) decays. The CP asymmetry in the four-body framework is dependent on the invariant mass of the final-state pairs, which results in the differences between the two-body and four-body frameworks in the PQCD approach. Meanwhile, we perform an angular analysis on four-body \(B_{(s)} \rightarrow \rho K^* \rightarrow (\pi \pi )(K\pi )\) decays to obtain the triple product asymmetries in detail. We found that most “true” TPAs are very small, which are consistent with the predictions of the standard model. A “true” TPA that is predicted to vanish provides an excellent place for looking for new physics because there is no suppression from the strong phases.