Localized direct CP violation in $B^\pm \rightarrow \rho ^0 (\omega )\pi ^\pm \rightarrow \pi ^+ \pi ^-\pi ^\pm$

Abstract

We study the localized direct CP violation in the hadronic decays $B^\pm \rightarrow \rho ^0 (\omega )\pi ^\pm \rightarrow \pi ^+ \pi ^-\pi ^\pm$, including the effect caused by an interesting mechanism involving the charge symmetry violating mixing between $\rho ^0$ and $\omega$. We calculate the localized integrated direct CP violation when the low invariant mass of $\pi ^+\pi ^-$ [$m(\pi ^+\pi ^-)_{\mathrm{low}}$] is near $\rho ^0(770)$. For the five models of form factors investigated, we find that the localized integrated direct CP violation varies from $-0.0752$ to $-0.0290$ in the ranges of parameters in our model when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800$ GeV. This result, especially the sign, agrees with the experimental data and is independent of the form factor models. The new experimental data shows that the signs of the localized integrated CP asymmetries in the regions $0.470<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.770$ and $0.770<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.920$ GeV are positive and negative, respectively. We find that $\rho$–$\omega$ mixing makes the localized integrated CP asymmetry move toward the negative direction, and therefore contributes to the sign change in those two regions. This behavior is also independent of the form factor models. We also calculate the localized integrated direct CP violating asymmetries in the regions $0.470<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.770$, $0.770<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.920$ and the whole region $0.470<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.920$ GeV and find that they agree with the experimental data in some models of the form factors.

1 Introduction

CP violation is one of the most fundamental and important properties of weak interactions. Even though it has been known since 1964 [1], we still do not know the source of CP violation completely. In the Standard Model, CP violation originates from the weak phase in the Cabibbo–Kobayashi–Maskawa (CKM) matrix [2, 3]. Besides the weak phase, a large strong phase is also needed for direct CP violation to occur in decay processes. Usually, this large phase is provided by QCD loop corrections and some phenomenological mechanisms. In the past few years, numerous theoretical studies have been conducted on CP violation. However, we need a lot of data to test these approaches because there are many theoretical uncertainties such as CKM matrix elements, hadronic matrix elements, and nonfactorizable effects. These uncertainties would be reduced by the increase of experimental data in the future and the improvement of theoretical methods.

Recently, the LHCb Collaboration focused on three-body final states in the decays of B and D mesons and a novel strategy to probe CP asymmetry in their Dalitz plots [4–6]. The local asymmetries in specific regions of the phase space of charmless three-body decays of bottom mesons, such as $B^\pm \rightarrow \pi ^\pm \pi ^+\pi ^-$ and $B^\pm \rightarrow K^\pm \pi ^+\pi ^-$, were measured. It was shown that the local asymmetry distributions in the Dalitz plots reveal rich structures and are not uniform [4–6]. These intriguing discoveries offer opportunities to search for different sources of CP violation, through the study of the signatures of these sources in certain phase spaces of the Dalitz plots. In fact, several theoretical studies have been made to explain these distributions, such as the interference between intermediate states [7–9] and final-state rescatterings [6, 10–12]. One can confirm that these complex structures originate from more than one source [6].

Charge symmetry is broken at the most fundamental level in strong interaction physics through the small mass difference between up and down quarks in the QCD Lagrangian. As a consequence the physical $\rho ^0$ and $\omega$ mesons are not eigenstates of isospin but, for example, the physical $\rho ^0$ contains a small admixture of an $I = 0$ state [13]. In previous works, this phenomenon, known as $\rho$–$\omega$ mixing, was considered to obtain a large strong phase in B decays [14–17] and it was found that such a mixing can lead to a peak of CP violation when the invariant mass of $\pi ^+\pi ^-$ is near $\omega$. The differential CP asymmetry was studied in the decays $B^\pm \rightarrow \rho ^0(\omega )\pi ^\pm \rightarrow \pi ^+\pi ^-\pi ^\pm$ before [18–20]. $\rho$–$\omega$ mixing provides additional complex terms to the amplitudes [21] and the strong phase passes through $90^{\circ }$ at the $\omega$ resonance [15–20, 22]. In our previous work, we found that it is more useful to investigate the localized integrated CP asymmetry and studied the localized integrated CP asymmetry in hadronic $\tau$ decays with this mechanism [21]. The newest LHCb experiments showed that the resonances are $\rho ^0$(770) when $m^2(\pi ^+\pi ^-)_{\mathrm{low}}<1\mathrm {GeV}^2/\mathrm {c}^4$ for $B^\pm \rightarrow \pi ^+\pi ^-\pi ^\pm$ decays [$m(\pi ^+\pi ^-)_{\mathrm{low}}$ is the low invariance mass of $\pi ^+\pi ^-$] [6]. It was claimed that the CP asymmetry in these decays changes sign around the $\rho ^0(770)$ peak of $m^2(\pi ^+\pi ^-)_{\mathrm{low}}$ [6], which contains the significant region of $\rho$–$\omega$ mixing. In the present paper, we aim at studying the localized integrated CP asymmetry in $B^\pm \rightarrow \pi ^+\pi ^-\pi ^\pm$ decays involving $\rho$–$\omega$ mixing and comparing it with the results obtained from the LHCb Collaboration.

In this paper, we will investigate the localized integrated CP violation with five phenomenological models of weak form factors for $B^\pm \rightarrow \rho ^0(\omega )\pi ^\pm \rightarrow \pi ^-\pi ^+\pi ^\pm$ with and without $\rho$–$\omega$ mixing. We will determine the allowed range of $N_c$, which is the effective color number describing nonfactorizable contributions. The model dependence of our results will be discussed in detail. We will see that for five models of the form factors, our result for the localized integrated CP asymmetry varies from $-0.0752$ ($-0.0626$) to $-0.0403$ ($-0.0290$) in the ranges of $N_c$, $2.07(2.09)<N_c<4.54(4.65)$ corresponding to $q^2/m_b^2=0.3(0.5)$ (q is the typical momentum transfer of the gluon or photon in the penguin diagrams) when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800$ GeV. From Fig. 4 of Ref. [6], we can see the localized integrated CP asymmetries have different signs in $\cos \theta >0$ and $\cos \theta <0$ regions when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800\, \text {GeV}$ ($\theta$ is the angle between the momenta of the unpaired $\pi ^{+(-)}$ and the $\rho ^0$ decay product with the same-sign charge). If one adds the events in these two experimental regions together, the total localized integrated CP asymmetry will be $-0.0294\pm 0.0285$. Our results agree with this experimental data. The experimental values of the localized integrated CP asymmetries in the regions $0.470<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.770\, \text {GeV}$ and $0.770<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.920\, \text {GeV}$ are $0.0508\pm 0.0171$ and $-0.0256\pm 0.0202$, respectively, with opposite signs [6]. We will find that $\rho$–$\omega$ mixing can make the localized integrated CP asymmetry move toward the negative direction, and therefore it will contribute to the sign change from the region $0.470<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.770\, \text {GeV}$ to $0.770<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.920\, \text {GeV}$. This behavior is independent of the form factor models. Furthermore, we will find that our results in these two regions and the whole region of them are consistent with the experimental data for several models of form factors.

The remainder of this paper is organized as follows. In Sect. 2, we present the form of the effective Hamiltonian and the values of Wilson coefficients and give the formalism for the CP violating asymmetry in $B^\pm \rightarrow \rho ^0(\omega )\pi ^\pm \rightarrow \pi ^+\pi ^-\pi ^\pm$. Then we show numerical results with several models of the form factors in this section. In Sect. 3, we calculate branching ratios for $B^+\rightarrow \rho ^0\pi ^+$ and $B^0\rightarrow \rho ^+\pi ^-$ and present numerical results for the range of $N_c$ allowed by the experimental data. In the last section, we give some discussions and summarize our results.

2 CP violation in $B^\pm \rightarrow \rho ^0 (\omega )\pi ^\pm \rightarrow \pi ^+ \pi ^-\pi ^\pm$

The amplitude of a decay process described by some amplitudes may have CP-even and -odd relative phases. Within the Standard Model, the CP-odd relative phase is always a weak phase difference which is directly determined by the CKM matrix. On the other hand, CP-even phases, which are called strong phases, usually originate from nonperturbative effects of strong interactions and are hard to handle. We consider a B meson weak decay process, $B\rightarrow M_1 M_2 M_3$, where $M_i(i=1,2,3)$ is a light pseudoscalar meson. For a weak decay process of a heavy meson, a typical form of the decay amplitude A and its CP conjugate one $\bar{A}$ are

$$\begin{aligned} A= & {} g_1r_1\mathrm {e}^{\mathrm {i}\phi _1}+g_2r_2\mathrm {e}^{\mathrm {i}\phi _2},\end{aligned}$$

(1)

$$\begin{aligned} \bar{A}= & {} g_1^*r_1\mathrm {e}^{\mathrm {i}\phi _1}+g_2^*r_2\mathrm {e}^{\mathrm {i}\phi _2}, \end{aligned}$$

(2)

where $g_1$ and $g_2$ represent CP-odd complex terms which involve CKM matrix elements, $r_1\mathrm{e}^{\mathrm {i}\phi _1}$ and $r_2\mathrm{e}^{\mathrm {i}\phi _2}$ terms are even under the CP transformation. Then one has

$$\begin{aligned} |A|^2-|\bar{A}|^2= & {} 4r_1r_2\,\mathfrak {I}\mathfrak {m}(g_1^*g_2)\,\sin (\phi _1-\phi _2)\nonumber \\ \!= & {} \!4r_1r_2|g_1||g_2|\,\sin [\,\mathrm {Arg}(g_2/g_1)]\,\sin (\phi _1-\phi _2),\nonumber \\ \end{aligned}$$

(3)

from which we can see explicitly that both the CP-odd phase difference $\mathrm {Arg}(g_2/g_1)$ and the CP-even phase difference $\phi _1-\phi _2$ are needed to produce CP violation.

2.1 The effective Hamiltonian

In order to calculate the direct CP violating asymmetry in hadronic decays, one can use the following effective weak Hamiltonian, based on the Operator Product Expansion [23–27]:

$$\begin{aligned} \mathcal {H}=\frac{G_F}{\sqrt{2}}\Bigg [V_{ub}V^*_{ud}(c_1O^u_1+c_2O^u_2)-V_{tb}V^*_{td}\sum ^{10}_{i=3}c_iO_i\Bigg ]+\mathrm{H.c.}, \end{aligned}$$

(4)

where $V_{ub}$, $V_{ud}$, $V_{tb}$, and $V_{td}$ are CKM matrix elements, and $c_i(i=1,2,\ldots ,10)$ are the Wilson coefficients, which are calculable in the renormalization group improved perturbation theory and are scale dependent. In the present case, we work with the renormalization scheme independent Wilson coefficients and use the values of the Wilson coefficients at the renormalization scale $\mu \approx m_b$. The operators $O_i$ have the following forms:

$$\begin{aligned}&O^u_1=\overline{d}_\alpha \gamma _\mu (1-\gamma _5)u_\beta \overline{u}_\beta \gamma ^\mu (1-\gamma _5)b_\alpha ,\nonumber \\&O^u_2=\overline{d}\gamma _\mu (1-\gamma _5)u\overline{u}\gamma ^\mu (1-\gamma _5)b,\nonumber \\&O_3=\overline{d}\gamma _\mu (1-\gamma _5)b\sum _{q^\prime }\overline{q^\prime }\gamma ^\mu (1-\gamma _5)q^\prime ,\nonumber \\&O_4=\overline{d}_\alpha \gamma _\mu (1-\gamma _5)b_\beta \sum _{q^\prime }\overline{q^\prime }_\beta \gamma ^\mu (1-\gamma _5)q^\prime _\alpha ,\nonumber \\&O_5=\overline{d}\gamma _\mu (1-\gamma _5)b\sum _{q^\prime }\overline{q^\prime }\gamma ^\mu (1+\gamma _5)q^\prime ,\nonumber \\&O_6=\overline{d}_\alpha \gamma _\mu (1-\gamma _5)b_\beta \sum _{q^\prime }\overline{q^\prime }_\beta \gamma ^\mu (1+\gamma _5)q^\prime _\alpha ,\nonumber \\&O_7=\frac{3}{2}\overline{d}\gamma _\mu (1-\gamma _5)b\sum _{q^\prime }e_{q^\prime }\overline{q^\prime }\gamma ^\mu (1+\gamma _5)q^\prime ,\nonumber \\&O_8=\frac{3}{2}\overline{d}_\alpha \gamma _\mu (1-\gamma _5)b_\beta \sum _{q^\prime }e_{q^\prime }\overline{q^\prime }_\beta \gamma ^\mu (1+\gamma _5)q^\prime _\alpha ,\nonumber \\&O_9=\frac{3}{2}\overline{d}\gamma _\mu (1-\gamma _5)b\sum _{q^\prime }e_{q^\prime }\overline{q^\prime }\gamma ^\mu (1-\gamma _5)q^\prime ,\nonumber \\&O_{10}=\frac{3}{2}\overline{d}_\alpha \gamma _\mu (1-\gamma _5)b_\beta \sum _{q^\prime }e_{q^\prime }\overline{q^\prime }_\beta \gamma ^\mu (1-\gamma _5)q^\prime _\alpha , \end{aligned}$$

(5)

where $\alpha$ and $\beta$ are color indices, and $q^\prime =u, d$ or s quark. In Eq. (5), $O_1^u$ and $O_2^u$ are the tree level operators, $O_3$–$O_6$ are QCD penguin operators, and $O_7$–$O_{10}$ arise from electroweak penguin diagrams.

The Wilson coefficients, $c_i$, which are known to the next-to-leading logarithmic order, take the following values [28–31]:

$$\begin{aligned} c_1= & {} -0.3125,\quad c_2=1.1502,\nonumber \\ c_3= & {} 0.0174,\quad c_4=-0.0373,\nonumber \\ c_5= & {} 0.0104,\quad c_6=-0.0459,\nonumber \\ c_7= & {} -1.050\times 10^{-5},\quad c_8=3.839\times 10^{-4},\nonumber \\ c_9= & {} -0.0101,\quad c_{10}=1.959\times 10^{-3}, \end{aligned}$$

(6)

at the scale $\mu = m_b=5\,\text {GeV}$.

To be consistent, the matrix elements of the operators $O_i$ should also be renormalized to the one-loop order. This results in the effective Wilson coefficients, $c^\prime _i$, which satisfy the constraint

$$\begin{aligned} c_i(m_b)\langle O_i (m_b) \rangle =c^\prime _i\langle O_i\rangle ^{\mathrm{tree}}, \end{aligned}$$

(7)

where $\langle O_i \rangle ^{\mathrm{tree}}$ is the matrix element at the tree level, which will be evaluated in the factorization approach. From Eq. (7), the relations between $c^\prime _i$ and $c_i$ are [28–31]

$$\begin{aligned} \begin{aligned} c_1^\prime&=c_1,\quad c_2^\prime =c_2,\\ c_3^\prime&=c_3-P_s/3,\quad c_4^\prime =c_4+P_s,\\ c_5^\prime&=c_5-P_s/3,\quad c_6^\prime =c_6+P_s,\\ c_7^\prime&=c_7+P_e,\quad c_8^\prime =c_8,\\ c_9^\prime&=c_9+P_e,\quad c_{10}^\prime =c_{10}, \end{aligned} \end{aligned}$$

(8)

where

$$\begin{aligned} P_s= & {} (\alpha _s/8 \pi )c_2[10/9+G(m_c,\mu ,q^2)], \\ P_e= & {} (\alpha _{em}/9 \pi )(3c_1+c_2)[10/9+G(m_c,\mu ,q^2)], \end{aligned}$$

with

$$\begin{aligned} G(m_c,\mu ,q^2)=4\int ^1_0\text {d}xx(x-1)\text {ln}\frac{m^2_c-x(1-x)q^2}{\mu ^2}. \end{aligned}$$

$G(m_c,\mu ,q^2)$ has the following explicit expression [32]:

$$\begin{aligned} \mathfrak {R}\mathfrak {e}G= & {} \frac{2}{3}\left[ \text {ln}\frac{m_c^2}{\mu ^2}-\frac{5}{3}-4\frac{m_c^2}{q^2}\right. \nonumber \\&\left. +\bigg (1+2\frac{m_c^2}{q^2}\bigg )\sqrt{1-4\frac{m_c^2}{q^2}}\text {ln}\frac{1+\sqrt{1-4\frac{m_c^2}{q^2}}}{1-\sqrt{1-4\frac{m_c^2}{q^2}}}\right] ,\nonumber \\ \mathfrak {I}\mathfrak {m}G= & {} -\frac{2}{3}\Bigg (1+2\frac{m_c^2}{q^2}\Bigg )\sqrt{1-4\frac{m_c^2}{q^2}}. \end{aligned}$$

(9)

Based on simple arguments at the quark level, the value of $q^2$ is chosen to be in the range $0.3<q^2/m^2_b<0.5$ [14]. From Eqs. (8) and (9) we can obtain numerical values for $c^\prime _i$.

When $q^2/m_b^2=0.3$,

$$\begin{aligned} \begin{aligned}&c^\prime _1=-0.3125,\quad c^\prime _2=1.1502,\\&c^\prime _3=2.433\times 10^{-2}+1.543\times 10^{-3}\text {i},\\&c^\prime _4=-5.808\times 10^{-2}-4.628\times 10^{-3}\text {i},\\&c^\prime _5=1.733\times 10^{-2}+1.543\times ^{-3}\text {i},\\&c^\prime _6=-6.668\times 10^{-2}-4.628\times ^{-3}\text {i},\\&c^\prime _7=-1.435\times 10^{-4}-2.963\times ^{-5}\text {i},\\&c^\prime _8=3.839\times 10^{-4},\\&c^\prime _9=-1.023\times 10^{-2}-2.963\times ^{-5}\text {i},\\&c^\prime _{10}=1.959\times 10^{-3}, \end{aligned} \end{aligned}$$

(10)

and when $q^2/m_b^2=0.5$, one has

$$\begin{aligned}&c^\prime _1=-0.3125,\quad c^\prime _2=1.1502,\nonumber \\&c^\prime _3=2.120\times 10^{-2}+2.174\times 10^{-3}\text {i},\nonumber \\&c^\prime _4=-4.869\times 10^{-2}-1.552\times 10^{-3}\text {i},\nonumber \\&c^\prime _5=1.420\times 10^{-2}+5.174\times ^{-3}\text {i},\nonumber \\&c^\prime _6=-5.729\times 10^{-2}-1.552\times ^{-2}\text {i},\\&c^\prime _7=-8.340\times 10^{-5}-9.938\times ^{-5}\text {i},\nonumber \\&c^\prime _8=3.839\times 10^{-4},\nonumber \\&c^\prime _9=-1.017\times 10^{-2}-9.938\times ^{-5}\text {i},\nonumber \\&c^\prime _{10}=1.959\times 10^{-3},\nonumber \end{aligned}$$

(11)

where we have taken $\alpha _s(m_Z)=0.112$, $\alpha _{\mathrm{em}}(m_b)=1/132.2$, $m_b=5$ GeV, and $m_c=1.35$ GeV.

2.2 Formalism

For the $B^-\rightarrow \rho ^0\pi ^-$ process, the amplitude can be written as $M_{B^-\rightarrow \rho ^0\pi ^-}^\lambda =\alpha p_B\cdot \epsilon ^*(\lambda )$, where $\epsilon$ is the polarization vector of $\rho ^0$ and $\lambda$ is its polarization, $p_B$ is the momenta of $B^-$ meson, and $\alpha$ is independent of $\lambda$. The amplitude for $\rho ^0\rightarrow \pi ^+\pi ^-$ is $M_{\rho ^0\rightarrow \pi ^+\pi ^-}^\lambda =g_\rho \epsilon (\lambda )(p_1-p_2)$, where $p_1$ and $p_2$ are the momenta of $\pi ^+$ and $\pi ^-$ produced by $\rho ^0$, respectively, and $g_\rho$ is the effective coupling constant for $\rho ^0\rightarrow \pi ^+\pi ^-$. Then, for the sequential decay $B^-\rightarrow \rho ^0\pi ^-\rightarrow \pi ^+\pi ^-\pi ^-$, the amplitude is [8, 9]:

$$\begin{aligned} A= & {} \alpha p_B^\mu \frac{\sum _\lambda \epsilon ^*_\mu (\lambda )\epsilon _\nu (\lambda )}{s_\rho }g_\rho (p_1-p_2)^\nu \nonumber \\= & {} \frac{g_\rho \alpha }{s_\rho }\cdot p_B^\mu \Bigg [ g_{\mu \nu }-\frac{(p_1+p_2)_\mu (p_1+p_2)_\nu }{s} \Bigg ](p_1-p_2)^\nu \nonumber \\= & {} \frac{g_\rho }{s_\rho }\cdot \frac{M_{B^+\rightarrow \rho ^0\pi ^+}^\lambda }{p_B\cdot \epsilon ^*}\cdot (\Sigma -s^\prime )\nonumber \\= & {} (\Sigma -s^\prime )\cdot \mathcal {M}, \end{aligned}$$

(12)

where $\sqrt{s^\prime }$ is the high invariance mass of the $\pi ^+\pi ^-$ pair, $\Sigma =\frac{1}{2}(s^\prime _{\mathrm{max}}+s^\prime _{\mathrm{min}})$ with $s^\prime _{\mathrm{max}}$ and $s^\prime _{\mathrm{min}}$ being the maximum and minimum values of $s^\prime$ for a fixed s, respectively, and $\sqrt{s}$ is the low invariant mass of the $\pi ^+\pi ^-$ pair [$m(\pi ^+\pi ^-)_{\mathrm{low}}$] and $s_V$ is from the inverse propagator of the vector meson V, $s_V=s-m_V^2+\mathrm {i}m_V\Gamma _V$. The factor $\Sigma -s^\prime$ is equal to $-2|\mathbf {p_2}||\mathbf {p_3}|\cos {\theta }$ in Ref. [33], which accounts for angular momentum conservation for the spin-1 resonance. $\mathcal {M}$ will be calculated in the following. According to the effective Hamiltonian, A can be divided into two parts [18]:

$$\begin{aligned} A= & {} \langle \pi ^+\pi ^-\pi ^-|H^T|B^-\rangle +\langle \pi ^+\pi ^-\pi ^-|H^P|B^-\rangle , \end{aligned}$$

(13)

where $H^T$ and $H^P$ are the Hamiltonians for the tree and penguin operators, respectively.

In order to obtain a large signal for direct CP violation, we need to appeal to some phenomenological mechanisms. $\rho$–$\omega$ mixing has the dual advantages that the strong phase difference is large (passes through $90^{\circ }$ at the $\omega$ resonance) and well known [15–17]. With this mechanism, to the first order in isospin violation, the amplitude for $B^-\rightarrow \rho ^0(\omega )\pi ^- \rightarrow \pi ^+\pi ^-\pi ^-$ takes the following form at a value of $\sqrt{s}$ close to the $\omega$ resonance mass [18]:

$$\begin{aligned} \langle \pi ^+\pi ^-\pi ^-|H^T|B^-\rangle= & {} (\Sigma -s^\prime )\nonumber \\&\times \left( \frac{g_\rho }{s_\rho s_\omega }\tilde{\Pi }_{\rho \omega }t_\omega +\frac{g_\rho }{s_\rho }t_{\rho }\right) ,\end{aligned}$$

(14)

$$\begin{aligned} \langle \pi ^+\pi ^-\pi ^-|H^P|B^-\rangle= & {} (\Sigma -s^\prime )\nonumber \\&\times \left( \frac{g_\rho }{s_\rho s_\omega }\tilde{\Pi }_{\rho \omega }p_\omega +\frac{g_\rho }{s_\rho }p_{\rho }\right) , \end{aligned}$$

(15)

where $t_V$ ($V=\rho ^0$ or $\omega$) is the tree amplitude and $p_V$ is the penguin amplitude for producing an intermediate vector meson V, $\tilde{\Pi }_{\rho \omega }$ is the effective $\rho$–$\omega$ mixing amplitude. From Eqs. (14) and (15), we note that $\rho$–$\omega$ mixing provides an additional complex term for the tree and penguin amplitudes (the first term in each equation), respectively. These complex terms will enlarge the CP-even phase and lead to a peak of CP asymmetry as mentioned before. We will show the difference between the CP asymmetries with and without $\rho$–$\omega$ mixing later. Here, we assume that the $B^\pm \rightarrow \pi ^+\pi ^-\pi ^\pm$ process is dominated by the resonance $\rho ^0$ in certain region of its Dalitz plot.

We stress that the direct coupling $\omega \rightarrow \pi ^+\pi ^-$ is effectively absorbed into $\tilde{\Pi }_{\rho \omega }$ [34, 35], leading to the explicit s dependence of $\tilde{\Pi }_{\rho \omega }$. Making the expansion $\tilde{\Pi }_{\rho \omega }(s)=\tilde{\Pi }_{\rho \omega }(m_\omega ^2)+(s-m_\omega ^2)\tilde{\Pi }_{\rho \omega }^\prime (m_\omega ^2)$, the $\rho$–$\omega$ mixing parameters were determined in the fit of Gardner and O’Connell [36]:

$$\begin{aligned} \mathfrak {R} \mathfrak {e}\tilde{\Pi }_{\rho \omega }(m^2_{\omega })= & {} {}-3500\pm 300\ \mathrm {MeV^2},\nonumber \\ \mathfrak {I} \mathfrak {m}\tilde{\Pi }_{\rho \omega }(m^2_{\omega })= & {} {}-300\pm 300\ \mathrm {MeV^2},\\ \tilde{\Pi }_{\rho \omega }^\prime (m^2_{\omega })= & {} 0.03\pm \ 0.04.\nonumber \end{aligned}$$

(16)

In practice, the effect of the derivative term is negligible.

In this work, we only consider $\rho ^0$ and $\omega$ resonances. Then, for a fixed s, the differential CP asymmetry parameter can be defined as

$$\begin{aligned} A_{\mathrm{CP}}=\frac{|\mathcal {M}|^2-|\bar{\mathcal {M}}|^2}{|\mathcal {M}|^2+|\bar{\mathcal {M}}|^2}. \end{aligned}$$

(17)

By integrating the denominator and numerator of $A_{\mathrm{CP}}$, respectively, in the region $\Omega$ ($s_1<s<s_2$, $s^\prime _1<s^\prime <s^\prime _2$), we obtain the localized integrated CP asymmetry, which can be measured by experiments and takes the following form:

$$\begin{aligned} A_{\mathrm{CP}}^\Omega = \frac{\int _{s_1}^{s_2} \mathrm {d} s \int _{s^\prime _1}^{s^\prime _2} \mathrm {d}s^\prime (\Sigma -s^\prime )^2 (|\mathcal {M}|^2-\bar{|\mathcal {M}|}^2)}{\int _{s_1}^{s_2} \mathrm {d} s \int _{s^\prime _1}^{s^\prime _2} \mathrm {d}s^\prime (\Sigma -s^\prime )^2 (|\mathcal {M}|^2+\bar{|\mathcal {M}|}^2)}. \end{aligned}$$

(18)

According to kinematics of the three-body decay, $\Sigma [=\frac{1}{2}(s^\prime _{\mathrm{max}}+s^\prime _{\mathrm{min}})]$ is related to s. In our calculations, s varies in a small region, and therefore $\Sigma$ can be treated as a constant approximately [6]. Then the terms $\int _{s^\prime _1}^{s^\prime _2} \mathrm {d} s^\prime (\Sigma -s^\prime )^2$ are canceled, and $A_{\mathrm{CP}}^\Omega$ becomes independent of the high invariance mass of $\pi ^+\pi ^-$. In practice, to be more precise, we take into account the s-dependence of $s^\prime _{\mathrm{max}}$ and $s^\prime _{\mathrm{min}}$ in our calculations. We choose $s^\prime _{\mathrm{min}}<s^\prime <s^\prime _{\mathrm{max}}$ as the integration interval of the high invariance mass of $\pi ^+\pi ^-$ and regard $\int _{s^\prime _{\mathrm{min}}}^{s^\prime _{\mathrm{max}}} \mathrm {d} s^\prime (\Sigma -s^\prime )^2$ as a factor which is dependent on s.

2.3 Calculational details

With the Hamiltonian given in Eq. (4), we are ready to evaluate the matrix elements for $B^-\rightarrow \rho ^0(\omega )\pi ^-$. According to the theory of QCD factorization, the naive factorization approach has been shown to be the leading order result in the framework of QCD factorization when the radiative QCD corrections of order $\mathcal O(\alpha _s(m_b))$ and the $\mathcal O(1/m_b)$ corrections in the heavy quark effective theory are neglected [24–27]. Since the b quark is very heavy and B meson decays are very energetic, so the quark–antiquark pair in a meson in the final state moves very fast away from the weak interaction point. The hadronization of the quark–antiquark pair occurs far away from the remaining quarks. Then the meson can be factorized out and the interaction between the quark pair in the meson and the remaining quark is tiny [37, 38]. The deviation of the value of $N_c$ from the color number, 3, measures the nonfactorizable effects in the naive factorization scheme [16–18]. In the factorization approximation, either $\rho ^0(\omega )$ or $\pi ^-$ is generated by one current which has appropriate quantum numbers in the Hamiltonian. For this decay process, two kinds of matrix element products are involved after factorization: $\langle \rho ^0(\omega )|J^\mu |0\rangle \langle \pi ^-|J_\mu |B^-\rangle$ and $\langle \pi ^-|J^\mu |0\rangle \langle \rho ^0(\omega )|J_\mu |B^-\rangle$. We will calculate them in some phenomenological quark models.

The matrix elements for $B\rightarrow P$ and $B\rightarrow V$ (where P and V denote pseudoscalar and vector mesons, respectively) can be decomposed as [39]

$$\begin{aligned} \langle P| J_\mu |B\rangle= & {} \Bigg ( p_B+p_P-\frac{m_B^2-m_P^2}{k^2}\Bigg )_\mu F_1^{BP}(k^2)\nonumber \\&+\,\frac{m_B^2-m_P^2}{k^2}k_\mu F_0^{BP}(k^2),\end{aligned}$$

(19)

$$\begin{aligned} \langle V|J_\mu |B\rangle= & {} \frac{2}{m_B+m_{V}}\varepsilon _{\mu \nu \rho \sigma }\epsilon ^{*\nu }p_B^\rho p^\sigma _{V}V^{BV}(k^2)\nonumber \\&+\,\text {i}\Bigg \{\epsilon ^*_\mu (m_B+m_{V})A_1^{BV}(k^2) \nonumber \\&-\frac{\epsilon ^* \cdot k}{m_B+m_{V}}(p_B+p_{V})_\mu A_2^{BV} (k^2)\nonumber \\&-\frac{\epsilon ^*\cdot k}{k^2}2m_{V}\cdot k_\mu A_3^{BV}(k^2)\Bigg \}\nonumber \\&+\,\text {i}\frac{\epsilon ^*\cdot k}{k^2}2m_{V}\cdot k_\mu A^{BV}_0(k^2), \end{aligned}$$

(20)

where $J_\mu$ is the weak current [$J_\mu =\bar{u} \gamma _\mu (1-\gamma _5)b$ or $\bar{d} \gamma _\mu (1-\gamma _5)b$], $k=p_B-p_{P(V)}$, and $\epsilon _\mu$ is the polarization vector of V. The form factors included in our calculations satisfy $F_1^{BP}(0)=F_0^{BP}(0)$, $A_3^{BV}(0)=A_0^{BV}(0)$, and $A_3^{BV}(k^2)=[(m_B+m_{V})/2m_{V}]A_1^{BV}(k^2)-[(m_B-m_{V})/2m_{V}]A_2^{BV}(k^2)$. We define the notation X for matrix elements. For example, $X^{(B^-\rho ^0,\pi ^-)}$ is defined as $\langle \pi ^-|\bar{d} \gamma ^\mu (1-\gamma _5) u|0\rangle \langle \rho ^0|\bar{u} \gamma _\mu (1-\gamma _5)b|B^-\rangle$. These matrix elements can be parameterized as the products of decay constants and form factors. Therefore, the factorized terms $X^{(BM_1,M_2)}$ have the following expressions [40]:

$$\begin{aligned} X^{(BP,V)}= & {} \langle V|\bar{q}_2 \gamma ^\mu (1-\gamma _5)q_3|0\rangle \langle P|\bar{q}_1 \gamma _\mu (1-\gamma _5)b|B\rangle \nonumber \\= & {} 2f_V m_V F_1^{BP}(m^2_V)(\epsilon ^*\cdot p_B), \end{aligned}$$

(21)

$$\begin{aligned} X^{(BV,P)}= & {} \langle P|\bar{q}_2 \gamma ^\mu (1-\gamma _5)q_3|0\rangle \langle V|\bar{q}_1 \gamma _\mu (1-\gamma _5)b|B\rangle \nonumber \\= & {} 2f_P m_V A_0^{BV}(m^2_P)(\epsilon ^*\cdot p_B), \end{aligned}$$

(22)

where $f_V$ and $f_P$ are the decay constants of vector and pseudoscalar mesons, respectively. Using the decomposition in Eqs. (19)–(22), one has

$$\begin{aligned} t_\rho= & {} V_{ub}V^*_{ud}\Bigg (a_1\frac{X^{(B^-\rho ^0,\pi ^-)}}{\epsilon ^*\cdot p_B}+a_2\frac{X^{(B^-\pi ^-,\rho ^0)}}{\epsilon ^*\cdot p_B}\Bigg ),\end{aligned}$$

(23)

$$\begin{aligned} t_\omega= & {} V_{ub}V^*_{ud}\Bigg (a_1\frac{X^{(B^-\omega ,\pi ^-)}}{\epsilon ^*\cdot p_B}+a_2\frac{X^{(B^-\pi ^-,\omega )}}{\epsilon ^*\cdot p_B}\Bigg ), \end{aligned}$$

(24)

where all the $a_i$ are built up from the effective Wilson coefficients $c_i^\prime$’s, and take the form $a_i=c^\prime _i+c^\prime _{i+1}/N_c$ for odd i and $a_i=c^\prime _i+c^\prime _{i-1}/N_c$ for even i. It is noted that in the factorization approach $N_c$ includes nonfactorizable contributions effectively, and the value of $N_c$ should be determined by experiments since we cannot evaluate nonfactorizable contributions. In the same way, we obtain the penguin operator contributions:

$$\begin{aligned} p_\rho= & {} -V_{tb}V^*_{td}\Bigg \{\Bigg [-a_4+\frac{3}{2}a_7+\frac{3}{2}a_9+\frac{1}{2}a_{10}\Bigg ]\frac{X^{(B^-\pi ^-,\rho ^0)}}{\epsilon ^*\cdot p_B}\nonumber \\&+\Bigg [a_4+a_{10}-2(a_6+a_8)\frac{m^2_\pi }{(m_d+m_u)(m_b+m_u)}\Bigg ]\nonumber \\&\times \frac{X^{(B^-\rho ^0,\pi ^-)}}{\epsilon ^*\cdot p_B} \Bigg \},\end{aligned}$$

(25)

$$\begin{aligned} p_\omega= & {} -V_{tb}V^*_{td}\Bigg \{\Bigg [2a_3+a_4+2a_5+\frac{1}{2}(a_7+a_9-a_{10})\Bigg ]\nonumber \\&\times \frac{X^{(B^-\pi ^-,\omega )}}{\epsilon ^*\cdot p_B}\nonumber \\&+\Bigg [a_4+a_{10}-2(a_6+a_8)\frac{m^2_\pi }{(m_d+m_u)(m_b+m_u)}\Bigg ]\nonumber \\&\times \frac{X^{(B^-\omega ,\pi ^-)}}{\epsilon ^*\cdot p_B} \Bigg \}. \end{aligned}$$

(26)

We adopt the same decay constants and form factors for the matrix elements producing $\rho ^0$ and $\omega$ mesons. Then we have $X^{(B^-\rho ^0,\pi ^-)}=X^{(B^-\omega ,\pi ^-)}=2f_\pi m_\rho A_0(m_\pi ^2)(\epsilon ^* \cdot p_B)$ and $X^{(B^-\pi ^-,\rho ^0)}=X^{(B^-\pi ^-,\omega )}=\sqrt{2} f_\rho m_\rho F_1(m_\rho ^2)(\epsilon ^* \cdot p_B)$, where $\langle \rho ^0(\omega )|J^\mu |0\rangle =1/\sqrt{2}f_\rho m_\rho \epsilon ^{*\mu }$ and $\langle \pi ^-|J_\mu |0\rangle =\text {i}f_\pi p_\mu$

2.4 Numerical results

In our numerical calculations we have several parameters: $q^2$, $N_c$, and the CKM matrix elements in the Wolfenstein parametrization. As mentioned in Sect. 2.1, the value of $q^2$ is conventionally chosen to be in the range $0.3<q^2/m_b^2<0.5$. The CKM matrix, which should be determined from the experimental data, has the following form in terms of the Wolfenstein parameters, A, $\lambda$, $\rho$, and $\eta$ [41]:

$$\begin{aligned} \left( \begin{array}{ccc} 1-\tfrac{1}{2}\lambda ^2 &{} \lambda &{}A\lambda ^3(\rho -\mathrm {i}\eta ) \\ -\lambda &{} 1-\tfrac{1}{2}\lambda ^2 &{}A\lambda ^2 \\ A\lambda ^3(1-\rho -\mathrm {i}\eta ) &{} -A\lambda ^2 &{}1\\ \end{array} \right) , \end{aligned}$$

(27)

where $O(\lambda ^4)$ corrections are neglected. Since $\lambda$ and A are well determined and the uncertainties due to the CKM matrix elements are mostly from $\rho$ and $\eta$, we take the central values of $\lambda (=0.225)$ and $A(=0.814)$ in the following. The ranges for $\rho$ and $\eta$ are

$$\begin{aligned} \bar{\rho }=0.117\pm 0.021,\quad \bar{\eta }|=0.353\pm 0.013, \end{aligned}$$

(28)

with

$$\begin{aligned} \bar{\rho }=\rho \Bigg (1-\frac{\lambda ^2}{2}\Bigg ),\quad \bar{\eta }=\eta \Bigg (1-\frac{\lambda ^2}{2}\Bigg ). \end{aligned}$$

The form factors $F_1(m^2_\rho )$ and $A_0(m^2_\pi )$ depend on the inner structure of the hadrons. Under the nearest pole dominance assumption, we take the following $k^2$ dependence of the form factors: for Model 1(2) [42–47]:

$$\begin{aligned} F_1(k^2)=\frac{h_1}{1-\frac{k^2}{m_1^2}},\quad A_0(k^2)=\frac{h_{A_0}}{1-\frac{k^2}{m_{A_0}^2}}, \end{aligned}$$

(29)

where $h_1=0.25(0.292)$, $h_{A_0}=0.30(0.366)$, $m_1=5.32\,\mathrm {GeV}$, and $m_{A_0}=5.27\,\mathrm {GeV}$; for Model 3(4) [43–48]:

$$\begin{aligned} F_1(k^2)=\frac{h_1}{\bigg (1-\frac{k^2}{m_1^2}\bigg )^2},\quad A_0(k^2)=\frac{h_{A_0}}{\bigg (1-\frac{k^2}{m_{A_0}^2}\bigg )^2}, \end{aligned}$$

(30)

where $h_1=0.25(0.292)$, $h_{A_0}=0.30(0.366)$, $m_1=5.32\,\mathrm {GeV}$, and $m_{A_0}=5.27\,\mathrm {GeV}$; for Model 5 [49, 50]:

$$\begin{aligned} \begin{aligned} F_1(k^2)&=\frac{h_1}{1-x_1\frac{k^2}{m_1^2}+y_1\bigg (\frac{k^2}{m_1^2}\bigg )^2},\\ A_0(k^2)&=\frac{h_{A_0}}{1-x_0\frac{k^2}{m_{A_0}^2}+y_0\bigg (\frac{k^2}{m_{A_0}^2}\bigg )^2}, \end{aligned} \end{aligned}$$

(31)

where $h_1=0.261$, $h_{A_0}=0.302$, $x_1=2.03$, $y_1=1.29$, $x_0=-1.49$, $y_0=6.61$, $m_1=5.32\,\mathrm {GeV}$, and $m_{A_0}=5.27\,\mathrm {GeV}$. The decay constants used in our calculations are $f_\rho =216\,\mathrm {MeV}$ and $f_\pi =132\,\mathrm {MeV}$ [47].

In the numerical calculations, it is found that for a fixed $N_c$, there is a maximum value for the differential CP violating parameter, when the low invariant mass of the $\pi ^+\pi ^-$ pair is in the vicinity of the $\omega$ resonance, $0.780-0.785\,\mathrm {GeV}$. Five models with different form factors were investigated to study the model dependence of $A_{\mathrm{CP}}$ in Ref. [18]. To be more specific, in Fig. 1a, b we display the results for the form factors in Model 1. These results show explicitly the dependence of the CP violating asymmetry on $q^2/m_b^2$, CKM matrix elements and the effective parameter $N_c$. The dependence on $N_c$ comes form the fact that $N_c$ is related to the hadronization effects, and consequently, we cannot exactly determine $N_c$ in our calculations. Therefore, we treat $N_c$ as a free effective parameter and take it in the range $2.07(2.09)<N_c<4.54(4.65)$ when $q^2/m_b^2=0.3(0.5)$ for reasons which will be explained later (Sect. 3).

Table 1 The localized integrated asymmetries $A^\Omega _{\mathrm{CP}}$ when $q^2/m_b^2=0.3(0.5)$. The first and second lines of each model corresponds to $A^\Omega _{\mathrm{CP}}$ with and without $\rho$–$\omega$ mixing in the region $0.750<\sqrt{s}<0.800$ GeV, respectively. The third and fourth lines of each model correspond to the low-mass region ($0.470<\sqrt{s}<0.770$ GeV) and the high-mass region ($0.770<\sqrt{s}<0.920$ GeV) near the resonance mass, respectively. The fifth and sixth lines of each model corresponds to $A^\Omega _{\mathrm{CP}}$ with and without $\rho$–$\omega$ mixing in the region $0.470<\sqrt{s}<0.920$ GeV, respectively. The second and third columns correspond to $N_c=2.07(2.09)$. The fourth and fifth columns correspond to $N_c=4.54(4.65)$. The second and fourth columns correspond to lower limiting values of the CKM matrix elements. The third and fifth columns correspond to upper limiting values of the CKM matrix elements

Fig. 1

The differential asymmetry, $A_{\mathrm{CP}}$, in Model 1. a For $q^2/m_b^2=0.3$: solid and dot lines correspond $N_c=2.78$ and $N_c=9.68$ with maximum CKM matrix elements, respectively. Dot-dashed and dashed line correspond $N_c=2.78$ and $N_c=9.68$ with minimum CKM matrix elements, respectively, b the same as a but for $q^2/m_b^2=0.5$ and $N_c=2.85$ and 9.02

Then we calculate the localized integrated CP asymmetries. According to Eq. (18), we integrate $A_{\mathrm{CP}}$ over the low invariant mass of $\pi ^+\pi ^-$ ($\sqrt{s}$) and obtain the localized integrated asymmetries $A_{\mathrm{CP}}^\Omega$. Considering the significant region of $\rho$–$\omega$ mixing, we choose the integration interval of $\sqrt{s}$ to be from 0.750 to 0.800 GeV. In order to compare with the newest result of the LHCb experiments, we also calculate $A_{\mathrm{CP}}^\Omega$ when $\sqrt{s}$ is in the low-mass region ($0.470<\sqrt{s}<0.770$ GeV), the high-mass region ($0.770<\sqrt{s}<0.920$ GeV) and the whole region ($0.470<\sqrt{s}<0.920$ GeV) near the $\rho ^0$ resonance [6]. The numerical results are displayed in Table 1. We also display $A^\Omega _{\mathrm{CP}}$ with and without $\rho$–$\omega$ mixing when $0.750<\sqrt{s}<0.800$ and $0.470<\sqrt{s}<0.920$ GeV in Table 1.

Table 1 shows that the values of $A^{\Omega }_{\mathrm{CP}}$ in our calculations vary from $-0.0752 (-0.0626)$ to $-0.0403 (-0.0290)$ corresponding to $q^2/m_b^2=0.3(0.5)$, in the regions of $N_c$, the CKM matrix elements, and the form factors in five models when $0.750<\sqrt{s}<0.800$ GeV. From Fig. 4 of Ref. [6], we can see the localized integrated CP asymmetries have different signs in $\cos \theta >0$ and $\cos \theta <0$ regions when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800\, \text {GeV}$. If one adds the events in these two experimental regions together, the total localized integrated CP asymmetry obtained from experiment becomes $-0.0294\pm 0.0285$ when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800\, \text {GeV}$. The values in our calculations agree with this experimental data. We stress that $A_{\mathrm{CP}}^\Omega$ in our calculations is always negative in this integration region and its sign is independent of the form factor models. We note that the signs of $A^\Omega _{\mathrm{CP}}$ are positive when $\rho$–$\omega$ mixing is not considered in this region. This indicates that $\rho$–$\omega$ mixing is vital for $A_{\mathrm{CP}}^\Omega$ to be negative in this region.

From Table 1, we can see $\rho$–$\omega$ mixing changes the sign of $A^\Omega _{\mathrm{CP}}$ from positive to negative. Figure 1a, b show that the peak of the differential asymmetry $A_{\mathrm{CP}}$ involving $\rho$–$\omega$ mixing is on the right of 0.770 GeV. Therefore, comparing with $A_{\mathrm{CP}}^\Omega$ in the range $0.470<\sqrt{s}<0.770$ GeV, the localized integrated CP asymmetries move toward the negative direction when $0.770<\sqrt{s}<0.920$ GeV due to $\rho$–$\omega$ mixing. This behavior is also independent of the form factor models. In fact, in our calculations the difference between the localized integrated CP asymmetries in the regions $0.470<\sqrt{s}<0.770$ and $0.770<\sqrt{s}<0.920$ GeV varies from 0.0076 to 0.0387. We add the experimental events for positive and negative $\cos \theta$ in the regions $0.470<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.770$ and $0.770<\sqrt{s}<0.920$ GeV and obtain $A^\Omega _{\mathrm{CP}}$ as $0.0508\pm 0.0171$ and $-0.0256\pm 0.0202$, respectively, from the data in Table IV of Ref. [6]. We can see they have opposite signs. After we combine these two regions together, $A^\Omega _{\mathrm{CP}}$ in the whole region $0.470<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.920$ GeV is $0.0173\pm 0.0130$ [6]. The values of $A^\Omega _{\mathrm{CP}}$ with $\rho$–$\omega$ mixing shown in Table 1 differ a lot between Model 1 (or 2, 3, 4) and Model 5. $A^\Omega _{\mathrm{CP}}$ is also sensitive to the choice of $q^2/m_b^2$. When $0.470<\sqrt{s}<0.770$ GeV, it appears that $A^\Omega _{\mathrm{CP}}$ varies from 0.0278 to 0.0353 in Models 1, 2, 3 and 4 for $q^2/m_b^2=0.5$ and $N_c=4.65$. This result is consistent with the experimental data. When $0.770<\sqrt{s}<0.920$ GeV, except for $q^2/m_b^2=0.5$ and $N_c=4.65$ in Models 1, 2, 3 and 4, $A^\Omega _{\mathrm{CP}}$ varies from $-0.0408$ to $-0.0110$. This result is consistent with the experimental data. In the whole region of $0.470<\sqrt{s}<0.920$ GeV, $A^\Omega _{\mathrm{CP}}$ varies from 0.0096 to 0.0146 in Models 1, 2, 3, and 4 for $q^2/m_b^2=0.5$ and $N_c=4.65$. This result is also consistent with the experimental data.

From above discussions, we can see $A^{\Omega }_{\mathrm{CP}}$ with $\rho$–$\omega$ mixing, especially the signs, agree with the experimental result when $0.750<\sqrt{s}<0.800$ GeV. We also find that the localized integrated CP asymmetries move toward the negative direction due to $\rho$–$\omega$ mixing. $A^{\Omega }_{\mathrm{CP}}$ in the region $0.770<\sqrt{s}<0.920$ GeV contains the contribution of $\rho$–$\omega$ mixing, while that in the region $0.470<\sqrt{s}<0.770$ GeV does not. Therefore, $\rho$–$\omega$ mixing contributes to the sign change of CP asymmetry around the $\rho ^0(770)$ peak of $m(\pi ^+\pi ^-)_{\mathrm{low}}$. Our results in these two regions and the whole region are consistent with the experimental data for several choices of $q^2/m^2$ and models of the form factors. One should take the effect of $\rho$–$\omega$ mixing into account in order to answer the question why the sign of CP asymmetry changes around the $\rho ^0(770)$ peak of $m(\pi ^+\pi ^-)_{\mathrm{low}}$.

3 Extraction of $N_c$ from data of branching ratios

3.1 Formalism

As mentioned before, $N_c$ includes nonfactorizable effects which are difficult to deal with at present. Therefore, we treat $N_c$ as an effective parameter to be determined by the experimental data. With the factorized decay amplitudes, we can calculate the decay rates by using the following expression [40]:

$$\begin{aligned} \Gamma (B \rightarrow VP)=\frac{|\mathbf {p}_\rho |^3}{8\pi m^2_V}\Bigg |\frac{A(B \rightarrow VP)}{\epsilon ^*\cdot p_B}\Bigg |^2, \end{aligned}$$

(32)

where

$$\begin{aligned} |\mathbf {p}_\rho |=\frac{\sqrt{[m_B^2-(m_V+m_P)^2][m_B^2-(m_V-m_P)^2]}}{2m_B} \end{aligned}$$

(33)

is the c.m. momentum of the decay particles, and $A(B\rightarrow VP)$ is the decay amplitude. In our case, to be consistent, we should also take into account the $\rho$–$\omega$ mixing contribution when we calculate the branching ratio since we are working to the first order of isospin violation. Explicitly, for $B^+\rightarrow \rho ^0\pi ^+$, we obtain

$$\begin{aligned}&BR(B^+\rightarrow \rho ^0\pi ^+)\nonumber \\&\quad =\frac{G_F^2|\mathbf {p}_\rho |^3}{16\pi m_{\rho }^2 \Gamma _{B^+}}\Bigg |[V_{ud}V^*_{ub}A^T_{\rho ^0}-V_{td}V^*_{tb}A^P_{\rho ^0}]/(\epsilon ^* \cdot p_B)\nonumber \\&\qquad +[V_{ud}V^*_{ub}A^T_{\omega }-V_{td}V^*_{tb}A^P_{\omega }]/(\epsilon ^*\cdot p_B)\nonumber \\&\qquad \times \frac{\tilde{\Pi }_{\rho \omega }}{(s_\rho -m_\omega ^2)+\text {i}m_\omega \Gamma _\omega }\Bigg |^2, \end{aligned}$$

(34)

where the tree and penguin amplitudes are

$$\begin{aligned} A^T_{\rho ^0}= & {} a_1X^{(B^+\rho ^0,\pi ^+)}+a_2X^{(B^+\pi ^+,\rho ^0)},\nonumber \\ A^P_{\rho ^0}= & {} \Bigg (-a_4+\frac{3}{2}a_7+\frac{3}{2}a_9+\frac{1}{2}a_{10}\Bigg )X^{(B^+\pi ^+,\rho ^0)}\nonumber \\&+\Bigg [a_4+a_{10}-2(a_6+a_8)\frac{m^2_{\pi }}{(m_u+m_d)(m_b+m_u)}\Bigg ]\nonumber \\&\qquad \qquad \times \, X^{(B^+\rho ^0,\pi ^+)},\nonumber \\ A^T_{\omega }= & {} a_1X^{(B^+\omega ,\pi ^+)}+a_2X^{(B^+\pi ^+,\omega )},\nonumber \\ A^P_{\omega }= & {} \Bigg [2a_3+a_4+2a_5+\frac{1}{2}(a_7+a_9-a_{10})\Bigg ]X^{(B^+\pi ^+,\omega )}\nonumber \\&+\Bigg [a_4+a_{10}-2(a_6+a_8)\frac{m^2_{\pi }}{(m_u+m_d)(m_b+m_u)}\Bigg ]\nonumber \\&\qquad \qquad \times X^{(B^+\omega ,\pi ^+)}. \end{aligned}$$

(35)

For $B^0\rightarrow \rho ^+\pi ^-$, we obtain

$$\begin{aligned}&BR(B^0\rightarrow \rho ^+\pi ^-)=\frac{G_F^2|\mathbf {p}_\rho |^3}{16\pi m_{\rho }^2 \Gamma _{B^0}}\nonumber \\&\qquad \times |(V_{ub}V^*_{ud}A^T_{\rho ^+}-V_{tb}V^*_{td}A^P_{\rho ^+})/(\epsilon ^* \cdot p_B)|^2, \end{aligned}$$

(36)

where

$$\begin{aligned} A^T_{\rho ^+}= & {} a_1X^{(B^0\pi ^+,\rho ^-)},\nonumber \\ A^P_{\rho ^+}= & {} (a_4+a_{10})X^{(B^0\rho ^-,\pi ^+)}, \end{aligned}$$

(37)

where $X^{(B^0\rho ^+,\pi ^-)}\!=\!2f_\pi m_\rho A_0(m_\pi ^2)(\epsilon ^* \cdot p_B)$ and $X^{(B^0\pi ^-,\rho ^+)}=\sqrt{2} f_\rho m_\rho F_1(m_\rho ^2)(\epsilon ^* \cdot p_B)$.

3.2 Numerical results

The latest experimental data of branching ratios of $B^+\rightarrow \rho ^0\pi ^+$ and $B^0\rightarrow \rho ^+\pi ^-$ from the Particle Data Group (PDG) are [41]

$$\begin{aligned} BR(B^+\rightarrow \rho ^0\pi ^+)= & {} (8.3\pm 1.2)\times 10^{-6},\\ BR(B^0\rightarrow \rho ^+\pi ^-)= & {} (2.3\pm 0.23)\times 10^{-5}. \end{aligned}$$

We can determine the range of $N_c$ by comparing the theoretical values of the branching ratios with those of two-body decay channels. We calculate the branching ratios with the formula given in Eqs. (32), (34), and (36) in five models for the weak form factors which are mentioned in the previous subsection. In Fig. 2, we show the results for $B^+\rightarrow \rho ^0\pi ^+$ in Model 1 as an example. The numerical results are sensitive to uncertainties coming from the experimental data. In addition, the branching ratio also depends on the CKM matrix elements which are parameterized by $\lambda$, A, $\rho$, and $\eta$. In the allowed ranges for the parameters $\rho$ and $\eta$, we obtain the range of $N_c$. We summarize the allowed range of $N_c$ in Table 2. It is found that the experimental data constrain the value of $N_c$ into two regions. We note that if $N_c$ approaches to zero, nonperturbative effects would be very large. Considering this, we drop the range $N_c<1$. Therefore, $N_c$ could be in the range $2.07(2.09)<N_c<4.54(4.65)$ for $q^2/m_b^2=0.3(0.5)$. These values have been used in Sect. 2.

Fig. 2

The branching ratio for $B^+\rightarrow \rho ^0 \pi ^+$ in Model 2. Solid (dashed) line stands for $q^2/m_b^2=0.3$ and maximum (maximum) CKM matrix elements. Dot (dot-dashed) line stands for $k^2/m_b^2=0.5$ and maximum (minimum) CKM matrix elements

4 Conclusion and discussion

The first aim of the present work is to study the localized integrated CP asymmetry for the decays $B^\pm \rightarrow \rho ^0(\omega )\pi ^\pm \rightarrow \pi ^+\pi ^-\pi ^\pm$ with the inclusion of $\rho$–$\omega$ mixing. The second aim is to study the sign change caused by $\rho$–$\omega$ mixing.

In the calculation of CP violating asymmetry parameters, we need the Wilson coefficients for the tree and penguin operators at the scale $m_b$. We worked with the renormalization scheme independent Wilson coefficients. One of the major uncertainties in our calculations is due to the fact that hadronic matrix elements of both tree and penguin operators involve nonperturbative QCD effects. We worked in the factorization approximation, with $N_c$ being treated as an effective parameter to include nonfactorizable contributions. We compared our theoretical results with the latest experimental data from PDG to determine the range of $N_c$ as $2.07(2.09)<N_c<4.54(4.65)$ for $q^2/m_b^2=0.3(0.5)$. It has been pointed out that the factorization approach is quite reliable in energetic weak decays [51–53].

Table 2 Summary of the ranges of $N_c$ which are determined from the experimental data for various models and input parameters [numbers outside (inside) brackets are for $q^2/m_b^2=0.3(0.5)$]. The notation (number, number) means the lower and upper limits for $N_c$. $(-, -)$ means that there is no range of $N_c$ which is consistent with the experimental data

We explicitly showed that the CP violating asymmetry is very sensitive to $N_c$, the CKM matrix elements and the form factors. There is a maximum value for the differential CP violating parameter when the low invariant mass of the $\pi ^+\pi ^-$ pair is near the vicinity of the $\omega$ resonance, $0.780-785\,\mathrm {GeV}$. We determined the range of the localized integrated CP asymmetry with and without $\rho$–$\omega$ mixing in the ranges of $N_c$, the CKM matrix elements, and $q^2/m_b^2$. For all the models investigated, we found that the localized integrated CP violating asymmetry with $\rho$–$\omega$ mixing varies from $-0.0403 (-0.0290)$ to $-0.0752 (-0.0626)$ corresponding to $q^2/m_b^2=0.3(0.5)$ when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800\, \text {GeV}$. If one adds the events in the $\cos \theta <0$ and $\cos \theta >0$ experimental regions together when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800\, \text {GeV}$, the total localized integrated CP asymmetry is $-0.0294\pm 0.0285$. Our results, especially the signs, agree with the experimental data when $0.750<m(\pi ^+\pi ^-)_{\mathrm{low}}<0.800\, \text {GeV}$. We note that the signs are positive in this region when $\rho$–$\omega$ mixing is not considered. This indicates that $\rho$–$\omega$ mixing is vital for $A_{\mathrm{CP}}^\Omega$ to be negative in this region. It was shown that the maximum localized integrated asymmetry in the range $0.750<\sqrt{s}<0.800$ GeV can reach $-0.0752$. We also found that $\rho$–$\omega$ mixing can make the localized integrated CP asymmetries move toward the negative direction, and therefore contributes to the sign change around the $\rho ^0(770)$ peak of $m(\pi ^+\pi ^-)_{\mathrm{low}}$. This behavior is independent of the form factor models. In our calculations, the difference between the localized integrated CP asymmetries in the regions $0.470<\sqrt{s}<0.770$ and $0.770<\sqrt{s}<0.920$ GeV varies from 0.0076 to 0.0387. Our results by including the $\rho$–$\omega$ mixing mechanism in these two regions and the whole region around the $\rho ^0(770)$ peak are consistent with the experimental results for some models of the form factors.

At this stage, we cannot explain the LHCb experimental data in the regions of positive and negative $\cos \theta$ individually. This is because three-body decays of heavy mesons are more complicated than two-body decays as they receive more contributions from different mechanisms, for example, nonresonants [54], the interference between intermediate resonances and final-state $KK\longleftrightarrow \pi \pi$ rescattering. We will investigate the angle distribution of $A_{\mathrm{CP}}^\Omega$ when considering both the $\rho$–$\omega$ mixing mechanism and the interference between different spin intermediate resonances [8]. We will also apply more accurate data in the future to further decrease the uncertainties in the calculations. With parameters with smaller uncertainties, we expect to be able to obtain the effects of $\rho$–$\omega$ mixing more precisely. This is important to interpret the angle distribution and the sign change of the CP asymmetry around the $\rho ^0(770)$ peak of $m^2(\pi ^+\pi ^-)_{\mathrm{low}}$ more accurately.