1 Introduction

In recent years, studies on B-meson decays have attracted increasing attention, as they enable the testing of the standard model (SM) and enrich the field of quantum chromodynamics (QCD). The three-body decays of the B meson involve both resonant and non-resonant contributions. Thus their calculations are more complicated than those for two-body decays. Because of mutual interferences between the resonant and non-resonant states, it is difficult to calculate them separately [1]. Based on the symmetry principles and factorization theorems, a few theoretical models for calculating the three-body decay have been developed. In this study, we have adopted the widely used perturbative QCD (pQCD) factorization approach [2,3,4,5]. The color-suppressed phenomenon occurs when a \(B^{0}_{s}\) meson decays into a kaon–pion pair and a charmonium. Thus, it is helpful to study the \(B^{0}_{s}\rightarrow \psi (2S,1D)K \pi \) decays. Recently, significant advances have been made in the research on heavy quarkonium generation mechanisms [6]. The Large Hadron Collider beauty experiment (LHCb) collaboration has detected the \(B^{0}_{s}\rightarrow \psi (2S)K ^{-}\pi ^{+}\) decay [7] and found that the main source of the decay branching ratio is the \(\overline{K }^{*}(892)^{0}\) resonance. These advances have allowed us to reliably calculate and test the \(B^{0}_{s}\rightarrow \psi (2S,1D)K \pi \) decays.

The pQCD factorization approach was proposed based on the \(k _{T}\) factorization theorem [8,9,10]. In this approach, a three-body problem can be simplified to a quasi-two-body problem by introducing two-hadron distribution amplitudes (DAs) [11, 12]. The predominant contributions in the decay process are from the parallel motion range, where the invariant mass of the double light meson pair is lower than \(O ({\bar{\Lambda }}M_{B})\), and \({\bar{\Lambda }}=M_{B}-m_{b}\) represents the mass difference between the B meson and the b-quark. Thus, the pQCD factorization formula for the three-body decay of the \(B^{0}_{s}\) meson can be generally described as [9, 10]

$$\begin{aligned} {{\mathcal {A}}}={{\mathcal {H}}}\otimes \phi _{B ^{0}_{s}} \otimes \phi _{h_{3}} \otimes \phi _{h_{1}h_{2}}, \end{aligned}$$
(1)

where the hard decay kernel, \({{\mathcal {H}}}\), represents the contribution of the Feynman diagram with only one gluon exchange in the leading order, which can be calculated using perturbation theory. The terms \(\phi _{B ^{0}_{s}}\), \(\phi _{h_{3}}\), and \(\phi _{h_{1}h_{2}}\) represent the wave functions of \(B^{0}_{s}\), \(h_3\), and the \(h_1h_2\) pair, respectively. They are considered as non-perturbative inputs, which can be constructed by extracting the relevant experimentally measured quantities or calculating them using the non-perturbative model.

Although the decay \(B^{0}_{s}\rightarrow \psi (3770)K \pi \) has not been observed experimentally, the mixing structure of \(\psi (3770)\) can be investigated by making a theoretical prediction for this decay channel. Since the charmonium mesons \(\psi (3686)\) and \(\psi (3770)\) are regarded as 2S–1D mixed states, the decays \(B^{0}_{s}\rightarrow \psi (2S)K \pi \) and \(B^{0}_{s}\rightarrow \psi (1D)K \pi \) should first be calculated, and then the fitting should be performed based on the 2S–1D mixing scheme to obtain the branching ratios of the decays \(B^{0}_{s}\rightarrow \psi (3686,3770)K \pi \). The \(\psi (1D)\) state denotes the orbital quantum number \(l=2\) and the principal quantum number \(n=1\), and \(\psi (2S)\) is the first radially excited state of the charmonium meson.

The 2S–1D mixing angle, \(\theta \), is related to the ratio of the lepton decay widths of \(\psi (3686)\) and \(\psi (3770)\) [13], and its value can be obtained from the fitting of the non-relativistic potential model [14,15,16]. The theoretical prediction for the \(B\rightarrow \psi (3770)K \) decay is in line with the experimental measurement when higher-twist effects are considered and the 2S–1D mixing angle of \(\theta =-(12\pm 2)^{\circ }\) has been adopted [17]. In addition, two mixing angle options, namely, \(\theta =(27\pm 2)^{\circ }\) and \(\theta =-(12\pm 2)^{\circ }\), have been offered [14,15,16]. Based on these views, \(\psi (3686)\) and \(\psi (3770)\) can be represented as follows [16, 18]:

$$\begin{aligned} \begin{aligned} \psi (3686)=\sin \theta |c{\bar{c}}(1D)\rangle +\cos \theta |c{\bar{c}}(2S)\rangle , \\ \psi (3770)=\cos \theta |c{\bar{c}}(1D)\rangle -\sin \theta |c{\bar{c}}(2S)\rangle . \end{aligned} \end{aligned}$$
(2)

The branching ratio is affected by the width of the resonant state and the interactions between the final-state meson pair, especially the direct CP violations. Hence, introducing an intermediate resonance, \(\overline{K }^{*0}\), is more appropriate [19,20,21]. We consider the contributions of the S-, P-, and D-wave resonances from the kaon–pion pair in the quasi-two-body decays \(B^{0}_{s}\rightarrow \psi (2S,1D)(\overline{K }^{*0}\rightarrow )K \pi \). In this work, the contributions of the following five intermediate resonances are included: \(\overline{K }^{*}_{0}(1430)^{0}\), \(\overline{K }^{*}(892)^{0}\), \(\overline{K }^{*}(1410)^{0}\), \(\overline{K }^{*}(1680)^{0}\), and \(\overline{K }^{*}_{2}(1430)^{0}\). According to Eq. (1), \(\phi _{h_{3}}\) denotes the wave functions of the charmonium \(\psi \), and \(\phi _{h_{1}h_{2}}\) represent the various partial-wave functions of the kaon–pion pair, such as S-wave \(\overline{K }^{*}_{0}(1430)^{0}\), P-wave \(\overline{K }^{*}(892)^{0}\), and D-wave \(\overline{K }^{*}_{2}(1430)^{0}\). We refer to the study by Rui and Wang [22] to obtain the information for the S-wave DAs. For the P-wave, there are three possible polarizations: longitudinal, parallel, and perpendicular amplitudes. Hence, we consider both the longitudinal and transverse polarization cases of the P-wave DAs. The P-wave DAs have been described analogously to the two-pion DAs [23], which include the longitudinal polarization fraction and the flavor-symmetry-breaking effect. At present, studies on the D-wave DAs are inadequate, and thus we have adopted the method used in the study by Rui et al. [24] to construct the D-wave DAs using a similar method of the \(K {} K \) pair.

The contents of this paper are organized as follows. In Sect. 2, we describe the computational framework and provide a list of the wave functions involved in this work. Expressions for the various decay amplitudes associated with the theoretical calculations are presented in Sect. 3. Section 4 presents the numerical results and the related discussion. The study is summarized in Sect. 5.

2 Computational framework

The weak-effective Hamiltonian of the \(B^{0}_{s}\rightarrow \psi (2S,1D)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) decays is expressed as [25]

$$\begin{aligned} \mathcal{H}_{\mathrm{eff}}= & {} \frac{G_{F}}{\sqrt{2}}\left\{ V^*_{cb}V_{cd}[C_{1}O_{1}+C_{2}O_{2}]\right. \nonumber \\&\left. -V^*_{tb}V_{td}\left[ \sum ^{10}_{i=3}C_{i}O_{i}\right] \right\} , \end{aligned}$$
(3)

where \(V^*_{cb}V_{cd}\) and \(V^{*}_{tb} V_{td}\) are the Cabibbo–Kobayashi–Maskawa (CKM) factors, \(O_{i}\) is the localized four-quark operator, and \(C_{i}\) is the Wilson coefficient corresponding to the quark operator.

Fig. 1
figure 1

Feynman diagrams of the \(B^{0}_{s}\rightarrow \psi (2S,1D)(\overline{K }^{*0}\rightarrow )K ^{-}\pi ^{+}\) decays

To simplify the calculation, we have chosen to describe the decay process in the light-cone coordinate system. Assuming that the initial state of the \(B^{0}_{s}\) meson is stationary, the charmonium \(\psi (2S,1D)\) and the \(K \pi \) pair move in the directions of the vectors \(v=(0,1,0_{T})\) and \(n=(1,0,0_{T})\), respectively. The Feynman diagrams of the decay are depicted in Fig. 1. \(p _{B}\), \(p \), and \(p _{3}\) represent the momenta of the \(B^0_{s}\), \(\overline{K }^{*0}\), and \(\psi (2S,1D)\) mesons, respectively.

$$\begin{aligned} \begin{aligned}&p _{B}=\frac{M_{B^{0}_{s}}}{\sqrt{2}}(1,1,0_{T}), \\&p =\frac{M_{B^{0}_{s}}}{\sqrt{2}}(1-r^2,\eta ,0_{T}), \\&p _{3}=\frac{M_{B^{0}_{s}}}{\sqrt{2}}(r^2,1-\eta ,0_{T}). \end{aligned} \end{aligned}$$
(4)

In addition, the momenta of the light quark corresponding to the \(B^0_{s}\), \(\overline{K }^{*0}\), and \(\psi (2S,1D)\) mesons, respectively, are as follows:

$$\begin{aligned}&k _{B}=\left( 0,\frac{M_{B^{0}_{s}}}{\sqrt{2}}x_{B},k _{BT}\right) ,\nonumber \\&k =\left( \frac{M_{B^{0}_{s}}}{\sqrt{2}}z(1-r^{2}),0,k _{T}\right) ,\nonumber \\&k _{3}=\left( \frac{M_{B^{0}_{s}}}{\sqrt{2}}r^{2}x_{3},\frac{M_{B^{0}_{s}}}{\sqrt{2}}(1-\eta )x_{3},k _{3T}\right) , \end{aligned}$$
(5)

where \(M_{B^{0}_{s}}\) represents the mass of the \(B^{0}_{s}\) meson, \(M_{\psi }\) is the mass of the charmonium \(\psi (2S,1D)\), \(r=\frac{M_\psi }{M_{B^{0}_{s}}}\), the variable \(\eta =\omega ^2/({M_{B^{0}_{s}}^2}-{M_\psi }^2)\), and \(\omega \) represents the invariant mass of the kaon–pion pair, which conforms to the relationship \(\omega ^2=p ^2\). \(x _{B}\), \(z \), and \(x _{3}\) are the proportions of the momenta of the spectator quark inside the \(B^{0}_{s}\), \(\overline{K }^{*0}\), and \(\psi (2S,1D)\) mesons, respectively, with values in the range of [0, 1].

Then, the momenta \(p _{1}\) and \(p _{2}\) are defined in the kaon–pion pair as follows:

$$\begin{aligned} \begin{aligned} p _{1}=(\zeta p ^{+},\eta (1-\zeta ) p ^{+},p _{1T}),\\ p _{2}=((1-\zeta )p ^{+},\eta \zeta p ^{+},p _{2T}). \end{aligned} \end{aligned}$$
(6)

The variable \(\zeta =\frac{p^{+}_{1}}{p^{+}}\) depicts the distribution of the longitudinal momentum of the kaon with \(p ^{2}_{1T}=p ^{2}_{2T}=(1-\zeta )\zeta \omega ^{2}\).

The \(B^{0}_{s}\) meson is considered a heavy-light model, and its wave function is expressed as [26,27,28]

$$\begin{aligned} \Phi _{B^{0}_{s}}=\frac{i }{\sqrt{2N_{c}}}({\not \!\!{p}}_{B}+M_{B^{0}_{s}}){\gamma _{5}}{\phi _{B_{s}}({x_{B},b_{B}})}. \end{aligned}$$
(7)

The DA \({\phi _{B_{s}}({x_{B},b_{B}})}\) is expressed as

$$\begin{aligned} \phi _{B_{s}}(x_{B},b_{B})= & {} N _{B}{{x_{B}}^2}(1-{x_{B}})^2\nonumber \\&\times \exp \bigg [-\frac{M^2_{B^{0}_{s}}{{x_{B}}^2}}{2\omega ^2_{B_{s}}}-\frac{1}{2}(\omega _{B_{s}}{b_{B}})^2\bigg ], \end{aligned}$$
(8)

where \(N _{B}\) is the normalization factor, and its value can be obtained using the normalization relation \(\int ^{1}_{0}{} \mathrm{d}x _{B}\phi _{B_{s}}(x_{B},b_{B}=0)=f _{B^{0}_{s}}/({2}{\sqrt{{2}{N }_{c}}})\). Here, the color factor \(N_c=3\), and we select the shape parameter \(\omega _{B_{s}}=0.50\pm 0.05\) GeV [29].

We have applied the wave function form described on the basis of the harmonic oscillator for the \(\psi (3686)\) and \(\psi (3770)\) states. This form has been successfully applied to many charmonium mesons, such as \(\psi (2S)\), \(\psi (3S)\), and \(J/\psi \) [18, 30,31,32]. The longitudinally and transversely polarized wave functions of \(\psi (2S)\) and \(\psi (1D)\) are expressed as follows [23, 31, 32]:

$$\begin{aligned} \begin{aligned}&\Phi ^{L}_{\psi }=\frac{1}{\sqrt{2N _{c}}}[M_{\psi }\not \!\!{\epsilon }_{3L}\psi ^L(x_{3},b_{3})+\not \!\!{\epsilon }_{3L}\not \!\! {p}_{3}\psi ^t(x_{3},b_{3})],\\&\Phi ^{T}_{\psi }=\frac{1}{\sqrt{2N _{c}}}[M_{\psi }\not \!\!{\epsilon }_{3T}\psi ^V(x_{3},b_{3})+\not \!\!{\epsilon }_{3T}\not \!\! {p}_{3}\psi ^T(x_{3},b_{3})], \end{aligned} \end{aligned}$$
(9)

where \(p _{3}\) represents the momentum of the \(\psi (2S,1D)\) meson and \(M_{\psi }\) is its mass. The longitudinal polarization vector \(\epsilon _{3L}=\frac{M_{B^0_{s}}}{\sqrt{2} M_{\psi }}(-{r^2},(1-\eta ),0_{T})\) and the transverse polarization vector \(\epsilon _{3T}=(0,0,1_{T})\). The twist-2 and twist-3 DAs are as follows [18, 31]:

$$\begin{aligned} \psi ^{L,T }(x _{3},b _{3})= & {} \frac{f _{(2S,1D)}}{2\sqrt{2N_{c}}}N^{L,T }{} x _{3}\overline{x }_{3} {{\mathcal {I}}}(x _{3})\nonumber \\&\times \exp \bigg [-x _{3}\overline{x }_{3}\frac{m _c}{w} \bigg [w^2b ^2_{3}+\bigg (\frac{x _{3}-\overline{x }_{3}}{2x _{3}\overline{x }_{3}}\bigg )^2\bigg ]\bigg ],\nonumber \\ \end{aligned}$$
(10)
$$\begin{aligned} \psi ^{t }(x _{3},b _{3})= & {} \frac{f _{(2S,1D)}}{2\sqrt{2N_{c}}}N^{t } ({x _{3}-\overline{x }_{3}})^2\mathcal{I}(x _{3})\nonumber \\&\times \exp \bigg [-x _{3}\overline{x }_{3}\frac{m _c}{w} \bigg [w^2b ^2_{3}+\bigg (\frac{x _{3}-\overline{x }_{3}}{2x _{3}\overline{x }_{3}}\bigg )^2\bigg ]\bigg ],\nonumber \\ \end{aligned}$$
(11)
$$\begin{aligned} \psi ^{V }(x _{3},b _{3})= & {} \frac{f _{(2S,1D)}}{2\sqrt{2N_{c}}}N^{V } [1+({x _{3}-\overline{x }_{3}})^2]\mathcal{I}(x _{3})\nonumber \\&\times \exp \bigg [-x _{3}\overline{x }_{3}\frac{m _c}{w} \bigg [w^2b ^2_{3}+\bigg (\frac{x _{3}-\overline{x }_{3}}{2x _{3}\overline{x }_{3}}\bigg )^2\bigg ]\bigg ],\nonumber \\ \end{aligned}$$
(12)

where \(\mathcal{I}(x _{3})=(\frac{1}{x _{3}\overline{x }_{3}}-m _{c}w b ^2_{3})(6x ^4_{3}-12x ^3_{3}+7x ^2_{3}-x _{3})-\frac{m _c(1-2x _{3})^2}{4w x _{3}\overline{x }_{3}}\) for \(\psi (1D)\) and \(\mathcal{I}(x _{3})=1-4{m _c}w {x _{3}\overline{x }_{3}}{} b ^2_{3}+\frac{m _c(1-2x _{3})^2}{w x _{3}\overline{x }_{3}}\) for \(\psi (2S)\). We have selected the shape parameters \(w_{1D}=0.5\pm 0.05\) GeV [18] and \(w_{2S}=0.2\pm 0.1\) GeV [31]. The normalization factor \(N^{i }(i =L,T,t,V )\) can be obtained using the normalization relationship \(\int ^1_0 \psi ^{i }(x _{3},b_{3}=0)\mathrm{d}{} x _{3}= \frac{f _{(2S,1D)}}{2\sqrt{2N_{c}}}\). Equations (10)–(12) are symmetric under the transformation of \(x _{3}\leftrightarrow \overline{x }_{3}\).

A form similar to the two-pion DA has been adopted for the S-wave of the kaon–pion pair DA [33]:

$$\begin{aligned} \Phi _{S}= & {} \frac{1}{\sqrt{2 N_{c}}}[\not \!\! {p}\phi ^{0}_{S}(z,\zeta ,\omega ^2)+\omega \phi ^{s}_{S}(z,\zeta ,\omega ^2) \nonumber \\&+\omega (\not \!\!{n}\not \!\! {v}-1)\phi ^{t}_{S}(z,\zeta ,\omega ^2)]. \end{aligned}$$
(13)

The subscripts S, P, and D denote the corresponding sub-waves, respectively, in the following description.

Using the description given by Wang et al. [34], the twist-2 DAs have been described in a form similar to the scalar meson [35, 36], whereas asymptotic forms for the twist-3 DAs have been adopted in this work. They can be expressed as follows:

$$\begin{aligned} \phi ^{0}_{S}(z,\zeta ,\omega ^2)= & {} \frac{6}{2\sqrt{2 N_{c}}}{} F _{S }(\omega ^2)(z-z^{2})\nonumber \\&\times \bigg [\frac{1}{\mu _{S }}+B _{1}{} C ^{3/2}_{1}(t)+B _{3}{} C ^{3/2}_{3}(t)\bigg ], \end{aligned}$$
(14)
$$\begin{aligned} \phi ^{s}_{S}(z,\zeta ,\omega ^2)= & {} \frac{1}{2\sqrt{2 N_{c}}}{} F _{S }(\omega ^2), \end{aligned}$$
(15)
$$\begin{aligned} \phi ^{t}_{S}(z,\zeta ,\omega ^2)= & {} \frac{1}{2\sqrt{2 N_{c}}}{} F _{S }(\omega ^2)(1-2z). \end{aligned}$$
(16)

The Gegenbauer polynomials are \(C ^{3/2}_{1}(t)=3t\) and \(C ^{3/2}_{3}(t)=\frac{5}{2}(7t^{3}-3t)\) with \(t=1-2z\). In addition, \(\mu _{S }=\frac{\omega }{m_{2}-m_{1}}\), \(m _{1}\), and \(m _{2}\) represent the corresponding current quark masses, and the Gegenbauer moments are \(B _{1}=-0.57\pm 0.13\) and \(B _{3}=-0.42\pm 0.22\) [35, 37, 38].

For the time-like scalar form factor, \(F _{S }(\omega ^2)\), we have adopted the parameterized fitting results of an improved LASS (Large Aperture Superconducting Solenoid) line type presented by Aston et al. [39]. \(F _{S }(\omega ^2)\) is expressed as [34]

$$\begin{aligned} F _{S }(\omega ^2)= & {} \frac{m^{2}_{0}\frac{\Gamma _{0}}{\mid {\overrightarrow{p_{0}}}\mid }}{m^{2}_{0}-\omega ^2-i m^{2}_{0}\frac{\Gamma _{0}}{\omega }\frac{\mid {\overrightarrow{p_{1}}}\mid }{\mid {\overrightarrow{p_{0}}}\mid }}{} e ^{2i \delta _{B }}+\frac{\omega }{\mid {\overrightarrow{p_{1}}}\mid [cot (\delta _{B })-i ]}, \end{aligned}$$
(17)
$$\begin{aligned} cot (\delta _{B })= & {} \frac{a{\mid {\overrightarrow{p_{1}}}\mid }}{2}+\frac{1}{l{\mid {\overrightarrow{p_{1}}}\mid }}. \end{aligned}$$
(18)

In Eq. (17), the first term contains the resonant contribution with a phase factor to maintain unitarity, and the second term is an empirical term of the elastic \(K \pi \) scattering. \(\Gamma _{0}\) and \(m_{0}\) represent the width and the pole mass, respectively, of the \(\overline{K }^{*}_{0}(1430)^{0}\) resonance, \(\mid {\overrightarrow{p_{1}}}\mid \) represents the momentum of the decay product of the intermediate resonance, and \(\mid {\overrightarrow{p_{0}}}\mid =\mid {\overrightarrow{p_{1}}}\mid \) is available when \(\omega =m_{K ^{*0}}\). \(a=(7.0\pm 2.4)\) \(\mathrm{GeV^{-1}}\) and \(l=(3.1\pm 1.0)\) \(\mathrm{GeV^{-1}}\) are the effective range and the scattering length, respectively, which are universal in describing the \(K \pi \) meson pair.

According to Li et al. [40], the P-wave kaon–pion DAs related to the longitudinal and transverse polarizations can be expressed as

$$\begin{aligned} \begin{aligned} \Phi ^{L}_{P}&=\frac{1}{\sqrt{2 N_{c}}}\bigg [\not \!\! {p}\phi ^{0}_{P}(z,\zeta ,\omega ^2)+\omega \phi ^{s}_{P}(z,\zeta ,\omega ^2)\\&\qquad +\frac{\not \!\!{p}_{1}\not \!\!{p}_{2}-\not \!\!{p}_{2}\not \!\!{p}_{1}}{\omega (2\zeta -1)}\phi ^{t}_{P}(z,\zeta ,\omega ^2)\bigg ],\\ \Phi ^{T}_{P}&=\frac{1}{\sqrt{2 N_{c}}}\bigg [\gamma _{5}\not \!\!{\epsilon }_{T}\not \!\! {p}\phi ^{T}_{P}(z,\zeta ,\omega ^2)+\omega \gamma _{5} \not \!\!{\epsilon }_{T} \phi ^{a}_{P}(z,\zeta ,\omega ^2)\\&\qquad +i \omega \frac{\epsilon ^{\mu \nu \rho \sigma }\gamma _{\mu }\epsilon _{T\nu }P_{\rho }n_{-\sigma }}{P\cdot n_{-}}\phi ^{v}_{P}(z,\zeta ,\omega ^2)\bigg ]. \end{aligned} \end{aligned}$$
(19)

The different twists in Eq. (19) when expanded using the Gegenbauer polynomial have the specific forms as follows:

$$\begin{aligned} \phi ^{0}_{P}(z,\zeta ,\omega ^2)= & {} \frac{3}{\sqrt{2 N_{c}}}{} F ^{\Vert }_{P }(\omega ^2)(z-z^{2})\nonumber \\&\times \bigg [1+3ta^{\Vert }_{1K ^{*}}+\frac{3}{2}(5t^{2}-1)a^{\Vert }_{2K ^{*}}\bigg ]\nonumber \\&(2\zeta -\alpha -1), \end{aligned}$$
(20)
$$\begin{aligned} \phi ^{s}_{P}(z,\zeta ,\omega ^2)= & {} \frac{3}{2\sqrt{2 N_{c}}}{} F ^{\bot }_{P } (\omega ^2)[t(1+ta^{\bot }_{1s})\nonumber \\&-(2z-2z^{2})a^{\bot }_{1s}](2\zeta -1), \end{aligned}$$
(21)
$$\begin{aligned} \phi ^{t}_{P}(z,\zeta ,\omega ^2)= & {} \frac{3}{2\sqrt{2 N_{c}}}{} F ^{\bot }_{P }(\omega ^2)[t^{2}+(3t^{3}-t)a^{\bot }_{1t}]\nonumber \\&(2\zeta -1), \end{aligned}$$
(22)
$$\begin{aligned} \phi ^{T}_{P}(z,\zeta ,\omega ^2)= & {} \frac{3}{\sqrt{2 N_{c}}}{} F ^{\bot }_{P }(\omega ^2)(z-z^{2})\nonumber \\&\times \bigg [1+3ta^{\bot }_{1K ^{*}}+\frac{3}{2}(5t^{2}-1)a^{\bot }_{2K ^{*}}\bigg ]\nonumber \\&\sqrt{\zeta -\zeta ^{2}}, \end{aligned}$$
(23)
$$\begin{aligned} \phi ^{a}_{P}(z,\zeta ,\omega ^2)= & {} \frac{3}{4\sqrt{2N_{c}}}{} F ^{\Vert } _{P }(\omega ^2) [t(1+ta^{\Vert }_{1a})\nonumber \\&-(2z-2z^{2})a^{\Vert }_{1a}]\sqrt{\zeta -\zeta ^{2}}, \end{aligned}$$
(24)
$$\begin{aligned} \phi ^{v}_{P}(z,\zeta ,\omega ^2)= & {} \frac{3}{8\sqrt{2 N_{c}}}{} F ^{\Vert }_{P }(\omega ^2)[1+t^{2}+t^{3}a^{\Vert }_{1v}]\nonumber \\&\sqrt{\zeta -\zeta ^{2}}. \end{aligned}$$
(25)

The SU(3) asymmetry factor \(\alpha =(m^{2}_{K ^{\pm }}-m^{2}_{\pi ^{\pm }})/\omega ^2\), and the Gegenbauer moments \(a^{\Vert }_{1K ^{*}}=0.2\pm 0.2\), \(a^{\Vert }_{2K ^{*}}=0.5\pm 0.5\), \(a^{\bot }_{1K ^{*}}=0.3\pm 0.3\), \(a^{\bot }_{2K ^{*}}=0.8\pm 0.8\), \(a^{\bot }_{1s}=-0.2\), \(a^{\bot }_{1t}=0.2\), \(a^{\Vert }_{1a}=-0.3\) and \(a^{\Vert }_{1v}=0.3\) [40] have been adopted in this work.

The time-like shape factor, \(F ^{\Vert }_{P }(\omega ^2)\), of the P-wave is expressed as [41]

$$\begin{aligned} F ^{\Vert }_{P }(\omega ^2)= & {} \frac{c_{1}m^{2}_{K ^{*}(892)^{0}}}{m^{2}_{K ^{*}(892)^{0}}-\omega ^2-i m_{K ^{*}(892)^{0}}\Gamma _{1}(\omega ^2)}\nonumber \\&+\frac{c_{2}m^{2}_{K ^{*}(1410)^{0}}}{m^{2}_{K ^{*}(1410)^{0}}-\omega ^2-i m_{K ^{*}(1410)^{0}}\Gamma _{2}(\omega ^2)}\nonumber \\&+\frac{c_{3}m^{2}_{K ^{*}(1680)^{0}}}{m^{2}_{K ^{*}(1680)^{0}}-\omega ^2-i m_{K ^{*}(1680)^{0}}\Gamma _{3}(\omega ^2)}. \end{aligned}$$
(26)

The three terms added together are derived from the \(K ^{*}(892)^{0}\), \(K ^{*}(1410)^{0}\), and \(K ^{*}(1680)^{0}\) resonant states, and their corresponding weight coefficients are \(c_{1}=0.72\), \(c_{2}=0.134\), and \(c_{3}=0.143\) [40], respectively.

The mass-related width is given by

$$\begin{aligned} \Gamma _{i }(\omega ^2)=\Gamma _{i } \bigg (\frac{m_{i }}{\omega }\bigg )\bigg (\frac{\mid {\overrightarrow{p_{1}}}\mid }{\mid {\overrightarrow{p_{0}}}\mid }\bigg ) ^{(2L _{R }+1)}, \end{aligned}$$
(27)

where \(\Gamma _{i }\) and \(m_{i }\) denote the width and the pole mass, respectively, of the corresponding resonance, and \(L _{R }\) represents the orbital angular momentum, with values of 0, 1, and 2 for the S-, P-, and D-wave, respectively. According to the study by Wang and Li [19], the following relation can be obtained

$$\begin{aligned} \frac{F ^{\bot }_{P }(\omega ^2)}{F ^{\Vert }_{P }(\omega ^2)}\approx \frac{f ^{T }_{K ^{*}}}{f _{K ^{*}}}, \end{aligned}$$
(28)

where \(f ^{T }_{K ^{*}}=0.185\pm 0.010\) GeV and \(f _{K ^{*}}=0.217\pm 0.005\) GeV [29]. We have adopted the procedure from the work by Li et al. [40]; studies on the decay constants of \(K ^{*}(1410)^{0}\) and \(K ^{*}(1680)^{0}\) are limited, and thus we have used the two decay constants of \(K ^{*}(892)^{0}\) to determine the ratio \(f ^{T }_{K ^{*}}/f _{K ^{*}}\).

A form similar to the two-kaon DAs has also been considered in the D-wave kaon–pion DAs [24]:

$$\begin{aligned} \begin{aligned} \Phi ^{L}_{D}&=\sqrt{\frac{2}{3}}\frac{1}{\sqrt{2 N_{c}}} \bigg [\not \!\! {p}\phi ^{0}_{D}(z,\zeta ,\omega ^2)+ \omega \phi ^{s}_{D}(z,\zeta ,\omega ^2)\\&\quad + \frac{\not \!\!{p}_{1}\not \!\!{p}_{2}-\not \!\!{p}_{2} \not \!\!{p}_{1}}{\omega (2\zeta -1)}\phi ^{t}_{D}(z,\zeta ,\omega ^2)\bigg ],\\ \Phi ^{T}_{D}&=\sqrt{\frac{1}{2}}\frac{1}{\sqrt{2 N_{c}}}\bigg [\gamma _{5}\not \!\!{\epsilon }_{T}\not \!\! {p}\phi ^{T}_{D}(z,\zeta ,\omega ^2)\\&\qquad +\omega \gamma _{5} \not \!\!{\epsilon }_{T} \phi ^{a}_{D}(z,\zeta ,\omega ^2)\\&\quad +i \omega \frac{\epsilon ^{\mu \nu \rho \sigma }\gamma _{\mu }\epsilon _{T\nu }P_{\rho }n_{-\sigma }}{P\cdot n_{-}}\phi ^{v}_{D}(z,\zeta ,\omega ^2)\bigg ], \end{aligned} \end{aligned}$$
(29)

where the coefficient \(\sqrt{\frac{2}{3}}(\sqrt{\frac{1}{2}})\) comes from the different definitions of the polarization vector between the vector and tensor mesons in the longitudinal (transverse) polarization.

The different twists in the D-wave DAs are [24, 42,43,44]

$$\begin{aligned} \phi ^{0}_{D}(z,\zeta ,\omega ^2)= & {} \frac{9}{\sqrt{2 N_{c}}}{} F ^{\Vert }_{D }(\omega ^2)(z-z^{2})(2z-1)\nonumber \\&\times a^{0}_{1} (1-6\zeta +6\zeta ^{2}), \end{aligned}$$
(30)
$$\begin{aligned} \phi ^{s}_{D}(z,\zeta ,\omega ^2)= & {} -\frac{9}{4\sqrt{2 N_{c}}}{} F ^{\bot }_{D }(\omega ^2) (1-6z+6z^{2})\nonumber \\&\times a^{0}_{1}(1-6\zeta +6\zeta ^{2}), \end{aligned}$$
(31)
$$\begin{aligned} \phi ^{t}_{D}(z,\zeta ,\omega ^2)= & {} \frac{9}{4\sqrt{2 N_{c}}}{} F ^{\bot }_{D }(\omega ^2) (2z-1)(1-6z+6z^{2})\nonumber \\&\times a^{0}_{1}(1-6\zeta +6\zeta ^{2}), \end{aligned}$$
(32)
$$\begin{aligned} \phi ^{T}_{D}(z,\zeta ,\omega ^2)= & {} \frac{9}{\sqrt{2 N_{c}}}{} F ^{\bot }_{D }(\omega ^2) (z-z^{2})(2z-1)\nonumber \\&\times a^{T}_{1}(2\zeta -1)\sqrt{\zeta -\zeta ^{2}}, \end{aligned}$$
(33)
$$\begin{aligned} \phi ^{a}_{D}(z,\zeta ,\omega ^2)= & {} \frac{3}{2\sqrt{2 N_{c}}}{} F ^{\Vert }_{D }(\omega ^2) (2z-1)^{3}\nonumber \\&\times a^{T}_{1}(2\zeta -1)\sqrt{\zeta -\zeta ^{2}}, \end{aligned}$$
(34)
$$\begin{aligned} \phi ^{v}_{D}(z,\zeta ,\omega ^2)= & {} -\frac{3}{2\sqrt{2 N_{c}}}{} F ^{\Vert }_{D }(\omega ^2) (1-6z+6z^{2})\nonumber \\&\times a^{T}_{1}(2\zeta -1)\sqrt{\zeta -\zeta ^{2}}. \end{aligned}$$
(35)

The Gegenbauer moments are \(a^{0}_{1}=0.4\pm 0.1\) and \(a^{T}_{1}=0.8\pm 0.2\), and a form similar to Eq. (26) has been adopted for the time-like shape factor, \(F ^{\Vert }_{D }(\omega ^2)\). Furthermore, the approximate relation \(F ^{\bot }_{D }(\omega ^2)/F ^{\Vert }_{D }(\omega ^2)\approx f ^{T }_{K ^{*}_{2}(1430)}/f _{K ^{*}_{2}(1430)}\) can also be found, with \(f ^{T }_{K ^{*}_{2}(1430)}=0.077\pm 0.014\) GeV and \(f _{K ^{*}_{2}(1430)}=0.118\pm 0.005\) GeV [42].

The differential decay ratios for the \(B^{0}_{s} \rightarrow \psi (2S,1D)K ^-\pi ^+\) decays in the \(B^{0}_{s}\) meson rest frame can be written as

$$\begin{aligned} \frac{d{\mathcal {B}}}{\mathrm{d}\omega }=\frac{\tau _{B^0_{s}}\omega \mid {\overrightarrow{p_{1}}} \mid \mid {\overrightarrow{p_{3}}} \mid }{32(\pi M_{B^0_{s}})^3} \sum _{i=0,\Vert ,\bot }\mid {{\mathcal {A}}}_{i} \mid ^2, \end{aligned}$$
(36)

where the three-momenta of \(K ^{-}\) and \(\psi (2S,1D)\) in the kaon–pion center-of-mass system are expressed as

$$\begin{aligned}&\mid {\overrightarrow{p_{1}}} \mid =\frac{1}{2\omega }\sqrt{\omega ^{4}+m^{4}_{K }+m^{4}_{\pi }-2(\omega ^{2}m^{2}_{K }+\omega ^{2}m^{2}_{\pi }+m^{2}_{K }m^{2}_{\pi })},\nonumber \\&\mid {\overrightarrow{p_{3}}} \mid =\frac{1}{2\omega }\sqrt{M^{4}_{B^0_{s}}+M^{4}_{\psi }+\omega ^{4}-2(M^{2}_{B^0_{s}}M^{2}_{\psi }+M^{2}_{B^0_{s}}\omega ^{2}+M^{2}_{\psi }\omega ^{2})}.\nonumber \\ \end{aligned}$$
(37)

The terms \({\mathcal {A}}_{0 }\), \({\mathcal {A}}_{\Vert }\), and \({\mathcal {A}}_{\bot }\) represent the longitudinal, parallel, and perpendicular polarization amplitudes, respectively. The related expressions are

$$\begin{aligned} \begin{aligned}&{\mathcal {A}}_{0 }={\mathcal {A}}_{L },\\&{\mathcal {A}}_{\Vert }=\sqrt{2 } {\mathcal {A}}_{N },\\&{\mathcal {A}}_{\bot }=\sqrt{2 } {\mathcal {A}}_{T }, \end{aligned} \end{aligned}$$
(38)

where the subscripts \(L \), \(N \), and \(T \) denote the longitudinal, normal, and transverse polarizations, respectively. The polarization fraction is defined as

$$\begin{aligned} f_{i}=\frac{\mid {{\mathcal {A}}}_{i} \mid ^2}{\mid \mathcal{A}_{0 }\mid ^2+\mid {{\mathcal {A}}}_{\Vert }\mid ^2+\mid \mathcal{A}_{\bot }\mid ^2}, \end{aligned}$$
(39)

with the normalization relation \(f_{0 }+f_{\Vert }+f_{\bot }=1\).

3 Decay amplitudes

Based on the pQCD approach, the decay amplitude of \(B^{0}_{s} \rightarrow \psi (2S,1D)K ^-\pi ^+\) is

$$\begin{aligned} {{\mathcal {A}}_{L ,N ,T }}= & {} \frac{G _{F }}{\sqrt{2}}\bigg [V^*_{cb}V_{cd} \big (a _{1}{} F ^{LL }_{L ,N ,T }+ C _{2}{} M ^{LL }_{L ,N , T }\big )-V^*_{tb}V_{td}\nonumber \\&\times \big (a _{2}{} F ^{LL }_{L ,N ,T }+a _{3}{} F ^{LR }_{L ,N ,T } +(C _{4}+C _{10})M ^{LL }_ {L ,N ,T }\nonumber \\&+(C _{6}+C _{8}) M ^{SP }_{L ,N ,T }\big )\bigg ], \end{aligned}$$
(40)

where \(F \) and \(M \) represent the factorization and non-factorization contributions, respectively. The superscripts \(LL \) and \(LR \) denote the weak vertices of the operators, and \(SP \) is the Fierz transformation of \(LR \). For the S-wave, the amplitude is only a longitudinal polarization. The total decay amplitudes of the P-wave and the D-wave are decomposed into

$$\begin{aligned} {\mathcal {A}}={\mathcal {A}}_{L }+{\mathcal {A}}_{N }\epsilon _{T }\cdot \epsilon _{3T }+i {\mathcal {A}}_{T }\epsilon _{\alpha \beta \rho \sigma } n ^{\alpha }_{+} n ^{\beta }_{-} \epsilon ^{\rho }_{T } \epsilon ^{\sigma }_{3T }. \end{aligned}$$
(41)

The decay amplitudes of the longitudinal polarization are as follows:

$$\begin{aligned} {F }^{LL }_{L }(S )= & {} 8\pi C _{F } f_{\psi }M^{4}_{B^0_{s}}\nonumber \\&\times \int ^{1}_{0}{\text {d}} x _{B}{\text {d}}{} z \int ^\infty _{0}{} b _{B}{} b {\text {d}}{} b _{B}{\text {d}}{} b \phi _{B_{s}}(x _{B},b_{B})\nonumber \\&\times \{[((1-\eta )(1+(1-2r^2)z)-r^2) \phi ^{0}_{S}\nonumber \\&+ \sqrt{(1-r^2)\eta } [(1-2z-\eta +2\eta z\nonumber \\&-(1-2z+2\eta z)r^2)\nonumber \\&\times ( \phi ^{s}_{S}+ \phi ^{t}_{S}) +2r^2 \phi ^{t}_{S}]] \times \alpha _{s}(t_{a})\nonumber \\&\exp [-S_{B^{0}_{s}}(t_{a})- S_{M}(t_{a})]\nonumber \\&\times S_{t}(z)h_{a}(x_{B}, z, b_{B}, b)\nonumber \\&+[[\eta ^{2}-\eta +(\eta -x_{B})r^2](1-r^2)\phi ^{0}_{S}\nonumber \\&+2\sqrt{(1-r^2)\eta }[1-\eta -r^2(1-x_{B})]\phi ^{s}_{S}]\nonumber \\&\times \alpha _{s}(t_{b})\exp [-S_{B^{0}_{s}}(t_{b})-S_{M}(t_{b})]S_{t}\nonumber \\&(|x_{B}-\eta |)h_{b}(x_{B }, z, b_{B }, b)\}, \end{aligned}$$
(42)
$$\begin{aligned} {F }^{LR }_{L }(S )= & {} {F }^{LL }_{L }(S ), \end{aligned}$$
(43)
$$\begin{aligned} {M }^{LL }_{L }(S )= & {} \frac{-32\pi C _{F }M^{4}_{B^{0}_{s}}}{\sqrt{2N_{c}}}\int ^{1}_{0}{\text {d}} x _{B}{\text {d}}{} z {\text {d}}{} x _{3}\nonumber \\&\times \int ^\infty _{0}{} b _{B}{} b _{3}{\text {d}}{} b _{B}{\text {d}}{} b _{3}\phi _{B_{s}}(x _{B},b_{B})\nonumber \\&\times \{[(1-r^2-\eta )[((1-x_{B}-x_{3})\nonumber \\&(1-r^2)+\eta ((1-2r^2)x_{3}\nonumber \\&-1+z-zr^2))\psi ^{L}(x_{3},b_{3})\nonumber \\&+(1-\eta )rr_{c}\psi ^{t}(x_{3},b_{3})]\phi ^{0}_{S}\nonumber \\&+\sqrt{(1-r^2)\eta }[((1-r^2)z+2(1-x_{3})r^2\nonumber \\&-x_{B}r^2)(1-\eta )\phi ^{t}_{S}\nonumber \\&-((1-r^2)z(1-\eta )+x_{B}r^2)\phi ^{s}_{S}]\psi ^{L}(x_{3},b_{3})]\nonumber \\&\times \alpha _{s}(t_{c})\exp [-S_{B^{0}_{s}}(t_{c})-S_{M}(t_{c})-S_{\psi }(t_{c})]\nonumber \\&h_{c}(x_{B}, z, x_{3}, b_{B}, b_{3})\nonumber \\&+[(1-r^2-\eta )[(x_{B}-z+zr^2\nonumber \\&-(1+r^2-\eta )x_{3})\psi ^{L}(x_{3},b_{3})\nonumber \\&+(1-\eta )rr_{c} \psi ^{t}(x_{3},b_{3})]\phi ^{0}_{S}-\sqrt{(1-r^2)\eta } [((x_{B}r^2\nonumber \\&-(2x_{3}r^2+(1-r^2)z)(1-\eta ))\psi ^{L} (x_{3},b_{3})\nonumber \\&+4(1-\eta )r r_{c}\psi ^{t}(x_{3},b_{3}))\phi ^{t}_{S}\nonumber \\&-(x_{B}r^2+(1-\eta )z(1-r^2))\psi ^{L}(x_{3},b_{3})\phi ^{s}_{S}]]\nonumber \\&\times \alpha _{s}(t_{d})\exp [-S_{B^{0}_{s}}(t_{d})-S_{M}(t_{d})\nonumber \\&-S_{\psi }(t_{d})]h_{d}(x_{B}, z, x_{3}, b_{B}, b_{3})\}, \end{aligned}$$
(44)
$$\begin{aligned} {M }^{SP }_{L }(S )= & {} \frac{32\pi C _{F }M^{4}_{B^{0}_{s}}}{\sqrt{2N_{c}}}\int ^{1}_{0}{\text {d}}{} x _{B} {\text {d}}{} z {\text {d}}{} x _{3}\nonumber \\&\times \int ^\infty _{0}{} b _{B}{} b _{3}{\text {d}}{} b _{B}{\text {d}}{} b _{3}\phi _{B_{s}}(x _{B},b_{B})\nonumber \\&\times \{[(1-\eta -r^2)[((1-x_{3})(1+r^2-\eta )-x_{B}\nonumber \\&+z(1-r^2))\psi ^{L}(x_{3},b_{3})\nonumber \\&-(1-\eta )r r_{c}\psi ^{t}(x_{3},b_{3})]\phi ^{0}_{S}\nonumber \\&+\sqrt{(1-r^2)\eta }[((1-\eta )(r^2-1)z-x_{B}r^2)\nonumber \\&\phi ^{s}_{S} \psi ^{L}(x_{3},b_{3})\nonumber \\&+[((1-\eta )((r^2-1)z-2(1-x_{3})r^2)\nonumber \\&+x_{B}r^2)\psi ^{L}(x_{3},b_{3})\nonumber \\&+4(1-\eta )rr_{c}\psi ^{t}(x_{3},b_{3})]\phi ^{t}_{S}]]\nonumber \\&\times \alpha _{s}(t_{c})\exp [-S_{B^{0}_{s}}(t_{c})-S_{M}(t_{c})-S_{\psi }(t_{c})]\nonumber \\&h_{c}(x_{B}, z, x_{3}, b_{B}, b_{3})\nonumber \\&+[(1-r^2-\eta )[((x_{B}-z\eta )(1-r^2)\nonumber \\&+x_{3}(\eta -1+r^2(1-2\eta )))\psi ^{L} (x_{3},b_{3}) \nonumber \\&-(1-\eta )rr_{c} \psi ^{t}(x_{3},b_{3})]\phi ^{0}_{S}\nonumber \\&+\sqrt{(1-r^2)\eta }[((1-r^2)z(1-\eta )+x_{B}r^2)\phi ^{s}_{S}\nonumber \\&+((z+2x_{3}r^{2}-zr^2)(\eta -1)\nonumber \\&+x_{B}r^2)\phi ^{t}_{S}] \psi ^{L}(x_{3},b_{3})]\nonumber \\&\times \alpha _{s}(t_{d})\exp [-S_{B^{0}_{s}}(t_{d})-S_{M}(t_{d}) -S_{\psi }(t_{d})]\nonumber \\&h_{d}(x_{B}, z, x_{3}, b_{B}, b_{3})\}. \end{aligned}$$
(45)

\({\mathcal {A}}_{L }(P )\) and \({\mathcal {A}}_{L }(D )\) can be expressed by the following replacement:

$$\begin{aligned} \begin{aligned}&{\mathcal {A}}_{L }(P )={\mathcal {A}}_{L }(S )|_{\phi ^{0 ,s }_{S }\rightarrow \phi ^{0 ,s }_{P },\phi ^{t }_{S }\rightarrow (1 -r ^{2 })\phi ^{t }_{P }},\\&{\mathcal {A}}_{L }(D )=\sqrt{\frac{2 }{3 }} {\mathcal {A}}_{L }(S )|_{\phi ^{0 ,s }_{S }\rightarrow \phi ^{0 ,s }_{D },\phi ^{t }_{S }\rightarrow (1 -r ^{2 })\phi ^{t }_{D }}. \end{aligned} \end{aligned}$$
(46)

The decay amplitudes of normal polarization are as follows:

$$\begin{aligned} {F }^{LL }_{N }(P )= & {} 8\pi C _{F }f_{\psi }M^{4}_{B^0_{s}}r \int ^{1}_{0}{\text {d}}{} x _{B}{\text {d}}{} z \nonumber \\&\times \int ^\infty _{0}{} b _{B}{} b {\text {d}}{} b _{B}{\text {d}}{} b \phi _{B_{s}}(x _{B},b_{B})\nonumber \\&\times \{[(r^2-1-(1-2z+2zr^2)\eta )\phi ^{T}_{P}\nonumber \\&+\sqrt{(1-r^2)\eta }((zr^2-2-z)\phi ^{a}_{P}+z(1-r^2)\phi ^{v}_{P})]\nonumber \\&\times \alpha _{s}(t_{a})\exp [-S_{B^{0}_{s}}(t_{a})-S_{M}(t_{a})]\nonumber \\&S_{t}(z)h_{a}(x_{B}, z, b_{B}, b)\nonumber \\&-\sqrt{(1-r^2)\eta }[(1+\eta -x_{B}-r^2)\phi ^{a}_{P}\nonumber \\&+(1+x_{B}-\eta -r^2)\phi ^{v}_{P}]\nonumber \\&\times \alpha _{s}(t_{b})\exp [-S_{B^{0}_{s}}(t_{b}) -S_{M}(t_{b})]\nonumber \\&\times S_{t}(|x_{B}-\eta |)h_{b}(x_{B }, z, b_{B }, b)\}, \end{aligned}$$
(47)
$$\begin{aligned} {F }^{LR }_{N }(P )= & {} {F }^{LL }_{N }(P ), \end{aligned}$$
(48)
$$\begin{aligned} {M }^{LL }_{N }(P )= & {} \frac{-64\pi C _{F }M^{4}_{B^{0}_{s}}}{\sqrt{2N_{c}}}\int ^{1}_{0}{\text {d}}{} x _{B} {\text {d}}{} z {\text {d}}{} x _{3}\nonumber \\&\times \int ^\infty _{0}{} b _{B}{} b _{3}{\text {d}}{} b _{B} {\text {d}}{} b _{3}\phi _{B_{s}}(x _{B},b_{B})\nonumber \\&\times \{[(x_{3}-x_{B}+z\eta -x_{3}\eta )r\psi ^{V} (x_{3},b_{3})\nonumber \\&-(1-\eta )r_{c}\psi ^{T}(x_{3},b_{3})]\phi ^{T}_{P}\nonumber \\&+\sqrt{(1-r^2)\eta }[(x_{B}-x_{3}-z+x_{3}\eta )r\psi ^{V} (x_{3},b_{3})\nonumber \\&+(1-\eta )r_{c}\psi ^{T}(x_{3},b_{3})]\phi ^{a}_{P}\}\nonumber \\&\times \alpha _{s}(t_{d})\exp [-S_{B^{0}_{s}}(t_{d})\nonumber \\&-S_{M}(t_{d})-S_{\psi }(t_{d})]h_{d}(x_{B}, z, x_{3}, b_{B}, b_{3}), \end{aligned}$$
(49)
$$\begin{aligned} {M }^{SP }_{N }(P )= & {} -{M }^{LL }_{N }(P ). \end{aligned}$$
(50)

\({\mathcal {A}}_{N }(D )\) can be expressed by the following replacement:

$$\begin{aligned} {\mathcal {A}}_{N }(D )=\sqrt{\frac{1 }{2 }} {\mathcal {A}}_{N }(P ) |_{\phi ^{T ,a ,v }_{P } \rightarrow \phi ^{T ,a ,v }_{D }}. \end{aligned}$$
(51)

The decay amplitudes of transverse polarization are as follows:

$$\begin{aligned} {F }^{LL }_{T }(P )= & {} 8\pi C _{F }f_ {\psi }M^{4}_{B^0_{s}}r\int ^{1}_{0}{\text {d}}{} x _{B} {\text {d}}{} z \nonumber \\&\times \int ^\infty _{0}{} b _{B}{} b {\text {d}}{} b _{B}{\text {d}}{} b \phi _{B_{s}}(x _{B},b_{B})\nonumber \\&\times \{[(r^2-1+(1-2z+2zr^2)\eta )\phi ^{T}_{P}\nonumber \\&+\sqrt{(1-r^2)\eta }((zr^2-2-z)\phi ^{v}_{P}+z(1-r^2)\phi ^{a}_{P})]\nonumber \\&\times \alpha _{s}(t_{a})\exp [-S_{B^{0}_{s}}(t_{a})-S_{M}(t_{a})]\nonumber \\&S_{t}(z)h_{a}(x_{B}, z, b_{B}, b)\nonumber \\&-\sqrt{(1-r^2)\eta }[(1+\eta -x_{B}-r^2)\phi ^{v}_{P}\nonumber \\&+(1+x_{B}-\eta -r^2)\phi ^{a}_{P}]\nonumber \\&\times \alpha _{s}(t_{b})\exp [-S_{B^{0}_{s}}(t_{b})-S_{M}(t_{b})]\nonumber \\&\times S_{t}(|x_{B}-\eta |)h_{b}(x_{B }, z, b_{B }, b)\}, \end{aligned}$$
(52)
$$\begin{aligned} {F }^{LR }_{T }(P )= & {} {F }^{LL }_{T }(P ), \end{aligned}$$
(53)
$$\begin{aligned} {M }^{LL }_{T }(P )= & {} \frac{-64\pi C _{F }M^{4}_{B^{0}_{s}}}{\sqrt{2N_{c}}}\nonumber \\&\times \int ^{1}_{0}{\text {d}}{} x _{B}{\text {d}}{} z {\text {d}}{} x _{3}\int ^\infty _{0}{} b _{B} b _{3}{\text {d}}{} b _{B}{\text {d}}{} b _{3} \phi _{B_{s}}(x _{B},b_{B})\nonumber \\&\times \{[(x_{3}-x_{B}-z\eta -x_{3}\eta )r\psi ^{V}(x_{3},b_{3})\nonumber \\&-(1-\eta )r_{c}\psi ^{T}(x_{3},b_{3})]\phi ^{T}_{P}\nonumber \\&+\sqrt{(1-r^2)\eta }[(x_{B}-x_{3}-z+x_{3}\eta )r\psi ^{V}(x_{3},b_{3})\nonumber \\&+(1-\eta )r_{c}\psi ^{T}(x_{3},b_{3})]\phi ^{v}_{P}\}\nonumber \\&\times \alpha _{s}(t_{d})\exp [-S_{B^{0}_{s}}(t_{d})-S_{M}(t_{d})-S_{\psi }(t_{d})]\nonumber \\&\times h_{d}(x_{B}, z, x_{3}, b_{B}, b_{3}), \end{aligned}$$
(54)
$$\begin{aligned} {M }^{SP }_{T }(P )= & {} -{M }^{LL }_{T }(P ). \end{aligned}$$
(55)

\({\mathcal {A}}_{T }(D )\) can be expressed by the following replacement:

$$\begin{aligned} \begin{aligned} {\mathcal {A}}_{T }(D )=\sqrt{\frac{1 }{2 }} {\mathcal {A}}_{T }(P )|_{\phi ^{T ,a ,v }_{P }\rightarrow \phi ^{T ,a ,v }_{D }}. \end{aligned} \end{aligned}$$
(56)

The mass ratio \(r_{c}=\frac{m_{c}}{M_{B^{0}_{s}}}\) and the group factor \(C _{F }=\frac{4}{3}\). The expressions for the Sudakov exponents \(S_{B^{0}_{s}}(t)\), \(S_{M}(t)\), and \(S_{\psi }(t)\), the threshold resummation factor \(S_{t}(x)\), the scattering kernel functions \(h_{i}(i=a, b, c, d)\), and the hard scales \(t_{i}\) are given in the Appendix.

Table 1 Various parameters used in the calculation [50,51,52]

Vertex correction is performed on the factorization diagrams in this work. According to the naïve dimensional regularization (NDR) scheme [45,46,47], the relevant Wilson coefficients are expressed as

$$\begin{aligned} {a }_{1}(S )= & {} C _{1}+\frac{C _{2}}{N_{c}}+\frac{\alpha _{s}}{9\pi }{} C _{2}[-18-12ln \bigg (\frac{\mu }{m _{b}}\bigg )\nonumber \\&+f _{I}+(1-r^{2})g _{I}],\nonumber \\ {a }_{2}(S )= & {} C _{3}+\frac{C _{4}}{N_{c}} +C _{9}+\frac{C _{10}}{N_{c}} +\frac{\alpha _{s}}{9\pi }(C _{4}+C _{10})\nonumber \\&\times \left[ -18-12ln \bigg (\frac{\mu }{m _{b}}\bigg )+f _{I}+(1-r^{2})g _{I}\right] ,\nonumber \\ {a }_{3}(S )= & {} C _{5}+\frac{C _{6}}{N_{c}} +C _{7}+\frac{C _{8}}{N_{c}} +\frac{\alpha _{s}}{9\pi }(C _{6}+C _{8})\nonumber \\&\times \left[ 6+12ln \bigg (\frac{\mu }{m _{b}}\bigg ) -f _{I}-(1-r^{2})g _{I}\right] ,\nonumber \\ \end{aligned}$$
(57)
$$\begin{aligned} {a }_{1}(P,D )= & {} C _{1}+\frac{C _{2}}{N_{c}}+\frac{\alpha _{s}}{9\pi }{} C _{2}[-18-12ln \bigg (\frac{\mu }{m _{b}}\bigg )+f^{h }],\nonumber \\ {a }_{2}(P,D )= & {} C _{3}+\frac{C _{4}}{N_{c}}+C _{9}+\frac{C _{10}}{N_{c}} +\frac{\alpha _{s}}{9\pi }(C _{4}+C _{10})\nonumber \\&\times \left[ -18-12ln \bigg (\frac{\mu }{m _{b}}\bigg )+f^{h }\right] ,\nonumber \\ {a }_{3}(P,D )= & {} C _{5}+\frac{C _{6}}{N_{c}} +C _{7}+\frac{C _{8}}{N_{c}}+ \frac{\alpha _{s}}{9\pi }(C _{6}+C _{8})\nonumber \\&\times \left[ 6+12ln \bigg (\frac{\mu }{m _{b}}\bigg )-f^{h }\right] .\nonumber \\ \end{aligned}$$
(58)

The renormalization scale, \(\mu \), has been selected to be of the order of \(m _{b}\). The Wilson coefficients \({a }_{1,2,3}(S )\) were applied to the decay amplitude \({\mathcal {A}}(S )\) with only longitudinal polarization, and the hard scattering functions, \(f _{I}\) and \(g _{I}\), are given in Ref. [48]. Meanwhile, the Wilson coefficients \({a }_{1,2,3}(P,D )\) were applied to the decay amplitudes \({\mathcal {A}}(P,D )\) with both longitudinal and transverse polarizations, the hard scattering function, \(f^{h }\), comes from the vertex corrections, and the superscript \(h \) denotes the polarization state \(h =0\) for the helicity 0 state, and \(h =\pm \) for the helicity ± states. The expressions for \(f^{0}\) and \(f^{\pm }\) can be found in Ref. [49].

According to the 2S–1D mixing scheme, the decay amplitudes of \(B^{0}_{s} \rightarrow \psi (3686,3770)K ^{-}\pi ^+\) can be constructed as

$$\begin{aligned}&{{\mathcal {A}}}(B^{0}_{s}\rightarrow \psi (3686)K ^{-}\pi ^{+})=\sin \theta \mathcal{A}(B^{0}_{s}\rightarrow \psi (1D)K ^{-}\pi ^+) \nonumber \\&\quad + \cos \theta {{\mathcal {A}}}(B^{0}_{s}\rightarrow \psi (2S)K ^{-}\pi ^+), \end{aligned}$$
(59)
$$\begin{aligned}&{{\mathcal {A}}}(B^{0}_{s}\rightarrow \psi (3770)K ^{-}\pi ^{+})=\cos \theta \mathcal{A}(B^{0}_{s}\rightarrow \psi (1D)K ^{-}\pi ^+) \nonumber \\&\quad - \sin \theta {{\mathcal {A}}}(B^{0}_{s}\rightarrow \psi (2S)K ^{-}\pi ^+). \end{aligned}$$
(60)
Table 2 Pole masses and widths for the different resonances [41]

4 Numerical results and discussion

The parameters used in the calculation are presented in Table 1, which include the masses of the involved mesons, their decay constants, the lifetime of the \(B^{0}_{s}\) meson, and the Wolfenstein parameters. The pole masses of the quarks were adopted in this study [52].

The data in Table 2 are taken from Ref. [41], and the relevant information that should be considered in the study for the S-, P-, and D-wave resonances is contained in the table. In this work, the dynamic limit of the invariant mass of the resonance is \(m_{K }+m_{\pi }< \omega < {M_{B_{s}^{0}}-M_{\psi }}\). In addition, although the mass of the \(\overline{K }^{*}(1680)^{0}\) resonance exceeds the upper limit, its decay channels should be considered in the study because of its large width(\(\Gamma _{K ^{*}(1680)^{0}}=322\pm 110\) MeV).

The decay branching ratios of the \(\overline{K }^{*}_{0}(1430)^{0}\) resonance of the S-wave were first calculated and the results obtained are given in Table 3. The errors were derived from the shape parameter, \(\omega _{B_{s}}\), in the wave function of the \(B^{0}_{s}\) meson, the Gegenbauer moments in the DAs of the kaon–pion pair, and the hard scale \(t (0.9t \thicksim 1.1t )\), respectively. The errors in the following tables were analyzed in the same order.

Next, the resonances of the P-wave were calculated considering \(\overline{K }^{*}(892)^{0}\), \(\overline{K }^{*}(1410)^{0}\), and \(\overline{K }^{*}(1680)^{0}\), and the results thus obtained are given in Table 4. The experimental measurement data \({\mathcal {B}}\big (B^{0}_{s} \rightarrow \psi (2S)\overline{K }^{*}(892)^{0}(\rightarrow K ^-\pi ^+)\big )=(2.2\pm 0.3)\times 10^{-5}\) was taken from the article by Zyla et al. [51]. Our pQCD prediction agrees well with it within errors.

Finally, the contributions of the \(\overline{K }^{*}_{2}(1430)^{0}\) intermediate resonance of the D-wave were considered, and the calculation results are presented in Table 5.

Table 3 Branching ratios of the S-wave resonance in the quasi-two-body decays \(B^{0}_{s} \rightarrow \psi (2S,1D)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) calculated using the pQCD factorization approach
Table 4 Branching ratios of the P-wave resonances in the quasi-two-body decays \(B^{0}_{s} \rightarrow \psi (2S,1D)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) calculated using the pQCD factorization approach
Table 5 Branching ratios of the D-wave resonance in the quasi-two-body decays \(B^{0}_{s} \rightarrow \psi (2S,1D)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) calculated using the pQCD factorization approach

The theoretical prediction for the branching ratio of the \(B^{0}_{s} \rightarrow \psi (2S)K ^-\pi ^+\) decay is \(3.67^{+1.56+1.42+0.15}_{-1.12-1.15-0.10}\times 10^{-5}\) in this work, which includes contributions from the intermediate resonances of the S-, P-, and D-wave. This result is consistent with the latest experimental data \((3.1\pm 0.4)\times 10^{-5}\) [51] within errors. From the numerical results, it is observed that \(\overline{K }^{*}(892)^{0}\) is the main contributor to the \(B^{0}_{s} \rightarrow \psi (2S)(K ^-\pi ^+)_{P }\) decay, accounting for approximately \(91.55\%\), whereas the contributions of the \(\overline{K }^{*}(1410)^{0}\) and \(\overline{K }^{*}(1680)^{0}\) resonances account for \(1.75\%\) and \(0.79\%\), respectively. Further, the interference contribution of the three resonances amounts to roughly \(5.91\%\). The \(\overline{K }^{*}(892)^{0}\) resonance is also the main source for the \(B^{0}_{s} \rightarrow \psi (1D)(K ^-\pi ^+)_{P }\) decay, accounting for approximately \(93.44\%\), whereas the \(\overline{K }^{*}(1410)^{0}\) and \(\overline{K }^{*}(1680)^{0}\) resonances account for \(1.29\%\) and \(0.47\%\), respectively. In addition, the interference contribution amounts to approximately \(4.80\%\). Referring to Table 4, the branching ratios of the \(\psi \overline{K }^{*}(1410)^{0}\) and \(\psi \overline{K }^{*}(1680)^{0}\) decay modes are of the same order, attributable to the large width of the \(\overline{K }^{*}(1680)^{0}\) resonance.

Fig. 2
figure 2

Differential branching ratios of the S-wave for a \(B^{0}_{s} \rightarrow \psi (2S)K ^{-}\pi ^{+}\) and b \(B^{0}_{s} \rightarrow \psi (1D)K ^{-}\pi ^{+}\)

Fig. 3
figure 3

Differential branching ratios of the P-wave for a \(B^{0}_{s} \rightarrow \psi (2S)K ^{-}\pi ^{+}\) and b \(B^{0}_{s} \rightarrow \psi (1D)K ^{-}\pi ^{+}\)

Fig. 4
figure 4

Differential branching ratios of the D-wave for a \(B^{0}_{s} \rightarrow \psi (2S)K ^{-}\pi ^{+}\) and b \(B^{0}_{s} \rightarrow \psi (1D)K ^{-}\pi ^{+}\)

In comparison, the branching ratio of the \(\psi (2S)\) decay channel is 2.76 times that of the \(\psi (1D)\) decay channel of the S-wave. Furthermore, the branching ratios of the \(\psi (2S)\) decay modes of the P-wave and D-wave are \(5.01\sim 8.57\) and 21.71 times those of the \(\psi (1D)\) decay modes, respectively. In our calculation, the main contributions of the \(\psi (2S)\) and \(\psi (1D)\) decay modes of the S-wave were the non-factorized diagrams, the amplitudes of which are slightly affected by the change in the wave function from \(\psi (2S)\) to \(\psi (1D)\), thus leading to the appearance of only a small gap between the branching ratios of the two decay modes of the S-wave. However, the amplitudes of the P- and D-wave decay channels are dominated by the factorized diagrams, especially the D-wave decay channels, which are significantly affected by the change in the decay constant from \(f_{\psi (2S)}\) to \(f_{\psi (1D)}\). Thus, a large gap can be observed between the branching ratios of the \(\psi (2S)\) and \(\psi (1D)\) decay modes. As mentioned above, the different effects of the factorized and non-factorized diagrams in the decay modes of the S-, P-, and D-wave might be related to the differences in the wave function models with regard to the scalar, vector, and tensor mesons.

In our study, the main uncertainty in the S-wave decay modes comes from the shape parameter \(\omega _{B_{s}}\). For the \(\psi (2S)\) decay modes of the P-wave, the errors from the shape parameter and the Gegenbauer moments are very close, whereas the maximum error term for the D-wave decay modes is from the Gegenbauer moments. These differences can be interpreted as the range of the values of the Gegenbauer moments of the P-wave kaon–pion DAs being larger than that of the S-wave kaon–pion DAs (for example, \(B _{1}=-0.57\pm 0.13\) and \(B _{3}=-0.42\pm 0.22\) for the S-wave, and \(a^{\Vert }_{1K ^{*}}=0.2\pm 0.2\) and \(a^{\Vert }_{2K ^{*}}=0.5\pm 0.5\) for the P-wave), and the single Gegenbauer moment of the D-wave dominating both the twist-2 and twist-3 DAs in the corresponding polarization case. The error caused by the hard scale, t, is the smallest among the three error terms, attributable to the selected range \((0.9t-1.1t)\).

The polarization fractions are defined by Eq. (39), and they are listed in Tables 4 and 5. For the P-wave \(\psi (2S)\) decay mode, the longitudinal polarization fraction is approximately \(43\%\), whereas in the \(\psi (1D)\) mode, it is about \(10\%\), with parallel and vertical fractions being approximately equal in both modes. For the D-wave \(\psi (2S)\) decay mode, the three polarization fractions are roughly at the same level of approximately \(33\%\), but they are distinctly different in the \(\psi (1D)\) mode. We expect additional abundant and detailed data to be obtained from future experiments so that our theoretical predictions can be accurately verified and more systematic analysis for \(B^{0}_{s} \rightarrow \psi (2S,1D)\overline{K }^{*}(\rightarrow K ^-\pi ^+)\) decays can be performed.

From the experimental data, the relative fraction between the branching ratios is [7]

$$\begin{aligned} \begin{aligned} \frac{\mathcal{B}({\overline{B}}^0_{s}\rightarrow \psi (2S)K ^{*}(892)^{0})}{\mathcal{B}({B^0\rightarrow \psi (2S)K }^{*}(892)^{0})}&=5.58\pm 0.57(stat )\\&\quad \pm 0.40({syst })\pm 0.32(f _{s }/f _{d })\%. \end{aligned} \end{aligned}$$
(61)

By comparing the branching ratio of the \(B^{0}_{s} \rightarrow \psi (2S)\overline{K }^{*}(892)^{0}(\rightarrow K ^-\pi ^+)\) decay calculated using the pQCD factorization approach with the pQCD prediction for the \(B^{0}\rightarrow \psi (2S)K ^{*}(892)^{0}(\rightarrow K ^+\pi ^-)\) decay [40], we obtain the relative fraction of the theoretical calculation as

$$\begin{aligned} \begin{aligned} \frac{{{\mathcal {B}}}(B^{0}_{s} \rightarrow \psi (2S)\overline{K }^{*}(892)^{0}(\rightarrow K ^{-}\pi ^{+}))}{\mathcal{B}(B^{0} \rightarrow \psi (2S)K ^{*}(892)^{0}(\rightarrow K ^{+}\pi ^{-}))}=8.01\%. \end{aligned} \end{aligned}$$
(62)

The discrepancy in the values comes from the vertex correction and the selection of different values for some of the parameters. However, this discrepancy is still within the acceptable limit. The relative fraction results predicted by the theory agree somewhat with the experimental data, which supports the pQCD factorization approach and also contributes to further studies on resonance mesons.

Figures 2, 3, and 4 depict the function images of the \(\omega \) dependence of the differential branching ratios of the S-, P-, and D-wave of the \(B^{0}_{s}\rightarrow \psi (2S,1D)K ^-\pi ^+\) decays, respectively. Figure 2 shows that a small peak can always be detected near the invariant mass \(\omega =0.892\)GeV, which can be attributed to the interference effect of the \(\overline{K }^{*}(892)^{0}\) resonance on the S-wave. On the other hand, the function images of the \(\psi (1D)\) mode drop faster at the end than those of the \(\psi (2S)\) mode due to the difference in the upper limit of their invariant masses \(\omega \) of \(K \pi \). Obviously, the peak values of all function images appear at the pole mass of the corresponding resonance. Therefore, the main part of the branching ratios is in the region around the resonance and almost in the range of \(\omega =[m_{K ^{*}}-\Gamma _{K ^{*}},m_{K ^{*}}+\Gamma _{K ^{*}}]\), and the branching ratios of the S-, P-, and D-wave decay modes in this range account for \(43.91\%\), \(74.73\%\), and \(78.68\%\) of the total branching ratios, respectively. The value of \(43.91\%\) can be interpreted as the interference effect of the \(\overline{K }^{*}(892)^{0}\) resonance on the S-wave that is not included.

Using Eqs. (59) and  (60),the branching ratios of the \(B^{0}_{s} \rightarrow \psi (3686,3770)K ^{-}\pi ^{+}\) decays were obtained using the fitting scheme based on the S-D mixing mechanism. The calculation results are presented in Tables 6 and 7, respectively.

Table 6 Branching ratios of the quasi-two-body decays \(B^{0}_{s} \rightarrow \psi (3686)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) under the 2S–1D mixing mechanism calculated using the pQCD factorization approach. The first three uncertainties are derived from the uncertainties in the previous tables, whereas the last one is derived from the mixing angle
Table 7 Branching ratios of the quasi-two-body decays \(B^{0}_{s} \rightarrow \psi (3770)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) under the 2S–1D mixing mechanism calculated using the pQCD factorization approach. The first three uncertainties are derived from the uncertainties in the previous tables, whereas the last one is derived from the mixing angle

Considering the Clebsch–Gordan coefficients, we can write the following relation:

$$\begin{aligned} \begin{aligned} \bigg |K \pi ,I =\frac{1}{2}\bigg \rangle =\sqrt{\frac{1}{3}}|\overline{K }^{0}\pi ^{0}\rangle -\sqrt{\frac{2}{3}}|K ^{-}\pi ^{+}\rangle . \end{aligned} \end{aligned}$$
(63)

In our calculation, for the quasi-two-body decay \(B^{0}_{s} \rightarrow \psi \overline{K }^{*0}\rightarrow \psi K ^-\pi ^+\), isospin conservation was assumed for the strong decays of an \(I =1/2\) intermediate resonance \(\overline{K }^{*0}\) to \(K \pi \), which can be expressed as follows:

$$\begin{aligned} \begin{aligned} \frac{\Gamma (\overline{K }^{*0}\rightarrow \overline{K }^{0}\pi ^{0})}{\Gamma (\overline{K }^{*0}\rightarrow K \pi )}=\frac{1}{3}, \frac{\Gamma (\overline{K }^{*0}\rightarrow K ^{-}\pi ^{+})}{\Gamma (\overline{K }^{*0}\rightarrow K \pi )}=\frac{2}{3}. \end{aligned} \end{aligned}$$
(64)

Therefore, the branching ratios of \(B^{0}_{s} \rightarrow \psi (2S,1D)\overline{K }^{*0}(\rightarrow K \pi )\) and \(B^{0}_{s} \rightarrow \psi (3686,3770)\overline{K }^{*0}(\rightarrow K \pi )\) decays can be directly extracted under the narrow-width approximation relation

$$\begin{aligned}&{{\mathcal {B}}}(B^{0}_{s}\rightarrow \psi \overline{K }^{*0} \rightarrow \psi K ^{-}\pi ^{+})= {{\mathcal {B}}}(B^{0}_{s}\rightarrow \psi \overline{K }^{*0}) \nonumber \\&\quad \cdot \mathcal{B}(\overline{K }^{*0}\rightarrow K \pi )\cdot \frac{2}{3}. \end{aligned}$$
(65)

A comparison of the branching ratios for \(\psi (3770)\) decay modes when the mixing angle is set to \(\theta =-12^{\circ }\) and \(\theta =27^{\circ }\) reveals a significant difference between the two choices, which can be attributed to the visibly small decay constant of \(\psi (1D)\) compared to that of \(\psi (2S)\). These results are in accordance with the analyses presented in other studies [16, 17, 50, 53, 54]. In addition, when the 2S–1D mixing scheme is considered for the \(B^{0}_{s}\rightarrow \psi (3686)K ^-\pi ^+\) decay, the numerical result is slightly different from that of the \(B^{0}_{s}\rightarrow \psi (2S)K ^-\pi ^+\) decay, indicating that the \(\psi (3686)\) state might be deemed the \(\psi (2S)\) state. Further, according to Eqs. (59) and  (60), the reason for the markedly different sensitivity of the \(\psi (3686)\) and \(\psi (3770)\) decay modes to the change in the mixing angle under the 2S–1D mixing scheme can be understood as follows. Numerically, \(\mathcal{A}(B^{0}_{s}\rightarrow \psi (2S)K ^{-}\pi ^+)\) is much larger than \({{\mathcal {A}}}(B^{0}_{s}\rightarrow \psi (1D)K ^{-}\pi ^+)\), and thus the former dominates the decay amplitudes of both the \(\psi (3686)\) and \(\psi (3770)\) decay modes. The value of the amplitude \(\sin \theta {{\mathcal {A}}}(B^{0}_{s}\rightarrow \psi (2S)K ^{-}\pi ^+)\) is greatly changed when the mixing angle is switched between \(\theta =-12^{\circ }\) and \(\theta =27^{\circ }\). On the contrary, the amplitude \(\cos \theta \mathcal{A}(B^{0}_{s}\rightarrow \psi (2S)K ^{-}\pi ^+)\) is relatively stable under this switch. Thus, the branching ratio of the decay \(B^{0}_{s} \rightarrow \psi (3686)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) is stable under the switch between the two values of the mixing angle, whereas the branching ratio of the decay \(B^{0}_{s} \rightarrow \psi (3770)\overline{K }^{*0}(\rightarrow K ^{-}\pi ^{+})\) is highly sensitive to the variation in the mixing angle. The running LHCb experiment is an excellent place to detect decays \(B^{0}_{s} \rightarrow \psi (3686,3770)\overline{K }^{*0}(\rightarrow K ^-\pi ^+)\) with branching ratios of the order of \(10^{-5}-10^{-8}\), which will help us gain a better understanding of the mixing mechanism of the charmonium mesons.

5 Summary

In this work, we have studied the \(B^{0}_{s}\rightarrow \psi (2S,1D)\overline{K }^{*0}(\rightarrow K ^-\pi ^+)\) decays using the pQCD factorization approach by introducing the kaon–pion DAs. We considered the S-wave resonance \(\overline{K }^{*}_{0}(1430)^{0}\), the P-wave resonances \(\overline{K }^{*}(892)^{0}\), \(\overline{K }^{*}(1410)^{0}\), and \(\overline{K }^{*}(1680)^{0}\), and the D-wave resonance \(\overline{K }^{*}_{2}(1430)^{0}\). This study covers three types of polarization amplitudes, namely, longitudinal, parallel, and vertical, which reflect the roles of the different polarization conditions in the decay in terms of the polarization fractions. Based on the 2S–1D mixing scheme, we have obtained the branching ratios of the \(B^{0}_{s} \rightarrow \psi (3686,3770)K ^{-}\pi ^{+}\) decays by fitting the decay amplitudes of the \(\psi (2S)\) and \(\psi (1D)\) decay modes. Finally, the pQCD predictions for the \(B^{0}_{s} \rightarrow \psi (2S,1D)K \pi \) and \(B^{0}_{s} \rightarrow \psi (3686,3770)K \pi \) decays have been obtained using a narrow-width approximation relation.

The pQCD predictions indicate that the \(\overline{K }^{*}(892)^{0}\) resonance is the main contributor to the total decay, and the branching ratios of the \(\psi (2S)\) decay modes agree well with the existing experimental data within acceptable errors. Our calculations show that the branching ratios of the \(\psi (3686)\) and \(\psi (2S)\) decay modes are very similar, suggesting that they can be regarded as the same state. Theoretical predictions for the branching ratios of the \(\psi (3686)\) and \(\psi (3770)\) decay channels are of the order of \(10^{-5}\) and \(10^{-6}\), respectively, which will be verified using the data from future experimental measurements. The detected data will help us to gain a further understanding of the internal structures of the \(\psi (3686)\) and \(\psi (3770)\) mesons.