1 Introduction

In 2005, the BaBar Collaboration observed a broad structure, Y(4260), near 4.26 GeV in the initial state radiation \(e^+e^- \rightarrow \gamma _{\mathrm {ISR}}\pi ^+\pi ^- J/\psi \) process [1]. Later, it was confirmed by the CLEO [2] and Belle [3] Collaborations. This state is considered as a non-\(q\bar{q}\) state by many authors, such as the hybrid state [4, 5], the tetraquark state [6,7,8,9], the molecular state [10,11,12,13], the hadrocharmonium [14] and so on, because its mass and decay properties seem to be in conflict with expectations. Conventionally, \(\psi (4040)\), \(\psi (4160)\) and \(\psi (4415)\) are usually assigned as the \(\psi (3S)\), \(\psi (2D)\) and \(\psi (4S)\) states, respectively, so a resonance with the mass near 4.26 GeV cannot fit in. In addition, the already confirmed excited charmonia usually decay into open-charm states dominantly once they exceed the open-charm threshold, but Y(4260) has never been found in two-body open-charm decay channels [15].Footnote 1 Another resonance, Y(4360), is in a quite similar situation as Y(4260). It was first observed by the BaBar Collaboration in 2007 [17], and was confirmed by the Belle Collaboration soon [18]. It is also considered as a good non-\(q\bar{q}\) candidate by many researchers [19,20,21,22,23,24,25]. (We refer the readers to Refs. [26,27,28,29,30,31] for a comprehensive review on Y(4260) and Y(4360).)

Although the masses of Y(4260) and Y(4360) are usually considered to be incompatible with the conventional charmonia picture, several studies predicted relatively compact mass spectra, with the masses of \(\psi (4S)\) and/or \(\psi (3D)\) in the \(~4.2-4.4\) GeV region. In Refs. [32, 33], Y(4260) is interpreted as a conventional charmonium. In Refs. [34,35,36], Y(4360) is considered to be a charmonium. And in Ref. [37], Y(4260) and Y(4360) are assigned as \(\psi (4S)\) and \(\psi (3D)\) respectively. For the decay properties, since the excited vector charmonia are usually mixtures of the \(^3S_1\) and \(^3D_1\) states, the author of Ref. [38] suggested that the non-observation of Y(4260) in \(e^+e^-\rightarrow \) hadrons may be due to the interference between the S-wave and D-wave contributions. (We shall call the mixing among the S-wave and D-wave vector charmona the \(S{-}D\) mixing for simplicity.) In addition, highly-excited charmonia may have unexpected decay properties due to their node structures. Thus, the common arguments on rejecting conventional interpretations of Y(4260) / Y(4360) may not stand up firmly. So, even though many studies favor exotic hadronic interpretations for these states, since no commonly accepted conclusions have occurred, we think it is still very interesting to investigate the mass spectrum and open-charm decays of vector charmonia incorporating the \(S{-}D\) mixing effect, in order to see whether the tension between the observed properties of Y(4260) / Y(4360) and their conventional charmonia interpretations can be softened, and to test particular assignments for Y(4260) / Y(4360) in the conventional charmonium picture.

Such an investigation should incorporate the coupled-channel effects [39, 40], because they have significant impacts on the excited charmonia, especially, the coupled-channel effects induce the \(S{-}D\) mixing of vector charmona. In coupled-channel models, the quarkonium couples to the continuum multi-particle states, and the physical states are mixtures among all these states. Many studies on coupled-channel effects in the vector charmonium sector focused on the mass shifts and ignored the \(S{-}D\) mixing, such as Refs. [41, 42]. The coupled-channel effects with all possible mixtures were studied in Refs. [39, 40, 43,44,45], either using the Cornell model or using the \(^3P_0\) model. For vector charmonia, it is expected that the mixing between the \((n+1)S\) and nD states is most significant compared to the mixing among other states, which was justified in Refs. [43]. So, in this work, only the mixing between the \((n+1)S\) and nD states are considered. The decay channels we studied are \(D^{(*)}\bar{D}^{(*)}\), \(D_s^{(*)}\bar{D}_s^{(*)}\). We assume Y(4260) and Y(4360) to be the lower and higher states of the mixture of \(\psi (4S)\) and \(\psi (3D)\) respectively, and compare the results with experimental data to test such an assumption. In addition, the coupled-channel-induced \(S{-}D\) mixing of vector charmonia on its own has physical significants, because it could be tested with experiments through di-leptonic decays.

This paper is organized as follows. The theoretical tools are introduced in Sect. 2. We use the instantaneous Bethe-Salpeter equation with the Cornell potential to calculate the wave functions of relevant mesons. Then the \(^3P_0\) model is used to evaluate decay amplitudes. The coupled-channel dynamics is reviewed in this section. In Sect. 3, the results and discussions are presented. The last section is devoted to conclusions.

2 Theoretical settings

2.1 Coupled-channel dynamics

In naive quark models, mesons are bound states of a quark and an anti-quark bounded by a QCD-inspired potential. The masses and wave functions can be obtained by solving the eigenvalue problem formally expressed as:

$$\begin{aligned} H_0|\psi _n\rangle =M_{0,n} |\psi _n\rangle , \end{aligned}$$
(1)

where the \(M_0\) and \(|\psi _n\rangle \) are usually referred as the bare mass and the bare state. The Hamiltonian \(H_0\) only includes the interaction described by the potential binding the quark-anti-quark pair. In coupled-channel formulism, the Hilbert space under consideration is enlarged to include the continuum states which the bare states can decay into. We are focusing on vector charmona decaying into two-body open-charm channels, and in this situation, the Hamiltonian can be formally written as [44]:

$$\begin{aligned} H=\left\{ \begin{array}{cc} H_0&{}H_{\mathrm {QPC}}^\dag \\ H_{\mathrm {QPC}}&{}H_{BC} \end{array} \right\} , \end{aligned}$$
(2)

where \(H_{BC}\) is the free Hamiltonian (by “free”, we mean that the interactions between the two mesons are neglected) for the continuum states with two particles B and C:

$$\begin{aligned} H_{BC}|B,C;\varvec{P}_B,\varvec{P}_C\rangle =E_{BC}|B,C;\varvec{P}_B,\varvec{P}_C\rangle , \end{aligned}$$
(3)

where

$$\begin{aligned} E_{BC}=\sqrt{M_B^2+\varvec{P}_B^2}+\sqrt{M_C^2+\varvec{P}_C^2}. \end{aligned}$$
(4)

The quark-pair creation Hamiltonian \(H_{\mathrm {QPC}}\) induces the decays. With the presence of the non-diagonal elements in the Hamiltonian, the physical states become mixtures of all bare states:

$$\begin{aligned} |\psi '_n\rangle = \sum _{i} a_{ni} |\psi _i\rangle + \sum _{BC} \int dk~c_{n,BC} |B,C;\varvec{P}_B,\varvec{P}_C\rangle , \end{aligned}$$
(5)

where \(\int dk\) denotes an integration over all three-momenta of B and C. The problem now turns into the eigenvalue problem [39, 40]:

$$\begin{aligned} \det |(M-M_{0,n})\delta _{mn}-\Pi _{mn}(M)|=0, \end{aligned}$$
(6)

where

$$\begin{aligned} \Pi _{mn}(M)=\sum _{BC}\int dk \frac{\langle m|H_{\mathrm {QPC}}^\dag |B,C;\varvec{P}_B,\varvec{P}_C\rangle \langle B,C;\varvec{P}_B,\varvec{P}_C|H_{\mathrm {QPC}}|n\rangle }{M-E_{BC}+i\epsilon }.\nonumber \\ \end{aligned}$$
(7)

Above the open-charm threshold, \(\Pi _{mn}\) develops an imaginary part, so in general, the equation allows complex value solutions. In this case, the physical mass and the width of a physical state \(|\psi '_n\rangle \) are related to the corresponding eigenvalue \(M_n\) as

$$\begin{aligned} M_{\mathrm {phys.},n}= & {} \mathrm {Re} M_n, \end{aligned}$$
(8)
$$\begin{aligned} \Gamma _n= & {} -2~\mathrm {Im} M_n. \end{aligned}$$
(9)

For real \(M_n\)’s, of cause, \(M_n\) is just the physical mass of the corresponding state, and these \(M_n\)’s should satisfy the condition \(M_n<2M_D\), where \(2M_D\) denotes the open-charm threshold.

In vector charmonium sector, the mass of nD state is closest to the \((n+1)S\) state. So it is expected that these two states should mix each other the most, which is verified in Ref. [43] and taken as granted by many authors [46, 47]. In this work, we follow these researches and only take into account the mixing between the nD and the \((n+1)S\) states, which means \(\Pi _{nD,(n+1)S}\) and \(\Pi _{(n+1)S,nD}\) are the only non-vanishing non-diagonal elements. With this simplification, Eq. (6) decomposes into several sectors. For 1S sector, we have

$$\begin{aligned} (M-M_{0,1})-\Pi _{1S1S}(M)=0. \end{aligned}$$
(10)

For \((n+1)S-nD\) sectors, we have

$$\begin{aligned} \det \left| \begin{array}{cc} M-M_{0,S}-\Pi _{SS}(M)&{}-\Pi _{SD}(M)\\ -\Pi _{DS}(M)&{}M-M_{0,D}-\Pi _{DD}(M) \end{array} \right| =0,\nonumber \\ \end{aligned}$$
(11)

where the principle quantum numbers in front of S and D are omitted. Since we are focusing on the \(S{-}D\) mixing in this work, and the continuum components in the physical states are irrelevant, we drop off these components in the physical states as Refs. [43, 47] did. So a mixed physical state is expressed as

$$\begin{aligned} |\psi '\rangle =a_S|S\rangle +a_D|D\rangle , \end{aligned}$$
(12)

with coefficients \(a_S\) and \(a_D\) satisfying \(|a_S|^2+|a_D|^2=1\).

2.2 Instantaneous Bethe-Salpeter equation and the \(^3P_0\) model

Now the remaining problem is to solve the naive quark model to obtain the bare masses and the wave functions, and to calculate the open-charm decay amplitudes \(\langle B,C;\varvec{P}_B,\varvec{P}_C|H_{\mathrm {QPC}}|n\rangle \). To this end, we make use of the instantaneous Bethe-Salpeter equation and the \(^3P_0\) model. The instantaneous BS equation, also known as the Salpeter equation, is a well developed relativistic two-body bound state equation, and is very suitable to apply on the heavy quarkonium system. The instantaneous BS wave function \(\varphi (q_{_{P_\perp }}^{\mu })\) may be decomposed into positive and negative energy parts: \(\varphi =\varphi ^{++}+\varphi ^{--}\), where \(q_{_{P_\perp }}\) is the perpendicular part of relative momentum q. For any momentum \(l^\mu \), we have \(l_{_{P_\perp }}=l-\frac{l_{_P}}{\sqrt{P^2}}P\) and \(l_{_P}\equiv \frac{l\cdot P}{\sqrt{P^2}}\), where \(P^\mu \) is the meson’s momentum. The Salpeter equation then takes the form as coupled equations for \(\varphi ^{++}\) and \(\varphi ^{--}\) [48]:

$$\begin{aligned}&(M_0-\omega _{1_P}-\omega _{2_P})\varphi ^{++}(q_{_{P_\perp }})\nonumber \\&\quad =\Lambda _{1}^{+}(p_{1_{P_\perp }})\eta (q_{_{P_\perp }}) \Lambda _{2}^{+}(p_{2_{P_\perp }}),\nonumber \\&(M_0+\omega _{1_P}+\omega _{2_P})\varphi ^{--}(q_{_{P_\perp }})\nonumber \\&\quad =-\Lambda _{1}^{-}(p_{1_{P_\perp }})\eta (q_{_{P_\perp }}) \Lambda _{2}^{-}(p_{2_{P_\perp }}), \end{aligned}$$
(13)

where

(14)
$$\begin{aligned} \eta \left( q_{_{P_\perp }}^{\mu }\right)\equiv & {} \int \frac{\mathrm {d}k_{_{P_\perp }}^{3}}{(2\pi )^{3}}V\left( k_{_{P_\perp }}^{\mu }-q_{_{P_\perp }}^{\mu }\right) \varphi \left( k_{_{P_\perp }}^{\mu }\right) ,\end{aligned}$$
(15)
$$\begin{aligned} \omega _{j_P}\equiv & {} \sqrt{m_{j}^{2}-p_{j_{P_\perp }}^{2}}, \end{aligned}$$
(16)

with \(j=1\) for quark and \(j=2\) for anti-quark. \(p_1\) is the quark momentum and \(p_2\) is the anti-quark momentum. The instantaneous interaction kernel \(V\left( k_{_{P_\perp }}^{\mu }-q_{_{P_\perp }}^{\mu }\right) \) now becomes the QCD-inspired potential between quark and anti-quark. In our model, we use a modified Cornell potential, which in the center of mass frame reads [49, 50]:

$$\begin{aligned} \displaystyle V(\varvec{q})= & {} V_s(\varvec{q})+V_v(\varvec{q})\gamma ^0\otimes \gamma _0,\\ \displaystyle V_s(\varvec{q})= & {} -\left( \frac{\lambda }{\alpha }+V_0\right) \delta ^3(\varvec{q}) +\frac{\lambda }{\pi ^2}\frac{1}{(\varvec{q}^2+\alpha ^2)^2},\\ \displaystyle V_v(\varvec{q})= & {} -\frac{2}{3\pi ^2}\frac{\alpha _s(\varvec{q})}{(\varvec{q}^2+\alpha ^2)}, \end{aligned}$$

where \(\varvec{q}\) is the three-momentum of \(q_{_{P_\perp }}^{\mu }\), i.e., \(q_{_{P_\perp }}^{\mu }=(0,\varvec{q})\) in the meson’s rest frame. \(\alpha _s(\varvec{q})=\frac{12\pi }{33-2N_f}\frac{1}{\mathrm {log}\left( a+\varvec{q}^2/\Lambda _{\mathrm {QCD}}^2 \right) }\) is the QCD running coupling constant; the constants \(\lambda ,\ \alpha ,\ a,\ V_0\) and \(\Lambda _{\mathrm {QCD}}\) are the parameters characterizing the potential.

The method of solving the full Salpeter equation is given in Refs. [49, 50]. After solving the Salpeter equation, we obtain the bare mass spectrum and the wave functions. Then we can use them to calculate the open-charm decay amplitudes. The \(A\rightarrow BC\) open-charm decay is a typical OZI-allowed strong decay process, which in essence is a non-perturbative QCD problem. Due to our pure knowledge of QCD in its non-perturbative region, such processes are usually evaluated using models. Among others, the \(^3P_0\) model is a widely accepted one [51, 52]. This model assumes the decay takes place via creating a quark anti-quark pair from the vacuum carrying the quantum number \(^3P_0\), so it is also known as the quark pair creation model. The \(^3P_0\) model is a non-relativistic model, and majority of works using this model stick on its non-relativistic form. On the other hand, the Salpeter wave function is a relativistic wave function containing the Dirac spinor of quark/anti-quark. So incorporating the Salpeter wave function into \(^3P_0\) model requires the relativistic extension of the original \(^3P_0\) model, which has been done in Ref. [53]. In this work we employ the formula derived in that reference and write the amplitude for the \(A\rightarrow BC\) open-charm decay as

(17)

where quantities referred to A, B and C are labeled with the subscript/superscript A, B and C respectively. \(g=2m_q \gamma \) with \(m_q\) being the the mass of the created quark q or anti-quark \(\bar{q}\) and \(\gamma \) being a universal constance characterizing the strength of the decay. Negative-energy contributions have been neglected due to their smallness comparing to positive-energy contributions.

For \(J^{P}=1^{-}\) meson, the wave function \(\varphi ^{++}\) can be decomposed into two parts: the S-wave part and the D-wave part, i.e.,

$$\begin{aligned} \varphi ^{++}_{1^-}\left( q_{_{P_\perp }}\right)= & {} \sqrt{\frac{m_1m_2}{\omega _{1_P}\omega _{2_P}}}\sum _{s_1,s_2,m,\tilde{M}}u_{s_1}(\varvec{p}_1)\bar{v}_{s_2}(\varvec{p}_2)\chi ^{1,\tilde{M}}_{s_1,s_2}\nonumber \\&\times \left( \sqrt{2M_04\pi }R_{n0}Y_{00}\delta _{m,0}\delta _{\tilde{M},\lambda }\right. \nonumber \\&\left. +\sqrt{2M_04\pi }R_{n2}Y_{2m}S_{m,\tilde{M};1,\lambda }^{2,1}\right) , \end{aligned}$$
(18)

where \(R_{nl}\) is the radial wave function with principle number n and orbital angular momentum l. \(Y_{lm}\) is the spherical harmonic function. m is the magnetic quantum number of orbital angular momentum. \(S,\tilde{M}\) represent the total spin of the quark-anti-quark pair. The \(\lambda \) appearing in Eq. (18) is the magnetic quantum number of the meson. \(s_{1(2)}\) is the spin of the quark (anti-quark) in the meson. \(S_{m,\tilde{M};J,M_{J}}^{l,S}=\langle \tilde{M},m|J, M_{J}\rangle \) and \(\chi _{s_1,s_{2}}^{S,\tilde{M}}=\langle s_1,s_{2}|S, \tilde{M}\rangle \) are Clebsch-Gordan (C-G) coefficients. \(u_{s}(\varvec{p})\) (\(\bar{v}_{s}(\varvec{p})\)) is the Dirac spinor of the quark (anti-quark) with spin s and momentum \(\varvec{p}\). This expression clearly shows the \(L-S\) coupling inside the meson, and it is the same as the corresponding non-relativistic wave function except that: (a) the non-relativistic spinor is replaced with the Dirac spinor; (b) the radial wave function is the solution of the Salpeter equation rather than the non-relativistic Schr\(\mathrm {\ddot{o}}\)dinger equation.

Inserting the S-wave part or the D-wave part of the wave function into Eq. (17), one arrives at the familiar form of the decay amplitude of the \(^3P_0\) model:

$$\begin{aligned} \mathcal {M}= & {} g\int \frac{d^3q_{_{P_\perp }}}{(2\pi )^3}\left\{ \sum _{\tilde{M},m}\sum _{\tilde{M}_{B},m_B}\sum _{\tilde{M}_{C},m_C}\psi _{nlm}(q_{_{P_\perp }})\nonumber \right. \\&\times (\psi _{n_Cl_Cm_C}(q^C_{_{P_\perp }}))^*(\psi _{n_Bl_Bm_B}(q^B_{_{P_\perp }}))^*\nonumber \\&\times S_{m,\tilde{M};J,M_{J}}^{l,S}S_{m_C,\tilde{M}_C;J_C,M_{J_C}}^{l_C,S_C}S_{m_B,\tilde{M}_B;J_B,M_{J_B}}^{l_B,S_B}\nonumber \\&\times \sum _{s_a,s_{\bar{a}},s_q,s_{\bar{q}}}\chi ^{S,\tilde{M}}_{s_a,s_{\bar{a}}}\chi ^{S_C,\tilde{M}_C}_{s_q,s_{\bar{a}}}\chi ^{S_B,\tilde{M}_B}_{s_a,s_{\bar{q}}}\nonumber \\&\times \frac{m_q}{\omega _{q_P}}\bar{u}_{s_q}(\varvec{p}_q)v_{s_{\bar{q}}}(\varvec{p}_{\bar{q}}), \end{aligned}$$
(19)

where \(\psi _{nlm}=\sqrt{2M_04\pi }R_{nl}Y_{lm}\) and \(\omega _{q_P}=\sqrt{\varvec{p}_q^2+m_q^2}\). The factor \(\frac{m_q}{\omega _{q_P}}\) is usually taken to be 1 in a non-relativistic \(^3P_0\) model calculation, however in this work, we keep this factor as it is and serve it as a relativistic correction in the decay amplitudes. The decay width is given by

$$\begin{aligned} \Gamma= & {} \frac{|\varvec{P}_B|}{8M_A^2\pi }\frac{1}{3}\sum _{\mathrm {pol}}|\mathcal {M}|^2\nonumber \\= & {} \frac{2\pi |\varvec{P}_B|E_BE_C}{M_A}\frac{1}{3}\sum _{\mathrm {pol}}|f_{A\rightarrow BC}|^2, \end{aligned}$$
(20)

where \(f_{A\rightarrow BC}\) is introduced to make connection with the convention which is widely used in literature (such as Ref. [54]). \(\sum _{\mathrm {pol}}\) means summation over polarizations. \(f_{A\rightarrow BC}\) is related to the matrix element as

$$\begin{aligned} \langle B,C;\varvec{P}_B,\varvec{P}_C|H_{\mathrm {QPC}}|A\rangle = \delta ^3(\varvec{P}_A-\varvec{P}_B-\varvec{P}_C)f_{A\rightarrow BC}.\nonumber \\ \end{aligned}$$
(21)
Table 1 Our theoretical results of bare masses, physical masses and mass shifts of vector charmonia. Experimental data and some theoretical predictions from literature are given for comparison. All quantities are given in the unit of MeV
Table 2 The coefficients \(a_S\) and \(a_D\) characterizing the \(S{-}D\) mixing for different states and the extracted mixing angles

Given these decay amplitudes, whenever \(\Pi _{mn}\) develops imaginary part, \(\Pi _{mn}\) can be calculated as [44]

$$\begin{aligned} \mathrm {Im} \Pi _{mn} (E)= & {} -\sum _{BC}\left\{ \frac{1}{3}\sum _{\mathrm {pol}} f_{m\rightarrow BC}^* f_{n\rightarrow BC}\right. \nonumber \\&\left. \times \frac{\pi |\varvec{P}_B|E_BE_C}{E} \theta (E-E_{BC}^{th})\right\} , \end{aligned}$$
(22)
$$\begin{aligned} \mathrm {Re} \Pi _{mn} (E)= & {} -\frac{1}{\pi }\int dE' \frac{\mathrm {Im} \Pi _{mn} (E')}{E-E'}, \end{aligned}$$
(23)

where \(E_{BC}^{th}\) represents the threshold energy of BC channel. \(\theta (x)\) is the step function. The expressions of \(\mathrm {Im} \Pi _{mn}\) for each decay channels are given in Appendix A.

3 Results and discussions

Previous section gives the formula for calculating the charmonium spectrum and the coupled-channel effects. To obtain the bare masses and the bare states of vector charmonia, we use the following parameters:

$$\begin{aligned} \begin{array}{ccc} m_c=1.71~\mathrm {GeV},~~~ &{} a=\mathrm {e}=2.7183, &{}~~~\alpha =0.12~ \mathrm {GeV},\\ \lambda =0.202~ \mathrm {GeV}^2,~~~&{} \Lambda _{\mathrm {QCD}}=0.40~ \mathrm {GeV}, &{} ~~~V_0=-0.204~ \mathrm {GeV}. \end{array} \end{aligned}$$

While all the other parameters lie in reasonable ranges, the parameter \(\Lambda _{\mathrm {QCD}}\) is a bit larger than its typical value. However, first of all, \(\Lambda _{\mathrm {QCD}}\) is a renormalization-scheme-dependent parameter, and its value varies from scheme to scheme. Secondly, in the popular \(\overline{MS}\) scheme, at two-loop order, \(\Lambda _{\overline{MS}}^{(4)}=325\) MeV corresponds to \(\alpha _s(M_z)=0.118\), where the superscript of \(\Lambda _{\overline{MS}}^{(4)}\) represents \(n_f=4\), while, \(\Lambda _{\overline{MS}}^{(4)}=413\) MeV corresponds to \(\alpha _s(M_z)=0.123\) [55]. So, the value we take for \(\Lambda _{\mathrm {QCD}}\) is not unreasonable large. Moreover, we have checked that varying the parameter \(\Lambda _{\mathrm {QCD}}\) has impacts on the mass spectrum, but much less impacts on the \(S{-}D\) mixing angles and the decay widths.

3.1 The mass spectrum

The results of bare masses are listed in Table 1. With the obtained wave functions, the coupled-channel effects are calculated. The decay strength parameter of \(^3P_0\) model is fitted to be \(\gamma =0.43\), and we also set the strength parameter of creating \(s\bar{s}\) to be \(\gamma _s=\frac{\gamma }{\sqrt{3}}\) as usual [56, 57]. For consistency, the wave functions of \(D^{(*)}\) and \(D^{(*)}_s\) are also calculated with the Salpeter equation. With constituent quark mass \(m_u=m_d=0.305\) GeV and \(m_s=0.500\) GeV, we obtain \(M_{D}=1.865\) GeV, \(M_{D^*}=2.008\) GeV, \(M_{D_s}=1.968\) GeV and \(M_{D_s^*}=2.112\) GeV. Now the physical states are mixtures of the bare \(\psi ((n+1)S)\) and \(\psi (nD)\) states. For \(\psi (1S)\), we assume that it does not mix with other bare states. We denote the physical state as \(\psi '(nS)/\psi '(nD)\), where nS / nD indicates the dominant component in this physical state. The physical masses are shifted by the coupled-channel effects and can be compared with the experimental data. The mass shifts are represented by \(\Delta M\). We also list some results from literature for comparison. These information are gathered in Table 1. One can see that from the \(\psi '(1S)\) through \(\psi '(3S)\), the masses are comparable with the experimental data, which justifies our model calculations. (\(\psi '(2D)\) is one exception, which is also a problem of another potential model calculation [37].)

We compare the masses of the two physical states \(\psi '(4S)\) and \(\psi '(3D)\) with the masses of Y(4260) and Y(4360) respectively. The mass of Y(4260) was measured by BaBar [1] and Belle [58] both close to 4260 MeV, but the most recent measurement performed by BESIII gives a mass of \(4222.0\pm 3.1\pm 1.4\) MeV [59]. So the world averaged mass of Y(4260) now becomes \(4230\pm 8\) MeV [15]. Our calculated mass of \(\psi '(4S)\) is larger than this value by \(\sim 50\) MeV. On the other hand, the mass of \(\psi '(3D)\) is lower than the world averaged mass of Y(4360) by \(\sim 50\) MeV. We indicate that although the Cornell potential with coupled-channel effects is a good modeling for QCD dynamics, there are still some uncertainties in the mass spectrum which could be of \(\sim 10\) MeV order or even larger. We find that the masses of \(\psi '(4S)\) and \(\psi '(3D)\) are close to the masses of Y(4260) and Y(4360) respectively, yet deviations still exist. If the mixing between the 4S and 3D charmonium states is larger, the masses of \(\psi '(4S)\) and \(\psi '(3D)\) would be closer to the masses of Y(4260) and Y(4360) respectively. This possibility will be discussed in subsection C. The bare mass of \(\psi (5S)\) state in our model is 4468 MeV. Considering that the mass shifts are in general tens MeV, we find that \(\psi (4415)\) should be assigned as the physical state dominated by 5S in our model.

3.2 The \(S{-}D\) mixing angles

In Table 2, we give the coefficients \(a_S\) and \(a_D\) as in Eq. (12) for different states. We also extracted the mixing angles defined as in

$$\begin{aligned} |\psi '(S)\rangle= & {} \cos \theta |S\rangle + \sin \theta |D\rangle , \end{aligned}$$
(24)
$$\begin{aligned} |\psi '(D)\rangle= & {} -\sin \theta |S\rangle + \cos \theta |D\rangle . \end{aligned}$$
(25)

In extracting the mixing angles, we neglect the phase of the complex number \(a_{S/D}\) and set the sign equal to the sign of its real part. The magnitude of mixing angles for \(2S-1D\) states in our model are smaller than the result from Ref. [40] where they obtained a mixing angle of \(-10^{\circ }\). The difference may due to different model settings in these two works. The magnitude of mixing angles for \(3S-2D\) states are larger than those for \(2S-1D\) states as expected. For \(4S-3D\) states, the mixing angles are around \(-6^{\circ }\sim -5^{\circ }\).

To provide more details, we plot the real and imaginary parts of \(\Pi _{mn}\) in Figs. 1, 2, 3. The real parts of diagonal elements roughly reflect the mass shifts at given energy scale. The imaginary part of diagonal elements roughly reflect half-widths at given energy scale. And non-diagonal elements may reflect how much the \(S{-}D\) mixing is at given energy scale. One can find that the non-diagonal elements are in general smaller than the diagonal ones. Actually, the diagonal elements \(\mathrm {Im} \Pi _{SS}^{BC}\) and \(\mathrm {Im} \Pi _{DD}^{BC}\) for each channel BC always \(\le 0\), so the contributions to \(\mathrm {Im}\Pi _{SS}\) and \(\mathrm {Im} \Pi _{DD}\) add up. But the non-diagonal elements \(\mathrm {Im} \Pi _{SD}^{BC}\) for different channels may have different signs, so \(\mathrm {Im}\Pi _{SD}\) is generally smaller than \(\mathrm {Im}\Pi _{SS}\) and \(\mathrm {Im} \Pi _{DD}\), and oscillates around zero as E varies. This in turn results in \(\mathrm {Re}\Pi _{SD}<\mathrm {Re}\Pi _{SS}\) and \(\mathrm {Re}\Pi _{DD}\) in general. The diagonal elements \(\mathrm {Im}\Pi _{SS}\) and \(\mathrm {Im}\Pi _{DD}\) also oscillate but without changing signs. The oscillation behavior of the diagonal elements of \(\Pi \) is a reflection of the node structures of initial states, so this behavior becomes more frequent (against E) for higher excitation states.

Fig. 1
figure 1

Real and imaginary parts of \(\Pi _{mn}\) in \(2S-1D\) sector and Real part of \(\Pi _{mn}\) in 1S sector

Fig. 2
figure 2

Real and imaginary parts of \(\Pi _{mn}\) in \(3S-2D\) sector

Fig. 3
figure 3

Real and imaginary parts of \(\Pi _{mn}\) in \(4S-3D\) sector

3.3 Decay widths

Table 3 Decay widths of \(D^{(*)}\bar{D}^{(*)},D_s^{(*)}\bar{D}_s^{(*)}\) channels for states above the open-charm threshold in the unit of MeV. \(\Gamma ^{\mathrm {theo.}}_{\mathrm {tot.}}\) denotes the summation of those in \(D^{(*)}\bar{D}^{(*)},D_s^{(*)}\bar{D}_s^{(*)}\) channels which are allowed by energy conservation
Table 4 The ratios \(\frac{\Gamma (Y(4260)\rightarrow X)}{\Gamma (Y(4260)\rightarrow J/\psi \pi ^+\pi ^-)}\), where X is one of the \(D^{(*)}\bar{D}^{(*)},D_s^{(*)}\bar{D}_s^{(*)}\) channels, comparing to the experimental upper bounds at \(90\%\) confidence level

In general, oscillations make \(\Pi \) and relevant observables sensitive to the energy scale E. In view of this, we calculate the decay widths of physical states at their real mass scales, i.e., the experimental masses. The decay widths of physical states are given in Table 3. \(\Gamma ^{\mathrm {theo.}}_{\mathrm {tot.}}\) denotes the summation of those in \(D^{(*)}\bar{D}^{(*)},D_s^{(*)}\bar{D}_s^{(*)}\) channels which are allowed by energy conservation. The decay widths of Y(4260) and Y(4360) are given under the assumption that they are the \(\psi '(4S)\) and \(\psi '(3D)\) respectively. The widths of \(\psi (3770)\), \(\psi (4040)\) and \(\psi (4160)\) are dominated by the two-body open-charm decays. So for these states, \(\Gamma ^{\mathrm {theo.}}_{\mathrm {tot.}}\) should be comparable to the observed width of the corresponding particle. From Table 3, one can see that our results are consistent with the experimental data. We would like to give some explanations about the open-charm channels of Y(4360) here. For Y(4360) (i.e., \(\psi '(3D)\) in our assignment), the open-charm decay channels with P wave D mesons, i.g., \(D_1D\), \(D^*_0D^*\), are open. The present work doesn’t take into account contributions from these channels. This is a drawback of this work. We argue that the most experimentally attainable charmed mesons are D and \(D^*\), so they are of primary interests to us. In addition, the impacts of \(D_1D\), \(D^*_0D^*\) channels on the mass spectrum could be weakened by re-adjusting the model parameters.

For \(\psi '(4S)\) and \(\psi '(3D)\), it is interesting to find that their \(\Gamma ^{\mathrm {theo.}}_{\mathrm {tot.}}\) are notably smaller than those of \(\psi '(3S)\) and \(\psi '(2D)\). Especially \(\Gamma _{D\bar{D}}\) of \(\psi '(4S)\) is less than 1 MeV. For further discussions, we estimate \(\Gamma (Y(4260)\rightarrow J/\psi \pi ^+\pi ^-)\) using the measured data \(\Gamma _{ee}Br(Y(4260)\rightarrow J/\psi \pi ^+\pi ^-)\approx 10\) eV [59]. In our assignment for Y(4260), the di-leptonic width is \(\lesssim 1\) KeV. For a larger \(S{-}D\) mixing, the di-leptonic width could be even smaller. So the decay width of \(Y(4260)\rightarrow J/\psi \pi ^+\pi ^-\) is estimated to be of \(\sim 1\) MeV order. With this estimation, we obtain the ratios of the decay widths of open-charm channels to the width of \(Y(4260)\rightarrow J/\psi \pi ^+\pi ^-\) and compare them with the upper limit given by the CLEO Collaboration in Table 4. We find that all but the \(D^*\bar{D}^*\) channel are lower than the upper limits. One can see that the open-charm widths of \(D\bar{D},D_s\bar{D}_s,etc\) are considerably smaller than that of \(J/\psi \pi ^+\pi ^-\) channel, which is contrary to the naive expectation for an excited vector charmonium, and the non-observations of Y(4260) in these channels may be explained. On the other hand, the ratio of \(D^*\bar{D}^*\) channel is above the experimental upper limit, which means the \(\psi '(4S)\) assignment for Y(4260) is not consistent with experiments on every aspect.

In a recent paper [61], the authors analyzed the light-quark SU(3) singlet and octet components of Y(4260) through the \(e^+e^-\rightarrow Y(4260)\rightarrow J/\psi \pi ^+\pi ^-\) process, and found a large octet component in Y(4260). They concluded that Y(4260) cannot be a conventional charmonium or a hybrid. Their work doesn’t invalidate our efforts, because their results still suffer from uncertainties, and studies from different points of views, such as what we presented here, is still worthy.

Fig. 4
figure 4

\(\mathrm {Im} \Pi ^{BC}_{SS}\) and \(\mathrm {Im} \Pi ^{BC}_{DD}\) of \(D\bar{D}\), \(D\bar{D}^{*}+c.c.\) and \(D^{*}\bar{D}^{*}\) channels

For Y(4360), \(\Gamma _{ee}Br(Y(4360)\rightarrow \psi (2S) \pi ^+\pi ^-)\approx 10\) eV [62] implies \(\Gamma (Y(4360)\rightarrow \psi (2S) \pi ^+\pi ^-)\) is of \(\sim 1\) MeV order. The decay widths of \(D^{(*)}\bar{D}^{(*)},D_s^{(*)}\bar{D}_s^{(*)}\) channels for Y(4360) are from \(\sim 0.1\) MeV to \(\lesssim 10\) MeV, which implies small branching ratios. So the tension of non-observations of Y(4360) in these channels and its charmonium assignment is softened. Using the experimental data of total widths, we show some of the branching ratios of open-charm decays which could be tested by further experimental data:

$$\begin{aligned}&Br(Y(4260)\rightarrow D\bar{D})=0.758\%\nonumber \\&Br(Y(4260)\rightarrow D^*\bar{D}+c.c.)=12.2\%\nonumber \\&Br(Y(4260)\rightarrow D^*\bar{D}^*)=35.6\% \end{aligned}$$
(26)
$$\begin{aligned}&Br(Y(4360)\rightarrow D\bar{D})=8.10\%\nonumber \\&Br(Y(4360)\rightarrow D^*\bar{D}+c.c.)=1.28\%\nonumber \\&Br(Y(4360)\rightarrow D^*\bar{D}^*)=5.76\% \end{aligned}$$
(27)

The reason of the smallness of decay widths of \(\psi '(4S)\) and \(\psi '(3D)\) are the oscillation behavior of decay amplitudes and the mixing between S and D wave components. The node structures of initial states make the decay amplitude of each channel oscillate, which can be reflected by \(\mathrm {Im} \Pi ^{BC}_{SS \mathrm {or} DD}\) in each channel. Figure 4 shows that the decay amplitudes (actually, the square of decay amplitudes) approach zero at particular energies. \(\psi (3D)\rightarrow D^*\bar{D}^*\) dose not reach zero but is close to zero at some energy scales. From Fig. 4, we can see that in the range \(E\simeq [4.2,4.4]\) GeV, where Y(4260) and Y(4360) lie in, the decay amplitude of each channel, except \(\psi (4S)\rightarrow D^*\bar{D}^*\), has one trough, and this exception has one crest. So, except \(\psi (4S)\rightarrow D^*\bar{D}^*\) process, \(\mathrm {Im} \Pi ^{BC}_{SS \mathrm {or} DD}\) of all channels at the mass scales of Y(4260) and Y(4360) have good chance to be small. One may wonder that the locations of the zeros of these decay amplitudes may vary as model parameters vary. But as long as we require a reasonable spectrum to be able to accommodate Y(4260) and Y(4360), the model parameters can not vary too far, and the locations of zeros are not sensitive to small variations in parameters.

Table 5 Di-leptonic decay widths, i.e., \(\Gamma _{ee}\) in the unit of keV

The \(S{-}D\) mixing then affect the final results in the following ways. For the case when the S wave amplitude and the D wave amplitude are not comparable to each other, for example in \(D^*\bar{D}^*\) channel the S wave amplitude is larger than the D wave amplitude, the decay width of the mixed state dominated by S wave is depressed by mixing with the smaller D wave amplitude and vice versa. For the case when the S wave amplitude is comparable to the D wave amplitude, the decay widths of mixed states either enhanced by the constructive interference or depressed by the destructive interference. \(Y(4260)\rightarrow D\bar{D}\) is an example of the constructive interference case and \(Y(4260)\rightarrow D\bar{D}^*+c.c.\) is an example of the destructive interference case. In Ref. [38], the authors noticed that the S wave amplitude and the D wave amplitude of \(Y(4260)\rightarrow D\bar{D}\) decay are of opposite signs, but since the mixing angle is negative, the two amplitudes actually interfere constructively.

Finally, we indicate that from the di-leptonic decay widths of vector charmonia, one may expect larger mixing angles than the coupled-channel-induced mixing angles presented in this work. To show this we calculate the \(\Gamma _{ee}\) for each physical states, and the results are shown in Table 5. In these calculations, the QCD correction factor \(1-\frac{16}{3}\frac{\alpha _s}{\pi }\) with \(\alpha _s=0.3\) is included. On the other hand, by fitting the experimental data of \(\Gamma _{ee}\) under the assumption of Eqs. (24) and (25), we obtain \(\theta _{2S-1D}=-11.5^{\circ }\) and \(\theta _{3S-2D}=-30.7^{\circ }\). This may imply that there are some unrevealed physical sources inducing \(S{-}D\) mixing. (The contributions in \(S{-}D\) mixing from tensor force in potential model are smaller than coupled-channel effects [40, 47].) If a larger mixing angle appears in the \(4S-3D\) sector, the mass splitting of \(\psi '(4S)\) and \(\psi '(3D)\) should become larger. In addition, the widths \(\Gamma _{\mathrm {tot.}}\) for both the resonances decrease as the magnitude of the mixing angle \(|\theta |\) becomes larger, as shown in Fig. 5. Particularly, the decay widths of \(Y(4360)\rightarrow D\bar{D}\) and \(Y(4260)\rightarrow D\bar{D}^*+c.c.\) decrease as \(|\theta |\) increases as expected, because the S wave and D wave amplitudes interfere destructively in these two channels. So the measurements on the di-leptonic decays of Y(4260) and Y(4360) are desired.

Fig. 5
figure 5

Decay widths of Y(4260) and Y(4360) as functions of the mixing angle. The widths of \(D_s^{(*)}\bar{D}_s^{(*)}\) are not plotted for they are generally small. The subscript “tot.” in the plot means summation of the \(D^{(*)}\bar{D}^{(*)},D_s^{(*)}\bar{D}_s^{(*)}\) channels

4 Conclusions

In this work, we calculated the mass spectrum and open-charm \(D^{(*)}\bar{D}^{(*)}\)/\(D_s^{(*)}\bar{D}_s^{(*)}\) decay widths of \(J^{PC}=1^{--}\) charmonium states with the coupled-channel effects taken into account. The mass shifts are found to be from tens MeV up to 100 MeV. We focused on the mixing between \((n+1)S\) and nD states induced by the coupled-channel effects. The mixing angles are extracted. We find that the mixing in \(3S-2D\) sector is larger than those in \(2S-1D\) sector and in \(4S-3D\) sector. The di-leptonic decay widths are also calculated. Most of the widths and the masses are consistent with the corresponding experimental data for states from \(J/\psi \) to \(\psi (4160)\). The calculations are performed using the instantaneous BS equation with the Cornell potential and the \(^3P_0\) model which has been reexpressed in the form suitable for the Salpeter wave functions.

Based on these calculations, we discussed the possibility of assigning the resonant state Y(4260) as the mixture of \(4S-3D\) with lower mass and Y(4360) as the mixture of \(4S-3D\) with higher mass. The masses of \(\psi '(4S)\) and \(\psi '(3D)\) are found to be 4285 MeV and 4319 MeV, which are close to the masses of Y(4260) and Y(4360) respectively, yet still deviate from them by \(\sim 50\) MeV. The open-charm decay widths of \(\psi '(4S)\) and \(\psi '(3D)\) in \(D^{(*)}\bar{D}^{(*)}\) and \(D_s^{(*)}\bar{D}_s^{(*)}\) channels are smaller than the naive expectations for the excited charmonia due to the oscillations of the decay amplitudes and the \(S{-}D\) mixing effects, except for the \(Y(4260)\rightarrow D^*\bar{D}^*\) channel. So the tension between the observed properties of Y(4260) / Y(4360) and their conventional charmonia interpretations is softened. But the present assignments cannot be consistent with experiments on every aspect. The branching ratios of open-charm \(D^{(*)}\bar{D}^{(*)}\) and \(D_s^{(*)}\bar{D}_s^{(*)}\) decays and the di-leptonic decay widths are given under the present assignments, which can be tested by further experimental data.