1 Introduction

Since the discovery of the heavy quarkonium, the research on it attracts more and more attentions from theorists and experimentalists. As an important way to study the QCD mechanism, the production of heavy quarkonium is very useful for testing perturbative QCD (pQCD) theory [1,2,3,4,5,6]. Many studies have been paid for them. For example, for the \(B_c\) meson production, many studies have been done through not only the ‘direct’ hadronic production [7,8,9,10,11,12,13], but also its ‘indirect’ production channels of top-quark [14,15,16,17], \(Z^0\)-boson [18,19,20,21], \(W^{\pm }\)-boson [22,23,24] and Higgs-boson [25, 26] decays in which sizable number of events can be detected at LHC or HL-LHC [27,28,29,30] which runs at the center-of mass energy \(\sqrt{S}=14\) TeV with the current integrated luminosity of \(3{~\mathrm ab}^{-1}\).

Fig. 1
figure 1

Feynman diagrams for the FCNC production channel, \(t(p_1)\rightarrow |(c{\bar{Q}})[n]\rangle (p_2) + Q(p_3) + Z^{0}(p_4)\), where n stands for a series of Fock states. \(d_m\) denotes as the generation of down-type quark with mass \(m_{d_m}\)

Being the heaviest fermion with a mass close to the electroweak symmetry breaking scale in standard model (SM), the top quark is helpful for analyzing the production of the heavy quarkonium and is also speculated to be a sensitive probe of new physics beyond the SM. A better understanding of those channels within the SM is helpful for judging whether there is really new physics, i.e. to deduct the SM background from the experimental data at a high confidence level such that to determine the right ranges for the new physics parameters. Following the top quark dominant decay channel, \(t\rightarrow b W^+\), it has been pointed out that sizable \(B^{-}_c\) mesons can be produced via the channel, \(t\rightarrow |(b{\bar{c}})[n]\rangle +c +W^{+}\) [14,15,16,17], where [n] stands for the \((b{\bar{c}})\)-quarkonium state via the velocity scaling rule of the non-relativistic QCD (NRQCD) theory [31].

The heavy quarkonium (\(B^-_{c}\), \(\eta _c\) and etc.) may also be produced via the top-quark decays through the flavor changing neutral current (FCNC) processes, i.e. \(t\rightarrow |(c{\bar{Q}})[n]\rangle + Q+ Z^{0}\) with Q equals to c or b respectively. The FCNC processes involving heavy hadrons are of significant interests and allow stringent tests of our current understanding of particle physics. The Glashow–Iliopoulos–Maiani (GIM) mechanism [32] forbids its production at the tree level and covers important information in the loop structure. There are many studies focused on the top-quark rare decays via FCNC in the SM [33,34,35,36,37,38,39,40,41,42,43] and other new models like the two-Higgs-doublet models (2HDM) [33], the minimal supersymmetric model (MSSM) [44,45,46], the Topcolor-assisted Technicolor Model (TC2) [47,48,49,50] and other models [51,52,53]. These researches confirmed that FCNC processes could be unambiguous small but also could provide a useful window in the quest for new physics signals. Thus to make a systematic study on the production of charmonium and \(c{\bar{b}}\)-quarkonium through the top-quark decays via the FCNC in the SM is requisite, it will provide useful guidance for future new physics research from the heavy quarkonium involved processes. As will be shown later, the decay width via FCNC is generally small and the contribution from the P-wave states is relatively smaller than that of the S-wave states. In the present paper, we shall only make a detailed discussion on the production of two color-singlet S-wave states \(^1S_0\) and \(^3S_1\).

The remaining parts of the paper are organized as follows. In Sect. 2, we present the calculation technology for the production of heavy quarkonium through the top-quark rare decays via FCNC. Numerical results for total and differential decay widths, together with their uncertainties, are presented in Sect. 3. Section 4 is reserved for a summary.

2 Calculation technology

The FCNC kernel of the top-quark decay is \(t\rightarrow c Z^0\), and the charmonium and the \((c{\bar{b}})\)-quarkonium production via FCNC is through the process

$$\begin{aligned} t(p_1) \rightarrow |(c{\bar{Q}})[n]\rangle (p_2) + Q(p_3) + Z^0(p_4), \end{aligned}$$
(1)

where Q stands for c or b, \(p_i~(i = 1,2,3,4)\) represent the four-momenta of initial and final state particles, respectively. Feynman diagrams for the production of heavy quarkonium via FCNC are depicted in Fig. 1, where the \(t\rightarrow cZ^{0}\) is realized via a weak interaction loop. Compared to b and s quarks, the d quark can be ignored for its small mass and the \(\mathrm {CKM}(1,3)\) is only 0.009. Because the intermediate gluon should be hard enough to generate a \(c{\bar{c}}\) pair or a \(b{\bar{b}}\) pair, those processes are pQCD calculable. The specific momenta of the two constitute quarks in \((c{\bar{Q}})\)-quarkonium are \(p_{21}\) and \(p_{22}\):

$$\begin{aligned} p_{21} = \frac{m_c}{M}{p_2} + q ,\;\;\;p_{22} = \frac{m_{Q}}{M}{p_2} - q, \end{aligned}$$
(2)

where q stands for the relative momentum between the two constituent quarks. The quarkonium mass \(M\simeq m_c + m_{Q}\) is adopted to ensure the gauge invariance of the hard scattering amplitude.

The decay width of the process \(t \rightarrow |(c{\bar{Q}})[n]\rangle + Q + Z^{0}\) can be written in the following factorized form

$$\begin{aligned} \Gamma =\sum _{n} {\hat{\Gamma }}(t\rightarrow |(c{\bar{Q}})[n]\rangle + Q+Z^{0}) \langle \mathcal{O}^H[n] \rangle , \end{aligned}$$
(3)

where n means a series of Fock states. Contributions from the color-octet states or the P-wave states are generally smaller than that from the color-singlet S-wave states, which are about \(10\%\) of the ground states via a general velocity scaling rule [31]. Thus in the present paper, we shall consider the color-singlet S-wave states’ contributions. The non-perturbative matrix element \(\langle \mathcal{O}^{H}[n]\rangle \) describes the hadronization process of a perturbative \((c{\bar{Q}})\) pair into an observable hadronic state. The color-singlet ones can be computed through potential models [54,55,56,57,58,59,60], e.g. the color-singlet S-wave states are related to the wavefunction at the origin, \(\Psi _S(0)^2=R_S(0)^2/{4\pi }\). The decay width \({\hat{\Gamma }}\) represents the short-distance coefficients which can be calculated perturbatively

$$\begin{aligned} {\hat{\Gamma }} =\int \frac{1}{2m_t} \overline{\sum } |M|^{2} d\Phi _3, \end{aligned}$$
(4)

where the symbol \(\overline{\sum }\) means to sum over the color and spin of final-state particles and to average over the spin and color of initial-state top quark. \(d\Phi _3\) is the three-body phase space which can be written as

$$\begin{aligned} d{\Phi _3}=(2\pi )^4 \delta ^{4}\left( p_1 - \sum _{f=2}^4 p_{f}\right) \prod _{f=2}^4 \frac{d^3{\mathbf {p}_f}}{(2\pi )^3 2p_f^0}, \end{aligned}$$
(5)

It is helpful to get the differential distributions, \(d\Gamma /ds_{ij}\) and \(d\Gamma /d\cos \theta _{ij}\), for experimental studies, where the invariant masses \(s_{ij}=(p_i+p_j)^2\) and \(\theta _{ij}\) is the angle between \(\mathbf {p}_i\) and \(\mathbf {p}_j\) for \(i,j=2,3,4\).

The amplitude can be generally expressed as

$$\begin{aligned} i M_{ss'}[n] = \mathcal{{C}}\; {\bar{u}_{s i}}({p_3}) \sum \limits _{l = 1}^{m} {\mathcal{A}_l[n] } {u_{s' j}}({p_1}), \end{aligned}$$
(6)

where \(m=10\) stands for the number of Feynman diagrams of this processes, s and \(s'\) are spin indices, i and j are color indices of the outgoing Q quark and the initial top quark, respectively. The color factor \(\mathcal{C}\) for the color-singlet production is \(\frac{4}{3 \sqrt{3} }\delta _{ij}\). The amplitude \(\mathcal{A}_{l}[n]\) for each hadronic state can be read out from Feynman diagrams in Fig. 1. It is worth mentioning that \(v(p_{22})\bar{u}(p_{21})\) for \((c{\bar{Q}})\)-quarkonium in \(\mathcal{A}_l[n]\) must be replaced by the projector \(\Pi _{p_2}[n]\) for each corresponding Fock state. And the projector \(\Pi _{p_2}[n]\) for the spin-singlet or spin-triplet S-wave states can be written as [61]:

(7)

where \(\epsilon [^1S_0]=\gamma _5\) and with \(\epsilon ^\rho \) is the polarization vector of \(^3S_1\) state.

As for the present considered one-loop triangle integrals with three internal masses, it is noted that there is no ultra-violet divergence [62], thus we can get the finite results by directly performing the loop integrals. More explicitly, the amplitudes \(A_{l}\) are given in Appendix A.

3 Numerical results

We use FeynArts 3.9 [63] to generate amplitudes and the modified FormCalc 7.3/LoopTools 2.1 [64] to do the algebraic and numerical calculations. We set the typical renormalization scale \(\mu _R\) to be \(2m_c\) (\(2m_b\)) for the production of charmonium (\((c{\bar{b}})\)-quarkonium) accordingly, leading to \(\alpha _s(2m_c)=0.259\) and \(\alpha _s(2m_b)=0.181\). Because the wavefunction at the zero is an overall factor and its uncertainty can be conventionally discussed when we know its exact values, thus we shall directly take the wavefunction at the zero to be the one derived from the QCD (Buchmuller-Type) potential model [60]. We set the masses of the ground states charmonium and (\(c{\bar{b}}\))-quarkonium as 3 GeV [65, 66] and 6.4 GeV [67,68,69] by default. As a summary, the relevant input parameters are as follows:

$$\begin{aligned}&m_{Z}=91.1876~{\mathrm{GeV}}, ~~m_{W}=80.385~{\mathrm{GeV}}, \\&m_t= 173.0~{\mathrm{GeV}},\,\,m_c=1.50~{\mathrm{GeV}}, ~~m_b=4.90~{\mathrm{GeV}}, \\&m_s=0.101~{\mathrm{GeV}},\,\,|R_S(c{\bar{c}})(0)|^2=0.810\;{\mathrm{GeV}}^3,\\&|R_S(c{\bar{b}})(0)|^2=1.642\;{\mathrm{GeV}}^3,\\&G_F=1.1663787\times 10^5. \end{aligned}$$

3.1 The charmonium and \((c{\bar{b}})\)-quarkonium production via FCNC

Total decay width for the process \(t\rightarrow cZ^{0}\) is \(9.59\times 10^{-13}\) GeV which is small due to the strong GIM suppression from the small values of the internal quark masses \(m_{b, s, d}\). As a subtle point, contribution from the d quark loop is negligible due to small CKM matrix element \(|V_{td}|\) and its small mass.

Table 1 The decay widths and the corresponding branching ratios for the \((c{\bar{Q}})\)-quarkonium production via the channel \(t\rightarrow |(c{\bar{Q}})[n]\rangle +Q+ Z^{0}\). The ratio \(R={\Gamma _{t\rightarrow |(c{\bar{Q}})[n]\rangle }}/{\Gamma _{t\rightarrow cZ^{0}}}\)
Fig. 2
figure 2

The differential decay widths \(d\Gamma /ds_{23}\) (a), \(d\Gamma /ds_{24}\) (b), and \(d\Gamma /ds_{34}\) (c) for \( t\rightarrow |(c{\bar{Q}})[n]\rangle +Q+Z^{0}\)

Fig. 3
figure 3

The differential decay widths \(d\Gamma /d\cos \theta _{23}\) (a), \(d\Gamma /d\cos \theta _{24}\) (b) and \(d\Gamma /d\cos \theta _{34}\) (c) for \( t\rightarrow |(c{\bar{Q}})[n]\rangle +Q+Z^{0}\)

The decay width and corresponding branching ratios for the production of the \((c{\bar{Q}})\)-quarkonium through the channel \(t \rightarrow |(c{\bar{Q}})[n]\rangle +Q +Z^{0}\) via FCNC are listed in Table 1. Table 1 shows the decay width of the charmonium production is almost two orders of magnitude larger than that of the \((c{\bar{b}})\)-quarkonium production.

We present the differential distributions over the invariant masses \(d\Gamma /ds_{23}\), \(d\Gamma /ds_{24}\) and \(d\Gamma /ds_{34}\) and the differential distributions over the angles \(d\Gamma /d\cos \theta _{23}\), \(d\Gamma /d\cos \theta _{24}\) and \(d\Gamma /d\cos \theta _{34}\) between the final particles for the \(|(c{\bar{Q}})[n]\rangle \) production in Figs. 2, 3, respectively. In Fig. 2 the sharp peaks in low region of \(d\Gamma /ds_{23}\) indicate the largest contribution emerges when the heavy quarkonium moves along with the same direction of the outgoing quark but with the opposite direction of the outgoing \(Z^0\) boson. This feature is consistent with angle distributions in Fig. 3.

3.2 Uncertainties for the charmonium and \((c{\bar{b}})\)-quarkonium production via FCNC

There are uncertainties from different choices of quark masses, renormalization scale and wavefunction uncertainties. In this subsection, we discuss the uncertainties from the quark masses and the renormalization scale.

Table 2 Uncertainties of the decay width for the process \(t\rightarrow |(c{\bar{Q}})[n]\rangle +Q +Z^0\) by varying \(m_c\in [1.25,1.75]\) GeV
Table 3 Uncertainties of the decay width for the process \(t\rightarrow |(c{\bar{Q}})[n]\rangle +Q +Z^0\) by varying \(m_b\in [4.50,5.30]\) GeV
Table 4 Uncertainties of the decay width for the process \(t\rightarrow |(c{\bar{Q}})[n]\rangle +Q +Z^0\) by varying \(m_t\in [169.0,177.0]\) GeV

In Tables 2, 3 and 4, we present the uncertainties caused by \(m_c\), \(m_b\) and \(m_t\) within the range of \(m_c=1.50\pm 0.25\) GeV, \(m_b=4.90\pm 0.40\) GeV and \(m_t=173.0\pm 4.0\) GeV. When varying one mass parameter, the other two parameters are fixed to be their central values. Tables 2, 3 and 4 indicate that the mass uncertainties are large. The decay width for the production of both charmonium and \((c{\bar{b}})\)-quarkonium will increase \(1\%\sim 10\%\) with the increment of \(m_t\). For the charmonium production, its decay width decreases with the increment of \(m_c\) and increases with the increment of \(m_b\). For the production of \((c{\bar{b}})\)-quarkonium, the decay width increases slower with the increment of \(m_b\). The total decay widthes with mass uncertainties are

$$\begin{aligned} \Gamma _{t\rightarrow \eta _c}= & {} 1.20^{+1.04}_{-0.51}\times 10^{-16}\; {\mathrm{GeV}}, \end{aligned}$$
(8)
$$\begin{aligned} \Gamma _{t\rightarrow J/\psi }= & {} 1.37^{+1.03}_{-0.51}\times 10^{-16}\; {\mathrm{GeV}}, \end{aligned}$$
(9)
$$\begin{aligned} \Gamma _{t\rightarrow B_c}= & {} 2.06^{+0.17}_{-0.17}\times 10^{-18}\; {\mathrm{GeV}}, \end{aligned}$$
(10)
$$\begin{aligned} \Gamma _{t\rightarrow B^*_c}= & {} 6.27^{+0.63}_{-0.62}\times 10^{-18}\; {\mathrm{GeV}}, \end{aligned}$$
(11)

where the uncertainties from various quark masses are summed up in quadrature.

Table 5 Scale uncertainties of the decay width for the process \(t\rightarrow |(c{\bar{Q}})[n]\rangle +Q +Z^0\) by varying the typical renormalization scale \(\mu _R\) from \(\frac{1}{2}\mu _R\) to \(2\mu _R\). \(\mu _R=2m_c\) for charmonium and \(\mu _R=2m_b\) for \((c{\bar{b}})\)-quarkonium
Fig. 4
figure 4

The differential decay width \(d\Gamma /ds_{23}\) (a), \(d\Gamma /ds_{24}\) (b) and \(d\Gamma /ds_{34}\) (c) with renormalization scale uncertainty for \( t\rightarrow |(c{\bar{c}})[n]\rangle +c+Z^{0}\)

Fig. 5
figure 5

The differential decay width \(d\Gamma /ds_{23}\) (a), \(d\Gamma /ds_{24}\) (b) and \(d\Gamma /ds_{34}\) (c) with renormalization scale uncertainty for \( t\rightarrow |(c{\bar{b}})[n]\rangle +b +Z^{0}\)

We present the scale uncertainties by varying the scale \(\mu _R\) within the range of \([\mu _R/2,2\mu _R]\) in Table 5. Generally, the scale uncertainty can be suppressed by including high-order terms or by using an optimized scaling-setting method [70, 71]. Here we set the renormalization scale to be \(\mu _R=2m_c\) for charmonium production and \(\mu _R=2m_b\) for \((c{\bar{b}})\)-quarkonium production. Scale uncertainties for total invariant mass distributions are shown for the production of charmonium and \((c{\bar{b}})\)-quarkonium in Figs. 4, 5. Considering that the selected renormalization scale is small for the production of charmonium, the uncertainty is relatively larger than that for the production of \((c{\bar{b}})\)-quarkonium.

Fig. 6
figure 6

The Feynman diagrams for \(t(p_1)\rightarrow |(c{\bar{b}})[n]\rangle (p_2) + b(p_3) + Z^{0}(p_4)\) without FCNC, where n stands for the two color-singlet S-wave states

3.3 Background for the \((c{\bar{b}})\)-quarkonium production

For the production of \((c{\bar{b}})\)-quarkonium with the same final states, there is another production channel, which could be treated as the background for observing the FCNC effect. The Feynman diagrams for the decay \(t(p_1)\rightarrow |(c{\bar{b}})[n]\rangle (p_2) + b(p_3) + Z^{0}(p_4)\) without FCNC are plotted in Fig. 6, where n stands for the two color-singlet S-wave states. For this channel, the short-distance amplitudes are

$$\begin{aligned} i M_{ss'}[n] = \mathcal{{C}}\; \sum \limits _{l = 11}^{16} {\mathcal{A}_l[n]}, \end{aligned}$$
(12)

where \(\mathcal {C}\) is \(\frac{3\delta _{ij}}{\sqrt{3}}\) and the amplitudes \(\mathcal{A}_{l}[n]\) are listed in Appendix B.

Fig. 7
figure 7

The differential decay width not via FCNC \(d\Gamma /ds_{23}\) (a), \(d\Gamma /ds_{24}\) (b) and \(d\Gamma /ds_{34}\) (c) for \( t\rightarrow |(c{\bar{b}})[n]\rangle +b+Z^{0}\)

Fig. 8
figure 8

The differential decay width not via FCNC \(d\Gamma /d\cos \theta _{23}\) (a), \(d\Gamma /d\cos \theta _{24}\) (b) and \(d\Gamma /d\cos \theta _{34}\) (c) for \( t\rightarrow |(c{\bar{b}})[n]\rangle +b+Z^{0}\)

We present the invariant mass and the angular distributions for the production of \(|(c{\bar{b}})[^1S_0]\) and \(|(c{\bar{b}})[^3S_1]\) without FCNC in Figs. 7, 8. Figures 3, 8 show the angular distributions \(d\Gamma /d\cos \theta _{24}\) and \(d\Gamma /d\cos \theta _{34}\) are close in shape, which the angular distribution \(d\Gamma /d\cos \theta _{23}\) is quite different for the decay channels with or without FCNC. For example, the distribution \(d\Gamma /d\cos \theta _{34}\) for the production without FCNC reaches its maximum value for \(\theta _{23}=0\), while the distribution \(d\Gamma /d\cos \theta _{34}\) for the production with FCNC reaches its maximum value for \(\theta _{23}=1\). This difference is caused by the fact that for the production without FCNC, the quark components of \((c{\bar{b}})\)-quarkonium are all from a off-shell \(W^+\) boson. After integration, the total decay widths for the background process are \(\Gamma (t\rightarrow B_c)=1.32\times 10^{-12}\) GeV and \(\Gamma (t\rightarrow B^*_c)=1.26\times 10^{-12}\) GeV, respectively. They are larger than those of FCNC channels by about \(10^5\sim 10^6\) times, thus when searching of new physics signals from the FCNC channels, those background should be taken into consideration.

3.4 New physics effects

To simply estimate the new physics effects, we adopted \(\Gamma =\Gamma _{t}\times BR(t\rightarrow cZ^0)\times R\), where \(\Gamma _{t}\) is the total decay width of top quark about 2 Gev, the related ratio R is given in subsection A and can be considered to be consistent with the SM on the order of magnitude. The branching ratio BR\((t\rightarrow cZ^0)\) has been studied in detail with many new models. Here we listed some estimated results in some new physics in Table 6. We can find that the production of charmonium and \((c{\bar{b}})\)-quarkonium through top quark decays may be accessible at LHC or HL-LHC running at \(\sqrt{s}\) = 14 TeV and with the integrated luminosity of 3 \(ab^{-1}\).

Table 6 The estimation of new physics effect with several new models

4 Summary

The rare FCNC process is generally forbidden at the tree level in the SM, which is small and is used for searching of new physics beyond the SM. Within the framework of NRQCD, we have done a detailed study on the production of heavy-quarkonium through top quark semi-exclusive decays via FCNC, \(t\rightarrow |(c{\bar{Q}})[n]\rangle +Q + Z^{0}\), where Q stands for c or b quark, respectively. If assuming the spin-triplet \(|(c{\bar{Q}})[^3S_1]\rangle \) decays to the ground \(|(c{\bar{Q}})[^1S_0]\rangle \) with \(100\%\) efficiency, the total decay width are as follows:

$$\begin{aligned} \Gamma _{t\rightarrow |(c{\bar{c}})[^1S_0]\rangle }= & {} 2.57^{+2.07+2.44}_{-1.02-0.96}\times 10^{-16}~{\mathrm{GeV}}, \end{aligned}$$
(13)
$$\begin{aligned} \Gamma _{t\rightarrow |(c{\bar{b}})[^1S_0]\rangle }= & {} 8.33^{+0.80+3.69}_{-0.79-2.18}\times 10^{-18}~{\mathrm{GeV}}, \end{aligned}$$
(14)

where the uncertainties from various quark masses and renormalization scales are summed up in quadrature. Various differential distributions have also been presented. Even though the decay widths are small, they are still important, which will provide useful guidance for searching of new physics beyond the SM from the heavy quarkonium involved processes.