1 Introduction

The rare B and D meson decay processes driven by a flavour changing neutral current (FCNC) transitions constitute a subject of great interest in the area of electroweak interactions and provide an excellent testing ground to look for new physics beyond the standard model (SM). The FCNC decays are highly suppressed in the SM and occur only at one-loop level. Of particular interest among the FCNC decays are the rare semileptonic B meson decays involving the transitions \(b \rightarrow s l^+ l^-\), where several anomalies at the level of few sigma have been observed recently in the LHCb experiment [1,2,3,4]. To complement these results, efforts should also be made towards the search for new physics signal in the up quark sector, mainly in the rare charm meson decays involving \(c \rightarrow u l^+ l^-\) quark level transitions. Recently the LHCb experiment has searched for the branching ratio of the lepton flavour violating (LFV) \(D^0 \rightarrow \mu ^{\mp } e^{\,\pm \,}\) decays and put the limit as \(\mathrm{BR}(D^0 \rightarrow \mu ^{\mp } e^{\,\pm \,})~ <~ 1.3 \times 10^{-8}\) [5] at \(90\%\) confidence level (CL). On the other hand, both the Belle and the BaBar experiments have reported significant deviations on the measured branching fractions of \({\bar{B}} \rightarrow D^{(*)} \tau \nu _{\tau }\) processes from the corresponding SM predictions. The ratio of these branching fractions, the so-called \(R(D^{(*)})\), defined as \(R(D^{(*)}) = \mathrm{BR}({\bar{B}} \rightarrow D^{(*)} \tau \nu _{\tau })/ \mathrm{BR}({\bar{B}} \rightarrow D^{(*)} l \nu _l)\), where \(l=e, \mu \), exceed the SM prediction by \(3.5 \sigma \) [6], thus opening an excellent window to search for new physics (NP) in the up quark sector.

Mixing between a neutral meson and its anti-meson with a specific flavour provides an useful tool to deal with problems in flavour sector. For example, in the past the \(K^0-{\bar{K}}^0\) and \(B^0-{\bar{B}}^0\) oscillations, involving mesons made of up- and down-type quarks, have provided information as regards the charm and top quark mass scale, much before the discovery of these particles in the collider. On the other hand, the \(D^0 - {\bar{D}}^0\) system involves mesons with up-type quarks and in the SM the mixing rate is sufficiently small, so that the new physics component might play an important role in this case. The mixing parameters required to describe the \(D^0 - {\bar{D}}^0\) mixing are defined by \(x = \Delta M{/}\Gamma \) and \(y= \Delta \Gamma / 2 \Gamma \), where \(\Delta M\) (\(\Delta \Gamma \)) is the mass (width) difference between the mass eigenstates.

In this paper, we focus on the analysis of rare charm meson decays induced by \(c \rightarrow u \mu ^+ \mu ^-\) and \(c \rightarrow u \mu ^{\mp } e^{\,\pm \,}\) FCNC transitions. We calculate the branching ratios, forward–backward asymmetry parameters and the flat terms in \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^+ \mu ^-\) processes both in the scalar leptoquark (LQ) and the generic \(Z^\prime \) model. These processes suffer from resonance background through \(c \rightarrow u M \rightarrow u l^+ l^-\), where M denotes \(\eta ^{(')}\) (pseudoscalar), \(\rho , \phi , \omega \) (vector) mesons. However, to reduce the background coming from these resonances, we work in the low and high \(q^2\) regimes, i.e., \( q^2 \in [0.0625, 0.275]\,\mathrm{GeV}^2\) and \( q^2 \in [1.56, 4.00]\,\mathrm{GeV}^2\), which lie outside the mass square range of the resonant mesons. We also compute the branching ratios of lepton flavour violating \(D^0 \rightarrow \mu e, \tau e\) and \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^- e^+\) processes. These LFV processes have negligible contributions from the SM, as they proceed through the box diagrams with tiny neutrino masses in the loop. However, they can occur at tree level in the LQ and \(Z^\prime \) models and are expected to have significantly large branching ratios. Leptoquarks are hypothetical colour triplet bosonic particles, which couple to quarks and leptons simultaneously and contain both baryon and lepton quantum numbers. It is interesting to study flavour physics with leptoquarks as they allow quark–lepton transitions at tree level, thus explaining several observed anomalies, e.g., the lepton non-universality (LNU) parameter \(R_K|_{q^2 \in [1,6]\,\mathrm{GeV}^2}=\mathrm{BR}(B \rightarrow K \mu ^+ \mu ^-)/\mathrm{BR}(B \rightarrow K e^+ e^-)\) in rare B decays. The existence of the scalar leptoquark is predicted in the extended SM theories, such as grand unified theory [7,8,9,10,11], the Pati–Salam model, the extended technicolour model [12, 13] and the composite model [14]. In this work, we consider the model which conserves baryon and lepton numbers and does not allow proton decay. Here we would like to see how this model affects the leptonic and semieptonic decays of the \(D^0\) meson induced by \(c \rightarrow u l^+ l^-\) transitions. The phenomenology of scalar leptoquarks and their implications to the B and D sector has been extensively studied in the literature [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51].

The \(Z^\prime \) boson is a colour singlet vector gauge boson and it is electrically neutral in nature. By adding an additional \(U(1)^\prime \) gauge symmetry, the new \(Z^\prime \) gauge boson could be naturally derived from the extension of electroweak symmetry of the SM, such as superstring theories, grand unified theories and theories with large extra dimensions. The processes mediated via \(c \rightarrow u\) FCNC transitions could be induced by the generic \(Z^\prime \) model at tree level. The theoretical framework of the heavy new \(Z^\prime \) gauge boson has been studied in the literature [52, 53, 56, 57]. In this paper, we investigate the \(Z^\prime \) contribution to the rare \(D^0\) meson decay processes within the parameter space constrained by \(D^0 - {\bar{D}}^0\) mixing and \(D^0 \rightarrow \mu ^+ \mu ^-\) processes.

The paper is organized as follows. In Sect. 2, we discuss the effective Hamiltonian describing \(\Delta C=1\) transitions i.e., \(c \rightarrow u l^+ l^-\), and \(\Delta C=2\) transition, which is responsible for \(D^0 - {\bar{D}}^0\) mixing. The new physics contribution to \(c \rightarrow u\) transitions and the constraint on leptoquark couplings from the \(D^0 - {\bar{D}}^0\) oscillation and the process \(D^0 \rightarrow \mu ^+ \mu ^-\) are discussed in Sect. 3. We calculate the constraint on \(Z^\prime \) couplings from \(D^0 - {\bar{D}}^0\) mixing and leptonic \(D^0 \rightarrow \mu ^+ \mu ^-\) decays in Sect. 4. In Sect. 5, we compute the branching ratios, forward–backward asymmetry parameters and the flat terms of the process \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^+ \mu ^-\) in both these models. The lepton flavour violating \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^- e^+\) and \(D^0 \rightarrow \mu e, \tau e \) processes are discussed in Sects. 6 and 7. Finally we summarize our findings in Sect. 8.

2 Effective Hamiltonian for \(\Delta C=1\) and \(\Delta C=2\) transitions

Though the rare charm decays are affected by large non-perturbative effects, the short distance structure of FCNC transitions can be investigated well theoretically. The change in charm quantum number for rare FCNC charm meson decays is either of two units or one unit, and hence, they involve either \(\Delta C=2\) or \(\Delta C=1\) transitions. The \(D^0 - {\bar{D}}^0\) mixing takes place via a \(\Delta C=2\) transition and the decay processes with \(\Delta C=1\) transitions are \(c \rightarrow u l^+ l^-\) and \(c \rightarrow u \gamma \).

If we integrate out the heavy degrees of freedom associated with the new interactions at a scale M, an effective Hamiltonian in the form of a series of operators of increasing dimensions can be obtained. However, the operators of dimension \(d=6\) have important contributions to charm meson decays or mixing. In general, one can write the complete basis of these effective operators in terms of chiral quark fields for both \(D^0 - {\bar{D}}^0\) mixing and the process \(D^0 \rightarrow l^+ l^-\) as [52, 53]

$$\begin{aligned} \langle f | \mathcal{H} | i \rangle = G \sum _{i=1} \mathrm{C}_i (\mu ) ~ \langle f | Q_i | i \rangle (\mu ), \end{aligned}$$
(1)

where G has inverse-mass squared dimensions, \(C_i\) are the Wilson coefficients.Footnote 1

The effective operators for \(D^{0}-{\bar{D}}^{0}\) mixing at the heavy mass scale M are given by [52, 53]

$$\begin{aligned} \begin{array}{l} Q_1 = (\overline{u}_L \gamma _\mu c_L) \ (\overline{u}_L \gamma ^\mu c_L), \\ Q_2 = (\overline{u}_L \gamma _\mu c_L) \ (\overline{u}_R \gamma ^\mu c_R), \\ Q_3 = (\overline{u}_L c_R) \ (\overline{u}_R c_L), \\ Q_4 = (\overline{u}_R c_L) \ (\overline{u}_R c_L), \end{array} \quad \begin{array}{l} Q_5 = (\overline{u}_R \sigma _{\mu \nu } c_L) \ ( \overline{u}_R \sigma ^{\mu \nu } c_L), \\ Q_6 = (\overline{u}_R \gamma _\mu c_R) \ (\overline{u}_R \gamma ^\mu c_R), \\ Q_7 = (\overline{u}_L c_R) \ (\overline{u}_L c_R), \\ Q_8 = (\overline{u}_L \sigma _{\mu \nu } c_R) \ (\overline{u}_L \sigma ^{\mu \nu } c_R), \end{array}\nonumber \\ \end{aligned}$$
(2)

where \(q_{L(R)}=L(R)q\) are the chiral quark fields with \(L(R)=(1{\mp } \gamma _5)/2\) as the projection operators.

In the standard model, the effective weak Hamiltonian for the \(c \rightarrow u\) transitions at the scale \(\mu =m_c\) can be written as the sum of three contributions [49, 54, 55],

$$\begin{aligned} \mathcal {H}_{\mathrm{eff}}=\lambda _d \mathcal {H}^d + \lambda _s \mathcal {H}^s + \lambda _b \mathcal {H}^\mathrm{peng}, \end{aligned}$$
(3)

where \(\lambda _q=V_{uq}V_{cq}^*\) is the product of Cabibbo–Kobayashi–Maskawa (CKM) matrix elements. The explicit form of \(\mathcal {H}^\mathrm{peng}\), which basically is responsible for the \(c \rightarrow u l^+ l^-\) transition is given by

$$\begin{aligned} \mathcal {H}^\mathrm{peng}= -\frac{4G_F}{\sqrt{2}}\left( \sum _{i=3,\ldots 10,S,P} C_i \mathcal {O}_i+ \sum _{i=7,\ldots 10,S,P}C_i' \mathcal {O}_i'\right) , \nonumber \\ \end{aligned}$$
(4)

where \(G_F\) is the Fermi constant, the \(C_i\)’s are the Wilson coefficients evaluated at the charm quark mass scale \((\mu =m_c)\) at Next-Next-to-Leading-Order (NNLO) [58]. We use the two loop result of Ref. [59] for the \(C_7^\mathrm{eff}(m_c)\) Wilson coefficients, \(V_{cb}^* V_{ub} C_7^\mathrm{eff} = V_{cs}^* V_{us} (0.007 + 0.020i)(1{\,\pm \,} 0.2)\) and the corresponding effective operators for the \(c \rightarrow u l^+ l^-\) transitions are given by [49]

$$\begin{aligned} \mathcal {O}_7^{(')}= & {} \frac{e}{16\pi ^2}m_c({\bar{u}}\sigma _{\mu \nu } R(L) c)F^{\mu \nu }\ , \nonumber \\ \mathcal {O}_9^{(')}= & {} \frac{e^2}{16\pi ^2} ({\bar{u}}\gamma _\mu L(R) c)( {\bar{\ell }}\gamma ^\mu \ell ) \ ,\nonumber \\ \mathcal {O}_{10}^{(')}= & {} \frac{e^2}{16\pi ^2} ({\bar{u}}\gamma _\mu L(R) c)( {\bar{\ell }}\gamma ^\mu \gamma _5\ell ), \nonumber \\ \mathcal {O}_S^{(')}= & {} \frac{e^2}{16\pi ^2} ({\bar{u}} R(L) c)( {\bar{\ell }} \ell ) \ , \nonumber \\ \mathcal {O}_{P}^{(')}= & {} \frac{e^2}{16\pi ^2} ({\bar{u}} R(L) c)( {\bar{\ell }} \gamma _5\ell ),\nonumber \\ \mathcal {O}_T= & {} \frac{e^2}{16\pi ^2} ({\bar{u}} \sigma _{\mu \nu } c)( {\bar{\ell }} \sigma ^{\mu \nu } \ell ) \ , \nonumber \\ \mathcal {O}_{T_5}= & {} \frac{e^2}{16\pi ^2} ({\bar{u}} \sigma _{\mu \nu } c)( {\bar{\ell }} \sigma ^{\mu \nu } \gamma _5\ell ). \end{aligned}$$
(5)

The contributions from the primed operators as well as the scalar, pseudoscalar and tensor operators are absent in the SM and arise only in scenarios beyond the standard model. The renormalization group running does not affect the \(\mathcal {O}_{10}\) operator, i.e., \(C_{10}(m_c) = C_{10}(M_W) \propto (m_{d,s}^2/m_W^2)\) and, hence, the Wilson coefficient \(C_{10}\) is negligible in the SM.

3 New physics contribution due to the exchange of scalar leptoquarks

The presence of leptoquarks can modify the SM effective Hamiltonian of \(c \rightarrow u\) transitions, giving appreciable deviation from the SM values. These colour triplet bosons can be either scalars or vectors. There exist three scalar and four vector relevant leptoquark states which potentially contribute to the \(c \rightarrow u l^+ l^-\) transitions and are invariant under the SM gauge group \(SU(3)_C \times SU(2)_L \times U(1)_Y\), where the hypercharge Y is related to the electric charge and weak isospin (I) through \(Y= Q-I_3\). Out of three possible scalar leptoquarks with the quantum numbers \((3,3,-1/3)\), \((3,1,-1/3)\) and (3, 2, 7 / 6) [49, 50], only the leptoquark with multiplet (3, 2, 7 / 6) conserves both baryon and lepton numbers and, thus, avoids rapid proton decay at the electroweak scale. Similarly out of the vector multiplets (3, 3, 2 / 3), (3, 1, 5 / 3), (3, 2, 1 / 6) and \((3,2,-5/6)\), only the first two leptoquark states do not allow baryon number violation and can be considered to study the observed anomalies in flavour sector. In this work we consider the baryon number conserving \(X=(3,2,7/6)\) scalar leptoquark which induces the interaction between the up-type quarks and charged leptons and, thus, contributes to the semileptonic decay amplitudes.

The interaction Lagrangian of the \(X=(3,2,7/6)\) scalar leptoquark with the SM bilinears is given by [49, 50]

$$\begin{aligned} \mathcal {L} ={\bar{l}}_R Y^L \Delta ^\dagger Q + {\bar{u}}_R Y^R \tilde{\Delta }^\dagger L +\mathrm h.c., \end{aligned}$$
(6)

where \(\tilde{\Delta } = i \tau _2 \Delta ^*\) represents the conjugate state. The transition of weak basis to mass basis divides the Yukawa couplings to two part of couplings pertinent for the upper and lower doublet components. The left-handed quark and lepton doublets are represented by Q and L and \(u_R(l_R)\) is the right-handed quark (charged-lepton) singlet. We use the basis where CKM and PMNS rotations are assigned to down-type quarks and neutrinos, i.e., \(d_L \rightarrow V_{CKM} d_L\) and \(\nu _L \rightarrow V_{PMNS} \nu _L\). Here \(Y^{L}\) and \(Y^R\) are the leptoquark couplings in the mass basis of the up-type quarks and charged leptons. Now writing the leptoquark doublets in terms of its components as \(\Delta =(\Delta ^{(5/3)}, \Delta ^{(2/3)})^T\), where the superscripts denote the electric charge of the LQ components and expanding the terms in Eq. (6), one can obtain the interaction Lagrangian for different components of LQs given as [50]

$$\begin{aligned} \mathcal {L}^{(2/3)}= & {} \left( {\bar{l}}_R [Y^L V_\mathrm{CKM}] d_L \right) {\Delta ^{(2/3)}}^*\nonumber \\&+\;\left( {\bar{u}}_R [Y^R V_\mathrm{PMNS}] \nu _L \right) \Delta ^{(2/3)} + h.c.,\nonumber \\ \mathcal {L}^{(5/3)}= & {} \left( {\bar{l}}_R Y^L u_L \right) {\Delta ^{(5/3)}}^* - \left( {\bar{u}}_R Y^R l_L \right) \Delta ^{(5/3)} + h.c. .\nonumber \\ \end{aligned}$$
(7)

Thus, one can see from (7) that only \(\Delta ^{(5/3)}\) component mediates the interaction between up-type quarks and charged lepton. Now applying the Fierz transformation, we obtain additional contributions to the SM Wilson coefficients for \(c \rightarrow u \mu ^+ \mu ^-\) transition as [49]

$$\begin{aligned} C_9^\mathrm{LQ}= & {} C_{10}^\mathrm{LQ}=-\frac{\pi }{2\sqrt{2}G_F \alpha _{em} \lambda _b} \frac{Y_{\mu c}^L Y_{\mu u}^{L *}}{m_\Delta ^2}, \nonumber \\ C_9^{\prime \mathrm{LQ}}= & {} -C_{10}^{\prime \mathrm{LQ}}=-\frac{\pi }{2\sqrt{2}G_F \alpha _{em} \lambda _b} \frac{Y_{c \mu }^{R *} Y_{u \mu }^{R}}{m_\Delta ^2}, \nonumber \\ C_S^\mathrm{LQ}= & {} C_{P}^\mathrm{LQ}=-\frac{\pi }{2 \sqrt{2}G_F \alpha _{em} \lambda _b} \frac{Y_{\mu u}^{L *} Y_{c \mu }^{R *}}{m_\Delta ^2}, \nonumber \\ C_S^{\prime \mathrm{LQ}}= & {} -C_{P}^{\prime \mathrm{LQ}}=-\frac{\pi }{2\sqrt{2}G_F \alpha _{em} \lambda _b} \frac{Y_{\mu c}^L Y_{u \mu }^{R}}{m_\Delta ^2}, \nonumber \\ C_T^\mathrm{LQ}= & {} -\frac{\pi }{8\sqrt{2}G_F \alpha _{em} \lambda _b} \frac{Y_{u\mu }^R Y_{\mu c}^L + {Y_{c \mu }^R}^* {Y_{\mu u}^L}^*}{m_\Delta ^2}, \nonumber \\ C_{T_5}^\mathrm{LQ}= & {} -\frac{\pi }{8\sqrt{2}G_F \alpha _{em} \lambda _b} \frac{-Y_{u\mu }^R Y_{\mu c}^L + {Y_{c \mu }^R}^* {Y_{\mu u}^L}^*}{m_\Delta ^2}, \end{aligned}$$
(8)

where \(\alpha _{em}\) is the fine structure constant. After having an idea about the new Wilson coefficients, we now proceed to constrain the combination of LQ couplings using the experimental data on \(D^0 - {\bar{D}}^0\) mixing and the process \(D^0 \rightarrow l^+ l^-\), where \(l=\mu , e\).

3.1 Constraint on leptoquark couplings from \(D^0 - {\bar{D}}^0\) mixing

In the standard model, \(D^0 - {\bar{D}}^0\) mixing proceeds through the box diagrams with an internal down-type quarks and W-boson exchange and the weak interaction boxes are suppressed due to the GIM mechanism because of the smallness of the down quark mass in comparison to the weak scale. In the LQ model, there will be a contribution to the \(D^0 - {\bar{D}}^0\) mass difference from the box diagrams with the leptoquark and leptons flowing in the loop. Since the SM contribution to the mass difference is very small, we consider its value to be saturated by new physics contributions. Furthermore, the couplings to the left-handed quarks are considered to be zero in order to avoid strict constraints in the down-type quark sector. Thus, considering only right-handed couplings, one can write the effective Hamiltonian due to the leptoquark X(3, 2, 7 / 6) and charged lepton/neutrinos in the loop as [25, 26]

$$\begin{aligned} \mathcal{H}_{\mathrm{eff}}= & {} \sum _{l} \frac{(Y_{lc}^{R} Y_{lu}^{R*})^{2}}{128 \pi ^2}\left[ \frac{1}{ M_{\Delta }^2}~I \left( \frac{m_l^2}{M_\Delta ^2}\right) +\frac{1}{M_\Delta ^2}\right] \nonumber \\&\times \;({\bar{c}} \gamma ^\mu P_R u) ({\bar{c}} \gamma _\mu P_R u)\;, \end{aligned}$$
(9)

where the first term is due to the charged lepton and second term is due to neutrinos in the loop (ignoring the effect of neutrino mixing). The loop function I(x) is given as

$$\begin{aligned} I(x)=\frac{1-x^2+2x \log x}{(1-x)^2}, \end{aligned}$$
(10)

which is very close to 1, i.e., \(I(0)=1\). Using the relation

$$\begin{aligned} \langle {\bar{D}}^0|({\bar{c}} \gamma ^\mu P_R u)({\bar{c}} \gamma _\mu P_R u)|D^0 \rangle = \frac{2}{3} B_D f_{D}^2 M_{D}^2, \end{aligned}$$
(11)

we obtain the contribution due to leptoquark exchange as

$$\begin{aligned} M_{12}^{LQ} = \frac{1}{2 M_D} \langle {\bar{D}}^0 | \mathcal{H}_{\mathrm{eff}}|D^0 \rangle =\frac{\sum _l(Y_{lc }^{R} {Y_{lu}^{R*})^2}}{192 \pi ^2 M_\Delta ^2} B_D f_{D}^2 M_{D}.\nonumber \\ \end{aligned}$$
(12)

Since \(\Delta M_D = 2 |M_{12}|\), we get

$$\begin{aligned} \Delta M_{D} = 2 |M_{12}^{LQ}|= \frac{2}{3} M_D f_D^2 B_D \frac{\left| \sum _l Y_{l c}^{R} {Y_{l u}^{R}}^* \right| ^2}{64 \pi ^2 M_\Delta ^2}, \end{aligned}$$
(13)

where l denotes the charged-lepton flavours. In our analysis, the mass of the \(D^0\) meson is taken from [60], the value of the decay constant \(f_D=222.6 {\,\pm \,} 16.7^{+2.3}_{-2.4}\) MeV [62] and \(B_D (3\,\mathrm{GeV}) = 0.757(27)(4)\) [63]. To obtain the bound on the leptoquark coupling, we assume that the individual leptoquark contribution to the mass difference does not exceed the \(1\sigma \) range of the experimental value. Since we are interested in obtaining the bounds on the \(Y_{\mu c}^{R} Y_{\mu u}^{R*}\) couplings, here we assume that the leptoquark has dominant coupling to muons and its coupling to electron or tau is negligible. The SM contribution to the mass difference is very small and hence can be neglected. The corresponding experimental value is given by [60]

$$\begin{aligned} \Delta M_{ D} = 0.0095^{+ 0.0041}_{-0.0044}\,\mathrm{ps}^{-1}. \end{aligned}$$
(14)

Now comparing the mass difference with the \(1\sigma \) range of experimental data, the bound on leptoquark coupling for a TeV scale LQ is given by

$$\begin{aligned}&7.73 \times 10^{-3} \left( \frac{M_\Delta }{1\,\mathrm{TeV}} \right) \le |Y_{\mu c }^{R} {Y_{\mu u}^{R}}^*| \nonumber \\&\quad \le 1.26 \times 10^{-2} \left( \frac{M_\Delta }{1\,\mathrm{TeV}} \right) , \end{aligned}$$
(15)

which can be translated with Eq. (8) to give the constraint on the new Wilson coefficients

$$\begin{aligned} 0.1 ~\left( \frac{M_\Delta }{1\,\mathrm{TeV}} \right) \le \lambda _b { C}_9^{\prime LQ} =-\lambda _b {C}_{10}^{\prime LQ} \le 0.17~ \left( \frac{M_\Delta }{1\,\mathrm{TeV}} \right) .\nonumber \\ \end{aligned}$$
(16)
Fig. 1
figure 1

The allowed region for the \(\tilde{C}_S {\,\pm \,} \tilde{C}_S^\prime \) Wilson coefficient obtained from \(D^0 \rightarrow \mu ^+ \mu ^-\) (left panel) and \(D^0 \rightarrow e^+ e^-\) processes (right panel)

3.2 Constraint from \(D^0 \rightarrow \mu ^+ \mu ^- (e^+ e^-)\) process

The rare leptonic \(D^0 \rightarrow \mu ^+ \mu ^- (e^+ e^-)\) processes, mediated by the FCNC transitions \(c \rightarrow u l^+ l^-\) at the quark level, are highly suppressed in the SM due to a negligible \(C_{10}\) Wilson coefficient and also suffer from CKM suppression. These processes occur only at one-loop level and are considered as some of the most powerful channels to constrain the new physics parameter space in the charm sector. Analogous to the leptonic B meson decay processes, the only non-perturbative quantity involved is the decay constant of the D meson, which can be reliably calculated using non-perturbative methods such as QCD sum rules, lattice gauge theory and so on. The branching ratio of the process \(D^0 \rightarrow l^+ l^-\) is given by [48, 49]

$$\begin{aligned} \mathrm{BR}\left( D^0 \rightarrow l^+ l^- \right)= & {} \tau _D \frac{G_F^2 \alpha _{em}^2 M_D^5 f_D^2 |\lambda _b |^2}{64 \pi ^3} \sqrt{1-\frac{4m_l^2}{M_D^2}}\nonumber \\&\times \;\left[ \left( 1-\frac{4m_l^2}{M_D^2} \right) \left| \frac{C_S^\mathrm{LQ}-C_S^{\prime \mathrm{LQ}}}{m_c} \right| ^2 \right. \nonumber \\&\left. +\left| \frac{C_P^\mathrm{LQ}-C_P^{\prime \mathrm{LQ}}}{m_c} + \frac{2m_l}{M_D^2} \left( C_{10}^\mathrm{LQ}-C_{10}^{\prime \mathrm{LQ}} \right) \right| ^2 \right] .\nonumber \\ \end{aligned}$$
(17)

The process \(D^0 \rightarrow \mu ^+ \mu ^-\) has a dominant intermediate \(\gamma ^* \gamma ^*\) state in the SM, which is electromagnetically converted to a \(\mu ^+ \mu ^-\) pair. After including the contribution of the \(\gamma ^* \gamma ^*\) intermediate state, the predicted branching ratio of this process is \(\mathrm{BR}(D^0 \rightarrow \mu ^+ \mu ^-)\simeq 2.7 \times 10^{-5} \times \mathrm{BR}(D^0 \rightarrow \gamma \gamma )\) [64]. Using the upper bound \(\mathrm{BR}(D^0 \rightarrow \gamma \gamma ) < 2.2 \times 10^{-6}\) at \(90\%\) CL reported in [65], the estimated limit on the branching ratio is \(\mathrm{BR} \left( D^0 \rightarrow \mu ^+ \mu ^- \right) ^{SM} \lesssim 10^{-10}\) [49]. The present experimental limits on the branching ratios of the dileptonic decays of the D meson are [60]

$$\begin{aligned} \mathrm{BR} \left( D^0 \rightarrow \mu ^+ \mu ^- \right)< & {} 6.2 \times 10^{-9},\nonumber \\ \mathrm{BR} \left( D^0 \rightarrow e^+ e^- \right)< & {} 7.9 \times 10^{-8}. \end{aligned}$$
(18)

Using the above experimental bounds, the constraint on the leptoquark coupling can be obtained by imposing the condition that the individual leptoquark contribution to the branching ratio does not exceed the experimental limit. In this analysis, we neglect the new physics contribution to the Wilson coefficient \(C_{10}\), as the scalar and pseudoscalar Wilson coefficients will be dominating due to the additional multiplication factor \(M_D/m_l\) as noted from Eq. (17). Now, redefining the Wilson coefficients by

$$\begin{aligned} \tilde{C}_i^{(')LQ}=\lambda _b C_i^{(')LQ}, \end{aligned}$$
(19)

we show in Fig. 1, the allowed region in the \(\tilde{C}_S^{LQ}-\tilde{C}_S^{\prime LQ}\), \(\tilde{C}_S^{LQ}+\tilde{C}_S^{\prime LQ}\) plane, obtained from the \(D^0 \rightarrow \mu ^+ \mu ^-\) (left panel) and \(D^0 \rightarrow e^+ e^-\) processes (right panel). Here we have used the relations \(C_S^{LQ}=C_P^{LQ}\) and \(C_S^{'LQ}=-C_P^{'LQ}\) from Eq. (8). From the figure, we find the allowed range for the above combinations of Wilson coefficients from the process \(D^0 \rightarrow \mu ^+ \mu ^-\) to be

$$\begin{aligned} \Big | \tilde{C}_S^{LQ}-\tilde{C}_S^{\prime LQ} \Big |~ \leqslant ~ 0.06, \quad \Big | \tilde{C}_S^{LQ}+\tilde{C}_S^{\prime LQ} \Big |~ \leqslant ~ 0.06, \end{aligned}$$
(20)

whereas the bounds obtained from the process \(D^0 \rightarrow e^+ e^-\) are rather weak, i.e.,

$$\begin{aligned} \Big | \tilde{C}_S^{LQ}-\tilde{C}_S^{\prime LQ} \Big |~ \leqslant ~ 0.2, \quad \Big | \tilde{C}_S^{LQ}+\tilde{C}_S^{\prime LQ} \Big |~ \leqslant ~ 0.2. \end{aligned}$$
(21)

It is obvious that the bounds obtained in Eqs. (20) and (21) could not give us proper information about the bounds on individual \(\tilde{C}_S^{LQ}\) and \(\tilde{C}_S^{\prime LQ}\) coefficients. Therefore, we consider only one Wilson coefficient at a time to extract the upper bound on individual coefficients. In Table 1, we report the constraint on the \(\tilde{C}_{S,P}^{LQ}\) Wilson coefficients obtained from the experimental bound on the branching fraction of the process \(D^0 \rightarrow \mu ^+ \mu ^- (e^+ e^-)\). The bounds on \(\tilde{C}_i^{\prime LQ}\) Wilson coefficients will be the same as those for \(\tilde{C}_i^{LQ}\).

If we impose chirality on the scalar leptoquarks i.e., they couple to either left-handed or right-handed quarks, but not to both, then the \(C_{S, P}^{(')}\) Wilson coefficients will vanish and we get only the additional contribution of the \(C_{9,10}^{(') LQ}\) Wilson coefficients to the SM. Now comparing the theoretical and experimental branching ratios, the allowed range of \(\tilde{C}_{10}^{(\prime ) LQ}\) Wilson coefficients are given in Table 1.

Table 1 The allowed values of the Wilson coefficients obtained from the upper bound of the process \(D^0 \rightarrow \mu ^+ \mu ^-(e^+ e^-)\). The constraint on the \(\tilde{C}_i^{ LQ}\) coefficients can also be applicable to \(\tilde{C}_i^{\prime { LQ}}\) Wilson coefficients

In order to evade the strict bounds in the down-type quark sector, we consider the leptoquark couplings to the left-handed quarks (\(Y^L\)) as zero. Therefore, the only contribution to the rare charm decays comes from the \(\tilde{C}_9^\prime = -\tilde{C}_{10}^\prime \) Wilson coefficients, which are related to the right-handed quark couplings. Now, to include the (pseudo)scalar and (pseudo)tensor Wilson coefficients and to extract the respective upper bound complying with the constraints from B and K physics, we consider a numerically tuned example as discussed in [49]. We assume that the \(Y^R\) coupling is perturbative, i.e., \(|Y^R| ~ <~ \sqrt{4\pi }\). In particular, we consider a large value for \(Y_{c\mu }^R\) coupling, e.g., \(Y_{c\mu }^R=3.5\). We compute the bound on \(Y_{u\mu }^R\) coupling by using the constraint on the \(\tilde{C}_{10}^{\prime LQ}\) Wilson coefficients from \(D^0 \rightarrow \mu ^+ \mu ^-\) process, which is found to be comparatively small, \(Y_{u\mu }^R~<~8.76 \times 10^{-3}\). Now we instigate a non-zero coupling to the left-handed quark \(Y_{u\mu }^L\), which along with the large \(Y_{c\mu }^R\) coupling provides non-zero values for the \(C_{S, P}\) and \(C_{T, T_5}\) coefficients. However, the process \(D^0 \rightarrow \mu ^+ \mu ^-\) imposes a strong bound on the coefficient \(C_S\), which, together with the large \(Y_{c\mu }^R\) coupling, limits the left-handed coupling to \(Y_{\mu u}^L ~<~1.14 \times 10^{-3}\). Thus, from the above discussion we observe that

$$\begin{aligned} \tilde{C}_9^{LQ}= & {} -\tilde{C}_{10}^{\prime LQ} = 0.8, \quad \tilde{C}_S^{LQ}=\tilde{C}_P^{LQ} \nonumber \\= & {} 4\tilde{C}_T^{LQ} = 4 \tilde{C}_{T_5}^{LQ} = -0.053. \end{aligned}$$
(22)

Our predicted bounds on the leptoquark coupling are in agreement with the constraints obtained in Refs. [48, 49] and also with the constraints obtained from B, K physics [66].

4 New physics contribution in \(Z^\prime \) model

The new heavy \(Z^\prime \) gauge boson can exist in many extended SM scenarios and can mediate the FCNC transitions among the fermions in the up quark sector at tree level. The most general Hamiltonian for \(c \rightarrow u\) transition in the \(Z^\prime \) model is given as [53]

$$\begin{aligned} \mathcal{H}^\mathrm{FCNC}_{Z'} = \mathcal{H}^{q}_{Z'} = g_{Z^\prime 1} \overline{u}_L \gamma _\mu c_L Z'^\mu + g_{Z^\prime 2} \overline{u}_R \gamma _\mu c_R Z'^\mu . \end{aligned}$$
(23)

Analogously, one can write the Hamiltonian for the leptonic sector \(\mathcal{H}^{ l}_{Z^\prime }\) as

$$\begin{aligned} \mathcal{H}^L_{Z'} = g_{Z^\prime 1}^\prime \overline{\ell }_L \gamma _\mu \ell _L Z'^\mu + g_{Z^\prime 2}^\prime \overline{\ell }_R \gamma _\mu \ell _R Z'^\mu . \end{aligned}$$
(24)

Here \(g_{Z^\prime i}\) and \(g_{Z^\prime i}^\prime \) are the couplings of the \(Z^\prime \) boson with the quarks and leptons, respectively, where \(i=1\) or 2 for the \(Z'^\mu \) vector boson coupled to left-handed or right-handed currents.

After having obtained the possible \(Z^\prime \) couplings with quarks and leptons, we proceed to constrain the new parameter space using the results from charm sector, e.g., the experimental data on \(D^0 - {\bar{D}}^0\) mixing and the branching ratios of \(D^0 \rightarrow l^+ l^-\) processes. The constraint on the coupling of \(Z^\prime \) with the leptonic part is obtained from the upper limit on the branching ratio of the lepton flavour violating \(\tau (\mu )^- \rightarrow e^- e^+ e^-\) processes.

4.1 Constraint from \(D^0 - {\bar{D}}^0\) mixing

In this subsection, we calculate the constraint on the \(Z^\prime \) couplings from the mass difference of the charm meson mass eigenstates, which characterizes the \(D^0 - {\bar{D}}^0\) mixing phenomenon. The \(D^0 - {\bar{D}}^0\) oscillation arises from the \(|\Delta C =2|\) transition that generates off-diagonal terms in the mass matrix for \(D^0\) and \({\bar{D}}^0\) mesons. The mass difference of \(D^0 - {\bar{D}}^0\) mixing at the scale \(\mu = m_c\) is given by [53]

$$\begin{aligned} \Delta M_\mathrm{D}^\mathrm{(Z^\prime )}= & {} { f_D^2 M_D B_D \over 2 M_{Z^\prime }^2 } \left[ {2 \over 3} \left( c_1 (m_c) + c_6 (m_c) \right) \right. \nonumber \\&\left. -\;\left( \displaystyle {1 \over 2} + \displaystyle {\eta \over 3} \right) c_2(m_c) + \left( \displaystyle {1 \over 12} + \displaystyle {\eta \over 2} \right) c_3(m_c) \right] .\nonumber \\ \end{aligned}$$
(25)

At the charm mass scale, the Wilson coefficients in terms of the \(Z^\prime \) couplings are expressed as

$$\begin{aligned} c_1(m_c)= & {} r (m_c,M_{Z^\prime })~ g_{Z^\prime 1}^2, \nonumber \\ c_3(m_c)= & {} \frac{4}{3} \left[ r(m_c,M_{Z^\prime })^{1/2} - r(m_c,M_{Z^\prime })^{-4} \right] g_{Z^\prime 1} g_{Z^\prime 2}, \nonumber \\ c_2(m_c)= & {} 2 \ r(m_c,M_{Z^\prime })^{1/2} g_{Z^\prime 1} g_{Z^\prime 2}, \nonumber \\ c_6(m_c)= & {} r (m_c,M_{Z^\prime }) ~ g_{Z^\prime 2}^2, \end{aligned}$$
(26)

where \(r (m_c,M_{Z^\prime })\) is the RG factor at the heavy mass scale and \(r (m_c,M_{Z^\prime }) =0.72~(0.71)\) for \(Z^\prime \) mass, \(M_{Z^\prime } = 1(2)\) TeV [52].

Now we consider two possible cases to constrain the couplings \(g_{Z^\prime 1}\) and \(g_{Z^\prime 2}\). One comes with only left-handed coupling present, i.e., (\(g_{Z^\prime 2} = 0\)) and the second with both left-handed and right-handed couplings present with equal strengths (\(g_{Z^\prime 1}=g_{Z^\prime 2}=g_{Z^\prime }\)). Here, we make the simple assumption that the NP part dominates over the SM contribution in \(D^0 - {\bar{D}}^0\) mixing. Thus, for the first case, on substitution of \(g_{Z^\prime 2}=0\) in Eqs. (25) and (26), the mass difference becomes

$$\begin{aligned} \Delta M^\mathrm{(Z')}_\mathrm{D} = {f_D^2 M_D B_D r (m_c, M_{Z'}) \over 3} ~ {g_{Z'1}^2 \over M_{Z'}^2}. \end{aligned}$$
(27)

Now varying the mass difference \(\Delta M_\mathrm{D}\) within its \(1\sigma \) allowed range [60], we obtain

$$\begin{aligned} {g_{Z'1} \over M_{Z'}} = (4.4-7.2) \times 10^{-7}\,\mathrm{GeV}^{-1}, \end{aligned}$$
(28)

and for a representative \(Z'\) mass \(M_{Z^\prime } =1\) TeV, the value of the coupling is found to be

$$\begin{aligned} g_{Z'1} = (4.4-7.2) \times 10^{-4}. \end{aligned}$$
(29)

Analogously for the second case, i.e., \(g_{Z^\prime 1}=g_{Z^\prime 2}=g_{Z^\prime }\), the constraint

$$\begin{aligned} g_{Z'} = (1.2 - 2.0) \times 10^{-4} \left( \frac{M_{Z^\prime }}{1\,\mathrm TeV} \right) \end{aligned}$$
(30)

is obtained.

4.2 Constraint from \(D^0 \rightarrow \mu ^+ \mu ^-\) process

The effective Hamiltonian for the process \(D^0 \rightarrow l^+ l^-\) in the \(Z^\prime \) model is given by [53]

$$\begin{aligned} \mathcal{H}_{c\rightarrow u\ell ^+\ell ^-}^{Z'}= & {} \frac{1}{M_{Z'}^2} \left[ g_{Z^\prime 1}g_{Z^\prime 1}' \widetilde{Q}_1 + g_{Z^\prime 1}g_{Z^\prime 2}' \widetilde{Q}_7\right. \nonumber \\&\left. +\;g_{Z^\prime 1}'g_{Z^\prime 2} \widetilde{Q}_2 + g_{Z^\prime 2}g_{Z^\prime 2}' \widetilde{Q}_6 \right] \ , \end{aligned}$$
(31)

where the operators \(\widetilde{Q}_{1,2}\) are

$$\begin{aligned} \widetilde{Q}_1= & {} \left( {\bar{l}}_L \gamma _\mu l_L \right) \left( {\bar{u}}_L \gamma ^\mu c_L \right) , \nonumber \\ \widetilde{Q}_2= & {} \left( {\bar{l}}_L \gamma _\mu l_L \right) \left( {\bar{u}}_R \gamma ^\mu c_R \right) , \end{aligned}$$
(32)

and \(\widetilde{Q}_{6, 7} \) can be obtained from \(\widetilde{Q}_{1, 2}\) by the substitutions of \(q_L \rightarrow q_R\) and \(q_R \rightarrow q_L\).

Comparing Eq. (31) with the SM effective Hamiltonian (4) yields the additional contributions to the Wilson coefficients \(C_{9,10}^{(\prime )Z'}\) as

$$\begin{aligned}&C_9^{Z^\prime } (C_{10}^{Z^\prime })=-\frac{\pi }{\sqrt{2}G_F \alpha _{em} \lambda _b} \frac{g_{Z^\prime 1} (g_{Z^\prime 2}^\prime {\,\pm \,} g_{Z^\prime 1}^\prime )}{M_{Z^\prime }^2}, \nonumber \\&{C_9^{\prime }}^{Z^\prime }({C_{10}^{{\prime }^{Z^\prime }}})=-\frac{\pi }{\sqrt{2}G_F \alpha _{em} \lambda _b} \frac{g_{Z^\prime 2} (g_{Z^\prime 2}^\prime {\,\pm \,} g_{Z^\prime 1}^\prime )}{M_{Z^\prime }^2}. \end{aligned}$$
(33)

The branching ratio for the process \(D^0 \rightarrow \mu ^+ \mu ^-\) in the \(Z^\prime \) model is given as [53]

$$\begin{aligned} \mathrm{BR}({D^0 \rightarrow \ell ^+\ell ^-})|_{Z^\prime }= & {} \tau _D\frac{f_D^2 m_\ell ^2 M_D}{32\pi M_{Z^\prime }^4 }\sqrt{1-\frac{4 m_\ell ^2}{M_D^2}}\nonumber \\&\times \left( g_{Z^\prime 1} - g_{Z^\prime 2}\right) ^2 \left( g_{Z^\prime 1}' - g_{Z^\prime 2}'\right) ^2.\nonumber \\ \end{aligned}$$
(34)

For simplicity, we consider here \(g_{Z'2}=0\). Now considering the couplings of the \(Z^\prime \) boson to the final leptons as the same form as the SM-like diagonal couplings of Z boson to leptons as discussed in [53], i.e.,

$$\begin{aligned} g_{Z'1}' = \frac{g}{\cos \theta _W} \left( -\frac{1}{2} + \sin ^2\theta _W \right) , \quad g_{Z'2}' = g\frac{ \sin ^2\theta _W }{\cos \theta _W},\nonumber \\ \end{aligned}$$
(35)

where g is the gauge coupling of Z boson and \(\theta _W\) is the Weinberg mixing angle. Now using the experimental upper limit on the branching ratio \(\mathrm{BR}(D^0 \rightarrow \mu ^+ \mu ^-) ~<~6.2 \times 10^{-9}\) [60], we obtain

$$\begin{aligned} {g_{Z'1} \over M_{Z'}^2} ~<~7.67 \times 10^{-8}\,\mathrm{GeV}^{-2}. \end{aligned}$$
(36)

For \(M_{Z^\prime }=1\) TeV, the constraint on the \(g_{Z'1}\) coupling is

$$\begin{aligned} g_{Z'1}~ <~0.077, \end{aligned}$$
(37)

which is rather weak compared with the constraint obtained from \(D^0 - {\bar{D}}^0\) mixing.

It should be noted that the constraint on \(Z^\prime \) couplings from the \(D^0 \rightarrow \mu ^+ \mu ^-\) decay process and the \(D^0 - {\bar{D}}^0\) mixing data have been computed in [53]. Similarly, the constraint on the couplings from \(D^0 - {\bar{D}}^0\) oscillation are obtained in Ref. [52]. We found that our constraints are consistent with the above predictions, if we use the updated values of various input parameters.

4.3 Constraints on \(g_{Z'1}^{\prime }\) from \(\tau ^-(\mu ^-) \rightarrow e^- e^+ e^-\) process

Considering only the left-handed coupling to the \(Z^\prime \) boson, the branching ratio of the process \(\mu ^- \rightarrow e^- e^+ e^-\) in the \(Z^\prime \) model is given by [67, 68]

$$\begin{aligned} \mathrm{BR}\left( \mu ^- \rightarrow e^- e^+ e^- \right) = \frac{\tau _\mu m_\mu ^5}{768 \pi ^3} \frac{ |{g'}_{\mu e}^L {g'}_{ee}^L |^2 }{M_{Z^\prime }^4}, \end{aligned}$$
(38)

where we have explicitly shown the indices in the couplings. The experimental upper limit on the branching ratio of this mode is \(\mathrm{BR} (\mu ^- \rightarrow e^- e^+ e^-) < 10^{-12}\) [60]. For the analysis, we use the mass and lifetime of the muon from [60] and consider the coupling \({g'}_{ee}^L=g_{Z' 1}^\prime \) as SM-like with its value as presented in Eq. (35). Thus, using the experimental upper limit, we get the bound on the \({g'}_{\mu e}^L\) coupling as

$$\begin{aligned} |{g'}_{\mu e}^L| ~<~5.69 \times 10^{-5} \left( \frac{M_{Z^\prime }}{1~ \mathrm TeV} \right) . \end{aligned}$$
(39)

Analogously using the branching ratio of \(\tau ^-\rightarrow e^- e^+ e^-\) process, \(\mathrm{BR} (\tau ^- \rightarrow e^- e^+ e^-) < 2.7 \times 10^{-8}\) [60], the constraint on the lepton flavour violating \({g'}_{\tau e}^L\) coupling is found to be

$$\begin{aligned} |{g'}_{\tau e}^L| ~<~0.02 \left( \frac{M_{Z^\prime }}{1 ~\mathrm TeV}\right) . \end{aligned}$$
(40)

5 \(D^{+ (0)} \rightarrow \pi ^{+ (0)} \mu ^+ \mu ^-\) process

In this section, we study the rare semileptonic decay process \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\), which is mediated by the quark level transition \(c \rightarrow u \mu ^+ \mu ^-\) and constitutes a suitable tool to search for new physics. The dominant resonance contributions come from the \(\phi \), \(\rho \) and \(\omega \) vector mesons and the effects of \(\eta ^{(')}\) mesons are comparatively negligible. These decay modes have recently been studied in Refs. [48, 49, 61] in various new physics scenarios and it is found that the model with scalar/vector leptoquarks and the minimal supersymmetric model with R-parity violation can give significant contributions. The matrix elements of various hadronic currents between the initial D meson and the final \(\pi \) meson can be parametrized in terms of three form factors \(f_0\), \(f_+\) and \(f_T\) [49]:

$$\begin{aligned}&\langle \pi (k)| \bar{u} \gamma ^\mu (1{\,\pm \,} \gamma _5) c | {D}(p_D) \rangle \nonumber \\&\quad =f_+\left( q^2\right) \left[ \left( p_D+k\right) ^\mu -\frac{M^2_D - M^2_\pi }{q^2}q^\mu \right] \nonumber \\&\qquad +\;f_0\left( q^2\right) \frac{M^2_D - M^2_\pi }{q^2}q^\mu , \end{aligned}$$
(41)
$$\begin{aligned}&\langle \pi (k)| \bar{u} \sigma ^{\mu \nu } (1{\,\pm \,} \gamma _5) c|{D}(p_D) \rangle \nonumber \\&\quad = i\frac{f_T(q^2)}{M_D+M_\pi }[\left( p_D+k\right) ^\mu q^\nu \nonumber \\&\qquad -\;\left( p_D+k\right) ^\nu q^\mu {\,\pm \,} i \epsilon ^{\mu \nu \alpha \beta } \left( p_D+k\right) _\alpha q_\beta ], \end{aligned}$$
(42)

where \(p_D\) and k are the four momenta of the D and \(\pi \) mesons, respectively, and \(q=p_D-k\) is the momentum transfer. The form factors for \(D^0 \rightarrow \pi ^0\) are scaled as \(f_i \rightarrow f_i/\sqrt{2}\) by isospin symmetry. For the \(q^2\)-dependence of the form factors, we use the parameterization from Refs. [69, 70], as

$$\begin{aligned} f_+(q^2)= & {} \frac{f_+(0)}{(1-x)(1-ax)}, \quad f_0(q^2) = \frac{f_+(0)}{(1-(x/b))},\nonumber \\ f_T(q^2)= & {} \frac{f_T(0)}{(1-x_T)(1-a_T x_T)}, \end{aligned}$$
(43)

where \(x=q^2/m_{pole}^2\) with \(m_{pole}=1.90(8)\) GeV, \(a=0.28(14)\) and \(b=1.27(17)\) are the shape parameters [49] measured from \(D \rightarrow \pi l \nu \) decay process and \(f_+(0)=0.67(3)\) [71]. The parameters in the \(f_T\) form factor are \(x_T=q^2/M_{D^*}^2\), \(f_T(0)=0.46(4)\) and \(a_T=0.18(16)\) [70]. Thus, one can write the transition amplitude for the process \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) as [49, 72]

$$\begin{aligned}&\mathcal{M}(D^+ \rightarrow \pi ^+ l^+ l^-) \nonumber \\&\quad = i \frac{G_F\lambda _b \alpha _{em}}{\sqrt{2} \pi }\left( V\, p_D^{\mu }\, [{\bar{l}}\gamma _{\mu } l] + A\, p_D^{\mu }\, [{\bar{l}} \gamma _{\mu }\gamma _5 l] \right. \nonumber \\&\qquad \left. +\;\left( S + T\cos \theta \right) [{\bar{l}}l] +\left( P + T_5 \cos \theta \right) [{\bar{l}}\gamma _5 l]\right) , \end{aligned}$$
(44)

where \(\theta \) is the angle between the D meson and the negatively charged lepton in the rest frame of the dilepton. The functions V, A, S and P are defined in terms of the Wilson coefficients as

$$\begin{aligned} V= & {} \frac{2 m_c f_T(q^2)}{M_D + M_\pi } C_7+ f_+(q^2) (C_9+C_9^{NP}+C_9^{\prime NP}) ,\nonumber \\ A= & {} f_+(q^2) \left( C_{10}+C_{10}^{NP}+C^{\prime NP}_{10}\right) ,\nonumber \\ S= & {} \frac{M_D^2 - M_\pi ^2}{2m_c} f_0(q^2) (C_S^{NP} + C^{\prime NP}_S),\nonumber \\ P= & {} \frac{M_D^2-M_\pi ^2}{2m_c} f_0(q^2) (C_P^{NP} + C^{\prime NP}_P) \nonumber \\&-\; m_l \left[ f_+(q^2)- \frac{M_D^2 - M_\pi ^2}{q^2} \left( f_0(q^2)-f_+(q^2) \right) \right] \nonumber \\&\times \;\left( C_{10}+C_{10}^{NP}+C^{\prime NP}_{10}\right) , \nonumber \\ T= & {} \frac{2f_T(q^2) \beta _l \lambda ^{1/2}}{M_D + M_\pi } C_T^{NP}, \nonumber \\ T_5= & {} \frac{2f_T(q^2) \beta _l \lambda ^{1/2}}{M_D + M_\pi } C_{T_5}^{NP}. \end{aligned}$$
(45)

Here \(C_{9, 10}^{(')NP}\), \(C_{S, P}^{(')NP}\) and \(C_{T, T_5}^{NP}\) are the new Wilson coefficients arising from either the scalar leptoquark model or the generic \(Z^\prime \) model. Using Eq. (44), the double differential decay distribution with respect to \(q^2\) and \(\theta \), for the lepton flavour l is given by [49, 72]

$$\begin{aligned} \frac{\mathrm{d}^2{\Gamma _l}}{\mathrm{d}q^2\, d\!\cos \theta } = a_l(q^2) + b_l(q^2) \cos \theta + c_l(q^2) \cos ^2\theta , \end{aligned}$$
(46)

where

$$\begin{aligned} a_l(q^2)= & {} \Gamma _0\, \sqrt{\lambda }\, \beta _l \left\{ 2q^2 \left( \beta ^2_l |S|^2 + |P|^2 \right) + \frac{\lambda }{2} (|A|^2 + |V|^2) \right. \nonumber \\&\left. +\;4 m_l (M_D^2 - M_\pi ^2 + q^2)\mathrm{Re}(A P^*) + 8 m_l^2 M_D^2 |A|^2 \phantom {\frac{1}{2}}\right\} , \nonumber \\ b_l(q^2)= & {} 4 \Gamma _0\, \sqrt{\lambda }\, \beta _l \left\{ q^2 \beta _l^2 \mathrm{Re}(S T^*) +q^2 \mathrm{Re} (P T_5^*) \right. \nonumber \\&\left. +\;m_l (M_D^2 - M_\pi ^2 +q^2) \mathrm{Re} (A T_5^*) +\sqrt{\lambda }\, \beta _l m_l \mathrm{Re}(V S^*)\right\} , \nonumber \\ c_l(q^2)= & {} \Gamma _0\, \sqrt{\lambda }\, \beta _l \left\{ - \frac{\lambda \beta _l^2}{2} (|V|^2 + |A|^2) \right. \nonumber \\&\left. +\;2q^2 (\beta _l^2 |T|^2 +|T_5|^2)+ 4m_l \beta _l \lambda ^{1/2} \mathrm{Re} (V T^*) \phantom {\frac{1}{2}}\right\} , \end{aligned}$$
(47)

with

$$\begin{aligned}&\lambda = M_D^4 + M_\pi ^4 + q^4 - 2 \left( M_D^2 M_\pi ^2 + M_D^2 q^2 + M_\pi ^2 q^2\right) ,\nonumber \\&\beta _l = \sqrt{1 - 4 \frac{m_l^2}{q^2}}, \end{aligned}$$
(48)

and

$$\begin{aligned} \Gamma _0 = \frac{G_F^2 \alpha _{em}^2 |\lambda _b|^2}{ (4\pi )^5 M_D^3}. \end{aligned}$$
(49)

Thus, the branching ratio is given by

$$\begin{aligned} \frac{\mathrm{d}\mathrm{BR}}{\mathrm{d}q^2} = 2\tau _D \left[ a_l(q^2)+\frac{1}{3} c_l(q^2)\right] . \end{aligned}$$
(50)

The forward–backward asymmetry (\(A_{FB}\)) is another useful observable to look for new physics; it is defined by [49]

$$\begin{aligned} A_{FB} (q^2)= & {} \Bigg [\int _0^1 \mathrm{d}\cos \theta \frac{\mathrm{d}^2\Gamma }{\mathrm{d}q^2 d \cos \theta }\nonumber \\&-\int _{-1}^0\mathrm{d}\cos \theta \frac{\mathrm{d}^2\Gamma }{\mathrm{d}q^2 d \cos \theta }\Bigg ] \Bigg /\frac{\mathrm{d}\Gamma }{\mathrm{d}q^2}\nonumber \\= & {} \frac{b_l(q^2)}{a_l(q^2)+\frac{1}{3}c_l(q^2)}. \end{aligned}$$
(51)

Since the coefficient \(b_l\) depends only on the scalar and pseudoscalar Wilson coefficients the forward–backward asymmetry is zero in SM. However, the additional new physics contribution can give a non-zero contribution to the forward–backward asymmetry parameter. Another interesting observable is the flat term, defined as [72]

$$\begin{aligned} F^l_H = \int _{q^2_{\mathrm{min}}}^{q^2_{\mathrm{max}}}\mathrm{d}q^2 \left( a_l + c_l\right) \Big / {\int _{q^2_{\mathrm{min}}}^{q^2_{\mathrm{max}}}\mathrm{d}q^2 \left( a_l + \frac{1}{3}c_l\right) }, \end{aligned}$$
(52)

where the uncertainties get reduced due to the cancelation between the numerator and denominator.

For numerical evaluation, we take the particle masses and the lifetime of the D meson from [60]. For the CKM matrix elements, we use the Wolfenstein parametrization with values \(A=0.814_{-0.024}^{+0.023}\), \(\lambda =0.22537{\,\pm \,} 0.00061\), \({\bar{\rho }} =0.117 {\,\pm \,} 0.021\) and \({\bar{\eta }} =0.353 {\,\pm \,} 0.013\) [60]. With these input parameters, we compute the resonant/non-resonant branching ratios of the process \(D^{+} \rightarrow \pi ^{+} \mu ^+ \mu ^-\) by integrating the decay distribution with respect to \(q^2\). We parametrize the contributions from the resonances with the Breit–Wigner shapes for \(C_9 \rightarrow C_9^\mathrm{res}\), for \( \rho , \omega , \phi \) (vector) and \(C_P \rightarrow C_P^\mathrm{res}\) for \(\eta ^{(\prime )}\) (pseudoscalar) mesons [48, 49]:

$$\begin{aligned}&C_9^\mathrm{res} \nonumber \\&\quad = a_\rho e^{i\delta _\rho } \left( \frac{1}{q^2-m_\rho ^2 +im_\rho \Gamma _\rho } - \frac{1}{3} \frac{1}{q^2-m_\omega ^2 +im_\omega \Gamma _\omega } \right) \nonumber \\&\quad \quad +\;\frac{a_\phi e^{i\delta _\phi }}{q^2-m_\phi ^2 +im_\phi \Gamma _\phi }, \nonumber \\&C_P^\mathrm{res} =\frac{ a_\eta e^{i\delta _\eta }}{q^2-m_\eta ^2 +im_\eta \Gamma _\eta } + \frac{a_{\eta ^\prime } }{q^2-m_{\eta ^\prime }^2 +im_{\eta ^\prime } \Gamma _{\eta ^\prime }}. \end{aligned}$$
(53)

Here \(m_{M} ~(\Gamma _{M})\) denotes the mass (total decay width) of the resonant state M, where M corresponds to \( \eta ^{(\prime )}, \rho , \omega , \phi \) mesons. With the approximation of \(\mathrm{BR}(D^+ \rightarrow \pi ^+ M(\rightarrow \mu ^+ \mu ^-)) \simeq \mathrm{BR}(D^+ \rightarrow \pi ^+ M) \mathrm{BR}(M \rightarrow \mu ^+ \mu ^-)\) and considering the experimental upper bound from [60], the magnitudes of the Breit–Wigner parameters are given by [48]

$$\begin{aligned} a_\phi= & {} 0.24^{+0.05}_{-0.06}\,\mathrm{GeV}^2, \quad a_\rho = 0.17 {\,\pm \,} 0.02\,\mathrm{GeV}^2, \nonumber \\&\quad a_\omega = a_\rho / 3, \nonumber \\ a_\eta= & {} 0.00060^{+0.00004}_{-0.00005}\,\mathrm{GeV}^2, \quad a_{\eta }^\prime \sim 0.0007\,\mathrm{GeV}^2. \end{aligned}$$
(54)
Fig. 2
figure 2

The resonant contributions to the branching ratio of \( D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) in the SM. The band arises due to the uncertainties in Breit–Winger parameters and the variation of relative phases. The horizontal black line represents the experimental upper bound from [60]

Table 2 The predicted branching ratios for \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^+ \mu ^-\) processes in both the low \(q^2\) and the high \(q^2\) region in the scalar X(3, 2, 7 / 6) LQ and \(Z^\prime \) model. This also contains the resonant and non-resonant SM branching ratios
Fig. 3
figure 3

The variation of branching ratio of \( D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) with respect to low \(q^2\) (left panel) and high \(q^2\) (right panel). The orange bands represent the contributions from the scalar leptoquark, the blue bands are due to the \(Z^\prime \) contributions, the red dashed lines are for the non-resonant SM and the cyan bands are for the resonant SM. The green bands stand for the theoretical uncertainties from the input parameters in the SM. The solid black line denotes the \(90\%\) CL experimental upper limit [73]

The detailed procedure of SM resonant contributions to the \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) process can be found in [48, 49, 70]. In Fig. 2, we show the \(q^2\) variation of the branching ratio of \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) process including the resonant contribution in the SM. The band in the figure is due to the uncertainties associated with the \(a_M\) parameters as given in (54) and the random variation of relative phases within \(-\pi \) and \(\pi \). For simplicity we have assumed the same phase for all the resonances. From the figure, one can observe that in the low and high \(q^2\) regions the long distance resonant contributions are approximately one order of magnitude below the current experimental sensitivity, and hence these regions are suitable to look for new physics beyond the SM. Thus, both in the SM and in the leptoquark and \(Z^\prime \) models, we study the process \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) only at the very low and high \(q^2\) regimes. However, it should be emphasized that the uncorrelated variation of the unknown resonant phases affects the branching ratio in the low \(q^2\) region significantly, which makes it quite difficult to infer the possible role of new physics.

Fig. 4
figure 4

The variation of the forward–backward asymmetry of \( D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) with respect to low \(q^2\) (left panel) and high \(q^2\) (right panel) in the scalar leptoquark model

Fig. 5
figure 5

The variation of the flat term of \( D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) with respect to low \(q^2\) (left panel) and high \(q^2\) (right panel) in the scalar leptoquark and \(Z^\prime \) models

With all the input parameters from [60] along with the SM Wilson coefficients [58, 59], we present in Table 2 the predicted values of the branching ratios for the \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^+ \mu ^-\) processes by integrating the decay distribution in low and high \(q^2\) bins. Here we have used the \(q^2\) regimes as \(q^2 \in [0.0625, 0.275]\,\mathrm{GeV}^{2}\) and \(q^2 \ge 1.56\,\mathrm{GeV}^2\) to reduce the background coming from the dominant resonances. The theoretical uncertainties in the SM are associated with the lifetime of the D meson, the CKM matrix elements and the hadronic form factors. In Fig. 3, the variation of the SM branching ratios of the process \(D^{+} \rightarrow \pi ^{+} \mu ^+ \mu ^-\) in the very low and high \(q^2\) regimes are shown in red dashed lines and the green bands represent the SM theoretical uncertainties.

Now using the constraint on the leptoquark parameter space obtained in Sect. 3, we show in Fig. 3, the \(q^2\) variation of the branching ratio of the process \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) in low \(q^2\) (left panel) and high \(q^2\) (right panel) regimes both in the scalar leptoquark and \(Z'\) models. Here the orange (blue) band represents the contributions from the scalar leptoquark (\(Z'\)) model. The \(90\%\) CL experimental upper bounds on the branching ratios from [73],

$$\begin{aligned} \mathrm{BR}(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-)|_\mathrm{low~q^2}< 2.0 \times 10^{-8},\nonumber \\ \mathrm{BR}(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-)|_\mathrm{high~q^2} < 2.6 \times 10^{-8}, \end{aligned}$$
(55)

are shown in thick black lines. In Table 2, we present the integrated branching ratios of the process \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^+ \mu ^- \)es in both the low and the high \(q^2\) regions in the leptoquark and \(Z^\prime \) models. We find that the predicted branching ratios in the leptoquark model have significant deviations from the corresponding SM values due to the effect of the scalar leptoquark and are well below the experimental upper limits. However, the effect of the \(Z^\prime \) boson to the branching ratios of \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^+ \mu ^-\) processes is very marginal.

In the leptoquark model, the variation of the forward–backward asymmetry for the process \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) in low \(q^2\) (left panel) and high \(q^2\) (right panel) is presented in Fig. 4. The forward–backward asymmetry depends on the combinations of the \(C_{S}^{(\prime )}\) and \(C_{T, T_5}\) Wilson coefficients, thus they have zero value in the SM. However, the additional contributions of \(C_{S,P}^{'}\) Wilson coefficients due to scalar leptoquark exchange give non-zero contribution to the forward–backward asymmetry, though it is not so significant. The integrated forward–backward asymmetry for the process \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) is given as

$$\begin{aligned} \langle A_{FB} \rangle= & {} -0.083 \rightarrow 0.042 ~~~\mathrm{~in ~ low ~q^2}, \nonumber \\ \langle A_{FB} \rangle= & {} -0.087 \rightarrow 0.062 ~~~\mathrm{~in ~ high ~q^2}, \nonumber \\ \langle A_{FB} \rangle= & {} -0.095 \rightarrow 0.06 ~~~\mathrm{~in ~ full ~q^2}. \end{aligned}$$
(56)

The \(Z^\prime \) model provides additional contributions only to the \(C_{9, 10}\) Wilson coefficients, and there are no new contributions to scalar or tensor terms. Thus, the forward–backward asymmetry vanishes in the \(Z^\prime \) model. In both the LQ and the \(Z^\prime \) model, the plot for flat term of \(D^+ \rightarrow \pi ^+ \mu ^+ \mu ^-\) process with respect to low \(q^2\) (left panel) and high \(q^2\) (right panel) is given in Fig. 5. The predicted values in low \(q^2\) range are

$$\begin{aligned} \langle F_H \rangle |_\mathrm{SM}= & {} 0.4 {\,\pm \,} 0.064, \quad \langle F_H \rangle |_\mathrm{LQ}=0.336 \rightarrow 0.46,\nonumber \\ \langle F_H \rangle |_\mathrm{Z^\prime }= & {} 0.4 \rightarrow 0.41, \end{aligned}$$
(57)

and in the region of high \(q^2\)

$$\begin{aligned} \langle F_H \rangle |_\mathrm{SM}= & {} 0.03 {\,\pm \,} 0.005, \quad \langle F_H \rangle |_\mathrm{LQ}=0.34 \rightarrow 0.5,\nonumber \\ \langle F_H \rangle |_\mathrm{Z^\prime }= & {} 0.08 \rightarrow 0.095. \end{aligned}$$
(58)

In addition to the leptoquark and \(Z^\prime \) models, the rare charm meson decays mediated by the \(c \rightarrow u\) transitions have also been investigated in various new physics models such as the minimal supersymmetric standard model [61, 64, 74, 75], the two Higgs doublet model [74], a warped extra dimensions model [75] and the up vector like quark singlet model [76, 77]. In Refs. [48, 49], the process \(D \rightarrow \pi \mu ^+ \mu ^-\) is studied in the context of both scalar and vector leptoquark models. Our results on predictions are found to be consistent with the literature.

6 \(D^{+ (0)} \rightarrow \pi ^{+ (0)} \mu ^- e^+\)

Since the individual lepton flavour number is conserved in the standard model, the observation of lepton flavour violation in the near future will provide unambiguous signal of new physics beyond the SM. The observation of neutrino oscillation implies the violation of lepton flavour in neutral sector and it is expected that there could be FCNC transitions in the charged lepton sector as well, such as \(l_i \rightarrow l_j \gamma \), \(l_i \rightarrow l_j l_k {\bar{l}}_k\), \(B \rightarrow l_i^{\mp } l_j^{\,\pm \,}\) and \(B \rightarrow K^{(*)} l_i^{\mp } l_j^{\,\pm \,}\) etc. The LFV decay modes proceed through box diagrams with tiny neutrino masses in the loop, thus become very rare in the SM. However, these modes can occur at tree level in the leptoquark and \(Z^\prime \) models, thus can provide observable signature in the high luminosity experiments. In this section, we would like to study the lepton flavour violating semileptonic decay process \(D^+ \rightarrow \pi ^+ \mu ^- e^+\). Due to the absence of intermediate states, these LFV processes have no long distance QCD contributions and dominant \(\phi \), \(\omega \) resonance backgrounds. The general expression for the transition amplitude of \(D^+ \rightarrow \pi ^+ \mu ^- e^+\) process in a generalized new physics model, is given by

$$\begin{aligned} \mathcal {M}= & {} -\frac{G_F \alpha _{em} \lambda _b}{\sqrt{2}\pi } f_+ (q^2) \left[ F_S ({\bar{\mu }}e) + F_P ({\bar{\mu }} \gamma _5 e)\right. \nonumber \\&\left. +\;F_V p_D^\mu \left( {\bar{\mu }} \gamma _\mu e \right) + F_A p_D^\mu \left( {\bar{\mu }} \gamma _\mu \gamma _5 e\right) \right] , \end{aligned}$$
(59)

where the functions \(F_i\), \(i=V, A, S, P\) are defined as

$$\begin{aligned} F_V= & {} K_9^{NP}+K_9^{\prime NP}\;, \nonumber \\ F_A= & {} K_{10}^{NP}+K_{10}^{\prime NP}\;, \nonumber \\ F_S= & {} \frac{1}{2} (K_S^{NP}+K_S^{\prime NP}) \frac{M_D^2-M_\pi ^2}{m_c}\frac{f_0 (q^2)}{f_+ (q^2)} \nonumber \\&+\;\frac{1}{2} \left( K_9^{NP}+K_9^{\prime NP}\right) (m_e - m_\mu )\nonumber \\&\times \;\left[ \frac{M_D^2 - M_\pi ^2}{q^2} \left( \frac{f_0 (q^2)}{f_+ (q^2)} - 1 \right) -1 \right] , \nonumber \\ F_P= & {} \frac{1}{2} (K_P^{NP}+K_P^{\prime NP}) \frac{M_D^2-M_\pi ^2}{m_c}\frac{f_0 (q^2)}{f_+ (q^2)}\nonumber \\&+\, \frac{1}{2}\left( K_{10}^{NP}+K_{10}^{\prime NP}\right) (m_\mu + m_e)\nonumber \\&\times \;\left[ \frac{M_D^2 - M_\pi ^2}{q^2} \left( \frac{f_0 (q^2)}{f_+ (q^2)} - 1 \right) -1 \right] . \end{aligned}$$
(60)

Here the Wilson coefficients \((K_i^{NP})\) involve the combination of LQ couplings as \(Y_{\mu c}^L Y_{e u}^{L/R*}\) instead of \(Y_{\mu c}^L Y_{\mu u}^{L/R*}\) in Eq. (8). Now using Eq. (59), the differential decay distribution for the \(D^+ \rightarrow \pi ^+ \mu ^- e^+\) process with respect to \(q^2\) and \(\cos \theta \) (\(\theta \) is the angle between the D and \(\mu ^-\) in the \(\mu -e\) rest frame) is given as

$$\begin{aligned} \frac{\mathrm{d}^2\Gamma }{\mathrm{d}q^2 \mathrm{d}\cos \theta } = A_l (q^2) + B_l (q^2) \cos \theta + C_l (q^2) \cos ^2\theta , \end{aligned}$$
(61)

where

$$\begin{aligned} A_l (q^2)= & {} 2\Gamma _0 \frac{\sqrt{\lambda _1 \lambda _2}}{q^2} f_+(q^2)^2 \left[ \frac{\lambda _1}{4}\left( |F_V|^2 + |F_A|^2 \right) \right. \nonumber \\&+\;|F_S|^2 \left( q^2 - (m_\mu + m_e)^2\right) \nonumber \\&+\;|F_P|^2 \left( q^2 - (m_\mu - m_e)^2 \right) \nonumber \\&+\;|F_A|^2 M_D^2 (m_\mu + m_e)^2 + |F_V|^2 M_D^2 (m_\mu - m_e)^2\nonumber \\&+\;\left( M_D^2 - M_\pi ^2 + q^2 \right) \Big ( (m_\mu + m_e) \mathrm{Re} (F_P F_A^*)\nonumber \\&\left. +\;(m_e - m_\mu ) \mathrm{Re} (F_S F_V^*)\Big ) \right] , \end{aligned}$$
(62)
$$\begin{aligned} B_l (q^2)&= 2\Gamma _0 \frac{\sqrt{\lambda _1 \lambda _2}}{q^2} f_+(q^2)^2 \Bigg [ (m_\mu + m_e) \mathrm{Re} (F_S F_V^*)\nonumber \\&\quad + (m_e - m_\mu ) \mathrm{Re} (F_P F_A^*) \Bigg ], \end{aligned}$$
(63)
Fig. 6
figure 6

The variation of branching ratio of the LFV process \( D^+ \rightarrow \pi ^+ \mu ^- e^+\) in the leptoquark model (left panel) and generic \(Z^\prime \) model (right panel) with respect to \(q^2\). The solid black lines represent the \(90\%\) CL experimental upper bound [60]

$$\begin{aligned} C_l (q^2) = - 2\Gamma _0 f_+(q^2)^2 \frac{(\lambda _1 \lambda _2)^{3/2}}{4 q^6} \left( |F_A|^2 + |F_V|^2 \right) \;, \end{aligned}$$
(64)

and

$$\begin{aligned} \lambda _1 = \lambda (M_D^2, M_\pi ^2, q^2), \lambda _2 = \lambda (q^2,m_\mu ^2, m_e^2). \end{aligned}$$
(65)

For numerical estimation in the leptoquark model, we use the constrained leptoquark couplings obtained from the process \(D^0 \rightarrow \mu ^+ \mu ^-\) and assume that the coupling between different generation of quarks and leptons follow the simple scaling laws, i.e. \(Y_{ij}^{L(R)} / Y_{ii}^{L(R)} = (m_i/m_j)^{1/2}\) with \(j > i\). As discussed in [47, 78, 79], such pattern of ansatz can explain the decay widths of radiative LFV decay \(\mu \rightarrow e \gamma \). Now using such ansatz, the variation of branching ratio with respect to \(q^2\) for \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^- e^+\) process in the leptoquark model is shown in left panel of Fig. 6 and the corresponding integrated value is given in Table 3. In this mode, the forward–backward asymmetry depends on \(K_{9, 10}^{(')NP}\) Wilson coefficients which give non-zero contribution. The left panel of Fig. 7 shows the \(q^2\) variation of the forward–backward asymmetry and the corresponding integrated value is found to be \((0.039 \rightarrow 0.047)\). The variation of the flat term with respect to \(q^2\) is presented in the left panel of Fig. 8 and the integrated value is \((0.137\rightarrow 0.33)\).

Fig. 7
figure 7

The variation of forward–backward asymmetry of the LFV process \( D^+ \rightarrow \pi ^+ \mu ^- e^+\) in the leptoquark model (left panel) and generic \(Z^\prime \) model (right panel) with respect to \(q^2\)

For the \(Z'\) model, we consider the constraint on the coupling of \(Z^\prime \) boson to the quarks, obtained from the \(D^0 - {\bar{D}}^0\) mixing and the process \(D^0 \rightarrow \mu ^+ \mu ^-\) as given in Eqs. (29) and (37). For the lepton flavour violating coupling, the constraint is taken from the process \(\mu ^- \rightarrow e^- e^+ e^-\), as discussed in Sect. 4. Thus, using Eqs. (29), (37) and (39), the predicted branching ratio of \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^- e^+\) process in the \(Z^\prime \) model is given in Table 3 and the \(q^2\) variation of the process \(D^+ \rightarrow \pi ^+ \mu ^- e^+\) is shown in Fig. 6 (right panel). The forward–backward asymmetry variation is shown in right panel of Fig. 7 and the predicted value is \(-1.15 \times 10^{-3}\), which is very small. In Fig. 8 (right panel), we show the plot for \(q^2\) variation of the flat term and the integrated value is 0.158.

Table 3 The predicted branching ratios for \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^- e^+\) lepton flavour violating processes in the scalar X(3, 2, 7 / 6) LQ and \(Z^\prime \) model. The present upper limit on the branching ratio \(\mathrm{BR}(D^{0} \rightarrow \pi ^{0} \mu ^{\mp } e^{\,\pm \,})= \mathrm{BR}(D^{0} \rightarrow \pi ^{0} \mu ^- e^+ + \pi ^0 \mu ^+ e^-) < 8.6 \times 10^{-5}\) [60]

From Table 3, one may note that the predicted branching ratios are well below the present experimental limit for the \(D^{+} \rightarrow \pi ^{+} \mu ^- e^+\) process. Although there is no experimental bound on the process \(D^{0} \rightarrow \pi ^{0} \mu ^- e^+\) so far, the experimental upper limit on the branching ratios of the \(D^{0} \rightarrow \pi ^{0} \mu ^{\mp } e^{\,\pm \,}\) process is well known, which is given as \(\mathrm{BR}(D^{0} \rightarrow \pi ^{0} \mu ^{\mp } e^{\,\pm \,})= \mathrm{BR}(D^{0} \rightarrow \pi ^{0} \mu ^- e^+ + \pi ^0 \mu ^+ e^-) < 8.6 \times 10^{-5}\). Our results for the process \(D^{0} \rightarrow \pi ^{0} \mu ^- e^+\) in both the leptoquark and the \(Z^\prime \) models are found to be within the above experimental bound.

Fig. 8
figure 8

The variation of flat term of the LFV process \( D^+ \rightarrow \pi ^+ \mu ^- e^+\) in the leptoquark model (left panel) and generic \(Z^\prime \) model (right panel) with respect to \(q^2\)

7 \(D^0 \rightarrow \mu ^- e^+ (\tau ^- e^+)\) LFV decay process

Recently LHCb put the upper limit on branching ratio of the \(D^0 \rightarrow \mu ^{\mp } e^{\,\pm \,} \) lepton flavour violating decay mode [5]:

$$\begin{aligned} \mathrm{BR}(D^0\rightarrow & {} \mu ^{\mp } e^{\,\pm \,}) \simeq \mathrm{BR}(D^0 \rightarrow \mu ^- e^+ + \mu ^+ e^-) \nonumber \\< & {} 1.3 \times 10^{-8}. \end{aligned}$$
(66)

Neglecting the mass of electron, the branching ratio of the process \(D^0\rightarrow \mu ^- e^+\) is given by [48]

$$\begin{aligned}&\mathrm{BR}(D^0\rightarrow \mu ^- e^+)\nonumber \\&\quad = \tau _{D} \frac{G_F^2\alpha _e^2 M_{D}^5f_{D}^2 |\lambda _b|^2}{64\pi ^3} \left( 1-\frac{m_\mu ^2}{M_{D}^2}\right) ^2\nonumber \\&\quad \quad \times \;\bigg [\left| \frac{K_S^{NP}-K_S^{\prime NP}}{m_c} + \frac{ m_\mu }{M_{D}^2} \left( K_9^{NP}-K_9^{\prime NP}\right) \right| ^2\nonumber \\&\quad \quad +\;\left| \frac{K_P^{NP}-K_P^{ \prime NP}}{m_c}+\frac{m_\mu }{M_{D}^2}\left( K_{10}^{NP}-K_{10}^{\prime NP}\right) \right| ^2\bigg ]. \end{aligned}$$
(67)

We use the scaling ansatz as discussed in the previous section to compute the required leptoquark coupling for \(D^0\rightarrow \mu ^- e^+\) process and the predicted branching ratio is found to be

$$\begin{aligned} \mathrm{BR}(D^0 \rightarrow \mu ^- e^+) = (3.18-4.8) \times 10^{-11}. \end{aligned}$$
(68)

Now using Eqs. (29), (37) and (39), the predicted branching ratio of this LFV process in the \(Z^\prime \) model is

$$\begin{aligned} \mathrm{BR}(D^0 \rightarrow \mu ^- e^+) \simeq 6.1 \times 10^{-17}. \end{aligned}$$
(69)

The predicted branching ratio is although small, but can be searched at LHCb experiment. The exploration/observation of this decay mode would definitely shed some light in the leptoquark scenarios.

Similarly using the new Wilson coefficient generated via leptoquark exchange, the branching ratio for the process \(D^0\rightarrow \tau ^- e^+\) is found to be

$$\begin{aligned} \mathrm{BR}(D^0 \rightarrow \tau ^- e^+) = (2.84-9.75) \times 10^{-14}. \end{aligned}$$
(70)

However, there is no experimental observation of the lepton flavour violating process \(D^0\rightarrow \tau ^- e^+\). The constraint on \(Z^\prime \) coupling to tau and electron is obtained from the process \(\tau ^- \rightarrow e^- e^+ e^-\). Using Eq. (40), the branching ratio for the process \(D^0\rightarrow \tau ^- e^+\) in \(Z^\prime \) model is given as

$$\begin{aligned} \mathrm{BR}(D^0 \rightarrow \tau ^- e^+) = (0.73-1.94) \times 10^{-15}. \end{aligned}$$
(71)

So far there is no experimental evidence on the LFV decay process \(D^0 \rightarrow \tau ^- e^+\). Our results for the process \(D^0 \rightarrow \tau ^- (\mu ^-) e^+\) is comparable with [48, 51, 53].

8 Conclusion

In this paper we have studied the rare decays of the D meson in both scalar leptoquark and generic \(Z^\prime \) models. We have considered the simple renormalizable scalar leptoquark model with the requirement that proton decay would not be induced in perturbation theory. The leptoquark parameter space is constrained using the present upper limit on branching ratio of \(D^0 \rightarrow \mu ^+ \mu ^-\) process and the \(D^0 - {\bar{D}}^0\) oscillation data. For the \(Z^\prime \) model, the constraints on the \(Z^\prime \) couplings are obtained from the mass difference of \(D^0 - {\bar{D}}^0\) mixing, the process \(D^0 \rightarrow \mu ^+ \mu ^-\) and the lepton flavour violating \(\tau ^-(\mu ^-) \rightarrow e^- e^+ e^-\) processes. Using the constrained parameter space, we estimated the branching ratios, forward–backward asymmetry parameters and the flat terms in the \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^+ \mu ^-\) processes. The branching ratios in the LQ model are found to be \(\sim \mathcal{O}(10^{-10})\), which are larger than the corresponding SM predictions in the very low and very high \(q^2\) regimes. If these branching ratios will be observed in near future they would provide indirect hints of leptoquark signal. Furthermore, we estimated the branching ratios of lepton flavour violating \(D^{+(0)} \rightarrow \pi ^{+(0)} \mu ^- e^+\) and \(D^0 \rightarrow \mu (\tau )^- e^+ \) processes, which are found to be rather small. We also estimated the forward–backward asymmetry parameter and the flat term for the LFV decays.