1 Introduction

Leptoquarks, arising in several extensions of the standard model (SM) are particles which can turn a lepton to a quark and vice versa. Beyond standard model (BSM) theories, which treat the leptons and quarks on the same basis, like SU(5) [1], \(SU(4)_C\times SU(2)_L\times SU(2)_R\) [2], or SO(10) [3, 4], contain such particles. The theories with composite model [5] and technicolor model [6] can also have such particles. Leptoquarks carry both baryon and lepton numbers simultaneously.

The discovery of the leptoquarks would be unambiguous signal of physics beyond the SM and hence searches for such particles were conducted in the past experiments and the hunt is still going on at the present collider. Unfortunately, so far, all searches have led to a negative result. However, these searches received further attention in view of the possibilities for leptoquarks to explain certain striking discrepancies observed in the flavor sector. The discrepancies are observed mostly in rare decay modes of B mesons by various experimental collaborations, like LHCb, Belle and BaBar, hinting towards lepton non-universality. Previous collider studies on leptoquark searches can be found in Refs. [7,8,9,10,11,12,13,14,15,16,17,18,19,20].

In this article we consider the LHC phenomenology of a scalar leptoquark which has the quantum numbers under the SM gauge group \((\mathbf{3}, \mathbf{1},-1/3)\). As mentioned above, the leptoquark can explain some of the observed anomalies [21, 22]; however, in this article we mainly focus on the collider perspective. The presence of the leptoquark also improves the stability of the electroweak vacuum significantly [23]. A study at ATLAS [24] with \(13\mathrm {\,TeV}\) data puts a bound on the scalar leptoquark mass \(\gtrsim 1,\,1.2 \, \hbox {TeV}\) when such leptoquark decays to \(u\,e\) and \(c\,\mu \) with \(100\%\) branching fraction, respectively. Another very recent study at 13 TeV data from the CMS collaboration [25] imposes a most stringent bound on the leptoquark mass of \(\ge 900\, \hbox {GeV}\) in the search through \(t\,\tau \) final states with \(100\%\) branching fraction. The previous results, with \(8\mathrm {\,TeV}\) data, from the search of single leptoquark production are much weaker \(\ge 660 \, \hbox {GeV}\) [26] for its decay to \(c\mu \).

As mentioned above, a leptoquark with a hypercharge of \(-1/3\) has been looked for at CMS experiments via its third generation decay mode, i.e., \(t\,\tau \) [25]. However, no searches are performed for the final states comprising the decays of the leptoquark involving both second and third generations. In this article we focus mainly on the third generation and also controlled second generation decay phenomenology for such leptoquarks that can probe the most favored region of the parameter space required by the other studies.

Preference of the third generation will promote the decays of the leptoquark to \(t\,\tau \) modes over other decay modes. This changes the search phenomenology drastically, which is the topic of this article. Apart from the decay, such a parameter space also allows single leptoquark production in association with \(\nu \) via b gluon fusion and in association with \(\mu \) via c gluon fusion. In this aspect we focus on the leptoquark pair production as well as the single leptoquark production at the LHC.

The paper is organized as follows. In Sect. 2 we briefly describe the model. The parameter spaces that is allowed when a leptoquark dominantly decays into second and/or third generations are studied in Sect. 3. The benchmark points and collider phenomenology are discussed in Sect. 4. The LHC simulation results for the final states coming from leptoquark pair production are presented in Sect. 5. In Sect. 6 we discuss the leptoquark mass reconstruction and the reach at current and future LHC. The last two discussions are repeated for single leptoquark productions in Sect. 7. Finally, in Sect. 8 we discuss the prospects of the leptoquark in future colliders and summarize the results.

2 The leptoquark model

We consider the SM extended with a colored, SU(2) singlet charged scalar \(\phi \), i.e., the leptoquark with the SM gauge quantum numbers \((\mathbf{3},\, \mathbf{1},\,-1/3)\). The relevant interaction terms are

$$\begin{aligned} {\mathcal {L}}_{\phi } \subset {\bar{Q}}^c Y^{\mathrm{L}} i \tau _2 L \phi ^*\, +\, {\bar{u}}^c_R Y^{\mathrm{R}} \ell _R \phi ^* + \mathrm{h.c.}. \end{aligned}$$
(2.1)

The Q, L are \(SU(2)_L\) quark–lepton doublets given by \(Q=\left( u_L,~d_L \right) ^T,~L= \left( \nu _L,~\ell _L \right) ^T\), and \(u^c_R\) and \(\ell _R\) are right-handed \(SU(2)_L\) singlet up type quark and right-handed charged lepton, respectively. The generation and color indices are suppressed here.

The leptoquark also interacts with the SM Higgs doublet \(\Phi \) via the scalar potential

$$\begin{aligned} V(\phi ,\Phi )= m_\phi ^2 |\phi |^2 + \,g_{h\phi }|\Phi |^2\phi ^2 +\lambda _{\phi } \phi ^4. \end{aligned}$$
(2.2)

It is shown in Ref. [23] that the coupling \(g_{h\phi }\) plays an important role in improving the stability of electroweak vacuum. The moderate value of \(g_{h\phi }\) (\(\ge 0.3\)) can make the vacuum (meta-) stable up to the Planck scale for the top quark mass measured at Tevatron [27].

The leptoquark \(\phi \) has an electric charge of \(-1/3\) unit and is also charged under \(SU(3)_c\). A similar state can also arise from a leptoquark triplet with gauge quantum numbers \((\mathbf{3},\, \mathbf{3},\, -1/3)\), which comprises three states with electric charges \(-4/3,~ -1/3\) and 2 / 3; however, the interactions are different in this case.

The Lagrangian in Eq. (2.1) is written in the flavor basis, and the rotation of fermion fields should be included in the definitions of \(Y^{\mathrm{L},\mathrm{R}}\) matrices while performing the phenomenology in their mass basis. Thus in general the matrices \(Y^{\mathrm{L}}\) and \(Y^{\mathrm{R}}\) have off-diagonal terms leading to lepton–quark flavor as well as generation violating couplings. The off-diagonal couplings are strongly constrained by various meson decay modes [28,29,30,31,32,33,34] and hence, for the analysis in our paper, we assume \(Y^{\mathrm{L},\mathrm{R}}\) to be diagonal. For simplicity, we introduce the following notation after performing the rotations via CKM (PMNS) matrix for down-type quarks (neutral leptons) for moving to the mass basis:

$$\begin{aligned} Y^{\mathrm{L},\mathrm{R}}\rightarrow Y^{\mathrm{L},\mathrm{R}}_{ij} \equiv Y^{\mathrm{L},\mathrm{R}}_{ij}\, \delta _{ij}. \end{aligned}$$
(2.3)
Fig. 1
figure 1

A comparison of cross-section limits on scalar leptoquark pair-production times branching fraction to \(b\,\nu \) (a), \(c\, \mu \) (b) and \(t\, \tau \) (c) final states as a function of leptoquark mass \(m_\phi \). The \(2\sigma \) allowed region from ATLAS searches at 8 TeV [37] (a), 13 TeV [24] (b) and CMS searches at 13 TeV [25] center of mass energy are shown in yellow bands. The NLO prediction is shown in blue curves for two different choices of renormalization/factorization scale with the corresponding chosen values of branching fraction to the final states

3 Revisiting leptoquark parameter space

The search for leptoquarks at the colliders especially at the LHC has drawn a lot of interest from the last few years. The subject has recently received further impetus from the possibility of explaining the lepton non-universal anomalies seen in B decays by leptoquarks. From the experimental point of view, it is much simpler to look for the final states involving a first or second generation of leptons. Unfortunately, no sign of excess has been seen in such searches, which eventually put bounds on the leptoquark mass as follows: a scalar leptoquark of a mass of \(\sim 1\, \hbox {TeV}\) is excluded at 95% confidence level assuming 100% branching ratio into a charged lepton (first and second generation) and a quark [24].

Depending upon the gauge quantum numbers, the leptoquark can also decay to \(b\, \tau \) final states. Searches for this type of leptoquarks have also been performed in Ref. [35] which excludes leptoquark mass up to 740 GeV with the assumption of 100% branching fraction. In this work we focus on the parameter space of a scalar leptoquark which decays predominantly to the \(t \,\tau \) and \(b\, \nu \) final states. Both CMS [25, 36] and ATLAS [37] have performed searches at 7–8 TeV and also in 13 TeV center of mass energy, where the lower bounds on the leptoquark mass are found to be 900 GeV and 625 GeV, respectively, for the final states mentioned.

In Fig. 1 we illustrate that a leptoquark mass \(>600\, \hbox {GeV}\) is still allowed, within 95% confidence level, for comparatively lower branching fractions to second and third generation final states. The \(2\sigma \) allowed region from ATLAS searches at center of mass energy of 8 TeV [37] in Fig. 1a, 13 TeV [24] in Fig. 1b and CMS results for 13 TeV [25] in Fig. 1c are shown in yellow bands where the leptoquark decays to the \(b\, \nu \), \(c\, \mu \) and \(t\,\tau \) final states, respectively. The blue solid and dashed curves denote the (next-to-leading order) NLO pair-production cross-sections for the choice of scale \(\mu = m_\phi \) and \(\mu = \sqrt{\hat{s}}\), respectively. We use the notation \(\beta ={\mathcal {B}}(\phi \rightarrow b\, \nu )=0.39\) (Fig. 1a), \(\beta ={\mathcal {B}}(\phi \rightarrow c \,\mu )=0.1\) (Fig. 1b) and \(\beta ={\mathcal {B}}(\phi \rightarrow t \,\tau )=0.61\) (Fig. 1c). Later we shall discuss the collider phenomenology for three specific choices of benchmark points.

4 Benchmark points and distributions

It is apparent from the previous section that a range of less than TeV for the leptoquark is still allowed for relatively lower branching fractions to second and third generation leptons and quarks. In this article we focus on the searches for the final states that arise from the combinations of the leptoquark decays to second (\(c\, \mu \)) and third (\(t\, \tau \)) generations. We select the three benchmark points presented in Table 1 motivated by such decays.

We consider two benchmark points with relatively lighter leptoquark mass of 650 GeV and the third one with 1.2 TeV in BP1, BP2 and BP3, respectively, for a collider study at the LHC with 14 TeV of center of mass energy. We have implemented the model in SARAH [38] and generated the model files for CalcHEP [39], which is then used for calculating the decay branching ratios, tree-level cross-section and event generation. Table 2 shows the decay branching fraction for the leptoquark, \(\phi \). For BP1 and BP3, the leptoquark dominantly decays into the third generation; 60.8%, 63.2% to \(t\, \tau \) and 39.2%, 36.8% to \(b\, \nu \) states. However, in the chosen BP2 the leptoquark also decays into the second generation, i.e., 10.4% into \(c \, \mu \) and \(s\, \nu \).

Table 1 The couplings and masses for three benchmark points
Table 2 Branching fractions of the leptoquark \(\phi \) to different decay modes for the benchmark points defined in Table 1
Table 3 The production cross-sections of \(\phi \) pair for the benchmark points at the LHC with \(E_{\mathrm{CM}}=14\,\)TeV, renormalization/factorization scale \(\mu =\sqrt{\hat{s}}\) and PDF = 6TEQ6L, where the associated k-factors are included [9, 10, 15, 16]
Fig. 2
figure 2

a The charged lepton (\(e, \, \mu \)) \(p_T\) distribution for the benchmark points and b charged lepton multiplicity distribution at the LHC with 14 TeV of center of mass energy

Table 3 shows the leptoquark pair-production cross-sections for the benchmark points where 6TEQ6L [40] is used as PDF and \(\sqrt{\hat{s}}\) is chosen as renormalization/factorization scale. The suitable k-factors for NLO cross-sections are implemented [9, 10, 15, 16]. The choice of \(\sqrt{\hat{s}}\) as a scale gives a conservative estimate, which can get an enhancement of \(\sim 40\%\) for the choice of \(m_{\phi }\) as renormalization/factorization scale.

Before going into the details of the collider simulation let us have a look at the different differential distributions to motivate the advanced cuts which will be used later on to reduce the SM backgrounds. Figure 2a shows the lepton \(p_T\) arising from the \(W^\pm \) in the case BP1 and BP3. However, for BP2 an additional source of muon is possible from the decay of the leptoquark, which can be very hard. The charged leptons coming from \(W^\pm \) decay in the case of BP3 are also relatively hard due to the higher mass of the leptoquark (\(m_{\phi }=1.2 \, \hbox {TeV}\)). Hence, eventually, we expect much harder charged leptons compared to the SM processes. Figure 2b shows the charged lepton (\(e,\, \mu \)) multiplicity distribution for the three benchmark points, where the third and fourth charged leptons come from the semileptonic decays of b or decays of \(\tau \), which could be hard enough to be detected as charged leptons in the electromagnetic calorimeter (ECAL) of the detector at the LHC.

Figure 3a describes the \(p_T\) of the first two \(p_T\) ordered jets for BP1 and BP3, respectively. The respective leptoquark masses are 650 and 1200 GeV for BP1 and BP3, resulting in relatively soft and hard jets for BP1 and BP3. The \(p_T\) distributions of BP2 are very similar to BP1 due to the same mass value chosen for the leptoquark. Nevertheless, irrespective of the benchmark points the requirement of a very hard first jet would be critical in reducing the SM backgrounds including \(t{\bar{t}}\), which can still give a high \(p_T\) tail. Figure 3b shows the jet multiplicity distribution for BP1 and BP3, and the peak values for both of them are at five.

Fig. 3
figure 3

a The jet \(p_T\) distribution of the first two \(p_T\) ordered jets for the benchmark points and \(t{\bar{t}}\) and b jet multiplicity distribution at the LHC with 14 TeV center of mass energy

Fig. 4
figure 4

a The \(\tau \)-jet \(p_T\) distribution for the benchmark points and \(t{\bar{t}}\), and b \(\tau \)-jet multiplicity distribution at the LHC with 14 TeV center of mass energy

The leptoquark decaying to \(t\, \tau \) gives rise to lots of hard \(\tau \)-jets, which can easily be identified from the relatively soft \(\tau \)-jets coming from the \(W^\pm \) decays. Figure 4a describes this feature, where we can see the \(\tau \)-jets coming from the decay of the leptoquark in BP3 is the hardest and for BP1 it is softer, and for the \(t{\bar{t}}\) background, the \(p_T\) of such \(\tau \)-jets are really low compared to the signal. A cut on such \(\tau \)-jets can be decisive to kill the dominant SM backgrounds. Figure 4b depicts the \(\tau \)-jet multiplicity in the final states and a maximum of four \(\tau \)-jets can be achieved when \(W^\pm \hbox {s}\) decay in \(\tau \, \nu \) mode. All these distributions will be crucial in the next section where we apply additional cuts to decide on the final state topologies.

Table 4 The number of events for \(2b+2\tau + 2 \ell + |m_{\ell \ell }-m_Z|\ge 5\, \mathrm{\,GeV}+ |m_{\tau \tau }-m_Z|\ge 10\, \hbox {GeV}\) final state for the benchmark points and the dominant SM backgrounds at the LHC with 14 TeV of center of mass energy and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). \(S_{\mathrm{sig}}\) denotes signal significance at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity and \(\int {\mathcal {L}}_5\) depicts the required integrated luminosity for \(5\sigma \) confidence level for the signal
Table 5 The number of events for the \(2b+2\tau + 4 j +|m_{\tau \tau }-m_Z|\ge 10\, \mathrm{GeV}\) final state for the three benchmark points and the dominant SM backgrounds at the LHC with 14 TeV of center of mass energy and an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). \(S_{\mathrm{sig}}\) denotes signal significance at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity and \(\int {\mathcal {L}}_5\) depicts the required integrated luminosity for \(5\sigma \) confidence level for the signal

5 Collider phenomenology

We focus on the phenomenology arising from the decays of the leptoquark into the second and third generations. The first part of the study is concentrated on the final states arising from the leptoquark pair production but the contributions from single leptoquark production are also being taken into account, whenever such contributions are non-negligible. For the simulation at LHC with center of mass energy of 14 TeV, we generate the events by CalcHEP [39]. The generated events are then mixed with their decay branching fraction written in the decay file in SLHA format, by the event_mixer routine [39] and converted into ‘lhe’ format. The ‘lhe’ events for all benchmark points then are simulated with PYTHIA [41] via the lhe interface [42]. The simulation at the hadronic level has been performed using the Fastjet-3.0.3 [43] with the CAMBRIDGE AACHEN algorithm. We have selected a jet size \(R=0.5\) for the jet formation. The following basic cuts have been implemented:

  • the calorimeter coverage is \(|\eta | < 4.5\);

  • the minimum transverse momentum of the jet \( p_{T,min}^{jet} = 20 \, \hbox {GeV}\) and jets are ordered in \(p_{T}\);

  • leptons (\(\ell =e,~\mu \)) are selected with \(p_T \ge 20 \, \hbox {GeV}\) and \(|\eta | \le 2.5\);

  • no jet should be accompanied by a hard lepton in the event;

  • \(\Delta R_{\ell \,j}\ge 0.4\) and \(\Delta R_{\ell \,\ell }\ge 0.2\);

  • since an efficient identification of the leptons is crucial for our study, we additionally require a hadronic activity within a cone of \(\Delta R = 0.3\) between two isolated leptons to be \(\le 0.15\, p^{\ell }_T \, \hbox {GeV}\), with \(p^{\ell }_T\) the transverse momentum of the lepton, in the specified cone.

In the following subsections, we discuss the phenomenology coming from the leptoquark pair production at the LHC as we describe the different final state topologies. For notational simplicity we refer to ‘b’, ‘c’ and ‘\(\tau \)’ as b-jet, c-jet and \(\tau \)-jet, respectively. As mentioned above, we include the single leptoquark contribution whenever it is necessary. Later we also shall investigate how single leptoquark production can generate different final state topologies.

5.1 \(2b+2\tau +2\ell \)

This final state occurs when both leptoquarks, which are pair produced, decay into a third generation lepton and quark, i.e., \(t\, \tau \). The top pair then further decay into \(2\, b\) quarks and \(2\, W^\pm \) bosons. This gives rise to the final states \(2b+2\tau + 2 \ell \) listed in Table 4, where the event numbers are given for the three benchmark points and dominant SM backgrounds, with the cumulative cuts at the 14 TeV LHC with an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). Here we collect both leptons (\(e\,, \mu \)) coming from the \(W^\pm \) decays. The \(\tau \)-jets are reconstructed from hadronic decays of \(\tau \) with at least one charged track within \(\Delta R \le 0.1\) of the candidate \(\tau \)-jet [44]. The b-jets are tagged via secondary vertex reconstruction and we take the single b-jet tagged efficiency of 0.5 [45]. The requirements of two b-jets, two \(\tau \)-jets and two opposite sign charged leptons, along with the invariant mass veto around the Z mass for di-\(\ell \) and di-\(\tau \)-jets, make the most dominant SM backgrounds such as \(t{\bar{t}}\), ZZZ, \(t{\bar{t}}b{\bar{b}}\) and gauge boson pair reducible ones. Some contributions coming from \(t{\bar{t}}Z\) and tZW also fade away after the invariant mass veto on di-\(\tau \)-jets. It is evident that BP1, having a leptoquark of a mass of 650 GeV, can be probed with very early data of \(\sim 100 \, {\hbox {fb}}^{-1}\) luminosity and for BP2 we need \(\sim 150 \, {\hbox {fb}}^{-1}\). However, in the case of BP3 the required luminosity is beyond the reach of LHC in its current design.

5.2 \(2b+2\tau +4j\)

In the scenario when both \(W^\pm \hbox {s}\) coming from the decays of top pair which are produced from leptoquarks decay hadronically, additional jets arise besides di-\(\ell \) jets. Here signal event numbers increase a lot due to the larger hadronic decay branching fraction of \(W^\pm \, (\sim 68\%)\). Table 5 describes the event numbers for the benchmark points and the dominant SM backgrounds for the \(2b+2\tau + 4 j\) final state at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). The \(\tau \)-jets invariant mass veto around the Z-mass, i.e., \(|m_{\tau \tau }-m_Z|\ge 10 \, \hbox {GeV}\), reduces the background contributions significantly. The significance of the final state is naturally enhanced compared to the leptonic final state (see Table 4) and can be probed with very early data of few \(\hbox {fb}^{-1}\) at the 14 TeV LHC. It seems that this particular final state can give the very first hint towards the discovery of the leptoquark if it dominantly decays into the third generation i.e., \(t\, \tau \). Even for BP3, which has a leptoquark of a mass of 1.2 TeV, it can be probed at an integrated luminosity of \(\sim 342 \, \hbox {fb}^{-1}\). In Tables 4 and 5, the single leptoquark production via \(c \, g\rightarrow \mu \phi \) does not contribute and thus these final states can probe leptoquarks via pair production only.

Table 6 The number of events for the \(1b+1j+1\tau +1\ell +1\mu \) final state for the benchmark points and the dominant SM backgrounds at the LHC with 14 TeV of center of mass energy and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). \(S_{\mathrm{sig}}\) denotes a signal significance at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity and \(\int {\mathcal {L}}_5\) depicts the required integrated luminosity for \(5\sigma \) confidence level for the signal. The ‘\(\dagger \)’ denotes the contribution from \(c\,g\rightarrow \phi \, \mu \) production process
Table 7 The number of events for the \(1b+3j+1\tau +1\mu \) final state for the bench mark points and the dominant SM backgrounds at the LHC with 14 TeV of center of mass energy and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). \(S_{\mathrm{sig}}\) denotes signal significance at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity and \(\int {\mathcal {L}}_5\) depicts the required integrated luminosity for \(5\sigma \) confidence level for the signal. A cumulative cut of \(p^{\mu }_T \ge 100 \, \hbox {GeV}\) is applied to reduce the SM backgrounds further. The ‘\(\dagger \)’ denotes the contribution from \(c\,g\rightarrow \phi \, \mu \) production process

5.3 \(1b+1j+1\tau +1\ell +1\mu \)

Now we focus on a scenario where both the second and the third generation decays contribute to the final state, i.e., one of the pair-produced leptoquark decays into \(t\, \tau \) and the other one into \(c\, \mu \). The c-jet coming from the leptoquark is tagged as a normal jet such that we do not lose events on its tagging efficiency [46]. We also require that the \(W^\pm \), arising from the top decay, decays leptonically. Selection of this kind of decay boils down to a final state composed of \(1b+1j+1\tau +1\ell +1\mu \). The event numbers for the final state \(1b+1j+1\tau +1\ell +1\mu \) for the benchmark points and backgrounds are given in Table 6 at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\) at the 14 TeV LHC. This combination is rich with charged leptons with all three flavors, i.e., \(e, \,\mu , \tau \), where \(\tau \) is tagged as a jet, making it a very unique signal. In the case of BP2, we get an additional contribution from the single leptoquark production via \(c\, g \rightarrow \mu \, \phi \). Both BP1 and BP2 will be explored with very early data of 14 TeV LHC. However, for BP3, this final state has less to offer.

5.4 \(1b+3j+1\tau +1\mu \)

Next we consider a similar case as the previous one except that one of the \(W^\pm \) bosons coming from the leptoquark, decays hadronically giving rise to two additional jets. One muon can come either from the decay of the leptoquark to \(c\, \mu \) or from the \(W^\pm \) boson when both leptoquarks decay into \(t\, \tau \). Such a scenario creates \(1b+3j+1\tau +1\mu \) final state and the number of events are given in Table 7 at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\) at the 14 TeV LHC. Here the potential muon is either coming from the decay of one leptoquark in the pair production or from the production of single leptoquark in association of muon. This is the reason for the given parameter space; single leptoquark production contributes only for BP2, where such a coupling is non-vanishing. However, due to the reduction of the final state tagged charged leptons from three to one, we have a reasonable amount of backgrounds coming from \(t{\bar{t}}\), tZW, \(t{\bar{t}}Z\) and \(t{\bar{t}}b{\bar{b}}\), even with the requirement that the di-jet invariant mass produces the \(W^\pm \) mass.

If we consider the fact that the muons coming directly from the decay of the leptoquark are hard enough, i.e., \(p^{\mu }_T \gtrsim 100 \, \hbox {GeV}\) (see Fig. 2a), then implementation of such an additional cut reduces the potential \(t{\bar{t}}\) background by a factor of \(\sim 7\). Contrary to that, the signal numbers get a minimal reduction. After all the cuts both BP1 and BP2 can be probed at the 14 TeV LHC with an integrated luminosities of \(\sim 175 \, {\hbox {fb}}^{-1}\) and \(\sim 54 \, {\hbox {fb}}^{-1}\), respectively.

5.5 \(1b+1\tau +2\mu \)

Motivated by the fact that the multileptonic final states have less SM backgrounds, we try to tag \(2\mu \) final state where one of them is very hard coming from the direct decay of the leptoquark to \(c\, \mu \) and the other can come from the \(W^\pm \) boson decay. Here, in order to keep the final state robust for all the BPs, we do not tag the c-jet. This choice corresponds to a final state \(1b+1\tau +2\mu \), where we only tag one b-jet and one \(\tau \)-jet coming from the decay of the leptoquark into third generation, and no additional jets are required. Table 8 reflects the number of events for the benchmark points and the dominant SM backgrounds at the LHC with 14 TeV of center of mass energy and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). The requirement of an additional muon reduces the dominant \(t{\bar{t}}\) background to a negligible level. Here additional cuts, like the veto of a di-muon invariant mass around the Z mass value and the requirement of at least one muon with \(p_T\ge 100 \, \hbox {GeV}\) are applied to reduce the backgrounds further. In this case, for BP2, both the pair and the single leptoquark production processes contribute. The single leptoquark production contribution in the case of BP2 is denoted by ‘\(\dagger \)’. We see now both BP1 and BP2 can be probed within \(\sim 41\, \hbox {fb}^{-1}\) and \(\sim 30 \, \hbox {fb}^{-1}\) integrated luminosity, respectively, at the 14 TeV LHC. However, BP3 remains elusive in this final state.

Table 8 The number of events for \(1b+1\tau +2\mu \) final states for the benchmark points and the dominant SM backgrounds at the LHC with 14 TeV of center of mass energy and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). \(S_{\mathrm{sig}}\) denotes signal significance at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity and \(\int {\mathcal {L}}_5\) depicts the required integrated luminosity for \(5\sigma \) confidence level for the signal. Here we requirement at least the hardest muon (say \(\mu _1\)) should have \(p^{\mu _1}_T \ge 100\) GeV. The ‘\(\dagger \)’ denotes the contribution from \(c\,g\rightarrow \phi \,\mu \) production process
Fig. 5
figure 5

Invariant mass distribution of \(2j\,b\,\tau \) for the selected final states (a) \(2b+2\tau +1\ell \), (b) \(1b+2\tau +1\ell \) and (c) \(2b+2\tau +4j\) as explained in the text at the LHC with center of mass energy of 14 TeV and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\) for total (signal plus backgrounds) BP1, BP2 and the dominant SM backgrounds. It should be noted that in order to clearly visualize the signal and background events, we have scaled the signal events by a factor of 4 in all three panels and \(t{\bar{t}}\) events by a factor of 1 / 2 in (c) only

6 Leptoquark mass reconstruction and reach at the LHC

Ensuring the final states with excess events, we now look for various invariant mass distributions for the resonance discovery of the leptoquark. In this section, we explore both the third and the second generation decay modes to reconstruct the leptoquark mass. Leptoquarks decay to the third generation namely, \(t\, \tau \) or \(b\,\nu \). In order to construct the leptoquark mass we focus on the \(t\,\tau \) mode and require that at least one leg of the leptoquark pair production should be tagged. In this process we also require that both t and \(\tau \) should be tagged via their hadronic decay. This is due to the fact that the leptonic decay of \(W^\pm \) will produce a neutrino as missing energy and will spoil the mass reconstruction. Hence for that one leg we construct \(W^\pm \) via its hadronic decay mode with the criteria that \(|m_{2j}-m_W|\le 10 \, \hbox {GeV}\) and that \(W^\mp \) from the other leg can decay hadronically or leptonically, depending on the additional tagging, required for the final states. We also tag the \(\tau \) coming from the leptoquark decay as hadronic \(\tau \)-jet [44]. In such a case the only amount of missing energy will arise from neutrinos originating from \(\tau \) decay and will have much less effect on the leptoquark mass reconstruction. After reconstructing the \(W^\pm \) mass, the top mass is reconstructed via the \(2j\,b\) invariant mass distribution, where the di-jets are coming from the \(W^\pm \) mass window and the b-jet originates from the top decay. Next we take the events from the top mass window, i.e. \(|m_{2j\,b}-m_t|\le 10 \, \hbox {GeV}\), for the reconstruction of \(m_{2j\,b\,\tau }\). These choices are sufficient to reconstruct the leptoquark mass peak via the \(m_{2j\,b\,\tau }\) distribution. However, some of the SM backgrounds, specially \(t{\bar{t}}\), overshadow the distribution. To reduce the most dominant SM background, \(t{\bar{t}}\), we invoke additional tagging by requiring \(2b+2\tau +2j+1\ell \) and \(1b+2\tau +2j+1\ell \) final states, where the extra b-jet, \(\tau \)-jet and \(\ell \) are coming from the other leg of the leptoquark pair production. The result is depicted in Fig. 5a, b. Here the additional charged leptons and \(\tau \)- or b-jet come from the other leg of the pair-produced leptoquark. It can be seen from Fig. 5a, b that a sort of smeared mass edges for BP1 and BP2 around 650 GeV are formed and the SM backgrounds are populated at the lower mass end only.

Table 9 The number of events around the leptoquark mass peak, i.e. \(|m_{2j\,b\,\tau }- m_{\phi }|\le 10 \, \hbox {GeV}\) for the benchmark points and the dominant SM backgrounds at the LHC with the center of mass energy of 14 TeV and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\) for three final states: (a) \(2b+2\tau +2j+1\ell \), (b) \(1b+2\tau + 2j+1\ell \), and (c) \(2b+2\tau +4j\), respectively. The ‘\(\dagger \)’ contributions are from \(c g \rightarrow \phi \mu \) process and ‘\(^{*}\)’ contributions are from leptoquark pair production. The criteria \( |m_{2j}-m_W|\le 10\hbox {GeV}\) and \(|m_{2j\,b}-m_t|\le 10~\hbox {GeV}\) are also required in order to achieve the leptoquark mass peak

The situation improves in terms of the statistics if we require both the \(W^\pm \)’s decay hadronically and thus giving rise to a final state \(2b+2\tau +4j\) and the corresponding \(m_{2j\,b\,\tau }\) mass distribution is shown in Fig. 5c. We can clearly see that the dominant SM backgrounds peak to the lower mass end and the signal mass peak for BP1 and BP2 are prominent. A suitable mass cut, i.e. a mass window around the 650 GeV for BP1 and BP2, will give us an accurate estimate for the discovery reach. In Table 9, we provide the number of events around the leptoquark mass peaks, i.e. \(|m_{2j\,b\,\tau }-m_{\phi }|\le 10 \, \hbox {GeV}\) for the benchmark points and the dominant SM backgrounds at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\) at the 14 TeV LHC. The mass reconstruction at \(100 \, \hbox {fb}^{-1}\) is highest for the \(2b+2\tau +4j\) final state, i.e., \(5.0\sigma \) and \(4.0\sigma \) for BP1 and BP2, respectively, while for the other two final states we need more luminosity to achieve \(5\sigma \) significance. A mass scale of \(\sim 1.3\, \hbox {TeV}\) can be probed at an integrated luminosity of \(3000 \, \hbox {fb}^{-1}\) for \(\beta = {\mathcal {B}} (\phi \rightarrow t\, \tau )=1.0\).

We have seen that the dominant decay modes of the leptoquark are in the third generation, specially to \(t\,\tau \). This gives rise to a very rich final state; however, in the presence of a large number of jets, and specially the missing momentum from neutrino, the peaks are smeared and we often encounter a mass edge of the distribution instead of a proper peak. A much cleaner mass peak reconstruction is possible via the invariant mass of the c-jet and the muon coming from the single leptoquark vertex because of the presence of a smaller number of jets and absence of potential missing momentum. This can happen in the case of BP2, where such a coupling has been introduced. However, due to the constraints from flavor observables [28,29,30,31,32,33,34], we choose the branching fraction of the leptoquark to \(c\,\mu \) to be only \(11\%\), which reduces the signal events. We improve the signal statistics by requiring one of the pair-produced leptoquarks to decay into \(c\,\mu \) and the other into \(t\, \tau \). To reduce the SM backgrounds, we tag the decay chain of the third generation by requiring one b-jet and at least one \(\tau \)-jet. In order to further enhance the signal number, we require \(W^\pm \) from this chain to decay hadronically, giving rise to two jets which are tagged with their invariant mass within \(\pm 10 \, \hbox {GeV}\) of the \(W^\pm \) mass, i.e., \(|m_{jj}-m_{W^\pm }| \le 10 \, \hbox {GeV}\). In addition, we insist on having one c-jet with \(p_T \ge 200 \, \hbox {GeV}\) and one muon with \(p_T \ge 100 \, \hbox {GeV}\) and also no spurious di-lepton coming from the Z boson, i.e., \(|m_{\ell \ell }- m_Z|\ge 5 \, \hbox {GeV}\).

Fig. 6
figure 6

Invariant mass distribution of one muon and one c-jet for the selected final state as explained in the text at the LHC with center of mass energy of 14 TeV and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\) for total (signal plus backgrounds) BP2 signal in orange and the dominant SM backgrounds in dark blue. The signal events are scaled by a factor of 4 in order to have clear visualization

After having considered the above-mentioned criteria, we plot the invariant mass distribution of the c-jet and muon in Fig. 6 for BP2Footnote 1 and the dominant SM backgrounds, namely \(t{\bar{t}}, \, t{\bar{t}}Z, \, tZW \). The detection efficiency of such c-jet is, however, not very high and for our simulation we choose the tagging efficiency of a c-jet is \(50\%\) [46]. The SM processes that contribute as backgrounds are mainly contributing due to faking of a b-jet as a c-jet, which we have taken as \(25\%\) per jet [46]. There are also possibilities of light-jets fake as c-jet [46]. Table 10 shows the numbers of such events around the peak, i.e. \(|m_{\mu \, c} - m_{\phi }|\le 10 \, \hbox {GeV}\) for signal events for BP2 and for the SM backgrounds. It is evident that the integrated luminosity of \(\sim 100 \, \hbox {fb}^{-1}\) at the LHC with 14 TeV center of mass energy can probe for this mode the peak at \(3\sigma \) level.

Table 10 The number of events for the benchmark points and the dominant SM backgrounds at the LHC with center of mass energy of 14 TeV and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). Here \(1c^{\circledast }\)-jet has \(p_T\ge 200 \, \hbox {GeV}\) and \(\mu ^{\circledcirc }\) has \(p_T \ge 100 \, \hbox {GeV}\). The ‘\(\dagger \)’ contributions are from the \(c g \rightarrow \phi \mu \) process and the ‘\(^{*}\)’ contributions are from leptoquark pair production

Naively, one can also look for the final state consisting of \(1c+2\mu \), by requiring the second muon of \(p_T\ge 100 \, \hbox {GeV}\), i.e., expecting it to come from the decay of the other leptoquark to the \(c\, \mu \) state. For BP2, as the branching fraction of the leptoquark to \(c\,\mu \) is only \(11\%\), the requirement of both the pair-produced leptoquarks to decay in \(c\,\mu \) will further reduce the effective branching fraction. To avoid further reduction from the c-jet tagging efficiency [46], we only tag one of the two c-jets as a c-jet. A cumulative requirement of is also assumed to reduce the SM di-muon backgrounds coming from the gauge boson decays as can be seen in the second final state of Table 10. Though this has reduced the contribution from leptoquark pair production, it enhanced the single leptoquark contribution via \(c\, g \rightarrow \phi \, \mu \). The signal reach for BP2 in this case is \(1.5\sigma \) at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity at the LHC with 14 TeV center of mass energy. If we proceed to tag the second c-jet, clearly the signal event reduces further, but the final state comprised of does not have any noticeable backgrounds as can be read from the third final state in Table 10. However, such a choice of final state yields only a reach of \(\sim 1.4\sigma \) at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity at the 14 TeV LHC.

Fig. 7
figure 7

Required integrated luminosity for \(5\sigma \) reach at the LHC with 14 TeV of center of mass energy for the final states defined in Table 5 (in panel a) and Table 6 (in panel b), respectively, where \(\beta \) and \(\beta _1\) correspond to the branching fraction to \(t\,\tau \) and \(\beta _2\) denotes the branching fraction to \(c\, \mu \)

It is apparent from the discussions in the preceding sections that the final state defined in Table 5 has the highest reach which probes the third generation decay mode. Figure 7a, b presents the reach for the scalar leptoquark mass in terms of integrated luminosity at the 14 TeV LHC corresponding to the final states given in Tables 5 and 6, respectively. It can be seen that, for BP1, where the leptoquark branching fraction to \(t\,\tau \) is 61%, a leptoquark mass of 1.6 TeV can be probed at the LHC with \(3000\, \hbox {fb}^{-1}\) of integrated luminosity. If such a branching ratio is 100%, the reach is enhanced to 1.8 TeV.

Similarly we can look into the final state defined in Table 6, where for BP2 both single and pair productions of the leptoquark contribute, and the final state is comprised of both the second and the third generation decay modes of the leptoquark. Here we define \(\beta _1={\mathcal {B}}(\phi \rightarrow t \, \tau )=0.50\) and \(\beta _2={\mathcal {B}}(\phi \rightarrow c\, \mu )=0.1\). We find a leptoquark mass scale reach of \(\sim 920\, \hbox {GeV}\) is desired at an integrated luminosity of \(3000 \, \hbox {fb}^{-1}\). However, if we take \(\beta _1=\beta _2=0.5\), the reach increases to 1.2 TeV. These calculations are done with the renormalization/factorization scale \(\mu =\sqrt{\hat{s}}\), which give a conservative estimate. A scale variation would enhance such a reach by 10–20%.

Fig. 8
figure 8

Single leptoquark production cross-section in association of lepton via quark gluon fusion verses leptoquark mass for the Yukawa couplings of BP1, BP2 and for universal coupling \(Y^{\mathrm{L},\mathrm{R}}_{ii}=0.5\), \(i\in \{1,2,3\}\) at the 14 TeV LHC. A NLO k-factor of 1.5 is considered [48]. The solid curves are obtained for renormalization/factorization scale \(\mu =\sqrt{\hat{s}}\) and the dashed curves depict the variation for \(\mu =[m_\phi /2,2m_\phi ]\)

7 Single leptoquark production and discovery reach

It is well known that the leptoquark pair-production cross-section is almost independent of the Yukawa type couplings \(Y^{\mathrm{L},\mathrm{R}}_{ii}\) except for very high values [47] and is actually determined by the leptoquark mass and strong coupling at a given scale. Due to the presence of the strong interaction, the pair-production cross-section range for the leptoquark is higher than the similar mass range for the weak scalar pair-production cross-section. Unlike the weakly charged scalar, there exists an additional mechanism that can produce a single leptoquark in association with leptons of a given flavor via Yukawa type couplings \(Y^{\mathrm{L},\mathrm{R}}_{ii}\). Quark fusion with a gluon can give rise to final states consisting of either \(\phi \, \ell \) or \(\phi \, \nu \).

In Fig. 8 we show the production cross-section of such a single leptoquark in fb with the variation of the leptoquark mass at the 14 TeV LHC. The cross-sections are calculated using CalcHEP [12], where we choose 6TEQ6L [40] as PDF and the variations for three different scale choices, i.e. \(\mu =\sqrt{\hat{s}},~m_{\phi }/2, ~ 2m_{\phi }\), are shown. The results for three different production cross-sections are shown: \(q \, g \rightarrow \phi +X\) in green, \(b \,g \rightarrow \phi \,\nu \) in red and \(c\, g \rightarrow \phi \, \mu \) in blue. The k-factor of 1.5 has been taken into account [48]. The leptoquark will decay to combinations of quark and lepton. However, among the chosen benchmark points only the couplings of BP2 can have single leptoquark production via \(c\, g \rightarrow \phi \, \mu \) and both BP2, BP3 contribute via the \(b\,g \rightarrow \phi \nu \) production channel. In the case of BP2, the leptoquark still dominantly decays to \(t\, \tau \) with a decay branching fraction of \(50\%\) and to \(c\, \mu \) only with \(10\%\). From a collider viewpoint, we also show the estimate of the inclusive single leptoquark production cross-section by considering universal Yukawa type couplings in all generations, namely, \(Y^{\mathrm{L},\mathrm{R}}_{ii}=0.5\) for \(i\in \{1,2,3\}\).

Table 11 The number of events for the benchmark points and the dominant SM backgrounds at the LHC with 14 TeV of center of mass energy and at an integrated luminosity of \(100 \, \hbox {fb}^{-1}\). Here \(1c^{\circledast }\)-jet has \(p_T\ge 200 \, \hbox {GeV}\) and \(\mu ^{\circledcirc }\) has \(p_T \ge 100 \, \hbox {GeV}\). The ‘\(^{*}\)’ contributions are from leptoquark pair production and ‘\(\dagger \)’ contributions are from the \(c g \rightarrow \phi \mu \) process. Here VVV,  VV are contributions from the SM gauge bosons where \(V=W^\pm , Z\)

In Table 11 we look for the final states coming from both the decay modes. The first final state deals with \(1b+ 1\tau \) arising from the decay of the leptoquark into \(t\, \tau \). We also tag the charged lepton \(e, \mu \) coming from the \(W^\pm \) decay along with a muon supposedly originating from one leptoquark decay with \(p_T\ge 100 \, \hbox {GeV}(\circledcirc )\). A requirement of \(p^{j_1}_T \ge 100 \, \hbox {GeV}\) for first \(p_T\) ordered jets, which mostly comes from the leptoquark decay, is also made to diminish the SM backgrounds further. For the first final state, the BP2 signal significance reaches \(3.9 \, \sigma \) at the LHC with 14 TeV of center of mass energy and \(100 \, \hbox {fb}^{-1}\) of integrated luminosity. If we tag both muons, coming from the leptoquark decay via \(c\, \mu \), with \(p_T \ge 100 \, \hbox {GeV}\) and the first \(p_T\) ordered jet with \(p_T \ge 200 \, \hbox {GeV}\), then the corresponding signal is given in the second row as \(1\le n_j \le 2 + p^{j_1}_T \ge 200 \, \mathrm{GeV}+ \ge 2\ell (2\mu ^{\circledcirc })\), where we do not tag any c-jet. However, due to the fact that the branching ratio to \(c\,\mu \) for BP2 is only 10%, the signal significance reaches only \(1.2\,\sigma \) at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity. If we further tag one of the two c-jets as c-jet, which is coming from leptoquark decay, then the signal significance for BP2 can reach only \(0.6\, \sigma \) at \(100 \, \hbox {fb}^{-1}\) of integrated luminosity. The c-jet tagging efficiency [46] also significantly affects the event numbers.

Fig. 9
figure 9

Required integrated luminosity for \(5\sigma \) reach at the LHC with 14 TeV of center of mass energy for the final states defined in Table 10 (in panel a) and Table 11 (in panel b), where \(\beta _1\) and \(\beta _2\) are the branching fraction to \(t\,\tau \) and \(c\, \mu \), respectively

The excess of events compared to the SM prediction provides a hint for some BSM physics. However, the conclusive discovery of a new particle can only happen via the reconstruction of its mass, through possible invariant mass reconstructions. Figure 9 shows the reach of the leptoquark mass reconstructed via \(c\,\mu \) for the final states given in Table 10 (in panel (a)) and Table 11 (in panel (b)). The requirement of such final states involves decay modes in both the second and the third generations. Similar to the previous reach plots (Fig. 7) here also, \(\beta _1={\mathcal {B}}(\phi \rightarrow t \, \tau )\) and \(\beta _2={\mathcal {B}}(\phi \rightarrow c\, \mu )\). The choice of \(\beta _1=\beta _2=0.5\) results in a reach of the leptoquark mass \(\sim 1.2\, \hbox {TeV}\) (in Fig. 9a) and 1 TeV (in Fig. 9b) at the 14 TeV LHC with \(3000 \, \hbox {fb}^{-1}\) of integrated luminosity. It should be noted that though the final reach is almost the same for the two cases, see Fig. 9a, which is for the final state given in Table 10, it mostly depends on the leptoquark pair production dominated by the gluon and quark fusion and thus is independent of \(Y^{\mathrm{L},\mathrm{R}}_{ii}\). On the other hand, Fig. 9b, which is for the final state given in Table 11, depends on both single and pair production of the leptoquark. As a consequence, this mode can be a good probe to the leptoquark Yukawa couplings \(Y^{\mathrm{L},\mathrm{R}}_{ii}\). A comparative study of both such reconstructions would certainly provide an upper hand understanding of the model parameters.

8 Summary

In this article we study the phenomenology of a scalar leptoquark via its dominant decay into third generation leptons and quarks and also from the combined decays into second and third generation channels. The leptoquark considered here has a hypercharge of \(-1/3\) units. By choosing some suitable benchmark points, we list the final states with well-defined cumulative cuts arising from leptoquark pair production, at the 14 TeV LHC with \(100\,\hbox {fb}^{-1}\) of integrated luminosity in Tables 4 and 5. These searches show that b and \(\tau \) jet tagging along with their invariant mass veto cuts helps to reduce the SM backgrounds immensely.

Next we discuss the phenomenology when one of the leptoquark decays into the third generation and other decays into the second generation. Due to the constraints from flavor data we conservatively allow, in BP2, for the leptoquark decays to \(c\,\mu \) with branching fraction by 10% only. Nevertheless from a collider perspective one can tune such a branching fraction while looking into a certain final state and can obtain independent limits. In Tables 6 and 7 we have analyzed the final states where both decay modes are reflected. For Table 6 the reach is comparable for BP1 and BP2, where only for BP2 single leptoquark production contributes. In Table 7 the significance drops due to lower branching fraction of \(W^\pm \) into leptons. Our study shows that a scalar leptoquark with hypercharge \(-1/3\) can be probed till \(\sim 2 \, \hbox {TeV}\) at the LHC with 14 TeV of center of mass energy and \(3000 \, \hbox {fb}^{-1}\) of integrated luminosity.

The leptoquark mass has been reconstructed via its decay to the third and second generations. For the decay into third generation states, we reconstruct \(m_{2j\,b\,\tau }\) and for BP1 it has a reach of \(\sim 1.3\, \hbox {TeV}\) that can be probed with the \(3000 \, \hbox {fb}^{-1}\) data. Next we reconstructed the leptoquark mass via \(c\,\mu \) invariant mass reconstruction. However, we require an environment that has additional tagging of b-jet and \(\tau \)-jet coming from third generation decays. This choice makes the final state almost background free and also increases the signal strength due to the higher branching fraction in the third generation.

We also study the single leptoquark production via b-gluon and c-gluon fusion in Fig. 8. The production cross-section improves significantly in the case of inclusive single leptoquark production while considering equal Yukawa type couplings for all generations. We highlight the reach of the leptoquark mass reconstruction from the single production in Fig. 9. For choices of couplings as in BP1 and BP2, we find that the reach is \(\sim 1.2\, \hbox {TeV}\) at the 14 TeV LHC with \(3000 \, \hbox {fb}^{-1}\) of integrated luminosity. As the limits obtained in this work are well within the current and future reach of the LHC, dedicated searches for the proposed final states will be important to confirm/falsify the existence of such a BSM particle.