1 Introduction

The Randall–Sundrum (RS) model of warped extra dimensions addresses issues of gauge and mass hierarchy in a simple manner and may yet provide some new observable consequences at the large hadron collider (LHC). The model, as originally conceived, provides a neat separation of gravity and standard model (SM) physics by localising these on two branes (UV and IR) located at the end-points of a segment of a five-dimensional bulk, respectively. The localisation of all SM particles on the IR brane has disastrous consequences for the model: the IR localised fields turn out to be composite and the model, consequently, fails miserably when confronted with electroweak precision measurements. One has to differentially localise the various SM fields in the bulk with only the Higgs field localised on or near the IR brane and the models thus constructed are what are known as the Bulk RS Models [1,2,3,4,5,6,7]. But even this is not enough to ensure agreement with electroweak measurements and one has to invoke a bulk custodial symmetry or to deform the AdS metric near the IR brane leading to two different Bulk Models: the custodial model [8] and the deformed model [9].

Even after making these modifications both the models demand that, the Kaluza–Klein (KK) modes of the bulk-localised SM particles which are constrained by electroweak data should be rather heavy. In the custodial model, these turn out to be of the order of 2.5–3 TeV, at least, for the first KK mode but, generically, the deformed model allows for lighter KK resonances. It is possible to accommodate a KK Higgs in the deformed model in the 1 TeV range and it is the KK Higgs in this mass range that will interest us in the rest of this paper. In other words, we are restricting ourselves to the deformed model and discussing KK Higgs production in the context of this model. In an earlier paper, we have discussed, in the context of the deformed model, the production of a KK Higgs resonance in gluon-gluon fusion through a triangle of top quarks [10], In this paper, we study KK Higgs production in association with a \(t {\bar{t}}\) pair. This, as we know from even the SM case, is a difficult mode to study because of the tiny cross-sections. Our attempt is to see how much of the KK Higgs mass range can be covered in the future runs of the LHC. At the outset, our intuition that this is not a hopeless task is built on the observation that the KK Higgs decays also into a boosted top-pair and the four-top final state with two tops boosted is shielded from very large SM backgrounds. Having said that, this is by no means an easy task and we devote the rest of the paper to describing what we can do to enhance the signal efficiency and have a reasonable number of events in this channel.

The paper is organised as follows: In Sect. 2 we have briefly discussed direct constraints on the four top final state obtained from colliders. Section 3 contains details of the signal and background simulation and the search strategies proposed to enhance the signal and achieve relevant background rejection. Finally in Sect.  4 we present results and our conclusion is given in Sect. 5.

2 Constraints on a four top final state search

The four-top final state studies at the colliders are rare due to the complexity of the final state and its very low cross sections. Even at the top factory like the LHC, the SM cross section for a four top final state is negligible (11 fb) in comparison to the huge SM two top background. Searches for such a final state have been done in Refs. [11,12,13,14]. We refer to most recent work by Aaboud et al. which provide the most stringent bounds. They put a limit on the production cross section of \(16 ^{+12}_{-9} fb \) on a massive resonance produced in association with top-quark pair resulting into a four-top final state. The cross-section for the process that we are considering is much smaller than the aforementioned limit and the KK-Higgs production does not show up at current luminosity. Even at higher luminosity the search has to be carefully conducted and this is what we now discuss.

3 Signal and background simulations

A top-quark decays to a W boson and a b-quark, where a W boson can decay either to a lepton and neutrino or two quarks . Thus for any four-top final state we will have a 12 parton final state, which should have a minimum of 4 b-quarks. Based on the most possible combinations, the final state is classified into four categories [12] namely, hadronic (all top-quarks decay hadronically), semileptonic (one of the four top-quarks decays leptonically), similarly the dilepton and trilepton final states are where two and three top-quarks decay leptonically respectively.

We consider a special four-top final state:

$$\begin{aligned} p p\rightarrow H_{1} t{\bar{t}} \rightarrow t{\bar{t}} t{\bar{t}}. \end{aligned}$$
(1)

In the above process the four-top final state is special in a sense that two of the top-quarks that are decay products of \(H_{1}\) are boosted. Such a differential distribution of boost among the four top-quarks in the final state makes our search interesting to explore.

For this special four-top final state we find that, the case where two boosted top-quarks decay hadronically, is the only one where, efficient mass reconstruction of \(H_{1}\) is possible, provided, other irreducible backgrounds are efficiently handled. We in the present work have performed a thorough inspection of this special four-top final state signal, where, two boosted top-quarks decay hadronically, with a thrust on selective reduction of multijet QCD backgrounds. We select events which do not have a lepton with \(p_{T} > \) 25 GeV in order to narrow down on events where boosted top-quarks decay hadronically. So we find that the special four-top final state that we are left with is either hadronic (all four top-quarks decaying hadronically) or special semileptonic, where, boosted top-quarks decay hadronically and the lepton that comes from one of two spectator top-quarks has a \(p_{T}\) less than 25 GeV.Footnote 1

We closely follow Ref. [13] for the choice of backgrounds. The Standard Model four-top and the top-quark pair with two additional heavy quark jets form two irreducible backgrounds for our signal. Their cross sections at 14 TeV centre of mass energy are given in the Table 1. Other Multijet backgrounds that have not been listed from Ref. [13] in the following table are QCD multijet background, \(t\bar{t}\)+jets,Footnote 2 \(t\bar{b}\)+jets and \(b\bar{b}\)+jets. We are able to tab them by our choice of cuts, what still remain substantial post cuts are the irreducible backgrounds given below:

Table 1 The simulated cross sections for the irreducible SM backgrounds for the hadronic decay channel in the four top final state
Full size table

In Table 1, \(t\bar{t}t\bar{t}\) is the SM four top final state and \(t\bar{t}bb\) is another multijet background. The parton-level amplitudes for both the signal and background were generated using MADGRAPH [15] at 14 TeV centre of mass energy using parton distribution function NNLO1 [16] and subsequently showering is done in PYTHIA [17] while the bulk higgs model files have been generated using FEYNRULES [18].

Once the simulation of the signal and background is completed using MADGRAPH and PYTHIA, then all final state particles in the signal and background are clustered into jets employing the \(anti-k_T\) [19] clustering algorithm in FASTJET [20, 21] with jet radius parameter set to \(R=0.4\). We accept only those jets which satisfy \(|\eta |<2.7\) and \(p_T>40\) GeV.

Fig. 1
figure 1

Number of jets to the left and the number of b-tagged jets as a discriminating variable to the right

Full size image
Fig. 2
figure 2

Scalar sum of \(p_{T}\) of jets to the left and the \(H_{1}\) mass reconstruction to the right

Full size image

As two top quarks are resulting from the decay of massive \(H_{1}\) they are boosted in comparison to other two top quarks. This as mentioned earlier is a unique feature of our signal. Along with this other features that are special for any four top final state in general are [11]:

  • \(N_{jets}\): The jet multiplicity or the total number of jets. This variable generally has a large value for a four-top final state, as jets are formed from a minimum of 12 partons. We expect \(N_{jets}\) to be atleast greater than 9 for any four-top final state.

  • \(N_{btags}\): The b-jet multiplicity the total number of b-tagged jets. A four top final state should by virtue have four b-quarks. Due to the limitation of b-tagging efficiency we can tag a minimum of three b-quarks per event and still have sufficient number of signal events passing this cut. So we expect \(N_{btags}\) greater than equal to 3.

  • \(H_{T}\): The total scalar sum of \(p_{T}\) for all the jets. As the \(H_{1}\) decays to boosted top-quarks the resulting decay products of such top-quarks are also boosted. Hence, \(p_{T}\) of jets is high in our case, thus this total sum \(H_{T}\) is very high. The maximum cut on \(H_{T}\) can be around 1200 GeV for \(M_{h_{1}}\) = 1 TeV, for higher mass of \(H_{1}\) this value will be still higher.

The above variables are the discriminating variables between our signal and the backgrounds. In Fig. 1 we show the plot for total number of jets after clustering and passing the set criteria along with number of jets that are b-tagged for our signal and all irreducible backgrounds. Our finding that the signal can be efficiently separated if the number of jets required are greater than equal to nine and b-tagged jets are greater than or equal to 3 can be clearly seen in the plots.

Further, as mentioned earlier, in a state of high boosted jet multiplicity, another variable that comes handy for segregation of the signal is \(H_{T}\). \(H_{T}\ge \) 1250 GeV is the best choice for the zero lepton case that we have considered. The plot for this variable is given in Fig. 2.

When \(H_{1}\) decays to a pair of boosted top quarks, each top quark further decays to three partons, thus we ideally would expect six partons to result into at-least a minimum of six high \(p_{T}\) jets after clustering. These jets could then be used to reconstruct the \(H_{1}\) mass. Contrary to our initial expectation we find that the leading jet is highly boosted and seems to accommodate two partons hence we are able to reconstruct the \(H_{1}\) mass with a total of five jets where four are normal jets and one is a b-tagged jet as shown in Fig. 2. Finally, we put a mass window cut around the \(M_{h_{1}}\)

In Table 2 we show the cut flow table for the signal with KK-Higgs mass \(M_{h_{1}}=1000\) GeV and the irreducible backgrounds.Footnote 3

Table 2 Cut flow table for hadronic decay of all top quarks for the \(M_{h_{1}}=1 \) TeV
Full size table

4 Results

Using the sequence of cuts described in the previous section for separation of the signal and the background, we obtain results for required luminosity to get a statistically significant result. These are given in Table 3.

Table 3 Integrated luminosity in fb\(^{-1}\) for 5\(\sigma \) and 3\(\sigma \) sensitivities
Full size table

We have focused only on the hadronic channel and we find that in this channel the search luminosity requirement is high. We find for \(M_{h_{1}}= 900\) GeV the 5\(\sigma \) luminosity is 1046 \(fb^{-1}\) and \(M_{h_{1}} = 1000\) GeV the 5 \(\sigma \) luminosity is 2276 \(fb^{-1}\) using our proposed search strategy. Our results suggest that future higher luminosity runs may be sensitive, at least at the 3\(\sigma \) level, to the production of \(H_{1}\).

In the present paper we have not performed a full detector level simulation. However the luminosity requirements may increase a little once we take into account the mistag rate in the QCD backgrounds.

5 Conclusion

In this paper, we have presented an effective search strategy for the \(H_{1}\), the first KK mode of Higgs, in the context of the deformed Randall–Sundrum model. We have considered the production of \(H_{1}\) in association with a \(t \bar{t}\) and have studied effective ways of unravelling this special four-top final state with two boosted top-quarks. We have focused only on the hadronic channel and we find that in this channel the search luminosity requirement is high, as shown in Table 3. We find that the luminosity requirement increases for higher masses, hence we limit our search to \(H_{1}\) mass of only 1200 GeV. For higher masses cross sections are extremely tiny and have no room for exploration at the existing colliders. This is also the reason that we have not been able to use this channel to study \(H_{1}\) production in the custodial model. We have to wait for future colliders that may promise higher luminosity in order to explore higher masses.