Towards a public analysis database for LHC new physics searches using MadAnalysis 5

We present the implementation, in the MadAnalysis 5 framework, of several ATLAS and CMS searches for supersymmetry in data recorded during the first run of the LHC. We provide extensive details on the validation of our implementations and propose to create a public analysis database within this framework.


Introduction
The LHC was designed as a machine of discovery. It was built to explore the TeV energy scale, in order to unravel the mechanism of electroweak symmetry breaking and shed light on new physics beyond the Standard Model (SM). The recent discovery [1,2] of a new particle with mass of 125 GeV and properties consistent with the SM Higgs boson is a first triumph for the LHC physics program and has profound implications for our understanding of the universe. We are, however, still left with many fundamental questions open, and to address them it is imperative that the search for new physics continue, at the LHC and elsewhere.
During Run I of the LHC at center-of-mass energies of √ s = 7 and 8 TeV, the ATLAS and CMS collaborations have carried out an extensive program searching for new physics in many different channels [3][4][5][6]. Since no signal was found, the experimental collaborations interpreted their results setting limits in terms of popular models, such as the CMSSM (Constrained Minimal Supersymmetric Standard Model, see e.g. [7]), or in terms of so-called a Present address: Department of Infectious Disease Epidemiology, Imperial College London, St Mary's Campus, Norfolk Place London, W2 1PG, UK.
Simplified Model Spectra (SMS). 1 These searches will be pursued further at higher energies, with first results to be expected soon after the start of Run II in 2015.
There exist, however, many different beyond-the-SM (BSM) theories, and each of them comes with a large variety of concrete realizations. This leads to a multitude of possible scenarios, with complex interrelations between parameters and signatures. It is a challenge for the whole community to work out the implications of the LHC results in the contexts of all these different models, to derive the relevant limits, point out possible loopholes in the current searches, and help design the analyses for the next phase of LHC running at higher energy.
To this end, many groups have been developing private codes for the interpretation of the LHC results. Moreover, recently some public tools became available, which serve the whole community. For the interpretation in the context of Simplified Models, there are SModelS [10] and Fastlim [11]. SModelS takes the spectrum of any BSM scenario, decomposes it into SMS topologies, and compares it to the cross section upper limits from more than 50 ATLAS and CMS SMS results. Fastlim reconstructs the visible cross sections from pre-calculated efficiency and cross section tables for simplified event topologies, currently taking into account 11 ATLAS analyses which mainly focus on searches for supersymmetric partners of the top and bottom quarks (stops and sbottoms, respectively). For confronting simulated events of any model to LHC results, there is CheckMATE [12]. This program currently has 8 ATLAS and 1 CMS SUSY analyses implemented, which it re-interprets based on fast simulation. Another tool, XQCAT [13], is designed specifically for testing scenarios with heavy extra quarks, based on a CMS search for top partners as well as two SUSY searches. Finally, ATOM [14] is being developed for calculating the efficiencies of signal and control regions of various LHC searches based on the Rivet [15] toolkit.
In this paper, we follow a complementary approach. We present the implementation of several ATLAS and CMS supersymmetry (SUSY) analyses in MadAnalysis 5 [16,17], with simulation of detector effects based on Delphes 3 [18], and propose to create a public analysis database (PAD) within this framework. MadAnalysis 5 offers a user-friendly platform for collider phenomenology, and the PAD we propose will be easily accessible to and extendible by the whole community.
In BSM searches, sets of selection criteria are designed in order to maximize the sensitivity to expected signals of new physics. These define so-called signal and control regions, described in the experimental publications. For interpreting a search in the context of a given new physics model, one has to implement these selection criteria together with a description of the detector performance (emulating the various object definitions and efficiencies) in a simulation tool. Based on simulated event samples for the model being tested, the expected number of signal events in the various signal regions (SRs) can then be computed and compared to the number of observed events and the number of expected SM background events, which are reported in the experimental publication.
Non-collaboration members however do not have access to the experimental data, nor the Monte Carlo (MC) event set simulated with an official collaboration detector simulation. This renders the implementation and validation of ATLAS and CMS analyses for re-interpretation of the experimental results in general contexts a tedious task, even more so as the information given in the experimental papers is often incomplete (we will comment more on this in Section 3). We therefore think that a common platform for collecting object definitions, cuts, and all other information necessary to reproduce or use the results of the analyses will be of great value to the high-energy physics community. Moreover, as our project follows an Open Access and Open Data approach, we hope that it will benefit the scientific communication and in particular motivate ATLAS and CMS to provide more information on their analyses, in line with the Les Houches Recommendations [19].
The rest of the paper is organized as follows. In Section 2, we briefly recall some new features in MadAnalysis 5, which are pertinent for implementing LHC analyses, and describe the modifications to the Delphes 3 detector simulation which we adopted for this project. In Section 3, we present some ATLAS and CMS analyses which we implemented in the MadAnalysis 5 framework and report in detail on their validation. The relevant C++ codes are all publicly available and may thus constitute the foundation of the PAD. A module for a simplified statistical interpretation of the simulated signals is presented in Section 4. Section 5 contains our conclusions.
2 New developments in MadAnalysis 5 and Delphes 3 2.1 Dealing with multiple signal regions in MadAnalysis 5 In most experimental analyses performed at the LHC, and in particular the searches considered in this work, a branching set of selection criteria ("cuts") is used to define several different sub-analyses ("regions") within the same analysis. In conventional coding frameworks, multiple regions are implemented with a nesting of conditions checking these cuts, which grows exponentially more complicated with the number of cuts. The scope of this project has therefore motivated us to extend the MadAnalysis 5 package to facilitate the handling of analyses with multiple regions, as first presented in [20] and described in detail in [17].
From version 1.1.10 onwards, the implementation of an analysis in the MadAnalysis 5 framework consists of implementing three basic functions: -Initialize, dedicated to the initialization of the signal regions, histograms, cuts and any user-defined variables; -Execute, containing the analysis cuts and weights applied to each event; and -Finalize, controlling the production of the results of the analysis, i.e., histograms and cut-flow charts.
The new functionalities of MadAnalysis 5 for implementing LHC analyses are described in detail in the new manual of its expert mode [17]. To illustrate the handling of multiple regions, we present a few snippets of our implementation [21] of the CMS search for stops in final states with one lepton [22] (see Section 3.1). This search comprises 16 SRs, all of which must be declared in the Initialize function. This is done through the AddRegionSelection method of the analysis manager class, of which Manager() is an instance provided by default with each analysis. It takes as its argument a string uniquely defining the SR under consideration. For instance, two of the 16 SRs of the CMS analysis are declared as Manager()->AddRegionSelection( "Stop->t+neutralino,LowDeltaM,MET>150"); Manager()->AddRegionSelection( "Stop->t+neutralino,LowDeltaM,MET>200"); The Initialize function should also contain the declaration of selection cuts. This is handled by the AddCut method of the analysis manager class. If a cut is common to all SRs, the AddCut method takes as a single argument a string that uniquely identifies the cut. An example of the declaration of two common cuts is Manager()->AddCut("1+ candidate lepton"); Manager()->AddCut("1 signal lepton"); If a cut is not common to all regions, the AddCut method requires a second argument, either a string or an array of strings, consisting of the names of all the regions to which the cut applies. For example, an E miss T > 150 GeV cut that applies to four SRs could be declared as Histograms are initialized in a similar fashion using the AddHisto method of the manager class. A string argument is hence required to act as a unique identifier for the histogram, provided together with its number of bins and bounds. A further optional argument consisting of a string or array of strings can then be used to associate it with specific regions. The exact syntax can be found in the manual [17].
Most of the logic of the analysis is implemented in the Execute function. This relies both on standard methods to declare particle objects and to compute the observables of interest for event samples including detector simulation [16] and on the new manner in which cuts are applied and histograms filled via the analysis manager class [17]. In particular, we emphasize the existence of a new isolCones method of the RecLeptonFormat class for testing the isolation of the leptons. This returns a vector of IsolationConeType objects describing the transverse activity in a cone of radius ∆R centered on the lepton and whose properties are the following: -deltaR(): returns the size of the cone; ntracks(): returns the number of tracks present in the cone; -sumPT(): returns the scalar sum of the transverse momenta of all tracks lying in the cone; -sumET(): returns the scalar sum of the transverse energy deposits in the cone.
In general, experimental analyses only consider tracks with a transverse momentum larger than a given threshold. It should be noted that MadAnalysis 5 does not control this last functionality so that the threshold must be specified at the level of the detector simulator. All these features should be used together with the modifications of Delphes 3 described in the next subsection. Below we provide a couple of examples for applying cuts and filling histograms. After having declared and filled two vectors, SignalElectrons and SignalMuons, with objects satisfying the signal lepton definitions used in the CMS-SUS-13-011 analysis, we require exactly one signal lepton with the following selection cut: if( !Manager()->ApplyCut( (SignalElectrons.size()+SignalMuons.size())>0, "1+ candidate lepton") ) return true; The if(...) syntax guarantees that a given event is discarded as soon as all regions fail the cuts applied so far.
Histogramming is as easy as applying a cut. For example, as we are interested in the transverse-momentum distribution of the leading lepton, our code contains This results in the filling of a histogram, previously declared with the name "pT(l)" in the Initialize method, but only when all cuts applied to the relevant regions are satisfied.
Finally, event weights often need to be applied at the analysis level to correct for the efficiency with which physical objects, such as electrons or jets, are identified or likely to trigger the event. In MadAnalysis 5, the weight of an event can easily be modified, if necessary, by using the SetCurrentEventWeight method of the manager class.
After the execution of the program, a set of Saf files (an Xml-inspired format used by MadAnalysis 5) is created. These files are organized in an automatically generated output directory with the same name as the input file (containing the path(s) to the event file(s) to consider), chosen to be input.txt for the sake of the example. At the root of this output directory, one finds a file named in our case input.txt.saf with general information on the analyzed events, such as the associated cross section, the number of events, etc. It comes together with a series of subdirectories named according to the different analyses that have been executed. In the case of an analysis denoted by cms sus 13 011, the corresponding subdirectory will contain: -a Saf file cms sus 13 011.saf listing the names of all the implemented SRs; -a subdirectory Histograms with a Saf file histos.saf describing all the histograms that have been implemented; and a subdirectory Cutflows with a series of Saf files (named according to the definition of the SRs) containing the cut flow tables of all declared SRs.
The structure of the various Saf files is detailed in [17].

The 'MA5tune' of Delphes 3
Delphes [18] is a C++ framework dedicated to the simulation of a generic detector such as those used in collider experiments. Contrary to full detector simulation software, Delphes does not simulate the particle-matter interactions, but uses instead a parameterization of the detector response and reconstructs the main physics objects considered in the analyses. This simplified picture results in much faster simulations, while the accuracy level is maintained suitable for realistic phenomenological investigations. From the computing side, Delphes is a modular framework where developers can both add their own contributions and tune the default parameterization according to their needs. This modularity is based on a division of the simulation process into modules inspired by the TTask Root class, and the addition and removal of new elements are easily achievable through a Tcl configuration file. Similarly, the content of the output Root files can be configured at will. In order to properly recast ATLAS and CMS analyses, a tuning of the version 3 of Delphes has been performed. In the original version of Delphes, an isolation criterion is applied to both leptons and photons, and only particles satisfying this requirement are stored in the output files. We have designed a new Delphes module named CalculationIsolation that allows one to move the isolation requirements in the analysis selection. This module computes several variables useful for the implementation of isolation cuts. Defining cone sizes of ∆R = 0.2, 0.3, 0.4 and 0.5, the number of tracks with a transverse momentum larger than a given threshold, the scalar sum of the transverse momenta of these tracks and the scalar sum of the calorimetric transverse energy deposits lying in the cones are evaluated and saved. In addition, the default module of Delphes dedicated to the filtering of non-isolated lepton and photon candidates is switched off so that all candidates are kept in the output Root files. For consistency reasons, the Delphes module UniqueObject-Finder giving a unique identification to all reconstructed objects is bypassed. Isolation selection cuts can then be performed at the analysis level by means of the isolCones method of the RecLeptonFormat class of MadAnalysis 5, described in the previous subsection and in [17].
Adding the isolation information to the output format yields an increase of the size of the output files. A cleaning of all collections is therefore in order to reduce the file sizes. First, collections such as calorimeter towers and particle-flow objects are not stored. Next, the (heavy) collection of all particles that have been generated at the different level of the simulation chain (hard scattering process, parton showering and hadronization) is pruned. Only particles produced at the hard-scattering process level, as well as final-state leptons and b quarks present after parton showering, are stored. In addition, the relations between generated and reconstructed leptons have been retained, together with information on the origin (the mother particle) of each lepton. All these changes result in a reduction of the size of the produced Root files by about a factor of ten when compared to the files produced with the original configuration of Delphes.
This tailored version of Delphes 3, which we internally call Delphes-MA5tune to avoid confusion with the original version, can conveniently be installed locally from the MadAnalysis 5 interpreter by typing in the command install delphesMA5tune Even if Delphes 3 is already installed on a given system, one will need this modified 'MA5tune' version of the program in order to run the MadAnalysis 5 analyses that we are presenting in this paper. Note however that for the moment MadAnalysis 5 is not able to run with both Delphes and Delphes-MA5tune installed in parallel. This means that the user must take care that only the directory tools/delphesMA5tune (but not the directory tools/delphes) be available in his/her local installation of MadAnalysis 5.
In order to process an (hadronized) event sample with the 'MA5tune' of Delphes, it is sufficient to start Mad-Analysis 5 in the reconstructed mode, import the considered sample and type set main.fastsim.package = delphesMA5tune set main.fastsim.detector = cms submit where cms can be replaced by atlas according to the needs of the user. Default detector parameters are employed and can be modified by the user, following the guidelines displayed on the screen. The output Root file can then be retrieved from the automatically generated working directory.

Implemented Analyses and their Validation
To start the analysis database, we have implemented and Several more analyses are currently being implemented and validated. Below we give some details on these analyses, the level of documentation by the experimental collaboration, and the validation of our MadAnalysis 5 implementations. We begin with the CMS stop search in the single-lepton channel, which also served as our template analysis for developing the extensions of MadAnalysis 5 described briefly in Section 2 and in detail in [17]. The related recast code [21] contains extensive comments, which should allow the interested reader to easily use it as template for implementing a different analysis.
A list of all available analyses (which will certainly evolve quickly), instructions on how to use them, as well as more detailed validation notes can be found on the Mad-Analysis 5 wiki page [27]. The recast codes themselves are published via Inspire [28], in order to make them citable (Inspire assigns each submission a DOI [29]) and to ensure that changes can be traced reliably through a rigorous versioning system.
Before proceeding, some general comments are in order. Generally, we cannot reproduce cleaning cuts (for, e.g., cosmic rays and beam effects). Moreover, some basic jet quality criteria must be skipped as we do not have vertex information. This is, however, expected to have a small impact on signal events. In addition, event weights are typically applied by ATLAS and CMS to correct simulated events with respect to data. We take such event weights into account whenever they are available. Otherwise they are neglected and contribute to the overall uncertainty of the procedure.
Finally, while the selection criteria that define the various SRs are usually clear and well documented, information on the preselection cuts is often missing. In particular, trigger efficiencies, information about isolation, efficiencies for leptons, and the order in which preselection cuts are applied is crucial for reliably reproducing an analysis, but this information is often incomplete in the experimental publications. We hope that this will improve over time and the necessary information will be given systematically either in the physics paper or in a performance note, as also advertised in [19].

CMS-SUS-13-011: search for stops in the single-lepton final state
The CMS search for stops in the single lepton and missing energy, + E miss T , final state with full luminosity at √ s = 8 TeV [22] has been taken as a "template analysis" to develop a common language and framework for the analysis implementation. It also allowed us to test the new developments in MadAnalysis 5 which were necessary for carrying out this project.
The analysis targets two possible decay modes of the stop:t → tχ 0 1 andt → bχ + 1 . Since the stops are pairproduced, their decays give rise to two W -bosons in each event, one of which is assumed to decay leptonically, whilst the other one is assumed to decay hadronically. In the cutbased version of the analysis, 2 two sets of signal regions with different cuts, each dedicated to one of the two decay modes, are defined. These two sets are further divided into "low ∆M " and "high ∆M " categories, targeting small and large mass differences with the lightest neutralinoχ 0 1 , respectively. Finally, each of these four categories are further sub-divided using four different E miss T requirements. In total, 16 different, potentially overlapping SRs are defined.
Two cuts are based on rather complex and specific kinematic variables designed to reduce the dilepton tt background: a χ 2 resulting from the full reconstruction of the hadronic top and M W T 2 -a variant of the m T 2 observable. The implementation of the χ 2 quantity in our code was straightforward thanks to the C++ Root code provided on the CMS Twiki page. The M W T 2 variable is calculated with the standard MadAnalysis 5 method, see [17], according to the algorithm presented in [30].
Overall, this analysis is very well documented. Some important pieces of information were however missing, in particular the detailed trigger efficiencies and the identification-only efficiencies for electron and muons. These were provided by the CMS collaboration upon request and are now available on the analysis Twiki page [31] in the section "Additional Material to aid the Phenomenology Community with Reinterpretations of these Results". In addition, the b-tagging efficiency as a function of p T is not given in the paper, but was taken from [32]. Another technical difficulty came from the isolation criteria. Indeed, the CMS analysis considers the sum of transverse momenta of so-called 'Particle Flow' particles in a cone of given ∆R. This is difficult to reproduce in our case. Instead, we only use tracks in the inner detector for the isolation. From the two benchmark points for which cut flows are available (see Table 3) we found that a weighting factor of 0.885, applied on the events at the same time as the isolation, is sufficient to correct our track-only isolation. Therefore we incorporate this correction to our analysis code.
The validation of the reimplementation of the analysis can be done using the eleven benchmark points presented in the experimental paper: four for the "T2tt" simplified model (in which the stop always decays ast → tχ 0 1 ), and seven for the "T2bW" simplified model (in which the stop always decays ast → bχ + 1 ), with different assumptions on the various masses. The distributions of the kinematic variables used in the analysis are given in Fig. 2 of [22] after the preselection cuts, with at least one benchmark point for illustration. Also provided are the corresponding histograms after the M T > 120 GeV cut, as supplementary material on the CMS Twiki page [31]. We use this information, together with the final number of events in the individual SRs (i.e., after all selection cuts) for given benchmark points provided in Tables 4 and 6 of [22].
The validation material both before and after cuts defining the SRs is truly valuable information since one can separately check on the one hand the implementation of the kinematic variables and the preselection/cleaning cuts, and on the other hand the series of cuts defining the SRs. Furthermore, the large number of benchmark points allows us to check in detail the quality of the reimplementation in complementary regions of phase space.
The validation process was based on (partonic) event samples, in LHE format [33,34], provided by the CMS collaboration. The provision of such event files greatly reduced the uncertainties in the first stage of validation since it avoided possible differences in the configuration of the used Monte Carlo tools. In the case of this CMS analysis, the setup of MadGraph 5 [35,36]-the event gen- erator employed for generating the necessary hard scattering matrix elements-is crucial, in particular with respect to the merging of samples with different (partonlevel) jet multiplicities. The LHE files were passed through Pythia 6.4 [37] for parton showering and hadronization, then processed by our modified version of Delphes 3 (see Section 2.2) for the simulation of the detector effects. The number of events after cuts and histograms produced by MadAnalysis 5 were then normalized to the correct luminosity after including cross sections at the next-to-leading order and next-to-leading logarithmic (NLO+NLL) accuracy [38], as tabulated by the LHC SUSY Cross Section Working Group [39].
Some examples of histograms reproduced for the validation are shown in Fig. 1. The shapes of the distributions shown -as well as all other distributions that we obtained but do not show here -follow closely the ones from CMS, which indicates the correct implementation of the analysis and all the kinematic variables. (Note that discrepancies in bins where the number of events is relatively small, as seen on a logarithmic scale, suffers from larger statistical uncertainties and hence should not be over-interpreted.) The expected yields for several benchmark points in their relevant SRs are given in Tables 1 and 2. The agreement is good for all tested benchmark points.
Upon our request, the CMS SUSY group furthermore provided detailed cut-flow tables, which are now also available at [31]. These proved extremely useful because they allowed us to verify our implementation step-by-step in the analysis. A comparison of our results with the official CMS ones is given in Table 3. (Note that here no trigger efficiency or initial state radiation, ISR, reweighting is applied.) For both cases shown, CMS results are reproduced within about 20%. On the whole, we conclude that our implementation gives reasonably accurate results (to the level that can be expected from fast simulation) and declare it as validated. As mentioned, the MadAnalysis 5 code for this analysis, including extensive comments, is published as [21]. More detailed validation material, including extra histograms and validation of the limitsetting procedure (see Section 4), is available at [27]. Table 1. Final number of events fort → bχ ± 1 in three SRs of the analysis CMS-SUS-13-011. The benchmark points are given in the format (mt, mχ0 1 , x) in GeV, with x setting the chargino mass according to mχ+  gluino-pair production withg → qqχ 0 1 , denoted as T1qqqq topology in the following; -gluino-pair production withg → ttχ 0 1 , denoted as T1tttt; -gluino-pair production withg → qqχ 0 2 /χ ± 1 , followed byχ 0 2 , χ ± 1 → Z/Wχ 0 1 , generically denoted as T5VV; and squark-pair production withq → qχ 0 1 , denoted as T2qq, following the CMS simplified models naming scheme [40].
The analysis comprises 36 non-overlapping signal regions, each one defined as a rectangular box volume in the space spanned by the variables n j , H T , and / H T . Here n j is the jet multiplicity of the event, H T is the scalar sum of the jet transverse momenta, and / H T is the magnitude of the vector sum of the jets transverse momenta. Explicitly, The event selection was primarily determined from the documentation in [25]. This document describes six baseline selection criteria on the events, named MET Cleaning, No Lepton, n j > 2, H T > 500 GeV, / H T > 200 GeV, and Min ∆φ(jets, / H T ). We note that the MET Cleaning cut involves a detailed consideration of spurious signals in the CMS detector, which we cannot simulate with Delphes. Instead, we simply multiply our event count by the efficiency given by CMS. (We stress again that such efficiencies being publicly available is extremely helpful.) We validated the recast code against cut-flow tables and distributions of the kinematic variables provided by the CMS analysis team as per our request. The benchmark scenarios used are (mg, mχ0 1 ) = (1100, 125) GeV for the T1qqqq, T1tttt and T5VV topologies, and (mq, mχ0 1 ) = (700, 100) GeV for the T2qq topology, with production cross sections of 10.2 fb and 63.4 fb, respectively [38,39]. For the T5VV topology, one also needs theχ ± 1 andχ 0 2 masses; they are set to 612.5 GeV for the (mg, mχ0 1 ) = (1100, 125) GeV benchmark point.
The complete validation material from CMS is available in form of the PDF documents T1qqqq.pdf, T1tttt.pdf, T2qq.pdf and T5VV.pdf in the "Attachments" section on the analysis' wiki page [41]. These files correspond to the simplified SUSY models of the same names. For each of the four simplified-model scenarios, the CMS collaboration provided us with 10 5 events in LHE format along with cut-flow tables and distributions in the variables n j , H T , and / H T after each cut. As before, we passed these LHE files to Pythia 6.4 [37] for showering and hadronization and finally to Delphes for detector simulation. The merging of the partonic events that exhibit different jet multiplicities was performed according to the setup read from the LHE files provided by CMS.
A detail that required additional correspondence with the CMS analysis team were the pseudorapidity (η) cuts on the electrons and muons used for the lepton veto. We learned that the only requirement on these leptons is that |η| < 2.4, and they are allowed to reside in the overlap region between the electromagnetic calorimeter barrel and the endcap. We also checked the dependence on the jet energy scale (JES) correction, which is set in the CMS Delphes card, to have good agreement in the n j , H T and / H T distributions, and found JES=1.0 to be optimal.
The results of our cut-flow counts for the various simplified models are shown alongside the official counts in Tables 4 and 5. The results were obtained by normalizing with the cross section for each of the benchmark points and for an integrated luminosity of 19.3 fb −1 . Moreover, some distributions after the baseline cuts for the case of the T2qq topology are shown in Figs. 2-4. The distribu- tions are normalized to unity and overlaid on the official plots obtained from the collaboration. The agreement between the official and MadAnalysis 5 results is better than 10% throughout the baseline cut flows. The largest discrepancy arises from the lepton veto cut, which leads to a difference of up to about 5% in the cut flow. The shapes of the distributions qualitatively match very well, and the peaking bins are in accordance with the official results. (This also holds for the other distributions not shown here for space considerations). The MadAnalysis 5 implementation is available as [42], and a detailed validation note comparing the recast results to the CMS ones can be found at [27].

CMS-SUS-13-016: search for gluinos in events with opposite-sign leptons, b-tagged jets and large missing energy
The CMS analysis [26] searches for new physics in the multi-top final state. The primary target is gluino-pair production followed byg → ttχ 0 1 , i.e. the T1tttt topology in the CMS simplified-model nomenclature. The dataset used corresponds to a total integrated luminosity of L = 19.7 fb −1 at √ s = 8 TeV.
The analysis is not published yet but available as a Public Analysis Summary (PAS), which is overall welldocumented. The signal selection requires two isolated leptons of opposite sign, a large number of jets, at least three b-tagged jets, and large missing transverse energy (E miss T > 180 GeV). Moreover, |η| < 1 is required for the two leading jets. As there is only one SR, the exclusion is directly obtained from the upper limit on the number of events in the SR.
Let us now turn to our MadAnalysis 5 implementation and its validation. For the lepton isolation, we follow the same procedure as described above for CMS-SUS-13-011 (see Section 3.1). Likewise, the b-tagging efficiency as function of p T is taken from [32]. The most important piece of missing information in this PAS was a cut flow, which was however provided by the collaboration upon request and is now available on the analysis Twiki page [43].
Along with the cut flows, CMS provided LHE files corresponding to two benchmark points for the T1tttt simplified model, one with (mg, mχ0 1 ) = (1150, 275) GeV, and one with (mg, mχ0 1 ) = (1150, 525) GeV. The gluino-pair production cross section for these points is 6.7 fb with an uncertainty of 25% [38,39]. Unfortunately, these benchmark points differ by 25 GeV in the neutralino mass from the ones used in the PAS, which have mχ0 1 = 300 and 500 GeV, respectively. Although this is likely to induce some small differences in the event numbers and distributions, we chose to use the provided LHE files for validation because it avoids more important discrepancies due to differences in the configuration of the MC tools (e.g. the exact version and setup of MadGraph as well as the matching of parton-showers with hard scattering matrix elements and the merging of event samples exhibiting different jet multiplicities). The LHE files were passed through Pythia 6.4 [37] for parton showering and hadronization, with the correct merging parameters (given in the LHE files) taken into account. The detector simulation was then performed using the modified version of Delphes, with the b-tagging efficiency taken from [32] incorporated in the CMS card. The numbers of events after all cuts were normalized using the cross section information tabulated by the LHC SUSY Cross Section Working Group and for an integrated luminosity of 19.7 fb −1 . Our cut flow is compared to the official CMS numbers in Table 6. Figures 5-8 show histograms of the kinematic selection variables for the (mg, mχ0 1 ) = (1150, 275) GeV benchmark point. Our MadAnalysis 5 results are overlaid on the official results from Fig. 1 of [26], which we digi-  tized. The plots were made by applying all cuts except the one represented, and all the histograms are normalized to unity. We note that the shapes of the distributions are in close agreement with the official ones, with the exception of the n j distribution, which is slightly shifted towards higher jet multiplicity. Note also that the CMS histogram is cut off at n j = 10, while the distribution in fact extends to higher n j . These differences can be attributed to various factors, one of which is the jet energy scale and resolution, for which a 8% uncertainty is quoted in [26]. Our results shown here were obtained with the JES parameter set to 1.0 in the CMS Delphes card. A change of this parameter to 0.95 does not change the results significantly, while a change to 0.9 changes the final event count by 5 % after all cuts, and brings our n j distribution closer to the official one. Additionally there can be effects like pile-up or jetlepton separation, which we cannot simulate reliably in this fast-simulation framework. Therefore we regard these effects as systematic uncertainties in our implementation.   Our final numbers of events for the two benchmark points agree within about 20% with the official CMS numbers, see Table 6. This is well within the 17-39% systematic uncertainty given in [26]. Moreover, the individual cut efficiencies do not differ by more than 8% for any cut for either of the benchmark points. This leads us to conclude that this implementation is well validated. The MadAnalysis 5 code for this analysis is available as [44] and a detailed validation note is available on [27].

ATLAS-SUSY-2013-05: search for third-generation squarks in final states with zero leptons and two b-jets
In this ATLAS analysis [23], stops and sbottoms are searched for in final states with large missing transverse momentum and two jets identified as b-jets. The results are presented for an integrated luminosity of 20.1 fb −1 at √ s = 8 TeV.
Two possible sets of SUSY mass spectra were investigated in this analysis: sbottomb 1 pair production withb 1 → bχ 0 1 , and stopt 1 pair production witht 1 → bχ ± 1 , where the subsequent decay of theχ ± 1 is invisible due to a small mass splitting with theχ 0 1 . Two sets of SRs, denoted by SRA and SRB, are defined to provide sensitivity to the kinematic topologies associated with the two sets of mass spectra. SRA targets signal events with large mass splittings between the squark and the neutralino by selecting two hard b-jets, while SRB is designed to enhance the sensitivity when the squarkneutralino mass difference is small by selecting a hard jet coming from ISR and two softer b-jets.
For both SRs, events are selected by requiring a large amount of missing transverse energy, E miss T > 150 GeV, and any event containing an identified muon or electron is vetoed. For the SR selections, all jets with a pseudorapidity |η| < 2.8 are ordered according to their p T , and two out of the n selected jets are required to be b-tagged.
In the SRA, the first two leading jets must be b-tagged. The event is vetoed if any additional central jet (|η| < 2.8) with p T > 50 GeV is found. To reject the multijet background, large ∆φ min and E miss T /m eff are required. 3 To reduce the SM background, a cut on the invariant mass of the b-jet pair, m bb > 200 GeV, is applied. As a final selection, five different thresholds on the contransverse mass In SRB, the sensitivity to small squark-neutralino mass difference is increased by selecting events whose leading jet has a very large p T , which is likely to have been produced by ISR, recoiling against the squark-pair system. High thresholds on the leading jet and on the missing transverse momentum, which are required to be almost back-to-back in φ, are imposed. The leading jet is required to be non-b-tagged and two additional jets are required to be b-tagged. Just like for SRA, large values of ∆φ min and E miss T /m eff are required, thereby suppressing the multijet background. The selection for SRB is finally completed by demanding that the additional hadronic activity is bounded from above, H T,3 < 50 GeV. Here, H T,3 is defined as the scalar sum of the p T of the jets, without including the three leading jets.
The analysis is very well documented regarding physics, but for recasting purposes more information than provided in the physics paper [23] and on the analysis Twiki page [45] was needed. Indeed this made the validation of the recast code seriously difficult in the earlier stages of the project. Since then, fortunately, two cut-flow tables were made public. Moreover, the ATLAS SUSY group provided us with general SUSY Les Houches Accord (SLHA) [47] input files which we used to simulate the signal, as well as with the exact versions of the MC tools used to generate the SUSY samples, which were not given in [23]. When we simulated the signal samples with Madgraph 5 1.4.8 [35,36] and Pythia 6.4 [37], we introduced nonetheless additional sources of uncertainties since the complete MC configuration which was used for the signal simulation in ATLAS was not known. For example, the run card for MadEvent [48] would be precious information. Also, we are missing information on the trigger only and b-tagging efficiencies.
The comparison between the official cut flows and the ones obtained within MadAnalysis 5 are presented in Tables 7 and 8. The numbers were normalized to 20.1 fb −1 of data using the cross sections tabulated by the LHC SUSY Cross Section Working Group [38,39]. Overall the agreement is quite satisfactory, considering the expected accuracy for a fast simulation. We observe the largest discrepancy in Table 8 in the final number of events in SRB after the H T,3 cut for the benchmark point (mt 1 , mχ± 1 , mχ0 1 ) = (500, 420, 400) GeV. This discrepancy will also be exhibited in the histogram of the H T,3 distribution. In the analysis paper [23] there are four histograms of distributions that we can compare against. For SRA, there are histograms of m CT and of m bb . Two benchmark points are   As far as the SRA distributions are concerned, see Fig. 9, the agreement between our recast analysis and the official one is very good.
The situation is less satisfactory in the SRB case. As already pointed out regarding the cut flow of Table 8, the treatment of the H T,3 variable seems problematic; we indeed observe a large excess of events in the very first bin of its distribution with respect to the official result from AT-LAS. The very first bin corresponds to events where there are no additional jets (H T,3 = 0 GeV) except the ones which are required to select the event. The second bin is empty since jets are required to have p T > 20 GeV. This shows that, after detector simulation, we do no get enough jet activity. One possible explanation for this might be that we do not account for pile-up effects. According to private communication with ATLAS, the discrepancy is however too large to be accounted for by the pile-up only. Varying the JES by a fixed factor does not improve much the situation for the very first bin but can lead to improvement in the next bins of the H T,3 distribution. However this also has a impact on the E miss T distribution, which gets significantly modified. A possible solution might be a parameterization of the JES in terms of the p T of the jets for these signal regions, since for low p T it may vary significantly. In any case, in [23], the JES uncertainty was carefully estimated and amounts to only 3% in SRB. To investigate more deeply a more detailed cut flow about the "Jet selection" line in Table 8 would also be appreciable, since it directly impacts the H T,3 variable.
We conclude that for SRA the agreement is quite good. For SRB the efficiency of the H T,3 cut differs from the official analysis by about 20%, which is acceptable from a fast-simulation viewpoint. Moreover, according to [49] the sensitivity of SRB is difficult to reproduce while the analysis is generally dominated by SRA, as can also be seen in Fig. 3 of the auxiliary figures of [45]. Overall this leads us to conclude that this implementation is validated to the best that could be done. The recast code is available as [50], and a detailed validation note can be found on [27].

ATLAS-SUSY-2013-11: search for charginos, neutralinos and leptons in di-lepton final states
We consider the ATLAS search for the electroweak production of charginos, neutralinos and sleptons in final states with two leptons (electrons and muons) and missing transverse momentum based on 20.3 fb −1 of data at 8 TeV [24]. The event selection requires two signal leptons of opposite charge, with p T > 35 GeV and p T > 20 GeV. Two kind of final states are considered: same flavor (SF = e + e − or µ + µ − ) and different flavors (DF = e ± µ ∓ ).
Three types of signal regions are defined in this analysis. First, the m T 2 and W W signal regions require the invariant mass of the lepton pair to be outside the Z window, and jets are vetoed. The m T 2 signal regions (SR m T 2 ) target direct slepton-pair production and chargino-pair production followed by slepton-mediated decays. Each m T 2 signal region is defined by its threshold on the m T 2 ("stransverse mass") variable [51,52] that is used for reducing the tt and W t backgrounds: m T 2 > 90, > 120 and > 150 GeV, for SR-m 90 T 2 , SR-m 120 T 2 , and SR-m 150 T 2 , respectively. The implementation of this requirement is straightforward as the m T 2 variable is available as a standard method in Mad-Analysis 5.
Next, the W W a, W W b and W W c signal regions (referred to as SR-W W ) are designed to provide sensitivity toχ + 1χ − 1 production followed by leptonic W decays. Each of these three regions is optimized for a given kinematic configuration, using cuts on the invariant mass and/or transverse momentum of the slepton pair (m and p T, , respectively), possibly combined with cuts on m T 2 and on the "relative missing transverse momentum" E miss,rel T .
Here, E miss,rel T is defined as the missing transverse momentum E miss T multiplied by sin ∆φ ,j of the azimuthal angle between the direction of p miss T and that of the closest lepton or jet, ∆φ ,j , is below π/2. This modified E miss T aims at suppressing events where missing transverse momentum is likely to come from mis-measured jets and leptons.
Finally, the Zjets signal region (SR-Zjets) targetsχ ± 1χ 0 2 production, followed byχ ± 1 → W ±χ0 1 andχ 0 2 → Zχ 0 1 , with hadronic W and leptonic Z decays. Unlike in the other regions, jets are not vetoed; instead at least two central "light" jets (non-b-tagged with |η| < 2.4) are required. In addition to m being consistent with leptonic Z decays, requirements are made on E miss,rel T , p T, , on the invariant mass of the two leading jets (m jj ) and on the separation between the two leptons (∆R ) in order to suppress, in particular, the Z + jets background.
All signal regions separately consider SF and DF leptons, except SR-Zjets where only SF leptons are considered. In total, 20 potentially overlapping signal regions are defined (considering ee and µµ signal regions separately, as required for comparison with the official ATLAS cut flows). Detailed electron efficiencies as a function of p T and η are available in [53]; we used the electron efficiencies as a function of p T for |η| < 2.47, while muon efficiencies were taken to be 100% as a good approximation. The analysis is very well-documented and gives clearly the various preselection criteria and signal region cuts. Moreover, an effort was made in the definition of the tested new physics scenarios: a whole section of the experimental publication is dedicated to the description of the different SUSY scenarios. Furthermore, SLHA files were uploaded to HepData in May 2014 after discussion with the ATLAS SUSY conveners.
For validation, at least one cut-flow table is given for every signal region and type of scenario tested, which is very good practice. In addition, several histograms are given and can be used to validate the distribution of, in particular, E miss,rel T and m T 2 . Finally, regarding the interpretations in terms of simplified models, not only the information on the 95% confidence level (CL) upper bound on the visible cross section is given, but also the CL s value, which is useful for validation of the limit-setting procedure. The only difficulty came from the benchmark points for direct slepton production. Given the SLHA files provided on HepData, it was not clear whether the slepton masses given as m˜ in the cut-flow charts and histograms really correspond to the physical masses or to the slepton soft terms. The difference can be of several GeV, inducing some uncertainty in the kinematic distributions and in the production cross sections for these scenarios.
Event samples used for the validation were generated with Herwig++ 2.5.2 [54], using as input the SLHA files provided on HepData. For each of the nine benchmark points we considered, 10 5 events were generated. In the case of chargino-pair production, non-leptonic decays of the intermediate W -boson were filtered to increase statistics. Similarly, for chargino-neutralino production, nonleptonic decays of the intermediate Z-boson were filtered. The cross sections for the benchmark points, evaluated at the NLO+NLL accuracy [55][56][57], were taken from the HepData entry. Tables 9-11 give some examples of cut flows for different benchmark points and signal regions, comparing the results obtained with our MadAnalysis 5 implementation to the official ATLAS numbers. (The complete list of cut flows for all nine benchmark points is available at [27].) We systematically find the jet veto to be less efficient than it should be, but did not find any explanation for this effect. This was also noted in [12]. Still, reasonably good agreement is observed for the available benchmark points. Distributions of E miss,rel T , m and m T 2 in some signal regions are shown in Figs. 11 and 12. Good agreement is observed. Note that the fluctuations in the ATLAS results in the left panel of Fig. 12 may correspond to statistical fluctuations and/or uncertainties when digitizing the ATLAS histogram (the results are extracted from a logarithmic scale that spans over six orders of magnitude).
We conclude that our MadAnalysis 5 implementation of ATLAS-SUSY-2013-11 reproduces well the experimental results. Our C++ code for this analysis is published as [58]; complete validation materials including validation of the limit-setting procedure (see next section) can be found at [27].   [59]. 5 This code can also be installed on a user system by typing in, from the MadAnalysis 5 interpreter, the command install RecastingTools which results in the file exclusion CLs.py being present at the root of any working directory created in the expert mode of MadAnalysis 5. We refer to [17,27] for details on the creation of MadAnalysis 5 working directories. The exclusion CLs.py code takes as input the acceptance × efficiency information from the cut flow Saf files generated when executing an analysis implemented in MadAnalysis 5 (see Section 2.1). Moreover, an Xml file, named analysis name.info (where analysis name stands for a generic analysis name), needs to be provided by the user in the Build/SampleAnalyzer/User/Analyzer directory, specifying the luminosity <lumi>, the number of observed events <nobs>, the nominal number of expected SM background events <nb>, and its uncertainty at 68% CL <deltanb> in each of the regions, as given in the experimental publication. The syntax of this file is as follows: <analysis id="cms_sus_13_011"> <lumi>19.5</lumi> <!--in fb^-1 --> <region type="signal" id="SRname"> <nobs>227</nobs> <nb>251</nb> <deltanb>50</deltanb> </region> ... ...

</analysis>
The attribute type of the root tag <analysis> can be signal or control and is optional (the default value is signal). The id of each <region> tag has to match the exact name of the SR used in the analysis code. When results are given after combining several SRs (for example, for same-flavor leptons instead of ee and µµ separately), the relevant SRs should all be listed in the attribute id separated by semicolons (without extra space). Taking the example of the ATLAS analysis presented in Section 3.5, this would read <region id="MT2-90 ee;MT2-90 mumu"> The last piece of information essential for calculating exclusions is the signal cross section. It can be provided by the user in the Saf file mypoint.txt.saf (automatically generated when executing an analysis, see Section 2.1), where mypoint.txt, stored in the Input folder of the working directory, is the input file for running the analysis under consideration. Alternatively, the cross section can be given as argument when calling exclusion CLs.py. Concretely, the limit-setting code is called as ./exclusion_CLs.py analysis_name mypoint.txt \ [run_number] [cross section in pb] where the run number and cross section value are optional arguments. The run number x (default zero) identifies the output directory to use, as each execution of the analysis code yields the creation of a new output directory, analysis name x, for the x th execution of the analysis code (starting from 0).
The procedure of exclusion CLs.py starts by selecting the most sensitive SR (i.e., the one that yields the best expected exclusion, assuming that the number of observed events is equal to the nominal number of background events). This is a standard procedure at the LHC whenever the SRs defined in the analysis are overlapping; here we use it as the default for all analyses. Then the actual exclusion is calculated, and the confidence level with which the tested scenario is excluded using the CL s prescription [59] is printed on the screen together with the name of the most sensitive SR. The same information is also stored in the file analysis name x.out located in the working directory of the Output folder. Last but not least, if a negative number is given for the cross section, the code returns instead the nominal cross section that is excluded at 95% CL, computed using a root-finding algorithm.
The core of the calculation works as follows. First, the number of signal events (n s ) is obtained as the product of the luminosity, signal cross section and acceptance × efficiency for the SR of interest. This is used, together with the number of observed events (n obs ) and the nominal number of background events (n b ) and its uncertainty (∆n b ) to compute the exclusion. A large number of toy MC experiments (10 5 by default) are then generated from the Poisson distribution poiss(n obs |n expected ), corresponding to the distribution of the total number of events in the SR under the background-only hypothesis on the one hand (n expected = n b ), and under the signal + background hypothesis (n expected = n s + n b ) on the other hand. We assume that the uncertainty on the number of background events is modeled as gauss(n b , ∆n b ), and for each toy MC the number of background events n b is randomly generated from this normal distribution. Under the two different hypotheses, p-values are then calculated using the number of events actually observed at the LHC, and finally used to compute the CL s value.
We have tested the limit-setting code on the analyses presented in this paper and generally found good agreement with the official exclusions from ATLAS and CMS. As an illustrative example, Fig. 13 shows the 95% CL exclusion limit in the neutralino versus gluino mass plane reproduced with the MadAnalysis 5 implementation [44] of CMS-SUS-13-016.

Conclusions
We have presented a new scheme for developing and deploying implementations of LHC analyses based on fast simulation within the MadAnalysis 5 framework. This can serve to create a public analysis database, which may be used and developed further by the whole community. The codes for the five analyses [21,42,44,50,58] that we published together with this paper are intended as a starting point for this database and may conveniently be used as templates for other analyses.
We propose that the C++ codes of new implementations within this scheme be published via Inspire [28], as done here, best together with the physics paper they have been developed for. This way, each analysis implementation is assigned a Digital Object Identifier (DOI) [29], ensuring that it is uniquely identifiable, searchable and citable. In addition it is very useful if a detailed validation note is made available on the MadAnalysis 5 wiki page [27].
The ease with which an experimental analysis can be implemented and validated may serve as a useful check for the experimental collaborations for the quality of their documentation. Note, finally, that the platform we are proposing might also be used by the experimental collaborations to directly provide implementations of their analyses for fast simulation, thereby assuring the maximum usability of their results, as for example envisaged in level 1 of the CMS statement on "data preservation, re-use and open access policy" [60].
It is important for the legacy of the LHC that its experimental results can be used by the whole high-energy physics community. We hope that our project contributes to this aim.