Autoencoders for Real-Time SUEP Detection

Confining dark sectors with pseudo-conformal dynamics can produce Soft Unclustered Energy Patterns (SUEP), at the Large Hadron Collider: the production of dark quarks in proton-proton collisions leading to a dark shower and the high-multiplicity production of dark hadrons. The final experimental signature is spherically-symmetric energy deposits by an anomalously large number of soft Standard Model particles with a transverse energy of O(100) MeV. Assuming Yukawa-like couplings of the scalar portal state, the dominant production mode is gluon fusion, and the dominant background comes from multi-jet QCD events. We have developed a deep learning-based Anomaly Detection technique to reject QCD jets and identify any anomalous signature, including SUEP, in real-time in the High-Level Trigger system of the Compact Muon Solenoid experiment at the Large Hadron Collider. A deep convolutional neural autoencoder network has been trained using QCD events by taking transverse energy deposits in the inner tracker, electromagnetic calorimeter, and hadron calorimeter sub-detectors as 3-channel image data. Due to the sparse nature of the data, only ~0.5% of the total ~300 k image pixels have non-zero values. To tackle this challenge, a non-standard loss function, the inverse of the so-called Dice Loss, is exploited. The trained autoencoder with learned spatial features of QCD jets can detect 40% of the SUEP events, with a QCD event mistagging rate as low as 2%. The model inference time has been measured using the Intel CoreTM i5-9600KF processor and found to be ~20 ms, which perfectly satisfies the High-Level Trigger system's latency of O(100) ms. Given the virtue of the unsupervised learning of the autoencoders, the trained model can be applied to any new physics model that predicts an experimental signature anomalous to QCD jets.


Introduction
Hidden valleys, referring to hidden sectors with a confining gauge group, can produce dark quarks in proton-proton collisions at the Large Hadron Collider (LHC), leading to a dark shower and the production of a large number of dark hadrons (ϕ D ), analogous to QCD jets [1][2][3].Depending on the details of the theory, the dark showers can follow large-angle emission, and dark hadrons do not arrange in narrow QCD-like jets.The decay of dark hadrons results in dark photons (Z D ), which further decay to low-energy Standard Model particles with transverse energy (E T ) of O( 102 ) MeV, with the final experimental signature being high-multiplicity spherically-symmetric Soft Unclustered Energy Patterns, or SUEPs.We focus on a well-motivated scenario where SUEP is produced in exotic Higgs (H) boson decays via gluon-gluon fusion and all dark hadrons decay promptly and exclusively to pions and leptons-an experimental nightmare scenario because of an overwhelming multi-jet QCD background.
We have developed an Anomaly Detection (AD) technique [4,5] exploiting unsupervised deep learning for real-time SUEP detection in the High-Level Trigger (HLT) system [6] of the Compact Muon Solenoid (CMS) experiment [7] at the LHC 1 .Most AD techniques developed for High-Energy Physics applications rely on the online or offline reconstruction of collision events, which could be ineffective for reconstructing new physics signatures.We utilise raw quantities reconstructed in the HLT system, i.e., energy deposits in different sub-detectors (input for the online particle reconstruction).We have trained a deep convolutional neural autoencoder network (ConvAE) using QCD events, which is designed to detect events significantly different from the training data without prior knowledge of the signal characteristics.The data are three-channel images: two-dimensional (in the η-ϕ plane, where η is the pseudorapidity) E T deposits in the inner tracker, electromagnetic calorimeter (ECAL), and hadron calorimeter (HCAL) sub-detectors of CMS.Even though our work targets SUEP event tagging, the trained model could identify any other non-QCD-like signature, such as semi-visible jets [8] and emerging jets [9], spanning a large part of the landscape of the hidden valley.We plan on covering those new physics signals in our next paper.This paper is structured as follows: Section 2 contains all the information about the QCD and SUEP event generation, simulation, and reconstruction.The event preselection is highlighted in the same section.The details about the ConvAE architecture and training are provided in Section 3. The trained ConvAE is applied to the QCD background and SUEP signal considering several Higgs boson mediator mass scenarios, and the performance results are given in Section 4. Section 5 concludes the paper.

Generation, simulation, and reconstruction
Both multi-jet QCD and SUEP events are generated using PYTHIA 8.244 [10].Detector effects are simulated and events are reconstructed with DELPHES 3.5 [11] using the CMS detector configuration.Proton-proton collisions at the LHC Run-3 centre-of-mass energy of 13.6 TeV are considered, with approximately 50 collisions per bunch crossing (pileup).A pileup reduction is applied to the reconstructed tracks: the E T deposits in the inner tracker by the pileup tracks are removed.No pileup reduction is applied for calorimeters, as per the HLT system's configuration during Run 3 [6].
The model parameters for the QCD event generation are identical to those in [12].For SUEP event generation, we used the SUEP_Generator plugin [13], and the following parameter settings have been used: • mediator masses (m H ) = 125 GeV, 400 GeV, 700 GeV, and 1000 GeV Please note that we denote SUEP with a mediator mass of m H GeV as SUEP(m H GeV) in the rest of the paper. 1 The CMS trigger system is a two-tiered system.The first level (L1) triggers, implemented on custom electronics, process information from the calorimeters and muon system and select 100 kHz events of physics interest from 40 MHz collision events within a time interval of 4 µs.The second level (HLT) triggers run an optimised version of full-event reconstruction software on a farm of processors and further reduce the event rate to 1 kHz with a latency of O(10 2 ) ms.

Event preselection
A preselection criterion is applied to both QCD and SUEP events based on the detector geometry and the typical characteristics of QCD events.The |η| of E T deposits is required to be less than 2.5, as defined by the inner tracker coverage.The scalar sum of E T of reconstructed electrons, muons, photons, and jets (denoted as H T ) in an event is required to be >500 GeV, which is a typical H T requirement at the HLT after L1 trigger selection during Run 3. The event preselection efficiency is found to be ∼85% for QCD and ∼2.1%, ∼10.4%, ∼18.4%, and ∼22.7% for SUEP(125 GeV), SUEP(400 GeV), SUEP(700 GeV), and SUEP(1000 GeV), respectively.
The plots showing the E T deposits for a QCD event and a SUEP(1000 GeV) event, which satisfy the preselection criterion, in the inner tracker, ECAL, and HCAL are given in Figures 1 and 2, respectively.The ECAL is 25 times more granular (0.0174×0.0174 rad 2 in η-ϕ) than the HCAL (0.087×0.087 rad 2 in η-ϕ).Hence, each HCAL pixel is divided into 25 equal pixels to match the ECAL granularity.The final shape of the images is (288, 360, 3).For this particular QCD event, we can appreciate the typical signature of two jets in the η-ϕ plane.Also, we can acknowledge the sparse nature of the data-only ∼15 k pixels out of a total of 3,11,040 image pixels have non-zero values.This observation is valid in general for all QCD events.On the other hand, for the SUEP(1000 GeV) event, there is a clear spherically-symmetric signature of a large number of low-E T particles in ϕ.We normalised the E T values such that they are of O(1), which facilitates more stable and faster ConvAE training.

Autoencoders
Our ultimate target is to reject QCD jets and identify anomalous signatures.The unsupervised learning nature of autoencoders (AEs) perfectly serves the purpose [14].In its simplest form, an AE is a neural network (NN) that maps the input (ground truth) to a latent compressed representation (encoding), the so-called bottleneck, and then back to the input dimensionality (decoding), providing the output (reconstruction) as an approximation of the input.The quality of the AE training is measured in terms of a distance metric, the so-called reconstruction loss; e.g., for images, the loss function can be defined as the pixel-wise summed mean-squared difference between input and output.Once the AE is trained using background events with the objective of minimising the reconstruction loss, the model is expected to poorly reconstruct anomalous signatures and result in large reconstruction loss values.A threshold on the reconstruction loss can be imposed to tag anomalies.

Architecture and training
The ConvAE architecture is detailed in Table 1.For encoding, there are five two-dimensional Convolutional NN (CNN) [15] layers with a kernel_size of (3, 3) and strides of (3, 3), (2, 2), or (2,3).Up to the bottleneck, the input shape of (288, 360, 3) compresses to (6,5,8).The decoder part, which is exactly the mirror image of the encoder part, is composed of five two-dimensional Transposed CNN [16] layers with the same kernel_size and exactly opposite strides.The padding is set to "same".After each (Transposed) CNN layer, a BatchNormalization layer is introduced, which facilitates a more stable and faster training by bringing the mean of the output close to zero and the output standard deviation close to unity.The Parametric Rectified Linear Unit (PReLU) and Rectified Linear Unit (ReLU) [17] activation functions are used for all the hidden layers and the output layer, respectively.This model architecture results in a total number of trainable hyperparameters equal to 3,574,065.The model was implemented in Keras/TensorFlow [18,19], and it was trained for 100 epochs (with early stopping enabled) with a batch_size of 128.The learning rate was set to 10 −2 , and the optimizer found to yield the best performance was Adam [20].The training was performed using the NVIDIA® Tesla® V100 PCIe 32 GB Graphics Processing Unit [21], and ∼25 minutes per epoch were consumed while model training.The biggest challenge we faced in accomplishing the task was data sparsity, as mentioned in Section 2.2.The model had a huge tendency to learn the zeros rather than the non-zero values.The standard loss functions, such as Mean Squared Error [22] and Cross-Entropy [23], did not work well.We exploited the inverse of the so-called Dice Loss [24] as the loss function, as given by equation 1.The x i and x ′ i denote the i th -pixel values for the input and output, respectively.The N is the total number of pixels.The denominator term contains the pixel-wise product of input and output, which forces the output to have the same non-zero pixels as the input to minimise the reconstruction loss.On the other hand, the numerator term and minus 1 ensure that the loss function gets its minimum at zero if the output becomes equal to the input.
We used two independent sets of 50 k QCD events for the model training and validation.During the model training, no overfitting was observed, and the loss minimisation was smooth and saturated after 57 epochs.

Autoencoder reconstruction
The trained ConvAE is tested using 4 k QCD and SUEP events each, which satisfy the event preselection criterion detailed in Section 2.2. Figure 3 represents a comparison of true E T (left) and reconstructed E T (right)-sum over all the pixels-between QCD and SUEP(1000 GeV).It can be noted that the model reconstructs the E T for QCD events quite well but struggles for SUEP(1000 GeV) E T reconstruction.75.6% for SUEP(125 GeV) to 86.9% for SUEP(1000 GeV).For a target of 40% signal-selection efficiency, the QCD mistagging rate ranges from 2% for SUEP(1000 GeV) to 11.9% for SUEP(125 GeV).The model inference time has been quantified using a CPU device, an Intel® Core TM i5-9600KF processor [25], and found to be ∼20 ms, which is very well within the processing-time limits of the HLT system (O(10 2 ) ms).

Conclusions
A deep convolutional neural autoencoder network, trained using background events by taking their signatures in different sub-detectors of an experiment at the Large Hadron Collider as n-channel image data, is highly effective as an Anomaly Detection (AD) technique.The training of the autoencoder using unlabeled background events makes this technique highly model-agnostic.We have developed an autoencoder for AD with the Compact Muon Solenoid experiment, which can easily be adapted for other Large Hadron Collider experiments.We have exploited the raw reconstructed

Figure 1 :
Figure 1: E T deposits for a QCD event in the inner tracker (left), ECAL (middle), and HCAL (right).

Figure 3 :
Figure 3: Comparisons of true E T (left) and reconstructed E T (right) (sum over all the pixels) between QCD and SUEP(1000 GeV).The combined true E T for a QCD event in the inner tracker, ECAL, and HCAL is given in the left plot of Figure4(same as Figure1but combined now).The reconstructed E T for the same event is shown in the right plot of Figure4.The model is able to reconstruct the non-zero input pixels quite well.Similar plots for a SUEP(1000 GeV) event are given in Figure5.In this case, the model reconstructed only the large E T clusters (in the top-left quadrant) but failed to reconstruct the whole spherically-symmetric signature of a large number of particles in ϕ.

Figure 4 :Figure 7 :
Figure 4: Combined true E T (left) and reconstructed E T (right) for a QCD event in the inner tracker, ECAL, and HCAL.