Energy flow networks: deep sets for particle jets

A key question for machine learning approaches in particle physics is how to best represent and learn from collider events. As an event is intrinsically a variable-length unordered set of particles, we build upon recent machine learning efforts to learn directly from sets of features or “point clouds”. Adapting and specializing the “Deep Sets” framework to particle physics, we introduce Energy Flow Networks, which respect infrared and collinear safety by construction. We also develop Particle Flow Networks, which allow for general energy dependence and the inclusion of additional particle-level information such as charge and flavor. These networks feature a per-particle internal (latent) representation, and summing over all particles yields an overall event-level latent representation. We show how this latent space decomposition unifies existing event representations based on detector images and radiation moments. To demonstrate the power and simplicity of this set-based approach, we apply these networks to the collider task of discriminating quark jets from gluon jets, finding similar or improved performance compared to existing methods. We also show how the learned event representation can be directly visualized, providing insight into the inner workings of the model. These architectures lend themselves to efficiently processing and analyzing events for a wide variety of tasks at the Large Hadron Collider. Implementations and examples of our architectures are available online in our EnergyFlow package.


Introduction
Collisions at accelerators like the Large Hadron Collider (LHC) produce multitudes of particles.Particles are the fundamental objects of interest in collider physics and provide an interface between theoretical calculations and experimental measurements, often reconstructed experimentally via "particle flow" algorithms [1][2][3].Analyses of collider data rely on observables to distill these complicated multiparticle events and capture essential aspects of the underlying physics.Because each collision event consists of a variable-length list of particles with no intrinsic ordering, collider observables must be sensibly defined as functions of sets of particles.In this paper, we develop a novel architecture for processing and learning from collider events in their natural set-based representation.
Two key choices must be made when using machine learning for a collider task: how to represent the event and how to analyze that representation.These choices are often made together, with examples from collider physics including calorimeter images paired with convolutional neural networks (CNNs) [5][6][7][8][9][10][11][12][13], particle lists paired with recurrent/recursive neural networks (RNNs) [14][15][16][17][18], collections of ordered inputs paired with dense neural networks (DNNs) [19][20][21][22][23][24][25][26], and Energy Flow Polynomials (EFPs) paired with linear methods [28].One lesson that emerges from this body of work is that any two sufficiently general models, given access to complete enough information, achieve similar performance.In light of this, criteria such as understandability of the model and closeness to theoretical and experimental constructs are of central importance.
Given that events are fundamentally sets of particles, particle-level inputs such as those used in Refs.[14][15][16][17][18][19][20][21] are a desirable way of representing an event for use in a model.That said, RNNs and DNNs, the two architectures typically used with particle-level inputs, each fail to be fully satisfactory methods for processing events: DNNs because they require a fixed-size input and RNNs because they are explicitly dependent on the ordering of the inputs.Though ad hoc workarounds for these problems exist, such as zero padding for DNNs or ordering particles by their transverse momentum (p T ) or clustering history for RNNs, an ideal architecture would manifestly respect the permutation symmetry of the problem.Such an architecture would be able to handle variable-length inputs while being inherently symmetric with respect to the ordering of the input particles.
The machine learning community has recently developed (and continues to explore) technology which is ideally suited for representing sets of objects for a model [56][57][58][59][60][61][62][63][64][65].One con-text where this appears is learning from point clouds, sets of data points in space, which have the same permutation symmetry of collider events and are directly relevant for applications such as self-driving car technologies.The Deep Sets framework, recently developed in Ref. [60], demonstrates how permutation-invariant functions of variable-length inputs can be parametrized in a fully general way.The key observation is that the summation function, which is clearly symmetric with respect to the order of the arguments, is general enough to encapsulate all symmetric functions if one is allowed a large enough internal (latent) space.
In the context of a physics observable O that is a symmetric function of an arbitrary number of particles each with d features, the result from Ref. [60] can be stated as: Observable Decomposition.An observable O can be approximated arbitrarily well as: where Φ : R d → R is a per-particle mapping and F : R → R is a continuous function.
A schematic representation of Eq. (1.1) is shown in Fig. 1.Inherent in the decomposition of Eq. (1.1) is a latent space of dimension that serves to embed the particles such that an overall latent event representation is obtained when the sum is carried out.One should think of the d features for each particle as possibly being kinematic information, such as the particle's p T , rapidity y, and azimuthal angle φ, or other quantum numbers such as the particle's charge or flavor.Sec. 2 contains additional mathematical details regarding this decomposition.With a suitable modification of Eq. (1.1), we can restrict the decomposition to infraredand collinear-safe (IRC-safe) observables: IRC-Safe Observable Decomposition.An IRC-safe observable O can be approximated arbitrarily well as: where z i is the energy (or p T ) and pi the angular information of particle i.
The energy-weighting factors z i as well as the energy-independent pi in Eq. (1.2) ensure that the event representation in the latent space is IRC-safe.
In this paper, we show that many common observables are naturally encompassed by simple choices of Φ and F from Eqs. (1.1) and (1.2).Furthermore, we can parametrize Φ and F by neural network layers, capable of learning essentially any function, in order to explore more complicated observables.In keeping with the naming convention of Ref. [28] for methods involving IRC-safe observables, we term a network architecture implementing Eq. (1.2) an Energy Flow Network (EFN).By contrast, we refer to the more general case of an architecture that implements Eq. (1.1) as a Particle Flow Network (PFN).Our framework manifestly respects the variable length and permutation invariance of particle sets, achieves performance Figure 1: A visualization of the decomposition of an observable via Eq.(1.1).Each particle in the event is mapped by Φ to an internal (latent) particle representation, shown here as three abstract illustrations for a latent space of dimension three.The latent representation is then summed over all particles to arrive at a latent event representation, which is mapped by F to the value of the observable.For the IRC-safe case of Eq. (1.2), Φ takes in the angular information of the particle and the sum is weighted by the particle energies or transverse momenta.
competitive with existing techniques on key collider tasks, and provides a platform for visualizing the information learned by the model.Beyond this, we demonstrate how our framework unifies the existing event representations of calorimeter images and radiation moments, and we showcase the extraction of novel analytic observables from the trained model.One ever-present collider phenomenon that involves complicated multiparticle final states is the formation and observation of jets, sprays of color-neutral hadrons resulting from the fragmentation of high-energy quarks and gluons in quantum chromodynamics (QCD).Numerous individual observables have been proposed to study jets including the jet mass, constituent multiplicity, image activity [66], N -subjettiness [67,68], track-based observables [69,70], generalized angularities [71], (generalized) energy correlation functions [72,73], soft drop multiplicity [74,75], and many more (see Refs. [51,[76][77][78][79][80] for reviews).Machine learning methods have found tremendous applicability to jet classification tasks, greatly outperforming individual standard observables.Jet classification provides an ideal case study for the Deep Sets method in a collider setting since jets, like events, are fundamentally variably sized and invariant under reorderings of their constituents.
To showcase the efficacy of EFNs and PFNs, we apply them to the task of distinguishing light-quark jets from gluon jets [81][82][83][84], finding that they achieve excellent classification performance.In general, the PFN model outperforms the EFN model, indicating that IRCunsafe information is helpful for discriminating quark and gluon jets.Additionally, including particle identification information improves the classification performance of the PFN.It would be interesting to apply all of these methods in a fully-data driven way [85][86][87] to test these conclusions beyond the limited scope of parton shower generators.
One fascinating aspect of EFNs is that they enable a natural visualization of the learned latent space, providing insights as to what exactly the machine is learning.In particular, since the function Φ of an EFN typically takes the two-dimensional angular information of a particle as input, this two-dimensional space is easily visualized.In the context of quark/gluon discrimination, we observe that the EFN learns a latent representation that "pixelates" the rapidity-azimuth plane, dynamically sizing the pixels to be smaller near the core of the jet and larger farther out.We also find qualitative and quantitative evidence that the EFN has in a sense "understood" the collinear singularity structure of QCD.
The rest of this paper is organized as follows.Sec. 2 provides a detailed mathematical discussion of the observable decompositions and explores Eqs.(1.1) and (1.2) in the context of specific observables and event representations.Sec. 3 discusses the implementation details of our EFN and PFN architectures, with other models discussed in App. A. Sec. 4 contains the case study discriminating quark-and gluon-initiated jets and demonstrates our new techniques for visualizing and analyzing the learned information.Conclusions are presented in Sec. 5. A supplementary top jet tagging study is presented in App.B, and additional visualizations of the models are provided in App. C. The EFN and PFN architectures are available online as part of our EnergyFlow package [88] along with example code.

A general framework for observables
Events consist of variable numbers of particles with no intrinsic ordering, so observables are described mathematically as functions of sets of particles.Such a mathematical formulation allows for a direct exploration of the space of observables.For instance, Ref. [28] exploited IRC safety to construct a linear approximating basis of all IRC-safe observables.Here, we treat the entire space of observables (both with and without IRC safety), using their mathematical structure to arrive at a general decomposition relevant for theoretically categorizing observables as well as developing machine learning techniques.

Observables as functions of sets of particles
The key mathematical fact that we exploit, due to Ref. [60], is that a generic function of a set of objects can be decomposed to arbitrarily good approximation in a practical and intuitive way.We state this result explicitly below: Deep Sets Theorem [60].Let X ⊂ R d be compact, X ⊂ 2 X be the space of sets with bounded cardinality of elements in X, and Y ⊂ R be a bounded interval.Consider a continuous function f : X → Y that is invariant under permutations of its inputs, i.e. f (x 1 , . . ., x M ) = f (x π(1) , . . ., x π(M ) ) for all x i ∈ X and π ∈ S M .Then there exists a sufficiently large integer and continuous functions Φ : X → R , F : R → Y such that the following holds to an arbitrarily good approximation: (2.1) We only rely on the Deep Sets Theorem to justify the generality of Eq. (2.1), which can otherwise be regarded as an interesting, manifestly permutation-invariant parameterization.
The Deep Sets Theorem can be immediately applied to the collider physics context where observables are viewed as functions of sets of particles.We denote an event with M particles as {p i } M i=1 , where p i ∈ R d contains the relevant attributes of particle i (momentum, charge, flavor, etc.).Phrased in the collider physics language, it states that an observable O can be approximated arbitrarily well as: where Φ : R d → R is a per-particle mapping and F : R → Y is a continuous function.This provides a mathematical justification for the Observable Decomposition stated in Eq. (1.1).
The content of the Observable Decomposition is that any observable can be viewed as linearly summing over the particles in some internal space and then mapping the result to an output space.We refer to R as the latent space and each component of the per-particle mapping Φ(p i ) as a filter.The latent space could be, for example, the pixel values of a detector image or a moment decomposition of the radiation pattern.Summing Φ(p i ) over the particles induces a latent description of the entire event, which is mapped by the function F to the value of the observable.
Many existing collider observables ranging from e + e − event shapes to jet substructure observables naturally fit into the decomposition of Eq. (2.2).In particular, observables that are defined directly in terms of the particles themselves (i.e.not algorithmically) can often be exactly encompassed.Several examples of such observables are summarized in Table 1.The fact that an exact decomposition holds in these cases indicates that the Observable Decomposition indeed captures an essential aspect of particle-level collider observables.
Table 1: A variety of common collider observables decomposed into per-particle maps Φ and functions F according to Eq. (1.1).Here I track is an indicator function over charged tracks.
In the last column, the arguments of F are placeholders for the summed output of Φ.

Enforcing infrared and collinear safety
We can formulate the Observable Decomposition specifically for a class of observables of particular theoretical interest, namely IRC-safe observables [92][93][94][95].IRC safety corresponds to robustness of the observable under collinear splittings of a particle or additions of soft particles, which makes the observable tractable in perturbative quantum field theory as well as robust to experimental resolution effects.Remarkably, building IRC safety into the latent representation simply corresponds to energy-weighting the contributions of each particle and restricting Φ to only depend on the particle geometry pi .The energy-weighting z i and geometry pi for particle i depends on the collider context.At an e + e − collider, it is natural to take z i = E i and pi = p µ i /E i , where E i is the energy and p µ i the four-momentum.At a hadron collider, it is natural to take z i = p T,i and pi = (y i , φ i ), where p T,i is the transverse momentum, y i is the rapidity, and φ i the azimuthal angle. 2 In practice, we typically focus on dimensionless observables and use the appropriate normalized weights: Any IRC-safe observable O can be approximated arbitrarily well by the decomposition: where Φ : R d → R is a per-particle angular mapping and F : R → R is continuous.All observables of the form in Eq. (2.3) are manifestly IRC safe due to the energy-weighted linear sum structure, the dependence of Φ on purely geometric inputs pi , and the fact that continuous functions of IRC-safe observables are IRC safe. 3he fact that the energy-weighted decomposition in Eq. (2.3) suffices to approximate all IRC-safe observables is intuitive from the fact that a continuous function of a sufficiently high-resolution calorimeter image can be used to approximate an IRC-safe observable arbitrarily well [98][99][100].As discussed in Sec.2.3, an image of the calorimeter deposits is exactly encompassed by the energy-weighted observable decomposition.
Here, we provide a direct argument to arrive at Eq. (2.3), building off the Deep Sets Theorem and following similar logic as Ref. [28].Given the decomposition of an IRC-safe observable O into F and Φ via Eq.(2.2), the IRC safety of the observable O corresponds to the following statements: C safety : where Eq. (2.4) holds for all directions p0 that a soft particle could be emitted and Eq.(2.5) holds for all energy fractions λ ∈ [0, 1] of the collinear splitting.In Eq. (2.5), we have selected particle 1 to undergo the collinear splitting but the statement holds for any of the particles by permutation symmetry.The equations here only hold to a specified accuracy of approximation in the Observable Decomposition, which we leave implicit since it does not alter the structure of our argument.We now make the following suggestive redefinition of Φ to ensure that the latent representation of a particle vanishes if the particle has zero energy: Φ(z, p) → Φ(z, p) − Φ(0, p). (2.6) Infrared safety via Eq.(2.4) ensures that the value of the observable is unchanged under this redefinition, so without loss of generality we may take Φ to vanish on arbitrarily soft particles.
Making another convenient redefinition of Φ, we choose a λ ∈ [0, 1] and let: Collinear safety via Eq.(2.5) ensures that the value of the observable is unchanged under such a redefinition, which holds for any λ ∈ [0, 1].We now show that the freedom to redefine the mapping Φ using Eqs.(2.6) and (2.7) for an IRC-safe observable leads to the IRC-safe Observable Decomposition in Eq. ( 2.3).To see this, consider approximating Φ in the energy argument z via the Stone-Weierstrass theorem.Calling the angular coefficients of each term C n (p) yields: for some large but finite N .How large N must be depends on the specified precision that we have been leaving implicit.
Invoking the soft redefinition in Eq. (2.6), Φ may be taken to vanish on arbitrarily soft particles, which allows C 0 (p) to be set to zero without changing the value of the observable.Implementing the collinear redefinition in Eq. (2.7) after the expansion in energy, we obtain: (2.9) From this equation, we seek to argue that C n (p) for n ≥ 2 may be taken to vanish.For λ ∈ (0, 1), this redefinition decreases the higher-order coefficients C n (p) by a factor of λ n + (1 − λ) n < 1 without changing the corresponding observable.Iterated application of this fact allows the higher-order coefficients to be removed while keeping the term linear in the energy.Thus, to an arbitrarily good approximation, we can take Φ(z, p) = z C 1 (p) for some angular function C 1 (p), which we subsequently rename to Φ(p).
To summarize, the Deep Sets Theorem, combined with IRC safety, shows that the map Φ can be taken to be linear in energy without loss of generality.Collinear safety was critical in arguing that Φ could be taken to be affine linear in the energy and infrared safety was critical in arguing that the constant piece could be set to zero without loss of generality.This is exactly the result needed to justify the IRC-safe Observable Decomposition in Eq. ( 2.3), thereby completing the argument.Beyond potential applications for building IRC safety directly into models, such an observable decomposition for IRC-safe observables may be useful for shedding light on the structure of these important observables more broadly.

Encompassing image and moment representations
Beyond the single observables tabulated in Table 1, entire event representations can be encompassed in the Observable Decomposition framework as well.
One common event representation is to view events as images by treating the energy deposits in a calorimeter as pixel intensities [4][5][6][7][8][9].Since typical pixelizations for jet classification are 33 × 33 1000, the images are quite sparse, with an order of magnitude more pixels than particles.Treating the detector as a camera and events as images allows for modern image recognition technology to be applied to collider physics problems.These images are typically fed into a convolutional neural network, which is trained to learn a function of the images optimized for a specific task.
The image-based event representation of a jet as a collection of pixels fits naturally into the Observable Decomposition.The energy (or transverse momentum) deposited in each pixel is simply a sum over the energies z i of the particles hitting that pixel.Letting I j,k (y, φ) be an indicator function of pixel (j, k) in the rapidity-azimuth plane, we have that the intensity P j,k of pixel (j, k) is: (2.10) Thus, having Φ be an indicator function for the location of the pixel directly allows the latent representation of the IRC-safe Observable Decomposition to be a detector image.We  illustrate this in Fig. 2 for the rapidity-azimuth plane relevant for a hadron collider.Here, the filters are a collection of localized square bumps evenly spaced throughout the rapidityazimuth plane.
Another way to represent an event or jet is as a collection of moments of its radiation pattern.Moments (or tensors) have been considered for analyzing hadronic energy flow patterns both for e + e − and hadron colliders [101][102][103].A moment-based representation has yet to be directly exploited for machine learning applications in collider physics, though is closely related to the EFPs [28]. 4Here we restrict to the collimated case of jets, but a similar discussion holds at the event level.The moments I m,n of the radiation pattern in the rapidity-azimuth plane are: This can be manifestly decomposed according to the IRC-safe Observable Decomposition by simply making each filter Φ(y, φ) = y m φ n , as illustrated in Fig. 3. Here, the filters are a collection of non-localized functions which weight different parts of the event differently.More generally, we can visualize Φ(y, φ) for learned IRC-safe latent spaces, where the model itself learns its own event representation.In interpreting these visualizations, it is worth keeping in mind that localized filters like Fig. 2 correspond to an image-like representation, while global filters like Fig. 3 correspond to a moment-like representation.The flexibility of the IRC-safe Observable Decomposition allows for more complicated filters as well.As we will see in Sec.4.4, visualizing the latent space is extremely useful in understanding the behavior of EFNs.Moreover, similar (albeit higher-dimensional) visualizations can be performed in the general PFN case of Φ(p) and have been explored in the point cloud context [57].

Preprocessing
The goal of preprocessing inputs is to assist the model in its effort to solve an optimization problem.Typically, preprocessing steps are optional, but are applied in order to improve the numerical convergence of the model, given the practical limitations of finite dataset and model size, as well as the particular choice of parameter initialization.The preprocessing described in this section was found to be helpful, and sometimes necessary, for achieving a well-trained EFN or PFN model for the applications considered in Sec. 4. It is likely that for further applications of EFNs or PFNs, such as event-level studies, the appropriate preprocessing steps may change.
For the models we construct, kinematic information-transverse momentum p T , rapidity y, and azimuthal angle φ-are always given for each particle.We preprocess these features as follows: the transverse momenta are divided by the total scalar sum p T and the rapidities and azimuthal angles are centered based on the rapidity and azimuthal angle of the jet, using the E-scheme jet axis.In terms of the four-momentum of each particle, this preprocessing step can be cast into the following suggestive form: with pi = p µ i /p T,i , where the subscripts indicate the rapidity and azimuth of the jet fourvector.This notation makes clear that the per-particle preprocessing of Eq. (3.1) solely relies on the scalar sum p T , rapidity, and azimuth of the jet, which itself can be written in terms of an IRC-safe Observable Decomposition with Φ(p) = (1, p).Alternative jet centerings, such as those based on the p T -weighted centroid, also fit nicely into this framework. 5ptionally, the inputs may also include particle identification (ID) information.Though typically encoded using the Particle Data Group (PDG) particle numbering scheme [104], the large and irregular integer values employed therein are not ideal inputs to a model expecting inputs roughly in the numerical range [−1, 1].Therefore, a mapping from PDG IDs to small floats is performed for each particle (the details of which are provided below).While this approach, which only uses a single feature to encode the particle ID information, should be sufficient to input this information to the model, alternative approaches using multiple categorical features may be easier for the model to interpret, since particle ID is inherently discrete rather than continuous.For instance, using two additional features per particle, one feature could indicate the charge of the particle {−1, 0, +1} and the other one could indicate {h, γ, e, µ} (where h corresponds to a hadron, one of π, K, n, p), covering an experimentally realistic particle ID scheme.One-hot encoding of the particle ID is another option.
In order to explore how particle identification is helpful to a model, we use it in four different ways, each with a PFN architecture.We describe each of the different models and levels of information used throughout Sec. 4 below: • PFN-ID: PFN, adding in the full particle ID information.For the case study in Sec. 4, particles are indicated as being one of γ, π + , π − , K + , K − , K L , n, n, p, p, e − , e + , µ − , µ + , which are represented to the model as a single float value starting at 0 and increasing by 0.1 for each distinct type, respectively.6 • PFN-Ex: PFN, adding in experimentally realistic particle ID information.For the case study in Sec. 4, particles are indicated as being one of γ, h + , h − , h 0 , e − , e + , µ − , µ + , which are represented to the model analogously to the PFN-ID case.7 • PFN-Ch: PFN, adding in the electric charge of the particles as an additional feature.
• PFN: The particle flow network using only three-momentum information via Eq.(1.1).
• EFN: The energy flow network using only IRC-safe latent space information via Eq.(1.2).

Network architecture
So far, there has not yet been any machine learning in our effort to apply the decompositions in Eqs.(1.1) and (1.2) to collider data.The machine learning enters by choosing to approximate the functions Φ and F with neural networks. 8Neural networks are a natural choice to use because sufficiently large neural networks can approximate any well-behaved function.
To parametrize the functions Φ and F in a sufficiently general way, we use several dense neural network layers as universal approximators, as shown in Fig. 4. For Φ, we employ three dense layers with 100, 100, and nodes, respectively, where is the latent dimension that will be varied in powers of 2 up to 256.For F , we use three dense layers, each with 100 nodes.We confirmed that several network architectures with more or fewer layers and nodes achieved similar performance.Each dense layer uses the ReLU activation function [105] and He-uniform parameter initialization [106].A two-unit layer with a softmax activation function is used as the output layer of the classifier.See App.A for additional details regarding the implementations of the EFN, PFN, and other networks.The EnergyFlow Python package [88] contains implementations and examples of EFN and PFN architectures.
For the PFN family, the latent observable is O a = i Φ a (y i , φ i , z i , pid i ), with different levels of particle-ID (PID) information.The output of F is a softmaxed signal (S) versus background (B) discriminant.

Discriminating quark and gluon jets
To demonstrate the EFN architecture in a realistic setting, we implement and train an EFN and several PFN variants to discriminate light-quark from gluon initiated jets [81][82][83][84], a problem relevant for new physics searches as well as precision measurements.See App.B for a similar study on classifying top jets from QCD jets using samples based on Ref. [21].

Event generation
The samples used for this study were Z(→ ν ν) + g and Z(→ ν ν) + (u, d, s) events generated with Pythia 8.226 [107,108] at √ s = 14 TeV using the WeakBosonAndParton:qqbar2gmZg and WeakBosonAndParton:qg2gmZq processes, ignoring the photon contribution and requiring the Z to decay invisibly to neutrinos.Hadronization and multiple parton interactions (i.e.underlying event) were turned on and the default tunings and shower parameters were used.Final state non-neutrino particles were clustered into R = 0.4 anti-k T jets [109] using FastJet 3.3.0[110].Jets with p T ∈ [500, 550] GeV and |y| < 2.0 were kept.No detector simulation was performed. 9While labeling these jets using quark/gluon parton labels is manifestly unphysical, applications of these techniques at colliders could rely on an operational jet flavor  The best model is PFN-ID, which uses full particle-type information, followed closely by PFN-Ex, which uses experimentally realistic particle-type information.The PFN without any extra information performs roughly the same as the PFN-Ch, which uses charge information.The fact that the EFN is lowest on this plot indicates that there is discrimination power to be found in IRC-unsafe information.

Classification performance
A standard tool to analyze a classifier is the receiver operating characteristic (ROC) curve, obtained from the true positive ε s and false positive ε b rates as the decision threshold is varied.This may also be plotted as a Significance Improvement (SI) curve [82], namely ε s / √ ε b as a function of ε s .To condense the performance of a classifier into a single quantity, the area under the ROC curve (AUC) is commonly used, which is also the probability that the classifier correctly sorts randomly drawn signal (quark jet) and background (gluon jet) samples.An AUC of 0.5 corresponds to a random classifier and an AUC of 1.0 corresponds to a perfect classifier.We also report the background rejection at 50% signal efficiency (1/ε b at ε s = 50%) as an alternative performance metric.The PFN-ID with full particle ID yields the best performance of all models, followed by the PFN-Ex using experimentally realistic particle ID.The EFN and EFP models perform comparably in terms of maximum SI, indicating that the available IRC-safe information is being captured consistently by these very different architectures.
For each of the models, we sweep the latent dimension of the internal representation from 2 to 256 in powers of 2. As discussed in Sec.3.1, four PFN models were trained each with different particle-type information.Models are trained ten times each to give a sense of the variation and stability of the training.The resulting model performances as quantified by the AUC are shown in Fig. 5.As anticipated, the performance of each model increases as the latent dimension increases, with good performance achieved by = 16.The higher variance at low latent dimensions arises because some of the filters fail to train to non-zero values in those cases.The performance of the models appears to saturate by the larger latent dimensions, which justifies our use of = 256 as our benchmark latent dimension size for additional explorations.
In Fig. 6, we show the full ROC and SI curves of these models with latent dimension 256.The best model performance of all tested techniques and models was the PFN-ID with full particle ID, followed closely by the PFN-Ex with experimentally realistic particle ID.Figs. 5 and 6 show a well-defined hierarchy of model performances at all latent dimension sizes based on the information available to the model.The fact that the PFNs outperform the EFN indicates that IRC-unsafe information is helpful for this discrimination task, which is not surprising in light of the fact that the constituent multiplicity is known to be a good quark/gluon discriminant [82].

Comparison to other architectures
Besides comparing the EFN and PFN architectures to each other, we also compare the = 256 models to a variety of other classifiers, summarized in Table 2 and described in more detail in App. A.
Of particular interest are the RNN-ID and RNN models, which also take particles as input (with and without full particle ID, respectively), but process them in a way which is dependent on the order the particles were fed into the network (decreasing p T ordering was used).In Fig. 7, ROC and SI curves are shown for the RNN-ID and RNN architectures, as well as their natural counterparts, PFN-ID and PFN.We see that PFN-ID slightly outperforms RNN-ID whereas the PFN and RNN are comparable, though we emphasize that making broad conclusions based on this one result is difficult given the variety of different RNN architectures we could have chosen.Since PFNs are less expressive than RNNs, which can learn order-dependent functions, it is satisfying that both the PFN and RNN architectures achieve comparable classification performance with similar information.
The other machine learning architectures we compare to are a DNN trained on the Nsubjettiness basis [23,67,68], a CNN trained on jet images [4,5,8], and a linear classifier trained on the energy flow basis [28].Their performance, as given by their AUC and background rejection at 50% signal efficiency, is summarized in Table 3. Classification improves with the addition of IRC-unsafe information, as seen in the gain that the various PFN and RNN models have over the EFN and EFP models.There is also a boost in performance from providing the model with ever-more specific particle-type information.
Fig. 8 shows ROC and SI curves for all of these models, as well as some common jet substructure observables.The best model is PFN-ID, followed by RNN-ID, and then (as shown in this figure) linear EFPs, which, somewhat remarkably, is the best architecture by AUC and SI curve height that does not take particles as direct inputs.We note that for the CNN, one can in principle include particle ID information via additional channels, though training a 14-channel CNN is computationally challenging as each channel comes with O(1000) additional numbers, most of which are zero.Similar to RNNs and CNNs, the EFN and PFN architectures endeavor to be efficient by reducing the number of trainable parameters using weight sharing by applying the same Φ network to each particle.Adding particle-type information to the EFPs or the N -subjettiness DNNs might be possible through a suitable generalization of jet charge [69], though we know of no concrete implementation of this in the literature.The fact that PFNs naturally incorporate particle ID information is a important aspect of this architecture.

Visualizing the singularity structure of QCD
Beyond their excellent classification performance, the EFN and PFN architectures have the additional benefit that the learned function Φ can be directly explored.As       Sec. 2.3, this is particularly true of the EFNs, where Φ(p) is a two-dimensional function of the angular information and thus can be directly visualized in the rapidity-azimuth plane.We take the learned Φ : R 2 → R network from the best EFN model, as determined by the AUC, and evaluate it at many rapidity-azimuth points (y, φ) in the range y, φ ∈ [−R, R] to form a set of filters representing the learned latent space.We show several of these filters from the = 256 EFN models in Fig. 9.These can be directly compared with the corresponding filters for the detector image representation in Fig. 2 and for the radiation moment representation in Fig. 3. Like the image representation, we see that the learned filters are localized bumps in the rapidity-azimuth plane, and thus we say that the model appears to have learned a "pixelization" of the rapidity-azimuth plane. 10Unlike the image representation, the "pixels" learned by the model are smaller near the core of the jet and larger near the edge of the jet.
Beyond showing individual filters, it is informative to attempt to visualize an entire EFN latent space at once.This strategy is illustrated in Fig. 10, where each filter is contoured around its 50% value and the contours are overlaid.With the contours plotted simultaneously, the filters learned by the model can all be shown on the same figure, allowing for a direct visualization of the latent space representation.
In Fig. 11, we show this visualization for EFN models with latent dimension varying from 8 to 256 in powers of 2. Some of the filters are zero in the region of interest, perhaps as a result of dying ReLUs, so these are not shown.It is evident from the simultaneous overlay of the filters that their sizes are correlated with their distance from the origin, which is especially clear for the larger latent dimensions.As quark and gluon jets are (approximately) fractal objects with radiation singularly enhanced near the core of the jet as a result of the collinear singularity of QCD, the dynamically-sized pixelization learned by the EFN suggests that the model, in a sense, has understood this fact and adjusted itself accordingly.
To quantify the tendency of the filters to change size as they approach the center of the jet, we plot the area of each filter as a function of its distance from the origin.To define the area of a filter, A, we integrate the region of the rapidity-azimuth for which the filter is greater than or equal to half of its maximum.To capture a notion of distance from the origin, θ, we take the distance from the origin to the maximum value of the filter.We exclude filters that have centers outside of the jet radius.The resulting plots of the filters in this space are shown in Fig. 12 for the models with latent space dimension from 8 to 256 in powers of 2. There is a clear linear relationship between the (log) pixel size and the (log) distance to the jet core.In particular, the slope between ln A and ln θ is around 1.6 in the cases studied.
We can attempt to understand why the slopes in Fig. 12 are around 2 by considering a uniform pixelization in (ln R θ , ϕ), where θ is the distance from the jet axis and ϕ is the azimuthal angle around the jet axis (not to be confused with φ).As discussed in Ref. [13], this is the natural emission space of the jet.Translating an area element from this natural  ).The localized nature of the filters leads to our interpretation that the model has learned an image-like "pixelization" of the rapidity-azimuth plane, albeit one that is not square as in Fig. 2.
emission space to the rapidity-azimuth (y, φ) plane yields: Thus, a uniform pixelization in (ln R θ , ϕ) yields the following relationship between the area element (or pixel) size in the rapidity-azimuth plane and its distance from the origin: explaining the slopes around 2 observed empirically in Fig. 12.This emergent behavior suggests an interesting connection with recent work on machine learning directly in the emission space of the jet [13].Deviations from the scale-invariant expectation of 2 are largest near the core of the jet, where non-perturbative physics or axis-recoil effects [116] become important.The emission plane is visualized directly in App.C, where the pixelization is indeed seen to be highly uniform and regular in that space.

Extracting new observables from the model
Given that we are able to interpret aspects of the trained EFN in the latent space, we can attempt to go even further and distill the learned filters into closed-form observables.Obtaining novel jet substructure observables from machine learning methods has been approached previously by parameterizing an observable and learning the optimal parameters for a particular task [27].Here, we go in a different direction and look directly at the latent observables learned by an EFN model.
To make this tractable, we focus on the simple case of a two-dimensional latent space.A trained = 2 EFN has two learned filters, Φ 1 (y, φ) and Φ 2 (y, φ), and a learned function . The filters can be visualized in the rapidity-azimuth plane and the function F can be viewed in the (O 1 , O 2 ) phase space.By studying these visualizations and noting their emergent properties, we can construct observables that reproduce the behavior and predictive power of the trained EFN.
In Figs.13a and 13b, we show the learned filters Φ 1 and Φ 2 of a trained = 2 EFN.It is evident that the filters exhibit approximate radial symmetry, with one of the filters concentrated at the center of the jet and the other activated at larger angular distances.Thus, we can restrict our attention to functional forms which depend only on the rapidityazimuth distance θ from the origin.In particular, due to its built-in IRC-safety, the EFN model has learned filters that correspond to observables of the following approximate form: These are of the general form of IRC-safe angularities [71] with a generic radially-symmetric angular weighting function [82]. 11To quantify the filters further, in Fig. 13c we plot the value of the learned filters as a function of the radial distance, taking an envelope over several radial slices.The complementary central and wide-angle nature of the two filters are clearly evident.
By observing the properties of the curves in Fig. 13c, we fit two IRC-safe observables to the learned profiles of the following forms: with values of r 0 = 0.018, β = 200, and r 1 = 0.015 found to be suitable.The observables in Eq. (4.4) are then multiplied by overall factors of 0.60 and 0.18, respectively, to match the arbitrary normalization of the learned filters.While the precise values and shapes of the observable profiles changed from training to training, these general forms emerged for several of the best-performing models.
The observables A r 0 and B r 1 ,β in Eq. (4.4) are IRC-safe angularities with a linear energy dependence and interesting angular weighting functions.A r 0 probes the collinear radiation near the core of the jet at angles θ r 0 , and B r 1 ,β probes wide-angle radiation away from the core of the jet at angles θ > r 1 .The separate treatment of collinear and wide-angle particles is unlike the behavior of the traditional angularities, which have explicit contributions from both collinear and wide-angle regions of phase space.Though, as will be shown, each is individually a comparable quark/gluon jet classifier to the traditional angularities, the model is able to combine them in such a way as to achieve a significantly better performance.It would be interesting to perform a first-principles QCD study to understand in what sense the separation of collinear and wide-angle behavior is beneficial for discrimination.We now proceed to obtain a closed-form estimate of the learned function F (O 1 , O 2 ).In Fig. 14a, we populate the (O 1 , O 2 ) phase space by quark and gluon jets and color most of the populated space (with a mild threshold against outliers) according to the value of F at that phase space point.We also indicate the truth labels of one hundred quark and gluon jets to guide the eye.Based on the transition from red to blue, we can see that the model selects a curved slice through the populated phase space region to obtain its predictions. 12orrespondingly, we choose a closed-form estimate based on the simple (squared) Euclidean distance in phase space from a reference point (a 0 , b 0 ), namely: with a 0 = 0.445 and b 0 = 0.545, where, for ease of visualization, a sigmoid is also applied to 50 (C(A, B) − 0.460 2 ) to monotonically rescale the predictions between 0 and 1 and approxi- BDT(λ (1) , λ (2) , λ (1/2) ) Angularity λ (1)   Angularity λ (2)   Angularity λ (1/2) (a) BDT(λ (1) , λ (2) , λ (1/2) ) Angularity λ (1)   Angularity λ (2)   Angularity λ (1/2) (b) mate the value of the learned function.We visualize C(A, B) in Fig. 14b, finding a satisfying correspondence between the closed-form observables and the learned case.Finally, we study the classification performance of these closed-form observables, compared to the trained = 2 EFN model and several IRC-safe angularities.We consider three IRC-safe angularities λ (β) = i z i θ β i with β = 2, β = 1, and β = 1/2, which are the jet mass, jet width, and Les Houches angularity, respectively [84].ROC curves for the various observables are shown in Fig. 15.As single observables, the closed-form observables A r 0 and B r 1 ,β of Eq. (4.4) perform similarly to the individual angularities.When combined via C(A, B) of Eq. (4.5), they outperform all of the single angularities and approach the trained = 2 EFN performance.By contrast, when the three angularities are combined with a Boosted Decision Tree (BDT), there is not a significant improvement in the classification performance. 13The fact that A r 0 and B r 1 ,β can be combined to significantly increase the 13 The BDT classifier was implemented using scikit-learn [118] using 100 estimators, trained and tested on 100,000 quark and gluon jets with a 50-50 train/test split.We also tested combining Ar 0 and B r 1 ,β with a BDT, which resulted in similar performance to the closed-form and learned cases, verifying that the information is being adequately captured by C(A, B).
We conclude by discussing possible extensions and additional applications of these methods.Pileup or underlying event mitigation could be an interesting avenue to explore with this architecture, similar to Ref. [29] but also to PUPPI [121] in that a per-particle correction factor could be applied.Such an effort would need to make use of the equivariant designs of Ref. [60], which have a separate output for each input.One may also consider adding high-level information -such as the jet four-momentum, individual substructure observables, or clustering histories -directly to Φ or F in order to improve the network performance.Further, while we used jet-level examples for our case study, the problem of learning functions of sets applies at the event level more broadly.At the event level, new challenges arise that the simple preprocessing performed here for the jet case studies may not completely solve.An iterative deep sets technique, where multiple latent spaces are constructed, may help to capture the local/global intuition that collider physicists regularly employ to study the hierarchical relationships between events, jets, and particles.
• EFN, PFNs: Both the EFN and PFN architectures are contained as part of our EnergyFlow Python package [88], which contains examples demonstrating their training, evaluation, and use.Keras requires contiguous NumPy [125] arrays as input, so the events are padded with all-zero particles to ensure they have equal length.This zero-padding is a technical trick, not a conceptual limitation.It is distinct from the zero-padding necessary to use variable-length inputs in a DNN.This operation can be done on a per-batch level, but to avoid repeating this procedure at each epoch, we zero-pad all the events once at the beginning.The Keras Masking layer is used to ignore all-zero particles during the training and evaluation of the network.The Keras TimeDistributed layer is used to apply the function Φ to each particle, which in the relevant language is a "time step".To carry out the summation in the latent space, we use the Keras Dot layer for the EFN architecture, to product and sum the transverse momentum fractions with the latent observables, and the Lambda layer for the PFN architecture, to sum over the particles.It may be interesting to consider alternative "pooling" functions to summation, such as max-pooling or average-pooling as in Refs.[57,61].
• RNN-ID, RNN: Recurrent Neural Networks provide an alternative way to process variable-length inputs, albeit one that is explicitly not agnostic to the order of the inputs.We choose to order the particles by their transverse momenta and train two variants: one with only kinematic information (RNN) and one with kinematic as well as full particle ID information (RNN-ID).The former should be compared with the PFN model and the latter to the PFN-ID model.We did not consider alternative particle orderings for the RNN, though other investigations have found that performance is robust to choices in particle ordering [15,42].Our RNN architecture consists of an LSTM layer with 64 nodes (the performance was insensitive to changing this value to 128 or 256) followed by three fully-connected layers each with 100 nodes and a fully-connected output layer with 2 nodes.Due to the Masking layer employed, the significantly faster CuDNNLSTM layer was not used and the batch size was taken to be 2000 to help speed up training.An architecture using two or three SimpleRNN layers was also tried but the performance was not as good as with an LSTM layer.The long training time of the RNN models was prohibitive in exploring additional hyperparameters.
• EFPs: The energy flow basis [28] is a linear basis for the space of IRC-safe jet substructure observables, allowing linear methods to be applied to a set of Energy Flow Polynomials with good performance.The EnergyFlow 0.8.2 Python package [88] was used to compute all EFPs with degree d ≤ 7 and complexity χ ≤ 3, using the normalized default hadronic measure with β = 0.5.The same EFPs with β = 1 were also tested and found to perform slightly worse.These 996 EFPs, including the trivial constant EFP, were used to train a Fisher's Linear Discriminant model from the scikit-learn package [118].
Each non-EFN/PFN model was trained twice and the model with the higher AUC on the test set is reported (except when ε g at ε q = 50% values are reported, in which case the model with the value of this statistic is reported).In Table 4, we report the typical training times for each of the models.

B Tagging top jets
In this appendix, we train EFN and PFN models to discriminate top jets from the QCD jet background to provide an additional demonstration of the excellent performance of these models.Since top jets do not have any singularities associated with radiation about their center, training EFN models on tops provides an important cross check about our conclusions in Sec.4.4 for how the model is learning to pixelize the rapidity-azimuth plane.
The top and QCD jets used in this study are based on the dataset used in Ref. [21], which were provided to us by those authors.The jets are Pythia-generated, anti-k T , R = 0.8 jets at √ s = 14 TeV with no underlying event or pileup using a Delphes [127] detector fast-simulation of the ATLAS detector.Jets are kept if they have transverse momentum p T ∈ [550, 650] GeV and pseudorapidity |η| < 2 and if they can be matched to a top parton to within a rapidity-azimuth distance of ∆R = 0.8.The jets are required to be fully merged, with the three quarks from the top decay contained within the jet radius.See Ref. [21] for additional information about the samples and their generation details, as well as information about other top-tagging algorithms.There are 1.2 million training events, with 400k validation events and 400k test events.The jet samples do not contain particle-level flavor or charge information, and thus we can only train EFN and PFN models that make use of kinematic information alone.A linear EFP model is also trained for comparison.Given the different topology typical of top jets compared to quark or gluon jets, we implement some additional preprocessing steps designed to help the model train more efficiently.As with the quark and gluon jets of Sec. 4, we center all of the jets in the rapidity-azimuth plane based on the four-momentum of the jet, and we normalize the transverse momenta of the particles to sum to one.Models were trained with just this minimal preprocessing, as well as with additional rotation and reflection (r.r.) operations.For the EFN-r.r. and PFNr.r.models, rotations were performed to align the leading eigenvector of the two-dimensional moment of the radiation pattern along the vertical axis, and reflections were performed about the horizontal and vertical axes to place the highest-p T quadrant in a consistent quadrant.
The results of training EFN and PFN models for top tagging are shown in Fig. 16 with the latent dimension of the models varying from 8 to 256 in powers of 2. ROC and SI curves of the trained models are shown in Fig. 17 and also compared to the linear EFP model.Performance competitive with the results in Ref. [21] is achieved, particularly for the PFN models.The preprocessing step of rotating and reflecting can be seen to notably improve both the EFN and PFN classification performance.In order to facilitate future comparisons with other models trained on these samples, in Table 5 we report the AUC and gluon background rejection factor at both 0.3 and 0.5 quark efficiency.For the EFNs, we can also visualize the learned filters using the technique of Sec.4.4.The resulting filter visualizations for latent dimension 256 are shown in Fig. 18.The learned filters have some tendency to be smaller near the center and larger near the periphery, but not  nearly as much as in the quark/gluon discrimination case in Fig. 10.This is expected because given the typical three-prong topology of a boosted top jet, the jet axis does not have any distinguished radiation associated with it, unlike for a QCD jet where the jet axis tends to lie along a core of radiation.To quantify this effect, we look at the size of the filters as a function of their distance from the origin, shown in Fig. 19.The relationship in the top tagging case is much weaker than the linear relationship present in the quark/gluon discrimination study, with significantly worse linear fits.

C Additional visualizations
For the quark/gluon EFN model in Sec. 4, the observed relationship between the size and location of the filters qualitatively (in Fig. 11) and quantitatively (in Fig. 12) suggested that the model learned a uniform pixelization in the ln R θ , ϕ emission plane.(The top tagging EFN model in App.B did not exhibit as clear of a relationship in either Fig. 18 or Fig. 19.)To directly visualize the EFN filters in the appropriate space, we implement a change of variables from Cartesian (y, φ) coordinates to polar (θ, ϕ) coordinates.The visualizations in Fig. 20 use the same contouring and overlaying technique of Fig. 10 to demonstrate the roughly uniform pixelation in the emission plane for the quark/gluon EFN models with latent dimensions of 16, 64, and 256.In this emission plane, the learned filters can be seen to be much more uniform in size and location compared to the rapidity-azimuth plane.The uniformity as a function of ϕ indicates that the set of filters approximately has rotational symmetry.
Finally, we show a larger version of the quark/gluon = 256 EFN model from Fig. 11 in Fig. 21, suitable for framing.

Figure 2 :
Figure 2: The calorimeter image representation decomposed into a collection of Φ(y, φ) filters according to the IRC-safe Observable Decomposition, shown here for the illustrative case of a 4 × 4 image.The energy deposits in each pixel can be decomposed via Eq.(1.2) into an indicator function Φ(y, φ) determining whether a particle in position (y, φ) hits the pixel.

Figure 4 :
Figure 4: The dense networks used to parametrize (a) the per-particle mapping Φ and (b) the function F , shown for the case of a latent space of dimension = 8.For the EFN, the latent observable is O a= i z i Φ a (y i , φ i ).For the PFN family, the latent observable is O a = i Φ a (y i , φ i , z i , pid i ), with different levels of particle-ID (PID) information.The output of F is a softmaxed signal (S) versus background (B) discriminant.

Figure 5 :
Figure 5: The AUC performance of the EFN and PFN models as a function of the latent dimension of the model, which is varied from 2 to 256 in powers of 2. The spread in values is due to training the model ten times with different initializations.The performance generally increases with larger latent dimensions, with saturation observed by latent dimension 256.The best model is PFN-ID, which uses full particle-type information, followed closely by PFN-Ex, which uses experimentally realistic particle-type information.The PFN without any extra information performs roughly the same as the PFN-Ch, which uses charge information.The fact that the EFN is lowest on this plot indicates that there is discrimination power to be found in IRC-unsafe information.

Figure 6 :
Figure 6: The (a) ROC and (b) SI curves of the median (selected by AUC) EFN and PFN models with latent dimensions of 256.The linear EFP model is shown for comparison.The PFN-ID with full particle ID yields the best performance of all models, followed by the PFN-Ex using experimentally realistic particle ID.The EFN and EFP models perform comparably in terms of maximum SI, indicating that the available IRC-safe information is being captured consistently by these very different architectures.

Figure 7 :
Figure 7: The (a) ROC and (b) SI curve classification performances of PFN and RNN models both with and without full particle ID information.From the SI curve, it appears that the PFN-ID model is doing better than the RNN-ID model, whereas the PFN and RNN models perform roughly equally.

Figure 8 :
Figure 8: The (a) ROC and (b) SI curve classification performances of several different models and observables.The PFN-ID and RNN-ID curves are shown in order to facilitate comparison with Figs. 6 and 7.The PFN-ID architecture compares well to existing techniques, often notably outperforming them.

Figure 9 :
Figure 9: Visualizations of 16 of the 256 filters learned by the = 256 EFN, with the filters sorted by their activated area.The domain is the rapidity-azimuth plane from −R to R in both y and φ, since the jets have been preprocessed by centering them at (0,0).The localized nature of the filters leads to our interpretation that the model has learned an image-like "pixelization" of the rapidity-azimuth plane, albeit one that is not square as in Fig.2.

Figure 10 :
Figure 10: An illustration of our simultaneous visualization procedure for example EFN filters.Contours of each filter are shown from 45% to 55% of its maximum value.These contours are then overlaid on the same figure with different colors.The resulting contour plot shows the dynamical pixelization of the plane determined by the model.

Figure 11 :
Figure 11: The learned EFN pixelization of the rapidity-azimuth plane around the jet center with latent dimensions between 8 and 256 in powers of 2. The learned filters are dynamically sized, with smaller filters probing the core of the jet and larger filters in the periphery.A large version of the last panel is shown in Fig. 21.

Figure 14 :
Figure 14: (a) The EFN model output F (O 1 , O 2 ) in the plane spanned by the learned latent space observables O 1 and O 2 .(b) The closed-form function C(A, B) in the plane of the analytic observables A r 0 and B r 1 ,β .One hundred quark jets (light blue circles) and gluon jets (dark red squares) are indicated to highlight the separation power.The distribution of the closed-form observables and output value faithfully reproduce those of the trained EFN.

Figure 15 :
Figure 15: The (a) ROC and (b) SI curves for the two closed-form observables A r 0 and B r 1 ,β as well as their combination C(A, B), compared to the trained = 2 EFN model.Three angularities are also shown for comparison, along with their corresponding performance when combined with a BDT.While the two learned observables perform similarly to the angularities on an individual basis, they are evidently more informative than the angularities when combined.The output of the trained EFN model and the closed-form estimate achieve similar performance.

AUCFigure 16 :Table 5 :
Figure16: The AUC performance of the EFN/PFN top tagging models as a function of the latent dimension, which is varied from 8 to 256 in powers of 2. Including rotation and reflection in the rapidity-azimuth plane as preprocessing steps (solid curves) improves the model performance significantly compared to only centering the jets (dashed curves).The spread in values is due to retraining the models ten times.

Figure 17 :
Figure 17: The (a) inverse ROC and (b) SI curves for each of the EFN/PFN top tagging models with latent dimension 256, compared to the EFP linear model.The curve with the median AUC out of ten independent trainings is shown.As expected, the PFN yields better performance than the EFN, with the additional rotation/reflection preprocessing steps providing a further boost in performance.

Figure 20 :
Figure 20: Visualization of the learned filters of a quark/gluon EFN in the (ln R θ , ϕ) plane with latent dimensions of (top) 16, (middle) 64, and (bottom) 256.The pixelization learned by the network is much more regular and uniform in this space compared to the rapidityazimuth plane in Fig. 11.

Figure 21 :
Figure 21: A larger scale visualization of the filters for the quark/gluon = 256 EFN model, shown originally in Fig. 11.

Table 2 :
The (top) PFN/EFN architectures, (middle) other machine learning models, and (bottom) jet substructure observables used in comparisons of quark/gluon discrimination performance, along with their corresponding symbols and short descriptions.A detailed discussion of model implementation and observable computation is given in App. A.

Table 3 :
The classification performances, quantified by the AUC and background rejection at 50% signal efficiency, for each of the models and observables in Table2.Reported uncertainties on the EFN and PFN family of models are half of the interquartile range over ten trainings.Performance uniformly improves with the inclusion of more particle-type information.