Abstract
Attention-based transformer models have become increasingly prevalent in collider analysis, offering enhanced performance for tasks such as jet tagging. However, they are computationally intensive and require substantial data for training. In this paper, we introduce a new jet classification network using an MLP mixer, where two subsequent MLP operations serve to transform particle and feature tokens over the jet constituents. The transformed particles are combined with subjet information using multi-head cross-attention so that the network is invariant under the permutation of the jet constituents. We utilize two clustering algorithms to identify subjets: the standard sequential recombination algorithms with fixed radius parameters and a new IRC-safe, density-based algorithm of dynamic radii based on HDBSCAN. The proposed network demonstrates comparable classification performance to state-of-the-art models while boosting computational efficiency drastically. Finally, we evaluate the network performance using various interpretable methods, including centred kernel alignment and attention maps, to highlight network efficacy in collider analysis tasks.
Article PDF
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
References
J.M. Butterworth, A.R. Davison, M. Rubin and G.P. Salam, Jet substructure as a new Higgs search channel at the LHC, Phys. Rev. Lett. 100 (2008) 242001 [arXiv:0802.2470] [INSPIRE].
D.E. Kaplan, K. Rehermann, M.D. Schwartz and B. Tweedie, Top Tagging: A Method for Identifying Boosted Hadronically Decaying Top Quarks, Phys. Rev. Lett. 101 (2008) 142001 [arXiv:0806.0848] [INSPIRE].
Y. Cui, Z. Han and M.D. Schwartz, W-jet Tagging: Optimizing the Identification of Boosted Hadronically-Decaying W Bosons, Phys. Rev. D 83 (2011) 074023 [arXiv:1012.2077] [INSPIRE].
T. Plehn, M. Spannowsky and M. Takeuchi, How to Improve Top Tagging, Phys. Rev. D 85 (2012) 034029 [arXiv:1111.5034] [INSPIRE].
D.E. Soper and M. Spannowsky, Finding top quarks with shower deconstruction, Phys. Rev. D 87 (2013) 054012 [arXiv:1211.3140] [INSPIRE].
C. Anders et al., Benchmarking an even better top tagger algorithm, Phys. Rev. D 89 (2014) 074047 [arXiv:1312.1504] [INSPIRE].
G. Kasieczka et al., Resonance Searches with an Updated Top Tagger, JHEP 06 (2015) 203 [arXiv:1503.05921] [INSPIRE].
J. Thaler and K. Van Tilburg, Identifying Boosted Objects with N-subjettiness, JHEP 03 (2011) 015 [arXiv:1011.2268] [INSPIRE].
J. Thaler and K. Van Tilburg, Maximizing Boosted Top Identification by Minimizing N-subjettiness, JHEP 02 (2012) 093 [arXiv:1108.2701] [INSPIRE].
A.J. Larkoski, G.P. Salam and J. Thaler, Energy Correlation Functions for Jet Substructure, JHEP 06 (2013) 108 [arXiv:1305.0007] [INSPIRE].
I. Moult, L. Necib and J. Thaler, New Angles on Energy Correlation Functions, JHEP 12 (2016) 153 [arXiv:1609.07483] [INSPIRE].
A.J. Larkoski, S. Marzani, G. Soyez and J. Thaler, Soft Drop, JHEP 05 (2014) 146 [arXiv:1402.2657] [INSPIRE].
A. Abdesselam et al., Boosted Objects: A Probe of Beyond the Standard Model Physics, Eur. Phys. J. C 71 (2011) 1661 [arXiv:1012.5412] [INSPIRE].
A. Altheimer et al., Jet Substructure at the Tevatron and LHC: New results, new tools, new benchmarks, J. Phys. G 39 (2012) 063001 [arXiv:1201.0008] [INSPIRE].
A. Altheimer et al., Boosted Objects and Jet Substructure at the LHC. Report of BOOST2012, held at IFIC Valencia, 23rd-27th of July 2012, Eur. Phys. J. C 74 (2014) 2792 [arXiv:1311.2708] [INSPIRE].
J. Cogan, M. Kagan, E. Strauss and A. Schwarztman, Jet-Images: Computer Vision Inspired Techniques for Jet Tagging, JHEP 02 (2015) 118 [arXiv:1407.5675] [INSPIRE].
L.G. Almeida et al., Playing Tag with ANN: Boosted Top Identification with Pattern Recognition, JHEP 07 (2015) 086 [arXiv:1501.05968] [INSPIRE].
L. de Oliveira et al., Jet-images — deep learning edition, JHEP 07 (2016) 069 [arXiv:1511.05190] [INSPIRE].
P. Baldi et al., Jet Substructure Classification in High-Energy Physics with Deep Neural Networks, Phys. Rev. D 93 (2016) 094034 [arXiv:1603.09349] [INSPIRE].
J. Barnard, E.N. Dawe, M.J. Dolan and N. Rajcic, Parton Shower Uncertainties in Jet Substructure Analyses with Deep Neural Networks, Phys. Rev. D 95 (2017) 014018 [arXiv:1609.00607] [INSPIRE].
P.T. Komiske, E.M. Metodiev and M.D. Schwartz, Deep learning in color: towards automated quark/gluon jet discrimination, JHEP 01 (2017) 110 [arXiv:1612.01551] [INSPIRE].
G. Kasieczka, T. Plehn, M. Russell and T. Schell, Deep-learning Top Taggers or The End of QCD?, JHEP 05 (2017) 006 [arXiv:1701.08784] [INSPIRE].
S. Macaluso and D. Shih, Pulling Out All the Tops with Computer Vision and Deep Learning, JHEP 10 (2018) 121 [arXiv:1803.00107] [INSPIRE].
S. Choi, S.J. Lee and M. Perelstein, Infrared Safety of a Neural-Net Top Tagging Algorithm, JHEP 02 (2019) 132 [arXiv:1806.01263] [INSPIRE].
J. Shlomi, P. Battaglia and J.-R. Vlimant, Graph Neural Networks in Particle Physics, arXiv:2007.13681 [https://doi.org/10.1088/2632-2153/abbf9a] [INSPIRE].
F. Mokhtar, R. Kansal and J. Duarte, Do graph neural networks learn traditional jet substructure?, in the proceedings of the 36th Conference on Neural Information Processing Systems: Workshop on Machine Learning and the Physical Sciences, New Orleans, U.S.A., December 03–03 (2022) [arXiv:2211.09912] [INSPIRE].
F. Ma, F. Liu and W. Li, Jet tagging algorithm of graph network with Haar pooling message passing, Phys. Rev. D 108 (2023) 072007 [arXiv:2210.13869] [INSPIRE].
S. Gong et al., An efficient Lorentz equivariant graph neural network for jet tagging, JHEP 07 (2022) 030 [arXiv:2201.08187] [INSPIRE].
F.A. Dreyer and H. Qu, Jet tagging in the Lund plane with graph networks, JHEP 03 (2021) 052 [arXiv:2012.08526] [INSPIRE].
D. Guest et al., Jet Flavor Classification in High-Energy Physics with Deep Neural Networks, Phys. Rev. D 94 (2016) 112002 [arXiv:1607.08633] [INSPIRE].
J. Pearkes, W. Fedorko, A. Lister and C. Gay, Jet Constituents for Deep Neural Network Based Top Quark Tagging, arXiv:1704.02124 [INSPIRE].
S. Egan et al., Long Short-Term Memory (LSTM) networks with jet constituents for boosted top tagging at the LHC, arXiv:1711.09059 [INSPIRE].
K. Fraser and M.D. Schwartz, Jet Charge and Machine Learning, JHEP 10 (2018) 093 [arXiv:1803.08066] [INSPIRE].
A. Butter, G. Kasieczka, T. Plehn and M. Russell, Deep-learned Top Tagging with a Lorentz Layer, SciPost Phys. 5 (2018) 028 [arXiv:1707.08966] [INSPIRE].
G. Kasieczka, N. Kiefer, T. Plehn and J.M. Thompson, Quark-Gluon Tagging: Machine Learning vs Detector, SciPost Phys. 6 (2019) 069 [arXiv:1812.09223] [INSPIRE].
P.T. Komiske, E.M. Metodiev and J. Thaler, Energy Flow Networks: Deep Sets for Particle Jets, JHEP 01 (2019) 121 [arXiv:1810.05165] [INSPIRE].
H. Qu and L. Gouskos, ParticleNet: Jet Tagging via Particle Clouds, Phys. Rev. D 101 (2020) 056019 [arXiv:1902.08570] [INSPIRE].
H. Qu, C. Li and S. Qian, Particle Transformer for Jet Tagging, arXiv:2202.03772 [INSPIRE].
T. Finke, M. Krämer, A. Mück and J. Tönshoff, Learning the language of QCD jets with transformers, JHEP 06 (2023) 184 [arXiv:2303.07364] [INSPIRE].
A. Shmakov et al., SPANet: Generalized permutationless set assignment for particle physics using symmetry preserving attention, SciPost Phys. 12 (2022) 178 [arXiv:2106.03898] [INSPIRE].
A. Hammad, S. Moretti and M. Nojiri, Multi-scale cross-attention transformer encoder for event classification, JHEP 03 (2024) 144 [arXiv:2401.00452] [INSPIRE].
M. He and D. Wang, Quark/gluon discrimination and top tagging with dual attention transformer, Eur. Phys. J. C 83 (2023) 1116 [arXiv:2307.04723] [INSPIRE].
M. Zaheer et al., Deep sets, Adv. Neural Inf. Process. Syst. 30 (2017) 3391.
E.A. Moreno et al., JEDI-net: a jet identification algorithm based on interaction networks, Eur. Phys. J. C 80 (2020) 58 [arXiv:1908.05318] [INSPIRE].
V. Mikuni and F. Canelli, Point cloud transformers applied to collider physics, Mach. Learn. Sci. Tech. 2 (2021) 035027 [arXiv:2102.05073] [INSPIRE].
A. Bogatskiy, T. Hoffman, D.W. Miller and J.T. Offermann, PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics, arXiv:2211.00454 [INSPIRE].
A. Vaswani et al., Attention is all you need, Adv. Neural Inf. Process. Syst. 30 (2017) 5998.
I. Tolstikhin et al., MLP-Mixer: An all-MLP Architecture for Vision, Adv. Neural Inf. Process. Syst. 34 (2021) 24261 [arXiv:2105.01601].
E. Buhmann et al., Full phase space resonant anomaly detection, Phys. Rev. D 109 (2024) 055015 [arXiv:2310.06897] [INSPIRE].
Y.L. Dokshitzer, G.D. Leder, S. Moretti and B.R. Webber, Better jet clustering algorithms, JHEP 08 (1997) 001 [hep-ph/9707323] [INSPIRE].
M. Cacciari, G.P. Salam and G. Soyez, The anti-kt jet clustering algorithm, JHEP 04 (2008) 063 [arXiv:0802.1189] [INSPIRE].
R.J.G.B. Campello, D. Moulavi and J. Sander, Density-based clustering based on hierarchical density estimates, in Advances in Knowledge Discovery and Data Mining, J. Pei et al. eds., Springer Berlin Heidelberg (2013), p. 160–172 [https://doi.org/10.1007/978-3-642-37456-2_14].
C.R. Qi, H. Su, K. Mo and L.J. Guibas, PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, arXiv:1612.00593 [INSPIRE].
J.R. Walsh and S. Zuberi, Factorization Constraints on Jet Substructure, arXiv:1110.5333 [INSPIRE].
C. Bierlich et al., A comprehensive guide to the physics and usage of PYTHIA 8.3, SciPost Phys. Codeb. 2022 (2022) 8 [arXiv:2203.11601] [INSPIRE].
DELPHES 3 collaboration, DELPHES 3, A modular framework for fast simulation of a generic collider experiment, JHEP 02 (2014) 057 [arXiv:1307.6346] [INSPIRE].
C. Li et al., Does Lorentz-symmetric design boost network performance in jet physics?, Phys. Rev. D 109 (2024) 056003 [arXiv:2208.07814] [INSPIRE].
M. Cacciari, G.P. Salam and G. Soyez, FastJet User Manual, Eur. Phys. J. C 72 (2012) 1896 [arXiv:1111.6097] [INSPIRE].
G. Cerro et al., Spectral clustering for jet physics, JHEP 02 (2022) 165 [arXiv:2104.01972] [INSPIRE].
B. Mukhopadhyaya, T. Samui and R.K. Singh, Dynamic radius jet clustering algorithm, JHEP 04 (2023) 019 [arXiv:2301.13074] [INSPIRE].
P.T. Komiske, E.M. Metodiev, B. Nachman and M.D. Schwartz, Pileup Mitigation with Machine Learning (PUMML), JHEP 12 (2017) 051 [arXiv:1707.08600] [INSPIRE].
D. Hendrycks and K. Gimpel, Gaussian Error Linear Units (GELUs), arXiv:1606.08415 [INSPIRE].
S. Kornblith, M. Norouzi, H. Lee and G. Hinton, Similarity of Neural Network Representations Revisited, in the proceedings of the International conference on machine learning (PMLR), Long Beach, U.S.A, June 9–15, (2019), p. 3519–3529 [arXiv:1905.00414].
W. Esmail, A. Hammad and S. Moretti, Sharpening the A → Z(∗)h signature of the Type-II 2HDM at the LHC through advanced Machine Learning, JHEP 11 (2023) 020 [arXiv:2305.13781] [INSPIRE].
D. Greenfeld and U. Shalit, Robust Learning with the Hilbert-Schmidt Independence Criterion, in International Conference on Machine Learning (PMLR), Vienna, Austria, July 12–18 PMLR (2020), p. 3759–3768 [arXiv:1910.00270].
H. Chefer, S. Gur and L. Wolf, Transformer Interpretability Beyond Attention Visualization, in the proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Nashville, U.S.A., June 20–25 (2021) p. 782–791 [arXiv:2012.09838].
Acknowledgments
This work is funded by grant number 22H05113, “Foundation of Machine Learning Physics”, Grant in Aid for Transformative Research Areas and 22K03626, Grant-in-Aid for Scientific Research (C).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
ArXiv ePrint: 2404.14677
Rights and permissions
Open Access . This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Hammad, A., Nojiri, M.M. Streamlined jet tagging network assisted by jet prong structure. J. High Energ. Phys. 2024, 176 (2024). https://doi.org/10.1007/JHEP06(2024)176
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/JHEP06(2024)176