Abstract
We deploy an advanced Machine Learning (ML) environment, leveraging a multi-scale cross-attention encoder for event classification, towards the identification of the gg → H → hh → \( b\overline{b}b\overline{b} \) process at the High Luminosity Large Hadron Collider (HL-LHC), where h is the discovered Standard Model (SM)-like Higgs boson and H a heavier version of it (with mH > 2mh). In the ensuing boosted Higgs regime, the final state consists of two fat jets. Our multi-modal network can extract information from the jet substructure and the kinematics of the final state particles through self-attention transformer layers. The diverse learned information is subsequently integrated to improve classification performance using an additional transformer encoder with cross-attention heads. We showcase that our approach surpasses current alternative methods used to establish sensitivity to this process in performance, whether solely based on kinematic analysis or combining this with mainstream ML approaches. Then, we employ various interpretive methods to evaluate the network results, including attention map analysis and visual representation of Gradient-weighted Class Activation Mapping (Grad-CAM). Finally, we note that the proposed network is generic and can be applied to analyse any process carrying information at different scales. Our code is publicly available for generic use (https://github.com/AHamamd150/Multi-Scale-Transformer-Encoder).
Article PDF
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
References
A. Chakraborty, S.H. Lim and M.M. Nojiri, Interpretable deep learning for two-prong jet classification with jet spectra, JHEP 07 (2019) 135 [arXiv:1904.02092] [INSPIRE].
Y.-L. Chung, S.-C. Hsu and B. Nachman, Disentangling boosted Higgs boson production modes with machine learning, 2021 JINST 16 P07002 [arXiv:2009.05930] [INSPIRE].
J. Guo, J. Li, T. Li and R. Zhang, Boosted Higgs boson jet reconstruction via a graph neural network, Phys. Rev. D 103 (2021) 116025 [arXiv:2010.05464] [INSPIRE].
C.K. Khosa and S. Marzani, Higgs boson tagging with the Lund jet plane, Phys. Rev. D 104 (2021) 055043 [arXiv:2105.03989] [INSPIRE].
K. Datta, A. Larkoski and B. Nachman, Automating the construction of jet observables with machine learning, Phys. Rev. D 100 (2019) 095016 [arXiv:1902.07180] [INSPIRE].
D. Cogollo et al., Deep learning analysis of the inverse seesaw in a 3-3-1 model at the LHC, Phys. Lett. B 811 (2020) 135931 [arXiv:2008.03409] [INSPIRE].
M. Grossi, J. Novak, B. Kersevan and D. Rebuzzi, Comparing traditional and deep-learning techniques of kinematic reconstruction for polarization discrimination in vector boson scattering, Eur. Phys. J. C 80 (2020) 1144 [arXiv:2008.05316] [INSPIRE].
V.S. Ngairangbam, A. Bhardwaj, P. Konar and A.K. Nayak, Invisible Higgs search through vector boson fusion: a deep learning approach, Eur. Phys. J. C 80 (2020) 1055 [arXiv:2008.05434] [INSPIRE].
C. Englert et al., Sensing Higgs boson cascade decays through memory, Phys. Rev. D 102 (2020) 095027 [arXiv:2008.08611] [INSPIRE].
F.F. Freitas, J. Gonçalves, A.P. Morais and R. Pasechnik, Phenomenology of vector-like leptons with deep learning at the Large Hadron Collider, JHEP 01 (2021) 076 [arXiv:2010.01307] [INSPIRE].
A. Stakia et al., Advances in multi-variate analysis methods for new physics searches at the Large Hadron Collider, Rev. Phys. 7 (2021) 100063 [arXiv:2105.07530] [INSPIRE].
F. Jorge et al., Top squark signal significance enhancement by different machine learning algorithms, Int. J. Mod. Phys. A 37 (2022) 2250197 [arXiv:2106.06813] [INSPIRE].
J. Ren et al., Detecting an axion-like particle with machine learning at the LHC, JHEP 11 (2021) 138 [arXiv:2106.07018] [INSPIRE].
D. Alvestad et al., Beyond cuts in small signal scenarios: enhanced sneutrino detectability using machine learning, Eur. Phys. J. C 83 (2023) 379 [arXiv:2108.03125] [INSPIRE].
S. Jung, Z. Liu, L.-T. Wang and K.-P. Xie, Probing Higgs boson exotic decays at the LHC with machine learning, Phys. Rev. D 105 (2022) 035008 [arXiv:2109.03294] [INSPIRE].
M. Drees, M. Shi and Z. Zhang, Machine learning optimized search for the Z′ from \( \textrm{U}{(1)}_{L_{\mu }-{L}_{\tau }} \) at the LHC, arXiv:2109.07674 [INSPIRE].
A.S. Cornell et al., Boosted decision trees in the era of new physics: a smuon analysis case study, JHEP 04 (2022) 015 [arXiv:2109.11815] [INSPIRE].
X.C. Vidal, L.D. Maroñas and Á.D. Suárez, How to use machine learning to improve the discrimination between signal and background at particle colliders, Appl. Sciences 11 (2021) 11076 [arXiv:2110.15099] [INSPIRE].
J. Lin, M. Freytsis, I. Moult and B. Nachman, Boosting H → \( b\overline{b} \) with machine learning, JHEP 10 (2018) 101 [arXiv:1807.10768] [INSPIRE].
E.A. Moreno et al., Interaction networks for the identification of boosted H → \( b\overline{b} \) decays, Phys. Rev. D 102 (2020) 012010 [arXiv:1909.12285] [INSPIRE].
Y.-L. Chung, K. Cheung and S.-C. Hsu, Sensitivity of two-Higgs-doublet models on Higgs-pair production via \( b\overline{b}b\overline{b} \) final state, Phys. Rev. D 106 (2022) 095015 [arXiv:2207.09602] [INSPIRE].
J.H. Kim et al., Portraying double Higgs at the Large Hadron Collider, JHEP 09 (2019) 047 [arXiv:1904.08549] [INSPIRE].
L. Huang et al., Portraying double Higgs at the Large Hadron Collider II, JHEP 08 (2022) 114 [arXiv:2203.11951] [INSPIRE].
W. Esmail, A. Hammad and S. Moretti, Sharpening the A → Z(*)h signature of the type-II 2HDM at the LHC through advanced machine learning, JHEP 11 (2023) 020 [arXiv:2305.13781] [INSPIRE].
K. Ban, K. Kong, M. Park and S.C. Park, Exploring the synergy of kinematics and dynamics for collider physics, arXiv:2311.16674 [INSPIRE].
A. Chakraborty et al., Revisiting jet clustering algorithms for new Higgs boson searches in hadronic final states, Eur. Phys. J. C 82 (2022) 346 [arXiv:2008.02499] [INSPIRE].
A. Chakraborty et al., Re-evaluating jet reconstruction techniques for new Higgs boson searches, PoS ICHEP2022 (2022) 503 [arXiv:2212.02246] [INSPIRE].
A. Chakraborty et al., Fat b-jet analyses using old and new clustering algorithms in new Higgs boson searches at the LHC, Eur. Phys. J. C 83 (2023) 347 [arXiv:2303.05189] [INSPIRE].
G. Cerro et al., Spectral clustering for jet reconstruction, PoS ICHEP2022 (2022) 771 [arXiv:2211.10164] [INSPIRE].
A. Vaswani et al., Attention is all you needs, in the proceedings of the 31st international conference on neural information processing systems, (2017) [arXiv:1706.03762] [INSPIRE].
B. Käch, D. Krücker and I. Melzer-Pellmann, Point cloud generation using transformer encoders and normalising flows, arXiv:2211.13623 [INSPIRE].
T. Finke, M. Krämer, A. Mück and J. Tönshoff, Learning the language of QCD jets with transformers, JHEP 06 (2023) 184 [arXiv:2303.07364] [INSPIRE].
H. Qu, C. Li and S. Qian, Particle transformer for jet tagging, arXiv:2202.03772 [INSPIRE].
P.T. Komiske, E.M. Metodiev and J. Thaler, Energy flow networks: deep sets for particle jets, JHEP 01 (2019) 121 [arXiv:1810.05165] [INSPIRE].
H. Qu and L. Gouskos, ParticleNet: jet tagging via particle clouds, Phys. Rev. D 101 (2020) 056019 [arXiv:1902.08570] [INSPIRE].
ATLAS collaboration, Search for resonant pair production of Higgs bosons in the \( b\overline{b}b\overline{b} \) final state using pp collisions at \( \sqrt{s} \) = 13 TeV with the ATLAS detector, Phys. Rev. D 105 (2022) 092002 [arXiv:2202.07288] [INSPIRE].
G.C. Branco et al., Theory and phenomenology of two-Higgs-doublet models, Phys. Rept. 516 (2012) 1 [arXiv:1106.0034] [INSPIRE].
T.D. Lee, A theory of spontaneous T violation, Phys. Rev. D 8 (1973) 1226 [INSPIRE].
S.L. Glashow and S. Weinberg, Natural conservation laws for neutral currents, Phys. Rev. D 15 (1977) 1958 [INSPIRE].
I.F. Ginzburg and M. Krawczyk, Symmetries of two Higgs doublet model and CP violation, Phys. Rev. D 72 (2005) 115013 [hep-ph/0408011] [INSPIRE].
S. Antusch, O. Fischer, A. Hammad and C. Scherb, Testing CP properties of extra Higgs states at the HL-LHC, JHEP 03 (2021) 200 [arXiv:2011.10388] [INSPIRE].
S. Antusch, O. Fischer, A. Hammad and C. Scherb, Explaining excesses in four-leptons at the LHC with a double peak from a CP violating two Higgs doublet model, JHEP 08 (2022) 224 [arXiv:2112.00921] [INSPIRE].
A. Arhrib et al., Double neutral Higgs production in the two-Higgs doublet model at the LHC, JHEP 08 (2009) 035 [arXiv:0906.0387] [INSPIRE].
A. Hammad, M. Park, R. Ramos and P. Saha, Exploration of parameter spaces assisted by machine learning, Comput. Phys. Commun. 293 (2023) 108902 [arXiv:2207.09959] [INSPIRE].
J. Alwall et al., The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations, JHEP 07 (2014) 079 [arXiv:1405.0301] [INSPIRE].
W. Porod, SPheno, a program for calculating supersymmetric spectra, SUSY particle decays and SUSY particle production at e+e− colliders, Comput. Phys. Commun. 153 (2003) 275 [hep-ph/0301101] [INSPIRE].
W. Porod and F. Staub, SPheno 3.1: extensions including flavour, CP-phases and models beyond the MSSM, Comput. Phys. Commun. 183 (2012) 2458 [arXiv:1104.1573] [INSPIRE].
T. Sjostrand, S. Mrenna and P.Z. Skands, PYTHIA 6.4 physics and manual, JHEP 05 (2006) 026 [hep-ph/0603175] [INSPIRE].
J. Alwall et al., Comparative study of various algorithms for the merging of parton showers and matrix elements in hadronic collisions, Eur. Phys. J. C 53 (2008) 473 [arXiv:0706.2569] [INSPIRE].
M.L. Mangano, M. Moretti, F. Piccinini and M. Treccani, Matching matrix elements and shower evolution for top-quark production in hadronic collisions, JHEP 01 (2007) 013 [hep-ph/0611129] [INSPIRE].
DELPHES 3 collaboration, DELPHES 3, a modular framework for fast simulation of a generic collider experiment, JHEP 02 (2014) 057 [arXiv:1307.6346] [INSPIRE].
M. Cacciari, G.P. Salam and G. Soyez, The anti-kt jet clustering algorithm, JHEP 04 (2008) 063 [arXiv:0802.1189] [INSPIRE].
S. Catani, Y.L. Dokshitzer, M.H. Seymour and B.R. Webber, Longitudinally invariant Kt clustering algorithms for hadron hadron collisions, Nucl. Phys. B 406 (1993) 187 [INSPIRE].
D. Krohn, J. Thaler and L.-T. Wang, Jet trimming, JHEP 02 (2010) 084 [arXiv:0912.1342] [INSPIRE].
ATLAS collaboration, ATLAS flavour-tagging algorithms for the LHC run 2 pp collision dataset, Eur. Phys. J. C 83 (2023) 681 [arXiv:2211.16345] [INSPIRE].
J.M. Butterworth, A.R. Davison, M. Rubin and G.P. Salam, Jet substructure as a new Higgs search channel at the LHC, Phys. Rev. Lett. 100 (2008) 242001 [arXiv:0802.2470] [INSPIRE].
D.E. Kaplan, K. Rehermann, M.D. Schwartz and B. Tweedie, Top tagging: a method for identifying boosted hadronically decaying top quarks, Phys. Rev. Lett. 101 (2008) 142001 [arXiv:0806.0848] [INSPIRE].
Y. Cui, Z. Han and M.D. Schwartz, W-jet tagging: optimizing the identification of boosted hadronically-decaying W bosons, Phys. Rev. D 83 (2011) 074023 [arXiv:1012.2077] [INSPIRE].
T. Plehn, M. Spannowsky and M. Takeuchi, How to improve top tagging, Phys. Rev. D 85 (2012) 034029 [arXiv:1111.5034] [INSPIRE].
D.E. Soper and M. Spannowsky, Finding top quarks with shower deconstruction, Phys. Rev. D 87 (2013) 054012 [arXiv:1211.3140] [INSPIRE].
C. Anders et al., Benchmarking an even better top tagger algorithm, Phys. Rev. D 89 (2014) 074047 [arXiv:1312.1504] [INSPIRE].
G. Kasieczka et al., Resonance searches with an updated top tagger, JHEP 06 (2015) 203 [arXiv:1503.05921] [INSPIRE].
J. Thaler and K. Van Tilburg, Identifying boosted objects with N-subjettiness, JHEP 03 (2011) 015 [arXiv:1011.2268] [INSPIRE].
J. Thaler and K. Van Tilburg, Maximizing boosted top identification by minimizing N-subjettiness, JHEP 02 (2012) 093 [arXiv:1108.2701] [INSPIRE].
A.J. Larkoski, G.P. Salam and J. Thaler, Energy correlation functions for jet substructure, JHEP 06 (2013) 108 [arXiv:1305.0007] [INSPIRE].
I. Moult, L. Necib and J. Thaler, New angles on energy correlation functions, JHEP 12 (2016) 153 [arXiv:1609.07483] [INSPIRE].
A.J. Larkoski, S. Marzani, G. Soyez and J. Thaler, Soft drop, JHEP 05 (2014) 146 [arXiv:1402.2657] [INSPIRE].
A. Abdesselam et al., Boosted objects: a probe of beyond the Standard Model physics, Eur. Phys. J. C 71 (2011) 1661 [arXiv:1012.5412] [INSPIRE].
A. Altheimer et al., Jet substructure at the Tevatron and LHC: new results, new tools, new benchmarks, J. Phys. G 39 (2012) 063001 [arXiv:1201.0008] [INSPIRE].
A. Altheimer et al., Boosted objects and jet substructure at the LHC. Report of BOOST2012, held at IFIC Valencia, 23rd–27th of July 2012, Eur. Phys. J. C 74 (2014) 2792 [arXiv:1311.2708] [INSPIRE].
J. Cogan, M. Kagan, E. Strauss and A. Schwarztman, Jet-images: computer vision inspired techniques for jet tagging, JHEP 02 (2015) 118 [arXiv:1407.5675] [INSPIRE].
L.G. Almeida et al., Playing tag with ANN: boosted top identification with pattern recognition, JHEP 07 (2015) 086 [arXiv:1501.05968] [INSPIRE].
L. de Oliveira et al., Jet-images — deep learning edition, JHEP 07 (2016) 069 [arXiv:1511.05190] [INSPIRE].
P. Baldi et al., Jet substructure classification in high-energy physics with deep neural networks, Phys. Rev. D 93 (2016) 094034 [arXiv:1603.09349] [INSPIRE].
J. Barnard, E.N. Dawe, M.J. Dolan and N. Rajcic, Parton shower uncertainties in jet substructure analyses with deep neural networks, Phys. Rev. D 95 (2017) 014018 [arXiv:1609.00607] [INSPIRE].
P.T. Komiske, E.M. Metodiev and M.D. Schwartz, Deep learning in color: towards automated quark/gluon jet discrimination, JHEP 01 (2017) 110 [arXiv:1612.01551] [INSPIRE].
G. Kasieczka, T. Plehn, M. Russell and T. Schell, Deep-learning top taggers or the end of QCD?, JHEP 05 (2017) 006 [arXiv:1701.08784] [INSPIRE].
S. Macaluso and D. Shih, Pulling out all the tops with computer vision and deep learning, JHEP 10 (2018) 121 [arXiv:1803.00107] [INSPIRE].
S. Choi, S.J. Lee and M. Perelstein, Infrared safety of a neural-net top tagging algorithm, JHEP 02 (2019) 132 [arXiv:1806.01263] [INSPIRE].
F. Mokhtar, R. Kansal and J. Duarte, Do graph neural networks learn traditional jet substructure?, in the proceedings of the 36th conference on neural information processing systems: workshop on machine learning and the physical sciences, (2022) [arXiv:2211.09912] [INSPIRE].
F. Ma, F. Liu and W. Li, Jet tagging algorithm of graph network with Haar pooling message passing, Phys. Rev. D 108 (2023) 072007 [arXiv:2210.13869] [INSPIRE].
S. Gong et al., An efficient Lorentz equivariant graph neural network for jet tagging, JHEP 07 (2022) 030 [arXiv:2201.08187] [INSPIRE].
D. Guest et al., Jet flavor classification in high-energy physics with deep neural networks, Phys. Rev. D 94 (2016) 112002 [arXiv:1607.08633] [INSPIRE].
J. Pearkes, W. Fedorko, A. Lister and C. Gay, Jet constituents for deep neural network based top quark tagging, arXiv:1704.02124 [INSPIRE].
S. Egan et al., Long Short-Term Memory (LSTM) networks with jet constituents for boosted top tagging at the LHC, arXiv:1711.09059 [INSPIRE].
K. Fraser and M.D. Schwartz, Jet charge and machine learning, JHEP 10 (2018) 093 [arXiv:1803.08066] [INSPIRE].
A. Butter, G. Kasieczka, T. Plehn and M. Russell, Deep-learned top tagging with a Lorentz layer, SciPost Phys. 5 (2018) 028 [arXiv:1707.08966] [INSPIRE].
G. Kasieczka, N. Kiefer, T. Plehn and J.M. Thompson, Quark-gluon tagging: machine learning vs detector, SciPost Phys. 6 (2019) 069 [arXiv:1812.09223] [INSPIRE].
G. Cowan, K. Cranmer, E. Gross and O. Vitells, Asymptotic formulae for likelihood-based tests of new physics, Eur. Phys. J. C 71 (2011) 1554 [Erratum ibid. 73 (2013) 2501] [arXiv:1007.1727] [INSPIRE].
LHC Dark Matter Working Group collaboration, LHC dark matter working group: next-generation spin-0 dark matter models, Phys. Dark Univ. 27 (2020) 100351 [arXiv:1810.09420] [INSPIRE].
E. Arganda, A. Delgado, R.A. Morales and M. Quirós, LHC search strategy for squarks in Higgsino-LSP scenarios with leptons and b-jets in the final state, Particles 5 (2022) 265 [arXiv:2206.05977] [INSPIRE].
H. Chefer, S. Gur and L. Wolf, Transformer interpretability beyond attention visualization, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (2021), p. 782 [arXiv:2012.09838].
R.R. Selvaraju et al., Grad-CAM: visual explanations from deep networks via gradient-based localization, in Proceedings of the IEEE international conference on computer vision, (2017), p. 618 [arXiv:1610.02391].
Y. Huang et al., SSiT: saliency-guided self-supervised image transformer for diabetic retinopathy grading, arXiv:2210.10969.
N. Duong-Trung, D.-M. Nguyen and D. Le-Phuoc, Temporal saliency detection towards explainable transformer-based timeseries forecasting, arXiv:2212.07771.
C. Lu, H. Zhu and P. Koniusz, From saliency to DINO: saliency-guided vision transformer for few-shot keypoint detection, arXiv:2304.03140.
A. Binder et al., Layer-wise relevance propagation for neural networks with local renormalization layers, in Artificial neural networks and machine learning — ICANN 2016: 25th international conference on artificial neural networks, Barcelona, Spain, 6–9 September 2016, Proceedings, part II 25, Springer, (2016), p. 63 [arXiv:1604.00825].
I. Cherepanov, A. Ulmer, J.G. Joewono and J. Kohlhammer, Visualization of class activation maps to explain AI classification of network packet captures, in 2022 IEEE symposium on visualization for cyber security (VizSec), IEEE (2022), p. 1 [https://doi.org/10.1109/VizSec56996.2022.9941392] [arXiv:2209.02045]
B. Zhou et al., Learning deep features for discriminative localization, in Proceedings of the IEEE conference on computer vision and pattern recognition, (2016), p. 2921 [arXiv:1512.04150].
D.P. Kingma and J. Ba, Adam: a method for stochastic optimization, arXiv:1412.6980 [INSPIRE].
J. Terven, D.M. Cordova-Esparza, A. Ramirez-Pedraza and E.A. Chavez-Urbiola, Loss functions and metrics in deep learning, arXiv:2307.02694.
Z. Lai et al., Rethinking skip connections in encoder-decoder networks for monocular depth estimation, arXiv:2208.13441.
D. Hendrycks and K. Gimpel, Gaussian Error Linear Units (GELUs), arXiv:1606.08415 [INSPIRE].
Acknowledgments
The work of SM is supported in part through the NExT Institute, the Knut and Alice Wallenberg Foundation under the Grant No. KAW 2017.0100 (SHIFT) and the STFC Consolidated Grant No. ST/L000296/1. AH and MN are funded by grant number 22H05113, “Foundation of Machine Learning Physics”, Grant in Aid for Transformative Research Areas and 22K03626, Grant-in-Aid for Scientific Research (C).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
ArXiv ePrint: 2401.00452
Rights and permissions
Open Access . This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Hammad, A., Moretti, S. & Nojiri, M. Multi-scale cross-attention transformer encoder for event classification. J. High Energ. Phys. 2024, 144 (2024). https://doi.org/10.1007/JHEP03(2024)144
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/JHEP03(2024)144