Abstract
Modern machine learning techniques can be used to construct powerful models for difficult collider physics problems. In many applications, however, these models are trained on imperfect simulations due to a lack of truth-level information in the data, which risks the model learning artifacts of the simulation. In this paper, we introduce the paradigm of classification without labels (CWoLa) in which a classifier is trained to distinguish statistical mixtures of classes, which are common in collider physics. Crucially, neither individual labels nor class proportions are required, yet we prove that the optimal classifier in the CWoLa paradigm is also the optimal classifier in the traditional fully-supervised case where all label information is available. After demonstrating the power of this method in an analytical toy example, we consider a realistic benchmark for collider physics: distinguishing quark- versus gluon-initiated jets using mixed quark/gluon training samples. More generally, CWoLa can be applied to any classification problem where labels or class proportions are unknown or simulations are unreliable, but statistical mixtures of the classes are available.
References
J. Cogan, M. Kagan, E. Strauss and A. Schwarztman, Jet-Images: Computer Vision Inspired Techniques for Jet Tagging, JHEP 02 (2015) 118 [arXiv:1407.5675] [INSPIRE].
L.G. Almeida, M. Backović, M. Cliche, S.J. Lee and M. Perelstein, Playing Tag with ANN: Boosted Top Identification with Pattern Recognition, JHEP 07 (2015) 086 [arXiv:1501.05968] [INSPIRE].
L. de Oliveira, M. Kagan, L. Mackey, B. Nachman and A. Schwartzman, Jet-images — deep learning edition, JHEP 07 (2016) 069 [arXiv:1511.05190] [INSPIRE].
P. Baldi, K. Bauer, C. Eng, P. Sadowski and D. Whiteson, Jet Substructure Classification in High-Energy Physics with Deep Neural Networks, Phys. Rev. D 93 (2016) 094034 [arXiv:1603.09349] [INSPIRE].
J. Barnard, E.N. Dawe, M.J. Dolan and N. Rajcic, Parton Shower Uncertainties in Jet Substructure Analyses with Deep Neural Networks, Phys. Rev. D 95 (2017) 014018 [arXiv:1609.00607] [INSPIRE].
G. Kasieczka, T. Plehn, M. Russell and T. Schell, Deep-learning Top Taggers or The End of QCD?, JHEP 05 (2017) 006 [arXiv:1701.08784] [INSPIRE].
P.T. Komiske, E.M. Metodiev and M.D. Schwartz, Deep learning in color: towards automated quark/gluon jet discrimination, JHEP 01 (2017) 110 [arXiv:1612.01551] [INSPIRE].
L. de Oliveira, M. Paganini and B. Nachman, Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis, arXiv:1701.05927 [INSPIRE].
P.T. Komiske, E.M. Metodiev, B. Nachman and M.D. Schwartz, Pileup Mitigation with Machine Learning (PUMML), arXiv:1707.08600 [INSPIRE].
G. Louppe, K. Cho, C. Becot and K. Cranmer, QCD-Aware Recursive Neural Networks for Jet Physics, arXiv:1702.00748 [INSPIRE].
ATLAS collaboration, Performance of Top Quark and W Boson Tagging in Run 2 with ATLAS, ATLAS-CONF-2017-064 (2017).
ATLAS collaboration, Optimisation and performance studies of the ATLAS b-tagging algorithms for the 2017-18 LHC run, ATL-PHYS-PUB-2017-013 (2017).
ATLAS collaboration, Identification of Hadronically-Decaying W Bosons and Top Quarks Using High-Level Features as Input to Boosted Decision Trees and Deep Neural Networks in ATLAS at \( \sqrt{s}= 13 \) TeV, ATL-PHYS-PUB-2017-004 (2017).
CMS collaboration, Heavy flavor identification at CMS with deep neural networks, CMS-DP-2017-005 (2017).
CMS collaboration, CMS Phase 1 heavy flavour identification performance and developments, CMS-DP-2017-013 (2017).
ATLAS collaboration, Identification of Jets Containing b-Hadrons with Recurrent Neural Networks at the ATLAS Experiment, ATL-PHYS-PUB-2017-003 (2017).
A. Butter, G. Kasieczka, T. Plehn and M. Russell, Deep-learned Top Tagging using Lorentz Invariance and Nothing Else, arXiv:1707.08966 [INSPIRE].
J. Pearkes, W. Fedorko, A. Lister and C. Gay, Jet Constituents for Deep Neural Network Based Top Quark Tagging, arXiv:1704.02124 [INSPIRE].
K. Datta and A. Larkoski, How Much Information is in a Jet?, JHEP 06 (2017) 073 [arXiv:1704.08249] [INSPIRE].
ATLAS collaboration, Quark versus Gluon Jet Tagging Using Jet Images with the ATLAS Detector, ATL-PHYS-PUB-2017-017 (2017).
CMS collaboration, New Developments for Jet Substructure Reconstruction in CMS, CMS-DP-2017-027 (2017).
ATLAS collaboration, Performance of b-Jet Identification in the ATLAS Experiment, 2016 JINST 11 P04008 [arXiv:1512.01094] [INSPIRE].
CMS collaboration, Identification of b-quark jets with the CMS experiment, 2013 JINST 8 P04013 [arXiv:1211.4462] [INSPIRE].
ATLAS collaboration, Light-quark and gluon jet discrimination in pp collisions at \( \sqrt{s}=7 \) TeV with the ATLAS detector, Eur. Phys. J. C 74(2014) 3023 [arXiv:1405.6583] [INSPIRE].
CMS collaboration, Performance of quark/gluon discrimination in 8 TeV pp data, CMS-PAS-JME-13-002 (2013).
CMS collaboration, Performance of quark/gluon discrimination in 13 TeV data, CMS-DP-2016-070 (2016).
ATLAS collaboration, Identification of boosted, hadronically decaying W bosons and comparisons with ATLAS data taken at √ s = 8 TeV, Eur. Phys. J. C 76 (2016) 154 [arXiv:1510.05821] [INSPIRE].
CMS collaboration, Identification techniques for highly boosted W bosons that decay into hadrons, JHEP 12 (2014) 017 [arXiv:1410.4227] [INSPIRE].
ATLAS collaboration, Identification of high transverse momentum top quarks in pp collisions at \( \sqrt{s}= 8 \) TeV with the ATLAS detector, JHEP 06 (2016) 093 [arXiv:1603.03127] [INSPIRE].
CMS collaboration, Boosted Top Jet Tagging at CMS, CMS-PAS-JME-13-007 (2014).
G. Louppe, M. Kagan and K. Cranmer, Learning to Pivot with Adversarial Networks, arXiv:1611.01046 [INSPIRE].
G. Blanchard, M. Flaska, G. Handy, S. Pozzi, and C. Scott, Classification with asymmetric label noise: consistency and maximal denoising, Electron. J. Stat. 10 (2016) 2780.
J. Hernández-González, I. Inza, and J.A. Lozano, Weak supervision and other non-standard classification problems: a taxonomy, Pattern Recogn. Lett. 69 (2016) 49.
L.M. Dery, B. Nachman, F. Rubbo and A. Schwartzman, Weakly Supervised Classification in High Energy Physics, JHEP 05 (2017) 145 [arXiv:1702.00414] [INSPIRE].
N. Quadrianto, A.J. Smola, T.S. Caetano and Q.V. Le, Estimating labels from label proportions, J. Mach. Learn. Res. 10 (2009) 2349.
G. Patrini, R. Nock, P. Rivera and T. Caetano, (Almost) no label no cry, in Advances in Neural Information Processing Systems, (2014), pp. 190-198.
P. Gras, S. Höche, D. Kar, A. Larkoski, L. Lönnblad, S. Plätzer et al., Systematics of quark/gluon tagging, JHEP 07 (2017) 091 [arXiv:1704.03878] [INSPIRE].
T. Cohen, M. Freytsis and B. Ostdiek, (Machine) Learning to Do More with Less, arXiv:1706.09451 [INSPIRE].
J. Neyman and E.S. Pearson, On the problem of the most efficient tests of statistical hypotheses, in Breakthroughs in statistics, Springer (1992), pp. 73-108.
N. Natarajan, I.S. Dhillon, P.K. Ravikumar and A. Tewari, Learning with noisy labels, in Advances in neural information processing systems 26 (NIPS 2013), (2013), pp. 1196-1204.
C. Scott, A rate of convergence for mixture proportion estimation, with application to learning from noisy labels, in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, (2015), pp. 838-846.
K. Cranmer, J. Pavez and G. Louppe, Approximating Likelihood Ratios with Calibrated Discriminative Classifiers, arXiv:1506.02169 [INSPIRE].
H.P. Nilles and K.H. Streng, quark-gluon Separation in Three Jet Events, Phys. Rev. D 23 (1981) 1944 [INSPIRE].
L.M. Jones, Tests for Determining the Parton Ancestor of a Hadron Jet, Phys. Rev. D 39 (1989) 2550 [INSPIRE].
Z. Fodor, How to See the Differences Between Quark and Gluon Jets, Phys. Rev. D 41 (1990) 1726 [INSPIRE].
L. Jones, Towards a systematic jet classification, Phys. Rev. D 42 (1990) 811 [INSPIRE].
L. Lönnblad, C. Peterson and T. Rognvaldsson, Using neural networks to identify jets, Nucl. Phys. B 349 (1991) 675 [INSPIRE].
J. Pumplin, How to tell quark jets from gluon jets, Phys. Rev. D 44 (1991) 2025 [INSPIRE].
J. Gallicchio and M.D. Schwartz, Quark and Gluon Tagging at the LHC, Phys. Rev. Lett. 107 (2011) 172001 [arXiv:1106.3076] [INSPIRE].
J. Gallicchio and M.D. Schwartz, Quark and Gluon Jet Substructure, JHEP 04 (2013) 090 [arXiv:1211.7038] [INSPIRE].
A.J. Larkoski, J. Thaler and W.J. Waalewijn, Gaining (Mutual) Information about Quark/Gluon Discrimination, JHEP 11 (2014) 129 [arXiv:1408.3122] [INSPIRE].
ATLAS collaboration, Measurement of the charged-particle multiplicity inside jets from \( \sqrt{s}=8 \) TeV pp collisions with the ATLAS detector, Eur. Phys. J. C 76(2016) 322 [arXiv:1602.00988] [INSPIRE].
J. Gallicchio and M.D. Schwartz, Pure Samples of Quark and Gluon Jets at the LHC, JHEP 10 (2011) 103 [arXiv:1104.1175] [INSPIRE].
J.M. Butterworth, A.R. Davison, M. Rubin and G.P. Salam, Jet substructure as a new Higgs search channel at the LHC, Phys. Rev. Lett. 100 (2008) 242001 [arXiv:0802.2470] [INSPIRE].
S.D. Ellis, C.K. Vermilion and J.R. Walsh, Techniques for improved heavy particle searches with jet substructure, Phys. Rev. D 80 (2009) 051501 [arXiv:0903.5081] [INSPIRE].
S.D. Ellis, C.K. Vermilion and J.R. Walsh, Recombination Algorithms and Jet Substructure: Pruning as a Tool for Heavy Particle Searches, Phys. Rev. D 81 (2010) 094023 [arXiv:0912.0033] [INSPIRE].
D. Krohn, J. Thaler and L.-T. Wang, Jet Trimming, JHEP 02 (2010) 084 [arXiv:0912.1342] [INSPIRE].
M. Dasgupta, A. Fregoso, S. Marzani and G.P. Salam, Towards an understanding of jet substructure, JHEP 09 (2013) 029 [arXiv:1307.0007] [INSPIRE].
A.J. Larkoski, S. Marzani, G. Soyez and J. Thaler, Soft Drop, JHEP 05 (2014) 146 [arXiv:1402.2657] [INSPIRE].
C. Frye, A.J. Larkoski, M.D. Schwartz and K. Yan, Precision physics with pile-up insensitive observables, arXiv:1603.06375 [INSPIRE].
C. Frye, A.J. Larkoski, M.D. Schwartz and K. Yan, Factorization for groomed jet substructure beyond the next-to-leading logarithm, JHEP 07 (2016) 064 [arXiv:1603.09338] [INSPIRE].
C.F. Berger, T. Kucs and G.F. Sterman, Event shape/energy flow correlations, Phys. Rev. D 68 (2003) 014012 [hep-ph/0303051] [INSPIRE].
L.G. Almeida, S.J. Lee, G. Perez, G.F. Sterman, I. Sung and J. Virzi, Substructure of high-p T Jets at the LHC, Phys. Rev. D 79 (2009) 074017 [arXiv:0807.0234] [INSPIRE].
S.D. Ellis, C.K. Vermilion, J.R. Walsh, A. Hornig and C. Lee, Jet Shapes and Jet Algorithms in SCET, JHEP 11 (2010) 101 [arXiv:1001.0014] [INSPIRE].
D. Bertolini, T. Chan and J. Thaler, Jet Observables Without Jet Algorithms, JHEP 04 (2014) 013 [arXiv:1310.7584] [INSPIRE].
A.J. Larkoski, D. Neill and J. Thaler, Jet Shapes with the Broadening Axis, JHEP 04 (2014) 017 [arXiv:1401.2158] [INSPIRE].
G. Salam, E ∞ t Scheme, unpublished.
CMS collaboration, Search for a Higgs boson in the decay channel \( H\to Z{Z^{\Big(}}^{\ast \Big)}\to q\overline{q}{\ell}^{-}{\ell}^{+} \) in pp collisions at \( \sqrt{s}=7 \) TeV, JHEP 04 (2012) 036 [arXiv:1202.1416] [INSPIRE].
J.R. Andersen et al., Les Houches 2015: Physics at TeV Colliders Standard Model Working Group Report, in 9th Les Houches Workshop on Physics at TeV Colliders (PhysTeV 2015), Les Houches, France, 1-19 June 2015, (2016) [arXiv:1605.04692] [INSPIRE].
T. Sjöstrand, S. Mrenna and P.Z. Skands, A Brief Introduction to PYTHIA 8.1, Comput. Phys. Commun. 178 (2008) 852 [arXiv:0710.3820] [INSPIRE].
M. Cacciari, G.P. Salam and G. Soyez, The Anti-k t jet clustering algorithm, JHEP 04 (2008) 063 [arXiv:0802.1189] [INSPIRE].
M. Cacciari, G.P. Salam and G. Soyez, FastJet User Manual, Eur. Phys. J. C 72 (2012) 1896 [arXiv:1111.6097] [INSPIRE].
F. Chollet, Keras, (2017) https://github.com/fchollet/keras.
M. Abadi et al., Tensorflow: a system for large-scale machine learning, proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16), volume 16, (2016), pp. 265-283.
K. He, X. Zhang, S. Ren, and J. Sun, Delving deep into rectifiers: surpassing human-level performance on imagenet classification, in proceedings of the IEEE international conference on computer vision, (2015), pp. 1026-1034.
D. Kingma and J. Ba, Adam: a method for stochastic optimization, arXiv:1412.6980.
Open Access
This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Additional information
ArXiv ePrint: 1708.02949
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Metodiev, E.M., Nachman, B. & Thaler, J. Classification without labels: learning from mixed samples in high energy physics. J. High Energ. Phys. 2017, 174 (2017). https://doi.org/10.1007/JHEP10(2017)174
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/JHEP10(2017)174
Keywords
- Jets