A theory of quark vs. gluon discrimination

  • Andrew J. Larkoski
  • Eric M. MetodievEmail author
Open Access
Regular Article - Theoretical Physics


Understanding jets initiated by quarks and gluons is of fundamental importance in collider physics. Efficient and robust techniques for quark versus gluon jet discrimination have consequences for new physics searches, precision αs studies, parton distribution function extractions, and many other applications. Numerous machine learning analyses have attacked the problem, demonstrating that good performance can be obtained but generally not providing an understanding for what properties of the jets are responsible for that separation power. In this paper, we provide an extensive and detailed analysis of quark versus gluon discrimination from first-principles theoretical calculations. Working in the strongly-ordered soft and collinear limits, we calculate probability distributions for fixed N -body kinematics within jets with up through three resolved emissions\( \left(\mathcal{O}\left({\alpha}_s^3\right)\right) \). This enables explicit calculation of quantities central to machine learning such as the likelihood ratio, the area under the receiver operating characteristic curve, and reducibility factors within a well-defined approximation scheme. Further, we relate the existence of a consistent power counting procedure for discrimination to ideas for operational flavor definitions, and we use this relationship to construct a power counting for quark versus gluon discrimination as an expansion in \( {e}^{C_F-{C}_A}\ll 1 \), the exponential of the fundamental and adjoint Casimirs. Our calculations provide insight into the discrimination performance of particle multiplicity and show how observables sensitive to all emissions in a jet are optimal. We compare our predictions to the performance of individual observables and neural networks with parton shower event generators, validating that our predictions describe the features identified by machine learning.




Open Access

This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited


  1. [1]
    H.P. Nilles and K.H. Streng, Quark-gluon Separation in Three Jet Events, Phys. Rev.D 23 (1981) 1944 [INSPIRE].
  2. [2]
    L.M. Jones, Tests for Determining the Parton Ancestor of a Hadron Jet, Phys. Rev.D 39 (1989) 2550 [INSPIRE].
  3. [3]
    Z. Fodor, How to See the Differences Between Quark and Gluon Jets, Phys. Rev.D 41 (1990) 1726 [INSPIRE].
  4. [4]
    L. Jones, Towards a systematic jet classification, Phys. Rev.D 42 (1990) 811 [INSPIRE].
  5. [5]
    J. Pumplin, How to tell quark jets from gluon jets, Phys. Rev.D 44 (1991) 2025 [INSPIRE].
  6. [6]
    J. Gallicchio and M.D. Schwartz, Pure Samples of Quark and Gluon Jets at the LHC, JHEP10 (2011) 103 [arXiv:1104.1175] [INSPIRE].ADSGoogle Scholar
  7. [7]
    J. Gallicchio and M.D. Schwartz, Quark and Gluon Tagging at the LHC, Phys. Rev. Lett.107 (2011) 172001 [arXiv:1106.3076] [INSPIRE].ADSGoogle Scholar
  8. [8]
    J. Gallicchio and M.D. Schwartz, Quark and Gluon Jet Substructure, JHEP04 (2013) 090 [arXiv:1211.7038] [INSPIRE].ADSGoogle Scholar
  9. [9]
    D. Ferreira de Lima, P. Petrov, D. Soper and M. Spannowsky, quark-gluon tagging with Shower Deconstruction: Unearthing dark matter and Higgs couplings, Phys. Rev.D 95 (2017) 034001 [arXiv:1607.06031] [INSPIRE].
  10. [10]
    C. Frye, A.J. Larkoski, J. Thaler and K. Zhou, Casimir Meets Poisson: Improved Quark/Gluon Discrimination with Counting Observables, JHEP09 (2017) 083 [arXiv:1704.06266] [INSPIRE].ADSGoogle Scholar
  11. [11]
    J. Davighi and P. Harris, Fractal based observables to probe jet substructure of quarks and gluons, Eur. Phys. J.C 78 (2018) 334 [arXiv:1703.00914] [INSPIRE].
  12. [12]
    P.T. Komiske, E.M. Metodiev and J. Thaler, Energy Flow Networks: Deep Sets for Particle Jets, JHEP01 (2019) 121 [arXiv:1810.05165] [INSPIRE].ADSGoogle Scholar
  13. [13]
    A. Banfi, G.P. Salam and G. Zanderighi, Infrared safe definition of jet flavor, Eur. Phys. J.C 47 (2006) 113 [hep-ph/0601139] [INSPIRE].
  14. [14]
    C. Frye, A.J. Larkoski, M.D. Schwartz and K. Yan, Factorization for groomed jet substructure beyond the next-to-leading logarithm, JHEP07 (2016) 064 [arXiv:1603.09338] [INSPIRE].
  15. [15]
    P. Gras et al., Systematics of quark/gluon tagging, JHEP07 (2017) 091 [arXiv:1704.03878] [INSPIRE].ADSGoogle Scholar
  16. [16]
    E.M. Metodiev and J. Thaler, Jet Topics: Disentangling Quarks and Gluons at Colliders, Phys. Rev. Lett.120 (2018) 241602 [arXiv:1802.00008] [INSPIRE].ADSGoogle Scholar
  17. [17]
    P.T. Komiske, E.M. Metodiev and J. Thaler, An operational definition of quark and gluon jets, JHEP11 (2018) 059 [arXiv:1809.01140] [INSPIRE].ADSGoogle Scholar
  18. [18]
    A.J. Larkoski, J. Thaler and W.J. Waalewijn, Gaining (Mutual) Information about Quark/Gluon Discrimination, JHEP11 (2014) 129 [arXiv:1408.3122] [INSPIRE].ADSGoogle Scholar
  19. [19]
    B. Bhattacherjee, S. Mukhopadhyay, M.M. Nojiri, Y. Sakaki and B.R. Webber, Associated jet and subjet rates in light-quark and gluon jet discrimination, JHEP04 (2015) 131 [arXiv:1501.04794] [INSPIRE].ADSGoogle Scholar
  20. [20]
    J. Mo, F.J. Tackmann and W.J. Waalewijn, A case study of quark-gluon discrimination at NNLL’ in comparison to parton showers, Eur. Phys. J.C 77 (2017) 770 [arXiv:1708.00867] [INSPIRE].ADSGoogle Scholar
  21. [21]
    Y. Sakaki, Quark jet rates and quark-gluon discrimination in multijet final states, Phys. Rev.D 99 (2019) 114012 [arXiv:1807.01421] [INSPIRE].
  22. [22]
    L. Lönnblad, C. Peterson and T. Rognvaldsson, Using neural networks to identify jets, Nucl. Phys.B 349 (1991) 675 [INSPIRE].
  23. [23]
    P.T. Komiske, E.M. Metodiev and M.D. Schwartz, Deep learning in color: towards automated quark/gluon jet discrimination, JHEP01 (2017) 110 [arXiv:1612.01551] [INSPIRE].
  24. [24]
    T. Cheng, Recursive Neural Networks in Quark/Gluon Tagging, Comput. Softw. Big Sci.2 (2018) 3 [arXiv:1711.02633] [INSPIRE].Google Scholar
  25. [25]
    H. Lüo, M.-x. Luo, K. Wang, T. Xu and G. Zhu, Quark jet versus gluon jet: fully-connected neural networks with high-level features, Sci. China Phys. Mech. Astron.62 (2019) 991011 [arXiv:1712.03634] [INSPIRE].
  26. [26]
    G. Kasieczka, N. Kiefer, T. Plehn and J.M. Thompson, quark-gluon Tagging: Machine Learning vs Detector, SciPost Phys.6 (2019) 069 [arXiv:1812.09223] [INSPIRE].
  27. [27]
    C.F. Berger, T. Kucs and G.F. Sterman, Event shape/energy flow correlations, Phys. Rev.D 68 (2003) 014012 [hep-ph/0303051] [INSPIRE].
  28. [28]
    L.G. Almeida, S.J. Lee, G. Perez, G.F. Sterman, I. Sung and J. Virzi, Substructure of high-p TJets at the LHC, Phys. Rev.D 79 (2009) 074017 [arXiv:0807.0234] [INSPIRE].
  29. [29]
    S.D. Ellis, C.K. Vermilion, J.R. Walsh, A. Hornig and C. Lee, Jet Shapes and Jet Algorithms in SCET, JHEP11 (2010) 101 [arXiv:1001.0014] [INSPIRE].ADSGoogle Scholar
  30. [30]
    A.J. Larkoski, S. Marzani, G. Soyez and J. Thaler, Soft Drop, JHEP05 (2014) 146 [arXiv:1402.2657] [INSPIRE].ADSGoogle Scholar
  31. [31]
    A.J. Larkoski, I. Moult and D. Neill, Power Counting to Better Jet Observables, JHEP12 (2014) 009 [arXiv:1409.6298] [INSPIRE].ADSGoogle Scholar
  32. [32]
    A.J. Larkoski, I. Moult and D. Neill, Building a Better Boosted Top Tagger, Phys. Rev.D 91 (2015) 034035 [arXiv:1411.0665] [INSPIRE].
  33. [33]
    I. Moult, L. Necib and J. Thaler, New Angles on Energy Correlation Functions, JHEP12 (2016) 153 [arXiv:1609.07483] [INSPIRE].ADSGoogle Scholar
  34. [34]
    J. Neyman and E.S. Pearson, IX. On the problem of the most efficient tests of statistical hypotheses, Phil. Trans. Roy. Soc. Lond.A 231 (1933) 289.Google Scholar
  35. [35]
    I.W. Stewart, F.J. Tackmann and W.J. Waalewijn, N-Jettiness: An Inclusive Event Shape to Veto Jets, Phys. Rev. Lett.105 (2010) 092002 [arXiv:1004.2489] [INSPIRE].
  36. [36]
    J. Thaler and K. Van Tilburg, Identifying Boosted Objects with N-subjettiness, JHEP03 (2011) 015 [arXiv:1011.2268] [INSPIRE].ADSGoogle Scholar
  37. [37]
    J. Thaler and K. Van Tilburg, Maximizing Boosted Top Identification by Minimizing N-subjettiness, JHEP02 (2012) 093 [arXiv:1108.2701] [INSPIRE].ADSGoogle Scholar
  38. [38]
    F.V. Tkachov, Measuring the number of hadronic jets, Phys. Rev. Lett.73 (1994) 2405 [hep-ph/9901332] [INSPIRE].
  39. [39]
    F.V. Tkachov, Measuring multi-jet structure of hadronic energy flow or What is a jet?, Int. J. Mod. Phys.A 12 (1997) 5411 [hep-ph/9601308] [INSPIRE].
  40. [40]
    A.J. Larkoski, G.P. Salam and J. Thaler, Energy Correlation Functions for Jet Substructure, JHEP06 (2013) 108 [arXiv:1305.0007] [INSPIRE].ADSMathSciNetzbMATHGoogle Scholar
  41. [41]
    P.T. Komiske, E.M. Metodiev and J. Thaler, Energy flow polynomials: A complete linear basis for jet substructure, JHEP04 (2018) 013 [arXiv:1712.07124] [INSPIRE].Google Scholar
  42. [42]
    B. Andersson, G. Gustafson, L. L¨onnblad and U. Pettersson, Coherence Effects in Deep Inelastic Scattering, Z. Phys.C 43 (1989) 625 [INSPIRE].
  43. [43]
    K. Datta and A. Larkoski, How Much Information is in a Jet?, JHEP06 (2017) 073 [arXiv:1704.08249] [INSPIRE].ADSGoogle Scholar
  44. [44]
    ATLAS collaboration, Identification of boosted, hadronically decaying W bosons and comparisons with ATLAS data taken at \( \sqrt{s} \) = 8 TeV, Eur. Phys. J.C 76 (2016) 154 [arXiv:1510.05821] [INSPIRE].
  45. [45]
    ATLAS collaboration, Search for dark matter produced in association with a hadronically decaying vector boson in pp collisions at \( \sqrt{s} \) = 13 TeV with the ATLAS detector, Phys. Lett.B 763 (2016) 251 [arXiv:1608.02372] [INSPIRE].
  46. [46]
    ATLAS collaboration, Performance of top-quark and W -boson tagging with ATLAS in Run 2 of the LHC, Eur. Phys. J.C 79 (2019) 375 [arXiv:1808.07858] [INSPIRE].
  47. [47]
    ATLAS collaboration, Measurement of jet-substructure observables in top quark, W boson and light jet production in proton-proton collisions at \( \sqrt{s} \) = 13 TeV with the ATLAS detector, JHEP08 (2019) 033 [arXiv:1903.02942] [INSPIRE].
  48. [48]
    M. Dasgupta, L. Schunk and G. Soyez, Jet shapes for boosted jet two-prong decays from first-principles, JHEP04 (2016) 166 [arXiv:1512.00516] [INSPIRE].ADSGoogle Scholar
  49. [49]
    R.K. Ellis, W.J. Stirling and B.R. Webber, QCD and collider physics, Camb. Monogr. Part. Phys. Nucl. Phys. Cosmol.8 (1996) 1.Google Scholar
  50. [50]
    G. Soyez, G.P. Salam, J. Kim, S. Dutta and M. Cacciari, Pileup subtraction for jet shapes, Phys. Rev. Lett.110 (2013) 162001 [arXiv:1211.2811] [INSPIRE].ADSGoogle Scholar
  51. [51]
    A.J. Larkoski and J. Thaler, Unsafe but Calculable: Ratios of Angularities in Perturbative QCD, JHEP09 (2013) 137 [arXiv:1307.1699] [INSPIRE].ADSGoogle Scholar
  52. [52]
    A.J. Larkoski, I. Moult and D. Neill, Toward Multi-Differential Cross Sections: Measuring Two Angularities on a Single Jet, JHEP09 (2014) 046 [arXiv:1401.4458] [INSPIRE].ADSGoogle Scholar
  53. [53]
    M. Procura, W.J. Waalewijn and L. Zeune, Resummation of Double-Differential Cross Sections and Fully-Unintegrated Parton Distribution Functions, JHEP02 (2015) 117 [arXiv:1410.6483] [INSPIRE].ADSGoogle Scholar
  54. [54]
    M. Procura, W.J. Waalewijn and L. Zeune, Joint resummation of two angularities at next-to-next-to-leading logarithmic order, JHEP10 (2018) 098 [arXiv:1806.10622] [INSPIRE].ADSGoogle Scholar
  55. [55]
    A.J. Larkoski, D. Neill and J. Thaler, Jet Shapes with the Broadening Axis, JHEP04 (2014) 017 [arXiv:1401.2158] [INSPIRE].ADSGoogle Scholar
  56. [56]
    D. Bertolini, T. Chan and J. Thaler, Jet Observables Without Jet Algorithms, JHEP04 (2014) 013 [arXiv:1310.7584] [INSPIRE].ADSGoogle Scholar
  57. [57]
    G. Salam, unpublished.Google Scholar
  58. [58]
    ATLAS collaboration, Light-quark and gluon jet discrimination in pp collisions at \( \sqrt{s} \) = 7 TeV with the ATLAS detector, Eur. Phys. J.C 74 (2014) 3023 [arXiv:1405.6583] [INSPIRE].
  59. [59]
    G.P. Salam, L. Schunk and G. Soyez, Dichroic subjettiness ratios to distinguish colour flows in boosted boson tagging, JHEP03 (2017) 022 [arXiv:1612.03917] [INSPIRE].ADSGoogle Scholar
  60. [60]
    D. Napoletano and G. Soyez, Computing N -subjettiness for boosted jets, JHEP12 (2018) 031 [arXiv:1809.04602] [INSPIRE].ADSGoogle Scholar
  61. [61]
    T. Hahn, CUBA: A library for multidimensional numerical integration, Comput. Phys. Commun.168 (2005) 78 [hep-ph/0404043] [INSPIRE].
  62. [62]
    T. Sjöstrand, S. Mrenna and P.Z. Skands, PYTHIA 6.4 Physics and Manual, JHEP05 (2006) 026 [hep-ph/0603175] [INSPIRE].
  63. [63]
    T. Sjöstrand et al., An Introduction to PYTHIA 8.2, Comput. Phys. Commun.191 (2015) 159 [arXiv:1410.3012] [INSPIRE].
  64. [64]
    M. Cacciari, G.P. Salam and G. Soyez, The anti-k tjet clustering algorithm, JHEP04 (2008) 063 [arXiv:0802.1189] [INSPIRE].
  65. [65]
    M. Cacciari, G.P. Salam and G. Soyez, FastJet User Manual, Eur. Phys. J.C 72 (2012) 1896 [arXiv:1111.6097] [INSPIRE].ADSzbMATHGoogle Scholar
  66. [66]
  67. [67]
    K. Datta and A.J. Larkoski, Novel Jet Observables from Machine Learning, JHEP03 (2018) 086 [arXiv:1710.01305] [INSPIRE].ADSGoogle Scholar
  68. [68]
    K. Datta, A. Larkoski and B. Nachman, Automating the Construction of Jet Observables with Machine Learning, arXiv:1902.07180 [INSPIRE].
  69. [69]
    F. Chollet, Keras,, (2015).
  70. [70]
    M. Abadi et al., Tensorflow: A system for large-scale machine learning, in proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 265–283, [arXiv:1605.08695].
  71. [71]
    V. Nair and G.E. Hinton, Rectified linear units improve restricted boltzmann machines, in proceedings of the 27th international conference on machine learning (ICML-10), (2010), pp. 807–814.Google Scholar
  72. [72]
    K. He, X. Zhang, S. Ren and J. Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in proceedings of the IEEE international conference on computer vision, (2015), pp. 1026–1034.Google Scholar
  73. [73]
    D.P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, arXiv:1412.6980 [INSPIRE].
  74. [74]
    L.M. Dery, B. Nachman, F. Rubbo and A. Schwartzman, Weakly Supervised Classification in High Energy Physics, JHEP05 (2017) 145 [arXiv:1702.00414] [INSPIRE].ADSzbMATHGoogle Scholar
  75. [75]
    T. Cohen, M. Freytsis and B. Ostdiek, (Machine) Learning to Do More with Less, JHEP02 (2018) 034 [arXiv:1706.09451] [INSPIRE].
  76. [76]
    E.M. Metodiev, B. Nachman and J. Thaler, Classification without labels: Learning from mixed samples in high energy physics, JHEP10 (2017) 174 [arXiv:1708.02949] [INSPIRE].ADSGoogle Scholar
  77. [77]
    P.T. Komiske, E.M. Metodiev, B. Nachman and M.D. Schwartz, Learning to classify from impure samples with high-dimensional data, Phys. Rev.D 98 (2018) 011502 [arXiv:1801.10158] [INSPIRE].
  78. [78]
    J.S. Conway, R. Bhaskar, R.D. Erbacher and J. Pilot, Identification of High-Momentum Top Quarks, Higgs Bosons, and W and Z Bosons Using Boosted Event Shapes, Phys. Rev.D 94 (2016) 094027 [arXiv:1606.06859] [INSPIRE].
  79. [79]
    J.A. Aguilar-Saavedra, J.H. Collins and R.K. Mishra, A generic anti-QCD jet tagger, JHEP11 (2017) 163 [arXiv:1709.01087] [INSPIRE].ADSGoogle Scholar
  80. [80]
    J.H. Collins, K. Howe and B. Nachman, Anomaly Detection for Resonant New Physics with Machine Learning, Phys. Rev. Lett.121 (2018) 241803 [arXiv:1805.02664] [INSPIRE].ADSGoogle Scholar
  81. [81]
    J. Hajer, Y.-Y. Li, T. Liu and H. Wang, Novelty Detection Meets Collider Physics, arXiv:1807.10261 [INSPIRE].
  82. [82]
    T. Heimel, G. Kasieczka, T. Plehn and J.M. Thompson, QCD or What?, SciPost Phys.6 (2019) 030 [arXiv:1808.08979] [INSPIRE].ADSGoogle Scholar
  83. [83]
    M. Farina, Y. Nakai and D. Shih, Searching for New Physics with Deep Autoencoders, arXiv:1808.08992 [INSPIRE].
  84. [84]
    J.H. Collins, K. Howe and B. Nachman, Extending the search for new resonances with machine learning, Phys. Rev.D 99 (2019) 014038 [arXiv:1902.02634] [INSPIRE].
  85. [85]
    T.S. Roy and A.H. Vijay, A robust anomaly finder based on autoencoder, arXiv:1903.02032 [INSPIRE].
  86. [86]
    R.D. Field and R.P. Feynman, A Parametrization of the Properties of Quark Jets, Nucl. Phys.B 136 (1978) 1 [INSPIRE].
  87. [87]
    D. Krohn, M.D. Schwartz, T. Lin and W.J. Waalewijn, Jet Charge at the LHC, Phys. Rev. Lett.110 (2013) 212001 [arXiv:1209.2421] [INSPIRE].ADSGoogle Scholar
  88. [88]
    W.J. Waalewijn, Calculating the Charge of a Jet, Phys. Rev.D 86 (2012) 094030 [arXiv:1209.3019] [INSPIRE].
  89. [89]
    European Muon collaboration, Quark Charge Retention in Final State Hadrons From Deep Inelastic Muon Scattering, Phys. Lett.144B (1984) 302 [INSPIRE].
  90. [90]
    ALEPH collaboration, Measurement of charge asymmetry in hadronic Z decays, Phys. Lett.B 259 (1991) 377 [INSPIRE].
  91. [91]
    ATLAS collaboration, Jet Charge with the ATLAS Detector using \( \sqrt{s} \) = 8 TeV pp Collision Data, in Proceedings, 2nd Conference on Large Hadron Collider Physics Conference (LHCP 2014): New York, U.S.A., June 2–7, 2014, arXiv:1409.0318 [INSPIRE].
  92. [92]
    ATLAS collaboration, Measurement of jet charge in dijet events from \( \sqrt{s} \) = 8 TeV pp collisions with the ATLAS detector, Phys. Rev.D 93 (2016) 052003 [arXiv:1509.05190] [INSPIRE].
  93. [93]
    CMS collaboration, Measurements of jet charge with dijet events in pp collisions at \( \sqrt{s} \) = 8 TeV, JHEP10 (2017) 131 [arXiv:1706.05868] [INSPIRE].
  94. [94]
    K. Fraser and M.D. Schwartz, Jet Charge and Machine Learning, JHEP10 (2018) 093 [arXiv:1803.08066] [INSPIRE].ADSGoogle Scholar
  95. [95]
    P. Mattig and W. Zeuner, Final state photon bremsstrahlung in e +e → Z 0→ hadrons as a tool for a precise measurement of the weak quark couplings, Z. Phys.C 52 (1991) 31 [INSPIRE].
  96. [96]
    OPAL collaboration, Measurement of the partial widths of the Z into up and down type quarks, Phys. Lett.B 586 (2004) 167 [hep-ex/0312043] [INSPIRE].
  97. [97]
    Z. Hall and J. Thaler, Photon isolation and jet substructure, JHEP09 (2018) 164 [arXiv:1805.11622] [INSPIRE].ADSGoogle Scholar

Copyright information

© The Author(s) 2019

Authors and Affiliations

  1. 1.Physics DepartmentReed CollegePortlandU.S.A.
  2. 2.Center for Theoretical PhysicsMassachusetts Institute of TechnologyCambridgeU.S.A.
  3. 3.Department of PhysicsHarvard UniversityCambridgeU.S.A.

Personalised recommendations