Skip to main content

The Tracking Machine Learning Challenge: Accuracy Phase

Part of the The Springer Series on Challenges in Machine Learning book series (SSCML)

Abstract

This paper reports the results of an experiment in high energy physics: using the power of the “crowd” to solve difficult experimental problems linked to tracking accurately the trajectory of particles in the Large Hadron Collider (LHC). This experiment took the form of a machine learning challenge organized in 2018: the Tracking Machine Learning Challenge (TrackML). Its results were discussed at the competition session at the Neural Information Processing Systems conference (NeurIPS 2018). Given 100,000 points, the participants had to connect them into about 10,000 arcs of circles, following the trajectory of particles issued from very high energy proton collisions. The competition was difficult with a dozen front-runners well ahead of a pack. The single competition score is shown to be accurate and effective in selecting the best algorithms from the domain point of view. The competition has exposed a diversity of approaches, with various roles for Machine Learning, a number of which are discussed in the document.

Participants, others are organizers: Jean-François Puget, Trian Xylouris, Sergey Gorbunov, Nicole Finnie, Liam Finnie, Diogo R. Ferreira, Johan Sokrates Wind, Yuval Reina.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-29135-8_9
  • Chapter length: 34 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-29135-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

References

  1. Aad, G., et al.: The ATLAS Simulation Infrastructure. Eur. Phys. J. C70, 823–874 (2010). DOI https://doi.org/10.1140/epjc/s10052-010-1429-9

    CrossRef  Google Scholar 

  2. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). URL https://www.tensorflow.org/. Software available from tensorflow.org

  3. Adam-Bourdarios, C., Cowan, G., Germain, C., Guyon, I., Kégl, B., Rousseau, D.: The Higgs boson machine learning challenge. In: HEPML@ NIPS, pp. 19–55 (2014). URL http://www.jmlr.org/proceedings/papers/v42/cowa14.pdf

  4. Amrouche, S., et al.: The Tracking Machine Learning Challenge: Throughput phase. in preparation

    Google Scholar 

  5. Amrouche, S., et al.: Track reconstruction at LHC as a collaborative data challenge use case with RAMP. EPJ Web Conf. 150, 00015 (2017). DOI https://doi.org/10.1051/epjconf/201715000015

    CrossRef  Google Scholar 

  6. Ben-Hur, A., Elisseeff, A., Guyon, I.: A stability based method for discovering structure in clustered data. In: Pacific Symposium on Biocomputing, pp. pp. 6–17 (2002)

    Google Scholar 

  7. Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975). DOI https://doi.org/10.1145/361002.361007. URL http://doi.acm.org/10.1145/361002.361007

    MathSciNet  CrossRef  Google Scholar 

  8. Flavours of Physics: Finding τ → μμμ. URL https://www.kaggle.com/c/flavours-of-physics

  9. the Large Hadron Collider at CERN. URL https://home.cern/topics/large-hadron-collider

  10. TrackML data set to appear on . URL http://opendata.cern.ch

  11. TrackML data set to appear on . URL https://archive.ics.uci.edu/ml/datasets.html

  12. TrackML particle tracking challenge main site. URL https://sites.google.com/site/trackmlparticle/

  13. TrackML Particle Tracking Challenge on Codalab. URL https://competitions.codalab.org/competitions/20112

  14. TrackML Particle Tracking Challenge on Kaggle. URL https://www.kaggle.com/c/trackml-particle-identification

  15. Chollet, F., et al.: Keras. https://keras.io (2015)

  16. CMS Collaboration: The Phase-2 Upgrade of the CMS Tracker. Tech. Rep. CMS-TDR-014, CERN, Geneva (2017)

    Google Scholar 

  17. Collaboration, A.: Technical Design Report for the ATLAS Inner Tracker Pixel Detector. Tech. Rep. ATLAS-TDR-030, CERN, Geneva (2017)

    Google Scholar 

  18. Duda, R.O., Hart, P.E.: Use of the hough transformation to detect lines and curves in pictures. Tech. rep., Artificial Intelligence Center (1971)

    Google Scholar 

  19. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd, pp. 226–231. AAAI Press (1996)

    Google Scholar 

  20. Farrell, S., et al.: The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking. EPJ Web Conf. 150, 00003 (2017). DOI https://doi.org/10.1051/epjconf/201715000003

    CrossRef  Google Scholar 

  21. Ferreira, D.R.: Code for TrackML Accuracy challenge. URL https://github.com/diogoff/trackml-100

  22. Finnie, L., Finnie, N.: Code for TrackML Accuracy challenge. URL https://github.com/jliamfinnie/kaggle-trackml

  23. Gorbunov, S.: Code for TrackML Accuracy challenge. URL https://github.com/sgorbuno/TrackML_CombinatorialTracker

  24. Gumpert, C., Salzburger, A., Kiehn, M., Hrdinka, J., Calace, N.: ACTS: from ATLAS software towards a common track reconstruction software. J. Phys. Conf. Ser. 898(4), 042011 (2017). DOI https://doi.org/10.1088/1742-6596/898/4/042011

    Google Scholar 

  25. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). DOI https://doi.org/10.1162/neco.1997.9.8.1735. URL http://dx.doi.org/10.1162/neco.1997.9.8.1735

    CrossRef  Google Scholar 

  26. Meilă, M.: Comparing clusterings—an information based distance. J. Multivar. Anal. 98(5), 873–895 (2007)

    MathSciNet  CrossRef  Google Scholar 

  27. Pérez, F., Granger, B.E.: IPython: a system for interactive scientific computing. Computing in Science and Engineering 9(3), 21–29 (2007). DOI https://doi.org/10.1109/MCSE.2007.53. URL https://ipython.org

    CrossRef  Google Scholar 

  28. Puget, J.F.: Code for TrackML Accuracy challenge. URL https://github.com/jfpuget/Kaggle_TrackML

  29. Rand, W.M.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66(336), 846–850 (1971)

    CrossRef  Google Scholar 

  30. Reina, Y., Xylouris, T.: Code for TrackML Accuracy challenge. URL https://github.com/tx1985/kaggle-trackML

  31. Sabes, D.: L1 track triggering with associative memory for the CMS HL-LHC tracker. JINST 9(11), C11014 (2014). URL https://cds.cern.ch/record/2025864

    CrossRef  Google Scholar 

  32. Sjöstrand, T., Mrenna, S., Skands, P.: A brief introduction to PYTHIA 8.1. Computer Physics Communications 178(11), 852–867 (2008). DOI https://doi.org/10.1016/j.cpc.2008.01.036

    CrossRef  Google Scholar 

  33. TrackML team: TrackML helper library . URL https://github.com/LAL/trackml-library

  34. Wilcoxon, F.: Individual Comparisons by Ranking Methods. Biometrics Bulletin 1(6), 80–83 (1945). DOI https://doi.org/10.2307/3001968. URL http://dx.doi.org/10.2307/3001968

    CrossRef  Google Scholar 

  35. Wind, J.S.: Code for TrackML Accuracy challenge. URL https://github.com/top-quarks/top-quarks

Download references

Acknowledgements

The team would like to thank CERN for allowing the use of the dataset, and Kaggle for hosting it. We are very grateful to our generous sponsors without which the challenges would not have been possible. Platinum sponsors: Kaggle, Nvidia and Université de Genève. Gold sponsors: Chalearn, ERC mPP and DataIA. Silver sponsors: CERN Openlab, Paris-Saclay CDS, INRIA, ERC RECEPT, Common Ground, Université Paris Sud, INQNET, Fermilab and pyTorch. TG acknowledges the support of the Swiss National Science Foundation under the grant 200020_181984. SG acknowledges the support of the German BMBF ministry. This project has received funding from the European Union Horizon 2020 research and innovation programme under grant agreement No 724777 “RECEPT”, No 772369 “mPP” and No 654168 “AIDA-2020”. In addition, the organizers would like to thank participant Pei-Lien Chou “outrunner” for major contributions, Maggie Demkin and Walter Reade at Kaggle and the members of the International Advisory Committee: Markus Elsing (CERN), Frank Gaede (DESY), Alison Lowndes (Nvidia), Maurizio Pierini (CERN), Danilo Rezende (Google DeepMind), Marc Schoenauer (INRIA-Saclay) and Svyatoslav Voloshynovskyy (U Genève).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to David Rousseau .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Amrouche, S. et al. (2020). The Tracking Machine Learning Challenge: Accuracy Phase. In: Escalera, S., Herbrich, R. (eds) The NeurIPS '18 Competition. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-29135-8_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29135-8_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29134-1

  • Online ISBN: 978-3-030-29135-8

  • eBook Packages: Computer ScienceComputer Science (R0)