Skip to main content

The Tracking Machine Learning Challenge: Accuracy Phase

Part of the The Springer Series on Challenges in Machine Learning book series (SSCML)


This paper reports the results of an experiment in high energy physics: using the power of the “crowd” to solve difficult experimental problems linked to tracking accurately the trajectory of particles in the Large Hadron Collider (LHC). This experiment took the form of a machine learning challenge organized in 2018: the Tracking Machine Learning Challenge (TrackML). Its results were discussed at the competition session at the Neural Information Processing Systems conference (NeurIPS 2018). Given 100,000 points, the participants had to connect them into about 10,000 arcs of circles, following the trajectory of particles issued from very high energy proton collisions. The competition was difficult with a dozen front-runners well ahead of a pack. The single competition score is shown to be accurate and effective in selecting the best algorithms from the domain point of view. The competition has exposed a diversity of approaches, with various roles for Machine Learning, a number of which are discussed in the document.

Participants, others are organizers: Jean-François Puget, Trian Xylouris, Sergey Gorbunov, Nicole Finnie, Liam Finnie, Diogo R. Ferreira, Johan Sokrates Wind, Yuval Reina.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-29135-8_9
  • Chapter length: 34 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-29135-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22


  1. Aad, G., et al.: The ATLAS Simulation Infrastructure. Eur. Phys. J. C70, 823–874 (2010). DOI

    CrossRef  Google Scholar 

  2. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). URL Software available from

  3. Adam-Bourdarios, C., Cowan, G., Germain, C., Guyon, I., Kégl, B., Rousseau, D.: The Higgs boson machine learning challenge. In: HEPML@ NIPS, pp. 19–55 (2014). URL

  4. Amrouche, S., et al.: The Tracking Machine Learning Challenge: Throughput phase. in preparation

    Google Scholar 

  5. Amrouche, S., et al.: Track reconstruction at LHC as a collaborative data challenge use case with RAMP. EPJ Web Conf. 150, 00015 (2017). DOI

    CrossRef  Google Scholar 

  6. Ben-Hur, A., Elisseeff, A., Guyon, I.: A stability based method for discovering structure in clustered data. In: Pacific Symposium on Biocomputing, pp. pp. 6–17 (2002)

    Google Scholar 

  7. Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975). DOI URL

    MathSciNet  CrossRef  Google Scholar 

  8. Flavours of Physics: Finding τ → μμμ. URL

  9. the Large Hadron Collider at CERN. URL

  10. TrackML data set to appear on . URL

  11. TrackML data set to appear on . URL

  12. TrackML particle tracking challenge main site. URL

  13. TrackML Particle Tracking Challenge on Codalab. URL

  14. TrackML Particle Tracking Challenge on Kaggle. URL

  15. Chollet, F., et al.: Keras. (2015)

  16. CMS Collaboration: The Phase-2 Upgrade of the CMS Tracker. Tech. Rep. CMS-TDR-014, CERN, Geneva (2017)

    Google Scholar 

  17. Collaboration, A.: Technical Design Report for the ATLAS Inner Tracker Pixel Detector. Tech. Rep. ATLAS-TDR-030, CERN, Geneva (2017)

    Google Scholar 

  18. Duda, R.O., Hart, P.E.: Use of the hough transformation to detect lines and curves in pictures. Tech. rep., Artificial Intelligence Center (1971)

    Google Scholar 

  19. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd, pp. 226–231. AAAI Press (1996)

    Google Scholar 

  20. Farrell, S., et al.: The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking. EPJ Web Conf. 150, 00003 (2017). DOI

    CrossRef  Google Scholar 

  21. Ferreira, D.R.: Code for TrackML Accuracy challenge. URL

  22. Finnie, L., Finnie, N.: Code for TrackML Accuracy challenge. URL

  23. Gorbunov, S.: Code for TrackML Accuracy challenge. URL

  24. Gumpert, C., Salzburger, A., Kiehn, M., Hrdinka, J., Calace, N.: ACTS: from ATLAS software towards a common track reconstruction software. J. Phys. Conf. Ser. 898(4), 042011 (2017). DOI

    Google Scholar 

  25. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). DOI URL

    CrossRef  Google Scholar 

  26. Meilă, M.: Comparing clusterings—an information based distance. J. Multivar. Anal. 98(5), 873–895 (2007)

    MathSciNet  CrossRef  Google Scholar 

  27. Pérez, F., Granger, B.E.: IPython: a system for interactive scientific computing. Computing in Science and Engineering 9(3), 21–29 (2007). DOI URL

    CrossRef  Google Scholar 

  28. Puget, J.F.: Code for TrackML Accuracy challenge. URL

  29. Rand, W.M.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66(336), 846–850 (1971)

    CrossRef  Google Scholar 

  30. Reina, Y., Xylouris, T.: Code for TrackML Accuracy challenge. URL

  31. Sabes, D.: L1 track triggering with associative memory for the CMS HL-LHC tracker. JINST 9(11), C11014 (2014). URL

    CrossRef  Google Scholar 

  32. Sjöstrand, T., Mrenna, S., Skands, P.: A brief introduction to PYTHIA 8.1. Computer Physics Communications 178(11), 852–867 (2008). DOI

    CrossRef  Google Scholar 

  33. TrackML team: TrackML helper library . URL

  34. Wilcoxon, F.: Individual Comparisons by Ranking Methods. Biometrics Bulletin 1(6), 80–83 (1945). DOI URL

    CrossRef  Google Scholar 

  35. Wind, J.S.: Code for TrackML Accuracy challenge. URL

Download references


The team would like to thank CERN for allowing the use of the dataset, and Kaggle for hosting it. We are very grateful to our generous sponsors without which the challenges would not have been possible. Platinum sponsors: Kaggle, Nvidia and Université de Genève. Gold sponsors: Chalearn, ERC mPP and DataIA. Silver sponsors: CERN Openlab, Paris-Saclay CDS, INRIA, ERC RECEPT, Common Ground, Université Paris Sud, INQNET, Fermilab and pyTorch. TG acknowledges the support of the Swiss National Science Foundation under the grant 200020_181984. SG acknowledges the support of the German BMBF ministry. This project has received funding from the European Union Horizon 2020 research and innovation programme under grant agreement No 724777 “RECEPT”, No 772369 “mPP” and No 654168 “AIDA-2020”. In addition, the organizers would like to thank participant Pei-Lien Chou “outrunner” for major contributions, Maggie Demkin and Walter Reade at Kaggle and the members of the International Advisory Committee: Markus Elsing (CERN), Frank Gaede (DESY), Alison Lowndes (Nvidia), Maurizio Pierini (CERN), Danilo Rezende (Google DeepMind), Marc Schoenauer (INRIA-Saclay) and Svyatoslav Voloshynovskyy (U Genève).

Author information

Authors and Affiliations


Corresponding author

Correspondence to David Rousseau .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Amrouche, S. et al. (2020). The Tracking Machine Learning Challenge: Accuracy Phase. In: Escalera, S., Herbrich, R. (eds) The NeurIPS '18 Competition. The Springer Series on Challenges in Machine Learning. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29134-1

  • Online ISBN: 978-3-030-29135-8

  • eBook Packages: Computer ScienceComputer Science (R0)