Skip to main content
SpringerLink
Log in

Review of real-time data processing for collider experiments

  • Regular Article
  • Published:
The European Physical Journal Plus Aims and scope Submit manuscript

Abstract

We review the status of, and prospects for, real-time data processing for collider experiments in experimental High Energy Physics. We discuss the historical evolution of data rates and volumes in the field and place them in the context of data in other scientific domains and commercial applications. We review the requirements for real-time processing of these data, and the constraints they impose on the computing architectures used for such processing. We describe the evolution of real-time processing over the past decades with a particular focus on the Large Hadron Collider experiments and their planned upgrades over the next decade. We then discuss how the scientific trends in the field and commercial trends in computing architectures may influence real-time processing over the coming decades.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability Statement

There is no data associated with this manuscript.

Notes

  1. For the purposes of this review, an event designates one nominal unit of data used in physics analysis. In collider experiments, it typically corresponds to the data produced during one crossing of the colliding beams.

  2. In practice this means that any deviations from perfect reproducibility should be small compared to other data-simulation differences and systematic uncertainties associated with the relevant physics analyses.

  3. As we discuss in Sections 3 and 4, several notable past experiments used multiple levels of fixed-latency triggers, but modern experiments which use fixed-latency triggers almost universally use only one level.

  4. Refs. [36, 45,46,47] contain more detailed and technical discussions of this topic.

  5. https://gptblogs.com/chatgpt-how-much-data-is-used-in-the-training-process.

  6. https://www.top500.org/.

  7. https://en.wikichip.org/wiki/WikiChip.

  8. In HEP, and at CERN in particular, data centre power consumption remains marginal relative to the accelerator’s own power consumption. There is nevertheless increasing pressure, which will in any case be amplified by commercial trends, to deploy applications which minimise this power consumption.

References

  1. V. Lindenstruth, I. Kisel, Nuclear instruments and methods in physics research section a: accelerators. Spectrom. Detectors Assoc. Equip. 535, 48 (2004). https://doi.org/10.1016/j.nima.2004.07.267

    Article  Google Scholar 

  2. N. Neufeld, Lhc trigger & daq - an introductory overview, in 2012 18th IEEE-NPSS Real Time Conference, (2012), pp. 1–4

  3. S. Cittolin, Phil. Trans. R. Soc. Lond. A 370, 950 (2012). https://doi.org/10.1098/rsta.2011.0464

    Article  ADS  Google Scholar 

  4. J. C. McCallum, Historical prices of computing equipment (2023) https://jcmit.net/index.htm

  5. LHCb collaboration collaboration (R. Aaij et al.), Int. J. Mod. Phys. A30, 1530022 (2014)

  6. ATLAS collaboration collaboration (G. Aad et al.), JINST3, S08003 (2008), https://doi.org/10.1088/1748-0221/3/08/S08003

  7. CMS collaboration collaboration (S. Chatrchyan et al.), JINST3, S08004361 (2008), https://doi.org/10.1088/1748-0221/3/08/S08004

  8. CMS collaboration (A. M. Sirunyan et al.), JINST15, P10017 (2020), arXiv:2006.10165 [hep-ex], https://doi.org/10.1088/1748-0221/15/10/P10017

  9. NA62 collaboration (E. Cortina Gil et al.), JINST12, P05025 (2017), arXiv:1703.08501 [physics.ins-det], https://doi.org/10.1088/1748-0221/12/05/P05025

  10. DUNE collaboration (B. Abi et al.), JINST15, T08008 (2020), arXiv:2002.02967 [physics.ins-det], https://doi.org/10.1088/1748-0221/15/08/T08008

  11. LHCb collaboration (R. Aaij et al.) (5 2023), arXiv:2305.10515 [hep-ex]

  12. ALICE collaboration collaboration (Ananya et al.), J. Phys. Conf. Ser.,513, 012037 (2014)

  13. ATLAS, Technical Design Report for the Phase-II Upgrade of the ATLAS TDAQ System (2017), https://doi.org/10.17181/CERN.2LBB.4IAL, https://cds.cern.ch/record/2285584

  14. CMS, The Phase-2 Upgrade of the CMS Level-1 Trigger (2020), Final version https://cds.cern.ch/record/2714892

  15. M. Dam, Eur. Phys. J. Plus 137, 81 (2022). https://doi.org/10.1140/epjp/s13360-021-02265-3. arXiv:2107.12837 [physics.ins-det]

    Article  Google Scholar 

  16. J. Dorenbosch, eConf. C851111, 134 (1985)

    Google Scholar 

  17. D. Decamp et al., Nuclear instruments and methods in physics research section a: accelerators. Spectrom. Detectors Assoc. Equip. 294, 121 (1990)

    Article  Google Scholar 

  18. V. Bocci et al. (1994), https://doi.org/10.1109/23.467783, https://cds.cern.ch/record/274052

  19. H1 collaboration (I. Abt et al.), Nucl. Instrum. Meth. A386, 310 (1997), https://doi.org/10.1016/S0168-9002(96)00893-5

  20. A. Baird et al., IEEE Trans. Nucl. Sci. 48, 1276 (2001). https://doi.org/10.1109/23.958765. arXiv:hep-ex/0104010

    Article  ADS  Google Scholar 

  21. CDF collaboration (D. Amidei et al.), Nucl. Instrum. Meth. A269, 51 (1988), https://doi.org/10.1016/0168-9002(88)90861-3

  22. CDF collaboration (B. Ashmanskas et al.), Nucl. Instrum. Meth. A518, 532 (2004), arXiv:physics/0306169, https://doi.org/10.1016/j.nima.2003.11.078

  23. ATLAS collaboration, ATLAS Level-1 Trigger: Technical Design Report (CERN, Geneva, 1998)

  24. ATLAS collaboration, P. Jenni, M. Nessi, M. Nordberg and K. Smith, ATLAS High-level Trigger, Data-Acquisition and Controls: Technical Design Report (CERN, Geneva, 2003)

  25. CMS collaboration, G. L. Bayatyan et al., CMS TriDAS Project: Technical Design Report, Volume 1: The Trigger Systems

  26. CMS collaboration, S. Cittolin, A. Rácz and P. Sphicas, CMS The TriDAS Project: Technical Design Report, Volume 2: Data Acquisition and High-Level Trigger. CMS trigger and data-acquisition project (CERN, Geneva, 2002)

  27. LHCb collaboration, R. Antunes-Nobrega et al., LHCb trigger system: Technical Design Report (CERN, Geneva, 2003)

  28. LHCb collaboration, T. Nakada, O. Ullaland and W. Witzelling, Expression of Interest for an LHCb Upgrade, tech. rep., CERN (2008)

  29. LHCb collaboration (R. Aaij et al.), JINST14, P04013 (2019), https://doi.org/10.1088/1748-0221/14/04/P04013, arXiv:1812.10790 [hep-ex]

  30. V. V. Gligorov (2011), https://inspirehep.net/literature/928797

  31. R. Aaij et al., JINST 8, P04022 (2013). https://doi.org/10.1088/1748-0221/8/04/P04022. arXiv:1211.3055 [hep-ex]

    Article  Google Scholar 

  32. CMS collaboration (A. Zabi, J. W. Berryhill, E. Perez and A. D. Tapper) (2020)

  33. CMS collaboration (G. Badaro et al.), EPJ Web Conf.251, 04023 (2021), https://doi.org/10.1051/epjconf/202125104023

  34. ATLAS collaboration (https://doi.org/10.1051/epjconf/202125104023) https://doi.org/10.17181/CERN.2LBB.4IAL

  35. CMS collaboration (V. Khachatryan et al.), Phys. Rev. Lett.117, 031802 (2016), arXiv:1604.08907 [hep-ex], https://doi.org/10.1103/PhysRevLett.117.031802

  36. R. Aaij et al., Comput. Phys. Commun. 208, 35 (2016). https://doi.org/10.1016/j.cpc.2016.07.022. arXiv:1604.05596 [physics.ins-det]

    Article  ADS  Google Scholar 

  37. ATLAS collaboration collaboration, Trigger-Object Level Analysis with the ATLAS Detector at the Large Hadron Collider: Summary and Perspectives, Tech. Rep. ATL-DAQ-PUB-2017-003, CERN (2017)

  38. ATLAS collaboration (M. Aaboud et al.), Phys. Rev. Lett.121, 081801 (2018), https://doi.org/10.1103/PhysRevLett.121.081801, arXiv:1804.03496 [hep-ex]

  39. CMS collaboration (A. M. Sirunyan et al.), Phys. Rev. Lett.124, 131802 (2020) https://doi.org/10.1103/PhysRevLett.124.131802, arXiv:1912.04776 [hep-ex]

  40. R. Nane, V.-M. Sima, C. Pilato, J. Choi, B. Fort, A. Canis, Y.T. Chen, H. Hsiao, S. Brown, F. Ferrandi, J. Anderson, K. Bertels, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 35, 1591 (2016). https://doi.org/10.1109/TCAD.2015.2513673

    Article  Google Scholar 

  41. T. Marc-André, Two FPGA Case Studies Comparing High Level Synthesis and Manual HDL for HEP Applications (2018). arXiv:1806.10672 [physics.ins-det]

  42. J. Marjanovic, Low vs high level programming for FPGA, in 7th International Beam Instrumentation Conference, (2019), p. THOA01

  43. S. Lahti, P. Sjövall, J. Vanne, T.D. Hämäläinen, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 38, 898 (2019). https://doi.org/10.1109/TCAD.2018.2834439

    Article  Google Scholar 

  44. J. Duarte et al., JINST 13, P07027 (2018). https://doi.org/10.1088/1748-0221/13/07/P07027. arXiv:1804.06913 [physics.ins-det]

    Article  Google Scholar 

  45. C. Fitzpatrick, V.V. Gligorov, LHCb-PUB-2014-027 (2014). https://inspirehep.net/literature/1920705

  46. V. V. Gligorov, Conceptualization, implementation, and commissioning of real-time analysis in the High Level Trigger of the LHCb experiment, PhD thesis, Paris U., VI-VII (2018)

  47. R. Aaij et al., JINST 14, P04006 (2019). https://doi.org/10.1088/1748-0221/14/04/P04006. arXiv:1903.01360 [hep-ex]

    Article  Google Scholar 

  48. I. Bird, Ann. Rev. Nucl. Part. Sci. 61, 99 (2011). https://doi.org/10.1146/annurev-nucl-102010-130059

    Article  ADS  Google Scholar 

  49. LHCb (2021), https://cds.cern.ch/record/2773174

  50. LHCb collaboration (R. Aaij et al.), Eur. Phys. J. C72, 2022 (2012)https://doi.org/10.1140/epjc/s10052-012-2022-1, arXiv:1202.4979 [hep-ex]

  51. V. Loncar et al., Mach. Learn. Sci. Tech. 2, 015001 (2021). https://doi.org/10.1088/2632-2153/aba042. arXiv:2003.06308 [cs.LG]

    Article  Google Scholar 

  52. LHCb collaboration (R. Aaij et al.), Comput. Softw. Big Sci., (2022)https://doi.org/10.1007/s41781-021-00070-2, arXiv:2105.04031 [physics.ins-det]

  53. R. Aaij, D. H. Cámpora Pérez, T. Colombo, C. Fitzpatrick, V. V. Gligorov, A. Hennequin, N. Neufeld, N. Nolte, R. Schwemmer and D. Vom Bruch, EPJ Web Conf.251, 04009 (2021)https://doi.org/10.1051/epjconf/202125104009, arXiv:2106.07701 [physics.ins-det]

  54. European strategy group collaboration, 2020 Update of the European Strategy for Particle Physics (CERN Council, Geneva, 2020)

    Google Scholar 

  55. R. Brenner, C. Leonidopoulos, Eur. Phys. J. Plus 136, 1198 (2021). https://doi.org/10.1140/epjp/s13360-021-02155-8. arXiv:2111.04168 [physics.ins-det]

    Article  Google Scholar 

  56. FCC collaboration (A. Abada et al.), Eur. Phys. J. ST228, 755 (2019), https://doi.org/10.1140/epjst/e2019-900087-0

  57. C. Accettura et al. (2023), arXiv:2303.08533 [physics.acc-ph]

Download references

Acknowledgements

VVG acknowledges support by the European Research Council under Grant Agreement number 724777 “RECEPT”. The authors would like to thank Alessandro Cerri (University of Sussex) for the kind permission to use Fig. 1 in this article.

Author information

Authors and Affiliations

  1. LPNHE, CNRS/IN2P3, Sorbonne Université, Paris Diderot Sorbonne Paris Cité, Paris, France

    V. V. Gligorov

  2. Vinca Institute of Nuclear Sciences - National Institute of the Republic of Serbia, University of Belgrade, M. Petrovica Alasa 12-14, Belgrade, Serbia

    V. Reković

Authors
  1. V. V. Gligorov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. Reković
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. V. Gligorov.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gligorov, V.V., Reković, V. Review of real-time data processing for collider experiments. Eur. Phys. J. Plus 138, 1005 (2023). https://doi.org/10.1140/epjp/s13360-023-04599-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1140/epjp/s13360-023-04599-6

Search

Navigation