Databases in High Energy Physics: A Critical Review

  • Jamie ShiersEmail author
Part of the The Frontiers Collection book series (FRONTCOLL)


This chapter traces the history of databases in High Energy Physics (HEP) over the past quarter century. It does not attempt to describe in detail all database applications, focusing primarily on their use related to physics data processing. In particular, the use of databases in the accelerator sector, as well as for administrative applications–extensively used by today’s large-scale collaborations–are only covered in passing.


Large Hadron Collider Application Program Interface High Energy Physics Grid Service Object Database 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Numerous people have contributed to the story of Databases in HEP, including the many who worked on various aspects of the CERN Program Library and the ZEBRA, FATMEN and HEPDB packages. Members of the PASS and RD45 projects, together with the ROOT and POOL and related projects as well as all those who have contributed to database deployment at the various HEP sites throughout the years also played key roles in this story. Particular thanks are due to Harry Renshall for his careful reading of this and many other documents during the past 25 years, together with frequent guidance and mentoring. Finally, the “Grid Guys & Gals” (3G), working on the various middleware components and associated services, for deploying a host of database applications and in anticipation of the many years of LHC data taking to come. LHC data archive anyone?


  1. 1.
    Databases and bookkeeping for HEP experiments, ECFA Working Group 11, ECFA/83/78 (Sept 83)Google Scholar
  2. 2.
  3. 3.
    KAPACK - Random Access I/O Using Keywords, CERN Program Library Entry Z303Google Scholar
  4. 4.
    Mount, R.P.: Database systems for HEP Experiments. Comput. Phys. Comm. 45, 299–310 (1987)CrossRefGoogle Scholar
  5. 5.
    The L3 database system, Nuclear Instruments and Methods in Physics Research A309, 318–330 (1991)Google Scholar
  6. 6.
    ZEBRA Data Structure Management Package, CERN Program Library Entry Q100Google Scholar
  7. 7.
    Computing at CERN in the 1990s, July 1989Google Scholar
  8. 8.
    The ALEPH DAta MOdel - ADAMO. In: The proceedings of [14].
  9. 9.
    ALEPH bookkeeping and SCANBOOK, JacquesGoogle Scholar
  10. 10.
    FATMEN - File and Tape Management Package, CERN Program Library Entry Q123Google Scholar
  11. 11.
    HEPDB - Database Management Package, CERN Program Library Entry Q180Google Scholar
  12. 12.
    Isnard, C., et al.: CHEOPS: A batch data dissemination system based on the OLYMPUS satellite. In: Proceeding of the International Conference on the Results of the OLYMPUS Utilisation Programme, Sevilla, 20-22 April, 1993Google Scholar
  13. 13.
  14. 14.
    Data Structures for Particle Physics Experiments: Evolution or Revolution? ISBN 981-02-0641-0Google Scholar
  15. 15.
    Goossens, M., Schaile, O.: DZDOC – Bank Documentation Tool. In: ZEBRA, CERN Program Library Long Writeups Q100/Q101Google Scholar
  16. 16.
    Kunz, P., Word, G.: The Cheetah Data Management System, SLAC-PUB-5450 March 1991 (E/I)Google Scholar
  17. 17.
    Barone, L.M.: Database Management and Distributed Data in High Energy Physics: Present and Future. In: The Proceedings of CHEP “91. ISBN 4-946443-09-6Google Scholar
  18. 18.
    Database Computing in High Energy Physics. In: The Proceedings of CHEP91, ISBN 4-946443-09-6Google Scholar
  19. 19.
    HEPDB Package, CERN Program Library Entry Q180Google Scholar
  20. 20.
  21. 21.
    Linder, B., Mount, R., Shiers, J.: Databases for High Energy Physics. In: The proceedings of CHEP92. ISBN 0007-8328Google Scholar
  22. 22.
    Bos, K.: The Moose project. Comput. Phys. Comm. 110(1-3), 160–163 (1998)CrossRefGoogle Scholar
  23. 23.
  24. 24.
  25. 25.
  26. 26.
    ROOT - an Object-Oriented Data Analysis Framework - see
  27. 27.
    May, E., et al.: The PASS Project: Database Computing for the SSC. Proposal to the DOE HPCC Initiative, Argonne National Lab, 1991, unpublishedGoogle Scholar
  28. 28.
  29. 29.
  30. 30.
    POOL - Persistency Framework - see
  31. 31.
    Malon, D., May, E.: Critical Database Technologies for High Energy Physics. In: The Proceedings of VLDB97, available from vldb.orgGoogle Scholar
  32. 32.
    Sphicas, P.: Conference summary talk, CHEP 2000, Padua, ItalyGoogle Scholar
  33. 33.
    Objectivity Data Migration. In: The Proceedings of CHEP 2003, La Jolla, CaliforniaGoogle Scholar
  34. 34.
    Nowak, M., Nienartowicz, K., Valassi, A., Lbeck, M., Geppert, D.: Objectivity Data Migration, Computing in High Energy and Nuclear Physics, La Jolla, California. March 2003 24–28Google Scholar
  35. 35.
    Becla, J., Wang, D.: Lessons learnt from Managing a Petabyte, SLAC. In: The Proceedings of the 2006 CIDR ConferenceGoogle Scholar
  36. 36.
    Girone, M.: Database services for Physics at CERN. In: The Proceedings of CHEP07Google Scholar
  37. 37.
  38. 38.
  39. 39.
    Computing at CERN in the LEP era, CERN, May 1983Google Scholar
  40. 40.
    Martelli, B., et al.: LHCb experience with LFC database replication. In: The Proceedings of CHEP07Google Scholar
  41. 41.
    The ALICE Collaboration, Technical Design Report of the Computing, CERN-LHCC-2005-01, ALICE TDR 012, 15 June 2005Google Scholar
  42. 42.
  43. 43.
    Lueking, L., et al.: FroNtier: High Performance Database Access Using Standard Web Components in a Scalable Multi-Tier Architecture. In: Proceedings of CHEP04, Interlaken (September 2004)Google Scholar
  44. 44.
    Oracle streams - see
  45. 45.
  46. 46.
    Düllmann, D., et al.: Production experience with Distributed Deployment of Databases for the LHC Computing Grid. In the Proceedings of CHEP07Google Scholar
  47. 47.
    Gaponenko, I., et al.: Using Multiple Persistent Technologies in the Conditions Database of BaBar. In: Proceedings of CHEP06, February 2006Google Scholar
  48. 48.
    Valassi, A., et al.: COOL Development and Deployment: Status and Plans. In: The Proceedings of the CHEP 2006 ConferenceGoogle Scholar
  49. 49.
    Papadopoulos, I., et al.: CORAL, A Software System for Vendor-neutral Access To Relation Databases. In: The Proceedings of the CHEP 2006 conferenceGoogle Scholar
  50. 50.
    Abadie, L., et al.: Grid-Enabled Standards-based Data Management, submitted to the 24th IEEE Conference on Mass Storage Systems and Technologies. Boucrot, in “The ALEPH Experience”, p 128Google Scholar
  51. 51.
    In Proceedings of Symposia On Advanced Computer Programs for Digital Computers, sponsored by ONR. Republished in Annals of the History of Computing, Oct. 1983, pp. 350–361. Reprinted at ICSE’87, Monterey, California, USA, March 30-April 2, 1987, June 1956Google Scholar
  52. 52.
    Girone, M., Shiers, J.: WLCG Operations and the First Prolonged LHC Run to appear in the proceedings of CHEP 2010Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.CERNGenevaSwitzerland

Personalised recommendations