Abstract
Scientific data intensive applications requiring simultaneous use of large amounts of computing resources are becoming quite common. Properties of applications coming from different scientific domains as well as their requirements to the computing resources are varying largely. Many scientific communities have access to different types of computing resources. Often their workflows can benefit from a combination of High Throughput (HTC) and High Performance (HPC) computing centers, cloud or volunteer computing power. However, all these resources have different user interfaces and access mechanisms, which are making their combined usage difficult for the users. This problem is addressed by projects developing software for integration of various computing centers into a single coherent infrastructure, the so-called interware. One of such software toolkits is the DIRAC interware. This product was very successful to solve problems of large High Energy Physics experiments and was reworked to offer a general-purpose solution suitable for other scientific domains. Services based on the DIRAC interware are now proposed to users of several distributed computing. One of these services is deployed at Joint Institute for Nuclear Research, Dubna. It aims at integration of computing resources of several grid and supercomputer centers as well as cloud providers. An overview of the DIRAC interware and its use for creating and operating of a hybrid distributed computing system at JINR is presented in this article.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tsaregorodtsev, A., Brook, N., Casajus Ramo, A., et al.: DIRAC3: the new generation of the LHCb grid software. J. Phys: Conf. Ser. 219, 6 (2010). doi:10.1088/1742-6596/219/6/062029
Klimentov, A., Buncic, P., De, K., et al.: Next generation workload management system for big data on heterogeneous distributed computing. J. Phys: Conf. Ser. 608, 1 (2015). doi:10.1088/1742-6596/608/1/012040
Bagnasco, S., Betev, L., Buncic, P., et al.: AliEn: ALICE environment on the grid. J. Phys: Conf. Ser. 119, 6 (2008). doi:10.1088/1742-6596/119/6/062012
Casajus Ramo, A., Graciani, R., Tsaregorodtsev, A.: DIRAC pilot framework and the DIRAC workload management system. J. Phys: Conf. Ser. 219, 6 (2010). doi:10.1088/1742-6596/219/6/062049
OpenScience Grid. https://www.opensciencegrid.org/
ARC project. http://www.nordugrid.org/arc/
Smith, A., Tsaregorodtsev, A.: DIRAC: data production management. J. Phys: Conf. Ser. 119, 6 (2008). doi:10.1088/1742-6596/119/6/062046
Gfal2 Project. https://dmc.web.cern.ch/projects-tags/gfal-2
Poss, S., Tsaregorodtsev, A.: DIRAC file replica and metadata catalog. J. Phys: Conf. Ser. 396, 3 (2012). doi:10.1088/1742-6596/396/3/032108
Stagni, F., Charpentier, P.: The LHCb DIRAC-based production and data management operations systems. J. Phys: Conf. Ser. 368, 1 (2012). doi:10.1088/1742-6596/368/1/012010
Zhang, X.M., Pelevanyuk, I., Korenkov, V., et al.: Design and operation of the BES-III distributed computing system. Procedia Comput. Sci. 66, 619–624 (2015). doi:10.1016/j.procs.2015.11.070
Kuhr, T.: Computing at Belle II. J. Phys: Conf. Ser. 396, 3 (2015). doi:10.1088/1742-6596/396/3/032063
Arrabito, L., Barbier, C., Graciani Diaz, R., et al.: Application of the DIRAC framework in CTA: first evaluation. J. Phys: Conf. Ser. 396, 3 (2015). doi:10.1088/1742-6596/396/3/032007
France-Grilles DIRAC portal. http://dirac.france-grilles.in2p3.fr
Korenkov, V., Pelevanyuk, I., Tsaregorodtsev, A., Zrelov, P.: Accessing distributed computing resources by scientific communities using DIRAC services. In: The XVIII International Conference on Data Analytics & Management in Data Intensive Domains (DAMDID/RCDL 2016), Ershovo, Moscow Region, Russia, CEUR Workshop Proceedings, vol. 1752, pp. 110–115 (2016)
DIRAC4EGI service portal. http://dirac.egi.eu
Trubnikov, G., Agapov, N., Alexandrov, V., et al.: Project of the nuclotron-based ion collider facility (NICA) at JINR. In: EPAC 2008 Conference Proceedings, Genoa, Italy (2008)
Wischnewski, R.: The Baikal neutrino telescope – results and plans. Int. J. Modern Phys. A 20(29), 6932–6936 (2005). doi:10.1142/S0217751X0503051X
Zweber, P.: Charm factories: present and future. AIP Conf. Proc. 1182(1), 406–409 (2009)
Alexandrov, E., Belyakov, D., Matveyev, M., et al.: Research of acceleration of calculation in solving scientific problems on the heterogeneous cluster HybriLIT. RUDN J. Math. Inf. Sci. Phys. 4, 20–27 (2015)
Lobachevsky supercomputer. https://www.top500.org/system/178472
Mesocentre of Aix-Marseille University. http://mesocentre.univ-amu.fr/en
DIRAC Project. http://diracgrid.org
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Gergel, V., Korenkov, V., Pelevanyuk, I., Sapunov, M., Tsaregorodtsev, A., Zrelov, P. (2017). Hybrid Distributed Computing Service Based on the DIRAC Interware. In: Kalinichenko, L., Kuznetsov, S., Manolopoulos, Y. (eds) Data Analytics and Management in Data Intensive Domains. DAMDID/RCDL 2016. Communications in Computer and Information Science, vol 706. Springer, Cham. https://doi.org/10.1007/978-3-319-57135-5_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-57135-5_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57134-8
Online ISBN: 978-3-319-57135-5
eBook Packages: Computer ScienceComputer Science (R0)