Abstract
The scale of large processing tasks being run within the scientific Grid and Cloud environments have introduced a need for stability guarantees from geographically spanning resources, to ensure that failures are detected and handled preemptively. Performance inefficiencies within stacked service environments are a challenge to detect, where failures stem from a multitude of causes, often requiring expert intervention. Online reporting and classification of performance fluctuations can aid experts, central services, and users to target service areas where optimizations can be introduced. This paper describes an approach for modeling performance states for production tasks running within the ALICE Grid. We first provide an overview for the ALICE data and software workflow, focusing on the production job computational profile. Data center event state is then developed, based on data center job, computing, storage, and user behavior. With across site analysis, we then train groups to classify service domain states. Our approach is able to detect periods of service instability and the affected service domains. This can guide users, central, and data center experts to take action in advance of service failure effects.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
ALICE Collaboration: Our Mission - http://aliceinfo.cern.ch/node/3935.
- 3.
Geant4 website: http://geant4.cern.ch/.
- 4.
ROOT Data Analysis Framework: https://root.cern.ch/.
References
Aerni, S., Farinas, H., Muralidhar, G.: Digital drivers in the age of massive datasets. XRDS 21(4), 60–66 (2015)
Bal, H., Casanova, H., Dongarra, J., Matsuoka, S.: Application-level tools. In: Kesselman, I.F.C. (ed.) The Grid 2, The Morgan Kaufmann Series in Computer Architecture and Design, 2 edn., pp. 463–489. Morgan Kaufmann, Burlington (2004)
Betev, L., Gheata, A., Gheata, M., Grigoras, C., Hristov, P.: Performance optimisations for distributed analysis in ALICE. J. Phys. Conf. Ser. 523, 012014 (2014)
Bird, I., Buncic, P., Carminati, F., Cattaneo, M., Clarke, P., Fisk, I., Girone, M., Harvey, J., Kersevan, B., Mato, P., Mount, R., Panzer-Steindel, B.: Update of the computing models of the WLCG and the LHC experiments. Technical report CERN-LHCC-2014-014. LCG-TDR-002, CERN, Geneva, April 2014
Buncic, P., Krzewicki, M., Vande Vyvre, P.: Technical design report for the upgrade of the online-offline computing system. Technical report CERN-LHCC-2015-006. ALICE-TDR-019, CERN, Geneva, April 2015
Cardwell, N., Cheng, Y., Gunn, C.S., Yeganeh, S.H., Jacobson, V.: BBR: congestion-based congestion control. Commun. ACM 60(2), 58–66 (2017)
Carminati, F.: A reflection on Software Engineering in HEP. J. Phys: Conf. Ser. 396(5), 052018 (2012)
Carminati, F., Schutz, Y.: Conception, realisation et exploitation du traitement de données de l’expérience ALICE pour la simulation, la reconstruction et l’analyse. Ph.D. thesis, Nantes U. December 2012. Presented 22 February 2013
Date, S.: Should you upload or ship big data to the cloud? Commun. ACM 59(7), 44–51 (2016)
Edwards, C.: Using patient data for personalized cancer treatments. Commun. ACM 57(4), 13–15 (2014)
Flix, J., Accion, A., Acin, V., Acosta, C., Casals, J., Caubet, M., Cruz, R., Delfino, M., Lopez, F., Pacheco, A., Yzquierdo Perez-Calero, A., Planas, E., Porto, M., Rodriguez, B., Sedov, A.: Getting prepared for the LHC Run2: the PIC Tier-1 case. J. Phys. Conf. Ser. 664(5), 052014 (2015)
Foster, I., Kesselman, C.: Chapter 4 - Concepts and architecture. In: Kesselman, C., Foster, I. (eds.) The Grid 2. The Morgan Kaufmann Series in Computer Architecture and Design, 2 edn., pp. 37–63. Morgan Kaufmann, Burlington (2004)
Kennedy, K.: Chapter 25 - languages, compilers, and run-time systems. In: Ian Foster Carl Kesselman, editor, The Grid 2. The Morgan Kaufmann Series in Computer Architecture and Design, 2 edn., pp. 491–512. Morgan Kaufmann, Burlington (2004)
Legrand, I., Newman, H., Voicu, R., Cirstoiu, C., Grigoras, C., Dobre, C., Muraru, A., Costan, A., Dediu, M., Stratan, C.: MonALISA: an agent based, dynamic service system to monitor, control and optimize distributed systems. Comput. Phys. Commun. 180(12), 2472–2498 (2009)
Legrand, I.C., Newman, H.B.: The MONARC toolset for simulating large network-distributed processing systems. In: Proceedings of the 32nd Conference on Winter Simulation, WSC 2000, pp. 1794–1801. Society for Computer Simulation International, San Diego (2000)
Limoncelli, T.A.: The small batches principle. Commun. ACM 59(7), 52–57 (2016)
Mattmann, C.A.: Computing: a vision for data science. Nature 493, 473–475 (2013)
O’Driscoll, A., Daugelaite, J., Sleator, R.D.: ‘Big data’, Hadoop and cloud computing in genomics. J. Biomed. Inform. 46(5), 774–781 (2013)
Rademakers, F.: Evolution of parallel computing in high energy physics, pp. 177–199. Springer, Heidelberg (2012)
Reed, D.A., Dongarra, J.: Exascale computing and Big Data. Commun. ACM 58(7), 56–68 (2015)
Robertson, L.: Computing services for LHC: from clusters to grids, pp. 69–89. Springer, Heidelberg (2012)
Zimmermann, M.: The ALICE analysis train system. J. Phys. Conf. Ser. 608, 012019. 5 p (2015)
Acknowledgments
The authors thank the ALICE collaboration and in particular Latchezar Betev for the valuable comments and suggestions. We also would like to thank Iossif Legrand and Federico Carminati for consultations in regards to monitoring data analytics, performance evaluation and statistical methods respectively.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Datskova, O., Shi, W. (2018). Modeling Instability for Large Scale Processing Tasks Within HEP Distributed Computing Environments. In: Xhafa, F., Caballé, S., Barolli, L. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2017. Lecture Notes on Data Engineering and Communications Technologies, vol 13. Springer, Cham. https://doi.org/10.1007/978-3-319-69835-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-69835-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69834-2
Online ISBN: 978-3-319-69835-9
eBook Packages: EngineeringEngineering (R0)