Abstract
The paper discusses the XP-COM hybrid software package. XP-COM is a combination of X-COM and Pyramid, two well-known packages for parallelization by data. The parallelization overhead introduced by X-COM and Pyramid has been presented in the previous studies. The X-COM software showed the best results in iterating over string values read from a file. The Pyramid software showed the least overhead when iterating through value combinations of several parameters. The authors used X-COM data transfer infrastructure as a basis for the XP-COM package development. The parameter value combination enumeration mechanism of the Pyramid software was used as the computational work separation procedure. The resulting XP-COM hybrid software was discovered to show better performance than both X-COM and Pyramid.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Tessema, M., Dunren, C.: Survey and taxonomy of volunteer computing. ACM Comput. Surv. 52(3), 1–35. ACM, New York (2019). https://doi.org/10.1145/3320073
Binnie, C.: Password cracking with hashcat. In: Linux Server Security (2016). https://doi.org/10.1002/9781119283096.ch9
Baranov, A., Kiselev, E., Chernyaev, D.: Experimental comparison of performance and fault tolerance of software packages Pyramid, X-COM, and BOINC. Commun. Comput. Inf. Sci. 687, 279–290 (2016). https://doi.org/10.1007/978-3-319-55669-7_22
Baranov, A., Kiselev, A., Kiselev, E., Korneev, V., Semenov, D.: The software package Pyramid for parallel computations arrangement with parallelization by data. In: Scientific Services & Internet: Supercomputing Centers and Applications: Proceedings of the International Supercomputing Conference, Novorossiysk, Russia, 20–25 September 2010, pp. 299–302. Publishing of Lomonosov Moscow State University, Moscow (2010). (in Russian)
Anderson, D.P.: BOINC: a system for public-resource computing and storage. In: 5th IEEE/ACM International Workshop on Grid Computing, pp. 4–10. IEEE/ACM, Pittsburgh (2004). https://doi.org/10.1109/GRID.2004.14
Filamofitsky, M.: The system X-Com for metacomputing support: architecture and tech-nology (in Russian). Numer. Methods Program. 5, 123–137. Publishing of the Research Computing Center of Lomonosov Moscow State University, Moscow (2004)
X-Com: Distributed computing software. http://x-com.parallel.ru/sites/x-com.parallel.ru/files/download/X-Com_tutorial.pdf. Accessed 11 Nov 2019
Baranov, A.V., Savin, G.I., Shabanov, B.M., Shitik, A.S., Svadkovskiy, I.A., Telegin, P.N.: Methods of jobs containerization for supercomputer workload managers. Lobachevskii J. Math. 40(5), 525–534 (2019). https://doi.org/10.1134/S1995080219050020
Savin, G.I., Shabanov, B.M., Telegin, P.N., Baranov, A.V.: Joint supercomputer center of the Russian Academy of Sciences: present and future. Lobachevskii J. Math. 40(11), 1853–1862 (2019). https://doi.org/10.1134/S1995080219110271
Voevodin, V., Sobolev, S.: X-COM technology for distributed computing: abilities, problems, perspectives. Mech. Control Inf. IKI Semin. 5, 183–191 (2011). (in Russian)
Khritankov, A.: Performance analysis of distributed computing systems using the X-Com system as an example. In: Scientific Services & Internet: Scalability. Parallelism, Efficiency: Proceedings of the International Supercomputing Conference, Novorossiysk, Russia, 21–26 September 2009, pp. 46–52. Publishing of Lomonosov Moscow State University, Moscow (2009). (in Russian)
Huang, S., Huang, J., Dai, J., Xie, T., Huang, B.: The HiBench benchmark suite: characterization of the MapReduce-based data analysis. In: Agrawal, D., Candan, K.S., Li, W.-S. (eds.) New Frontiers in Information and Software as Services. LNBIP, vol. 74, pp. 209–228. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19294-4_9
Zhang, Z., Cherkasova, L., Loo, B.T.: Parameterizable benchmarking framework for designing a MapReduce performance model. Concurrency Comput. Pract. Experience 26, 2005–2026 (2014). https://doi.org/10.1002/cpe.3229
Costa, F., Silva, L., Dahlin, M.: Volunteer cloud computing: MapReduce over the internet. In: 2011 IEEE International Parallel & Distributed Processing Symposium, pp. 1850–1857 (2011). http://hpcolsi.dei.uc.pt/hpcolsi/publications/mapreduce-pcgrid2011.pdf. Accessed 21 Nov 2019
Naegele, T.: Mapreduce framework performance comparison (2013). http://www.cs.ru.nl/bachelorscripties/2013/Thomas_Naegele___4031253___MapReduce_Framework_Performance_Comparison.pdf. Accessed 21 Nov 2019
Pavlo, A., Paulson, E., Rasin, A., et al.: A comparison of approaches to large-scale data analysis. https://doi.org/10.1145/1559845.1559865. Accessed 21 Nov 2019
Sobolev, S.: Hierarchical methods for improving the scalability and efficiency of distributed computing in the X-Com metacomputing system. In: Parallel Computational Technologies (PCT 2010), pp. 346–352 (2010). (in Russian)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Baranov, A., Fedotov, A., Aladyshev, O., Kiselev, E., Ovsyannikov, A. (2020). XP-COM Hybrid Software Package for Parallelization by Data. In: Sokolinsky, L., Zymbler, M. (eds) Parallel Computational Technologies. PCT 2020. Communications in Computer and Information Science, vol 1263. Springer, Cham. https://doi.org/10.1007/978-3-030-55326-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-55326-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55325-8
Online ISBN: 978-3-030-55326-5
eBook Packages: Computer ScienceComputer Science (R0)