Abstract
Processing huge amounts of data is currently of concern in various fields of science and commercial data processing, such as pharmaceutical drug development, astronomical probe data processing, security analysis of large amounts of communication data, etc. Generally, centrally administered methods are used, but their employment and operation are very expensive. The aim of this paper is to present a model of high-capacity data processing that is based on the technology of Apache Hadoop with emphasis on use of volunteer host devices with the service distribution via the Internet.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lu, Q., Li, S., Zhang, W., Zhang, L.: A genetic algorithm-based job scheduling model for big data analytics. EURASIP J. Wireless Commun. Netw. 2016(1), 152 (2016)
Hashem, I.A.T., Anuar, N.B., Gani, A., Yaqoob, I., Xia, F., Khan, S.U.: MapReduce: review and open challenges. Scientometrics 109(1), 389–422 (2016)
Kobayashi, K., Kaito, K.: Big data-based deterioration prediction models and infrastructure management: towards assetmetrics. Struct. Infrastruct. Eng. 13(1), 84–93 (2017)
Govindarajan, K., Somasundaram, T.S., Boulanger, D., Kumar, V.S.: Kinshuk: A framework for scheduling and managing big data applications in a distributed infrastructure. In: ICoAC 2015 - 7th International Conference on Advanced Computing, art. no. 7562784 (2016)
Alekseev, A.A., Osipova, V.V., Ivanov, M.A., Klimentov, A., Grigorieva, N.V., Nalamwar, H.S.: Efficient data management tools for the heterogeneous big data warehouse. Phys. Part. Nucl. Lett. 13(5), 689–692 (2016)
Hadoop wiki - faq. http://wiki.apache.org/hadoop/faq
Kranjc, J., Orač, R., Podpečan, V., Lavrač, N., Robnik-Šikonja, M.: ClowdFlows: Online workflows for distributed big data mining. Future Gener. Comput. Syst. 68, 38–58 (2017)
Sobeslav, V., Maresova, P., Krejcar, O., Franca, T.C.C., Kuca, K.: Use of cloud computing in biomedicine. J. Biomol. Struct. Dyn. 34(12), 1–10 (2016). Article in Press
Sobeslav, V., Komarek, A.: Opensource automation in cloud computing. In: Wong, W.E. (ed.) Proceedings of the 4th International Conference on Computer Engineering and Networks, pp. 805–812. Springer, Cham (2015). doi:10.1007/978-3-319-11104-9_93
Bao, X., Xiao, N., Lu, Y., Chen, Z.: A configuration management study to fast massive writing for distributed NoSQL systém. In: IEICE Transactions on Information and Systems, vol. E99D (9), pp. 2269–2282 (2016)
Li, C.-S., Franke, H., Parris, C., Abali, B., Kesavan, M., Chang, V.: Composable architecture for rack scale big data computing. Future Gener. Comput. Syst. 67, 180–193 (2017)
Kobayashi, K., Kaito, K.: Big data-based deterioration prediction models and infrastructure management: towards assetmetrics. Struct. Infrastruct. Eng. 13(1), 84–93 (2017)
Tran, M.C., Nakamura, Y.: Classification of HTTP automated software communication behaviour using NoSql database. In: International Conference on Electronics, Information, and Communications, ICEIC 2016, art. no. 7562957 (2016)
Holik, F., Horalek, J., Neradova, S., Zitta, S., Novak, M.: Methods of deploying security standards in a business environment. In: Proceedings of 25th International Conference Radioelektronika, RADIOELEKTRONIKA 2015, art. no. 7128984, pp. 411–414 (2015)
Holik, F., Horalek, J., Neradova, S., Zitta, S., Marik, O.: The deployment of security information and event management in cloud infrastructure. In: Proceedings of 25th International Conference Radioelektronika, RADIOELEKTRONIKA 2015, art. no. 7128982, pp. 399–404 (2015)
Acknowledgment
This article has been produced in cooperation with Petr Volf, a graduate at FIM of UHK, to whom we hereby give our thanks for extraordinary assistance with proposing the solution using Hadoop in effective distributed computing. We would also like to thank to Ondrej Horning and Ladislav Balik, doctoral students who were helping with the simulation tests in laboratories. This work and the contribution were also supported by project of specific science, Faculty of informatics and Management, University of Hradec Kralove, Czech Republic.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Horalek, J., Soběslav, V. (2017). Analysis and Solution Model of Distributed Computing in Scientific Calculations. In: Younas, M., Awan, I., Holubova, I. (eds) Mobile Web and Intelligent Information Systems. MobiWIS 2017. Lecture Notes in Computer Science(), vol 10486. Springer, Cham. https://doi.org/10.1007/978-3-319-65515-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-65515-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65514-7
Online ISBN: 978-3-319-65515-4
eBook Packages: Computer ScienceComputer Science (R0)