Abstract
Considering the increased hard disk capacity on desktop PCs, we examine, by the modeling and simulation technique, the feasibility of exploiting the idle computational storage in a large Cluster of Workstations (COW). The model built is architecturally based on the Hadoop Distributed File System (HDFS) and was implemented in the CPN Tools using the Coloured Petri Nets combined with the CPN ML programming language. To characterize the workstations’ availability in the model, a statistical study was realized by collecting data from computer laboratories in our academic institution over a period of 40 days. From the simulation results, we propose a small modification in the source code of HDFS and a specific number of replicas in order to achieve a reliable service for writing and reading files despite the random failures due to the turning on and off of the computers in a COW with hundreds of machines.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bolosky, W.J., Douceur, J.R., Ely, D., Theimer, M.: Feasibility of a serverless distributed file system deployed on an existing set of desktop pcs. SIGMETRICS Perform. Eval. Rev. 28(1), 34–43 (2000)
Anderson, D.P., Fedak, G.: The computational and storage potential of volunteer computing. In: Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid. CCGRID 2006, pp. 73–80. IEEE Computer Society, Washington, DC (2006)
Levy, E., Silberschatz, A.: Distributed file systems: concepts and examples. ACM Comput. Surv. 22(4), 321–374 (1990)
Thanh, T., Mohan, S., Choi, E., Kim, S., Kim, P.: A taxonomy and survey on distributed file systems. In: Fourth International Conference on Networked Computing and Advanced Information Management, NCM 2008, vol. 1, pp. 144–149 (2008)
Bai, S., Wu, K.: The performance study on several distributed file systems. In: 2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), pp. 226–229 (2011)
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–10 (2010)
Borthakur, D., Gray, J., Sarma, J.S., Muthukkaruppan, K., Spiegelberg, N., Kuang, H., Ranganathan, K., Molkov, D., Menon, A., Rash, S., Schmidt, R., Aiyer, A.: Apache hadoop goes realtime at facebook. In: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. SIGMOD 2011, pp. 1071–1080. ACM, New York (2011)
Novaes, R.C., Roisenberg, P., Scheer, R., Northfleet, C., Jornada, J.H., Cirne, W.: Non-dedicated distributed environment: A solution for safe and continuous exploitation of idle cycles. In: Proceedings of the Workshop on Adaptive Grid Middleware, pp. 107–115 (2003)
Jensen, K.: An introduction to the practical use of coloured petri nets. In: Reisig, W., Rozenberg, G. (eds.) APN 1998. LNCS, vol. 1492, pp. 237–292. Springer, Heidelberg (1998)
Kristensen, L.M., Christensen, S., Jensen, K.: The practitioner’s guide to coloured petri nets. International Journal on Software Tools for Technology Transfer 2, 98–132 (1998)
Jensen, K.: An introduction to the theoretical aspects of coloured petri nets. In: de Bakker, J.W., de Roever, W.-P., Rozenberg, G. (eds.) REX 1993. LNCS, vol. 803, pp. 230–272. Springer, Heidelberg (1994)
Vinter Ratzer, A., Wells, L., Lassen, H.M., Laursen, M., Qvortrup, J.F., Stissing, M.S., Westergaard, M., Christensen, S., Jensen, K.: CPN tools for editing, simulating, and analysing coloured petri nets. In: van der Aalst, W.M.P., Best, E. (eds.) ICATPN 2003. LNCS, vol. 2679, pp. 450–462. Springer, Heidelberg (2003)
Jensen, K., Kristensen, L.M., Wells, L.: Coloured petri nets and cpn tools for modelling and validation of concurrent systems. International Journal on Software Tools for Technology Transfer. 2007 (2007)
Wells, L.: Performance analysis using cpn tools (2006)
Borthakur, D.: The Hadoop Distributed File System: Architecture and Design. The Apache Software Foundation (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Aguilera-Mendoza, L., Llorente-Quesada, M.T. (2013). Modeling and Simulation of Hadoop Distributed File System in a Cluster of Workstations. In: Cuzzocrea, A., Maabout, S. (eds) Model and Data Engineering. MEDI 2013. Lecture Notes in Computer Science, vol 8216. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41366-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-41366-7_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41365-0
Online ISBN: 978-3-642-41366-7
eBook Packages: Computer ScienceComputer Science (R0)