Consistency Models for Global Scalable Data Access Services

  • Michał WrzeszczEmail author
  • Darin Nikolow
  • Tomasz Lichoń
  • Rafał Słota
  • Łukasz Dutka
  • Renata G. Słota
  • Jacek Kitowski
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10777)


Developing and deploying a global and scalable data access service is a challenging task. We assume that the globalization is achieved by creating and maintaining appropriate metadata while the scalability is achieved by limiting the number of entities taking part in keeping the metadata consistency. In this paper, we present different consistency and synchronization models for various metadata types chosen for implementation of global and scalable data access service.


Data consistency model Global data access Distributed data storage 



This research is supported partly by the European Regional Development Fund program no. POIG.02.03.00-12-137/13 as part of the PLGrid Core. R. G. Słota, D. Nikolow and J. Kitowski acknowledge AGH-UST statutory Grant no. Support by IndigoDC project no. RIA 653549 is also acknowledged by Ł. Dutka, M. Wrzeszcz. T. Lichoń acknowledges Polish Ministry of Science and Higher Education under AGH University of Science and Technology, Faculty of Computer Science, Electronics and Telecommunications statutory project no., and owes special thanks to ACK Cyfronet AGH computing center for providing computing and storage infrastructure for this research.


  1. 1.
    Baud, J.P.B., Casey, J., Lemaitre, S., Nicholson, C., Smith, D., Stewart, G.: LCG data management: from EDG to EGEE. In: UK e-Science All Hands Meeting, Nottingham, UK (2005).
  2. 2.
    Drago, I., Mellia, M., Munafo, M., Sperotto, A., Sadre, R., Pras, A.: Inside dropbox: understanding personal cloud storage services. In: Proceedings of the 2012 ACM Conference on Internet Measurement, IMC 2012, pp. 481–494. ACM, New York (2012)Google Scholar
  3. 3.
    Dutka, Ł., Wrzeszcz, M., Lichoń, T., Słota, R., Zemek, K., Trzepla, K., Opiola, Ł., Słota, R., Kitowski, J.: Onedata - a step forward towards globalization of data access for computing infrastructures. In: Proceedings of the International Conference on Computational Science, ICCS 2015, Computational Science at the Gates of Nature, Reykjavík, Iceland, pp. 2843–2847, 1–3 June 2015Google Scholar
  4. 4.
    Hünich, D., Müller-Pfefferkorn, R.: Managing large datasets with iRODS - a performance analysis. In: Proceedings of the 2010 International Multiconference on Computer Science and Information Technology (IMCSIT), pp. 647–654. IEEE (2010)Google Scholar
  5. 5.
    Pacheco, L., Halalai, R., Schiavoni, V., Pedone, F., Riviere, E., Felber, P.: GlobalFS: a strongly consistent multi-site file system. In: Proceedings of the 35th IEEE Symposium on Reliable Distributed Systems, SRDS 2016, Budapest, Hungary, 26–29 September 2016, pp. 147–156. IEEE Computer Society (2016).
  6. 6.
    Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), MSST 2010, pp. 1–10. IEEE Computer Society, Washington (2010).
  7. 7.
    Slota, R., Nikolow, D., Skalkowski, K., Kitowski, J.: Management of data access with quality of service in PL-Grid environment. Comput. Inf. 31(2), 463–479 (2012). Google Scholar
  8. 8.
    Słota, R., Nikolow, D., Skitał, Ł., Kitowski, J.: Implementation of replication methods in the grid environment. In: Sloot, P.M.A., Hoekstra, A.G., Priol, T., Reinefeld, A., Bubak, M. (eds.) EGC 2005. LNCS, vol. 3470, pp. 474–484. Springer, Heidelberg (2005). CrossRefGoogle Scholar
  9. 9.
    Słota, R., Skitał, Ł., Nikolow, D., Kitowski, J.: Algorithms for automatic data replication in grid environment. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Waśniewski, J. (eds.) PPAM 2005. LNCS, vol. 3911, pp. 707–714. Springer, Heidelberg (2006). CrossRefGoogle Scholar
  10. 10.
    Thain, D., Livny, M.: Parrot: an application environment for data-intensive computing. Scalable Comput. Pract. Exp. 6(3), 9–18 (2005)Google Scholar
  11. 11.
    Viotti, P., Vukolić, M.: Consistency in non-transactional distributed storage systems. ACM Comput. Surv. 49(1), 19:1–19:34 (2016). CrossRefGoogle Scholar
  12. 12.
    Wang, F., Oral, S., Shipman, G., Drokin, O., Wang, T., Huang, I.: Understanding Lustre Filesystem Internals. Technical report ORNL/TM-2009/117, Oak Ridge National Lab., National Center for Computational Sciences (2009)Google Scholar
  13. 13.
    Weil, S.A., Brandt, S.A., Miller, E.L., Long, D.D.E., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI), pp. 307–320 (2006)Google Scholar
  14. 14.
    Weil, S.A., Brandt, S.A., Miller, E.L., Maltzahn, C.: CRUSH: controlled, scalable, decentralized placement of replicated data. In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC 2006, Tampa. ACM, New York (2006).
  15. 15.
    Wrzeszcz, M., Trzepla, K., Słota, R., Zemek, K., Lichoń, T., Opioła, Ł., Nikolow, D., Dutka, Ł., Słota, R., Kitowski, J.: Metadata organization and management for globalization of data access with Onedata. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds.) PPAM 2015. LNCS, vol. 9573, pp. 312–321. Springer, Cham (2016). CrossRefGoogle Scholar
  16. 16.
    Zhang, J., Wu, Y., Chung, Y.C.: PROAR: a weak consistency model for Ceph. In: 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS), pp. 347–353 (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Michał Wrzeszcz
    • 1
    • 2
    Email author
  • Darin Nikolow
    • 2
  • Tomasz Lichoń
    • 1
    • 2
  • Rafał Słota
    • 2
  • Łukasz Dutka
    • 1
  • Renata G. Słota
    • 2
  • Jacek Kitowski
    • 1
    • 2
  1. 1.Academic Computer Centre Cyfronet AGHAGH University of Science and TechnologyKrakowPoland
  2. 2.Department of Computer Science, Faculty of Computer Science, Electronics and TelecommunicationsAGH University of Science and TechnologyKrakowPoland

Personalised recommendations