Advertisement

An Overview of the Sirocco Parallel Storage System

  • Matthew L. Curry
  • H. Lee Ward
  • Geoff Danielson
  • Jay Lofstead
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9945)

Abstract

Sirocco is a massively parallel, high performance storage system that breaks from the classical Zebra-style file system design paradigm. Its architecture is inspired by peer-to-peer and victim-cache architectures, and emphasizes client-to-client coordination, low server-side coupling, and free data movement and placement. By leveraging these ideas, Sirocco natively supports automatic migration between several media types, including RAM, flash, disk, and archival storage.

Sirocco provides advanced storage interfaces, enabling clients to efficiently use key-value storage or block-based storage through a single interface. It also provides several levels of transactional data updates, up to and including ACID-compliant updates across several objects. Further support is provided for concurrency control, enabling greater performance during safe concurrent modification.

By pioneering these and other techniques, Sirocco is well-poised to fulfill a need for a massively scalable, write-optimized storage system. This paper provides an overview of Sirocco’s current system design.

Keywords

Parallel file systems High performance computing I/O 

References

  1. 1.
    Hartman, J.H., Ousterhout, J.K.: The Zebra striped network file system. ACM Trans. Comput. Syst. 13(3), 274–310 (1995). http://doi.acm.org/10.1145/210126.210131 CrossRefGoogle Scholar
  2. 2.
    Oldfield, R., Ward, L., Riesen, R., Maccabe, A., Widener, P., Kordenbrock, T.: Lightweight I/O for scientific applications. In: 2006 IEEE International Conference on in Cluster Computing, pp. 1–11, September 2006Google Scholar
  3. 3.
  4. 4.
    Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. In: SIGCOMM 2001, pp. 149–160 (2001)Google Scholar
  5. 5.
    Timpanaro, J., Cholez, T., Chrisment, I., Festor, O.: Bittorrent’s mainline DHT security assessment. In: 2011 4th IFIP International Conference on New Technologies, Mobility and Security (NTMS), pp. 1–5, February 2011Google Scholar
  6. 6.
    Feldman, M., Papadimitriou, C., Chuang, J., Stoica, I.: Free-riding and whitewashing in peer-to-peer systems. IEEE J. Sel. Areas Commun. 24(5), 1010–1019 (2006)CrossRefGoogle Scholar
  7. 7.
    Karakoyunlu, C., Kimpe, D., Carns, P., Harms, K., Ross, R., Ward, L.: Toward a unified object storage foundation for scalable storage systems. In: Proceedings of the 5th Workshop on Interfaces and Architectures for Scientific Data Storage (IASDS 2013) (2013)Google Scholar
  8. 8.
    Ching, A., Choudhary, A., Liao, W.-K., Ross, R., Gropp, W.: Noncontiguous I/O through PVFS. In: Proceedings of 2002 IEEE International Conference on Cluster Computing, 2002, pp. 405–414 (2002)Google Scholar
  9. 9.
    Sun, Z., Skjellum, A., Ward, L., Curry, M.L.: A lightweight data location service for nondeterministic exascale storage systems. Trans. Storage 10(3), 1–22 (2014). http://doi.acm.org/10.1145/2629451 CrossRefGoogle Scholar
  10. 10.
    Kung, H.T., Robinson, J.T.: On optimistic methods for concurrency control. ACM Trans. Database Syst. 6(2), 213–226 (1981). http://doi.acm.org/10.1145/319566.319567 CrossRefGoogle Scholar
  11. 11.
    Carns, P., Harms, K., Kimpe, D., Wozniak, J., Ross, R., Ward, L., Curry, M., Klundt, R., Danielson, G., Karakoyunlu, C., Chandy, J., Settlemyer, B., Gropp, W.: A case for optimistic coordination in HPC storage systems,. In: 2012 SC Companion in High Performance Computing, Networking, Storage and Analysis (SCC), pp. 48–53, November 2012Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Matthew L. Curry
    • 1
  • H. Lee Ward
    • 1
  • Geoff Danielson
    • 2
  • Jay Lofstead
    • 1
  1. 1.Sandia National LaboratoriesAlbuquerqueUSA
  2. 2.Hewlett Packard EnterprisePalo AltoUSA

Personalised recommendations