Advertisement

TxForest: A DSL for Concurrent Filestores

  • Jonathan DiLorenzoEmail author
  • Katie Mancini
  • Kathleen Fisher
  • Nate Foster
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11893)

Abstract

Many systems use ad hoc collections of files and directories to store persistent data. For consumers of this data, the process of properly parsing, using, and updating these filestores using conventional APIs is cumbersome and error-prone. Making matters worse, most filestores are too big to fit in memory, so applications must process the data incrementally while managing concurrent accesses by multiple users. This paper presents Transactional Forest (TxForest), which builds on earlier work on Forest to provide a simpler, more powerful API for managing filestores, including a mechanism for managing concurrent accesses using serializable transactions. Under the hood, TxForest implements an optimistic concurrency control scheme using Huet’s zippers to track the data associated with filestores. We formalize TxForest in a core calculus, develop a proof of serializability, and describe our OCaml prototype, which we have used to build several practical applications.

Keywords

Data description languages File systems Ad hoc data Concurrency Transactions Zippers 

Notes

Acknowledgments

The authors wish to thank the anonymous APLAS reviewers for helpful comments on an earlier draft of this paper. Our work is supported in part by the National Science Foundation under grant CNS-1413972 and by the Defense Advanced Research Projects Agency.

References

  1. 1.
    Buraga, S.C.: An XML-based semantic description of distributed file systems. In: RoEduNet (2003)Google Scholar
  2. 2.
    DiLorenzo, J., Mancini, K., Fisher, K., Foster, N.: TxForest: A DSL for Concurrent Filestores. Technical report (2019). https://arxiv.org/abs/1908.10273
  3. 3.
    DiLorenzo, J., Zhang, R., Menzies, E., Fisher, K., Foster, N.: Incremental forest: a DSL for efficiently managing filestores. In: ACM SIGPLAN Notices, OOPSLA 2016, vol. 51, pp. 252–271 (2016)CrossRefGoogle Scholar
  4. 4.
    Escriva, R., Sirer, E.G.: The design and implementation of the warp transactional filesystem. In: Proceedings of the 13th Usenix Conference on Networked Systems Design and Implementation, NSDI 2016, pp. 469–483. USENIX Association, Berkeley (2016)Google Scholar
  5. 5.
    Fisher, K., Foster, N., Walker, D., Zhu, K.Q.: Forest: a language and toolkit for programming with filestores. In: Proceedings of the 16th ACM SIGPLAN International Conference on Functional Programming, ICFP 2011, pp. 292–306. ACM, New York (2011).  https://doi.org/10.1145/2034773.2034814
  6. 6.
    Fisher, K., Gruber, R.: PADS: a domain-specific language for processing ad hoc data. In: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2005, pp. 295–304. ACM, New York (2005).  https://doi.org/10.1145/1065010.1065046
  7. 7.
    Fisher, K., Walker, D.: The PADS project: an overview. In: Proceedings of the 14th International Conference on Database Theory, ICDT 2011, ACM, New York (2011).  https://doi.org/10.1145/1938551.1938556
  8. 8.
    Foster, J.N., Greenwald, M.B., Moore, J.T., Pierce, B.C., Schmitt, A.: Combinators for bidirectional tree transformations: a linguistic approach to the view update problem. ACM Trans. Program. Lang. Syst. (TOPLAS) 29(3) (2007). Short version in POPL 2005CrossRefGoogle Scholar
  9. 9.
    Garcia, J., Ferreira, P., Guedes, P.: The PerDiS FS: a transactional file system for a distributed persistent store. In: Proceedings of the 8th ACM SIGOPS European Workshop on Support for Composing Distributed Applications, EW 8, pp. 189–194. ACM, New York (1998).  https://doi.org/10.1145/319195.319224
  10. 10.
    Huet, G.: The Zipper. J. Funct. Program. 7(5), 549–554 (1997).  https://doi.org/10.1017/S0956796897002864MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Kiselyov, O.: Tool demonstration: a zipper based file/operating system. In: Proceedings of the 2005 ACM SIGPLAN Workshop on Haskell, Haskell 2005 (2005). http://okmij.org/ftp/continuations/ZFS/zfs-talk.pdf
  12. 12.
    LINQ: NET language-integrated query, February 2007. http://msdn.microsoft.com/library/bb308959.aspx
  13. 13.
    Liskov, B., Rodrigues, R.: Transactional file systems can be fast. In: Proceedings of the 11th Workshop on ACM SIGOPS European Workshop, EW 2004, ACM, New York (2004).  https://doi.org/10.1145/1133572.1133592
  14. 14.
    Moore, K.F., Grossman, D.: High-level small-step operational semantics for transactions. In: Proceedings of the 35th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2008, pp. 51–62. ACM, New York (2008).  https://doi.org/10.1145/1328438.1328448
  15. 15.
    Schmuck, F., Wylie, J.: Experience with transactions in QuickSilver. In: Proceedings of the Thirteenth ACM Symposium on Operating Systems Principles, SOSP 1991, pp. 239–253. ACM, New York (1991).  https://doi.org/10.1145/121132.121171
  16. 16.
    Syme, D.: Looking ahead with F#: taming the data deluge. Presentation at the Workshop on F# in Education, November 2010Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jonathan DiLorenzo
    • 1
    Email author
  • Katie Mancini
    • 1
  • Kathleen Fisher
    • 2
  • Nate Foster
    • 1
  1. 1.Cornell UniversityIthacaUSA
  2. 2.Tufts UniversityMedfordUSA

Personalised recommendations