Advertisement

Spores: A Type-Based Foundation for Closures in the Age of Concurrency and Distribution

  • Heather Miller
  • Philipp Haller
  • Martin Odersky
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8586)

Abstract

Functional programming (FP) is regularly touted as the way forward for bringing parallel, concurrent, and distributed programming to the mainstream. The popularity of the rationale behind this viewpoint has even led to a number of object-oriented (OO) programming languages outside the Smalltalk tradition adopting functional features such as lambdas and thereby function closures. However, despite this established viewpoint of FP as an enabler, reliably distributing function closures over a network, or using them in concurrent environments nonetheless remains a challenge across FP and OO languages. This paper takes a step towards more principled distributed and concurrent programming by introducing a new closure-like abstraction and type system, called spores, that can guarantee closures to be serializable, thread-safe, or even have custom user-defined properties. Crucially, our system is based on the principle of encoding type information corresponding to captured variables in the type of a spore. We prove our type system sound, implement our approach for Scala, evaluate its practicality through a small empirical study, and show the power of these guarantees through a case analysis of real-world distributed and concurrent frameworks that this safe foundation for closures facilitates.

Keywords

closures functions distributed programming concurrent programming type systems 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Budimlić, Z., Burke, M., Cavé, V., Knobe, K., Lowney, G., Newton, R., Palsberg, J., Peixotto, D., Sarkar, V., Schlimbach, F.: et al. Concurrent collections. Scientific Programming 18(3) (2010)Google Scholar
  2. 2.
    Chakravarty, M.M.T., Leshchinskiy, R., Peyton Jones, S., Keller, G., Marlow, S.: Data Parallel Haskell: A status report. In: Proc. DAMP Workshop, pp. 10–18. ACM (2007)Google Scholar
  3. 3.
    Collins, A., Grewe, D., Grover, V., Lee, S., Susnea, A.: NOVA: A functional language for data parallelism. Technical Report NVR-2013-002, NVIDIA Corporation (July 2013)Google Scholar
  4. 4.
    Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  5. 5.
    Epstein, J., Black, A.P., Peyton-Jones, S.: Towards Haskell in the cloud. In: Proc. Haskell Symposium, pp. 118–129. ACM (2011)Google Scholar
  6. 6.
    Germain, G.: Concurrency oriented programming in Termite Scheme. In: Erlang Workshop, p. 20. ACM (2006)Google Scholar
  7. 7.
    Goetz, B.: JSR 335: Lambda expressions for the Java programming language (2013), https://jcp.org/en/jsr/detail?id=335
  8. 8.
    Herhut, S., Hudson, R.L., Shpeisman, T., Sreeram, J.: River trail: a path to parallelism in JavaScript. In: OOPSLA, pp. 729–744 (2013)Google Scholar
  9. 9.
    International Standard ISO/IEC 14882:2011. Programming Languages – C++. International Organization for Standards (2011)Google Scholar
  10. 10.
    Lewis, J.R., Launchbury, J., Meijer, E., Shields, M.: Implicit parameters: Dynamic scoping with static types. In: POPL, pp. 108–118 (2000)Google Scholar
  11. 11.
    Maier, P., Trinder, P.: Implementing a high-level distributed-memory parallel Haskell in Haskell. In: Gill, A., Hage, J. (eds.) IFL 2011. LNCS, vol. 7257, pp. 35–50. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Marlow, S., Newton, R., Peyton Jones, S.: A monad for deterministic parallelism. In: Proc. Haskell Symposium, pp. 71–82. ACM (2011)Google Scholar
  13. 13.
    Matsakis, N.: Fn types in Rust, take 3 (2013), http://smallcultfollowing.com/babysteps/blog/2013/10/10/fn-types-in-rust
  14. 14.
    Matsakis, N.D.: Parallel closures: a new twist on an old idea. In: HotPar. USENIX (2012)Google Scholar
  15. 15.
    McKerns, M.M., Strand, L., Sullivan, T., Fang, A., Aivazis, M.A.: Building a framework for predictive science. In: Proc. of the 10th Python in Science Conf. (2011)Google Scholar
  16. 16.
    Meijer, E.: Confessions of a used programming language salesman. In: OOPSLA (2007)Google Scholar
  17. 17.
    Meyerovich, L.A., Rabkin, A.S.: Empirical analysis of programming language adoption. In: OOPSLA (2013)Google Scholar
  18. 18.
    Miller, H., Haller, P.: Spores, formally. Technical Report EPFL-REPORT-191240, Department of Computer Science, EPFL, Lausanne, Switzerland (December 2013)Google Scholar
  19. 19.
    Miller, H., Haller, P., Burmako, E., Odersky, M.: Instant pickles: Generating object-oriented pickler combinators for fast and extensible serialization. In: OOPSLA, pp. 183–202 (2013)Google Scholar
  20. 20.
    Miller, H., Haller, P., Rytz, L., Odersky, M.: Functional programming for all! Scaling a MOOC for students and professionals alike. In: ICSE, pp. 265–263 (2014)Google Scholar
  21. 21.
    Morrisett, J.G., Walker, D., Crary, K., Glew, N.: From system F to typed assembly language. ACM Trans. Program. Lang. Syst 21(3), 527–568 (1999)CrossRefGoogle Scholar
  22. 22.
    Murphy VII, T., Crary, K., Harper, R.: Type-safe distributed programming with ML5. In: Barthe, G., Fournet, C. (eds.) TGC 2007. LNCS, vol. 4912, pp. 108–123. Springer, Heidelberg (2008)Google Scholar
  23. 23.
    Odersky, M.: The Scala language specification (2013)Google Scholar
  24. 24.
    Pierce, B.C.: Types and programming languages. MIT Press (2002)Google Scholar
  25. 25.
    Prokopec, A., Bagwell, P., Rompf, T., Odersky, M.: A generic parallel collection framework. In: Jeannot, E., Namyst, R., Roman, J. (eds.) Euro-Par 2011, Part II. LNCS, vol. 6853, pp. 136–147. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  26. 26.
    Prokopec, A., Miller, H., Schlatter, T., Haller, P., Odersky, M.: FlowPools: A lock-free deterministic concurrent dataflow abstraction. In: Kasahara, H., Kimura, K. (eds.) LCPC 2012. LNCS, vol. 7760, pp. 158–173. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  27. 27.
    Rompf, T., Maier, I., Odersky, M.: Implementing first-class polymorphic delimited continuations by a type-directed selective CPS-transform. In: ICFP, pp. 317–328. ACM (2009)Google Scholar
  28. 28.
    Schwendner, A.: Distributed functional programming in Scheme. Master’s thesis, Massachusetts Institute of Technology (2009)Google Scholar
  29. 29.
    Typesafe. Akka (2009), http://akka.io/
  30. 30.
    Wright, A.K., Felleisen, M.: A syntactic approach to type soundness. Inf. Comput. 115(1), 38–94 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Zaharia, M., Chowdhury, M., Das, T., Dave, A., McCauley, M., Franklin, M., Shenker, S., Stoica, I.: Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In: NSDI. USENIX (2012)Google Scholar
  32. 32.
    Zibin, Y., Potanin, A., Li, P., Ali, M., Ernst, M.D.: Ownership and immutability in generic java. In: OOPSLA, pp. 598–617. ACM (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Heather Miller
    • 1
  • Philipp Haller
    • 2
  • Martin Odersky
    • 1
  1. 1.EPFLSwitzerland
  2. 2.Typesafe, Inc.USA

Personalised recommendations