Skip to main content
SpringerLink
Log in

Multi-ML: Programming Multi-BSP Algorithms in ML

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

bsp is a bridging model between abstract execution and concrete parallel systems. Structure and abstraction brought by bsp allow to have portable parallel programs with scalable performance predictions, without dealing with low-level details of architectures. In the past, we designed bsml for programming bsp algorithms in ml. However, the simplicity of the bsp model does not fit the complexity of today’s hierarchical architectures such as clusters of machines with multiple multi-core processors. The multi-bsp model is an extension of the bsp model which brings a tree-based view of nested components of hierarchical architectures. To program multi-bsp algorithms in ml, we propose the multi-ml language as an extension of bsml where a specific kind of recursion is used to go through a hierarchy of computing nodes. We define a formal semantics of the language and present preliminary experiments which show performance improvements with respect to bsml.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Beran, M.: Decomposable bulk synchronous parallel computers. In: Pavelka, J., Tel, G., Bartoek, M. (eds.) Theory and Practice of Informatics (SOFSEM), Volume 1725 of LNCS, pp. 349–359. Springer, Berlin (1999)

    Google Scholar 

  2. Bisseling, R.H.: Parallel Scientific Computation: A Structured Approach Using bsp and mpi. Oxford University Press, Oxford (2004)

    Book  MATH  Google Scholar 

  3. Blelloch, G.E.: NESL. In: Padua, D.A. (ed.) Encyclopedia of Parallel Computing, pp. 1278–1283. Springer, Berlin (2011)

    Google Scholar 

  4. Bonorden, O., Judoiink, B., von Otte, I., Rieping, O.: The Paderborn University bsp (pub) library. Parallel Comput. 29(2), 187–207 (2003)

    Article  Google Scholar 

  5. Bourgoin, M., Chailloux, E., Lamotte, J.-L.: SPOC: GPGPU programming through stream processing with ocaml. Parallel Process. Lett. 22(2), 1–12 (2012)

    Article  MathSciNet  Google Scholar 

  6. Cappello, F., Guermouche, A., Snir, M.: On communication determinism in parallel hpc applications. In: Computer Communications and Networks (ICCCN). IEEE (2010)

  7. Cha, H., Lee, D.: h-bsp: a hierarchical bsp computation model. J. Supercomput. 18(2), 179–200 (2001)

    Article  MATH  Google Scholar 

  8. Chailloux, E., Foisy, C.: A portable implementation for Objective Caml Flight. Parallel Process. Lett. 13(3), 425–436 (2003)

    Article  MathSciNet  Google Scholar 

  9. Chailloux, E., Ravet, V., Verlaguet, J.: HirondML: Fair threads migrations for Objective Caml. Parallel Process. Lett. 18(1), 55–69 (2008)

    Article  MathSciNet  Google Scholar 

  10. Chakravarty, M.M.T., Leshchinskiy, R., Jones, S.L.P., Keller, G., Marlow, S.: Data parallel Haskell: a status report. In: Glew, N., Blelloch, G.E. (eds.) Declarative aspects of multicore programming (DAMP), part of POPL, pp. 10–18. ACM, London (2007)

    Google Scholar 

  11. Cosmo, R.D., Li, Z., Pelagatti, S., Weis, P.: Skeletal Parallel Programming with ocaml P3l 2.0. Parallel Process. Lett. 18(1), 149–164 (2008)

    Article  MathSciNet  Google Scholar 

  12. da Rosa Righi, R., Pilla, L.L., Carissimi, A., Navaux, P.O.A., Heiss, H.-U.: MigBSP: a novel migration model for bulk-synchronous parallel processes rescheduling. In: High Performance Computing and Communications (HPCC), pp. 585–590. IEEE (2009)

  13. Fluet, M., Rainey, M., Reppy, J., Shaw, A.: Implicitly-threaded parallelism in manticore. SIGPLAN Not. 43(9), 119–130 (2008)

    Article  MATH  Google Scholar 

  14. Fortin, J., Gava, F.: BSP-Why: a tool for deductive verification of bsp algorithms with subgroup synchronisation. Int. J. Parallel Program. (2015). doi:10.1007/s10766-015-0360-y

  15. Gava, F.: Formal proofs of functional bsp programs. Parallel Process. Lett. 13(3), 365–376 (2003)

    Article  MathSciNet  Google Scholar 

  16. Gesbert, L., Gava, F., Loulergue, F., Dabrowski, F.: Bulk synchronous parallel ml with exceptions. Future Gener. Comput. Syst. 26, 486–490 (2010)

    Article  Google Scholar 

  17. Gesbert, L., Hu, Z., Loulergue, F., Matsuzaki, K., Tesson, J.: Systematic development of correct bulk synchronous parallel programs. In: Parallel and Distributed Computing, Applications and Technologies (PDCAT), pp. 334–340. IEEE (2010)

  18. González-Vélez, H., Leyton, M.: A survey of algorithmic skeleton frameworks: high-level structured parallel programming enablers. Software, Practrice & Experience 40(12), 1135–1160 (2010)

    Article  Google Scholar 

  19. Grelck, C., Scholz, S.-B.: SAC: a functional array language for efficient multi-threaded execution. Int. J. Parallel Prog. 34(4), 383–427 (2006)

    Article  MATH  Google Scholar 

  20. Gu, Y., Le, B.-S., Wentong, C.: JBSP: A bsp programming library in java. J. Parallel Distrib. Comput. 61(8), 1126–1142 (2001)

    Article  MATH  Google Scholar 

  21. Hamidouche, K., Falcou, J., Etiemble, D.: A framework for an automatic hybrid mpi+open-mp code generation. In: Watson, L.T., Howell, G.W., Thacker, W.I., Seidel, S. (eds.) Simulation Multi-conference (SpringSim) on High Performance Computing Symposia (HPC), pp. 48–55. SCS/ACM, London (2011)

    Google Scholar 

  22. Hammond, K.: The dynamic properties of hume: a functionally-based concurrent language with bounded time and space behaviour. In: Mohnen, M., Koopman, P.W.M. (eds.) Implementation of Functional Languages (IFL), Volume 2011 of LNCS, pp. 122–139. Springer, Berlin (2000)

    Google Scholar 

  23. Hammond, K., Michaelson, G. (eds.): Research Directions in Parallel Functional Programming. Springer, Berlin (2000)

    Google Scholar 

  24. Hill, J.M.D., McColl, B., Stefanescu, D.C., Goudreau, M.W., Lang, K., Rao, S.B., Suel, T., Tsantilas, T., Bisseling, R.: bsplib: the bsp programming library. Parallel Comput. 24, 1947–1980 (1998)

    Article  Google Scholar 

  25. Hinsen, K., Langtangen, H.P., Skavhaug, O., Ødegård, Å.: Using bsp and python to simplify parallel programming. Future Gener. Comput. Syst. 22(1–2), 123–157 (2006)

    Article  Google Scholar 

  26. Hou, Q., Zhou, K., Guo, B.: BSGP: Bulk-Synchronous gpu Programming. ACM Trans. Graph. 27(3), 19:1–19:12 (2008)

    Article  Google Scholar 

  27. Keßler, C.W.: NestStep: nested parallelism and virtual shared memory for the bsp model. J. Supercomput. 17(3), 245–262 (2000)

    Article  MATH  Google Scholar 

  28. Leroy, X., Grall, H.: Coinductive big-step operational semantics. Inf. Comput. 207(2), 284–304 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  29. Li, C., Hains, G.: SGL: towards a bridging model for heterogeneous hierarchical platforms. IJHPCN 7(2), 139–151 (2012)

    Article  Google Scholar 

  30. Loidl, H.-W., et al.: Comparing parallel functional languages: programming and performance. Higher Order and Symb. Comp. 16(3), 203–251 (2003)

    Article  MATH  Google Scholar 

  31. Loogen, R., Ortega-Mallén, Y., Peña Marí, R.: Parallel functional programming in Eden. J. Funct. Program. 15(3), 431–475 (2005)

    Article  MATH  Google Scholar 

  32. Mandel, L., Maranget, L.: Programming in jocaml. In: Drossopoulou, S. (ed.) European Symposium on Programming (ESOP), Volume 4960 of LNCS, pp. 108–111. Springer, Berlin (2008)

    Google Scholar 

  33. Miller, Q.: BSP in a Lazy Functional Context. In: Hammond, K., Curtis, S. (eds.) Trends in Functional Programming, pp. 37–50. Intellect Books (2002)

  34. Rossberg, A.: Typed open programming: a higher-order, typed approach to dynamic modularity and distribution. PhD thesis, Universität des Saarlandes (2007)

  35. Scaife, N., Michaelson, G., Horiguchi, S.: Empirical parallel performance prediction from semantics-based profiling. Scalable Comput. Pract. Exp. 7(3), 781–789 (2006)

    Google Scholar 

  36. Seo, S., Yoon, E.J., Kim, J.-H., Jin, S., Kim, J.-S., Maeng, S.: hama: an efficient matrix computation with the mapreduce framework. In: Cloud Computing (CloudCom), pp. 721–726. IEEE (2010)

  37. Valiant, L.G.: A bridging model for parallel computation. Comm. ACM 33(8), 103–111 (1990)

    Article  Google Scholar 

  38. Valiant, L.G.: A bridging model for multi-core computing. J. Comput. Syst. Sci. 77(1), 154–166 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  39. Yzelman, A.N., Bisseling, R.H.: An object-oriented bulk synchronous parallel library for multicore programming. Concurr. Comput. Pract. Exp. 24(5), 533–553 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Université Paris-Est, LACL (EA 4219), UPEC, 94010, Créteil, France

    V. Allombert, F. Gava & J. Tesson

Authors
  1. V. Allombert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Gava
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Tesson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Allombert.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Allombert, V., Gava, F. & Tesson, J. Multi-ML: Programming Multi-BSP Algorithms in ML. Int J Parallel Prog 45, 340–361 (2017). https://doi.org/10.1007/s10766-016-0417-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-016-0417-6

Keywords

Search

Navigation