International Journal of Parallel Programming

, Volume 45, Issue 2, pp 340–361 | Cite as

Multi-ML: Programming Multi-BSP Algorithms in ML

  • V. Allombert
  • F. Gava
  • J. Tesson


bsp is a bridging model between abstract execution and concrete parallel systems. Structure and abstraction brought by bsp allow to have portable parallel programs with scalable performance predictions, without dealing with low-level details of architectures. In the past, we designed bsml for programming bsp algorithms in ml. However, the simplicity of the bsp model does not fit the complexity of today’s hierarchical architectures such as clusters of machines with multiple multi-core processors. The multi-bsp model is an extension of the bsp model which brings a tree-based view of nested components of hierarchical architectures. To program multi-bsp algorithms in ml, we propose the multi-ml language as an extension of bsml where a specific kind of recursion is used to go through a hierarchy of computing nodes. We define a formal semantics of the language and present preliminary experiments which show performance improvements with respect to bsml.


bsp multi-bsp ml Parallel programming 


  1. 1.
    Beran, M.: Decomposable bulk synchronous parallel computers. In: Pavelka, J., Tel, G., Bartoek, M. (eds.) Theory and Practice of Informatics (SOFSEM), Volume 1725 of LNCS, pp. 349–359. Springer, Berlin (1999)Google Scholar
  2. 2.
    Bisseling, R.H.: Parallel Scientific Computation: A Structured Approach Using bsp and mpi. Oxford University Press, Oxford (2004)CrossRefzbMATHGoogle Scholar
  3. 3.
    Blelloch, G.E.: NESL. In: Padua, D.A. (ed.) Encyclopedia of Parallel Computing, pp. 1278–1283. Springer, Berlin (2011)Google Scholar
  4. 4.
    Bonorden, O., Judoiink, B., von Otte, I., Rieping, O.: The Paderborn University bsp (pub) library. Parallel Comput. 29(2), 187–207 (2003)CrossRefGoogle Scholar
  5. 5.
    Bourgoin, M., Chailloux, E., Lamotte, J.-L.: SPOC: GPGPU programming through stream processing with ocaml. Parallel Process. Lett. 22(2), 1–12 (2012)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Cappello, F., Guermouche, A., Snir, M.: On communication determinism in parallel hpc applications. In: Computer Communications and Networks (ICCCN). IEEE (2010)Google Scholar
  7. 7.
    Cha, H., Lee, D.: h-bsp: a hierarchical bsp computation model. J. Supercomput. 18(2), 179–200 (2001)CrossRefzbMATHGoogle Scholar
  8. 8.
    Chailloux, E., Foisy, C.: A portable implementation for Objective Caml Flight. Parallel Process. Lett. 13(3), 425–436 (2003)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Chailloux, E., Ravet, V., Verlaguet, J.: HirondML: Fair threads migrations for Objective Caml. Parallel Process. Lett. 18(1), 55–69 (2008)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chakravarty, M.M.T., Leshchinskiy, R., Jones, S.L.P., Keller, G., Marlow, S.: Data parallel Haskell: a status report. In: Glew, N., Blelloch, G.E. (eds.) Declarative aspects of multicore programming (DAMP), part of POPL, pp. 10–18. ACM, London (2007)Google Scholar
  11. 11.
    Cosmo, R.D., Li, Z., Pelagatti, S., Weis, P.: Skeletal Parallel Programming with ocaml P3l 2.0. Parallel Process. Lett. 18(1), 149–164 (2008)MathSciNetCrossRefGoogle Scholar
  12. 12.
    da Rosa Righi, R., Pilla, L.L., Carissimi, A., Navaux, P.O.A., Heiss, H.-U.: MigBSP: a novel migration model for bulk-synchronous parallel processes rescheduling. In: High Performance Computing and Communications (HPCC), pp. 585–590. IEEE (2009)Google Scholar
  13. 13.
    Fluet, M., Rainey, M., Reppy, J., Shaw, A.: Implicitly-threaded parallelism in manticore. SIGPLAN Not. 43(9), 119–130 (2008)CrossRefzbMATHGoogle Scholar
  14. 14.
    Fortin, J., Gava, F.: BSP-Why: a tool for deductive verification of bsp algorithms with subgroup synchronisation. Int. J. Parallel Program. (2015). doi: 10.1007/s10766-015-0360-y
  15. 15.
    Gava, F.: Formal proofs of functional bsp programs. Parallel Process. Lett. 13(3), 365–376 (2003)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Gesbert, L., Gava, F., Loulergue, F., Dabrowski, F.: Bulk synchronous parallel ml with exceptions. Future Gener. Comput. Syst. 26, 486–490 (2010)CrossRefGoogle Scholar
  17. 17.
    Gesbert, L., Hu, Z., Loulergue, F., Matsuzaki, K., Tesson, J.: Systematic development of correct bulk synchronous parallel programs. In: Parallel and Distributed Computing, Applications and Technologies (PDCAT), pp. 334–340. IEEE (2010)Google Scholar
  18. 18.
    González-Vélez, H., Leyton, M.: A survey of algorithmic skeleton frameworks: high-level structured parallel programming enablers. Software, Practrice & Experience 40(12), 1135–1160 (2010)CrossRefGoogle Scholar
  19. 19.
    Grelck, C., Scholz, S.-B.: SAC: a functional array language for efficient multi-threaded execution. Int. J. Parallel Prog. 34(4), 383–427 (2006)CrossRefzbMATHGoogle Scholar
  20. 20.
    Gu, Y., Le, B.-S., Wentong, C.: JBSP: A bsp programming library in java. J. Parallel Distrib. Comput. 61(8), 1126–1142 (2001)CrossRefzbMATHGoogle Scholar
  21. 21.
    Hamidouche, K., Falcou, J., Etiemble, D.: A framework for an automatic hybrid mpi+open-mp code generation. In: Watson, L.T., Howell, G.W., Thacker, W.I., Seidel, S. (eds.) Simulation Multi-conference (SpringSim) on High Performance Computing Symposia (HPC), pp. 48–55. SCS/ACM, London (2011)Google Scholar
  22. 22.
    Hammond, K.: The dynamic properties of hume: a functionally-based concurrent language with bounded time and space behaviour. In: Mohnen, M., Koopman, P.W.M. (eds.) Implementation of Functional Languages (IFL), Volume 2011 of LNCS, pp. 122–139. Springer, Berlin (2000)Google Scholar
  23. 23.
    Hammond, K., Michaelson, G. (eds.): Research Directions in Parallel Functional Programming. Springer, Berlin (2000)Google Scholar
  24. 24.
    Hill, J.M.D., McColl, B., Stefanescu, D.C., Goudreau, M.W., Lang, K., Rao, S.B., Suel, T., Tsantilas, T., Bisseling, R.: bsplib: the bsp programming library. Parallel Comput. 24, 1947–1980 (1998)CrossRefGoogle Scholar
  25. 25.
    Hinsen, K., Langtangen, H.P., Skavhaug, O., Ødegård, Å.: Using bsp and python to simplify parallel programming. Future Gener. Comput. Syst. 22(1–2), 123–157 (2006)CrossRefGoogle Scholar
  26. 26.
    Hou, Q., Zhou, K., Guo, B.: BSGP: Bulk-Synchronous gpu Programming. ACM Trans. Graph. 27(3), 19:1–19:12 (2008)CrossRefGoogle Scholar
  27. 27.
    Keßler, C.W.: NestStep: nested parallelism and virtual shared memory for the bsp model. J. Supercomput. 17(3), 245–262 (2000)CrossRefzbMATHGoogle Scholar
  28. 28.
    Leroy, X., Grall, H.: Coinductive big-step operational semantics. Inf. Comput. 207(2), 284–304 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Li, C., Hains, G.: SGL: towards a bridging model for heterogeneous hierarchical platforms. IJHPCN 7(2), 139–151 (2012)CrossRefGoogle Scholar
  30. 30.
    Loidl, H.-W., et al.: Comparing parallel functional languages: programming and performance. Higher Order and Symb. Comp. 16(3), 203–251 (2003)CrossRefzbMATHGoogle Scholar
  31. 31.
    Loogen, R., Ortega-Mallén, Y., Peña Marí, R.: Parallel functional programming in Eden. J. Funct. Program. 15(3), 431–475 (2005)CrossRefzbMATHGoogle Scholar
  32. 32.
    Mandel, L., Maranget, L.: Programming in jocaml. In: Drossopoulou, S. (ed.) European Symposium on Programming (ESOP), Volume 4960 of LNCS, pp. 108–111. Springer, Berlin (2008)Google Scholar
  33. 33.
    Miller, Q.: BSP in a Lazy Functional Context. In: Hammond, K., Curtis, S. (eds.) Trends in Functional Programming, pp. 37–50. Intellect Books (2002)Google Scholar
  34. 34.
    Rossberg, A.: Typed open programming: a higher-order, typed approach to dynamic modularity and distribution. PhD thesis, Universität des Saarlandes (2007)Google Scholar
  35. 35.
    Scaife, N., Michaelson, G., Horiguchi, S.: Empirical parallel performance prediction from semantics-based profiling. Scalable Comput. Pract. Exp. 7(3), 781–789 (2006)Google Scholar
  36. 36.
    Seo, S., Yoon, E.J., Kim, J.-H., Jin, S., Kim, J.-S., Maeng, S.: hama: an efficient matrix computation with the mapreduce framework. In: Cloud Computing (CloudCom), pp. 721–726. IEEE (2010)Google Scholar
  37. 37.
    Valiant, L.G.: A bridging model for parallel computation. Comm. ACM 33(8), 103–111 (1990)CrossRefGoogle Scholar
  38. 38.
    Valiant, L.G.: A bridging model for multi-core computing. J. Comput. Syst. Sci. 77(1), 154–166 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Yzelman, A.N., Bisseling, R.H.: An object-oriented bulk synchronous parallel library for multicore programming. Concurr. Comput. Pract. Exp. 24(5), 533–553 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Université Paris-Est, LACL (EA 4219), UPECCréteilFrance

Personalised recommendations