Advertisement

The Journal of Supercomputing

, Volume 75, Issue 3, pp 1396–1409 | Cite as

Distributed execution of communicating sequential process-style concurrency: Golang case study

  • James Whitney
  • Chandler Gifford
  • Maria PantojaEmail author
Article
  • 103 Downloads

Abstract

In the last decade, the majority of new central processing units (CPU) have become multicore. To take advantage of these new architectures, we need programming languages that can express parallelisms. The programming language Golang is well known for providing developers with an easy programming model for communicating sequential process-style concurrency enabling programmers to easily write functions that will execute on the different cores of a modern multicore CPUs. Unfortunately, Golang does not support distributed execution of goroutines on clusters or distributed systems. In this paper, we extend the concurrency capabilities of Golang to a distributed cluster by providing a library called Gluster that is simple and easy to use. We developed a programming model that allows users to easily distribute work between machines, in a similar way as workloads are distributed in multicore CPUs by using operating system threads with libraries similar to Pthreads and OpenMP. Our Gluster solution is based on a single master node that connects to peers over a network and distributes work to these peers. The master node is able to send function arguments over the network to worker nodes as well as receive return values. Results using matrix multiplication show that our distributed implementation can speed up the goroutines by 5\(\times \) in a small 16 nodes cluster, but more importantly, it shows that the results are scalable to cluster size.

Keywords

Concurrent programming Distributed computing Cluster HPC 

References

  1. 1.
    GoLang (2018) Go -Lang reference. https://golang.org/doc/. Accessed 1 May 2018
  2. 2.
    Hoare CAR (1978) Communicating sequential processes. Commun. ACM 21(8):666–677.  https://doi.org/10.1145/359576.359585 CrossRefzbMATHGoogle Scholar
  3. 3.
    Go usage (2018) https://research.swtch.com/gophercount. Accessed 1 May 2018
  4. 4.
  5. 5.
    Jones S (2018) CUDA—new features and beyond, developer talk. GTC 2018—ID S8278Google Scholar
  6. 6.
    Prasertsang A, Pradubsuwun D (2016) Formal verification of concurrency in go. In: 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE), Khon Kaen, pp 1–4Google Scholar
  7. 7.
    Togashi N, Klyuev V (2014) Concurrency in Go and Java: performance analysis. In: 2014 4th IEEE International Conference on Information Science and Technology, Shenzhen, pp 213–216Google Scholar
  8. 8.
    Ueda Y, Ohara M (2017) Performance competitiveness of a statically compiled language for server-side Web applications. In: 2017 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp 13–22Google Scholar
  9. 9.
    Lange J, Ng N, Toninho B, Yoshida N (2018) A static verification framework for message passing in Go using behavioural types. Accepted draft at ICSE 2018Google Scholar
  10. 10.
    Midtgaard J, Nielson F, Nielson H (2018) Process-local static analysis of synchronous processes. In: 25th Static Analysis SymposiumGoogle Scholar
  11. 11.
    Jenkins L, Zhou T, Spear M (2017) Redesigning Gos built-in map to support concurrent operations. In: 2017 26th International Conference on Parallel Architectures and Compilation Techniques (PACT), Portland, OR, pp 14–26Google Scholar
  12. 12.
    Pasarella E, Vidal ME, Zoltan C (2016) Comparing Mapreduce and pipeline implementations for counting triangles. In: Proceedings XVI Jornadas sobre Programación y Lenguajes, PROLE 2016, Salamanca, Spain, 14–16th September 2016Google Scholar
  13. 13.
    Binet S (2018) Go-HEP: writing concurrent software with ease and Go. arXiV:1808.06529 https://go-hep.org
  14. 14.
    Togashi N, Klyuev V (2015) A novel approach for web development: a schedule management system using GAE/Go. In: 2015 IEEE 7th International Conference on Awareness Science and Technology (iCAST), Qinhuangdao, pp 55–59Google Scholar
  15. 15.
    Fang Z, Luo M, Anwar FM, Zhuang H, Gupta RK (2018) Go-realtime: a lightweight framework for multiprocessor real-time system in user space. SIGBED Rev 14(4):46–52CrossRefGoogle Scholar
  16. 16.
    Scionti A, Mazumdar S (2017) Let’s go: a data-driven multi-threading support. In: Proceedings of the Computing Frontiers Conference (CF’17). ACM, New York, NY, USA, pp 287–290Google Scholar
  17. 17.
    Anurag VN (2018) Distributed computing with Go: practical concurrency and parallelism for Go applications. Packt Publishing, BirminghamGoogle Scholar
  18. 18.
    Proto-actor (2018) http://proto.actor/. Accessed 1 May 2018
  19. 19.
    Hewitt C, Bishop P, Steiger R (1973) A universal modular actor formalism for artificial intelligence. IJCAIGoogle Scholar
  20. 20.
    Lu C (2017) Gleam. https://github.com/chrislusf. Accessed 1 Jan 2018
  21. 21.
    Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. In: OSDI04 Proceedings of the 6th Conference on Symposium on Operating Systems Design and Implementation, vol 6Google Scholar
  22. 22.
    OpenMPI forum website. http://www.mpi-forum.org/. Accessed 1 Feb 2018
  23. 23.
    Dagum I, Menon R (1998) OpenMP: an industry standard API for shared-memory programming. IEEE Comput Sci Eng 5(1):46–55CrossRefGoogle Scholar
  24. 24.
    Lewis B, Berg D (1998) Multithreaded programming with Pthreads. Prentice-Hall, Inc., Upper Saddle RiverGoogle Scholar
  25. 25.
    Pike R (2011) Gobs of data. The Go blog. https://blog.golang.org/gobs-of-data. Accessed 24 Mar 2011
  26. 26.
    Algee-Hewitt M (2016) Counting words in HathiTrust with Python and MPI. Stanford Literary Lab, StanfordGoogle Scholar
  27. 27.
    Github Gluster. https://github.com/James-Whitney/gluster. Accessed 1 Oct 2018

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Cal Poly San Luis Obispo College of EngineeringSan Luis ObispoUSA

Personalised recommendations