Abstract
The Go programming language offers a wide range of primitives to coordinate lightweight threads, e.g., channels, waitgroups, and mutexes—all of which may cause concurrency bugs. Static checkers that guarantee the absence of bugs are essential to help programmers avoid these costly errors before their code is executed. However existing tools either miss too many bugs or cannot handle large programs, and do not support programs that rely on statically unknown parameters that affect their concurrent structure (e.g., number of threads). To address these limitations, we propose a static checker for Go programs which relies on performing bounded model checking of their concurrent behaviours. In contrast to previous works, our approach deals with large codebases, supports programs that have statically unknown parameters, and is extensible to additional concurrency primitives. Our work includes a detailed presentation of the extraction algorithm from Go programs to models, an algorithm to automatically check programs with statically unknown parameters, and a large scale evaluation of our approach. The latter shows that our approach outperforms the state-of-the-art on 220 synthetic programs and 78 buggy programs adapted from existing codebases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Following our report, the bug was fixed in https://github.com/google/trillian/pull/2272.
We have reported this bug in https://github.com/kubernetes/kubernetes/issues/97461. At the time of writing, it has not been fixed.
See https://spinroot.com/spin/Man/if.html and https://spinroot.com/spin/Man/do.html for the full description of the semantics of these constructs.
Here we target only blocking and safety bugs, but our approach is not tied to these types of bugs, e.g., one could also consider data-races.
All experiments reported in this paper were performed on an Intel Xeon E5 6-Core @ 3.5 GHz machine with 32GB RAM, using Go version go1.15.2 (darwin/amd64).
References
Borges, H., Valente, M.T.: What’s in a GitHub star? Understanding repository starring practices in a social coding platform. J. Syst. Softw. 146, 112–129 (2018). https://doi.org/10.1016/j.jss.2018.09.016
Castro-Perez, D., Hu, R., Jongmans, S., Ng, N., Yoshida, N.: Distributed programming using role-parametric session types in Go: statically-typed endpoint apis for dynamically-instantiated communication structures. Proc. ACM Program. Lang. 3(POPL), 29–12930 (2019)
Chabbi, M., Ramanathan, M.K.: A study of real-world data races in golang. In: PLDI, pp. 474–489. ACM, New York, NY, USA (2022)
Clarke, E.M., Kroening, D., Lerda, F.: A tool for checking ANSI-C programs. In: TACAS. Lecture notes in computer science, vol. 2988, pp. 168–176. Springer, Berlin, Heidelberg (2004)
Cordeiro, L.C., Kesseli, P., Kroening, D., Schrammel, P., Trtík, M.: JBMC: A bounded model checking tool for verifying Java bytecode. In: CAV (1). Lecture Notes in Computer Science, vol. 10981, pp. 183–190. Springer, Berlin, Heidelberg (2018)
Cranen, S., Groote, J.F., Keiren, J.J.A., Stappers, F.P.M., de Vink, E.P., Wesselink, W., Willemse, T.A.C.: An overview of the mCRL2 toolset and its recent advances. In: TACAS. Lecture Notes in Computer Science, vol. 7795, pp. 199–213. Springer, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36742-7_15
Dilley, N., Lange, J.: An empirical study of messaging passing concurrency in Go projects. In: SANER, pp. 377–387. IEEE, Piscataway, NJ, USA (2019). https://doi.org/10.1109/SANER.2019.8668036
Dilley, N., Lange, J.: Automated verification of go programs via bounded model checking (Artifact). https://zenodo.org/record/5222045#.Y9FJ-ezP2rN (2021)
Dilley, N., Lange, J.: Automated verification of go programs via bounded model checking. In: ASE, pp. 1016–1027. IEEE, Piscataway, NJ, USA (2021)
Dilley, N., Lange, J.: Bounded verification of message-passing concurrency in Go using promela and spin. In: PLACES@ETAPS. EPTCS. 314, pp. 34–45 (2020)
Dilley, N., Lange, J.: Concurrency in Go Benchmark Maker. https://github.com/nicolasdilley/benchmark-maker (2022)
Dilley, N., Lange, J.: Gomela. https://github.com/nicolasdilley/gomela/ (2022)
Dilley, N., Lange, J.: Gomela. https://github.com/nicolasdilley/gomela-ase21/ (2021)
Gabet, J., Yoshida, N.: Static race detection and mutex safety and liveness for go programs. In: ECOOP. LIPIcs, vol. 166, pp. 4–1430. Schloss Dagstuhl: Leibniz-Zentrum für Informatik, Saarbrücken (2020)
Gadelha, M.Y.R., Monteiro, F.R., Morse, J., Cordeiro, L.C., Fischer, B., Nicole, D.A.: ESBMC 5.0: an industrial-strength C model checker. In: ASE, pp. 888–891. ACM, New York, NY, USA (2018)
golang: go Github project. https://github.com/golang/go (2021)
golang: TestRaceIssue12664_2 code. https://github.com/golang/go/blob/2580d0e08d5e9f979b943758d3c49877fb2324cb/src/runtime/race/testdata/issue12664_test.go#L35 (2021)
Google: FindAll code. https://github.com/google/gops/blob/6fb0d860e5fa50629405d9e77e255cd32795967e/goprocess/gp.go#L29 (2020)
Google: preload code. https://github.com/google/trillian/blob/c92fa63aaa6c133eb8383f2727524421bea420c4/storage/cache/subtree_cache.go#L108 (2021)
gopls, the Go language server. https://github.com/golang/tools/tree/master/gopls
Havelund, K., Pressburger, T.: Model checking JAVA programs using JAVA pathfinder. STTT 2(4), 366–381 (2000). https://doi.org/10.1007/s100090050043
Holzmann, G.J.: The model checker SPIN. IEEE Trans. Software Eng. 23(5), 279–295 (1997)
Honnef, D.: Staticcheck. https://staticcheck.io/
Jhala, R., Majumdar, R.: Software model checking. ACM Comput. Surv. 41(4), 21–12154 (2009)
Kroening, D., Tautschnig, M.: CBMC - C bounded model checker - (competition contribution). In: TACAS. Lecture notes in computer science, vol. 8413, pp. 389–391. Springer, Berlin, Heidelberg (2014)
Kubernetes: GenerateNodeMap code. https://github.com/kubernetes/kubernetes/blob/d70ee902fddc682863a3cc4f0d8eac0223ebf70b/test/e2e/storage/vsphere/nodemapper.go#L62 (2021)
Kubernetes: Kubernetes (k8s). https://github.com/kubernetes/kubernetes (2021)
Kusano, M., Wang, C.: CCmutator: A mutation generator for concurrency constructs in multithreaded C/C++ applications. In: ASE, pp. 722–725. IEEE, Piscataway, NJ, USA (2013)
Lange, J., Ng, N., Toninho, B., Yoshida, N.: A static verification framework for message passing in Go using behavioural types. In: ICSE, pp. 1137–1148. ACM, New York, NY, USA (2018). https://doi.org/10.1145/3180155.3180157
Lange, J., Ng, N., Toninho, B., Yoshida, N.: Fencing off Go: liveness and safety for channel-based programming. In: POPL, pp. 748–761. ACM, New York, NY, USA (2017). https://doi.org/10.1145/3009837.3009847
Liu, Z., Xia, S., Liang, Y., Song, L., Hu, H.: Who goes first? detecting Go concurrency bugs via message reordering. In: Falsafi, B., Ferdman, M., Lu, S., Wenisch, T.F. (eds.) ASPLOS ’22: 27th ACM International conference on architectural support for programming languages and operating systems, Lausanne, Switzerland, 28 February 2022–4 March 2022, pp. 888–902. ACM, New York, NY, USA (2022). https://doi.org/10.1145/3503222.3507753
Liu, Z., Zhu, S., Qin, B., Chen, H., Song, L.: Automatically detecting and fixing concurrency bugs in Go software systems. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. ASPLOS 2021, pp. 616–629. Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3445814.3446756
Magee, J., Kramer, J.: Concurrency: State Models and Java Programs. Wiley, Hoboken (1999)
Midtgaard, J., Nielson, F., Nielson, H.R.: Process-local static analysis of synchronous processes. In: SAS. Lecture notes in computer science, vol. 11002, pp. 284–305. Springer, Berlin, Heidelberg (2018). https://doi.org/10.1007/978-3-319-99725-4_18
Ng, N., Yoshida, N.: Static deadlock detection for concurrent Go by global session graph synthesis. In: CC, pp. 174–184. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2892208.2892232
Padhiyar, S., Sivaramakrishnan, K.C.: ConFuzz: Coverage-guided property fuzzing for event-driven programs. In: PADL. Lecture Notes in Computer Science, vol. 12548, pp. 127–144. Springer, Berlin, Heidelberg (2021)
sasha-s: go-deadlock. https://github.com/sasha-s/go-deadlock
Siebenmann, C.: Even in Go, concurrency is still not easy (with an example). https://utcc.utoronto.ca/~cks/space/blog/programming/GoConcurrencyStillNotEasy (2020)
Snail007: Keys code. https://github.com/snail007/goproxy/blob/7e0406bdb90960fa0c0d9c89a770ef206c4c02d8/utils/map.go#L243 (2022)
Stadtmüller, K., Sulzmann, M., Thiemann, P.: Static trace-based deadlock analysis for synchronous mini-go. In: APLAS. Lecture notes in computer science, vol. 10017, pp. 116–136. Berlin, Heidelberg (2016). https://doi.org/10.1007/978-3-319-47958-3_7
Stiévenart, Q., Madsen, M.: Fuzzing channel-based concurrency runtimes using types and effects. Proc. ACM Program. Lang. 4(OOPSLA), 186–118627 (2020)
Sulzmann, M., Stadtmüller, K.: Trace-based run-time analysis of message-passing Go programs. In: Haifa Verification Conference. Lecture Notes in Computer Science, vol. 10629, pp. 83–98. Springer, Berlin, Heidelberg (2017). https://doi.org/10.1007/978-3-319-70389-3_6
Sulzmann, M., Stadtmüller, K.: Two-phase dynamic analysis of message-passing Go programs based on vector clocks. In: PPDP, pp. 22–12213. ACM, New York, NY, USA (2018). https://doi.org/10.1145/3236950.3236959
Taheri, S., Gopalakrishnan, G.: GoAT: Automated concurrency analysis and debugging tool for Go. In: IEEE international symposium on workload characterization, IISWC 2021, Storrs, CT, USA, November 7–9, 2021, pp. 138–150. IEEE, Piscataway, NJ, USA (2021). https://doi.org/10.1109/IISWC53511.2021.00023
Tu, T., Liu, X., Song, L., Zhang, Y.: Understanding real-world concurrency bugs in Go. In: ASPLOS, pp. 865–878. ACM, New York, NY, USA (2019). https://doi.org/10.1145/3297858.3304069
Uber: goleak. https://github.com/uber-go/goleak
Veileborg, O.H., Saioc, G., Møller, A.: Detecting blocking errors in go programs using localized abstract interpretation. In: ASE, pp. 32–13212. ACM, New York, NY, USA (2022)
von Gleissenthall, K., Kici, R.G., Bakst, A., Stefan, D., Jhala, R.: Pretend synchrony: synchronous verification of asynchronous distributed programs. Proc. ACM Program. Lang. 3(POPL), 59–15930 (2019). https://doi.org/10.1145/3290372
Yuan, T., Li, G., Lu, J., Liu, C., Li, L., Xue, J.: Gobench: a benchmark suite of real-world Go concurrency bugs. In: CGO, pp. 187–199. IEEE, Piscataway, NJ, USA (2021)
Zaks, A., Joshi, R.: Verifying multi-threaded C programs with SPIN. In: SPIN. Lecture notes in computer science, vol. 5156, pp. 325–342. Springer, Berlin, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85114-1_22
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Dilley, N., Lange, J. Automated verification of concurrent go programs via bounded model checking. Autom Softw Eng 30, 29 (2023). https://doi.org/10.1007/s10515-023-00391-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10515-023-00391-z