Skip to main content
SpringerLink
Log in

Synthesizing Precise and Useful Commutativity Conditions

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

Reasoning about commutativity between data-structure operations is an important problem with many applications. In the sequential setting, commutativity can be used to reason about the correctness of refactoring, compiler transformations, and identify instances of non-determinism. In parallel contexts, commutativity dates back to the database (Weihl in IEEE Trans Comput 37(12):1488–1505, 1988) and compilers (Rinard and Diniz in ACM Trans Program Lang Syst 19(6):942–991, 1997) communities and, more recently, appears in optimistic parallelization (Herlihy and Koskinen in Proceedings of the 13th ACM SIGPLAN symposium on principles and practice of parallel programming, 2008), dynamic concurrency (Tripp et al. in Proceedings of the 33rd ACM SIGPLAN conference on programming language design and implementation, PLDI ’12, New York, NY, USA, ACM, pp 145–156, 2012; Dimitrov et al. in Proceedings of the 35th ACM SIGPLAN conference on programming language design and implementation, 2014), scalable systems (Clements et al. in ACM Trans Comput Syst 32(4):10, 2015) and even smart contracts (Dickerson et al. in Proceedings of the ACM symposium on principles of distributed computing, PODC ’17, New York, NY, USA, ACM, pp 303–312, 2017). There have been research results on automatic generation of commutativity conditions, yet we are unaware of any fully automated technique to generate conditions that are both sound and effective. We have designed such a technique, driven by an algorithm that iteratively refines a conservative approximation of the commutativity (and non-commutativity) condition for a pair of methods into an increasingly precise version. The algorithm terminates if/when the entire state space has been considered, and can be aborted at any time to obtain a partial yet sound commutativity condition. We have generalized our work to left-/right-movers (Lipton in Commun ACM 8(12):717–721, 1975) and proved relative completeness. We describe aspects of our technique that lead to useful commutativity conditions, including how predicates are selected during refinement and heuristics that impact the output shape of the condition. We have implemented our technique in a prototype open-source tool Servois. Our algorithm produces quantifier-free queries that are dispatched to a back-end SMT solver. We evaluate Servois first by synthesizing commutativity conditions for a range of data structures including Set, HashTable, Accumulator, Counter, and Stack. We then show several applications of our work including reasoning about memories and locks, finding vulnerabilities in Ethereum smart contracts, improving transactional memory performance, distributed applications, code refactoring, verification, and synthesis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://github.com/kbansal/servois/blob/master/src/lift.py.

References

  1. Abadi, M., Lamport, L.: The existence of refinement mappings. Theor. Comput. Sci. 82, 253–284 (1991)

    Article  MathSciNet  Google Scholar 

  2. Aleen,F., Clark, N.: Commutativity analysis for software parallelization: letting program transformations see the big picture. In: Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-XII), pp. 241–252. ACM (2009)

  3. Bansal, K., Koskinen, E., Tripp, O.: Automatic generation of precise and useful commutativity conditions. In: Tools and Algorithms for the Construction and Analysis of Systems—24th International Conference, TACAS 2018, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2018, Greece, April 2018, Proceedings, Part II (2017)

  4. Bansal, K., Reynolds, A., Barrett, C., Tinelli, C.: A new decision procedure for finite sets and cardinality constraints in SMT. In: Proceedings of the 8th International Joint Conference on Automated Reasoning, vol. 9706, pp. 82–98. Springer (2016)

  5. Bansal, K.: Decision Procedures for Finite Sets with Cardinality and Local Theory Extensions. Ph.D. Thesis, New York University (Jan. 2016)

  6. Barnett, M., Leino, K.R.M., Schulte, W.: The spec# programming system: An overview. In: Proceedings of the 2004 International Conference on Construction and Analysis of Safe, Secure, and Interoperable Smart Devices, CASSIS’04, pp. 49–69 (2005)

  7. Barrett, C., Conway, C.L., Deters, M., Hadarean, L., Jovanović, D., King, T., Reynolds, A., Tinelli, C.: CVC4. In: Gopalakrishnan, G., Qadeer, S. (eds.), Proceedings of the 23rd International Conference on Computer Aided Verification (CAV ’11), vol. 6806, pp. 171–177. Springer (July 2011)

  8. Chechik, M., Stavropoulou, I., Disenfeld, C., Rubin, J.: FPH: efficient non-commutativity analysis of feature-based systems. In: Fundamental Approaches to Software Engineering, 21st International Conference, FASE 2018, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2018, Thessaloniki, Greece, April 14–20, 2018, Proceedings, pp. 319–336 (2018)

  9. Clements, A.T., Kaashoek, M.F., Zeldovich, N., Morris, R.T., Kohler, E.: The scalable commutativity rule: designing scalable software for multicore processors. ACM Trans. Comput. Syst. 32(4), 10 (2015)

    Article  Google Scholar 

  10. Cook, B., Koskinen, E.: Making prophecies with decision predicates. In: Proceedings of the 38th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2011, Austin, TX, USA, January 26–28, 2011, pp. 399–410 (2011)

  11. Dickerson, T., Gazzillo, P., Herlihy, M., Koskinen, E.: Adding concurrency to smart contracts. In: Proceedings of the ACM Symposium on Principles of Distributed Computing, PODC ’17, New York, NY, USA, pp. 303–312 (2017). ACM

  12. Dimitrov, D., Raychev, V., Vechev, M.T., Koskinen, E.: Commutativity race detection. In: O’Boyle, M.F.P., Pingali, K. (eds.) ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom, June 09–11, 2014, p. 33. ACM (2014)

  13. Dimitrov, D., Raychev, V., Vechev, M., Koskinen, E.: Commutativity race detection. In: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’14) (2014)

  14. Ernst, G.W., Ogden, W.F.: Specification of abstract data types in modula. ACM Trans. Program. Lang. Syst. 2(4), 522–543 (1980)

    Article  Google Scholar 

  15. Ethereum. https://ethereum.org/. Accessed 26 Aug 2020

  16. Ettinger, R.: Program sliding. In: ECOOP 2012—Object-Oriented Programming—26th European Conference, Beijing, China, June 11–16, 2012. Proceedings, pp. 713–737 (2012)

  17. Flon, L., Misra, J.: A unified approach to the specification and verification of abstract data types. In: Proceedings of Specifications of Reliable Software Conference. IEEE Computer Society (1979)

  18. Gehr, T., Dimitrov, D., Vechev, M.T.: Learning commutativity specifications. In: Computer Aided Verification—27th International Conference, CAV 2015, San Francisco, CA, USA, July 18–24, 2015, Proceedings, Part I, pp. 307–323 (2015)

  19. Herlihy, M., Koskinen, E.: Transactional boosting: a methodology for highly concurrent transactional objects. In: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP’08) (2008)

  20. Hoare, C.A.R.: Software pioneers. In: Broy, M., Denert, E. (eds.) Proof of Correctness of Data Representations, pp. 385–396. Springer, New York (2002)

    Google Scholar 

  21. Hu, Y., Barrett, C., Goldberg, B.: Theory and algorithms for the generation and validation of speculative loop optimizations. In: Proceedings of the 2nd IEEE International Conference on Software Engineering and Formal Methods (SEFM ’04), pp. 281–289. IEEE Computer Society (Sept. 2004)

  22. Itzhaky, S., Gulwani, S., Immerman, N., Sagiv, M.: A simple inductive synthesis methodology and its applications. In: Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA ’10, New York, NY, USA, pp. 36–46 (2010). ACM

  23. Kim, D., Rinard, M.C.: Verification of semantic commutativity conditions and inverse operations on linked data structures. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, pp. 528–541. ACM (2011)

  24. Koskinen, E., Parkinson, M.J., Herlihy, M.: Coarse-grained transactions. In: Hermenegildo, M.V., Palsberg, J. (eds.) Proceedings of the 37th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2010, pp. 19–30. ACM (2010)

  25. Koskinen, E., Parkinson, M.J.: The push/pull model of transactions. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’15, Portland, OR, USA, June (2015)

  26. Kulkarni, M., Nguyen, D., Prountzos, D., Sui, X., Pingali, K.: Exploiting the commutativity lattice. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, pp. 542–555. ACM (2011)

  27. Leino, K.R.M.: Specifying and verifying programs in spec#. In: Proceedings of the 6th International Perspectives of Systems Informatics, Andrei Ershov Memorial Conference, PSI 2006, p. 20 (2006)

  28. Lipton, R.J.: Reduction: a method of proving properties of parallel programs. Commun. ACM 18(12), 717–721 (1975)

    Article  MathSciNet  Google Scholar 

  29. Meyer, B.: Applying “design by contract”. IEEE Comput. 25(10), 40–51 (1992)

    Article  Google Scholar 

  30. Ni, Y., Menon, V., Adl-Tabatabai, A., Hosking, A.L., Hudson, R.L., Moss, J.E.B., Saha, B., Shpeisman, T.: Open nesting in software transactional memory. In: Proceedings of the 12th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP 2007, pp. 68–78. ACM (2007)

  31. Rinard, M.C., Diniz, P.C.: Commutativity analysis: a new analysis technique for parallelizing compilers. ACM Trans. Program. Lang. Syst. 19(6), 942–991 (1997)

    Article  Google Scholar 

  32. Sergey, I., Hobor, A.: A concurrent perspective on smart contracts. In: 1st Workshop on Trusted Smart Contracts (2017)

  33. Servois homepage. http://cs.nyu.edu/~kshitij/projects/servois. Accessed 26 Aug 2020

  34. Servois source code. https://github.com/kbansal/servois. Accessed 26 Aug 2020

  35. Solar-Lezama, A., Jones, C. G., Bodík, R.: Sketching concurrent data structures. In: Proceedings of the ACM SIGPLAN 2008 Conference on Programming Language Design and Implementation PLDI 2008, pp. 136–148 (2008)

  36. Solidity programming language. https://solidity.readthedocs.io/en/develop/. Accessed 26 Aug 2020

  37. Tripp, O., Manevich, R., Field, J., Sagiv, M.: Janus: exploiting parallelism via hindsight. In: Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’12, New York, NY, USA, pp. 145–156 (2012). ACM

  38. Tripp, O., Yorsh, G., Field, J., Sagiv, M.: HAWKEYE: effective discovery of dataflow impediments to parallelization. In: Proceedings of the 26th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2011, part of SPLASH 2011, Portland, OR, USA, October 22–27, 2011, pp. 207–224 (2011)

  39. Vechev, M.T., Yahav, E., Yorsh, G.: Abstraction-guided synthesis of synchronization. In: Proceedings of the 37th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2010, pp. 327–338 (2010)

  40. Vechev, M.T., Yahav, E.: Deriving linearizable fine-grained concurrent objects. In: Proceedings of the ACM SIGPLAN 2008 Conference on Programming Language Design and Implementation, pp. 125–135 (2008)

  41. Wang, C., Yang, Z., Kahlon, V., Gupta, A.: Peephole partial order reduction. In: Vojnar, T., Zhang, L. (eds.) Tools and Algorithms for the Construction and Analysis of Systems, pp. 382–396. Springer, Berlin (2008)

    Chapter  Google Scholar 

  42. Weihl, W.: Commutativity-based concurrency control for abstract data types. IEEE Trans. Comput. 37(12), 1488–1505 (1988)

    Article  MathSciNet  Google Scholar 

  43. Xiao, T., Zhang, J., Zhou, H., Guo, Z., McDirmid, S., Lin, W., Chen, W., Zhou, L.: Nondeterminism in mapreduce considered harmful? an empirical study on non-commutative aggregators in mapreduce programs. In: Companion Proceedings of the 36th International Conference on Software Engineering, ICSE Companion 2014, New York, NY, USA, pp. 44–53 (2014). ACM

Download references

Author information

Authors and Affiliations

  1. Google Inc., Mountain View, CA, USA

    Kshitij Bansal

  2. Stevens Institute of Technology, Hoboken, NJ, USA

    Eric Koskinen

  3. Amazon.com, East Palo Alto, CA, USA

    Omer Tripp

Authors
  1. Kshitij Bansal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Koskinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Omer Tripp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Omer Tripp.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article extends our prior work [3], with the addition of proofs (Theorems 1 and 2), support for left- and right-movers (Sect. 6), and a new set of applications (Sect. 9), including memories and lock-based synchronization, transactional memory, distributed systems, refactoring, verification, and code synthesis.

K. Bansal was partially supported by NSF award #1228768. E. Koskinen was partially supported by NSF CCF Award #1421126, CCF Award #1618542, and CCF Award #1813745, and some of the work was done while he was at New York University and at IBM Research. K. Bansal was at New York University when part of the work was completed. The work by O. Tripp was done prior to him joining Amazon (while he was at IBM Research and Google).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bansal, K., Koskinen, E. & Tripp, O. Synthesizing Precise and Useful Commutativity Conditions. J Autom Reasoning 64, 1333–1359 (2020). https://doi.org/10.1007/s10817-020-09573-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10817-020-09573-w

Keywords

Search

Navigation