Abstract
Testing presents a daunting challenge for concurrent programs, as non-deterministic scheduling defeats reproducibility. The problem is even harder if, rather than testing entire systems, one tries to test individual components, for example to assess them for thread-safety. We present demonic testing, a technique combining the tangible results of unit testing with the rigour of formal rely-guarantee reasoning to provide deterministic unit testing for concurrent programs. Deterministic execution is provided by abstracting threads away via rely-guarantee reasoning, and replacing them with “demonic” sequences of interfering instructions that drive the program to break invariants. Demonic testing reuses existing unit tests to drive the routine under test, using the execution to discover demonic interference. Programs carry contract-based rely-guarantee style specifications to express what sort of thread interference should be tolerated. Aiding the demonic testing technique is an interference synthesis tool we have implemented based on SMT solving. The technique is shown to find errors in contracted versions of several benchmark applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Araujo, W., Briand, L., Labiche, Y.: On the effectiveness of contracts as test oracles in the detection and diagnosis of race conditions and deadlocks in concurrent object-oriented software. In: Proc. ESEM 2011. IEEE Computer Society (2011)
Barnett, M., Chang, B.-Y.E., DeLine, R., Jacobs, B., Leino, K.R.M.: Boogie: A Modular Reusable Verifier for Object-Oriented Programs. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.-P. (eds.) FMCO 2005. LNCS, vol. 4111, pp. 364–387. Springer, Heidelberg (2006)
Blundell, C., Giannakopoulou, D., Pǎsǎreanu, C.S.: Assume-guarantee testing. In: Proc. SAVCBS 2005. ACM (2005)
Choi, J.-D., Srinivasan, H.: Deterministic replay of Java multithreaded applications. In: Proc. SPDT 1998, pp. 48–59. ACM (1998)
Code contracts (2011), http://research.microsoft.com/en-us/projects/contracts/
Collection of Concurrency Bugs (2011), http://www.eecs.umich.edu/~jieyu/bugs.html
Demonic test case downloads (2011), http://se.inf.ethz.ch/people/west/demonic-cases/
Dingel, J.: Computer-assisted assume/guarantee reasoning with VeriSoft. In: Proc. ICSE 2003, pp. 138–148. IEEE Computer Society (2003)
Dutertre, B., de Moura, L.: A Fast Linear-Arithmetic Solver for DPLL(T). In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 81–94. Springer, Heidelberg (2006)
Edelstein, O., Farchi, E., Goldin, E., Nir, Y., Ratsaby, G., Ur, S.: Framework for testing multi-threaded Java programs. Concurrency and Computation: Practice and Experience 15(3-5), 485–499 (2003)
EVE project (2011), https://svn.eiffel.com/eiffelstudio/branches/eth/eve/
Godefroid, P.: Model checking for programming languages using VeriSoft. In: Proc. POPL 1997, pp. 174–186. ACM (1997)
Jones, C.B.: Development Methods for Computer Programs including a Notion of Interference. PhD thesis, Oxford University (June 1981)
Kundu, S., Ganai, M.K., Wang, C.: Contessa: Concurrency Testing Augmented with Symbolic Analysis. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 127–131. Springer, Heidelberg (2010)
Leavens, G.T., Baker, A.L., Ruby, C.: Preliminary design of JML: A behavioral interface specification language for Java. SIGSOFT Software Engineering Notes 31, 1–38 (2006)
McDermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld, D., Wilkins, D.: PDDL: The planning domain definition language. Technical Report CVC TR-98-003, Yale Center for Computational Vision and Control (1998)
Meyer, B.: Object-Oriented Software Construction, 2nd edn. Prentice-Hall (1997)
Meyer, B., Fiva, A., Ciupa, I., Leitner, A., Wei, Y., Stapf, E.: Programs that test themselves. IEEE Computer 42, 46–55 (2009)
Musuvathi, M., Qadeer, S., Ball, T., Basler, G., Nainar, P.A., Neamtiu, I.: Finding and reproducing Heisenbugs in concurrent programs. In: Proc. OSDI 2008, pp. 267–280. USENIX Association (2008)
Narayanasamy, S., Wang, Z., Tigani, J., Edwards, A., Calder, B.: Automatically classifying benign and harmful data racesallusing replay analysis. ACM SIGPLAN Notices 42(6), 22–31 (2007)
Park, S., Lu, S., Zhou, Y.: CTrigger: Exposing atomicity violation bugs from their hiding places. In: Proc. ASPLOS 2009, pp. 25–36. ACM (2009)
Sen, K.: Race directed random testing of concurrent programs. In: Proc. PLDI 2008, pp. 11–21. ACM (2008)
Sen, K., Agha, G.: CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools. In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 419–423. Springer, Heidelberg (2006)
Shacham, O., Bronson, N.G., Aiken, A., Sagiv, M., Vechev, M.T., Yahav, E.: Testing atomicity of composed concurrent operations. In: Proc. OOPSLA 2011, pp. 51–64 (2011)
Wang, C., Said, M., Gupta, A.: Coverage guided systematic concurrency testing. In: Proc. ICSE 2011, pp. 221–230. ACM (2011)
Yu, J., Narayanasamy, S.: A case for an interleaving constrained shared-memory multi-processor. In: Proc. ICSA 2009, pp. 325–336. ACM (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
West, S., Nanz, S., Meyer, B. (2012). Demonic Testing of Concurrent Programs. In: Aoki, T., Taguchi, K. (eds) Formal Methods and Software Engineering. ICFEM 2012. Lecture Notes in Computer Science, vol 7635. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34281-3_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-34281-3_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34280-6
Online ISBN: 978-3-642-34281-3
eBook Packages: Computer ScienceComputer Science (R0)