Abstract
Concolic testing is a method for test input generation where a given program is executed both concretely and symbolically at the same time. This paper introduces the LIME Concolic Tester (LCT), an open source concolic testing tool for sequential Java programs. It discusses the design choices behind LCT as well as its use in automated unit test generation for the JUnit testing framework. As the main experimental contribution we report on an empirical evaluation of LCT for testing smart card Java applets. In particular, we focus on the problem of differential testing, where a Java class implementation is tested against a reference implementation. Two different concolic unit test generation approaches are presented and their effectiveness is compared with random testing. The experiments show that concolic testing is able to find significantly more bugs than random testing in the testing domain at hand.
Work financially supported by Tekes - Finnish Funding Agency for Technology and Innovation, Conformiq Software, Elektrobit, Nokia, Space Systems Finland, and Academy of Finland (projects 126860 and 128050).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Godefroid, P., Klarlund, N., Sen, K.: DART: Directed automated random testing. In: Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI 2005), pp. 213–223. ACM, New York (2005)
Sen, K.: Scalable automated methods for dynamic program analysis. Doctoral thesis, University of Illinois (2006)
Sen, K., Agha, G.: CUTE and jCUTE: Concolic unit testing and explicit path model-checking tools. In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 419–423. Springer, Heidelberg (2006)
Cadar, C., Ganesh, V., Pawlowski, P.M., Dill, D.L., Engler, D.R.: EXE: automatically generating inputs of death. In: Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS 2006), pp. 322–335. ACM, New York (2006)
Tillmann, N., de Halleux, J.: Pex – White box test generation for .NET. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 134–153. Springer, Heidelberg (2008)
Cadar, C., Dunbar, D., Engler, D.R.: KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2008), pp. 209–224. USENIX Association (2008)
Chen, Z.: Java Card Technology for Smart Cards: Architecture and Programmer’s Guide. Prentice-Hall, Englewood Cliffs (2000)
Sun Microsystems: Java Card Development Kit 2.2.2 (2009), http://java.sun.com/javacard/devkit
Brummayer, R., Biere, A.: Boolector: An efficient SMT solver for bit-vectors and arrays. In: Kowalewski, S., Philippou, A. (eds.) TACAS 2009. LNCS, vol. 5505, pp. 174–177. Springer, Heidelberg (2009)
Kähkönen, K., Lampinen, J., Heljanko, K., Niemelä, I.: The LIME Interface Specification Language and Runtime Monitoring Tool. In: Peled, D. (ed.) RV 2009. LNCS, vol. 5779, pp. 93–100. Springer, Heidelberg (2009)
Godefroid, P., Levin, M.Y., Molnar, D.A.: Automated whitebox fuzz testing. In: Proceedings of the Network and Distributed System Security Symposium, NDSS 2008, pp. 151–166. The Internet Society (2008)
Godefroid, P., Levin, M.Y., Molnar, D.A.: Active property checking. In: Proceedings of the 8th ACM & IEEE International Conference on Embedded Software, EMSOFT 2008, pp. 207–216. ACM, New York (2008)
Majumdar, R., Sen, K.: Hybrid concolic testing. In: Proceedings of the 29th International Conference on Software Engineering (ICSE 2007), pp. 416–426. IEEE Computer Society, Los Alamitos (2007)
Molnar, D., Li, X.C., Wagner, D.A.: Dynamic test generation to find integer bugs in x86 binary Linux programs. In: Proceedings of the 18th USENIX Security Symposium (USENIX Security 2009), pp. 67–81. USENIX Association (2009)
Dutertre, B., de Moura, L.: A Fast Linear-Arithmetic Solver for DPLL(T). In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 81–94. Springer, Heidelberg (2006)
de Moura, L.M., Bjørner, N.: Z3: An efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)
Lampinen, J., Liedes, S., Kähkönen, K., Kauttio, J., Heljanko, K.: Interface specification methods for software components. Technical Report TKK-ICS-R25, Helsinki University of Technology, Department of Information and Computer Science, Espoo, Finland (December 2009)
Vallée-Rai, R., Co, P., Gagnon, E., Hendren, L.J., Lam, P., Sundaresan, V.: Soot - a Java bytecode optimization framework. In: Proceedings of the 1999 Conference of the Centre for Advanced Studies on Collaborative Research (CASCON 1999), p. 13. IBM (1999)
Kähkönen, K.: Automated test generation for software components. Technical Report TKK-ICS-R26, Helsinki University of Technology, Department of Information and Computer Science, Espoo, Finland (December 2009)
Factor, M., Schuster, A., Shagin, K.: Instrumentation of standard libraries in object-oriented languages: The twin class hierarchy approach. In: Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2004), pp. 288–300. ACM, New York (2004)
Person, S., Dwyer, M.B., Elbaum, S.G., Pasareanu, C.S.: Differential symbolic execution. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT FSE 2008), pp. 226–237. ACM, New York (2008)
Ma, Y.S., Offutt, J., Kwon, Y.R.: MuJava: An automated class mutation system. Software Testing, Verification and Reliability 15(2), 97–133 (2005)
Kindermann, R.: Testing a Java Card applet using the LIME Interface Test Bench: A case study. Technical Report TKK-ICS-R18, Helsinki University of Technology, Department of Information and Computer Science, Espoo, Finland (September 2009)
Holmström, P., Höglund, S., Sirén, L., Porres, I.: Evaluation of Specification-based Testing Approaches. Technical report, Åbo Akademi University, Department of Information Technologies (September 2009), https://poseidon.cs.abo.fi/trac/gaudi/lime/raw-attachment/wiki/MainResults/t34-report.pdf
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kähkönen, K., Kindermann, R., Heljanko, K., Niemelä, I. (2010). Experimental Comparison of Concolic and Random Testing for Java Card Applets . In: van de Pol, J., Weber, M. (eds) Model Checking Software. SPIN 2010. Lecture Notes in Computer Science, vol 6349. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16164-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-16164-3_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16163-6
Online ISBN: 978-3-642-16164-3
eBook Packages: Computer ScienceComputer Science (R0)