Science China Information Sciences

, 60:012105

An empirical study on constraint optimization techniques for test generation

  • Zhiyi Zhang
  • Zhenyu Chen
  • Ruizhi Gao
  • Eric Wong
  • Baowen Xu
Research Paper

DOI: 10.1007/s11432-015-0450-5

Cite this article as:
Zhang, Z., Chen, Z., Gao, R. et al. Sci. China Inf. Sci. (2017) 60: 012105. doi:10.1007/s11432-015-0450-5
  • 51 Downloads

Abstract

Constraint solving is a frequent, but expensive operation with symbolic execution to generate tests for a program. To improve the efficiency of test generation using constraint solving, four optimization techniques are usually applied to existing constraint solvers, which are constraint independence, constraint set simplification, constraint caching, and expression rewriting. In this paper, we conducted an empirical study, using these four constraint optimization techniques in a well known test generation tool KLEE with 77 GNU Coreutils applications, to systematically investigate how these optimization techniques affect the efficiency of test generation. The experimental results show that these constraint optimization techniques as well as their combinations cannot improve the efficiency of test generation significantly for ALL-SIZED programs. Moreover, we studied the constraint optimization techniques with respect to two static metrics, lines of code (LOC) and cyclomatic complexity (CC), of programs. The experimental results show that the “constraint set simplification” technique can improve the efficiency of test generation significantly for the programs with high LOC and CC values. The “constraint caching” optimization technique can improve the efficiency of test generation significantly for the programs with low LOC and CC values. Finally, we propose four hybrid optimization strategies and practical guidelines based on different static metrics.

Keywords

test generation symbolic execution constraint solving constraint optimization static metric 

约束优化技术在测试用例生成中的实证研究

创新点

我们使用KLEE执行了77个GNU Coreutils程序,用于系统的研究4种流行的约束优化技术如何影响测试用例生成的效率。结果表明,对于所有程序,这些约束优化技术及它们的组合不能大幅提高测试用例生成的效率。 此外,我们研究了程序的两个静态指标(代码行和圈复杂度)跟约束优化技术的关系。结果表明,对于大规模及高圈复杂度程序,使用“约束集简化”技术,对于小规模及低圈复杂度程序,使用“约束缓存”技术,都可以显著提高测试生成效率。最后,基于不同的静态指标,我们提出了四种混合优化策略和指导方针。

测试用例生成 符号执行 约束求解 约束优化 静态度量 
012105 

Copyright information

© Science China Press and Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Zhiyi Zhang
    • 1
  • Zhenyu Chen
    • 1
  • Ruizhi Gao
    • 2
  • Eric Wong
    • 2
  • Baowen Xu
    • 1
  1. 1.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.Department of Computer ScienceUniversity of Texas at DallasRichardsonUSA

Personalised recommendations