Software Quality Journal

, Volume 23, Issue 1, pp 171–202

PBCOV: a property-based coverage criterion

  • Kassem Fawaz
  • Fadi Zaraket
  • Wes Masri
  • Hamza Harkous

DOI: 10.1007/s11219-014-9237-3

Cite this article as:
Fawaz, K., Zaraket, F., Masri, W. et al. Software Qual J (2015) 23: 171. doi:10.1007/s11219-014-9237-3


Coverage criteria aim at satisfying test requirements and compute metrics values that quantify the adequacy of test suites at revealing defects in programs. Typically, a test requirement is a structural program element, and the coverage metric value represents the percentage of elements covered by a test suite. Empirical studies show that existing criteria might characterize a test suite as highly adequate, while it does not actually reveal some of the existing defects. In other words, existing structural coverage criteria are not always sensitive to the presence of defects. This paper presents PBCOV, a Property-Based COVerage criterion, and empirically demonstrates its effectiveness. Given a program with properties therein, static analysis techniques, such as model checking, leverage formal properties to find defects. PBCOV is a dynamic analysis technique that also leverages properties and is characterized by the following: (a) It considers the state space of first-order logic properties as the test requirements to be covered; (b) it uses logic synthesis to compute the state space; and (c) it is practical, i.e., computable, because it considers an over-approximation of the reachable state space using a cut-based abstraction.We evaluated PBCOV using programs with test suites comprising passing and failing test cases. First, we computed metrics values for PBCOV and structural coverage using the full test suites. Second, in order to quantify the sensitivity of the metrics to the absence of failing test cases, we computed the values for all considered metrics using only the passing test cases. In most cases, the structural metrics exhibited little or no decrease in their values, while PBCOV showed a considerable decrease. This suggests that PBCOV is more sensitive to the absence of failing test cases, i.e., it is more effective at characterizing test suite adequacy to detect defects, and at revealing deficiencies in test suites.


Software testing Coverage criteria Property-based coverage State space coverage Specification-based coverage Test suite evaluation Reachability analysis Logic synthesis 

Supplementary material

11219_2014_9237_MOESM1_ESM.pdf (322 kb)
Supplementary material 1 (PDF 323 kb)

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Kassem Fawaz
    • 1
  • Fadi Zaraket
    • 2
  • Wes Masri
    • 2
  • Hamza Harkous
    • 3
  1. 1.University of MichiganAnn ArborUSA
  2. 2.American University of BeirutBeirutLebanon
  3. 3.Swiss Federal Institute of Technology in Lausanne (EPFL)LausanneSwitzerland

Personalised recommendations