Property-Based Testing and Verification: A Catalog of Classroom Examples

Abstract

For the past a decade, John Hughes of Chalmers University and Quviq has pioneered property-based testing in debugging industrial software. Three undergraduate courses at the University of Oklahoma have paralleled this work, but on the pedagogical side and with the additional goal of verification of properties by mathematical proof, using both paper-and-pencil methods and mechanized logic. An essential part of both efforts, and probably the most important part, is the formalization in predicate logic of the expectations of software developers. This is central to engineering software quality in the defect dimension. This report discusses property-based testing in course work and provides some examples from the classroom. Hughes has observed that software properties useful for testing functional software often (1) compare the results of different ways of computing the same thing or (2) check that forward and inverse transformations restore the original data. What percentage of useful software properties fall into these categories? Our collection of examples from course records may help shed some light on how closely pedagogy on property-based testing matches observations of relative frequencies of various categories of properties in industry and may also serve as an educational aid in learning to use property-based testing effectively in software development.

Partially supported by National Science Foundation Award Number 1016532.