Advertisement

Assisted Coverage Closure

  • Adam Nellis
  • Pascal Kesseli
  • Philippa Ryan Conmy
  • Daniel Kroening
  • Peter Schrammel
  • Michael Tautschnig
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9690)

Abstract

Malfunction of safety-critical systems may cause damage to people and the environment. Software within those systems is rigorously designed and verified according to domain specific guidance, such as ISO26262 for automotive safety. This paper describes academic and industrial co-operation in tool development to support one of the most stringent of the requirements — achieving full code coverage in requirements-driven testing. We present a verification workflow supported by a tool that integrates the coverage measurement tool RapiCover with the test-vector generator FShell. The tool assists closing the coverage gap by providing the engineer with test vectors that help in debugging coverage-related code quality issues and creating new test cases, as well as justifying the presence of unreachable parts of the code in order to finally achieve full effective coverage according to the required criteria. We illustrate the tool’s practical utility on automotive industry benchmarks. It generates 8\(\times \) more MC/DC coverage than random search.

Keywords

Test Suite Random Search Test Vector Code Coverage Bound Model Check 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    ISO26262 road vehicles – functional safety, Part 6: Product development at the software level, Annex B: Model-based development (2011)Google Scholar
  2. 2.
    Biere, A., Cimatti, A., Clarke, E., Zhu, Y.: Symbolic model checking without BDDs. In: Cleaveland, W.R. (ed.) TACAS 1999. LNCS, vol. 1579, p. 193. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  3. 3.
    Clarke, E., Kroening, D., Lerda, F.: A tool for checking ANSI-C programs. In: Jensen, K., Podelski, A. (eds.) TACAS 2004. LNCS, vol. 2988, pp. 168–176. Springer, Heidelberg (2004)Google Scholar
  4. 4.
    Dupuy, A., Leveson, N.: An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software. In: Digital Avionics Systems Conference, vol. 1, pp. 1B6/1–1B6/7 (2000)Google Scholar
  5. 5.
    Fraser, G., Wotawa, F., Ammann, P.: Testing with model checkers: a survey. Softw. Test., Verification Reliab. 19(3), 215–261 (2009)CrossRefGoogle Scholar
  6. 6.
    Ghani, K., Clark, J.A.: Automatic test data generation for multiple condition and MCDC coverage. In: ICSEA, pp. 152–157 (2009)Google Scholar
  7. 7.
    Holzer, A., Schallhart, C., Tautschnig, M., Veith, H.: FShell: systematic test case generation for dynamic analysis and measurement. In: Gupta, A., Malik, S. (eds.) CAV 2008. LNCS, vol. 5123, pp. 209–213. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. Trans. Software Eng. 37(5), 649–678 (2011)CrossRefGoogle Scholar
  9. 9.
    Kandl, S., Kirner, R.: Error detection rate of MC/DC for a case study from the automotive domain. In: Min, S.L., Pettit, R., Puschner, P., Ungerer, T. (eds.) SEUS 2010. LNCS, vol. 6399, pp. 131–142. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Kroening, D., Strichman, O.: Efficient computation of recurrence diameters. In: Zuck, L.D., Attie, P.C., Cortesi, A., Mukhopadhyay, S. (eds.) VMCAI 2003. LNCS, vol. 2575, pp. 298–309. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. 11.
    Rierson, L.: Developing Safety-Critical Software: A Practical Guide for Aviation Software and DO-178C Compliance. CRC Press, Boca Raton (2013). Chapter 14.3 Potential Risks of Model-Based Development and VerificationGoogle Scholar
  12. 12.
    Schrammel, P., Melham, T., Kroening, D.: Chaining test cases for reactive system testing. In: Yenigün, H., Yilmaz, C., Ulrich, A. (eds.) ICTSS 2013. LNCS, vol. 8254, pp. 133–148. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    Sheeran, M., Singh, S., Stålmarck, G.: Checking safety properties using induction and a SAT-solver. In: Johnson, S.D., Hunt Jr., W.A. (eds.) FMCAD 2000. LNCS, vol. 1954, pp. 108–125. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  14. 14.
    Tallam, S., Gupta, N.: A concept analysis inspired greedy algorithm for test suite minimization. In: PASTE, pp. 35–42 (2005)Google Scholar
  15. 15.
    Zamli, K.Z., Al-Sewari, A.A., Hassin, M.H.M.: On test case generation satisfying the MC/DC criterion. Int. J. Adv. Soft Comput. Appl. 5(3) (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Adam Nellis
    • 1
  • Pascal Kesseli
    • 2
  • Philippa Ryan Conmy
    • 1
  • Daniel Kroening
    • 2
  • Peter Schrammel
    • 2
    • 4
  • Michael Tautschnig
    • 3
  1. 1.Rapita Systems LtdYorkUK
  2. 2.University of OxfordOxfordUK
  3. 3.Queen Mary University of LondonLondonUK
  4. 4.University of SussexBrightonUK

Personalised recommendations