Skip to main content
SpringerLink
Log in

Coverage Criteria for State Based Specifications

  • Chapter
Formal Methods and Testing

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 4949))

Abstract

Test engineers often face the task of developing a set of test cases that are appropriate for a given software artefact. The software testing literature is replete with testing methods tailored to the various specification, design, and implementation methods used in software engineering. This chapter takes a novel inverted view. Instead of starting with the specific artefact at hand, we identify two general sets of coverage criteria – one based on graphs and the other based on predicates. We then ask two questions with respect to the specific artefact under test: (1) What graphs are suitable abstractions of the artefact for the purpose of testing? (2) What predicates should be extracted from this artefact for the purpose of testing? Combining the answers to these two questions with the standard graph-based and logic-based coverage criteria yields test requirements. The test engineer can then proceed to identify test cases that satisfy the various requirements. This chapter illustrate this technique in the context of testing software that is modelled by state-based specifications. We present a representative sample of graph-based and logic-based test coverage criteria. We extract appropriate graphs and predicates from state based specifications and apply the coverage criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akers, S.B.: On a theory of boolean functions. Journal Society Industrial Applied Mathematics 7(4), 487–498 (1959)

    Article  MATH  Google Scholar 

  2. Ammann, P., Offutt, J., Huang, H.: Coverage criteria for logical expressions. In: Proceedings of the 14th International Symposium on Software Reliability Engineering, Denver, CO, November 2003, pp. 99–107. IEEE Computer Society Press, Los Alamitos (2003)

    Google Scholar 

  3. Atlee, J.M.: Native model-checking of SCR requirements. In: Fourth International SCR Workshop (November 1994)

    Google Scholar 

  4. Atlee, J.M., Gannon, J.: State-based model checking of event-driven system requirements. IEEE Transactions on Software Engineering 19(1), 24–40 (1993)

    Article  Google Scholar 

  5. Binder, R.: Testing Object-oriented Systems. Addison-Wesley Publishing Company Inc., New York (2000)

    Google Scholar 

  6. Borzovs, J., Kalniņš, A., Medvedis, I.: Automatic construction of test sets: Practical approach. In: Barzdins, J., Bjorner, D. (eds.) Baltic Computer Science. LNCS, vol. 502, pp. 360–432. Springer, Heidelberg (1991)

    Chapter  Google Scholar 

  7. Boyer, R.S., Elpas, B., Levitt, K.N.: Select–a formal system for testing and debugging programs by symbolic execution. In: Proceedings of the International Conference on Reliable Software, June 1975, SIGPLAN Notices, vol. 10(6) (1975)

    Google Scholar 

  8. Briand, L., Labiche, Y.: A UML-based approach to system testing. In: Gogolla, M., Kobryn, C. (eds.) UML 2001. LNCS, vol. 2185, pp. 194–208. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  9. Special Interest Group in Software Testing British Computer Society. Standard for Software Component Testing, Working Draft 3.3. British Computer Society (1997), http://www.rmcs.cranfield.ac.uk/~cised/sreid/BCS_SIG/

  10. Cheatham, T.E., Holloway, G.H., Townley, J.A.: Symbolic evaluation and the analysis of programs. IEEE Transactions on Software Engineering 4 (July 1979)

    Google Scholar 

  11. Cherniavsky, J.C.: On finding test data sets for loop free programs. Information Processing Letters 8(2), 106–107 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  12. Chilenski, J.J., Miller, S.P.: Applicability of modified condition/decision coverage to software testing. Software Engineering Journal 9(5), 193–200 (1994)

    Article  Google Scholar 

  13. Chilenski, J., Richey, L.A.: Definition for a masking form of modified condition decision coverage (MCDC). Technical report, Boeing, Seattle, WA (1997), http://www.boeing.com/nosearch/mcdc/

  14. Chilenski, J.J.: Personal communication (March 2003)

    Google Scholar 

  15. Chow, T.: Testing software designs modeled by finite-state machines. IEEE Transactions on Software Engineering SE-4(3), 178–187 (1978)

    Article  Google Scholar 

  16. Clarke, L.A.: A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering 2(3), 215–222 (1976)

    Article  Google Scholar 

  17. Clarke, L.A., Richardson, D.J.: Applications of symbolic evaluation. The Journal of Systems and Software 5(1), 15–35 (1985)

    Article  Google Scholar 

  18. Darringer, J.A., King, J.C.: Applications of symbolic execution to program testing. IEEE Computer 4 (April 1978)

    Google Scholar 

  19. DeMillo, R.A., Guindi, D.S., King, K.N., McCracken, W.M., Offutt, J.: An extended overview of the Mothra software testing environment. In: Proceedings of the Second Workshop on Software Testing, Verification, and Analysis, Banff, Alberta, July 1988, pp. 142–151. IEEE Computer Society Press, Los Alamitos (1988)

    Chapter  Google Scholar 

  20. DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: Help for the practicing programmer. IEEE Computer 11(4), 34–41 (1978)

    Google Scholar 

  21. DeMillo, R.A., Offutt, J.: Constraint-based automatic test data generation. IEEE Transactions on Software Engineering 17(9), 900–910 (1991)

    Article  Google Scholar 

  22. Dupuy, A., Leveson, N.: An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software. In: Proceedings of the Digital Aviations Systems Conference (DASC) (October 2000)

    Google Scholar 

  23. Fairley, R.E.: An experimental program testing facility. IEEE Transactions on Software Engineering SE-1, 350–3571 (1975)

    Google Scholar 

  24. Ferguson, R., Korel, B.: The chaining approach for software test data generation. ACM Transactions on Software Engineering Methodology 5(1), 63–86 (1996)

    Article  Google Scholar 

  25. Fujiwara, S., Bochman, G., Khendek, F., Amalou, M., Ghedasmi, A.: Test selection based on finite state models. IEEE Transactions on Software Engineering 17(6), 591–603 (1991)

    Article  Google Scholar 

  26. Gonenc, G.: A method for the design of fault-detection experiments. IEEE Transactions on Computers C-19, 155–558 (1970)

    Article  Google Scholar 

  27. Gourlay, J.S.: A mathematical framework for the investigation of testing. IEEE Transactions on Software Engineering 9(6), 686–709 (1983)

    Article  Google Scholar 

  28. Henninger, K.: Specifying software requirements for complex systems: New techniques and their applications. IEEE Transactions on Software Engineering SE-6(1), 2–12 (1980)

    Article  Google Scholar 

  29. Howden, W.E.: Methodology for the generation of program test data. IEEE Transactions on Software Engineering SE-24 (May 1975)

    Google Scholar 

  30. Howden, W.E.: Symbolic testing and the DISSECT symbolic evaluation system. IEEE Transactions on Software Engineering 3(4) (July 1977)

    Google Scholar 

  31. Huang, J.C.: An approach to program testing. ACM Computing Surveys 7(3), 113–128 (1975)

    Article  Google Scholar 

  32. Jasper, R., Brennan, M., Williamson, K., Currier, B., Zimmerman, D.: Test data generation and feasible path analysis. In: Proceedings of the 1994 International Symposium on Software Testing, and Analysis, August 1994, August 1994, pp. 95–107. ACM Press, New York (1994)

    Chapter  Google Scholar 

  33. Jones, J.A., Harrold, M.J.: Test-suite reduction and prioritizaion for modified condition / decision coverage. IEEE Transactions on Software Engineering 29(3), 195–209 (2003)

    Article  Google Scholar 

  34. Kim, Y.G., Hong, H.S., Cho, S.M., Bae, D.H., Cha, S.D.: Test cases generation from UML state diagrams. IEE Proceedings – Software 146(4), 187–192 (1999)

    Article  Google Scholar 

  35. King, K.N., Offutt, J.: A Fortran language system for mutation-based software testing. Software – Practice and Experience 21(7), 685–718 (1991)

    Article  Google Scholar 

  36. Korel, B.: Automated software test data generation. IEEE Transactions on Software Engineering 16(8), 870–879 (1990)

    Article  Google Scholar 

  37. Korel, B.: Dynamic method for software test data generation. Software Testing, Verification, and Reliability 2(4), 203–213 (1992)

    Article  Google Scholar 

  38. Kosaraju, S.: Analysis of structured programs. Journal of Computer Systems and Science 9, 232–255 (1974)

    MATH  MathSciNet  Google Scholar 

  39. Kuhn, D.R.: Fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering Methodology 8(4), 411–424 (1999)

    Article  Google Scholar 

  40. Legard, H., Marcotty, M.: A generalogy of control structures. Communications of the ACM 18, 629–639 (1975)

    Article  Google Scholar 

  41. McCabe, T.J.: A complexity measure. IEEE Transactions on Software Engineering SE-2(4), 308–320 (1976)

    Article  MathSciNet  Google Scholar 

  42. McMinn, P.: Search-based software test data generation: A survey. Software Testing, Verification, and Reliability 13(2), 105–156 (2004)

    Article  Google Scholar 

  43. Myers, G.: The Art of Software Testing. John Wiley and Sons, New York (1979)

    Google Scholar 

  44. Naito, S., Tsunoyama, M.: Fault detection for sequential machines by transition tours. In: Proceedings Fault Tolerant Computing Systems, pp. 238–243. IEEE Computer Society Press, Los Alamitos (1981)

    Google Scholar 

  45. Offutt, J., Jin, Z., Pan, J.: The dynamic domain reduction approach to test data generation. Software – Practice and Experience 29(2), 167–193 (1999)

    Article  Google Scholar 

  46. Offutt, J., Liu, S., Abdurazik, A., Ammann, P.: Generating test data from state-based specifications. Software Testing, Verification, and Reliability 13(1), 25–53 (2003)

    Article  Google Scholar 

  47. Paige, M.R.: Program graphs, an algebra, and their implication for programming. IEEE Transactions on Software Engineering SE-1(3), 286–291 (1975)

    MathSciNet  Google Scholar 

  48. Paige, M.R.: In: Proc. of IEEE 2nd Annual International Computer Software and Applications Conference (COMPSAC 1978), pp. 527–532 (1978)

    Google Scholar 

  49. Pimont, S., Rault, J.C.: A software reliability assessment based on a structural behavioral analysis of programs. In: Proceedings of the Second International Conference on Software Engineering, San Francisco, CA (October 1976)

    Google Scholar 

  50. Ramamoorthy, C.V., Ho, S.F., Chen, W.T.: On the automated generation of program test data. IEEE Transactions on Software Engineering 2(4), 293–300 (1976)

    Article  Google Scholar 

  51. RTCA-DO-178B. Software considerations in airborne systems and equipment certification (December 1992)

    Google Scholar 

  52. Sabnani, K., Dahbura, A.: A protocol testing procedure. Computer Networks and ISDN Systems 14(4), 285–297 (1988)

    Article  Google Scholar 

  53. Tip, F.: A survey of program slicing techniques. Technical report CS-R-9438, Computer Science/Department of Software Technology, Centrum voor Wiskunde en Informatica (1994)

    Google Scholar 

  54. Vilkomir, S.A., Bowen, J.P.: Reinforced condition/decision coverage (RC/DC): A new criterion for software testing. In: Bert, D., P. Bowen, J., C. Henson, M., Robinson, K. (eds.) B 2002 and ZB 2002. LNCS, vol. 2272, pp. 295–313. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  55. Weiser, M.: Program slicing. IEEE Transactions on Software Engineering SE-10(4), 352–357 (1984)

    Article  Google Scholar 

  56. Weyuker, E., Goradia, T., Singh, A.: Automatically generating test data from a boolean specification. IEEE Transactions on Software Engineering 20(5), 353–363 (1994)

    Article  MATH  Google Scholar 

  57. Weyuker, E.J., Weiss, S.N., Hamlet, R.G.: Data flow-based adequacy analysis for languages with pointers. In: Proceedings of the Fourth Symposium on Software Testing, Analysis, and Verification, Victoria, British Columbia, Canada, October 1991, pp. 74–86. IEEE Computer Society Press, Los Alamitos (1991)

    Google Scholar 

  58. White, L., Wiszniewski, B.: Path testing of computer programs with loops using a tool for simple loop patterns. Software – Practice and Experience 21(10), 1075–1102 (1991)

    Article  Google Scholar 

  59. Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Computing Surveys 29(4), 366–427 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information and Software Engineering, George Mason University, USA

    Paul Ammann, Jeff Offutt & Wuzhi Xu

Authors
  1. Paul Ammann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeff Offutt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wuzhi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Robert M. Hierons Jonathan P. Bowen Mark Harman

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ammann, P., Offutt, J., Xu, W. (2008). Coverage Criteria for State Based Specifications. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds) Formal Methods and Testing. Lecture Notes in Computer Science, vol 4949. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78917-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78917-8_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78916-1

  • Online ISBN: 978-3-540-78917-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation