Skip to main content
SpringerLink
Log in

FAST: a framework for automating statistics-based testing

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

To achieve software quality, testing is an essential component in all software development. It involves the execution of a deterministic software system with test data and a comparison of the results with the expected output, which must satisfy the users' requirements. This accounts for over 25% of the cost of a software development. Therefore, automation has considerable potential. The quality programming introduced by Cho can automatically generate data for testing, based on a so-called ‘SIAD tree’ which is used to represent the hierarchical and ‘network’ relation between input elements and also incorporates rules into the tree for using the inputs. However, it lacks a clear framework which would show how automated testing can be achieved. To address this problem, we present a Framework for Automating Statistics-based Testing (FAST), which is an extension of the testing concept in quality programming to achieve automated testing. In FAST, we propose a SOAD tree, which is similar to the structure of the SIAD tree, to describe the syntactic structure of the product unit and its defectiveness. Based on this tool, the inspection of test results can be automatically achieved by lexical and syntax analysis. The implementation of automated software testing or Command File Interpreter (CFI) software which incorporates the framework is also described.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J.D. Musa, and W.W. Everett. Software-reliability engineering: technology for the 1990s, 1EEE Software, 7(6) (1990) 36-43.

    Article  Google Scholar 

  2. M.S. Deutsch and R.R. Willis. Software Quality Engineering - A Total Technical and Management Approach (Prentice Hall International, London, 1988).

    Google Scholar 

  3. D. Ince. The automatic generation of test data, Computer Journal, 30(1987) 63-69.

    Google Scholar 

  4. G.J. Myers. The Art of Software Testing (John Wiley & Sons, New York, 1978).

    Google Scholar 

  5. S. Norman. Software Testing Tools (Ovum Ltd, London, 1993).

    Google Scholar 

  6. B. Marre, P. Thévenod-Fosse, H. Waeselynck, P.L. Gall, and Y. Grouzet. An experimental evaluation of formal testing and statistical testing, in Predictably Dependable Computing Systems (Springer, London, 1995), pp. 273-281.

    Google Scholar 

  7. P.A. Curritt, M Dyer and H.D. Mills. Certifying the reliability of software, IEEE Transactions on Software Engineering, SE-12(1986) 3-11.

    Google Scholar 

  8. J.A. Whittaker and M.G. Thomason. A Markov chain model for statistical software testing, IEEE Transactions on Software Engineering, SE-20(1994) 812-824.

    Article  Google Scholar 

  9. M. Dyer. The Cleanroom Approach to Quality Software Development (John Wiley & Sons, New York, 1992).

    Google Scholar 

  10. P. Thévenod-Fosse, P. Waeselynck, H. and Y. Crouzet. Software statistical testing. In Predictably Dependable Computing Systems (Springer, London, 1995), pp. 253-272.

    Google Scholar 

  11. C.K. Cho. Quality Programming - Development and Testing Software with Statistical Quality Control (John Wiley & Sons, New York, 1988).

    Google Scholar 

  12. P. Thévenod-Fosse and H. Waeselynck. An investigation of statistical software testing, Journal of Software Testing, Verification and Reliability, 1(2) (1991), 5-25.

    Google Scholar 

  13. M.A. Ould and C. Unwin. Testing in Software Development (Cambridge University Press, Cambridge, 1986).

    Google Scholar 

  14. M. Roper. Software Testing (McGraw-Hill, London, 1994)

    Google Scholar 

  15. I. Sommerville. Software Engineering, 5th edn (Addison-Wesley, Wokingham, 1996).

    Google Scholar 

  16. H.V. Vliet. Software Engineering: Principles and Practice (John Wiley & Sons, New York, 1994.

    Google Scholar 

  17. W.S. Humphrey. Characterizing the software process: a maturity framework, IEEE Software, 5(2) (1988) 73-79.

    Article  Google Scholar 

  18. M.E. Fagan. Design and code inspections to reduce errors in program development, IBM Systems Journal, 15(1976) 182-211.

    Google Scholar 

  19. R.A. DeMillo, R.J. Lipton and A.J. Perlis. Social processes and the proofs of theorems and programs, Communications of the ACM, 22(1979) 271-280.

    Article  Google Scholar 

  20. M.E. Fagan. Advances in inspections, IEEE Transactions on Software Engineering, SE-12(1986) 744-751.

    Google Scholar 

  21. H.D. Mills, M. Dyer and R. Linger. Cleanroom software engineering, IEEE Software 4(1987) 19-25.

    Google Scholar 

  22. B. Beizer. Black-Box Testing (John Wiley & Sons, New York, 1995).

    Google Scholar 

  23. V.R. Basili and R.W. Selby. Comparing the effectiveness of software testing strategies, IEEE Transactions on Software Engineering, SE-13(1987) 1278-1296.

    Google Scholar 

  24. R.M. Poston. Automating Specification-Based Software Testing (IEEE Computer Society Press, Los Alamitos, CA, 1996).

    Google Scholar 

  25. E. Kit. Software Testing in the Real World: Improving the Process (Addison-Wesley, Wokingham, 1995).

    Google Scholar 

  26. D. Hamlet. Special section on software testing. Communication of the ACM, 31(6) (1988) 662-667.

    Article  Google Scholar 

  27. J.D. Musa and A.F. Ackerman. Quantifying software validation: when to stop testing? IEEE Software, 6(3) (1989) 19-27.

    Article  Google Scholar 

  28. D. Parnas, J.V. Schouwen and S.P. Kwan. Evaluation of safety-critical software. Communication of the CAM, 33(6) (1990) 636-648.

    Google Scholar 

  29. H.M. Hörcher, and J. Peleska. Using formal specifications to support software testing, Software Quality Journal, 4(1995) 309-327.

    Google Scholar 

  30. H. Zhu. A formal analysis of the subsume relation between software testing adequacy criteria, IEEE Transactions on Software Engineering, SE-22(1996) 248-255.

    Google Scholar 

  31. M. Deck. Cleanroom practice: a theme and variations, in Proceedings of the 9th International Software Quality Week (Software Research Institute, San Francisco, 1996).

    Google Scholar 

  32. I.C. Liu, R.D. Yang, H.D. Chu, F.H Liu, and C.S. Chang. Software statistical quality assurance, Technical Report, Institute for Information and Industry, Taiwan, 1992.

    Google Scholar 

  33. R.V. Megen, and D.B. Meyerhoff. Costs and benefits of early defect detection: experiences from developing client serve and host applications, Software Quality Journal, 4(1995) 247-256.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Software Reliability, Department of Computing Science, University of Newcastle upon Tyne, Newcastle upon Tyne, UK

    Huey-Der Chu & John E. Dobson

  2. Department of Information Management, National Defense Management College, Chungho, 23500, Taiwan

    I-Chiang Liu

Authors
  1. Huey-Der Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John E. Dobson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I-Chiang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chu, HD., Dobson, J.E. & Liu, IC. FAST: a framework for automating statistics-based testing. Software Quality Journal 6, 13–36 (1997). https://doi.org/10.1023/A:1018535229843

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018535229843

Search

Navigation