Abstract
Systematic validation and verification of safety-critical software is of crucial importance. A key precaution is intensive testing at several levels, from the entire system down to individual functional elements, the latter often carried out as unit testing. This paper presents results from a unit test performed on a C++ package from a testbed of a safety critical application at the ARC Seibersdorf research lab. After outlining the test environment and relevant characteristics of the tested software package, a detailed analysis of the test results is given. This analysis comprises fault categorization, fault distribution, relations between software metrics (like McCabe’s cyclomatic complexity or the risk categories of NASA SATC), software faults, and testing efforts, and yields clues about the significance of these measures for fault probabilities. A summary of the findings and related work conclude the paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adams, E.N.: Optimising Preventive Service of Software Products. IBM J. Res. Develop. 28(1) (January 1984)
Basili, V.R., Perricone, B.T.: Software Errors and Complexity: An Empirical Investigation. Communications of the ACM 27(1) (January 1984)
Briand, L., Labiche, Y.: Empirical Studies of Software Testing Techniques: Challenges, Practical Strategies, and Future Research. In: Workshop on Empirical Research in Software Testing, WERST 2004 (2004), Online [PDF Document] (16/02/2005), http://www.sce.carleton.ca/squall/WERST2004/accepted_papers/LB-YL.pdf
Ellims, M., Bridges, J., Ince, D.C.: Unit Testing in Practice. In: Proceedings of the 15th International Symposium on Software Reliability Engineering, ISSRE 2004 (2004), 1071-9458/04
Fenton, N.E., Ohlsson, N.: Quantitative Analysis of Faults and Failures in a Complex Software System. IEEE Transactions on Software Engineering 26(8), 797–814 (2000)
Gittens, M., Lutfiyya, H., Bauer, M., Godwin, D., Yong, W.K., Gupta, P.: An Empirical Evaluation of System and Regression Testing. In: University of Western Ontario and IBM, Online (PDF Document) (2/16/2005), http://www.cleanscape.net/docs_lib/ibm_cas02.pdf
Hatton, L.: Software Failures: Follies and Fallacies. IEE Review 43(2), 49–52 (1997)
IEC 61508, Functional Safety of Electric/Electronic/Programmable Electronic Systems, Part 1 – Part 7 (1998 – 2001)
IEEE 982.2-1988, IEEE Guide for the Use of IEEE Standard Dictionary of Measures to Produce Reliable Software. IEEE Press, New York (June 1989)
IPL, This web-page also provides links to technical papers, http://www.ipl.com/products/tools/pt400.uk.php
Kenney, G.Q., Vouk, M.A.: Measuring the Field Quality of Wide-Distribution Commercial Software. In: Proc. of 3rd Int. Symposium on Software Reliability Engineering, Research Triangle Park, NC, USA, pp. 351–357 (1992), Online (PDF-Document) (2/16/2005), http://renoir.csc.ncsu.edu/Faculty/Vouk/Papers/Kenney/Kenney_ISSRE92.pdf
McCabe, T.J.: Structural Testing: A Software Testing Methodology Using the Cyclomatic Complexity Metric, National Bureau of Standards, Washington (1982)
Moschitz, M., Thuswald, M., Weber, E.: WWQM2 im Jahr 2000 (in German only). Internal report, ARC Seibersdorf research (2001)
Moschitz, M., Studer, M., Fuhrmann, S., Weber, E., Zoffmann, G.: WWQM im Jahr 2002 (in German only). Internal report, ARC Seibersdorf research (2003)
Ohlsson, N., Alberg, H.: Predicting Fault-Prone Software Modules in Telefone Switches. IEEE Transactions on Software Engineering 22(12) (December 1996)
Ramberger, S., Gruber, T., Herzner, W.: Experience Report: Fault Distribution in Safety-Critical Software and Software Risk Analysis Based on Unit Tests. In: Dadam, S., Reichert, M. (Hrsg.) (eds.), INFORMATIK 2004, Band 1. Lecture Notes in Informatics (LNI) - Proceedings, Series of the Gesellschaft für Informatik (GI), vol. P-50 (2004)
Rosenberg, L.H.: Applying and Interpreting Object Oriented Metrics. In: Software Technology Conference (April 1998), http://satc.gsfc.nasa.gov/metrics , as of Feburary 2003
Software Assurance Technology Center. as of February 2003, http://satc.gsfc.nasa.gov
Public WWQM test version at, with ”guest” account (no password), http://www.wwqm.at
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Herzner, W., Ramberger, S., Länger, T., Reumann, C., Gruber, T., Sejkora, C. (2005). Comparing Software Measures with Fault Counts Derived from Unit-Testing of Safety-Critical Software. In: Winther, R., Gran, B.A., Dahll, G. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2005. Lecture Notes in Computer Science, vol 3688. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11563228_7
Download citation
DOI: https://doi.org/10.1007/11563228_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29200-5
Online ISBN: 978-3-540-32000-5
eBook Packages: Computer ScienceComputer Science (R0)