Prioritizing Tests for Fault Localization
Maritime Safety and Security Systems of Systems (MSS SoS) evolve dynamically during operation, i.e., at runtime. After each runtime evolution, the quality assurance of the integrated system of systems has to be verified again. It is therefore necessary to devise an appropriate verification strategy that not only achieves this goal, but also minimizes the cost, e.g., time, resources, disruption, of checking after each modification. During testing, test prioritization techniques heuristically select test cases to minimize the time to detect the presence of a fault. However, this obviates that once a fault has been detected, it must be localized and isolated/repaired. Test suites prioritized for fault detection can reduce the amount of useful information for fault localization, increasing the cost of fault localization, e.g., with respect to randomly chosen tests. In this chapter we introduce fault localization prioritization and two new test case prioritization heuristics that greatly reduce the cost of fault localization (up to 80%) with almost no increase on the fault detection cost.
KeywordsFault Detection Similarity Coefficient Fault Localization Information Gain Test Case Prioritization
This research has been carried out as a part of the Poseidon project at Thales under the responsibilities of the Embedded Systems Institute (ESI). This project is partially supported by the Dutch Ministry of Economic Affairs under the BSIK program.
- 1.Abreu R, Zoeteweij P, van Gemund AJC (2009) Spectrum-based multiple fault localization. In: ASE ’09: proceedings of the 2009 IEEE/ACM international conference on automated software engineering. IEEE Computer Society, Washington, DC, pp 88–99Google Scholar
- 3.González-Sánchez A, Abreu R, Gross H-G, van Gemund AJC (2011) Spectrum-based sequential diagnosis. In: 25th international conference on artificial intelligence (AAAI’11), pp 189–196, 2011. http://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/download/3565/3847.
- 4.González-Sánchez A, Piel E, Abreu R, Gross H-G, van Gemund AJC (2011) Prioritizing tests for software fault diagnosis. Software 41(10):1105–1129Google Scholar
- 5.Hutchins M, Foster H, Goradia T, Ostrand T (1994) Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In: ICSE ’94: proceedings of the 16th international conference on software engineering. IEEE Computer Society Press, Los Alamitos, pp 191–200Google Scholar
- 6.Jiang B, Zhang Z, Tse TH, Chen TY (2009) How well do test case prioritization techniques support statistical fault localization. In: COMPSAC ’09: proceedings of the 2009 33rd annual IEEE international computer software and applications conference. IEEE Computer Society, Washington, DC, pp 99–106Google Scholar
- 7.Johnson RA (1960) An information theory approach to diagnosis. In: Symposium on reliability and quality control, Washington, DC. DOI: http://dx.doi.org/10.1109/IRE-PGRQC.1960.5007263