Advertisement

A Generic Fault Model for Quality Assurance

  • Alexander Pretschner
  • Dominik Holling
  • Robert Eschbach
  • Matthias Gemmar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8107)

Abstract

Because they are comparatively easy to implement, structural coverage criteria are commonly used for test derivation in model- and code-based testing. However, there is a lack of compelling evidence that they are useful for finding faults, specifically so when compared to random testing. This paper challenges the idea of using coverage criteria for test selection and instead proposes an approach based on fault models. We define a general fault model as a transformation from correct to incorrect programs and/or a partition of the input data space. Thereby, we leverage the idea of fault injection for test assessment to test derivation.

We instantiate the developed general fault model to describe existing fault models. We also show by example how to derive test cases.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arcuri, A., Briand, L.: Formal analysis of the probability of interaction fault detection using random testing. IEEE Transactions on Software Engineering 38(5), 1088–1099 (2012)CrossRefGoogle Scholar
  2. 2.
    Arcuri, A., Iqbal, M.Z., Briand, L.: Random testing: Theoretical results and practical implications. IEEE Transactions on Software Engineering 38(2), 258–277 (2012)CrossRefGoogle Scholar
  3. 3.
    Binder, R.V.: Testing Object-Oriented Systems: Models, Patterns, and Tools. Object Technology Series. Addison-Wesley (1999)Google Scholar
  4. 4.
    Bochmann, G.v., Das, A., Dssouli, R., Dubuc, M., Ghedamsi, A., Luo, G.: Fault models in testing. In: Proceedings of the IFIP TC6/WG6.1 Fourth International Workshop on Protocol Test Systems IV, pp. 17–30. North-Holland Publishing Co., Amsterdam (1992), http://dl.acm.org/citation.cfm?id=648126.747577 Google Scholar
  5. 5.
    Büchler, M., Oudinet, J., Pretschner, A.: Semi-automatic security testing of web applications from a secure model. In: 2012 IEEE Sixth International Conference on Software Security and Reliability (SERE), pp. 253–262 (2012)Google Scholar
  6. 6.
    Ceccato, M., Tonella, P., Ricca, F.: Is aop code easier or harder to test than oop code? In: On-line Proceedings of the First Workshop on Testing Aspect-Oriented Programs (WTAOP 2005) (March 2005)Google Scholar
  7. 7.
    Chillarege, R., Bhandari, I.S., Chaar, J.K., Halliday, M.J., Moebus, D.S., Ray, B.K., Wong, M.Y.: Orthogonal Defect Classification-A Concept for In-Process Measurements. IEEE Trans. Softw. Eng. 18(11), 943–956 (1992), http://dx.doi.org/10.1109/32.177364 CrossRefGoogle Scholar
  8. 8.
    Ciupa, I., Pretschner, A., Oriol, M., Leitner, A., Meyer, B.: On the number and nature of faults found by random testing. Softw. Test. Verif. Reliab. 21(1), 3–28 (2011)CrossRefGoogle Scholar
  9. 9.
    Dupuy, A., Leveson, N.: An empirical evaluation of the mc/dc coverage criterion on the hete-2 satellite software. In: Proceedings of the 19th Digital Avionics Systems Conference, DASC 2000, vol. 1, pp. 1B6/1–1B6/7 (2000)Google Scholar
  10. 10.
    Fagan, M.E.: Design and code inspections to reduce errors in program development. IBM Syst. J. 15(3), 182–211 (1976), http://dx.doi.org/10.1147/sj.153.0182 CrossRefGoogle Scholar
  11. 11.
    Gutjahr, W.J.: Partition testing vs. random testing: The influence of uncertainty. IEEE Trans. Softw. Eng. 25(5), 661–674 (1999), http://dx.doi.org/10.1109/32.815325 CrossRefGoogle Scholar
  12. 12.
    Harris, I.G.: Fault models and test generation for hardware-software covalidation. IEEE Des. Test 20(04), 40–47 (2003), http://dx.doi.org/10.1109/MDT.2003.1214351, doi:10.1109/MDT.2003.1214351CrossRefGoogle Scholar
  13. 13.
    Heimdahl, M., Whalen, M., Rajan, A., Staats, M.: On mc/dc and implementation structure: An empirical study. In: IEEE/AIAA 27th Digital Avionics Systems Conference, DASC 2008, pp. 5.B.3–1–5.B.3–13 (2008)Google Scholar
  14. 14.
    Kuhn, D.R., Wallace, D.R., Gallo Jr., A.M.: Software fault interactions and implications for software testing. IEEE Trans. Softw. Eng. 30(6), 418–421 (2004)CrossRefGoogle Scholar
  15. 15.
    Lu, S., Park, S., Seo, E., Zhou, Y.: Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In: Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIII, pp. 329–339. ACM, New York (2008), http://dx.doi.org/10.1145/1346281.1346323 CrossRefGoogle Scholar
  16. 16.
    Ma, Y.-S., Kwon, Y.-R., Offutt, J.: Inter-class mutation operators for java. In: ISSRE, pp. 352–366 (2002)Google Scholar
  17. 17.
    Malaiya, Y., Li, M., Bieman, J., Karcich, R.: Software reliability growth with test coverage. IEEE Transactions on Reliability 51(4), 420–426 (2002)CrossRefGoogle Scholar
  18. 18.
    Martin, E., Xie, T.: A fault model and mutation testing of access control policies. In: Proceedings of the 16th International Conference on World Wide Web, WWW 2007, pp. 667–676. ACM, New York (2007), http://dx.doi.org/10.1145/1242572.1242663 CrossRefGoogle Scholar
  19. 19.
    McCluskey, E., Clegg, F.W.: Fault equivalence in combinational logic networks. IEEE Transactions on Computers C-20(11), 1286–1293 (1971)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Mockus, A., Nagappan, N., Dinh-Trong, T.T.: Test coverage and post-verification defects: A multiple case study. In: Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement, ESEM 2009, pp. 291–301 (2009)Google Scholar
  21. 21.
    Nagaraja, K., Li, X., Bianchini, R., Martin, R.P., Nguyen, T.D.: Using fault injection and modeling to evaluate the performability of cluster-based services. In: Proceedings of the 4th Conference on USENIX Symposium on Internet Technologies and Systems, USITS 2003, vol. 4, p. 2. USENIX Association, Berkeley (2003), http://dl.acm.org/citation.cfm?id=1251460.1251462 Google Scholar
  22. 22.
    Offutt, J., Alexander, R., Wu, Y., Xiao, Q., Hutchinson, C.: A fault model for subtype inheritance and polymorphism. In: Proceedings of the 12th International Symposium on Software Reliability Engineering, ISSRE 2001, pp. 84–95. IEEE Computer Society, Washington, DC (2001), http://dl.acm.org/citation.cfm?id=851028.856258 CrossRefGoogle Scholar
  23. 23.
    Ostrand, T.J., Balcer, M.J.: The category-partition method for specifying and generating fuctional tests. Commun. ACM 31(6), 676–686 (1988), http://dx.doi.org/10.1145/62959.62964 CrossRefGoogle Scholar
  24. 24.
    Pretschner, A., Prenninger, W., Wagner, S., Kühnel, C., Baumgartner, M., Sostawa, B., Zölch, R., Stauner, T.: One evaluation of model-based testing and its automation. In: Proceedings of the 27th International Conference on Software Engineering, pp. 392–401 (2005)Google Scholar
  25. 25.
    Weyuker, E., Jeng, B.: Analyzing partition testing strategies. IEEE Transaction on Software Engineering 17(7), 703–711 (1991)CrossRefGoogle Scholar
  26. 26.
    Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Comput. Surv. 29(4), 366–427 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Alexander Pretschner
    • 1
  • Dominik Holling
    • 1
  • Robert Eschbach
    • 2
  • Matthias Gemmar
    • 2
  1. 1.Technische Universität MünchenGermany
  2. 2.itk EngineeringGermany

Personalised recommendations