Skip to main content
Log in

Injecting software faults in Python applications

The OpenStack case study

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software fault injection techniques have been largely used as means for evaluating the dependability of systems in presence of certain types of faults. Despite the large diversity of tools offering the possibility of emulating the presence of software faults, there is little practical support for emulating the presence of software faults in Python applications, which are increasingly being used to support business critical cloud services. In this paper, we present FIT4Python, a tool for injecting software faults in Python code and then use it, in a mutation testing campaign, to analyse the effectiveness of OpenStack’s test suite against new probable software faults. We begin by analysing the types of faults affecting Nova Compute, the core component of OpenStack. We use our tool to emulate the presence of new faults in Nova Compute API to understand how well OpenStack’s battery of unit, functional, and integration tests cover these new, but probable, situations. Results show clear limitations in the effectiveness of OpenStack developers’ test suites, with many cases of injected faults passing undetected through all three types of tests and that nearly half of the analysed problems could be detected with trivial changes or additions to the unit tests.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Listing 1
Listing 2
Listing 3
Fig. 3

Similar content being viewed by others

References

Download references

Acknowledgments

This work has been supported by national funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the project CISUC - UID/CEC/00326/2020 and by European Social Fund, through the Regional Operational Program Centro 2020; by project VALU3S - Verification and Validation of Automated Systems’ Safety and Security, H2020-EU.2.1.1.7. - ECSEL, Grant agreement ID: 876852; and by project AIDA: Adaptive, Intelligent and Distributed Assurance Platform project, co-financed by the European Regional Development Fund (ERDF) through the Operational Program for Competitiveness and Internationalization – COMPETE 2020 (POCI-01-0247-FEDER-045907) and by the Portuguese Foundation for Science and Technology under CMU Portugal Program. We also would like to acknowledge the National Distributed Computing Infrastructure (INCD) for providing access to computing resources.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nuno Laranjeiro.

Additional information

Communicated by: Paolo Tonella

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marques, H., Laranjeiro, N. & Bernardino, J. Injecting software faults in Python applications. Empir Software Eng 27, 20 (2022). https://doi.org/10.1007/s10664-021-10047-9

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-021-10047-9

Keywords

Navigation