PROFES 2016: Product-Focused Software Process Improvement pp 683-688 | Cite as
Exploring Expectations About Risk-Based Testing: Towards Increasing Effectiveness and Efficiency
Abstract
Risk-based testing is sometimes reduced to an approach that focuses on cutting costs and time in testing. While the high effort involved in testing makes efficiency an important issue, for many companies the main concern is still to find the critical defects in their software products. Such defects can cause costly remedial upgrades and fixes, and they threaten the company’s long-term business success. In this paper we explore how the two goals “effectiveness” and “efficiency” motivate a risk-based testing approach in different organizations. Furthermore, we identify a third goal, summarized as “management support”. In a survey conducted as part of a tutorial on risk-based testing we investigated common expectations and potential benefits associated with these three goals. The results indicate that the main motivation for a risk-based approach is making testing more efficient. Nevertheless, efficiency and effectiveness are not conflicting goals and the main challenge is therefore finding strategies that increase the overall benefit of including risk information in testing.
Keywords
Test management Software risk management Software testing Risk-based testing Test process improvement Effectiveness EfficiencyNotes
Acknowledgements
This work has been supported by the COMET Competence Center program of the Austrian Research Promotion Agency (FFG), and the project QE LaB – Living Models for Open Systems funded by the Austrian Federal Ministry of Science, Research and Economy.
References
- 1.Ramler, R., Felderer, M.: A process for risk-based test strategy development and its industrial evaluation. In: Abrahamsson, P., Corral, L., Oivo, M., Russo, B. (eds.) PROFES 2015. LNCS, vol. 9459, pp. 355–371. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-26844-6_26 Google Scholar
- 2.Felderer, M., Schieferdecker, I.: A taxonomy of risk-based testing. Int. J. Softw. Tools Technol. Transf. 16(5), 559–568 (2014). SpringerCrossRefGoogle Scholar
- 3.Felderer, M., Ramler, R.: A multiple case study on risk-based testing in industry. Int. J. Softw. Tools Technol. Transf. 16(5), 609–625 (2014). SpringerCrossRefGoogle Scholar
- 4.Felderer, M., Ramler, R.: Risk orientation in software testing processes of small and medium enterprises: an exploratory and comparative study. Softw. Q. J. 24(3), 519–548 (2016). SpringerCrossRefGoogle Scholar
- 5.Felderer, M., Ramler, R.: Integrating risk-based testing in industrial test processes. Softw. Q. J. 22(3), 543–575 (2014). SpringerCrossRefGoogle Scholar
- 6.Boehm, B., Basili, V.R.: Software defect reduction top 10 list. IEEE Comput. 34(1), 135–137 (2001). IEEECrossRefGoogle Scholar
- 7.Shull, F., Basili, V.R., Boehm, B., Brown, A.W., Costa, P., Lindvall, M., Port, D., Rus, I., Tesoriero, R., Zelkowitz, M.: What we have learned about fighting defects. In: 8th Symposium on Software Metrics, METRICS 2002, p. 249. IEEE (2002)Google Scholar
- 8.Huang, L.G., Boehm, B.: How much software quality investment is enough: a value-based approach. IEEE Softw. 23(5), 88–95 (2006)CrossRefGoogle Scholar
- 9.Ramler, R., Biffl, S., Grünbacher, P.: Value-based management of software testing. In: Value-Based Software Engineering, pp. 225–244. Springer, Heidelberg (2006)Google Scholar
- 10.Ramler, R., Wolfmaier, K.: Economic perspectives in test automation: balancing automated and manual testing with opportunity cost. In: International Workshop on Automation of Software Test, AST 2006, pp. 85–91. ACM (2006)Google Scholar
- 11.Amland, S.: Risk-based testing: risk analysis fundamentals and metrics for software testing including a financial application case study. J. Syst. Softw. 53(3), 287–295 (2000)CrossRefGoogle Scholar
- 12.Redmill, F.: Theory and practice of risk-based testing. Softw. Test. Verification Reliab. 15(1), 3–20 (2005). WileyCrossRefGoogle Scholar