Skip to main content
SpringerLink
Log in

Risk orientation in software testing processes of small and medium enterprises: an exploratory and comparative study

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Risk orientation in testing is an important means to balance quality, time-to-market, and cost of software. Especially for small and medium enterprises (SME) under high competitive and economic pressure, risk orientation can help to focus testing activities on critical areas of a software product. Although several risk-based approaches to testing are available, the topic has so far not been investigated in the context of SME, where risks are often associated with business critical issues. This article fills the gap and explores the state of risk orientation in the testing processes of SME. Furthermore, it compares the state of risk-based testing in SME to the situation in large enterprises. The article is based on a multiple case study conducted with five SME. A previous study on risk-based testing in large enterprises is used as reference for investigating the differences between risk orientation in SME and large enterprises. The findings of our study show that a strong business focus, the use of informal risk concepts, as well as the application of risk knowledge to reduce testing cost and time are key differences of risk-based testing in SME compared to large enterprises.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Ahern, D., Clouse, A., & Turner, R. (2008). CMMI distilled: A practical introduction to integrated process improvement. Boston: Addison-Wesley Professional.

    Google Scholar 

  • Amland, S. (2000). Risk-based testing: Risk analysis fundamentals and metrics for software testing including a financial application case study. Journal of Systems and Software, 53(3), 287–295.

    Article  Google Scholar 

  • Azar, J., Smith, R. K., & Cordes, D. (2007). Value-oriented requirements prioritization in a small development organization. IEEE Software, 24(1), 32–37.

    Article  Google Scholar 

  • Bath, G., & van Veenendaal, E. (2014). Improving the test process. Rocky Nook: Massachusetts.

    Google Scholar 

  • Dorling, A. (1993). SPICE: Software process improvement and capability determination. Software Quality Journal, 2(4), 209–224.

    Article  Google Scholar 

  • Erdogan, G., Li, Y., Runde, R. K., Seehusen, F., & Stølen, K. (2014). Approaches for the combined use of risk analysis and testing: A systematic literature review. International Journal on Software Tools for Technology Transfer, 16(5), 627–642.

    Article  Google Scholar 

  • Felderer, M., Haisjackl, C., Breu, R., & Motz, J. (2012). Integrating manual and automatic risk assessment for risk-based testing. Software quality. In Process automation in software development. 4th international conference SWQD (pp. 159–180).

  • Felderer, M., & Ramler, R. (2013). Experiences and challenges of introducing risk-based testing in an industrial project. In Software quality. Increasing value in software and systems development. 5th international conference SWQD (pp. 10–29).

  • Felderer, M., & Ramler, R. (2014a). Integrating risk-based testing in industrial test processes. Software Quality Journal, 22(3), 543–575.

    Article  Google Scholar 

  • Felderer, M., & Ramler, R. (2014b). A multiple case study on risk-based testing in industry. International Journal on Software Tools for Technology Transfer, 16(5), 609–625.

    Article  Google Scholar 

  • Felderer, M., & Schieferdecker, I. (2014). A taxonomy of risk-based testing. International Journal on Software Tools for Technology Transfer, 16(5), 559–568.

    Article  Google Scholar 

  • Felderer, M., Wendland, M-F., & Schieferdecker, I. (2014). Risk-based testing. In Leveraging applications of formal methods, verification and validation. Specialized techniques and applications (pp. 274–276). Berlin, Heidelberg: Springer.

  • García, I., Pacheco, C., Mendoza, E., Calvo Manzano, J. A., Cuevas, G., & San Feliu, T. (2012). Managing the software process with a software process improvement tool in a small enterprise. Journal of Software: Evolution and Process, 24(5), 481–491.

    Google Scholar 

  • Gerrard, P., & Thompson, N. (2002). Risk based e-business testing. Norwood: Artech House Inc.

    Google Scholar 

  • Gleirscher, M., Golubitskiy, D., Irlbeck, M., & Wagner, S. (2014). Introduction of static quality analysis in small-and medium-sized software enterprises: Experiences from technology transfer. Software Quality Journal, 22(3), 499–542.

    Article  Google Scholar 

  • Harrold, M. J. (2000). Testing: A roadmap. In Proceedings of the conference on the future of software engineering (pp. 61–72).

  • ISO/IEC. (2011). Software engineering—Lifecycle profiles for Very Small Entities (VSEs). Available online at http://www.iso.org/iso/catalogue_detail?csnumber=51150. Accessed on July 15, 2015.

  • ISO/IEC/IEEE. (2013). ISO/IEC/IEEE 29119 Software testing. Draft available online at http://www.softwaretestingstandard.org/. Accessed on July 15, 2015.

  • ISTQB. (2012). Standard glossary of terms used in software testing. Version 2.2. Brussels: International Software Testing Qualifications Board.

    Google Scholar 

  • Karlström, D., Runeson, P., & Norden, S. (2005). A minimal test practice framework for emerging software organizations. Software Testing, Verification and Reliability, 15(3), 145–166.

    Article  Google Scholar 

  • Karolak, D. W. (1995). Software engineering risk management. Hoboken: Wiley-IEEE Computer Society Press.

    Google Scholar 

  • Kautz, K. (1999). Making sense of measurement for small organizations. IEEE Software, 16, 14–20.

    Article  Google Scholar 

  • Koomen, T., & Pol, M. (1999). Test process improvement: A practical step-by-step guide to structured testing. Boston: Addison-Wesley Professional.

    MATH  Google Scholar 

  • Koomen, T., van der Aalst, L., Broekman, B., & Vroon, M. (2006). TMap next, for result-driven testing. ‘s-Hertogenbosch: UTN Publishers.

    Google Scholar 

  • Martin, K., & Hoffman, B. (2007). An open source approach to developing software in a small organization. IEEE Software, 24(1), 46–53.

    Article  Google Scholar 

  • Martin, D., Rooksby, J., Rouncefield, M., & Sommerville, I. (2007). ‘Good’ organisational reasons for ‘Bad’ software testing: An ethnographic study of testing in a small software company. In 29th international conference on software engineering (ICSE ‘07).

  • Mc Caffery, F., Taylor, P. S., & Coleman, G. (2007). Adept: A unified assessment method for small software companies. IEEE Software, 24(1), 24–31.

    Article  Google Scholar 

  • Mishra, D., & Mishra, A. (2008). Software process improvement methodologies for small and medium enterprises. In 9th international conference on product-focused software process improvement (PROFES 2008).

  • Pino, F. J., García, F., & Piattini, M. (2008). Software process improvement in small and medium software enterprises: A systematic review. Software Quality Journal, 16(2), 237–261.

    Article  Google Scholar 

  • Pino, F. J., Pardo, C., García, F., & Piattini, M. (2010). Assessment methodology for software process improvement in small organizations. Information and Software Technology, 52(10), 1044–1061.

    Article  Google Scholar 

  • Redmill, F. (2004). Exploring risk-based testing and its implications. Software Testing Verification and Reliability, 14(1), 3–15.

    Article  Google Scholar 

  • Redmill, F. (2005). Theory and practice of risk-based testing: Research Articles. Software Testing Verification and Reliability, 15(1), 3–20.

    Article  Google Scholar 

  • Richardson, I., & von Wangenheim, C. G. (2007). Why are small software organizations different. IEEE Software, 24(1), 18–22.

    Article  Google Scholar 

  • Runeson, P., Höst, M., Rainer, A., & Regnell, B. (2012). Case study research in software engineering: Guidelines and examples. Hoboken: Wiley.

    Book  Google Scholar 

  • Sanchez-Gordon, M. L., O’Connor, R. V., & Colomo-Palacios, R. (2015). Evaluating VSEs viewpoint and sentiment towards the ISO/IEC 29110 standard: A two country grounded theory study. In 15th international conference on software process improvement and capability determination (SPICE 2015).

  • Souza, E., Gusmão, C., & Venâncio, J. (2010). Risk-based testing: A case study. In seventh international conference on information technology: New generations (ITNG) (pp. 1032–1037).

  • Souza, E., Gusmão, C., Venâncio, J., & Melo, R. (2009). Measurement and control for risk-based test cases and activities. In 10th Latin American Test Workshop (LATW’09) (pp. 1–6).

  • Steiner, M., Blaschke, M., Philipp, M., & Schweigert, T. (2012). Make test process assessment similar to software process assessment—The test SPICE approach. Journal of Software: Evolution and Process, 24(5), 471–480.

    Google Scholar 

  • van Veenendaal, E., Goslin, A., Olsen, K., O’Hara, F., Miller, M., Thompson, G., & Wells, B. (2008). Test Maturity Model integration (TMMi) Version 1.0.. TMMi Foundation: Princeton.

    Google Scholar 

  • Yin, R. K. (2014). Case study research: Design and methods. Thousand Oaks: Sage Publications.

    Google Scholar 

  • Yoon, H., & Choi, B. (2011). A test case prioritization based on degree of risk exposure and its empirical evaluation. International Journal of Software Engineering and Knowledge Engineering, 21(02), 191–209.

    Article  Google Scholar 

Download references

Acknowledgments

This work has been supported by the research project Smart Testing funded by the Austrian Research Promotion Agency (FFG), the COMET Competence Center program of the Austrian Research Promotion Agency (FFG), and the project QE LaB—Living Models for Open Systems (www.qe-lab.at) funded by the Austrian Federal Ministry of Economics (Bundesministerium für Wirtschaft und Arbeit).

Author information

Authors and Affiliations

  1. Institute of Computer Science, University of Innsbruck, Innsbruck, Austria

    Michael Felderer

  2. Software Competence Center Hagenberg, Hagenberg im Mühlkreis, Austria

    Rudolf Ramler

Authors
  1. Michael Felderer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rudolf Ramler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Felderer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Felderer, M., Ramler, R. Risk orientation in software testing processes of small and medium enterprises: an exploratory and comparative study. Software Qual J 24, 519–548 (2016). https://doi.org/10.1007/s11219-015-9289-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-015-9289-z

Keywords

Search

Navigation