A Business View on Testing ERP Systems with Value-Based Requirements Coverage

  • Rudolf Ramler
  • Theodorich Kopetzky
  • Wolfgang Platz
Conference paper
Part of the Lecture Notes in Information Systems and Organisation book series (LNISO, volume 4)


Testing has been identified as a critical factor for a successful implementation of ERP systems. However, most testing activities are still value-neutral and do not utilize the information about the system’s achievable business value, which is a particularly promising improvement for testing of business software and ERP systems. In this paper we therefore present an approach for value-based coverage measurement that can be used to align the testing effort to the value associated with requirements and typical usage scenarios. It has been implemented as part of the commercial test tool TOSCA Testsuite by Tricentis and was successfully applied in real-world projects. The results demonstrated its ability to adequately capture the distribution of the business value involved in different functional units. Furthermore, when compared with a value-neutral and a pure requirements-based approach for test case prioritization, it produced a higher benefit curve and an early positive ROI from testing.


System testing Requirements-based testing Value-based testing Test case prioritization 


  1. 1.
    Al-Mashari, M., Al-Mudimigh, A., Zairi, M.: Enterprise resource planning: a taxonomy of critical factors. Eur. J. Oper. Res. 146(2), 352–364 (2003)CrossRefGoogle Scholar
  2. 2.
    Bach, J.: Risk and requirements-based testing. IEEE Comput. 32(6), 113–114 (1999)Google Scholar
  3. 3.
    Boehm, B.: Value-based software engineering: overview and agenda. In: Biffl, S., et al. (eds.) Value-Based Software Engineering, pp. 3–14. Springer, Berlin (2006)Google Scholar
  4. 4.
    Boehm, B., Basili, V.R.: Software defect reduction top 10 list. Computer 34(1), 135–137 (2001)Google Scholar
  5. 5.
    Brooks, P.A., Memon, A.M.: Automated GUI testing guided by usage profiles. In: Proceedings of the Twenty-Second IEEE/ACM International Conference on Automated Software Engineering. ASE’07, pp. 333–342. ACM, New York (2007)Google Scholar
  6. 6.
    Chen, C.C., Law, C., Yang, S.C.: Managing ERP implementation failure: a project management perspective. IEEE Trans. Eng. Manage. 56(1), 157–170 (2009)CrossRefGoogle Scholar
  7. 7.
    Daneva, M.: ERP requirements engineering practice: lessons learned. Softw. IEEE 21(2), 26–33 (2004)CrossRefGoogle Scholar
  8. 8.
    Farago, D.: Nondeterministic coverage metrics as key performance indicator for model- and value-based testing. 31. Treffen der GI-Fachgruppe Test, Analyse and Verifikation von Software (TAV) (2011)Google Scholar
  9. 9.
    Felderer, M., et al.: Integrating manual and automatic risk assessment for risk-based testing. In: Software Quality. Process Automation in Software Development. Software Quality Days 2012, pp. 159–180. Vienna (2012)Google Scholar
  10. 10.
    Gerrard, P.: Test methods and tools for ERP implementations. In: IEEE, pp. 40–46 (2007)Google Scholar
  11. 11.
    Gittens, M., Romanufa, K., Godwin, D., Racicot, J.: All code coverage is not created equal: a case study in prioritized code coverage. In: Proceedings of the 2006 Conference of the Center for Advanced Studies on Collaborative Research (CASCON ‘06) (2006)Google Scholar
  12. 12.
    Holland, C.R., Light, B.: A critical success factors model for ERP implementation. Softw. IEEE 16(3), 30–36 (1999)CrossRefGoogle Scholar
  13. 13.
    Huang, L., Boehm, B.: How much software quality investment is enough: a value-based approach. IEEE Softw. 23(5), 88–95 (2006)CrossRefGoogle Scholar
  14. 14.
    Krishnamoorthi, R., Mary, S.A.: Factor oriented requirement coverage based system test case prioritization of new and regression test cases. Inf. Softw. Technol. 51(4), 799–808 (2009)Google Scholar
  15. 15.
    Kumar, M., Suresh, A.V., Prashanth, P.: Analyzing the quality issues in ERP implementation: a case study. In: Proceedings of the Second International Conference on Emerging Trends in Engineering and Technology, ICETET 2009, pp. 759–764 (2009)Google Scholar
  16. 16.
    Lehtola, L., Kauppinen, M., Kujala, S.: Linking the business view to requirements engineering: long-term product planning by roadmapping. In Proceedings of the 13th IEEE International Conference on Requirements Engineering. RE’05, pp. 439–446. IEEE Computer Society, Washington, DC (2005)Google Scholar
  17. 17.
    Li, Q., Li, M., Yang, Y., Wang, Q., Tan, T., Boehm, B., Hu, C.: Bridge the gap between software test process and business value: a case study. Proceedings of the International Conference on Software Process: Trustworthy Software Development Processes (ICSP ‘09) (2009)Google Scholar
  18. 18.
    Li, Q., Yang, Y., Li, M., Wang, Q., Boehm, B., Hu, C.: Improving software testing process: feature prioritization to make winners of success-critical stakeholders. J. Softw. Maint. Evol. Res. Pract. (2010). doi:  10.1002/smr.512
  19. 19.
    Pandian, C.P.: Applied Software Risk Management: A Guide for Software Project Managers. Auerbach Publications, Boston (2006)Google Scholar
  20. 20.
    Petschenik, N. H.: Practical priorities in system testing. IEEE Softw. 2(5), 18–23 1985Google Scholar
  21. 21.
    Ramler, R., Biffl, S. Grünbacher, P.: Value-based management of software testing. In: Biffl, S., et al. (eds.) Value-Based Software Engineering, pp. 225–244. Springer, Berlin (2006)Google Scholar
  22. 22.
    Ramler, R., Kopetzky, T., Platz, W.: Combinatorial test design in the TOSCA testsuite: lessons learned and practical implications. In: Proceedings of the 2012 Workshop on Combinatorial Testing, ICST 2012 (2012)Google Scholar
  23. 23.
    Srikanth, H., Williams, L.: On the economics of requirements-based test case prioritization. In: Proceedings of the Seventh International Workshop on Economics-Driven Software Engineering Research (EDSER ‘05) (2005)Google Scholar
  24. 24.
    Srivastva, P.R., Kumar, K., Raghurama, G.: Test case prioritization based on requirements and risk factors. SIGSOFT Softw. Eng. Notes 33(4), Article 7 (2008)Google Scholar
  25. 25.
    Urem, F., Mikulic, Z.: Developing operational profile for ERP software module reliability prediction. In: 2010 Proceedings of the 33rd International Convention MIPRO, pp. 409–413 (2010)Google Scholar
  26. 26.
    Zhang, D.: Machine learning in value-based software test data generation. In: Proceedings of the 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI ‘06) (2006)Google Scholar
  27. 27.
    Zhang, D.A.: Value-based framework for software evolutionary testing. Int J Softw Sci Comput Intell 3(2), 62–82 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Rudolf Ramler
    • 1
  • Theodorich Kopetzky
    • 1
  • Wolfgang Platz
    • 2
  1. 1.Software Competence Center HagenbergHagenberg im MühlkreisAustria
  2. 2.TRICENTIS Technology and Consulting GmbHViennaAustria

Personalised recommendations