Abstract
Test management aims at organizing, documenting and executing test cases, and at generating execution reports. The adoption of a support tool and of a standard process for such activities is expected to improve the current practice. ITALO is a European project devoted to the evaluation of the benefits coming from test management automation. In this paper the experiences collected and the lessons learned during ITALO are summarized. A formal methodology was adopted for the selection of a support tool among those available from the market. A survey of the current practice in component testing was conducted to adapt the existing process model so as to obtain the greatest benefits from automation. An empirical study was then designed to measure the effects that are expected to be produced by the new test process complemented with the introduction of the support tool. Three pilot projects were conducted to measure the benefits obtained from tool usage and process modification. Results are presented and discussed in this paper.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Antoniol, G., La Commare, G., Giraudo, G., and Tonella, P. 1999. Effective feature analysis for tool selection. Proceedings of Profes'99, International Conference on Product Focused Software Process Improvement, Oulu, Finland, pp. 103-117.
Basili, V. R. 1993. The experience factory and its relationship to other improvement paradigms. Lecture Notes in Computer Science 717, Software Engineering ESEC'93, 4th European Software Engineering Conference, Garmish-Partenkirchen, Germany.
Basili, V., Shull, F., and Lanubile, F. 1999. Building knowledge through families of experiments. IEEE Transactions on Software Engineering 25(4): 456-473.
Bolzan, E. and Correrini, F. 1996. Verification and Validation Testing Activity Guidelines. Sodalia Internal Report.
Gaburro, P. 1996. Automated testing for the man machine interface of a training simulator. ESSI PIE 10824, SIMTEST, Final Report.
Giraudo, G., Tonella, P., and La Commare, G. 1998. Component test guidelines. ESSI PIE 27912, ITALO, Deliverable D2.
Kemerer, C. F. 1992. How the learning curve affects CASE tool adoption. IEEE Software 23-28.
Kitchenham, B., Pickard, L., and Pfleeger, S. L., 1995. Case studies for method and tool evaluation. IEEE Software 52-62.
Kitchenham, B. 1996. A method for evaluating software engineering methods and tools. Technical Report TR96-09, DESMET project, UK DTI.
La Commare, G., Giraudo, G., and Tonella, P. 2000. Test management automation: lessons learned from a process improvement experiment. Proceedings of EWSPT'2000, European Workshop on Software Process Technology, Kaprun, Austria, pp. 156-160.
Martin, M., and Gallagher, J. 1996. Mature Software Application Adaptation Project. ESSI PIE 10360, PHOENIX, Final Report.
Pfleeger, S. L. Experimental design and analysis in software engineering. SIGSOFT Notes, Parts 1 to 5, 1994 and 1995.
Tonella, P., Giraudo, G., Antoniol, G., and Mambella, E. 1998. Sodalia. Tool selection feature analysis. ESSI PIE 27912-ITALO-Deliverable D1.
Tonella, P., and Giraudo, G. 1998. Experimental design. ESSI PIE 27912-ITALO-Deliverable D3.
Wright, T. P. 1936. Factors affecting the cost of airplanes. Journal of Aeronautical Science 3(2).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Giraudo, G., Tonella, P. Designing and Conducting an Empirical Study on Test Management Automation. Empirical Software Engineering 8, 59–81 (2003). https://doi.org/10.1023/A:1021720916127
Issue Date:
DOI: https://doi.org/10.1023/A:1021720916127