Skip to main content
SpringerLink
Log in

Empirical studies in reverse engineering: state of the art and future trends

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Starting with the aim of modernizing legacy systems, often written in old programming languages, reverse engineering has extended its applicability to virtually every kind of software system. Moreover, the methods originally designed to recover a diagrammatic, high-level view of the target system have been extended to address several other problems faced by programmers when they need to understand and modify existing software. The authors’ position is that the next stage of development for this discipline will necessarily be based on empirical evaluation of methods. In fact, this evaluation is required to gain knowledge about the actual effects of applying a given approach, as well as to convince the end users of the positive cost–benefit trade offs. The contribution of this paper to the state of the art is a roadmap for the future research in the field, which includes: clarifying the scope of investigation, defining a reference taxonomy, and adopting a common framework for the execution of the experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Basili VR (1996) The role of experimentation in software engineering: past, current, and future. In: ICSE ’96: Proceedings of the 18th international conference on software engineering. IEEE Computer Society, Washington, DC, USA, 1996. ISBN 0-8186-7246-3, pp 442–449

    Chapter  Google Scholar 

  • Basili VR, Shull F, Lanubile F (1999) Building knowledge through families of experiments. In: IEEE transactions on software engineering. IEEE Computer Society, pp 456–473

  • Brooks R (1977) Towards a theory of the cognitive processes in computer programming. Int J Man-Mach Stud 9(6):737–741

    Article  Google Scholar 

  • Chikofsky E, Cross II J (1990) Reverse engineering and design recovery: a taxonomy. IEEE Softw 7(1):13–17

    Article  Google Scholar 

  • Conradi R, Wang AI (eds) (2003) Empirical methods and studies in software engineering, experiences from ESERNET. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  • Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart 13(3):319–340

    Article  Google Scholar 

  • Do H, Elbaum SG, Rothermel G (2005) Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir Softw Eng 10(4):405–435

    Article  Google Scholar 

  • Erlikh L (2000) Leveraging legacy system dollars for e-business. In: IEEE IT Pro, pp 17–23

  • IEEE (1983) Glossary of software engineering terminology. Technical Report

  • ISO (1995) ISO/IEC DIS 14102: evaluation and selection of computer-aided software engineering (CASE) tools

  • ISO (1998) ISO 9241: ergonomic requirements for office work with visual display terminals (Part 11. Guidance on Usability)

  • John B, Marks S (1997) Tracking the effectiveness of usability evaluation methods. Behav Inf Technol 16(4–5):188–202

    Article  Google Scholar 

  • Jones TC (1998) Estimating software costs. McGraw Hill

  • Kitchenham B (2004) Procedures for performing systematic reviews. Technical Report TR/SE-0401, Keele University and NICTA

  • Kitchenham B, Travassos G, von Mayrhauser A, Niessink F, Schneidewind NF, Singer J, Takada S, Vehvilainen R, Yang H (1999) Towards an ontology of software maintenance. J Softw Maint 11(6):365–389

    Article  Google Scholar 

  • Kitchenham BA, Pfleeger SL, Pickard LM, Jones PW, Hoaglin DC, Emam KE, Rosenberg J (2002) Preliminary guidelines for empirical research in software engineering. IEEE Trans Softw Eng 28(8):721–734

    Article  Google Scholar 

  • Lethbridge T, Singer J (1997) Understanding software maintenance tools: some empirical research. In: Proceedings of the IEEE workshop on empirical studies in software maintenance (WESS), pp 157–162

  • Letovsky S (1986) Cognitive processes in program comprehension. In: Soloway E, Iyengar S (eds) Proceedings of the 1st workshop on empirical studies of programmers. Ablex Publishing Company, Norwood, NJ

    Google Scholar 

  • Littman DC, Pinto J, Letovsky S, Soloway E (1987) Mental models and software maintenance. J Syst Softw 7(4):341–355

    Article  Google Scholar 

  • Miller J (2000) Applying meta-analytical procedures to software engineering experiments. J Syst Softw 54(1):29–39

    Article  Google Scholar 

  • Muller H, Jahnke J, Smith D, Storey M-A, Tilley S, Wong K (2000) Reverse engineering: a roadmap. In: Proc. of the 22nd international conference on software engineering, future of software engineering track. ACM Press, New York, pp 47–60

    Chapter  Google Scholar 

  • Nelson ML (1996) Software maintenance cost estimation and modernization support. Technical Report, Old Dominion University

  • Nielsen J (1993) Usability engineering. Academic, Boston, MA

    MATH  Google Scholar 

  • Pennington N (1987) Comprehension strategies in programming. In: Olson G, Sheppard S, Soloway E (eds) Proceedings of the 2nd workshop on empirical studies of programmers. Ablex Publishing Company, Norwood, NJ

    Google Scholar 

  • Pfleeger SL, Kitchenham B (2001–2002) Principles of survey research – parts 1–5. ACM Software Engineering Notes 26,27(6,1,2,3,5):16–18,18–20,20–24,20–23,17–20

  • Pickard LM, Kitchenham BA, Jones PW (1998) Combining empirical results in software engineerig. Inf Softw Technol 40(1):811–821

    Article  Google Scholar 

  • Sayyad Shirabad J, Menzies T (2005) The PROMISE Repository of software engineering databases. School of Information Technology and Engineering, University of Ottawa, Canada

  • Shadish WR, Cook TD, Campbell DT (2001) Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin, Boston, MA

    Google Scholar 

  • Shaw M (2002) What makes good research in software engineering? Int J Softw Tools Technol Transf 4(1):1–7. ISSN 1433-2779 (print), 1433-2787 (electronic)

    Article  Google Scholar 

  • Shull F, Basili VR, Carver J, Maldonado JC, Travassos GH, Mendonça MG, Fabbri SCPF (2002) Replicating software engineering experiments: addressing the tacit knowledge problem. In: Proc. of the international symposium on empirical software engineering (ISESE). IEEE Computer Society, Los Alamitos, CA, pp 7–16

    Chapter  Google Scholar 

  • Shull F, Mendonça MG, Basili VR, Carver J, Maldonado JC, Fabbri SCPF, Travassos GH, de Oliveira MCF (2004) Knowledge-sharing issues in experimental software engineering. Empir Softw Eng 9(1–2):111–137

    Article  Google Scholar 

  • Sim SE, Di Penta M (eds) (2006) Proceedings of the 13th working conference on reverse engineering. IEEE Computer Society, Benevento, Italy, 2006. ISBN 0-7695-2719-1

    Google Scholar 

  • Sim SE, Easterbrook SM, Holt RC (2003) Using benchmarking to advance research: a challenge to software engineering. In: Proceedings of the 25th international conference on software engineering. IEEE Computer Society, Los Alamitos, CA, pp 74–83

    Chapter  Google Scholar 

  • Sim SE, Storey M-AD (2000) A structured demonstration of program comprehension tools. In: Proceedings of the working conference on reverse engineering (WCRE), pp 184–193

  • Sim SE, Storey M-AD, Winter A (2000) A structured demonstration of five program comprehension tools: lessons learnt. In: Proceedings of the working conference on reverse engineering (WCRE), pp 210–212

  • Singer J (1999) Using the American Psychological Association (APA) style guidelines to report experimental results. In: Proceedings of the IEEE workshop on empirical studies in software maintenance (WESS), pp 71–75

  • Sjoberg D, Hannay J, Hansen O, Kampenes V, Karahasanovic A, Liborg N, Rekdal A (2005) A survey of controlled experiments in software engineering. IEEE Trans Softw Eng 31(9):733–753

    Article  Google Scholar 

  • Sneed H (1996) Encapsulating legacy software for use in client/server systems. In: Proceedings of the working conference on reverse engineering, Monterey, CA, pp 104–119

  • Soloway E, Ehrlich K (1984) Empirical studies of programming knowledge. IEEE Trans Softw Eng 10(5):595–609

    Article  Google Scholar 

  • Storey M-AD, Fracchia FD, Mueller HA (1997) Cognitive design elements to support the construction of a mental model during software visualization. In: Proceedings of the 5th international workshop on program comprehension (IWPC), p 17

  • Tichy WF, Lukowicz P, Prechelt L, Heinz EA (1995) Experimental evaluation in computer science: a quantitative study. J Syst Softw 28(1):9–18

    Article  Google Scholar 

  • Venkatesh V, Davis FD (2000) A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci 46(2):186–204

    Article  Google Scholar 

  • von Mayrhauser A, Vans A (1994) Comprehension processes during large scale maintenance. In: Proceedings of the international conference on software engineering, Sorrento, Italy. IEEE Computer Society Press, Los Alamitos, CA, pp 39–48

    Google Scholar 

  • Walenstein A (2002) Cognitive support in software engineering tools: a distributed cognition framework. Ph.D. thesis, School of Computing Science, Simon Fraser University

  • Waters RG, Chikofsky E (1994) Reverse engineering: progress along many dimensions. Commun ACM 37(5):22–25

    Article  Google Scholar 

  • Wohlin C, Runeson P, Host M, Ohlsson MC, Regnell B, Wesslen A (2000) Experimentation in software engineering: an introduction. Kluwer, Boston, MA

    MATH  Google Scholar 

  • Zayour I, Lethbridge T (2001) Adoption of reverse engineering tools: a cognitive perspective and methodology. In: Proc. of the international workshop on program comprehension (IWPC). IEEE Computer Society, Los Alamitos, CA, pp 245–258

    Google Scholar 

  • Zelkowitz M, Wallace D (1997) Experimental validation in software engineering. Inf Softw Technol 39(11):735–743

    Article  Google Scholar 

  • Zelkowitz M, Wallace D (1998) Experimental models for validating technology. IEEE Computer 31(5):23–31

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ITC-irst, Centro per la Ricerca Scientifica e Tecnologica, Povo, Trento, Italy

    Paolo Tonella

  2. Politecnico di Torino, Torino, Italy

    Marco Torchiano

  3. University of Antwerp, Antwerp, Belgium

    Bart Du Bois

  4. Tampere University of Technology, Tampere, Finland

    Tarja Systä

Authors
  1. Paolo Tonella
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Torchiano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bart Du Bois
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tarja Systä
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paolo Tonella.

Additional information

Editor: Daniel M. Berry

This work has been partially sponsored by Eureka ∑ 2023 Programme; under grants of the ITEA project if04032 entitled Software Evolution, Refactoring, Improvement of Operational & Usable Systems (SERIOUS).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tonella, P., Torchiano, M., Du Bois, B. et al. Empirical studies in reverse engineering: state of the art and future trends. Empir Software Eng 12, 551–571 (2007). https://doi.org/10.1007/s10664-007-9037-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-007-9037-5

Keywords

Search

Navigation