Skip to main content
SpringerLink
Log in

Experiments with Test Case Generation and Runtime Analysis

  • Conference paper
  • First Online:
Abstract State Machines 2003 (ASM 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2589))

Included in the following conference series:

Abstract

Software testing is typically an ad hoc process where human testers manually write many test inputs and expected test results, perhaps automating their execution in a regression suite. This process is cumbersome and costly. This paper reports preliminary results on an approach to further automate this process. The approach consists of combining automated test case generation based on systematically exploring the program’s input domain, with runtime analysis, where execution traces are monitored and verified against temporal logic specifications, or analyzed using advanced algorithms for detecting concurrency errors such as data races and deadlocks. The approach suggests to generate specifications dynamically per input instance rather than statically once-and-for-all. The paper describes experiments with variants of this approach in the context of two examples, a planetary rover controller and a space craft fault protection system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. S. Bensalem and K. Havelund. Reducing False Positives in Runtime Analysis of Deadlocks. Submitted for publication, October 2002. 91, 93, 95

    Google Scholar 

  2. C. Boyapati, S. Khurshid, and D. Marinov. Korat: Automated Testing Based on Java Predicates. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), July 2002. 90

    Google Scholar 

  3. G. Brat, D. Giannakopoulou, A. Goldberg, K. Havelund, M. Lowry, C. Pasareanu, A. Venet, and W. Visser. A Comparative Field Study of Advanced Verification Technologies. Internal report, in preparation for submission, November 2002. 87, 95

    Google Scholar 

  4. M. Clavel, F. J. Durán, S. Eker, P. Lincoln, N. Martí-Oliet, J. Meseguer, and J. F. Quesada. Maude: Specification and Programming in Rewriting Logic, March 1999. Maude System documentation at maude.csl.sri.com/papers. 94

    Google Scholar 

  5. S. Cohen. Jtrek. Compaq, http://www.compaq.com/java/download/jtrek. 92

  6. D. Drusinsky. The Temporal Rover and the ATG Rover. In SPIN Model Checking and Software Verification, volume 1885 of LNCS, pages 323–330. Springer, 2000. 89, 91, 93, 94

    Chapter  Google Scholar 

  7. D. Drusinsky. Monitoring Temporal Rules with Temporal Data. Submitted for publication, October 2002. 94

    Google Scholar 

  8. M. Feather, S. Fickas, and N. Razermera-Mamy. Model-Checking for Validation of a Fault Protection System. In Proceedings of Sixth IEEE International Symposium on High Assurance System Engineering. IEEE Computer Society, October 2001. 103

    Google Scholar 

  9. E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns-Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995. 92

    Google Scholar 

  10. W. Grieskamp, Y. Gurevich, W. Schulte, and M. Veanes. Generating Finite State Machines from Abstract State Machines. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), July 2002. 89

    Google Scholar 

  11. A. Groce and W. Visser. Model Checking Java Programs using Structural Heuristics. In Proceedings of the 2002 International Symposium on Software Testing and Analysis (ISSTA). ACM Press, July 2002. 90

    Google Scholar 

  12. A. Hartman. Model Based Test Generation Tools. http://www.agedis.de/documents/ModelBasedTestGenerationToolsc s.pdf+.89

  13. K. Havelund, S. Johnson, and G. Roşu. Specification and Error Pattern Based Program Monitoring. In Proceedings of the European Space Agency workshop on On-Board Autonomy, Noordwijk, The Netherlands, October 2001. 95

    Google Scholar 

  14. K. Havelund and G. Roşu. Monitoring Java Programs with Java PathExplorer. In K. Havelund and G. Roşu, editors, Proceedings of the First International Workshop on Runtime Verification (RV’01), volume 55 of Electronic Notes in Theoretical Computer Science, pages 97–114, Paris, France, July 2001. Elsevier Science. 91, 93

    Google Scholar 

  15. K. Havelund and G. Roşu. Monitoring Programs using Rewriting. In Proceedings of the International Conference on Automated Software Engineering (ASE’01), pages 135–143. IEEE CS Press, 2001. Coronado Island, California. 95

    Google Scholar 

  16. K. Havelund and G. Roşu. A Rewriting-based Approach to Trace Analysis. Submitted for journal publication, September 2002. 95

    Google Scholar 

  17. K. Havelund and G. Roşu. Synthesizing Monitors for Safety Properties. In Tools and Algorithms for Construction and Analysis of Systems (TACAS’02), volume 2280 of Lecture Notes in Computer Science, pages 342–356. Springer, 2002. EASST best paper award at ETAPS’02. 95

    Chapter  Google Scholar 

  18. H. Hong, I. Lee, O. Sokolsky, and H. Ural. A Temporal Logic Based Theory of Test Coverage and Generation. In Proceedings of the 8th International Conference on Tools and Algorithms for Construction and Analysis of Systems (TACAS), April 2002. 90

    Google Scholar 

  19. J. C. King. Symbolic Execution and Program Testing. Communications of the ACM, 19(7):385–394, 1976. 89

    Article  MATH  Google Scholar 

  20. B. Korel. Automated Software Test Data Generation. IEEE Transaction on Software Engineering, 16(8):870–879, August 1990. 89

    Article  Google Scholar 

  21. Parasoft. http://www.parasoft.com. 89

  22. A. Pnueli. The Temporal Logic of Programs. In Proceedings of the 18th IEEE Symposium on Foundations of Computer Science, pages 46–77, 1977. 93

    Google Scholar 

  23. S. Savage, M. Burrows, G. Nelson, P. Sobalvarro, and T. Anderson. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. ACM Transactions on Computer Systems, 15(4):391–411, November 1997. 95

    Article  Google Scholar 

  24. N. Tracey, J. Clark, and K. Mander. The Way Forward for Unifying Dynamic Test-Case Generation: The Optimisation-Based Approach. In International Workshop on Dependable Computing and Its Applications (DCIA), pages 169–180. IFIP, January 1998. 89

    Google Scholar 

  25. T-VEC. http://www.t-vec.com. 89

  26. W. Visser, K. Havelund, G. Brat, and S. Park. Model Checking Programs. In Proceedings of ASE’2000: The 15th IEEE International Conference on Automated Software Engineering. IEEE CS Press, September 2000. 90

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Systems Institute, ETH Zurich, CH-8092, Zurich, Switzerland

    Cyrille Artho

  2. Naval Postgraduate School, Monterey, California, USA

    Doron Drusinksy

  3. Kestrel Technology, NASA Ames Research Center, Moffett Field, California, USA

    Allen Goldberg, Klaus Havelund & Corina Pasareanu

  4. NASA Ames Research Center, Moffett Field, California, USA

    Mike Lowry

  5. Department of Computer Science, University of Illinois at Urbana-Champaign, Illinios

    Grigore Roşu

  6. RIACS, NASA Ames Research Center, Moffett Field, California, USA

    Willem Visser

Authors
  1. Cyrille Artho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Doron Drusinksy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Allen Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Klaus Havelund
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mike Lowry
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Corina Pasareanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Grigore Roşu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Willem Visser
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Informatica, Università di Pisa, Via Buonarotti 2, 56127, Pisa, Italy

    Egon Börger

  2. CEA, Università di Catania, Piazza Università 2, 95124, Catania, Italy

    Angelo Gargantini

  3. Dipartimento di Matematica e Informatica, Università di Catania, Città Universitaria, viale A. Doria 6, 95125, Catania, Italy

    Elvinia Riccobene

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Artho, C. et al. (2003). Experiments with Test Case Generation and Runtime Analysis. In: Börger, E., Gargantini, A., Riccobene, E. (eds) Abstract State Machines 2003. ASM 2003. Lecture Notes in Computer Science, vol 2589. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36498-6_5

Download citation

  • DOI: https://doi.org/10.1007/3-540-36498-6_5

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00624-4

  • Online ISBN: 978-3-540-36498-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation