A Combined Testing and Verification Approach for Software Reliability

  • Natasha Sharygina
  • Doron Peled
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2021)


Automatic and manual software verification is based on applying mathematical methods to a model of the software. Modeling is usually done manually, thus it is prone to modeling errors. This means that errors found in the model may not correspond to real errors in the code, and that if the model is found to satisfy the checked properties, the actual code may still have some errors. For this reason, it is desirable to be able to perform some consistency checks between the actual code and the model. Exhaustive consistency checks are usually not possible, for the same reason that modeling is necessary. We propose a methodology for improving the throughput of software verification by performing some consistency checks between the original code and the model, specifically, by applying software testing. In this paper we present such a combined testing and verification methodology and demonstrate how it is applied using a set of software reliability tools. We introduce the notion of a neighborhood of an error trace, consisting of a tree of execution paths, where the original error trace is one of them. Our experience with the methodology shows that traversing the neighborhood of an error is extremely useful in locating its cause. This is crucial not only in understanding where the error stems from, but in getting an initial idea of how to redesign the code. We use as a case study a robot control system, and report on several design and modeling errors found during the verification and testing process.


Model Check Joint Angle Software Reliability Execution Path Actual Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cetin, M., Kapoor, C., Tesar, D.: Performance based robot redundancy resolution with multiple criteria, Proc. of ACME Design Engineering Technical Conference, Georgia (1998)Google Scholar
  2. 2.
    Clarke, E.M., Emerson, E.A.: Design and synthesis of synchronization skeletons using branching time temporal logic. Workshop on Logic of Programs, Yorktown Heights, NY. Lecture Notes in Computer Science, Vol. 131. Springer-Verlag, Berlin Heidelberg New York (1981)Google Scholar
  3. 3.
    Clarke, E.M., Grunberg, O., and Peled, D.: Model Checking, MIT Press (1999)Google Scholar
  4. 4.
    Dijkstra, E.W.: Guarded commands, nondeterminacy and formal derivation of programs, Comm. ACM, Vol. 18(8) 1975 453–457zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Emerson, E.A., Clarke, E.M.: Characterizing correctness properties of parallel programs using fixpoints, Lecture Notes in Computer Science, Vol. 85, Springer-Verlag, Berlin Heidelberg New York (1980) 169–181Google Scholar
  6. 6.
    Graig, J.J: Introduction to Robotics: Mechanics and Control. Addison-Wesley (1989)Google Scholar
  7. 7.
    Gunter, E.L., Peled, D.: Path Exploration Tool, Proceeding of TACAS 1999, Amsterdam, The Netherlands, (1999) 405–419Google Scholar
  8. 8.
    Harel, D.: From Play-In Scenarios to Code: An Achievable Dream, Proceedings of FASE 2000, Berlin, Germany, Lecture Notes in Computer Science, Vol. 1783, Springer-Verlag, Berlin Heidelberg New York (2000) 22–34Google Scholar
  9. 9.
    Hoare, C.A.R.: Communicating Sequential Processes, Comm. ACM, Vol. 21(8) (1978) 666–677zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Hoare, C.A.R.: An axiomatic basis for computer programming, Comm. ACM, Vol. 12 (1969) 576–580zbMATHCrossRefGoogle Scholar
  11. 11.
    Holzmann, G.J.: Design and Validation of Computer Protocols, Prentice Hall Software Series, (1992)Google Scholar
  12. 12.
    Jackson, D.: Aspect: Detecting Bugs with Abstract Dependencies. ACM Transactions on Software Engineering and Methodology, Vol. 4(2) (1995) 279–295CrossRefGoogle Scholar
  13. 13.
    Kapoor, C., and Tesar, D.: A Reusable Operational Software Architecture for Advanced Robotics (OSCAR), The University of Texas at Austin, Report to U.S. Dept. of Energy, Grant No. DE-FG01 94EW37966 and NASA Grant No. NAG 9-809 (1998)Google Scholar
  14. 14.
    Kernighan, B., and Ritchie, D.: The C programming Language, Prentice Hall (1988)Google Scholar
  15. 15.
    Myers, G.J.: The Art of Software Testing, Wiley (1979)Google Scholar
  16. 16.
    Murphy, G., Notkin, D., and Sullivan, K: Software Reflexion Models: Bridging the Gap between Source and High-Level Models, In Proceedings of SIGSOFT’95 Third ACM SIGSOFT Symposium on the Foundations of Software Engineering, ACM (1995) 18–28Google Scholar
  17. 17.
    Pnueli, A.: The temporal logic of programs, Proc. of the 18th IEEE Symp. on Foundation of Computer Science (1977) 46–57Google Scholar
  18. 18.
    Quielle, J.P., and Sifakis, J.: Specification and verification of concurrent systems in CESAR, Proceedings of the 5th International Symposium on Programming (1981) 337–350Google Scholar
  19. 19.
    SES inc., ObjectBench Technical Reference, SES Inc. (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Natasha Sharygina
    • 1
  • Doron Peled
    • 1
  1. 1.Bell LaboratoriesMurray HillUSA

Personalised recommendations