Advertisement

On the Complexity of Error Explanation

  • Nirman Kumar
  • Viraj Kumar
  • Mahesh Viswanathan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3385)

Abstract

When a system fails to satisfy its specification, the model checker produces an error trace (or counter-example) that demonstrates an undesirable behavior, which is then used in debugging the system. Error explanation is the task of discovering errors in the system or the reasons why the system exhibits the error trace. While there has been considerable recent interest in automating this task and developing tools based on different heuristics, there has been very little effort in characterizing the computational complexity of the problem of error explanation.

In this paper, we study the complexity of two popular heuristics used in error explanation. The first approach tries to compute the smallest number of system changes that need to be made in order to ensure that the given counter-example is no longer exhibited, with the intuition being that these changes are the errors that need fixing. The second approach relies on the observation that differences between correct and faulty runs of a system shed considerable light on the sources of errors. In this approach, one tries to compute the correct trace of the system that is closest to the counter-example. We consider three commonly used abstractions to model programs and systems, namely, finite state Mealy machines, extended finite state machines and pushdown automata. We show that the first approach of trying to find the fewest program changes is NP-complete no matter which of the three formal models is used to represent the system. Moreover we show that no polynomial factor approximation algorithm for computing the smallest set of changes is possible, unless P = NP. For the second approach, we present a polynomial time algorithm that finds the closest correct trace, when the program is represented by a Mealy machine or a pushdown automata. When the program is represented by an extended finite state machine, the problem is once again NP-complete, and no polynomial factor approximation algorithm is likely.

Keywords

State Machine Model Check Polynomial Time Algorithm Hamiltonian Cycle Input String 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Clarke, E., Grumberg, O., Peled, D.: Model Checking. MIT Press, Cambridge (2000)Google Scholar
  2. 2.
    Zeller, A.: Isolating cause-effect chains for computer programs. In: Proceedings of the ACM Symposium on the Foundations of Software Engineering, pp. 1–10 (2002)Google Scholar
  3. 3.
    Zeller, A., Hildebrandt, R.: Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering 28, 183–200 (2002)CrossRefGoogle Scholar
  4. 4.
    Jin, H., Ravi, K., Somenzi, F.: Fate and free will in error traces. In: TACAS 2001. LNCS, vol. 2031, pp. 445–459. Springer, Heidelberg (2001)Google Scholar
  5. 5.
    Renieris, M., Reiss, S.: Fault localization with nearest neighbor queries. In: Proceedings of the Conference on Automated Software Engineering (2003) Google Scholar
  6. 6.
    Ball, T., Naik, M., Rajamani, S.: From symptom to cause: Localizing errors in counterexample traces. In: Proceedings of the ACM Symposium on the Principles of Programming Languages, pp. 97–105 (2003)Google Scholar
  7. 7.
    Groce, A., Visser, W.: What went wrong: Explaining counterexamples. In: Proceedings of the SPIN Workshop on Model Checking of Software, pp. 121–135 (2003)Google Scholar
  8. 8.
    Groce, A.: Error explanation with distance metrics. In: Proceedings of Conference on Tools and Algorithms for Construction and Analysis of Systems, pp. 108–122 (2004)Google Scholar
  9. 9.
    Ball, T., Rajamani, S.K.: The SLAM project: Debugging system software via static analysis. In: Proceedings of the ACM Symposium on the Principles of Programming Languages, pp. 1–3 (2002)Google Scholar
  10. 10.
    Brat, G., Havelund, K., Park, S., Visser, W.: Java PathFinder – A second generation of a Java model checker. In: Proceedings of the Workshop on Advances in Verification (2000)Google Scholar
  11. 11.
    Lewis, D.: Causation. Journal of Philosophy 70, 556–567 (1973)CrossRefGoogle Scholar
  12. 12.
    Zeller, A.: Yesterday, my program worked. Today, is does not. Why? In: Proceedings of the ACM Symposium on the Foundations of Software Engineering, pp. 253–267 (1999)Google Scholar
  13. 13.
    Tip, F., Dinesh, T.B.: A slicing-based approach for locating type errors. ACM Transactions on Software Engineering and Methodology 10, 5–55 (2001)CrossRefGoogle Scholar
  14. 14.
    Bhargavan, K., Gunter, C.A., Kim, M., Lee, I., Obradovic, D., Sokolsky, O., Viswanathan, M.: Verisim: Formal analysis of network simulations. IEEE: Transactions on Software Engineering 28, 129–145 (2002)CrossRefGoogle Scholar
  15. 15.
    Hopcroft, J.E., Ullman, J.D.: Introduction to Automata Theory, Languages and Computation. Addison-Wesley, Reading (1979)MATHGoogle Scholar
  16. 16.
    Alur, R., Madhusudan, P.: Visibly pushdown languages. In: Proceedings of the ACM Symposium on the Theory of Computation (2004)Google Scholar
  17. 17.
    Reps, T., Horwitz, S., Sagiv, M.: Precise interprocedural dataflow analysis via graph reachability. In: Proceedings of the ACM Symposium on the Principles of Programming Languages, pp. 49–61 (1995)Google Scholar
  18. 18.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms. McGraw-Hill Higher Education, New York (2001)MATHGoogle Scholar
  19. 19.
    Pitt, L., Warmuth, M.K.: The minimum consistent DFA problem cannot be approximated within any polynomial. Journal of the ACM 40, 95–142 (1993)MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Lawler, E.L.: A procedure for computing the K best solutions to discrete optimization problems and its application to the shortest path problem. Management Science 18, 401–405 (1972)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Nirman Kumar
    • 1
  • Viraj Kumar
    • 2
  • Mahesh Viswanathan
    • 2
  1. 1.Oracle CorporationRedwood ShoresUSA
  2. 2.University of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations