Taming False Alarms from a Domain-Unaware C Analyzer by a Bayesian Statistical Post Analysis

  • Yungbum Jung
  • Jaehwang Kim
  • Jaeho Shin
  • Kwangkeun Yi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3672)

Abstract

We present our experience of combining, in a realistic setting, a static analyzer with a statistical analysis. This combination is in order to reduce the inevitable false alarms from a domain-unaware static analyzer. Our analyzer named Airac(Array Index Range Analyzer for C) collects all the true buffer-overrun points in ANSI C programs. The soundness is maintained, and the analysis’ cost-accuracy improvement is achieved by techniques that static analysis community has long accumulated. For still inevitable false alarms (e.g. Airac raised 970 buffer-overrun alarms in commercial C programs of 5.3 million lines and 737 among the 970 alarms were false), which are always apt for particular C programs, we use a statistical post analysis. The statistical analysis, given the analysis results (alarms), sifts out probable false alarms and prioritizes true alarms. It estimates the probability of each alarm being true. The probabilities are used in two ways: 1) only the alarms that have true-alarm probabilities higher than a threshold are reported to the user; 2) the alarms are sorted by the probability before reporting, so that the user can check highly probable errors first. In our experiments with Linux kernel sources, if we set the risk of missing true error is about 3 times greater than false alarming, 74.83% of false alarms could be filtered; only 15.17% of false alarms were mixed up until the user observes 50% of the true alarms.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    CERT/CC advisories, http://www.cert.org/advisories
  3. 3.
    Berger, J.O.: Statistical Decision Theory and Bayesian Analysis, 2nd edn. Springer, Heidelberg (1985)MATHGoogle Scholar
  4. 4.
    Blanchet, B., Cousot, P., Cousot, R., Feret, J., Mauborgne, L., Mine, A., Monnizux, D., Rival, X.: A static analyzer for large safety-critical software. In: Proceedings of the SIGPLAN Conference on Programming Language Design and Implementation, June 2003, pp. 196–207 (2003)Google Scholar
  5. 5.
    Cousot, P., Cousot, R.: Comparing the galois connection and widening/narrowing approaches to abstract interpretation. In: Bruynooghe, M., Wirsing, M. (eds.) PLILP 1992. LNCS, vol. 631, pp. 269–295. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  6. 6.
    Cousot, P., Cousot, R., Feret, J., Mauborgne, L., Miné, A., Monniaux, D., Rival, X.: The astrée analyzer. In: Sagiv, M. (ed.) ESOP 2005. LNCS, vol. 3444, pp. 21–30. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Dor, N., Rodeh, M., Sagiv, M.: Cssv: towards a realistic tool for statically detecting all buffer overflows in c. In: PLDI 2003: Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, pp. 155–167. ACM Press, New York (2003)CrossRefGoogle Scholar
  8. 8.
    Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B.: Bayesian Data Analysis, 2nd edn. Text in Statistical Science. Chapman & Hall/CRC (2004)Google Scholar
  9. 9.
    Hovemeyer, D., Pugh, W.: Finding bugs is easy. SIGPLAN Not. 39(12), 92–106 (2004)CrossRefGoogle Scholar
  10. 10.
    Kremenek, T., Engler, D.: Z-ranking: Using statistical analysis to counter the impact of static analysis approximations. In: Cousot, R. (ed.) SAS 2003. LNCS, vol. 2694, pp. 295–315. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  11. 11.
    Mauborgne, L., Rival, X.: Trace partitioning in abstract interpretation based static analyzers. In: Sagiv, M. (ed.) ESOP 2005. LNCS, vol. 3444, pp. 5–20. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Metropolis, N., Ulam, S.: The Monte Carlo method. Journal of the American Statistical Association 44(247), 335–341 (1949)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Xie, Y., Chou, A., Engler, D.: Archer: using symbolic, path-sensitive analysis to detect memory access errors. In: ESEC/FSE-11: Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, pp. 327–336. ACM Press, New York (2003)CrossRefGoogle Scholar
  14. 14.
    Zitser, M., Lippmann, R., Leek, T.: Testing static analysis tools using exploitable buffer overflows from open source code. In: SIGSOFT 2004/FSE-12: Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering, pp. 97–106. ACM Press, New York (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Yungbum Jung
    • 1
  • Jaehwang Kim
    • 1
  • Jaeho Shin
    • 1
  • Kwangkeun Yi
    • 1
  1. 1.Programming Research Laboratory, School of Computer Science and EngineeringSeoul National University 

Personalised recommendations