Advertisement

Automated Software Engineering

, Volume 21, Issue 1, pp 107–143 | Cite as

Behind the scenes in SANTE: a combination of static and dynamic analyses

  • Omar Chebaro
  • Pascal Cuoq
  • Nikolai KosmatovEmail author
  • Bruno Marre
  • Anne Pacalet
  • Nicky Williams
  • Boris Yakobowski
Article

Abstract

While the development of one software verification tool is often seen as a difficult task, the realization of a tool combining various verification techniques is even more complex. This paper presents an innovative tool for verification of C programs called Sante (Static ANalysis and TEsting). We show how several tools based on heterogeneous techniques such as abstract interpretation, dependency analysis, program slicing, constraint solving and test generation can be combined within one tool. We describe the integration of these tools and discuss particular aspects of each underlying tool that are beneficial for the whole combination.

Keywords

C program verification SANTE tool Frama-C toolset Static analysis Program slicing Test generation Constraint solving 

Notes

Acknowledgements

The authors thank Patrick Baudin, Bernard Botella, Loïc Correnson, Benjamin Monate, Virgile Prevosto and Julien Signoles for their support and advice, as well as the editors and anonymous referees for profound analysis of the paper and lots of valuable comments. Special thanks to Alain Giorgetti and Jacques Julliand for their contribution on the theoretical aspects of the Sante method.

References

  1. Ball, T.: A theory of predicate-complete test coverage and generation. In: The Third International Symposium on Formal Methods for Components and Objects (FMCO 2004). LNCS, vol. 3657, pp. 1–22. Springer, Berlin (2004) CrossRefGoogle Scholar
  2. Bardin, S., Herrmann, P.: Structural testing of executables. In: The First International Conference on Software Testing, Verification, and Validation (ICST 2008), pp. 22–31. IEEE Computer Society, Los Alamitos (2008) CrossRefGoogle Scholar
  3. Bardin, S., Herrmann, P., Perroud, F.: An alternative to SAT-based approaches for bit-vectors. In: The 16th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2010). LNCS, vol. 6015, pp. 84–98. Springer, Berlin (2010) CrossRefGoogle Scholar
  4. Baudin, P., Filliâtre, J.C., Hubert, T., Marché, C., Monate, B., Moy, Y., Prevosto, V.: In: ACSL: ANSI/ISO C Specification Language, v1.6 (2012). URL:http://frama-c.com/acsl.html Google Scholar
  5. Beckman, N.E., Nori, A.V., Rajamani, S.K., Simmons, R.J.: Proofs from tests. In: The ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008), pp. 3–14. ACM, New York (2008) CrossRefGoogle Scholar
  6. Berthomé, P., Heydemann, K., Kauffmann-Tourkestansky, X., Lalande, J.F.: Attack model for verification of interval security properties for smart card c codes. In: The 5th ACM SIGPLAN Workshop on Programming Languages and Analysis for Security (PLAS 2010). ACM, New York (2010) Google Scholar
  7. Beyer, D., Henzinger, T.A., Jhala, R., Majumdar, R.: The software model checker Blast: applications to software engineering. Int. J. Softw. Tools Technol. Transf. 9(5–6), 505–525 (2007) CrossRefGoogle Scholar
  8. Bonichon, R., Cuoq, P.: A mergeable interval map. Studia Inform. Universalis 9(1), 5–37 (2011) Google Scholar
  9. Cadar, C., Ganesh, V., Pawlowski, P.M., Dill, D.L., Engler, D.R.: EXE: automatically generating inputs of death. In: The 13th ACM Conference on Computer and Communications Security (CCS 2006), pp. 322–335. ACM, New York (2006) Google Scholar
  10. Cadar, C., Dunbar, D., Engler, D.R.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: The 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2008), pp. 209–224. USENIX Association, Berkeley (2008) Google Scholar
  11. Canet, G., Cuoq, P., Monate, B.: A value analysis for C programs. In: The Ninth IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2009), pp. 123–124. IEEE Computer Society, Los Alamitos (2009) CrossRefGoogle Scholar
  12. Chebaro, O., Kosmatov, N., Giorgetti, A., Julliand, J.: Combining static analysis and test generation for C program debugging. In: The 4th International Conference on Tests and Proofs (TAP 2010). LNCS, pp. 652–666. Springer, Berlin (2010) Google Scholar
  13. Chebaro, O., Kosmatov, N., Giorgetti, A., Julliand, J.: The SANTE tool: value analysis, program slicing and test generation for C program debugging. In: The 5th International Conference on Tests and Proofs (TAP 2011). LNCS, pp. 78–83. Springer, Berlin (2011) Google Scholar
  14. Chebaro, O., Kosmatov, N., Giorgetti, A., Julliand, J.: Program slicing enhances a verification technique combining static and dynamic analysis. In: The ACM Symposium on Applied Computing (SAC 2012), pp. 1284–1291. ACM, New York (2012) Google Scholar
  15. Choi, J.D., Ferrante, J.: Static slicing in the presence of goto statements. ACM Trans. Program. Lang. Syst. 16(4), 1097–1113 (1994) CrossRefGoogle Scholar
  16. Cok, D.R., Kiniry, J.R.: ESC/Java2: uniting ESC/Java and JML. In: The International Workshop on Construction and Analysis of Safe, Secure and Interoperable Smart Devices (CASSIS 2004). LNCS, vol. 3362, pp. 108–128. Springer, Berlin (2004) CrossRefGoogle Scholar
  17. Correnson, L., Signoles, J.: Combining analyses for C program verification. In: The 17th International Workshop on Formal Methods for Industrial Critical Systems (FMICS 2012). LNCS, vol. 7437, pp. 108–130. Springer, Berlin (2012) CrossRefGoogle Scholar
  18. Correnson, L., Cuoq, P., Kirchner, F., Prevosto, V., Puccetti, A., Signoles, J., Yakobowski, B.: Frama-C User Manual (2012). URL:http://frama-c.com
  19. Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: The 4th Symposium on Principles of Programming Languages (POPL 1977), pp. 238–252 (1977) Google Scholar
  20. Csallner, C., Smaragdakis, Y.: JCrasher: an automatic robustness tester for Java. Softw. Pract. Exp. 34(11), 1025–1050 (2004) CrossRefGoogle Scholar
  21. Csallner, C., Smaragdakis, Y.: Dynamically discovering likely interface invariants. In: The 28th ACM/IEEE International Conference on Software Engineering (ICSE 2006), Emerging Results Track, pp. 861–864. ACM, New York (2006) Google Scholar
  22. Cuoq, P., Doligez, D.: Hashconsing in an incrementally garbage-collected system: a story of weak pointers and hashconsing in OCaml 3.10.2. In: The ACM Workshop on ML, pp. 13–22. ACM, New York (2008) Google Scholar
  23. Cuoq, P., Signoles, J., Baudin, P., Bonichon, R., Canet, G., Correnson, L., Monate, B., Prevosto, V., Puccetti, A.: Experience report: OCaml for an industrial-strength static analysis framework. In: The 14th ACM SIGPLAN International Conference on Functional Programming (ICFP 2009), pp. 281–286. ACM, New York (2009) CrossRefGoogle Scholar
  24. Cuoq, P., Monate, B., Pacalet, A., Prevosto, V.: Functional dependencies of C functions via weakest pre-conditions. Int. J. Softw. Tools Technol. Transf. 13(5), 405–417 (2011) CrossRefGoogle Scholar
  25. Cuoq, P., Delmas, D., Duprat, S., Moya Lamiel, V.: Fan-C, a Frama-C plug-in for data flow verification. In: The Embedded Real-Time Software and Systems Congress (ERTS2 2012) (2012a) Google Scholar
  26. Cuoq, P., Hilsenkopf, P., Kirchner, F., Labbé, S., Thuy, N., Yakobowski, B.: Formal verification of software important to safety using the Frama-C tool suite. In: The 8th International Conference on Nuclear Plant Instrumentation and Control (NPIC 2012) (2012b) Google Scholar
  27. Cuoq, P., Kirchner, F., Kosmatov, N., Prevosto, V., Signoles, J., Yakobowski, B.: Frama-C, a program analysis perspective. In: The 10th International Conference on Software Engineering and Formal Methods (SEFM 2012). LNCS, vol. 7504, pp. 233–247. Springer, Berlin (2012c) Google Scholar
  28. Cuoq, P., Monate, B., Pacalet, A., Prevosto, V., Regehr, J., Yakobowski, B., Yang, X.: Testing static analyzers with randomly generated programs. In: The 4th International NASA Formal Methods Symposium (NFM 2012). LNCS, vol. 7226, pp. 120–125. Springer, Berlin (2012d) CrossRefGoogle Scholar
  29. Dragoi, C., Sighireanu, M.: CELIA User manual (2011). http://www.liafa.jussieu.fr/celia/
  30. Ernst, M.D., Perkins, J.H., Guo, P.J., McCamant, S., Pacheco, C., Tschantz, M.S., Xiao, C.: The Daikon system for dynamic detection of likely invariants. Sci. Comput. Program. 69(1–3), 35–45 (2007) CrossRefzbMATHMathSciNetGoogle Scholar
  31. Ferrante, J., Ottenstein, K.J., Warren, J.D.: The program dependence graph and its use in optimization. ACM Trans. Program. Lang. Syst. 9(3), 319–349 (1987) CrossRefzbMATHGoogle Scholar
  32. Ge, X., Taneja, K., Xie, T., Tillmann, N.: DyTa: dynamic symbolic execution guided with static verification results. In: The 33rd International Conference on Software Engineering (ICSE 2011), pp. 992–994. ACM, New York (2011) Google Scholar
  33. Godefroid, P., Levin, M.Y., Molnar, D.A.: Automated whitebox fuzz testing. In: The Network and Distributed System Security Symposium (NDSS 2008). The Internet Society, Washington (2008) Google Scholar
  34. Gotlieb, A.: Euclide: a constraint-based testing platform for critical C programs. In: The Second International Conference on Software Testing Verification and Validation (ICST 2009), pp. 151–160. IEEE Computer Society, Los Alamitos (2009) CrossRefGoogle Scholar
  35. Gotlieb, A., Botella, B., Watel, M.: INKA: ten years after the first ideas. In: The International Conference on Software and Systems Engineering and Their Applications (ICSSEA 2006) (2006) Google Scholar
  36. Gotlieb, A., Leconte, M., Marre, B.: Constraint solving on modular integers. In: The CP 2010 Workshop on Constraint Modelling and Reformulation (ModRef 2010) (2010) Google Scholar
  37. Granger, P.: Static analysis of linear congruence equalities among variables of a program. In: The International Joint Conference on Theory and Practice of Software Development (TAPSOFT 1991), vol. 1: Colloquium on Trees in Algebra and Programming (CAAP 1991). LNCS, pp. 169–192. Springer, Berlin (1991) Google Scholar
  38. Grieskamp, W., Tillmann, N., Schulte, W.: Xrt-exploring runtime for .net architecture and applications. Electron. Notes Theor. Comput. Sci. 144(3), 3–26 (2006) CrossRefGoogle Scholar
  39. Gulavani, B.S., Henzinger, T.A., Kannan, Y., Nori, A.V., Rajamani, S.K.: SYNERGY: a new algorithm for property checking. In: The 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2006), pp. 117–127. ACM, New York (2006) CrossRefGoogle Scholar
  40. Horwitz, S., Reps, T., Binkley, D.: Interprocedural slicing using dependence graphs. In: The ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 1988), vol. 23, pp. 35–46. ACM, New York (1988) CrossRefGoogle Scholar
  41. IEEE Std 754-2008: IEEE standard for floating-point arithmetic. Tech. rep. (2008) http://dx.doi.org/10.1109/IEEESTD.2008.4610935
  42. International Organization for Standardization: ISO/IEC 9899:TC3: Programming Languages—C (2007). http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf
  43. Iosif, R., Garnier, F. (eds.):. Flata-C (2013). http://www-verimag.imag.fr/FLATA-C.html
  44. Kosmatov, N.: All-paths test generation for programs with internal aliases. In: The 19th International Symposium on Software Reliability Engineering (ISSRE 2008), pp. 147–156. IEEE Computer Society, Los Alamitos (2008) CrossRefGoogle Scholar
  45. Kosmatov, N.: On complexity of all-paths test generation. From practice to theory. In: Proceedings of the Testing: Academic and Industrial Conference—Practice and Research Techniques (TAIC PART 2009), pp. 144–153. IEEE Computer Society Press, Los Alamitos (2009) CrossRefGoogle Scholar
  46. Kosmatov, N.: Online version of PathCrawler (2010–2012). http://pathcrawler-online.com/
  47. Kosmatov, N., Legeard, B., Peureux, F., Utting, M.: Boundary coverage criteria for test generation from formal models. In: The 15th International Symposium on Software Reliability Engineering (ISSRE 2004), pp. 139–150. IEEE Computer Society, Los Alamitos (2004) CrossRefGoogle Scholar
  48. Ku, K., Hart, T.E., Chechik, M., Lie, D.: A buffer overflow benchmark for software model checkers. In: The 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007), pp. 389–392. ACM, New York (2007) Google Scholar
  49. Leconte, M., Berstel, B.: Extending a cp solver with congruences as domains for software verification. In: The CP 2006 Workshop on Constraints in Software Testing, Verification and Analysis, CSTVA 2006 (2006) Google Scholar
  50. Lee, C., Potkonjak, M., Mangione-Smith, W.H.: MediaBench: a tool for evaluating and synthesizing multimedia and communications systems. In: The 30th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 1997), pp. 330–335. IEEE Computer Society, Los Alamitos (1997) Google Scholar
  51. Mark Harman, D.B., Danicic, S.: Amorphous program slicing. J. Syst. Softw. 68(1), 45–64 (2003) CrossRefGoogle Scholar
  52. Marre, B., Arnould, A.: Test sequences generation from Lustre descriptions: GATeL. In: The 15th IEEE International Conference on Automated Software Engineering (ASE 2000), pp. 229–237. IEEE Computer Society, Los Alamitos (2000) Google Scholar
  53. Marre, B., Blanc, B.: Test selection strategies for Lustre descriptions in GATeL. Electron. Notes Theor. Comput. Sci. 111, 93–111 (2005) CrossRefGoogle Scholar
  54. Marre, B., Michel, C.: Improving the floating point addition and subtraction constraints. In: The 16th International Conference on Principles and Practice of Constraint Programming (CP 2010). LNCS, vol. 6308, pp. 360–367. Springer, Berlin (2010) CrossRefGoogle Scholar
  55. Michel, C.: Exact projection functions for floating point number constraints. In: The 7th International Symposium on Artificial Intelligence and Mathematics (AIMA 2002) (2002) Google Scholar
  56. Mouy, P., Marre, B., Willams, N., Le Gall, P.: Generation of all-paths unit test with function calls. In: The First International Conference on Software Testing, Verification, and Validation (ICST 2008), pp. 32–41. IEEE Computer Society, Los Alamitos (2008) CrossRefGoogle Scholar
  57. Necula, G.C., Mcpeak, S., Rahul, S.P., Weimer, W.: Cil: intermediate language and tools for analysis and transformation of C programs. In: The International Conference on Compiler Construction (CC 2002). LNCS, vol. 2304, pp. 213–228. Springer, Berlin (2002) CrossRefGoogle Scholar
  58. Ottenstein, K.J., Ottenstein, L.M.: The program dependence graph in a software development environment. In: The First ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments (SDE 1984), pp. 177–184. ACM, New York (1984) CrossRefGoogle Scholar
  59. Pariente, D., Ledinot, E.: Formal verification of industrial C code using Frama-C: a case study. In: The International Conference on Formal Verification of Object-Oriented Software (FoVeOOS 2010), pp. 205–218 (2010) Google Scholar
  60. Pasareanu, C., Pelanek, R., Visser, W.: Concrete model checking with abstract matching and refinement. In: The 17th International Conference on Computer Aided Verification (CAV 2005). LNCS, vol. 3576, pp. 52–66. Springer, Berlin (2005) CrossRefGoogle Scholar
  61. Reps, T., Turnidge, T.: Program specialization via program slicing. In: The Dagstuhl Seminar on Partial Evaluation. LNCS, vol. 1110, pp. 409–429. Springer, Berlin (1996) CrossRefGoogle Scholar
  62. Schimpf, J., Shen, K.: ECLiPSe—from LP to CLP. Theory Pract. Log. Program. 12(1–2), 127–156 (2011) MathSciNetGoogle Scholar
  63. Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: The 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE 2005), pp. 263–272. ACM, New York (2005) Google Scholar
  64. Signoles, J.: Foncteurs impératifs et composés: la notion de projets dans Frama-C. Studia Inform. Universalis 7(2), 20–51 (2009) Google Scholar
  65. Signoles, J.: Une bibliothèque de typage dynamique en OCaml. In: Journées Francophones des Langages Applicatifs (JFLA 2011), pp. 209–242. Hermann, Studia Informatica Universalis, Paris (2011) Google Scholar
  66. Smaragdakis, Y., Csallner, C.: Combining static and dynamic reasoning for bug detection. In: The First International Conference on Tests and Proofs (TAP 2007). LNCS, vol. 4454, pp. 1–16. Springer, Berlin (2007) Google Scholar
  67. Tillmann, N., de Halleux, J.: White box test generation for .NET. In: The Second International Conference on Tests and Proofs (TAP 2008). LNCS, vol. 4966, pp. 133–153. Springer, Berlin (2008) Google Scholar
  68. Tip, F.: A survey of program slicing techniques. J. Prog. Lang. 3(3) (1995) Google Scholar
  69. Weiser, M.: Program slicing. In: The 5th International Conference on Software Engineering (ICSE 1981), pp. 439–449. IEEE Computer Society, Los Alamitos (1981) Google Scholar
  70. Weiser, M.: Programmers use slices when debugging. Commun. ACM 25(7), 446–452 (1982) CrossRefGoogle Scholar
  71. Williams, N.: WCET measurement using modified path testing. In: The 5th International Workshop on Worst-Case Execution Time Analysis (WCET 2005) (2005) Google Scholar
  72. Williams, N., Marre, B., Mouy, P.: On-the-fly generation of k-paths tests for C functions: towards the automation of grey-box testing. In: The 19th IEEE International Conference on Automated Software Engineering (ASE 2004), pp. 290–293. IEEE Computer Society, Los Alamitos (2004) CrossRefGoogle Scholar
  73. Williams, N., Marre, B., Mouy, P., Roger, M.: PathCrawler: automatic generation of path tests by combining static and dynamic analysis. In: The 5th European Dependable Computing Conference on Dependable Computing (EDCC 2005). LNCS, vol. 3463, pp. 281–292. Springer, Berlin (2005) Google Scholar
  74. Williams, N., Roger, M.: Test generation strategies to measure worst-case execution time. In: The 4th International Workshop on Automation of Software Test (AST 2009), pp. 88–96. IEEE Computer Society, Los Alamitos (2009) CrossRefGoogle Scholar
  75. Yorsh, G., Ball, T., Sagiv, M.: Testing, abstraction, theorem proving: better together! In: The ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2006), pp. 145–156. ACM, New York (2006) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Omar Chebaro
    • 1
  • Pascal Cuoq
    • 2
  • Nikolai Kosmatov
    • 2
    Email author
  • Bruno Marre
    • 2
  • Anne Pacalet
    • 3
    • 4
  • Nicky Williams
    • 2
  • Boris Yakobowski
    • 2
  1. 1.ASCOLA (EMN-INRIA, LINA)École des Mines de NantesNantesFrance
  2. 2.CEA, LISTSoftware Reliability LaboratoryGif-sur-YvetteFrance
  3. 3.INRIA-Sophia-AntipolisSophia AntipolisFrance
  4. 4.SafeRiverParisFrance

Personalised recommendations