Skip to main content
SpringerLink
Log in

Behind the scenes in SANTE: a combination of static and dynamic analyses

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

While the development of one software verification tool is often seen as a difficult task, the realization of a tool combining various verification techniques is even more complex. This paper presents an innovative tool for verification of C programs called Sante (Static ANalysis and TEsting). We show how several tools based on heterogeneous techniques such as abstract interpretation, dependency analysis, program slicing, constraint solving and test generation can be combined within one tool. We describe the integration of these tools and discuss particular aspects of each underlying tool that are beneficial for the whole combination.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The French word santé means health, and sometimes also Cheers!

  2. The C standard itself does not specify the layout of bit-fields, but then again, it does not specify the layout of any kind of data.

  3. Type-punning is allowed when the lvalue used to access is of type char (International Organization for Standardization: ISO/IEC 9899:TC3 2007, §6.5.7) or a union type (this has been made explicit in Technical Corrigendum 3 footnote 82 (International Organization for Standardization: ISO/IEC 9899:TC3 2007, §6.5.2.3)).

  4. In particular, for program slicing, without a dependency S→I, I can be sliced out, while S is still present in the slice. It may introduce new control flow not present in the original program, and lead to incorrect slices (Choi and Ferrante 1994; Tip 1995).

  5. http://frama-c.com/support.html.

  6. http://c.happycodings.com/Mathematics/code4.html.

  7. http://freshmeat.net/projects/eurocheck.

References

  • Ball, T.: A theory of predicate-complete test coverage and generation. In: The Third International Symposium on Formal Methods for Components and Objects (FMCO 2004). LNCS, vol. 3657, pp. 1–22. Springer, Berlin (2004)

    Chapter  Google Scholar 

  • Bardin, S., Herrmann, P.: Structural testing of executables. In: The First International Conference on Software Testing, Verification, and Validation (ICST 2008), pp. 22–31. IEEE Computer Society, Los Alamitos (2008)

    Chapter  Google Scholar 

  • Bardin, S., Herrmann, P., Perroud, F.: An alternative to SAT-based approaches for bit-vectors. In: The 16th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2010). LNCS, vol. 6015, pp. 84–98. Springer, Berlin (2010)

    Chapter  Google Scholar 

  • Baudin, P., Filliâtre, J.C., Hubert, T., Marché, C., Monate, B., Moy, Y., Prevosto, V.: In: ACSL: ANSI/ISO C Specification Language, v1.6 (2012). URL:http://frama-c.com/acsl.html

    Google Scholar 

  • Beckman, N.E., Nori, A.V., Rajamani, S.K., Simmons, R.J.: Proofs from tests. In: The ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008), pp. 3–14. ACM, New York (2008)

    Chapter  Google Scholar 

  • Berthomé, P., Heydemann, K., Kauffmann-Tourkestansky, X., Lalande, J.F.: Attack model for verification of interval security properties for smart card c codes. In: The 5th ACM SIGPLAN Workshop on Programming Languages and Analysis for Security (PLAS 2010). ACM, New York (2010)

    Google Scholar 

  • Beyer, D., Henzinger, T.A., Jhala, R., Majumdar, R.: The software model checker Blast: applications to software engineering. Int. J. Softw. Tools Technol. Transf. 9(5–6), 505–525 (2007)

    Article  Google Scholar 

  • Bonichon, R., Cuoq, P.: A mergeable interval map. Studia Inform. Universalis 9(1), 5–37 (2011)

    Google Scholar 

  • Cadar, C., Ganesh, V., Pawlowski, P.M., Dill, D.L., Engler, D.R.: EXE: automatically generating inputs of death. In: The 13th ACM Conference on Computer and Communications Security (CCS 2006), pp. 322–335. ACM, New York (2006)

    Google Scholar 

  • Cadar, C., Dunbar, D., Engler, D.R.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: The 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2008), pp. 209–224. USENIX Association, Berkeley (2008)

    Google Scholar 

  • Canet, G., Cuoq, P., Monate, B.: A value analysis for C programs. In: The Ninth IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2009), pp. 123–124. IEEE Computer Society, Los Alamitos (2009)

    Chapter  Google Scholar 

  • Chebaro, O., Kosmatov, N., Giorgetti, A., Julliand, J.: Combining static analysis and test generation for C program debugging. In: The 4th International Conference on Tests and Proofs (TAP 2010). LNCS, pp. 652–666. Springer, Berlin (2010)

    Google Scholar 

  • Chebaro, O., Kosmatov, N., Giorgetti, A., Julliand, J.: The SANTE tool: value analysis, program slicing and test generation for C program debugging. In: The 5th International Conference on Tests and Proofs (TAP 2011). LNCS, pp. 78–83. Springer, Berlin (2011)

    Google Scholar 

  • Chebaro, O., Kosmatov, N., Giorgetti, A., Julliand, J.: Program slicing enhances a verification technique combining static and dynamic analysis. In: The ACM Symposium on Applied Computing (SAC 2012), pp. 1284–1291. ACM, New York (2012)

    Google Scholar 

  • Choi, J.D., Ferrante, J.: Static slicing in the presence of goto statements. ACM Trans. Program. Lang. Syst. 16(4), 1097–1113 (1994)

    Article  Google Scholar 

  • Cok, D.R., Kiniry, J.R.: ESC/Java2: uniting ESC/Java and JML. In: The International Workshop on Construction and Analysis of Safe, Secure and Interoperable Smart Devices (CASSIS 2004). LNCS, vol. 3362, pp. 108–128. Springer, Berlin (2004)

    Chapter  Google Scholar 

  • Correnson, L., Signoles, J.: Combining analyses for C program verification. In: The 17th International Workshop on Formal Methods for Industrial Critical Systems (FMICS 2012). LNCS, vol. 7437, pp. 108–130. Springer, Berlin (2012)

    Chapter  Google Scholar 

  • Correnson, L., Cuoq, P., Kirchner, F., Prevosto, V., Puccetti, A., Signoles, J., Yakobowski, B.: Frama-C User Manual (2012). URL:http://frama-c.com

  • Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: The 4th Symposium on Principles of Programming Languages (POPL 1977), pp. 238–252 (1977)

    Google Scholar 

  • Csallner, C., Smaragdakis, Y.: JCrasher: an automatic robustness tester for Java. Softw. Pract. Exp. 34(11), 1025–1050 (2004)

    Article  Google Scholar 

  • Csallner, C., Smaragdakis, Y.: Dynamically discovering likely interface invariants. In: The 28th ACM/IEEE International Conference on Software Engineering (ICSE 2006), Emerging Results Track, pp. 861–864. ACM, New York (2006)

    Google Scholar 

  • Cuoq, P., Doligez, D.: Hashconsing in an incrementally garbage-collected system: a story of weak pointers and hashconsing in OCaml 3.10.2. In: The ACM Workshop on ML, pp. 13–22. ACM, New York (2008)

    Google Scholar 

  • Cuoq, P., Signoles, J., Baudin, P., Bonichon, R., Canet, G., Correnson, L., Monate, B., Prevosto, V., Puccetti, A.: Experience report: OCaml for an industrial-strength static analysis framework. In: The 14th ACM SIGPLAN International Conference on Functional Programming (ICFP 2009), pp. 281–286. ACM, New York (2009)

    Chapter  Google Scholar 

  • Cuoq, P., Monate, B., Pacalet, A., Prevosto, V.: Functional dependencies of C functions via weakest pre-conditions. Int. J. Softw. Tools Technol. Transf. 13(5), 405–417 (2011)

    Article  Google Scholar 

  • Cuoq, P., Delmas, D., Duprat, S., Moya Lamiel, V.: Fan-C, a Frama-C plug-in for data flow verification. In: The Embedded Real-Time Software and Systems Congress (ERTS2 2012) (2012a)

    Google Scholar 

  • Cuoq, P., Hilsenkopf, P., Kirchner, F., Labbé, S., Thuy, N., Yakobowski, B.: Formal verification of software important to safety using the Frama-C tool suite. In: The 8th International Conference on Nuclear Plant Instrumentation and Control (NPIC 2012) (2012b)

    Google Scholar 

  • Cuoq, P., Kirchner, F., Kosmatov, N., Prevosto, V., Signoles, J., Yakobowski, B.: Frama-C, a program analysis perspective. In: The 10th International Conference on Software Engineering and Formal Methods (SEFM 2012). LNCS, vol. 7504, pp. 233–247. Springer, Berlin (2012c)

    Google Scholar 

  • Cuoq, P., Monate, B., Pacalet, A., Prevosto, V., Regehr, J., Yakobowski, B., Yang, X.: Testing static analyzers with randomly generated programs. In: The 4th International NASA Formal Methods Symposium (NFM 2012). LNCS, vol. 7226, pp. 120–125. Springer, Berlin (2012d)

    Chapter  Google Scholar 

  • Dragoi, C., Sighireanu, M.: CELIA User manual (2011). http://www.liafa.jussieu.fr/celia/

  • Ernst, M.D., Perkins, J.H., Guo, P.J., McCamant, S., Pacheco, C., Tschantz, M.S., Xiao, C.: The Daikon system for dynamic detection of likely invariants. Sci. Comput. Program. 69(1–3), 35–45 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  • Ferrante, J., Ottenstein, K.J., Warren, J.D.: The program dependence graph and its use in optimization. ACM Trans. Program. Lang. Syst. 9(3), 319–349 (1987)

    Article  MATH  Google Scholar 

  • Ge, X., Taneja, K., Xie, T., Tillmann, N.: DyTa: dynamic symbolic execution guided with static verification results. In: The 33rd International Conference on Software Engineering (ICSE 2011), pp. 992–994. ACM, New York (2011)

    Google Scholar 

  • Godefroid, P., Levin, M.Y., Molnar, D.A.: Automated whitebox fuzz testing. In: The Network and Distributed System Security Symposium (NDSS 2008). The Internet Society, Washington (2008)

    Google Scholar 

  • Gotlieb, A.: Euclide: a constraint-based testing platform for critical C programs. In: The Second International Conference on Software Testing Verification and Validation (ICST 2009), pp. 151–160. IEEE Computer Society, Los Alamitos (2009)

    Chapter  Google Scholar 

  • Gotlieb, A., Botella, B., Watel, M.: INKA: ten years after the first ideas. In: The International Conference on Software and Systems Engineering and Their Applications (ICSSEA 2006) (2006)

    Google Scholar 

  • Gotlieb, A., Leconte, M., Marre, B.: Constraint solving on modular integers. In: The CP 2010 Workshop on Constraint Modelling and Reformulation (ModRef 2010) (2010)

    Google Scholar 

  • Granger, P.: Static analysis of linear congruence equalities among variables of a program. In: The International Joint Conference on Theory and Practice of Software Development (TAPSOFT 1991), vol. 1: Colloquium on Trees in Algebra and Programming (CAAP 1991). LNCS, pp. 169–192. Springer, Berlin (1991)

    Google Scholar 

  • Grieskamp, W., Tillmann, N., Schulte, W.: Xrt-exploring runtime for .net architecture and applications. Electron. Notes Theor. Comput. Sci. 144(3), 3–26 (2006)

    Article  Google Scholar 

  • Gulavani, B.S., Henzinger, T.A., Kannan, Y., Nori, A.V., Rajamani, S.K.: SYNERGY: a new algorithm for property checking. In: The 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2006), pp. 117–127. ACM, New York (2006)

    Chapter  Google Scholar 

  • Horwitz, S., Reps, T., Binkley, D.: Interprocedural slicing using dependence graphs. In: The ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 1988), vol. 23, pp. 35–46. ACM, New York (1988)

    Chapter  Google Scholar 

  • IEEE Std 754-2008: IEEE standard for floating-point arithmetic. Tech. rep. (2008) http://dx.doi.org/10.1109/IEEESTD.2008.4610935

  • International Organization for Standardization: ISO/IEC 9899:TC3: Programming Languages—C (2007). http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf

  • Iosif, R., Garnier, F. (eds.):. Flata-C (2013). http://www-verimag.imag.fr/FLATA-C.html

  • Kosmatov, N.: All-paths test generation for programs with internal aliases. In: The 19th International Symposium on Software Reliability Engineering (ISSRE 2008), pp. 147–156. IEEE Computer Society, Los Alamitos (2008)

    Chapter  Google Scholar 

  • Kosmatov, N.: On complexity of all-paths test generation. From practice to theory. In: Proceedings of the Testing: Academic and Industrial Conference—Practice and Research Techniques (TAIC PART 2009), pp. 144–153. IEEE Computer Society Press, Los Alamitos (2009)

    Chapter  Google Scholar 

  • Kosmatov, N.: Online version of PathCrawler (2010–2012). http://pathcrawler-online.com/

  • Kosmatov, N., Legeard, B., Peureux, F., Utting, M.: Boundary coverage criteria for test generation from formal models. In: The 15th International Symposium on Software Reliability Engineering (ISSRE 2004), pp. 139–150. IEEE Computer Society, Los Alamitos (2004)

    Chapter  Google Scholar 

  • Ku, K., Hart, T.E., Chechik, M., Lie, D.: A buffer overflow benchmark for software model checkers. In: The 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007), pp. 389–392. ACM, New York (2007)

    Google Scholar 

  • Leconte, M., Berstel, B.: Extending a cp solver with congruences as domains for software verification. In: The CP 2006 Workshop on Constraints in Software Testing, Verification and Analysis, CSTVA 2006 (2006)

    Google Scholar 

  • Lee, C., Potkonjak, M., Mangione-Smith, W.H.: MediaBench: a tool for evaluating and synthesizing multimedia and communications systems. In: The 30th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 1997), pp. 330–335. IEEE Computer Society, Los Alamitos (1997)

    Google Scholar 

  • Mark Harman, D.B., Danicic, S.: Amorphous program slicing. J. Syst. Softw. 68(1), 45–64 (2003)

    Article  Google Scholar 

  • Marre, B., Arnould, A.: Test sequences generation from Lustre descriptions: GATeL. In: The 15th IEEE International Conference on Automated Software Engineering (ASE 2000), pp. 229–237. IEEE Computer Society, Los Alamitos (2000)

    Google Scholar 

  • Marre, B., Blanc, B.: Test selection strategies for Lustre descriptions in GATeL. Electron. Notes Theor. Comput. Sci. 111, 93–111 (2005)

    Article  Google Scholar 

  • Marre, B., Michel, C.: Improving the floating point addition and subtraction constraints. In: The 16th International Conference on Principles and Practice of Constraint Programming (CP 2010). LNCS, vol. 6308, pp. 360–367. Springer, Berlin (2010)

    Chapter  Google Scholar 

  • Michel, C.: Exact projection functions for floating point number constraints. In: The 7th International Symposium on Artificial Intelligence and Mathematics (AIMA 2002) (2002)

    Google Scholar 

  • Mouy, P., Marre, B., Willams, N., Le Gall, P.: Generation of all-paths unit test with function calls. In: The First International Conference on Software Testing, Verification, and Validation (ICST 2008), pp. 32–41. IEEE Computer Society, Los Alamitos (2008)

    Chapter  Google Scholar 

  • Necula, G.C., Mcpeak, S., Rahul, S.P., Weimer, W.: Cil: intermediate language and tools for analysis and transformation of C programs. In: The International Conference on Compiler Construction (CC 2002). LNCS, vol. 2304, pp. 213–228. Springer, Berlin (2002)

    Chapter  Google Scholar 

  • Ottenstein, K.J., Ottenstein, L.M.: The program dependence graph in a software development environment. In: The First ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments (SDE 1984), pp. 177–184. ACM, New York (1984)

    Chapter  Google Scholar 

  • Pariente, D., Ledinot, E.: Formal verification of industrial C code using Frama-C: a case study. In: The International Conference on Formal Verification of Object-Oriented Software (FoVeOOS 2010), pp. 205–218 (2010)

    Google Scholar 

  • Pasareanu, C., Pelanek, R., Visser, W.: Concrete model checking with abstract matching and refinement. In: The 17th International Conference on Computer Aided Verification (CAV 2005). LNCS, vol. 3576, pp. 52–66. Springer, Berlin (2005)

    Chapter  Google Scholar 

  • Reps, T., Turnidge, T.: Program specialization via program slicing. In: The Dagstuhl Seminar on Partial Evaluation. LNCS, vol. 1110, pp. 409–429. Springer, Berlin (1996)

    Chapter  Google Scholar 

  • Schimpf, J., Shen, K.: ECLiPSe—from LP to CLP. Theory Pract. Log. Program. 12(1–2), 127–156 (2011)

    MathSciNet  Google Scholar 

  • Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: The 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE 2005), pp. 263–272. ACM, New York (2005)

    Google Scholar 

  • Signoles, J.: Foncteurs impératifs et composés: la notion de projets dans Frama-C. Studia Inform. Universalis 7(2), 20–51 (2009)

    Google Scholar 

  • Signoles, J.: Une bibliothèque de typage dynamique en OCaml. In: Journées Francophones des Langages Applicatifs (JFLA 2011), pp. 209–242. Hermann, Studia Informatica Universalis, Paris (2011)

    Google Scholar 

  • Smaragdakis, Y., Csallner, C.: Combining static and dynamic reasoning for bug detection. In: The First International Conference on Tests and Proofs (TAP 2007). LNCS, vol. 4454, pp. 1–16. Springer, Berlin (2007)

    Google Scholar 

  • Tillmann, N., de Halleux, J.: White box test generation for .NET. In: The Second International Conference on Tests and Proofs (TAP 2008). LNCS, vol. 4966, pp. 133–153. Springer, Berlin (2008)

    Google Scholar 

  • Tip, F.: A survey of program slicing techniques. J. Prog. Lang. 3(3) (1995)

  • Weiser, M.: Program slicing. In: The 5th International Conference on Software Engineering (ICSE 1981), pp. 439–449. IEEE Computer Society, Los Alamitos (1981)

    Google Scholar 

  • Weiser, M.: Programmers use slices when debugging. Commun. ACM 25(7), 446–452 (1982)

    Article  Google Scholar 

  • Williams, N.: WCET measurement using modified path testing. In: The 5th International Workshop on Worst-Case Execution Time Analysis (WCET 2005) (2005)

    Google Scholar 

  • Williams, N., Marre, B., Mouy, P.: On-the-fly generation of k-paths tests for C functions: towards the automation of grey-box testing. In: The 19th IEEE International Conference on Automated Software Engineering (ASE 2004), pp. 290–293. IEEE Computer Society, Los Alamitos (2004)

    Chapter  Google Scholar 

  • Williams, N., Marre, B., Mouy, P., Roger, M.: PathCrawler: automatic generation of path tests by combining static and dynamic analysis. In: The 5th European Dependable Computing Conference on Dependable Computing (EDCC 2005). LNCS, vol. 3463, pp. 281–292. Springer, Berlin (2005)

    Google Scholar 

  • Williams, N., Roger, M.: Test generation strategies to measure worst-case execution time. In: The 4th International Workshop on Automation of Software Test (AST 2009), pp. 88–96. IEEE Computer Society, Los Alamitos (2009)

    Chapter  Google Scholar 

  • Yorsh, G., Ball, T., Sagiv, M.: Testing, abstraction, theorem proving: better together! In: The ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2006), pp. 145–156. ACM, New York (2006)

    Chapter  Google Scholar 

Download references

Acknowledgements

The authors thank Patrick Baudin, Bernard Botella, Loïc Correnson, Benjamin Monate, Virgile Prevosto and Julien Signoles for their support and advice, as well as the editors and anonymous referees for profound analysis of the paper and lots of valuable comments. Special thanks to Alain Giorgetti and Jacques Julliand for their contribution on the theoretical aspects of the Sante method.

Author information

Author notes

  1. Anne Pacalet

    Present address: SafeRiver, 55 rue Boissonade, 75014, Paris, France

Authors and Affiliations

  1. ASCOLA (EMN-INRIA, LINA), École des Mines de Nantes, 44307, Nantes, France

    Omar Chebaro

  2. CEA, LIST, Software Reliability Laboratory, PC 174, 91191, Gif-sur-Yvette, France

    Pascal Cuoq, Nikolai Kosmatov, Bruno Marre, Nicky Williams & Boris Yakobowski

  3. INRIA-Sophia-Antipolis, BP 93, 06902, Sophia Antipolis, France

    Anne Pacalet

Authors
  1. Omar Chebaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pascal Cuoq
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nikolai Kosmatov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bruno Marre
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anne Pacalet
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nicky Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Boris Yakobowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolai Kosmatov.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chebaro, O., Cuoq, P., Kosmatov, N. et al. Behind the scenes in SANTE: a combination of static and dynamic analyses. Autom Softw Eng 21, 107–143 (2014). https://doi.org/10.1007/s10515-013-0127-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-013-0127-x

Keywords

Search

Navigation