Formal Verification of the rank Algorithm for Succinct Data Structures

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10009)


Succinct data structures are designed to use a minimal amount of computer memory in a time-efficient way. Their correct implementation is essential to big data analysis. Yet, succinct data structures are difficult to verify because they rely on bit-level manipulations better achieved with low-level languages. In this paper, we report on the formal verification of the standard Jacobson rank algorithm using the Coq proof-assistant and extract an OCaml implementation from it. This requires overcoming the mismatch between Coq being a purely functional programming language and succinct data structures being inherently imperative. To enjoy the best of both worlds, we propose to use code extraction from Coq to OCaml but with an original (tested but unverified) implementation of bitstrings. We can then use Coq to formalize correctness, including important claims about storage requirements, and still obtain efficient native code. To the best of our knowledge, this is the first application of formal verification to succinct data structures.





The authors are grateful to the anonymous reviewers for their helpful comments. This work is partially supported by a JSPS Grant-in-Aid for Scientific Research (Project Number: 15K12013).


  1. 1.
    Affeldt, R., Marti, N.: An approach to formal verification of arithmetic functions in assembly. In: Okada, M., Satoh, I. (eds.) ASIAN 2006. LNCS, vol. 4435, pp. 346–360. Springer, Heidelberg (2008)Google Scholar
  2. 2.
    Agarwal, R., Khandelwal, A., Stoica, I.: Succinct: enabling queries on compressed data. In: NSDI 2015, pp. 337–350. USENIX Association (2015). Technical report:
  3. 3.
    Armand, M., Grégoire, B., Spiwack, A., Théry, L.: Extending Coq with imperative features and its application to SAT verification. In: Kaufmann, M., Paulson, L.C. (eds.) ITP 2010. LNCS, vol. 6172, pp. 83–98. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Clark, D.: Compact pat trees. Doctoral dissertation, University of Waterloo (1996)Google Scholar
  5. 5.
    The Coq Development Team: Reference Manual. Version 8.5. INRIA (2004–2016).
  6. 6.
    Free Software Foundation: GCC 4.9.2 Manual (2014).
  7. 7.
    Gonthier, G., Mahboubi, A., Tassi, E.: A small scale reflection extension for the Coq system. Version 16. Technical report RR-6455, INRIA (2015)Google Scholar
  8. 8.
    Intel Advanced Vector Extensions Programming Reference, June 2011Google Scholar
  9. 9.
    Intel 64 and IA-32 Architectures Optimization Reference Manual, September 2015Google Scholar
  10. 10.
    Intel SSE4 Programming Reference, April 2007Google Scholar
  11. 11.
    Jacobson, G.: Succinct static data structures. Doctoral dissertation, Carnegie Mellon University (1988)Google Scholar
  12. 12.
    Jones, R.W.M.: A beginners guide to OCaml internals (2009).
  13. 13.
    Kim, D.-K., Na, J.C., Kim, J.E., Park, K.: Efficient implementation of rank and select functions for succinct representation. In: Nikoletseas, S.E. (ed.) WEA 2005. LNCS, vol. 3503, pp. 315–327. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Klein, G., Andronick, J., Elphinstone, K., Heiser, G., Cock, D., Derrin, P., Elkaduwe, D., Engelhardt, K., Kolanski, R., Norrish, M., Sewell, T., Tuch, H., Winwood, S.: seL4: formal verification of an operating-system kernel. Commun. ACM 53(6), 107–115 (2010)CrossRefGoogle Scholar
  15. 15.
    Nipkow, T.: Amortized complexity verified. In: Urban, C., Zhang, X. (eds.) ITP 2015. LNCS, vol. 9236, pp. 310–324. Springer, Berlin (2015)Google Scholar
  16. 16.
    SDSL: Succinct Data Structure Library.
  17. 17.
    OUnit: Unit test framework for OCaml.
  18. 18.
    Okanohara, D.: The world of fast character string analysis. Iwanami Shoten (2012). (in Japanese)Google Scholar
  19. 19.
    Tanaka, A., Affeldt, R., Garrigue, J.: Formal Verification of the Rank Function for Succinct Data Structures.

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Akira Tanaka
    • 1
  • Reynald Affeldt
    • 1
  • Jacques Garrigue
    • 2
  1. 1.National Institute of Advanced Industrial Science and Technology (AIST)TsukubaJapan
  2. 2.Nagoya UniversityNagoyaJapan

Personalised recommendations