CastSan: Efficient Detection of Polymorphic C++ Object Type Confusions with LLVM

  • Paul MunteanEmail author
  • Sebastian Wuerl
  • Jens Grossklags
  • Claudia Eckert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11098)


C++ object type confusion vulnerabilities as the result of illegal object casting have been threatening systems’ security for decades. While there exist several solutions to address this type of vulnerability, none of them are sufficiently practical for adoption in production scenarios. Most competitive and recent solutions require object type tracking for checking polymorphic object casts, and all have prohibitively high runtime overhead. The main source of overhead is the need to track the object type during runtime for both polymorphic and non-polymorphic object casts. In this paper, we present CastSan, a C++ object type confusion detection tool for polymorphic objects only, which scales efficiently to large and complex code bases as well as to many concurrent threads. To considerably reduce the object type cast checking overhead, we employ a new technique based on constructing the whole virtual table hierarchy during program compile time. Since CastSan does not rely on keeping track of the object type during runtime, the overhead is drastically reduced. Our evaluation results show that complex applications run insignificantly slower when our technique is deployed, thus making CastSan a real-world usage candidate. Finally, we envisage that based on our object type confusion detection technique, which relies on ordered virtual tables (vtables), even non-polymorphic object casts could be precisely handled by constructing auxiliary non-polymorphic function table hierarchies for static classes as well.


Static cast Type confusion Bad casting Type safety Type casting 



We thank Mathias Payer from EPFL, CH; for insights which helped to improve paper quality. We thank Dimitar Bounov from the University of California, San Diego, USA; and Benjamin Johnson from the Technical University of Munich, Germany for reviewing an early version of this paper. Jens Grossklags’ research is supported by the German Institute for Trust and Safety on the Internet (DIVSI). Further, we thank the anonymous reviewers for their rich feedback.


  1. 1.
    2016 Working Draft, Standard for Programming Language C++ N4618.
  2. 2.
    Abadi, M., Budiu, M., Erlingsson, Ú., Ligatti, J.: Control flow integrity. In: CCS (2005)Google Scholar
  3. 3.
    Abadi, M., Budiu, M., Erlingsson, Ú., Ligatti, J.: Control flow integrity principles, implementations, and applications. In: TISSEC (2009)Google Scholar
  4. 4.
    Balls Browser Benchmark (2017).
  5. 5.
    Bounov, D., Kici, R.G., Lerner, S.: Protecting C++ dynamic dispatch through VTable interleaving. In: NDSS (2016)Google Scholar
  6. 6.
    Buchanan, E., Roemer, R., Shacham, H., Savage, S.: When good instructions go bad: generalizing return-oriented programming to RISC. In: CCS (2008)Google Scholar
  7. 7.
    Clang. Clang 3.9 Documentation - Control Flow Integrity.
  8. 8.
    Clang. Clang 5 Documentation - Control Flow Integrity (2017).
  9. 9.
    Clang-CFI Cast Checker Metadata.
  10. 10.
    Crane, S., et al.: It’s a TRaP: table randomization and protection against function-reuse attacks. In: CCS (2015)Google Scholar
  11. 11.
    CVE-2016-1612: Bug Description and reward (2016).
  12. 12.
    CVE-2017-3106: Object Type Confusion in Adobe F. Player v. (2017).
  13. 13.
    Dewey, D., Giffin, J.: Static detection of C++ VTable escape vulnerabilities in binary code. In: NDSS (2012)Google Scholar
  14. 14.
    Dromaeo Browser Benchmark (2017).
  15. 15.
    Google. Undefined Behavior Sanitizer (2017).
  16. 16.
    Google. The Chromium Projects, Chromium (2017).
  17. 17.
    Haller, I., Goktas, E., Athanasopoulos, E., Portokalidis, G., Bos, H.: ShrinkWrap: VTable protection without loose ends. In: ACSAC (2015)Google Scholar
  18. 18.
    Haller, I., Jeon, Y., Peng, H., Payer, M., Giuffrida, C.: TypeSan: practical type confusion detection. In: CCS (2016)Google Scholar
  19. 19.
    Jeon, Y., Biswas, P., Carr, S., Lee, B., Payer, M.: HexType: efficient detection of type confusion errors for C++. In: CCS (2017)Google Scholar
  20. 20.
    JetStream Browser Benchmark (2017).
  21. 21.
    Kraken JavaScript Benchmark (2017).
  22. 22.
    Lee, B., Song, C., Kim, T., Lee, W.: Type casting verification: stopping an emerging attack vector. In: USENIX Security (2015)Google Scholar
  23. 23.
    LLVM. The LLVM Gold Plugin (2017).
  24. 24.
    LLVM. LLVM Team, The LLVM compiler infrastructure project.
  25. 25.
    LLVM. LLVM link time optimization: design and implementation.
  26. 26.
    Microsoft. Changes to Functionality in Microsoft Windows XP SP 2.
  27. 27.
    Octane Browser Benchmark (2017).
  28. 28.
    PaX Team: Address Space Layout Randomization (2001).
  29. 29.
    Prakash, A., Hu, X., Yin, H.: Strict protection for virtual function calls in COTS C++ binaries. In: NDSS (2015)Google Scholar
  30. 30.
    Schuster, F., Tendyck, T., Liebchen, C., Davi, L., Sadeghi, A.-R., Holz, T.: Counterfeit object-oriented programming. In: S&P (2015)Google Scholar
  31. 31.
    Standard Performance Evaluation Corporation. SPEC CPU 2006 (2017).
  32. 32.
    SunSpider 1.0.2 JavaScript Benchmark (2017).
  33. 33.
    Zhang, C., et al.: Practical control flow integrity & randomization for binary executables. In: S&P (2013)Google Scholar
  34. 34.
    Zhao, M., Grossklags, J., Liu, P.: An empirical study of web vulnerability discovery ecosystems. In: CCS (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Paul Muntean
    • 1
    Email author
  • Sebastian Wuerl
    • 1
  • Jens Grossklags
    • 1
  • Claudia Eckert
    • 1
  1. 1.Technical University of MunichMunichGermany

Personalised recommendations