Systematic Analysis of Defenses against Return-Oriented Programming

  • Richard Skowyra
  • Kelly Casteel
  • Hamed Okhravi
  • Nickolai Zeldovich
  • William Streilein
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8145)


Since the introduction of return-oriented programming, increasingly complex defenses and subtle attacks that bypass them have been proposed. Unfortunately the lack of a unifying threat model among code reuse security papers makes it difficult to evaluate the effectiveness of defenses, and answer critical questions about the interoperability, composability, and efficacy of existing defensive techniques. For example, what combination of defenses protect against every known avenue of code reuse? What is the smallest set of such defenses? In this work, we study the space of code reuse attacks by building a formal model of attacks and their requirements, and defenses and their assumptions. We use a SAT solver to perform scenario analysis on our model in two ways. First, we analyze the defense configurations of a real-world system. Second, we reason about hypothetical defense bypasses. We prove by construction that attack extensions implementing the hypothesized functionality are possible even if a ‘perfect’ version of the defense is implemented. Our approach can be used to formalize the process of threat model definition, analyze defense configurations, reason about composability and efficacy, and hypothesize about new attacks and defenses.


System Call Address Space Threat Model Code Reuse Brute Force Attack 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abadi, M., Budiu, M., Erlingsson, U., Ligatti, J.: Control-flow integrity principles, implementations, and applications. ACM Trans. Inf. Syst. Secur. 13(1), 4:1–4:40 (2009)Google Scholar
  2. 2.
    Arnold, S.: Security/features (March 2013),
  3. 3.
  4. 4.
    Bletsch, T., Jiang, X., Freeh, V., Liang, Z.: Jump-oriented programming: A new class of code-reuse attack. In: Proc. of the 6th ACM CCS (2011)Google Scholar
  5. 5.
    Bray, B.: Compiler security checks in depth (2002),
  6. 6.
    Buchanan, E., Roemer, R., Shacham, H., Savage, S.: When good instructions go bad: generalizing return-oriented programming to RISC. In: Proc. of the 15th ACM CCS (2008)Google Scholar
  7. 7.
    Checkoway, S., Davi, L., Dmitrienko, A., Sadeghi, A., Shacham, H., Winandy, M.: Return-oriented programming without returns. In: Proc. of the 17th ACM CCS, pp. 559–572 (2010)Google Scholar
  8. 8.
    Cowan, C., Beattie, S., Johansen, J., Wagle, P.: Pointguard: protecting pointers from buffer overflow vulnerabilities. In: Proceedings of the 12th USENIX Security Symposium (2003)Google Scholar
  9. 9.
  10. 10.
    Etoh, H.: Propolice: Gcc extension for protecting applications from stack-smashing attacks. IBM ( April 2003),
  11. 11.
    Hiser, J., Nguyen, A., Co, M., Hall, M., Davidson, J.: ILR: Where’d my gadgets go. In: IEEE Symposium on Security and Privacy (2012)Google Scholar
  12. 12.
    Homescu, A., Stewart, M., Larsen, P., Brunthaler, S., Franz, M.: Microgadgets: size does matter in turing-complete return-oriented programming. In: Proceedings of the 6th USENIX Conference on Offensive Technologies, p. 7. USENIX Association (2012)Google Scholar
  13. 13.
    Katebi, H., Sakallah, K.A., Marques-Silva, J.P.: Empirical study of the anatomy of modern sat solvers. In: Sakallah, K.A., Simon, L. (eds.) SAT 2011. LNCS, vol. 6695, pp. 343–356. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Kayaalp, M., Ozsoy, M., Abu-Ghazaleh, N., Ponomarev, D.: Branch regulation: low-overhead protection from code reuse attacks. In: Proceedings of the 39th International Symposium on Computer Architecture, pp. 94–105 (2012)Google Scholar
  15. 15.
    Kil, C., Jun, J., Bookholt, C., Xu, J., Ning, P.: Address space layout permutation (ASLP): Towards fine-grained randomization of commodity software. In: Proc. of ACSAC 2006 (2006)Google Scholar
  16. 16.
    Kornau, T.: Return oriented programming for the ARM architecture. Ph.D. thesis, Master’s thesis, Ruhr-Universitat Bochum (2010)Google Scholar
  17. 17.
    Li, J., Wang, Z., Jiang, X., Grace, M., Bahram, S.: Defeating return-oriented rootkits with “return-less” kernels. In: EuroSys (2010)Google Scholar
  18. 18.
    de Moura, L., Bjørner, N.: Z3: An efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Nergal: The advanced return-into-lib(c) exploits (pax case study). Phrack Magazine 58(4),  54 (2001)Google Scholar
  20. 20.
    Onarlioglu, K., Bilge, L., Lanzi, A., Balzarotti, D., Kirda, E.: G-free: Defeating return-oriented programming through gadget-less binaries. In: Proc. of ACSAC 2010 (2010)Google Scholar
  21. 21.
    One, A.: Smashing the stack for fun and profit. Phrack Magazine 7(49), 14–16 (1996)Google Scholar
  22. 22.
    Pappas, V., Polychronakis, M., Keromytis, A.: Smashing the gadgets: Hindering return-oriented programming using in-place code randomization. In: Proc. of IEEE Symposium on Security and Privacy (2012)Google Scholar
  23. 23.
    PaX: PaX non-executable pages design & implem.,
  24. 24.
    Polychronakis, M., Anagnostakis, K.G., Markatos, E.P.: Emulation-based detection of non-self-contained polymorphic shellcode. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 87–106. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  25. 25.
    Roemer, R.: Finding the bad in good code: Automated return-oriented programming exploit discovery. Ph.D. thesis, UCSD (2009)Google Scholar
  26. 26.
    Roglia, G., Martignoni, L., Paleari, R., Bruschi, D.: Surgically returning to randomized lib (c). In: Proc. of ACSAC 2009 (2009)Google Scholar
  27. 27.
    Russinovich, M.: Windows internals. Microsoft, Washington, DC (2009)Google Scholar
  28. 28.
    Shacham, H.: The geometry of innocent flesh on the bone: Return-into-libc without function calls (on the x86). In: ACM CCS (2007)Google Scholar
  29. 29.
    Shacham, H., Page, M., Pfaff, B., Goh, E.J., Modadugu, N., Boneh, D.: On the effectiveness of address-space randomization. In: Proc. of ACM CCS, pp. 298–307 (2004)Google Scholar
  30. 30.
    Sinnadurai, S., Zhao, Q., fai Wong, W.: Transparent runtime shadow stack: Protection against malicious return address modifications (2008)Google Scholar
  31. 31.
    Snow, K., Monrose, F., Davi, L., Dmitrienko, A.: Just-in-time code reuse: On the effectiveness of fine-grained address space layout randomization. In: Proc. of IEEE Symposium on Security and Privacy (2013)Google Scholar
  32. 32.
    Strackx, R., Younan, Y., Philippaerts, P., Piessens, F., Lachmund, S., Walter, T.: Breaking the memory secrecy assumption. In: Proc. of EuroSec 2009 (2009)Google Scholar
  33. 33.
    Team, P.: Pax address space layout randomization, aslr (2003)Google Scholar
  34. 34.
    Tran, M., Etheridge, M., Bletsch, T., Jiang, X., Freeh, V., Ning, P.: On the expressiveness of return-into-libc attacks. In: Sommer, R., Balzarotti, D., Maier, G. (eds.) RAID 2011. LNCS, vol. 6961, pp. 121–141. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  35. 35.
    Twitch: Taking advantage of non-terminated adjacent memory spaces. Phrack 56 (2000)Google Scholar
  36. 36.
    van de Ven, A.: New security enhancements in red hat enterprise linux v. 3, update 3. Raleigh (2004)Google Scholar
  37. 37.
    Wachter, M., Haenni, R.: Propositional dags: a new graph-based language for representing boolean functions. KR 6, 277–285 (2006)Google Scholar
  38. 38.
    Wartell, R., Mohan, V., Hamlen, K.W., Lin, Z.: Binary stirring: self-randomizing instruction addresses of legacy x86 binary code. In: Proc. of ACM CCS, pp. 157–168 (2012)Google Scholar
  39. 39.
    Xu, H., Chapin, S.: Improving address space randomization with a dynamic offset randomization technique. In: Proc. of the 2006 ACM Symposium on Applied Computing (2006)Google Scholar
  40. 40.
    Younan, Y., Joosen, W., Piessens, F.: Code injection in C and C++: A survey of vulnerabilities and countermeasures. Technical Report CW386, Katholieke Universiteit Leuven (July 2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Richard Skowyra
    • 1
  • Kelly Casteel
    • 2
  • Hamed Okhravi
    • 2
  • Nickolai Zeldovich
    • 3
  • William Streilein
    • 2
  1. 1.Boston UniversityUSA
  2. 2.MIT Lincoln LaboratoryUSA

Personalised recommendations