Diversifying the Software Stack Using Randomized NOP Insertion

  • Todd Jackson
  • Andrei Homescu
  • Stephen Crane
  • Per Larsen
  • Stefan Brunthaler
  • Michael Franz
Conference paper
Part of the Advances in Information Security book series (ADIS, volume 100)


Software monoculture is a significant liability from a computer security perspective. Single attacks can ripple through networks and affect large numbers of vulnerable systems. A simple but unusually powerful idea to solve this problem is to use artificial diversity in software systems. After discussing the design space of introducing artificial diversity, we present an in-depth performance analysis of our own technique: randomly inserting non-alignment NOP instructions. We observe that this technique has a moderate performance impact and demonstrate its real world applicability by diversifying a full system stack.


Performance Impact Performance Overhead Insertion Probability Translation Lookaside Buffer Instruction Stream 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Parts of this effort have been sponsored by the Defense Advanced Research Projects Agency (DARPA) under agreement number D11PC20024, and by a generous gift by Google.

The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Any opinions, findings, and conclusions or recommendations expressed here are those of the authors and do not necessarily reflect the views of DARPA or Google.


  1. 1.
    M. Abadi, M. Budiu, Ú. Erlingsson, and J. Ligatti. Control-flow integrity principles, implementations, and applications. ACM Transactions on Information System Security, 13:4:1–4:40, 2009.Google Scholar
  2. 2.
    A. Avizienis and L. Chen. On the implementation of n-version programming for software fault tolerance during execution. In Proceedings of the International Computer Software and Applications Conference, pages 149–155, 1977.Google Scholar
  3. 3.
    Aleph One. Smashing the stack for fun and profit. Phrack Magazine, Issue 49, 1996.Google Scholar
  4. 4.
    Internet Explorer “Aurora” Attack, 2010. (CVE-2010-0249).Google Scholar
  5. 5.
    E.G. Barrantes, D.H. Ackley, S. Forrest, and D. Stefanović. Randomized Instruction Set Emulation. ACM Transactions on Information and System Security, 8(1):3–40, 2005.Google Scholar
  6. 6.
    D. Bruschi, L. Cavallaro, and A. Lanzi. Diversified process replicae for defeating memory error exploits. In Proceedings of the International Workshop on Information Assurance, pages 434–441, 2007.Google Scholar
  7. 7.
    S. Bhatkar, D.C. DuVarney, and R. Sekar. Address Obfuscation: An Efficient Approach to Combat a Broad Range of Memory Error Exploits. In Proceedings of the 12th USENIX Security Symposium, pages 105–120, 2003.Google Scholar
  8. 8.
    T. Bletsch, X. Jiang, and V. Freeh. Mitigating code-reuse attacks with control-flow locking. In Proceedings of the 27th Annual Computer Security Applications Conference, pages 353–362. ACM, 2011.Google Scholar
  9. 9.
    T. Bletsch, X. Jiang, V. Freeh, and Z. Liang. Jump-oriented programming: a new class of code-reuse attack. In Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security, pages 30–40, 2011.Google Scholar
  10. 10.
    E. Buchanan, R. Roemer, H. Shacham, and S. Savage. When good instructions go bad: generalizing return-oriented programming to RISC. In Proceedings of the 15th ACM Conference on Computer and Communications Security, pages 27–38, 2008.Google Scholar
  11. 11.
    S. Checkoway, L. Davi, A. Dmitrienko, A. Sadeghi, H. Shacham, and M. Winandy. Return-Oriented Programming without Returns. In Proceedings of the 17th ACM Conference on Computer and Communications Security, pages 559–72, 2010.Google Scholar
  12. 12.
    B. Cox, D. Evans, A. Filipi, J. Rowanhill, W. Hu, J. Davidson, J. Knight, A. Nguyen-Tuong, and J. Hiser. N-variant systems: A Secretless Framework for Security through Diversity. In Proceedings of the 15th USENIX Security Symposium, pages 105–120, 2006.Google Scholar
  13. 13.
    C. Cowan, C. Pu, D. Maier, J. Walpole, P. Bakke, D. Beattie, A. Grier, P. Wagle, Q. Zhang, and H. Hinton. StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks. In Proceedings of the 7th USENIX Security Symposium, pages 63–78, 1998.Google Scholar
  14. 14.
    P. Chen, X. Xing, H. Han, B. Mao, and L. Xie. Efficient Detection of the Return-oriented Programming Malicious Code. In Proceedings of the 6th International Conference on Information Systems Security, pages 140–155, 2010.Google Scholar
  15. 15.
    M. Franz. E unibus pluram: Massive-Scale Software Diversity as a Defense Mechanism. In Proceedings of the 2010 Workshop on New Security Paradigms, NSPW ’10, pages 7–16, New York, NY, USA, 2010. ACM.Google Scholar
  16. 16.
    Jin Han, Debin Gao, and Robert H. Deng. On the effectiveness of software diversity: A systematic study on real-world vulnerabilities. In Proceedings of the 6th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pages 127–146, 2009.Google Scholar
  17. 17.
    R. Hund, T. Holz, and F.C. Freiling. Return-oriented rootkits: Bypassing kernel code integrity protection mechanisms. In Proceedings of the 18th USENIX Security Symposium, pages 383–398, 2009.Google Scholar
  18. 18.
    Intel Corporation. Intel 64 and IA-32 architectures optimization reference manual.Google Scholar
  19. 19.
    M. Jacob, M. Jakubowski, P. Naldurg, C. Saw, and R. Venkatesan. The superdiversifier: Peephole individualization for software protection. In K. Matsuura and E. Fujisaki, editors, Advances in Information and Computer Security, volume 5312 of Lecture Notes in Computer Science, pages 100–120. Springer Berlin / Heidelberg, 2008.Google Scholar
  20. 20.
    Todd Jackson, Babak Salamat, Andrei Homescu, Karthikeyan Manivannan, Gregor Wagner, Andreas Gal, Stefan Brunthaler, Christian Wimmer, and Michael Franz. Compiler-generated software diversity. In Sushil Jajodia, Anup K. Ghosh, Vipin Swarup, Cliff Wang, and X. Sean Wang, editors, Moving Target Defense, volume 54 of Advances in Information Security, pages 77–98. Springer New York, 2011.Google Scholar
  21. 21.
    G.S. Kc, A.D. Keromytis, and V. Prevelakis. Countering Code-Injection Attacks with Instruction-Set Randomization. In Proceedings of the 10th ACM Conference on Computer and Communications Security, pages 272–280, 2003.Google Scholar
  22. 22.
    S. Krahmer. x86-64 buffer overflow exploits and the borrowed code chunks exploitation techniques. 2005.
  23. 23.
    Richard C. Linger. Systematic generation of stochastic diversity as an intrusion barrier in survivable systems software. In Proceedings of the Thirty-Second Annual Hawaii International Conference on System Sciences, pages 3062–, 1999.Google Scholar
  24. 24.
    H. Massalin. Superoptimizer: a look at the smallest program. In Proceedings of the Second International Conference on Architectual Support for Programming Languages and Operating Systems, pages 122–126, 1987.Google Scholar
  25. 25.
    S. McCamant and G. Morrisett. Evaluating SFI for a CISC architecture. In Proceedings of the 15th USENIX Security Symposium, pages 209–224, 2006.Google Scholar
  26. 26.
    A. Matrosov, E. Rodionov, D. Harley, and J. Malcho. Stuxnet Under the Microscope, 2010. Accessed 01/09/2012.Google Scholar
  27. 27.
    Nergal. The advanced return-into-lib(c) exploits: PaX case study. Phrack Magazine, Issue 58, 2001.Google Scholar
  28. 28.
    Anh Nguyen-Tuong, Andrew Wang, Jason D. Hiser, John C. Knight, and Jack W. Davidson. On the effectiveness of the metamorphic shield. In Proceedings of the Fourth European Conference on Software Architecture: Companion Volume, pages 170–174, 2010.Google Scholar
  29. 29.
    K. Onarlioglu, L. Bilge, A. Lanzi, D. Balzarotti, and E. Kirda. G-free: defeating return-oriented programming through gadget-less binaries. In Proceedings of the 26th Annual Computer Security Applications Conference, pages 49–58, 2010.Google Scholar
  30. 30.
    PaX. Homepage of The PaX Team, 2009.
  31. 31.
    R. Roemer, E. Buchanan, H. Shacham, and S. Savage. Return-oriented programming: Systems, languages, and applications. ACM Transactions in Information and Systems Security, 2011. To appear.Google Scholar
  32. 32.
    E. J. Schwartz, T. Avgerinos, and D. Brumley. Q: Exploit Hardening Made Easy. In Proceedings of the 20th USENIX Security Symposium, 2011.Google Scholar
  33. 33.
    B. Salamat, A. Gal, and M. Franz. Reverse Stack Execution in a Multi-Variant Execution Environment. In Workshop on Compiler and Architectural Techniques for Application Reliability and Security, 2008.Google Scholar
  34. 34.
    H. Shacham. The Geometry of Innocent Flesh on the Bone: Return-into-libc without Function Calls (on the x86). In Proceedings of the 14th ACM Conference on Computer and Communications Security, pages 552–561, 2007.Google Scholar
  35. 35.
    B. Salamat, T. Jackson, G. Wagner, C. Wimmer, and M. Franz. Run-Time Defense against Code Injection Attacks using Replicated Execution. IEEE Transactions on Dependable and Secure Computing, 2011.Google Scholar
  36. 36.
    P. Sole. Hanging on a ROPe. In ekoParty Security Conference, 2010.
  37. 37.
    scut / team teso. Exploiting Format String Vulnerabilities. 2001.
  38. 38.
    M. Tran, M. Etheridge, T. Bletsch, X. Jiang, V. W. Freeh, and P. Ning. On the Expressiveness of Return-into-libc Attacks. In Proceedings of the 14th Interntional Symposium on Recent Advances in Intrusion Detection, 2011.Google Scholar
  39. 39.
    D. W. Williams, W. Hu, J. W. Davidson, J. Hiser, J. C. Knight, and A. Nguyen-Tuong. Security through diversity: Leveraging virtual machine technology. IEEE Security & Privacy, 7(1): 26–33, 2009.Google Scholar
  40. 40.
    B. Yee, D. Sehr, G. Dardyk, J. B. Chen, R. Muth, T. Ormandy, S. Okasaka, N. Narula, and N. Fullagar. Native client: A sandbox for portable, untrusted x86 native code. In IEEE Symposium on Security and Privacy, pages 79–93, 2009.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Todd Jackson
    • 1
  • Andrei Homescu
    • 1
  • Stephen Crane
    • 1
  • Per Larsen
    • 1
  • Stefan Brunthaler
    • 1
  • Michael Franz
    • 1
  1. 1.Department of Computer ScienceUniversity of CaliforniaIrvineUSA

Personalised recommendations