eavesROP: Listening for ROP Payloads in Data Streams

  • Christopher Jämthagen
  • Linus Karlsson
  • Paul Stankovski
  • Martin Hell
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8783)


We consider the problem of detecting exploits based on return-oriented programming. In contrast to previous works we investigate to which extent we can detect ROP payloads by only analysing streaming data, i.e., we do not assume any modifications to the target machine, its kernel or its libraries. Neither do we attempt to execute any potentially malicious code in order to determine if it is an attack. While such a scenario has its limitations, we show that using a layered approach with a filtering mechanism together with the Fast Fourier Transform, it is possible to detect ROP payloads even in the presence of noise and assuming that the target system employs ASLR. Our approach, denoted eavesROP, thus provides a very lightweight and easily deployable mitigation against certain ROP attacks. It also provides the added merit of detecting the presence of a brute-force attack on ASLR since library base addresses are not assumed to be known by eavesROP.


Return-Oriented Programming ROP Pattern Matching ASLR 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    One, A.: Smashing the stack for fun and profit, phrack, 49 (1996)Google Scholar
  2. 2.
    Bletsch, T., Jiang, X., Freeh, V.W., Liang, Z.: Jump-oriented programming: A new class of code-reuse attack. In: Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security, ASIACCS 2011, pp. 30–40. ACM, New York (2011)Google Scholar
  3. 3.
    Bracewell, R.: The Fourier Transform and its Applications, 3rd edn. McGraw-Hill Series in Electrical and Computer Engineering. McGraw-Hill Science/Engineering/Math. (June 1999)Google Scholar
  4. 4.
    c0ntex: Bypassing non-executable-stack during exploitation using return-to-libc,
  5. 5.
    Cantoni, L.: BigAnt Server 2.52 SP5 - SEH Stack Overflow ROP-based exploit (ASLR + DEP bypass),
  6. 6.
    Checkoway, S., Davi, L., Dmitrienko, A., Sadeghi, A.R., Shacham, H., Winandy, M.: Return-oriented programming without returns. In: Proceedings of the 17th ACM Conference on Computer and Communications Security, CCS 2010, pp. 559–572. ACM, New York (2010)Google Scholar
  7. 7.
    Chen, P., Xiao, H., Shen, X., Yin, X., Mao, B., Xie, L.: DROP: Detecting return-oriented programming malicious code. In: Prakash, A., Sen Gupta, I. (eds.) ICISS 2009. LNCS, vol. 5905, pp. 163–177. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Cheng, Y., Zhou, Z., Miao, Y., Ding, X., Deng, R.: ROPecker: A generic and practical approach for defending against ROP attack. In: NDSS. Research Collection School of Information Systems (2014)Google Scholar
  9. 9.
    Cormen, T., Leiserson, C., Rivest, R., Stein, C.: Introduction to Algorithms, 3rd edn. MIT Press (2009)Google Scholar
  10. 10.
    Davi, L., Sadeghi, A., Winandy, M.: ROPdefender: A detection tool to defend against return-oriented programming attacks. In: Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security, ASIACCS 2011 (2011)Google Scholar
  11. 11.
    Durden, T.: Bypassing PaX ASLR protection, phrack, 59 (2002)Google Scholar
  12. 12.
    Fratric, I.: Ropguard: Runtime prevention of return-oriented programming attacks (2012)Google Scholar
  13. 13.
    Gupta, A., Kerr, S., Kirkpatrick, M., Bertino, E.: Marlin: Making it harder to fish for gadgets. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012. ACM (2012)Google Scholar
  14. 14.
  15. 15.
    Hiser, J., Nguyen-Tuong, A., Co, M., Hall, M., Davidson, J.: Ilr: Where’d my gadgets go? In: 2012 IEEE Symposium on Security and Privacy (SP) (2012)Google Scholar
  16. 16.
    Jämthagen, C., Karlsson, L., Stankovski, P., Hell, M.: eavesROP: Listening for ROP payloads in data streams (full version) (2014),
  17. 17.
    Li, J., Wang, Z., Jiang, X., Grace, M., Bahram, S.: Defeating return-oriented rootkits with “return-less” kernels. In: Proceedings of the 5th European Conference on Computer Systems, EuroSys 2010. ACM (2010)Google Scholar
  18. 18.
    Lu, K., Zou, D., Wen, W., Gao, D.: Packed, printable, and polymorphic return-oriented programming. In: Sommer, R., Balzarotti, D., Maier, G. (eds.) RAID 2011. LNCS, vol. 6961, pp. 101–120. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Onarlioglu, K., Bilge, L., Lanzi, A., Balzarotti, D., Kirda, E.: G-free: Defeating return-oriented programming through gadget-less binaries. In: Proceedings of the 26th Annual Computer Security Applications Conference, ACSAC 2010, pp. 49–58. ACM (2010)Google Scholar
  20. 20.
    Pappas, V., Polychronakis, M., Keromytis, A.: Smashing the gadgets: Hindering return-oriented programming using in-place code randomization. In: IEEE Symposium on Security and Privacy. IEEE Computer Society (2012)Google Scholar
  21. 21.
    Pappas, V., Polychronakis, M., Keromytis, A.: Transparent ROP exploit mitigation using indirect branch tracing. Presented as part of the 22nd USENIX Security Symposium (USENIX Security 2013). USENIX (2013)Google Scholar
  22. 22.
    PaX Team: Address space layout randomization (2003),
  23. 23.
    Polychronakis, M., Keromytis, A.: ROP payload detection using speculative code execution. In: Proceedings of the 2011 6th International Conference on Malicious and Unwanted Software, MALWARE 2011. IEEE Computer Society (2011)Google Scholar
  24. 24.
    Schwartz, E., Avgerinos, T., Brumley, D.: Q: Exploit hardening made easy. In: Proceedings of USENIX Security 2011 (2011)Google Scholar
  25. 25.
    Serna, F.J.: CVE-2012-0769, the case of the perfect info leak (2009),
  26. 26.
    Shacham, H.: The geometry of innocent flesh on the bone: Return-into-libc without function calls (on the x86). In: Proceedings of the 14th ACM Conference on Computer and Communications Security, CCS 2007, pp. 552–561. ACM (2007)Google Scholar
  27. 27.
    Shacham, H., Page, M., Pfaff, N., Goh, E., Modadugu, N., Boneh, D.: On the effectiveness of address-space randomization. In: Proceedings of the 11th ACM Conference on Computer and Communications Security, CCS 2004, pp. 298–307. ACM (2004)Google Scholar
  28. 28.
    Snow, K., Monrose, F., Davi, L., Dmitrienko, A., Liebchen, C., Sadeghi, A.: Just-in-time code reuse: On the effectiveness of fine-grained address space layout randomization. In: 2013 IEEE Symposium on Security and Privacy (SP), pp. 574–588 (May 2013)Google Scholar
  29. 29.
    Stancill, B., Snow, K.Z., Otterness, N., Monrose, F., Davi, L., Sadeghi, A.-R.: Check my profile: Leveraging static analysis for fast and accurate detection of ROP gadgets. In: Stolfo, S.J., Stavrou, A., Wright, C.V. (eds.) RAID 2013. LNCS, vol. 8145, pp. 62–81. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  30. 30.
    Sud0: Audio converter 8.1 0day stack buffer overflow PoC exploit ROP/WPM,
  31. 31.
    Vreugdenhil, P.: Pwn2Own 2010 Windows 7 Internet Explorer 8 exploit (2010),
  32. 32.
    Wartell, R., Mohan, V., Hamlen, K., Lin, Z.: Binary stirring: Self-randomizing instruction addresses of legacy x86 binary code. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Christopher Jämthagen
    • 1
  • Linus Karlsson
    • 1
  • Paul Stankovski
    • 1
  • Martin Hell
    • 1
  1. 1.Dept. of Electrical and Information TechnologyLund UniversityLundSweden

Personalised recommendations