Journal in Computer Virology

, Volume 3, Issue 4, pp 253–265 | Cite as

Software transformations to improve malware detection

  • Mihai Christodorescu
  • Somesh Jha
  • Johannes Kinder
  • Stefan Katzenbeisser
  • Helmut Veith
Original Paper

Abstract

Malware is code designed for a malicious purpose, such as obtaining root privilege on a host. A malware detector identifies malware and thus prevents it from adversely affecting a host. In order to evade detection, malware writers use various obfuscation techniques to transform their malware. There is strong evidence that commercial malware detectors are susceptible to these evasion tactics. In this paper, we describe the design and implementation of a malware transformer that reverses the obfuscations performed by a malware writer. Our experimental evaluation demonstrates that this malware transformer can drastically improve the detection rates of commercial malware detectors.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    McGraw G. and Morrisett G. (2000). Attacking malicious code: report to the Infosec research council. IEEE Softw. 17(5): 33–41 CrossRefGoogle Scholar
  2. 2.
    Szor, P. 11. In: The Art of Computer Virus Research and Defense. Addison-Wesley, Reading, pp. 425–494 (2005)Google Scholar
  3. 3.
    Detristan, T., Ulenspiegel, T., Malcom, Y., von Underduk, M.S.: Polymorphic shellcode engine using spectrum analysis. Phrack 11(61) (August 2003) published online at http://www.phrack.org. Last accessed on 14 April (2006)Google Scholar
  4. 4.
    Christodorescu, M., Jha, S.: Testing malware detectors. In: Proc. of the ACM SIGSOFT International Symposium on Software Testing and Analysis 2004 (ISSTA’04) July 2004, pp. 34–44 (2004)Google Scholar
  5. 5.
    Mohanty, D.: Anti-virus evasion techniques and countermeasures. Published online at http://www.hackingspirits.com/eth-hac/papers/whitepapers.asp. Last accessed on 18 August (2005)Google Scholar
  6. 6.
    AVV: Antiheuristics. 29A Magazine 1(1) (1999)Google Scholar
  7. 7.
    Rajaat: Polymorphism. 29A Magazine 1(3) (1999)Google Scholar
  8. 8.
    Julus, L.: Metamorphism. 29A Magazine 1(5) (2000)Google Scholar
  9. 9.
    Mental Driller: Metamorphism in practice. 29A Magazine 1(6) (2002)Google Scholar
  10. 10.
    Ször, P.: Advanced Code Evolution Techniques and Computer Virus Generator Kits. Symantec Press. In: The Art of Computer Virus Research and Defense. 1st edn. Addison Wesley, Reading February (2005)Google Scholar
  11. 11.
    Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. Technical Report 148, Dept. of Computer Science, Univ. of Auckland, New Zealand July (1997)Google Scholar
  12. 12.
    Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Proceedings of the 25th Annual ACM SIGPLAN-SIGACT symposium on principles of programming languages (POPL’98) January (1998)Google Scholar
  13. 13.
    Henry, S.M., Kafura, D.G.: Software structure metrics based on information flow. IEEE Trans. Softw. Eng. 7(5): (1981)Google Scholar
  14. 14.
    McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. 2(4) (1976)Google Scholar
  15. 15.
    Munson, J.C., Khoshgoftaar, T.M.: Measurement of data structure complexity. J. Syst. Softw. 20(3) (1993)Google Scholar
  16. 16.
    Jordan, M.: Dealing with metamorphism. pp. 4–6, Virus Bulletin (October 2002)Google Scholar
  17. 17.
    Komondoor, R., Horwitz, S.: Semantics-preserving procedure extraction. In: Proceedings of the 27th ACM SIGPLAN-SIGACT symposium on principles of programming languages (POPL’00), pp. 155–169 (2000)Google Scholar
  18. 18.
    Muchnick S. (1997). Advanced Compiler Design and Implementation. Morgan Kaufmann, San Francisco Google Scholar
  19. 19.
    DataRescue sa/nv: IDA Pro–interactive disassembler Published online at http://www.datarescue.com/idabase/. Last accessed on 14 April (2006)Google Scholar
  20. 20.
    Bellard, F.: Qemu. Published online at http://fabrice.bellard.free.fr/qemu/. Last accessed on 14 April (2006)Google Scholar
  21. 21.
    Wehner, S.: Analyzing worms and network traffic using compression. Published online at http://arxiv.org/abs/cs.CR/0504045. Last accessed on 14 April (2006)Google Scholar
  22. 22.
    McDaniel, M., Heydari, M.H.: Content based file type detection algorithms. In: Proceedings of the 36th annual hawaii international conference on system sciences (HICCSS’03) January (2003)Google Scholar
  23. 23.
    Li, W.J., Wang, K., Stolfo, S.J.: Fileprints: identifying file types by n-gram analysis. In: Proceedings of the 6th annual IEEE information assurance workshop, United States Military Academy, West Point, pp. 64–71 (June 2005)Google Scholar
  24. 24.
    Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: Proceedings of the 10th ACM conference on computer and communications security (CCS’03) October (2003)Google Scholar
  25. 25.
    Collberg, C., Thomborson, C., Low, D.: Breaking abstractions and unstructuring data structures. In: Proceedings of the international conference on computer languages 1998 (ICCL’98) May 1998, pp. 28–39 (1998)Google Scholar
  26. 26.
    Christodorescu, M., Jha, S., Seshia, S.A., Song, D., Bryant, R.E.: Semantics-aware malware detection. In: Proceedigs of the 2005 IEEE symposium on security and privacy (Oakland 2005) May 2005, pp. 32–46 (2005)Google Scholar
  27. 27.
    Detlefs, D., Nelson, G., Saxe, J.: The Simplify theorem prover Published online at http://www.hpl.hp.com/downloads/crl/jtk/download-simplify.html. Last accessed on 14 April (2006)Google Scholar
  28. 28.
    Lahiri, S.K., Seshia, S.A.: The UCLID decision procedure. In Alur, R., Peled, D.A., (eds.) Proceedings of the 16th International Conference on Computer Aided Verification (CAV’04) July 2004. Volume 3114 of Lecture Notes in Computer Science, pp. 475–478 (2004)Google Scholar
  29. 29.
    z0mbie: Automated reverse engineering: Mistfall engine. Published online at http://z0mbie.host.sk /autorev.txt. Last accessed: 16 January (2004)Google Scholar
  30. 30.
    Oberhumer, M.F., Molnár, L.: The Ultimate Packer for eXecutables (UPX). Published online at http://upx.sourceforge.net/. Last accessed on 14 April (2006)Google Scholar
  31. 31.
    Chow, S., Gu, Y., Johnson, H., Zakharov, V.: An approach to the obfuscation of control-flow of sequential computer programs. In Davida, G., Frankel, Y., (eds.) Proceedings of the 4th international information security conference (ISC’01). Volume 2200 of Lecture Notes in Computer Science October 2001, pp. 144–155 (2001)Google Scholar
  32. 32.
    Beaucamps P. and Filiol E. (2007). On the possibility of practically obfuscating programs towards a unified perspective of code protection. J. Comput. Virol. 3(1): 3–21 CrossRefGoogle Scholar
  33. 33.
    Nachenberg, C.: Polymorphic virus detection module. United States Patent # 5,826,013 (October 1998), (1998)Google Scholar
  34. 34.
    Natvig, K.: Sandbox technology inside AV scanners. In: Proceedings of the 2001 virus bulletin conference September 2001, pp. 475–487 (2001)Google Scholar
  35. 35.
    Natvig, K.: Sandbox II: Internet. In: Proceedings of the 2002 virus bulletin conference, pp. 1–18 (2002)Google Scholar
  36. 36.
    Royal, P., Halpin, M., Dagon, D., Edmonds, R., Lee, W.: PolyUnpack: automating the hidden-code extraction of unpack-executing malware. In: the 22nd annual computer security applications conference (ACSAC’06), December 2006, pp. 289–300 (2006)Google Scholar
  37. 37.
    Josse, S.: Secure and advanced unpacking using computer emulation. J. Comput. Virol. (2007)Google Scholar
  38. 38.
    Udupa, S.K., Debray, S.K., Madou, M.: Deobfuscation: reverse engineering obfuscated code. In: Proceedings of the 12th IEEE working conference on reverse engineering (WCRE’05), November 2005 (2005)Google Scholar
  39. 39.
    Walenstein, A., Mathur, R., Chouchane, M.R., Lakhotia, A.: Normalizing metamorphic malware using term rewriting. In: Proceedings of the 6th IEEE international workshop on source code analysis and manipulation (SCAM ’06) September 2006, pp. 75–84 (2006)Google Scholar
  40. 40.
    Lakhotia, A., Mohammed, M.: Imposing order on program statements and its implication to AV scanners. In: Proceedings of the 11th IEEE working conference on reverser engineering (WCRE’04) November 2004, pp. 161–171 (2004)Google Scholar
  41. 41.
    Perriot, F.: Defeating polymorphism through code optimization. In: Proceedings of the 2003 Virus Bulletin Conference (VB2003) September 2003, pp. 1–18 (2003)Google Scholar
  42. 42.
    Bruschi, D., Martignoni, L., Monga, M.: Using code normalization for fighting self-mutating malware. In: Proceedings of the international symposium of secure software engineering (ISSSE’06) March (2006)Google Scholar
  43. 43.
    Kapoor, A.: An approach towards disassembly of malicious binary executables. Master’s thesis, The Center for Advanced Computer Studies, University of Louisiana at Lafayette, November (2004)Google Scholar
  44. 44.
    Kruegel, C., Robertson, W., Valeur, F., Vigna, G.: Static disassembly of obfuscated binaries. In: Proceedings of the 13th usenix security symposium (USENIX’04), San Diego August (2004)Google Scholar
  45. 45.
    Kinder, J., Katzenbeisser, S., Schallhart, C., Veith, H.: Detecting malicious code by model checking. In Julisch, K., Krügel, C., (eds.) Proceedings of the 2nd international conference on intrusion and malware detection and vulnerability assessment (DIMVA’05). Volume 3548 of Lecture Notes in Computer Science July 2005, pp. 174–187 (2005)Google Scholar

Copyright information

© Springer-Verlag France 2007

Authors and Affiliations

  • Mihai Christodorescu
    • 1
  • Somesh Jha
    • 1
  • Johannes Kinder
    • 2
  • Stefan Katzenbeisser
    • 2
  • Helmut Veith
    • 2
  1. 1.University of WisconsinMadisonUSA
  2. 2.Technische Universität MünchenMunichGermany

Personalised recommendations