Skip to main content
SpringerLink
Log in

Software transformations to improve malware detection

  • Original Paper
  • Published:
Journal in Computer Virology Aims and scope Submit manuscript

Abstract

Malware is code designed for a malicious purpose, such as obtaining root privilege on a host. A malware detector identifies malware and thus prevents it from adversely affecting a host. In order to evade detection, malware writers use various obfuscation techniques to transform their malware. There is strong evidence that commercial malware detectors are susceptible to these evasion tactics. In this paper, we describe the design and implementation of a malware transformer that reverses the obfuscations performed by a malware writer. Our experimental evaluation demonstrates that this malware transformer can drastically improve the detection rates of commercial malware detectors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. McGraw G. and Morrisett G. (2000). Attacking malicious code: report to the Infosec research council. IEEE Softw. 17(5): 33–41

    Article  Google Scholar 

  2. Szor, P. 11. In: The Art of Computer Virus Research and Defense. Addison-Wesley, Reading, pp. 425–494 (2005)

  3. Detristan, T., Ulenspiegel, T., Malcom, Y., von Underduk, M.S.: Polymorphic shellcode engine using spectrum analysis. Phrack 11(61) (August 2003) published online at http://www.phrack.org. Last accessed on 14 April (2006)

  4. Christodorescu, M., Jha, S.: Testing malware detectors. In: Proc. of the ACM SIGSOFT International Symposium on Software Testing and Analysis 2004 (ISSTA’04) July 2004, pp. 34–44 (2004)

  5. Mohanty, D.: Anti-virus evasion techniques and countermeasures. Published online at http://www.hackingspirits.com/eth-hac/papers/whitepapers.asp. Last accessed on 18 August (2005)

  6. AVV: Antiheuristics. 29A Magazine 1(1) (1999)

  7. Rajaat: Polymorphism. 29A Magazine 1(3) (1999)

  8. Julus, L.: Metamorphism. 29A Magazine 1(5) (2000)

  9. Mental Driller: Metamorphism in practice. 29A Magazine 1(6) (2002)

  10. Ször, P.: Advanced Code Evolution Techniques and Computer Virus Generator Kits. Symantec Press. In: The Art of Computer Virus Research and Defense. 1st edn. Addison Wesley, Reading February (2005)

  11. Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. Technical Report 148, Dept. of Computer Science, Univ. of Auckland, New Zealand July (1997)

  12. Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Proceedings of the 25th Annual ACM SIGPLAN-SIGACT symposium on principles of programming languages (POPL’98) January (1998)

  13. Henry, S.M., Kafura, D.G.: Software structure metrics based on information flow. IEEE Trans. Softw. Eng. 7(5): (1981)

  14. McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. 2(4) (1976)

  15. Munson, J.C., Khoshgoftaar, T.M.: Measurement of data structure complexity. J. Syst. Softw. 20(3) (1993)

  16. Jordan, M.: Dealing with metamorphism. pp. 4–6, Virus Bulletin (October 2002)

  17. Komondoor, R., Horwitz, S.: Semantics-preserving procedure extraction. In: Proceedings of the 27th ACM SIGPLAN-SIGACT symposium on principles of programming languages (POPL’00), pp. 155–169 (2000)

  18. Muchnick S. (1997). Advanced Compiler Design and Implementation. Morgan Kaufmann, San Francisco

    Google Scholar 

  19. DataRescue sa/nv: IDA Pro–interactive disassembler Published online at http://www.datarescue.com/idabase/. Last accessed on 14 April (2006)

  20. Bellard, F.: Qemu. Published online at http://fabrice.bellard.free.fr/qemu/. Last accessed on 14 April (2006)

  21. Wehner, S.: Analyzing worms and network traffic using compression. Published online at http://arxiv.org/abs/cs.CR/0504045. Last accessed on 14 April (2006)

  22. McDaniel, M., Heydari, M.H.: Content based file type detection algorithms. In: Proceedings of the 36th annual hawaii international conference on system sciences (HICCSS’03) January (2003)

  23. Li, W.J., Wang, K., Stolfo, S.J.: Fileprints: identifying file types by n-gram analysis. In: Proceedings of the 6th annual IEEE information assurance workshop, United States Military Academy, West Point, pp. 64–71 (June 2005)

  24. Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: Proceedings of the 10th ACM conference on computer and communications security (CCS’03) October (2003)

  25. Collberg, C., Thomborson, C., Low, D.: Breaking abstractions and unstructuring data structures. In: Proceedings of the international conference on computer languages 1998 (ICCL’98) May 1998, pp. 28–39 (1998)

  26. Christodorescu, M., Jha, S., Seshia, S.A., Song, D., Bryant, R.E.: Semantics-aware malware detection. In: Proceedigs of the 2005 IEEE symposium on security and privacy (Oakland 2005) May 2005, pp. 32–46 (2005)

  27. Detlefs, D., Nelson, G., Saxe, J.: The Simplify theorem prover Published online at http://www.hpl.hp.com/downloads/crl/jtk/download-simplify.html. Last accessed on 14 April (2006)

  28. Lahiri, S.K., Seshia, S.A.: The UCLID decision procedure. In Alur, R., Peled, D.A., (eds.) Proceedings of the 16th International Conference on Computer Aided Verification (CAV’04) July 2004. Volume 3114 of Lecture Notes in Computer Science, pp. 475–478 (2004)

  29. z0mbie: Automated reverse engineering: Mistfall engine. Published online at http://z0mbie.host.sk /autorev.txt. Last accessed: 16 January (2004)

  30. Oberhumer, M.F., Molnár, L.: The Ultimate Packer for eXecutables (UPX). Published online at http://upx.sourceforge.net/. Last accessed on 14 April (2006)

  31. Chow, S., Gu, Y., Johnson, H., Zakharov, V.: An approach to the obfuscation of control-flow of sequential computer programs. In Davida, G., Frankel, Y., (eds.) Proceedings of the 4th international information security conference (ISC’01). Volume 2200 of Lecture Notes in Computer Science October 2001, pp. 144–155 (2001)

  32. Beaucamps P. and Filiol E. (2007). On the possibility of practically obfuscating programs towards a unified perspective of code protection. J. Comput. Virol. 3(1): 3–21

    Article  Google Scholar 

  33. Nachenberg, C.: Polymorphic virus detection module. United States Patent # 5,826,013 (October 1998), (1998)

  34. Natvig, K.: Sandbox technology inside AV scanners. In: Proceedings of the 2001 virus bulletin conference September 2001, pp. 475–487 (2001)

  35. Natvig, K.: Sandbox II: Internet. In: Proceedings of the 2002 virus bulletin conference, pp. 1–18 (2002)

  36. Royal, P., Halpin, M., Dagon, D., Edmonds, R., Lee, W.: PolyUnpack: automating the hidden-code extraction of unpack-executing malware. In: the 22nd annual computer security applications conference (ACSAC’06), December 2006, pp. 289–300 (2006)

  37. Josse, S.: Secure and advanced unpacking using computer emulation. J. Comput. Virol. (2007)

  38. Udupa, S.K., Debray, S.K., Madou, M.: Deobfuscation: reverse engineering obfuscated code. In: Proceedings of the 12th IEEE working conference on reverse engineering (WCRE’05), November 2005 (2005)

  39. Walenstein, A., Mathur, R., Chouchane, M.R., Lakhotia, A.: Normalizing metamorphic malware using term rewriting. In: Proceedings of the 6th IEEE international workshop on source code analysis and manipulation (SCAM ’06) September 2006, pp. 75–84 (2006)

  40. Lakhotia, A., Mohammed, M.: Imposing order on program statements and its implication to AV scanners. In: Proceedings of the 11th IEEE working conference on reverser engineering (WCRE’04) November 2004, pp. 161–171 (2004)

  41. Perriot, F.: Defeating polymorphism through code optimization. In: Proceedings of the 2003 Virus Bulletin Conference (VB2003) September 2003, pp. 1–18 (2003)

  42. Bruschi, D., Martignoni, L., Monga, M.: Using code normalization for fighting self-mutating malware. In: Proceedings of the international symposium of secure software engineering (ISSSE’06) March (2006)

  43. Kapoor, A.: An approach towards disassembly of malicious binary executables. Master’s thesis, The Center for Advanced Computer Studies, University of Louisiana at Lafayette, November (2004)

  44. Kruegel, C., Robertson, W., Valeur, F., Vigna, G.: Static disassembly of obfuscated binaries. In: Proceedings of the 13th usenix security symposium (USENIX’04), San Diego August (2004)

  45. Kinder, J., Katzenbeisser, S., Schallhart, C., Veith, H.: Detecting malicious code by model checking. In Julisch, K., Krügel, C., (eds.) Proceedings of the 2nd international conference on intrusion and malware detection and vulnerability assessment (DIMVA’05). Volume 3548 of Lecture Notes in Computer Science July 2005, pp. 174–187 (2005)

Download references

Author information

Authors and Affiliations

  1. University of Wisconsin, Madison, USA

    Mihai Christodorescu & Somesh Jha

  2. Technische Universität München, Munich, Germany

    Johannes Kinder, Stefan Katzenbeisser & Helmut Veith

Authors
  1. Mihai Christodorescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Somesh Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Johannes Kinder
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Katzenbeisser
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Helmut Veith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mihai Christodorescu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Christodorescu, M., Jha, S., Kinder, J. et al. Software transformations to improve malware detection. J Comput Virol 3, 253–265 (2007). https://doi.org/10.1007/s11416-007-0059-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11416-007-0059-8

Keywords

Search

Navigation