Advertisement

Detecting Self-mutating Malware Using Control-Flow Graph Matching

  • Danilo Bruschi
  • Lorenzo Martignoni
  • Mattia Monga
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4064)

Abstract

Next generation malware will by be characterized by the intense use of polymorphic and metamorphic techniques aimed at circumventing the current malware detectors, based on pattern matching. In order to deal with this new kind of threat, novel techniques have to be devised for the realization of malware detectors. Recent papers started to address such an issue and this paper represents a further contribution in such a field. More precisely in this paper we propose a strategy for the detection of metamorphic malicious code inside a program P based on the comparison of the control flow graphs of P against the set of control flow graphs of known malware. We also provide experimental data supporting the validity of our strategy.

Keywords

Malicious Code Code Fragment Subgraph Isomorphism Executable Code Host Program 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
  3. 3.
    Aho, A.V., Sethi, R., Ullman, J.D.: Compilers: Principles, Techniques and Tools. Addison-Wesley, Reading (1986)Google Scholar
  4. 4.
    C. Associates. Security advisor center glossary, http://www3.ca.com/securityadvisor/glossary.aspx
  5. 5.
    Bruschi, D., Martignoni, L., Monga, M.: Using code normalization for fighting self-mutating malware. In: Proceedings of the International Symposium of Secure Software Engineering, Arlington, VA. IEEE Computer Society, Los Alamitos (2006)Google Scholar
  6. 6.
    Chess, D.M., White, S.R.: An undetectable computer virus. In: Proceedings of Virus Bulletin Conference (September 2000)Google Scholar
  7. 7.
    Christodorescu, M., Jha, S.: Static analysis of executables to detect malicious patterns. In: Proceedings of USENIX Security Symposium (August 2003)Google Scholar
  8. 8.
    Christodorescu, M., Jha, S.: Testing malware detectors. In: Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2004), Boston, MA, USA, pp. 34–44. ACM Press, New York (2004)CrossRefGoogle Scholar
  9. 9.
    Christodorescu, M., Jha, S., Seshia, S.A., Song, D., Bryant, R.E.: Semantics-aware malware detection. In: Proceedings of the 2005 IEEE Symposium on Security and Privacy (Oakland 2005), Oakland, CA, USA (May 2005)Google Scholar
  10. 10.
    Cohen, F.B.: A Short Course on Computer Viruses. Wiley Professional Computing, Chichester (1994)MATHGoogle Scholar
  11. 11.
    Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. Technical Report 148, Department of Computer Science, University of Auckland (July 1997)Google Scholar
  12. 12.
    Debray, S.K., Evans, W., Muth, R., Sutter, B.D.: Compiler techniques for code compaction. ACM Trans. Program. Lang. Syst. 22(2), 378–415 (2000)CrossRefGoogle Scholar
  13. 13.
    Ferrie, P., Ször, P.: Zmist opportunities. Virus Bullettin (March 2001)Google Scholar
  14. 14.
    Foggia, P.: The VFLib graph matching library, version 2.0, http://amalfi.dis.unina.it/graph/db/vflib-2.0/
  15. 15.
    Kapoor, A.: An approach towards disassembly of malicious binaries. Master’s thesis, University of Louisiana at Lafayette (2004)Google Scholar
  16. 16.
    Krügel, C., Kirda, E., Mutz, D., Robertson, W., Vigna, G.: Polymorphic worm detection using structural information of executables. In: Valdes, A., Zamboni, D. (eds.) RAID 2005. LNCS, vol. 3858, pp. 207–226. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Kruegel, C., Robertson, W., Valeur, F., Vigna, G.: Static disassembly of obfuscated binaries. In: Proceedings of USENIX Security 2004, San Diego, CA, pp. 255–270 (August 2004)Google Scholar
  18. 18.
    Lakhotia, A., Kumar, E.U., Venable, M.: A method for detecting obfuscated calls in malicious binaries. IEEE Transactions on Software Engineering 31(11), 955–968 (2005)CrossRefGoogle Scholar
  19. 19.
    Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: CCS 2003: Proceedings of the 10th ACM conference on Computer and communications security, pp. 290–299. ACM Press, New York (2003)CrossRefGoogle Scholar
  20. 20.
    Muchnick, S.S.: Advanced compiler design and implementation. Morgan Kaufmann Publishers Inc., San Francisco (1997)Google Scholar
  21. 21.
    Newsome, J., Karp, B., Song, D.X.: Polygraph: Automatically generating signatures for polymorphic worms. In: IEEE Symposium on Security and Privacy, pp. 226–241 (2005)Google Scholar
  22. 22.
    Pearce, S.: Viral polymorphism. Sans Institute (2003)Google Scholar
  23. 23.
    Ször, P., Ferrie, P.: Hunting for metamorphic. In: Proceedings of Virus Bulletin Conference (September 2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Danilo Bruschi
    • 1
  • Lorenzo Martignoni
    • 1
  • Mattia Monga
    • 1
  1. 1.Dip. Informatica e ComunicazioneUniversità degli Studi di MilanoMilanItaly

Personalised recommendations