Advertisement

A Layered Architecture for Detecting Malicious Behaviors

  • Lorenzo Martignoni
  • Elizabeth Stinson
  • Matt Fredrikson
  • Somesh Jha
  • John C. Mitchell
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5230)

Abstract

We address the semantic gap problem in behavioral monitoring by using hierarchical behavior graphs to infer high-level behaviors from myriad low-level events. Our experimental system traces the execution of a process, performing data-flow analysis to identify meaningful actions such as “proxying”, “keystroke logging”, “data leaking”, and “downloading and executing a program” from complex combinations of rudimentary system calls. To preemptively address evasive malware behavior, our specifications are carefully crafted to detect alternative sequences of events that achieve the same high-level goal. We tested eleven benign programs, variants from seven malicious bot families, four trojans, and three mass-mailing worms and found that we were able to thoroughly identify high-level behaviors across this diverse code base. Moreover, we effectively distinguished malicious execution of high-level behaviors from benign by identifying remotely-initiated actions.

Keywords

Dynamic Semantic Gap Malware Behavior Data-Flow 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Symantec Internet Security Threat Report, Trends for January-June 07, Volume XII (September 2007)Google Scholar
  2. 2.
    Keizer, G.: Bot Networks Behind Big Boos. In: Phishing Attacks. TechWeb (November 2004)Google Scholar
  3. 3.
    Parizo, E.: New bots, worm threaten AIM network. SearchSecurity (December 2005)Google Scholar
  4. 4.
    Naraine, R.: Money Bots: Hackers Cas. In on Hijacked PCs. eWeek (September 2006)Google Scholar
  5. 5.
    Overton, M.: Bots and Botnets: Risks, Issues, and Prevention. In: Virus Bulletin Conference (October 2005)Google Scholar
  6. 6.
    Ianelli, N., Hackworth, A.: Botnets as a Vehicle for Online Crime. CERT Coordination Center (December 2005)Google Scholar
  7. 7.
    Ilett, D.: Most spam generated by botnets, says expert. ZDNet UK (September 22, 2004)Google Scholar
  8. 8.
    Christodorescu, M., Jha, S.: Testing Malware Detectors. In: Proc. of the International Symposium on Software Testing and Analysis (July 2004)Google Scholar
  9. 9.
    SRI Honeynet and BotHunter Malware Analysis Automatic Summary AnalysisGoogle Scholar
  10. 10.
    Jevans, D.: The Latest Trends in Phishing, Crimeware and Cash-Out Schemes. Private correspondenceGoogle Scholar
  11. 11.
    Christodorescu, M., Jha, S., Kruegel, C.: Mining specifications of malicious behavior. In: Proc. of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (August 2007)Google Scholar
  12. 12.
    NoAH Foundation: Containment Environment DesignGoogle Scholar
  13. 13.
    Chen, P., Noble, B.: When Virtual is Better than Real. In: Proceedings of HotOS-VIII: 8th Workshop on Hot Topics in Operating SystemsGoogle Scholar
  14. 14.
    Petritsch, H.: Understanding and Replaying Network Traffic in Windows XP for Dynamic Malware Analysis. Master’s Thesis (February 2007)Google Scholar
  15. 15.
    Christodorescu, M., Jha, S., Seshia, S., Song, D., Bryant, R.: Semantics-Aware Malware Detection. In: IEEE Symposium on Security and Privacy (May 2005)Google Scholar
  16. 16.
    Stinson, E., Mitchell, J.: Characterizing Bots’ Remote Control Behavior. In: Proc. of the 4th DIMVA Conference (July 2007)Google Scholar
  17. 17.
    Newsome, J., Song, D.: Dynamic Taint Analysis for Automatic Detection, Analysis, and Signature Generation of Exploits on Commodity Software. In: Network and Distributed Systems Symposium (February 2005)Google Scholar
  18. 18.
    Yin, H., Song, D., Egele, M., Kruegel, C., Kirda, E.: Panorama: capturing system-wide information flow for malware detection and analysis. In: Proc. of the 14th ACM conference on Computer and communications security (October 2007)Google Scholar
  19. 19.
    Cui, W., Katz, R., Tan, W.: BINDER: An Extrusion-based Break-in Detector for Personal Computers. In: Proc. of the 21st Annual Computer Security Applications Conference (December 2005)Google Scholar
  20. 20.
    Kirda, E., Kruegel, C., Banks, G., Vigna, G., Kemmerer, R.: Behavior-based Spyware Detection. In: Proc. of the 15th USENIX Security Symposium (August 2006)Google Scholar
  21. 21.
    United States Patent Application 20070067843 M̈ethod and apparatus for removing harmful software: Williamson, Matthew; Gorelik, Vladimir (March 22, 2007)Google Scholar
  22. 22.
    Strider GhostBuster Rootkit DetectionGoogle Scholar
  23. 23.
    Wang, Y., Beck, D., Vo, B., Roussev, R., Verbowski, C.: Detecting Stealth Software with Strider GhostBuster. Microsoft Technical Report MSR-TR-2005-25Google Scholar
  24. 24.
    Garfinkel, T.: Traps and Pitfalls: Practical Problems in System Call Interposition Based Security Tools. In: Network and Distributed System Security (Feburary 2003)Google Scholar
  25. 25.
    Garfinkel, T., Rosenblum, M.: A Virtual Machine Introspection Based Architecture for Intrusion Detection. In: Network and Distributed Systems Symp. (Feburary 2003)Google Scholar
  26. 26.
    Nilsson, N.: Problem-Solving Methods in Artificial Intelligence. McGraw-Hill, New York (1971)Google Scholar
  27. 27.
    Bayer, U., Moser, A., Kruegel, C., Kirda, E.: Dynamic Analysis of Malicious Code. Journal in Computer Virology 2(1) (August 2006)Google Scholar
  28. 28.
    Jiang, X., Xu, D., Wang, X.: Stealthy Malware Detection Through VMM-Based ”Out-of-the-Box” Semantic View Reconstruction. In: Proceedings of the 14th ACM Conference on Computer and Communications Security (CCS 2007), Alexandria, VA (November 2007)Google Scholar
  29. 29.
    Jiang, X., Wang, X.: ’Out-of-the-box’ Monitoring of VM-based High-Interaction Honeypots. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 198–218. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  30. 30.
    Egele, M., Kruegel, C., Kirda, E., Yin, H., Son, D.: Dynamic Spyware Analysis. In: Proceedings of Usenix Annual Technical Conference, USA (June 2007)Google Scholar
  31. 31.
    Moser, A., Kruegel, C., Kirda, E.: Exploring Multiple Execution Paths for Malware Analysis. In: Proceedings of IEEE Symposium on Security and Privacy, May 2007, IEEE Computer Society Press, USA (2007)Google Scholar
  32. 32.
    Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Poosankam, P., Song, D., Yin, H.: In: Lee, W., et al. (eds.) Botnet Analysis (2007)Google Scholar
  33. 33.
    Norman SandboxGoogle Scholar
  34. 34.
    Willems, C.: Automatic Behaviour Analysis of Malware. Master Thesis. University of MannheimGoogle Scholar
  35. 35.
    Jones, S.: Implicit Operating System Awareness in a Virtual Machine Monitor. Ph.D. Thesis, University of Wisconsin - Madison (April 2007)Google Scholar
  36. 36.
    Jones, S., Arpaci-Dusseau, A., Arpaci-Dusseau, R.: VMM-based Hidden Process Detection and Identification using Lycosid. In: ACM International Conference on Virtual Execution Environments (March 2008)Google Scholar
  37. 37.
    Vasudevan, A., Yerraballi, R.: Cobra: Fine-grained Malware Analysis using Stealth Localized-executions. In: Proceedings of IEEE Symposium on Security and Privacy, May 2006, IEEE Computer Society Press, USA (2006)Google Scholar
  38. 38.
    Bellard, F.: QEMU Accelerator (KQEMU)Google Scholar
  39. 39.
    Bellard, F.: QEMU, a Fast and Portable Dynamic TranslatorGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Lorenzo Martignoni
    • 1
  • Elizabeth Stinson
    • 2
  • Matt Fredrikson
    • 3
  • Somesh Jha
    • 3
  • John C. Mitchell
    • 2
  1. 1.Università degli Studi di Milano 
  2. 2.Stanford University 
  3. 3.University of Wisconsin 

Personalised recommendations