Skip to main content

Automatically Identifying Trigger-based Behavior in Malware

  • Chapter
Botnet Detection

Part of the book series: Advances in Information Security ((ADIS,volume 36))

Summary

Malware often contains hidden behavior which is only activated when properly triggered. Well known examples include: the MyDoom worm which DDoS’s on particular dates, keyloggers which only log keystrokes for particular sites, and DDoS zombies which are only activated when given the proper command. We call such behavior trigger-based behavior

Currently, trigger-based behavior analysis is often performed in a tedious, manual fashion. Providing even a small amount of assistance would greatly assist and speed-up the analysis. In this chapter, we propose that automatic analysis of trigger-based behavior in malware is possible. In particular, we design an approach for automatic trigger-based behavior detection and analysis using dynamic binary instrumentation and mixed concrete and symbolic execution. Our approach shows that in many cases we can:

(1) detect the existence of trigger-based behavior, (2) find the conditions that trigger such hidden behavior, and (3) find inputs that satisfy those conditions, allowing us to observe the triggered malicious behavior in a controlled environment. We have implemented MineSweeper, a system utilizing this approach. In our experiments, MineSweeper has successfully identified trigger-based behavior in real-world malware. Although there are many challenges presented by automatic trigger-based behavior detection, MineSweeper shows us that such automatic analysis is possible and encourages future work in this area

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Blazingtools perfect keylogger. http://www.blazingtools.com/bpk.html.

    Google Scholar 

  2. QEMU. http://www.qemu.org.

    Google Scholar 

  3. Tribal flood network. http://www.cert.org/incident_notes/IN-99-07.html.

    Google Scholar 

  4. David Brumley, Cody Hartwig, Min Gyang Kang, Zhenkai Liang, James Newsome, Pongsin Poosankam, Dawn Song, and Heng Yin. Automatically dissecting malicious binaries. Technical Report CMU-CS-07-133, 2007.

    Google Scholar 

  5. David Brumley and James Newsome. Alias analysis for assembly. Technical Report CMU-CS-06-180, Carnegie Mellon University School of Computer Science, 2006.

    Google Scholar 

  6. Cristian Cadar, Vijay Ganesh, Peter Pawlowski, David Dill, and Dawson Engler. EXE: A system for automatically generating inputs of death using symbolic execution. In Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS), October 2006.

    Google Scholar 

  7. Edmund Clarke, Daniel Kroening, and Flavio Lerda. A tool for checking ANSI-C programs. In Kurt Jensen and Andreas Podelski, editors, Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2004), volume 2988 of Lecture Notes in Computer Science, pages 168–176. Springer, 2004.

    Google Scholar 

  8. Jedidiah R. Crandall, Gary Wassermann, Daniela A. S. de Oliveira, Zhendong Su, S. Felix Wu, and Frederic T. Chong. Temporal search: Detecting hidden malware timebombs with virtual machines. In Proceedings of the Twelfth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XII), October 2006.

    Google Scholar 

  9. Tony LeePeter Ferrie. Win32.Netsky.C. http://www.symantec.com/ security_response/writeup.jsp?docid=2004-022417%-4628-99.

    Google Scholar 

  10. C. Flanagan and J.B. Saxe. Avoiding exponential explosion: Generating compact verification conditions. In Proceedings of the 28th ACM Symposium on the Principles of Programming Languages (POPL), 2001.

    Google Scholar 

  11. Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, and Raymie Stata. Estended static checking for java. In ACM Conference on the Programming Language Design and Implementation (PLDI), 2002.

    Google Scholar 

  12. Vijay Ganesh and David Dill. STP: A decision procedure for bitvectors and arrays. http://theory.stanford.edu/~vganesh/stp.html.

    Google Scholar 

  13. Scott Gettis. W32.Mydoom.B@mm. http://www.symantec.com/security_ response/writeup.jsp?docid=2004-022011%-2447-99.

    Google Scholar 

  14. Patrice Godefroid, Nils Klarlund, and Koushik Sen. DART: Directed automated random testing. In Proc. of the 2005 Programming Language Design and Implementation Conference (PLDI), 2005.

    Google Scholar 

  15. Kevin Ha. Keylogger.Stawin. http://www.symantec.com/security_ response/writeup.jsp?docid=2004-012915%-2315-99.

    Google Scholar 

  16. Neal Hindocha. Win32.Netsky.D. http://www.symantec.com/security_ response/writeup.jsp?docid=2004-030110%-0232-99.

    Google Scholar 

  17. James King. Symbolic execution and program testing. Communications of the ACM, 19:386–394, 1976.

    Google Scholar 

  18. McAfee. W97M/Opey.C. ttp://vil.nai.com/vil/content/v_10290.htm.

    Google Scholar 

  19. Andreas Moser, Christopher Kruegel, and Engin Kirda. Exploring multiple execution paths for malware analysis. In IEEE Symposium on Security and Privacy. IEEE Press, 2007.

    Google Scholar 

  20. James Newsome, David Brumley, Jason Franklin, and Dawn Song. Replayer: Automatic protocol replay by binary analysis. In Proceedings of the13$th$ACM Conference on Computer and and Communications Security (CCS), October 2006.

    Google Scholar 

  21. Benjamin C Pierce. Types and Programming Languages. The MIT Press, 2002.

    Google Scholar 

  22. Koushik Sen, Darko Marinov, and Gul Agha. CUTE: A concolic unit testing engine for c. In ACM SIGSOFT Sympsoium on the Foundations of Software Engineering, 2005.

    Google Scholar 

  23. Symantec. Spyware.e2give. http://www.symantec.com/security response/ writeup.jsp?docid=2004-102614-1006-99.

    Google Scholar 

  24. Symantec. Xeram.1664. http://www.symantec.com/security_response/ writeup.jsp?docid=2000-121913-2839-99.

    Google Scholar 

  25. United States Department of Justice Press Release. Former computer network administrator at new jersey high-tech firm sentenced to 41 months for unleashing $10 million computer “time bomb”. http://www.usdoj.gov/criminal/cybercrime/lloydSent.htm.

    Google Scholar 

  26. United States Department of Justice Press Release. Former lance, inc. employee sentenced to 24 months and ordered to pay $194,609 restitution in computer fraud case. http://www.usdoj.gov/criminal/cybercrime/SullivanSent.htm.

    Google Scholar 

  27. United States Department of Justice Press Release. Former technology manager sentenced to a year in prison for computer hacking offense. http://www.usdoj.gov/criminal/cybercrime/sheaSent.htm.

    Google Scholar 

  28. Yichen Xie and Alex Aiken. Context-and path-sensitive memory leak detection. ACM SIGSOFT Software Engineering Notes, 30, 2005.

    Google Scholar 

  29. Junfeng Yang, Can Sar, Paul Twohey, Cristian Cadar, and Dawson Engler. Automatically generating malicious disks using symbolic execution. In IEEE Symposium on Security and Privacy, 2006.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., Yin, H. (2008). Automatically Identifying Trigger-based Behavior in Malware. In: Lee, W., Wang, C., Dagon, D. (eds) Botnet Detection. Advances in Information Security, vol 36. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-68768-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-68768-1_4

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-68766-7

  • Online ISBN: 978-0-387-68768-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics