An Improved Method to Unveil Malware’s Hidden Behavior

  • Qiang Li
  • Yunan Zhang
  • Liya Su
  • Yang Wu
  • Xinjian Ma
  • Zeming Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10726)

Abstract

Sandbox technique is widely used in automated malware analysis. However, it can only see one path during its analysis. This is fatal when meeting the targeted malware. The challenge is how to unleash the hidden behaviors of targeted malware. Many works have been done to mitigate this problem. However, these solutions either use limited and fixed sandbox environments or introduce time and space consuming multi-path exploration. To address this problem, this paper proposes a new hybrid dynamic analysis scheme by applying function summary based symbolic execution of malware. Specifically, by providing Windows APIs’ summary stub and using unicorn CPU emulator, we can effectively extract malware’s hidden behavior which are not shown in sandbox environment. Without the usage of full system emulation, our approach achieve much higher speed than existing schemes. We have implemented a prototype system, and evaluated it with typical real-world malware samples. The experiment results show that our system can effectively and efficiently extract malware’s hidden behavior.

Keywords

Dynamic malware analysis Function summary Symbolic execution 

Notes

Acknowledgments

This work was partially supported by The National Key Research and Development Program of China (2016YFB0801004 and 2016YFB0801604).

References

  1. 1.
    Balzarotti, D., Cova, M., Karlberger, C., Kruegel, C., Kirda, E., Vigna, G.: Efficient detection of split personalities in malware. In: NDSS 2010, 17th Annual Network and Distributed System Security Symposium, February 2010Google Scholar
  2. 2.
    Bayer, U., Moser, A., Kruegel, C., Kirda, E.: Dynamic analysis of malicious code. J. Comput. Virol. 2(1), 67–77 (2006)CrossRefGoogle Scholar
  3. 3.
    Bilge, L., Dumitras, T.: Before we knew it: an empirical study of zero-day attacks in the real world. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012, pp. 833–844. ACM, New York (2012)Google Scholar
  4. 4.
    Brumley, D., Hartwig, C., Kang, M.G., Liang, Z., Newsome, J., Poosankam, P., Song, D., Yin, H.: Bitscope: automatically dissecting malicious binaries. Technical report, In CMU-CS-07-133 (2007)Google Scholar
  5. 5.
    Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., Yin, H.: Automatically identifying trigger-based behavior in malware. In: Lee, W., Wang, C., Dagon, D. (eds.) Botnet Detection. Advances in Information Security, vol. 36. Springer, Boston (2008).  https://doi.org/10.1007/978-0-387-68768-1_4
  6. 6.
    Cadar, C., Dunbar, D., Engler, D.R.: Klee: unassisted and automatic generation of high-coverage tests for complex systems programs. In: OSDI, vol. 8, pp. 209–224 (2008)Google Scholar
  7. 7.
    Cha, S.K., Avgerinos, T., Rebert, A., Brumley, D.: Unleashing mayhem on binary code. In: Proceedings of the 2012 IEEE Symposium on Security and Privacy, SP 2012, pp. 380–394. IEEE Computer Society, Washington, DC (2012)Google Scholar
  8. 8.
    Chen, X., Andersen, J., Mao, Z.M., Bailey, M., Nazario, J.: Towards an understanding of anti-virtualization and anti-debugging behavior in modern malware. In: 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN), pp. 177–186, June 2008Google Scholar
  9. 9.
    Comparetti, P.M., Salvaneschi, G., Kirda, E., Kolbitsch, C., Kruegel, C., Zanero, S.: Identifying dormant functionality in malware programs. In: 2010 IEEE Symposium on Security and Privacy (SP), pp. 61–76. IEEE (2010)Google Scholar
  10. 10.
    Cuckoo: Automated malware analysis - cuckoo sandbox (2016). http://www.cuckoosandbox.org/
  11. 11.
    Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: malware analysis via hardware virtualization extensions. In: CCS 2008, pp. 51–62. ACM (2008)Google Scholar
  12. 12.
  13. 13.
    Fleck, D., Tokhtabayev, A., Alarif, A., Stavrou, A., Nykodym, T.: Pytrigger: a system to trigger & extract user-activated malware behavior. In: 2013 Eighth International Conference on Availability, Reliability and Security (ARES), pp. 92–101. IEEE (2013)Google Scholar
  14. 14.
    GeorgiaTech: Open malware (2016). http://www.offensivecomputing.net/
  15. 15.
  16. 16.
    Godefroid, P.: Compositional dynamic test generation. In: Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2007, pp. 47–54. ACM, New York (2007)Google Scholar
  17. 17.
    Google: Virustotal (2016). https://www.virustotal.com/
  18. 18.
    Graziano, M., Leita, C., Balzarotti, D.: Towards network containment in malware analysis systems. In: Proceedings of the 28th Annual Computer Security Applications Conference, ACSAC 2012, pp. 339–348. ACM, New York (2012)Google Scholar
  19. 19.
  20. 20.
  21. 21.
    Kirat, D., Vigna, G., Kruegel, C.: Barecloud: bare-metal analysis-based evasive malware detection. In: Proceedings of the 23rd USENIX conference on Security Symposium (SEC 2014), pp. 287–301. USENIX Association, Berkeley (2014)Google Scholar
  22. 22.
    Kolbitsch, C., Comparetti, P.M., Kruegel, C., Kirda, E., Zhou, X., Wang, X.: Effective and efficient malware detection at the end host. In: Proceedings of the 18th Conference on USENIX Security Symposium, SSYM 2009, pp. 351–366. USENIX Association, Berkeley (2009)Google Scholar
  23. 23.
    Kolbitsch, C., Kirda, E., Kruegel, C.: The power of procrastination: detection and mitigation of execution-stalling malicious code (2011)Google Scholar
  24. 24.
    Kolbitsch, C., Livshits, B., Zorn, B., Seifert, C.: Rozzle: de-cloaking internet malware. In: Proceedings of the 2012 IEEE Symposium on Security and Privacy, SP 212, pp. 443–457. IEEE Computer Society, Washington, DC (2012)Google Scholar
  25. 25.
    Lindorfer, M., Kolbitsch, C., Milani Comparetti, P.: Detecting environment-sensitive malware. In: Sommer, R., Balzarotti, D., Maier, G. (eds.) RAID 2011. LNCS, vol. 6961, pp. 338–357. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-23644-0_18 CrossRefGoogle Scholar
  26. 26.
    Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: IEEE Symposium on Security and Privacy, SP 2007, pp. 231–245 (2007)Google Scholar
  27. 27.
    Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: Twenty-Third Annual Computer Security Applications Conference, ACSAC 2007, pp. 421–430 (2007)Google Scholar
  28. 28.
    Nappa, A., Xu, Z., Rafique, M.Z., Caballero, J., Gu, G.: Cyberprobe: towards internet-scale active detection of malicious servers. In: Proceedings of the 2014 Network and Distributed System Security Symposium (NDSS 2014), pp. 1–15 (2014)Google Scholar
  29. 29.
  30. 30.
    Peng, F., Deng, Z., Zhang, X., Xu, D., Lin, Z., Su, Z.: X-force: force-executing binary programs for security applications. In: Proceedings of the 23rd USENIX Conference on Security Symposium, SEC 2014, pp. 829–844. USENIX Association, Berkeley (2014)Google Scholar
  31. 31.
    Porras, P., Saïdi, H., Yegneswaran, V.: A foray into conficker’s logic and rendezvous points. In: Proceedings of the 2nd USENIX Conference on Large-scale Exploits and Emergent Threats: Botnets, Spyware, Worms, and More, LEET 2009, p. 7. USENIX Association, Berkeley (2009)Google Scholar
  32. 32.
    Shin, S., Xu, Z., Gu, G.: Effort: efficient and effective bot malware detection. In: 2012 Proceedings IEEE INFOCOM, pp. 2846–2850, March 2012Google Scholar
  33. 33.
    Song, C., Royal, P., Lee, W.: Impeding automated malware analysis with environmentsensitive malware. In: USENIX Workshop on Hot Topics in Security (2012)Google Scholar
  34. 34.
  35. 35.
  36. 36.
  37. 37.
    Symantec: Symantec intelligence quarterly (2016). http://www.symantec.com/threatreport/quarterly.jsp
  38. 38.
    Symantec: Triage analysis of targeted attacks (2016). http://www.symantec.com/threatreport/topic.jsp?id=malicious_code_trend
  39. 39.
  40. 40.
  41. 41.
    UCSB: Angr (2016). https://github.com/angr/angr
  42. 42.
    Unicorn: The ultimate CPU emulator (2016). http://www.unicorn-engine.org/
  43. 43.
  44. 44.
    Wikipedia: Stuxnet (2016). http://en.wikipedia.org/wiki/Stuxnet
  45. 45.
    Wikipedia: Trojan backdoor.flashback (2016). http://en.wikipedia.org/wiki/Trojan_BackDoor.Flashback
  46. 46.
    Wilhelm, J., Chiueh, T.C.: A forced sampled execution approach to kernel rootkit identification (2007)Google Scholar
  47. 47.
    Willems, C., Holz, T., Freiling, F.: Toward automated dynamic malware analysis using cwsandbox. IEEE Secur. Privacy 5(2), 32–39 (2007)CrossRefGoogle Scholar
  48. 48.
    Xu, Z., Zhang, J., Gu, G., Lin, Z.: Autovac: automatically extracting system resource constraints and generating vaccines for malware immunization. In: 2013 IEEE 33rd International Conference on Distributed Computing Systems (ICDCS), pp. 112–123, July 2013Google Scholar
  49. 49.
    Xu, Z., Chen, L., Gu, G., Kruegel, C.: Peerpress: utilizing enemies’ P2P strength against them. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 212, pp. 581–592. ACM, New York (2012)Google Scholar
  50. 50.
    Xu, Z., Zhang, J., Gu, G., Lin, Z.: GoldenEye: efficiently and effectively unveiling malware’s targeted environment. In: Stavrou, A., Bos, H., Portokalidis, G. (eds.) RAID 2014. LNCS, vol. 8688, pp. 22–45. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-11379-1_2 Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Qiang Li
    • 1
    • 2
  • Yunan Zhang
    • 1
    • 2
  • Liya Su
    • 1
    • 2
  • Yang Wu
    • 1
  • Xinjian Ma
    • 1
    • 2
  • Zeming Yang
    • 1
    • 2
  1. 1.Institute of Information Engineering, Chinese Academy of SciencesBeijingChina
  2. 2.School of Cyber SecurityUniversity of Chinese Academy of SciencesBeijingChina

Personalised recommendations