Skip to main content
SpringerLink
Log in

Forced continuation of malware execution beyond exceptions

  • Original Paper
  • Published:
Journal of Computer Virology and Hacking Techniques Aims and scope Submit manuscript

Abstract

A malware program often terminates on its own owing to exceptions. Some of these exceptions occur only under certain execution conditions. To understand the potential threat posed by the malware, analysts need to collect information on its behavior observed when these exceptions do not occur. However, most analysis systems do not provide a mechanism to capture this, which makes exception-raising malware analysis an extremely challenging task. In this paper, we propose a method for dynamic analysis of malware programs that raise exceptions. This method intercepts exceptions and “nullifies” so that the malware behaves as if the exceptions did not occur in the first place. This is achieved by modifying the memory and registers of malware at the time of exceptions and flexibly controlling the delivery of intercepted exceptions depending on the exception type and program state. Analysts using this method can continue malware execution beyond critical exceptions and when exceptions do not occur. We developed a sandbox system by extending Cuckoo Sandbox using the proposed method and compared the execution results of 2592 malware samples between the original and extended sandboxes. The results of the experiments indicated that our system increased the number of invoked API calls in 37.8% of the samples, and the number of accessed resources was 32.0%. We believe that our system provides key insights into malware execution, which will help analysts better understand the behavior of malware that was once unobservable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://cuckoosandbox.org/.

References

  1. Alsaleh, M.N., Wei, J., Al-Shaer, E., et al.: gExtractor: Towards automated extraction of malware deception parameters. In: Proceedings of the 8th Software Security, Protection, and Reverse Engineering Workshop (2018)

  2. Avllazagaj, E., Zhu, Z., Bilge, L., et al.: When malware changed its mind: an empirical study of variable program behaviors in the real world. In: Proceedings of the 30th USENIX Security Symposium, pp. 3487–3504 (2021)

  3. Branco, R.R., Barbosa, G.N., Neto, P.D.: Scientific but not academical overview of malware anti-debugging, anti-disassembly and anti-VM technologies. Black Hat USA 2012 (2012)

  4. Bremer, J.: Cuckoo sandbox and its recent developments. SECURE 2014 (2014)

  5. Chailytko, A., Skuratovich, S.: Defeating sandbox evasion: How to increase successful emulation rate in your virtualized environment. In: Proceedings of the 26th Virus Bulletin Conference (2016)

  6. Chen, P., Huygens, C., Desmet, L., et al.: Advanced or not? A comparative study of the use of anti-debugging and anti-VM techniques in generic and targeted malware. In: Proceedings of the 31st IFIP International Conference on ICT Systems Security and Privacy Protection, pp 323–336 (2016)

  7. Cheng, B., Ming, J., Fu, J., et al.: Towards paving the way for large-scale Windows malware analysis: Generic binary unpacking with orders-of-magnitude performance boost. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp 395–411 (2018)

  8. Choi, S., Chang, T., Yoon, S., et al.: Hybrid emulation for bypassing anti-reversing techniques and analyzing malware. J. Supercomput. 77, 471–497 (2021)

    Article  Google Scholar 

  9. Egele, M., Woo, M., Chapman, P., et al.: Blanket execution: dynamic similarity testing for program binaries and components. In: Proceedings of the 23rd USENIX Security Symposium, pp 303–317 (2014)

  10. Galloro, N., Polino, M., Carminati, M., et al.: A systematical and longitudinal study of evasive behaviors in windows malware. Comput. Secur. 113 (2022)

  11. Gao, X., Mechtaev, S., Roychoudhury, A.: Crash-avoiding program repair. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 8–18 (2019)

  12. Gao, Q., Zhang, W., Tang, Y., et al.: First-Aid: surviving and preventing memory management bugs during production runs. In: Proceedings of the 4th ACM European conference on Computer systems, pp. 159–172 (2009)

  13. Guarnieri, C., Schloesser, M., Bremer, J.: Mo Malware Mo Problems - Cuckoo Sandbox to the rescue. Black Hat 2013 (2013)

  14. Hatada, M., Akiyama, M., Matsuki, T., et al.: Empowering anti-malware research in Japan by sharing the MWS datasets. J. Inf. Process. 23(5), 579–588 (2015)

    Google Scholar 

  15. Igor, V. Popov GRASaumya, Debray, K.: Binary obfuscation using signals. In: Proceedings of the 16th USENIX Security Symposium, pp. 275–290 (2007)

  16. Lee, B., Kim, Y., Kim, J.: binOb+: a framework for potent and stealthy binary obfuscation. In: Proceedings of the 5th ACM Symposium on Information, Computer and Communications Security, pp. 271–281 (2010)

  17. Lin, H., Zhang, X., Yong, M., et al.: Branch obfuscation using binary code side effects. In: Proceedings of the International Conference on Computer, Networks and Communication Engineering, pp. 152–157 (2013)

  18. Long, F., Sidiroglou-Douskos, S., Rinard, M.: Automatic runtime error repair and containment via recovery shepherding. In: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 227–238 (2014)

  19. Microsoft: Debugging Tools for Windows: Specific Exceptions (2021) https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/specific-exceptions. Last accessed 5 Jan 2022

  20. Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: Proceedings of the 2007 IEEE Symposium on Security and Privacy, pp. 231–245 (2007)

  21. Rapid7 Blog: Fooling malware like a boss with Cuckoo Sandbox (2013) https://www.rapid7.com/blog/post/2013/04/16/fooling-malware-like-a-boss-with-cuckoo-sandbox/

  22. Oyama, Y.: How does malware use RDTSC? a study on operations executed by malware with CPU cycle measurement. In: Proceedings of the 16th Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 197–218 (2019)

  23. Oyama, Y.: Trends of anti-analysis operations of malwares observed in API call logs. J. Comput. Virol. Hack. Tech. 14(1), 69–85 (2018)

    Article  Google Scholar 

  24. Peng, F., Deng, Z., Zhang, X., et al.: X-Force: Force-executing binary programs for security applications. In: Proceedings of the 23rd USENIX Security Symposium, pp. 829–844 (2014)

  25. Plumerault, F., David, B.: Exploiting flaws in Windbg: how to escape or fool debuggers from existing flaws. J. Comput. Virolo. Hack. Tech. 16, 173–183 (2020)

    Article  Google Scholar 

  26. Plumerault, F., David, B.: DBI, debuggers, VM: gotta catch them all. J. Comput. Virol. Hack. Tech. 17, 105–117 (2021)

    Article  Google Scholar 

  27. Qin, F., Tucek, J., Zhou, Y., et al.: Rx: Treating bugs as allergies—a safe method to survive software failures. ACM Trans. Comput. Syst. 25(3) (2007)

  28. Rinard, M., Cadar, C., Dumitran, D., et al.: Enhancing server availability and security through failure-oblivious computing. In: Proceedings of the 6th Symposium on Operating Systems Design and Implementation, pp. 303–316 (2004)

  29. Roundy, K.A., Miller, B.P.: Binary-code obfuscations in prevalent packer tools. ACM Comput. Surv. 46(1) (2013)

  30. Tamboli, T., Austin, T.H., Stamp, M.: Metamorphic code generation from LLVM bytecode. J. Comput. Virol. Hack. Tech. 10, 177–187 (2014)

    Article  Google Scholar 

  31. Wilhelm, J., Chiueh, T.: A forced sampled execution approach to kernel rootkit identification. In: Proceedings of the 10th International Symposium on Recent Advances in Intrusion Detection, pp. 219–235 (2007)

  32. Xu, Z., Zhang, J., Gu, G., et al.: GoldenEye: efficiently and effectively unveiling malware’s targeted environment. In: Proceedings of the 17th International Symposium on Research in Attacks, Intrusions, and Defenses, pp. 22–45 (2014)

  33. Yadegari, B., Stephens, J., Debray, S.: Analysis of exception-based control transfers. In: Proceedings of the 7th ACM Conference on Data and Application Security and Privacy, pp. 205–216 (2017)

  34. Yao, X., Pang, J., Zhang, Y., et al.: A method and implementation of control flow obfuscation using SEH. In: Proceedings of the 2012 Fourth International Conference on Multimedia Information Networking and Security, pp. 336–339 (2012)

Download references

Author information

Authors and Affiliations

  1. University of Tsukuba, Tennoudai 1-1-1, Tsukuba, Ibaraki, 3058573, Japan

    Yoshihiro Oyama & Hirotaka Kokubo

  2. FUJITSU, Kamikodanaka 4-1-1, Kawasaki, Kanagawa, 2118588, Japan

    Hirotaka Kokubo

Authors
  1. Yoshihiro Oyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hirotaka Kokubo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoshihiro Oyama.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oyama, Y., Kokubo, H. Forced continuation of malware execution beyond exceptions. J Comput Virol Hack Tech 19, 483–501 (2023). https://doi.org/10.1007/s11416-022-00457-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11416-022-00457-8

Keywords

Search

Navigation