Abstract
Present malicious threats have been consolidated in past few years by incorporating diverse stealthy techniques. Detecting these malwares on the basis of their dynamic behavior has become a potential approach as it suppresses the shortcomings of static approaches raised due to the obfuscated malware binaries. Additionally, existing behavior based malware detection approaches are resilient to zero–day malware attacks. These approaches rely on isolated analysis environment to monitor and capture the run–time malware behavior. Malware bundled with environment–aware payload may degrade detection accuracy of such approaches. These malicious programs detect the presence of execution environment and thus inspite of having their malicious payload they mimic a benign behavior to avoid detection. In this paper, we have presented an approach using system–calls to identify a malware on the basis of their malignant and environment–reactive behavior. The proposed approach offers an automated screening mechanism to segregate malware samples on the basis of aforementioned behaviors. We have built a decision model which is based on multi–layer perceptron learning with back propagation algorithm. Our proposed model decides the candidacy of a sample to be put into one of the four classes (clean, malignant, guest–crashing and infinite–running). Clean behavior denotes benign sample and rest of the behaviors denote the presence of malware sample. The proposed technique has been evaluated with known and unknown instances of real malware and benign programs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anderson, B., Quist, D., Neil, J., Storlie, C., Lane, T.: Graph-based malware detection using dynamic analysis. J. Comput. Virol. 7(4), 247–258 (2011)
Balzarotti, D., Cova, M., Karlberger, C., Kirda, E., Kruegel, C., Vigna, G.: Efficient detection of split personalities in malware. In: Proceedings of the Network and Distributed System Security Symposium, NDSS, San Diego, California, USA, pp. 1–16 (2010)
Bethencourt, J., Song, D., Waters, B.: Analysis-resistant malware. In: Proceedings of the Network and Distributed System Security Symposium, NDSS, San Diego, California, USA, pp. 1–13 (2008)
Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., Yin, H.: Automatically identifying trigger-based behavior in malware. In: Lee, W., Wang, C., Dagon, D. (eds.) Botnet Detection. Advances in Information Security, vol. 36, pp. 65–88. Springer, New York (2008)
Chen, X., Andersen, J., Mao, Z., Bailey, M., Nazario, J.: Towards an understanding of anti-virtualization and anti-debugging behavior in modern malware. In: Dependable Systems and Networks With FTCS and DCC, DSN, pp. 177–186, June 2008
Chester, D.L.: Why two hidden layers are better than one. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 1990, Washington, DC (1990)
Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: Malware analysis via hardware virtualization extensions. In: Proceedings of the 15th ACM Conference on Computer and Communications Security, CCS 2008, pp. 51–62. ACM, New York (2008)
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)
J00ru: Windows win32k.sys system call table, April 2014
Jacob, G., Hund, R., Kruegel, C., Holz, T.: Jackstraws: picking command and control connections from bot traffic. In: Proceedings of the 20th USENIX Conference on Security, SEC 2011, pp. 29–48. USENIX Association, Berkeley (2011)
Kang, M.G., Yin, H., Hanna, S., McCamant, S., Song, D.: Emulating emulation-resistant malware. In: Proceedings of the 1st ACM Workshop on Virtual Machine Security, VMSec 2009. ACM, New York, pp. 11–22 (2009)
Kevin, L., Bryce, D., David, G., Volker, R., Christophe, B.: Bochs user manual (2010)
Kolbitsch, C., Comparetti, P.M., Kruegel, C., Kirda, E., Zhou, X., Wang, X.: Effective and efficient malware detection at the end host. In: Proceedings of the 18th Conference on USENIX Security Symposium, SSYM 2009, pp. 351–366. USENIX Association (2009)
Lindorfer, M., Kolbitsch, C., Milani Comparetti, P.: Detecting environment-sensitive malware. In: Sommer, R., Balzarotti, D., Maier, G. (eds.) RAID 2011. LNCS, vol. 6961, pp. 338–357. Springer, Heidelberg (2011)
Mark, R., David A, s., Alex, L.: Windows internal part 2
Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: Proceedings of the 2007 IEEE Symposium on Security and Privacy, SP 2007, Washington, DC, pp. 231–245 (2007)
Ou, G., Murphey, Y.L.: Multi-class pattern classification using neural networks. Pattern Recogn. 40(1), 4–18 (2007)
Park, Y., Reeves, D.S., Stamp, M.: Deriving common malware behavior through graph clustering. Comput. Secur. 39, 419–430 (2013)
Pék, G., Bencsáth, B., Buttyán, L.: nEther: in-guest detection of out-of-the-guest malware analyzers. In: Proceedings of the Fourth European Workshop on System Security, EUROSEC 2011, pp. 3:1–3:6. ACM, New York (2011)
Quist, D., Liebrock, L., Neil, J.: Improving antivirus accuracy with hypervisor assisted analysis. J. Comput. Virol. 7(2), 121–131 (2011)
Rieck, K., Holz, T., Willems, C., Düssel, P., Laskov, P.: Learning and classification of malware behavior. In: Zamboni, D. (ed.) DIMVA 2008. LNCS, vol. 5137, pp. 108–125. Springer, Heidelberg (2008)
Rutkowska, J.: Red pill... or how to detect vmm using (almost) one cpu instruction
Srivastava, A., Lanzi, A., Giffin, J.T.: System call API obfuscation (extended abstract). In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 421–422. Springer, Heidelberg (2008)
Sun, M.K., Lin, M.J., Chang, M., Laih, C.S., Lin, H.T.: Malware virtualization-resistant behavior detection. In: Proceedings of the 17th International Conference on Parallel and Distributed Systems (IEEE), ICPADS 2011, Washington, DC, USA, pp. 912–917 (2011)
Vinod, P., Laxmi, V., Gaur, M.S.: REFORM: relevant feature for malware analysis. In: Proceedings of Sixth IEEE International Conference of Security and Multimodality in Pervasive Environment (SMPE 2012), pp. 26–29. Fukuoka Institute of technology (FIT), Fukuoka, Japan (2012)
Acknowledgments
Mauro Conti is supported by a Marie Curie Fellowship funded by the European Commission under the agreement No. PCIG11-GA-2012-321980. This work is also partially supported by the TENACE PRIN Project 20103P34XC funded by the Italian MIUR, and by the Project “Tackling Mobile Malware with Innovative Machine Learning Techniques” funded by the University of Padua.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Naval, S., Laxmi, V., Gaur, M.S., Raja, S., Rajarajan, M., Conti, M. (2015). Environment–Reactive Malware Behavior: Detection and Categorization. In: Garcia-Alfaro, J., et al. Data Privacy Management, Autonomous Spontaneous Security, and Security Assurance. DPM QASA SETOP 2014 2014 2014. Lecture Notes in Computer Science(), vol 8872. Springer, Cham. https://doi.org/10.1007/978-3-319-17016-9_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-17016-9_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-17015-2
Online ISBN: 978-3-319-17016-9
eBook Packages: Computer ScienceComputer Science (R0)