Computational Intelligent Techniques and Similarity Measures for Malware Classification

Abstract

One of the major problems concerning information security is malicious code. To evade detection, malware (unwanted malicious piece of code) is packed, encrypted, and obfuscated to produce variants that continue to plague properly defended and patched systems and networks with zero day exploits. Zero day exploits are used by the attackers to compromise victims computer before the developer of the target software knows about the vulnerability.

In this chapter we present a method of functionally classifying malicious code that might lead to automated attacks and intrusions using computational intelligent techniques and similarity measures. We study the performance of kernel methods in the context of robustness and generalization capabilities of malware classification.

Results from our recent experiments indicate that similarity measures can be utilized to determine the likelihood that a piece of code or binary under inspection contains a particular malware. Malware variants of a particular malware family show very high similarity scores (over 85%). Interestingly Trojans and hacking tools have high similarity scores with other Trojans and hacking tools.

Our results also show that malware analysis based on the API calling sequence and API frequency that reflects the behavior of a particular piece of code gives good accuracy to classify malware. We also show that classification accuracy varies with the kernel type and the parameter values; thus, with appropriately chosen parameter values, malware can be detected by support vector machines (SVM) with higher accuracy and lower rates of false alarms.