Computational intelligence anti-malware framework for android OS

1.1 Android security model

One of the most important features that distinguish the android OS, is the adoption of user identifiers (UIDs) which imparts sophisticated security capabilities, compared to the modes of traditional OS. In particular, the android applications run as separate processes with different UIDs and different permissions each. In this way, there is no application capable to read/write data or code to another, whereas if it is necessary to make data exchange with another application, it requires the assignment of specific permission. It should be noted that the android uses the mandatory access control (MAC) model in all of the processes, even in those that run with root/superuser privileges [1]. Specifically, this model is based on a security label system, which is attributed to both “subjects” (e.g. applications, users) and “objects”, which are the categories that manage information, related to different needs. This means that they can be assigned to sections of individual information within a system. Specific types of security clearance (classification labels) are assigned to the applications and to their corresponding data. In this way, the android is based on security clearances, on the classification data labels and on the system’s security policies, to reach a decision on which subject is entitled to access an object.

The classification data labels are assigned to each type of object (file, directory, device, network). Based on the security policies, the system checks the security clearances of a “subject” (e.g. user, application) by comparing them to the classification data labels of an “object” (e.g. data, files) to which access is sought. Access is not approved, if security policies are not met. The MAC model is called mandatory [1], because the classification of “subjects” and “objects” is performed automatically by the system, without the intervention of users who theoretically cannot change these rating levels.

1.2 Android rootkits

An android rootkit [2] is a set of executable scripts and configuration files, allowing the continuous access to the root/superuser privileges. It should be mentioned that they actively hide their presence from the system administrators. This is done by their incorporation into basic android OS files or other legitimate applications.

Thus, they enable secret maintenance of the system’s control by executing commands or by stealing important data (e.g. credit card numbers, passwords, banking applications) totally unnoticed. Typically, an attacker installs an android rootkit by exploiting known security loopholes (zero-day exploit, unpatched), to obtain passwords (e.g. phishing, clickjacking), or to perform direct attack on encryption (brute-force attack, hijack attack) or through close-in attack (social engineering).

The action of the rootkit, starts after the installation of the android OS, which is equivalent to simultaneous acquisition root/superuser privileges and after the installation of the necessary “Payloads”.

Thus, the user cannot revoke the full administrator rights from the malicious software, even if he uninstalls all applications that turned his phone a pawn of unknown forces, capable of an imperceptible files interception. It should be mentioned that there are cases which even require the full cancellation of the operating system.

The firmware malware [3], a special category of android rootkits, is extremely difficult to detect because the traditional virus scanners will not detect firmware threats.

Android rootkits ransomware encrypt data and then they demand money to unlock the victim’s files. If the money is not paid within the period specified by the criminals, they threaten to hold the decryption key, which is kept only on the hacker’s C & C server.

Finally, the android rootkits are mainly mobile banking malware, which have been developed with the objective of financial fraud. They are conducting illegal financial transactions and they steal money.

The memory dumps analysis method is the most serious approach of treating these threats. It performs a forced dump of the operating system’s virtual memory to identify an active rootkit. However, this technique is highly specific, it requires access to private source code, it is time consuming and it requires specialized personnel with the respective tools (digital forensic investigation tools). Moreover, it does not have the ability to detect every type of threat, as a hypervisor rootkit is able to monitor and to overturn the lower level of the system in an attempt to read the memory [3].

1.3 Tor-based botnets and Tor traffic analysis

The objective of Tor [4] is to conceal the user IDs and their activity in the network to prevent the monitoring and analysis of the traffic and to separate the detection from the routing using virtual circuits, or overlays, which change periodically.

It is the implementation of onion routing [5], in which multiple layers of encryption are employed, to ensure perfect forward secrecy between the nodes and the hidden services of Tor, while launching randomly the communication via Tor nodes (consensus) operated by volunteers worldwide. Although the Tor network is operating in the Transport layer of the OSI, the onion proxy software shows customers the secure socket interface (SOCKS) which operates in the session layer.

Also, a continuous redirection of traffic requests between the relays (entry guards, middle relays and exit relays), takes place in this network. Both the sender and recipient addresses and the information are in the form of encrypted text, so that no one at any point along the communication channel can decrypt the information or identify both ends directly [5]. The most famous types of malware are seeking communication recovery and its maintenance with the C & C remote servers on a regular basis, so that botmasters can collect or transfer information and upgrades to the compromised devices (bots). This communication is usually performed using hardcoded address or default lists address (pool addresses) controlled by the creator of the.

The mode of communication of the latest, sophisticated malware generations, lies in the creation of an encrypted communication channel, based on the chaotic architecture of Tor, to alter the traces and to distort the elements that define an attack and eventually to increase the complexity of the botnets.

Although modern programming techniques enable the malware creators to use thousands, alternating and different subnet IP address, to communicate with their C2 servers, the trace of those IPs is relatively straightforward for the network engineers, or for the responsible security analysts. Once identified, they are included in a blacklist and eventually they are blocked as spam. On the other hand, the limitation of the Tor-based botnets is extremely difficult because the movement of the Tor network resembles that of the HTTPS protocol.

1.4 Tor vs HTTPS

The Tor network not only performs encryption, but it is also designed to simulate normal HTTPS protocol traffic, which makes the identification of its channels an extremely complex and specialized process, even for experienced engineers or network analyzers. Specifically, the Tor network can use the TCP port 443, which is used by the HTTPS, so that the supervision and interpretation of a session exclusively with the determination of the door cannot constitute a reliable method.

A successful method for detecting Tor traffic is the statistical analysis and the identification of the secure sockets layer protocol differences (SSL) [6]. The SSL protocol uses a combination of public and symmetric key encryption. Each SSL connection always starts with the exchange of messages by the server and the client until the secure connection is established (handshake). The handshake allows the server to prove its identity to the client using public-key encryption techniques and then allows the client and the server to cooperate in the creation of a symmetric key to be used to quickly encrypt and decrypt data exchanged between them. Optionally, the handshake also allows the client to prove its identity to the server [6]. Given that each Tor client creates self-signed SSL, using a random domain name that changes around every 30 min, a statistical analysis of the network traffic based on the specific SSL characteristics can identify the Tor sessions, in a network full of HTTPS traffic.

4.1 Smart anti-malware extension (SAME)

In our previous work [45], we have proposed the SAME which introduces intelligence to the compiler and classifies malicious java classes in time to spot the android malwares. This is done by applying the java class file analysis (JCFA) approach and based on the effective BBO optimization algorithm, which is used to train a MLP.

Generally, the source code java files (.java) of a java application are compiled to byte code files (.class) which are platform independent and they can be executed by a JVM just like ART which is an ahead-of-time (AOT) compiler. The classes are organized in the .java files with each file containing at least one public class. The name of the file is identical to the name of the contained public class. The ART loads the classes required to execute the java program (class loader) and then it verifies the validity of the byte code files before execution (byte code verifier) [3]. The JCFA process includes also the analysis of the classes, methods and specific characteristics included in an application. The SAME, introduces advanced artificial intelligence (AI) methods, applied on specific parameters and data (obtained after the JCFA process) to perform binary classification of the classes comprising an application, in benign or malicious. More specifically the SAME system employs the biogeography-based optimizer to train a MLP which classifies the java classes of an application successfully in benign or malicious.

The architectural design of the SAME introduces an additional functional level inside the ARTJVM, which analyzes the java classes before their loading and before the execution of the java program (class loader). The introduction of the files in the ARTJVM, always passes from the above level, where the check for malicious classes is done. If malicious classes are detected, decisions are done depending on the accuracy of the classification. If the accuracy is high, then the decisions are done automatically, otherwise the actions are imposed by the user regarding the acceptance or rejection of the application installation. In the case that the classes are benign, the installation is performed normally and the user is notified that this is a secure application [45].

A basic innovation of the SAME is the inclusion of a machine learning approach as an extension of the ART JVM used by the android OS. This joint with the JCFA and the fact that the ART JVM resolves ahead-of-time all of the dependencies during the loading of classes, introduces Intelligence in compiler level. This fact enhances the defensive capabilities of the system significantly. It is important that the dependencies and the structural elements of an application are checked before its installation enabling the malware cases.

An another important innovative part of this research is related to the choice of the independent parameters, which was done after several exhaustive tests, to ensure the maximum performance and generalization of the algorithm and the consumption of the minimum resources.

Finally, it is worth mentioning that the BBO optimization algorithm (popular for engineering cases) is used for the first time to train an artificial neural network (ANN) for a real information security problem.

4.2 Online Tor traffic identification extension (OTTIE)

The TTIE is essentially a tool for analysis of web streaming traffic in fixed intervals, to extract timely conclusions in which some or all of the incoming data are not available for access from any permanent or temporary storage medium, but those arrive in a form of consecutive flows. For these data there is no control over the order in which they arrive, their size may vary and many of them offer no real information. Also the examination of individual IP packets or TCP segments can extract only a few conclusions and therefore the interdependence of the individual packets to each other, their analysis cannot be done with simple static methods, but it requires further modeling of traffic and the use of advanced analytical methods for the extraction of knowledge from complex data sets. This modeling in TTIE is achieved by the use of the computational intelligence online sequential extreme learning machine (OSELM) algorithm.

The extreme learning machine (ELM) as an emerging biologically inspired learning technique provides efficient unified solutions to “generalized” single-hidden layer feed forward networks (SLFNs) but the hidden layer (or called feature mapping) in ELM need not be tuned [46]. Such SLFNs include but are not limited to support vector machine, polynomial network, RBF networks, and the conventional feed forward neural networks. All the hidden node parameters are independent from the target functions or the training datasets and the output weights of ELMs may be determined in different ways (with or without iterations, with or without incremental implementations). ELM has several advantages, ease of use, faster learning speed, higher generalization performance, suitable for many nonlinear activation function and kernel functions.

The proposed method uses an OSELM that can learn data chunk-by-chunk with a fixed chunk size of \(20\times 20\), with RBF kernel classification approach to perform malware localization, Tor traffic identification and botnets prohibition in an energetic security mode that needs minimum computational resources and time [7]. The OSELM consists of two main phases namely: boosting phase (BPh) and sequential learning phase (SLPh). The BPh used to train the SLFNs using the primitive ELM method with some batch of training data in the initialization stage and these boosting training data will be discarded as soon as boosting phase is completed. The required batch of training data is very small, which can be equal to the number of hidden neurons [46, 47, 48].

The general classification process with OSELM classifier is described below:

Phase 1 (BPh) [47, 48]

Step 1 Perform malware localization by OSELM with malware localization dataset (MLD). If the malware analysis gives a positive result (Malware) the network traffic is blocked and the process is terminated. If the malware analysis gives a negative result (Benign), no action is required and goes to step 2.

Step 2 Perform network traffic analysis by OSELM with network traffic classification dataset (NTCD). If the network traffic classification result is not a HTTPS, no action is required. If the network traffic classification result is a HTTPS, go to step 3.

Step 3 Performs Tor-traffic identification by OSELM with Tor-traffic identification dataset (TTID). If the botnet classification result gives a positive result (Botnet) the network traffic blocked and the process terminated. If the botnet classification result gives a negative result (HTTPS), no action is required.

The overall algorithmic approach of TTIE that is proposed herein is described clearly and in detail in the following Fig. 2.

Open image in new window

5.1 Datasets

The selection of the data was the result of an extensive research on the functionality of the protocol SSL, combined with a deep analysis of the independent variables, to obtain the ones that give the maximum precision under the strict condition of minimal computing resources consumption. This effort resulted in the creation of training sets, capable to properly train the employed learning algorithms.

Three datasets with high complexity were constructed and used for testing by the CIantiMF. The first MLD comprised 32 independent variables and 2 classes (benign or malware). This dataset containing 73,469 patterns (37,127 benign samples they were chosen from the Pcaps from National Cyber Watch Mid-Atlantic Collegiate Cyber Defense Competition and 36,342 malicious samples they were chosen from http://malware-traffic-analysis.net/) [49]. The idea of the network traffic analysis and the features extraction approach were based on the functional mode of the TCP protocol and moreover on the acknowledgement method of the reliable submission and receipt of the data. Also it relies on the error-free data transfer mechanisms between the network layer and the application layer, of the TCP header structure and the three-way handshake process [50].

The full list of the 32 features with the class is detailed in Table 1.

The second NTCD (Network Traffic Classification Dataset) comprised 22 independent variables and 12 network traffic classes (TELNET, FTP, HTTP, HTTPS, DNS, Lime, Local Forwarding, Remote Forwarding, SCP, SFTP, x11 and Shell). This dataset containing 137,050 patterns they were chosen from the Pcaps from Information Technology Operations Center (ITOC), US Military Academy [51].

The feature management and export (for Tables 2, 3) was based on the analysis of the network traffic and specifically on the methodology used in [52]. The full list of the 22 features with the corresponding classes is presented in the following Table 2.

Finally, the third TTID comprised 45 independent variables and 2 classes (Tor or HTTPS). This dataset containing 217,483 patterns they were chosen from the Pcaps from [53]. The full list of 45 features with their corresponding classes is presented in the following Table 3.

In the preprocessing process the duplicate records and records with missing values were removed. Also the datasets were determined and normalized to the interval [\(-1,1\)] to phase the problem of prevalence of features with wider range over the ones with a narrower range, without being more important [54].

Taking into account the limited capacity of resources and computing power of mobile devices and the limitations posed by their dependence on the battery, we have made a transformation of the TTID independent variables vector space, which is a very complex and extensive dataset. Principal components analysis (PCA) has been performed to obtain new linear combinations, capable to contain the largest possible part of the variance of the original information, without limiting the predictive capability and accuracy of the learning algorithm. However, due to the fact that the results have deteriorated enough (almost 12% less accuracy was obtained) the approach was abandoned.

Then, we have performed correlation analysis on various subsets of features which had the highest correlation with the obtained class, regardless of their interrelation. Also other subsets were used, which were highly correlated with the class and they appeared to have high cross-correlation (correlation attribute evaluation) [55].

Eventually, the subset chosen was based on the method of correlation-based feature subset selection (subsets of features that correlate highly with the class value and has low correlation with each other). From this dataset we have obtained the minimum error of the classifier in the training and test data, in relation to the value of each feature (attribute evaluation with particle swarm optimization) [55].