RunMax: fake profile classification using novel nonlinear activation in CNN

Abstract

Online social networks (OSN) are well-known platforms for exchanging various information. However, one of the existing OSN challenges is the issue of fake accounts. The attacker harnesses malicious accounts in the infected system to spread false information, such as malware, viruses, and harmful URLs. Based on the vast triumphs of deep learning in several fields, mainly automated representation, we propose RunFake, a convolutional neural network (CNN) to handle malicious account classification. We build a dynamic CNN to train a classification model instead of using regular machine learning. In particular, we create a general activation function called RunMax as a new element of the neural network's final layer. We improve accuracy in the training and testing procedure by utilizing the proposed activation layer instead of the traditional function. Based on the experimental result, our method can yield Precision = 94.00, Recall = 93.21, and F1-Score = 93.42 with a better area under curve (AUC) score = 0.9547 using user profile data as features. We harvest a promising outcome with greater accuracy with tiny loss than common learning architecture in a fake account classification problem.

Appendix

1.1 RunMax: complement material

To deal with the SoftMax bottleneck problem, we propose RunMax given as follows:

Definition 1

RunMax is defined as

$$\left[ {f \left( z \right)_{i} } \right] = \frac{{\exp (z_{i} )\varphi (z_{i} )}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp (z_{m} )\varphi (z_{m} )}},\quad i = 1 \ldots M$$

(10)

where ${\mathcal{G}}\left( \cdot \right)$ represents a Gaussian function $f\left( z \right) = \exp \left( { - z^{2} } \right)$ with derivative $f^{\prime}\left( z \right) = - 2z \left( {\exp \left( { - z^{2} } \right)} \right)$

Theorem 1

Let $z \in S$ as the input of RunMax $f\left( z \right)$ and SoftMax $f_{s} \left( z \right)$. Let S as a d-dimensional vector space $1 \in S$, thus the range of Softmax is a subset of RunMax.

$$\left\{ {f_{s } \left( z \right)\left| {z \in S \subseteq f\left( z \right)} \right|z \in S} \right\}$$

(11)

Proof

If we have $1 \in S$, it can be described as $S = \left\{ {\mathop \sum \limits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} + k^{^{\prime}\left( d \right)} 1 | k^{^{\prime}\left( l \right)} \in R} \right\}$ where $u^{^{\prime}\left( l \right)}$ is $\left( {l = 1, \ldots ,d - 1} \right)$ and 1 are linearly independent vectors. The arbitrary part of S can be represented as $\mathop \sum \limits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} + k^{^{\prime}\left( d \right)} 1$, and thus, we can write $z = \mathop \sum \limits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} + k^{^{\prime}\left( d \right)} 1$. For the output of softmax,

$$\left[ {f_{s} \left( z \right)_{i} } \right] = \frac{{\exp \left( {z_{i} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {z_{m} } \right)}}$$

(12)

By substituting $z = \mathop \sum \limits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} + k^{^{\prime}\left( d \right)} 1$ to the $\left[ {f_{s} \left( z \right)_{i} } \right]$ function, we have:

$$\left[ {f_{s} \left( z \right)_{i} } \right] = \frac{{\exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} + k^{^{\prime}\left( d \right)} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} + k^{^{\prime}\left( d \right)} } \right)}} = \frac{{\exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} } \right)}}$$

(13)

Thus, the range of SoftMax become as follows:

$$\left\{ {f_{s} \left( {\left[ {\mathop \sum \limits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right] + k^{^{\prime}\left( d \right)} 1} \right)k^{^{\prime}\left( d \right)} \in {\mathbb{R}}} \right\} = \left\{ {\frac{{\exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} } \right)}}|k^{^{\prime}\left( l \right)} \in {\mathbb{R}}} \right\}$$

(14)

Besides, by replacing $z = \mathop \sum \limits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} + k^{^{\prime}\left( d \right)} 1$ to the $\left[ {f\left( z \right)_{i} } \right]$ RunMax function, output of RunMax becomes as follows:

$$\left[ {f\left( z \right)_{i} } \right] = \frac{{\exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} } \right)\varphi \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} + k^{^{\prime}\left( d \right)} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} } \right)\varphi \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} + k^{^{\prime}\left( d \right)} } \right)}}$$

(15)

When $k^{^{\prime}\left( l \right)}$ are fixed for $\left( {l = 1, \ldots ,d - 1} \right)$ and $k^{^{\prime}\left( d \right)} \to + \infty$, we get the following equality:

$$\begin{gathered} \mathop {\lim }\limits_{{k^{^{\prime}\left( d \right) \to + \infty } }} \frac{{\exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} } \right) \varphi \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} + k^{^{\prime}\left( d \right)} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} } \right) \varphi \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} + k^{^{\prime}\left( d \right)} } \right)}} \hfill \\ \quad \quad = \frac{{\exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} } \right) }} \hfill \\ \end{gathered}$$

(16)

hence ${\text{lim}}_{{{\text{k}} \to + \infty }} {\mathcal{G} }\left( {{\text{v}},{\text{k}}} \right) = 1$ when $v$ is fixed. Considering Eq. 17, RunMax has following relation:

$$\left\{ {\left( {\frac{{\exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{i} } \right)}}{{\mathop \sum \nolimits_{m = 1}^{M} \exp \left( {\left[ {\mathop \sum \nolimits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} } \right]_{m} } \right) }}} \right)k^{^{\prime}\left( l \right)} \in {\mathbb{R}}} \right\} = \left\{ {f\left( z \right)|z \in s^{\prime}} \right\} \subseteq f\left( z \right)|z \in S\}$$

(17)

We can calculate $S^{\prime} = \left\{ {\mathop \sum \limits_{l = 1}^{d - 1} k^{^{\prime}\left( l \right)} u^{^{\prime}\left( l \right)} + k^{^{\prime}\left( d \right)} 1 | k^{^{\prime}\left( l \right)} \in R for \left( {l = 1, \ldots ,d - 1, k^{^{\prime}\left( d \right)} \to + \infty } \right) \subset S} \right\}$. Based on Eq. (16), we can look that the range of RunMax contains the range of SoftMax. Therefore, we get $\left\{ {f_{s} \left( z \right)\left| { z \in S \subseteq f\left( z \right)} \right| z \in S} \right\}$. Theorem 1 describes that if $1 \in S$, so the range of RunMax is able to be larger than that of SoftMax. The assumption $1 \in S$ means that there exist inputs of which outputs are the equal probabilities for all labels as $p_{\theta } \left( {y_{i} |x} \right) = \frac{1}{M}$ for all $i$.