A multi-task attention tree neural net for stance classification and rumor veracity detection

Abstract

Identifying public’s stances to rumors and detecting rumors’ veracity in each rumor conversation are two closely related tasks for distinguishing fake news on social networks. However, very few methods are specially designed for rumor conversations to extract their structural features and jointly address the two tasks in a single machine learning model. In this paper, we propose a Multi-task Attention Tree Neural Network (MATNN) to jointly classify stances and to detect rumor veracity. Specifically, the proposed MATNN specially designs a structural representation called Regular rumor Conversation Trees (RC-Trees) for rumor conversations, and irregular rumor conversation trees are transformed into regular RC-Trees. To extract features from RC-Trees, the proposed MATNN designs Tree Self-Attention mechanism to extract local structural features to classify stances; Based on extracted stance features, the MATNN designs Tree convolution and Tree pooling operations to extract global structural features to detect rumor veracity. Experiments on both stance classification and rumor veracity detection tasks show that the proposed RC-Trees are useful for the two tasks and the proposed MATNN achieves remarkable performance over the current state-of-arts.

Appendix

In this paper, we propose a MATNN for stance classification and rumor veracity detection, and the MATNN is a kind of neural network, which aims to model the data distribution. For different datasets, a well-trained neural network can fit its data distribution and build a joint distribution of the data and the labels ${p_{i}}\left ({dat{a_{i}},label} \right )$. For different data_i, the joint distributions are different. Therefore, a well-trained neural network usually need to be adjusted for other datasets. Moreover, a neural network is usually stable for its training and testing dataset. In this section, we use F-test to verify that the proposed MATNN is statistically stable.

For rumor analysis dataset, we stochastically divide the dataset into two parts: X₁ and X₂, the two sub-datasets come from the same environment, and it is assumed that they have the same variance, which can be expressed as:

$$\sigma_{{X_{1}}}^{2} \approx \sigma_{{X_{2}}}^{2}$$

If the proposed MATNN successfully model the joint distribution of data and labels, the variances of the output of the MATNN $S_{MTNN\left ({{X_{1}}} \right )}^{2}$ and $S_{MTNN\left ({{X_{2}}} \right )}^{2}$ should also be consistent. Therefore, we use F-test of the MATNN output to verify that the proposed MATNN is statistically stable. To ensure the consistency of the $\sigma _{{X_{1}}}^{2}$ and $\sigma _{{X_{2}}}^{2}$, we randomly sample 10 times from the rumor veracity detection dataset, the sampling ratio in every sampling process is 80%. The RMSEs of the proposed MATNN with 10 times training are used for calculating the $S_{MTNN\left ({{X_{1}}} \right )}^{2}$ and $S_{MTNN\left ({{X_{2}}} \right )}^{2}$. In every training, the variances are calculated 10 times, and 20 times, and then averaged. The ratio of $S_{MTNN\left ({{X_{1}}} \right )}^{2}$ and $S_{MTNN\left ({{X_{2}}} \right )}^{2}$ are shown in the following Table.

As Table 1 shows, the ratios of $S_{MTNN\left ({{X_{1}}} \right )}^{2}$ and $S_{MTNN\left ({{X_{2}}} \right )}^{2}$ are similar for 10 trained MATNNs, and the ratios are close to 1, which verify that the proposed MATNN learns the joint distribution and it is statistically stable.