A comparison of text preprocessing techniques for hate and offensive speech detection in Twitter

Abstract

Preprocessing is a crucial step for each task related to text classification. Preprocessing can have a significant impact on classification performance, but at present there are few large-scale studies evaluating the effectiveness of preprocessing techniques and their combinations. In this work, we explore the impact of 26 widely used text preprocessing techniques on the performance of hate and offensive speech detection algorithms. We evaluate six common machine learning models, such as logistic regression, random forest, linear support vector classifier, convolutional neural network, bidirectional encoder representations from transformers (BERT), and RoBERTa, on four common Twitter benchmarks. Our results show that some preprocessing techniques are useful for improving the accuracy of models while others may even cause a loss of efficiency. In addition, the effectiveness of preprocessing techniques varies depending on the chosen dataset and the classification method. We also explore two ways to combine the techniques that have proved effective during a separate evaluation. Our results show that combining techniques can produce different results. In our experiments, combining techniques works better for traditional machine learning methods than for other methods.

Appendices

Appendix A: Lists of emoticons, contractions, and acronyms

• Converting emoticons to words (17)::D, )),:),:)), etc. $\rightarrow$ happy; ((,:(,:((, etc. $\rightarrow$ sad; XD $\rightarrow$ laugh;:* $\rightarrow$ kiss.

• Decontraction (22): won’t $\rightarrow$ will not; shan’t $\rightarrow$ shall not; can’t $\rightarrow$ can not; couldn’t $\rightarrow$ could not; ’re $\rightarrow$ are; ’s $\rightarrow$ is; ’d $\rightarrow$ would not; ’ll $\rightarrow$ will not; ’ve $\rightarrow$ have; ’m $\rightarrow$ am.

• Replacing acronyms (23): ?4U $\rightarrow$ question for you, 2moro $\rightarrow$ tomorrow, 2nte $\rightarrow$ tonight, asap $\rightarrow$ as soon as possible, atm $\rightarrow$ at the moment, awsm $\rightarrow$ awesome, b2w $\rightarrow$ back to work, bff $\rightarrow$ best friends forever, brb $\rightarrow$ be right back, btw $\rightarrow$ by the way, cu $\rightarrow$ see you, cus $\rightarrow$ see you soon, deti $\rightarrow$ don’t even think it, diky $\rightarrow$ do i know you, g2g $\rightarrow$ got to go, gr8 $\rightarrow$ great, idc $\rightarrow$ i don’t care, idk $\rightarrow$ i don’t care, lmk $\rightarrow$ let me know, lol $\rightarrow$ laughing out loud, lu $\rightarrow$ love you, ly $\rightarrow$ love you, myob $\rightarrow$ mind your own business, omg $\rightarrow$ oh my god, pls $\rightarrow$ please, plz $\rightarrow$ please, rip $\rightarrow$ rest in peace, rofl $\rightarrow$ rolling on the floor, sep $\rightarrow$ someone else’s problem, sup $\rightarrow$ what’s up, tnx $\rightarrow$ thanks, ttyl $\rightarrow$ talk to you later.

Appendix B: The main parameters of LR, RF, and LSVC

See Table 13.

Table 13 Parameters for LR, RF, and LSVC

Table 14 Parameters for grid search

Fig. 3

CNN architecture

Appendix C: The architecture of CNN

See Table 14 and Fig. 3.

Appendix D: The values of standard deviation across the folds for separate evaluation of preprocessing techniques

See Tables 15 and 16.

Table 15 Separate evaluation of techniques for logistic regression, random forest, and linear support vector classifier, standard deviation across the folds (%)

Table 16 Separate evaluation of techniques for Convolutional Neural Network, BERT, and RoBERTa (RB), standard deviation across the folds (%)