Abstract
In this paper we studied the most typical characteristics of toxic images on the web. To get a set of toxic images we collected a set of 8800 images from 4chan.org. Then we trained a BERT-based classifier to find toxic texts with accompanying images. We manually labelled approximately 2000 images accompanying these texts. This revealed that toxic content in images does not correlate with toxic content in texts. On top of manually annotated images there was trained a neural network that inferred labels for unannotated pictures. Neural network layer activations for these images were clustered and manually classified to find the most typical ways of expressing aggression in images. We find that racial stereotypes are the main cause of toxicity in images (https://github.com/denis-gordeev/specom20).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cheng, L., Guo, R., Silva, Y., Hall, D., Liu, H.: Hierarchical attention networks for cyberbullying detection on the instagram social network. In: Proceedings of the 2019 SIAM International Conference on Data Mining, pp. 235–243. SIAM (2019)
Cheng, L., Li, J., Silva, Y.N., Hall, D.L., Liu, H.: XBully: cyberbullying detection within a multi-modal context. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 339–347 (2019)
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20(1), 37–46 (1960)
Davidson, T., Warmsley, D., Macy, M., Weber, I.: Automated hate speech detection and the problem of offensive language. In: Proceedings of the ICWSM (2017)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding, October 2018. http://arxiv.org/abs/1810.04805
DeWall, C.N., Anderson, C.A., Bushman, B.J.: The general aggression model: theoretical extensions to violence. Psychol. Violence 1(3), 245 (2011)
Duggan, M.: Online harassment, vol. 2017 (2017)
Hitkul, H., Shah, R.R., Kumaraguru, P., Satoh, S.: Maybe look closer? Detecting trolling prone images on instagram. In: 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pp. 448–456. IEEE (2019)
Khandagale, S., Xiao, H., Babbar, R.: Bonsai-diverse and shallow trees for extreme multi-label classification. arXiv preprint arXiv:1904.08249 (2019)
Lapidot-Lefler, N., Barak, A.: Effects of anonymity, invisibility, and lack of eye-contact on toxic online disinhibition. Comput. Hum. Behav. 28(2), 434–443 (2012)
Malmasi, S., Zampieri, M.: Detecting Hate Speech in Social Media. In: Proceedings of the International Conference on Recent Advances in Natural Language Processing, pp. 467–472 (2017)
McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018)
Niam, I.M.A., Irawan, B., Setianingsih, C., Putra, B.P.: Hate speech detection using latent semantic analysis (LSA) method based on image. In: 2018 International Conference on Control, Electronics, Renewable Energy and Communications (ICCEREC), pp. 166–171. IEEE (2018)
Potapova, R., Potapov, V.: Human as acmeologic entity in social network discourse (multidimensional approach). In: Karpov, A., Potapova, R., Mporas, I. (eds.) SPECOM 2017. LNCS (LNAI), vol. 10458, pp. 407–416. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66429-3_40
Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987)
Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Suler, J.: The online disinhibition effect. Cyberpsychol. Behav. 7(3), 321–326 (2004)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826 (2016)
Van Horn, G., et al.: The inaturalist species classification and detection dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8769–8778 (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processsing Systems, vol. 2017, pp. 5999–6009, December 2017. http://papers.nips.cc/paper/7181-attention-is-all-you-need
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Brew, J.: HuggingFace’s Transformers: State-of-the-art Natural Language Processing. arXiv abs/1910.0 (2019)
Yao, M., Chelmis, C., Zois, D.S.: Cyberbullying ends here: Towards robust detection of cyberbullying in social media. In: The World Wide Web Conference. pp. 3427–3433 (2019)
Acknowledgements
This research was supported by the Russian Science Foundation (RSF) according to the research project 18-18-00477.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gordeev, D., Potapov, V. (2020). Toxicity in Texts and Images on the Internet. In: Karpov, A., Potapova, R. (eds) Speech and Computer. SPECOM 2020. Lecture Notes in Computer Science(), vol 12335. Springer, Cham. https://doi.org/10.1007/978-3-030-60276-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-60276-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60275-8
Online ISBN: 978-3-030-60276-5
eBook Packages: Computer ScienceComputer Science (R0)