Abstract
The paper describes the lab on Sexism identification in social networks (EXIST 2023) that will be hosted as a lab at the CLEF 2023 conference. The lab consists of three tasks, two of which are continuation of EXIST 2022 (sexism detection and sexism categorization) and a third and novel one on source intention identification. For this edition new test and training data will be provided and some novelties are introduced in order to tackle two central problems of Natural Language Processing (NLP): bias and fairness. Firstly, the sampling and data gathering process will take into account different sources of bias in data: seed, temporal and user bias. During the annotation process we will also consider some sources of “label bias” that come from the social and demographic characteristics of the annotators. Secondly, we will adopt the “learning with disagreements” paradigm by providing datasets containing also pre-aggregated annotations, so that systems can make use of this information to learn from different perspectives. The general goal of the EXIST shared tasks is to advance the state of the art in online sexism detection and categorization, as well as investigating to what extent bias can be characterized in data and whether systems may take fairness decisions when learning from multiple annotations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The Oxford English Dictionary defines sexism as “prejudice, stereotyping or discrimination, typically against women, on the basis of sex”.
- 2.
- 3.
https://gab.com/. Accessed 14 Oct 2022.
- 4.
Everyday sexism project: https://everydaysexism.com/. Accessed 14 October 2022.
References
Social Media and the Silencing Effect: Why Misogyny Online is a Human Rights Issue. NewStatesman. http://bit.ly/3n3ox68. Accessed 25 Sept 2022
Burgos, A., et al.: Violencias de Género 2.0, pp. 13–27 (2014)
Twitter’s Famous Racist Problem. The Atlantic. https://bit.ly/38EnFPw. Accessed 25 Sept 2022
Anzovino, M., Fersini, E., Rosso, P.: Automatic identification and classification of misogynistic language on twitter. In Proceedings of the International Conference on Applications of Natural Language to Information Systems (NLDB 2018), pp. 57–64 (2018)
Pamungkas, E., Basile, V., Patti, V.: Misogyny detection in twitter: a multilingual and cross-domain study. Inf. Process. Manag. 57(6), 102360 (2020)
Guest, E., Vidgen, B., Mittos, A., Sastry, N., Tyson, G., Margetts, H.: An expert annotated dataset for the detection of online misogyny. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume (EACL 2021), pp. 1336–1350 (2021)
Rodríguez-Sánchez, F., et al.: Overview of EXIST 2021: sexism identification in social networks. Procesamiento del Lenguaje Natural 67, 195–207 (2021)
Rodríguez-Sánchez, F., et al.: Overview of EXIST 2022: sEXism identification in social neTworks. Procesamiento del Lenguaje Natural 69, 229–240 (2022)
Uma, A., et al.: SemEval-2021 Task 12: learning with disagreement. In: Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pp. 338–347 (2021)
Hovy, D., Prabhumoye, S.: Five sources of bias in natural language processing. Lang. Linguist. Compass 15(8), e12432 (2021)
Sheng, V.S., Provost, F., Ipeirotis, P. G.: get another label? improving data quality and data mining using multiple, noisy labelers. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2008), pp. 614–622 (2008)
Rodrigues, F., Pereira. F. C.: Deep learning from crowds. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pp. 1611–1618 (2018)
Peterson, J. C., Battleday, R.M., Griffiths, T.L., Russakovsky, O.: Human uncertainty makes classification more robust. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, pp. 9616–9625 (2019)
Roselli, D., Matthews, J., Talagala, N.: Managing bias in AI. In: Companion Proceedings of The 2019 World Wide Web Conference (WWW 2019), pp. 539–544 (2019)
Acknowledgments
This work is supported by the Spanish Ministry of Science and Innovation (project FairTransNLP (PID2021-124361OB-C31 and PID2021-124361OB-C32)) and the Spanish Ministry of Economy and Competitiveness (project pace for Observation of AI in Spanish (UNED and RED.ES, M.P., ref. C039/21- OT)).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Plaza, L. et al. (2023). Overview of EXIST 2023: sEXism Identification in Social NeTworks. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13982. Springer, Cham. https://doi.org/10.1007/978-3-031-28241-6_68
Download citation
DOI: https://doi.org/10.1007/978-3-031-28241-6_68
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28240-9
Online ISBN: 978-3-031-28241-6
eBook Packages: Computer ScienceComputer Science (R0)