Abstract
Decision-making and opinion-forming are everyday tasks that involve weighing pro and con arguments. The goal of Touché is to foster the development of support-technologies for decision-making and opinion-forming and to improve our understanding of these processes. This fifth edition of the lab features three shared tasks: (1) Human value detection (ValueEval), where participants detect (implicit) references to human values and their attainment in text; (2) Multilingual Ideology and Power Identification in Parliamentary Debates, where participants identify from a speech the political leaning of the speaker’s party and whether it was governing at the time of the speech (new task); and (3) Image retrieval or generation in order to convey the premise of an argument with visually. In this paper, we briefly describe the planned setup for the fifth lab edition at CLEF 2024 and summarize the results of the 2023 edition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
‘touché’ confirms “a hit in fencing or the success or appropriateness of an argument, an accusation, or a witty point.” [https://merriam-webster.com/dictionary/touche].
- 2.
Demo of best-performing approach: https://values.args.me.
- 3.
- 4.
- 5.
- 6.
References
Bondarenko, A., et al.: Overview of touché 2020: argument retrieval. In: Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum (CLEF 2020), CEUR Workshop Proceedings, vol. 2696. CEUR-WS.org (2020). http://ceur-ws.org/Vol-2696/paper_261.pdf
Bondarenko, A., et al.: Overview of touché 2023: argument and causal retrieval. In: Arampatzis, A., et al. (eds.) 14th International Conference of the CLEF Association (CLEF 2023). LNCS, vol. 14163, pp. 507–530, Springer, Heidelberg (2023). https://doi.org/10.1007/978-3-031-42448-9_31
Bondarenko, A., et al.: Overview of Touché 2022: argument retrieval. In: Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum (CLEF 2022), CEUR Workshop Proceedings, vol. 3180, pp. 2867–2903. CEUR-WS.org (2022). http://ceur-ws.org/Vol-3180/paper-247.pdf
Bondarenko, A., et al.: Overview of Touché 2021: argument retrieval. In: Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum (CLEF 2021), CEUR Workshop Proceedings, vol. 2936, pp. 2258–2284. CEUR-WS.org (2021). http://ceur-ws.org/Vol-2936/paper-205.pdf
Deckers, N., et al.: The infinite index: information retrieval on generative text-to-image models. In: Gwizdka, J., Rieh, S.Y. (eds.) ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2023), pp. 172–186. ACM (2023). https://doi.org/10.1145/3576840.3578327
Erjavec, T., et al.: The parlamint corpora of parliamentary proceedings. Lang. Resour. Eval. 57, 415–448 (2022). https://doi.org/10.1007/s10579-021-09574-0
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20(4), 422–446 (2002). https://doi.org/10.1145/582415.582418
Kiesel, J., et al.: SemEval-2023 Task 4: ValueEval: identification of human values behind arguments. In: Kumar, R., Ojha, A.K., Doğruöz, A.S., Martino, G.D.S., Madabushi, H.T. (eds.) 17th International Workshop on Semantic Evaluation (SemEval 2023), pp. 2287–2303. Association for Computational Linguistics, Toronto (2023). https://doi.org/10.18653/v1/2023.semeval-1.313
Kiesel, J., Reichenbach, N., Stein, B., Potthast, M.: Image retrieval for arguments using stance-aware query expansion. In: 8th Workshop on Argument Mining (ArgMining 2021). ACL (2021)
MacAvaney, S., Yates, A., Feldman, S., Downey, D., Cohan, A., Goharian, N.: Simplified data wrangling with ir_datasets. In: 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2022), pp. 2429–2436. ACM (2021). https://doi.org/10.1145/3404835.3463254
Potthast, M., Gollub, T., Wiegmann, M., Stein, B.: TIRA integrated research architecture. In: Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF, The Information Retrieval Series, vol. 41, pp. 123–160. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22948-1_5
Robertson, S.E., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M.: Okapi at TREC-3. In: 3rd Text REtrieval Conference (TREC 1994), NIST Special Publication, vol. 500–225, pp. 109–126. NIST (1994)
Robertson, S.E., Zaragoza, H., Taylor, M.J.: Simple BM25 extension to multiple weighted fields. In: 13th International Conference on Information and Knowledge Management (CIKM 2004), pp. 42–49. ACM (2004). https://doi.org/10.1145/1031171.1031181
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), pp. 10674–10685. IEEE (2022). https://doi.org/10.1109/CVPR52688.2022.01042
Scharfbillig, M., et al.: Values and Identities - a Policymaker’s Guide. Tech. Rep. KJ-NA-30800-EN-N, European Commission’s Joint Research Centre, Luxembourg (2021). https://doi.org/10.2760/349527
Schwartz, S.H., et al.: Refining the theory of basic individual values. J. Personal. Soc. Psychol. 103(4), 663–688 (2012). https://doi.org/10.1037/a0029393
Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., Gurevych, I.: BEIR: a heterogeneous benchmark for zero-shot evaluation of information retrieval models. In: Vanschoren, J., Yeung, S. (eds.) Neural Information Processing Systems (NeurIPS 2021). NeurIPS (2021)
Acknowledgements
This work was partially supported by the European Commission under grant agreement GA 101070014 (https://openwebsearch.eu).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kiesel, J. et al. (2024). Overview of Touché 2024: Argumentation Systems. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. Lecture Notes in Computer Science, vol 14612. Springer, Cham. https://doi.org/10.1007/978-3-031-56069-9_64
Download citation
DOI: https://doi.org/10.1007/978-3-031-56069-9_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-56068-2
Online ISBN: 978-3-031-56069-9
eBook Packages: Computer ScienceComputer Science (R0)