Skip to main content
SpringerLink
Log in

Overview of Touché 2022: Argument Retrieval

  • Conference paper
  • First Online:
Experimental IR Meets Multilinguality, Multimodality, and Interaction (CLEF 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13390))

Abstract

This paper is a condensed report on the third year of the Touché lab on argument retrieval held at CLEF 2022. With the goal to foster and support the development of technologies for argument mining and argument analysis, we organized three shared tasks in the third edition of Touché: (a) argument retrieval for controversial topics, where participants retrieve a gist of arguments from a collection of online debates, (b) argument retrieval for comparative questions, where participants retrieve argumentative passages from a generic web crawl, and (c) image retrieval for arguments, where participants retrieve images from a focused web crawl that show support or opposition to some stance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    ‘touché’ is commonly “used to acknowledge a hit in fencing or the success or appropriateness of an argument” [https://merriam-webster.com/dictionary/touche]

  2. 2.

    https://trec.nist.gov/tracks.html

  3. 3.

    http://commoncrawl.org

  4. 4.

    Three teams declined to proceed in the task after submitting the results

  5. 5.

    The expected format of submissions was also described at https://touche.webis.de

  6. 6.

    https://webis.de/data.html#args-me-corpus

  7. 7.

    https://www.args.me/api-en.html

  8. 8.

    https://opennlp.apache.org/

  9. 9.

    https://pypi.org/project/pyserini/

  10. 10.

    https://www.elastic.co/

  11. 11.

    https://lemurproject.org/clueweb12/index.php

  12. 12.

    https://github.com/grill-lab/trec-cast-tools

  13. 13.

    Also available as a Python library: https://pypi.org/project/targer-api/

  14. 14.

    http://www.dcs.gla.ac.uk/~craigm/colbert.dnn.zip

  15. 15.

    https://github.com/MatthiasWinkelmann/english-words-names-brands-places

  16. 16.

    https://lemurproject.org/clueweb12/related-data.php

  17. 17.

    https://www.phash.org/

  18. 18.

    Archived using https://github.com/webis-de/scriptor

  19. 19.

    https://github.com/tesseract-ocr/tesseract

  20. 20.

    https://www.mturk.com

  21. 21.

    https://github.com/justinshenk/fer

References

  1. Aigrain, P., Zhang, H., Petkovic, D.: Content-based representation and retrieval of visual media: a state-of-the-art review. Multimed. Tools Appl. 3(3), 179–202 (1996). https://doi.org/10.1007/BF00393937

    Article  Google Scholar 

  2. Ajjour, Y., Braslavski, P., Bondarenko, A., Stein, B.: Identifying argumentative questions in web search logs. In: 45th International ACM Conference on Research and Development in Information Retrieval (SIGIR 2022). ACM, July 2022. https://doi.org/10.1145/3477495.3531864

  3. Ajjour, Y., Wachsmuth, H., Kiesel, J., Potthast, M., Hagen, M., Stein, B.: Data acquisition for argument search: the args.me corpus. In: Benzmüller, C., Stuckenschmidt, H. (eds.) KI 2019. LNCS (LNAI), vol. 11793, pp. 48–59. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30179-8_4

    Chapter  Google Scholar 

  4. Alhamzeh, A., Bouhaouel, M., Egyed-Zsigmond, E., Mitrovic, J.: DistilBERT-based argumentation retrieval for answering comparative questions. In: Proceedings of the Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings, vol. 2936, pp. 2319–2330, CEUR-WS.org (2021). http://ceur-ws.org/Vol-2936/paper-209.pdf

  5. Alshomary, M., Düsterhus, N., Wachsmuth, H.: Extractive snippet generation for arguments. In: Proceedings of the 43nd International ACM Conference on Research and Development in Information Retrieval, SIGIR 2020, pp. 1969–1972. ACM (2020). https://doi.org/10.1145/3397271.3401186

  6. Aristotle, Kennedy, G.A.: On Rhetoric: A Theory of Civic Discourse. Oxford University Press, Oxford (2006)

    Google Scholar 

  7. Bar-Haim, R., Kantor, Y., Venezian, E., Katz, Y., Slonim, N.: Project debater APIs: decomposing the AI grand challenge. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2021, Online and Punta Cana, Dominican Republic, 7–11 November 2021, pp. 267–274. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.emnlp-demo.31

  8. Bevendorff, J., et al.: Webis at TREC 2020: Health Misinformation track. In: Voorhees, E., Ellis, A. (eds.) Proceedings of the 29th International Text Retrieval Conference, TREC 2020, NIST, November 2020

    Google Scholar 

  9. Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly (2009). ISBN 978-0-596-51649-9. http://www.oreilly.de/catalog/9780596516499/index.html

  10. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003). http://jmlr.org/papers/v3/blei03a.html

  11. Bondarenko, A., Ajjour, Y., Dittmar, V., Homann, N., Braslavski, P., Hagen, M.: Towards understanding and answering comparative questions. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining, WSDM 2022. ACM (2022). https://doi.org/10.1145/3488560.3498534

  12. Bondarenko, A., et al.: Comparative web search questions. In: Proceedings of the 13th ACM International Conference on Web Search and Data Mining, WSDM 2020, pp. 52–60. ACM (2020). https://dl.acm.org/doi/abs/10.1145/3336191.3371848

  13. Bondarenko, A., et al.: Overview of Touché 2020: argument retrieval. In: Working Notes Papers of the CLEF 2020 Evaluation Labs. CEUR Workshop Proceedings, vol. 2696 (2020). http://ceur-ws.org/Vol-2696/

  14. Bondarenko, A., Fröbe, M., Kasturia, V., Völske, M., Stein, B., Hagen, M.: Webis at TREC 2019: decision track. In: Voorhees, E., Ellis, A. (eds.) Proceedings of the 28th International Text Retrieval Conference, TREC 2019, NIST, November 2019

    Google Scholar 

  15. Bondarenko, A., et al.: Overview of touché 2022: argument retrieval. In: Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. CEUR Workshop Proceedings. CEUR-WS.org, Berlin, Heidelberg, New York (2022, to appear)

    Google Scholar 

  16. Bondarenko, A., et al.: Overview of touché 2021: argument retrieval. In: Working Notes Papers of the CLEF 2021 Evaluation Labs. CEUR Workshop Proceedings, vol. 2936 (2021). http://ceur-ws.org/Vol-2936/

  17. Chang, N., Fu, K.: Query-by-pictorial-example. IEEE Trans. Softw. Eng. 6(6), 519–524 (1980). https://doi.org/10.1109/TSE.1980.230801

    Article  Google Scholar 

  18. Chekalina, V., Bondarenko, A., Biemann, C., Beloucif, M., Logacheva, V., Panchenko, A.: Which is better for deep learning: Python or MATLAB? Answering comparative questions in natural language. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, EACL 2021, pp. 302–311. Association for Computational Linguistics (2021). https://www.aclweb.org/anthology/2021.eacl-demos.36/

  19. Chernodub, A., et al.: TARGER: neural argument mining at your fingertips. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019, pp. 195–200. ACL (2019). https://doi.org/10.18653/v1/p19-3031

  20. Cormack, G.V., Clarke, C.L.A., Büttcher, S.: Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In: Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009, pp. 758–759. ACM (2009). https://doi.org/10.1145/1571941.1572114

  21. Cormack, G.V., Smucker, M.D., Clarke, C.L.A.: Efficient and effective spam filtering and re-ranking for large web datasets. Inf. Retr. 14(5), 441–465 (2011). https://doi.org/10.1007/s10791-011-9162-z

    Article  Google Scholar 

  22. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, pp. 4171–4186. ACL (2019). https://doi.org/10.18653/v1/n19-1423

  23. Dimitrov, D., et al.: SemEval-2021 task 6: detection of persuasion techniques in texts and images. In: 15th International Workshop on Semantic Evaluation (SemEval 2021), pp. 70–98. Association for Computational Linguistics, August 2021. https://doi.org/10.18653/v1/2021.semeval-1.7, https://aclanthology.org/2021.semeval-1.7

  24. Dove, I.J.: On images as evidence and arguments. In: van Eemeren, F.H., Garssen, B. (eds.) Topical Themes in Argumentation Theory: Twenty Exploratory Studies. Argumentation Library, pp. 223–238. Springer, Dordrecht (2012). https://doi.org/10.1007/978-94-007-4041-9_15

    Chapter  Google Scholar 

  25. Dumani, L., Neumann, P.J., Schenkel, R.: A framework for argument retrieval - ranking argument clusters by frequency and specificity. In: Jose, J.M., et al. (eds.) ECIR 2020. LNCS, vol. 12035, pp. 431–445. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45439-5_29

    Chapter  Google Scholar 

  26. Dumani, L., Schenkel, R.: Quality aware ranking of arguments. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM 2020, pp. 335–344. Association for Computing Machinery (2020). https://doi.org/10.1007/978-3-030-45439-5_29

  27. Dunaway, F.: Images, emotions, politics. Mod. Am. Hist. 1(3), 369–376 (2018). https://doi.org/10.1017/mah.2018.17. ISSN 2515-0456, 2397-1851

    Article  Google Scholar 

  28. Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004). https://doi.org/10.1613/jair.1523

    Article  Google Scholar 

  29. Fröbe, M., et al.: CopyCat: near-duplicates within and between the ClueWeb and the common crawl. In: Proceedings of the 44th International ACM Conference on Research and Development in Information Retrieval, SIGIR 2021, pp. 2398–2404. ACM (2021). https://dl.acm.org/doi/10.1145/3404835.3463246

  30. Fröbe, M., Bevendorff, J., Reimer, J., Potthast, M., Hagen, M.: Sampling bias due to near-duplicates in learning to rank. In: Proceedings of the 43rd International ACM Conference on Research and Development in Information Retrieval, SIGIR 2020. ACM (2020). https://dl.acm.org/doi/10.1145/3397271.3401212

  31. Fröbe, M., Bittner, J.P., Potthast, M., Hagen, M.: The effect of content-equivalent near-duplicates on the evaluation of search engines. In: Jose, J.M., et al. (eds.) ECIR 2020. LNCS, vol. 12036, pp. 12–19. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45442-5_2

    Chapter  Google Scholar 

  32. Gienapp, L., Stein, B., Hagen, M., Potthast, M.: Efficient pairwise annotation of argument quality. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5772–5781. Association for Computational Linguistics, July 2020. https://doi.org/10.18653/v1/2020.acl-main.511, https://aclanthology.org/2020.acl-main.511

  33. Gienapp, L., Stein, B., Hagen, M., Potthast, M.: Efficient pairwise annotation of argument quality. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp. 5772–5781. Association for Computational Linguistics (2020). https://www.aclweb.org/anthology/2020.acl-main.511/

  34. Google: Google images best practices. Google Developers (2021). https://support.google.com/webmasters/answer/114016

  35. Grancea, I.: Types of visual arguments. Argumentum. J. Seminar Discursive Log. Argument. Theory Rhetoric 15(2), 16–34 (2017)

    Google Scholar 

  36. Gretz, S., et al.: A large-scale dataset for argument quality ranking: construction and analysis. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, pp. 7805–7813. AAAI Press (2020). https://ojs.aaai.org/index.php/AAAI/article/view/6285

  37. Ho, T.K.: Random decision forests. In: Third International Conference on Document Analysis and Recognition, ICDAR 1995, 14–15 August 1995, Montreal, Canada, vol. I, pp. 278–282. IEEE Computer Society (1995). https://doi.org/10.1109/ICDAR.1995.598994

  38. Hovy, D., Berg-Kirkpatrick, T., Vaswani, A., Hovy, E.: Learning whom to trust with MACE. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HTL 2013), pp. 1120–1130. Association for Computational Linguistics, Atlanta, June 2013. https://aclanthology.org/N13-1132

  39. Jindal, N., Liu, B.: Identifying comparative sentences in text documents. In: Proceedings of the 29th Annual International Conference on Research and Development in Information Retrieval, SIGIR 2006, pp. 244–251. ACM (2006). https://doi.org/10.1145/1148170.1148215

  40. Jindal, N., Liu, B.: Mining comparative sentences and relations. In: Proceedings of the 21st National Conference on Artificial Intelligence and the 18th Innovative Applications of Artificial Intelligence Conference, AAAI 2006, pp. 1331–1336. AAAI Press (2006). http://www.aaai.org/Library/AAAI/2006/aaai06-209.php

  41. Kaszkiel, M., Zobel, J.: Passage retrieval revisited. In: Belkin, N.J., Narasimhalu, A.D., Willett, P., Hersh, W.R., Can, F., Voorhees, E.M. (eds.) Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1997, Philadelphia, PA, USA, 27–31 July 1997, pp. 178–185. ACM (1997). https://doi.org/10.1145/258525.258561

  42. Kessler, W., Kuhn, J.: A corpus of comparisons in product reviews. In: Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014, pp. 2242–2248. European Language Resources Association (ELRA) (2014). http://www.lrec-conf.org/proceedings/lrec2014/summaries/1001.html

  43. Khattab, O., Zaharia, M.: ColBERT: efficient and effective passage search via contextualized late interaction over BERT. In: Huang, J., et al. (eds.) Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, pp. 39–48. ACM (2020). https://doi.org/10.1145/3397271.3401075

  44. Kiesel, J., Reichenbach, N., Stein, B., Potthast, M.: Image retrieval for arguments using stance-aware query expansion. In: Proceedings of the 8th Workshop on Argument Mining, ArgMining 2021 at EMNLP, pp. 36–45. ACL (2021)

    Google Scholar 

  45. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: a lite BERT for self-supervised learning of language representations. In: Proceedings of the 8th International Conference on Learning Representations, ICLR 2020. OpenReview.net (2020). https://openreview.net/forum?id=H1eA7AEtvS

  46. Latif, A., et al.: Content-based image retrieval and feature extraction: A comprehensive review. Math. Probl. Eng. 2019, 21 (2019). https://doi.org/10.1155/2019/9658350

    Article  Google Scholar 

  47. Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on Machine Learning, pp. 1188–1196. PMLR (2014)

    Google Scholar 

  48. Levy, R., Bogin, B., Gretz, S., Aharonov, R., Slonim, N.: Towards an argumentative content search engine using weak supervision. In: Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, pp. 2066–2081. Association for Computational Linguistics (2018). https://www.aclweb.org/anthology/C18-1176/

  49. Lin, J., Ma, X., Lin, S., Yang, J., Pradeep, R., Nogueira, R.: Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021, pp. 2356–2362. ACM (2021). https://doi.org/10.1145/3404835.3463238

  50. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019). http://arxiv.org/abs/1907.11692

  51. Lovins, J.B.: Development of a stemming algorithm. Mech. Transl. Comput. Linguist. 11(1–2), 22–31 (1968). http://www.mt-archive.info/MT-1968-Lovins.pdf

  52. Ma, N., Mazumder, S., Wang, H., Liu, B.: Entity-aware dependency-based deep graph attention network for comparative preference classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp. 5782–5788. Association for Computational Linguistics (2020). https://www.aclweb.org/anthology/2020.acl-main.512/

  53. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the 1st International Conference on Learning Representations, ICLR 2013 (2013). http://arxiv.org/abs/1301.3781

  54. Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  55. Nadamoto, A., Tanaka, K.: A comparative web browser (CWB) for browsing and comparing web pages. In: Proceedings of the 12th International World Wide Web Conference, WWW 2003, pp. 727–735. ACM (2003). https://doi.org/10.1145/775152.775254

  56. Nguyen, T., et al.: MS MARCO: a human generated MAchine reading COmprehension dataset. In: Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches 2016 at NIPS, CEUR Workshop Proceedings, vol. 1773. CEUR-WS.org (2016). http://ceur-ws.org/Vol-1773/CoCoNIPS_2016_paper9.pdf

  57. Nogueira, R., Lin, J., Epistemic, A.: From doc2query to docTTTTTquery. Online preprint (2019). https://cs.uwaterloo.ca/~jimmylin/publications/Nogueira_Lin_2019_docTTTTTquery-v2.pdf

  58. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web. Technical report 1999–66, Stanford InfoLab (1999). http://ilpubs.stanford.edu:8090/422/

  59. Palotti, J.R.M., Scells, H., Zuccon, G.: TrecTools: an open-source Python library for information retrieval practitioners involved in TREC-like campaigns. In: Proceedings of the 42nd International Conference on Research and Development in Information Retrieval, SIGIR 2019, pp. 1325–1328. ACM (2019). https://doi.org/10.1145/3331184.3331399

  60. Panchenko, A., Bondarenko, A., Franzek, M., Hagen, M., Biemann, C.: Categorizing comparative sentences. In: Proceedings of the 6th Workshop on Argument Mining, ArgMining@ACL 2019, pp. 136–145. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/w19-4516

  61. Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980). https://doi.org/10.1108/eb046814

    Article  Google Scholar 

  62. Porter, M.F.: Snowball: a language for stemming algorithms (2001). http://snowball.tartarus.org/texts/introduction.html

  63. Potthast, M., et al.: Argument search: assessing argument relevance. In: Proceedings of the 42nd International Conference on Research and Development in Information Retrieval, SIGIR 2019, pp. 1117–1120. ACM (2019). https://doi.org/10.1145/3331184.3331327

  64. Potthast, M., Gollub, T., Wiegmann, M., Stein, B.: TIRA integrated research architecture. In: Information Retrieval Evaluation in a Changing World. TIRS, vol. 41, pp. 123–160. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22948-1_5

    Chapter  Google Scholar 

  65. Pradeep, R., Nogueira, R., Lin, J.: The expando-mono-duo design pattern for text ranking with pretrained sequence-to-sequence models. CoRR abs/2101.05667 (2021). https://arxiv.org/abs/2101.05667

  66. Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)

    Google Scholar 

  67. Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 140:1–140:67 (2020). http://jmlr.org/papers/v21/20-074.html

  68. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pp. 3980–3990. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/D19-1410

  69. Reimers, N., Schiller, B., Beck, T., Daxenberger, J., Stab, C., Gurevych, I.: Classification and clustering of arguments with contextualized word embeddings. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 567–578. Association for Computational Linguistics, Florence, July 2019. https://doi.org/10.18653/v1/P19-1054, https://aclanthology.org/P19-1054

  70. Robertson, S.E., Walker, S., Jones, S., Hancock-Beaulieu, M., Gatford, M.: Okapi at TREC-3. In: Proceedings of The Third Text REtrieval Conference, TREC 1994, NIST Special Publication, vol. 500-225, pp. 109–126. NIST (1994). https://trec.nist.gov/pubs/trec3/papers/city.ps.gz

  71. Robertson, S.E., Zaragoza, H., Taylor, M.J.: Simple BM25 extension to multiple weighted fields. In: Proceedings of the 13th International Conference on Information and Knowledge Management, CIKM 2004, pp. 42–49. ACM (2004). https://doi.org/10.1145/1031171.1031181

  72. Rocchio, J.: Relevance feedback in information retrieval. In: The Smart Retrieval System-Experiments in Automatic Document Processing, pp. 313–323 (1971)

    Google Scholar 

  73. Roque, G.: Visual argumentation: a further reappraisal. In: van Eemeren, F.H., Garssen, B. (eds.) Topical Themes in Argumentation Theory. Argumentation Library, vol. 22, pp. 273–288. Springer, Dordrecht (2012). https://doi.org/10.1007/978-94-007-4041-9_18. ISBN 978-94-007-4040-2

  74. Rose, S., Engel, D., Cramer, N., Cowley, W.: Automatic keyword extraction from individual documents. Text Min.: Appl. Theory 1(1–20), 10–1002 (2010)

    Google Scholar 

  75. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019). http://arxiv.org/abs/1910.01108

  76. Sanh, V., et al.: Multitask prompted training enables zero-shot task generalization. CoRR abs/2110.08207 (2021). https://arxiv.org/abs/2110.08207

  77. Schildwächter, M., Bondarenko, A., Zenker, J., Hagen, M., Biemann, C., Panchenko, A.: Answering comparative questions: better than ten-blue-links? In: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, CHIIR 2019, pp. 361–365. ACM (2019). https://doi.org/10.1145/3295750.3298916

  78. Solli, M., Lenz, R.: Color emotions for multi-colored images. Color Res. Appl. 36(3), 210–221 (2011). https://doi.org/10.1002/col.20604

    Article  Google Scholar 

  79. Stab, C., et al.: ArgumenText: searching for arguments in heterogeneous sources. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2018, pp. 21–25. Association for Computational Linguistics (2018). https://www.aclweb.org/anthology/N18-5005

  80. Sun, J., Wang, X., Shen, D., Zeng, H., Chen, Z.: CWS: a comparative web search system. In: Proceedings of the 15th International Conference on World Wide Web, WWW 2006, pp. 467–476. ACM (2006). https://doi.org/10.1145/1135777.1135846

  81. Wachsmuth, H., et al.: Argumentation quality assessment: theory vs. practice. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, pp. 250–255. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-2039

  82. Wachsmuth, H., et al.: Computational argumentation quality assessment in natural language. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, pp. 176–187 (2017). http://aclweb.org/anthology/E17-1017

  83. Wachsmuth, H., et al.: Building an argument search engine for the web. In: Proceedings of the Fourth Workshop on Argument Mining (ArgMining), pp. 49–59. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/w17-5106

  84. Wachsmuth, H., Stein, B., Ajjour, Y.: “PageRank” for argument relevance. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, pp. 1117–1127. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/e17-1105

  85. Wachsmuth, H., Syed, S., Stein, B.: Retrieval of the best counterargument without prior topic knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, pp. 241–251. Association for Computational Linguistics (2018). https://www.aclweb.org/anthology/P18-1023/

  86. Wang, W., He, Q.: A survey on emotional semantic image retrieval. In: International Conference on Image Processing (ICIP 2008), pp. 117–120. IEEE (2008). https://doi.org/10.1109/ICIP.2008.4711705

  87. Wu, A.: Learn more about what you see on google images. Google Blog (2020). https://support.google.com/webmasters/answer/114016

  88. Yanai, K.: Image collector: an image-gathering system from the world-wide web employing keyword-based search engines. In: International Conference on Multimedia and Expo (ICME 2001). IEEE (2001). https://doi.org/10.1109/ICME.2001.1237772

Download references

Acknowledgments

We are very grateful to the CLEF 2022 organizers and the Touché participants, who allowed this lab to happen. We also want to thank our volunteer annotators who helped to create the relevance and argument quality assessments and our reviewers for their valuable feedback on the participants’ notebooks.

This work was partially supported by the Deutsche Forschungsgemeinschaft (DFG) through the projects “ACQuA 2.0” (Answering Comparative Questions with Arguments; project number 376430233) and “OASiS: Objective Argument Summarization in Search” (grant WA 4591/3-1), all part of the priority program “RATIO: Robust Argumentation Machines” (SPP 1999), and the German Ministry for Science and Education (BMBF) through the project “Shared Tasks as an Innovative Approach to Implement AI and Big Data-based Applications within Universities (SharKI)” (grant FKZ 16DHB4021). We are also grateful to Jan Heinrich Reimer for developing the TARGER Python library and Erik Reuter for expanding a document collection for Task 2 with docT5query.

Author information

Authors and Affiliations

  1. Martin-Luther-Universität Halle-Wittenberg, Halle, Germany

    Alexander Bondarenko, Maik Fröbe & Matthias Hagen

  2. Bauhaus-Universität Weimar, Weimar, Germany

    Johannes Kiesel & Benno Stein

  3. Leipzig University, Leipzig, Germany

    Shahbaz Syed & Martin Potthast

  4. Paderborn University, Paderborn, Germany

    Timon Gurcke & Henning Wachsmuth

  5. Uppsala University, Uppsala, Sweden

    Meriem Beloucif

  6. Skolkovo Institute of Science and Technology, Moscow, Russia

    Alexander Panchenko

  7. Universität Hamburg, Hamburg, Germany

    Chris Biemann

Authors
  1. Alexander Bondarenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maik Fröbe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Johannes Kiesel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shahbaz Syed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Timon Gurcke
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Meriem Beloucif
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alexander Panchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Chris Biemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Benno Stein
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Henning Wachsmuth
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Martin Potthast
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Matthias Hagen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Bondarenko .

Editor information

Editors and Affiliations

  1. University of Bologna, Forlì, Italy

    Alberto Barrón-Cedeño

  2. University of Padua, Padova, Italy

    Giovanni Da San Martino

  3. University of Bologna, Bologna, Italy

    Mirko Degli Esposti

  4. Instituto di Scienza e Tecnologie dell' Informazione “Alessandro Faedo”, Pisa, Italy

    Fabrizio Sebastiani

  5. University of Glasgow, Glasgow, UK

    Craig Macdonald

  6. University Milano-Bicocca, Milan, Italy

    Gabriella Pasi

  7. TU Wien, Vienna, Austria

    Allan Hanbury

  8. Leipzig University, Leipzig, Germany

    Martin Potthast

  9. University of Padua, Padova, Italy

    Guglielmo Faggioli

  10. University of Padua, Padova, Italy

    Nicola Ferro

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bondarenko, A. et al. (2022). Overview of Touché 2022: Argument Retrieval. In: Barrón-Cedeño, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2022. Lecture Notes in Computer Science, vol 13390. Springer, Cham. https://doi.org/10.1007/978-3-031-13643-6_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-13643-6_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-13642-9

  • Online ISBN: 978-3-031-13643-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation