Skip to main content
SpringerLink
Log in
Book cover

International Conference of the Cross-Language Evaluation Forum for European Languages

CLEF 2022: Experimental IR Meets Multilinguality, Multimodality, and Interaction pp 447–469Cite as

Overview of JOKER@CLEF 2022: Automatic Wordplay and Humour Translation Workshop

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13390))

Abstract

While humour and wordplay are among the most intensively studied problems in the field of translation studies, they have been almost completely ignored in machine translation. This is partly because most AI-based translation tools require a quality and quantity of training data (e.g., parallel corpora) that has historically been lacking for humour and wordplay. The goal of the JOKER@CLEF 2022 workshop was to bring together translators and computer scientists to work on an evaluation framework for wordplay, including data and metric development, and to foster work on automatic methods for wordplay translation. To this end, we defined three pilot tasks: (1) classify and explain instances of wordplay, (2) translate single terms containing wordplay, and (3) translate entire phrases containing wordplay (punning jokes). This paper describes and discusses each of these pilot tasks, as well as the participating systems and their results.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://www.joker-project.com/pun-translation-contest/.

  2. 2.

    Unlike in the SemEval-2017 task, we simply list the word(s) in question rather than indicating their position within the instance.

  3. 3.

    https://www.nltk.org/_modules/nltk/stem/wordnet.html.

  4. 4.

    See Sect. 7 for further explanation.

  5. 5.

    On closer inspection, we determined that Example 12 was very close to an example from a train set.

References

  1. Blinov, V., Bolotova-Baranova, V., Braslavski, P.: Large dataset and language model fun-tuning for humor recognition. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4027–4032. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/P19-1394

  2. Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

  3. Castro, S., Chiruzzo, L., Rosá, A., Garat, D., Moncecchi, G.: A crowd-annotated Spanish corpus for humor analysis. In: Proceedings of the Sixth International Workshop on Natural Language Processing for Social Media, pp. 7–11. Association for Computational Linguistics, July 2018. https://doi.org/10.18653/v1/W18-3502. https://www.aclweb.org/anthology/W18-3502

  4. Cattle, A., Ma, X.: Recognizing humour using word associations and humour anchor extraction. In: Proceedings of the 27th International Conference on Computational Linguistics, Association for Computational Linguistics, Santa Fe, New Mexico, USA, pp. 1849–1858 (2018). https://www.aclweb.org/anthology/C18-1157

  5. Delabastita, D.: There’s a Double Tongue: an Investigation into the Translation of Shakespeare’s Wordplay, with Special Reference to Hamlet. Rodopi, Amsterdam (1993)

    Google Scholar 

  6. Delabastita, D.: Introduction to the special issue on wordplay and translation. Translator: Stud. Intercultural Commun. 2(2), 1–22 (1996). https://doi.org/10.1080/13556509.1996.10798970

    Article  Google Scholar 

  7. Delabastita, D.: Wordplay as a translation problem: a linguistic perspective. In: Ein internationales Handbuch zur Übersetzungsforschung, vol. 1, pp. 600–606. De Gruyter Mouton, July 2008. https://doi.org/10.1515/9783110137088.1.6.600

  8. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, vol. 1, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/n19-1423

  9. Ermakova, L., et al.: CLEF workshop JOKER: automatic wordplay and humour translation. In: Hagen, M., et al. (eds.) ECIR 2022. LNCS, vol. 13186, pp. 355–363. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-99739-7_45

    Chapter  Google Scholar 

  10. Ermilov, A., Murashkina, N., Goryacheva, V., Braslavski, P.: Stierlitz meets SVM: humor detection in Russian. In: Ustalov, D., Filchenkov, A., Pivovarova, L., Žižka, J. (eds.) AINL 2018. CCIS, vol. 930, pp. 178–184. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01204-5_17

    Chapter  Google Scholar 

  11. Faggioli, G., Ferro, N., Hanbury, A., Potthast, M. (eds.): Proceedings of the Working Notes of CLEF 2022: Conference and Labs of the Evaluation Forum. CEUR Workshop Proceedings. CEUR-WS.org (2022)

    Google Scholar 

  12. Farwell, D., Helmreich, S.: Pragmatics-based MT and the translation of puns. In: Proceedings of the 11th Annual Conference of the European Association for Machine Translation, pp. 187–194, June 2006. http://www.mt-archive.info/EAMT-2006-Farwell.pdf

  13. Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  14. Francesconi, C., Bosco, C., Poletto, F., Sanguinetti, M.: Error analysis in a hate speech detection task: the case of HaSpeeDe-TW at EVALITA 2018. In: Bernardi, R., Navigli, R., Semeraro, G. (eds.) Proceedings of the 6th Italian Conference on Computational Linguistics, November 2018. http://ceur-ws.org/Vol-2481/paper32.pdf

  15. Ghanem, B., Karoui, J., Benamara, F., Moriceau, V., Rosso, P.: IDAT@FIRE2019: overview of the track on irony detection in Arabic tweets. In: Proceedings of the 11th Forum for Information Retrieval Evaluation, pp. 10–13. Association for Computing Machinery (2019). https://doi.org/10.1145/3368567.3368585

  16. Giorgadze, M.: Linguistic features of pun, its typology and classification. Eur. Sci. J. 10(10) (2014). https://eujournal.org/index.php/esj/article/view/4819

  17. Gottlieb, H.: You got the picture? On the polysemiotics of subtitling wordplay. In: Delabastita, D. (ed.) Traductio: Essays on Punning and Translation, pp. 207–232. St. Jerome, Manchester (1997)

    Google Scholar 

  18. Guibon, G., Ermakova, L., Seffih, H., Firsov, A., Le Noé-Bienvenu, G.: Multilingual fake news detection with satire. In: CICLing: International Conference on Computational Linguistics and Intelligent Text Processing, La Rochelle, France, April 2019. https://halshs.archives-ouvertes.fr/halshs-02391141

  19. Hempelmann, C.F., Miller, T.: Puns: taxonomy and phonology. In: Attardo, S. (ed.) The Routledge Handbook of Language and Humor. Routledge Handbooks in Linguistics, pp. 95–108. Routledge, New York, February 2017. https://doi.org/10.4324/9781315731162-8

  20. Hong, B.A., Ong, E.: Automatically extracting word relationships as templates for pun generation. In: Computational Approaches to Linguistic Creativity: Proceedings of the Workshop, pp. 24–31. Association for Computational Linguistics, June 2009

    Google Scholar 

  21. Hossain, N., Krumm, J., Gamon, M., Kautz, H.: SemEval-2020 task 7: assessing humor in edited news headlines. In: Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp. 746–758. International Committee for Computational Linguistics, December 2020. https://aclanthology.org/2020.semeval-1.98

  22. Jain, A., Yadav, P., Javed, H.: Equivoque: detection and interpretation of English puns. In: Proceedings of the 8th International Conference System Modeling and Advancement in Research Trends, pp. 262–265 (2019). https://doi.org/10.1109/SMART46866.2019.9117433

  23. Jiang, C., Maddela, M., Lan, W., Zhong, Y., Xu, W.: Neural CRF model for sentence alignment in text simplification. arXiv:2005.02324 [cs], June 2020

  24. Karoui, J., Benamara, F., Moriceau, V., Patti, V., Bosco, C., Aussenac-Gilles, N.: Exploring the impact of pragmatic phenomena on irony detection in tweets: a multilingual corpus study. In: 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 1, pp. 262–272. Association for Computational Linguistics (2017). https://oatao.univ-toulouse.fr/18921/

  25. Karoui, J., Farah, B., Moriceau, V., Aussenac-Gilles, N., Hadrich-Belguith, L.: Towards a contextual pragmatic model to detect irony in tweets. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, vol. 2, pp. 644–650. Association for Computational Linguistics (2015). https://doi.org/10.3115/v1/P15-2106. http://aclweb.org/anthology/P15-2106

  26. Kolb, W., Miller, T.: Human-computer interaction in pun translation. In: Hadley, J., Taivalkoski-Shilov, K., Teixeira, C.S.C., Toral, A. (eds.) Using Technologies for Creative-Text Translation. Routledge (2022, to appear)

    Google Scholar 

  27. Lieber, O., Sharir, O., Lentz, B., Shoham, Y.: Jurassic-1: technical details and evaluation. White paper, AI21 Labs, August 2021. https://uploads-ssl.webflow.com/60fd4503684b466578c0d307/61138924626a6981ee09caf6_jurassic_tech_paper.pdf

  28. Meaney, J.A., Wilson, S., Chiruzzo, L., Lopez, A., Magdy, W.: SemEval-2021 task 7: HaHackathon, detecting and rating humor and offense. In: Proceedings of the 15th International Workshop on Semantic Evaluation, pp. 105–119. Association for Computational Linguistics, August 2021. https://doi.org/10.18653/v1/2021.semeval-1.9. https://aclanthology.org/2021.semeval-1.9

  29. Mihalcea, R., Strapparava, C.: Making computers laugh: investigations in automatic humor recognition. In: Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing: Proceedings of the Conference, Stroudsburg, PA, pp. 531–538. Association for Computational Linguistics, October 2005. https://doi.org/10.3115/1220575.1220642. http://www.aclweb.org/anthology/H/H05/H05-1067

  30. Miller, T.: The punster’s amanuensis: the proper place of humans and machines in the translation of wordplay. In: Proceedings of the Second Workshop on Human-Informed Translation and Interpreting Technology, pp. 57–64, September 2019. https://doi.org/10.26615/issn.2683-0078.2019_007

  31. Miller, T., Hempelmann, C.F., Gurevych, I.: SemEval-2017 task 7: detection and interpretation of English puns. In: Proceedings of the 11th International Workshop on Semantic Evaluation, pp. 58–68, August 2017. https://doi.org/10.18653/v1/S17-2005

  32. Nijholt, A., Niculescu, A., Valitutti, A., Banchs, R.E.: Humor in human-computer interaction: a short survey. In: Proceedings of INTERACT 2017 (2017)

    Google Scholar 

  33. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002). https://doi.org/10.3115/1073083.1073135. https://www.aclweb.org/anthology/P02-1040

  34. Potash, P., Romanov, A., Rumshisky, A.: SemEval-2017 task 6: #HashtagWars: learning a sense of humor. In: Proceedings of the 11th International Workshop on Semantic Evaluation, pp. 49–57. Association for Computational Linguistics, August 2017. https://doi.org/10.18653/v1/S17-2004

  35. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. Technical report (2019). https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

  36. Reyes, A., Buscaldi, D., Rosso, P.: An analysis of the impact of ambiguity on automatic humour recognition. In: Matoušek, V., Mautner, P. (eds.) TSD 2009. LNCS (LNAI), vol. 5729, pp. 162–169. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04208-9_25

    Chapter  Google Scholar 

  37. Reyes, A., Rosso, P., Buscaldi, D.: From humor recognition to irony detection: the figurative language of social media. Data Knowl. Eng. 74, 1–12 (2012). https://doi.org/10.1016/j.datak.2012.02.005

    Article  Google Scholar 

  38. Valitutti, A., Toivonen, H., Doucet, A., Toivanen, J.M.: “Let everything turn well in your wife”: generation of adult humor using lexical constraints. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 243–248. Association for Computational Linguistics, August 2013. https://aclanthology.org/P13-2044

  39. Vaswani, A., et al.: Attention is all you need. arXiv:1706.03762 [cs], December 2017

  40. Viennot, E.: Le langage inclusif: pourquoi, comment. Les Éditions iXe (2020)

    Google Scholar 

  41. Vrticka, P., Black, J.M., Reiss, A.L.: The neural basis of humour processing. Nat. Rev. Neurosci. 14(12), 860–868 (2013). https://doi.org/10.1038/nrn3566

    Article  Google Scholar 

  42. Xue, L., et al.: mT5: a massively multilingual pre-trained text-to-text transformer. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 483–498. Association for Computational Linguistics, June 2021. https://doi.org/10.18653/v1/2021.naacl-main.41. https://aclanthology.org/2021.naacl-main.41

  43. Yang, D., Lavie, A., Dyer, C., Hovy, E.: Humor recognition and humor anchor extraction. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2367–2376. Association for Computational Linguistics, September 2015. https://doi.org/10.18653/v1/D15-1284. https://www.aclweb.org/anthology/D15-1284

  44. Yu, Z., Tan, J., Wan, X.: A neural approach to pun generation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 1650–1660. Association for Computational Linguistics, July 2018. https://doi.org/10.18653/v1/P18-1153. https://aclanthology.org/P18-1153

Download references

This work has been funded in part by the National Research Agency under the program Investissements d’avenir (Reference ANR-19-GURE-0001) and by the Austrian Science Fund under project M 2625-N31. JOKER is supported by La Maison des sciences de l’homme en Bretagne. We thank Orlane Puchalski, Adrien Couaillet, Ludivine Grégoire and Paul Campen for data collection as well as Eric Sanjuan for providing a server. We also thank the PC members: Monika Bokiniec, Ġorġ Mallia, Gordan Matas, Mohamed Saki, Alain Kerhervé, Grigori Sidorov, Victor Manuel Palma Preciado, Fabrice Antoine, and Danica Škara.

Author information

Authors and Affiliations

  1. Université de Bretagne Occidentale, HCTI, 29200, Brest, France

    Liana Ermakova, Élise Mathurin, Julien Boccou, Albin Digue, Aurianne Damoy & Benoît Jeanjean

  2. Maison des sciences de l’homme en Bretagne, 35043, Rennes, France

    Liana Ermakova, Élise Mathurin, Gaëlle Le Corre, Sílvia Araújo, Radia Hannachi & Benoît Jeanjean

  3. Austrian Research Institute for Artificial Intelligence, Vienna, Austria

    Tristan Miller

  4. Dipartimento DILL, Universitá degli Studi di Udine, 33100, Udine, Italy

    Fabio Regattin

  5. École Nationale d’Ingénieurs de Brest, Lab-STICC CNRS UMR 6285, Brest, France

    Anne-Gwenn Bosser

  6. University of Malta, Msida, MSD 2020, Malta

    Claudine Borg

  7. Université de Bretagne Occidentale, CRBC, 29200, Brest, France

    Gaëlle Le Corre

  8. Universidade do Minho, CEHUM, 4710-057, Braga, Portugal

    Sílvia Araújo

  9. Université de Bretagne Sud, HCTI, 56321, Lorient, France

    Radia Hannachi

Authors
  1. Liana Ermakova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tristan Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabio Regattin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anne-Gwenn Bosser
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Claudine Borg
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Élise Mathurin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gaëlle Le Corre
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sílvia Araújo
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Radia Hannachi
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Julien Boccou
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Albin Digue
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Aurianne Damoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Benoît Jeanjean
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liana Ermakova .

Editor information

Editors and Affiliations

  1. University of Bologna, Forlì, Italy

    Alberto Barrón-Cedeño

  2. University of Padua, Padova, Italy

    Giovanni Da San Martino

  3. University of Bologna, Bologna, Italy

    Mirko Degli Esposti

  4. Instituto di Scienza e Tecnologie dell' Informazione “Alessandro Faedo”, Pisa, Italy

    Fabrizio Sebastiani

  5. University of Glasgow, Glasgow, UK

    Craig Macdonald

  6. University Milano-Bicocca, Milan, Italy

    Gabriella Pasi

  7. TU Wien, Vienna, Austria

    Allan Hanbury

  8. Leipzig University, Leipzig, Germany

    Martin Potthast

  9. University of Padua, Padova, Italy

    Guglielmo Faggioli

  10. University of Padua, Padova, Italy

    Nicola Ferro

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ermakova, L. et al. (2022). Overview of JOKER@CLEF 2022: Automatic Wordplay and Humour Translation Workshop. In: Barrón-Cedeño, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2022. Lecture Notes in Computer Science, vol 13390. Springer, Cham. https://doi.org/10.1007/978-3-031-13643-6_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-13643-6_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-13642-9

  • Online ISBN: 978-3-031-13643-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation