Skip to main content
SpringerLink
Log in

Overview of BioASQ 2023: The Eleventh BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

  • Conference paper
  • First Online:
Experimental IR Meets Multilinguality, Multimodality, and Interaction (CLEF 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14163))

Abstract

This is an overview of the eleventh edition of the BioASQ challenge in the context of the Conference and Labs of the Evaluation Forum (CLEF) 2023. BioASQ is a series of international challenges promoting advances in large-scale biomedical semantic indexing and question answering. This year, BioASQ consisted of new editions of the two established tasks b and Synergy, and a new task (MedProcNER) on semantic annotation of clinical content in Spanish with medical procedures, which have a critical role in medical practice. In this edition of BioASQ, 28 competing teams submitted the results of more than 150 distinct systems in total for the three different shared tasks of the challenge. Similarly to previous editions, most of the participating systems achieved competitive performance, suggesting the continuous advancement of the state-of-the-art in the field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://doi.org/10.5281/zenodo.7817745.

  2. 2.

    https://doi.org/10.5281/zenodo.7817666.

  3. 3.

    https://spacy.io/.

  4. 4.

    https://plantl.mineco.gob.es.

  5. 5.

    http://participants-area.bioasq.org/Tasks/b/eval_meas_2022/.

  6. 6.

    http://participants-area.bioasq.org/results/11b/phaseA/.

  7. 7.

    http://participants-area.bioasq.org/results/11b/phaseB/.

  8. 8.

    http://participants-area.bioasq.org/results/synergy_v2023/.

References

  1. Aksenova, A., Asamov, T., Boytcheva, S., Ivanov, P.: Improving Biomedical Question Answering with Sentence-Based Ranking at BioASQ-11b (2023)

    Google Scholar 

  2. Aksenova, A., Ivanov, P., Asamov, T., Boytcheva, S.: Leveraging biomedical ontologies for clinical procedures recognition in Spanish at BioASQ MedProcNER. In: Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum (2023)

    Google Scholar 

  3. Almeida, T., Jonker, R.A.A., Poudel, R., Silva, J.M., Matos, S.: BIT.UA at MedProcNER: discovering medical procedures in Spanish using transformer models with MCRF and augmentation. In: Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum (2023)

    Google Scholar 

  4. Almeida, T., Jonker, R.A.A., Poudel, R., Silva, J.M., Matos, S.: Two-stage IR with synthetic training and zero-shot answer generation at BioASQ 11 (2023)

    Google Scholar 

  5. Ateia, S.: Is ChatGPT a Biomedical Expert? - Exploring the Zero-Shot Performance of Current GPT Models in Biomedical Tasks (2023)

    Google Scholar 

  6. Baldwin, B., Carpenter, B.: Lingpipe (2003). Available from World Wide Web: http://alias-i.com/lingpipe

  7. Balikas, G., et al.: Evaluation framework specifications. Project deliverable D4.1, UPMC (2013)

    Google Scholar 

  8. Chizhikova, M., Collado-Montañez, J., Díaz-Galiano, M.C., Ureña-López, L.A., Martín-Valdivia, M.T.: Coming a long way with pre-trained transformers and string matching techniques: clinical procedure mention recognition and normalization. In: Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum (2023)

    Google Scholar 

  9. Croce, D., Castellucci, G., Basili, R.: GAN-BERT: generative adversarial learning for robust text classification with a bunch of labeled examples. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2114–2119 (2020)

    Google Scholar 

  10. Galat, D., Rizoiu, M.A.: Enhancing Biomedical Text Summarization and Question-Answering: On the Utility of Domain-Specific Pre-training (2023)

    Google Scholar 

  11. Gasco, L., et al.: Overview of BioASQ 2021-MESINESP track. Evaluation of advance hierarchical classification techniques for scientific literature, patents and clinical trials (2021)

    Google Scholar 

  12. Hsueh, C.Y., Zhang, Y., Lu, Y.W., Han, J.C., Meesawad, W., Tsai, R.T.H.: NCU-IISR: Prompt Engineering on GPT-4 to Stove Biological Problems in BioASQ 11b Phase B (2023)

    Google Scholar 

  13. Kim, H., Hwang, H., Lee, C., Seo, M., Yoon, W., Kang, J.: Exploration of Various Techniques in Biomedical Question Answering: From Pre-processing to GPT-4 (2023)

    Google Scholar 

  14. Krithara, A., Nentidis, A., Bougiatiotis, K., Paliouras, G.: BioASQ-QA: a manually curated corpus for biomedical question answering. Sci. Data 10(1), 170 (2023)

    Article  Google Scholar 

  15. Krithara, A., Nentidis, A., Paliouras, G., Krallinger, M., Miranda, A.: BioASQ at CLEF2021: large-scale biomedical semantic indexing and question answering. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12657, pp. 624–630. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72240-1_73

    Chapter  Google Scholar 

  16. Lesavourey, M., Hubert, G.: BioASQ 11B: integrating domain specific vocabulary to BERT-based model for biomedical document ranking (2023)

    Google Scholar 

  17. Lima-López, S., et al.: Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023. In: Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum (2023)

    Google Scholar 

  18. Liu, F., Shareghi, E., Meng, Z., Basaldella, M., Collier, N.: Self-alignment pretraining for biomedical entity representations. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4228–4238. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.naacl-main.334. https://aclanthology.org/2021.naacl-main.334

  19. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

  20. Miranda-Escalada, A., Farré-Maduell, E., Lima-López, S., Estrada, D., Gascó, L., Krallinger, M.: Mention detection, normalization & classification of species, pathogens, humans and food in clinical documents: overview of LivingNER shared task and resources. Procesamiento del Lenguaje Natural (2022)

    Google Scholar 

  21. Miranda-Escalada, A., et al.: Overview of DISTEMIST at BioASQ: automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources (2022)

    Google Scholar 

  22. Mollá, D.: Query-focused extractive summarisation for biomedical and Covid-19 complex question answering. In: 2022 Conference and Labs of the Evaluation Forum, CLEF 2022, pp. 305–314 (2022)

    Google Scholar 

  23. Nentidis, A., et al.: Overview of BioASQ 2021: the ninth BioASQ challenge on large-scale biomedical semantic indexing and question answering. In: Candan, K.S., et al. (eds.) CLEF 2021. LNCS, vol. 12880, pp. 239–263. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85251-1_18

    Chapter  Google Scholar 

  24. Nentidis, A., et al.: Overview of BioASQ 2022: the tenth BioASQ challenge on large-scale biomedical semantic indexing and question answering. In: Barrón-Cedeño, A., et al. (eds.) CLEF 2022. LNCS, vol. 13390, pp. 337–361. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-13643-6_22

    Chapter  Google Scholar 

  25. Nentidis, A., et al.: Overview of BioASQ 2020: the eighth BioASQ challenge on large-scale biomedical semantic indexing and question answering. In: Arampatzis, A., et al. (eds.) CLEF 2020. LNCS, vol. 12260, pp. 194–214. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58219-7_16

    Chapter  Google Scholar 

  26. Nentidis, A., Krithara, A., Paliouras, G., Farre-Maduell, E., Lima-Lopez, S., Krallinger, M.: BioASQ at CLEF2023: the eleventh edition of the large-scale biomedical semantic indexing and question answering challenge. In: Kamps, J., et al. (eds.) ECIR 2023. LNCS, vol. 13982, pp. 577–584. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-28241-6_66

    Chapter  Google Scholar 

  27. Nentidis, A., Krithara, A., Paliouras, G., Gasco, L., Krallinger, M.: BioASQ at CLEF2022: the tenth edition of the large-scale biomedical semantic indexing and question answering challenge. In: Hagen, M., et al. (eds.) ECIR 2022. LNCS, vol. 13186, pp. 429–435. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-99739-7_53

    Chapter  Google Scholar 

  28. Ozyurt, I.B.: End-to-end biomedical question answering via bio-answerfinder and discriminative language representation models (2021)

    Google Scholar 

  29. Panou, D., Reczko, M.: Semi-supervised training for biomedical question answering (2023)

    Google Scholar 

  30. R, R., Rauchwerk, J., Rajwade, P., Gummadi, T.: Biomedical Question Answering using Transformer Ensembling (2023)

    Google Scholar 

  31. Reczko, M.: ELECTROLBERT: combining replaced token detection and sentence order prediction. In: CLEF (Working Notes) (2022)

    Google Scholar 

  32. Rosso-Mateus, A., Montes-Y-Gómez, M., Munoz Serna, L.A., Gonzalez, F.: Deep Metric Learning for Effective Passage Retrieval in the BioASQ Challenge (2023)

    Google Scholar 

  33. Shin, A.D., Jin, Q., Lu, Z.: Multi-stage Literature Retrieval System Trained by PubMed Search Logs for Biomedical Question Answering (2023)

    Google Scholar 

  34. Tsatsaronis, G., et al.: An overview of the BioASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinform. 16, 138 (2015). https://doi.org/10.1186/s12859-015-0564-6

    Article  Google Scholar 

  35. Vassileva, S., Grazhdanski, G., Boytcheva, S., Koychev, I.: Fusion @ BioASQ MedProcNER: transformer-based approach for procedure recognition and linking in Spanish clinical text. In: Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum (2023)

    Google Scholar 

  36. Wei, C.H., Leaman, R., Lu, Z.: Beyond accuracy: creating interoperable and scalable text-mining web services. Bioinformatics (Oxford, England) 32(12), 1907–10 (2016). https://doi.org/10.1093/bioinformatics/btv760

    Article  Google Scholar 

  37. Yang, Z., Zhou, Y., Eric, N.: Learning to answer biomedical questions: OAQA at BioASQ 4b. In: ACL 2016, p. 23 (2016)

    Google Scholar 

  38. Zotova, E., García-Pablos, A., Cuadros, M., Rigau, G.: VICOMTECH at MedProcNER 2023: transformers-based sequence-labelling and cross-encoding for entity detection and normalisation in Spanish clinical texts. In: Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum (2023)

    Google Scholar 

Download references

Acknowledgments

Google was a proud sponsor of the BioASQ Challenge in 2022. The eleventh edition of BioASQ is also sponsored by Ovid. Atypon Systems Inc. is also sponsoring this edition of BioASQ. The MEDLINE/PubMed data resources considered in this work were accessed courtesy of the U.S. National Library of Medicine. BioASQ is grateful to the CMU team for providing the exact answer baselines for task 11b, as well as to Georgios Moschovis and Ion Androutsopoulos, from the Athens University of Economics and Business, for providing the ideal answer baselines. The MedProcNER track was partially funded by the Encargo of Plan TL (SEDIA) to the Barcelona Supercomputing Center. Due to the relevance of medical procedures for implants/devices specially in the case cardiac diseases this project is also supported by the European Union’s Horizon Europe Coordination & Support Action under Grant Agreement No 101058779 (BIOMATDB) and DataTools4Heart Grant Agreement No. 101057849. We also acknowledge the support from the AI4PROFHEALTH project (PID2020-119266RA-I00).

Author information

Authors and Affiliations

  1. National Center for Scientific Research “Demokritos”, Athens, Greece

    Anastasios Nentidis, Georgios Katsimpras, Anastasia Krithara & Georgios Paliouras

  2. Aristotle University of Thessaloniki, Thessaloniki, Greece

    Anastasios Nentidis

  3. Barcelona Supercomputing Center, Barcelona, Spain

    Salvador Lima López, Eulália Farré-Maduell, Luis Gasco & Martin Krallinger

Authors
  1. Anastasios Nentidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georgios Katsimpras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anastasia Krithara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salvador Lima López
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eulália Farré-Maduell
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Luis Gasco
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Martin Krallinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Georgios Paliouras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anastasios Nentidis .

Editor information

Editors and Affiliations

  1. Democritus University of Thrace, Xanthi, Greece

    Avi Arampatzis

  2. University of Amsterdam, Amsterdam, The Netherlands

    Evangelos Kanoulas

  3. CERTH-ITI, Thessaloniki, Greece

    Theodora Tsikrika

  4. CERTH-ITI, Thessaloniki, Greece

    Stefanos Vrochidis

  5. Utrecht University, Utrecht, The Netherlands

    Anastasia Giachanou

  6. Elsevier, Amsterdam, The Netherlands

    Dan Li

  7. University of Amsterdam, Amsterdam, The Netherlands

    Mohammad Aliannejadi

  8. University of Lausanne, Lausanne, Switzerland

    Michalis Vlachos

  9. University of Padua, Padova, Italy

    Guglielmo Faggioli

  10. University of Padua, Padova, Italy

    Nicola Ferro

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nentidis, A. et al. (2023). Overview of BioASQ 2023: The Eleventh BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering. In: Arampatzis, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2023. Lecture Notes in Computer Science, vol 14163. Springer, Cham. https://doi.org/10.1007/978-3-031-42448-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42448-9_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42447-2

  • Online ISBN: 978-3-031-42448-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation