Abstract
Automatic indexing is a challenging task in which computers must emulate the behaviour of professional indexers to assign to a document some keywords or keyphrases that represent concisely the content of the document. While most of the existing algorithms are based on a select-and-rank strategy, it has been shown that selecting only keywords from text is not ideal as human annotators tend to assign keywords that are not present in the source. This problem is more evident in scholarly literature. In this work we leverage a transformer-based language model to approach the automatic indexing task from a generative point of view. In this way we overcome the problem of keywords that are not in the original document, as the neural language models can rely on knowledge acquired during their training process. We apply our method to a French collection of annotated scientific articles.
Supported by: the AMIC-PoC project (PDC2021-120846-C44), funded by MCIN/AEI/10.13039/501100011033 and the European Union “NextGenerationEU”/“PRTR”, a public grant overseen by ANR as part of the program “Investissements d’Avenir” (reference: ANR-10-LABX-0083) and the Vicerrectorado de Investigación de la Universitat Politècnica de València (PAID-11-21).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ammar, W., Peters, M., Bhagavatula, C., Power, R.: The AI2 system at SemEval-2017 Task 10 (ScienceIE): semi-supervised end-to-end entity and relation extraction. In: Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval-2017, pp. 592–596 (2017)
Augenstein, I., Das, M., Riedel, S., Vikraman, L., McCallum, A.: SemEval 2017 Task 10: ScienceIE - extracting keyphrases and relations from scientific publications. In: Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval-2017, pp. 546–555 (2017)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015 (January 2015)
Boudin, F., Daille, B., Jacquey, É., Nie, J.: The DELICES project: indexing scientific literature through semantic expansion. arXiv arXiv:abs/2106.14731 (2020)
Bougouin, A., Boudin, F., Daille, B.: TopicRank: graph-based topic ranking for keyphrase extraction. In: International Joint Conference on Natural Language Processing (IJCNLP), pp. 543–551 (2013)
Cabanac, G., Frommholz, I., Mayr, P.: Scholarly literature mining with information retrieval and natural language processing: preface. Scientometrics 125(3), 2835–2840 (2020). https://doi.org/10.1007/s11192-020-03763-4
Campos, R., Mangaravite, V., Pasquali, A., Jorge, A., Nunes, C., Jatowt, A.: YAKE! keyword extraction from single documents using multiple local features. Inf. Sci. 509, 257–289 (2020)
Daille, B., Barreaux, S., Bougouin, A., Boudin, F., Cram, D., Hazem, A.: Automatic indexing of scientific papers presentation and results of DEFT 2016 text mining challenge. Inf. Retrieval Doc. Semant. Web 17(2), 1–17 (2017)
Eddine, M.K., Tixier, A.J.P., Vazirgiannis, M.: BARThez: a skilled pretrained French sequence-to-sequence model. arXiv preprint arXiv:2010.12321 (2020)
Hasan, K.S., Ng, V.: Automatic keyphrase extraction: a survey of the state of the art. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1262–1273 (2014)
Hernandez, S.D., Buscaldi, D., Charnois, T.: LIPN at SemEval-2017 Task 10: filtering candidate keyphrases from scientific publications with part-of-speech tag sequences to train a sequence labeling model. In: Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval-2017, pp. 995–999 (2017)
Huang, H., Wang, X., Wang, H.: NER-RAKE: an improved rapid automatic keyword extraction method for scientific literatures based on named entity recognition. Proc. Assoc. Inf. Sci. Technol. 57(1), 71–91 (2020)
Li, G., Wang, H.: Improved automatic keyword extraction based on TextRank using domain knowledge. In: Zong, C., Nie, J.-Y., Zhao, D., Feng, Y. (eds.) NLPCC 2014. CCIS, vol. 496, pp. 403–413. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45924-9_36
Lopez, P., Romary, L.: HUMB: automatic key term extraction from scientific articles in GROBID. In: Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden, July 2010, pp. 248–251. Association for Computational Linguistics (2010)
Lu, Y., Li, R., Wen, K., Lu, Z.: Automatic keyword extraction for scientific literatures using references. In: Proceedings of the 2014 International Conference on Innovative Design and Manufacturing (ICIDM), pp. 78–81. IEEE (2014)
Marchand, M., Fouquier, G., Marchand, E., Pitel, G.: Document vector embeddings for bibliographic records indexing. Inf. Retrieval Doc. Semant. Web 17(2) (2017)
Meng, R., Zhao, S., Han, S., He, D., Brusilovsky, P., Chi, Y.: Deep keyphrase generation. In: 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, ACL 2017. vol. 1, pp. 582–592 (2017)
Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 404–411 (2004)
Rose, S., Engel, D., Cramer, N., Cowley, W.: Automatic keyword extraction from individual documents. In: Text Mining: Applications and Theory, vol. 1, pp. 1–20 (2010)
Rousseau, F., Vazirgiannis, M.: Graph-of-word and TW-IDF: new approach to Ad Hoc IR. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 59–68 (2013)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada, July 2017, pp. 1073–1083. Association for Computational Linguistics (2017)
Spärck Jones, K.: A statistical interpretation of term specificity and its application in retrieval. J. Doc. 28, 11–21 (1972)
Turney, P.D.: Learning algorithms for keyphrase extraction. Inf. Retrieval 2(4), 303–336 (2000)
Wang, R., Liu, W., McDonald, C.: Using word embeddings to enhance keyword identification for scientific publications. In: Sharaf, M.A., Cheema, M.A., Qi, J. (eds.) ADC 2015. LNCS, vol. 9093, pp. 257–268. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19548-3_21
Xu, S., Li, H., Yuan, P., Wu, Y., He, X., Zhou, B.: Self-attention guided copy mechanism for abstractive summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, July 2020, pp. 1355–1362. Association for Computational Linguistics (2020). Online
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
González, J.Á., Buscaldi, D., Sanchis, E., Hurtado, LF. (2022). Transformer-Based Models for the Automatic Indexing of Scientific Documents in French. In: Rosso, P., Basile, V., Martínez, R., Métais, E., Meziane, F. (eds) Natural Language Processing and Information Systems. NLDB 2022. Lecture Notes in Computer Science, vol 13286. Springer, Cham. https://doi.org/10.1007/978-3-031-08473-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-08473-7_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08472-0
Online ISBN: 978-3-031-08473-7
eBook Packages: Computer ScienceComputer Science (R0)