ChEMU: Named Entity Recognition and Event Extraction of Chemical Reactions from Patents
- 3.8k Downloads
We introduce a new evaluation lab named ChEMU (Cheminformatics Elsevier Melbourne University), part of the 11th Conference and Labs of the Evaluation Forum (CLEF-2020). ChEMU involves two key information extraction tasks over chemical reactions from patents. Task 1—Named entity recognition—involves identifying chemical compounds as well as their types in context, i.e., to assign the label of a chemical compound according to the role which the compound plays within a chemical reaction. Task 2—Event extraction over chemical reactions—involves event trigger detection and argument recognition. We briefly present the motivations and goals of the ChEMU tasks, as well as resources and evaluation methodology.
KeywordsNamed entity recognition Event extraction Chemical reactions Patents
The chemical industry undoubtedly depends on the discovery of new chemical compounds. However, new chemical compounds are often initially disclosed in patent documents, and only a small fraction of these compounds are published in journals, usually taking an additional 1–3 years after the patent . Therefore, most chemical compounds are only available through patent documents . In addition, chemical patent documents contain unique information, such as reactions, experimental conditions, mode of action, which is essential for the understanding of compound prior art, providing a means for novelty checking and validation as well as pointers for chemical research in both academia and industry [1, 2]. As the number of new chemical patent applications has been drastically increasing , it is becoming crucial to develop natural language processing (NLP) approaches that enable automatic extraction of key information from the chemical patents .
In this paper, we propose a new evaluation lab (called ChEMU) focusing on information extraction over chemical reactions from patents. In particular, we will focus on two key information extraction tasks of chemical named entity recognition (NER) and chemical reaction event extraction. While previous related shared tasks focusing on chemicals or drugs such as CHEMDNER  have also included chemical named entity recognition as a task, those have primarily focused on PubMed abstracts. The CHEMDNER patents task  was limited to entity mentions and chemical entity passage detection, and only considered titles and abstracts of patents. For our ChEMU lab, we extend the existing corpora in several directions: first, we go beyond chemical NER to require labeling of the role of a chemical with respect to a reaction, and to consider complete chemical reactions in addition to entities. The ChEMU website is available at: http://chemu.eng.unimelb.edu.au.
2 Goals and Importance
What are the Goals of This Evaluation Lab? Our goals are: (1) To develop tasks that impact chemical research in both academia and industry, (2) To provide the community with a new dataset of chemical entities, enriched with relational links between chemical event triggers and arguments, and (3) To advance the state-of-the-art in information extraction over chemical patents.
Why is This Lab Needed? For evaluating information extraction developments in the scientific literature domain, there have been a large number of labs/shared tasks offered within previous i2b2/n2c2, SemEval, BioNLP, BioCreative, TREC and CLEF workshops. However, less attention has been paid to the chemical patent domain. In particular, there has previously been only one shared task on this domain, which is the CHEMDNER patents task at the BioCreative V workshop, involving detection of mentions of chemical compounds and genes/proteins in patent text .
Information extraction approaches developed for the scientific literature domain may not apply directly to the chemical patent domain. This is because as legal documents, patents are written very differently as compared to scientific literature. When writing scientific papers, authors strive to make their words as clear and straightforward as possible, whereas patent authors often seek to protect their knowledge from being fully disclosed . In tension with this is the need to claim broad scope for intellectual property reasons, and hence patents typically contain more details and are more exhaustive than scientific papers .
There are also a number of characteristics of patent texts that create challenges for NLP in this context. Long sentences listing names of compounds are frequently used in chemical patents. The structure of sentences in patent claims is usually complex, and syntactic parsing in patents can be difficult . A quantitative analysis from  showed that the average sentence length in a patent corpus is much longer than in general language use. That work also showed that the lexicon used in patents usually includes domain-specific and novel terms that are difficult to understand.
Brief definitions of ChEMU chemical entity types, organised into chemical entity types, a reaction label introduced in the text, and reaction properties.
A product is a substance that is formed during a chemical reaction
A substance that is consumed in the course of a chemical reaction providing atoms to products is considered as starting material
A reagent is a compound added to a system to cause or help with a chemical reaction. Compounds like catalysts, bases to remove protons or acids to add protons must be also annotated with this tag
A solvent is a chemical entity that dissolves a solute resulting in a solution
Other chemical compounds that are not the products, starting materials, reagents, catalysts and solvents
A label associated with a reaction specification
The temperature at which the reaction was carried out must be annotated with this tag
The reaction time of the reaction
Yield given in percent values
Yields provided in other units than %
The ChEMU lab at CLEF-20201 offers the two information extraction tasks of Named entity recognition (Task 1) and Event extraction (Task 2) over chemical reactions from patent documents. Teams may participate in one or both tasks.
3.1 Task 1: Named Entity Recognition
In general, a chemical reaction is a process leading to the transformation of one set of chemical substances to another . Task 1 involves identifying chemical compounds and their specific types, i.e. to assign the label of a chemical compound according to the role which it plays within a chemical reaction. In addition to chemical compounds, this task also requires identification of the temperatures and reaction times at which the chemical reaction is carried out, as well as yields obtained for the final chemical product and the label of the reaction.
3.2 Task 2: Event Extraction
As illustrated in Figs. 1 and 2, a chemical reaction leading to an end product often consists of a sequence of individual event steps. Task 2 is to identify those steps which involve chemical entities recognized from Task 1. Unlike a conventional event extraction problem  which involves event trigger word detection, event typing and argument prediction, our Task 2 requires identification of event trigger words (e.g. “added” and “stirred”) which all have the same type of “EVENT_TRIGGER”, and then determination of the chemical entity arguments of these events.2
An end-to-end process incorporating both Task 1 and Task 2 can be equivalently viewed as a relation extraction task which identifies 11 entity types including 10 types defined in Table 1 plus “EVENT_TRIGGER”, and extracts relations between the “EVENT_TRIGGER” entities and the remaining entities.
4 Data and Evaluation
Data: For system development and evaluation, a new corpus of 1500 chemical reaction snippets will be provided for both tasks (an example of a chemical reaction snippet is shown in Table 2). These snippets are sampled from 170 English document patents from the European Patent Office and the United States Patent and Trademark Office. We will mark up every chemical compound or event trigger with both text spans and IDs, and highlight relations and event arguments, as illustrated in Figs. 1 and 2. We have begun preparing the corpus and will make available strong baselines for the tasks. Initial publications related to the data and Task 1 appear at the 2019 ALTA and BioNLP workshops, respectively [18, 19].
The corpus will be split into 70%/10%/20% training/development/test. Gold annotations for the training and development sets will be provided to task participants in the BRAT standoff format  during the development phase. The raw test set will be provided for final test phase.
To support teams who are interested in Task 2 only, a pre-trained chemical NER tagger is provided as a resource .
In this paper, we have presented a brief description of the upcoming ChEMU lab at CLEF-2020. ChEMU will focus on two new tasks of named entity recognition and event extraction over chemical reactions from patents. We expect participants from both academia and industry. We will advertise our ChEMU lab via social media as well as NLP-related mailing lists.
This work is supported by an Australian Research Council Linkage Project, LP160101469, and Elsevier. We would like to thank Estrid He, Zubair Afzal and Mark Sheehan for supporting this work, as well as the anonymous reviewers for their feedback.
- 4.Hu, M., Cinciruk, D., Walsh, J.M.: Improving automated patent claim parsing: dataset, system, and experiments. CoRR abs/1605.01744 (2016)Google Scholar
- 5.Jurafsky, D., Martin, J.H.: Semantic Role Labeling and Argument Structure. In: Speech and Language Processing, 3rd edn. (2019)Google Scholar
- 6.Kim, J.D., Ohta, T., Pyysalo, S., Kano, Y., Tsujii, J.: Overview of BioNLP’09 shared task on event extraction. In: Proceedings of the BioNLP 2009 Workshop Companion Volume for Shared Task, pp. 1–9 (2009)Google Scholar
- 8.Krallinger, M., et al.: Overview of the CHEMDNER patents task. In: Proceedings of the Fifth BioCreative Challenge Evaluation Workshop, pp. 63–75 (2015)Google Scholar
- 14.Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., Tsujii, J.: brat: a web-based tool for NLP-assisted text annotation. In: Proceedings of the Demonstrations Session at EACL 2012 (2012)Google Scholar
- 16.Verberne, S., D’hondt, E., Oostdijk, N., Koster, C.: Quantifying the challenges in parsing patent claims. In: Proceedings of the 1st International Workshop on Advances in Patent Information Retrieval at ECIR 2010, pp. 14–21 (2010)Google Scholar
- 18.Yoshikawa, H., et al.: Detecting chemical reactions in patents. In: Proceedings of the 17th Annual Workshop of the Australasian Language Technology Association, pp. 100–110 (2019)Google Scholar
- 19.Zhai, Z., et al.: Improving chemical named entity recognition in patents with contextualized word embeddings. In: Proceedings of the 18th BioNLP Workshop, pp. 328–338 (2019)Google Scholar