Abstract
Named Entity Recognition (NER) serves as the foundation for several natural language applications like question answering, chatbots and intent classification. Identification of entity boundaries and its categorization into entity types poses a significant challenge in domain-dependent and low-resource settings, with limited training data availability. To this end, we propose AtEnA, a novel NER framework utilizing entity class attributes from external knowledge source for few-shot learning. We use a two-stage fine-tuning process, wherein a language model is initially trained to “attend” to the different entity class attributes along with the textual context, and is then fine-tuned for the downstream application data with few annotated training examples. Experiments on benchmark NER datasets depict AtEnA to perform around 10 F1 score points better than the existing NER methodologies, specifically for few-shot limited training scenarios.
Work done while the author was at Huawei Research, Ireland
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aliod, D.M., van Zaanen, M., Smith, D.: Named entity recognition for question answering. In: ALTA, pp. 51–58 (2006)
Aone, C., Halverson, L., Hampton, T., Ramos-Santacruz, M.: SRA: description of the IE2 system used for MUC-7. In: MUC-7 (1998)
Aone, C., Okurowski, M.E., Gorlinsky, J.: A trainable summarizer with knowledge acquired from robust NLP techniques, pp. 71–80 (2022)
Bocklisch, T., Faulkner, J., Pawlowski, N., Nichol, A.: Rasa: open source language understanding and dialogue management. In: NIPS Workshop on Conversational AI (2017)
Bouarroudj, W., Boufaida, Z., Bellatreche, L.: Named entity disambiguation in short texts over knowledge graphs. Knowl. Inf. Syst. 64, 325–351 (2022)
Cheng, P., Erk, K.: Attending to entities for better text understanding. In: AAAI, pp. 7554–7561 (2020)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Eddy, S.R.: Hidden markov models. Curr. Opin. Struct. Biol. 6(3), 361–365 (1996)
Etzioni, O., Cafarella, M., Downey, D., Popescu, A.M., Shaked, T., Soderland, S., Weld, D.S., Yates, A.: Unsupervised named entity extraction from the web: an experimental study. Artif. Intell. 165(1), 91–134 (2005)
Goyal, A., Gupta, V., Kumar, M.: Recent named entity recognition and classification techniques: a systematic review. Comput. Sci. Rev. 29, 21–43 (2018)
Hearst, M.A., Dumais, S.T., Osuna, E., Platt, J., Scholkopf, B.: Support vector machines. IEEE Intell. Syst. & Their Appl. 13(4), 18–28 (1998)
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging (2015). arXiv:1508.01991
Krupka, G., IsoQuest, K.: Description of the NetOWL extractor system as used for MUC-7. In: MUC-7, pp. 21–28 (2005)
Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: ICML, pp. 282–289 (2001)
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: NAACL, pp. 260–270 (2016)
Li, J., Sun, A., Han, J., Li, C.: A Survey on Deep Learning for Named Entity Recognition. IEEE Trans. Knowl, Data Eng (2020)
Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., Li, J.: A unified MRC framework for named entity recognition. In: ACL, pp. 5849–5859 (2020)
Lin, B.Y., Xu, F., Luo, Z., Zhu, K.: Multi-channel BiLSTM-CRF model for emerging named entity recognition in social media. In: W-NUT, pp. 160–165 (2017)
Liu, W., Zhou, P., Zhao, Z., Wang, Z., Qi, J., Deng, H.: and Ping Wang. Enabling Language Representation with Knowledge Graph. In: AAAI, K-BERT (2020)
Nadeau, D., Sekine, S.: A survey of named entity recognition and classification. Lingvisticae Investig. 30(1), 3–26 (2007)
Panchendrarajan, R., Amaresan, A.: Bidirectional LSTM-CRF for named entity recognition. In: PACLIC, pp. 531–540 (2018)
Peters, M.E., Ammar, W., Bhagavatula, C., Power, R.: Semisupervised sequence tagging with bidirectional language models. In: ACL, pp. 1756–1765 (2017)
Peters, M.E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., Smith, N.A.: Knowledge enhanced contextual word representations. In: EMNLP-IJCNLP, pp. 43–54 (2019)
Ri, R., Yamada, I., Tsuruoka, Y.: mLUKE: the power of entity representations in multilingual pretrained language models. In: ACL, pp. 7316–7330 (2022)
Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition (2003). arXiv:cs/0306050
Swietojanski, P., Liu, X., Eshghi, A., Rieser, V.: Benchmarking natural language understanding services for building conversational agents. In: Proceedings of the Tenth International Workshop on Spoken Dialogue Systems Technology (IWSDS), Ortigia, Siracusa (SR), Italy, April 2019. Springer
Yamada, I., Asai, A., Shindo, H., Takeda, H., Matsumoto, Y.: LUKE: deep contextualized entity representations with entity-aware self-attention. In: EMNLP, pp. 6442–6454 (2020)
Yosef, M.A., Hoffart, J., Bordino, I., Spaniol, M., Weikum, G.: AIDA: an online tool for accurate disambiguation of named entities in text and tables. Proc. VLDB Endow. 4(12), 1450–1453 (2011)
Zhang, S., Elhadad, N.: Unsupervised biomedical named entity recognition: experiments with clinical and biological texts. J. Biomed. Inform. 46(6), 1088–1098 (2013)
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., Liu, Q.: ERNIE: enhanced language representation with informative entities. In: ACL, pp. 1441–1451 (2019)
Zhou, G., Su, J.: Named entity recognition using an HMM based chunk tagger. In: ACL, pp. 473–480 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Patel, R.N., Dutta, S., Assem, H. (2024). Attending to Entity Class Attributes for Named Entity Recognition with Few-Shot Learning. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2023. Lecture Notes in Networks and Systems, vol 824. Springer, Cham. https://doi.org/10.1007/978-3-031-47715-7_57
Download citation
DOI: https://doi.org/10.1007/978-3-031-47715-7_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47714-0
Online ISBN: 978-3-031-47715-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)