Abstract
Category systems play an essential role in knowledge bases by groupings of semantically related entities. Category generation task aims to produce category suggestions which can help knowledge editors to expand a category system. Most past research has focused on solving coarse-grained problems, not fine-grained scenarios. In this paper, we propose a two-stage framework to generate fine-grained categories for sets of entities. In the category generation stage, we extract conceptual texts from the context of entities and then employ the Seq2Seq model to generate candidate categories. In the category selection stage, we cluster the entities and design discrete patterns using entity names for prompt ranking, which are further ensembled to preserve the final categories. We construct a new fine-grained category generation dataset based on Wikipedia. Experimental results demonstrate the effectiveness of the framework over the state-of-the-art abstractive summarization methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In the experiment, the best result is achieved on 0.15.
- 2.
We determined a threshold of 0.85 which provided the best result.
- 3.
- 4.
- 5.
- 6.
- 7.
We calculate text similarity by using SequenceMatcher.ratio(). Prec@1 and Prec@5 indicate whether the top1 and top5 results include a ratio value of 1. Dist@1 and Dist@5 indicate the largest ratio among the top1 and top5 results.
References
Aiken, E.G., Thomas, G.S., Shennum, W.A.: Memory for a lecture: effects of notes, lecture rate, and informational density. J. Educ. Psychol. 67(3), 439 (1975)
Ben-David, E., Oved, N., Reichart, R.: Pada: a prompt-based autoregressive approach for adaptation to unseen domains. arXiv preprint arXiv:2102.12206 (2021)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)
Carlsson, F., Öhman, J., Liu, F., Verlinden, S., Nivre, J., Sahlgren, M.: Fine-grained controllable text generation using non-residual prompting. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6837–6857 (2022)
Choi, E., Levy, O., Choi, Y., Zettlemoyer, L.: Ultra-fine entity typing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 87–96 (2018)
Ding, N., et al.: Prompt-learning for fine-grained entity typing. arXiv preprint arXiv:2108.10604 (2021)
Han, X., et al.: Cross-lingual contrastive learning for fine-grained entity typing for low-resource languages. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2241–2250 (2022)
Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C (Appl. Stati.) 28(1), 100–108 (1979)
Hearst, M.A.: Automatic acquisition of hyponyms from large text corpora. In: COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics (1992)
Jelinek, F., Mercer, R.L., Bahl, L.R., Baker, J.K.: Perplexity-a measure of the difficulty of speech recognition tasks. J. Acoust. Soc. Am. 62(S1), S63–S63 (1977)
Kenton, J.D.M.W.C., Toutanova, L.K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)
Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3045–3059 (2021)
Lewis, M., et al.: Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871–7880 (2020)
Lin, C.Y.: Rouge: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81 (2004)
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. ACM Comput. Surv. 55(9), 1–35 (2023)
Liu, X., et al.: GPT understands, too. arXiv preprint arXiv:2103.10385 (2021)
Liu, Y., Shen, S., Lapata, M.: Noisy self-knowledge distillation for text summarization. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 692–703 (2021)
Min, E., Guo, X., Liu, Q., Zhang, G., Cui, J., Long, J.: A survey of clustering with deep learning: from the perspective of network architecture. IEEE Access 6, 39501–39514 (2018)
Mota, T., Sridharan, M.: Commonsense reasoning and knowledge acquisition to guide deep learning on robots. In: Robotics: Science and Systems (2019)
Obeidat, R., Fern, X., Shahbazi, H., Tadepalli, P.: Description-based zero-shot fine-grained entity typing. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 807–814 (2019)
Pang, K., Zhang, H., Zhou, J., Wang, T.: Divide and denoise: learning from noisy labels in fine-grained entity typing with cluster-wise loss correction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1997–2006 (2022)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)
Reddy, D.R., et al.: Speech understanding systems: a summary of results of the five-year research effort. Department of Computer Science. Camegie-Mell University, Pittsburgh, PA, vol. 17, p. 138 (1977)
Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Inf. Process. Manag. 24(5), 513–523 (1988)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073–1083 (2017)
Shen, J., Wu, Z., Lei, D., Shang, J., Ren, X., Han, J.: Setexpan: corpus-based set expansion via context feature selection and rank ensemble. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Dzeroski, S. (eds.) Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 288–304. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71249-9_18
Shi, T., Keneshloo, Y., Ramakrishnan, N., Reddy, C.K.: Neural abstractive text summarization with sequence-to-sequence models. ACM Trans. Data Sci. 2(1), 1–37 (2021)
Shin, T., Razeghi, Y., Logan IV, R.L., Wallace, E., Singh, S.: Autoprompt: eliciting knowledge from language models with automatically generated prompts. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222–4235 (2020)
Shleifer, S., Rush, A.M.: Pre-trained summarization distillation. arXiv preprint arXiv:2010.13002 (2020)
Wallace, E., Feng, S., Kandpal, N., Gardner, M., Singh, S.: Universal adversarial triggers for attacking and analyzing NLP. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2153–2162 (2019)
Zhang, J., Zhao, Y., Saleh, M., Liu, P.: Pegasus: pre-training with extracted gap-sentences for abstractive summarization. In: International Conference on Machine Learning, pp. 11328–11339. PMLR (2020)
Zhang, S., Balog, K., Callan, J.: Generating categories for sets of entities. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 1833–1842 (2020)
Zhang, Y., Shen, J., Shang, J., Han, J.: Empower entity set expansion via language model probing. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8151–8160 (2020)
Zhou, B., Khashabi, D., Tsai, C.T., Roth, D.: Zero-shot open entity typing as type-compatible grounding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2065–2076 (2018)
Acknowledgments
This work is supported by National Key R &D Program of China (2020AAA0105203).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Du, Y., Yu, J., Wan, J., Xu, J., Hou, L. (2024). Fine-Grained Category Generation for Sets of Entities. In: Song, X., Feng, R., Chen, Y., Li, J., Min, G. (eds) Web and Big Data. APWeb-WAIM 2023. Lecture Notes in Computer Science, vol 14334. Springer, Singapore. https://doi.org/10.1007/978-981-97-2421-5_26
Download citation
DOI: https://doi.org/10.1007/978-981-97-2421-5_26
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2420-8
Online ISBN: 978-981-97-2421-5
eBook Packages: Computer ScienceComputer Science (R0)