Skip to main content
SpringerLink
Log in

Knowledge enhanced graph inference network based entity-relation extraction and knowledge graph construction for industrial domain

  • Research Article
  • Published:
Frontiers of Engineering Management Aims and scope Submit manuscript

Abstract

With the escalating complexity in production scenarios, vast amounts of production information are retained within enterprises in the industrial domain. Probing questions of how to meticulously excavate value from complex document information and establish coherent information links arise. In this work, we present a framework for knowledge graph construction in the industrial domain, predicated on knowledge-enhanced document-level entity and relation extraction. This approach alleviates the shortage of annotated data in the industrial domain and models the interplay of industrial documents. To augment the accuracy of named entity recognition, domain-specific knowledge is incorporated into the initialization of the word embedding matrix within the bidirectional long short-term memory conditional random field (BiLSTM-CRF) framework. For relation extraction, this paper introduces the knowledge-enhanced graph inference (KEGI) network, a pioneering method designed for long paragraphs in the industrial domain. This method discerns intricate interactions among entities by constructing a document graph and innovatively integrates knowledge representation into both node construction and path inference through TransR. On the application stratum, BiLSTM-CRF and KEGI are utilized to craft a knowledge graph from a knowledge representation model and Chinese fault reports for a steel production line, specifically SPOnto and SPFRDoc. The F1 value for entity and relation extraction has been enhanced by 2% to 6%. The quality of the extracted knowledge graph complies with the requirements of real-world production environment applications. The results demonstrate that KEGI can profoundly delve into production reports, extracting a wealth of knowledge and patterns, thereby providing a comprehensive solution for production management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability Statements The datasets generated and analyzed within the current study are available from China Baowu Steel Group Corporation Limited. Restrictions apply to the availability of steel production data, which were used under license for this study. Steel production data are available from the corresponding author with the permission of China Baowu Steel Group Corporation Limited.

References

  • Bahdanau D, Cho K, Bengio Y (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint. arXiv: 1409.0473

  • Cao P, Chen Y, Liu K, Zhao J, Liu S (2018). Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Brussels: ACL, 182–192

    Google Scholar 

  • Christopoulou F, Miwa M, Ananiadou S (2019). Connecting the dots: Document-level neural relation extraction with edge-oriented graphs. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing / 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong: ACL, 4925–4936

    Google Scholar 

  • Deng J, Wang T, Wang Z, Zhou J, Cheng L (2022). Research on event logic knowledge graph construction method of robot transmission system fault diagnosis. IEEE Access, 10: 17656–17673

    Article  Google Scholar 

  • Dong J, Wang J, Chen S (2021). Knowledge graph construction based on knowledge enhanced word embedding model in manufacturing domain. Journal of Intelligent & Fuzzy Systems, 41(2): 3603–3613

    Article  Google Scholar 

  • Gui W, Zeng Z, Chen X, Xie Y, Sun Y (2020). Knowledge-driven process industry smart manufacturing. Scientia Sinica Informationis, 50(9): 1345–1360 (in Chinese)

    Article  Google Scholar 

  • Guo Z, Zhang Y, Lu W (2019). Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 241–251

    Chapter  Google Scholar 

  • Han X, Gao T, Lin Y, Peng H, Yang Y, Xiao C, Liu Z, Li P, Zhou J, Sun M (2020). More data, more relations, more context and more openness: A review and outlook for relation extraction. In: Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing. Suzhou: ACL, 745–758

    Google Scholar 

  • Hu L, Wu G, Xing Y, Wang F (2020). Things2Vec: Semantic modeling in the Internet of Things with graph representation learning. IEEE Internet of Things Journal, 7(3): 1939–1948

    Article  Google Scholar 

  • Huang Z, Xu W, Yu K (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint. arXiv:1508.01991

  • Huet A, Pinquié R, Véron P, Mallet A, Segonds F (2021). CACDA: A knowledge graph for a context-aware cognitive design assistant. Computers in Industry, 125: 103377

    Article  Google Scholar 

  • Kamble S, Gunasekaran A, Gawankar S (2018). Sustainable Industry 4.0 framework: A systematic literature review identifying the current trends and future perspectives. Process Safety and Environmental Protection, 117: 408–425

    Article  CAS  Google Scholar 

  • Kipf T N, Welling M (2016). Semi-supervised classification with graph convolutional networks. arXiv preprint. arXiv:1609.02907

  • Lafferty J, McCallum A, Pereira F (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann Publishers Inc., 282–289

    Google Scholar 

  • Li G, Pan R, Mao J, Cao Y (2020). Entity recognition of Chinese electronic medical records based on BiLSTM-CRF network and dictionary resources. Journal of Modern Information, 40(4): 3–12, 58 (in Chinese)

    CAS  Google Scholar 

  • Li J, Sun A, Han J, Li C (2022). A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, 34(1): 50–70

    Article  Google Scholar 

  • Lin H, Liu Y, Wang W, Yue Y, Lin Z (2017). Learning entity and relation embeddings for knowledge resolution. Procedia Computer Science, 108: 345–354

    Article  Google Scholar 

  • Lin Y, Tsai T, Chou W, Wu K, Sung T, Hsu W (2004). A maximum entropy approach to biomedical named entity recognition. In: Proceedings of the 4th International Conference on Data Mining in Bioinformatics. Seattle WA: Springer-Verlag, 56–61

    Google Scholar 

  • Liu M, Li X, Li J, Liu Y, Zhou B, Bao J (2022). A knowledge graph-based data representation approach for IIoT-enabled cognitive manufacturing. Advanced Engineering Informatics, 51: 101515

    Article  Google Scholar 

  • Liu W, Zhou P, Zhao Z, Wang Z, Ju Q, Deng H, Wang P (2020). K-BERT: Enabling language representation with knowledge graph. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, NY: AAAI, 2901–2908

    Google Scholar 

  • Lyu Z, Wu Y, Lai J, Yang M, Li C, Zhou W (2023). Knowledge enhanced graph neural networks for explainable recommendation. IEEE Transactions on Knowledge and Data Engineering, 35(5): 4954–4968

    Google Scholar 

  • Nan G, Guo Z, Sekulic I, Lu W (2020). Reasoning with latent structure refinement for document-level relation extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). Online: ACL, 1546–1557

  • Qi A, Sin T, Fathullah M, Lee C (2017). The impact of fit manufacturing on green manufacturing: A review. In: Proceedings of the 3rd Electronic and Green Materials International Conference (EGM). Krabi: AIP, 020083

    Google Scholar 

  • Ren H, Chen Z, Jiang Z, Yang C, Gui W (2021). An industrial multilevel knowledge graph-based local–global monitoring for plant-wide processes. IEEE Transactions on Instrumentation and Measurement, 70: 1–15

    Google Scholar 

  • Sahu S, Christopoulou F, Miwa M, Ananiadou S (2019). Inter-sentence relation extraction with document-level graph convolutional neural network. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL). Florence: ACL, 4309–4316

    Chapter  Google Scholar 

  • Shi H, Huang D, Wang L, Wu M, Xu Y, Zeng B, Pang C (2021). An information integration approach to spacecraft fault diagnosis. Enterprise Information Systems, 15(8): 1128–1161

    Article  ADS  Google Scholar 

  • Strubell E, Verga P, Belanger D, McCallum A (2017). Fast and accurate entity recognition with iterated dilated convolutions. arXiv preprint. arXiv:1702.02098

  • Wan Z, Ge P, Zhang X, Yin G (2018). Research on equipment manufacturing industry upgrading under intelligent manufacturing. World Sci-Tech R & D, 40(3): 316–327 (in Chinese)

    Google Scholar 

  • Wang B, Yi B, Liu Z, Zhou Y, Zhou Y (2021). Evolution and state-of-the-art of intelligent manufacturing from HCPS perspective. Computer Integrated Manufacturing Systems, 27(10): 2749–2761 (in Chinese)

    Google Scholar 

  • Wang B, Zang J, Qu X, Dong J, Zhou Y (2018). Research on new-generation intelligent manufacturing based on human-cyber-physical systems. Strategic Study of CAE, 20(4): 29–34

    Article  Google Scholar 

  • Wang H, Focke C, Sylvester R, Mishra N, Wang W (2019). Fine-tune BERT for DocRED with two-step process. arXiv preprint. arXiv: 1909.11898 i]Xiao C, Yao Y, Xie R, Han X, Liu Z, Sun M, Lin F, Lin L (2020). Denoising relation extraction from document-level distant supervision. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: ACL, 3683–3688

  • Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z (2021). Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction. In: Proceedings of the AAAI Conference on Artificial Intelligence. Online: AAAI, 14149–14157

  • Xu Z, Dang Y, Zhang Z, Chen J (2020). Typical short-term remedy knowledge mining for product quality problem-solving based on bipartite graph clustering. Computers in Industry, 122: 103277

    Article  Google Scholar 

  • Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, Liu Z, Huang L, Zhou J, Sun M (2019). DocRED: A large-scale document-level relation extraction dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL). Florence: ACL, 764–777

    Chapter  Google Scholar 

  • Zeng S, Xu R, Chang B, Li L (2020). Double graph based reasoning for document-level relation extraction. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: ACL, 1630–1640

  • Zhang D, Liu Z, Jia W, Liu H, Tan J (2021). A review on knowledge graph and its application prospects to intelligent manufacturing. Journal of Mechanical Engineering, 57(5): 90–113 (in Chinese)

    Article  Google Scholar 

  • Zhang Z, Han X, Liu Z, Jiang X, Sun M, Liu Q (2019). ERNIE: Enhanced language representation with informative entities. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL). Florence: ACL, 1441–1451

    Chapter  Google Scholar 

  • Zheng P, Xia L, Li C, Li X, Liu B (2021). Towards Self-X cognitive manufacturing network: An industrial knowledge graph-based multi-agent reinforcement learning approach. Journal of Manufacturing Systems, 61: 16–26

    Article  Google Scholar 

  • Zhou J, Zhou Y, Wang B, Zang J (2019). Human-Cyber-Physical Systems (HCPSs) in the context of new-generation intelligent manufacturing. Engineering, 5(4): 624–636

    Article  Google Scholar 

  • Zhou W, Huang K, Ma T, Huang J (2021). Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI Conference on Artificial Intelligence. Online: AAAI, 14612–14620

  • Zhou Y, Huang H, Liu H, Hao Z (2022). Survey on document-level relation extraction. Journal of South China University of Technology (Natural Science Edition), 50(4): 10–25 (in Chinese)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Electronic and Information Engineering, Tongji University, Shanghai, 200092, China

    Zhulin Han & Jian Wang

Authors
  1. Zhulin Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhulin Han.

Ethics declarations

Competing Interests The authors declare that they have no competing interests.

Additional information

This work was supported by the National Science and Technology Innovation 2030 New Generation Artificial Intelligence Major Project (Grant No. 2018AAA0101800), and the National Natural Science Foundation of China (Grant No. 72271188).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, Z., Wang, J. Knowledge enhanced graph inference network based entity-relation extraction and knowledge graph construction for industrial domain. Front. Eng. Manag. 11, 143–158 (2024). https://doi.org/10.1007/s42524-023-0273-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42524-023-0273-1

Keywords

Search

Navigation