Skip to main content

Chinese toponym recognition with variant neural structures from social media messages based on BERT methods

Abstract

Many natural language tasks related to geographic information retrieval (GIR) require toponym recognition, and identifying Chinese toponyms from social media messages to share real-time information is a critical problem for many practical applications, such as natural disaster response and geolocating. In this article, we focused on toponym recognition from social media messages in Chinese. While existing off-the-shelf Chinese named entity recognition (NER) tools could be applied to identify toponyms, these approaches cannot address a variety of language irregularities taken from social media messages, including location name abbreviations, informal sentence structures and combination toponyms. We present a deep neural network named BERT-BiLSTM-CRF, which extends a basic bidirectional recurrent neural network model (BiLSTM) with the pretraining bidirectional encoder representation from transformers (BERT) representation to handle the toponym recognition task in Chinese text. Using three datasets taken from lists of alternative location names, the experimental results showed that the proposed model can significantly outperform previous Chinese NER models/algorithms and a set of state-of-the-art deep learning models.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Code availability

https://github.com/HumarTxx/toponym-recognition-code.git

References

  • Alex B, Byrne K, Grover C, Tobin R (2015) Adapting the Edinburgh geoparser for historical georeferencing. Int J Humanities Arts Comput 9(1):15–35

    Article  Google Scholar 

  • Arribas-Bel D, Green M, Rowe F, Singleton A (2021) Open data products-A framework for creating valuable analysis ready data. J Geogr Syst 23(4):497–514

    Article  Google Scholar 

  • Borges KA, Davis CA Jr, Laender AH, Medeiros CB (2011) Ontology-driven discovery of geospatial evidence in web pages. GeoInformatica 15:609–631

    Article  Google Scholar 

  • Brunsdon C, Comber A (2020) Big issues for big data: challenges for critical spatial data analytics. arXiv preprint arXiv:2007.11281

  • Chen Y, Ouyang Y, Li WJ, Zheng DQ, Zhao TJ (2010) Using deep belief nets for Chinese named entity categorization. In: Proceedings of the 2010 Named Entities Workshop, Uppsala, Sweden, 16 July 2010; pp. 102–109.

  • Delboni TM, Borges KAV, Laender AHF, Davis CA Jr (2007) Semantic expansion of geographic web queries based on natural language positioning expressions. Trans GIS 11:377–397

    Article  Google Scholar 

  • DeLozier G, Baldridge J, London L (2015) Gazetteer-independent toponym resolution using geographic word profiles. In: Twenty-Ninth AAAI conference on artificial intelligence

  • Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  • Gao P, Zhang Q, Wang F, Xiao L, Fujita H, Zhang Y (2020) Learning reinforced attentional representation for end-to-end visual tracking. Inf Sci 517:52–67. https://doi.org/10.1016/j.ins.2019.12.084

    Article  Google Scholar 

  • Gelernter J, Balaji S (2013) An algorithm for local geoparsing of microtext. GeoInformatica 17(4):635–667

    Article  Google Scholar 

  • Gelernter J, Mushegian N (2011) Geo-parsing messages from microtext. Trans GIS 15(6):753–773

    Article  Google Scholar 

  • Gritta M, Pilehvar MT, Collier N (2018) Which Melbourne? Augmenting geocoding with maps. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia (Vol. 1: Long Papers, pp. 1285–1296). Stroudsburg, PA: ACL.

  • Hahmann S, Burghardt D (2013) How much information is geospatially referenced? Networks and cognition. Int J Geogr Inf Sci 27(6):1171–1189. https://doi.org/10.1080/13658816.2012.743664

    Article  Google Scholar 

  • Hill, LL (2000) Core elements of digital gazetteers: placenames, categories, and footprints. In: Borbinha J, Baker T (eds) Research and advanced technology for digital libraries. Springer Lecture Notes in Computer Science, Germany, Berlin, Vol. 1923, pp. 280–290

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 (PMID:9377276)

    Article  Google Scholar 

  • Hu Y, Mao H, McKenzie G (2019) A natural language processing and geospatial clustering framework for harvesting local place names from geotagged housing advertisements. Int J Geogr Inf Sci 33(4):714–738

    Article  Google Scholar 

  • Huynh-The T, Bae S-H, Lee S (2020) Cross-attentional bracket-shaped convolutional network for semantic image segmentation. Inf Sci. https://doi.org/10.1016/j.ins.2020.06.023

    Article  Google Scholar 

  • Ju Y, Adams B, Janowicz K, Hu Y, Yan B, McKenzie G (2016) Things and strings: improving place name disambiguation from short texts by combining entity co-occurrence with topic modeling. In: Blomqvist E, Ciancarini P, Poggi F, Vitali F (eds) Knowledge engineering and knowledge management: EKAW 2016 (Lecture notes in computer science, vol 10024. Springer, Cham, Switzerland, pp 353–367

    Chapter  Google Scholar 

  • Karimzadeh M, Huang W, Banerjee S, Wallgrün JO, Hardisty F, Pezanowski S, MacEachren AM (2013) GeoTxt: a web API to leverage place references in text. In: Proceedings of the Seventh Workshop on Geographic Information Retrieval, Orlando, FL (pp. 72–73). New York, NY: ACM.

  • Karimzadeh M, Pezanowski S, MacEachren AM, Wallgrün JO (2019) GeoTxt: A scalable geoparsing system for unstructured text geolocation. Trans GIS 23(1):118–136

    Article  Google Scholar 

  • Kuai Xi, Guo R, Zhang Z, He B, Zhigang Z, Guo H (2020) Spatial context-based local toponym extraction and chinese textual address segmentation from Urban POI data. ISPRS Int J Geo Inf 9:147. https://doi.org/10.3390/ijgi9030147

    Article  Google Scholar 

  • Levow G-A (2006) The third international Chinese language processing bakeoff: word segmentation and named entity recognition. In: Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, pp 108–117

  • Li H, Wang M, Baldwin T, Tomko M, Vasardani M (2019) UniMelb at SemEval-2019 Task 12: multi-model combination for toponym resolution. In: Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, MN (pp. 1313–1318). Stroudsburg, PA: ACL.

  • Liu G, Guo J (2019) Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing. https://doi.org/10.1016/j.neucom.2019.01.078

    Article  Google Scholar 

  • Lovelace R (2021) Open source tools for geographic analysis in transport planning. J Geogr Syst 23(4):547–578

    Article  Google Scholar 

  • Mcdonough K, Moncla L, Camp MVD (2019) Named entity recognition goes to old regime france: geographic text analysis for early modern french corpora. Int J Geograph Inf ence(1).

  • McCurley KS (2001) Geospatial mapping and navigation of the web. In: Proceedings of the 10th International Conference on World Wide Web, Hong Kong, China, 221–229.

  • Melo F, Martins B (2016) Automated geocoding of textual documents: a survey of current approaches: automated geocoding of textual documents. Trans GIS. https://doi.org/10.1111/tgis.12212

    Article  Google Scholar 

  • Mikolov T, Karafiát M, Burget L, Černocký J, Khudanpur S (2010) Recurrent neural network based language model. In: Interspeech, vol 2, No. 3, pp 1045–1048

  • Mikolov T, Deoras A, Povey D, Burget L, Černocký J (2011) Strategies for training large scale neural network language models. In: 2011 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). IEEE. pp. 196–201.

  • Moura TH VM, Davis CA Jr, Fonseca FT (2017) Reference data enhancement for geographic information retrieval using linked data. Trans GIS 21(4):683–700

    Article  Google Scholar 

  • Pennington J, Socher R, Manning CD (2014) GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar (pp. 1532–1543). Stroudsburg, PA: ACL.

  • Purves RS, Clough P, Jones CB, Hall MH, Murdock V (2018) Geographic information retrieval: progress and challenges in spatial search of text. Found Trends Inf Retr 12(2&3):164–318

    Article  Google Scholar 

  • Qiu Q, Xie Z, Wu L (2018a) A cyclic self-learning Chinese word segmentation for the geoscience domain. Geomatica. https://doi.org/10.1139/geomatica-2018-0007

    Article  Google Scholar 

  • Qiu Q, Xie Z, Wu L, Li W (2018b) DGeoSegmenter: a dictionary-based Chinese word segmenter for the geoscience domain. Comput Geosci. https://doi.org/10.1016/j.cageo.2018.08.006

    Article  Google Scholar 

  • Qiu Q, Xie Z, Wu L, Li W (2019) Geoscience keyphrase extraction algorithm using enhanced word embedding. Expert Syst Appl 125:157–169

    Article  Google Scholar 

  • Quercini G, Samet H (2014) Uncovering the spatial relatedness in Wikipedia. In: Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Dallas, Texas, pp 153–162

  • Qu J, Ouyang D, Hua W, Ye Y, Li X (2018) Distant supervision for neural relation extraction integrated with word attention and property features. Neural Netw. https://doi.org/10.1016/j.neunet.2018.01.006

    Article  Google Scholar 

  • Santos J, Anastácio I, Martins B (2015) Using machine learning methods for disambiguating place references in textual documents. GeoJ 80(3):375–392. https://doi.org/10.1007/s10708-014-9553-y

    Article  Google Scholar 

  • Santos R, Murrieta-Flores P, Martins B (2017) Learning to combine multiple string similarity metrics for effective toponym matching. Int J Digital Earth. https://doi.org/10.1080/17538947.2017.1371253

    Article  Google Scholar 

  • Santos R, Murrieta-Flores P, Calado P, Martins B (2018) Toponym matching through deep neural networks. Int J Geogr Inf Sci 32(2):324–348

    Article  Google Scholar 

  • Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681. https://doi.org/10.1109/78.650093

    Article  Google Scholar 

  • Speriosu M, Baldridge J (2013) Text-driven toponym resolution using indirect supervision. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria (pp. 1466–1476). Stroudsburg, PA: ACL

  • Wang J, Hu Y (2019) Enhancing spatial and textual analysis with EUPEG: an extensible and unified platform for evaluating geoparsers. Trans GIS 23(6):1393–1419

    Article  Google Scholar 

  • Wang S, Zhang X, Ye P, Du M (2018) Deep belief networks based toponym recognition for Chinese text. ISPRS Int J Geo-Inf 7(6):217

    Article  Google Scholar 

  • Wang J, Hu Y, Joseph K (2020) NeuroTPR: a neuro-net toponym recognition model for extracting locations from social media messages. Trans GIS. https://doi.org/10.1111/tgis.12627

    Article  Google Scholar 

  • Weissenbacher D, Magge A, O’Connor K, Scotch M, Gonzalez G (2019) SemEval-2019 Task 12: Toponym resolution in scientific papers. In: Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, MN (pp. 907–916). Stroudsburg, PA: ACL

  • Yadav V, Laparra E, Wang T-T, Surdeanu M, Bethard S (2019) University of Arizona at SemEval-2019 Task 12: Deep-affix named entity recognition of geolocation entities. In: Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, MN (pp. 1319–1323). Stroudsburg, PA: ACL

  • Yi X, Raghavan H, Leggetter C (2009) Discovering users’ specific geo intention in web search. In: Proceedings of the 18th International Conference on World Wide Web, Madrid, Spain, 481–490

  • Zhou D, Qian M, Hua M, Liu D, Tang X (2011) Structural analysis and computation of Chinese toponyms. Int J Knowl Lang Process 2(3):36–47

    Google Scholar 

Download references

Acknowledgments

This study was financially supported by the National Natural Science Foundation of China (42050101, U1711267, 41871311, 41871305), the China Postdoctoral Science Foundation (No.2021M702991), Open Research Project of The Hubei Key Laboratory of Intelligent Geo-Information Processing (No. KLIGIP-2021A01), National Key Research and Development Program (2018YFB0505500, 2018YFB0505504), and the Fundamental Research Funds for the Central Universities, China University of Geosciences (Wuhan) (No. CUG2106116)).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qinjun Qiu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ma, K., Tan, Y., Xie, Z. et al. Chinese toponym recognition with variant neural structures from social media messages based on BERT methods. J Geogr Syst 24, 143–169 (2022). https://doi.org/10.1007/s10109-022-00375-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10109-022-00375-9

Keywords

  • Chinese toponym recognition
  • Deep learning
  • BERT representation
  • Geographic information retrieval

JEL Classification

  • C10
  • C18