Skip to main content
SpringerLink
Log in

Linking unknown characters via oracle bone inscriptions retrieval

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Retrieving useful information from existing collections of oracle bone rubbing images plays a pivotal role in the study of oracle bone inscription decipherment. However, current systems for processing oracle bone information rely on expert-curated databases, which entail a time-consuming and labor-intensive process. Moreover, solely depending on oracle bone databases fails to yield any relevant information about undeciphered characters. Therefore, to address these challenges, in this paper, we present a deep learning retrieval framework named LUC, specifically designed for searching arbitrary oracle bone characters (both deciphered and undeciphered). Specifically, LUC takes clear glyph images as input, which can be handwritten by users or downloaded from websites, and extracts similar characters from raw oracle bone rubbing images through feature extraction and metric learning. Furthermore, unlike conventional image retrieval frameworks, we introduce an additional domain-aware embedding module to bridge the significant domain gap between clear glyphs and image patches. This module utilizes domain-specific information to generate a set of oracle bone radical prototypes, enhancing the structural features of oracle bone characters. Lastly, to mitigate the impact of increased feature output dimensions on retrieval performance, we construct a novel loss function. This loss function, based on the principle of maximum coding rate in metric learning, alleviates the performance degradation caused by dimensionality increase. Importantly, we establish a customized oracle bone retrieval benchmark comprising known characters for training and unknown characters for testing. Extensive comparative experiments demonstrate that LUC achieves superior performance compared to other classical retrieval methods. Furthermore, experiments on three publicperson ReID benchmarks also verify the effectiveness and generalization of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Data available on request from the authors.

References

  1. Arandjelovic, R., Gronat, P., Torii, A. et al: Netvlad: Cnn architecture for weakly supervised place recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5297–5307 (2016)

  2. Carlucci, F.M., D’Innocente, A., Bucci, S. et al: Domain generalization by solving jigsaw puzzles. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2229–2238 (2019)

  3. Chang, X., Chao, F., Shang, C., et al.: Sundial-gan: A cascade generative adversarial networks framework for deciphering oracle bone inscriptions. In: Proceedings of the ACM International Conference on Multimedia, pp 1195–1203 (2022)

  4. Chen, T., Ding, S., Xie, J., et al.: Abd-net: Attentive but diverse person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, pp 8350–8360 (2019)

  5. Chen, W., Chen, X., Zhang, J., et al.: Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 403–412 (2017)

  6. Chen, X., Fu, C., Zhao, Y., et al.: Salience-guided cascaded suppression network for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, pp 3297–3307 (2020)

  7. Chi, Y., Giunchiglia, F., Shi, D., et al.: Zinet: Linking chinese characters spanning three thousand years. In: Proceedings of the Findings of the Association for Computational Linguistics, pp 3061–3070 (2022)

  8. Cui, Z., Zhou, J., Peng, Y., et al.: Dcr-reid: Deep component reconstruction for cloth-changing person re-identification. IEEE Transactions on Circuits and Systems for Video Technology (2023)

  9. Deng, J., Dong, W., Socher, R., et al.: Imagenet: A large-scale hierarchical image database. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Ieee, pp 248–255 (2009)

  10. Diao, X., Shi, D., Tang, H., et al.: Rezcr: A zero-shot character recognition method via radical extraction. arXiv preprint arXiv:2207.05842 (2022)

  11. Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  12. Dutta, T., Singh, A., Biswas, S.: Styleguide: Zero-shot sketch-based image retrieval using style-guided image generation. IEEE Trans. Multimedia 23, 2833–2842 (2020)

    Article  Google Scholar 

  13. El-Nouby, A., Neverova, N., Laptev, I., et al.: Training vision transformers for image retrieval. arXiv preprint arXiv:2102.05644 (2021)

  14. Feng, Q., Kang, G., Fan, H., et al.: Attract or distract: Exploit the margin of open set. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 7990–7999 (2019)

  15. Fu, X., Yang, Z., Zeng, Z., et al.: Improvement of oracle bone inscription recognition accuracy: A deep learning perspective. ISPRS Int. J. Geo Inf. 11(1), 45 (2022)

    Article  Google Scholar 

  16. Fujikawa, Y., Li, H., Yue, X., et al.: Recognition of oracle bone inscriptions by using two deep learning models. Proceedings of the International Journal of Digital Humanities pp 1–15 (2022)

  17. Gao, F., Zhang, J., Liu, Y., et al.: Image translation for oracle bone character interpretation. Symmetry 14(4), 743 (2022)

    Article  Google Scholar 

  18. Ge, W., Liu, G., Lv, J.: Oracle bone inscriptions extraction by using weakly supervised instance segmentation under deep network. In: Proceedings of the International Conference on Information Systems and Computer Aided Education, IEEE, pp 229–233 (2021)

  19. Gkelios S, Boutalis Y, Chatzichristofis SA (2021) Investigating the vision transformer model for image retrieval tasks. In: Proceedings of the International Conference on Distributed Computing in Sensor Systems, IEEE, pp 367–373

  20. Han, X., Bai, Y., Qiu, K., et al.: Isobs: An information system for oracle bone script. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp 227–233 (2020)

  21. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2016)

  22. He, S., Luo, H., Wang. P., et al.: Transreid: Transformer-based object re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 15013–15022 (2021)

  23. Huang, S., Wang, H., Liu, Y., et al.: Obc306: A large-scale oracle bone character recognition dataset. In: Proceedings of the International Conference on Document Analysis and Recognition, IEEE, pp 681–688 (2019)

  24. Jégou, H., Douze, M., Schmid, C., et al.: Aggregating local descriptors into a compact image representation. In: Proceedings of the IEEE/CVF on Computer Vision and Pattern Recognition, IEEE, pp 3304–3311 (2010)

  25. Jin, X., Lan, C., Zeng, W., et al.: Semantics-aligned representation learning for person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, pp 11173–11180 (2020)

  26. Li, B., Dai, Q., Gao, F., et al.: Hwobc-a handwriting oracle bone character recognition database. In: Journal of Physics: Conference Series, IOP Publishing, p 012050 (2020)

  27. Li, X., Li, J., Du, Z., et al.: Interpretable open-set domain adaptation via angular margin separation. In: Proceedings of the European Conference on Computer Vision, Springer, pp 1–18 (2022)

  28. Li, Z., Shi, Y., Ling, H., et al.: Viewpoint disentangling and generation for unsupervised object re-id. ACM Trans Multimedia Comput Commun Appl 20(5). https://doi.org/10.1145/3632959, (2024)

  29. Lin, X., Chen, S., Zhao, F., et al.: Radical-based extract and recognition networks for oracle character recognition. Proceedings of the International Journal on Document Analysis and Recognition 25(3), 219–235 (2022)

    Article  Google Scholar 

  30. Liu, G., Song, X., Ge, W., et al.: Oracle-bone-inscription image segmentation based on simple fully convolutional networks. In: Proceedings of the MIPPR 2019: Pattern Recognition and Computer Vision, SPIE, pp 293–296 (2020a)

  31. Liu, G., Xing, J., Xiong, J.: Spatial pyramid block for oracle bone inscription detection. In: Proceedings of the International Conference on Software and Computer Applications, pp 133–140 (2020b)

  32. Liu, G., Chen, S., Xiong, J., et al.: An oracle bone inscription detector based on multi-scale gaussian kernels. Appl. Math. 12(3), 224–239 (2021)

    Article  Google Scholar 

  33. Liu, J., Ni, B., Yan, Y., et al.: Pose transferrable person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4099–4108 (2018)

  34. Liu, Z., Lin, Y., Cao, Y., et al.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the International Conference on Computer Vision (2021b)

  35. Lou, Y., Bai, Y., Liu, J., et al.: Veri-wild: A large dataset and a new method for vehicle re-identification in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3235–3243 (2019)

  36. Luo, H., Gu, Y., Liao, X., et al.: Bag of tricks and a strong baseline for deep person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition workshops, pp 0–0 (2019)

  37. Meng, D., Li, L., Wang, S., et al.: Fine-grained feature alignment with part perspective transformation for vehicle reid. In: Proceedings of the ACM International Conference on Multimedia, pp 619–627 (2020)

  38. Meng, L.: Two-stage recognition for oracle bone inscriptions. In: Proceedings of the International Conference on Image Analysis and Processing, Springer, pp 672–682 (2017)

  39. Meng, L., Kamitoku, N., Yamazaki, K.: Recognition of oracle bone inscriptions using deep learning based on data augmentation. In: Proceedings of the Metrology for Archaeology and Cultural Heritage, IEEE, pp 33–38 (2018)

  40. Meng, L., Lyu, B., Zhang, Z., et al.: Oracle bone inscription detector based on ssd. In: Proceedings of the International Conference on Image Analysis and Processing Workshops, Springer, pp 126–136 (2019)

  41. Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Ieee, pp 2161–2168 (2006)

  42. Rao, Y., Zhao, W., Chen, G., et al.: Denseclip: Language-guided dense prediction with context-aware prompting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 18082–18091 (2022)

  43. Ren, H., Zheng, Z., Lu, H.: Energy-guided feature fusion for zero-shot sketch-based image retrieval. Neural Processing Letters pp 1–10 (2022)

  44. Ristani, E., Solera, F., Zou, R., et al.: Performance measures and a data set for multi-target, multi-camera tracking. In: Proceedings of the European Conference on Computer Vision, Springer, pp 17–35 (2016)

  45. Shi, D., Diao, X., Shi, L., et al.: Charformer: A glyph fusion based attentive framework for high-precision character image denoising. In: Proceedings of the ACM International Conference on Multimedia, pp 1147–1155 (2022)

  46. Wang, G., Yang, S., Liu, H., et al.: High-order information matters: Learning relation and topology for occluded person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, pp 6448–6457 (2020)

  47. Wang, H., Niu, Y., Chen, L., et al.: Dair-v2xreid: A new real-world vehicle-infrastructure cooperative re-id dataset and cross-shot feature aggregation network perception method. IEEE Transactions on Intelligent Transportation Systems (2024)

  48. Wang, M., Deng, W., Liu, C.L.: Unsupervised structure-texture separation network for oracle character recognition. IEEE Trans. Image Process. 31, 3137–3150 (2022)

    Article  Google Scholar 

  49. Wang, Z., Yue, X., Meng, L.: An effective approach for noise reduction from shirakawa’s hand-notated documents on obis research. In: ATAIT, pp 28–37 (2021)

  50. Wei, L., Zhang, S., Gao, W., et al.: Person transfer gan to bridge domain gap for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 79–88 (2018)

  51. Xing, J., Liu, G., Xiong, J.: Oracle bone inscription detection: a survey of oracle bone inscription detection based on deep learning algorithm. In: Proceedings of the International Conference on Artificial Intelligence, Information Processing and Cloud Computing, pp 1–8 (2019)

  52. Xiong, J., Liu, G., Liu, Y., et al.: Oracle bone inscriptions information processing based on multi-modal knowledge graph. Computers & Electrical Engineering 92, 107173 (2021)

    Article  Google Scholar 

  53. Yang, F., Wu, Y., Wang, Z., et al.: Instance-level heterogeneous domain adaptation for limited-labeled sketch-to-photo retrieval. IEEE Trans. Multimedia 23, 2347–2360 (2020)

    Article  Google Scholar 

  54. Yu, Y., Chan, K.H.R., You, C., et al.: Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Proceedings of the Advances in Neural Information Processing Systems 33, 9422–9434 (2020)

    Google Scholar 

  55. Yue-Hei Ng, J., Yang, F., Davis, L.S.: Exploiting local features from deep networks for image retrieval. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 53–61 (2015)

  56. Zhang, C., Zong, R., Cao, S., et al.: Ai-powered oracle bone inscriptions recognition and fragments rejoining. In: Proceedings of the International Conference on International Joint Conferences on Artificial Intelligence, pp 5309–5311 (2021)

  57. Zhang, C., Wang, B., Chen, K., et al.: Data-driven oracle bone rejoining: A dataset and practical self-supervised learning scheme. In: Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp 4482–4492 (2022)

  58. Zhang, G., Liu, J., Chen, Y., et al.: Multi-biometric unified network for cloth-changing person re-identification. IEEE Trans. Image Process. 32, 4555–4566 (2023)

    Article  Google Scholar 

  59. Zhang, J., Du, J., Dai, L.: Radical analysis network for learning hierarchies of chinese characters. Pattern Recogn. 103, 107305 (2020)

    Article  Google Scholar 

  60. Zhang, Y., Zhang, F., Jin, Y., et al.: Local correlation ensemble with gcn based on attention features for cross-domain person re-id. ACM Trans Multimedia Comput Commun Appl 19(2). https://doi.org/10.1145/3542820 (2023b)

  61. Zheng, L., Shen, L., Tian, L., et al.: Scalable person re-identification: A benchmark. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1116–1124 (2015)

  62. Zheng, L., Yang, Y., Tian, Q.: SIFT meets CNN: A decade survey of instance retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 40(5), 1224–1244 (2018)

    Article  Google Scholar 

  63. Zhong, Z., Zheng, L., Kang, G., et al.: Random erasing data augmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 13001–13008 (2020)

  64. Zhou, C., Loy, C.C., Dai, B.: Extract free dense labels from clip. In: Proceedings of the European Conference on Computer Vision, Springer, pp 696–712 (2022)

  65. Zhou, K., Yang, Y., Cavallaro, A., et al.: Omni-scale feature learning for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, pp 3701–3711 (2019)

  66. Zhu, K., Guo, H., Liu, Z., et al.: Identity-guided human semantic parsing for person re-identification. In: Proceedings of the European Conference Computer Vision, Lecture Notes in Computer Science, vol 12348. Springer, pp 346–363 (2020)

  67. Zhuang, Z., Wei, L., Xie, L., et al.: Rethinking the distribution gap of person re-identification with camera-based batch normalization. In: Proceedings of the European Conference on Computer Vision, vol 12357. Springer, pp 140–157 (2020)

Download references

Acknowledgements

Feng Gao is supported by the Henan Province Science and technology research Project (NO.232102320169). Bang Li is supported by the Natural Science Foundation of Henan Province (NO. 242300420680) and the Henan Province Science and Technology Research Project (NO. 222102210257). Yongge Liu is supported by the Paleography and Chinese Civilization Inheritance and Development Program (NO. G1807 and G1806).

Author information

Authors and Affiliations

  1. The School of Computer and Information Engineering, Anyang Normal University, Anyang, 455000, Henan, China

    Feng Gao, Bang Li & Yongge Liu

  2. The College of Intelligence and Computing, Tianjin University, Tianjin, 300072, China

    Xu Chen, Runhua Jiang & Yahong Han

  3. Key Laboratory of Oracle Bone Inscriptions Information Processing, Ministry of Education, Anyang, China

    Feng Gao, Bang Li & Yongge Liu

Authors
  1. Feng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongge Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Runhua Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yahong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Feng Gao: Conceptualization, Methodology, Writing – original draft. Xu Chen and Bang Li: Formal analysis, Data curation, Software. Yongge Liu: Writing – review & editing. Runhua Jiang: Visualization, Supervision. Yahong Han: Funding acquisition, Investigation, Resources.

Corresponding author

Correspondence to Yahong Han.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Communicated by B. Bao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, F., Chen, X., Li, B. et al. Linking unknown characters via oracle bone inscriptions retrieval. Multimedia Systems 30, 125 (2024). https://doi.org/10.1007/s00530-024-01327-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01327-7

Keywords

Search

Navigation