Skip to main content
SpringerLink
Log in

FFD Augmentor: Towards Few-Shot Oracle Character Recognition from Scratch

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13845))

Included in the following conference series:

  • 325 Accesses

Abstract

Recognizing oracle characters, the earliest hieroglyph discovered in China, is recently addressed with more and more attention. Due to the difficulty of collecting labeled data, recognizing oracle characters is naturally a Few-Shot Learning (FSL) problem, which aims to tackle the learning problem with only one or a few training data. Most current FSL methods assume a disjoint but related big dataset can be utilized such that one can transfer the related knowledge to the few-shot case. However, unlike common phonetic words like English letters, oracle bone inscriptions are composed of radicals representing graphic symbols. Furthermore, as time goes, the graphic symbols to represent specific objects were significantly changed. Hence we can hardly find plenty of prior knowledge to learn without negative transfer. Another perspective to solve this problem is to use data augmentation algorithms to directly enlarge the size of training data to help the training of deep models. But popular augment strategies, such as dividing the characters into stroke sequences, break the orthographic units of Chinese characters and destroy the semantic information. Thus simply adding noise to strokes perform weakly in enhancing the learning capacity.

To solve these issues, we in this paper propose a new data augmentation algorithm for oracle characters such that (1) it will introduce informative diversity for the training data while alleviating the loss of semantics; (2) with this data augmentation algorithm, we can train the few-shot model from scratch without pre-training and still get a powerful recognition model with superior performance to models pre-trained with a large dataset. Specifically, our data augmentation algorithm includes a B-spline free form deformation method to randomly distort the strokes of characters but maintain the overall structures. We generate 20–40 augmented images for each training data and use this augmented training set to train a deep neural network model in a standard pipeline. Extensive experiments on several benchmark datasets demonstrate the effectiveness of our augmentor. Code and models are released in https://github.com/Hide-A-Pumpkin/FFDAugmentor.

This paper is the final project of Neural Network and Deep Learning (DATA130011.01, Course Instructor: Dr. Yanwei Fu; TA: Yikai Wang), School of Data Science, Fudan University.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Keightley, D.N.: Graphs, words, and meanings: three reference works for Shang oracle-bone studies, with an excursus on the religious role of the day or sun. J. Am. Oriental Soc. 117, 507–524 (1997)

    Google Scholar 

  2. Xing, J., Liu, G., Xiong, J.: Oracle bone inscription detection: a survey of oracle bone inscription detection based on deep learning algorithm. In: Proceedings of the International Conference on Artificial Intelligence, Information Processing and Cloud Computing. AIIPCC 2019, New York, NY, USA, Association for Computing Machinery (2019)

    Google Scholar 

  3. Guo, J., Wang, C., Roman-Rangel, E., Chao, H., Rui, Y.: Building hierarchical representations for oracle character and sketch recognition. IEEE Trans. Image Process. 25, 104–118 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  4. Lu, X., Cai, H., Lin, L.: Recognition of oracle radical based on the capsule network. CAAI Trans. Intell. Syst. 15, 243–254 (2020)

    Google Scholar 

  5. Han, W., Ren, X., Lin, H., Fu, Y., Xue, X.: Self-supervised learning of Orc-Bert augmentor for recognizing few-shot oracle characters. In: Ishikawa, H., Liu, C.-L., Pajdla, T., Shi, J. (eds.) ACCV 2020. LNCS, vol. 12627, pp. 652–668. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-69544-6_39

    Chapter  Google Scholar 

  6. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems 29 (2016)

    Google Scholar 

  7. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning (2017)

    Google Scholar 

  8. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning, pp. 1126–1135. PMLR (2017)

    Google Scholar 

  9. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. In: Balcan, M.F., Weinberger, K.Q. (eds.) Proceedings of The 33rd International Conference on Machine Learning. Volume 48 of Proceedings of Machine Learning Research., New York, New York, USA, PMLR, pp. 1842–1850 (2016)

    Google Scholar 

  10. Oreshkin, B., López, P.R., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning (2018)

    Google Scholar 

  11. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: Relation network for few-shot learning (2018)

    Google Scholar 

  12. Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999 (2018)

  13. Rusu, A.A., et al.: Meta-learning with latent embedding optimization (2019)

    Google Scholar 

  14. Mishra, N., Rohaninejad, M., Chen, X., Abbeel, P.: A simple neural attentive meta-learner (2018)

    Google Scholar 

  15. Chen, Z., Fu, Y., Chen, K., Jiang, Y.G.: Image block augmentation for one-shot learning. Proceed. AAAI Conf. Artif. Intell. 33, 3379–3386 (2019)

    Google Scholar 

  16. Antoniou, A., Storkey, A., Edwards, H.: Augmenting image classifiers using data augmentation generative adversarial networks. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11141, pp. 594–603. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01424-7_58

    Chapter  Google Scholar 

  17. Ren, M., et al.: Meta-learning for semi-supervised few-shot classification (2018)

    Google Scholar 

  18. Liu, Y., et al.: Learning to propagate labels: transductive propagation network for few-shot learning (2019)

    Google Scholar 

  19. Li, X., et al.: Learning to self-train for semi-supervised few-shot classification (2019)

    Google Scholar 

  20. Wang, Y., Xu, C., Liu, C., Zhang, L., Fu, Y.: Instance credibility inference for few-shot learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  21. Wang, Y., Zhang, L., Yao, Y., Fu, Y.: How to trust unlabeled data? instance credibility inference for few-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

    Google Scholar 

  22. Yalniz, I.Z., Jégou, H., Chen, K., Paluri, M., Mahajan, D.: Billion-scale semi-supervised learning for image classification. arXiv preprint arXiv:1905.00546 (2019)

  23. Springenberg, J.T.: Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv preprint arXiv:1511.06390 (2015)

  24. Wang, M., Deng, W., Liu, C.L.: Unsupervised structure-texture separation network for oracle character recognition. IEEE Trans. Image Process. 31, 3137–3150 (2022)

    Article  Google Scholar 

  25. Rueckert, D., Sonoda, L., Hayes, C., Hill, D., Leach, M., Hawkes, D.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18, 712–721 (1999)

    Article  Google Scholar 

  26. Keszei, A.P., Berkels, B., Deserno, T.M.: Survey of non-rigid registration tools in medicine. J. Digit. Imaging 30, 102–116 (2017)

    Article  Google Scholar 

  27. Bendou, Y., et al.: Easy: ensemble augmented-shot y-shaped learning: State-of-the-art few-shot classification with simple ingredients. arXiv preprint arXiv:2201.09699 (2022)

  28. Li, B., Dai, Q., Gao, F., Zhu, W., Li, Q., Liu, Y.: HWOBC-a handwriting oracle bone character recognition database. J. Phys: Conf. Ser. 1651, 012050 (2020)

    Google Scholar 

  29. Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. 31, 1–10 (2012)

    Google Scholar 

  30. Zhou, X.L., Hua, X.C., Li, F.: A method of Jia Gu wen recognition based on a two-level classification. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 2, pp. 833–836. IEEE (1995)

    Google Scholar 

  31. Li, F., Woo, P.-Y.: The coding principle and method for automatic recognition of Jia Gu wen characters. Int. J. Human-Comput. Stud. 53(2), 289–299 (2000)

    Google Scholar 

  32. Qingsheng, L.: Recognition of inscriptions on bones or tortoise shells based on graph isomorphism. Computer Engineering and Applications (2011)

    Google Scholar 

  33. Yang, Z., et al.: Accurate oracle classification based on deep convolutional neural network. In: 2018 IEEE 18th International Conference on Communication Technology (ICCT), pp. 1188–1191 (2018)

    Google Scholar 

  34. Zhang, Y.K., Zhang, H., Liu, Y.G., Yang, Q., Liu, C.L.: Oracle character recognition by nearest neighbor classification with deep metric learning. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 309–314 (2019)

    Google Scholar 

  35. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199–1208 (2018)

    Google Scholar 

  36. Qiao, S., Liu, C., Shen, W., Yuille, A.L.: Few-shot image recognition by predicting parameters from activations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7229–7238 (2018)

    Google Scholar 

  37. Chen, D., Chen, Y., Li, Y., Mao, F., He, Y., Xue, H.: Self-supervised learning for few-shot image classification. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1745–1749. IEEE (2021)

    Google Scholar 

  38. Rusu, A.A., et al.: Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960 (2018)

  39. Chen, H., Li, H., Li, Y., Chen, C.: Multi-scale adaptive task attention network for few-shot learning. CoRR arXiv:abs/2011.14479 (2020)

  40. Qi, H., Brown, M., Lowe, D.G.: Low-shot learning with imprinted weights. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  41. Shyam, P., Gupta, S., Dukkipati, A.: Attentive recurrent comparators. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Volume 70 of Proceedings of Machine Learning Research, pp. 3173–3181. PMLR (2017)

    Google Scholar 

  42. DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)

  43. Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: learning augmentation strategies from data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 113–123 (2019)

    Google Scholar 

  44. Yue, X., Li, H., Fujikawa, Y., Meng, L.: Dynamic dataset augmentation for deep learning-based oracle bone inscriptions recognition. J. Comput. Cult. Herit. 15, 3532868 (2022)

    Google Scholar 

  45. Woods, R.P., Grafton, S.T., Holmes, C.J., Cherry, S.R., Mazziotta, J.C.: Automated image registration: I. general methods and intrasubject, intramodality validation. J. Comput. Assist. Tomogr. 22, 139–152 (1998)

    Google Scholar 

  46. Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. Neuroimage 45, S61–S72 (2009)

    Article  Google Scholar 

  47. Oliveira, F.P., Tavares, J.M.R.: Medical image registration: a review. Comput. Methods Biomech. Biomed. Engin. 17, 73–93 (2014). PMID: 22435355

    Article  Google Scholar 

  48. Ziyi, G., et al.: An improved neural network model based on inception-v3 for oracle bone inscription character recognition. Scientific Programming (2022)

    Google Scholar 

  49. Jiang, J., Luk, W., Rueckert, D.: FPGA-based computation of free-form deformations in medical image registration. In: Proceedings. 2003 IEEE International Conference on Field-Programmable Technology (FPT)(IEEE Cat. No. 03EX798), pp. 234–241. IEEE (2003)

    Google Scholar 

  50. Rohlfing, T., Maurer, C.R.: Nonrigid image registration in shared-memory multiprocessor environments with application to brains, breasts, and bees. IEEE Trans. Inf Technol. Biomed. 7, 16–25 (2003)

    Article  Google Scholar 

  51. Gribbon, K., Bailey, D.: A novel approach to real-time bilinear interpolation. In: Proceedings. DELTA 2004. Second IEEE International Workshop on Electronic Design, Test and Applications, pp. 126–131 (2004)

    Google Scholar 

  52. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  53. Zagoruyko, S., Komodakis, N.: Wide residual networks. CoRR arXiv:abs/1605.07146 (2016)

  54. Mangla, P., Kumari, N., Sinha, A., Singh, M., Krishnamurthy, B., Balasubramanian, V.N.: Charting the right manifold: manifold mixup for few-shot learning. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2218–2227 (2020)

    Google Scholar 

  55. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems 32, pp. 8024–8035. Curran Associates, Inc. (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Data Science, Fudan University, Shanghai, China

    Xinyi Zhao, Siyuan Liu, Yikai Wang & Yanwei Fu

Authors
  1. Xinyi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siyuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yikai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanwei Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanwei Fu .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, X., Liu, S., Wang, Y., Fu, Y. (2023). FFD Augmentor: Towards Few-Shot Oracle Character Recognition from Scratch. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13845. Springer, Cham. https://doi.org/10.1007/978-3-031-26348-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26348-4_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26347-7

  • Online ISBN: 978-3-031-26348-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation