Skip to main content
SpringerLink
Log in
Book cover

European Conference on Computer Vision

ECCV 2020: Computer Vision – ECCV 2020 pp 751–767Cite as

AutoSTR: Efficient Backbone Search for Scene Text Recognition

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12369))

Abstract

Scene text recognition (STR) is challenging due to the diversity of text instances and the complexity of scenes. However, no STR methods can adapt backbones to different diversities and complexities. In this work, inspired by the success of neural architecture search (NAS), we propose automated STR (AutoSTR), which can address the above issue by searching data-dependent backbones. Specifically, we show both choices on operations and the downsampling path are very important in the search space design of NAS. Besides, since no existing NAS algorithms can handle the spatial constraint on the path, we propose a two-step search algorithm, which decouples operations and downsampling path, for an efficient search in the given space. Experiments demonstrate that, by searching data-dependent backbones, AutoSTR can outperform the state-of-the-art approaches on standard benchmarks with much fewer FLOPS and model parameters. (Code is available at https://github.com/AutoML-4Paradigm/AutoSTR.git).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Almazán, J., Gordo, A., Fornés, A., Valveny, E.: Word spotting and recognition with embedded attributes. IEEE Trans. Pattern Anal. Mach. Intell. 36, 2552–2566 (2014)

    Article  Google Scholar 

  2. Baek, J., et al.: What is wrong with scene text recognition model comparisons? Dataset and model analysis. In: International Conference on Computer Vision (2019)

    Google Scholar 

  3. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Technical report arXiv preprint arXiv:1409.0473 (2014)

  4. Bai, F., Cheng, Z., Niu, Y., Pu, S., Zhou, S.: Edit probability for scene text recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  5. Cai, H., Zhu, L., Han, S.: ProxylessNAS: direct neural architecture search on target task and hardware. In: International Conference on Learning Representations (2019)

    Google Scholar 

  6. Chen, Y., Yang, T., Zhang, X., Meng, G., Xiao, X., Sun, J.: DetNAS: backbone search for object detection. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  7. Cheng, Z., Bai, F., Xu, Y., Zheng, G., Pu, S., Zhou, S.: Focusing attention: towards accurate text recognition in natural images. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)

    Google Scholar 

  8. Cheng, Z., Xu, Y., Bai, F., Niu, Y., Pu, S., Zhou, S.: Aon: towards arbitrarily-oriented text recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  9. Dutta, K., Mathew, M., Krishnan, P., Jawahar, C.: Localizing and recognizing text in lecture videos. In: International Conference on Frontiers in Handwriting Recognition (2018)

    Google Scholar 

  10. Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: a survey. J. Mach. Learn. Res. (2019)

    Google Scholar 

  11. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: International Conference on Machine Learning (2006)

    Google Scholar 

  12. Guo, Z., et al.: Single path one-shot neural architecture search with uniform sampling. Technical report arXiv preprint arXiv:1904.00420 (2019)

  13. Gupta, A., Vedaldi, A., Zisserman, A.: Synthetic data for text localisation in natural images. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2315–2324 (2016)

    Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  15. Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Synthetic data and artificial neural networks for natural scene text recognition. Technical report arXiv preprint arXiv:1406.2227 (2014)

  16. Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Reading text in the wild with convolutional neural networks. Int. J. Comput. Vis. 116, 1–20 (2016). https://doi.org/10.1007/s11263-015-0823-z

    Article  MathSciNet  Google Scholar 

  17. Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems (2015)

    Google Scholar 

  18. Karatzas, D., et al.: ICDAR 2015 competition on robust reading. In: International Conference on Document Analysis and Recognition (2015)

    Google Scholar 

  19. Karatzas, D., et al.: ICDAR 2013 robust reading competition. In: International Conference on Document Analysis and Recognition (2013)

    Google Scholar 

  20. Lee, C.Y., Osindero, S.: Recursive recurrent nets with attention modeling for OCR in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  21. Li, H., Wang, P., Shen, C., Zhang, G.: Show, attend and read: a simple and strong baseline for irregular text recognition. In: AAAI Conference on Artificial Intelligence (2019)

    Google Scholar 

  22. Li, L., Talwalkar, A.: Random search and reproducibility for neural architecture search. In: Uncertainty in Artificial Intelligence (2019)

    Google Scholar 

  23. Liao, M., et al.: Scene text recognition from two-dimensional perspective. In: AAAI Conference on Artificial Intelligence (2019)

    Google Scholar 

  24. Liu, C., et al.: Auto-deeplab: hierarchical neural architecture search for semantic image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  25. Liu, H., Simonyan, K., Yang, Y.: DARTS: differentiable architecture search. In: International Conference on Learning Representations (2019)

    Google Scholar 

  26. Liu, W., Chen, C., Wong, K.Y.K.: Char-net: a character-aware neural network for distorted scene text recognition. In: AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  27. Liu, Y., Wang, Z., Jin, H., Wassell, I.: Synthetically supervised feature learning for scene text recognition. In: European Conference on Computer Vision (2018)

    Google Scholar 

  28. Long, S., He, X., Yao, C.: Scene text detection and recognition: the deep learning era. Technical report arXiv preprint arXiv:1811.04256 (2018)

  29. Lucas, S.M., et al.: ICDAR 2003 robust reading competitions: entries, results, and future directions. Int. J. Doc. Anal. Recogn. 7, 105–122 (2005). https://doi.org/10.1007/s10032-004-0134-3

    Article  Google Scholar 

  30. Luo, C., Jin, L., Sun, Z.: MORAN: a multi-object rectified attention network for scene text recognition. Pattern Recogn. 90, 109–118 (2019)

    Article  Google Scholar 

  31. Mishra, A., Alahari, K., Jawahar, C.: Top-down and bottom-up cues for scene text recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)

    Google Scholar 

  32. Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Efficient neural architecture search via parameter sharing. In: International Conference on Machine Learning (2018)

    Google Scholar 

  33. Quy Phan, T., Shivakumara, P., Tian, S., Lim Tan, C.: Recognizing text with perspective distortion in natural scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (2013)

    Google Scholar 

  34. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  35. Sciuto, C., Yu, K., Jaggi, M., Musat, C., Salzmann, M.: Evaluating the search phase of neural architecture search. In: International Conference on Learning Representations (2020)

    Google Scholar 

  36. Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2298–2304 (2017)

    Article  Google Scholar 

  37. Shi, B., Wang, X., Lyu, P., Yao, C., Bai, X.: Robust scene text recognition with automatic rectification. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  38. Shi, B., Yang, M., Wang, X., Lyu, P., Yao, C., Bai, X.: Aster: an attentional scene text recognizer with flexible rectification. IEEE Trans. Pattern Anal. Mach. Intell. 41, 2035–2048 (2019)

    Article  Google Scholar 

  39. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Technical report arXiv preprint arXiv:1409.1556 (2014)

  40. Wang, K., Babenko, B., Belongie, S.: End-to-end scene text recognition. In: International Conference on Computer Vision (2011)

    Google Scholar 

  41. Xie, F., Zhang, M., Zhao, J., Yang, J., Liu, Y., Yuan, X.: A robust license plate detection and character recognition algorithm based on a combined feature extraction model and BPNN. J. Adv. Transp. 2018, 14 (2018)

    Google Scholar 

  42. Xie, L., Yuille, A.: Genetic CNN. In: IEEE International Conference on Computer Vision (2017)

    Google Scholar 

  43. Yang, M., et al.: Symmetry-constrained rectification network for scene text recognition. In: IEEE International Conference on Computer Vision (2019)

    Google Scholar 

  44. Yang, X., He, D., Zhou, Z., Kifer, D., Giles, C.L.: Learning to read irregular text with attention mechanisms. In: International Joint Conferences on Artificial Intelligence (2017)

    Google Scholar 

  45. Yao, Q., Xu, J., Tu, W.W., Zhu, Z.: Efficient neural architecture search via proximal iterations. In: AAAI Conference on Artificial Intelligence (2020)

    Google Scholar 

  46. Yao, Q., Wang, M.: Taking human out of learning applications: a survey on automated machine learning. Technical report arXiv preprint arXiv:1810.13306 (2018)

  47. Zeiler, M.: Adadelta: an adaptive learning rate method. Technical report arXiv preprint arXiv:1212.5701 (2012)

  48. Zhan, F., Lu, S.: ESIR: end-to-end scene text recognition via iterative image rectification. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  49. Zhu, Y., Yao, C., Bai, X.: Scene text detection and recognition: recent advances and future trends. Front. Comput. Sci. 10, 19–36 (2016). https://doi.org/10.1007/s11704-015-4488-0

    Article  Google Scholar 

  50. Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. In: International Conference on Learning Representations (2017)

    Google Scholar 

Download references

Acknowledgments

The work is performed when H. Zhang was an intern in 4Paradigm Inc. mentored by Dr. Q. Yao. This work was partially supported by National Key R&D Program of China (No. 2018YFB1004600), to Dr. Xiang Bai by the National Program for Support of Top-notch Young Professionals and the Program for HUST Academic Frontier Youth Team 2017QYTD08.

Author information

Authors and Affiliations

  1. Department of Electronics and Information Engineering, Huazhong University of Science and Technology, Wuhan, China

    Hui Zhang, Mingkun Yang, Yongchao Xu & Xiang Bai

  2. 4Paradigm Inc., Beijing, China

    Hui Zhang & Quanming Yao

Authors
  1. Hui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quanming Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingkun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongchao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quanming Yao .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, H., Yao, Q., Yang, M., Xu, Y., Bai, X. (2020). AutoSTR: Efficient Backbone Search for Scene Text Recognition. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12369. Springer, Cham. https://doi.org/10.1007/978-3-030-58586-0_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58586-0_44

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58585-3

  • Online ISBN: 978-3-030-58586-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation