Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Visual Computing

ISVC 2016: Advances in Visual Computing pp 11–22Cite as

Guided Text Spotting for Assistive Blind Navigation in Unfamiliar Indoor Environments

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10073))

Abstract

Scene text in indoor environments usually preserves and communicates important contextual information which can significantly enhance the independent travel of blind and visually impaired people. In this paper, we present an assistive text spotting navigation system based on an RGB-D mobile device for blind or severely visually impaired people. Specifically, a novel spatial-temporal text localization algorithm is proposed to localize and prune text regions, by integrating stroke-specific features with a subsequent text tracking process. The density of extracted text-specific feature points serves as an efficient text indicator to guide the user closer to text-likely regions for better recognition performance. Next, detected text regions are binarized and recognized by off-the-shelf optical character recognition methods. Significant non-text indicator signage can also be matched to provide additional environment information. Both recognized results are then transferred to speech feedback for user interaction. Our proposed video text localization approach is quantitatively evaluated on the ICDAR 2013 dataset, and the experimental results demonstrate the effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://tinyurl.com/who-blindness.

  2. 2.

    https://get.google.com/tango.

  3. 3.

    https://github.com/tesseract-ocr.

  4. 4.

    http://tinyurl.com/android-tts.

References

  1. Xiong, B., Grauman, K.: Text detection in stores using a repetition prior. In: WACV (2016)

    Google Scholar 

  2. Qin, S., Manduchi, R.: A fast and robust text spotter. In: WACV (2016)

    Google Scholar 

  3. Yin, X., Zuo, Z., Tian, S., Liu, C.: Text detection, tracking and recognition in video: a comprehensive survey. IEEE Trans. Image Process. (2016)

    Google Scholar 

  4. Busta, M., Neumann, L., Matas, J.: FASText: efficient unconstrained scene text detector. In: ICCV (2015)

    Google Scholar 

  5. Jaderberg, M., Vedaldi, A., Zisserman, A.: Deep features for text spotting. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 512–528. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10593-2_34

    Google Scholar 

  6. Yin, X., Yin, X., Huang, K., Hao, H.: Robust text detection in natural scene images. IEEE Trans. Pattern Anal. Mach. Intell. (2014)

    Google Scholar 

  7. Rakshit, S., Basu, S.: Recognition of handwritten roman script using tesseract open source ocr engine. arXiv.org (2010)

  8. Munõz, J.P., Li, B., Rong, X., Xiao, J., Tian, Y., Arditi, A.: Demo: assisting visually impaired people navigate indoors. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 4260–4261 (2016)

    Google Scholar 

  9. Lees, Y., Medioni, G.: RGB-D camera based wearable navigation system for the visually impaired. Comput. Vis. Image Underst. 149, 3–20 (2016)

    Article  Google Scholar 

  10. Li, B., Muñoz, J.P., Rong, X., Xiao, J., Tian, Y., Arditi, A.: ISANA: wearable context-aware indoor assistive navigation with obstacle avoidance for the blind. In: Hua, G., Jégou, H. (eds.) ECCV 2016 Workshop. LNCS, vol. 9914, pp. 448–462. Springer, Heidelberg (2016)

    Chapter  Google Scholar 

  11. Li, B., Zhang, X., Muñoz, J.P., Xiao, J., Rong, X., Tian, Y.: Assisting blind people to avoid obstacles: an wearable obstacle stereo feedback system based on 3D detection. In: IEEE International Conference on Robotics and Biomimetics (ROBIO) (2015)

    Google Scholar 

  12. Rong, X., Yi, C., Yang, X., Tian, Y.: Scene text recognition in multiple frames based on text tracking. In: IEEE International Conference on Multimedia and Expo (2014)

    Google Scholar 

  13. Rong, X., Yi, C., Tian, Y.: Recognizing text-based traffic guide panels with cascaded localization network. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 109–121. Springer, Heidelberg (2016). doi:10.1007/978-3-319-46604-0_8

    Chapter  Google Scholar 

  14. Yi, C., Tian, Y.: Text string detection from natural scenes by structure-based partition and grouping. IEEE Trans. Image Process. 20, 2594–2605 (2011)

    Article  MathSciNet  Google Scholar 

  15. Yi, C., Tian, Y., Arditi, A.: Portable camera-based assistive text and product label reading from hand-held objects for blind persons. IEEE Trans. Mechatron. 19, 808–817 (2014)

    Article  Google Scholar 

  16. Balntas, V., Tang, L., Mikolajczyk, K.: Bold - binary online learned descriptor for efficient image matching. In: CVPR (2015)

    Google Scholar 

  17. Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32, 448–461 (2010)

    Article  Google Scholar 

  18. Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: ICCV (2005)

    Google Scholar 

  19. Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression a statistical view of boosting. Ann. Stat. 28, 337–407 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  20. Karatzas, D.: ICDAR 2013 robust reading competition. In: ICDAR (2013)

    Google Scholar 

  21. Goto, H., Tanaka, M.: Text-tracking wearable camera system for the blind. In: ICDAR (2009)

    Google Scholar 

  22. Wu, L., Shivakumara, P., Lu, T.: A new technique for multi-oriented scene text line detection and tracking in video. IEEE Trans. Multimed. 17, 1137–1152 (2015)

    Article  Google Scholar 

  23. Cambra, A., Murillo, A.: Towards robust and efficient text sign reading from a mobile phone (2011)

    Google Scholar 

  24. Li, H., Doermann, D., Kia, O.: Automatic text detection and tracking in digital video. IEEE Trans. Image Process. 9, 147–156 (2000)

    Article  Google Scholar 

  25. Mosleh, A., Bouguila, N., Hamza, A.: Automatic inpainting scheme for video text detection and removal. IEEE Trans. Image Process. 22, 4460–4472 (2013)

    Article  MathSciNet  Google Scholar 

  26. Zhao, X., Lin, K., Fu, Y., Hu, Y., Liu, Y.: Text from corners: a novel approach to detect text and caption in videos. IEEE Trans. Image Process. 20, 790–799 (2011)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported in part by U.S. Federal Highway Administration (FHWA) grant DTFH 61-12-H-00002, National Science Foundation (NSF) grants CBET-1160046, EFRI-1137172 and IIP-1343402, National Institutes of Health (NIH) grant EY023483.

Author information

Authors and Affiliations

  1. The City College, City University of New York, New York, New York, USA

    Xuejian Rong, Bing Li, Jizhong Xiao & Yingli Tian

  2. The Graduate Center, City University of New York, New York, New York, USA

    J. Pablo Muñoz, Jizhong Xiao & Yingli Tian

  3. Visibility Metrics LLC, Chappaqua, New York, USA

    Aries Arditi

Authors
  1. Xuejian Rong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Pablo Muñoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jizhong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aries Arditi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yingli Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuejian Rong .

Editor information

Editors and Affiliations

  1. University of Nevada, Reno, Nevada, USA

    George Bebis

  2. NASA Ames Research Center, Moffett Field, California, USA

    Richard Boyle

  3. Lawrence Berkeley National Laboratory, Berkeley, California, USA

    Bahram Parvin

  4. Desert Research Institute, Reno, Nevada, USA

    Darko Koracin

  5. The Australian National University, O’Malley, Aust Capital Terr, Australia

    Fatih Porikli

  6. Pilot AI Labs, Redwood City, California, USA

    Sandra Skaff

  7. University of Florida, Gainesville, Florida, USA

    Alireza Entezari

  8. Google Inc., Mountain View, California, USA

    Jianyuan Min

  9. Osaka University, Osaka, Japan

    Daisuke Iwai

  10. The MOVES Institute, Monterey, California, USA

    Amela Sadagic

  11. University of Arizona, Tucson, Arizona, USA

    Carlos Scheidegger

  12. Université Paris-Sud, Orsay, France

    Tobias Isenberg

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Rong, X., Li, B., Muñoz, J.P., Xiao, J., Arditi, A., Tian, Y. (2016). Guided Text Spotting for Assistive Blind Navigation in Unfamiliar Indoor Environments. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2016. Lecture Notes in Computer Science(), vol 10073. Springer, Cham. https://doi.org/10.1007/978-3-319-50832-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50832-0_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50831-3

  • Online ISBN: 978-3-319-50832-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation