Skip to main content
SpringerLink
Log in

A dataset for the recognition of obstacles on blind sidewalk

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Recently, the technology of assisting the navigation of visually impaired persons with computer vision has been greatly developed. A number of scholars have conducted related research, including indoor and outdoor object detection for blind people. However, there are still problems with some existing methods or datasets. Our work mainly proposes a dataset (OD) for assisting the detection and recognition of outdoor obstacles for blind people on blind sidewalk. We classify some common obstacles, train the dataset with state-of-the-art detectors to obtain detection models, and then analyze and compare these models in detail. The results show that our proposed dataset is very challenging. The OD and the detection model can be obtained at the following URL: https://github.com/TW0521/Obstacle-Dataset.git.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1
Fig.2
Fig.3
Fig.4
Fig.5
Fig.6
Fig.7
Fig.8
Fig.9

Similar content being viewed by others

References

  1. Katika, B.R., Karthik, K.: Face anti-spoofing by identity masking using random walk patterns and outlier detection. Pattern Anal. Appl. 23, 1735–1754 (2020). https://doi.org/10.1007/s10044-020-00875-8

    Article  Google Scholar 

  2. Sajjad, M., Nasir, M., Muhammad, K., Khan, S., Jan, Z., Sangaiah, A.K., Elhoseny, M., Baik, S.W.: Raspberry Pi assisted face recognition framework for enhanced law-enforcement services in smart cities. Futur. Gener. Comput. Syst. 108, 995–1007 (2020). https://doi.org/10.1016/j.future.2017.11.013

    Article  Google Scholar 

  3. Zhang, J., Wu, X., Hoi, S.C.H., Zhu, J.: Feature agglomeration networks for single stage face detection. Neurocomputing 380, 180–189 (2020). https://doi.org/10.1016/j.neucom.2019.10.087

    Article  Google Scholar 

  4. Chen, X., Wang, T., Zhu, Y., Jin, L., Luo, C.: Adaptive embedding gate for attention-based scene text recognition. Neurocomputing 381, 261–271 (2020). https://doi.org/10.1016/j.neucom.2019.11.049

    Article  Google Scholar 

  5. Wang, T., Zhu, Y., Jin, L., Luo, C., Chen, X., Wu, Y., Wang, Q., Cai, M.: Decoupled attention network for text recognition. (2019)

  6. Liao, M., Wan, Z., Yao, C., Chen, K., Bai, X.: Real-time scene text detection with differentiable binarization. arXiv. (2019). https://doi.org/10.1609/aaai.v34i07.6812

  7. Hao, Y., Xu, Z.J., Liu, Y., Wang, J., Fan, J.L.: Effective crowd anomaly detection through spatio-temporal texture analysis. Int. J. Autom. Comput. 16, 27–39 (2019). https://doi.org/10.1007/s11633-018-1141-z

    Article  Google Scholar 

  8. Krumm, J.C., Horvitz, E.J., Wolk, J.K.: Localized Anomaly Detection Using Contextual Signals. WO 2017048585 A1[P]

  9. Song, W., Jia, G., Zhu, H., Jia, D., Gao, L.: Automated pavement crack damage detection using deep multiscale convolutional features. J. Adv. Transp. (2020). https://doi.org/10.1155/2020/6412562

    Article  Google Scholar 

  10. Hassaballah, M., Kenk, M.A., El-Henawy, I.M.: Local binary pattern-based on-road vehicle detection in urban traffic scene. Pattern Anal. Appl. 23, 1505–1521 (2020). https://doi.org/10.1007/s10044-020-00874-9

    Article  Google Scholar 

  11. Bu, Q., Yang, G., Ming, X., Zhang, T., Feng, J., Zhang, J.: Deep transfer learning for gesture recognition with WiFi signals. Pers. Ubiquitous Comput. (2020). https://doi.org/10.1007/s00779-019-01360-8

    Article  Google Scholar 

  12. Hosni Mahmoud, H.A., Mengash, H.A.: A novel technique for automated concealed face detection in surveillance videos. Pers. Ubiquitous Comput. (2020). https://doi.org/10.1007/s00779-020-01419-x

    Article  Google Scholar 

  13. Xiaomeng, C.: A case study on the difficulty of outdoor activities in the college students with visual impairments. J. Suihua Univ. 37, 1–6 (2017)

    Google Scholar 

  14. KR-VISION Technology Co., L.: Krvision, http://www.krvision.cn/offical/page/assist1.html

  15. Tapu, R., Mocanu, B., Bursuc, A., Zaharia, T.: A smartphone-based obstacle detection and classification system for assisting visually impaired people. Proc. IEEE Int. Conf. Comput. Vis. 444–451 (2013). https://doi.org/10.1109/ICCVW.2013.65

  16. Gorapudi, R., Darsini, P.P., Kavya, U.N., Jaswanthi, O.: Product label, obstacle and sign boards detection for visually impaired people. SSRN Electron. J. (2020). https://doi.org/10.2139/ssrn.3643597

    Article  Google Scholar 

  17. Yadav, S., Joshi, R.C., Dutta, M.K., Kiac, M., Sikora, P.: Fusion of object recognition and obstacle detection approach for assisting visually challenged person. 2020 43rd Int. Conf. Telecommun. Signal Process. TSP 2020. 537–540 (2020). https://doi.org/10.1109/TSP49548.2020.9163434

  18. Jarraya, S.K., Al-Shehri, W.S., Ali, M.S.: Deep multi-layer perceptron-based obstacle classification method from partial visual information: application to the assistance of visually impaired people. IEEE Access. 8, 26612–26622 (2020). https://doi.org/10.1109/ACCESS.2020.2970979

    Article  Google Scholar 

  19. Afif, M., Ayachi, R., Said, Y., Pissaloux, E., Atri, M.: Recognizing signs and doors for indoor wayfinding for blind and visually impaired persons. 2020 Int. Conf. Adv. Technol. Signal Image Process. ATSIP 2020. 10–13 (2020). https://doi.org/10.1109/ATSIP49331.2020.9231933

  20. Afif, M., Ayachi, R., Said, Y., Pissaloux, E., Atri, M.: An evaluation of retinanet on indoor object detection for blind and visually impaired persons assistance navigation. Neural Process. Lett. 51, 2265–2279 (2020). https://doi.org/10.1007/s11063-020-10197-9

    Article  Google Scholar 

  21. Park, H., Lee, J.: Implementation of an obstacle recognition system for the blind. 2nd ieee eurasia conf. IOT, Commun. Eng. 2020, ECICE 2020. 125–128 (2020). https://doi.org/10.1109/ECICE50847.2020.9302019

  22. Park, H., Lee, J.: Implementation and evaluation of obstacle recognition system for the blind. 2nd IEEE Eurasia Conf. IOT, Commun. Eng. 2020, ECICE 2020. 125–128 (2020). https://doi.org/10.1109/ECICE50847.2020.9302019

  23. Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111, 98–136 (2015). https://doi.org/10.1007/s11263-014-0733-5

    Article  Google Scholar 

  24. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common objects in context. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 8693 LNCS, 740–755 (2014). https://doi.org/10.1007/978-3-319-10602-1_48

  25. Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34, 743–761 (2012). https://doi.org/10.1109/TPAMI.2011.155

    Article  Google Scholar 

  26. Zhang, S., Benenson, R., Schiele, B.: CityPersons: a diverse dataset for pedestrian detection. Proc. 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017. 2017-Janua, 4457–4465 (2017). https://doi.org/10.1109/CVPR.2017.474

  27. Braun, M., Krebs, S., Flohr, F., Gavrila, D.M.: EuroCity persons: a novel benchmark for person detection in traffic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1844–1861 (2019). https://doi.org/10.1109/TPAMI.2019.2897684

    Article  Google Scholar 

  28. Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Synthetic data and artificial neural networks for natural scene text recognition. 1–10 (2014)

  29. Veit, A., Matera, T., Neumann, L., Matas, J., Belongie, S.: COCO-Text: dataset and benchmark for text detection and recognition in natural images. (2016)

  30. Behrendt, K., Novak, L., Botros, R.: A deep learning approach to traffic lights: detection, tracking, and classification. Proc. IEEE Int. Conf. Robot. Autom. 1370–1377 (2017). https://doi.org/10.1109/ICRA.2017.7989163

  31. Zhu, Z., Liang, D., Zhang, S., Huang, X., Li, B., Hu, S.: Traffic-sign detection and classification in the wild. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2016-Decem, 2110–2118 (2016). https://doi.org/10.1109/CVPR.2016.232

  32. Yucel, M.K., Bilge, Y.C., Oguz, O., Ikizler-Cinbis, N., Duygulu, P., Cinbis, R.G.: Wildest faces: face detection and recognition in violent settings. arXiv. (2018)

  33. Nada, H., Sindagi, V.A., Zhang, H., Patel, V.M.: Pushing the limits of unconstrained face detection: a challenge dataset and baseline results. 2018 IEEE 9th Int. Conf. Biometrics Theory, Appl. Syst. BTAS 2018. 1–10 (2018). https://doi.org/10.1109/BTAS.2018.8698561

  34. Lam, D., Kuzma, R., McGee, K., Dooley, S., Laielli, M., Klaric, M., Bulatov, Y., McCord, B.: xView: Objects in context in overhead imagery. arXiv. (2018)

  35. Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: DOTA: a large-scale dataset for object detection in aerial images. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 3974–3983 (2018). https://doi.org/10.1109/CVPR.2018.00418

  36. Ta, T.L.: LabelImg, https://github.com/tzutalin/labelImg

  37. Jocher, G., Stoken, A., Borovec, J., NanoCode012, ChristopherSTAN, Changyu, L., Laughing, tkianai, Hogan, A., lorenzomammana, yxNONG, AlexWang1900, Diaconu, L., Marc, wanghaoyang0106, ml5ah, Doug, Ingham, F., Frederik, Guilhen, Hatovix, Poznanski, J., Fang, J., Yu, L., changyu98, Wang, M., Gupta, N., Akhtar, O., PetrDvoracek, Rai, P.: ultralytics/YOLO v5: v3.1 - Bug Fixes and Performance Improvements (2020). https://doi.org/10.5281/zenodo.4154370

  38. Girshick, R.: Fast R-CNN. Proc. IEEE Int. Conf. Comput. Vis. 2015 Inter, 1440–1448 (2015). https://doi.org/10.1109/ICCV.2015.169

  39. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: Single shot multibox detector. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 9905 LNCS, 21–37 (2016). https://doi.org/10.1007/978-3-319-46448-0_2

  40. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2016-Decem, 779–788 (2016). https://doi.org/10.1109/CVPR.2016.91

  41. Fu, C.Y., Liu, W., Ranga, A., Tyagi, A., Berg, A.C.: DSSD: Deconvolutional single shot detector. arXiv. (2017)

  42. Kolekar, A., Dalal, V.: Barcode detection and classification using SSD (single shot multibox detector) deep learning algorithm. SSRN Electron. J. (2020). https://doi.org/10.2139/ssrn.3568499

    Article  Google Scholar 

  43. Du, Y., Pan, N., Xu, Z., Deng, F., Shen, Y., Kang, H.: Pavement distress detection and classification based on YOLO network. Int. J. Pavement Eng. (2020). https://doi.org/10.1080/10298436.2020.1714047

    Article  Google Scholar 

  44. Huang, Z., Wang, J., Fu, X., Yu, T., Guo, Y., Wang, R.: DC-SPP-YOLO: Dense connection and spatial pyramid pooling based YOLO for object detection. Inf. Sci. (Ny) 522, 241–258 (2020). https://doi.org/10.1016/j.ins.2020.02.067

    Article  MathSciNet  Google Scholar 

  45. Zhu, X., Chen, C., Zheng, B., Yang, X., Gan, H., Zheng, C., Yang, A., Mao, L., Xue, Y.: Automatic recognition of lactating sow postures by refined two-stream RGB-D faster R-CNN. Biosyst. Eng. 189, 116–132 (2020). https://doi.org/10.1016/j.biosystemseng.2019.11.013

    Article  Google Scholar 

  46. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLO v4: Optimal speed and accuracy of object detection. arXiv. (2020)

  47. Parikh, N., Shah, I., Vahora, S.: Android smartphone based visual object recognition for visually impaired using deep learning. Proc. 2018 IEEE Int. Conf. Commun. Signal Process. ICCSP 2018. 420–425 (2018). https://doi.org/10.1109/ICCSP.2018.8524493

  48. Ying, J.C., Li, C.Y., Wu, G.W., Li, J.X., Chen, W.J., Yang, D.L.: A deep learning approach to sensory navigation device for blind guidance. In: Proceedings—20th international conference on high performance computing and communications, 16th international conference on smart city and 4th international conference on data science and systems, HPCC/SmartCity/DSS 2018. pp. 1195–1200 (2019)

  49. Zhou, Z., Lan, X., Li, S., Zhu, C., Chang, H.: Feature pyramid SSD: outdoor object detection algorithm for blind people. 2019 IEEE 5th Int. Conf. Comput. Commun. ICCC 2019. 650–654 (2019). https://doi.org/10.1109/ICCC47050.2019.9064251

  50. Arora, A., Grover, A., Chugh, R., Reka, S.S.: Real time multi object detection for blind using single shot multibox detector. Wirel. Pers. Commun. (2019). https://doi.org/10.1007/s11277-019-06294-1

    Article  Google Scholar 

  51. Shah, S., Bandariya, J., Jain, G., Ghevariya, M., Dastoor, S.: CNN based auto-assistance system as a boon for directing visually impaired person. Proc. Int. Conf. Trends Electron. Inf. (2019). https://doi.org/10.1109/ICOEI.2019.8862699

    Article  Google Scholar 

  52. Joshi, R., Tripathi, M., Kumar, A., Gaur, M.S.: Object recognition and classification system for visually impaired. In: Proceedings of the 2020 IEEE International Conference on Communication and Signal Processing, ICCSP 2020. pp. 1568–1572 (2020)

  53. Abraham, L., Mathew, N.S., George, L., Sajan, S.S.: VISION: wearable speech based feedback system for the visually impaired using computer vision. In: Proceedings of the 4th international conference on trends in electronics and informatics, ICOEI 2020. pp. 972–976 (2020)

Download references

Acknowledgements

We would like to acknowledge the anonymous reviewers and authors of cited papers for their detailed comments, without which this work would not have been possible. This work was supported by the National Natural Science Foundation of China (Nos. 41361077, 41561085) and the National Natural Science Foundation of Jiangxi Provence, China (No. 20202BAB202025).

Author information

Authors and Affiliations

  1. School of Civil and Surveying & Mapping Engineering, Jiangxi University of Science and Technology, Ganzhou, 341000, Jiangxi, China

    Wu Tang, De-er Liu & Zenghui Chen

  2. School of Economics and Management, Jiangxi University of Science and Technology, Ganzhou, 341000, Jiangxi, China

    Xiaoli Zhao

  3. Fujian Jingwei surveying and mapping information CO., Ltd, Fuzhou, 350000, Fujian, China

    Chen Zhao

Authors
  1. Wu Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. De-er Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoli Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zenghui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to De-er Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, W., Liu, De., Zhao, X. et al. A dataset for the recognition of obstacles on blind sidewalk. Univ Access Inf Soc 22, 69–82 (2023). https://doi.org/10.1007/s10209-021-00837-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-021-00837-9

Keywords

Search

Navigation