Crosswalk Recognition Through Point-Cloud Processing and Deep-Learning Suited to a Wearable Mobility Aid for the Visually Impaired

  • Matteo PoggiEmail author
  • Luca Nanni
  • Stefano Mattoccia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9281)


In smart-cities, computer vision has the potential to dramatically improve the quality of life of people suffering of visual impairments. In this field, we have been working on a wearable mobility aid aimed at detecting in real-time obstacles in front of a visually impaired. Our approach relies on a custom RGBD camera, with FPGA on-board processing, worn as traditional eyeglasses and effective point-cloud processing implemented on a compact and lightweight embedded computer. This latter device also provides feedback to the user by means of an haptic interface as well as audio messages. In this paper we address crosswalk recognition that, as pointed out by several visually impaired users involved in the evaluation of our system, is a crucial requirement in the design of an effective mobility aid. Specifically, we propose a reliable methodology to detect and categorize crosswalks by leveraging on point-cloud processing and deep-learning techniques. The experimental results reported, on 10000+ frames, confirm that the proposed approach is invariant to head/camera pose and extremely effective even when dealing with large occlusions typically found in urban environments.


Wearable Embedded 3d vision Deep learning Crosswalk detection 


  1. 1.
    Haselhoff, A., Kummert, A.: On visual crosswalk detection for driver assistance systems. In: 2010 IEEE Intelligent Vehicles Symposium (IV), pp. 883–888, June 2010Google Scholar
  2. 2.
    Wu, T., Ranganathan, A.: A practical system for road marking detection and recognition. In: 2012 IEEE Intelligent Vehicles Symposium (IV), pp. 25–30, June 2012Google Scholar
  3. 3.
    Mancini, A., Frontoni, E., Zingaretti, P.: Automatic road object extraction from mobile mapping systems. In: 2012 IEEE/ASME International Conference on Mechatronics and Embedded Systems and Applications (MESA), pp. 281–286, July 2012Google Scholar
  4. 4.
    Hata, A., Wolf, D.: Road marking detection using lidar reflective intensity data and its application to vehicle localization. In: 2014 IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), pp. 584–589, October 2014Google Scholar
  5. 5.
    Suzuki, S., Raksincharoensak, P., Shimizu, I., Nagai, M., Adomat, R.: Sensor fusion-based pedestrian collision warning system with crosswalk detection. In: 2010 IEEE Intelligent Vehicles Symposium (IV), pp. 355–360, June 2010Google Scholar
  6. 6.
    Ishizaki, R., Morimoto, M., Fujii, K.: An evaluation method of driving behavior by in-vehicle data camera. In: 2012 Fifth International Conference on Emerging Trends in Engineering and Technology (ICETET), pp. 293–297, November 2012Google Scholar
  7. 7.
    Shioyama, T., Wu, H., Nishibe, Y., Nakamura, N., Kitawaki, S.: Image analysis of crosswalk. In: proceedings of the 11th International Conference on Image Analysis and Processing, pp. 168–173, September 2001Google Scholar
  8. 8.
    Ivanchenko, V., Coughlan, J., Shen, H.: Detecting and locating crosswalks using a camera phone. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2008, pp. 1–8, June 2008Google Scholar
  9. 9.
    Ahmetovic, D., Bernareggi, C., Gerino, A., Mascetti, S.: Zebrarecognizer: efficient and precise localization of pedestrian crossings. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp. 2566–2571, August 2014Google Scholar
  10. 10.
    Radvanyi, M., Varga, B., Karacs, K.: Advanced crosswalk detection for the bionic eyeglass. In: 2010 12th International Workshop on Cellular Nanoscale Networks and Their Applications (CNNA), pp. 1–5, February 2010Google Scholar
  11. 11.
    Wang, S., Tian, Y.: Detecting stairs and pedestrian crosswalks for the blind by rgbd camera. In: 2012 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW), pp. 732–739, October 2012Google Scholar
  12. 12.
    Murali, V.N., Coughlan, J.M.: Smartphone-based crosswalk detection and localization for visually impaired pedestrians. In: 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–7, July 2013Google Scholar
  13. 13.
    Mattoccia, S., Macri’, P.: 3d glasses to improve autonomous mobility of people visually impaired. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV Workshop. LNCS, vol. 8927, pp. 539–554. Springer, Switzerland (2014)Google Scholar
  14. 14.
    Mattoccia, S., Marchio, I., Casadio, M.: A compact 3d camera suited for mobile and embedded vision applications. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 195–196, June 2014Google Scholar
  15. 15.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRefGoogle Scholar
  16. 16.
    Hard-Kernel: Odroid u3.
  17. 17.
    Choi, S., Kim, T., Yu, W.: Performance evaluation of ransac family. In: BMVC (2009)Google Scholar
  18. 18.
    Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, pp. 1–22 (2004)Google Scholar
  19. 19.
    Jarrett, K., Kavukcuoglu, K., Ranzato, M.A., LeCun, Y.: What is the best multi-stage architecture for object recognition? In: Proc. International Conference on Computer Vision (ICCV 2009). IEEE (2009)Google Scholar
  20. 20.
    Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: a matlab-like environment for machine learning. In: BigLearn, NIPS Workshop (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Computer Science and Engineering (DISI)University of BolognaBolognaItaly

Personalised recommendations