KrNet: A Kinetic Real-Time Convolutional Neural Network for Navigational Assistance

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10897)


Over the past years, convolutional neural networks (CNN) have not only demonstrated impressive capabilities in computer vision but also created new possibilities of providing navigational assistance for people with visually impairment. In addition to obstacle avoidance and mobile localization, it is helpful for visually impaired people to perceive kinetic information of the surrounding. Road barrier, as a specific obstacle as well as a sign of entrance or exit, is an underlying hazard ubiquitously in daily environments. To address the road barrier recognition, this paper proposes a novel convolutional neural network named KrNet, which is able to execute scene classification on mobile devices in real time. The architecture of KrNet not only features depthwise separable convolution and channel shuffle operation to reduce computational cost and latency, but also takes advantage of Inception modules to maintain accuracy. Experimental results are presented to demonstrate qualified performance for the meaningful and useful applications of navigational assistance within residential and working area.


Convolutional neural network Scene classification Mobile navigational assistance Visually impaired 


  1. 1.
    Bourne, R.R.A., Flaxman, S.R.: Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. Lancet Glob. Health 5, e888–e897 (2017)CrossRefGoogle Scholar
  2. 2.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1–9 (2012)Google Scholar
  3. 3.
    Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 640–651 (2017)CrossRefGoogle Scholar
  4. 4.
    Lin, J., Wang, W.J., Huang, S.K., Chen, H.C.: Learning based semantic segmentation for robot navigation in outdoor environment. In: 2017 Joint 17th World Congress of International Fuzzy Systems Association and 9th International Conference on Soft Computing and Intelligent Systems (IFSA-SCIS), pp. 1–5 (2017)Google Scholar
  5. 5.
    Arroyo, R., Alcantarilla, P.F., Bergasa, L.M., Romera, E.: Fusion and binarization of CNN features for robust topological localization across seasons. In: IEEE International Conference on Intelligent Robots and Systems, pp. 4656–4663 (2016)Google Scholar
  6. 6.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. ImageNet Challenge, pp. 1–10 (2014)Google Scholar
  8. 8.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 7-12-NaN-2015, pp. 1–9 (2015)Google Scholar
  9. 9.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)Google Scholar
  10. 10.
    Han, S., Mao, H., Dally, W.J.: A Deep Neural Network Compression Pipeline: Pruning, Quantization, Huffman Encoding. arXiv:1510.00149 [cs], p. 13 (2015)
  11. 11.
    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. Comput. Sci. 1–9 (2015).
  12. 12.
    Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., Keutzer, K.: SqueezeNet. arXiv, pp. 1–5 (2016)Google Scholar
  13. 13.
    Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv, p. 9 (2017)Google Scholar
  14. 14.
    Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. arXiv, pp. 1–10 (2017)Google Scholar
  15. 15.
    Yang, K., Wang, K., Hu, W., Bai, J.: Expanding the detection of traversable area with RealSense for the visually impaired. Sensors 16, 1954 (2016)CrossRefGoogle Scholar
  16. 16.
    Yang, K., Wang, K., Cheng, R., Hu, W., Huang, X., Bai, J.: Detecting traversable area and water hazards for the visually impaired with a pRGB-D sensor. Sensors 17, 1890 (2017)CrossRefGoogle Scholar
  17. 17.
    Cheng, R., Wang, K., Yang, K., Long, N., Hu, W.: Crosswalk navigation for people with visual impairments on a wearable device. J. Electron. Imaging 26, 1 (2017)Google Scholar
  18. 18.
    Cheng, R., Wang, K., Yang, K., Long, N., Bai, J., Liu, D.: Real-time pedestrian crossing lights detection algorithm for the visually impaired. Multimedia Tools Appl. 1–21 (2017).
  19. 19.
  20. 20.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the Inception Architecture for Computer Vision (2015)Google Scholar
  21. 21.
    Chollet, F.: Xception: Deep Learning with Separable Convolutions. arXiv Preprint arXiv:1610.02357, pp. 1–14 (2016)
  22. 22.
    Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: International Conference for Learning Representations, pp. 1–15 (2015)Google Scholar
  23. 23.
    Road barrier dataset.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.State Key Laboratory of Modern Optical InstrumentationZhejiang UniversityHangzhouChina

Personalised recommendations