ICIAR 2016: Image Analysis and Recognition pp 290-295 | Cite as
Multiple Object Scene Description for the Visually Impaired Using Pre-trained Convolutional Neural Networks
Abstract
This paper introduces a new method for multiple object scene description as part of a system to guide the visually impaired in an indoor environment. Here we are interested in a coarse scene description, where only the presence of certain objects is indicated regardless of its position in the scene. The proposed method is based on the extraction of powerful features using pre-trained convolutional neural networks (CNN), then training a Neural Network regression to predict the content of any unknown scene based on its CNN feature. We have found the CNN feature to be highly descriptive, even though it is trained on auxiliary data from a completely different domain.
The proposed methodology was assessed on four datasets representing different indoor environments. It achieves better results in terms of both accuracy and processing time when compared to state-of-the art.
Keywords
Visual impaired Multiple object indoor scene description Image multiple labeling Convolutional neural networks (CNN) NN regressionReferences
- 1.Mekhalfi, M.L., Melgani, F., Bazi, Y., Alajlan, N.: Toward an assisted indoor scene perception for blind people with image multilabeling strategies. Expert Syst. Appl. 42(6), 2907–2918 (2015)CrossRefGoogle Scholar
- 2.Mekhalfi, M.L., Melgani, F., Bazi, Y., Alajlan, N.: A compressive sensing approach to describe indoor scenes for blind people. IEEE Trans. Circuits Syst. Video Technol. 25(7), 1246–1257 (2015)CrossRefGoogle Scholar
- 3.Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)CrossRefGoogle Scholar
- 4.Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
- 5.Razavi, N., Gall, J., Van Gool, L.: Scalable multi-class object detection. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 1505–1512 (2011)Google Scholar
- 6.Caffe | Deep Learning Framework. http://caffe.berkeleyvision.org/. Accessed 09 Mar 2016
- 7.ConvNet: Deep Convolutional Networks. http://libccv.org/doc/doc-convnet/. Accessed 09 Mar 2016
- 8.Torch | Scientific computing for LuaJIT. http://torch.ch/. Accessed 09 Mar 2016
- 9.Nervana Neon - Scalable Deep Learning library. http://www.nersc.gov/users/data-analytics/data-analytics/neon-scalable-deep-learning-library/. Accessed 09 Mar 2016