Abstract
Although image captioning has made great progress in describing images, current methods are limited in 2D recognition of salient and moving objects. This leads to sentences that lack information about static and background objects, with poor performance on words’ order and prepositions, which cannot be enough for blind people to completely understand a scene. They also don’t give precise information about spatial relationships of the detected objects, reducing by that the amount of information that the scene contains. In this paper, we will first explore the existing methods of state-of-the-art for image captioning, we will learn about the approaches, and highlight their shortcomings considering the egocentric guidance of blind and visually impaired people. We will explore and test some tools that may be needed in the future to augment them with new information and new functionalities that are needed by the visually impaired to be able to not only understand their environment, but to also move around it and have a complete awareness of the objects present in their scene.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Although used in [9], it is only estimated whereas nowadays we can collect the information with high precision using depth cameras such as the Microsoft Kinect.
References
Investigative Ophthalmology & Visual Science, vol. 61, p. 2317, June 2020
Merchan, F., Poveda, M., Cáceres-Hernández, D., Sanchez Galan, F.J.: Indoor navigation aid systems for the blind and visually impaired based on depth sensors. IGI Global, 2021, pp. 187–223 (2021). https://doi.org/10.4018/978-1-7998-6522-3.ch007
Zatout, C., Slimane, L., Ilyes, M., Ablam, S.: Ego-semantic labeling of scene from depth image for visually impaired and blind people. ICCVW 2019, 4376–4384 (2019). https://doi.org/10.1109/ICCVW.2019.00538
https://www.azuremagazine.com/article/tech-products-for-the-visually-impaired/
Zongjian, Z., Qiang, W., Yang, W., Fang, C.: Exploring region relationships implicitly: image captioning with visual relationship attention. Image Vis. Comput. 109, 104146 (2021). https://doi.org/10.1016/j.imavis.2021.104146
Aditya, S., Yang, Y., Baral, C., Aloimonos, Y., Fermüller, C.: Image understanding using vision and reasoning through scene description graph. Comput. Vis. Image Underst. (2017). https://doi.org/10.1016/j.cviu.2017.12.004
Su, J., Tang, J., Lu, Z., Han, X., Zhang, H.: A neural image captioning model with caption-to-images semantic constructor. Neurocomputing 367 (2019). https://doi.org/10.1016/j.neucom.2019.08.012
Wei, Y., Wu, C., Jia, Z., Hu, X., Guo, S., Shi, H.: Past is important: Improved image captioning by looking back in time. Signal Process. Image Commun. 94, 116183 (2021). https://doi.org/10.1016/j.image.2021.116183
Wang, Z., Luo, Y., Li, Y., Huang, Z., Yin, H.: Look deeper see richer: depth-aware image paragraph captioning, pp. 672–680 (2018). https://doi.org/10.1145/3240508.3240583
Li, R., Liang, H., Shi, Y., Feng, F., Wang, X.: Dual-CNN: a convolutional language decoder for paragraph image captioning. Neurocomputing 396 (2020). https://doi.org/10.1016/j.neucom.2020.02.041
Chunpu, X., Min, Y., Xiang, A., Ying, S., Ruifeng, X., Jinwen, T.: Retrieval-enhanced adversarial training with dynamic memory-augmented attention for image paragraph captioning. Knowl.-Based Syst. 214, 106730 (2020). https://doi.org/10.1016/j.knosys.2020.106730
Kelvin, X., et al.: Show, attend and tell: neural image caption generation with visual attention (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Delloul, K., Larabi, S. (2022). Image Captioning State-of-the-Art: Is It Enough for the Guidance of Visually Impaired in an Environment?. In: Senouci, M.R., Boulahia, S.Y., Benatia, M.A. (eds) Advances in Computing Systems and Applications. CSA 2022. Lecture Notes in Networks and Systems, vol 513. Springer, Cham. https://doi.org/10.1007/978-3-031-12097-8_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-12097-8_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-12096-1
Online ISBN: 978-3-031-12097-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)