Abstract
Currently, the demand for automation in information technology industry is increasing at rapid rate. Leading companies in artificial intelligence and computer vision have started investing in the research and making new products that can easily do the redundant work done by humans, ultimately increasing the productivity and reducing the cost of doing work. Industrial artificial intelligence can create new business models, can automate the industrial tasks which are redundant and can be easily done by specifically trained machines. The work is experimented with various feature extraction methods, like Visual Geometry Group (VGG-16) and VGG-19, and a smaller proposed five-layer convolution neural network (CNN-5) to generate captions for an image. The performances of these methods are compared using BLEU-1, BLEU-2, BLEU-3 and BLEU-4 scores. Various experiments have been conducted to see the effect of different hyper-parameters, like number of layers, learning rate, dropout, on the model accuracy measured by BLEU score. The effect of the input length (15, 34 and 40) of the sequence of text data on the LSTM network is also being analyzed and reported in the results section. The experiments are conducted on benchmark Flickr-8 K dataset which contains 8000 images and their respective descriptions in a text file. Each image was described by five different people and the text file contains image id and their respective five descriptions for each of the 8000 images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. (2010). Every picture tells a story: Generating sentences from images. In Proceedings of the 11th European Conference on Computer Vision: Part IV, ECCV’10 (pp. 15–29). Berlin, Heidelberg: Springer.
Kuznetsova, P., Ordonez, V., Berg, A. C., Berg, T. L. , & Choi, Y. (2012). Collective generation of natural image descriptions. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers. ACL’12 (Vol. 1, pp. 359–368). Stroudsburg, PA, USA. Association for Computational Linguistics.
Li, S., Kulkarni, G., Berg, T. L., Berg, A. C., & Choi, Y. (2011). Composing simple image descriptions using web-scale n-grams. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning, CoNLL’11 (pp. 220–228). Stroudsburg, PA, USA: Association for Computational Linguistics.
Chen, X., & Zitnick, C. L. (2014). Learning a recurrent visual representation for image caption generation. CoRR, abs/1411.5654.
Mao, J., Xu, W., Yang, Y., Wang, J., & Yuille, A. L. (2014). Deep captioning with multimodal recurrent neural networks (m-rnn). CoRR, abs/1412.6632.
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2014). Show and tell: A neural image caption generator. CoRR, abs/1411.4555.
Mandal, I. (2015). Developing new machine learning ensembles for quality spine diagnosis. Knowledge-based systems, 73, 298–310. https://doi.org/10.1016/j.knosys.2014.10.012. ISSN 0950-7051.
Mandal, I., & Sairam, N. (2013). Accurate telemonitoring of Parkinson’s disease diagnosis using robust inference system. International Journal of Medical Informatics, 82(5), 359–377. https://doi.org/10.1016/j.ijmedinf.2012.10.006. ISSN 1386-5056.
Mandal, Indrajit, & Sairam, N. (2012). New machine-learning algorithms for prediction of Parkinson’s disease. International Journal of Systems Science, 45(3), 647–666. https://doi.org/10.1080/00207721.2012.724114.
Mandal, I., & Sairam, N. (2012). Accurate prediction of coronary artery disease using reliable diagnosis system. Journal of Medical Systems, 36, 3353. https://doi.org/10.1007/s10916-012-9828-0.
Mandal, Indrajit. (2014). A novel approach for accurate identification of splice junctions based on hybrid algorithms. Journal of Biomolecular Structure & Dynamics, 33(6), 1281–1290. https://doi.org/10.1080/07391102.2014.944218.
Mandal, Indrajit. (2016). Machine learning algorithms for the creation of clinical healthcare enterprise systems. Enterprise Information Systems, 11(9), 1374–1400. https://doi.org/10.1080/17517575.2016.1251617.
Mandal, I. (2015). A novel approach for predicting DNA splice junctions using hybrid machine learning algorithms. Soft Computing, 19, 3431. https://doi.org/10.1007/s00500-014-1550-z.
https://econsultancy.com/blog/61991-83-of-online-shoppers-need-support-to-complete-a-purchase-stats.
Kisilev, P., Sason, E., Barkan, E., & Hashoul, S. (2011). Medical image captioning: Learning to describe medical image findings using multi-task-loss CNN.
Tanti, M., Gatt, A., & Camilleri, K. P. (2017). What is the role of recurrent neural networks (RNNs) in an image caption generator? In INLG.
Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. In ICLR.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735. PMID 9377276.
Li, X., & Wu, X. (2014). Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition, October 15, 2014.
Rashtchian, C., Young, P., Hodosh, M., & Hockenmaier, J. (2010). Collecting image annotations using Amazon’s mechanical turk. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk.
Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002). BLEU: a method for automatic evaluation of machine translation. In ACL-2002: 40th Annual meeting of the Association for Computational Linguistics (pp. 311–318). CiteSeerX 10.1.1.19.9416 .
Marc, T., Albert, G., & Kenneth, C. (2017). Where to put the image in an image caption generator.
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015). Show and tell: A neural image caption generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3156–3164).
Johnson, J., Karpathy, A., & Fei-Fei, L. (2015). Densecap: Fully convolutional localization networks for dense captioning. arXiv preprint arXiv:1511.07571.
Wang, C., Yang, H., Bartz, C., & Meinel, C. (2016). Image captioning with deep bidirectional LSTMS. ArXiv preprint arXiv:1604.00790.
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2014). Show and tell: A neural image caption generator. CoRR, abs/1411.4555, 2014.
Young, P., Lai, A., Hodosh, M., & Hockenmaier, J. (2014). From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. TACL, 2, 67–78.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common objects in context. In Proceedings of the ECCV’14 (pp. 740–755).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Mandal, I., Dwivedi, A. (2020). Deep Learning Algorithms for Accurate Prediction of Image Description for E-commerce Industry. In: Sharma, N., Chakrabarti, A., Balas, V. (eds) Data Management, Analytics and Innovation. Advances in Intelligent Systems and Computing, vol 1016. Springer, Singapore. https://doi.org/10.1007/978-981-13-9364-8_29
Download citation
DOI: https://doi.org/10.1007/978-981-13-9364-8_29
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9363-1
Online ISBN: 978-981-13-9364-8
eBook Packages: EngineeringEngineering (R0)