Skip to main content

Improvising the performance of image-based recommendation system using convolution neural networks and deep learning


Recommendation systems now hold special significance, for in a world full of choices, order is the need of the hour. Without proper sorting, the gift of choice means nothing. The online retail world is fast-paced and ever-growing. With the exponential waning of attention span, it has become crucial to convert a casual visitor into a buyer within a limited window. Different ways can be used to do this: analysing buying patterns, surveys, user–user relationships, user-item relationships, and so on. This can be done with simple data analysis or with complex algorithms—the data must be harnessed one way or another. Deep learning is a branch of machine learning that has now become synonymous with computer vision, as these deep architectures closely emulate the biological process of vision. In this paper, the primary focus is the incorporation of a recommendation system with the visual features of products. This is done with the help of a deep architecture and a series of “convolution” operations that cause the overlapping of edges and blobs in images. We find that when the dimensionality problem has been dealt with, the features extracted serve as good quality representations of the images. Our empirical study compares the different linear and nonlinear reduction techniques on convolutional neural network features for building a recommendation model entirely based on the images.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15


  • Andreeva E, Ignatov DI, Grachev A, Savchenko AV (2018). Extraction of visual features for recommendation of products via deep learning. In: International conference on analysis of images, social networks and texts. Springer, Cham 2018, July, pp 201–210

  • Babenko A, Slesarev A, Chigorin A, Lempitsky V (2014) Neural codes for image retrieval. In: European conference on computer vision. Springer, Cham, Sept 2014, pp 584–599

  • Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009). Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, 2009 June. IEEE, pp. 248–255

  • Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014). Decaf: A deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, Jan 2014, pp 647–655

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Hoo-Chang S, Roth HR, Gao M, Lu L, Xu Z, Nogues I et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285

    Article  Google Scholar 

  • Hu J, Shen L, Sun G (2017) Squeeze-and-excitation networks. arXiv preprint arXiv:1709.01507, 7

  • Jing Y, Baluja S (2008) Visualrank: applying pagerank to large-scale image search. IEEE Trans Pattern Anal Mach Intell 30(11):1877–1890

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  • LeCun Y et al (1998) Gradient-based learning applied to document recognition. In: Proceedings of the IEEE 86.11, pp 2278–2324

  • Maaten LVD, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  • McInnes L, Healy J, Melville J (2018) Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426

  • Ramesh N, Moh T (2018) Outfit recommender system. 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM).

  • Shankar D, Narumanchi S, Ananya HA, Kompalli P, Chaudhury K (2017) Deep learning based large scale visual recommendation and search for e-commerce. arXiv preprint arXiv:1703.02344

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  • Sun A (2017) Deep learning based recommender system: a survey and new perspectives. arXiv preprint arXiv:1707.07435

  • Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  • Wang J, Song Y, Leung T, Rosenberg C, Wang J, Philbin J et al (2014) Learning fine-grained image similarity with deep ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp 1386–1393

  • Wattenberg M, Viégas F, Johnson I (2016) How to use t-SNE effectively. Distill 1(10):e2

    Article  Google Scholar 

  • Yosinski J, Clune J, BengioY, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in neural information processing systems, pp. 3320–3328

  • Zhai A, Kislyuk D, Jing Y, Feng M, Tzeng E, Donahue J, et al (2017) Visual discovery at pinterest. In: Proceedings of the 26th international conference on world wide web companion. International world wide web conferences steering committee, April 2017, pp 515–524

  • Zhang R, Isola P, Efros AA, Shechtman E, Wang O (2018) The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 586–595

  • Zheng L, Zhao Y, Wang S, Wang J, Tian Q (2016) Good practice in CNN feature transfer. arXiv preprint arXiv:1604.00133

  • Zhong G, Wang LN, Ling X, Dong J (2016) An overview on data representation learning: from traditional feature learning to recent deep learning. J Finance Data Sci 2(4):265–278

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to A. Razia Sulthana.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

We used our own data.

Human/animal rights

Humans/animals are not involved in this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by V. Loia.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sulthana, A.R., Gupta, M., Subramanian, S. et al. Improvising the performance of image-based recommendation system using convolution neural networks and deep learning. Soft Comput 24, 14531–14544 (2020).

Download citation

  • Published:

  • Issue Date:

  • DOI:


  • Deep learning
  • Convolution neural network
  • Recommendation system
  • Dimensionality reduction
  • Tensor flow