Recently, recommendation system has become popular in many e-commerce websites. It helps users by suggesting products which they could buy. Existing work till now uses past feedback of user, similarity of other users’ buying pattern, or a hybrid approach in which both type of information is used. But the pitfall of these approaches is that there is a need to collect and process huge amount of data for good recommendation. This paper is aimed at developing an efficient recommendation system by incorporating user’s emotion and interest to provide good recommendations. The proposed system does not require any of aforementioned data and works without the continuous and interminable attention of the user. In this framework, we capture user’s eye-gaze and facial expression while exploring websites through inexpensive, visible light “webcam”. The eye-gaze detection method uses pupil-center extraction of both eyes and calculates the reference point through a joint probability. The facial expression uses landmark points of face and analyzes the emotion of the user. Both methods work in approximate real time and the proposed framework thus provides intelligent recommendations on-the-fly without requirement of feedback and buying patterns of users.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749
Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. Comput Vis–ECCV 2006:404–417
Beel J, Gipp B, Langer S, Breitinger C (2016) Paper recommender systems: a literature survey. Int J Digit Libr 17(4):305–338
Beilin L, Yi S (2013) Survey of personalized recommendation based on society networks analysis. In: 2013 6th International conference on information management, innovation management and industrial engineering (ICIII), Vol. 3, IEEE, pp 337–340
Breese JS, Heckerman D, Kadie C (1998) Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc., pp 43–52
Chen H-H, Gou L, Zhang X, Giles CL (2011) Collabseer: a search engine for collaboration discovery. In: Proceedings of the 11th annual international ACM/IEEE joint conference on digital libraries. ACM, pp 231–240
Chen H-H, Ororbia I, Alexander G, Giles CL (2015) Expertseer: a keyphrase based expert recommender for digital libraries. arXiv:1511.02058
Cohen I, Garg A, Huang TS et al (2000) Emotion recognition from facial expressions using multilevel hmm. In: Neural information processing systems, vol 2
Felfernig A, Isak K, Szabo K, Zachar P (2007) The vita financial services sales support environment. In: Proceedings of the national conference on artificial intelligence, vol 22, p 1692
Guo Y, Tao D, Yu J, Xiong H, Li Y, Tao D et al (2016) Deep neural networks with relativity learning for facial expression recognition. In: 2016 IEEE international conference on multimedia & expo workshops, ICMEW. IEEE, pp 1–6
Gupta P, Goel A, Lin J, Sharma A, Wang D, Zadeh R (2013) Wtf: the who to follow service at twitter. In: Proceedings of the 22nd international conference on World Wide Web. ACM, pp 505–514
Hansen DW, Ji Q (2010) In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans Pattern Anal Mach Intell 32(3):478–500
Ishwar D, Singh BK (2015) Emotion detection using facial expression. Int J Emerg Res Manage Technol 4(6):144–151
Jafarkarimi H, Sim ATH, Saadatdoost R (2012) A naive recommendation model for large databases. Intern J Inform Educ Technol 2(3):216
Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: Proceedings of the 4th IEEE international conference on automatic face and gesture recognition, 2000. IEEE, pp 46–53
Karray F, Alemzadeh M, Saleh JA, Arab MN Human-computer interaction: overview on state of the art. International Journal on Smart Sensing and Intelligent Systems
Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1867–1874
Kermany NR, Alizadeh SH (2017) A hybrid multi-criteria recommender system using ontology and neuro-fuzzy techniques. Electron Commer Res Appl 21:50–64
Liao C-L, Lee S-J (2016) A clustering based approach to improving the efficiency of collaborative filtering recommendation. Electron Commer Res Appl 18:1–9
Liu W, Zhang L, Tao D, Cheng J (2018) Reinforcement online learning for emotion prediction by using physiological signals. Pattern Recogn Lett 107:123–130
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), 2010. IEEE, pp 94–101
Ma X, Lu H, Gan Z, Zeng J (2017) An explicit trust and distrust clustering based collaborative filtering recommendation approach. Electron Commer Res Appl 25:29–39
Mase K (1991) Recognition of facial expression from optical flow. IEICE Trans 74(E):3474–3483
Melville P, Sindhwani V (2011) Recommender systems. In: Encyclopedia of machine learning, Springer, pp 829–838
Mooney RJ, Roy L (2000) Content-based book recommending using learning for text categorization. In: Proceedings of the fifth ACM conference on digital libraries. ACM, pp 195–204
Pourgholamali F, Kahani M, Bagheri E, Noorian Z (2017) Embedding unstructured side information in product recommendation. Electron Commer Res Appl 25:70–85
Timm F, Barth E (2011) Accurate eye centre localisation by means of gradients. Visapp 11:125–130
Valenti R, Gevers T (2008) Accurate eye center location and tracking using isophote curvature. In: 2008 IEEE conference on computer vision and pattern recognition, CVPR 2008. IEEE, pp 1–8
Venkateswarlu R (2003) Eye gaze estimation from a single image of one eye. In: Proceedings of the 9th IEEE international conference on computer vision, 2003. IEEE, pp 136–143
Viola P, Jones M (2001) Robust real-time object detection. Intern J Comput Vis 57(2):137–154
Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 1. IEEE, pp I–I
Yacoob Y, Davis LS (1996) Recognizing human facial expressions from long image sequences using optical flow. IEEE Trans Pattern Anal Mach Intell 18(6):636–642
Yadava M, Kumar P, Saini R, Roy PP, Dogra DP (2017) Analysis of eeg signals and its application to neuromarketing. Multimed Tool Appl 76(18):19087–19111
Zhao WX, Li S, He Y, Chang EY, Wen J-R, Li X (2016) Connecting social media to e-commerce: Cold-start product recommendation using microblogging information. IEEE Trans Knowl Data Eng 28(5):1147–1159
Zhao Z-L, Wang C-D, Wan Y-Y, Lai J-H Recommendation in feature space sphere. Electronic Commerce Research and Applications
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Jaiswal, S., Virmani, S., Sethi, V. et al. An intelligent recommendation system using gaze and emotion detection. Multimed Tools Appl 78, 14231–14250 (2019). https://doi.org/10.1007/s11042-018-6755-1
- E-commerce recommendation system
- Eye gaze
- Emotion detection