Advertisement

Design of normalized fractional SGD computing paradigm for recommender systems

  • Zeshan Aslam Khan
  • Syed Zubair
  • Naveed Ishtiaq Chaudhary
  • Muhammad Asif Zahoor Raja
  • Farrukh A. Khan
  • Nebojsa DedovicEmail author
Original Article
  • 72 Downloads

Abstract

Fast and effective recommender systems are fundamental to fulfill the growing requirements of the e-commerce industry. The strength of matrix factorization procedure based on stochastic gradient descent (SGD) algorithm is exploited widely to solve the recommender system problem. Modern computing paradigms are designed by utilizing the concept of fractional gradient in standard SGD and outperform the standard counterpart. The performance of fractional SGD improves considerably by adaptively tuning the learning rate parameter. A nonlinear computing paradigm based on normalized version of fractional SGD is developed in this paper to investigate the adaptive behavior of learning rate with novel application to recommender systems. The accuracy of the proposed approach is verified through root mean square error metric by using different latent features, learning rates, fractional orders and datasets. The superiority of the designed method is validated through comparison with the state-of-the-art counterparts.

Keywords

Recommender systems Normalized adaptive algorithms E-commerce Fractional calculus Stochastic gradient descent 

Notes

Acknowledgements

For the last author, this paper was supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia (project: Improvement of the quality of tractors and mobile systems with the aim of increasing competitiveness and preserving soil and environment, no. TR-31046).

Compliance with ethical standards

Conflict of interest

All authors declared that there are no potential conflicts of interest.

Human and animal rights statements

All authors declared that there is no research involving human and/or animal.

Informed consent

All authors declared that there is no material that required informed consent.

References

  1. 1.
    Bobadilla J, Ortega F, Hernando A, Gutiérrez A (2013) Recommender systems survey. Knowl Based Syst 46:109–132CrossRefGoogle Scholar
  2. 2.
    Logesh R, Subramaniyaswamy V, Malathi D, Sivaramakrishnan N, Vijayakumar V (2019) Enhancing recommendation stability of collaborative filtering recommender system through bio-inspired clustering ensemble method. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3891-5 CrossRefGoogle Scholar
  3. 3.
    Katarya R, Verma OP (2018) Recommender system with grey wolf optimizer and FCM. Neural Comput Appl 30(5):1679–1687CrossRefGoogle Scholar
  4. 4.
    Yin H, Wang W, Chen L, Du X, Nguyen QVH, Huang Z (2018) Mobi-SAGE-RS: a sparse additive generative model-based mobile application recommender system. Knowl Based Syst 157:68–80CrossRefGoogle Scholar
  5. 5.
    Zhou H, Hirasawa K (2019) Evolving temporal association rules in recommender system. Neural Comput Appl 31(7):2605–2619CrossRefGoogle Scholar
  6. 6.
    Katarya R (2018) Movie recommender system with metaheuristic artificial bee. Neural Comput Appl 30(6):1983–1990CrossRefGoogle Scholar
  7. 7.
    Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 8:30–37CrossRefGoogle Scholar
  8. 8.
    Luo X, Zhou M (2019) Effects of extended stochastic gradient descent algorithms on improving latent factor-based recommender systems. IEEE Robot Autom Lett 4(2):618–624CrossRefGoogle Scholar
  9. 9.
    Khan ZA, Chaudhary NI, Zubair S (2019) Fractional stochastic gradient descent for recommender systems. Electron Mark 29:275–285CrossRefGoogle Scholar
  10. 10.
    Chaudhary NI, Ahmed M, Khan ZA, Zubair S, Raja MAZ, Dedovic N (2018) Design of normalized fractional adaptive algorithms for parameter estimation of control autoregressive autoregressive systems. Appl Math Model 55:698–715MathSciNetCrossRefGoogle Scholar
  11. 11.
    Chaudhary NI, Zubair S, Raja MAZ, Dedovic N (2019) Normalized fractional adaptive methods for nonlinear control autoregressive systems. Appl Math Model 66:457–471MathSciNetCrossRefGoogle Scholar
  12. 12.
    Zhang Y, Abbas H, Sun Y (2019) Smart e-commerce integration with recommender systems. Electron Mark 29(2):219–220CrossRefGoogle Scholar
  13. 13.
    Kunaver M, Požrl T (2017) Diversity in recommender systems—a survey. Knowl Based Syst 123:154–162CrossRefGoogle Scholar
  14. 14.
    Aggarwal CC (2016) An introduction to recommender systems. In: Recommender systems. Springer, pp 1–28Google Scholar
  15. 15.
    Narayan S, Sathiyamoorthy E (2019) A novel recommender system based on FFT with machine learning for predicting and identifying heart diseases. Neural Comput Appl 31(1):93–102CrossRefGoogle Scholar
  16. 16.
    Xie X, Wang B (2018) Web page recommendation via twofold clustering: considering user behavior and topic relation. Neural Comput Appl 29(1):235–243MathSciNetCrossRefGoogle Scholar
  17. 17.
    He C, Parra D, Verbert K (2016) Interactive recommender systems: a survey of the state of the art and future research challenges and opportunities. Expert Syst Appl 56:9–27CrossRefGoogle Scholar
  18. 18.
    Hong B, Yu M (2018) A collaborative filtering algorithm based on correlation coefficient. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3857-7 CrossRefGoogle Scholar
  19. 19.
    Wu X, Yuan X, Duan C, Wu J (2019) A novel collaborative filtering algorithm of machine learning by integrating restricted Boltzmann machine and trust information. Neural Comput Appl 31(9):4685–4692CrossRefGoogle Scholar
  20. 20.
    Schafer JB, Konstan JA, Riedl J (2001) E-commerce recommendation applications. In: Applications of data mining to electronic commerce. Springer, pp 115–153Google Scholar
  21. 21.
    Wang J, De Vries AP, Reinders MJ (2006) Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 501–508Google Scholar
  22. 22.
    Salakhutdinov R, Mnih A (2007) Probabilistic matrix factorization. In: Nips, vol 1, no. 1, pp 2-1Google Scholar
  23. 23.
    Wang S, Tang J, Wang Y, Liu H (2015) Exploring implicit hierarchical structures for recommender systems. In: IJCAI, pp 1813–1819Google Scholar
  24. 24.
    Gao L, Li C (2008) Hybrid personalized recommended model based on genetic algorithm. In: 4th International conference on wireless communications, networking and mobile computing, 2008. WiCOM’08. IEEE, pp 1–4Google Scholar
  25. 25.
    Wang S, Gong M, Li H, Yang J, Wu Y (2017) Memetic algorithm based location and topic aware recommender system. Knowl Based Syst 131:125–134CrossRefGoogle Scholar
  26. 26.
    Roh TH, Oh KJ, Han I (2003) The collaborative filtering recommendation based on SOM cluster-indexing CBR. Expert Syst Appl 25(3):413–423CrossRefGoogle Scholar
  27. 27.
    Park MH, Hong JH, Cho SB (2007) Location-based recommendation system using bayesian user’s preference model in mobile devices. In: International conference on ubiquitous intelligence and computing. Springer, Berlin, pp 1130–1139Google Scholar
  28. 28.
    Zhong J, Li X (2010) Unified collaborative filtering model based on combination of latent features. Expert Syst Appl 37(8):5666–5672CrossRefGoogle Scholar
  29. 29.
    Luo X, Xia Y, Zhu Q (2012) Incremental collaborative filtering recommender based on regularized matrix factorization. Knowl Based Syst 27:271–280CrossRefGoogle Scholar
  30. 30.
    Luo X, Xia Y, Zhu Q (2013) Applying the learning rate adaptation to the matrix factorization based collaborative filtering. Knowl Based Syst 37:154–164CrossRefGoogle Scholar
  31. 31.
    Funk S (2006) Netflix update: try this at home. http://sifter.org/simon/journal/20061211.html. Accessed 10 Dec 2017
  32. 32.
    Paterek A (2007) Improving regularized singular value decomposition for collaborative filtering. In: Proceedings of KDD cup and workshop, vol 2007, pp 5–8Google Scholar
  33. 33.
    Sarwar B, Karypis G, Konstan J, Riedl J (2000) Application of dimensionality reduction in recommender system-a case study (No. TR-00-043). Minnesota Univ Minneapolis Dept of Computer ScienceGoogle Scholar
  34. 34.
    Srebro N, Rennie JD, Jaakkola TS (2004) Maximum-margin matrix factorization. In: NIPS, vol 17, pp 1329–1336Google Scholar
  35. 35.
    Hofmann T (2004) Latent semantic models for collaborative filtering. ACM Trans Inf Syst (TOIS) 22(1):89–115CrossRefGoogle Scholar
  36. 36.
    Bell RM, Koren Y (2007) Scalable collaborative filtering with jointly derived neighborhood interpolation weights. In: Seventh IEEE international conference on data mining, 2007. ICDM 2007. IEEE, pp 43–52Google Scholar
  37. 37.
    Zhou Y, Wilkinson D, Schreiber R, Pan R (2008) Large-scale parallel collaborative filtering for the netflix prize. In: International conference on algorithmic applications in management. Springer, Berlin, pp 337–348Google Scholar
  38. 38.
    Takács G, Pilászy I, Németh B, Tikk D (2009) Scalable collaborative filtering approaches for large recommender systems. J Mach Learn Res 10(Mar):623–656Google Scholar
  39. 39.
    Yu HF, Hsieh CJ, Si S, Dhillon I (2012) Scalable coordinate descent approaches to parallel matrix factorization for recommender systems. In 2012 IEEE 12th international conference on data mining (ICDM). IEEE, pp 765–774Google Scholar
  40. 40.
    Gemulla R, Nijkamp E, Haas PJ, Sismanis Y (2011) Large-scale matrix factorization with distributed stochastic gradient descent. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 69–77Google Scholar
  41. 41.
    Chin WS, Zhuang Y, Juan YC, Lin CJ (2015) A fast parallel stochastic gradient method for matrix factorization in shared memory systems. ACM Trans Intell Syst Technol (TIST) 6(1):2Google Scholar
  42. 42.
    Chin WS, Zhuang Y, Juan YC, Lin CJ (2015) A learning-rate schedule for stochastic gradient methods to matrix factorization. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, Cham, pp 442–455CrossRefGoogle Scholar
  43. 43.
    Sun R, Luo ZQ (2016) Guaranteed matrix completion via non-convex factorization. IEEE Trans Inf Theory 62(11):6535–6579MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Jafari H, Jassim HK, Moshokoa SP, Ariyan VM, Tchier F (2016) Reduced differential transform method for partial differential equations within local fractional derivative operators. Adv Mech Eng 8(4):1687814016633013CrossRefGoogle Scholar
  45. 45.
    Jafari H, Jassim HK (2016) A new approach for solving a system of local fractional partial differential equations. Appl Appl Math 11:162–173MathSciNetzbMATHGoogle Scholar
  46. 46.
    Hosseini VR, Chen W, Avazzadeh Z (2014) Numerical solution of fractional telegraph equation by using radial basis functions. Eng Anal Bound Elem 38:31–39MathSciNetzbMATHCrossRefGoogle Scholar
  47. 47.
    Hosseini VR, Shivanian E, Chen W (2016) Local radial point interpolation (MLRPI) method for solving time fractional diffusion-wave equation with damping. J Comput Phys 312:307–332MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Chaudhary NI, Manzar MA, Raja MAZ (2019) Fractional Volterra LMS algorithm with application to Hammerstein control autoregressive model identification. Neural Comput Appl 31(9):5227–5240CrossRefGoogle Scholar
  49. 49.
    Chaudhary NI, Raja MAZ, Aslam MS, Ahmed N (2018) Novel generalization of Volterra LMS algorithm to fractional order with application to system identification. Neural Comput Appl 29(6):41–58CrossRefGoogle Scholar
  50. 50.
    Cheng S, Wei Y, Sheng D, Chen Y, Wang Y (2018) Identification for Hammerstein nonlinear ARMAX systems based on multi-innovation fractional order stochastic gradient. Sig Process 142:1–10CrossRefGoogle Scholar
  51. 51.
    Chaudhary NI, Aslam MS, Baleanu D, Raja MAZ (2019) Design of sign fractional optimization paradigms for parameter estimation of nonlinear Hammerstein systems. Neural Comput Appl.  https://doi.org/10.1007/s00521-019-04328-0 CrossRefGoogle Scholar
  52. 52.
    Raja MAZ, Akhtar R, Chaudhary NI, Zhiyu Z, Khan Q, Rehman AU, Zaman F (2019) A new computing paradigm for the optimization of parameters in adaptive beamforming using fractional processing. Eur Phys J Plus 134(6):275CrossRefGoogle Scholar
  53. 53.
    Chaudhary NI, Zubair S, Aslam MS, Raja MAZ, Machado JT (2019) Design of momentum fractional LMS for Hammerstein nonlinear system identification with application to electrically stimulated muscle model. Eur Phys J Plus 134(8):407CrossRefGoogle Scholar
  54. 54.
    Shah SM, Samar R, Khan NM, Raja MAZ (2017) Design of fractional-order variants of complex LMS and NLMS algorithms for adaptive channel equalization. Nonlinear Dyn 88(2):839–858zbMATHCrossRefGoogle Scholar
  55. 55.
    Chaudhary NI, Zubair S, Raja MAZ (2017) A new computing approach for power signal modeling using fractional adaptive algorithms. ISA Trans 68:189–202CrossRefGoogle Scholar
  56. 56.
    Zubair S, Chaudhary NI, Khan ZA, Wang W (2018) Momentum fractional LMS for power signal parameter estimation. Sig Process 142:441–449CrossRefGoogle Scholar
  57. 57.
    Podlubny I (1998) Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications, vol 198. Elsevier, AmsterdamzbMATHGoogle Scholar
  58. 58.
    Hajipour M, Jajarmi A, Baleanu D (2018) An efficient nonstandard finite difference scheme for a class of fractional chaotic systems. J Comput Nonlinear Dyn 13(2):021013CrossRefGoogle Scholar
  59. 59.
    Baleanu D, Machado JAT, Luo AC (eds) (2011) Fractional dynamics and control. Springer, BerlinzbMATHGoogle Scholar
  60. 60.
    Baleanu D (2012) Fractional calculus: models and numerical methods, vol 3. World Scientific, SingaporezbMATHCrossRefGoogle Scholar
  61. 61.
    Valério D, Trujillo JJ, Rivero M, Machado JT, Baleanu D (2013) Fractional calculus: a survey of useful formulas. Eur Phys J Spec Top 222(8):1827–1846CrossRefGoogle Scholar
  62. 62.
    Zhang F, Yang C, Zhou X, Gui W (2018) Fractional-order PID controller tuning using continuous state transition algorithm. Neural Comput Appl 29(10):795–804CrossRefGoogle Scholar
  63. 63.
    Baleanu D, Yusuf A, Aliyu AI (2018) Time fractional third-order evolution equation: symmetry analysis, explicit solutions, and conservation laws. J Comput Nonlinear Dyn 13(2):021011CrossRefGoogle Scholar
  64. 64.
    Zheng Y, Huang M, Lu Y, Li W (2018) Fractional stochastic resonance multi-parameter adaptive optimization algorithm based on genetic algorithm. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3910-6 CrossRefGoogle Scholar
  65. 65.
    Maxwell HF, Konstan JA (2015) The movielens datasets: history and context. ACM Trans Interact Intell Syst (TIIS) 5:19Google Scholar
  66. 66.
    Li K, Zhou X, Lin F, Zeng W, Alterovitz G (2019) Deep probabilistic matrix factorization framework for online collaborative filtering. IEEE Access 7:56117–56128CrossRefGoogle Scholar
  67. 67.
    Nguyen DM, Tsiligianni E, Deligiannis N (2018) Learning discrete matrix factorization models. IEEE Signal Process Lett 25(5):720–724CrossRefGoogle Scholar
  68. 68.
    Hoi SC, Sahoo D, Lu J, Zhao P (2018) Online learning: a comprehensive survey. arXiv preprint arXiv:1802.02871
  69. 69.
    Kim D, Park C, Oh J, Lee S, Yu H (2016) Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 233–240Google Scholar
  70. 70.
    Lin F, Zhou X, Zeng W (2016) Sparse online learning for collaborative filtering. Int J Comput Commun Control 11(2):248–258CrossRefGoogle Scholar
  71. 71.
    Huang J, Nie F, Huang H (2013) Robust discrete matrix completion. In: Twenty-seventh AAAI conference on artificial intelligenceGoogle Scholar
  72. 72.
    Huo Z, Liu J, Huang H (2016) Optimal discrete matrix completion. In: Thirtieth AAAI conference on artificial intelligenceGoogle Scholar
  73. 73.
    Xue HJ, Dai X, Zhang J, Huang S, Chen J (2017) Deep matrix factorization models for recommender systems. In: IJCAI, pp 3203–3209Google Scholar
  74. 74.
    Nguyen DM, Tsiligianni E, Deligiannis N (2018) Extendable neural matrix completion. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 6328–6332Google Scholar
  75. 75.
    Dassios IK, Baleanu DI (2018) Caputo and related fractional derivatives in singular systems. Appl Math Comput 337:591–606MathSciNetGoogle Scholar
  76. 76.
    Dassios I, Baleanu D (2018) Optimal solutions for singular linear systems of Caputo fractional differential equations. Math Methods Appl Sci.  https://doi.org/10.1002/mma.5410 CrossRefGoogle Scholar
  77. 77.
    Dassios IK (2018) A practical formula of solutions for a family of linear non-autonomous fractional nabla difference equations. J Comput Appl Math 339:317–328MathSciNetzbMATHCrossRefGoogle Scholar
  78. 78.
    Atangana A (2016) On the new fractional derivative and application to nonlinear Fisher’s reaction–diffusion equation. Appl Math Comput 273:948–956MathSciNetzbMATHGoogle Scholar
  79. 79.
    Atangana A, Baleanu D (2017) Caputo-Fabrizio derivative applied to groundwater flow within confined aquifer. J Eng Mech 143(5):D4016005CrossRefGoogle Scholar
  80. 80.
    Atangana A, Gómez-Aguilar JF (2018) Fractional derivatives with no-index law property: application to chaos and statistics. Chaos Solitons Fractals 114:516–535MathSciNetzbMATHCrossRefGoogle Scholar
  81. 81.
    Yang XJ, Gao F, Machado JA, Baleanu D (2017) A new fractional derivative involving the normalized sinc function without singular kernel. arXiv preprint arXiv:1701.05590
  82. 82.
    Firoozjaee MA, Jafari H, Lia A, Baleanu D (2018) Numerical approach of Fokker–Planck equation with Caputo–Fabrizio fractional derivative using Ritz approximation. J Comput Appl Math 339:367–373MathSciNetzbMATHCrossRefGoogle Scholar
  83. 83.
    Mehmood A, Chaudhary NI, Zameer A, Raja MAZ (2019) Novel computing paradigms for parameter estimation in Hammerstein controlled auto regressive auto regressive moving average systems. Appl Soft Comput 80:263–284CrossRefGoogle Scholar
  84. 84.
    Mehmood A, Chaudhary NI, Zameer A, Raja MAZ (2019) Novel computing paradigms for parameter estimation in power signal models. Neural Comput Appl.  https://doi.org/10.1007/s00521-019-04133-9 CrossRefGoogle Scholar
  85. 85.
    Khan WU, Ye Z, Chaudhary NI, Raja MAZ (2018) Backtracking search integrated with sequential quadratic programming for nonlinear active noise control systems. Appl Soft Comput 73:666–683CrossRefGoogle Scholar
  86. 86.
    Mehmood A, Zameer A, Raja MAZ, Bibi R, Chaudhary NI, Aslam MS (2018) Nature-inspired heuristic paradigms for parameter estimation of control autoregressive moving average systems. Neural Comput Appli.  https://doi.org/10.1007/s00521-018-3406-4 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Zeshan Aslam Khan
    • 1
  • Syed Zubair
    • 1
  • Naveed Ishtiaq Chaudhary
    • 1
  • Muhammad Asif Zahoor Raja
    • 2
  • Farrukh A. Khan
    • 3
  • Nebojsa Dedovic
    • 4
    Email author
  1. 1.Department of Electrical EngineeringInternational Islamic UniversityIslamabadPakistan
  2. 2.Department of Electrical and Computer EngineeringCOMSATS University IslamabadAttockPakistan
  3. 3.Department of Computer ScienceNational University of Computer and Emerging SciencesIslamabadPakistan
  4. 4.Department of Agricultural Engineering, Faculty of AgricultureUniversity of Novi SadNovi SadSerbia

Personalised recommendations