Website Navigation Recommendation Based on Reinforcement Learning Technique

  • Yin-Ling TangEmail author
  • I-Hsien Ting
  • Shyue-Liang Wang
Conference paper
Part of the Springer Proceedings in Complexity book series (SPCOM)


The explosive growth of the Internet has made information on the web large and complicated. If the structure of a website is not optimized, users could easily get lost and could not find the most important information at the first time. The adaptive website can present the information that users needed by analyzing the users’ behavior. However, visitors may have different needs at different times. Most of recommended methods are not considerate of dynamic or time-dependent needs. This paper presents a recommender system based on reinforcement learning. We assume that five parameters are on recommendation, which include clicks of the page, time that spent on viewing the page, paths to find the page, hierarchy of the page, and the rank of the page. With the help of reinforcement learning to adjust the weight of five parameters, we aim to reduce the paths that user needed to find the object page.


Web usage mining Adaptive web sites Reinforcement learning 


  1. 1.
    Schwarzkopf E (2001) An Adaptive web site for the UM2001 conference. In: Proceedings of the UM2001 workshop on machine learning for user modeling, p 77–86Google Scholar
  2. 2.
    Perkowitz M, Etzioni O (1997) Adaptive web sites: an AI challenge. In: Proceedings of international joint conferences on artificial intelligence, NagoyaGoogle Scholar
  3. 3.
    Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  4. 4.
    Catledge L, Pitkow J (1995) Characterizing browsing behaviors on the World Wide Web. In: Proceedings of computer networks and ISDN systems, vol 27(6)Google Scholar
  5. 5.
    Srivastava J, Cooley R, Deshpande M, Tan P-T (2000) Web usage mining: discovery and applications of usage patterns from web data. In Proceedings of SIGKDD explorations, vol 1(2), pp 1–12Google Scholar
  6. 6.
    Perkowitz M, Etzioni O (1998) Adaptive web sites: automatically synthesizing web pages. In: Proceedings of association for the advancement of artificial intelligence, Madison, pp 727–732Google Scholar
  7. 7.
    Brusilovsky P (1996) Methods and techniques of adaptive hypermedia. User Model User-Adap Inter J 6:87–129CrossRefGoogle Scholar
  8. 8.
    Pierrakos D, Paliouras G, Papatheodorou C, Spyropoulos CD (2003) Web usage mining as a tool for personalization: a survey. In Proceedings of user modeling and user-adapted interaction, vol 13, pp 311–372Google Scholar
  9. 9.
    Teíeni D, Feldman R (2001) Performance and satisfaction in aadaptive websites: an experiment on searches within a task-adapted website. J Assoc Inf Syst, 2(3):1–30 Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.Department of Information ManagementNational University of Kaohsiung KaohsiungKaohsiungTaiwan

Personalised recommendations