Abstract
The contextual bandit problem has been studied in the recommender system community, but without paying much attention to the contextual aspect of the recommendation. We introduce in this paper an algorithm that tackles this problem by modeling the Mobile Context-Aware Recommender Systems (MCRS) as a contextual bandit algorithm and it is based on dynamic exploration/exploitation. Within a deliberately designed offline simulation framework, we conduct extensive evaluations with real online event log data. The experimental results and detailed analysis demonstrate that our algorithm outperforms surveyed algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alexandre, S., Moira Norrie, C., Grossniklaus, M., Signer, B.: Spatio-Temporal Proximity as a Basis for Collaborative Filtering in Mobile Environments. In: CAiSE (2006)
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite Time Analysis of the Multiarmed Bandit Problem. Machine Learning 47, 235–256 (2002)
Bader, R., Neufeld, E., Woerndl, W., Prinz, V.: Context-aware POI recommendations in an automotive scenario using multi-criteria decision making methods. In: Proceedings of the 2011 Workshop on Context-awareness in Retrieval and Recommendation, USA, pp. 23–30 (2011)
Baltrunas, L., Ludwig, B., Peer, S., Ricci, F.: Context relevance assessment and exploitation in mobile recommender systems. Personal and Ubiquitous Computing 16(5) (2011)
Bellotti, V., Begole, B., Chi, E.H., Ducheneaut, N., Fang, J., Isaacs, E.: Activity-Based Serendipitous Recommendations. In: Proceedings on the Move, pp. 1157–1166. ACM, USA (2008)
Bouneffouf, D., Bouzeghoub, A., Gançarski, A.L.: Following the User’s Interests in Mobile Context-Aware recommender systems. In: AINA Workshops, pp. 657–662. IEEE Computer Society, USA (2012)
Adomavicius, G., Mobasher, B., Ricci, F., Tuzhilin, A.: Context-Aware Recommender Systems. AI Magazine 32(3), 67–80 (2011)
Chu, W., Park, S.: Personalized recommendation on dynamic content using predictive bilinear models. In: Proc. of World Wide Web, pp. 691–700. ACM, Spain (2009)
Even-Dar, E., Mannor, S., Mansour, Y.: PAC Bounds for Multi-armed Bandit and Markov Decision Processes. In: Kivinen, J., Sloan, R.H. (eds.) COLT 2002. LNCS (LNAI), vol. 2375, pp. 255–270. Springer, Heidelberg (2002)
Kivinen, J., Manfred, K.W.: Exponentiated gradient versus gradient descent for linear predictors. In: Information and Computation, vol. 132, pp. 132–163. Academic Press, USA (1997)
Langford, J., Zhang, T.: The epoch-greedy algorithm for contextual multi-armed bandits. In: Advances in Neural Information Processing Systems, vol. 20, pp. 1096–1103. Curran Associates, Canada (2008)
Lakshmish, R., Deepak, P., Ramana, P., Kutila, G., Garg, D., Karthik, V., Shivkumar, K.: A Mobile context-aware, Social Recommender System for Low-End Mobile Device. In: Mobile Data Management: Systems, Services and Middleware CAESAR, pp. 338–347. IEEE Computer Society, USA (2009)
Lihong, L., Wei, C., Langford, J., Schapire, E.: A Contextual-Bandit Approach to Personalized News Document Recommendation. CoRR, Presented at the Nineteenth International Conference on World Wide Web, Raleigh, pp. 661–670. ACM, USA (2010)
Mannor, S., Tsitsiklis, J.: The Sample Complexity of Exploration in the Multi-Armed Bandit Problem. Computational Learning Theory 5, 623–648 (2004)
Mladenic, D.: Text-learning and related intelligent agents: A survey. IEEE Intelligent Agents 14, 44–54 (1999)
Robbins, H.: Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society (1952)
Samaras, G., Panayiotou, C.: Personalized portals for the wireless user based on mobile agents. In: Proc. 2nd Int’l Workshop on Mobile Commerce, pp. 70–74. ACM, USA (2002)
Schafer, J.B., Konstan, J.: Riedi. J.: Recommender systems in e-commerce. In: Proc. of the 1st ACM Conf. on Electronic Commerce, pp. 158–166. ACM, USA (1999)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press (1998)
Watkins, C.J.: Learning from Delayed Rewards. Ph.D. thesis. Cambridge University (1989)
Wei, L., Wang, X., Zhang, R., Cui, Y., Mao, J., Jin, R.: Exploitation and Exploration in a Performance based Contextual Advertising System. In: KDD, pp. 133–138. ACM, USA (2010)
Woerndl, W., Schulze, F.: Capturing, Analyzing and Utilizing Context-Based Information About User Activities on Smartphones. In: Activity Context Representation Techniques and Languages: Papers from the 2011 AAAI Workshop (2011)
Wu, Z., Palmer, M.: Verb Semantics and Lexical Selection. In: Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, pp. 133–138 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bouneffouf, D., Bouzeghoub, A., Gançarski, A.L. (2012). Exploration / Exploitation Trade-Off in Mobile Context-Aware Recommender Systems. In: Thielscher, M., Zhang, D. (eds) AI 2012: Advances in Artificial Intelligence. AI 2012. Lecture Notes in Computer Science(), vol 7691. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35101-3_50
Download citation
DOI: https://doi.org/10.1007/978-3-642-35101-3_50
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35100-6
Online ISBN: 978-3-642-35101-3
eBook Packages: Computer ScienceComputer Science (R0)