Importance of Recommendation Policy Space in Addressing Click Sparsity in Personalized Advertisement Display

  • Sougata Chaudhuri
  • Georgios Theocharous
  • Mohammad Ghavamzadeh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10358)


We study a specific case of personalized advertisement recommendation (PAR) problem, which consist of a user visiting a system (website) and the system displaying one of K ads to the user. The system uses an internal ad recommendation policy to map the user’s profile (context) to one of the ads. The user either clicks or ignores the ad and correspondingly, the system updates its recommendation policy. The policy space in large scale PAR systems are generally based on classifiers. A practical problem in PAR is extreme click sparsity, due to very few users actually clicking on ads. We systematically study the drawback of using classifier-based policies, in face of extreme click sparsity. We then suggest an alternate policy, based on rankers, learnt by optimizing the Area Under the Curve (AUC) ranking loss, which can significantly alleviate the problem of click sparsity. We create deterministic and stochastic policy spaces and conduct extensive experiments on public and proprietary datasets to illustrate the improvement in click-through-rate (CTR) obtained by using the ranker-based policy over classifier-based policy.


  1. 1.
    Agarwal, A., Hsu, D., Kale, S., Langford, J., Li, L., Schapire, R.: Taming the monster: a fast and simple algorithm for contextual bandits. In: Proceedings of the 31st International Conference on Machine Learning, pp. 1638–1646 (2014)Google Scholar
  2. 2.
    Agarwal, D., Gabrilovich, E., Hall, R., Josifovski, V., Khanna, R.: Translating relevance scores to probabilities for contextual advertising. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 1899–1902. ACM (2009)Google Scholar
  3. 3.
    Beygelzimer, A., Langford, J., Ravikumar, P.: Multiclass classification with filter trees. Preprint, 2 June 2007Google Scholar
  4. 4.
    Calders, T., Jaroszewicz, S.: Efficient AUC optimization for classification. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 42–53. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74976-9_8 CrossRefGoogle Scholar
  5. 5.
    Cao, P., Zhao, D., Zaiane, O.: An optimized cost-sensitive SVM for imbalanced data learning. In: Pei, J., Tseng, V.S., Cao, L., Motoda, H., Xu, G. (eds.) PAKDD 2013. LNCS, vol. 7819, pp. 280–292. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-37456-2_24 CrossRefGoogle Scholar
  6. 6.
    Chapelle, O., Li, L.: An empirical evaluation of thompson sampling. In: Advances in Neural Information Processing Systems, pp. 2249–2257 (2011)Google Scholar
  7. 7.
    Chawla, N., Japkowicz, N., Kotcz, A.: Editorial: special issue on learning from imbalanced data sets. ACM SIGKDD Explor. Newsl. 6(1), 1–6 (2004)CrossRefGoogle Scholar
  8. 8.
    Cheng, T., Wang, Y., Bryant, S.: FSelector: a ruby gem for feature selection. Bioinformatics 28(21), 2851–2852 (2012)CrossRefGoogle Scholar
  9. 9.
    Cortes, C., Mohri, M.: AUC optimization vs. error rate minimization. Adv. Neural Inf. Proces. Syst. 16(16), 313–320 (2004)Google Scholar
  10. 10.
    Dudik, M., Hsu, D., Kale, S., Karampatziakis, N., Langford, J., Reyzin, L., Zhang, T.: Efficient optimal learning for contextual bandits. In: Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (2011)Google Scholar
  11. 11.
    He, X., et al.: Practical lessons from predicting clicks on ads at facebook. In: Proceedings of 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1–9. ACM (2014)Google Scholar
  12. 12.
    Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6(5), 429–449 (2002)MATHGoogle Scholar
  13. 13.
    Koh, E., Gupta, N.: An empirical evaluation of ensemble decision trees to improve personalization on advertisement. In: Proceedings of KDD 14 Second Workshop on User Engagement Optimization (2014)Google Scholar
  14. 14.
    Langford, J., Li, L., Dudik, M.: Doubly robust policy evaluation and learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 1097–1104 (2011)Google Scholar
  15. 15.
    Langford, J., Zhang, T.: The epoch-greedy algorithm for multi-armed bandits with side information. In: Advances in Neural Information Processing Systems, pp. 817–824 (2008)Google Scholar
  16. 16.
    Li, L., Chu, W., Langford, J., Wang, X.: Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In: Proceedings of the fourth ACM International Conference on Web Search and Data Mining, pp. 297–306. ACM (2011)Google Scholar
  17. 17.
    McMahan, H., et al.: Ad click prediction: a view from the trenches. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1222–1230. ACM (2013)Google Scholar
  18. 18.
    Richardson, M., Dominowska, E., Ragno, R.: Predicting clicks: estimating the click-through rate for new ads. In: Proceedings of the 16th International Conference on World Wide Web, pp. 521–530. ACM (2007)Google Scholar
  19. 19.
    Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)MathSciNetMATHGoogle Scholar
  20. 20.
    Shamir, O., Zhang, T.: Stochastic gradient descent for non-smooth optimization: convergence results and optimal averaging schemes. In: Proceedings of the 30th International Conference on Machine Learning, pp. 71–79 (2013)Google Scholar
  21. 21.
    Theocharous, G., Thomas, P., Ghavamzadeh, M.: Ad recommendation systems for life-time value optimization. In: Proceedings of the 24th International Conference on World Wide Web Companion, pp. 1305–1310 (2015)Google Scholar
  22. 22.
    Zhao, P., Jin, R., Yang, T., Hoi, S.C.: Online AUC maximization. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 233–240 (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Sougata Chaudhuri
    • 1
  • Georgios Theocharous
    • 2
  • Mohammad Ghavamzadeh
    • 2
  1. 1.University of MichiganAnn ArborUSA
  2. 2.Adobe ResearchSan JoseUSA

Personalised recommendations