Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
Book cover

Joint European Conference on Machine Learning and Knowledge Discovery in Databases

ECML PKDD 2012: Machine Learning and Knowledge Discovery in Databases pp 51–66Cite as

  1. Home
  2. Machine Learning and Knowledge Discovery in Databases
  3. Conference paper
A Live Comparison of Methods for Personalized Article Recommendation at Forbes.com

A Live Comparison of Methods for Personalized Article Recommendation at Forbes.com

  • Evan Kirshenbaum21,
  • George Forman21 &
  • Michael Dugan22 
  • Conference paper
  • 4908 Accesses

  • 22 Citations

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7524)

Abstract

We present the results of a multi-phase study to optimize strategies for generating personalized article recommendations at the Forbes.com web site. In the first phase we compared the performance of a variety of recommendation methods on historical data. In the second phase we deployed a live system at Forbes.com for five months on a sample of 82,000 users, each randomly assigned to one of 20 methods. We analyze the live results both in terms of click-through rate (CTR) and user session lengths. The method with the best CTR was a hybrid of collaborative-filtering and a content-based method that leverages Wikipedia-based concept features, post-processed by a novel Bayesian remapping technique that we introduce. It both statistically significantly beat decayed popularity and increased CTR by 37%.

Keywords

  • personalization
  • recommender systems
  • collaborative filtering
  • content analysis
  • live user trial

Download conference paper PDF

References

  1. Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems. IEEE Trans. on Knowl. and Data Eng. 17(6), 734–749 (2005)

    CrossRef  Google Scholar 

  2. Billsus, D., Pazzani, M.J.: Adaptive News Access. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) Adaptive Web 2007. LNCS, vol. 4321, pp. 550–570. Springer, Heidelberg (2007)

    CrossRef  Google Scholar 

  3. Chen, J., Nairn, R., Nelson, L., Bernstein, M., Chi, E.: Short and tweet: experiments on recommending content from information streams. In: CHI 2010 (2010)

    Google Scholar 

  4. Cremonesi, P., Koren, Y., Turrin, R.: Performance of recommender algorithms on top-n recommendation tasks. In: RecSys 2010, pp. 39–46 (2010)

    Google Scholar 

  5. Das, A.S., Datar, M., Garg, A., Rajaram, S.: Google news personalization: scalable online collaborative filtering. In: WWW 2007, pp. 271–280 (2007)

    Google Scholar 

  6. de Wit, J.: Evaluating recommender systems: an evaluation framework to predict user satisfaction for recommender systems in an electronic programme guide context. Master’s thesis, University of Twente, The Netherlands (May 2008)

    Google Scholar 

  7. Ge, M., Delgado-Battenfeld, C., Jannach, D.: Beyond accuracy: evaluating recommender systems by coverage and serendipity. In: RecSys 2010, p. 257 (2010)

    Google Scholar 

  8. Herlocker, J., Konstan, J., Terveen, L., Riedl, J.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22(1), 5–53 (2004)

    CrossRef  Google Scholar 

  9. Hurley, N., Zhang, M.: Novelty and diversity in top-N recommendation – analysis and evaluation. ACM Trans. Internet Technol. 14, 14:1–14:30 (2011)

    CrossRef  Google Scholar 

  10. Jahrer, M., Töscher, A., Legenstein, R.: Combining predictions for accurate recommender systems. In: KDD 2010, pp. 693–702 (2010)

    Google Scholar 

  11. Katz, G., Ofek, N., Shapira, B., Rokach, L., Shani, G.: Using Wikipedia to boost collaborative filtering techniques. In: RecSys 2011, pp. 285–288 (2011)

    Google Scholar 

  12. Kirshenbaum, E.: A Wikipedia-based concept extractor for unstructured text. Technical report, HP Labs (in preparation)

    Google Scholar 

  13. Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing user studies with mechanical turk. In: CHI 2008, pp. 453–456 (2008)

    Google Scholar 

  14. Koenigstein, N., Dror, G., Koren, Y.: Yahoo! music recommendations: modeling music ratings with temporal dynamics and item taxonomy. In: RecSys 2011 (2011)

    Google Scholar 

  15. Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: KDD 2008, pp. 426–434 (2008)

    Google Scholar 

  16. Lerman, K., Hogg, T.: Using a model of social dynamics to predict popularity of news. In: WWW 2010, pp. 621–630 (2010)

    Google Scholar 

  17. Liu, J., Dolan, P., Pedersen, E.R.: Personalized news recommendation based on click behavior. In: IUI 2010, pp. 31–40 (2010)

    Google Scholar 

  18. Maidel, V., Shoval, P., Shapira, B., Taieb-Maimon, M.: Evaluation of an ontology-content based filtering method for a personalized newspaper. In: RecSys 2008 (2008)

    Google Scholar 

  19. Pan, R., Zhou, Y., Cao, B., Liu, N.N., Lukose, R., Scholz, M., Yang, Q.: One-class collaborative filtering. In: ICDM 2008, pp. 502–511 (2008)

    Google Scholar 

  20. Pu, P., Chen, L., Hu, R.: A user-centric evaluation framework for recommender systems. In: RecSys 2011, pp. 157–164 (2011)

    Google Scholar 

  21. Sandholm, T., Ung, H., Aperjis, C., Huberman, B.A.: Global budgets for local recommendations. In: RecSys 2010, pp. 13–20 (2010)

    Google Scholar 

  22. Steck, H.: Item popularity and recommendation accuracy. In: RecSys 2011 (2011)

    Google Scholar 

  23. Su, X., Khoshgoftaar, T.M.: A survey of collaborative filtering techniques. In: Advances in Artificial Intelligence, pp. 4:2–4:2 (January 2009)

    Google Scholar 

  24. Zheng, H., Wang, D., Zhang, Q., Li, H., Yang, T.: Do clicks measure recommendation relevancy? an empirical user study. In: RecSys 2010, pp. 249–252 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. HP Labs, Palo Alto, CA, USA

    Evan Kirshenbaum & George Forman

  2. Forbes Media, New York, NY, USA

    Michael Dugan

Authors
  1. Evan Kirshenbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George Forman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Dugan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Intelligent Systems Laboratory, University of Bristol, Merchant Venturers Building, Woodland Road, BS8 1UB, Bristol, UK

    Peter A. Flach

  2. Intelligent Systems Laboratory, University of Bristol, Merchant Venturers Building, Woodland Road,, BS8 1UB, Bristol, UK

    Tijl De Bie & Nello Cristianini & 

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kirshenbaum, E., Forman, G., Dugan, M. (2012). A Live Comparison of Methods for Personalized Article Recommendation at Forbes.com. In: Flach, P.A., De Bie, T., Cristianini, N. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2012. Lecture Notes in Computer Science(), vol 7524. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33486-3_4

Download citation

  • .RIS
  • .ENW
  • .BIB
  • DOI: https://doi.org/10.1007/978-3-642-33486-3_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33485-6

  • Online ISBN: 978-3-642-33486-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature