Skip to main content

Input Enhanced Logarithmic Factorization Network for CTR Prediction

  • 289 Accesses

Part of the Lecture Notes in Computer Science book series (LNAI,volume 13282)

Abstract

Factorization-based methods, which can automatically model second-order or higher-order cross features, have been the benchmark models for click-through rate (CTR) prediction. In general, they enumerate all cross features with a predetermined order and filter out useless interactions through model training. However, two significant challenges remain. First, a maximum order for feature interactions needs to be defined in advance, imposing a trade-off between computational cost and the expression ability of models; Second, enumerating all feature interactions may introduce unwanted noise. In this work, we propose a novel model called Input Enhanced Logarithmic Factorization Network (ILFN), which can effectively learn arbitrary-order feature interactions and identify useful cross features. More importantly, ILFN can take full advantage of the original feature distribution in a discriminative way.

The core of ILFN is the Input Enhanced Component (IEC), which can represent the impact of each feature in a feature interaction as a trainable coefficient to learn arbitrary-order cross features. Moreover, IEC can reduce the demand for logarithmic neurons by exploiting the essential raw information and does not need to incorporate the deep neural network (DNN) to model high-order interactions. Therefore, ILFN is more effective and efficient and can converge to satisfactory results faster. Extensive experiments on four real-world datasets demonstrate that our ILFN model can outperform start-of-the-art methods. The effectiveness of each proposed component is also verified by hyper-parameter and ablation studies.

Keywords

  • Factorization machines
  • Logarithmic neural networks
  • Recommender systems
  • Feature interactions

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-05981-0_4
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-031-05981-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

Notes

  1. 1.

    The models in bold are baselines in the experimental part.

  2. 2.

    http://labs.criteo.com/2014/02/.

  3. 3.

    https://www.kaggle.com/c/avazu-ctr-prediction.

  4. 4.

    https://grouplens.org/datasets/movielens/.

  5. 5.

    http://baltrunas.info/research-menu/frappe.

  6. 6.

    https://www.tensorflow.org/.

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)

    Google Scholar 

  2. Wang, R., Shivanna, R., et al.: DCN V2: improved deep & cross network and practical lessons for web-scale learning to rank systems. In: WWW, pp. 1785–1797 (2021)

    Google Scholar 

  3. Cheng, H.-T., Koc, L., et al.: Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, pp. 7–10 (2016)

    Google Scholar 

  4. Cheng, W., Shen, Y., Huang, L.: Adaptive factorization network: learning adaptive-order feature interactions. In: AAAI, pp. 3609–3616 (2020)

    Google Scholar 

  5. McMahan, H.B., Holt, G., et al.: Ad click prediction: a view from the trenches. In: The 19th SIGKDD, 11–14 August 2013 (2013)

    Google Scholar 

  6. Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: DeepFM: a factorization-machine based neural network for CTR prediction. In: IJCAI, pp. 1725–1731 (2017)

    Google Scholar 

  7. Graepel, T., Candela, J.Q., Borchert, T., Herbrich, R.: Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft’s Bing search engine. Omnipress (2010)

    Google Scholar 

  8. Wang, Z., Zhang, R., Qi, J., Yuan, B.: DBSVEC: density-based clustering using support vector expansion. In: ICDE, pp. 280–291. IEEE (2019)

    Google Scholar 

  9. Lee, K., Orten, B., Dasdan, A., Li, W.: Estimating conversion rate in display advertising from past erformance data. In: KDD, pp. 768–776 (2012)

    Google Scholar 

  10. Wang, Z., Liu, L., Tao, D.: Deep streaming label learning. In: International Conference on Machine Learning (ICML) (2020)

    Google Scholar 

  11. Qu, Y., et al.: Product-based neural networks for user response prediction. In: ICDM, pp. 1149–1154. IEEE (2016)

    Google Scholar 

  12. Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., Sun, G.: xDeepFM: combining explicit and implicit feature interactions for recommender systems. In: KDD, pp. 1754–1763 (2018)

    Google Scholar 

  13. He, X., Chua, T.-S.: Neural factorization machines for sparse predictive analytics. In: SIGIR, pp. 355–364 (2017)

    Google Scholar 

  14. Xiao, J., Ye, H., He, X., Zhang, H., et al.: Attentional factorization machines: learning the weight of feature interactions via attention networks. In: IJCAI (2017)

    Google Scholar 

  15. Hines, J.W.: A logarithmic neural network architecture for unbounded non-linear function approximation. In: IEEE ICNN 1996 (1996)

    Google Scholar 

  16. Juan, Y., Zhuang, Y., Chin, W.-S., Lin, C.-J.: Field-aware factorization machines for CTR prediction. In: RecSys (2016)

    Google Scholar 

  17. Song, W., Shi, C., et al.: AutoInt: automatic feature interaction learning via self-attentive neural networks. In: CIKM (2019)

    Google Scholar 

  18. Wang, R., Fu, B., Fu, G., Wang, M.: Deep & cross network for ad click predictions. In: KDD, pp. 1–7 (2017)

    Google Scholar 

  19. Huang, T., Zhang, Z., Zhang, J.: FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction. In: ACM RecSys (2019)

    Google Scholar 

  20. Rendle, S.: Factorization machines. In: 2010 IEEE International Conference on Data Mining, pp. 995–1000. IEEE (2010)

    Google Scholar 

  21. Blondel, M., Fujino, A., Ueda, N., Ishihata, M.: Higher-order factorization machines. In: NIPS (2016)

    Google Scholar 

  22. Yu, Y., Wang, Z., Yuan, B.: An input-aware factorization machine for sparse prediction. In: IJCAI (2019)

    Google Scholar 

  23. Zhang, W., Du, T., Wang, J.: Deep learning over multi-field categorical data. In: Ferro, N., et al. (eds.) ECIR 2016. LNCS, vol. 9626, pp. 45–57. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1_4

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xueqian Wang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Li, X., Wang, Z., Wu, X., Yuan, B., Wang, X. (2022). Input Enhanced Logarithmic Factorization Network for CTR Prediction. In: Gama, J., Li, T., Yu, Y., Chen, E., Zheng, Y., Teng, F. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2022. Lecture Notes in Computer Science(), vol 13282. Springer, Cham. https://doi.org/10.1007/978-3-031-05981-0_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05981-0_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05980-3

  • Online ISBN: 978-3-031-05981-0

  • eBook Packages: Computer ScienceComputer Science (R0)