Abstract
The e-commerce fashion industry is booming and comes with the need for proper search and recommendation. However, sufficient user personalization is still a challenging task. In this paper, we introduce a personalized fashion recommendation system based on high-dimensional input of user- and environment information. The proposed framework is used to estimate suitable categories and style of clothing depending on customized settings such as body type, age, occasion, or season. The goal is to recommend a full fitting outfit from the estimated suggestions. However, various personal attributes add up to a high dimensionality, and datasets are often very unbalanced or biased, making it difficult to do a proper recommendation. To solve this, we propose a pairwise-attention module to improve the performance of our framework. Our model can improve the performance up to 53.29% over the comparison method on MSE, mAP, and Recall. Moreover, in a subjective evaluation with human participants, the recommendations of the proposed method are preferred over the comparison method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amed, I., Balchandani, A., Berg, A., Hedrich, S., Jensen, J.E., Rölkens, F.: The State of Fashion 2021. McKinsey & Company (2021)
Amed, I., et al.: The State of Fashion 2020. Business of Fashion and McKinsey & Company (2020)
Amed, I., et al.: The State of Fashion 2019; Business of Fashion Mckinsey & Company (2018)
Dai, Y., Gieseke, F., Oehmcke, S., Wu, Y., Barnard, K.: Attentional feature fusion. In: IEEE Winter Conference on Applications Computer Vision, WACV, pp. 3559–3568 (2021). https://doi.org/10.1109/WACV48630.2021.00360
Gu, X., Wong, Y., Peng, P., Shou, L., Chen, G., Kankanhalli, M.S.: Understanding fashion trends from street photos via neighbor-constrained embedding learning. In: Proceedings of the 2017 ACM Multimedia Conference, pp. 190–198 (2017). https://doi.org/10.1145/3123266.3123441
Han, X., Wu, Z., Jiang, Y., Davis, L.S.: Learning fashion compatibility with bidirectional LSTMs. In: Proceedings of the 2017 ACM Multimedia Conference, pp. 1078–1086 (2017). https://doi.org/10.1145/3123266.3123394
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
He, X., Du, X., Wang, X., Tian, F., Tang, J., Chua, T.: Outer product-based neural collaborative filtering. In: Lang, J. (ed.) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI, pp. 2227–2233. ijcai.org (2018)
Hidayati, S.C., Hsu, C., Chang, Y., Hua, K., Fu, J., Cheng, W.: What dress fits me best?: fashion recommendation on the clothing style for personal body shape. In: 2018 ACM Multimedia Conference, pp. 438–446 (2018). https://doi.org/10.1145/3240508.3240546
Hsiao, W., Grauman, K.: Vibe: dressing for diverse body shapes. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020, pp. 11056–11066 (2020). https://doi.org/10.1109/CVPR42600.2020.01107
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: 26th Annual Conference on Neural Information Processing Systems (NIPS), pp. 1106–1114 (2012)
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference BMVC, pp. 41.1–41.12 (2015). https://doi.org/10.5244/C.29.41
Pavlakos, G., et al.: Expressive body capture: 3D hands, face, and body from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 10975–10985 (2019). Computer Vision Foundation/IEEE. https://doi.org/10.1109/CVPR.2019.01123
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp. 1532–1543 (2014). https://doi.org/10.3115/v1/d14-1162
Rawat, Y.S., Kankanhalli, M.S.: ConTagNet: exploiting user context for image tag recommendation. In: Proceedings of the 2016 ACM Conference on Multimedia, pp. 1102–1106. ACM (2016)
Rendle, S.: Factorization machines with libFM. ACM Trans. Intell. Syst. Technol. 3(3), 57:1–57:22 (2012)
Rong, Y., Shiratori, T., Joo, H.: FrankMocap: fast monocular 3D hand and body motion capture by regression and integration. CoRR arXiv:2008.08324 (2020)
Serengil, S.I., Ozpinar, A.: LightFace: a hybrid deep face recognition framework. In: 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), pp. 1–5 (2020)
Thomee, B., et al.: Yfcc100m: the new data in multimedia research. Commun. ACM 59(2), 64–73 (2016)
Verma, D., Gulati, K., Goel, V., Shah, R.R.: Fashionist: personalising outfit recommendation for cold-start scenarios. In: 28th ACM International Conference on Multimedia, pp. 4527–4529 (2020). https://doi.org/10.1145/3394171.3414446
Zheng, L., Noroozi, V., Yu, P.S.: Joint deep modeling of users and items using reviews for recommendation. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM, pp. 425–434 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Trakulwaranont, D., Kastner, M.A., Satoh, S. (2022). Personalized Fashion Recommendation Using Pairwise Attention. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. https://doi.org/10.1007/978-3-030-98355-0_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-98355-0_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98354-3
Online ISBN: 978-3-030-98355-0
eBook Packages: Computer ScienceComputer Science (R0)