Abstract
The task of combining complimentary pieces of clothing into an outfit is familiar to most people, but has thus far proved difficult to automate. We present a model that uses multimodal embeddings of pieces of clothing based on images and textual descriptions. The embeddings and a shared style space are trained end to end in a novel deep neural network architecture. The network is trained on the largest and richest labelled outfit dataset made available to date, which we open source. This is the first public expert created, labelled dataset and contains 586,320 labelled outfits. We evaluate the performance of our model using an AB test and compare it to a template based model that selects items from the correct classes, but ignores style. Our experiments show that our model outperforms by 21% and 34% for womenswear and menswear respectively.
Supported by ASOS.com.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Dataset available at http://osf.io/cg6eq/?view_only=f446b0f51bca409da4175a4ce8d9000a.
References
Bachrach, Y., et al.: Speeding up the Xbox recommender system using a euclidean transformation for inner-product spaces. In: Proceedings of the 8th ACM Conference on Recommender Systems, pp. 257–264. ACM (2014)
Blondel, V.D., Guillaume, J.L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. 2008, 1–12 (2008)
Cardoso, Â., Daolio, F., Vargas, S.: Product characterisation towards personalisation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining - KDD ’18, pp. 80–89 (2018)
Han, X., Wu, Z., Jiang, Y.G., Davis, L.S.: Learning fashion compatibility with bidirectional LSTMs. In: Proceedings of the 2017 ACM on Multimedia Conference - MM ’17, pp. 1078–1086 (2017)
He, T., Hu, Y.: FashionNet: Personalized Outfit Recommendation with Deep Neural Network (2018). http://arxiv.org/abs/1810.02443
Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: Feragen, A., Pelillo, M., Loog, M. (eds.) SIMBAD 2015. LNCS, vol. 9370, pp. 84–92. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24261-3_7
Hu, Y., Yi, X., Davis, L.S.: Collaborative fashion recommendation: a functional tensor factorization approach. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 129–138. Brisbane, Australia (2015)
Jimenez, A., Alvarez, J.M., Giro-i Nieto, X.: Class-weighted convolutional features for visual instance search. In: 28th British Machine Vision Conference (BMVC) (Sept 2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference for Learning Representations, San Diego (2015)
Li, Y., Cao, L., Zhu, J., Luo, J.: Mining fashion outfit composition using an end-to-end deep learning approach on set data. IEEE Trans. Multimedia 19(8), 1946–1955 (2017). https://doi.org/10.1109/TMM.2017.2690144
Ma, Y., Jia, J., Zhou, S., Fu, J., Liu, Y., Tong, Z.: Towards better understanding the clothing fashion styles: a multimodal deep learning approach. In: AAAI Conference on Artificial Intelligence, pp. 38–44 (2017)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
McAuley, J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: SIGIR Conference on Research and Development in Information Retrieval, pp. 43–52 (2015)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Neural Information Processing Systems, pp. 3111–3119 (2013). https://doi.org/10.1162/jmlr.2003.3.4-5.951
Nakamura, T., Goto, R.: Outfit generation and style extraction via bidirectional LSTM and autoencoder. In: The Third International Workshop on Fashion and KDD (2018). http://arxiv.org/abs/1807.03133
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014). https://doi.org/10.3115/v1/D14-1162
Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Conference on Uncertainty in Artificial Intelligence, vol. 1120, pp. 452–461 (2009)
Ribeiro, F., Florêncio, D., Zhang, C., Seltzer, M.: CROWDMOS: an approach for crowdsourcing mean opinion score studies. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 2416–2419 (2011). https://doi.org/10.1109/ICASSP.2011.5946971
Shih, Y.S., Chang, K.Y., Lin, H.T., Sun, M.: Compatibility family learning for item recommendation and generation. In: AAAI Conference on Artificial Intelligence, pp. 2403–2410 (2018). https://arxiv.org/pdf/1712.01262.pdf
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)
Song, X., Feng, F., Han, X., Yang, X., Liu, W., Nie, L.: Neural compatibility modeling with attentive knowledge distillation. In: SIGIR Conference on Research & Development in Information Retrieval, pp. 5–14 (2018)
Tangseng, P., Yamaguchi, K., Okatani, T.: Recommending outfits from personal closet. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 269–277 (2018)
Vasileva, M.I., Plummer, B.A., Dusad, K., Rajpal, S., Kumar, R., Forsyth, D.: Learning type-aware embeddings for fashion compatibility. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, Part XVI. LNCS, vol. 11220, pp. 405–421. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_24
Veit, A., Kovacs, B., Bell, S., Mcauley, J., Bala, K., Belongie, S.: Learning visual clothing style with heterogeneous dyadic co-occurrences. In: IEEE International Conference on Computer Vision, pp. 4642–4650 (2015)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Computer Vision and Pattern Recognition (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Bettaney, E.M., Hardwick, S.R., Zisimopoulos, O., Chamberlain, B.P. (2021). Fashion Outfit Generation for E-Commerce. In: Dong, Y., Ifrim, G., Mladenić, D., Saunders, C., Van Hoecke, S. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12461. Springer, Cham. https://doi.org/10.1007/978-3-030-67670-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-67670-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67669-8
Online ISBN: 978-3-030-67670-4
eBook Packages: Computer ScienceComputer Science (R0)
