Discrete Binary Hashing Towards Efficient Fashion Recommendation

  • Luyao Liu
  • Xingzhong Du
  • Lei Zhu
  • Fumin Shen
  • Zi Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10827)


How to match clothing well is always a troublesome problem in our daily life, especially when we are shopping online to select a pair of matched pieces of clothing from tens of thousands available selections. To help common customers overcome selection difficulties, recent studies in the recommender system area have started to infer the fashion matching results automatically. The conventional fashion recommendation is normally achieved by considering visual similarity of clothing items or/and item co-purchase history from existing shopping transactions. Due to the high complexity of visual features and the lack of historical item purchase records, most of the existing work is unlikely to make an efficient and accurate recommendation. To address the problem, in this paper we propose a new model called Discrete Supervised Fashion Coordinates Hashing (DSFCH). Its main objective is to learn meaningful yet compact high level features of clothing items, which are represented as binary hash codes. In detail, this learning process is supervised by a clothing matching matrix, which is initially constructed based on limited known matching pairs and subsequently on the self-augmented ones. The proposed model jointly learns the intrinsic matching patterns from the matching matrix and the binary representations from the clothing items’ images, where the visual feature of each clothing item is discretized into a fixed-length binary vector. The binary representation learning significantly reduces the memory cost and accelerates the recommendation speed. The experiments compared with several state-of-the-art approaches have evidenced the superior performance of the proposed approach on efficient fashion recommendation.


  1. 1.
    Al-Halah, Z., Stiefelhagen, R., Grauman, K.: Fashion forward: forecasting visual style in fashion. In: ICCV, October 2017Google Scholar
  2. 2.
    Andoni, A., Indyk, P.: Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Commun. ACM 51(1), 117–122 (2008)CrossRefGoogle Scholar
  3. 3.
    Andoni, A., Razenshteyn, I.: Optimal data-dependent hashing for approximate near neighbors. In: STOC, STOC 2015, pp. 793–801. ACM (2015)Google Scholar
  4. 4.
    Bracher, C., Heinz, S., Vollgraf, R.: Fashion DNA: merging content and sales data for recommendation and article mapping. CoRR abs/1609.02489 (2016)Google Scholar
  5. 5.
    Chen, K., Chen, K., Cong, P., Hsu, W.H., Luo, J.: Who are the devils wearing Prada in New York city? In: Proceedings of the 23rd ACM international conference on Multimedia, pp. 177–180. ACM (2015)Google Scholar
  6. 6.
    Gong, Y., Lazebnik, S., Gordo, A., Perronnin, F.: Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. TPAMI 35(12), 2916–2929 (2013)CrossRefGoogle Scholar
  7. 7.
    Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. CVPR 2, 1735–1742 (2006)Google Scholar
  8. 8.
    He, R., Packer, C., McAuley, J.: Learning compatibility across categories for heterogeneous item recommendation. In: ICDM, pp. 937–942. IEEE (2016)Google Scholar
  9. 9.
    Iwata, T., Wanatabe, S., Sawada, H.: Fashion coordinates recommender system using photographs from fashion magazines. In: IJCAI, vol. 22, p. 2262 (2011)Google Scholar
  10. 10.
    Jagadeesh, V., Piramuthu, R., Bhardwaj, A., Di, W., Sundaresan, N.: Large scale visual recommendations from street fashion images. In: SIGKDD, KDD 2014, pp. 1925–1934. ACM (2014)Google Scholar
  11. 11.
    Jiang, Q.Y., Li, W.J.: Scalable graph hashing with feature transformation. In: IJCAI, IJCAI 2015, pp. 2248–2254. AAAI Press (2015)Google Scholar
  12. 12.
    Kang, W.C., Fang, C., Wang, Z., McAuley, J.: Visually-aware fashion recommendation and design with generative image models. arXiv preprint arXiv:1711.02231 (2017)
  13. 13.
    Kiapour, M.H., Han, X., Lazebnik, S., Berg, A.C., Berg, T.L.: Where to buy it: matching street clothing photos in online shops. In: ICCV, pp. 3343–3351 (2015)Google Scholar
  14. 14.
    Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)CrossRefGoogle Scholar
  15. 15.
    Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. TOMM 2(1), 1–19 (2006)CrossRefGoogle Scholar
  16. 16.
    Liong, V.E., Lu, J., Wang, G., Moulin, P., Zhou, J.: Deep hashing for compact binary codes learning. In: CVPR, pp. 2475–2483 (2015)Google Scholar
  17. 17.
    Liu, L., Lin, Z., Shao, L., Shen, F., Ding, G., Han, J.: Sequential discrete hashing for scalable cross-modality similarity retrieval. TIP 26(1), 107–118 (2017)MathSciNetGoogle Scholar
  18. 18.
    Liu, L., Zhu, L., Li, Z.: Learning robust graph hashing for efficient similarity search. In: Huang, Z., Xiao, X., Cao, X. (eds.) ADC 2017. LNCS, vol. 10538, pp. 110–122. Springer, Cham (2017). Scholar
  19. 19.
    Liu, W., Wang, J., Ji, R., Jiang, Y.G., Chang, S.F.: Supervised hashing with kernels. In: CVPR, pp. 2074–2081 (2012)Google Scholar
  20. 20.
    Liu, W., Mu, C., Kumar, S., Chang, S.F.: Discrete graph hashing. In: NIPS, NIPS 2014, pp. 3419–3427. MIT Press (2014)Google Scholar
  21. 21.
    Liu, W., Wang, J., Ji, R., Jiang, Y.G., Chang, S.F.: Supervised hashing with kernels. In: CVPR, pp. 2074–2081. IEEE (2012)Google Scholar
  22. 22.
    Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: DeepFashion: powering robust clothes recognition and retrieval with rich annotations. In: CVPR (2016)Google Scholar
  23. 23.
    Liu, Z., Yan, S., Luo, P., Wang, X., Tang, X.: Fashion landmark detection in the wild. In: ECCV (2016)CrossRefGoogle Scholar
  24. 24.
    McAuley, J., Pandey, R., Leskovec, J.: Inferring networks of substitutable and complementary products. In: SIGKDD, pp. 785–794. ACM (2015)Google Scholar
  25. 25.
    McAuley, J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: SIGIR, SIGIR 2015, pp. 43–52. ACM (2015)Google Scholar
  26. 26.
    McAuley, J., Yang, A.: Addressing complex and subjective product-related queries with customer reviews. In: Proceedings of the 25th International Conference on World Wide Web, pp. 625–635. International World Wide Web Conferences Steering Committee (2016)Google Scholar
  27. 27.
    Shen, F., Shen, C., Liu, W., Shen, H.T.: Supervised discrete hashing. In: CVPR, pp. 37–45 (2015)Google Scholar
  28. 28.
    Shen, F., Liu, W., Zhang, S., Yang, Y., Tao Shen, H.: Learning binary codes for maximum inner product search. In: ICCV, pp. 4148–4156 (2015)Google Scholar
  29. 29.
    Shrivastava, A., Li, P.: Asymmetric LSH (ALSH) for sublinear time maximum inner product search (mips). In: NIPS, pp. 2321–2329 (2014)Google Scholar
  30. 30.
    Simo-Serra, E., Fidler, S., Moreno-Noguer, F., Urtasun, R.: Neuroaesthetics in fashion: modeling the perception of fashionability. In: CVPR, pp. 869–877 (2015)Google Scholar
  31. 31.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  32. 32.
    Song, J., Yang, Y., Yang, Y., Huang, Z., Shen, H.T.: Inter-media hashing for large-scale retrieval from heterogeneous data sources. In: SIGMOD, SIGMOD 2013, pp. 785–796. ACM (2013)Google Scholar
  33. 33.
    Veit, A., Kovacs, B., Bell, S., McAuley, J., Bala, K., Belongie, S.: Learning visual clothing style with heterogeneous dyadic co-occurrences. In: ICCV, pp. 4642–4650 (2015)Google Scholar
  34. 34.
    Wan, M., McAuley, J.: Modeling ambiguity, subjectivity, and diverging viewpoints in opinion question answering systems. In: ICDM, pp. 489–498. IEEE (2016)Google Scholar
  35. 35.
    Wang, C., Blei, D.M.: Collaborative topic modeling for recommending scientific articles. In: SIGKDD, pp. 448–456. ACM (2011)Google Scholar
  36. 36.
    Wang, H., Wang, N., Yeung, D.Y.: Collaborative deep learning for recommender systems. In: SIGKDD, pp. 1235–1244. ACM (2015)Google Scholar
  37. 37.
    Wang, J., Kumar, S., Chang, S.F.: Semi-supervised hashing for scalable image retrieval. In: CVPR, pp. 3424–3431 (2010)Google Scholar
  38. 38.
    Wang, J., Kumar, S., Chang, S.F.: Semi-supervised hashing for large-scale search. TPAMI 34(12), 2393–2406 (2012)CrossRefGoogle Scholar
  39. 39.
    Wang, J., Xu, X.S., Guo, S., Cui, L., Wang, X.L.: Linear unsupervised hashing for ANN search in Euclidean space. Neurocomputing 171, 283–292 (2016)CrossRefGoogle Scholar
  40. 40.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) NIPS, pp. 1753–1760. Curran Associates, Inc. (2009)Google Scholar
  41. 41.
    Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. AAAI 1, 2156–2162 (2014)Google Scholar
  42. 42.
    Xu, H., Wang, J., Li, Z., Zeng, G., Li, S., Yu, N.: Complementary hashing for approximate nearest neighbor search. In: ICCV, pp. 1631–1638 (2011)Google Scholar
  43. 43.
    Zhang, P., Zhang, W., Li, W.J., Guo, M.: Supervised hashing with latent factor models. In: SIGIR, SIGIR 2014, pp. 173–182. ACM (2014)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Luyao Liu
    • 1
  • Xingzhong Du
    • 1
  • Lei Zhu
    • 2
  • Fumin Shen
    • 3
  • Zi Huang
    • 1
  1. 1.School of ITEEThe University of QueenslandBrisbaneAustralia
  2. 2.School of Information Science and EngineeringShandong Normal UniversityJinanChina
  3. 3.School of Computer Science and EngineeringUESTCChengduChina

Personalised recommendations