Skip to main content
SpringerLink
Log in

Learning icons appearance similarity

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Selecting an optimal set of icons is a crucial step in the pipeline of visual design to structure and navigate through content. However, designing the icons sets is usually a difficult task for which expert knowledge is required. In this work, to ease the process of icon set selection to the users, we propose a similarity metric which captures the properties of style and visual identity. We train a Siamese Neural Network with an on-line dataset of icons organized in visually coherent collections that are used to adaptively sample training data and optimize the training process. As the dataset contains noise, we further collect human-rated information on the perception of icon’s similarity which will be used for evaluating and testing the proposed model. We present several results and applications based on searches, kernel visualizations and optimized set proposals that can be helpful for designers and non-expert users while exploring large collections of icons.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://material.google.com/

  2. https://thenounproject.com/

References

  1. Airey D (2016) Logo design love a guide to creating iconic brand identities. Peachpit Press, Berkeley

    Google Scholar 

  2. Bai X, Yang X, Latecki LJ, Liu W, Tu Z (2010) Learning context-sensitive shape similarity by graph transduction. IEEE Trans Pattern Anal Mach Intell 32(5):861–874

    Article  Google Scholar 

  3. Barnard M (2013) Graphic design as communication. Routledge, Abingdon

    Book  Google Scholar 

  4. Bates CL, Cragun BJ, Day PR (2002) Automatic icon generation, US Patent 6,456,307

  5. Bell S, Bala K (2015) Learning visual similarity for product design with convolutional neural networks. ACM Trans Graph (Proc SIGGRAPH) 34:4

    Article  Google Scholar 

  6. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522

    Article  Google Scholar 

  7. Bernstein GL, Li W (2015) Lillicon: using transient widgets to create scale variations of icons. ACM Trans Graph (TOG) 34(4):144

    Article  Google Scholar 

  8. Bober M (2001) Mpeg-7 visual shape descriptors. IEEE Trans Circ Syst Video Technol 11(6):716–719

    Article  Google Scholar 

  9. Bromley J, Guyon I, Lecun Y, Säckinger E, Shah R (1994) Signature verification using a ”siamese” time delay neural network. In: NIPS Proceedings

  10. Crandall DJ, Huttenlocher DP (2006) Weakly supervised learning of part-based spatial models for visual object recognition. In: Leonardis A, Bischof H, Pinz A (eds) Computer Vision - ECCV 2006, pp 16–29

  11. Demiralp Ç, Bernstein MS, Heer J (2014) Learning perceptual kernels for visualization design. IEEE Trans Vis Comput Graph 20(12):1933–1942

    Article  Google Scholar 

  12. Doulamis A, Doulamis N (2004) Generalized nonlinear relevance feedback for interactive content-based retrieval and organization. IEEE Trans Circ Syst Video Technol 14(5):656–671

    Article  Google Scholar 

  13. El-Naqa I, Yang Y, Galatsanos NP, Nishikawa RM, Wernick MN (2004) A similarity learning approach to content-based image retrieval: application to digital mammography. IEEE Trans Med Imaging 23(10):1233–1244

    Article  Google Scholar 

  14. Farnung-Laursen L, Koyama Y, Chen H-T, Garces E, Gutierrez D, Harper R, Igarashi T (2016) Icon Set Selection via Human Computation. In: Pacific Graphics Short Papers

  15. Fried O, Avidan S, Cohen-Or D (2017) Patch2vec: Globally consistent image patch representation. Pac Graph 36:7

    Google Scholar 

  16. Garces E, Agarwala A, Gutierrez D, Hertzmann A (2014) A similarity measure for illustration style. ACM Trans Graph (Proc SIGGRAPH) 33:4

    Article  Google Scholar 

  17. Garces E, Agarwala A, Hertzmann A, Gutierrez D (2017) Style-based Exploration of Illustration Datasets. Multimed Tools Appl 76:11

    Article  Google Scholar 

  18. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, vol 9 of Proceedings of Machine Learning Research, PMLR, pp 249–256

  19. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning. MIT Press, Cambridge. http://www.deeplearningbook.org

    MATH  Google Scholar 

  20. Gramazio CC, Laidlaw DH, Schloss KB (2017) Colorgorical Creating discriminable and preferable color palettes for information visualization. IEEE Trans Vis Comput Graph 23(1):521–530

    Article  Google Scholar 

  21. Horton WK (1994) The icon book: Visual symbols for computer systems and documentation. Wiley, NY

    Google Scholar 

  22. Hu M-K (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8(2):179–187

    Article  Google Scholar 

  23. Ioffe S, Szegedy C (2015) Batch normalization Accelerating deep network training by reducing internal covariate shift. CoRR arXiv:1502.03167

  24. Khotanzad A, Hong YH (1990) Invariant image recognition by zernike moments. IEEE Trans Pattern Anal Mach Intell 12(5):489–497

    Article  Google Scholar 

  25. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. CoRR arXiv:1412.6980

  26. Kleiman Y, van Kaick O, Sorkine-Hornung O, Cohen-Or D (2015) Shed: shape edit distance for fine-grained shape similarity. ACM Trans Graph (TOG) 34(6):235

    Article  Google Scholar 

  27. Kwan KC, Sinn LT, Han C, Wong T-T, Fu C-W (2016) Pyramid of arclength descriptor for generating collage of shapes. ACM Trans Graph (Proc SIGGRAPH Asia) 35(6):1–12

    Article  Google Scholar 

  28. Latecki LJ, Lakamper R, Eckhardt T (2000) Shape descriptors for non-rigid shapes with a single closed contour. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE, vol 1, pp 424–429

  29. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  30. Lewis JP, Rosenholtz R, Fong N, Neumann U (2004) VisualIDs Automatic Distinctive Icons for Desktop Interfaces. ACM Trans Graph (Proc SIGGRAPH) 1(212):416–423

    Article  Google Scholar 

  31. Liu T, Hertzmann A, Li W, Funkhouser T (2015) Style compatibility for 3D furniture models. ACM Trans Graph (Proc SIGGRAPH) 34(4):1–9

    Article  Google Scholar 

  32. Liu Y, Agarwala A, Lu J, Rusinkiewicz S (2016) Data-driven iconification. In: Proceedings of Computational Aesthetics and Sketch Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering. Eurographics Association, pp 113–124

  33. Lun Z, Kalogerakis E, Sheffer A (2015) Elements of style learning perceptual shape style similarity. ACM Trans Graph (TOG) 34 4:84:1–14

    Article  Google Scholar 

  34. Lupton E (2004) Thinking with type Critical Guide for Designers, Writers, Editors & Students

  35. Lupton E, Phillips JC (2015) Graphic design: The New Basics: Revised and Expanded. Chronicle Books

  36. Mohler DS, Vick JH (2015) Screen icon manipulation by context and frequency of use, US Patent 8,972,878

  37. O’Donovan P, Agarwala A, Hertzmann A (2011) Color Compatibility From Large Datasets. ACM Trans Graph 30:4

    Google Scholar 

  38. O’Donovan P, Lı̄beks J, Agarwala A, Hertzmann A (2014) Exploratory font selection using crowdsourced attributes. ACM Trans Graph (TOG) 33(4):92

    Google Scholar 

  39. Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. ACM Trans Graph (TOG) 21(4):807–832

    Article  MathSciNet  Google Scholar 

  40. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: Proceedings of the British Machine Vision Conference (BMVC)

  41. Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. ACM Trans Graph (Proc SIGGRAPH Asia) 29 6:160:1–160:10

    Google Scholar 

  42. Saleh B, Dontcheva M, Hertzmann A, Liu Z (2015) Learning style similarity for searching infographics. In: Proceedings of the 41st graphics interface conference. GI ’15. Canadian Information Processing Society, pp 59–64

  43. Schroff F, Kalenichenko D, Philbin J (2015) FaceNet: A unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, vol 07-12-June, pp 815–823

  44. Setlur V, Mackinlay JD (2014) Automatic generation of semantic icon encodings for visualizations. Inproceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’14, pp 541–550

  45. Setlur V, Albrecht-Buehler C, Gooch AA, Rossoff S, Gooch B (2005) Semanticons Visual metaphors as file icons. Comput Graph Forum (Proc Eurographics) 24(3):647–656

    Article  Google Scholar 

  46. Shugrina M, Lu J, Diverdi S (2017) Playful palette: an interactive parametric color mixer for artists. ACM Trans Graph (TOG) 36(4):61

    Article  Google Scholar 

  47. Sikora T (2001) The mpeg-7 visual standard for content description-an overview. IEEE Trans Circ Syst Video Technol 11(6):696–702

    Article  Google Scholar 

  48. Simo-Serra E, Ishikawa H (2016) Fashion Style in 128 Floats: Joint Ranking and Classification using Weak Data for Feature Extraction. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR)

  49. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. CoRR arXiv:1409.1556

  50. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  51. Torresani L (2014) Weakly supervised learning. Springer, Boston, pp 883–885

    Google Scholar 

  52. Upchurch P, Snavely N, Bala K (2016) From A to Z: supervised transfer of style and content using deep neural network generators. CoRR arXiv:1603.02003

  53. van der Maaten L, Hinton GE (2008) Visualizing high-dimensional data using t-sne. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  54. Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E Deep learning for computer vision: A brief review, pp 1–13

  55. Wu P, Hoi SC, Xia H, Zhao P, Wang D, Miao C (2013) Online multimodal deep similarity learning with application to image retrieval. In: Proceedings of the 21st ACM International Conference on Multimedia, pp 153–162

  56. Xia H, Hoi SCH, Jin R, Zhao P (2014) Online multiple kernel similarity learning for visual search. IEEE Trans Pattern Anal Mach Intell 36(3):536–549

    Article  Google Scholar 

  57. Yin W, Schütze H, Xiang B, Zhou B (2015) ABCNN: Attention-based convolutional neural network for modeling sentence pairs. CoRR arXiv:1512.05193

  58. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In European conference on computer vision. Springer, pp 818–833

  59. Zhang D, Lu G (2002) Shape-based image retrieval using generic fourier descriptor. Signal Process Image Commun 17(10):825–848

    Article  Google Scholar 

Download references

Acknowledgments

We want to thank the anonymous reviewers and Adrian Jarabo for their insightful comments on the manuscript. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (CHAMELEON project, grant agreement No 682080).

Author information

Authors and Affiliations

  1. Universidad de Zaragoza, I3A, Zaragoza, Spain

    Manuel Lagunas & Diego Gutierrez

  2. Technicolor, 975 Avenue des Champs Blancs, 35576, Cesson-Sévigné, France

    Elena Garces

Authors
  1. Manuel Lagunas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elena Garces
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diego Gutierrez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuel Lagunas.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lagunas, M., Garces, E. & Gutierrez, D. Learning icons appearance similarity. Multimed Tools Appl 78, 10733–10751 (2019). https://doi.org/10.1007/s11042-018-6628-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6628-7

Keywords

Search

Navigation