A review of design intelligence: progress, problems, and challenges

Abstract

Design intelligence is an important branch of artificial intelligence (AI), focusing on the intelligent models and algorithms in creativity and design. In the context of AI 2.0, studies on design intelligence have developed rapidly. We summarize mainly the current emerging framework of design intelligence and review the state-of-the-art techniques of related topics, including user needs analysis, ideation, content generation, and design evaluation. Specifically, the models and methods of intelligence-generated content are reviewed in detail. Finally, we discuss some open problems and challenges for future research in design intelligence.

This is a preview of subscription content, access via your institution.

References

  1. Arjovsky M, Chintala S, Bottou L, 2017. Wasserstein generative adversarial networks. Proc 34th Int Conf on Machine Learning, p.298–321.

  2. Aubry M, Maturana D, Efros AA, et al., 2014. Seeing 3D chairs: exemplar part-based 2D-3D alignment using a large dataset of CAD models. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.3762–3769. https://doi.org/10.1109/CVPR.2014.487

  3. Ballester C, Bertalmio M, Caselles V, et al., 2001. Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans Image Process, 10(8):1200–1211. https://doi.org/10.1109/83.935036

    MathSciNet  MATH  Google Scholar 

  4. Bertalmio M, Sapiro G, Caselles V, et al., 2000. Image in-painting. Proc 27th Annual Conf on Computer Graphics and Interactive Techniques, p.417–424. https://doi.org/10.1145/344779.344972

  5. Bharadhwaj H, Park H, Lim BY, 2018. RecGAN: recurrent generative adversarial networks for recommendation systems. Proc 12th ACM Conf on Recommender Systems, p.372–376. https://doi.org/10.1145/3240323.3240383

  6. Boden MA, 2009. Computer models of creativity. AI Mag, 30(3):23–34. https://doi.org/10.1609/aimag.v30i3.2254

    Google Scholar 

  7. Brock A, Donahue J, Simonyan K, 2018. Large scale GAN training for high fidelity natural image synthesis. https://doi.org/1809.11096

  8. Bruna J, Sprechmann P, LeCun Y, 2015. Super-resolution with deep convolutional sufficient statistics. https://doi.org/1511.05666

  9. Chakrabarti A, Siddharth L, Dinakar M, et al., 2017. Idea inspire 3.0—a tool for analogical design. In: Chakrabarti A, Chakrabarti D (Eds.), Research into Design for Communities. Springer, Singapore, p.475–485. https://doi.org/10.1007/978-981-10-3521-0_41

    Google Scholar 

  10. Champandard AJ, 2016. Semantic style transfer and turning two-bit doodles into fine artworks. https://doi.org/1603.01768

  11. Chan C, Ginosar S, Zhou TH, et al., 2018. Everybody dance now. https://doi.org/1808.07371

  12. Chen DD, Yuan L, Liao J, et al., 2018. Stereoscopic neural style transfer. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.6654–6663. https://doi.org/10.1109/CVPR.2018.00696

  13. Chen LQ, Wang P, Dong H, et al., 2019. An artificial intelligence based data-driven approach for design ideation. J Vis Commun Image Represent, 61:10–22. https://doi.org/10.1016/j.jvcir.2019.02.009

    Google Scholar 

  14. Ciesielski V, Barile P, Trist K, 2013. Finding image features associated with high aesthetic value by machine learning. Proc 2nd Int Conf on Evolutionary and Biologically Inspired Music, Sound, Art and Design, p.47–58. https://doi.org/10.1007/978-3-642-36955-1_5

    Google Scholar 

  15. Cooper A, 1999. The Inmates Are Running the Asylum. SAMS, Indianapolis, USA.

    Google Scholar 

  16. Cooper A, Reimann RM, 2003. About Face 2.0: the Essentials of Interaction Design. John Wiley & Sons, Indianapolis, USA.

    Google Scholar 

  17. Dash A, Gamboa JCB, Ahmed S, et al., 2017. TAC-GAN-text conditioned auxiliary classifier generative adversarial network. https://doi.org/1703.06412

  18. Datta R, Joshi D, Li J, et al., 2006. Studying aesthetics in photographic images using a computational approach. Proc 9th European Conf on Computer Vision, p.288–301. https://doi.org/10.1007/11744078_23

    Google Scholar 

  19. de Gómez Silva Garza A, Maher ML, 1999. An evolutionary approach to case adaptation. Proc 3rd Int Conf on Case-Based Reasoning, p.162–173. https://doi.org/10.1007/3-540-48508-2_12

    Google Scholar 

  20. de Silva Garza AG, 2019. An introduction to and comparison of computational creativity and design computing. Artif Intell Rev, 51(1):61–76. https://doi.org/10.1007/s10462-017-9557-3

    Google Scholar 

  21. Deng J, Dong W, Socher R, et al., 2009. ImageNet: a large-scale hierarchical image database. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.248–255. https://doi.org/10.1109/CVPR.2009.5206848

  22. Deng YB, Loy CC, Tang XO, 2018. Aesthetic-driven image enhancement by adversarial learning. Proc 26th ACM Int Conf on Multimedia, p.870–878. https://doi.org/10.1145/3240508.3240531

  23. Donahue J, Krähenbühl P, Darrell T, 2016. Adversarial feature learning. https://doi.org/1605.09782

  24. Dou Q, Zheng XS, Sun TF, et al., 2019. Webthetics: quantifying webpage aesthetics with deep learning. Int J Hum Comput Stud, 124:56–66. https://doi.org/10.1016/j.ijhcs.2018.11.006

    Google Scholar 

  25. Dugosh KL, Paulus PB, Roland EJ, et al., 2000. Cognitive stimulation in brainstorming. J Pers Soc Psychol, 79(5):722–735. https://doi.org/10.1037/0022-3514.79.5.722

    Google Scholar 

  26. Dumoulin V, Visin F, 2016. A guide to convolution arithmetic for deep learning. https://doi.org/1603.07285

  27. Edelman RR, Hesselink JR, Zlatkin MB, 1996. MRI: Clinical Magnetic Resonance Imaging. Saunders, Philadelphia.

    Google Scholar 

  28. Efros AA, Freeman WT, 2001. Image quilting for texture synthesis and transfer. Proc 28th Annual Conf on Computer Graphics and Interactive Techniques, p.341–346. https://doi.org/10.1145/383259.383296

  29. Elgammal A, Liu B, Elhoseiny M, et al., 2017. CAN: creative adversarial networks, generating “art” by learning about styles and deviating from style norms. https://doi.org/1706.07068

  30. Fang H, Zhang M, 2017. Creatism: a deep-learning photographer capable of creating professional work. https://doi.org/1707.03491

  31. Faste H, Rachmel N, Essary R, et al., 2013. Brainstorm, chainstorm, cheatstorm, tweetstorm: new ideation strategies for distributed HCI design. Proc Conf on Human Factors in Computing Systems, p.1343–1352. https://doi.org/10.1145/2470654.2466177

  32. Fu K, Murphy J, Yang M, et al., 2015. Design-by-analogy: experimental evaluation of a functional analogy search methodology for concept generation improvement. Res Eng Des, 26(1):77–95. https://doi.org/10.1007/s00163-014-0186-4

    Google Scholar 

  33. Garabedian CA, 1934. Birkhoff on aesthetic measure. Bull Amer Math Soc, 40(1):7–10. https://doi.org/10.1090/S0002-9904-1934-05764-1

    MathSciNet  Google Scholar 

  34. Gatys L, Ecker A, Bethge M, 2016a. Image style transfer using convolutional neural networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2414–2423. https://doi.org/10.1109/CVPR.2016.265

  35. Gatys L, Ecker A, Bethge M, 2016b. A neural algorithm of artistic style. J Vis, 16(12):326. https://doi.org/10.1167/16.12.326

    Google Scholar 

  36. Gero JS, 1990. Design prototypes: a knowledge representation schema for design. AI Mag, 11(4):26–36.

    Google Scholar 

  37. Gilon K, Chan J, Ng FY, et al., 2018. Analogy mining for specific design needs. Proc CHI Conf on Human Factors in Computing Systems, p.121. https://doi.org/10.1145/3173574.3173695

  38. Goel AK, Rugaber S, Vattam S, 2009. Structure, behavior, and function of complex systems: the structure, behavior, and function modeling language. AI Edam, 23(1):23–35. https://doi.org/10.1017/S0890060409000080

    Google Scholar 

  39. Goldschmidt G, Smolkov M, 2006. Variances in the impact of visual stimuli on design problem solving performance. Des Stud, 27(5):549–569. https://doi.org/10.1016/j.destud.2006.01.002

    Google Scholar 

  40. Gooch B, Gooch A, 2001. Non-photorealistic Rendering. A K Peters/CRC Press, New York, USA. https://doi.org/10.1201/9781439864173

    Google Scholar 

  41. Goodfellow I, Pouget-Abadie J, Mirza M, et al., 2014. Generative adversarial nets. Proc 27th Int Conf on Neural Information Processing Systems, p.2672–2680.

  42. Grudin J, Pruitt J, 2002. Personas, participatory design, and product development: an infrastructure for engagement. Proc 7th Biennial Participatory Design Conf, p.144–152.

  43. Gulrajani I, Ahmed F, Arjovsky M, et al., 2017. Improved training of Wasserstein GANs. Advances in Neural Information Proc Systems, p.5767–5777.

  44. Han J, Shi F, Chen LQ, et al., 2018. A computational tool for creative idea generation based on analogical reasoning and ontology. Artif Intell Eng Des Anal Manuf, 32(4):462–477. https://doi.org/10.1017/S0890060418000082

    Google Scholar 

  45. Hao J, Zhou YJ, Zhao QF, et al., 2019. An evolutionary computation based method for creative design inspiration generation. J Intell Manuf, 30(4):1673–1691. https://doi.org/10.1007/s10845-017-1347-x

    Google Scholar 

  46. Hartson R, Pyla PS, 2012. The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Elsevier, Amsterdam. https://doi.org/10.1016/C2010-0-66326-7

    Google Scholar 

  47. He KM, Sun J, 2014. Image completion approaches using the statistics of similar patches. IEEE Trans Patt Anal Mach Intell, 36(12):2423–2435. https://doi.org/10.1109/TPAMI.2014.2330611

    Google Scholar 

  48. Hertzmann A, Jacobs CE, Oliver N, et al., 2001. Image analogies. Proc 28th Annual Conf on Computer Graphics and Interactive Techniques, p.327–340. https://doi.org/10.1145/383259.383295

  49. Hong YJ, Hwang U, Yoo J, et al., 2019. How generative adversarial networks and their variants work: an overview. ACM Comput Surv, 52(1):10. https://doi.org/10.1145/3301282

    Google Scholar 

  50. Huang HZ, Wang H, Luo WH, et al., 2017. Real-time neural style transfer for videos. IEEE Conf on Computer Vision and Pattern Recognition, p.7044–7052. https://doi.org/10.1109/CVPR.2017.745

  51. Huang X, Belongie S, 2017. Arbitrary style transfer in realtime with adaptive instance normalization. Proc IEEE Int Conf on Computer Vision, p.1501–1510. https://doi.org/10.1109/ICCV.2017.167

  52. Iizuka S, Simo-Serra E, Ishikawa H, 2017. Globally and locally consistent image completion. ACM Trans Graph, 36(4), Article 107. https://doi.org/10.1145/3072959.3073659

    Google Scholar 

  53. Isola P, Zhu JY, Zhou TH, et al., 2017. Image-to-image translation with conditional adversarial networks. IEEE Conf on Computer Vision and Pattern Recognition, p.5967–5976. https://doi.org/10.1109/CVPR.2017.632

  54. Jansen BJ, Jung SG, Salminen J, et al., 2017. Viewed by too many or viewed too little: using information dissemination for audience segmentation. Proc Assoc Inform Sci Technol, 54(1):189–196. https://doi.org/10.1002/pra2.2017.14505401021

    Google Scholar 

  55. Jansson DG, Smith SM, 1991. Design fixation. Des Stud, 12(1):3–11. https://doi.org/10.1016/0142-694X(91)90003-F

    Google Scholar 

  56. Jia J, Huang J, Shen GY, et al., 2016. Learning to appreciate the aesthetic effects of clothing. Proc 30th AAAI Conf on Artificial Intelligence, p.1216–1222.

  57. Jia L, Becattini N, Cascini G, et al., 2020. Testing ideation performance on a large set of designers: effects of analogical distance. Int J Des Creat Innov, 8(1):31–45. https://doi.org/10.1080/21650349.2019.1618736

    Google Scholar 

  58. Jiang SH, Fu Y, 2017. Fashion style generator. Proc 26th Int Joint Conf on Artificial Intelligence, p.3721–3727. https://doi.org/10.24963/ijcai.2017/520

  59. Jing YC, Yang YZ, Feng ZL, et al., 2019. Neural style transfer: a review. IEEE Trans Vis Comput Graph, in press. https://doi.org/10.1109/tvcg.2019.2921336

  60. Jo Y, Park J, 2019. SC-FEGAN: face editing generative adversarial network with user’s sketch and color. https://doi.org/1902.06838

  61. Johnson J, Alahi A, Li FF, 2016. Perceptual losses for real-time style transfer and super-resolution. Proc 14th European Conf, p.694–711. https://doi.org/10.1007/978-3-319-46475-6_43

    Google Scholar 

  62. Karras T, Laine S, Aila T, 2019. A style-based generator architecture for generative adversarial networks. The IEEE Conf on Computer Vision and Pattern Recognition, p.4401–4410.

  63. Kaufman JC, Sternberg RJ, 2006. The International Handbook of Creativity. Edward Elgar Publishing, Cheltenham, UK.

    Google Scholar 

  64. Keys R, 1981. Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process, 29(6):1153–1160. https://doi.org/10.1109/TASSP.1981.1163711

    MathSciNet  MATH  Google Scholar 

  65. Kim J, Lee JK, Lee KM, 2016. Accurate image superresolution using very deep convolutional networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1646–1654. https://doi.org/10.1109/CVPR.2016.182

  66. Kingma DP, Welling M, 2013. Auto-encoding variational Bayes. https://arxiv.org/abs/1312.6114

  67. Kong S, Shen XH, Lin Z, et al., 2016. Photo aesthetics ranking network with attributes and content adaptation. Proc 14th European Conf on Computer Vision, p.662–679. https://doi.org/10.1007/978-3-319-46448-0_40

    Google Scholar 

  68. Krizhevsky A, Hinton G, 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report, University of Toronto, Toronto.

    Google Scholar 

  69. Kwak H, An J, Jansen BJ, 2017. Automatic generation of personas using YouTube social media data. Proc 50th Hawaii Int Conf on System Sciences, p.833–842.

  70. Larsen ABL, Sønderby SK, Larochelle H, et al., 2016. Autoencoding beyond pixels using a learned similarity metric. Proc 33rd Int Conf on Machine Learning, p.1558–1566.

  71. LeCun Y, Bottou L, Bengio Y, et al., 1998. Gradient-based learning applied to document recognition. Proc IEEE, 86(11):2278–2323. https://doi.org/10.1109/5.726791

    Google Scholar 

  72. Ledig C, Theis L, Huszár F, et al., 2017. Photo-realistic single image super-resolution using a generative adversarial network. IEEE Conf on Computer Vision and Pattern Recognition, p.105–114. https://doi.org/10.1109/CVPR.2017.19

  73. Li C, Wand M, 2016. Precomputed real-time texture synthesis with Markovian generative adversarial networks. Proc 14th European Conf on Computer Vision, p.702–716. https://doi.org/10.1007/978-3-319-46487-9_43

    Google Scholar 

  74. Li CC, Chen T, 2009. Aesthetic visual quality assessment of paintings. IEEE J Sel Top Signal Process, 3(2):236–252. https://doi.org/10.1109/JSTSP.2009.2015077

    Google Scholar 

  75. Li HH, Wang JG, Tang MM, et al., 2017. Polarization-dependent effects of an Airy beam due to the spin-orbit coupling. J Opt Soc Am A, 34(7):1114–1118. https://doi.org/10.1364/JOSAA.34.001114

    Google Scholar 

  76. Li XT, Liu SF, Kautz J, et al., 2019. Learning linear transformations for fast arbitrary style transfer. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.3809–3817.

  77. Li YJ, Fang C, Yang JM, et al., 2017. Universal style transfer via feature transforms. Proc 31st Conf on Neural Information Processing Systems, p.386–396.

  78. Liu GL, Reda FA, Shih KJ, et al., 2018. Image inpainting for irregular holes using partial convolutions. Proc 15th European Conf on Computer Vision, p.85–105. https://doi.org/10.1007/978-3-030-01252-6_6

    Google Scholar 

  79. Liu H, Singh P, 2004. ConceptNet—a practical commonsense reasoning tool-kit. BT Technol J, 22(4):211–226. https://doi.org/10.1023/B:BTTJ.0000047600.45421.6d

    Google Scholar 

  80. Liu MY, Huang X, Mallya A, et al., 2019. Few-shot unsupervised image-to-image translation. https://doi.org/1905.01723

  81. Liu ZW, Luo P, Wang XG, et al., 2015. Deep learning face attributes in the wild. Proc IEEE Int Conf on Computer Vision, p.3730–3738. https://doi.org/10.1109/ICCV.2015.425

  82. Lowdermilk T, 2013. User-Centered Design: a Developer’s Guide to Building User-Friendly Applications. O’Reilly, Beijing, China.

    Google Scholar 

  83. Lu X, Lin Z, Shen XH, et al., 2015. Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. Proc IEEE Int Conf on Computer Vision, p.990–998. https://doi.org/10.1109/ICCV.2015.119

  84. Luo YW, Tang XO, 2008. Photo and video quality evaluation: focusing on the subject. Proc 10th European Conf on Computer Vision, p.386–399.

    Google Scholar 

  85. Ma S, Liu J, Chen WC, 2017. A-lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. Proc 30th IEEE Conf on Computer Vision and Pattern Recognition, p.722–731. https://doi.org/10.1109/CVPR.2017.84

  86. Maguire M, Bevan N, 2002. User requirements analysis. In: Hammond J, Gross T, Wesson J (Eds.), Usability: Gaining a Competitive Edge. Springer, Boston, USA, p.133–148. https://doi.org/10.1007/978-0-387-35610-5_9

    Google Scholar 

  87. Mai L, Jin HL, Liu F, 2016. Composition-preserving deep photo aesthetics assessment. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.497–506. https://doi.org/10.1109/CVPR.2016.60

  88. Matthews T, Judge T, Whittaker S, 2012. How do designers and user experience professionals actually perceive and use personas? Proc Conf on Human Factors in Computing Systems, p.1219–1228. https://doi.org/10.1145/2207676.2208573

  89. McCaffrey T, Krishnamurty S, 2015. The obscure features hypothesis in design innovation. Int J Des Creat Innov, 3(1):1–28. https://doi.org/10.1080/21650349.2014.893840

    Google Scholar 

  90. McGinn J, Kotamraju N, 2008. Data-driven persona development. Proc Conf on Human Factors in Computing Systems, p.1521–1524. https://doi.org/10.1145/1357054.1357292

  91. Miaskiewicz T, Kozar KA, 2011. Personas and user-centered design: how can personas benefit product design processes? Des Stud, 32(5):417–430. https://doi.org/10.1016/j.destud.2011.03.003

    Google Scholar 

  92. Mikolov T, Chen K, Corrado G, et al., 2013. Efficient estimation of word representations in vector space. https://doi.org/1301.3781

  93. Miller GA, 1995. Wordnet: a lexical database for English. Commun ACM, 38(11):39–41. https://doi.org/10.1145/219717.219748

    Google Scholar 

  94. Mirza M, Osindero S, 2014. Conditional generative adversarial nets. https://doi.org/1411.1784

  95. Miyato T, Kataoka T, Koyama M, et al., 2018. Spectral normalization for generative adversarial networks. Int Conf on Learning Representations.

  96. Murray N, Marchesotti L, Perronnin F, 2012. AVA: a large-scale database for aesthetic visual analysis. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2408–2415. https://doi.org/10.1109/CVPR.2012.6247954

  97. Nazeri K, Ng E, Joseph T, et al., 2019. Edgeconnect: generative image inpainting with adversarial edge learning. https://doi.org/1901.00212

  98. Nelson BA, Wilson JO, Rosen D, et al., 2009. Refined metrics for measuring ideation effectiveness. Des Stud, 30(6):737–743. https://doi.org/10.1016/j.destud.2009.07.002

    Google Scholar 

  99. Nielsen L, Hansen KS, Stage J, et al., 2015. A template for design personas: analysis of 47 persona descriptions from Danish industries and organizations. Int J Sociotechnol Knowl Dev, 7(1):45–61. https://doi.org/10.4018/ijskd.2015010104

    Google Scholar 

  100. Niles I, Pease A, 2001. Towards a standard upper ontology. Proc Int Conf on Formal Ontology in Information Systems, p.2–9. https://doi.org/10.1145/505168.505170

  101. Nilsback ME, Zisserman A, 2008. Automated flower classification over a large number of classes. Proc 6th Indian Conf on Computer Vision, Graphics & Image Processing, p.722–729. https://doi.org/10.1109/ICVGIP.2008.47

  102. Odena A, Olah C, Shlens J, 2017. Conditional image synthesis with auxiliary classifier GANs. Proc 34th Int Conf on Machine Learning, p.4043–4055.

  103. Pan YH, 2017. Special issue on artificial intelligence 2.0. Front Inform Technol Electron Eng, 18(1):1–2. https://doi.org/10.1631/FITEE.1710000

    Google Scholar 

  104. Park T, Liu MY, Wang TC, et al., 2019. Semantic image synthesis with spatially-adaptive normalization. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2337–2346.

  105. Pathak D, Krähenbühl P, Donahue J, et al., 2016. Context encoders: feature learning by inpainting. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2536–2544. https://doi.org/10.1109/CVPR.2016.278

  106. Peeters JR, Verhaegen PA, Vandevenne D, et al., 2010. Refined metrics for measuring novelty in ideation. ID-MME Virtual Concept Research in Interaction Design, Article 4.

  107. Perera D, Zimmermann R, 2019. CNGAN: generative adversarial networks for cross-network user preference generation for non-overlapped users. World Wide Web Conf, p.3144–3150. https://doi.org/10.1145/3308558.3313733

  108. Pruitt J, Adlin T, 2005. The Persona Lifecycle: Keeping People in Mind Throughout Product Design. Elsevier, Amsterdam, p.724.

    Google Scholar 

  109. Radford A, Metz L, Chintala S, 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. Proc 4th Int Conf on Learning Representations.

  110. Reed SE, Akata Z, Yan XC, et al., 2016a. Generative adversarial text to image synthesis. Proc 33rd Int Conf on Machine Learning, p.1681–1690.

  111. Reed SE, Akata Z, Mohan S, et al., 2016b. Learning what and where to draw. Advances in Neural Information Processing Systems, p.217–225.

  112. Rigau J, Feixas M, Sbert M, 2008. Informational aesthetics measures. IEEE Comput Graph Appl, 28(2):24–34. https://doi.org/10.1109/MCG.2008.34

    Google Scholar 

  113. Russell SJ, Norvig P, 2016. Artificial Intelligence: a Modern Approach. Pearson Education Limited, Harlow, Essex.

    Google Scholar 

  114. Saleh B, Elgammal A, 2015. Large-scale classification of fine-art paintings: learning the right metric on the right feature. https://doi.org/1505.00855

  115. Salimans T, Goodfellow IJ, Zaremba W, et al., 2016. Improved techniques for training GANs. Advances in Neural Information Processing Systems, p.2226–2234.

  116. Salminen J, Sengün S, Kwak H, et al., 2017. Generating cultural personas from social data: a perspective of middle eastern users. Proc 5th Int Conf on Future Internet of Things and Cloud Workshops, p.120–125. https://doi.org/10.1109/FiCloudW.2017.97

  117. Salminen J, Jansen BJ, An J, et al., 2018a. Are personas done? Evaluating their usefulness in the age of digital analytics. Persona Stud, 4(2):47–65. https://doi.org/10.21153/psj2018vol4no2art737

    Google Scholar 

  118. Salminen J, Jung SG, An J, et al., 2018b. Findings of a user study of automatically generated personas. Proc Conf on Human Factors in Computing Systems, p.LBW097. https://doi.org/10.1145/3170427.3188470

  119. Salminen J, Engün S, Jung SG, et al., 2019. Design issues in automatically generated persona profiles: a qualitative analysis from 38 think-aloud transcripts. Proc Conf on Human Information Interaction and Retrieval, p.225–229. https://doi.org/10.1145/3295750.3298942

  120. Schwarz K, Wieschollek P, Lensch HPA, 2018. Will people like your image? Learning the aesthetic space. Proc IEEE Winter Conf on Applications of Computer Vision, p.2048–2057. https://doi.org/10.1109/WACV.2018.00226

  121. Shah JJ, Kulkarni SV, Vargas-Hernandez N, 2000. Evaluation of idea generation methods for conceptual design: effectiveness metrics and design of experiments. J Mech Des, 122(4):377–384. https://doi.org/10.1115/1.1315592

    Google Scholar 

  122. Simonyan K, Zisserman A, 2014. Very deep convolutional networks for large-scale image recognition. https://doi.org/1409.1556

  123. Strohmann T, Siemon D, Robra-Bissantz S, 2017. brAInstorm: intelligent assistance in group idea generation. Proc 12th Int Conf on Design Science Research in Information System and Technology, p.457–461. https://doi.org/10.1007/978-3-319-59144-5_31

    Google Scholar 

  124. Strothotte T, Schlechtweg S, 2002. Non-photorealistic Computer Graphics: Modeling, Rendering, and Animation. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

    Google Scholar 

  125. Tang X, Wang ZW, Luo WX, et al., 2018. Face aging with identity-preserved conditional generative adversarial networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.7939–7947. https://doi.org/10.1109/CVPR.2018.00828

  126. Tang XO, Luo W, Wang XG, 2013. Content-based photo quality assessment. IEEE Trans Multim, 15(8):1930–1943. https://doi.org/10.1109/TMM.2013.2269899

    Google Scholar 

  127. Vandevenne D, Verhaegen PA, Dewulf S, et al., 2015. A scalable approach for ideation in biologically inspired design. Artif Intell Eng Des Anal Manuf, 29(1):19–31. https://doi.org/10.1017/S0890060414000122

    Google Scholar 

  128. Varshney LR, Pinel F, Varshney KR, et al., 2019. A big data approach to computational creativity: the curious case of Chef Watson. IBM J Res Dev, 63(1):7:1–7:18. https://doi.org/10.1147/JRD.2019.2893905

    Google Scholar 

  129. Verma P, Smith JO, 2018. Neural style transfer for audio spectograms. https://doi.org/1801.01589

  130. Wang J, Yu LT, Zhang WN, et al., 2017. IRGAN: a minimax game for unifying generative and discriminative information retrieval models. Proc 40th Int ACM SI-GIR Conf on Research and Development in Information Retrieval, p.515–524. https://doi.org/10.1145/3077136.3080786

  131. Wang TC, Liu MY, Zhu JY, et al., 2018. Video-to-video synthesis. https://doi.org/1808.06601

  132. Wang WG, Shen JB, 2017. Deep cropping via attention box prediction and aesthetics assessment. Proc IEEE Int Conf on Computer Vision, p.2205–2213. https://doi.org/10.1109/ICCV.2017.240

  133. Wang WN, Cai D, Wang L, et al., 2016. Synthesized computational aesthetic evaluation of photos. Neurocomputing, 172:244–252. https://doi.org/10.1016/j.neucom.2014.12.106

    Google Scholar 

  134. Wang WS, Yang S, Zhang WS, et al., 2018. Neural aesthetic image reviewer. https://doi.org/1802.10240

  135. Wang XT, Yu K, Wu SX, et al., 2018. ESRGAN: enhanced super-resolution generative adversarial networks. European Conf on Computer Vision, p.63–79. https://doi.org/10.1007/978-3-030-11021-5_5

    Google Scholar 

  136. Wu JJ, Zhang CK, Xue TF, et al., 2016. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. Advances in Neural Information Processing Systems, p.82–90.

  137. Xu T, Zhang PC, Huang QY, et al., 2018. AttnGAN: fine-grained text to image generation with attentional generative adversarial networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1316–1324. https://doi.org/10.1109/CVPR.2018.00143

  138. Yan Y, Wang JR, Tang C, et al., 2019. Research on the development of contemporary design intelligence driven by neural network technology. In: Marcus A, Wang WT (Eds.), Design, User Experience, and Usability. Design Philosophy and Theory. Springer, Cham, p.368–381. https://doi.org/10.1007/978-3-030-23570-3_27

    Google Scholar 

  139. Yang HY, Huang D, Wang YH, et al., 2018. Learning face age progression: a pyramid architecture of GANs. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.31–39. https://doi.org/10.1109/CVPR.2018.00011

  140. Yang WM, Zhang XC, Tian YP, et al., 2019. Deep learning for single image super-resolution: a brief review. IEEE Trans Multim, 21(12):3106–3121. https://doi.org/10.1109/tmm.2019.2919431

    Google Scholar 

  141. Yang Y, Zhuang YT, Wu F, et al., 2008. Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Trans Multim, 10(3):437–446. https://doi.org/10.1109/TMM.2008.917359

    Google Scholar 

  142. Yi ZL, Zhang H, Tan P, et al., 2017. DualGAN: unsupervised dual learning for image-to-image translation. Proc IEEE Int Conf on Computer Vision, p.2868–2876. https://doi.org/10.1109/ICCV.2017.310

  143. Yoon Y, Jeon HG, Yoo D, et al., 2015. Learning a deep convolutional network for light-field image super-resolution. Proc IEEE Int Conf on Computer Vision, p.57–65. https://doi.org/10.1109/ICCVW.2015.17

  144. You S, You N, Pan MX, 2019. PI-REC: progressive image reconstruction network with edge and color domain. https://doi.org/1903.10146

  145. Yu F, Zhang YD, Song SR, et al., 2015. LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. https://doi.org/1506.03365

  146. Yu JH, Lin Z, Yang JM, et al., 2018a. Free-form image inpainting with gated convolution. https://doi.org/1806.03589

  147. Yu JH, Lin Z, Yang JM, et al., 2018b. Generative image inpainting with contextual attention. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.5505–5514. https://doi.org/10.1109/CVPR.2018.00577

  148. Zakharov E, Shysheya A, Burkov E, et al., 2019. Fewshot adversarial learning of realistic neural talking head models. https://doi.org/1905.08233

  149. Zeiler MD, Taylor GW, Fergus R, 2011. Adaptive deconvolutional networks for mid and high level feature learning. Proc IEEE Int Conf on Computer Vision, p.2018–2025. https://doi.org/10.1109/ICCV.2011.6126474

  150. Zhang H, Xu T, Li H, et al., 2017. StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. Proc IEEE Int Conf on Computer Vision, p.5907–5915.

  151. Zhang H, Xu T, Li H, et al., 2019. StackGAN++: realistic image synthesis with stacked generative adversarial networks. IEEE Trans Patt Anal Mach Intell, 41(8):1947–1962. https://doi.org/10.1109/TPAMI.2018.2856256

    Google Scholar 

  152. Zhang JJ, Yu JH, Zhang K, et al., 2017. Computational aesthetic evaluation of logos. ACM Trans Appl Perc, 14(3), Article 20. https://doi.org/10.1145/3058982

    Google Scholar 

  153. Zhang R, Isola P, Efros AA, 2016. Colorful image colorization. Proc 14th European Conf on Computer Vision, p.649–666. https://doi.org/10.1007/978-3-319-46487-9_40

    Google Scholar 

  154. Zhao H, Gallo O, Frosio I, et al., 2016. Loss functions for image restoration with neural networks. IEEE Trans Comput Imag, 3(1):47–57. https://doi.org/10.1109/tci.2016.2644865

    Google Scholar 

  155. Zhu JY, Park T, Isola P, et al., 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proc IEEE Int Conf on Computer Vision, p.2242–2251. https://doi.org/10.1109/ICCV.2017.244

Download references

Acknowledgements

Figs. 5c, 5d, and 8 in this study were generated by the pre-trained models of Runway toolkit (https://runwayml.com).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yong-chuan Tang.

Additional information

Compliance with ethics guidelines

Yong-chuan TANG, Jiang-jie HUANG, Meng-ting YAO, Jia WEI, Wei LI, Yong-xing HE, and Ze-jian LI declare that they have no conflict of interest.

Project supported by the National Science and Technology Innovation 2030 Major Project of the Ministry of Science and Technology of China (No. 2018AAA0100703), the National Natural Science Foundation of China (Nos. 61773336 and 91748127), the Chinese Academy of Engineering Consulting Project (No. 2018-ZD-12-06), the Provincial Key Research and Development Plan of Zhejiang Province, China (No. 2019C03137), and the Ng Teng Fong Charitable Foundation in the form of ZJU-SUTD IDEA Grant

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tang, Yc., Huang, Jj., Yao, Mt. et al. A review of design intelligence: progress, problems, and challenges. Front Inform Technol Electron Eng 20, 1595–1617 (2019). https://doi.org/10.1631/FITEE.1900398

Download citation

Key words

  • Design intelligence
  • Creativity
  • Personas
  • Ideation
  • AI-generated content
  • Computational aesthetics

CLC number

  • TP183