Crowdsourcing intelligent design

  • Wei Xiang
  • Ling-yun Sun
  • Wei-tao You
  • Chang-yuan Yang


Design intelligence, namely, artificial intelligence to solve creative problems and produce creative ideas, has improved rapidly with the new generation artificial intelligence. However, existing methods are more skillful in learning from data and have limitations in creating original ideas different from the training data. Crowdsourcing offers a promising method to produce creative designs by combining human inspiration and machines’ computational ability. We propose a crowdsourcing intelligent design method called ‘flexible crowdsourcing design’. Design ideas produced through crowdsourcing design can be unreliable and inconsistent because they rely solely on selection among participants’ submissions of ideas. In contrast, the flexible crowdsourcing design method employs a cultivation procedure that integrates the ideas from crowd participants and cultivates these ideas to improve design quality at the same time. We introduce a series of studies to show how flexible crowdsourcing design can produce original design ideas consistently. Specifically, we will describe the typical procedure of flexible crowdsourcing design, the refined crowdsourcing tasks, the factors that affect the idea development process, the method for calculating idea development potential, and two applications of the flexible crowdsourcing design method. Finally, it summarizes the design capabilities enabled by crowdsourcing intelligent design. This method enhances the performance of crowdsourcing design and supports the development of design intelligence.


Crowdsourcing Flexible crowdsourcing design Design intelligence 

CLC number



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ball LJ, Ormerod TC, 1995. Structured and opportunistic processing in design: a critical discussion. Int J Hum-Comput Stud, 43(1):131–151. Scholar
  2. Chan J, Dang S, Dow SP, 2016. Improving crowd innovation with expert facilitation. Proc 19th ACM Conf on Computer-Supported Cooperative Work & Social Computing, p.1223–1235. Scholar
  3. Chang DN, Chen CH, Lee KM, 2014. A crowdsourcing development approach based on a neuro-fuzzy network for creating innovative product concepts. Neurocomputing, 142:60–72. Scholar
  4. Cross N, 2006. Designerly Ways of Knowing. Springer, London. Scholar
  5. Dontcheva M, Morris RR, Brandt JR, et al., 2014. Combining crowdsourcing and learning to improve engagement and performance. Proc SIGCHI Conf on Human Factors in Computing Systems, p.3379–3388. Scholar
  6. Flores RL, Belaud JP, le Lann JM, et al., 2015. Using the collective intelligence for inventive problem solving: a contribution for open computer aided innovation. Expert Syst Appl, 42(23):9340–9352. Scholar
  7. Gatys LA, Ecker AS, Bethge M, 2016. Image style transfer using convolutional neural networks. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2414–2423. Scholar
  8. Glickman ME, 1999. Parameter estimation in large dynamic paired comparison experiments. J Roy Stat Soc Ser C, 48(3):377–394. Scholar
  9. Goldschmidt G, 2015. Ubiquitous serendipity: potential visual design stimuli are everywhere. In: Gero JS (Ed.), Studying Visual and Spatial Reasoning for Design Creativity. Springer Dordrecht Netherlands, p.205–214. Scholar
  10. Ikeda K, Morishima A, Rahman H, et al., 2016. Collaborative crowdsourcing with crowd4U. Proc VLDB Endowm, 9(13):1497–1500. Scholar
  11. Kim J, Dontcheva M, Li W, et al., 2015. Motif: supporting novice creativity through expert patterns. Proc 33rd Annual ACM Conf on Human Factors in Computing Systems, p.1211–1220. Scholar
  12. Lafreniere B, Grossman T, Anderson F, et al., 2016. Crowdsourced fabrication. Proc 29th Annual Symp on User Interface Software and Technology, p.15–28. Scholar
  13. Li W, Wu WJ, Wang HM, et al., 2017. Crowd intelligence in AI 2.0 era. Front Inform Technol Electron Eng, 18(1): 15–43. Scholar
  14. Michelucci P, Dickinson JL, 2016. The power of crowds. Science, 351(6268):32–33. Scholar
  15. O’Donovan P, Agarwala A, Hertzmann A, 2014. Learning layouts for single-pagegraphic designs. IEEE Trans Vis Comput Graph, 20(8):1200–1213. Scholar
  16. Pan YH, 2017. Special issue on artificial intelligence 2.0. Front Inform Technol Electron Eng, 18(1):1–2. Scholar
  17. Park CH, Son KH, Lee JH, et al., 2013. Crowd vs. crowd: large-scale cooperative design through open team competition. Proc Conf on Computer Supported Cooperative Work, p.1275–1284. Scholar
  18. Pauwels P, de Meyer R, van Campenhout J, 2013. Design thinking support: information systems versus reasoning. Des Iss, 29(2):42–59. Scholar
  19. Pinel F, Varshney LR, Bhattacharjya D, 2015. A culinary computational creativity system. In: Besold TR, Schorlemmer M, Smaill A (Eds.), Computational Creativity Research: Towards Creative Machines. Springer, Paris, p.327–346. Scholar
  20. Prats M, Earl CF, 2006. Exploration through drawings in the conceptual stage of product design. In: Gero JS (Ed.), Design Computing and Cognition. Springer Dordrecht Netherlands, p.83–102. Scholar
  21. Ren J, Nickerson JV, Mason W, et al., 2014. Increasing the crowd’s capacity to create: how alternative generation affects the diversity, relevance and effectiveness of generated ads. Dec Supp Syst, 65:28–39. Scholar
  22. Schneider OS, Seifi H, Kashani S, et al., 2016. HapTurk: crowdsourcing affective ratings of vibrotactile icons. Proc CHI Conf on Human Factors in Computing Systems, p.3248–3260. Scholar
  23. Sun LY, Xiang W, Chai CL, et al., 2014a. Creative segment: a descriptive theory applied to computer-aided sketching. Des Stud, 35(1):54–79. Scholar
  24. Sun LY, Xiang W, Chai CL, et al., 2014b. Designers’ perception during sketching: an examination of creative segment theory using eye movements. Des Stud, 35(6): 593–613. Scholar
  25. Sun LY, Xiang W, Chen S, et al., 2015. Collaborative sketching in crowdsourcing design: a new method for idea generation. Int J Technol Des Educat, 25(3):409–427. Scholar
  26. Suzuki R, Salehi N, Lam MS, et al., 2016. Atelier: repurposing expert crowdsourcing tasks as micro-internships. Proc CHI Conf on Human Factors in Computing Systems, p.2645–2656. Scholar
  27. van der Maaten L, Weinberger K, 2012. Stochastic triplet embedding. IEEE Int Workshop on Machine Learning for Signal Processing, p.1–6. Scholar
  28. Wah C, van Horn G, Branson S, et al., 2014. Similarity comparisons for interactive fine-grained categorization. IEEE Conf on Computer Vision and Pattern Recognition, p.859–866. Scholar
  29. Warby SC, Wendt SL, Welinder P, et al., 2014. Sleep-spindle detection: crowdsourcing and evaluating performance of experts, non-experts and automated methods. Nat Methods, 11(4):385–392. Scholar
  30. Wiltschnig S, Christensen BT, Ball LJ, 2013. Collaborative problem–solution co-evolution in creative design. Des Stud, 34(5):515–542. Scholar
  31. Xiang W, Sun LY, Xia SC, et al., 2017. An evolutionary computation method of crowdsourcing ideation that integrates the balanced-exploration pattern. J Mech Eng, 53(15):73–80 (in Chinese). Scholar
  32. Xu AB, Rao HM, Dow SP, et al., 2015. A classroom study of using crowd feedback in the iterative design process. Proc 18th ACM Conf on Computer Supported Cooperative Work & Social Computing, p.1637–1648. Scholar
  33. Yu LX, Nickerson JV, 2011. Cooks or cobblers?: crowd creativity through combination. Proc SIGCHI Conf on Human Factors in Computing Systems, p.1393–1402. Scholar
  34. Yu LX, Kittur A, Kraut RE, 2014. Distributed analogical idea generation: inventing with crowds. Proc SIGCHI Conf on Human Factors in Computing Systems, p.1245–1254. Scholar
  35. Yu LX, Kraut RE, Kittur A, 2016. Distributed analogical idea generation with multiple constraints. Proc 19th ACM Conf on Computer-Supported Cooperative Work & Social Computing, p.1236–1245. Scholar
  36. Zhao Q, Huang ZH, Harper FM, et al., 2016. Precision crowdsourcing: closing the loop to turn information consumers into information contributors. Proc 19th ACM Conf on Computer-Supported Cooperative Work & Social Computing, p.1615–1625. Scholar
  37. Zhu JY, Krähenbühl P, Shechtman E, et al., 2016. Generative visual manipulation on the natural image manifold. Proc 14th European Conf on Computer Vision, p.597–613. Scholar

Copyright information

© Zhejiang University and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.State Key Lab of CAD & CGZhejiang UniversityHangzhouChina
  2. 2.Modern Industrial Design InstituteZhejiang UniversityHangzhouChina
  3. 3.International User Experience Business Unit, Alibaba GroupHangzhouChina

Personalised recommendations