Key Crowdsourcing Technologies for Product Design and Development


Traditionally, small and medium enterprises (SMEs) in manufacturing rely heavily on a skilled, technical and professional workforce to increase productivity and remain globally competitive. Crowdsourcing offers an opportunity for SMEs to get access to online communities who may provide requested services such as generating design ideas or problem solutions. However, there are some barriers preventing them from adopting crowdsourcing into their product design and development (PDD) practice. In this paper, we provide a literature review of key crowdsourcing technologies including crowdsourcing platforms and tools, crowdsourcing frameworks, and techniques in terms of open call generation, rewarding, crowd qualification for working, organization structure of crowds, solution evaluation, workflow and quality control and indicate the challenges of integrating crowdsourcing with a PDD process. We also explore the necessary techniques and tools to support the crowdsourcing PDD process. Finally, we propose some key guidelines for coping with the aforementioned challenges in the crowdsourcing PDD process.


  1. [1]

    E. Simperl. How to use crowdsourcing effectively: Guidelines and examples. LIBER Quarterly, vol. 25, no. 1, pp. 18–39, 2015. DOI: 10.18352/lq.9948.

    Article  Google Scholar 

  2. [2]

    Open Innovation Defintion, [Online], Available:, September 20, 2017.

  3. [3]

    A. Wichman. Challenges, Crowdsourcing, Citizen Science: What’s the Dfi? [Online], Available:, September 20, 2017.

  4. [4]

    Citizen Science: Definition, [Online], Available:, September 15, 2017.

  5. [5]

    J. Howe. The rise of crowdsourcing. Wired magazine, vol. 14, no. 6, pp. 1–4, 2006.

    Google Scholar 

  6. [6]

    A. Kittur, J. V. Nickerson, M. Bernstein, E. Gerber, A. Shaw, J. Zimmerman, M. Lease, J. Horton. The future of crowd work. In Proceedings of the Conference on Computer Supported Cooperative Work, ACM, San Antonio, USA, pp. 1301–1318, 2013. DOI: 10.1145/2441776. 2441923.

    Google Scholar 

  7. [7]

    X. J. Niu, S. F. Qin. A review of crowdsourcing technology for product design and development. In Proceedings of the 23rd International Conference on Automation and Computing, IEEE, Huddersfield, UK, 2017. DOI: 10.23919/IConAC.2017.8081981.

    Google Scholar 

  8. [8]

    Open Innovation & Crowdsourcing Examples, [Online], Available:, September 25, 2017.

  9. [9]

    Cocreate with Your Fans. [Online], Available:, September 26, 2017.

  10. [10]

    J. Jin, P. Ji, Y. Liu. Prioritising engineering characteristics based on customer online reviews for quality function deployment. Journal of Engineering Design vol. 25, no. 7–9, pp. 303–324, 2014. DOI: 10.1080/09544828.2014. 984665.

    Article  Google Scholar 

  11. [11]

    S. N. Pedersen, M. E. Christensen, T. J. Howard. Robust design requirements specification: A quantitative method for requirements development using quality loss functions. Journal of Engineering Design, vol. 27, no. 8, pp. 544–567, 2016. DOI: 10.1080/09544828.2016.1183163.

    Article  Google Scholar 

  12. [12]

    Open IDEO, [Online], Available:, September 26, 2017.

  13. [13]

    Herox. Where Innovators Come to Compete and Businesses Come for Solutions, [Online], Available: September 28, 2017.

  14. [14], [Online], Available:, September 28, 2017.

  15. [15]

    E.A.T. School Lunch UX Challenge, [Online], Available: September 28, 2017.

  16. [16]

    H. To. Task assignment in spatial crowdsourcing: challenges and approaches. In Proceedings of the 3rd ACM Sigspatial Ph.D. Symposium, ACM, Burlingame, USA, pp. 1–4, 2016. DOI: 10.1145/3003819.3003820.

    Google Scholar 

  17. [17]

    R. R. Morris, D. McDuff. Crowdsourcing techniques for affective computing. The Oxford Handbook of Affective Computing, R. Calvo, S. D’Mello, J. Gratch, A. Kappas, Eds., Oxford, UK: Oxford University Press, pp. 384–394, 2014.

    Google Scholar 

  18. [18]

    J. Thebault-Spieker, L. G. Terveen, B. Hecht. Avoiding the south side and the suburbs: The geography of mobile crowdsourcing markets. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, Vancouver, Canada, pp. 265–275, 2015. DOI: 10.1145/2675133.2675278.

    Google Scholar 

  19. [19]

    H. To, R. Geraldes, C. Shahabi, S. H. Kim, H. Prendinger. An empirical study of workers’ behavior in spatial crowdsourcing. In Proceedings of the 3rd International ACM SIGMOD Workshop on Managing and Mining Enriched Geo-spatial Data, San Francisco, USA, 2016. DOI: 10.1145/2948649.2948657.

    Google Scholar 

  20. [20]

    S. Djelassi, I. Decoopman. Customers’ participation in product development through crowdsourcing: Issues and implications. Industrial Marketing Management, vol. 42, no. 5, pp. 683–692, 2013. DOI: 10.1016/j.indmarman.2013.05.006.

    Article  Google Scholar 

  21. [21]

    H. Carpenter. Motivating the crowd to participate in your innovation initiative. A Guide to Open Innovation and Crowdsourcing, pp. 76–84, 2011.

    Google Scholar 

  22. [22]

    H. Simula, T. Ahola. A network perspective on idea and innovation crowdsourcing in industrial firms. Industrial Marketing Management, vol. 43, no. 3, pp. 400–408, 2014. DOI: 10.1016/j.indmarman.2013.12.008.

    Article  Google Scholar 

  23. [23]

    W. Mason, D. J. Watts. Financial incentives and the “performance of crowds”. ACM Special Interest Group on Knowledge Discovery and Data Mining Explorations Newsletter, vol. 11, no. 2, pp. 100–108, 2009. DOI: 10.1145/1809400.1809422.

    Google Scholar 

  24. [24]

    J. Rogstadius, V. Kostakos, A. Kittur, B. Smus, J. Laredo, M. Vukovic. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. In Proceedings of the 5th International AAAI Conference on Web and Social Media, Barcelona, Spain, pp. 17–21, 2011.

    Google Scholar 

  25. [25]

    J. Howe. Crowdsousrcing: Why the Power of the Crowd is Driving the Future of Business. New York, USA: Crown Business, 2009.

    Google Scholar 

  26. [26]

    L. Von Ahn, L. Dabbish. Designing games with a purpose. Communications of the ACM, vol. 51, no. 8, pp. 58–67, 2008. DOI: 10.1145/1378704.1378719.

    Article  Google Scholar 

  27. [27]

    B. Jonathan Mausam, D. S. Weld. Optimal testing for crowd workers. In Proceedings of the International Conference on Autonomous Agents & Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, Singapore, pp. 966–974, 2016.

    Google Scholar 

  28. [28]

    D. N. Chang, C. H. Chen. Product concept evaluation and selection using data mining and domain ontology in a crowdsourcing environment. Advanced Engineering Informatics, vol. 29, no. 4, pp. 759–774, 2015. DOI: 10.1016/j.aei.2015.06.003.

    Article  Google Scholar 

  29. [29]

    P. G. Ipeirotis, F. Provost, J. Wang. Quality management on amazon mechanical Turk. In Proceedings of the ACM SIGKDD Workshop on Human Computation, Washington DC, USA, pp. 64–67, 2010. DOI: 10.1145/1837885.1837906.

    Google Scholar 

  30. [30]

    S. Sedhain, S. Sanner, D. Braziunas, L. X. Xie, J. Christensen. Social collaborative filtering for cold-start recommendations. In Proceedings of the 8th ACM Conference on Recommender Systems, Foster City, USA, pp. 345–348, 2014. DOI: 10.1145/2645710.2645772.

    Google Scholar 

  31. [31]

    N. Holladay. Achieving quality through teamwork, [Online], Available:, August 15, 2017.

  32. [32]

    J. Pedersen, D. Kocsis, A. Tripathi, A. Tarrell, A. Weerakoon, N. Tahmasbi, J. Xiong, W. Deng, O. Oh, G. J. de Vreede. Conceptual foundations of crowdsourcing: A review of IS research. In Proceedings of the 46th Hawaii International Conference on System Sciences, IEEE, Wailea, USA, pp. 579–588, 2013. DOI: 10.1109/HICSS.2013.143.

    Google Scholar 

  33. [33]

    Quirky, [Online], Available:, September 26, 2017.

  34. [34]

    99designs, [Online], Available:, September 26, 2017.

  35. [35]

    Jovoto, [Online], Available: September 26, 2017.

  36. [36]

    T. P. Walter, A. Back. A text mining approach to evaluate submissions to crowdsourcing contests. In Proceedings of the 46th Hawaii International Conference on System Sciences, IEEE, Wailea, USA, pp. 3109–3118, 2013. DOI: 10.1109/HICSS.2013.64.

    Google Scholar 

  37. [37]

    T. P. Walter, A. Back. Towards Measuring Crowdsourcing Success: An Empirical Study on Effects of External Factors in Online Idea Contest, [Online], Available:, September 26, 2017.

  38. [38]

    J. Bjork, M. Magnusson. Where do good innovation ideas come from? Exploring the influence of network connectivity on innovation idea quality Journal of Product Innovation Management, vol. 26, no. 6, pp. 662–670, 2009. DOI: 10.1111/j.1540-5885.2009.00691.x.

    Google Scholar 

  39. [39]

    G. Wang, X. M. Liu, W. G. Fan. A knowledge adoption model based framework for finding helpful user-generated contents in online communities. In Proceedings of the 32nd International Conference on Information Systems, Shanghai, China, pp. 1–11, 2011.

    Google Scholar 

  40. [40]

    K. Aniket, E. H. Chi, B. Suh. Crowdsourcing user studies with Mechanical Turk. In Proceedings of the Special Interest Group on Computer-Human Interaction Conference on Human Factors in Computing Systems, ACM, Florence, Italy, pp. 453–456, 2008. DOI: 10.1145/1357054. 1357127.

    Google Scholar 

  41. [41]

    A. D. Shaw, J. J. Horton, D. L. Chen. Designing incentives for inexpert human raters. In Proceedings of the ACM Conference on Computer Supported Cooperative Work, ACM, Hangzhou, China, pp. 275–284, 2011. DOI: 10.1145/1958824.1958865.

    Google Scholar 

  42. [42]

    A. B. Xu, H. M. Rao, S. P. Dow, B. P. Bailey. A classroom study of using crowd feedback in the iterative design process. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, ACM, Vancouver, Canada, pp. 1637–1648, 2015. DOI: 10.1145/2675133.2675140.

    Google Scholar 

  43. [43]

    P. Dai, J. M. Rzeszotarski, P. Paritosh, E. H. Chi. And now for something completely different: Improving crowdsourcing workflows with micro-diversions. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, ACM, Vancouver, Canada, pp. 628–638, 2015. DOI: 10.1145/2675133. 2675260.

    Google Scholar 

  44. [44]

    P. Dai, Mausam, D. S. Weld. Decision-theoretic control of crowd-sourced workflows. In Proceedings of the 24th AAAI Conference on Artificial Intelligence, AAAI, Atlanta, USA, pp. 1168–1174, 2010.

    Google Scholar 

  45. [45]

    A. Mohammad, B. Benatallah, A. Ignjatovic, H. R. Motahari-Nezhad, E. Bertino, S. Dustdar. Quality control in crowdsourcing systems: Issues and directions. IEEE Internet Computing, vol. 17, no. 2, pp. 76–81, 2013. DOI: 10.1109/MIC.2013.20.

    Article  Google Scholar 

  46. [46]

    M. S. Bernstein, G. Little, R. Miller, B. Hartmann, M. S. Ackerman, D. R. Karger, D. Crowell, D. Crowell. Soylent: A word processor with a crowd inside. Communications of the ACM, vol. 58, no. 8, pp. 85–94, 2015. DOI: 10. 1145/2791285.

    Article  Google Scholar 

  47. [47]

    D. Ofer, O. Shamir. Vox populi: Collecting high-quality labels from a crowd. In Proceedings of the 22nd Annual Conference on Learning Theory, Montreal, Canada, 2009.

    Google Scholar 

  48. [48]

    What Are the Best Tools for Crowdsourcing Ideas? [Online], Available:, February 15, 2018.

  49. [49]

    Crowdsourcing Tool, [Online], Available:, February 15, 2018.

  50. [50]

    S. F. Qin, D. van der Velde, E. Chatzakis, T. McStea, N. Smith. Exploring barriers and opportunities in adopting crowdsourcing based new product development in manufacturing SMEs. Chinese Journal of Mechanical Engineering, vol. 29, no. 6, pp. 1052–1066, 2016. DOI: 10.3901/CJME.2016.0808.089.

    Article  Google Scholar 

  51. [51]

    C. Y. Xu, S. F. Qin, Z. P. Xiao. Crowdsourcing based product innovation design service model for small-sized and medium-sized enterprises. In Proceedings of 18th International Conference on Automation and Computing, IEEE, Leicestershire, UK, pp. 1–5, 2012.

    Google Scholar 

  52. [52]

    Product Design, [Online], Available:, October 20, 2017.

  53. [53]

    K. Ulrich, S. Eppinger. Product Design and Development. 6th ed. New York, USA: McGraw-Hill Higher Education, 2015.

    Google Scholar 

  54. [54]

    J. L. Nevins, D. E. Whitney. Concurrent Design of Products and Processes: A Strategy for the Next Generation in Manufacturing. New York, USA: McGraw-Hill Companies, 1989.

    Google Scholar 

  55. [55]

    Y. L. Tu, S. Q. Xie, R. Y. K. Fung. Product development cost estimation in mass customization. IEEE Transactions on Engineering Management, vol. 54, no. 1, pp. 29–40, 2007. DOI: 10.1109/TEM.2006.889065.

    Article  Google Scholar 

  56. [56]

    H. P. Shang, Z. X. Zhao. Integration of manufacturing services into virtual environments over the Internet. International Journal of Automation and Computing, vol. 1, no. 1, pp. 89–106, 2004. DOI: 10.1007/s11633-004-0089-3.

    Article  Google Scholar 

  57. [57]

    M. L. Gray, S. Suri, S. S. Ali, D. Kulkarni. The crowd is a collaborative network. In Proceedings of the 19th ACM Conference on Computer-supported Cooperative Work & Social Computing, ACM, San Francisco, USA, pp. 134–147, 2016. DOI: 10.1115/DETC2002/DTM-34020.

    Google Scholar 

  58. [58]

    W. Y. Song, X. G. Ming, Z. Y. Wu. An integrated rough number-based approach to design concept evaluation under subjective environments. Journal of Engineering Design, vol. 24, no. 5, pp. 320–341, 2013. DOI: 10.1080/ 09544828.2012.732994.

    Article  Google Scholar 

  59. [59]

    Z. Ayag, R. G. Ozdem. An analytic network process-based approach to concept evaluation in a new product development environment. Journal of Engineering Design, vol. 18, no. 3, pp. 209–226, 2007. DOI: 10.1080/09544820600752740.

    Article  Google Scholar 

  60. [60]

    E. Alex, N. Rahmati, N. Zhu. Structured handoffs in expert crowdsourcing improve communication and work output. In Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology, ACM, Honolulu, USA, pp. 99–100, 2014.

    Google Scholar 

  61. [61]

    H. P. Zhang, R. Q. Zhang, Y. P. Zhao, B. J. Ma. Big data modeling and analysis of microblog ecosystem. International Journal of Automation and Computing, vol. 11, no. 2, pp. 119–127, 2014. DOI: 10.1007/s11633-014-0774-9.

    Article  Google Scholar 

  62. [62]

    L. Y. Zhai, L. P. Khoo, Z. W. Zhong. Design concept evaluation in product development using rough sets and grey relation analysis. Expert Systems with Applications, vol. 36, no. 3, pp. 7072–7079, 2009. DOI: 10.1016/j.eswa. 2008.08.068.

    Article  Google Scholar 

  63. [63]

    V. Girotto. Collective creativity through a micro-tasks crowdsourcing approach. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion, ACM, San Francisco, USA, pp. 143–146, 2016. DOI: 10.1145/2818052.2874356.

    Google Scholar 

  64. [64]

    Z. Ayag. An integrated approach to concept evaluation in a new product development. Journal of Intelligent Manufacturing, vol. 27, no. 5, pp. 991–1005, 2016. DOI: 10.1007/ s10845-014-0930-7.

    Article  Google Scholar 

  65. [65]

    V. Tiwari, P. K. Jain, P. Tandon. Product design concept evaluation using rough sets and VIKOR method. Advanced Engineering Informatics, vol. 30, no. 1, pp. 16–25, 2016. DOI: 10.1016/j.aei.2015.11.005.

    Article  Google Scholar 

  66. [66]

    Z. Ayag. An integrated approach to evaluating conceptual design alternatives in a new product development environment. International Journal of Production Research, vol. 43, no. 4, pp. 687–713, 2005. DOI: 10.1080/00207540512 331311831.

    Article  Google Scholar 

  67. [67]

    H. Z. Huang, R. F. Bo, W. Chen. An integrated computational intelligence approach to product concept generation and evaluation. Mechanism and Machine Theory, vol. 41, no. 5, pp. 567–583, 2006. DOI: 10.1016/j.mechmachtheory. 2005.07.006.

    MATH  Article  Google Scholar 

  68. [68]

    T. L. Saaty. Decision Making with Dependence and Feedback: The Analytic Network Process. Pittsburgh, USA: RWS publications, 1996.

    Google Scholar 

  69. [69]

    D. N. Chang, C. H. Chen. Exploration of a concept screening method in a crowdsourcing environment. Moving Integrated Product Development to Service Clouds in the Global Economy, J. Z. Cha, S. Y. Chou, J. Stjepandic, R. Curran, W. S. Xu, Eds, Netherlands: IOS Press, pp. 861–870, 2014. DOI: 10.3233/978-1-61499-440-4-861.

    Google Scholar 

  70. [70]

    W. Qi, Z. W. Ren, Z. F. Guo. XML-based data processing in network supported collaborative design. International Journal of Automation and Computing, vol. 7, no. 3, pp. 330–335, 2010. DOI: 10.1007/s11633-010-0511-y.

    Article  Google Scholar 

  71. [71]

    K. Gautam. Advanced Methods of Teaching “Feedback”. Pondicherry University, [Online], Available:, November 5, 2017.

  72. [72]

    P. L. Jackson. Getting Design Right: A Systems Approach. Boca Raton, USA: CRC Press, 2009.

    Google Scholar 

  73. [73]

    Z. Y. Ma, A. X. Sun, Q. Yuan, G. Cong. Topic-driven reader comments summarization. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, Maui, USA, pp. 265–274, 2012. DOI: 10.1145/2396761.2396798.

    Google Scholar 

  74. [74]

    J. Hui, A. Glenn, R. Jue, E. Gerber, S. Dow. Using anonymity and communal efforts to improve quality of crowdsourced feedback. In Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing, AAAI, San Diego, USA, pp. 72–82. 2015.

    Google Scholar 

  75. [75]

    A. B. Xu, B. Bailey. What do you think? A case study of benefit, expectation, and interaction in a large online critique community. In Proceedings of the ACM Conference on Computer Supported Cooperative Work, Seattle, USA, pp. 295–304, 2012. DOI: 10.1145/2145204.2145252.

    Google Scholar 

  76. [76]

    S. Dow, E. Gerber, A. Wong. A pilot study of using crowds in the classroom. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Paris, France, pp. 227–236, 2013. DOI: 10.1145/2470654. 2470686.

    Google Scholar 

  77. [77]

    A. B. Xu, S. W. Huang, B. Bailey. Voyant: Generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, ACM, Baltimore, USA, pp. 1433–1444, 2014. DOI: 10.1145/2531602.2531604.

    Google Scholar 

  78. [78]

    K. Luther, A. Pavel, W. Wu, J. L. Tolentino, M. Agrawala, B. Hartmann, S. P. Dow. CrowdCrit: Crowdsourcing and aggregating visual design critique. In Proceedings of the Companion Publication of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, Baltimore, USA, pp. 21–24, 2014. DOI: 10.1145/2556420.2556788.

    Google Scholar 

  79. [79]

    S. Salimun, N. Janom, N. H. Arshad. Quality factors of crowdsourcing system: Paper review. In Proceedings of the 6th IEEE Control and System Graduate Research Colloquium, Shah Alam, Malaysia, pp. 82–86, 2015. DOI: 10.1109/ICSGRC.2015.7412469.

    Google Scholar 

  80. [80]

    R. Khazankin, D. Schall, S. Dustdar. Predicting QoS in scheduled crowdsourcing. Advanced Information Systems Engineering, J. Ralyte, X. Franch, S. Brinkkemper, S. Wrycza, Eds., Berlin Heidelberg, Germang: Springer, 2012. DOI: 10.1007/978-3-642-31095-9—30.

    Google Scholar 

  81. [81]

    R. Alabduljabbar, H. Al-Dossari. A task ontology-based model for quality control in crowdsourcing systems. In Proceedings of the International Conference on Research in Adaptive and Convergent Systems, ACM, Odense, Denmark, pp. 22–28, 2016. DOI: 10.1145/2987386.2987413.

    Google Scholar 

Download references


This work was supported by the China Scholarship Council and State Key Laboratory of Traction Power at Southwest Jiaotong University (No. TPL1501). We thank anonymous reviewers for their helpful comments which helped to improve the paper.

Author information



Corresponding author

Correspondence to Xiao-Jing Niu.

Additional information

Recommended by Associate Editor Jie Zhang

Xiao-Jing Niu received the B. Eng. and M. Eng. degrees in computer science from Northwest A&F University, China in 2013 and 2016, respectively. She is currently a Ph. D. degree candidate in industrial design of Department of Northumbria School of Design, Northumbria University Newcastle, UK.

Her research interests include human-computer interaction, computer aided design and machine learning.

Sheng-Feng Qin received the Ph. D. degree in product design from University of Wales Institute, UK in 2000. Now, he is a professor of digital design at Northumbria University with an extensive career in design academia. He is a member of IEEE and the Design Society. He has published more than 150 papers in journals and conferences and 2 books.

His research interests include computer-aided conceptual design, sketch-based interface and modeling, interface and interaction design, simulation modelling and virtual manufacturing, smart product and sustainable design, digital design methods and tools.

John Vines received the Ph. D. degree in interaction design for older people from University of Plymouth, UK in 2010. He is a professor at Department of Northumbria School of Design, Northumbria University, UK.

His research interests include human-computer interaction, methods and processes for participatory, collaborative and experience-centered design and research, experience-centered security and privacy.

Rose Wong received the BA degree in product design from Northumbria University, UK in 2000. She has worked as a senior lecturer on the Bachelor (Hons) 3D design and Bachelor (Hons) design for industry courses at Northumbria University, UK, and is currently the programme leader on Bachelor (Hons) design for industry course. Prior to this, she worked as a “Designer in Residence” at Northumbria University, UK, which operates as an in-house design consultancy employing successful students graduating from 3D Design. This venture has helped many of Northumbria University′s Design School graduates become successful freelance designers, demonstrating a nurture of entrepreneurial skills in our alumni.

Her research interests include in-house design and 3D design.

Hui Lu received the B. Sc. degree in business administration from Zhejiang University, China in 2009, and the M. Sc. degree in operations and supply chain management from the University of Liverpool, UK in 2010. Currently, she is an associated lecturer, research assistant and Ph. D. degree candidate in business management, Newcastle Business School, Northumbria University, UK. She once worked in the Integrated Supply Chain Department in IBM China for 4 years.

Her research interests include sustainable manufacturing and supply chain management, process optimal and control, and technology innovation.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Niu, X., Qin, S., Vines, J. et al. Key Crowdsourcing Technologies for Product Design and Development. Int. J. Autom. Comput. 16, 1–15 (2019).

Download citation


  • Crowdsourcing technologies
  • product design and development (PDD)
  • communication
  • information sharing
  • design evaluation
  • feedback