Skip to main content
SpringerLink
Log in

Crowd Intelligence in Requirements Engineering: Current Status and Future Directions

  • Conference paper
  • First Online:
Requirements Engineering: Foundation for Software Quality (REFSQ 2019)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11412))

Abstract

Software systems are the joint creative products of multiple stakeholders, including both designers and users, based on their perception, knowledge and personal preferences of the application context. The rapid rise in the use of Internet, mobile and social media applications make it even more possible to provide channels to link a large pool of highly diversified and physically distributed designers and end users, the crowd. Converging the knowledge of designers and end users in requirements engineering process is essential for the success of software systems. In this paper, we report the findings of a survey of the literature on crowd-based requirements engineering research. It helps us understand the current research achievements, the areas of concentration, and how requirements related activities can be enhanced by crowd intelligence. Based on the survey, we propose a general research map and suggest the possible future roles of crowd intelligence in requirements engineering.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adepetu, A., Ahmed, K.A., Abd, Y.A., Zaabi, A.A., Svetinovic, D.: CrowdREquire: a requirements engineering crowdsourcing platform. In: Proceedings of the AAAI Spring Symposium: Wisdom of the Crowd (2012)

    Google Scholar 

  2. Ali, R., Solis, C., Nuseibeh, B., Maalej, W.: Social sensing: when users become monitors. In: 19th ACM SIGSOFT Symposium and the 13th Conference on ESEC/FSE (2011)

    Google Scholar 

  3. Almaliki, M., Ncube, C., Ali, R.: Adaptive software-based feedback acquisition: a persona-based design. In. Proceedings of the IEEE 9th International Conference on RCIS 2015, pp. 100–111 (2015)

    Google Scholar 

  4. Bakiu, E., Guzman, E.: Which feature is unusable? Detecting usability and user experience issues from user reviews. In: RE Workshops, pp. 182–187 (2017)

    Google Scholar 

  5. Bano, M., Zowghi, D.: Crowd vigilante. In: Kamalrudin, M., Ahmad, S., Ikram, N. (eds.) APRES 2017. CCIS, vol. 809, pp. 114–120. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-7796-8_9

    Google Scholar 

  6. Bozzon, A., Brambilla, M., Ceri, S., Silvestri, M., Vesci, G.: Choosing the right crowd: expert finding in social networks. In: EDBT, pp. 637–648 (2013)

    Google Scholar 

  7. Brabham, D.C.: Crowdsourcing as a model for problem solving: an introduction and cases. Converg.: Int. J. Res. New Media Technol. 14(1), 75–90 (2008)

    Google Scholar 

  8. Breaux, T.D., Schaub, T.D.: Scaling requirements extraction to the crowd: experiments with privacy policies. In. Preceding of IEEE Requirement Engineering Conference (RE 2014), pp. 163–172 (2014)

    Google Scholar 

  9. Burnay, C., Horkoff, J., Maiden, N.: Stimulating stakeholders’ imagination: new creativity triggers for eliciting novel requirements. In: Proceedings of IEEE RE Conference (RE 2016), pp. 36–45 (2016)

    Google Scholar 

  10. Byren, E.: Internal crowdsourcing for innovation development. Chalmers University of Technology, Sweden (2013)

    Google Scholar 

  11. Clifton, J.: State of the American WorkPlace (2013)

    Google Scholar 

  12. Dabrowski, J., Kifetew, F.M., Muñante, D., Letier, E., Siena, A., Susi, A.: Discovering requirements through goal-driven process mining. In: RE 2017 Workshops (2017)

    Google Scholar 

  13. Dalpiaz, F., Korenko, M., Salay, R., Chechik, M.: Using the crowds to satisfy unbounded requirements. In: Proceedings of CrowdRE@RE 2015, pp. 19–24 (2015)

    Google Scholar 

  14. Dheepa, V., Aravindhar, D.J., Vijayalakshmi, C.: A novel method for large scale requirement elicitation. Int. J. Eng. Innov. Technol. 2, 375–379 (2013)

    Google Scholar 

  15. Do, A.Q., Bhowmik, T.: Refinement and resolution of just-in-time requirements in open source software: a case study. In: RE Workshops 2017, pp. 407–410 (2017)

    Google Scholar 

  16. Ferrari, A., Donati, B., Gnesi, S.: Detecting domain-specific ambiguities: an NLP approach based on Wikipedia crawling and word embeddings. In: Proceedings of RE Workshops 2017, pp. 393–399 (2017)

    Google Scholar 

  17. Fricker, S.A., Wallmüller, E., Paschen, I.: Requirements engineering as innovation journalism: a research preview. In: Proceedings of RE 2016, pp. 335–340 (2016)

    Google Scholar 

  18. Fricker, S.A.: Systematic mapping of technology-enabled product innovations. In: RE 2016 Workshops (2016)

    Google Scholar 

  19. Greenwood, P., Rashid, A., Walkerdine, J.: UdesignIt: towards social media for community-driven design. In. Proceedings of the 34th International Conference on SE, pp. 1321–1324

    Google Scholar 

  20. Groen, E.C., Doerr, J., Adam, S.: Towards crowd-based requirements engineering: a research preview. In: Fricker, S.A., Schneider, K. (eds.) REFSQ 2015. LNCS, vol. 9013, pp. 247–253. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16101-3_16

    Google Scholar 

  21. Groen, E.C., et al.: The crowd in requirement engineering - the landscape and challenges. IEEE Softw. 34(2), 44–52 (2017)

    Google Scholar 

  22. Groen, E.C.: Crowd out the competition gaining market advantage through crowd-based requirements engineering. In: IEEE Ist Workshop on CrowdRE, pp. 13–18 (2015)

    Google Scholar 

  23. Groen, E.C., Kopczynska, S., Hauer, M.P., Krafft, T.D., Dörr, J.: Users - the hidden software product quality experts?: A study on how app users report quality aspects in online reviews. In: RE 2017 Conference (2017)

    Google Scholar 

  24. Guzman, E., Alkadhi, R., Seyff, N.: A needle in a haystack: what do Twitter users say about software?. In: Proceedings of RE 2016 International Conference, pp. 96–105 (2016)

    Google Scholar 

  25. Guzman, E., Ibrahim, M., Glinz, M.: A little bird told me: mining tweets for requirements and software evolution. In: Proceedings of RE 2017, pp. 11–20 (2017)

    Google Scholar 

  26. Hamidi, S., Andritsos, P., Liaskos, S.: Constructing adaptive configuration dialogs using crowd data. In: ASE (2014)

    Google Scholar 

  27. Hu, W., Jiau, H.C.: UCFrame: a use case framework for crowd-centric requirement acquisition. ACM SIGSOFT Softw. Eng. Notes 41(2), 1–13 (2016)

    Google Scholar 

  28. Hosseini, M., Shahri, A., Phalp, K., Taylor, J., Ali, R., Dalpiaz, F.: Configuring crowdsourcing for requirements elicitation. In: RCIS, pp. 133–138 (2015)

    Google Scholar 

  29. Hosseini, M., Phalp, K., Taylor, J. Ali, R.: Towards crowdsourcing for requirement engineering. In. 20th International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ), Springer, Heidelberg (2014)

    Google Scholar 

  30. Hosseini, M., Phalp, K., Taylor, J., Ali, R.: The four pillars of crowdsourcing: a reference model. In: IEEE 8th RCIS (2014)

    Google Scholar 

  31. Hosseini, M., Groen, E.C., Shahri, A., Ali, R.: CRAFT: a crowed-annotated feedback technique. In. IEEE 25th International RE Conference Workshops, pp. 170–175 (2017)

    Google Scholar 

  32. Johann, T., Maalej, W.: Democratic mass participation of users in requirements engineering? In: Proceedings of the 23rd IEEE RE Conference, pp. 256–261 (2015)

    Google Scholar 

  33. Johann, T., Stanik C., Alireza, M., Alizadeh, B., Maalej, W.: SAFE: a simple approach for feature extraction from app descriptions and app reviews. In: RE 2017 (2017)

    Google Scholar 

  34. Kanchev, G.M., Murukannaiah, P.K., Chopra, A.K., Sawyer, P.: Canary: an interactive and query-based approach to extract requirements from online forums. In: RE 2017 (2017)

    Google Scholar 

  35. Kanchev, G.M., Chopra, A.K.: Social media through the requirements lens: a case study of Google maps. In: CrowdRE@RE 2015, pp. 7–12 (2015)

    Google Scholar 

  36. Kifetew, F.M., et al.: Gamifying collaborative prioritization: does pointsification work. In: RE 2017, pp. 322–331 (2017)

    Google Scholar 

  37. Kurtanovic, Z., Maalej, W.: Mining user rationale from software reviews. In: RE 2017 (2017)

    Google Scholar 

  38. Lakhani, K.R., Garvin, D.A., Lonstein, E.: TopCoder (A): developing software through crowdsourcing. Harvard Business School (2010)

    Google Scholar 

  39. Levy, M., Hadar, I., Teeni, D.: A gradual approach to crowd-base requirements engineering: the case of conference online social networks. In: IEEE 2nd CrowdRE, pp. 26–30 (2017)

    Google Scholar 

  40. Li, W., et al.: Crowd intelligence in AI 2.0 era. Front. IT EE 18(1), 15–43 (2017)

    Google Scholar 

  41. Li, C., Huang, L., Luo, J.G.B., Ng, V.: Automatically classifying user requests in crowdsourcing requirements engineering. JSS 138, 108–123 (2018)

    Google Scholar 

  42. Liang, W., Qian, W., Wu, Y., Peng, X., Zhao, W.: Mining context-aware user requirements from crowd contributed mobile data. In: Internetware, pp. 132–140 (2015)

    Google Scholar 

  43. Lim, S.L., Finkelstein, A.: StakeRare: using social networks and collaborative filtering for large-scale requirements elicitation. IEEE Trans. SE 38, 707–735 (2012)

    Google Scholar 

  44. Linåker, J., Wnuk, K.: Requirements analysis and management for benefiting openness. In: RE Workshops 2016 (2016)

    Google Scholar 

  45. Liu, L., et al.: Requirements cybernetics: elicitation based on user behavioral data. JSS 124, 187–194 (2017)

    Google Scholar 

  46. Lutz, R., Schäfer, S., Diehl, S.: Using mobile devices for collaborative requirements engineering. In: ASE 2012 (2012)

    Google Scholar 

  47. Maalej, W., Nayebi, M., Johann, T., Ruhe, G.: Toward data-driven requirements engineering. IEEE Softw. 33, 48–54 (2016)

    Google Scholar 

  48. Maalej, W., Happel, H.-J., Rashid, A.: When users become collaborators: towards continuous and context-aware user input. In: Proceedings of the 24th ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA 2009) (2009)

    Google Scholar 

  49. Maalej, W., Nabil, H.: Bug report, feature request, or simply praise? On automatically classifying app reviews. In: Proceedings of the 23rd RE Conference (RE 2015), pp. 116–125 (2015)

    Google Scholar 

  50. Mao, K., Capra, L., Harman, M., Jia, Y.: A survey of the use of crowdsourcing in software engineering. J. Syst. Softw. 126, 57–84 (2016)

    Google Scholar 

  51. Mead, N., Shull, F., Spears, J., Heibl, S., Weber, S., Cleland-Huang, J.: Crowd sourcing the creation of personae non gratae for requirements-phase threat modeling. In: RE 2017, pp. 412–417 (2017)

    Google Scholar 

  52. Merten, T., Falis, M., Hübner, P., Quirchmayr, T., Bürsner, S., Paech, B.: Software feature request detection in issue tracking systems. In: Proceedings of RE 2016, pp. 166–175 (2016)

    Google Scholar 

  53. Misra, J., Sengupta, S., Podder, S.: Topic cohesion preserving requirements clustering. In: RAISE@ICSE 2016 (2016)

    Google Scholar 

  54. Moketar, N.A., Kamalrudin, M., Sidek, S., Robinson, M., Grundy, J.C.: An automated collaborative requirements engineering tool for better validation of requirements. In: ASE 2016, pp. 864–869 (2016)

    Google Scholar 

  55. Munante, D., Siena, A., Kifetew, F.M., Susi, A., Stade, M., Seyff, N.: Gathering requirements for software configuration from the crowd. In: RE Workshops (2017)

    Google Scholar 

  56. Murukannaiah, P.K., Ajmeri, N., Singh, M.P.: Towards automating Crowd RE. In: Proceedings of IEEE 25th International RE Conference Workshops, pp. 512–515 (2017)

    Google Scholar 

  57. Murukannaiah, P.K., Ajmeri, N., Singh, M.P.: Acquiring creative requirements from the crowd: understanding the influences of personality and creative potential in crowd RE. In: RE 2016, pp. 176–185 (2016)

    Google Scholar 

  58. Nascimento, P., Aguas, R., Schneider, D.S., Souza, J.M.: An approach to requirements categorization using Kano’s model and crowds. In: CSCWD 2012, pp. 387–392 (2012)

    Google Scholar 

  59. Niu, N., Koshoffer, A., Newman, L., Khatwani, C., Samarasinghe, C., Savolainen, J.: Advancing repeated research in requirements engineering: a theoretical replication of viewpoint merging. In: RE 2016, pp. 186–195 (2016)

    Google Scholar 

  60. Nuseibeh, B., Easterbrook, S.: Requirements engineering: a roadmap. In: FOSE 2000 (2000)

    Google Scholar 

  61. Piras, L., Giorgini, P., Mylopoulos, J.: Acceptance requirements and their gamification solutions. In: RE 2016 (2016)

    Google Scholar 

  62. Qi, J., Zhang, Z., Jeon, S., Zhou, Y.: Mining customer requirements from online reviews: a product improvement perspective. Inf. Manag. 53(8), 951–963 (2016)

    Google Scholar 

  63. Regnell, B., Brinkkemper, S.: Market-driven requirements engineering for software products. In: Aurum, A., Wohlin, C. (eds.) Engineering and Managing Software Requirements, pp. 287–308. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-28244-0_13

    Google Scholar 

  64. Rooijen, L., Bäumer, F.S., Platenius, M.C., Geierhos, M., Hamann, H., Engels, G.: From user demand to software service: using machine learning to automate the requirements specification process. In: RE Workshops, March 2017

    Google Scholar 

  65. Saito, S., Iimura, Y., Massey, A.K., Antón, A.I.: How much undocumented knowledge is there in agile software development?: case study on industrial project using issue tracking system and version control system. In: RE 2017, pp. 194–203 (2017)

    Google Scholar 

  66. Salay, R., Dalpiaz, F., Chechik, M.: Integrating crowd intelligence into software. In: CSI-SE 2015, pp. 1–7 (2015)

    Google Scholar 

  67. Schneider, K., Karras, O., Finger, A., Zibell, B.: Reframing societal discourse as requirements negotiation: vision statement. In: RE Workshops 2017, pp. 188–193 (2017)

    Google Scholar 

  68. Sherief, N., Abdelmoez, W., Phalp, K., Ali, R.: Modelling users feedback in crowd-based requirements engineering: an empirical study. In: Ralyté, J., España, S., Pastor, Ó. (eds.) PoEM 2015. LNBIP, vol. 235, pp. 174–190. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-25897-3_12

    Google Scholar 

  69. Shi, L., Chen, C., Wang, Q., Boehm, B.M.: Is it a new feature or simply “Don’t Know Yet”?: on automated redundant OSS feature requests identification. In: RE 2016, pp. 377–382 (2016)

    Google Scholar 

  70. Snijders, R., Dalpiaz, F., Hosseini, M., Shahri, A., Ali, R.: Crowd-centric requirement engineering. In. IEEE/ACM 7th International Conference on Utility and Cloud Computing (2014)

    Google Scholar 

  71. Snijders, R., Dalpiaz, F., Hosseini, M., Ali, R., Ozum, A.: REfine: a gamified platform for participatory requirement engineering. In: 23rd IEEE RE Conference, pp. 1–6 (2015)

    Google Scholar 

  72. Srivastava, P.K., Sharma, R.: Crowdsourcing to elicit requirements for MyERP application. In: IEEE Ist International Workshop on CrowdRE, pp. 31–35 (2015)

    Google Scholar 

  73. Stade, M., et al.: Providing a user forum is not enough: first experiences of a software company with CrowdRE. In: IEEE 2nd International Workshop on CrowdRE, pp. 164–169 (2017)

    Google Scholar 

  74. Stol, K., Fitzgerald, B.: Two’s company, three’s a crowd: a case study of crowdsourcing software development. In: Proceedings of 36th ICSE (2014)

    Google Scholar 

  75. Sutcliffe, A., Sawyer, P.: Beyond awareness requirements. In: RE 2016, pp. 383–388 (2016)

    Google Scholar 

  76. Surowiecki, J.: Why the Many are Smarter than the Few and How Collective Wisdom Shapes Business, Economics, Society and Nations. Brown, Little, New York (2004)

    Google Scholar 

  77. Todoran, I., Seyff, N., Glinz, M.: How cloud providers elicit consumer requirements: an exploratory study of nineteen companies. In: 21st IEEE RE Conference (2013)

    Google Scholar 

  78. Wikipedia: Crowdsourcing (2017). https://en.wikipedia.org/wiki/Crowdsourcing. Accessed 28 Nov 2017

  79. Williams, G., Mahmoud, A.: Mining Twitter feeds for software user requirements. In: RE 2017, pp. 1–10 (2017)

    Google Scholar 

  80. Xiao, S., Wei, C., Dong, M.: Crowd intelligence: analyzing online product reviews for preference measurement. Inf. Manag. 53(2), 169–182 (2016)

    Google Scholar 

  81. Xie, H., et al.: A statistical analysis approach to predict user’s changing requirements for software service evolution. J. Syst. Softw. 132, 147–164 (2017)

    Google Scholar 

  82. Zhang, W., Mei, H.: Software development based on collective intelligence on the internet: feasibility, state-of-the-practise, and challenges. SCIENTIA SINICA Informationis (2017). (in Chinese)

    Google Scholar 

  83. Muganda, N., Asmelash, D., Samali, M.: Groupthink decision making deficiency in the requirements engineering process: towards a crowdsourcing model (2012)

    Google Scholar 

  84. Wang, H., Wang, Y., Wang, J.: A participant recruitment framework for crowdsourcing based software requirement acquisition. In: Conference on Global Software Engineering (2014)

    Google Scholar 

  85. Seyff, N., et al.: Crowd-focused semi-automated requirements engineering for evolution towards sustainability. In: Proceedings of 26th RE@Next Conference (RE 2018) (2018)

    Google Scholar 

  86. Sarro, F., Harmna, M., Jia, Y., Zhang, Y.: Customer rating reactions can be predicted purely using app features. In: Proceedings of 26 IEEE RE Conference (RE 2018) (2018)

    Google Scholar 

  87. Williams, G., Mahmoud, A.: Modeling user concerns in the app store: a case study on the rise and fall of Yik Yak. In: Proceedings of 26 IEEE RE Conference (RE 2018) (2018)

    Google Scholar 

  88. Dhinakaran, V.T., Pulle, R., Ajmeri, N., Murukannaiah, K.P.: App review analysis via active learning: reducing supervision effort without compromising classification accuracy. In: RE 2018 (2018)

    Google Scholar 

  89. Gemkow, T., Conzelmann, M., Hartig, K., Volesang, A.: Automatically glossary term extraction form large-scale requirements specifications. In: 26th IEEE RE Conference (2018)

    Google Scholar 

  90. Khan, J.A., Lin, L., Jia, Y., W, L.: Linguistic analysis of crowd requirements: an experimental study. In: Proceedings of 27th IEEE RE Workshop (Empri 2018), pp. 24–31 (2018)

    Google Scholar 

  91. Liu, X., Leng, Y., Yang, W., Zhai, C., Xie, T.: Mining android app descriptions for permission requirements recommendation. In: 26th IEEE RE Conference (2018)

    Google Scholar 

  92. Hosseini, M., Moore, J., Almaliki, M., Shahri, A., Phalp, K.T., Ali, R.: Wisdom of the crowd within enterprises: practices and challenges. Comput. Netw. 90, 121–132 (2015)

    Google Scholar 

  93. Oriol, M., et al.: FAME: supporting continuous requirements elicitation by combining user feedback and monitoring. In: Proceedings of 26th RE Conference (RE 2018) (2018)

    Google Scholar 

  94. Surowiecki, J.: The Wisdom of Crowds. Anchor Books, New York (2005)

    Google Scholar 

  95. Shahri, A., Hosseini, M., Almaliki, M., Phalp, K., Taylor, J., Ali, R.: Engineering software-based motivation: a persona-based approach. In: The 10th IEEE Conference RCIS (2016)

    Google Scholar 

  96. Almaliki, M., Ali, R.: Persuasive and culture-aware feedback acquisition. In: Meschtscherjakov, A., De Ruyter, B., Fuchsberger, V., Murer, M., Tscheligi, M. (eds.) PERSUASIVE 2016. LNCS, vol. 9638, pp. 27–38. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31510-2_3

    Google Scholar 

  97. Ryan, R.M., Deci, E.L.: Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55(1), 68 (2000)

    Google Scholar 

Download references

Acknowledgment

Financial support from the Natural Science Foundation of China Project no. 61432020 is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Tsinghua University, Beijing, 100084, China

    Javed Ali Khan, Lin Liu & Lijie Wen

  2. Bournemouth University, Poole, BH12 5BB, UK

    Raian Ali

Authors
  1. Javed Ali Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lijie Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raian Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Liu .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden

    Eric Knauss

  2. paluno – The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany

    Michael Goedicke

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khan, J.A., Liu, L., Wen, L., Ali, R. (2019). Crowd Intelligence in Requirements Engineering: Current Status and Future Directions. In: Knauss, E., Goedicke, M. (eds) Requirements Engineering: Foundation for Software Quality. REFSQ 2019. Lecture Notes in Computer Science(), vol 11412. Springer, Cham. https://doi.org/10.1007/978-3-030-15538-4_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-15538-4_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-15537-7

  • Online ISBN: 978-3-030-15538-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation