Effect of Crowd Composition on the Wisdom of Artificial Crowds Metaheuristic

  • Christopher J. LowranceEmail author
  • Dominic M. Larkin
  • Sang M. Yim
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11346)


This paper investigates the impact that task difficulty and crowd composition have on the success of the Wisdom of Artificial Crowds metaheuristic. The metaheuristic, which is inspired by the wisdom of crowds phenomenon, combines the intelligence from a group of optimization searches to form a new solution. Unfortunately, the aggregate formed by the metaheuristic is not always better than the best individual solution within the crowd, and little is known about the variables which maximize the metaheuristic’s success. Our study offers new insights into the influential factors of artificial crowds and the collective intelligence of multiple optimization searches performed on the same problem. The results show that favoring the opinions of experts (i.e., the better searches) improves the chances of the metaheuristic succeeding by more than 15% when compared to the traditional means of equal weighting. Furthermore, weighting expertise was found to require smaller crowd sizes for the metaheuristic to reach its peak chances of success. Finally, crowd size was discovered to be a critical factor, especially as problem complexity grows or average crowd expertise declines. However, crowd size matters only up to a point, after which the probability of success plateaus.


Wisdom of crowds Combinatorial optimization Collective intelligence Metaheuristic optimization Crowd factors 


  1. 1.
    Ashby, L.H., Yampolskiy, R.V.: Genetic algorithm and wisdom of artificial crowds algorithm applied to light up. In: International Conference on Computer Games 2011, pp. 27–32 (2011)Google Scholar
  2. 2.
    Chen, J., Ren, Y., Riedl, J.: The effects of diversity on group productivity and member withdrawal in online volunteer groups. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 821–830 (2010)Google Scholar
  3. 3.
    Hughes, R., Yampolskiy, R.V.: Solving sudoku puzzles with wisdom of artificial crowds. Int. J. Intell. Games Simul. 7(1), 24–29 (2012)Google Scholar
  4. 4.
    Hundley, M., Yampolskiy, R.V.: Shortest total path length spanning tree via wisdom of artificial crowds algorithm. In: Proceedings of the 28th Modern Artificial Intelligence and Cognitive Science Conference (2017)Google Scholar
  5. 5.
    Kantardzic, M.: Data Mining: Concepts, Models, Methods, and Algorithms. Wiley, Hoboken (2011)CrossRefGoogle Scholar
  6. 6.
    Khalifa, A.B., Yampolskiy, R.V.: Ga with wisdom of artificial crowds for solving mastermind satisfiability problem. Int. J. Intell. Games Simul. 6(2), 12–17 (2011)Google Scholar
  7. 7.
    Kittur, A., Kraut, R.E.: Harnessing the wisdom of crowds in wikipedia: quality through coordination. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, pp. 37–46 (2008)Google Scholar
  8. 8.
    Lowrance, C.J., Abdelwahab, O., Yampolskiy, R.V.: Evolution of a metaheuristic for aggregating wisdom from artificial crowds. In: Pereira, F., Machado, P., Costa, E., Cardoso, A. (eds.) EPIA 2015. LNCS (LNAI), vol. 9273, pp. 238–249. Springer, Cham (2015). Scholar
  9. 9.
    Mäntylä, M.V., Itkonen, J.: The effect of crowd size and time restriction in software testing. Inf. Softw. Technol. 55(6), 986–1003 (2013)CrossRefGoogle Scholar
  10. 10.
    Moore, T., Clayton, R.: Evaluating the wisdom of crowds in assessing phishing websites. In: Tsudik, G. (ed.) FC 2008. LNCS, vol. 5143, pp. 16–30. Springer, Heidelberg (2008). Scholar
  11. 11.
    Port, A.C., Yampolskiy, R.V.: Using a GA and wisdom of artificial crowds to solve solitaire battleship puzzles. In: International Conference on Computer Games, pp. 25–29 (2012)Google Scholar
  12. 12.
    Robert, L., Romero, D.M.: Crowd size, diversity and performance. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1379–1382 (2015)Google Scholar
  13. 13.
    Surowiecki, J.: The Wisdom of Crowds. Random House LLC, New York City (2005)Google Scholar
  14. 14.
    Trainor, P.J., Yampolskiy, R.V., DeFilippis, A.P.: Wisdom of artificial crowds feature selection in untargeted metabolomics: an application to the development of a blood-based diagnostic test for thrombotic myocardial infarction. J. Biomed. Inform. 81, 53–60 (2017)CrossRefGoogle Scholar
  15. 15.
    Velic, M., Grzinic, T., Padavic, I.: Wisdom of crowds algorithm for stock market predictions. In: Proceedings of the International Conference on Information Technology Interfaces, pp. 137–144 (2013)Google Scholar
  16. 16.
    von der Gracht, H.A., Hommel, U., Prokesch, T., Wohlenberg, H.: Testing weighting approaches for forecasting in a group wisdom support system environment. J. Bus. Res. 69(10), 4081–4094 (2016)CrossRefGoogle Scholar
  17. 17.
    Wagner, C., Suh, A.: The wisdom of crowds - impact of collective size and expertise transfer on collective performance. In: 47th Hawaii International Conference on System Sciences, pp. 594–603 (2014)Google Scholar
  18. 18.
    Welsh, M.: Expertise and the wisdom of crowds - whose judgments to trust and when. In: 34th Annual Meeting of the Cognitive Science Society, pp. 594–603 (2012)Google Scholar
  19. 19.
    Yampolskiy, R.V., Ashby, L., Hassan, L.: Wisdom of artificial crowds—a metaheuristic algorithm for optimization. J. Intell. Learn. Syst. Appl. 4(02), 98–107 (2012)Google Scholar
  20. 20.
    Yampolskiy, R.V., El-Barkouky, A.: Wisdom of artificial crowds algorithm for solving NP-hard problems. Int. J. Bio-inspired Comput. 3(6), 358–369 (2011)CrossRefGoogle Scholar
  21. 21.
    Concorde TSP Solver. Accessed 10 July 2018

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Electrical Engineering and Computer ScienceUnited States Military AcademyWest PointUSA

Personalised recommendations