Skip to main content
SpringerLink
Log in

Quantifying effectiveness of team recommendation for collaborative software development

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

It is undeniable that software development is a team-based activity. The quality of the delivered product highly depends on the team configuration. However, selecting an appropriate team to complete a software task is non-trivial, as it needs to consider team compatibility in multiple aspects. While extensive literature introduced multiple team recommendation algorithms, such algorithms are not designed to support the specific roles in software teams. This paper proposes a novel set of metrics for measuring five dimensions of a software team’s effectiveness, including historical collaboration, team cohesiveness, teammate interaction, team members’ expertise, and role experience. Furthermore, Wining Experience-based Software Team RECommendation (WESTREC) is introduced to solve the software team recommendation problem. WESTREC considers multiple aspects of team characteristics, including historical collaboration, team cohesiveness, teammate interaction, project description, team members’ expertise, and role experience. Specifically, given a software project, a machine learning based team scoring function is used along with the Max-Logit algorithm to approximate and recommend suitable software team configurations for the given task. We validate the effectiveness of the WESTREC on real-world software development datasets (i.e., Atlassian and Apache). Furthermore, we study the factors that affect the performance of collaborative software development and propose a method to evaluate the effectiveness of a software team. The results show that WESTREC outperforms state-of-the-art baseline approaches in three out of five groups of team effectiveness metrics associated with different team characteristics in large software systems. Our research findings not only illustrate the efficacy of automatic software team evaluation using machine learning techniques but also serve as building blocks for potential applications that involve automatic team formation and evaluation, such as automatic recommendation of research collaborators and grouping personnel for team-based projects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://www.apache.org/.

  2. https://www.atlassian.com/.

  3. https://www.atlassian.com/software/jira.

  4. https://issues.apache.org/jira.

  5. https://jira.atlassian.com/secure.

  6. http://sentistrength.wlv.ac.uk/.

References

  • Akbar, M.A., Sang, J., Khan, A.A., Mahmood, S., Qadri, S.F., Hu, H., Xiang, H.: Success factors influencing requirements change management process in global software development. J. Comput. Lang. 51, 112–130 (2019)

    Article  Google Scholar 

  • Alberola, J.M., Del Val, E., Sanchez-Anguix, V., Palomares, A., Teruel, M.D.: An artificial intelligence tool for heterogeneous team formation in the classroom. Knowl.-Based Syst. (2016). https://doi.org/10.1016/j.knosys.2016.02.010

    Article  Google Scholar 

  • Alsharo, M., Gregg, D., Ramirez, R.: Virtual team effectiveness: the role of knowledge sharing and trust. Inf. Manag. 54, 11 (2016). https://doi.org/10.1016/j.im.2016.10.005

    Article  Google Scholar 

  • Assavakamhaenghan, N., Choetkiertikul, M., Tuarob, S., Kula, R.G., Hata, H., Ragkhitwetsagul, C., Sunetnanta, T., Matsumoto K.: Software team member configurations: a study of team effectiveness in moodle. In: Proceedings of the 10th International Workshop on Empirical Software Engineering in Practice (IWESEP), pp. 19–195 (2019). https://doi.org/10.1109/IWESEP49350.2019.00012

  • Beaver, J., Schiavone, G.: The effects of development team skill on software product quality. ACM SIGSOFT Softw. Eng. Notes 31, 1–5 (2006). https://doi.org/10.1145/1127878.1127882

    Article  Google Scholar 

  • Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3(Jan), 993–1022 (2003)

    MATH  Google Scholar 

  • Chen, R., Liang, C., Gu, D., Leung, J.Y.: A multi-objective model for multi-project scheduling and multi-skilled staff assignment for it product development considering competency evolution. Int. J. Prod. Res. 55(21), 6207–6234 (2017)

    Article  Google Scholar 

  • Chipulu, M., Ojiako, U., Gardiner, P., Williams, T., Mota, C., Maguire, S., Shou, Y., Stamati, T., Marshall, A.: Exploring the impact of cultural values on project performance—the effects of cultural values, age and gender on the perceived importance of project success/failure factors. Int. J. Oper. Prod. Manag. 34, 364–389 (2014). https://doi.org/10.1108/IJOPM-04-2012-0156

    Article  Google Scholar 

  • Choetkiertikul, M., Dam, H.K., Tran, T., Ghose, A.: Predicting the delay of issues with due dates in software projects. Empir. Softw. Eng. 22(3), 1223–1263 (2017). https://doi.org/10.1007/s10664-016-9496-7

    Article  Google Scholar 

  • Chow, T., Cao, D.-B.: A survey study of critical success factors in agile software projects. J. Syst. Softw. 81(6), 961–971 (2008). https://doi.org/10.1016/j.jss.2007.08.020

    Article  Google Scholar 

  • Colazo, J.: Collaboration structure and performance in new software development: findings from the study of open source projects. Int. J. Innov. Manag. 14, 735–758 (2010). https://doi.org/10.1142/S1363919610002866

    Article  Google Scholar 

  • Datta, A., Tan Teck Yong, J., Ventresque, A.: T-recs: team recommendation system through expertise and cohesiveness. In: Proceedings of the 20th International Conference Companion on World Wide Web, WWW ’11, New York, NY, USA, pp. 201–204. ACM. ISBN 978-1-4503-0637-9 (2011a). https://doi.org/10.1145/1963192.1963289

  • Datta, A., Yong, J., Ventresque, A.: T-recs: team recommendation system through expertise and cohesiveness, pp. 201–204 (2011b). https://doi.org/10.1145/1963192.1963289

  • Dingsøyr, T., Dybå, T.: Team effectiveness in software development: human and cooperative aspects in team effectiveness models and priorities for future studies (2012). https://doi.org/10.1109/CHASE.2012.6223016

  • Dingsøyr, T., Fægri, T., Dybå, T., Haugset, B., Lindsjørn, Y.: Team performance in software development: research results versus agile principles. IEEE Softw. 33, 106–110 (2016). https://doi.org/10.1109/MS.2016.100

    Article  Google Scholar 

  • Dubinsky, Y., Hazzan, O.: Roles in agile software development teams, pp. 157–165 (2004). https://doi.org/10.1007/978-3-540-24853-8_18

  • Fagerholm, F., Ikonen, M., Kettunen, P., Münch, J., Roto, V., Abrahamsson, P.: How do software developers experience team performance in Lean and Agile environments? In: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, pp. 1–10. ISBN 9781450324762 (2014). https://doi.org/10.1145/2601248.2601285

  • Faraj, S., Sproull, L.: Coordinating expertise in software development teams. Manag. Sci. 46, 1554–1568 (2000). https://doi.org/10.1287/mnsc.46.12.1554.12072

    Article  Google Scholar 

  • Foster, E.C.: Human resource management. In: Software Engineering, pp. 253–269. Springer (2014)

  • Franzago, M., Di Ruscio, D., Malavolta, I., Muccini, H.: Collaborative model-driven software engineering: a classification framework and a research map. IEEE Trans. Softw. Eng. 1–1, 09 (2017). https://doi.org/10.1109/TSE.2017.2755039

    Article  Google Scholar 

  • Grigore, M., Rosenkranz, C.: Increasing the willingness to collaborate online: an analysis of sentiment-driven interactions in peer content production. In: Galletta, D.F., Liang, T. (eds.) Proceedings of the International Conference on Information Systems, ICIS 2011, Shanghai, China, December 4–7, 2011. Association for Information Systems (2011)

  • Guha, R., Kumar, R., Raghavan, P., Tomkins, A.: Propagation of trust and distrust. In: Proceedings of the 13th International Conference on World Wide Web, pp. 403–412 (2004)

  • Günsel, A., Açikgšz, A., Tükel, A., Öğüt, E.T.: The role of flexibility on software development performance: an empirical study on software development teams. Procedia Soc. Behav. Sci. 58, 853–860 (2012). https://doi.org/10.1016/j.sbspro.2012.09.1063

    Article  Google Scholar 

  • Han, W.-M., Huang, S.-J.: An empirical analysis of risk components and performance on software projects. J. Syst. Softw. 80(1), 42–50 (2007). https://doi.org/10.1016/j.jss.2006.04.030

    Article  Google Scholar 

  • Huckman, R., Staats, B., Upton, D.: Team familiarity, role experience, and performance: evidence from Indian software services. Manag. Sci. 55, 85–100 (2009). https://doi.org/10.1109/EMR.2012.6172773

    Article  Google Scholar 

  • Hupa, A., Rzadca, K., Wierzbicki, A., Datta, A.: Interdisciplinary matchmaking: choosing collaborators by skill, acquaintance and trust, pp. 319–347 (2010)

  • iDalko: A guide to Jira workflow best practices (2018). https://www.idalko.com/jira-workflow-best-practices/

  • Jiang, J., Klein, G.: Software development risks to project effectiveness. J. Syst. Softw. 52, 3–10 (2000). https://doi.org/10.1016/S0164-1212(99)00128-4

    Article  Google Scholar 

  • Jiang, J.J., Klein, G., Means, T.L.: Project risk impact on software development team performance. Proj. Manag. J. 31(4), 19–26 (2000). https://doi.org/10.1177/875697280003100404

    Article  Google Scholar 

  • Kale, A.: Modeling trust and influence in blogosphere using link polarity. Master’s thesis, April (2007)

  • Khan, A.A., Basri, S., Dominc, P.: A proposed framework for communication risks during RCM in GSD. Procedia—Social and Behavioral Sciences 129, 496–503 (2014). In: 2nd International Conference on Innovation, Management and Technology Research

  • Khan, A.A., Keung, J., Hussain, S., Niazi, M., Tamimy, M.M.I.: Understanding software process improvement in global software development: a theoretical framework of human factors. SIGAPP Appl. Comput. Rev. 17(2), 5–15 (2017)

    Article  Google Scholar 

  • Khan, A.A., Keung, J., Niazi, M., Hussain, S., Ahmad, A.: Systematic literature review and empirical investigation of barriers to process improvement in global software development: client–vendor perspective. Inf. Softw. Technol. 87, 180–205 (2017)

    Article  Google Scholar 

  • Lappas, T., Liu, K., Terzi, E.: Finding a team of experts in social networks, pp. 467–476 (2009). https://doi.org/10.1145/1557019.1557074

  • Lindsjørn, Y., Sjøberg, D.I., Dingsøyr, T., Bergersen, G.R., Dybå, T.: Teamwork quality and project success in software development: a survey of agile development teams. J. Syst. Softw. 122, 274–286 (2016). https://doi.org/10.1016/j.jss.2016.09.028

    Article  Google Scholar 

  • Liu, H., Qiao, M., Greenia, D., Akkiraju, R., Dill, S., Nakamura, T., Song, Y., Motahari Nezhad, H.R.: A machine learning approach to combining individual strength and team features for team recommendation (2014). https://doi.org/10.13140/2.1.4558.4966

  • Maalej, W., Ellmann, M., Robbes, R.: Using contexts similarity to predict relationships between tasks. J. Syst. Softw. 128, 267–284 (2017). https://doi.org/10.1016/j.jss.2016.11.033

    Article  Google Scholar 

  • Monderer, D., Shapley, L.: Potential games. Games Econ. Behav. 14, 124–143 (1996). https://doi.org/10.1006/game.1996.0044

    Article  MathSciNet  MATH  Google Scholar 

  • Mudrack, P.: Defining group cohesiveness: a legacy of confusion? Small Group Res 20, 37–49 (1989). https://doi.org/10.1177/104649648902000103

    Article  Google Scholar 

  • Naguib, H., Narayan, N., Brugge, B., Helal, D..: Bug report assignee recommendation using activity profiles. In: Proceeding of the 10th Working Conference on Mining Software Repositories (MSR), pp. 22–30. IEEE, May 2013. ISBN 978-1-4673-2936-1. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6623999 (2013). https://doi.org/10.1109/MSR.2013.6623999

  • Niazi, M., Babar, M.A., Verner, J.M.: Software process improvement barriers: A cross-cultural comparison. Inf. Softw. Technol., 52 (11):1204–1216 (2010). Special Section on Best Papers PROMISE 2009

  • Niazi, M., Mahmood, S., Alshayeb, M., Qureshi, A.M., Faisal, K., Cerpa, N.: Toward successful project management in global software development. Int. J. Proj. Manag. 34(8), 1553–1567 (2016)

    Article  Google Scholar 

  • Oliver Bossert,J. L., Kretzberg, Alena.: Agile compendium, chapter 1.3, p 30. McKinsey Quarterly, 10 (2018)

  • Qiao, W., Yan, Z., Wang, X.: Join or not: The impact of physicians’ group joining behavior on their online demand and reputation in online health communities. Inf. Process. Manag. 58(5), 102634 (2021)

    Article  Google Scholar 

  • Rahman, M.M., Roy, C.K., Redl, J., Collins, J.A.: Correct: code reviewer recommendation at github for vendasta technologies. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, New York, NY, USA, pp. 792–797. Association for Computing Machinery. ISBN 9781450338455 (2016). https://doi.org/10.1145/2970276.2970283. URL

  • Rebai, S., Amich, A., Molaei, S., Kessentini, M., Kazman, R.: Multi-objective code reviewer recommendations: balancing expertise, availability and collaborations. Autom. Softw. Eng. 27(3), 301–328 (2020). https://doi.org/10.1007/s10515-020-00275-6

    Article  Google Scholar 

  • Sokolov, E.: On software development product management: feature selection and model analysis for predicting Jira issue attributes (2017)

  • Sommerville, I.: Software Engineering, 9th edn. Pearson Education, London (2011)

    MATH  Google Scholar 

  • Song, Y., Wong, S., Lee, K.-W.: Optimal gateway selection in multi-domain wireless networks: a potential game perspective, pp. 325–336 (2011). https://doi.org/10.1145/2030613.2030650

  • Storey, M., Zagalsky, A., Filho, F.F., Singer, L., German, D.M.: How social and communication channels shape and challenge a participatory culture in software development. IEEE Transa. Softw. Eng. 43(2), 185–204 (2017)

    Article  Google Scholar 

  • Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  • The Standish Group: Chaos report 2015. Technical report, The Standish Group International, Inc (2015)

  • Tian, Y. Wijedasa, D., Lo, D., Le Gouesy, C.: Learning to rank for bug report assignee recommendation. In: Proceedings of the 24th International Conference on Program Comprehension (ICPC), pp. 1–10. ISBN 9781509014286 (2016). https://doi.org/10.1109/ICPC.2016.7503715

  • Tuarob, S., Assavakamhaenghan, N., Tanaphantaruk, W., Suwanworaboon, P., Hassan, S.-U., Choetkiertikul, M.: Automatic team recommendation for collaborative software development. Empir. Softw. Eng. 26(4), 1–53 (2021)

    Article  Google Scholar 

  • Wang, X., Zhao, Z., Ng, W.: A comparative study of team formation in social networks, pp. 389–404, 2015. ISBN 978-3-319-18119-6. https://doi.org/10.1007/978-3-319-18120-2_23

  • Wang, X., Zhao, Z., Ng, W.: Ustf: a unified system of team formation. IEEE Trans. Big Data 2(1), 70–84 (2016)

    Article  Google Scholar 

  • Wick, C.T.: The importance of team skills for software development. PhD thesis (1999)

  • Wieland, K., Langer, P., Seidl, M., Wimmer, M., Kappel, G.: Turning conflicts into collaboration. Comput. Supported Cooperative Work (CSCW) 22(2), 181–240 (2013). https://doi.org/10.1007/s10606-012-9172-4

    Article  Google Scholar 

  • Xu, C., Sun, X., Li, B., Lu, X., Guo, H.: MULAPI: improving API method recommendation with API usage location. J. Syst. Softw. 142, 195–205 (2018). https://doi.org/10.1016/j.jss.2018.04.060

    Article  Google Scholar 

  • Yang, H., Yan, Z., Jia, L., Liang, H.: The impact of team diversity on physician teams’ performance in online health communities. Inf. Process. Manag. 58(1), 102421 (2021)

    Article  Google Scholar 

  • Yasrab, R., Ferzund, J., Razzaq, S.: Challenges and issues in collaborative software developments (2011)

  • Ye, L., Sun, H., Wang, X., Wang, J.: Personalized Teammate Recommendation for Crowdsourced Software Developers, New York, NY, USA. Association for Computing Machinery, pp. 808–813 (2018). https://doi.org/10.1145/3238147.3240472

  • Zhang, Z., Sun, H., Zhang, H.: Developer recommendation for topcoder through a meta-learning based policy model. Empir. Softw. Eng. 25(1), 859–889 (2020)

    Article  Google Scholar 

  • Zhu, H., Zhou, M., Seguin, P.: Supporting software development with roles. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 36(6), 1110–1123 (2006). https://doi.org/10.1109/TSMCA.2006.883170

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the Thailand Science Research and Innovation (TSRI), formerly known as Thailand Research Fund (TRF), and the National Research Council of Thailand (NRCT) through Grant No. RSA6280105. We also appreciate computing resources from Mahidol University (Grant No. MU-MiniRC02/2564).

Author information

Authors and Affiliations

  1. Faculty of Information and Communication Technology, Mahidol University, Nakhon Pathom, Thailand

    Noppadol Assavakamhaenghan, Waralee Tanaphantaruk, Ponlakit Suwanworaboon, Morakot Choetkiertikul & Suppawong Tuarob

Authors
  1. Noppadol Assavakamhaenghan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Waralee Tanaphantaruk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ponlakit Suwanworaboon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Morakot Choetkiertikul
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Suppawong Tuarob
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Suppawong Tuarob.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Assavakamhaenghan, N., Tanaphantaruk, W., Suwanworaboon, P. et al. Quantifying effectiveness of team recommendation for collaborative software development. Autom Softw Eng 29, 51 (2022). https://doi.org/10.1007/s10515-022-00357-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-022-00357-7

Keywords

Search

Navigation