Empirical Software Engineering

, Volume 19, Issue 4, pp 822–856 | Cite as

Data collection in global software engineering research: learning from past experience

  • Rafael Prikladnicki
  • Alexander Boden
  • Gabriela Avram
  • Cleidson R. B. de Souza
  • Volker Wulf
Article

Abstract

Global Software Engineering has become a standard in today’s software industry. Research in distributed software development poses severe challenges that are due to the spatial and temporal distribution of the actors, as well as to language, intercultural and organizational aspects. These challenges occur in addition to “traditional” challenges of the domain itself in large-scale software projects, like coordination and communication issues, requirements volatily, lack of domain knowledge, among others. While several authors have reported empirical studies of global software development projects, the methodological difficulties and challenges of this type of studies have not been sufficiently discussed. In this paper, we share our experiences of collecting and analysing qualitative data in the context of Global Software Engineering projects. We discuss strategies for gaining access to field sites, building trust and documenting distributed and complex work practices in the context of several research projects we have conducted in the past 9 years. The experiences described in this paper illustrate the need to deal with fundamental problems, such as understanding local languages and different cultures, observing synchronous interaction, or dealing with barriers imposed by political conflicts between the sites. Based on our findings, we discuss some practical implications and strategies that can be used by other researchers and provide some recommendations for future research in methodological aspects of Global Software Engineering.

Keywords

Global software engineering Research methods Qualitative research Data collection 

Notes

Acknowledgments

The studies reported in this paper were partially supported by: the Research Group on Distributed Software Development of the PDTI program, financed by Dell Computers of Brazil Ltd. (Law 8.248/91); CAPES (project BEX 426006-6); CAPES (project BEX 1312/99-5); FAPERGS; CNPq (project 479206/2006-6, project 550130/2011-06, project 560037/2010-4, project 483125/2010-5); Science Foundation Ireland and Lero—The Irish Software Engineering Research Centre under PI grant 03/IN3/1408C; and the German Research Foundation (DFG) within the Project ARTOS—Articulation work in Offshoring Projects of Small and Medium- sized Enterprises of the Software Branch.

References

  1. Aranda J, Venolia G (2009) The secret life of bugs: going past the errors and omissions in software repositories. Proceedings of the 2009 IEEE 31st International Conference on Software Engineering, IEEE Computer Society, pp 298–308Google Scholar
  2. Argyris C, Putnam R, Smith DM (1985) Action science. San FranciscoGoogle Scholar
  3. Avram G (2007a) Of deadlocks and peopleware: colaborative work practices in global software development. In: ICGSE, Munich, GermanyGoogle Scholar
  4. Avram G (2007b) Knowledge work practices in global software development. The European Conference on Knowledge Management, BarcelonaGoogle Scholar
  5. Avram G, Wulf V (2011) Guest editorial: Studying work practices in Global Software Engineering, Information and Software Technology 53(9):949–954, ISSN 0950-5849, 10.1016/j.infsof.2011.01.010. (http://www.sciencedirect.com/science/article/pii/S0950584911000371)
  6. Avram G, Bannon L, Bowers J, Sheehan A, Sullivan D (2009) Bridging, patching and keeping the work flowing: defect resolution in distributed software development. doi:10.1007/s10606-009-9099-6. Special Issue dedicated to ‘Software Engineering as Cooperative Work’ Guest Edited by Yvonne Dittrich, Dave W. Randall and Janice Singer, Journal of Computer Supported Cooperative Work, Volume 18, Numbers 5–6, pp 477–507
  7. Biolchini J, Mian PG, Natali ACC, Travassos GH (2005) Systematic review in software engineering. Technical Report TR—ES 679 / 05, COPPE/UFRJGoogle Scholar
  8. Boden A, Nett B, Wulf V (2007) Coordination practices in distributed software development of small enterprises. In: ICGSE, Munich, Germany, pp 235–244Google Scholar
  9. Boden A, Nett B, Wulf V (2008) Articulation work in small-scale offshore software development projects. Proceedings of the 2008 international workshop on Cooperative and human aspects of Software Engineering. ACM, Leipzig, Germany, pp 21–24Google Scholar
  10. Boden A, Avram G, Bannon L, Wulf V (2009) Knowledge management in distributed software development teams—does culture matter? Proceedings of the 2009 International Conference on Global Software Engineering (ICGSE), Limerick, Ireland, pp 18–27Google Scholar
  11. Boden A, Avram G, Bannon L, Wulf V (2011) Knowledge sharing practices and the impact of cultural factors: reflections on two case studies of offshoring in SME. J Softw Maint Evol Res PractGoogle Scholar
  12. Brown B, Lundin J, Rost M, Lymer G, Holmquist L (2007) Seeing ethnographically: teaching ethnography as part of CSCW. ECSCW 2007:411–430Google Scholar
  13. Carmel E (1999) Global software teams—collaborating across borders and time-zones. Prentice Hall, Upper Saddle RiverGoogle Scholar
  14. Carmel E (2006) Building your information systems from the other side of the world: How Infosys manages time zone differences. MISQ 5(1):43–53Google Scholar
  15. Carver J, Seaman C, Jeffery R (2004) Using qualitative methods in software engineering. In: International Advanced School of Empirical Software Engineering (IASESE04), Los Angeles, CA, USAGoogle Scholar
  16. Creswell JW (2003) Research design: qualitative, quantitative, and mixed methods approaches. SAGE Publications, USAGoogle Scholar
  17. Cross R, Parker A (2004) The hidden power of social networks: understanding how work really gets done in organizations. Harvard Business School Press, 304 ppGoogle Scholar
  18. Curtis B, Krasner H et al (1988) A field study of the software design process for large systems. Commun ACM 31(11):1268–1287CrossRefGoogle Scholar
  19. Damian D, Moitra D (2006) Guest editors’ introduction: global software development: how far have we come? IEEE Softw 23(5):17–19CrossRefGoogle Scholar
  20. Dawson R, Bones P, Oates BJ, Brereton P, Azuma M, Jackson ML (2003) Empirical methodologies in software engineering. In: 11th Annual International Workshop on Software Technology and Engineering Practice, Washington, DC, pp 52–58Google Scholar
  21. De Souza CRB, Redmiles DF (2011) The awareness network. To whom should I display my actions? And, whose actions should I monitor? IEEE T Software Eng 37(3):325–340Google Scholar
  22. De Souza CRB, Hildenbrand T, Redmiles DF (2007) Towards visualization and analysis of traceability relationships in distributed and offshore software development projects. In: SEAFOOD, LNCSGoogle Scholar
  23. de Souza CRB, Sharp H, Singer J, Cheng L-T, Venolia G (2009) Guest editors’ introduction: cooperative and human aspects of software engineering. IEEE Software 26(6):17–19CrossRefGoogle Scholar
  24. Dittrich Y, John M, Singer J, Tessem B (2007) For the special issue on qualitative software engineering research. Inf Softw Technol 49(6):531–539CrossRefGoogle Scholar
  25. Dourish P (2006) Re-space-ing place: “place ” and “ space ” ten years on. In: Proceedings of the Conference on Computer Supported Cooperative WorkGoogle Scholar
  26. Dybå T, Prikladnicki R, Rönkkö K, Seaman CB, Sillito J (2011) Qualitative research in software engineering. Empir Softw Eng 16(4):425–429CrossRefGoogle Scholar
  27. Easterbrook SM, Singer J, Storey M, Damian D (2007) Selecting empirical methods for software engineering research. In: Shull F, Singer J (eds) Guide to advanced empirical software engineering. SpringerGoogle Scholar
  28. Espinosa JA, Nan N, Carmel E (2007) Do gradations of time zone separation make a difference in performance? A first laboratory study. In: ICGSE, Munich, GermanyGoogle Scholar
  29. Geertz C (1973) Thick description: towards an interpretive theory of culture. In: Geertz C (ed) The interpretation of cultures: selected essays. Basic Books, New York, pp 3–30Google Scholar
  30. Gobo G (2008) Doing ethnography. Sage Publications, Los AngelesGoogle Scholar
  31. Gonçalves MK, De Souza CRB, Gonzales VMG (2011) Collaboration, information seeking and communication: an observational study of software developers work practices. J Univers Comput Sci 17:1913–1930Google Scholar
  32. Gupta A, Ferguson J (1997) Culture, power, place: explorations in critical anthropology. Duke University Press, DurhamCrossRefGoogle Scholar
  33. Herbsleb JD (2005) Beyond computer science. 27th ICSE, 23–27Google Scholar
  34. Herbsleb JD (2007) Global software engineering: the future of socio-technical coordination. 29th ICSE, 188–198Google Scholar
  35. Herbsleb JD, Grinter RE (1999) Architectures, coordination, and distance: Conway’s Law and beyond. IEEE Software: 63–70Google Scholar
  36. Herbsleb JD, Moitra D (2001) Guest editors’ introduction: global software development. IEEE Softw 18(2):16–20CrossRefGoogle Scholar
  37. Herbsleb JD, Mockus A, Finholt T, Grinter RE (2001) An empirical study of global software development: distance and speed. In 23rd ICSE, pp 81–90Google Scholar
  38. Hine C (2000) Virtual ethnography. Sage, LondonGoogle Scholar
  39. Jacobson I, Booch G, Rumbaugh J (1999) The unified software development process. Addison Wesley Longman, Inc., ReadingGoogle Scholar
  40. Jarvenpaa SL (1998) Communication and trust in global virtual teams. J Comput-Mediat Commun 3(4). http://jcmc.huji.ac.il
  41. King WR, Torkzadeh G (2008) Information systems offshoring: research status and issues. MIS Q 32(2):205–225Google Scholar
  42. Klein HK, Myers MD (1999) A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Q 23(1):67–93CrossRefGoogle Scholar
  43. Ko AJ, DeLine R, Venolia G (2007) Information needs in collocated software development teams. International Conference on Software Engineering (ICSE), Minneapolis, MN, May 20–26, 344–353Google Scholar
  44. Kraut RE, Streeter LA (1995) Coordination in software development. Commun ACM 38(3):69–81CrossRefGoogle Scholar
  45. Kraut R, Egido C, Galegher J (1990) Patterns of contact and communication in scientific research collaborations. In: Galegher J, Egido C, Kraut R (eds) Intellectual teamwork: social and technological foundations of cooperative work. Lawrence Erlbaum, pp 149–172Google Scholar
  46. Lethbridge T, Sim S, und Singer J (2005) Studying software engineers: data collection techniques for software field studies. Empir Softw Eng 10:311–341CrossRefGoogle Scholar
  47. Lutz B (2009) Linguistic challenges in global software development: lessons learned in an International SW Development Division. Global Software Engineering, International Conference on, Los Alamitos, CA, USA: IEEE Computer Society, pp 249–253Google Scholar
  48. Marcus GE (1998) Ethnography through thick and thin. Princeton University Press, Princeton, New JerseyGoogle Scholar
  49. Oates BJ (2006) Researching information systems and computing. Sage Publications, Thousand OaksGoogle Scholar
  50. Onwuegbuzie A, Leech N (2007) Validity and qualitative research: an oxymoron? Qual Quant 41(2):233–249CrossRefGoogle Scholar
  51. Paré G. Enhancing the rigor of qualitative research: application of a case methodology to build theories of IT implementation. The Qualitative Report, 7(4), retrieved from http://www.nova.edu/ssss/QR/QR7-4/pare.html, April of 2007
  52. Parnas DL (2003) The limits of empirical studies of software engineering. In: Proceedings of the 2003 International Symposium on Empirical Software Engineering (September 30–October 01, 2003). International Symposium on Empirical Software Engineering. IEEE Computer Society, Washington, DC, 2Google Scholar
  53. Patil S, Kobsa A, John A, Seligmann D (2011) Methodological reflections on a field study of a globally distributed software project. Inf Softw Technol 53(9):969–980. doi:10.1016/j.infsof.2011.01.013 CrossRefGoogle Scholar
  54. Perry DE, Staudenmayer NA, Votta LG (1994) People, organizations, and process improvement. IEEE Softw 11(4):36–45CrossRefGoogle Scholar
  55. Plonka L, Sharp H, van der Linden J (2012) Disengagement in pair programming: does it matter? International Conference on Software Engineering, ZurichGoogle Scholar
  56. Prikladnicki R, Audy JLN, Evaristo R (2003) Global software development in practice: lessons learned. J Softw Process Improv Pract 8(4):267–282CrossRefGoogle Scholar
  57. Prikladnicki R, Audy JLN, Evaristo R (2006) A reference model for GSE: findings from a case study. In: ICGSE 2006, Florianopolis, BrazilGoogle Scholar
  58. Prikladnicki R, Audy JLN, Damian D, Oliveira TC (2007) Distributed software development: practices and challenges in different business strategies of offshoring and onshoring. In: ICGSE, Munich, GermanyGoogle Scholar
  59. Prikladnicki R, Evaristo R, Damian D, Audy JLN (2008) Conducting qualitative research in an international and distributed research team: challenges and lessons learned. In: HICSS 2008, Hawaii, USAGoogle Scholar
  60. Randall D, Harper R, Rouncefield M (2007) Fieldwork for design: theory and practice. SpringerGoogle Scholar
  61. Richardson I, Avram G, Deshpande S, Casey V (2008) Having a foot on each shore—bridging global software development in the case of SMEs. In: Proceedings of the 3rd IEEE International Conference on Global Software Engineering, Bangalore, India, 19–21st August, 2008, IEEE Computer Society Washington, DC, USA, 13–22Google Scholar
  62. Rigby PC, Storey MA (2011) Understanding broadcast based peer review on open source software projects. International Conference on Software Engineering, HawaiiGoogle Scholar
  63. Rönkkö K (2000) Ethnography and distributed software development. In: ICSE Workshop Beg, Borrow or Steal: Using Multidisciplinary Approaches In Empirical Software Engineering Research, Limerick, IrelandGoogle Scholar
  64. Seaman CB (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572CrossRefGoogle Scholar
  65. Sengupta B, Chandra S, Sinha V (2006) A research agenda for distributed software development. In: 28th ICSE, pp 731–740, Shanghai, ChinaGoogle Scholar
  66. Sharp H, Robinson H (2004) An ethnographic study of XP practice. Empir Softw Eng 9(4):353–375CrossRefGoogle Scholar
  67. Spanoudakis G, Zisman A (2004) Software traceability: a roadmap. Handbook of Software Engineering and Knowledge Engineering. In: Chang SK (ed). World Scientific Publishing CoGoogle Scholar
  68. Strauss A, Corbin J (1998) Basics of qualitative research: techniques and procedures for developing grounded theory, 2nd edn. Sage Publications, USAGoogle Scholar
  69. Suchman L (1987) Plans and situated actions: the problem of human-machine communication. Cambridge University Press, New YorkGoogle Scholar
  70. Taylor SJ, Bogdan R (1984) Introduction to qualitative research methods. Wiley, New YorkGoogle Scholar
  71. Tosun A, Turhan B, Bener A (2008) Ensembe of software defect predictors: a case study. Proceedings of the 2nd Symposium on Empirical Software Engineering and MeasurementGoogle Scholar
  72. Wittel A (2000) Ethnography on the move: from field to net to internet. Forum Qualitative Sozialforschung, vol 1; http://www.qualitative-research.net/fqs-texte/1-00/1-00wittel-e.htm
  73. Wohlin C, Höst M, Henningsson K (2003) Empirical research methods in software engineering. In: Conradi R, Wang AI (eds) ESERNET 2001-2003, LNCS 2765. Springer, pp 7–23Google Scholar
  74. Wulf V, Rohde M, Pipek V, Stevens G (2011) Engaging with practices: design case studies as a research framework in CSCW. In: Proceedings of the 2011 Conference on Computer Supported Cooperative Work, pp 505–512Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Rafael Prikladnicki
    • 1
  • Alexander Boden
    • 2
  • Gabriela Avram
    • 3
  • Cleidson R. B. de Souza
    • 4
  • Volker Wulf
    • 2
  1. 1.PUCRSPorto AlegreBrazil
  2. 2.University of SiegenSiegenGermany
  3. 3.University of LimerickLimerickIreland
  4. 4.Vale Institute of Technology and Universidade Federal do ParáBelémBrazil

Personalised recommendations