Skip to main content
SpringerLink
Log in

The Focus Group Method as an Empirical Tool in Software Engineering

  • Chapter
Guide to Advanced Empirical Software Engineering

This chapter presents the focus group method and discusses its use for empirical research in the software engineering context. The background, process and main characteristics of the method are presented, as well as guidelines for its use. Moreover, the traditional as well computer-mediated focus group variations are compared to each other. The chapter concludes in with a discussion of the applicability of the method for software engineering research. In summary, the focus group method is a cost-effective and quick empirical research approach for obtaining qualitative insights and feedback from practitioners. It can be used in several phases and types of research. However, a major limitation of the method is that it is useful only in studying concepts that can be understood by knowledgeable participants in a limited time. We also emphasize the importance of empirical rigor when the method is used in scholarly work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Anon., Focus Group Kit, Vol. 1–6, Sage Publications, Thousand Oaks, CA, 1997.

    Google Scholar 

  • Adler, M., Ziglio, E., Gazing into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, Jessica Kingsley Pub, 1996.

    Google Scholar 

  • Austin, T., Drakos, N., Mann, J., Web Conferencing Amplifies Dysfunctional Meeting Practices. Gartner Research Report Nr. G00138101, Gartner Inc., 2006.

    Google Scholar 

  • Baker, S.L., Improving Business Services through the Use of Focus Groups, Reference Quarterly, 30(Spring):377–385, 1991.

    Google Scholar 

  • Basili, V.R., Selby, R.W., Hutchens, D.H., Experimentation in Software Engineering, IEEE Transactions on Software Engineering, 12(7):758–773, 1986.

    Google Scholar 

  • Benbasat, I., Rethinking Diversity in Information Systems Research, Information Systems Research, 7(4):389–399, 1996.

    Article  Google Scholar 

  • Boehm, B., Gruenbacher, P., Briggs, R.O., Developing Groupware for Requirements Negotiation: Lessons Learned, IEEE Software, 18(3):46–55, 2001.

    Article  Google Scholar 

  • Bogdan, R.C., Biklen, S.K., Qualitative Research for Education: An Introduction to Theory and Methods, Allyn and Bacon Inc., Boston, MA, 1982.

    Google Scholar 

  • Bragge, J., den Hengst, M., Tuunanen, T., Virtanen, V., A Repeatable Collaboration Process for Developing a Road Map for Mobile Marketing. In Proceedings of the 11th Americas Conference on Information Systems AMCIS, 2005a.

    Google Scholar 

  • Bragge, J., Marttiin, P., Tuunanen, T., Developing Innovative IS Services Together with Wide Audience End-Users, In Proceedings of the 38th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1–10, 2005b.

    Google Scholar 

  • Bragge, J., Merisalo-Rantanen, H., Hallikainen, P., Gathering Innovative End-User Feedback for Continuous Development of Information Systems: A Repeatable and Transferable E-Collaboration Process, IEEE Transactions on Professional Communication, 48(1):55–67, 2005c.

    Article  Google Scholar 

  • Bragge, J., Merisalo-Rantanen, H., Nurmi, A., Tanner, L., A Repeatable E-Collaboration Process Based on ThinkLets for Multi-Organization Strategy Development, Group Decision and Negotiation, 16(4):363–379, 2007a.

    Article  Google Scholar 

  • Bragge, J., Relander, S., Sunikka, A., Mannonen, P., Enriching Literature Reviews with Computer-Assisted Research Mining. Case: Profiling Group Support Systems Research. In Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS’07), IEEE, Los Alamitos, CA, pp. 1–10, 2007b.

    Google Scholar 

  • Briggs, R.O., De Vreede, G.J., Nunamaker, J.F., Collaboration Engineering with ThinkLets to Pursue Sustained Success with Group Support Systems, Journal of Management Information Systems, 19(4):31–64, 2003.

    Google Scholar 

  • Buhrer, H.K., Software Development: What It is, What It should be, and How to get There, ACM SIGSOFT Software Engineering Notes, 28(2):1–4, 2007.

    Google Scholar 

  • Chen, M., Nunamaker, J.F., The Architecture and Design of a Collaborative Environment for Systems Definition, Data-Base, 22(1–2):22–29, 1991.

    Google Scholar 

  • Clapper, D.L., Massey, A.P., Electronic Focus Groups: A Framework for Exploration, Information and Management, 30(1):43–50, 1996.

    Article  Google Scholar 

  • Daft, R., Lengel, R., Organizational Information Requirements, Media Richness, and Structural Design, Management Science, 32(5):554–570, 1986.

    Article  Google Scholar 

  • De Vreede, G.J., Fruhling, A., Chakrapani, A., A Repeatable Collaboration Process for Usability Testing, In Proceedings of the 38th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1–10, 2005.

    Google Scholar 

  • Easton, G., Easton, A., Belch, M., An Experimental Investigation of Electronic Focus Groups, Information and Management, 40:717–727, 2003.

    Article  Google Scholar 

  • Edmunds, H., The Focus Group Research Handbook, NTC Business Books, Lincolnwood, IL, 1991.

    Google Scholar 

  • Edmunds, H., The Focus Group Research Handbook, NTC Business Books and American Marketing Association, Lincolnwood, IL, 1999.

    Google Scholar 

  • Elfvengren, K., Karkkainen, H., Torkkeli, M., Tuominen, M., A GDSS Based Approach for the Assessment of Customer Needs in Industrial Markets, International Journal of Production Economics, 89(3):272–292, 2004.

    Article  Google Scholar 

  • Feig, B., How to Run a Focus Group, American Demographics, 11(December):36–37, 1989.

    Google Scholar 

  • Fern, E.F., The Use of Focus Groups for Idea Generation: The Effects of Group Size, Acquaintanceship, and Moderator on Response Quantity and Quality, Journal of Marketing Research, 19(1):1–13, 1982.

    Article  Google Scholar 

  • Fjermestad, J., Hiltz, S.R., An Assessment of Group Support Systems Experimental Research: Methodology and Results, Journal of Management Information Systems, 15(3):7–150, 1999.

    Google Scholar 

  • Fjermestad, J., Hiltz, S.R., Group Support Systems: A Descriptive Evaluation of Case and Field Studies, Journal of Management Information Systems, 17(3):112–157, 2000.

    Google Scholar 

  • Fraunhofer USA Inc., Summary of the Third eWorkshop on Agile Methods, Center for Experimental Software Engineering, http://fc-md.umd.edu/projects/Agile/3rd-eWorkshop/summary3rdeWorksh.htm, 2002.

  • Galliers, R.D., Choosing Appropriate Information Systems Research Approaches: A Revised Taxonomy, Information Systems Research: Contemporary Approaches and Emerging Traditions, Nissen H.-E. , H.K. Klein and R. Hirschheim, eds. Elsevier Science Publishers, Amsterdam, pp. 327–345, 1991.

    Google Scholar 

  • Ghauri, P., Grønhaug, K., Kristianslund, I., Research Methods in Business Studies, Prentice-Hall, Englewood Cliffs, NJ, 1995.

    Google Scholar 

  • Glass, R.A., A Structure-Based Critique of Contemporary Computing Research, Journal of Systems and Software, 28(1):3–7, 1995.

    Article  Google Scholar 

  • Gruenbacher, P., Halling, M., Biffl, S., Kitapci, H., Boehm, B.W., Repeatable Quality Assurance Techniques for Requirements Negotiation, In Proceedings of the 36th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1–9, 2003.

    Google Scholar 

  • Halling, M., Gruenbacher, P., Biffl, S., Tailoring a COTS Group Support System for Software Requirements Inspection, In Proceedings of the 16th Annual International Conference on Automated Software Engineering, IEEE, Los Alamitos, CA 201–208, 2001.

    Chapter  Google Scholar 

  • Hansen, K., Hansen, R.S., Using an Asynchronous Discussion Board for Online Focus Groups: A Protocol and Lessons Learned, In Proceedings of the College Teaching and Learning Conference, Clute Institute for Academic Research, Littleton, Colorado 1–8, 2006.

    Google Scholar 

  • Judd, C.M., Smith, E.R., Kidder, L.H., Research Methods in Social Relations, Harcourt Brace Jovanovich College Publishers, New York, 1991.

    Google Scholar 

  • Juristo, N., Moreno, A.M., Basics of Software Engineering Experimentation, Kluwer Academic Publishers, Boston, MA, 2001.

    MATH  Google Scholar 

  • Kitchenham, B., Dyba, T., Jorgensen, M., Evidence-Based Software Engineering. In Proceedings of 26th International Conference on Software Engineering, IEEE, Los Alamitos, CA, pp. 273–281, 2004.

    Google Scholar 

  • Klein, E.E., Tellefsen, T., Herskovitz, P.J., The Use of Group Support Systems in Focus Groups: Information Technology Meets Qualitative Research, Computers in Human Behavior, 23(5):2113–2132, 2007.

    Article  Google Scholar 

  • Kontio, J., The Riskit Method for Software Risk Management, version 1.00. (College Park, MD, University of Maryland, 1997) CS-TR-3782/UMIACS-TR-97-38, Computer Science Technical Reports.

    Google Scholar 

  • Kontio, J., Software Engineering Risk Management: A Method, Improvement Framework, and Empirical Evaluation. Doctoral dissertation. (2001), Helsinki University of Technology, publisher: Center of Excellence, ISBN: 952-5136-22-1.

    Google Scholar 

  • Kontio, J., Bragge, J., Lehtola, L., Using the Focus Group Method in Software Engineering: Obtaining Practitioner and User Experiences. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE), ACM-IEEE, Los Alamitos, CA pp. 271–280, 2004.

    Google Scholar 

  • Krueger, R.A., Casey, M.A., Focus Groups: A Practical Guide for Applied Research, Sage Publications, Thousand Oaks, CA, 2000.

    Google Scholar 

  • Langford, J., McDonaugh, D., Focus Groups. Supporting Effective Product Development, Taylor and Francis, London, 2003.

    Google Scholar 

  • Lehtola, L., Kauppinen, M., Kujala, S., Requirements-Prioritization-Challenges-in-Practice. In Fifth International Conference on Product Focused Software Process Improvement, 2004.

    Google Scholar 

  • Liou, Y.I., Chen, M., Using Group Support Systems and Joint Application Development for Requirements Specification, Journal of Management Information Systems, 10(3):25–41, 1993.

    Google Scholar 

  • Massey, A.P., Wallace, W.A., Focus Groups as a Knowledge Elicitation Technique, IEEE Transactions on Knowledge and Data Engineering, 3(2):193–200, 1991.

    Article  Google Scholar 

  • McQuarrie, E.F., New Books in Review: The Handbook for Focus Group Research & Successful Focus Groups: Advancing the State of the Art, Journal of Marketing Research, 31:377–380, 1994.

    Article  Google Scholar 

  • McQuarrie, E.F., New Books in Review: The Mirrored Window: Focus Groups from a Moderator’s Point of View & Advanced Focus Group Research, Journal of Marketing Research, 38(November):515–516, 2001.

    Article  Google Scholar 

  • McQuarrie, E.F., McIntyre, S.H., Focus Groups and the Development of New Products by Technologically Driven Companies: Some Guidelines, Journal of Product Innovation Management, 1:40–47, 1986.

    Article  Google Scholar 

  • Miles, M.B., Huberman, A.M., Qualitative Data Analysis: A Sourcebook of New Methods, Sage Publications, Thousand Oaks, CA, 1984.

    Google Scholar 

  • Montoya-Weiss, M.M., Massey, A.P., Clapper, D.L., On-line Focus Groups: Conceptual Issues and a Research Tool, European Journal of Marketing, 32(7/8):713–723, 1998.

    Article  Google Scholar 

  • Morgan, D.L., Designing Focus Group Research, in: Tools for Primary Care Research. Volume 2: Research Methods for Primary Care, M. Stewart, F. Tudiver, M.J. Bass, E.V. Dunn and P.G. Norton, eds. Sage Publications, Thousand Oaks, CA, 1992.

    Google Scholar 

  • Morgan, D.L., Focus Groups, Annual Review of Sociology, 22(August):129–152, 1996.

    Article  Google Scholar 

  • Morgan, D.L., Focus Groups as Qualitative Research, Sage Publications, Thousand Oaks, CA, 1997.

    Google Scholar 

  • Myers, M., Qualitative Research in Information Systems, http://www.qual.auckland.ac.nz, 2004.

  • Nambisan, S., Information Systems as a Reference Discipline for New Product Development, MIS Quarterly, 27(1):1–18, 2003.

    Google Scholar 

  • Nambisan, S., Wilemon, D., Software Development and New Product Development: Potentials for Cross-Domain Knowledge Sharing, IEEE Transactions on Engineering Management, 47(2):211–220, 2000.

    Article  Google Scholar 

  • Neter, J., Waksberg, J., A Study of Response Errors in Expenditure Data from Household Interviews, Journal of the American Statistical Association, 59:18–55, 1964.

    Article  Google Scholar 

  • Newby, R., Soutar, G., Watson, J., Comparing Traditional Focus Groups with a Group Support Systems (GSS) Approach for Use in SME Research, International Small Business Journal, 21(4):421–433, 2003.

    Article  Google Scholar 

  • Nielsen, J., The Use and Misuse of Focus Groups, IEEE Software, 14(January):94–95, 1997.

    Article  Google Scholar 

  • Nunamaker, J.F., Dennis, A.R., Valacich, J.S., Vogel, D.R., George, J.F., Electronic Meeting Systems to Support Group Work, Communications of the ACM, 34(7):40–61, 1991.

    Article  Google Scholar 

  • O’Connor, H., Madge, C., “Focus Groups in Cyberspace”: Using the Internet for Qualitative Research, Qualitative Market Research, 6(2):133–143, 2003.

    Article  Google Scholar 

  • Oringderff, J., ‘My Way’: Piloting and Online Focus Group, International Journal of Qualitative Methods, 3(3):1–10, 2004.

    Google Scholar 

  • Parent, M., Gallupe, R.B., Salisbury, W.D., Handelman, J.M., Knowledge Creation in Focus Groups: Can Group Technologies Help?, Information and Management, 38(1):47–58, 2000.

    Article  Google Scholar 

  • Patton, M.Q., Qualitative Evaluation and Research Methods, Sage Publications, Thousand Oaks, CA, 1990.

    Google Scholar 

  • Reid, D.J., Reid, F.J.M., Online Focus Groups. An In-Depth Comparison of Computer-Mediated and Conventional Focus Group Discussions, International Journal of Market Research, 47(2):131–162, 2005.

    Google Scholar 

  • Rodgers, T.L., Dean, D.L., Nunamaker, J.F., Increasing Inspection Efficiency through Group Support Systems, In Proceedings of the 37th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1–10, 2004.

    Google Scholar 

  • Rubin, J., Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, Wiley, New York, 1994.

    Google Scholar 

  • Stewart, D.W., Shamdasani, P.N., Focus Groups: Theory and Practice, Sage Publications, Thousand Oaks, CA, 1990.

    Google Scholar 

  • Sunikka, A., Usability evaluation of the Helsinki School of Economics Website. Master’s thesis, Helsinki School of Economics, 2004.

    Google Scholar 

  • Sweet, C., Designing and Conducting Virtual Focus Groups, Qualitative Market Research, 4(3):130–135, 2001.

    Article  MathSciNet  Google Scholar 

  • Taylor, S.J., Bogdan, R., Introduction to Qualitative Research Methods, Wiley, New York, 1984.

    Google Scholar 

  • Templeton, J.F., The Focus Group: A Strategic Guide to Organizing, Conducting and Analyzing the Focus Group Interview, McGraw-Hill Professional Publishing, New York, 1994.

    Google Scholar 

  • Ten-Pow, J., Fundamentals for Those Considering Online Focus Groups, On Survey Research Intelligence, http://www.onsurvey.ca/supplemental/onfocus.pdf, 2003.

  • Tichy, W.F., Should Computer Scientists Experiment More?, IEEE Computer, 31(5):32–40, 1998.

    MathSciNet  Google Scholar 

  • Tichy, W.F., Lukowicz, P., Prechelt, L., Heinz, E.A., Experimental Evaluation in Computer Science: A Quantitative Study, Journal of Systems and Software, 28(1):9–18, 1995.

    Article  Google Scholar 

  • Turney, L., Pocknee, C., Virtual focus groups: New technologies, new opportunities, new learning environments, Proceedings of the 21st ASCILITE Conference, University of Wollongong, New South Wales, Australia, pp. 905–912.

    Google Scholar 

  • Tuunanen, T., Rossi, M., Engineering a Method for Wide Audience Requirements Elicitatation and Integrating It to Software Development, In Proceedings of the 37th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, 2004.

    Google Scholar 

  • Van Genuchten, M., Cornelissen, W., Van Dijk, C., Supporting Inspections with an Electronic Meeting System, Journal of Management Information Systems, 14(3):165–178, 1997.

    Google Scholar 

  • Van Genuchten, M., Van Dijk, C., Scholten, H., Using Group Support Systems for Software Inspections, IEEE Software, 18(3):60–65, 2001.

    Article  Google Scholar 

  • Vitharana, P., Ramamurthy, K., Computer-Mediated Group Support, Anonymity, and the Software Inspection Process: An Empirical Investigation, IEEE Transactions on Software Management, 29(2):167–180, 2003.

    Article  Google Scholar 

  • Wellner, A.S., The New Science of Focus Groups, American Demographics, March 1 29–33, 2003.

    Google Scholar 

  • Widdows, R., Hensler, T.A., Wyncott, M.H., The Focus Group Interview: A Method for Assessing User’s Evaluation of Library Service, College and Research Libraries, 52(July):352–359, 1991.

    Google Scholar 

  • Wohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Experimentation in Software Engineering: An Introduction, Kluwer Academic Publishers, Boston, MA, 1999.

    Google Scholar 

  • Wohlin, C., Höst, M., Henningsson, K., Empirical Research Methods in Software Engineering, Lecture Notes in Computer Science, Vol. 2765 7–23, 2003.

    Article  Google Scholar 

  • Zelkowitz, M.V., Wallace, D.R., Experimental Models for Validating Technology, IEEE Computer, 31(5):23–31, 1998.

    Google Scholar 

  • Zinchiak, M., Online Focus Group FAQs, Quirk’s Marketing Research Review, http://www.quirks.com/articles/a2001/20010712.aspx?searchID=2619905, July/August 2001.

Download references

Author information

Authors and Affiliations

  1. Software Business Laboratory, Helsinki University of Technology, FIN-02015, Helsinki, Finland

    Jyrki Kontio

  2. Helsinki School of Economics and Business Administration, FIN-00101, Helsinki, Finland

    Johanna Bragge

  3. Software Business and Engineering Institute, Helsinki University of Technology, FIN-02015, Helsinki, Finland

    Laura Lehtola

Authors
  1. Jyrki Kontio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johanna Bragge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laura Lehtola
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Fraunhofer Center Maryland, 20742, College Park, MD, USA

    Forrest Shull

  2. National Research Council Canada, K1A 0R6, Ottawa, Ontario, Canada

    Janice Singer

  3. Simula Research Laboratory, NO-1325, Lysaker, Norway

    Dag I. K. Sjøberg

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag London Limited

About this chapter

Cite this chapter

Kontio, J., Bragge, J., Lehtola, L. (2008). The Focus Group Method as an Empirical Tool in Software Engineering. In: Shull, F., Singer, J., Sjøberg, D.I.K. (eds) Guide to Advanced Empirical Software Engineering. Springer, London. https://doi.org/10.1007/978-1-84800-044-5_4

Download citation

  • DOI: https://doi.org/10.1007/978-1-84800-044-5_4

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-84800-043-8

  • Online ISBN: 978-1-84800-044-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation