Abstract
For over four decades user involvement has been considered intuitively to lead to user satisfaction, which plays a pivotal role in successful outcome of a software project. The objective of this paper is to explore the notion of user satisfaction within the context of the user involvement and system success relationship. We have conducted a longitudinal case study of a software development project and collected qualitative data by means of interviews, observations and document analysis over a period of 3 years. The analysis of our case study data revealed that user satisfaction significantly contributes to the system success even when schedule and budget goals are not met. The case study data analysis also presented additional factors that contribute to the evolution of user satisfaction throughout the project. Users’ satisfaction with their involvement and the resulting system are mutually constituted while the level of user satisfaction evolves throughout the stages of software development process. Effective management strategies and user representation are essential elements of maintaining an acceptable level of user satisfaction throughout software development process.
Similar content being viewed by others
Notes
Only the first case study is presented in this paper; the data from the second case study is currently undergoing analysis and the results will be published in the future.
The real name of this organization cannot be mentioned due to ethical considerations and a non-disclosure agreement signed by the researchers.
References
Abelein U, Paech B (2013) Understanding the influence of user participation and involvement on system success–A systematic mapping study. Empir Softw Eng 20(1):28–81
Au N, Ngai EW, Cheng TE (2008) Extending the understanding of end user information systems satisfaction formation: An equitable needs fulfillment model approach. MIS Quarterly 43–66
Bailey JE, Pearson SW (1983) Development of a tool for measuring and analyzing computer user satisfaction. Manag Sci 29(5):530–545
Ballantine J, Bonner M, Levy M, Martin A, Munro I, Powell P (1996) The 3-D model of information systems success: the search for the dependent variable continues. Info Res Manag J (IRMJ) 9(4):5–15
Bano M, Zowghi D (2013) ‘Users' involvement in requirements engineering and system success’ In 3rd IEEE International Workshop on Empirical Requirements Engineering (EmpiRE) at International Requirements Engineering Conference 2013, Rio de Janeiro, Brazil. 24–31
Bano M, Zowghi D (2015) A systematic review on the relationship between user involvement and system success. Inf Softw Technol 58:148–169
Barki H, Hartwick J (1989) Rethinking the concept of user involvement. MIS Quarterly 53–63
Baronas A-MK, Louis MR (1988) Restoring a sense of control during implementation: how user involvement leads to system acceptance. MIS Quarterly 111–24
Bhattacherjee A (2001) Understanding information systems continuance: an expectation-confirmation model. MIS Quarterly 351–70
Briggs RO, Reinig BA, de Vreede G-J (2006) Meeting Satisfaction for Technology-Supported Groups An Empirical Validation of a Goal-Attainment Model. Small Group Res 37(6):585–611
Briggs RO, Reinig B, de VreedeG-J (2012) The yield shift theory of satisfaction and its application to the IS/IT domain. Info Syst Theory, Springer, 185–217
Cavaye AL (1995) User participation in system development revisited. Info Manag 28(5):311–323
Charmaz K (2006) Constructing Grounded Theory. SAGE
Cruzes DS, Dybå T (2011) Recommended steps for thematic synthesis in software engineering. Int Symp Emp Software Eng Measure (ESEM), IEEE, 275–84
Dalcher D (2009) Software project success: moving beyond failure. Eur J Info Profession 10(1):42–50
DeLone WH, McLean ER (1992) Information systems success: The quest for the dependent variable. Inf Syst Res 3(1):60–95
Delone WH, McLean ER (2003) The DeLone and McLean model of information systems success: a ten-year update. J Manag Inf Syst 19(4):9–30
Doll WJ, Torkzadeh G (1988) The measurement of end-user computing satisfaction. MIS Quarterly 259–74
Doll WJ, Xia W, Torkzadeh G (1994) A confirmatory factor analysis of the end-user computing satisfaction instrument. MIS Quarterly 453–61
Dybå T, Dingsøyr T, Hanssen GK (2007) Applying Systematic Reviews to Diverse Study Types: An Experience Report. First Int Symp Emp Software Eng Measure, ESEM 2007., IEEE. 225–34
Eason K (1989) Information Technology and Organizational Change, Taylor and Francis, Inc
El Emam K (2008) A replicated survey of IT software project failures. IEEE Softw 25(5):84–90
Gregor S (2006) The nature of theory in information systems. MIS Quarterly 611–42
Harris MA, Weistroffer HR (2009) A new look at the relationship between user involvement in systems development and system success. Commun Assoc Inf Syst 24(1):42
Hartwick J, Barki H (1994) Explaining the role of user participation in information system use. Manag Sci 40(4):440–465
He J, King WR (2008) The role of user participation in information systems development: implications from a meta-analysis. J Manag Inf Syst 25(1):301–331
Hirschheim RA (1989) User participation in practice: Experiences with participative systems design. Participat Syst Dev 194–212
Hirschheim R, Klein HK (1989) Four paradigms of information systems development. Commun ACM 32(10):1199–1216
Hwang MI, Thorn RG (1999) The effect of user engagement on system success: a meta-analytical integration of research findings. Info Manag 35(4):229–236
Igbaria M, Wormley WM (1992) Organizational experiences and career success of MIS professionals and managers: an examination of race differences. MIS Quarterly 507–29
Iivari N (2006) Representing the User’ in software development—a cultural analysis of usability work in the product development context. Inter Comput 18(4):635–664
Iivari J, Isomäki H, Pekkola S (2010) The user–the great unknown of systems development: reasons, forms, challenges, experiences and intellectual contributions of user involvement. Inf Syst J 20(2):109–117
Ives B, Olson MH (1984) User involvement and MIS success: A review of research. Manag Sci 30(5):586–603
Ives B, Olson MH, Baroudi JJ (1983) The measurement of user information satisfaction. Commun ACM 26(10):785–793
Klobas JE, McGill TJ (2010) The role of involvement in learning management system success. J Comput High Educ 22(2):114–134
Kujala S (2003) User involvement: a review of the benefits and challenges. Behav Info Technol 22(1):1–16
Lawrence M, Low G (1993) Exploring individual user satisfaction within user-led development. MIS Quarterly 195–208
Lee AS, Baskerville RL (2003) Generalizing generalizability in information systems research. Inf Syst Res 14(3):221–243
Liu V, Khalifa M (2003) Determinants of satisfaction at different adoption stages of Internet-based services. J Assoc Inf Syst 4(1):12
McHaney R, Hightower R, Pearson J (2002) A validation of the end-user computing satisfaction instrument in Taiwan. Info Manag 39(6):503–511
McKeen JD, Guimaraes T (1997) Successful strategies for user participation in systems development. J Manag Inform Syst 133–50
McKeen JD, Guimaraes T, Wetherbe JC (1994) The relationship between user participation and user satisfaction: an investigation of four contingency factors. MIS Quarterly 427–51
McKinney V, Yoon K, Zahedi FM (2002) The measurement of web-customer satisfaction: An expectation and disconfirmation approach. Inf Syst Res 13(3):296–315
Melone NP (1990) A theoretical assessment of the user-satisfaction construct in information systems research. Manag Sci 36(1):76–91
Montesdioca GPZ, Maçada ACG (2015) Measuring user satisfaction with information security practices. Comput Security 48:267–280
Muller MJ, Haslwanter JH, Dayton T (1997) Participatory practices in the software lifecycle. Handbook Human-Comput Int 2:255–297
Myers MD (1997) Qualitative Research in Information Systems. MIS Quarterly 241
Oliver RL (1996) Varieties of value in the consumption satisfaction response. Adv Consum Res 23:143–147
Palanisamy R (2001) User involvement in information systems planning leads to strategic success: an empirical study’. J Serv Res 1(2):125
Palmer JW, Griffith DA (1998) An emerging model of Web site design for marketing. Commun ACM 41(3):44–51
Petter S, DeLone W, McLean E (2008) Measuring information systems success: models, dimensions, measures, and interrelationships. Eur J Inf Syst 17(3):236–263
Procaccino JD, Verner JM, Overmyer SP, Darter ME (2002) Case study: factors for early prediction of software development success. Inf Softw Technol 44(1):53–62
Procaccino JD, Verner JM, Lorenzet SJ (2006) Defining and contributing to software development success. Commun ACM 49(8):79–83
Ralph P, Kelly P (2014) The dimensions of software engineering success. Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, ACM. 24–35
Reinig BA (2003) Toward an understanding of satisfaction with the process and outcomes of teamwork. J Manag Inf Syst 19(4):65–83
Robey D, Farrow D (1982) User involvement in information system development: A conflict model and empirical test. Manag Sci 28(1):73–85
Rouibah K, Hamdy HI, Al-Enezi MZ (2009) Effect of management support, training, and user involvement on system usage and satisfaction in Kuwait. Ind Manag Data Syst 109(3):338–356
Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empir Softw Eng 14(2):131–164
Rushinek A, Rushinek SF (1986) What makes users happy? Commun ACM 29(7):594–598
Savolainen P, Ahonen JJ, Richardson I (2012) Software development project success and failure from the supplier's perspective: A systematic literature review. Int J Proj Manag 30(4):458–469
Seddon PB (1997) A respecification and extension of the DeLone and McLean model of IS success. Inf Syst Res 8(3):240–253
Srijumpa R, Speece M, Paul H (2002) Satisfaction drivers for Internet service technology among stock brokerage customers in Thailand. J Financ Serv Mark 6(3):240–253
Suh K, Kim S, Lee J (1994) End-user’s disconfirmed expectations and the success of information systems. Info Res Manag J 7(4):30
Susarla A, Barua A, Whinston AB (2003) Understanding the service component of application service provision: empirical analysis of satisfaction with ASP services. MIS Q 27(1):91–123
Wagner EL, Piccoli G (2007) Moving beyond user participation to achieve successful IS design. Commun ACM 50(12):51–55
Walsham G (1995) The emergence of interpretivism in IS research. Inf Syst Res 6(4):376–394
Walsham G (2006) Doing interpretive research. Eur J Inf Syst 15(3):320–330
Wixom BH, Todd PA (2005) A theoretical integration of user satisfaction and technology acceptance. Inf Syst Res 16(1):85–102
Wu JTB, Marakas GM (2006) The impact of operational user participation on perceived system implementation success: An empirical investigation. J Comput Info Syst 46(5):127–140
Yin RK (2013) Case study research: Design and methods. Sage publications
Yoon,Y, Guimaraes T, O’Neal Q (1995) Exploring the factors associated with expert systems success. MIS Quarterly 83–106
Zowghi D, da Rimini F, Bano M (2015) Problems and challenges of user involvement in software development: an empirical study', Proceedings of the 19th ACM International Conference on Evaluation and Assessment in Software Engineering, EASE 2015 9
Acknowledgments
We would like to express our gratitude to the ASG management and staff for allowing us to conduct this study. We also acknowledge the funding provided to Muneera Bano by the University of Technology Sydney, and Schlumberger award of the Faculty For The Future (FFTF).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Magne Jørgensen, Mika Mäntylä, Paul Ralph and Hakan Erdogmus
Appendix A: Interview Guide Questions
Appendix A: Interview Guide Questions
1.1 Questions for First Phase of Interviews
1.1.1 General Questions
1. What is your position title in the organization? Perhaps explain what are the core tasks you perform daily.
2. How long have you been with this organization? Have you always been in the same position?
3. What is your role in this project?
4. Are you going to be a main user of the system being developed?
5. Are you a key stakeholder in the system being developed?
6. If the answer is yes to 4 and 5, how would you/your team benefit from the system? Your anticipation of overall benefit of the system.
1.1.2 Involvement in the Software Development Lifecycle
7. Please explain your involvement in this project from the beginning till now? When did you start? What is the nature of your involvement?
8. What would be the nature of your involvement from now till the end of the software development?
9. Who in the organization decides about the involvement of appropriate personnel in the software development project?
10. To what extent have you contributed to the scoping and requirements gathering process in this project so far?
11. Are you likely to be consulted again about the system from now on?
1.1.3 Communications
12. Can you comment on the communication channels between the core project team? And between this team and those external to the project team, in particular stakeholders.
13. What form of communications is mostly used?
E.g. (a) Face to Face: One on one (BA talking to individual stakeholders to elicit requirements), One to many (BA talking to a group of stakeholders), Facilitated Workshops (where an independent facilitator is involved between development team and stakeholders), focused groups,
(b) Emails, Telephone, Fax
(c) feedback via web sites/SNS, e.g., skype
14. Are you satisfied with the extent and degree of users involvement in this project so far?
15. If not, why not? If yes, why?
16. Is there anything that could be done differently to improve the process of scoping and requirements gathering and analysis?
1.1.4 Change/Change Management
17. Are there any/many business process changes in the new system?
18. Have these changes been identified at the outset and documented?
19. Is there a change management process in place to ensure the smooth roll out of the new system? If so, how are the users involved, informed, consulted about the potential changes to:
Business processes
New/modified workflows
20. Is there any training planned for the users? If so, who is in charge? Has the plan been articulated and consulted with users?
1.2 Post Implementation Interview Questions
1.2.1 UAT
1. How was this done? Process followed?
2. Who was involved? Level of involvement? Sign off?
3. How many rounds of UAT?
4. Did you develop any test scenarios?
1.2.2 Training
5. How was it carried out? Who was involved?
6. Quality?
7. Quantity?
8. Are you satisfied?
9. Documentation?
1.2.3 User Manual
10. Is there any?
11. When was it prepared/given to users?
12. Quality?
1.2.4 Post Implementation Review
13. When? How? Will they be involved?
14. User feedback sought? How? When?
1.2.5 Changes to Business Process/work Flow
15. Were there many?
16. Did you know about them before the system went live?
17. Any frustrations, interruptions to existing work, impacts over your work, possible delays?
1.2.6 Overall Satisfaction with the System
18. Are you generally satisfied with the delivered system?
19. Do you feel that your voices as users have been heard and taken into account?
Rights and permissions
About this article
Cite this article
Bano, M., Zowghi, D. & da Rimini, F. User satisfaction and system success: an empirical exploration of user involvement in software development. Empir Software Eng 22, 2339–2372 (2017). https://doi.org/10.1007/s10664-016-9465-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-016-9465-1