Abstract
Determining the contribution of information system (IS) projects is a difficult endeavour. This research presents a framework for evaluating and measuring IS project performance that seek to address this problem while moving away from the reliance on the traditional assessment method (time, cost, specifications). When IS projects fail or succeed is difficult to determine what conditions enabled the outcome. However, if we are to develop a higher level of IS project management competence we need frameworks that can help in assessing IS project outcomes while critically reflecting and learning from IS project management practices. The Project Performance Scorecard (PPS) recommended here is one framework that can help to advance our understanding IS project management and evaluation approaches. This framework integrates concepts from existing models of IS success, the Balanced Scorecard and project management practices. A case study approach is used to illustrate the use of the PPS, and offers insights into how it can be useful in assessing the performance of IS projects.
Similar content being viewed by others
References
Agarwal, N., & Rathod, U. (2006). Defining ‘success’ for software projects: An exploratory revelation. International Journal of Project Management, 24(4), 358–370.
Alleman, G. B. (2005). Agile project management for IT projects. In Y. H. K. Elias, G. Carayannis, & F. T. Anbari (Eds.), The story of managing projects: An interdisciplinary approach (p. 373). Westport: Greenwood Press.
Atkinson, R. (1999). Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. International Journal of Project Management, 17(6), 337–342.
Baccarini, D. (1999). The logical framework method for defining project success. Project Management Journal, 30(4), 25–32.
Bandi, R. K., Vaishnavi, V. K., & Turk, D. E. (2003). Predicting maintenance performance using object-oriented design complexity metrics. IEEE Transactions on Software Engineering, 29(1), 77–87.
Barclay, C. (2007). Towards an integrated framework for project performance IRMA 2007. Vancouver: IGI Publishing.
Basili, V. R., Caldiera, G., & Rombach, H. D. (1994). The goal question metric approach. Encyclopedia of software engineering. Hoboken: Wiley.
Bourne, M., Franco, M., & Wilkes, J. (2003). Corporate performance measurement. Measuring Business Excellence, 7, 15–21.
Brancheau, J. C., Janz, B. D., et al. (1996). Key issues in information systems management: 1994–95 SIM Delphi results. MIS Quarterly, 20(2), 225–242.
Brancheau, J. C., & Wetherbe, J. C. (1987). Key issues in information systems management. MIS Quarterly, 11(1), 23–44.
Brandon, D. (2006). Project management for modern information systems. Hershey: IRM.
Bryde, D. J. (2003). Modelling project management performance. International Journal of Quality & Reliability Management, 20(2), 229–254.
Bryde, D. J. (2005). Methods for managing different perspectives of project success. British Journal of Management, 16(2), 119–131.
Brynjolfsson, E. (1993). The productivity paradox of information technology. Communications of the ACM, 36(12), 67–77.
Cohen, D. J., & Graham, R. J. (2001). The project manager’s MBA: How to translate project decisions into business success. Hoboken: Wiley.
Collins, A., & Baccarini, D. (2004). Project success - A survey. Journal of Construction Research, 5(2), 211–231.
Cooke-Davies, T. (2002). The real success factors on projects. International Journal of Project Management, 20(3), 185–190.
DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60, 36p.
DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: ten year update. Journal of Management Information Systems, 19(4), 9–30.
deWit, A. (1988). Measurement of project management success. International Journal of Project Management, 6(3), 164–170.
Duggan, E. W., & Reichgelt, H. (2006). Measuring information systems delivery quality. Hershey: Idea Group Publishing.
Dvir, D., Lipovetsky, S., Shenhar, A. J., & Tishler, A. (2003). What is really important for project success? A refined, multivariate, comprehensive analysis. International Journal of Management and Decision Making, 4(4), 382–404.
Ewusi-Mensah, K., & Przasnyski, Z. H. (1991). On information systems project abandonment: An exploratory study of organisational practices. MIS Quarterly, 15(1), 67–89.
Fitzgerald, G. (1998). Evaluating information systems project: A multidimensional approach. Journal of Information Technology, 13(1), 15–27.
Freeman, M., & Beale, P. (1992). Measuring project success. Project Management Journal, 23(1), 8–17.
Garvin, D. A. (1987). Competing on the eight dimensions of quality. Harvard Business Review, pp 101–109.
Glass, R. L. (1999). Evolving a new theory of project success. Communications of the ACM, 42(11), 17–19.
Glass, R. L. (2005). IT failure rates—70% or 10–15%. IEEE Software, 22(3), 112, 110–111.
Glass, R. L. (2006). The Standish report: Does it really describe a software crisis. Communications of the ACM, 49(8), 15–16.
ISO/IEC (2001). ISO/IEC standard 9126-1—Software Engineering—Product Quality—Part 1: quality model.
Jugdev, K., & Muller, R. (2005). A retrospective look at our evolving understanding of project success. Project Management Journal, 36(4), 19–31.
Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard: measures that drive performance. Harvard Business Review, 70(1), 71–79.
Kaplan, R. S., & Norton, D. P. (1996a). Balanced scorecard: Translating strategy into action. Boston: Harvard Business School Press.
Kaplan, R. S., & Norton, D. P. (1996b). Linking the balance scorecard to strategy. California Management Review, 39(1), 53–79.
Kleist, V. F. (2003). An approach to evaluating E-business information systems projects. Information Systems Frontiers, 5(3), 249–263.
KPMG. (2005). Global IT project management survey, KPMG information risk management, (pp 1–35).
Kumar, R. L. (2003). Understanding the value of information technology enabled responsiveness. EJISE, 1(1).
Lee, A. S. (1989). A scientific methodology for MIS case studies. MIS Quarterly, 13(1), 32.
Leonard-Barton, D. (1990). A dual methodology for case studies. synergistic use of longitudinal single site with replicated multiple sites. Organisation Science, 1(1), 248–266.
Lim, C. S., & Mohamed, M. Z. (1999). Criteria of project success: An exploratory re-examination. International Journal of Project Management, 17(4), 243–248.
Lin, C., & Pervan, G. (2001). Review of IS/IT investment evaluation and benefits management issues, problems, and processes. New York: Wiley.
Lu, J. (2003). A model for evaluating e-commerce based on cost/benefit and customer satisfaction. Information Systems Frontiers, 5(3), 265–277.
Lyytinen, K. (1988). Expectation failure concept and systems analysts’ view of information system failures: results of an exploratory study. Information & Management, 14(1), 45–56.
Markus, M. L., & Keil, M. (1994). If we build it they will come: Designing information systems that users want to use. Sloan Management Review, 35(4), 11–25.
Martinsons, M., Davison, R., & Tse, D. (1999). The balanced scorecard: A foundation for the strategic management of information systems. Decision Support Systems, 25, 71–88.
Morris, P. W. G., & Hough, G. H. (1987). The anatomy of major projects: A study of the reality of project management. Hoboken: Wiley.
Nelson, R. R. (2005). Project retrospectives: Evaluating project success, failure and everything in between. MIS Quarterly Executive, 4(3), 361–372.
Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management, 42, 15–29.
Palvia, S. C., Sharma, R. S., & Conrath, D. W. (2001). A socio-technical framework for quality assessment of computer information systems. Industrial Management & Data Systems, 101(5), 237–251.
Pinto, J. K., & Slevin, D. P. (1987). Critical success factors in successful project implementation. IEEE Transactions Engineering Management, 34(1), 22–27.
Rai, A., Lang, S. S., & Welker, R. B. (2002). Assessing the validity of IS success models: An empirical testand theoretical analysis. Information Systems Research, 13(1), 50–69.
Ravichandran, R., & Rai, A. (1994). The dimensions and correlates of system development quality. In Annual SIG Computer Personnel Research Conference on Reinventing IS, Virginia.
Remenyi, D. (2004). The elusive nature of delivering benefits from IT investment. EJISE, 3(1).
Robey, D., & Markus, M. L. (1984). Rituals in information system design. MIS Quarterly, 8(1), 5.
Saaty, T. L. (1987). Risk—Its priority and probability: The analytic hierarchy process. Risk Analysis, 7(2), 159–172.
Saaty, T. L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48(1), 9–26.
Seddon, P. B. (1997). A respecification and extension of the Delone and McLean model of IS success. Information Systems Research, 8(3), 240–253.
Seddon, P. B., Staples, S., Patnayakuni, R., & Bowtell, M. (1999). Dimensions of information systems success. CAIS, 2(20), 5.
Sheng, H., Nah, F. F.-H., & Siau, K. (2005). Strategic implications of mobile technology: A case study using Value-Focused Thinking. Journal of Strategic Information Systems, 14, 269–290.
Shenhar, A. J., Dvir, D., et al. (2001). Project success: A multi-dimensional strategic perspective. Long Range Planning, 34, 699–725.
Shenhar, A. J., & Levy, O. (1997). Mapping the dimensions of project success. Project Management Journal, 28(2), 5–13.
Shenhar, A. J., Tisler, A., Dvir, D., Lipostevsky, S., & Lechler, T. (2002). Refining the search for project success factors: A multivariate typographical approach. R & D Management, 32(2), 111–126.
Standish (1994). The chaos report. Boston: The Standish Group International.
Standish (2004). 2004 third quarter research report. Boston: The Standish Group International.
Teltumbde, A. (2000). A framework for evaluating ERP projects. International Journal of Production Research, 38(17), 4507–4520.
Wateridge, J. (1998). How can IS/IT projects be measured for success. International Journal of Project Management, 16(1), 59–63.
White, D., & Fortune, J. (2002). Current practice in project management—An empirical study. International Journal of Project Management, 20, 1–11.
Winter, M., Smith, C., Morris, P., & Cicmil, S. (2006). Directions for future research in project management: The main findings of a UK government-funded research network. International Journal of Project Management, 24, 638–649.
Yin, R. K. (1994). Case study research: Design and methods. Beverly Hills: Sage Publishing.
Yin, R. K. (2003). Case study research: Design and methods. Beverly Hills: Sage Publishing.
Zayed, T., Amer, M., and Pan, J. (2007). Assessing risk and uncertainty inherent in Chinese highway projects using AHP. International Journal of Project Management. (in press) DOI 10:1016/j.ijproman.2007.05.012.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Barclay, C. Towards an integrated measurement of IS project performance: The project performance scorecard. Inf Syst Front 10, 331–345 (2008). https://doi.org/10.1007/s10796-008-9083-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10796-008-9083-6