Skip to main content

The post mortem paradox: a Delphi study of IT specialist perceptions

Abstract

While post mortem evaluation (PME) has long been advocated as a means of improving development practices by learning from IT project failures, few organizations conduct PMEs. The purpose of the study is to explain this discrepancy between theory and practice. This paper integrates findings from a Delphi study of what experienced practitioners perceive as the most important barriers to conducting PMEs with insights from organizational learning theory. The results suggest that there are critical tensions between development practices and learning contexts in many organizations, and adopting PMEs in these cases is likely to reinforce organizational learning dysfunctions rather than improve current development practices. Based on these findings, we argue that the PME literature has underestimated the limits to learning in most IT organizations and we propose to explore paradoxical thinking to help researchers frame continued inquiry into PME and to help managers overcome learning dysfunctions as they push for more widespread use of PMEs.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2

References

  • Argyris C and Schoen DA (1996) Organizational Learning ii Theory, Method and Practice. Addison-Wesley, Reading, MA.

    Google Scholar 

  • Benko C and Mcfarlan W (2003) Connecting the Dots: Aligning Projects with Objectives in Unpredictable Times. Harvard Business School Press, Boston, MA.

    Google Scholar 

  • Birk A, Dingsoyr T and Stalhane T (2002) Post mortem: never leave a project without it. IEEE Software 19 (3), 43–45.

    Article  Google Scholar 

  • Brancheau JC, Janz BD and Wetherbe JC (1996) Key issues in information systems management: 1994–95 sim delphi results. MIS Quarterly 20 (2), 225.

    Article  Google Scholar 

  • Brancheau JC and Wetherbe JC (1987) Key issues in information systems management. MIS Quarterly 11 (1), 23.

    Article  Google Scholar 

  • Brown AD and Jones MR (1998) Doomed to failure: narratives of inevitability and conspiracy in a failed is project. Organization Studies 19 (1), 73–88.

    Article  Google Scholar 

  • Cameron KS (1986) Effectiveness as paradox: consensus and conflict in conceptions of organizational effectiveness. Management Science 32 (5), 539–553.

    Article  Google Scholar 

  • Collier B, Demarco T and Fearey P (1996) A defined process for project postmortem review. IEEE Software 13 (4), 65–72.

    Article  Google Scholar 

  • Dalkey N (1969) The Delphi Method: An Experimental Study of Group Opinion. Rand Corp, Santa Monica, CA.

    Google Scholar 

  • Dalkey N and Helmer O (1963) An experimental application of the delphi method to the use of experts. Management Science 9, 458–467.

    Article  Google Scholar 

  • Douglas M (1992) Rish and Blame: Essays in Cultural Theory. Routledge, London, UK.

    Book  Google Scholar 

  • Ellmer E (1999) Collaborative technologies & organizational learning. European Journal of Information Systems 8 (2), 158.

    Article  Google Scholar 

  • Fiol CM and Lyles MA (1985) Organizational learning. Academy of Management. The Academy of Management Review 10 (4), 803.

    Google Scholar 

  • Glass RL (2002) Project retrospectives, and why they never happen. IEEE Software 19 (5), 111–112.

    Article  Google Scholar 

  • Gupta UM and Clarke ER (1996) Theory and applications of the delphi technique: a bibliography (1975–1994). Technological Forecasting and Social Change 53 (2), 185–211.

    Article  Google Scholar 

  • Gwillim D, Dovey K and Wieder B (2005) The politics of post-implementation reviews. Information Systems Journal 15 (4), 307–319.

    Article  Google Scholar 

  • Johnson J (1999) Turning chaos into success. Software Magazine 19 (3), 30–34, 39.

    Google Scholar 

  • Jones MM and Mclean ER (1970) Management problems in large-scale software development projects. Industrial Management Review 11 (3), 1.

    Google Scholar 

  • Kalle L and Gregory MR (2006) Information system development agility as organizational learning. European Journal of Information Systems 15 (2), 183.

    Article  Google Scholar 

  • Keil M, Tiwana A and Bush A (2002) Reconciling user and project manager perceptions of it project risk: a delphi study. Information Systems Journal 12 (2), 103–119.

    Article  Google Scholar 

  • Kerth NL (2000) Project Retrospectives: A Handbook for Team Reviews. Dorset House Publishing, New York.

    Google Scholar 

  • Kumar K (1990) Post implementation evaluation of computer-based information systems: current practices. Communications of ACM 33 (2), 203–212.

    Article  Google Scholar 

  • Kwak YH and Stoddard J (2004) Project risk management: lessons learned from software development environment. Technovation 24 (11), 915–920.

    Article  Google Scholar 

  • Lewis MW (2000) Exploring paradox: toward a more comprehensive guide. Academy of Management Review 25 (4), 760–776.

    Google Scholar 

  • Linstone HA and Turoff M (1975) Delphi Method. Techniques and Applications. Addison-Wesley, Reading, MA.

    Google Scholar 

  • Love Z (2001) Transforming failure into success through organisational learning: an analysis of a manufacturing information system. European Journal of Information Systems 10 (1), 55.

    Article  Google Scholar 

  • Lyytinen K and Robey D (1999) Learning failure in information systems development. Information Systems Journal 9 (2), 85–101.

    Article  Google Scholar 

  • March JG, Sproull LS and Tamuz M (1991) Learning from samples of one or fewer. Organization Science: A Journal of the Institute of Management Sciences 2 (1), 58.

    Article  Google Scholar 

  • Mason J (2004) Qualitative Researching. Sage Publications, Thousand Oaks, CA.

    Google Scholar 

  • Mclellan K and Marcolin B (1994) Information technology outsourcing. Business Quarterly 59, 95–104.

    Google Scholar 

  • Nolan AJ (1999) Learning from success. IEEE Software 16 (1), 97–105.

    Article  Google Scholar 

  • Norris G (1996) Post-investment appraisal. In Investing in Information Systems (WILLCOCKS L, Ed), pp 193–223, Chapman and Hall, London, UK.

    Google Scholar 

  • Panagiotidis P and Edwards JS (2001) Organisational learning – a critical systems thinking discipline. European Journal of Information Systems 10 (3), 135.

    Article  Google Scholar 

  • Pedersen K (2004) Barriers for post mortem evaluations in system development. In UKAIS 2004 Conference, Glasgow, UK.

  • Pedersen K (2005) Managing learning in systems development projects. PhD Thesis, Computer Science Department, Aalborg Univesity, Denmark.

  • Petroski H (1994) Design paradigms: Case Histories of Error and Judgment in Engineering. Cambridge University Press, Cambridge.

    Book  Google Scholar 

  • Poole MS and Van De Ven AH (1989) Using paradox to build management and organization theories. Academy of Management Review 14 (4), 562–578.

    Google Scholar 

  • Popper M and Lipshitz R (1998) Organizational learning mechanisms. Journal of Applied Behavioral Science 34 (2), 161–179.

    Article  Google Scholar 

  • Power M (2004) The Risk Management of Everything: Rethinking The Politics of Uncertainty. Demons, London, UK.

    Google Scholar 

  • Robey D, Boudreau M-C and Rose GM (2000) Information technology and organizational learning: a review and assessment of research. Accounting, Management and Information Technologies 10, 125–155.

    Article  Google Scholar 

  • Schmidt RC (1997) Managing delphi surveys using nonparametric statistical techniques. Decision Sciences 28 (3), 763–774.

    Article  Google Scholar 

  • Scott SV and Walsham G (2005) Reconceptualizing and managing reputation risk in the knowledge economy: toward reputable action. Organization Science 16 (3), 308–322.

    Article  Google Scholar 

  • Seddon P, Graeser V and Willcocks L (2002) Measuring organizational is effectiveness: an overview and update of senior management perspectives. Data Base for Advances in Information Systems 33, 11–27.

    Article  Google Scholar 

  • Smith WK and Tushman ML (2005) Managing strategic contradictions: a top management model for managing innovation streams. Organization science 16 (5), 522–536.

    Article  Google Scholar 

  • Smithson S and Hirschheim R (1998) Analysing information systems evaluation: another look at an old problem. European Journal of Information Systems 7 (3), 158–174.

    Article  Google Scholar 

  • Snyder WM and Cummings TG (1998) Organization learning disorders: conceptual model and intervention hypotheses. Human Relations 51 (7), 873.

    Google Scholar 

  • Statz J (1999) Leverage your lessons. IEEE Software 16 (2), 30–32.

    Article  Google Scholar 

  • Taylor RE and Meinhardt DJ (1985) Defining computer information needs for small business: a delphi method. Journal of Small Business Management 23, 3–9.

    Google Scholar 

  • Tiedeman MJ (1990) Post-mortems – methodology and experiences. IEEE Journal on Selected Areas in Communications 8 (2), 176–180.

    Article  Google Scholar 

  • Turoff M and Hiltz SR (1996) Computer-based delphi processes In Gazing into the Oracle: The Delphi Method and its Application to Social Policy and Public Health (ADLER M and ZIGLIO E, Eds), Kingsley Publishers, London, UK.

    Google Scholar 

  • Vaughan D (1996) The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA. University of Chicago Press, Chicago, IL.

    Google Scholar 

  • Wastell DG (1999) Learning dysfunctions in information systems development: overcoming the social defenses with transitional objects. MIS Quarterly 23 (4), 581.

    Article  Google Scholar 

  • Zedtwitz M (2002) Organizational learning through post project reviews in r&d. R&D Management 32 (3), 255–268.

    Article  Google Scholar 

  • Zollo M and Winter SG (2002) Deliberate learning and the evolution of dynamic capabilities. Organization Science INFORMS: Institute for Operations Research 13 (3), 339.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Vijay Kasi.

Appendix A

Appendix A

Table A1

Table a1 Comprehensive list of PME barriers

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Kasi, V., Keil, M., Mathiassen, L. et al. The post mortem paradox: a Delphi study of IT specialist perceptions. Eur J Inf Syst 17, 62–78 (2008). https://doi.org/10.1057/palgrave.ejis.3000727

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/palgrave.ejis.3000727

Keywords

  • PME
  • organizational learning
  • paradoxical thinking
  • Delphi study
  • post mortem paradox