Skip to main content

The post mortem paradox: a Delphi study of IT specialist perceptions

Abstract

While post mortem evaluation (PME) has long been advocated as a means of improving development practices by learning from IT project failures, few organizations conduct PMEs. The purpose of the study is to explain this discrepancy between theory and practice. This paper integrates findings from a Delphi study of what experienced practitioners perceive as the most important barriers to conducting PMEs with insights from organizational learning theory. The results suggest that there are critical tensions between development practices and learning contexts in many organizations, and adopting PMEs in these cases is likely to reinforce organizational learning dysfunctions rather than improve current development practices. Based on these findings, we argue that the PME literature has underestimated the limits to learning in most IT organizations and we propose to explore paradoxical thinking to help researchers frame continued inquiry into PME and to help managers overcome learning dysfunctions as they push for more widespread use of PMEs.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2

References

  1. Argyris C and Schoen DA (1996) Organizational Learning ii Theory, Method and Practice. Addison-Wesley, Reading, MA.

    Google Scholar 

  2. Benko C and Mcfarlan W (2003) Connecting the Dots: Aligning Projects with Objectives in Unpredictable Times. Harvard Business School Press, Boston, MA.

    Google Scholar 

  3. Birk A, Dingsoyr T and Stalhane T (2002) Post mortem: never leave a project without it. IEEE Software 19 (3), 43–45.

    Article  Google Scholar 

  4. Brancheau JC, Janz BD and Wetherbe JC (1996) Key issues in information systems management: 1994–95 sim delphi results. MIS Quarterly 20 (2), 225.

    Article  Google Scholar 

  5. Brancheau JC and Wetherbe JC (1987) Key issues in information systems management. MIS Quarterly 11 (1), 23.

    Article  Google Scholar 

  6. Brown AD and Jones MR (1998) Doomed to failure: narratives of inevitability and conspiracy in a failed is project. Organization Studies 19 (1), 73–88.

    Article  Google Scholar 

  7. Cameron KS (1986) Effectiveness as paradox: consensus and conflict in conceptions of organizational effectiveness. Management Science 32 (5), 539–553.

    Article  Google Scholar 

  8. Collier B, Demarco T and Fearey P (1996) A defined process for project postmortem review. IEEE Software 13 (4), 65–72.

    Article  Google Scholar 

  9. Dalkey N (1969) The Delphi Method: An Experimental Study of Group Opinion. Rand Corp, Santa Monica, CA.

    Google Scholar 

  10. Dalkey N and Helmer O (1963) An experimental application of the delphi method to the use of experts. Management Science 9, 458–467.

    Article  Google Scholar 

  11. Douglas M (1992) Rish and Blame: Essays in Cultural Theory. Routledge, London, UK.

    Book  Google Scholar 

  12. Ellmer E (1999) Collaborative technologies & organizational learning. European Journal of Information Systems 8 (2), 158.

    Article  Google Scholar 

  13. Fiol CM and Lyles MA (1985) Organizational learning. Academy of Management. The Academy of Management Review 10 (4), 803.

    Google Scholar 

  14. Glass RL (2002) Project retrospectives, and why they never happen. IEEE Software 19 (5), 111–112.

    Article  Google Scholar 

  15. Gupta UM and Clarke ER (1996) Theory and applications of the delphi technique: a bibliography (1975–1994). Technological Forecasting and Social Change 53 (2), 185–211.

    Article  Google Scholar 

  16. Gwillim D, Dovey K and Wieder B (2005) The politics of post-implementation reviews. Information Systems Journal 15 (4), 307–319.

    Article  Google Scholar 

  17. Johnson J (1999) Turning chaos into success. Software Magazine 19 (3), 30–34, 39.

    Google Scholar 

  18. Jones MM and Mclean ER (1970) Management problems in large-scale software development projects. Industrial Management Review 11 (3), 1.

    Google Scholar 

  19. Kalle L and Gregory MR (2006) Information system development agility as organizational learning. European Journal of Information Systems 15 (2), 183.

    Article  Google Scholar 

  20. Keil M, Tiwana A and Bush A (2002) Reconciling user and project manager perceptions of it project risk: a delphi study. Information Systems Journal 12 (2), 103–119.

    Article  Google Scholar 

  21. Kerth NL (2000) Project Retrospectives: A Handbook for Team Reviews. Dorset House Publishing, New York.

    Google Scholar 

  22. Kumar K (1990) Post implementation evaluation of computer-based information systems: current practices. Communications of ACM 33 (2), 203–212.

    Article  Google Scholar 

  23. Kwak YH and Stoddard J (2004) Project risk management: lessons learned from software development environment. Technovation 24 (11), 915–920.

    Article  Google Scholar 

  24. Lewis MW (2000) Exploring paradox: toward a more comprehensive guide. Academy of Management Review 25 (4), 760–776.

    Google Scholar 

  25. Linstone HA and Turoff M (1975) Delphi Method. Techniques and Applications. Addison-Wesley, Reading, MA.

    Google Scholar 

  26. Love Z (2001) Transforming failure into success through organisational learning: an analysis of a manufacturing information system. European Journal of Information Systems 10 (1), 55.

    Article  Google Scholar 

  27. Lyytinen K and Robey D (1999) Learning failure in information systems development. Information Systems Journal 9 (2), 85–101.

    Article  Google Scholar 

  28. March JG, Sproull LS and Tamuz M (1991) Learning from samples of one or fewer. Organization Science: A Journal of the Institute of Management Sciences 2 (1), 58.

    Article  Google Scholar 

  29. Mason J (2004) Qualitative Researching. Sage Publications, Thousand Oaks, CA.

    Google Scholar 

  30. Mclellan K and Marcolin B (1994) Information technology outsourcing. Business Quarterly 59, 95–104.

    Google Scholar 

  31. Nolan AJ (1999) Learning from success. IEEE Software 16 (1), 97–105.

    Article  Google Scholar 

  32. Norris G (1996) Post-investment appraisal. In Investing in Information Systems (WILLCOCKS L, Ed), pp 193–223, Chapman and Hall, London, UK.

    Google Scholar 

  33. Panagiotidis P and Edwards JS (2001) Organisational learning – a critical systems thinking discipline. European Journal of Information Systems 10 (3), 135.

    Article  Google Scholar 

  34. Pedersen K (2004) Barriers for post mortem evaluations in system development. In UKAIS 2004 Conference, Glasgow, UK.

  35. Pedersen K (2005) Managing learning in systems development projects. PhD Thesis, Computer Science Department, Aalborg Univesity, Denmark.

  36. Petroski H (1994) Design paradigms: Case Histories of Error and Judgment in Engineering. Cambridge University Press, Cambridge.

    Book  Google Scholar 

  37. Poole MS and Van De Ven AH (1989) Using paradox to build management and organization theories. Academy of Management Review 14 (4), 562–578.

    Google Scholar 

  38. Popper M and Lipshitz R (1998) Organizational learning mechanisms. Journal of Applied Behavioral Science 34 (2), 161–179.

    Article  Google Scholar 

  39. Power M (2004) The Risk Management of Everything: Rethinking The Politics of Uncertainty. Demons, London, UK.

    Google Scholar 

  40. Robey D, Boudreau M-C and Rose GM (2000) Information technology and organizational learning: a review and assessment of research. Accounting, Management and Information Technologies 10, 125–155.

    Article  Google Scholar 

  41. Schmidt RC (1997) Managing delphi surveys using nonparametric statistical techniques. Decision Sciences 28 (3), 763–774.

    Article  Google Scholar 

  42. Scott SV and Walsham G (2005) Reconceptualizing and managing reputation risk in the knowledge economy: toward reputable action. Organization Science 16 (3), 308–322.

    Article  Google Scholar 

  43. Seddon P, Graeser V and Willcocks L (2002) Measuring organizational is effectiveness: an overview and update of senior management perspectives. Data Base for Advances in Information Systems 33, 11–27.

    Article  Google Scholar 

  44. Smith WK and Tushman ML (2005) Managing strategic contradictions: a top management model for managing innovation streams. Organization science 16 (5), 522–536.

    Article  Google Scholar 

  45. Smithson S and Hirschheim R (1998) Analysing information systems evaluation: another look at an old problem. European Journal of Information Systems 7 (3), 158–174.

    Article  Google Scholar 

  46. Snyder WM and Cummings TG (1998) Organization learning disorders: conceptual model and intervention hypotheses. Human Relations 51 (7), 873.

    Google Scholar 

  47. Statz J (1999) Leverage your lessons. IEEE Software 16 (2), 30–32.

    Article  Google Scholar 

  48. Taylor RE and Meinhardt DJ (1985) Defining computer information needs for small business: a delphi method. Journal of Small Business Management 23, 3–9.

    Google Scholar 

  49. Tiedeman MJ (1990) Post-mortems – methodology and experiences. IEEE Journal on Selected Areas in Communications 8 (2), 176–180.

    Article  Google Scholar 

  50. Turoff M and Hiltz SR (1996) Computer-based delphi processes In Gazing into the Oracle: The Delphi Method and its Application to Social Policy and Public Health (ADLER M and ZIGLIO E, Eds), Kingsley Publishers, London, UK.

    Google Scholar 

  51. Vaughan D (1996) The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA. University of Chicago Press, Chicago, IL.

    Google Scholar 

  52. Wastell DG (1999) Learning dysfunctions in information systems development: overcoming the social defenses with transitional objects. MIS Quarterly 23 (4), 581.

    Article  Google Scholar 

  53. Zedtwitz M (2002) Organizational learning through post project reviews in r&d. R&D Management 32 (3), 255–268.

    Article  Google Scholar 

  54. Zollo M and Winter SG (2002) Deliberate learning and the evolution of dynamic capabilities. Organization Science INFORMS: Institute for Operations Research 13 (3), 339.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Vijay Kasi.

Appendix A

Appendix A

Table A1

Table a1 Comprehensive list of PME barriers

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Kasi, V., Keil, M., Mathiassen, L. et al. The post mortem paradox: a Delphi study of IT specialist perceptions. Eur J Inf Syst 17, 62–78 (2008). https://doi.org/10.1057/palgrave.ejis.3000727

Download citation

Keywords

  • PME
  • organizational learning
  • paradoxical thinking
  • Delphi study
  • post mortem paradox