Toward a Theory of Debiasing Software Development

  • Paul Ralph
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 93)

Abstract

Despite increasingly sophisticated programming languages, software developer training, testing tools, integrated development environments and project management techniques, software project failure, abandonment and overrun rates remain high. One way to address this is to focus on common systematic errors made by software project participants. In many cases, such errors are manifestations of cognitive biases. Consequently this paper proposes a theory of the role of cognitive biases in software development project success. The proposed theory posits that such errors are mutual properties of people and tasks; they may therefore be avoided by modifying the person-task system using specific sociotechnical interventions. The theory is illustrated using the case of planning poker, a task estimation technique designed to overcome anchoring bias.

Keywords

Design Science Software Engineering Theory Development Cognitive Bias Debiasing Heuristics Illusions 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Desmond, J.P.: System Integrators, Outsourcers Gain Again as IT Guards in-House Resources. Software Magazine, King Content Co. (2010), http://www.softwaremag.com/focus-areas/the-software-500-industry/featured-articles/system-integrators-outsourcers-gain-again-as-it-guards-in-house-resources/
  2. 2.
    Gartner: Gartner Says Worldwide IT Spending to Grow 5.3 Percent in 2010. Gartner, Inc. (2010), http://www.gartner.com/it/page.jsp?id=1339013
  3. 3.
    Ewusi-Mensah, K.: Software Development Failures. MIT Press (2003)Google Scholar
  4. 4.
    Standish Group: Chaos Summary 2009, Boston, MA, USA (2009)Google Scholar
  5. 5.
    Moløkken-Østvold, K.: A Review of Surveys on Software Effort Estimation. In: Proceedings of International Symposium on Empirical Software Engineering (ISESE 2003), pp. 223–230. IEEE, Rome (2003)Google Scholar
  6. 6.
    Ralph, P., Wand, Y.: A Proposal for a Formal Definition of the Design Concept. In: Lyytinen, K., Loucopoulos, P., Mylopoulos, J., Robinson, B. (eds.) Design Requirements Engineering. LNBIP, vol. 14, pp. 103–136. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Van de Ven, A.H.: Engaged Scholarship: A Guide for Organizational and Social Research. Oxford University Press, Oxford (2007)Google Scholar
  8. 8.
    Celdman, J.: Designing with Web Standards. New Riders, Berkely (2007)Google Scholar
  9. 9.
    Charette, R.N.: Why Software Fails. IEEE Spectrum Online (2005), http://www.spectrum.ieee.org/sep05/1685
  10. 10.
    Koppel, R., Metlay, J.P., Cohen, A., Abaluck, B., Localio, A.R., Kimmel, S.E., Strom, B.L.: Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors. Journal of the American Medical Association 293, 1197–1203 (2005)CrossRefGoogle Scholar
  11. 11.
    Keizer, G.: A Third of New Pcs Being Downgraded to XP, Says Metrics Researcher. Computerworld (2008)Google Scholar
  12. 12.
    Ralph, P.: Introducing an Empirical Model of Design. In: Proceedings of The 6th Mediterranean Conference on Information Systems, Limassol, Cyprus (2011)Google Scholar
  13. 13.
    Arnott, D.: Cognitive Biases and Decision Support Systems Development: A Design Science Approach. Information Systems Journal 16, 55–78 (2006)CrossRefGoogle Scholar
  14. 14.
    Samuelson, W., Zeckhauser, R.J.: Status Quo Bias in Decision Making. Journal of Risk and Uncertainty 1, 7–59 (1988)CrossRefGoogle Scholar
  15. 15.
    Gheorghiu, V., Molz, G., Pohl, R.: Suggestion and Illusion. In: Pohl, R. (ed.) Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory, pp. 399–421. Psychology Press, Hove (2004)Google Scholar
  16. 16.
    Kahneman, D., Tversky, A.: Prospect Theory: An Analysis of Decision under Risk. Econometrica 47, 263–291 (1979)CrossRefGoogle Scholar
  17. 17.
    Buehler, R., Griffin, D., Ross, M.: Exploring the “Planning Fallacy": Why People Underestimate Their Task Completion Times. Journal of Personality and Social Psychology 67, 366–381 (1994)CrossRefGoogle Scholar
  18. 18.
    Taylor, S.E., Kemeny, M.E., Aspinwall, L.G., Schneider, S.G., Rodriguez, R., Herbert, M.: Optimism, Coping, Psychological Distress, and High-Risk Sexual Behavior among Men at Risk for Acquired Immunodeficiency Syndrome (Aids). Journal of Personality and Social Psychology 63, 460–473 (1992)CrossRefGoogle Scholar
  19. 19.
    Tversky, A., Kahneman, D.: Judgment under Uncertainty: Heuristics and Biases. Science 185, 1124–1131 (1974)CrossRefGoogle Scholar
  20. 20.
    Parsons, J., Saunders, C.: Cognitive Heuristics in Software Engineering: Applying and Extending Anchoring and Adjustment to Artifact Reuse. IEEE Transactions on Software Engineering 30, 873–888 (2004)CrossRefGoogle Scholar
  21. 21.
    Pohl, R. (ed.): Cognitive Illusions. Psychology Press, East Sussex (2004)Google Scholar
  22. 22.
    Langer, E.: The Illusion of Control. Journal of Personality and Social Psychology 32, 311–328 (1975)CrossRefGoogle Scholar
  23. 23.
    Janis, I.L.: Groupthink: Psychological Studies of Policy Decisions and Fiascoes. Houghton Mifflin (1982)Google Scholar
  24. 24.
    Matlin, M.: Pollyanna Principle. In: Pohl, R.F. (ed.) Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory, pp. 255–272. Psychology Press, Hove (2004) Google Scholar
  25. 25.
    Fischoff, B.: Debiasing. In: Kahneman, D., Slovic, P., Tversky, A. (eds.) Judgment under Uncertainty: Heuristics and Biases. Cambridge University Press, Cambridge (1982)Google Scholar
  26. 26.
    Larrick, R.P.: Debiasing. In: Blackwell Handbook of Judgment and Decision Making, pp. 316–338. Blackwell Publishing Ltd. (2008)Google Scholar
  27. 27.
    Wilson, T., Centerbar, D., Brekke, N.: Mental Contamination and the Debiasing Problem. In: Gilovich, T., Griffin, D., Kahneman, D. (eds.) Heuristics and Biases: The Psychology of Intuitive Judgment, pp. 185–200. Cambridge University Press, Cambridge (2002)CrossRefGoogle Scholar
  28. 28.
    Pronin, E., Lin, D.Y., Ross, L.: The Bias Blind Spot: Perceptions of Bias in Self Versus Others. Personality and Social Psychology Bulletin 28, 369–381 (2002)CrossRefGoogle Scholar
  29. 29.
    Pronin, E.: Perception and Misperception of Bias in Human Judgment. Trends in Cognitive Sciences 11, 37–43 (2006)CrossRefGoogle Scholar
  30. 30.
    Jørgensen, M.: Identification of More Risks Can Lead to Increased Over-Optimism of and Over-Confidence in Software Development Effort Estimates. Information & Software Technology 52, 506–516 (2009)CrossRefGoogle Scholar
  31. 31.
    Thaler, R., Benartzi, S.: Save More Tomorrow: Using Behavioral Economics to Increase Employee Saving. The Journal of Political Economy 112, S164–S187 (2004)Google Scholar
  32. 32.
    Leventhal, L., Teasley, B., Rohlman, D., Instone, K.: Positive Test Bias in Software Testing among Professionals: A Review. In: Bass, L.J., Unger, C., Gornostaev, J. (eds.) EWHCI 1993. LNCS, vol. 753, pp. 210–218. Springer, Heidelberg (1993)CrossRefGoogle Scholar
  33. 33.
    Stacy, W., MacMillan, J.: Cognitive Bias in Software Engineering. Communications of ACM 38, 57–63 (1995)CrossRefGoogle Scholar
  34. 34.
    Siau, K., Wand, Y., Benbasat, I.: When Parents Need Not Have Children — Cognitive Biases in Information Modeling. In: Constantopoulos, P., Vassiliou, Y., Mylopoulos, J. (eds.) CAiSE 1996. LNCS, vol. 1080, pp. 402–420. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  35. 35.
    Busby, J., Payne, K.: The Situated Nature of Judgement in Engineering Design Planning. Journal of Engineering Design 9, 271–291 (1998)CrossRefGoogle Scholar
  36. 36.
    Snow, A.P., Keil, M., Wallace, L.: The Effects of Optimistic and Pessimistic Biasing on Software Project Status Reporting. Information & Management 44, 130–141 (2007)CrossRefGoogle Scholar
  37. 37.
    Browne, G.J., Ramesh, V.: Improving Information Requirements Determination: A Cognitive Perspective. Information & Management 39, 625–645 (2002)CrossRefGoogle Scholar
  38. 38.
    Schwaber, K.: Agile Project Management with Scrum. Microsoft Press (2004)Google Scholar
  39. 39.
    Nils, C.H.: An Empirical Study of Using Planning Poker for User Story Estimation. In: Proceedings of Agile 2006, pp. 23–34. IEEE Computer Society, Visegrád (2006)Google Scholar
  40. 40.
    Flyvbjerg, B.: From Nobel Prize to Project Management: Getting Risks Right. Project Management Journal 37, 5 (2006)Google Scholar
  41. 41.
    Lovallo, D., Kahneman, D.: Delusions of Success. Harvard Business Review 81, 56–63 (2003)Google Scholar
  42. 42.
    Molokken, K., Jorgensen, M.: A Review of Software Surveys on Software Effort Estimation. In: Proceedings of International Symposium on Empirical Software Engineering, pp. 223–230. ACM-IEEE, Rome, Italy (2003)Google Scholar
  43. 43.
    Hevner, A.R., March, S.T., Park, J., Ram, S.: Design Science in Information Systems Research. MIS Quarterly 28, 75–105 (2004)Google Scholar
  44. 44.
    Hevner, A., Chatterjee, S.: Design Research in Information Systems: Theory and Practice. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  45. 45.
    Sein, M., Henfridsson, O., Purao, S., Rossi, M., Lindgren, R.: Action Design Research. MIS Quarterly 35 (2011)Google Scholar
  46. 46.
    Gilovich, T., Griffin, D., Kahneman, D. (eds.): Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge University Press, Cambridge (2002)Google Scholar
  47. 47.
    Newell, A., Simon, H.: Human Problem Solving. Prentice-Hall, Inc. (1972)Google Scholar
  48. 48.
    Stanovich, K.: What Intelligence Tests Miss: The Psychology of Rational Thought. Yale University Press, New Haven (2009)Google Scholar
  49. 49.
    Oliver, R.L.: A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions. Journal of Marketing Research 17, 460–469 (1980)CrossRefGoogle Scholar
  50. 50.
    Taylor, N.: Making Actuaries Less Human: Lessons from Behavioural Finance. The Staple Inn Actuarial Society Meeting (2000)Google Scholar
  51. 51.
    Standish Group: Chaos Database: Chaos Surveys Conducted from 1994 to Fall 2004 (2006)Google Scholar
  52. 52.
    Hartmann, D.: Interview: Jim Johnson of the Standish Group. InfoQueue (2006)Google Scholar
  53. 53.
    Poses, R.M., Anthony, M.: Availability, Wishful Thinking, and Physicians’ Diagnostic Judgments for Patients with Suspected Bacteremia. Medical Decision Making 11, 159–168 (1991)CrossRefGoogle Scholar
  54. 54.
    Armor, D., Taylor, S.: When Predictions Fail: The Dilemma of Unrealistic Optimism. In: Gilovich, T., Griffin, D., Kahneman, D. (eds.) Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge University Press, Cambridge (2002)Google Scholar
  55. 55.
    Colman, A.: A Dictionary of Psychology. Oxford University Press, Oxford (2009)Google Scholar
  56. 56.
    VandenBos, G. (ed.): APA Dictionary of Psychology. American Psychological Association, Washington, DC, USA (2007)Google Scholar
  57. 57.
    Todd, P.M., Gigerenzer, G.: Environments That Make Us Smart: Ecological Rationality. Current Directions in Psychological Science 16, 167–171 (2007)CrossRefGoogle Scholar
  58. 58.
    Renner, C.H.: Validity Effect. In: Pohl, R.F. (ed.) Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory, pp. 201–213. Psychology Press, Hove (2004)Google Scholar
  59. 59.
    Jost, J.T., Banaji, M.R.: The Role of Stereotyping in System-Justification and the Production of False Consciousness. British Journal of Social Psychology 33, 1–27 (1994)CrossRefGoogle Scholar
  60. 60.
    Bornstein, R., Carver-Lemley, C.: Mere Exposure Effect. In: Pohl, R.F. (ed.) Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory, pp. 215–234. Psychology Press, Hove (2004)Google Scholar
  61. 61.
    Shore, B.: Systematic Biases and Culture in Project Failures. Project Management Journal 39, 5–16 (2008)CrossRefGoogle Scholar
  62. 62.
    Omer, H., Alon, N.: The Continuity Principle: A Unified Approach to Disaster and Trauma. American Journal of Community Psychology 22, 273–287 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Paul Ralph
    • 1
  1. 1.Lancaster UniversityLancasterUK

Personalised recommendations