Accountability in a computerized society

Abstract

This essay warns of eroding accountability in computerized societies. It argues that assumptions about computing and features of situations in which computers are produced create barriers to accountability. Drawing on philosophical analyses of moral blame and responsibility, four barriers are identified: 1) the problem of many hands, 2) the problem of bugs, 3) blaming the computer, and 4) software ownership without liability. The paper concludes with ideas on how to reverse this trend.

This is a preview of subscription content, access via your institution.

References

  1. [1]

    Borning A (1987) Computer System Reliability and Nuclear War.Communications of the ACM 30, 2: 112–131.

    Article  Google Scholar 

  2. [2]

    Leveson N (1986) Software Safety: Why, What, and How.Computing Surveys 18, 2: 125–163.

    Article  Google Scholar 

  3. [3]

    Leveson N, and Turner C (1993) An Investigation of the Therac-25 Accidents.Computer 26, 7: 18–41.

    Article  Google Scholar 

  4. [4]

    Littlewood B and Strigini L (1992) The Risks of Software.Scientific American, November: 62–75.

  5. [5]

    Neumann P G. (monthly column) Inside Risks.Communications of the ACM.

  6. [6]

    Parnas D, Schouwen J and Kwan S P (1990) Evaluation of Safety-Critical Software.Communications of the ACM 33, 6: 636–648.

    Article  Google Scholar 

  7. [7]

    Feinberg, J (1985) Sua Culpa, In: Johnson D G and Snapper J, eds.Ethical Issues in the Use of Computers. Wadsworth, Belmont.

    Google Scholar 

  8. [8]

    Thompson D (1987)Political Ethics and Public Office. Harvard University Press, Cambridge.

    Google Scholar 

  9. [9]

    De George R (1991) Ethical Responsibilities of Engineers in Large Organizations: The Pinto Case, In: May L and Hoffman S, eds.Collective Responsibility. Rowman and Littlefield: 151–166.

  10. [10]

    Feinberg J. (1970) Collective Responsibility. In Feinberg J, ed.Doing and Deserving, Princeton University Press, Princeton.

    Google Scholar 

  11. [11]

    Ladd J. (1989) Computers and Moral Responsibility: A Framework for an Ethical Analysis. In Gould C, ed.The Information Web: Ethical and Social Implications of Computer Networking. Westview Press, Boulder.

    Google Scholar 

  12. [12]

    Thompson D (1987) The Moral Responsibility of Many Hands. In:Political Ethics and Public Office, Harvard University Press, Cambridge: 46–60.

    Google Scholar 

  13. [13]

    Velasquez M (1991) Why Corporations Are Not Morally Responsible for Anything They Do. In: May L and Hoffman S, eds.Collective Responsibility. Rowman and Littlefield: 111–131.

  14. [14]

    Johnson D G and Mulvey J M (1993)Computer Decisions: Ethical Issues of Responsibility and Bias. Statistics and Operations Research Series, Princeton University, SOR-93-11.

  15. [15]

    Weizenbaum J (1972) On the Impact of the Computer on Society.Science 176, 12: 609–614.

    Article  Google Scholar 

  16. [16]

    Jacky J (1989)Safety-Critical Computing: Hazards, Practices, Standards and Regulations (Unpublished Manuscript). University of Washington.

  17. [17]

    Corbato F J (1991) On Building Systems That Will Fail.Communications of the ACM 34, 9: 73–81.

    Article  Google Scholar 

  18. [18]

    Smith, B C (1985)The Limits of Correctness. Center for the Study of Language and Information, Stanford University, CSLI-85-35.

  19. [19]

    McCullough D (1972)The Great Bridge. Simon & Schuster, New York.

    Google Scholar 

  20. [20]

    Turkle, Sherry (1984)The Second Self. Simon & Schuster, New York.

    Google Scholar 

  21. [21]

    Friedman B and Millett L (1995) “It’s the computer’s fault—reasoning about computers as moral agents.” Paper presented at CHI ’95, New York.

  22. [22]

    Fitzgerald S (1992) Hospital Computer Predicts Patients’ Chance of Survival.The Miami Herald, Sunday, July 19.

  23. [23]

    Snapper J W (1985) Responsibility for Computer-Based Errors.Metaphilosophy 16: 289–295.

    Google Scholar 

  24. [24]

    Friedman B and Kahn P H (1992) Human Agency and Responsible Computing: Implications for Computer System Design.Journal of System Software 17, January: 7–14.

    Article  Google Scholar 

  25. [25]

    Nissenbaum H. Should I Copy My Neighbor’s Software? In: Johnson D G and Nissenbaum H (1995)Computers, Ethics, and Social Values. Prentice-Hall, Englewood.

    Google Scholar 

  26. [26]

    Samuelson P (1992)Adapting Intellectual Property Law to New Technologies: A Case Study on Computer Programs. National Research Council Report.

  27. [27]

    Stallman R M (1987) The GNU Manifesto.GNU Emacs Manual: 175–84.

  28. [28]

    Samuelson P (1993) Liability for Defective Information.Communications of the ACM 36, 1: 21–26.

    Article  Google Scholar 

  29. [29]

    Fried J P (1993) Maximum Terms for Two Youths in Red Hook Murder.New York Times, July 7: 3.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Helen Nissenbaum PhD.

Additional information

Several people have contributed generously to this work. Michael Davis, Deborah G. Johnson, Arthur Kuflik, Pamela Samuelson, Debra Satz, Richard De George, Larry May, and Dennis Thompson read drafts and made invaluable suggestions. Reviewers forScience and Engineering Ethics offered thorough and challenging commentary. An earlier version of the paper was presented at The American Philosophical Association, Eastern Division Meeting, December 1993, where audience comments and questions led to clarification of several key issues.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nissenbaum, H. Accountability in a computerized society. Sci Eng Ethics 2, 25–42 (1996). https://doi.org/10.1007/BF02639315

Download citation

Key Words

  • accountability
  • bugs
  • computer ethics
  • liability
  • moral responsibility
  • standard of care