Skip to main content
SpringerLink
Log in

An Ethical Inquiry of the Effect of Cockpit Automation on the Responsibilities of Airline Pilots: Dissonance or Meaningful Control?

  • Original Paper
  • Published:
Journal of Business Ethics Aims and scope Submit manuscript

Abstract

Airline pilots are attributed ultimate responsibility and final authority over their aircraft to ensure the safety and well-being of all its occupants. Yet, with the advent of automation technologies, a dissonance has emerged in that pilots have lost their actual decision-making authority as well as their ability to act in an adequate fashion towards meeting their responsibilities when unexpected circumstances or emergencies occur. Across the literature in human factor studies, we show how automated algorithmic technologies have wrestled control away from airline pilots. The inter-related ethical and pragmatic consequences resulting from such a dissonance are presented. This includes pilots serving as ‘legal sponges’, being caught in ‘moral crumple zones’, as well as real impacts on safety and well-being of passengers and crew. One potential avenue presented in the literature involves a redistribution of responsibility towards all agential entities (pilots and technology) within the cockpit system. We discuss the possible pitfalls to this approach, including the reinforcement of pilot skills erosion as well as technology’s own limitations in adaptively dealing with unexpected circumstances. We justify our position by showing how algorithmic (AI) technology’s representational frames mask or severely limit the pilots’ natural reflex to ‘practically cope’ and gain an ‘optimal grip’ with a given context/situation at hand. Here, we present the concept of relevant or meaningful control, which pilots, when allowed to develop their natural brain-body reflexes, gain across gradual mastery of the aircraft. Finally, we propose a socio-technical design approach in which pilots can re-appropriate full human meaningful control of the aircraft, thus allowing them to meet their attributed responsibilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abbink, D. A., & Mulder, M. (2009). Exploring the dimensions of haptic feedback support in manual control. Journal of Computing and Information Science in Engineering, 9(1), 011006-1–011006-9.

    Google Scholar 

  • Abbink, D. A., Mulder, M., & Boer, E. R. (2012). Haptic shared control: Smoothly shifting control authority? Cognition Technology and Work, 14, 19–28.

    Google Scholar 

  • Adams, R. J., & Ericsson, A. E. (2000). Introduction to cognitive processes of expert pilots. Journal of Human Performance in Extreme Environments, 5(1), 44–62. https://doi.org/10.7771/2327-2937.1006.

    Article  Google Scholar 

  • Amalberti, R. (1998). Automation in aviation: A human factors perspective. In J. A. Wise, V. D. Hopkin, & D. J. Garland (Eds.), Handbook of aviation human factors (pp. 173–192). Boca Raton, FL: CRC Press.

    Google Scholar 

  • Angier, T. (2010). Techné in Aristotle’s ethics: Crafting the moral life. New York: Continuum International.

    Google Scholar 

  • Aristotle. (1955). The ethics of Aristotle (trans: Thomson, J. A. K.). Harmondsworth: Penguin.

  • Bainbridge, L. (1983). Ironies of automation. Automatica, 19, 775–779.

    Google Scholar 

  • Bailey, N., & Scerbo, M. (2008). Automation induced complacency for monitoring highly reliable systems; the role of task complexity, system experience, and operator trust. Theoretical Issues in Ergonomics Science, 8(4), 321–348.

    Google Scholar 

  • Beaty, D. (1995). The naked pilot: The human factor in aircraft accidents. London: Airlife.

    Google Scholar 

  • Benton, P. A. (1995). Ethics in aviation. Journal of Aviation/Aerospace Education and Research, 5(2), 22–25.

    Google Scholar 

  • Billings, C. E., Grayson, R., Hecht, W., & Curry, R. (1980). A study of midair collisions in US terminal airspace. NASA Aviation Safety Reporting System: Quarterly Report No. 11 (NASA TM81225).

  • Bilz, K., & Nadler, J. (2009). Law, psychology, and morality. In D. M. Bartels, C. W. Bauman, L. J. Skitka & D. L. Medin (Eds), The psychology of learning and motivation. Moral judgment and decision making (Vol. 50, pp. 101–131). Elsevier Academic Press. https://doi.org/10.1016/S0079-7421(08)00403-9.

  • Boden, M. A. (2010). Creativity and art: Three roads to surprise. Oxford: Oxford University Press.

    Google Scholar 

  • Boden, M. A. (2016). AI: Its nature and future. Oxford: Oxford University Press.

    Google Scholar 

  • Boudreau, M.-C., & Robey, D. (2005). Enacting integrated information technology: A human agency perspective. Organization Science, 16(1), 3–18. https://doi.org/10.1287/orsc.1040.0103.

    Article  Google Scholar 

  • Bourdieu, P. (1994). Practical reason. Cambridge: Polity.

    Google Scholar 

  • Bovens, M. (1998). The quest for responsibility. In Accountability and citizenship in complex organisations. Cambridge: Cambridge University Press.

  • CAR. (1988). Civil Aviation Regulation. https://lawlex.com.au/tempstore/consolidated/7460.pdf.

  • Carr, N. (2015). The glass cage—How our computers are changing us. New York: W.W. Norton and Company.

    Google Scholar 

  • Casner, S. M., & Schooler, J. (2014). Thoughts in flight: Automation use and pilots’ task-related and task-unrelated thought. Human Factors, 56(3), 433–422.

    Google Scholar 

  • Chanter, T. (2001). The problematic normative assumptions of Heidegger’s ontology. In N. Holland & P. Huntington (Eds.), Feminist interpretations of Martin Heidegger. New York: Routledge.

    Google Scholar 

  • Chemero, A. (2009). Radical embodied cognitive science. Cambridge, MA: The MIT Press.

    Google Scholar 

  • Chemero, A. (2013). Radical embodied cognitive science. Review of General Psychology, 17(2), 145–150.

    Google Scholar 

  • Chia, R. (2009). The nature of knowledge and knowing in the context of management learning, education and development. In S. J. Armstrong & C. V. Fukami (Eds.), The SAGE handbook of management learning, education and development (pp. 25–41). London: SAGE Publications Ltd. https://doi.org/10.4135/9780857021038.n2.

    Chapter  Google Scholar 

  • Chialastri, A. (2012). Automation in aviation. In F. Kongoli (Ed), Automation (pp. 79–102). InTech. https://doi.org/10.5772/49949.

  • Clark, A. (2001). Being there: Putting the brain, body, and world together again. Cambridge, MA: MIT Press.

    Google Scholar 

  • Coeckelbergh, M. (2009). Virtual moral agency, virtual moral responsibility. AI and Society, 24(2), 181–189.

    Google Scholar 

  • Coeckelbergh, M. (2019). Artificial intelligence, responsibility attribution, and a relational justification of explainability. Science and Engineering Ethics. https://doi.org/10.1007/s11948-019-00146-8.

    Article  Google Scholar 

  • Collins, H. (2010). Tacit and explicit knowledge. Chicago: University of Chicago Press.

    Google Scholar 

  • Cooling, J. E., & Herbers, P. V. (1983). Considerations in autopilot litigation. Journal of Air Law and Commerce, 48, 693–723.

    Google Scholar 

  • Croskerry, P. (2018). Adaptive expertise in medical decision making. Medical Teacher, 40(8), 803–808. https://doi.org/10.1080/0142159X.2018.1484898.

    Article  Google Scholar 

  • Cummings, M. L. (2006). Automation and accountability in decision support system interface design. The Journal of Technology Studies, 32(1), 23–31.

    Google Scholar 

  • Dalcher, D. (2007). Why the pilot cannot be blamed: A cautionary note about excessive reliance on technology. International Journal of Risk Assessment and Management, 7(3), 350–366.

    Google Scholar 

  • Dao, A.-Q., et al. (2009). The impact of automation assisted aircraft separation on situation awareness. In M. J. Smith & G. Salvendy (Eds.), Human interface, Part II, HCII 2009, LNCS 5618 (pp. 738–747). Berlin: Springer.

    Google Scholar 

  • Dehais, F. et al. (2015). ‘Automation Surprise’ in aviation: Real-time solutions. In CHI 2015, April 18–23, Seoul, Republic of Korea (pp. 2525–2534).

  • Dekker, S. (2006). The field guide to understanding human error. Burlington: Ashgate.

    Google Scholar 

  • Dekker, S. (2010). Pilots, controllers and mechanics on trial: Cases, concerns and countermeasures. International Journal of Applied Aviation Studies, 10(1), 31–49.

    Google Scholar 

  • Dekker, S. (2011). The criminalization of human error in aviation and healthcare: A review. Safety Science, 49, 121–127.

    Google Scholar 

  • Deming, W. E. (1986). Out of the crisis. Cambridge, MA: MIT/CAES.

    Google Scholar 

  • Doherty, G., & Massink, M. (1999). Continuous interaction and human control. In J. Alty (Ed), Proceedings of the XVIII European annual conference on human decision making and manual control (pp. 80–96).

  • Dreyfus, H. L. (1972). What computers can’t do. New York, NY: MIT Press.

    Google Scholar 

  • Dreyfus, H. L. (1991). Being-in-the-world: A commentary on Heidegger’s being and time. Cambridge, MA: MIT Press.

    Google Scholar 

  • Dreyfus, H. L. (1992). What computers still can’t do: A critique of artificial reason. London: MIT Press.

    Google Scholar 

  • Dreyfus, H. L. (2002). Refocusing the question: can there be skillful coping without propositional representations or brain representations? Phenomenology and the Cognitive Sciences, 1, 413–425.

    Google Scholar 

  • Dreyfus, H. L. (2007). Why Heideggerian AI failed and how fixing it would require making it more Heidegerrian. Artificial Intelligence, 171(18), 1137–1160.

    Google Scholar 

  • Dreyfus, H. L. (2014). Skillful coping: Essays on the phenomenology of everyday perception and action. Oxford: Oxford University Press.

    Google Scholar 

  • Dreyfus, H. L., & Dreyfus, S. E. (1980). A five-stage model of the mental activities involved in directed skill acquisition. ORC-80-2, AD-A084551. Berkeley, CA: US Air Force Operations Research Center, University of California.

  • Dreyfus, H. L., & Dreyfus, S. E. (1986). Mind over machine. New York: The Free Press.

    Google Scholar 

  • Dreyfus, H. L., & Dreyfus, S. E. (2005). Peripheral vision expertise in real world contexts. Organization Studies, 26(5), 779–792.

    Google Scholar 

  • Dreyfus, H. L., & Kreisler, H. (2005). Meaning, relevance and the limits of technology—Conversation with H.L. Dreyfus. Conversations with History. Institute of International Studies, UC Berkeley.

  • Dubnick, M. (2003). Accountability and ethics: Reconsidering the relationships. International Journal of Organization Theory and Behavior, 6(3), 405–441.

    Google Scholar 

  • Elish, M. C. (2019). Moral crumple zones: Cautionary tales in human–robot interaction. Engaging Science, Technology, and Society, 5, 40–60.

    Google Scholar 

  • Elish, M. C., & Hwang, T. (2015). Praise the Machine! Punish the Human! The contradictory history of accountability in automated aviation. Data and Society Working Paper. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2720477.

  • Esfeld, M. (2000). Aristotle’s direct realism in “De Anima”. The Review of Metaphysics, 54(2), 321–336.

    Google Scholar 

  • FAA. (2011). AC 23-17C—Systems and Equipment Guide for Certification of Part 23 Airplanes and Airships. Retrieved July 1, 2018, from https://www.faa.gov/regulations_policies/advisory_circulars/index.cfm/go/document.information/documentID/1019689.

  • Flyvbjerg, B. (2001). Making social science matter: Why social enquiry fails and how it can succeed again. Cambridge: Cambridge University Press.

    Google Scholar 

  • Foushee, H. C. (1982). The role of communications, socio-psychological and personality factors in the maintenance of crew coordination. Aviation, Space and Environmental Medicine, 53(11), 1062–1066.

    Google Scholar 

  • Fischer, J. M., & Ravizza, M. S. J. (1998). Responsibility and control. A theory of moral responsibility. Cambridge: Cambridge University Press.

    Google Scholar 

  • Jeannot, E., Kelly, C., & Thompson, D. (2003). The development of situation awareness measures in ATM systems. European Air Traffic Management Programme HRS/HSP-005-REP-01.

  • FAA FAR. (2014). Clause 91.4 “Responsibility and Authority of Pilot in Command”. US Federal Aviation Authority. https://web.archive.org/web/20140914065448/, https://www.ecfr.gov/cgi-bin/text-idx?rgn=div5&node=14:2.0.1.3.10#se14.2.91_13.

  • Flemisch, F., et al. (2012). Towards a dynamic balance between humans and automation: Authority, ability, responsibility and control in shared and cooperative control situations. Cognition, Technology and Work, 14, 3–18.

    Google Scholar 

  • Freeman, W. J. (1991). The physiology of perception. Scientific American, 264, 78–85.

    Google Scholar 

  • Gaffary, Y., & Lécuyer, A. (2018). The use of haptic and tactile information in the car to improve driving safety: A review of current technologies. Frontiers in ICT, 5(5), 1–11.

    Google Scholar 

  • Gallagher, S. (2009). The philosophical antecedents of situated cognition. In P. Robbins & M. Aydede (Eds.), The Cambridge handbook of situated cognition. Cambridge: Cambridge University Press.

    Google Scholar 

  • Gawron, V. (2019). Automation in aviation accident analyses. Center for Advanced Aviation System Development: MITRE Technical Report MTR190013. The MITRE Corporation.

  • German, E. S., & Rhodes, D. H. (2016). Human-model interactivity: What can be learned from the experience of pilots with the glass cockpit? In Conference on systems engineering research.

  • Gibson, J. J. (1979). The ecological approach to visual perception. Reading, MA: Houghton Mifflin.

    Google Scholar 

  • Giddens, A. (1984). The constitution of society: Outline of the theory of structuration. Cambridge: Polity Press.

    Google Scholar 

  • Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilistic mental models: A Brunswickian theory of confidence. Psychological Review, 98, 506–528.

    Google Scholar 

  • Glasziou, P., Moynihan, R., Richards, T., & Godlee, F. (2013). Too much medicine; too little care. BMJ, 347, f4247.

    Google Scholar 

  • Greenhalgh, T., Howick, J., & Maskrey, N. (2014). Evidence based medicine: A movement in crisis? BMJ, 348, g3725.

    Google Scholar 

  • Heidegger, M. (1962). Being and time. New York: Harper and Row.

    Google Scholar 

  • Heidegger, M. (1997). Plato’s sophist (trans: Rojcewicz, R., and Schuwer, A.). Bloomington, IN: Indiana University Press.

  • Hicks, N. M., Bumbaco, A. E., & Douglas, E. P. (2014). Critical thinking, reflective practice, and adaptive expertise in engineering. In: Conference proceedings—121st ASEE annual conference and exposition, Indianapolis, IN.

  • Holden, R. J. (2009). People or systems? To blame is human. The fix is to engineer. Professional Safety, 54(12), 34–41.

    Google Scholar 

  • Horowitz, M. C., & Scharre, P. (2015). Meaningful human control in weapon systems: A primer. https://www.cnas.org/publications/reports/meaningful-human-control-in-weapon-systems-a-primer.

  • Hubbard, T., & Bor, R. (2012). Aviation mental health: Psychological implications for air transportation. Hampshire: Ashgate.

    Google Scholar 

  • ICAO. (2005). Rules of the Air, Annex 2. International Civil Aviation Organisation. https://www.icao.int/Meetings/anconf12/Document%2520Archive/an02_cons%5B1%5D.pdf.

  • IATA. (2020). Command Training, Guidance Material and Best Practices, Edition 1. International Air Transport Association. ISBN 978-92-9229-976-7.

  • Kallinikos, J., Leonardi, P. M., & Nardi, B. A. (2012). The challenge of materiality: Origins, scope, and prospects. In P. Leonardi, B. Nardi, & J. Kalliniko (Eds.), Materiality and organizing: Social interaction in a technological world (pp. 1–22). Oxford: Oxford University Press.

    Google Scholar 

  • Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, 6, 175–183.

    Google Scholar 

  • McBride, S. E., Rogers, W. E., & Fisk, A. D. (2014). Understanding human management of automation errors. Theoretical Issues in Ergonomic Sciences, 15(6), 545–577.

    Google Scholar 

  • McKenna, M. (2008). Putting the lie on the control condition for moral responsibility. Philosophical Studies, 139(1), 29–37.

    Google Scholar 

  • Merleau-Ponty, M. (1962). Phenomenology of perception (trans: Smith, C.). London: Routledge and Kegan Paul.

  • Meshkati, N., & Khashe, Y. (2015). Operators’ improvisation in complex technological systems: Successfully tackling ambiguity, enhancing resiliency and the last resort to averting disaster. Journal of Contingencies and Crisis Management, 23(2), 90–96.

    Google Scholar 

  • Michaelides-Mateou, S., & Mateou, A. (2010). Flying in the face of criminalization. Farnham: Ashgate Publishing Co.

    Google Scholar 

  • Miller, C., & Parasuraman, R. (2007). Designing for flexible interaction between humans and automation: Delegation interfaces for supervisory control. Human Factors, 49, 57–75.

    Google Scholar 

  • Mosier, K. L., & Skitka, L. J. (1996). Human decision makers and automated decision aids: Made for each other? In R. Parasuraman & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp. 201–220). Boca Raton, FL: CRC Press.

    Google Scholar 

  • Mosier, K. L., Skitka, L. J., Heers, S., & Burdick, M. D. (1998). Automation bias: Decision making and performance in high-tech cockpits. The International Journal of Aviation Psychology, 8(1), 47–63.

    Google Scholar 

  • North, D. M. (2002). Oil and water, cats and dogs. Aviation Week & Space Technology, 156(5), 70–70.

    Google Scholar 

  • Oguz, S. O., Kucukyilmaz, A., Sezgin, T. M., & Basdogan, C. (2010). Haptic negotiation and role exchange for collaboration in virtual environments. In IEEE Haptics symposium (pp. 371–378). https://doi.org/10.1109/HAPTIC.2010.5444628

  • Oliver, N., Calvard, T., & Potocnik, K. (2017). Cognition, technology and organizational limits: Lessons from the Air France 447 disaster. Organization Science, 28(4), 597–780.

    Google Scholar 

  • Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253.

    Google Scholar 

  • Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man and Cybernetics, 30(3), 286–297.

    Google Scholar 

  • Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381–410.

    Google Scholar 

  • Reuter, M. (1999). Merleau-Ponty’s notion of pre-reflective intentionality. Synthese, 118(1), 69–88.

    Google Scholar 

  • Roe, E., & Schulman, P. R. (2008). High reliability management: Operating on the edge. Palo Alto, CA: Stanford University Press.

    Google Scholar 

  • Sackett, D. L., Rosenberg, W. M., Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. BMJ, 312, 71–72.

    Google Scholar 

  • Santonio de Sio, F., & van den Hoven, J. (2018). Meaningful human control over autonomous systems: A philosophical account. Frontiers in Robotics and AI, 5(15), 1–14.

    Google Scholar 

  • Sarter, N. B., & Woods, D. D. (1994). Decomposing automation: Autonomy, authority, observability and perceived animacy. In M. Mouloua & R. Parasuraman (Eds.), Human performance in automated systems: current research and trends (pp. 22–27). Hilldale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Sarter, N. B., Woods, D. D., & Billings, C. E. (1997). Automation surprises. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (2nd ed.). New York: Wiley.

    Google Scholar 

  • Sayer, A. (2011). Why things matter to people: Social science, values and ethical life. Cambridge: Cambridge University Press.

    Google Scholar 

  • Schultz, T. P. (2018). The problem with pilots: How physicians, engineers and airpower enthusiasts redefined flight. Baltimore, MA: John Hopkins University Press.

    Google Scholar 

  • Schwartz, D., Fischhoff, B., Krishnamurti, T., & Sowell, F. (2013). The Hawthorne effect and energy awareness. Proceedings of the National Academy of Sciences of USA, 110, 15242–15246.

    Google Scholar 

  • Schwartzman, H. B. (1993). Ethnography in organizations. Newbury Park, CA: Sage Publications.

    Google Scholar 

  • Skitka, L. J., Mosier, K. L., & Burdick, M. D. (2000). Accountability and automation bias. International Journal of Human–Computer Studies, 2000, 701–717.

    Google Scholar 

  • Stoffregen, T. A. (2003). Affordances as properties of the animal–environment system. Ecological Psychology, 15(2), 115–134.

    Google Scholar 

  • Strauch, B. (2017). The automation-by-expertise-by-training interaction: Why automation-related accidents continue to occur in sociotechnical systems. Human Factors, 59(2), 204–228.

    Google Scholar 

  • Timmermans, S., & Berg, M. (2003). The gold standard: The challenge of evidence-based medicine and standardization in health care. Philadelphia: Temple University Press.

    Google Scholar 

  • Wachter, B. (2015). My interview with Capt. Sully Sullenberger: On aviation, medicine and technology. The Hospital Leader, February 23, 2015. https://thehospitalleader.org/my-interview-with-capt-sully-sullenberger-on-aviation-medicine-and-technology/.

  • Ward, P., Gore, J., Hutton, R., Conway, G., & Hoffman, R. (2018). Adaptive skill as the conditio sine qua non of expertise. Journal of Applied Research in Memory and Cognition, 7(1), 35–50.

    Google Scholar 

  • Wickens, C. D., Li, H., Santamaria, A., Sebok, A., & Sarter, N. B. (2010). Stages and levels of automation: An integrated meta-analysis. In Proceedings of the Human Factors and Ergonomics Society 54th annual meeting (pp. 389–393).

  • Young, J. (2000). What is dwelling? The homelessness of Modernity and the worlding of the world. In M. Wrathall & J. Mapas (Eds.), Heidegger, authenticity and modernity: Essays in Honor of H.L. Dreyfus (Vol. 1, pp. 187–204). London: MIT Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management, School of Management Sciences (ESG), University of Quebec at Montreal, Downtown Station, P.O. Box 8888, Montreal, QC, H3C 3P8, Canada

    W. David Holford

Authors
  1. W. David Holford
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to W. David Holford.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Holford, W.D. An Ethical Inquiry of the Effect of Cockpit Automation on the Responsibilities of Airline Pilots: Dissonance or Meaningful Control?. J Bus Ethics 176, 141–157 (2022). https://doi.org/10.1007/s10551-020-04640-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10551-020-04640-z

Keywords

Search

Navigation