Skip to main content
SpringerLink
Log in

User Rights and Adaptive A/IS – From Passive Interaction to Real Empowerment

  • Conference paper
  • First Online:
Adaptive Instructional Systems (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12214))

Included in the following conference series:

Abstract

Adaptive autonomous intelligent systems (A/IS) may satisfy design functionality and user experiential requirements but prior to deployment an assessment must be made of their impact on user rights. A/IS systems may assist rather than replace humans but it is unclear where the line is drawn between supplementing human endeavour and knowledge, on the one hand, and gradual erosion of human cognitive abilities on the other. This paper makes the case for development of ethical standards for user awareness of A/IS in operation, taking account of rights under the EU General Data Protection Regulation (GDPR) and the Council of Europe Modernised Convention for the Protection of Individuals with Regard to Automatic Processing of Personal Data (Convention 108+). It sets out three main user awareness stages (pre-use, during-use, and post-use) along with consideration of commensurate rights. In the pre-use stage potential users will need to be aware that an A/IS is either fully or partially in operation, and consent to such an operation or have the option to opt out. During A/IS use if there is a part of the A/IS operation which involves a “black box” scenario, that is, it is difficult for a human to discern what the system is doing and why, then appropriate risk-based parameters need to be set for the systems use. Post-use requires users to be aware of how their data and information shared with the A/IS will be used by the system and any third parties.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Respect for human autonomy is one of four ethical principles established under the EU AI Guidelines - Ethics Guidelines for Trustworthy AI, High-Level Expert Group on Artificial Intelligence (EU AI HLEG), European Commission. https://ec.europa.eu/futurium/en/ai-alliance-consultation/guidelines/2. Accessed 03 Feb 2020, p. 12; cf. Articles 32 (which implicitly predicates the security framework for data processing on guaranteeing the human dignity of natural persons), and 88(2) (which refers to safeguarding the data subject’s human dignity in the processing of data for employment purposes) of the European Parliament and Council of the European Union, Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, repealing Directive 95/46/EC (General Data Protection Regulation) (GDPR), (OJ L 119, 4.5.2016), 27 April 2016; Preamble of the Council of Europe Modernised Convention for the Protection of Individuals with Regard to Automatic Processing of Personal Data (ETS No. 108+), Amending Protocol to the Convention, adopted by the Committee of Ministers at its 128th Session in Elsinore on 18 May 2018 (Convention 108+) (which refers to securing human dignity and personal autonomy, and predicates the legal framework on safeguarding these)

  2. Human agency and oversight represents a key requirement to implement the ethical principle of human autonomy under the EU AI HLEG, pp. 15–16; cf. Principle (d) of the 2019 Guiding Principles on Lethal Autonomous Weapons Systems by the UN Group of Governmental Experts on Lethal Autonomous Weapons Systems (UNGGE Principles). https://undocs.org/en/CCW/GGE.1/2019/3. Accessed 03 Feb 2020, which refers to “the operation of such systems within a responsible chain of human command and control”; Preamble of Convention 108+ (which refers to a person’s right to control their personal data and the processing of it)

  3. Privacy and data governance represents a key requirement to implement the ethical principle of prevention of harm under the EU AI HLEG, pp. 17; cf. Recital 116, Articles 1(2), 12–18, 20, 21, and 22 GDPR; Preamble and Article 1 of Convention 108+; Principles 1 and 3 of 2019 IEEE Ethically Aligned Design for Autonomous and Intelligent Systems - The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems (Final version, 4 April 2019) (IEEE EAD), pp. 19–20, 23–24. https://ethicsinaction.ieee.org. Accessed 03 Feb 2020; IEEE P7002, IEEE Standards Project for Data Privacy Process. https://standards.ieee.org/project/7002.html. Accessed 03 Feb 2020

  4. Prevention of harm is the second ethical principle established under the EU AI HLEG, p. 12; generally expressed as violations of privacy and data protection rules under the GDPR; generally expressed as interference with the fundamental rights and freedoms of the individual under Convention 108+

    Google Scholar 

  5. Fairness is the third ethical principle established under the EU AI HLEG, p. 12; cf. Recitals 39, 60, and 71, Articles 5(1)(a), 13(2), and 14(2) GDPR; Articles 5(4)(a) and 8(1) of the Council of Europe Modernised Convention for the Protection of Individuals with Regard to Automatic Processing of Personal Data (ETS No. 108+), Amending Protocol to the Convention, adopted by the Committee of Ministers at its 128th Session in Elsinore on 18 May 2018 (Convention 108+)

    Google Scholar 

  6. Recitals 39, 58, 60, and 71, Articles 5(1)(a) and 12(1) GDPR; cf. Articles 5(4)(a), 8, and 9(b) Convention 108+; Principle 5 IEEE EAD, pp. 27–28; IEEE P7001, IEEE Standards Project for Transparency of Autonomous Systems. https://standards.ieee.org/project/7001.html. Accessed 03 Feb 2020

  7. Explicability, as the fourth ethical principle established under the EU AI HLEG, p. 13, is closely associated with auditability particularly where it concerns “black box” scenarios; cf. GDPR transparency provisions, and safeguarding measures under Article 22; Article 9(1)(c) Convention 108+; generally considered under Principle 5 IEEE EAD

    Google Scholar 

  8. Accountability represents a key requirement to implement the ethical principle of fairness under the EU AI HLEG, pp. 19–20; cf Article 5(2) GDPR; Principle 6 IEEE EAD, pp. 29–30

    Google Scholar 

  9. Legal responsibility is subsumed in the accountability requirement under the EU AI HLEG, pp. 19–20; Recitals 74 and 79, Article 24 GDPR; Article 15(1) Convention 108+; Principle 6 IEEE EAD

    Google Scholar 

  10. 2018 UK House of Lords Select Committee AI Report - AI in the UK: ready, willing and able? (16 April 2018). https://publications.parliament.uk/pa/ld201719/ldselect/ldai/100/100.pdf. Accessed 03 Feb 2020; 2018 China’s AI Standardisation White Paper - China’s White Paper on Artificial Intelligence Standardisation (January 2018, Standards Administration of China) - 人工智能标准化白皮书 (白皮书) (2018年1月, 中国标准管理局); 2018 Canada’s White Paper on Responsible AI in Government - Responsible Artificial Intelligence in the Government of Canada (Digital Disruption White Paper Series, Version 2.0, 2018-04-10); 2019 EU AI Guidelines - Ethics Guidelines for Trustworthy AI, High-Level Expert Group on Artificial Intelligence (AI HLEG), European Commission. https://ec.europa.eu/futurium/en/ai-alliance-consultation/guidelines/2. Accessed 03 Feb 2020; 2016 EU Study on Civil Law Rules in Robotics - European Civil Law Rules in Robotics (October 2016), PE 571.379. http://www.europarl.europa.eu/RegData/etudes/STUD/2016/571379/IPOL_STU(2016)571379_EN.pdf. Accessed 03 Feb 2020; 2017 EU Parliament Resolution for further study on the implications of creating legal status for robots - European Parliament resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics (2015/2103(INL)); 2019 IEEE Ethically Aligned Design for Autonomous and Intelligent Systems - The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems (Final version, 4 April 2019). https://ethicsinaction.ieee.org. Accessed 03 Feb 2020; 2019 Guiding Principles on Lethal Autonomous Weapons Systems by the UN Group of Governmental Experts on Lethal Autonomous Weapons Systems. https://undocs.org/en/CCW/GGE.1/2019/3. Accessed 03 Feb 2020

  11. European Parliament and Council of the European Union, Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, repealing Directive 95/46/EC (General Data Protection Regulation) (GDPR), (OJ L 119, 4.5.2016), 27 April 2016

    Google Scholar 

  12. Council of Europe Modernised Convention for the Protection of Individuals with Regard to Automatic Processing of Personal Data (ETS No. 108+), Amending Protocol to the Convention, adopted by the Committee of Ministers at its 128th Session in Elsinore on 18 May 2018 (Convention 108+)

    Google Scholar 

  13. Cf. Wachter, S., Mittelstadt, B., Floridi, L.: Why a right to explanation of automated decision-making does not exist in the General Data Protection Regulation. Int. Data Privacy Law 7(2), 78 (2017); Malgieri, G., Comandé, G.: Why a right to legibility of automated decision-making exists in the General Data Protection Regulation. Int. Data Privacy Law 7(4), 265 (2017)

    Google Scholar 

  14. Under GDPR Article 4(4) “profiling” means “any form of automated processing of personal data consisting of the use of personal data to evaluate certain personal aspects relating to a natural person, in particular to analyse or predict aspects concerning that natural person’s performance at work, economic situation, health, personal preferences, interests, reliability, behaviour, location or movements”

    Google Scholar 

  15. Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 (“WP Guidelines”). The WP Guidelines were endorsed by the European Data Protection Board in May 2018, p. 21. https://edpb.europa.eu/node/71. Accessed 31 Jan 2020

  16. WP Guidelines, p. 21

    Google Scholar 

  17. GDPR Article 9(1)

    Google Scholar 

  18. GDPR Article 9(2)(a)

    Google Scholar 

  19. GDPR Article 9(2)(g)

    Google Scholar 

  20. Convention 108+ Convention for the protection of individuals with regard to the processing of personal data (Council of Europe, June 2018), Explanatory Report (Convention 108+ Explanatory Report), pp. 19–20. https://rm.coe.int/convention-108-convention-for-the-protection-of-individuals-with-regar/16808b36f1. Accessed 31 Jan 2020

  21. See examples of practical application of these different interpretations to the information needed: Automated decision-making on the basis of personal data that has been transferred from the EU to companies certified under the EU-U.S. Privacy Shield Fact-finding and assessment of safeguards provided by U.S. law (European Commission Final Report, Directorate-General for Justice and Consumers, October 2018), pp. 45–47

    Google Scholar 

  22. WP Guidelines, pp. 34–35

    Google Scholar 

  23. WP Guidelines, p. 26

    Google Scholar 

  24. Convention 108+ Explanatory Report, p. 24

    Google Scholar 

  25. Convention 108+ Explanatory Report, p. 20

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Birmingham City University, Birmingham, UK

    Ozlem Ulgen

Authors
  1. Ozlem Ulgen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ozlem Ulgen .

Editor information

Editors and Affiliations

  1. Soar Technology, Inc., Orlando, FL, USA

    Robert A. Sottilare

  2. Fraunhofer FKIE, Wachtberg, Germany

    Jessica Schwarz

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ulgen, O. (2020). User Rights and Adaptive A/IS – From Passive Interaction to Real Empowerment. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. HCII 2020. Lecture Notes in Computer Science(), vol 12214. Springer, Cham. https://doi.org/10.1007/978-3-030-50788-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-50788-6_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-50787-9

  • Online ISBN: 978-3-030-50788-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation