1 Introduction

The example of Mr. Alois shows that the use of AI-based MATs in the care of people with dementia (PwD) touches on various ethical aspects like self-determination, privacy, and social participation, but also safety, efficiency, and relief of caregiver burden. Against the backdrop of different interests of, for example, residents and caregivers or facility management, value conflicts can arise.

AI-based MATs themselves have long been ethically relevant actors in these conflict situations because they cause moral conflicts like the one described in the use case above. Yet, they are now also being designed to contribute to the solution of moral conflicts (Allen et al. 2005; Anderson et al. 2006; Sharkey 2020; Dodig Crnkovic and Çürüklü 2012; van Wynsberghe 2013). They differ from earlier, non-intelligent assistive technologies precisely in that they fulfill certain goals somewhat autonomously (HLEGoAI 2019). In this context, they need to take ethically relevant decisions. As the use case shows, it is crucial for the acceptance of these technologies that they consider the value preferences of their users, i.e., that their general functioning aligns with users’ moral preferences (Ienca et al. 2018; Kunze and König 2017). If the decisions of AI-based MATs ignore these preferences, they will be perceived as useless, unethical, or even dangerous. The question of how ethical principles can be integrated into new technologies is the subject of the ethics-by-design debate. Ethics-by-design describes the approach of including ethical considerations in the development process of AI-based technologies (Anderson et al. 2006; Misselhorn 2022). A number of different approaches can be distinguished here, such as bottom-up, top-down or participatory approaches.

Participative approaches in particular are associated with challenges in the context of assisted technical care for people with dementia, as stakeholder engagement is required here. Since design process works under the assumption that the user is cognitively and technically able to operate the system independently, people with cognitive impairments such as PwD are excluded from this process, as they increasingly lose the cognitive ability to make and communicate ethical judgments as their illness progresses. However, due to demographic shifts, resulting in an increased number of PwD and changes in family care structures, intelligent assistive technologies are also intended to be used in the care of PwD. This poses both theoretical and practical challenges to participatory approaches in the ethics-by-design debate. In particular the question arises: How can the value preferences of users be integrated into AI-based MATs in a way that overcomes the challenges commonly associated with impairments in cognition and especially communication?

This paper addresses the aforementioned question. We develop an approach that facilitates the integration of users’ moral preferences into the operation of MATs—even for people with cognitive impairments. To this end, we introduce value preference profiles (VPPs) as a new approach for ethics-by-design. To create VPPs, we take an empirically informed ethics approach, i.e., we use research methods of social sciences in combination with ethical theory to tackle ethical issues (Sugarman 2004). Through content analysis of qualitative interviews with PwD and their relatives—combined with subsequent ethical reflection—VPPs can provide technology developers with crucial insights into the moral attitudes of potential users. VPPs are to be understood as ideal types and can be used in the development of new technologies (Kuckartz 2014). To this end, we test the value-sensitive application of the new products in a computer-based simulation. By developing these profiles, multiple settings, i.e., adjustments regarding the functioning of the MATs that align with the respective moral preferences of the users, can be defined. Going forward, users could be enabled to choose the value profile that fits their moral preferences best, thereby having a direct influence on the settings that control the functioning of the MAT. Our approach differs from similar ethics-by-design approaches in two respects: first, we not only consider the ethical principles proposed by the relevant stakeholders (Aizenberg and van den Hoven 2020; Gerdes 2022; Stahl and Wright 2018; The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems 2017), but also analyze the underlying argumentation. This enables us to elicit different meanings of the same principle, e.g., autonomy. Secondly, we test the VPPs as a result of the first analysis process in a computer-based simulation.

The implementation of ethical principles in an AI-based technology or in a computer-based simulation requires the translation of these principles into the notation of computer science. In the notation of computer science, intelligent assistive systems need to be programmed to choose an optimal action (e.g., an action that aligns with the moral preferences of the user) \({\varvec{a}}\) (from a set of possible actions \({{\varvec{A}}}_{{\varvec{t}}}\)). An example is the decision to send a caregiver to a disoriented patient immediately or delaying the intervention and letting the patient try to reorient on their own. The former (action) is more likely to ensure the safety of the patient while the latter leans more towards preserving their autonomy. The goal of the intelligent assistive system is to take the best possible action given the current information at hand. Such information can be simply the current cognitive state of the patient or more elaborate information, e.g., on the duration of the current disorientation episode, age and history of the patient and number of the available nursing staff. Here, the respective information is referred to as the state. Given a set of possible states the intelligent system aims to learn optimal actions for each state \(({{\varvec{a}} \mathord{\left/ {\vphantom {{\varvec{a}} {\varvec{s}}}} \right. \kern-0pt} {\varvec{s}}})\) here referred to as the optimal policy \({\varvec{\pi}}\).The approach we describe below employs preference profiles and simulation-based ethical compliance quantification (see Sect. 4.2), providing a method by which systems can learn such action policy. We outline how preference profiles, extrapolated from end-users, can be employed to learn the effects of different moral preferences on the system’s actions and their outcomes—e.g., a priority for autonomy will lead to a different operating mode of the system than a priority for safety. This approach is an important step towards developing user-centered technologies and (ethically) evaluating technical assistance in terms of user’s ethical preferences.

Below, we describe the basis for a new tool that can be used to better consider user preferences and, consequentially, to contribute to a higher ethical acceptability and social acceptance of new technologies. The aim is to define, based on empirically identified argumentation profiles, multiple ideal–typical modes of operation of the assistive systems. By involving PwD in this survey, we also take a step towards greater inclusion.

We present our approach below in three sections: theoretical ethics (2), empirical ethics (3), and computer science (4). In a subsequent step, we discuss how the profiles so developed can be used to computationally simulate a smartwatch application for supporting disoriented PwD like Mr. Alois in a value-sensitive way.

2 Dementia as a challenge for participatory ethics-by-design approaches

Due to progress in the field of artificial intelligence (AI), technologies can operate ever more independently. Especially in the field of healthcare, these AI-enabled technologies and the decisions taken by them directly affect human beings. Thereby, they themselves become ethically relevant actors (Allen et al. 2005; Anderson et al. 2006; Sharkey 2020; Dodig Crnkovic and Çürüklü 2012; van Wynsberghe 2013). Dodig Crnkovic and Çürüklü (2012) speak of “artifactual ethical” technologies in this regard. The term “artifactual” emphasizes that technology is not itself ethical but only behaves in accordance with ethical principles as a result of deliberate ethical programming by technology developers. Against this backdrop, it is reasonable to expect that machines should be designed with the ability to make ethically justifiable decisions. Beyond this theoretical consideration, however, stands the fact that user acceptance of new technologies will depend on whether machine decisions align with users’ moral preferences (Ienca et al. 2018; Schweda et al. 2019; Kunze and König 2017; Schicktanz and Schweda 2021).

Ethics-by-design approaches address this by proposing various ways in which ethical considerations can be integrated into the development process of AI-based technologies (Anderson et al. 2006; Misselhorn 2022). Initially, the ethics by design approach was conceived more broadly for technologies. More recent work such as a contribution from (Brey and Dainow 2023) develop an approach to “ethics by design for artificial intelligence”. The approach of value-sensitive design also aims to identify potential ethical challenges that arise in the use of technology, such as discrimination against people of color or women in the context of automated algorithms (Zhu et al. 2018). Inherent in both approaches is the iterative process of technology development, in which ethical principles are to be considered from the outset (Friedman et al. 2013). Unlike Iphofen and Kritikos (2021), we do not pursue the formulation of ethical guidelines for developers and users with our ethics-by-design approach.

In order to design “artifactual ethical robots” (Dodig Crnkovic and Çürüklü 2012), various methods have been used. These can be divided into three ethics-by-design approaches corresponding to the way the principles and values to be implemented are elicited. While in bottom-up approaches, the machines are trained in (a simulation of) their future environment to learn operating in an ethically appropriate way (Sharkey 2020), in top-down approaches, values are usually preselected by experts (Allen et al. 2005; Anderson et al. 2006; McLennan et al. 2022; van Wynsberghe 2013). Participatory design approaches are a third way to elicit principles, values, and norms. For this method, relevant stakeholders participate in the development process of a new technology, articulating needs, preferences, and wishes themselves (Ienca et al. 2018; Friedman and Hendry 2019).

Due to context-specific factors, all three approaches face challenges and limits in dementia care. Typical symptoms of dementia are loss of memory, impaired orientation, loss of judgment and reasoning ability, and loss of speech (Atri 2019). Behavioral changes, such as apathy, avoidance of social interaction, and loss of affect control are also common (Lyketsos et al. 2011). All of these symptoms interfere with individual planning and problem-solving abilities (Atri 2019). Since participatory ethics-by-design approaches require the involvement of stakeholders, the cognitive and communicative limitations associated with dementia pose a challenge for such approaches. Another challenge for ethics-by-design approaches that want to take user preferences into account is inherent in the problem of different selves in the context of dementia. This problem describes the increasing loss of memory and language ability on the part of people with dementia, which is also accompanied by the dwindling ability to express one’s own self in narration and judgment, so that it seems as if people with dementia have lost their self or have adopted a different self (Hutmacher 2021). In this context, Walsh (2020) speaks of a cognitive transformative experience. This affects not least the temporal stability of moral preferences.

With regard to bottom-up approaches, it may be problematic or even risky to allow untrained machines to assist PwD, as these persons have an increased risk of anxiety and panic attacks (Atri 2019). Although top-down approaches seem to provide an adequate solution to this challenge, they introduce the risk of paternalism because they dictate fundamental decisions regarding which ethical principles are most appropriate without consulting the persons most directly affected. Further, it is important to emphasize that the choice of guiding ethical principles may also be influenced by further biases that exist in power relations, e.g., gender and race bias with regard to the data used and the people involved (Friedman et al. 2013). Especially for the technology-assisted care of people with dementia, this contributes to further marginalization and disempowerment of an already stigmatized group by ignoring their right to participate in decisions affecting them. Additionally, there is no consensus among ethicists regarding which ethical principles should guide machine learning (Allen et al. 2005; Anderson et al. 2006; Dodig Crnkovic and Çürüklü, 2012; Misselhorn 2022; Sharkey 2020; van Wynsberghe 2013).

Addressing the aforementioned challenges, so-called hybrid approaches have gained traction over the last years in scientific and public discourses (Misselhorn 2019; Sharkey 2020). They aim to combine the benefits of bottom-up and top-down approaches by allowing intended users to participate in technology development. The approaches also include ethical reflection on the results of participation. Due to the nature of their symptoms, participation in complex technology-development processes is challenging or even impossible for PwD. This problem is not yet resolved, neither by Köhler et al. (2022), who attempt to catalog relevant value concepts for technological assistance for PwD, nor by Misselhorn (2019), who suggests building a “moral framework” (ibid., p. 61) by listing “moral values which are considered important for elderly care by the affected persons” (ibid.). However, Köhler et al. do not consider individual preferences, and Misselhorn does not address dementia-specific challenges.

Given this dementia-specific critique of bottom-up, top-down, and hybrid/participatory ethics-by-design approaches, we see a need for a new, dementia-sensitive approach for aligning AI-enabled technologies with user preferences.

While numerous approaches obtain the perspectives of stakeholders or recommend their integration into the development process of AI (e.g., Aizenberg and van den Hoven 2020; Gerdes 2022; Stahl and Wright 2018; The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems 2017), our approach not only elicits their value preferences, but also analyzes the underlying reasoning of the stakeholders. This allows for a more precise consideration of the preferences of the stakeholders surveyed and also addresses the fact that although there is a “fragile consensus” on relevant ethics-by-design principles, their concrete form is the subject of controversial debate (Morley et al. 2020, 2145). In addition, our approach includes testing the corresponding principles in a simulation in order to check the intended effect on the functioning of the MATs. This delineates our approach from similar models. Since our approach discusses both ethical aspects of concrete AI-based assistance systems and methods for solving these challenges as well as proposing a corresponding workflow, our approach can be categorized in the typology of Prem (2023) as a case study and as a process model. Our work is characterized by the combination of both approaches (Fig. 1).

Fig. 1
figure 1

Proposed workflow of VPP and ECQ as participatory ethics-by-design approach

Full size image

3 Taking people with dementia seriously: value preference profiling as a new approach for participatory design in dementia care

3.1 Method of constructing value preference profiles

In the previous section, we argued that the respective ethics-by-design approaches each face different challenges. For developing the value preference profiles, we first conducted 27 semi-structured interviews with people with dementia (n = 12) and their relatives (n = 15). A qualitative methodological approach is particularly appropriate here, as it allows for a basic exploration of participants’ preferences by being open to their subjective perspectives, a more nuanced elicitation for participants’ values including their underlying reasoning. This does not imply that empiricism directly provides answers to ethical questions; rather, conducting empirical studies helps to broaden our ethical reasoning by including the perspective of those affected (Schicktanz et al. 2012). The participants were recruited through local self-help groups and nursing homes in Oldenburg and Bremen, Germany, the REHA Center Oldenburg, and the German Center for Neurodegenerative Diseases (DZNE), Rostock as project partner in the project EIDEC.Footnote 1 The interviews were conducted in the respective nursing homes in Oldenburg and Bremen, the DZNE, or in the interviewees’ homes. Participants’ ages ranged from 37 to 84 years, and their educational attainment ranged from a secondary school diploma to university degrees. Of the family caregivers, 12 participants identified as female and 3 as male. Of those affected, 5 identified as female and 7 as male. The study was approved by the ethics committee of the University of Oldenburg on February 26, 2020 (2020-025).

The interview guideline contained questions about everyday life, the interviewee’s living and care situation and their experience with health-related technical devices. The main focus of the interviews was four different technical assistive systems and respondents’ attitudes towards them: a GPS tracker, a dressing assistant in two different versions, and an emotion recognition system. The technologies were described with explanatory illustrations and example applications to facilitate understanding. Respondents were asked about basic attitudes, their willingness to use the respective devices, the presumed influence of the devices on their independence, safety, privacy, quality of life, caregiver relationships, and, finally, their evaluation of each technical assistive system for the care of people with dementia.

The statements of the interviewees were audiotaped, transcribed, coded with the help of a coding guideline, and then interpreted using qualitative content analysis (Kuckartz 2014). The guideline was developed in accordance with the basic coding framework of Saldaña (2015) and contained descriptive codes referring to the interview questions. Before EB and JW coded the interview material in independent settings, we tested the guideline with an external researcher who was not familiar with the interviews to ensure intercoder reliability.

As a first step of analysis, we structured the interview data according to thematic foci of the interviewees. We identified patterns in how respondents evaluated the individual assistive systems. By first openly asking for a basic evaluation of the respective assistive systems and then also asking about the perceived impact of the systems on users’ autonomy, privacy, security, and quality of life, we were able to identify the underlying rationale of the respondent’s evaluation as well as the hierarchy of their corresponding values. Since data regarding the individual’s understanding and weighting of the evaluation criteria was directly derived from the responses, the collected data also shed light on moral attitudes. We observed patterns in the way respondents conceptualized principles and also in the way they linked those principles to evaluations of the assistive systems. Further, we observed different lines of argumentation linking principles and evaluations across the interviews. Consequently, we decided to group these arguments into distinct profiles.

In a next step, we clustered the observed patterns into types (Kuckartz 2014). The types result from comparing and contrasting respondents’ evaluations of the assistive systems, including the lines of reasoning that underlie them. A central process in this method is the creation of an “attribute space” in which the respective types can be located. In this study, the attribute space is characterized by the hierarchy of the mentioned values, their conceptualization, their application in the assessment of each assistance system as well as their underlying lines of argumentation. The aim of the type-building was to identify typical lines of argumentation and condense them into profiles so that a generalized overview of potential preferences of affected persons regarding the use and implementation of technical assistive systems in the care of PwD could be obtained. Hence, the VPPs we developed do not represent the values of specific persons but are rather ideal-types. In the following, we will show how such profiles can help to circumvent the challenges described above.

The initial challenge emerged from the factor that not all participants’ statements could be understood as moral judgments in the narrow sense. That means, for example, that they were not presented in the form of a practical syllogism, i.e., the logical structure of an argument. Rather, they were couched in narrative, evaluative, or descriptive passages. Therefore, we first identified these passages and their underlying lines of argumentation and translated them into moral judgments (Table 1). A distinctive feature of our approach is that we not only interpreted respondents’ statements as an expression of their value preferences toward technical assistive systems, but also elicited and analyzed the underlying reasoning from the participants’ statements and their respective contexts. Finally, we put these judgments to the test by checking whether the underlying reasoning is based on fallacies or misconceptions. In this context, we applied basic logical principles and normative theories. It should be noted that the respondent’s statements listed below also represent radical statements and partly correspond to one of the profiles in its most extreme form. The statements are not to be understood as the authors’ position, but the table exemplifies the method of translating (descriptive) statements of the respondents into ethical argumentation.

Table 1 Examples of identified moral reasoning in respondents’ statements
Full size table

Consequently, this approach is not to be understood merely as a sociological survey but also as a classification of argumentation structures. The individual statements of the participants were translated into generalizable ethical ways of reasoning. The table below exemplifies this procedure. The method of value preference profiling can thus be seen as an empirically informed approach to ethics in which ethics and empiricism interact with each other in various ways (Schicktanz et al. 2012). The method of qualitative interview research is particularly suitable for this approach as it not only facilitates the fundamental exploration of a thematic field, thereby generating hypotheses, but it also allows for testing of argumentation and for detailed attention focus on statements of the participants. (Mey and Mruck 2020; Roest et al. 2021). Applied to the case of Mr. Alois described above, this means considering not only the high value he places on autonomy but also revealing the reasoning for this specific value choice. Mr. Alois’ preferences regarding how assistive technologies should function could be met by a consideration of the appropriate VPP. Examples are discussed in the next section.

3.2 Examples of value preference profiles

In this section, we present three example VPPs. After reviewing the material, we were able to construct several types of preferences regarding the use of technical assistive systems. These differed in the selection of individual values, their weighting, and the underlying line of reasoning. In addition, it was possible to identify types that were similar in the weighting of the respective values but whose underlying reasoning differed, as shown below. This highlights the need to include and analyze the argumentation while structuring the types. In the following, we present such profiles as “natural types” (Kuckartz 2014, 110), based on interview data with people with dementia and their relatives.

The first profile is “safety.” For respondents who fit this profile, safety is central to the evaluation of technical assistive systems (“I rate that positively because I think that also gives the person who wears the bracelet a kind of security” (Rel9)). This profile values, first, the actual safety of the PwD including the preservation of their health. But also, an increased sense of responsibility for the physical integrity of the person with dementia seems to represent a decisive criterion for this type (“… as long as he is at home I feel responsible for him” (Rel3)). Autonomy is not understood here as freedom of choice, but rather as independence in everyday life. Freedom of choice plays a subordinate role in the evaluation of assistive systems. Rather, the focus is on a safe daily routine. This means that from the point of view of those affected, safety is also understood in terms of feeling safe. For relatives who fit this type, independence is conceived as a feeling of safety for those affected, when they go shopping independently for example. Violations of privacy are accepted if necessary to gain safety. When asked if they wished for the GPS tracking device to send signals only when leaving a predefined area, a caring relative replied: “That would be too unsafe for me. […] [S]o, no. Not only in these situations. Then I would want to know already, every time” (Rel3). However, such evaluations are made in consideration of the disease severity of the person concerned. That is, intrusions into privacy and freedom of choice are more likely to be tolerated when the relative’s dementia is more advanced. The “safety” type conceives privacy primarily in terms of its informational dimension and attaches great importance to data protection (Welsch and Buhr 2022), as is evident in the following statement by a respondent: “Of course, this is a security issue, and I would of course also welcome this and would always tend to use a device […] that is also based here in Europe or in Germany. That it is so protected, and no one can access it” (Rel4).

In contrast, the “individual self-determination” type is characterized by a high valuation of autonomy in both its positive and negative dimensions. The assessment of assistive systems was more positive the more they served to maintain the self-efficacy of the users or provided them with independence from other personal support services. To this end, relatives who fit to this type were also prepared to accept reductions in the safety of their family members affected by dementia. For example, one respondent would, in fact, rather run the risk of their relative with dementia committing suicide because they were unmonitored than to constantly monitor them via a GPS-Tracker: “I think, one must also concede to Mrs. M. that she may take her own life in this way” (Rel1). One person with dementia also stated that she would rather risk death than have her children constantly watch over her: “I also don’t think it’s bad at all if I die…just because the [daughter] or whoever is not [alerted] when I’m in distress …” (Aff31). For the “individual self-determination” type, security is primarily an instrumental value. It is conceptualized as a sense of safety for those affected which serves to allow them to remain independent for longer, i.e. it boosts self-confidence. The same is true for privacy. The functioning of assistive systems is seen as a violation of privacy primarily if the surveillance or the data collected is intended to induce a change in behavior on the part of those affected (Welsch and Buhr 2022).

The “considerateness” type illustrates well the added value of capturing and analyzing the underlying reasoning behind value statements. Respondents who fit to this type also place a high value on autonomy, but the underlying argumentation reveals that the focus lies on the autonomy of the care-taking relatives rather than the autonomy of the PwD. Thus, for this type, assistive systems were given higher assessments to the degree they served to maintain users’ independence and provided relatives with more freedom from their care-taking duties which results in a relieve of caregiver-burden, as the following quote illustrates: “And I think that would give the patient’s caregivers some peace of mind, because he shouldn’t be locked up, he has to be able to go outside, and he wants to go out …” (Rel3). However, in contrast to the “individual self-determination” type, safety continues to play an overriding role in the evaluation of assistive systems. For persons who fit to the “considerateness” type, safety is conceptualized not only as a feeling of safety, but as actual physical safety and the preservation of health as in the “safety” type. A heightened sense of responsibility for the physical vulnerability of those affected seems to play a role here. To this end, persons who fit to the “considerateness” type are sometimes also prepared to restrict the privacy of those affected, but only if it serves their safety, the use of the device has been agreed upon in advance, and the ability to monitor those affected is restricted to the circle of relatives or, considering the progression of dementia, to medical or nursing staff. In summary, autonomy is still the central value of this type, but autonomy here means primarily the autonomy of family caregivers. While people with dementia who fit this type also value the independence that using a MAT affords them, here too, the focus is on relieving the burden on caregivers, resulting in an increased level of their autonomy. This corresponds to the common desire of people with incipient dementia to not be a burden on anyone (Aldridge et al. 2019). This type differs from the “individual self-determination” type primarily in that security plays a greater role and intrusions into privacy are tolerated in order to promote and maintain it.

These examples show how VPPs can contribute to an answer to our initial question of how the value preferences of cognitively impaired users can be integrated into the functioning of AI-based MATs. The types discussed above are ideal types of preferences regarding the implementation and functioning of technical assistive systems as identified empirically in our sample of individuals with dementia and their caregivers. The identification of the reasoning underlying specific value preferences also enables a more fine-grained alignment with value beliefs to ultimately increase the perceived benefits of assistive systems for patients and caregivers. In an area as ethically fraught as automated forms of dementia care, however, this process would have to be informed additionally by experts from medicine, ethics, and information technology in actual applications.

4 Towards putting VPPs into practice: a simulation of a caregiver technical notification system to support disoriented persons with dementia

Prior to integrating VPPs into assistive systems in practice, we need to measure the extent to which the profiles actually guide the actions of assistive systems. The challenging parts are A) operationalizing the VPPs for a particular technical assistive system, B) formalizing a methodology to measure assistive systems’ adherence to the operationalized profiles, C) testing the proposed operationalization and measurement on a data set that includes usage of a technical assistive system in a caregiving scenario. We address A and B by proposing the ethical compliance quantification (ECQ) method explained below in Sect. 4.2. For C, we propose a simulation approach. Since our focus is on providing assistance to PwD, we require a behavior simulation tool to model navigation and assistance (personal and technical) for PwD. For the sake of simplicity, we will limit our focus to a nursing-home setting.

To further understand the requirements for such a simulation, let us refer to the case of Mr. Alois and Ms. Beatrice. Mr. Alois and Ms. Beatrice are permanent residents at a nursing home. In this nursing home, they have their private rooms and they can perform daily activities such as going to clinical therapy, hosting visitors, and dining. In addition to the in-house nursing staff, they are also provided with a personal smartwatch that can help them navigate and alert caregivers if needed. To understand what type of assistance we model, let us suppose that Mr. Alois has the “individual self-determination” profile and Ms. Beatrice the “safety” profile (Sect. 3). Assume that Mr. Alois and Ms. Beatrice leave their private rooms at 09:00 for a medical appointment and breakfast, respectively. While navigating on their own, they eventually become disoriented. The smartwatch tries to help them by issuing navigation interventions. Assuming that the smartwatch’s first intervention fails in both cases, the smartwatch can either alert caregivers or issue another navigation intervention. As Ms. Beatrice belongs to the “safety” profile and would not oppose human assistance, the smartwatch alerts caregivers to provide a physical intervention. Mr. Alois’ smartwatch, in contrast, initiates several attempts to reorient him and eventually succeeds. Both reach their final destinations. Ms. Beatrice reached her destination quicker and with lower risk to safety, as the smartwatch alerted human assistance at the onset of her state of helplessness. Mr. Alois arrived at his destination temporally delayed, after having spent more time in a vulnerable state, yet, his preference for autonomy was accounted for.

4.1 Behavior simulation

Reflecting on the case-study above, we formalize the requirements needed to simulate such scenarios. First, we require a spatial environment (e.g., a nursing home) where different agents of the simulation, such as PwD, nurses, and smartwatches can communicate and interact with each other. In this context, a number of similar behavior-modelling tools have been proposed (Huang et al. 2013; Li et al. 2016). However, these tools either do not consider cognitive aspects of spatial navigation (e.g., they assume cognitively healthy individuals) or their spatial navigation produces random walk action (Andresen et al. 2016; Manning et al. 2014). Because adapting existing tools was not an option, we developed our own domain-specific simulator SimDem (Shaukat et al. 2021). In the following, we describe those aspects of the simulator relevant to this research.

SimDem is a multi-agent-based tool that aims to model navigation and assistance of PwD in indoor environments. It includes a PwD agent, nursing agents, smartwatch assistance, and a spatial environment. We briefly discuss these features below.

Environment: SimDem simulates the spatial environment as a two-dimensional grid where agents interact with each other. It also provides spatial features such as patient rooms, appointment locations, and a nurses’ room. Figure 2 shows an example environment.

Fig. 2
figure 2

SimDem spatial environment

Full size image

PwD agents: PwD agents reside in a private room (e.g., P1 in Fig. 2) and leave for their appointments, e. g. physiotherapy, as provided in a daily schedule. While navigating to these appointments (and returning to their private room), PwD agents can be spatially oriented or disoriented, where the probability of disorientation is defined by the parameter \({p}_{d}\in [\mathrm{0,1}]\). Essential PwD agent parameters are:

  • \(T\) is the schedule of each PwD agent and contains a set of appointments for each simulation.

  • \(q\in \{{q}_{O},{q}_{D},{q}_{G}\}\) is the cognitive state of the PwD agent i.e., whether the agent is currently oriented (\({q}_{O}\)), disoriented (\({q}_{D}\)) or guided (\({q}_{G}\)) by a nurse agent.

  • \({p}_{d}\in [\mathrm{0,1}]\) is the disorientation level and models the probability of disorientation while navigating to an appointment.

  • \({p}_{i}\in [\mathrm{0,1}]\) denotes the probability that a navigation intervention of a smartwatch will be successful (i.e., helps PwD to regain orientation).

Smartwatch agents: Each PwD agent can be equipped with a smartwatch agent. A smartwatch agent can attempt to reorient a disoriented patient by issuing a navigation intervention which provides an instruction to the PwD agent or a help intervention which calls a nurse agent to the location of the PwD agent. Smartwatch parameters are:

  • \({p}_{detect}\) ∈ [0,1] denotes the probability with which disorientation of the patient agent will be detected correctly.

  • \({n}_{help}\in {N}^{*}\) denotes the maximum number of consecutive, failed navigation interventions (\(n\in N\)) before a help intervention is triggered (i.e., before a nurse agent is called).

Nurse agents: In addition to the technical assistance, SimDem also models human assistance via nurse agents. A nursing agent can guide a PwD agent if they perceive a disoriented PwD agent within their visual field of view (modelled by the ray-casting algorithm (Amanatides and Woo 1987)) or if a PwD agent’s smartwatch issues a help intervention and provides their location to the nurse agent.

4.2 Ethical compliance quantification (ECQ) and its utilization in the simulated environment

As outlined in Sect. 5, an operationalization and measurement mechanism is needed to analyze VPPs for use in an assistance scenario. In order to evaluate functioning from the perspective of the users, VPPs must be translated into numerical values. This does not promote “moral arithmetic”, which is the attempt to translate moral values into numbers to quantify and simply net them against each other. Thus, the claim is not that VPPs can be used to program outcomes, i.e., functions of the MATs, which are ethical in themselves. Rather, the VPPs represent an attempt to map different value concepts with regard to the implementation and functioning of technical assistive systems in their application and behavior. The application of VPPs in the simulation described below, with the introduction of the concept of ethical compliance quantification (ECQ), shows one way this can be accomplished.

The ECQ method evaluates action(s) executed by an assistive system against ethical preferences (e.g., VPPs) provided by the stakeholders. It consists of two steps, value selection and value measurement:

  1. 1.

    Value selection comprises defining the ethical value set against which the actions of system can be evaluated, e.g., autonomy of PwD.

  2. 2.

    Value measurement outlines the precise criteria for each ethical value as defined in step 1 that can be used to evaluate a system’s compliance in regard to that particular ethical value. For instance, being guided by the nurse violates PwD’s autonomy.

We used the VPPs and the SimDem simulator to perform ECQ and defined the process for this particular use-case as follows.

Value selection: As a proof-of-concept, we used the preference profiles “individual self-determination,” “considerateness” and “safety” as the ethical value set. In a next step, we evaluated the assistive system’s compliance against the respective value set.

Value measurement: We define the violation criteria (i.e., what action(s) violate a preference profile) for ethical values as follows.

Individual self-determination: As the respondents of the “individual self-determination” profile see both human and technical support as a violation to their autonomy, we define violation of “individual self-determination” (\({V}_{{\text{ISD}}}\)) as:

$${V}_{{\text{ISD}}}={n}_{{\text{si}}}+{n}_{{\text{ge}}}$$

In this definition, \({n}_{{\text{si}}}\) is the number of the smartwatch interventions per simulation (i.e., technical intervention) and \({n}_{{\text{ge}}}\) denotes the number of guidance episodes per simulation (i.e., human intervention). Each guidance episode represents a time period for which a PwD agent is being guided by a nurse agent.

Considerateness: We assume that respondents of the “considerateness” profile deem the notification of a human caregiver as a violation to the caregivers’ autonomy. We define the violation of “considerateness” (\({V}_{{\text{RA}}}\)) as the ratio of time a PwD agent is guided by the nurse agent (\({t}_{{\text{guided}}}\)) relative to the total duration (\({t}_{{\text{total}}}\)) of the simulation (e.g. 9 h):

$${V}_{{\text{RA}}}=\frac{{{\text{t}}}_{{\text{guided}}}}{{t}_{{\text{total}}}}$$

Safety: We measure safety in terms of the average time it takes a PwD agent to get reoriented. This measure tells us how long it takes for a PwD agent to get reoriented (\({q}_{O}\)) after getting disoriented (\({q}_{D}\)). Formally, we define violation of safety (\({V}_{S}\)) as:

$${V}_{S}=\frac{1}{n}\stackrel{n}{\sum_{i=1}}({t}_{{q}_{D}\to {q}_{O}}{)}_{i}$$

where \({t}_{{q}_{D}\to {q}_{O}}\) states transition time \(({q}_{D}\to {q}_{O})\) for each disorientation instance \(i\) and \(n\) is the total number of \({q}_{D}\to {q}_{O}\) instances per simulation.

To perform the ECQ against selected values (\({V}_{ISD},{V}_{RA},{V}_{S}\)), the following experimental design was used for SimDem.

PwD Agents: For all experiments, we simulated five PwD agents whose home location was as shown in Fig. 2. Tuning parameters were set as:

  • \(T\): For each simulation run and PwD agent, 6 unique appointments were selected. Type and location of these appointments were a subset of locations shown in Fig. 2 (e.g. dining).

  • \({p}_{d}\): Five disorientation levels were selected as 0, 0.25, 0.50, 0.75, and 1.00 for all PwD agents.

  • \({p}_{i}\): Probability of a successful intervention was set as 0.20 and 0.50 for each patient.

Smartwatch Agents: Each PwD agent was equipped with a smartwatch using following parameters:

  • \({p}_{{\text{detect}}}\) was set as 0.5 and 0.2 for all experiments.

  • \({n}_{{\text{help}}}\): Six values of \({n}_{{\text{help}}}\) were experimented from 0 to 5.

Nurse Agents: Each simulation (where nurse assistance was simulated) had three nurse agents.

4.3 Evaluation method and simulation outcomes

We investigate the ECQ for the selected preference profiles using different assistive strategies. These strategies are modelled by the smartwatch parameter \({n}_{{\text{help}}}\). By increasing the \({n}_{{\text{help}}}\) value, the calling the-nurse-agent action is delayed. For instance, if \({n}_{{\text{help}}}\) = 2, the smartwatch attempts to reorient the PwD agent two times before calling a nurse. Whereas if \({n}_{{\text{help}}}\) = 0, the smartwatch immediately informs a nurse agent. Additionally, we also report the results without smartwatch assistance (i.e., only nurse agents) as well as without any type of assistance.

Figure 3 shows individual self-determination violations (\({V}_{{\text{ISD}}}\)) versus different types of (or lack of) assistance. In the case of no assistance, the number of smartwatch interventions and guided episodes are zero (i.e., \({n}_{{\text{si}}}={n}_{g}=0\)), therefore \({V}_{{\text{ISD}}}\) = 0. In case of assistance, the lowest number of violations occurred when only caregiver assistance was provided. Moreover, violations (\({V}_{{\text{ISD}}}\)) increase with delays in calls to a nurse agent (i.e., increase the \({n}_{{\text{help}}}\)). This is because delaying caregiver intervention (i.e., setting higher \({n}_{{\text{help}}}\) value) allows the smartwatch to make a higher number of navigation attempts (\({n}_{{\text{si}}}\)).

Fig. 3
figure 3

Individual self-determination violations (\({V}_{{\text{ISD}}}\)) with different assistive strategies

Full size image

Figure 4 depicts considerateness violations (\({V}_{{\text{RA}}}\)) versus assistance. It is interesting to observe that violations (\({V}_{{\text{RA}}}\)) are higher for \({n}_{{\text{help}}}\) = 0 (i.e., immediately calling for caregiver intervention upon disorientation) as compared to having no smartwatch assistance at all (i.e., nurse-only assistance). In other words, PwD agents get more guidance time if the smartwatch acts only as an alarming mechanism and informs the caregivers immediately. However, in the case of assistance without a smartwatch, caregivers are unaware of disoriented PwD agents unless they visually perceive them. Moreover, violations (\({V}_{{\text{RA}}}\)) decrease as we increase \({n}_{help}\) (delayed caregiver intervention) because more smartwatch attempts are initiated to reorient PwD agents before asking for nurse agent’s “physical” help.

Fig. 4
figure 4

Considerateness violations (\({V}_{{\text{RA}}}\)) with different assistive strategies

Full size image

Figure 5 shows safety violations (\({V}_{S}\)) versus assistance type. First, the addition of the smartwatch assistant improves reorientation time \(({t}_{{q}_{D}\to {q}_{O}})\) and consequently safety, as compared to no assistance and nurse-only assistance. In the case of no assistance, PwD agents rely on themselves to get reoriented (depending on their disorientation level \({p}_{d}\)). Whereas, for the nurse-only case PwD need to be guided first (\({q}_{G}\)) before getting reoriented (\({q}_{O}\)). Furthermore, increasing \({n}_{{\text{help}}}\) decreases safety violations (\({V}_{S}\)) because PwD agents are given more time to recover \(({t}_{{q}_{D}\to {q}_{O}})\) before a nurse agent is called.

Fig. 5
figure 5

Safety violations (\({V}_{{\text{S}}}\)) with different assistive strategies

Full size image

Regarding behavior simulation and value preference profiles, two important points should be noted. First, simulation-based ECQ allows for the comparison of how different assistive strategies perform when guided by different preference profiles. Here, we also checked whether and to what extent the preferences expressed in the interviews and their transfer into the VPPs could be accommodated. That is, the results of the simulation should correspond with the preferences expressed in the interviews and represented in the VPPs. The simulation also allows for experimentation, which would be impossible in real-world scenarios. This includes situations that would be unethical in real life, such as providing no assistance at all. Lastly, the most critical aspect of ECQ is to precisely define value measurement criteria. Such an objective function is required to accurately estimate what action(s) of an assistive system violate which ethical preference(s). The main contribution of our work is thus neither to perfectly define preference profiles for specific scenarios nor to provide a flawless behavior simulator. Rather, we provide a methodology to apply preference profiles using a simulation approach. Before we turn to the discussion of our approach in general, we briefly review the results of the individual profiles in the simulation with regard to their plausibility.

The profile “individual self-determination” is characterized by a high appreciation of autonomy and independence. This is also reflected in the simulation. Thus, any intervention—with or without a smartwatch—is perceived as a violation of autonomy. Thereby, the degree of perceived violations increases with the number of unsuccessful reorientation-attempts by the smart watch. According to the results of the simulation, this means that the ideal setting of a smartwatch for persons who fit to the type “individual self-determination” is to directly notify a support person. One could argue, that thereby, only one intervention—perceived as a violation of autonomy—is necessary. However, a notification and subsequent assistance from a caregiver may also be perceived as a more severe intrusion on autonomy than a reorientation attempt made by the smartwatch. Here, a differentiation between an autonomy violation by caregiver and by smartwatch would be necessary. It was also not possible to clearly deduce from the profiles whether, for every possible representative of this idealized type, assistance by a human being would always be preferable to a smartwatch. Rather, the number of attempts to bring about a change in behavior seemed to be the decisive factor. In this context, it is important to discuss whether the support provided by a caregiver can be regarded as an expression of relational autonomy, following (Stoljar and Mackenzie 2023) and (Childress 2022). This means that the autonomy of the cared-for person can be guaranteed precisely by the support provided by the caregiver. The concept of relational autonomy includes others in the description of a person’s autonomy and thus needs to be distinguished from conventional individualistic concepts of autonomy. This aspect would have to be further distinguished from support by an assistance system, which could also increase the independence and autonomy of the user but rather in the sense of empowerment, i.e., the process of gaining control over health-related decisions and actions (Welsch and Buhr 2022).

The next profile tested in the simulation is the “considerateness” profile. Individuals who fit this profile are very considerate regarding the autonomy of their relatives and, thus, assess technical MATs primarily on the grounds of the extent to which they provide relief for relatives. It is striking that, according to the simulation, the perceived interference with the smartwatch users’ understanding of autonomy is highest when a smartwatch is worn, and it directly alerts a caregiver without making any reorientation attempts. In this case, the smartwatch is perceived as an interference with both the wearer’s and the caregiver’s autonomy. The violations of autonomy by the smartwatch are, therefore, not to be understood primarily as violations of users’ autonomy but rather as violations (also) of caregivers’ autonomy. The simulation was able to numerically map these preferences, thus showing that it is possible to integrate user preferences relating to third parties into the mode of operation and also that it is possible to differentiate between the “individual self-determination” and “considerateness” types of autonomy. This again demonstrates the benefit of eliciting and considering the underlying reasoning of interview participants regarding their value preferences. Here, the negative dimension of autonomy is the main concern. That is, any interference in a person’s course of action is perceived as a violation of their autonomy (May 1994). Also, regarding the results of this profile, it can be argued that MATs are evaluated positively when they offer independence to the provider, so that the caregivers do not experience any restriction in their negative freedom. Further data collection would be needed to sharpen this profile. This could allow for a distinction between professional and informal caregivers, if necessary.

In this model, perceived autonomy violations decrease successively with unsuccessful orientation attempts made by the smartwatch. The lowest value can be found at five unsuccessful reorientation attempts. This means that as the smartwatch makes more orientation attempts, perceived interference falls. This raises the question whether this process can be continued infinitely. It seems implausible to assume that a smartwatch user would prefer one hundred unsuccessful reorientation attempts to three, for example. This suggests that the inclusion of the caregivers’ perspective is missing here. It is questionable whether not wearing a smartwatch—meaning that receiving help is only possible if the caregiver physically sees a disoriented resident—represents a higher interference on caregiver autonomy. It is possible that caregivers would, on the contrary, also perceive knowing that a smartwatch will alert them when necessary, as increased autonomy because they would be relieved of the need to physically surveil their charges. This means that in order to improve the results of the simulation, further parameters need to be included in the calculation.

The third VPP checked for possible preference violations using the simulation is the “safety” profile. Persons who fit to this profile evaluate MATs according to the extent to which they increase user safety. Thus, the greatest violation of the subjects’ preference occurs when no assistance is provided at all, either by a smartwatch or by a caregiver. Likewise, it is evident from the simulation that for those who fit this profile, the triggering of assistance by a caregiver is in itself an insufficient outcome. Rather, it is necessary for the caregiver to monitor or accompany the person continually. The scenario described above, however, describes a nursing home setting, where the uninterrupted surveillance of every resident is neither necessary nor possible. Accordingly, the settings in which a smartwatch is worn are consistently rated better. Strikingly, the ratings increase with the number of unsuccessful reorientation attempts before a caregiver is alerted. It is unclear whether this actually contributes to an increase in safety or whether persons who fit to this type would actually prefer that a nurse is alerted more quickly, for example in order to check whether reorientation attempts might have been unsuccessful due to a health complication. The results seem to indicate that multiple reorientation attempts would be assessed as an increase in safety. This is implausible due to the limited technical capabilities of a smartwatch. Here, it could make sense to define a threshold above which the safety parameter does not increase with each successive smartwatch reorientation attempt.

5 Discussion and outlook

Which insights for the development and implementation of technical assistive systems in the care of PwD can be gained from VPPs and their application in the simulation, described from the perspectives of ethics and computer science? Technology development requires developers to design their products and solutions in a way that their functioning aligns with the preferences and needs of end-users (Zhu et al. 2018; Ienca et al. 2018; Floridi et al. 2018). To this end, our approach takes a crucial step towards accommodating users’ preference profiles in the decision-making process. In order to do this, assistive systems need to be evaluated in different situations to learn what actions lead to preferred outcomes for different value preferences (Prem 2023; Lee et al. 2021). Our preference profiles combined with a simulation-based ECQ approach provide a method for systems to learn such action policies. As the actions of assistance are (ethically) evaluated against users’ preferences, we outline how preference profiles extrapolated from end-users can eventually be used to learn about the consequences of the decision-making strategies of assistive technologies. This is an important step towards developing user-centered technologies—especially for people with cognitive impairments whose perspectives were usually left out by conventional ethics-by-design approaches.

With the development of VPPs from our interview material, we explored a method for incorporating the preferences and values of people with cognitive impairments in general and PwD in particular in the development and use of assistive technologies. As it may be difficult for PwD to articulate their preferences—especially during later stages of the disease—it was crucial to develop a method to discover actual preferences and translate them into generalizable modes of functioning. The VPPs thus represent generalized ideal types of value-based preferences regarding the functioning of assistive technologies. Through a qualitative analysis of interviews with persons with dementia and their caregivers, we were able to identify several ideal value preference types with different core values, value weights, and/or lines of reasoning underlying those values. In particular, the finding that different lines of reasoning can underlie the same values and value weights underscores the importance of qualitative research in this context. Considering the underlying reasoning of value settings sets VPPs apart from other ethics-by-design approaches. While the latter merely ask about principles without ascertaining their concrete meaning in application (Prem 2023), our approach aims to explicate the principles by analyzing the underlying argumentation, to identify distinct dimensions of meaning and to simulate their application. The behavior simulation supports the iterative process of ethics-by-design approaches and allows us to test the implemented preferences on the part of the stakeholders with regard to their intended functionality. This increases the control that future users have over the functioning of the assistive systems and strengthens their social acceptability (Floridi and Taddeo 2016). While constructing the VPPs, one major challenge was to quantify expressed preferences into numerical values and weights. This fundamental techno-ethical problem was not resolved here. It is also not foreseeable that it can be solved in the near future. The implementation of ethical principles into the functioning of AI-based technologies necessarily requires a quantification of moral preferences. Similar to a linguistic text-to-text translation, aspects of meaning are inevitably lost in this process. We attempted to mitigate this problem by translating these preferences transparently. In doing so, we address the issue that many approaches propose rather basic principles and do not suggest any details regarding their implementation (Prem 2023). This is not least in response to the demand of a majority of tech workers for practical resources for the implementation of ethical principles (Miller and Coldicutt 2019).

However, to improve our approach and minimize the loss of meaning in translation, further steps are necessary. While the simple behavioral simulation used here is suitable for testing our exploratory approach and for a first test of VPPs (Schicktanz et al. 2023), it is important to further refine our approach in the future. Thus, broader data sets are needed comprising both qualitative data to enable the identification of additional relevant ethical principles (see, for example, (Köhler et al. 2022)), and quantitative data to further sharpen the weighing of individual principles within individual VPPs and to facilitate the translation of preferences into numerical values. Furthermore, dimensions of meaning of specific terms need to be differentiated, such as positive, negative, and relational autonomy (Moffat 2023; Stoljar and Mackenzie 2023). Moreover, it still appears desirable for proxy-by-peer approaches to give final adjustment options to end-users and their caregivers. With this in mind, the algorithm could also be further refined—either through manual calibration or through an application of AI that uses the results of the simulation and further survey data to improve the translation of VPPs into technical functionality. Another possibility would be to train the simulation via data augmentation, a technique in which values are repetitively changed slightly, resulting in a minimally different setting of the MATs (Shorten and Khoshgoftaar 2019) in an effort to find optimal outcomes, i.e., functioning of the MATs. For this, however, it would first be crucial to collect more real-life data, i.e., not only data generated in a simulation but of people using MATs and giving feedback to its functioning. Eventually, due to the small sample size, the VPPs presented here cannot be generalized without further research, either in the form of qualitative studies that explore further possible dimensions of evaluation on the part of those affected or in the form of quantitative studies, that identify additional ethical principles from the perspective of relevant stakeholders.

The discussion of our simulation results showed that, in some cases, unclear or ambivalent results occur. Only in the case of the “individual self-determination” profile, the simulation was able to align outcomes with the preferences of this type appropriately—under the assumption that an intervention by a human caregiver is always preferred to that of a smartwatch. However, this seems to be an unusually unambivalent preference: any kind of assistance is rejected as an unjustified interference with one’s own autonomy. This shows that clear-cut, bimodal preferences can be more easily translated into technical modes of operation. For further testing of the ex-ante preference setting of MATs in the care of PwD, it is therefore crucial to feed a variety of easily calculable but also competing values into the simulation. Furthermore, it is important to distinguish between the understandings of autonomy identified here and to utilize these different understandings for the design of MATs. Thus, a distinction would have to be made between the negative dimension of autonomy, as addressed primarily in the “considerateness” profile, and the emphasis on the positive dimension of autonomy, with MATs serving an empowering function. Furthermore, relational aspects of autonomy (Mackenzie 2008; Stoljar and Mackenzie 2023), as addressed primarily in the individual self-determination profile, should be considered.

Another aspect, touched upon earlier, concerns the challenge that the problem of “different selves” in the context of dementia poses for the temporally stable setting of the functioning of MATs. It should be noted that while VPPs do not solve this challenge either, we believe that they present a way to mitigate it since they allow for a switch between different ideal–typical settings. Although the possibility of a change in moral preferences is not exclusive to PwD, the diminishing ability to speak as the disease progresses makes it increasingly difficult for outsiders to perceive such changes. This means that caretakers need to be aware that a change of preferences on the part of the PwD regarding the functioning of a MAT might occur. However, it can also be noted here that even if people with dementia are sometimes no longer able to express their self in narration and judgment, it can still be persistently reflected in their behavior over time (Hutmacher 2021).

Overall, it should be emphasized once again that the approach chosen here represents an attempt to translate individual preferences and value concepts into ethically relevant machine functionalities. To our knowledge, there have been only few attempts to develop ethical algorithms (Anderson et al. 2006; Meier et al. 2022). We are not aware of any work that has thought this through for people with dementia or non-autonomous and non-consenting individuals. While the Patient Preference Predictor approach also addresses cognitively impaired individuals (Rid and Wendler 2014; Biller-Andorno and Biller 2019), and VPPs can also serve as a support for ethically relevant decisions, the focus of VPPs should be distinguished. In contrast to the Patient Preference Predictor, it firstly targets the functioning of assistance systems in accordance with the moral preferences of users and, secondly, also considers the underlying reasoning of value decisions. Against this background, our approach should be understood as an approximation attempting to incorporate the preferences of persons with dementia and cognitively disabled persons in the functioning of MATs. This represents a particularly difficult case for an ethics-by-design approach. By identifying possible evaluation dimensions of MATs and determining different types (VPPs) it should be possible to map the preferences in an ideal–typical form and thus enable participation via proxy-by-peer participation. This requires further efforts, both empirically and ethically, as well as from a computer science perspective. The interdisciplinary approach pursued here can be regarded as a first step on the long road ahead.