1 Introduction

Aging brings cognitive and physical challenges [1], which may hamper agency and choice [2]. Older adults might have reduced agency in their lives; this can be exacerbated when they live in nursing homes or other residential facilities. In these contexts, older adults may experience reduced communication and interaction with others due to personality traits, motor or cognitive impairments, or even an unfavorable institutional environment. These factors might negatively impact their ability to decide and influence the environment, decreasing the sense of agency [3, 4]. That, in turn, may translate to decreased social integration, reduced capacity of adaptation to new situations, and a limited ability to express themselves independently [2].

Previous work has looked at assistive robots as a way to allow people to live more independently at home or actively in a care home. Examples include assistive robots that take the role of a nurse [5, 6] or companion [7], robots as exercise coaches in care homes [8, 9], or robots as promoters of routines and communication [10]. While many approaches often position the robot in a caregiving or guiding role [8, 11, 12], the dynamics of agency in these interactions can vary significantly. For instance, telepresence robots demonstrate scenarios where older adults exert control and agency, as highlighted in, [13] and [14]. These robots have been employed to reduce loneliness and social isolation among the elderly, especially during the COVID-19 pandemic, emphasizing user-led interaction and autonomy. Fiorini et. al., (2022) research with frail older adults living with a telepresence robot underscores the high user experience and system reliability, showcasing how these systems can empower older adults in care services [14]. These studies provide compelling evidence that older adults can effectively control robots, challenging the traditional notion of robots having predominant agency.

Given these insights, we argue for a nuanced understanding of agency distribution in human-robot interactions. In some cases, shifting more agency towards users, particularly older adults, may be beneficial to enhance their independence and engagement. Potentially, in the future, robots can serve as an extension tool, a vehicle to amplify the actions that people want to accomplish according to their wills and intentions. In this qualitative work, we take the first steps toward our vision by increasing the agency of older adults in a care home in a specific task by providing them with a set of tools to define the goals of the robot.

This leads to our first research question:

RQ1. How do older adults extend their communication and agency through a controlled robot?

We designed an activity where participants build sequences of actions, consisting of sending different verbal messages and objects (e.g., cookies) to other people, by using tangible blocks to command the robot, enabling the user to lead the interactions. We selected tangible blocks to bridge possible digital gaps among older adults and for the ease of use and simplified interaction with technology without prior computer knowledge [15].

This leads to our second research question:

RQ2. What challenges and barriers do older people face in controlling a robot through tangible blocks?

Following preliminary discussions with professionals, we deployed this activity in a care home with 22 older adults, with two groups of people positioned in adjacent rooms. The groups were encouraged to autonomously program sequences of actions to control the robot and remotely interact with the other group. The sessions with older people were video-recorded and later analysed using thematic analysis. Additionally, we conducted interviews with the care home professionals to review and discuss excerpts of the sessions to triangulate and enrich our analysis.

We observed that participants enjoyed the activity and, upon understanding how to control the actions of the robot, appropriated it to create a playful environment. Asymmetries in cognitive ability and personality had a great impact on how involved people were, suggesting future approaches should explore how the robot can be active in balancing control [16]. We contribute findings and reflections that can inform the design of future human-robot interactions, and human-human interactions through robots, for older adults. Additionally, we argue for a paradigm shift where the values of older adults are at the core of the design of human-robot interaction.

2 Related Work

As people age, the need for relationships and maintaining strong ties remains as in youth [17, 18]. However, older people often experience reductions in their social spheres, which may be aggravated due to physical or physiological impairments [19]. It has been shown how strengthened social spheres lead to greater perceived emotional satisfaction [20] and aid in slowing down cognitive deterioration. [21, 22].

Socially Assistive Robotics (SAR) has been widely examined for use with older adults, providing emotional support [23,24,25], helping manage routines [26, 27], offering new and sustained social activities, and providing therapeutic support [28,29,30]. Some studies have focused on older adults’ conversations with a robot, demonstrating that people tend to respond to a robot as if it was a human being [31], due to its perceived presence [32]. As social robots gain more aptitude to enroll in society, more complexity will exist in social contexts [33]. In this way, the perception of robots as tools is being diluted and is being reframed instead as agents that interact with humans to jointly accomplish tasks [34]. This extension can be seen from the perspective of teammates [33], who might help in more industrial [35] or social tasks, disaster environments [36], hospital contexts [37], schools [38], day centers and nursing homes [39].

Past research [33] focuses on the perception of complementarity and agency—for the robot to be perceived as a social agent, people have to perceive it as a team member. Additionally, research points to the importance of assigning social agency to robots [40]. One of the ways of attributing social agency to robots is related to turn-taking between robots and humans [40]. When there is an interaction by turns, the perceptions of the robot as a social actor proliferate, although there may be fluctuations in this perception depending on the robot’s limitations [40].

Further studies also show a higher sense of control of the robot when people command a robot [41]. However, if people do not have a successful experience controlling the robot and it is perceived as a failure repeatedly, it can generate frustration [42] which is essential to anticipate and minimize.

A number of HRI studies with older adults explore the agency of robots [9, 43,44,45]. Much of this focus is placed on the robot’s autonomy as an agent interacting with a human [9, 43, 44]. Often, robots are mentioned as agents who will act in users’ environment to change it through decision-making [41]. Accordingly, several robotic approaches put older adults exposed to actions previously programmed to be specific [6, 9]. Commonly, with these interaction methods, people are not in control of the robot’s initiative and actions.

To the best of our knowledge, there are only few examples of older adults leading the interaction. In [46], older adults controlled a teleoperated robot to greet children in nursery schools. Nevertheless, the focus was more on the acceptance and perceptions of children than on how older adults explored the interaction.

This body of research helps us understand the current status of agency in HRI and provides valuable insights into the transfer of agency from robots to humans. Despite advances in HRI regarding agency transfer to humans, it is crucial to establish a more robust conceptualization of agency transfer in HRI, thereby facilitating further progress in this direction.

Previous studies have explored giving people agency of control [47], allowing them to program the robot’s behavior. One of the adopted strategies is using tangible blocks. These have perceived advantages of engagement and attention span [48]. Tangible blocks allow people to have a holistic and physical view of simple programming or a sequence of actions, having shown to be successful with children [48,49,50]. This component is beneficial from a learning perspective as it allows the person to visualize and rationalize the order of actions one is creating [48]. Specifically, with an older population, Gooch et al. [51] focused on designing tangible prototypes to express emotions and communicate with family and friends. In our work, we explore how older adults living in nursing homes use tangible blocks to send messages and objects to their peers using a robot as a carrier-pigeon, aiming to understand how people appropriate the activity and perceive the role of the robot.

Fig. 1
figure 1

Tangible blocks used in the activity: Tasks blocks (the ‘cookie’ block used to send cookies or sweets; the ‘greet’, ‘music’, ‘proverbs’, and ‘ask for lunch’ blocks used to send pre-recorded audio messages), Record block, Target blocks, and Play block

Full size image

3 A Robot as a Carrier-Pigeon

Our concept builds on the early days of remote communication: a carrier pigeon. The robot is a communication vehicle, purposefully limited in its agency but still rich in the ability to act when commanded to do so. The robot is (conceptually) autonomous in navigation but requires the initiative of the person to define the task goal.

The concepts of autonomy and agency in HCI are ambiguous and overlap in the literature [52]. Agency can be identified as a feeling in control of one’s own actions [53], overcoming barriers to acting and choosing, and finding meaning, whether individually or in a group [52]. In our work, we consider that humans have the agency to define the robot’s goals, while the robot has the agency to make lower-level decisions to ensure the goal is achieved autonomously.

Although the robot can be commanded to perform several tasks, the carrier pigeon metaphor offers a simple and user-friendly approach to message delivery. This simplicity is emphasized through tangible blocks, allowing individuals to engage with the system easily, even if they have limited technological experience. Additionally, the metaphor captures the efficiency and reliability associated with carrier pigeons. Our approach allows a robot to reach the intended target destination, delivering messages and objects sent by a human.

With a conventional carrier pigeon, people have to write a message on paper and send it. In our scenario, to send a message, older adults have to compose their message or sequence of messages through tangible blocks (Fig. 1). Tangible blocks refer to physical building blocks or objects that can be touched and manipulated, typically used for educational or therapeutic purposes [54]. There are four groups of blocks: white blocks with different tasks from pre-recorded voice messages to carrying items; a red block to record audio; dark grey target blocks that define whether the message is sent to the other group or a nursing-home staff; and, finally, the blue starting piece for initiating the sequence. The blocks fit together to produce a 3 to 7 blocks sequence. They were printed in 3D as miniatures representing their corresponding actions and painted with a plain color associated with the type of block.

First, users have to define what actions they want the robot to perform by building a sequence with the tasks and ‘recording’ block. Then, they have to choose the target by choosing one of the grey blocks. When satisfied with the sequence assembled, people connect the blue starting piece so that the robot can start its journey. The robot, in our case a Dash robot,Footnote 1 autonomously travels to its destination and deploys the messages/objects in the order they were created. An example of a possible sequence is: (i) arrive and speak out ‘Hi There!’ (‘greet’ block), (ii) play a recorded audio (a riddle recorded by the sender with the ‘record’ block), and (iii) ask the receiver to pick the cookie on its head plate (‘cookie’ block). While the message is being delivered, a video stream is being shown to the sender of the message.

4 User Study

We contacted a local institution that provides daycare and residential nursing home services for older adults. Initially, we conducted a 60-minute session with two professionals, where we presented our approach and refined a script for the proposed activity. Then, we scheduled two sessions on two different days in the institution, where we deployed the activity with groups of care home users recruited by the professionals who also participated in the sessions. The sessions were video recorded. Following the qualitative analysis of the recordings, we identified fundamental aspects of the experience to reflect on and discuss. As such, we conducted ensuing sessions; one group interview with the two aforementioned professionals and, an individual interview with a psychologist. These sessions were essential to triangulate our perspectives on the experience and ended up significantly enriching our thematic analysis. The study was approved by the ethics committee of our school (“Comissão de Ética para a Recolha e Protecção de Dados de Ciências") and all participants provided written informed consent.

4.1 Procedure

The activity was deployed in two contiguous rooms within the institution’s building, with an open door giving access from one room to the other. In each room, one group of three or four participants sat around a set of clustered tables. There was one researcher to accompany each group and a third researcher to remotely control the robot. The two professionals were also present for most of the sessions, mobilizing participants and assisting the research team in onboarding and mediating the groups. These were rooms where care home users routinely rest, talk, and watch television. As such, other users, who were not recruited, were also present in the room.

There was a set of tangible blocks for each group and a sheet with a schematic showing the different blocks and a caption indicating their function (Fig. 1). The blocks were representative of each action. We followed a Wizard of Oz procedure, where a researcher controlled the robot, observing the sequence of participants’ actions, and commanding the robot accordingly with a tablet.Footnote 2 Each group had at their front a laptop showing them the video feed of the other group. This content was recorded and used for analysis.

At the start of the activity, the research team presented themselves and explained the goal of the activity (i.e. to interact with the group in the other room through the robot) and how participants were able to command the robot, detailing the type of actions each tangible block represented (leveraging the schematic sheet). In each session, one group was selected to start with the robot and send the first message to the second group by building the action sequence with the tangible blocks. Then, the robot would be placed on the floor, commanded to go to the other room, and perform the actions in the order corresponding to the sequence. This happened in turns, with groups alternatively sending and receiving a message through the robot. When exchanging a message, the sending group was encouraged to watch their colleagues’ reactions, through the video call (Fig. 2).

Fig. 2
figure 2

Activity setup with group A, during the second session

Full size image

4.2 Debriefing Interviews with Professionals

Interviews with professionals were conducted through an online video conference call. An initial interview was conducted with the two professionals who were current employees—a specialist in occupational therapy (T1) and one in sociocultural animation (T2), both experienced and trained to work with older adults. A second interview was conducted with a psychologist (T3) who previously worked in the institution. These sessions began with a presentation that described the approach and procedure of the activity. Afterwards, excerpts from the video recordings were shown through screen sharing, which we further discussed with the professionals. These excerpts were selected based on our initial videos analysis and corresponded to aspects of the experience that we wanted to focus on. These aspects were: 1) the onboarding of participants, 2) interaction with blocks, 3) building sequences, 4) interaction with the robot, 5) competition and collaboration, 6) future perspectives, and 7) negative reactions. The goal of these sessions was to confront our initial analysis with the interviewees’ perceptions as professionals who dealt with the population daily and were familiar with the domain. Also, as they knew participants directly, they could delve into how their behavior was affected by individual factors, such as abilities and personality traits, and how they reacted in comparison to other activities and challenges.

4.3 Participants

The institution we went to is a Private Social Solidarity Institution (meaning people pay according to their income), which has 50 people in daycare and 150 residents. Of the people who interacted with us, 15 are daily visitors and 7 are residents. We had 22 older adults in total taking part in the user study (P1-P22), with 3–4 people at a time per group in each of the two sessions. 17 participants were female, and 5 were male, with ages ranging from 68 to 88 (Mean=78.19, SD=5.65). The groups changed multiple times during the activity, with some participants leaving and others taking their place. All the participants participated voluntarily and did not receive any compensation for participating in the study.

4.4 Data Analysis

We followed the process of an inductive thematic analysis to examine our data [55]. Before the interview sessions, we initiated our analysis by watching the video recordings and annotating recurring ideas in a sheet with corresponding timestamps — these recurring ideas ranged from common behaviors observed in participants’ verbal and nonverbal interactions to purely descriptive codes (e.g., indicating when participants chose the ’greet’ action). One of the recordings (portraying group B of the first session) had no audio due to a technical issue. This recording was still coded, fundamentally based on participants’ gestural communication and observed use of the blocks and the robot. Two researchers (who were present in both sessions) were involved in this process from the beginning, first independently annotating the data and then meeting on multiple occasions to discuss codes with corresponding descriptions and examples. We used video coding software on later coding iterations, which allowed us to better associate labels to timestamps and visualize them. Codes were iteratively added, removed, and merged, eventually reaching a code book, which was then discussed among all authors.

Before advancing to the next phase of the analysis (searching for themes), we decided to expand the study by scheduling the interview sessions with the professionals, aiming to substantiate and enrich our perspectives. As mentioned above, for these sessions, we categorized participants’ interactions into seven dimensions of the experience that we found particularly relevant for our analysis. These dimensions and the selection of video excerpts were based on the codes we identified and the central concepts that characterize our approach. The interviews were recorded and then transcribed. Again, the same two researchers annotated the text independently, following the code book already established but adding new codes as needed. After reaching a new version of the code book, we proceeded to rationalize and discuss the overarching themes that encapsulated the experience observed in the user study and the perspectives given by professionals. The themes were discussed among authors, better defined, refined, and finally named. The final codebook is available.Footnote 3

5 Findings

We first present the themes that are inherently connected with the observed participants’ behavior during the activity while giving examples of their interactions with the components and people involved in the experience. Each theme presented intertwines perspectives given by professionals that provide a broader context to the situations encountered in the user study. The last two themes are primarily derived from insights from the interviews, in particular from professionals’ past experiences when developing and conducting activities to support older adults’ well-being.

5.1 Affection, Collaboration, and Competition

In turn, groups started by experimenting with the various actions, first opting for the simpler ones (greeting, asking for lunch). After sending the robot, participants were encouraged to observe the reactions of their colleagues through the video call, which they started to spontaneously do after the first exchanges. As participants could see who was in the other group, some were keen on sending messages to specific colleagues—“I wanna send a message to my friend, [name]" (P21). For example, two different groups decided to show their appreciation for the occupational therapists by sending them affectionate messages—“I want to tell [him] he is a good person and we like him" (P1). Especially on these occasions, participants were attentive to the reactions of the targeted colleague, by focusing on the video call. Also, while the activity prompted groups to communicate through the robot, there were instances where participants waved and gesticulated to the other group through the video call.

Within each group, some participants started to show more will and autonomy than others, when deciding which message to send and building the action sequences. During the activity, these participants ended up guiding most tasks, giving more suggestions of what to send, and leading the interaction with the blocks and engagement with the researchers. The therapists mentioned this is a typical behavior in other group activities, as care home users have different cognitive abilities (with some having more difficulties in keeping up) and interests (e.g., some not being receptive to activities with new technologies). However, in all groups, cooperative behavior was also observed, with participants passing blocks among themselves, and helping verbally individuals who showed difficulty understanding the tasks. As the people in each group were changing during the activity, the researchers had to repeatedly explain the activity to newcomers. Yet, on some occasions, this onboarding was actually ensured by the more active participants, who were willing to explain the tasks involved in the activity to their colleagues.

In both sessions, as the groups started to use the music block and the proverb block, participants were increasingly participative and seemed more captivated. Encouraged by the researchers and staff, participants started to record their own custom messages, by using the ‘record’ block. While this was the block that all groups tried last, the recording action was the most frequently used across sessions. As such, the interaction rapidly shifted from exchanges with pre-recorded messages to recording and sending custom messages. The interaction flowed around riddles, incomplete songs, and proverbs the other group would have to guess. Incrementally, riddles became more complex, with people purposely making them more complicated. This competitive behavior was evident, especially in the second session, and generated several moments of liveliness, with participants discussing with enthusiasm and laughing frequently. For one participant (P22), it also gave rise to robot personification as if it were responsible for the increased difficulty of the charades “Oh yeah? Since you are so smart, take it now".

5.2 Directed and Non-oriented Activities

One of the goals of the activity was to incentivize people to think about building sequences. For some people, the blocks and the schematic sheet urged curiosity, as people spontaneously started questioning the function of each block. However, most participants were hesitant in trying to use the blocks, with one or two proactive participants per group being the ones to show initiative in sequence building. At the beginning of the activity, we witnessed a great dependence on the research team and professionals, as assistance was often provided to program the robot. The psychologist, T3, mentions that “The task of sequencing, understanding the sequence, is one of the most difficult tasks”.

While initial interactions were primarily focused on the robot’s action blocks, sequence building became secondary as the sessions evolved, with the dialogue between groups through custom-recorded messages gaining prominence. Given that communication rapidly became the propelling aspect of the activity, participants also became unconcerned with programming the robot through the blocks. In the last exchanges of the second session, sequence building even started to be totally ignored, as researchers were unsuccessful in inducing participants to use the blocks. When inquired about this, T3 expounded how using the microphone to record messages is simpler in comparison with sequence building:

“The recording action is much simpler and they understand that. Despite their age, they are aware of what a recording is and they already know what it is for, what is going to happen. They are surprised if someone responds back to the recording they made. It is a completely different stimulus. The sequencing task is harder, more complicated”—T3

In the last third of each session, participants started to exclusively resort to the recording action, which motivated them to create verbalizations and develop more autonomy in their interactions. This also allowed participants, that were hesitant at first, to participate more actively. The initial orientation guidelines through the blocks that designated the activity were increasingly infringed by participants and alleviated by researchers, giving space to autonomy and agency, the actual key concepts of the approach. Similarly, therapists recount occasions where protocoled activities they conduct are adapted to meet the needs and interests of the group. T3 explained how the unpredictability factor can also be fully embraced, giving examples of activities she conducted in the past where participants were able to freely express themselves (e.g., prompting them to talk about a set of subjects or draw on blank sheets of paper). T3 adds that, while this type of less-guided activities might give rise to unique experiences among older adults, it might fail to stimulate some individuals, who prefer activities with defined tasks and objectives:

“I gathered them in a room with white A4 sheets in front of them and said ‘now do what you want’ and many of them did nothing. They left the sheet blank. And they could only do something when the task was direct, when I said ‘so now, draw a house’ and they drew a house”—T3

The tasks we asked for the participants to perform were initially more restricted, following a protocol and focused on sequencing actions. However, people preferred creative freedom when generating custom messages. One participant (P21) was particularly enthusiastic after recording a message to deliver to her friend, drawing the attention of her fellow group members when the robot was delivering her message: “Oh, did you hear? It is mine! I said it!"

5.3 Unwanted Harm

Professionals refer to the aging population as a population who is likely to inaccurately perceive their capabilities [56, 57]. This exacerbates insecurities about their self-efficacy. Namely, when faced with the activity, two participants stated they were not capable of commanding the robot, even before starting the interactions. While some had insecurities derived from cognitive and motor changes (according to the therapists), others mentioned other factors, such as P18, who stated she was not able to program the robot, due to the fact that she could not read, as she did not attend school. As shown in prior research, older adults’ perceived self-efficacy is low [56, 57], and poor technology acceptance is standard, as they assume they will not succeed [56, 57]: “Many older people are not very accepting of new technologies. [...] This resistance to new activities is normal. They are afraid of not succeeding, afraid of failing” (T1). This lack of confidence and self-doubts in some participants proves to be a challenge in their activity’s performance.

Although people gradually understood the mechanics and logic of the system, professionals warned us of the importance of continuity. Any activity conducted in the institution is done, at least, weekly. This systematization is ensured as a means to combat existing cognitive changes. Occupational therapists mentioned how activities targeting this population should be done on a regular basis, ensuring that people with diverse levels of cognitive decline, such as in memory and reasoning, are able to learn and participate in the activities. Accordingly, during the sessions, some participants showed returning expectations regarding the activity—for instance, when P14 shared she would bring more challenging riddles “next time”.

A few participants had a somewhat alienated posture (e.g., remaining quiet, when the group was discussing the actions to program). However, when watching one of the excerpts that focused on these behaviors, T3 stated that does not necessarily mean people are reacting in a negative way. She explained that those participants could actually be trying to understand the tasks involved, by observing their colleagues. However, three participants in particular (P2, P5, and P19) seemed actually detached during the whole activity. T1 also reasoned that they were also not reacting negatively to the activity. These participants could be having difficulties in understanding and/or giving their contribution, due to cognitive decline or other conditions (in the case of P2, aphasia).

5.4 The Role of Mediation

Given that this population is usually very heterogeneous when it comes to abilities and perspectives towards new technologies (including the groups that took part in the sessions), several asymmetries end up emerging, in their understanding of the activity and, consequently, in their participation. From an inclusion point of view, an external party is needed to mediate the interaction and balance participation. This requirement arises daily in the activities developed by professionals, with them mentioning that “It is very complicated. This is our daily challenge because they are very different". In addition to guiding the activity and answering questions people may have during the activities, mediation dilutes unbalanced conquests: “when I see that someone is not participating, I call them to the activity" (T1). Especially at the beginning of each activity, it is essential to have a suitable introduction that makes it easy for everyone to be aware and understand what the activity is, how they participate, and what tasks they can perform.

“I also often work with a tablet, but on an individual basis. However, if presented well, they would interact [...] and I presented it in different ways”—T3

A few participants were peremptorily quick to help other colleagues with more difficulties before doing the task as a group. However, professionals mention that “sometimes we have to separate them” due to fracturing differences in cognitive performance: “We do not like to do it, but sometimes we have to do it". When this separation occurs, the norm is for people with dementia to form a separate group as they usually come to understand activities more slowly. As mentioned above, the systematization and repetition of tasks are vital to work with this population. Repetition also allows professionals to perceive specific behavioral and cognitive functioning and more readily adapt tasks to the different needs and preferences: “We already know them, and that is why we know the approach that works best" (T3).

5.5 Design Specifications

Some people had difficulties immediately perceiving the blocks and their functions. T1 mentioned that our tangible blocks focus on a more abstract language and can represent an understanding barrier. According to him, older adults easily perceive visual cues if they are figurative and “closer to what is real" because it is easier for them to identify. For instance, while the block used to send cookies is sculpted like a plate with cookies, professionals mentioned that its color and size makes it difficult to understand its function: “Perhaps they should have the real color. Try to get closer to the real object” (T1); “If the cookie block was actually a cookie, people would identify the cookie right away. [...] It has to be the cookie” (T2).

The first reactions can also be optimized if a personalized approach exists in the familiarization process. The research team was responsible for introducing the tangible blocks and the robot verbally and with a schematic sheet illustrating the use of each block. This sheet turned out to be irrelevant due to its depurated visual language and “even more because some of them cannot read—try to bring the object and some relevant indications to reality with visual clues”.

In parallel, the robot captured the participants’ full attention, making it more difficult for them to concentrate on learning the commands. The robot itself represented a significant novelty factor, and when moving around, it ended up attracting most of the attention. Several studies have shown that older people have a more limited ability to concentrate on a specific task [58].

The proverbs and sections of music that we chose are widely known in our country. As aforementioned, these actions were successful in involving participants who were hesitant at first. For most exchanges, participants were able to recognize and complete the proverbs and sang traditional songs. Therapists argued these actions stimulated a feeling of nostalgia in participants, as they were connected to their childhood which are stored in long-term memory. Therapists added that these actions captured participants’ attention: “As soon as the robot played the music, it grabbed them all” (T3). It is common in current practices for professionals to use tools to induce emotions, whether autobiographical or not. Aspects such as the country they live in, that everyone is aware of, or marriages and births that are usually successful in capturing older adults’ attention, stimulating social interactions, and bringing them together as a group.

When asked about the possible benefits of having a robot in the institution, the perspective given was always utilitarian, both by participants and professionals. The functions vary between the “escort robot" in charge of security and monitoring the person; “the waitress keeper robot" who checks if the person needs help and who would compensate for the reduced number of professionals; or even “1 to 1 robot", mentioned by the psychologist, who would periodically perform cognitive and physical checkups. However, in any of these scenarios, the psychologist fears a transformation in the institution dynamics into the dystopian setting of the Disney-Pixar movie, ’Wall-E’, where people are highly sedentary and have a panoply of robots to perform their tasks, highlighting the need for future scenarios to be thoughtful of physical and cognitive consequences.

6 Discussion

While the vision and even the reality of robots in care homes focus largely on physical assistance, we focus on the role of robots in supporting social contexts among humans. We discuss insights from the study regarding possible interaction strands in using a robot controlled by older adults as a tool to increase agency in these contexts. These results inform advantages and precautions to consider in using social robots as controlled communication vehicles.

6.1 Extending Agency and Communication

Communication is essential to well-being [59] and, in an institution for older adults, it becomes especially relevant in combating cognitive decline and agency maintenance [60]. Even when done remotely through technology, this communication brings benefits in combating isolation, especially if done with friends and family [61].

Our experience was focused on agency, essentially in more utilitarian, oriented, restricted tasks. However, one of the relevant points that we could observe was the manifestation of people regarding human relationships. The communication that existed in the last third of the interaction reflected affection, collaboration, and challenge. Organically, people began to explore their communication to privilege messages of affection to other people and entertainment and to play with each other through the ‘record’ block.

Most of today’s robotic systems for older people may be neglecting some values that people want to achieve. Many of the current systems focus on healthcare promotion [62, 63] through cognitive tasks [64, 65], physical exercise promotion tasks [9, 66], and occupational tasks [67]. These systems are frequently stereotypically task-oriented and protocoled. For example, the most common value that older people want to see fulfilled is helping others [68, 68,69,70,71], and, as far as we know, there are no systems that allow older people to help other people.

Our initial explorations evidence that it is possible to take an agency shift to older adults while using the robot’s potential to fulfill the goals they want to achieve, unveiling that, ultimately, people want to express what they value and what represents them in their core.

We understand there is room to explore new approaches with robots in which they enhance and prioritize people’s values and desires. In particular, aspects of the activity that were connected with participants’ childhood (e.g., traditional proverbs and music) acted as anchors that captivated most people to the activity.

A factor that could be essential in the permanence and continuation of the use of this system is the perception of authorship. Participants were engaged by the fact the actions and verbalizations conveyed by the robot were put together by them. Previous studies show that engagement in an activity is dependent on the communion people feel with the activity [72]. Additionally, some degrees of competition were developed, which may have significant effects on factors such as an individual’s motivation for continued participation, or their optimism for the aging process in general [73].

6.2 Including Passive Participants

Inclusion is seen as critical in the activities carried out by professionals. Inclusion promotes the spirit of unity [74, 75] and belonging to a community [75, 76], which are beneficial to people’s well-being [76]. The main barrier to the fullness of this approach is the idiosyncrasies of each person that make the group highly heterogeneous. Care home professionals highlighted how the goal of inclusiveness is a daily challenge when promoting and conducting recreational activities in the institution. Asymmetries might derive from personality traits [77, 78] or different cognitive abilities [79, 80], leading some participants to feel detached from the activities proposed. However, we realized through the approach taken by the professionals that, at the level of cognitive stimulation, the inclusion of more passive people is equally important. People end up profiting from being exposed to activity through socialization, learning through observation, or simply contemplation. From a perspective of inclusion and more balanced participation, the robot’s control system must have different degrees of complexity. As the professionals mentioned, there is a constant adaptation and simplification of the materials depending on the person. For example, the game ’Bingo’ is currently played with pictures to include people who can no longer identify all the numbers to participate as well.

The idealization of a robot or multiple robots that can be at the individual command of different people, also leads us to envision how this autonomous control can be facilitated and personalized by the system itself. Closest to our approach, we have a robot, devoid of any programming, merely complying with the commands given by users. This design decision is assumed as a way of transferring and emphasizing the feeling of choice and, ultimately, agency for users. However, given the usual heterogeneity of this population, it could be essential that the system is able to adapt to the user’s individual needs and preferences. The system is designed to identify the individual operating it and utilize a record of previously executed sequences specific to that user, and act more actively to support the users in achieving their goals.

6.3 Supporting Self-Efficacy and Agency

Institutionalized older adults need great external support, whether in recreational activities [81, 82] or in carrying out basic daily activities [83]. In both sessions, participants were not completely autonomous when commanding the robot as they were, for the most part, dependent on the onboarding, regular guidance, and assistance given by the research team and staff, especially when building sequences. In fact, it was out of scope to achieve a scenario where participants were able to use the system without any help. However, we should consider that the perceived self-efficacy of anyone is influenced by the ability to autonomously perform tasks [84, 85]. In this regard, it is important to ponder how these types of systems could succeed in mitigating or reducing external help. For instance, it would be important to simplify some aspects of the activity (e.g., fewer blocks, verbal commands) to ensure a smoother learning curve. Eventually, the system could allow more complicated sequences after prolonged use. Similarly, the psychologist mentioned how she would introduce puzzles to care home users, initially 5–10 pieces, and how people would, after a period of time, be autonomously able and spontaneously interested in assembling 1000-piece puzzles. This incremental advance in the degree of difficulty is only allowed with familiarization and continued use.

To promote greater independence from external elements such as the staff, one of the possibilities that emerged is that the robot could be the mediator. Although we have presented the blocks and have introduced the various components of the system and operating logic, it is not straightforward for people to know right away what to do or how to get started. While the professionals or we were the fuse for the activity’s promotion and initiation, it could eventually be the robot that played this monitoring role. Proactively, the robot can help the person learn to use the system, serving as an aid in developing sequences and rationality of actions that will be performed sequentially.

The concept of mixed-initiative interaction, as discussed by J. E. Allen et al. (1999), gains significant relevance and serves as a design consideration [86]. This approach emphasizes a fluid exchange of agency between the user and the robot, symbolized by the interplay of ‘robot push’ and ‘user pull,’ enabling collaborative and adaptive interactions. Such a framework is particularly pertinent in elder care, where balancing the robot’s role in guiding or assisting (robot push) with the user’s autonomy and decision-making capabilities (user pull) is crucial. By embracing mixed-initiative principles, we can better design interactions responsive to the users’ changing needs and contexts while promoting active participation and a sense of control over the interaction process.

6.4 Revisiting Our Research Questions

How do people extend their communication and agency through a controlled robot? Older people took ownership of the robot to do tasks such as communicating more, cracking jokes, and challenging each other. They also improvise and start to play with each other by personalizing actions. By being more in control, people gain an implicit agency through the tasks. Essentially, people are more prone to act according to their wishes and prioritize what they value.

What challenges and barriers do people face in controlling a robot through tangible blocks? The diversity and heterogeneity of participants raised some inclusion barriers that will have to be considered in the future. One of the significant challenges is to simplify the mechanics and sequential logic. Furthermore, we only performed two sessions which made some participants, who were able to comprehend the task faster, dominate the interaction. These control asymmetries can be diluted if the approach is slower, more methodical, and consistent with some participants characteristics so that, at their own pace, they can understand the logic and become familiarized with the sequential process.

6.5 Limitations

In our study, we scrutinized interactions with the robot and tangible blocks, as well as interpersonal communication, with verbal expressions of enthusiasm and personal views informing our analysis. Despite limited oral communication, several participants showed changes in engagement, as evaluated by professionals, due to our insufficient understanding of their daily routines. Our methodology, informed by past engagements, emphasized observational techniques and discussions with professionals.

The number of tangible blocks that people could use was also a limitation. Quickly, there was a saturation of the number and type of interactions people could have in the first few interactions with the other group. However, this limitation proved to be a shortcut for a more improvised use by the participants through the ‘record’ block.

Furthermore, we were only present at one institution, and therefore, socio-economic and cultural inputs may diverge depending on the widening of the sample. The increase in the diversity of participants can influence the established communication dynamics and determine new knowledge in the design of tasks to be translated into tangibles. For example, we might have more complex robotic scenarios if we interact with more technologically savvy people.

Although we used tangible blocks, it was not clear if it was the best option to command the robot, given the hesitation on the part of some participants and advice provided by professionals. We used this interface, taking into account past research on computational thinking that shows tangible interaction and the visualization of sequences facilitate the learning process [87, 88] and increase collaboration [89, 90]. Finally, longitudinal studies of the use of the robot and tangible blocks will be relevant to ascertain their use over time, older people’s learning, and improvements in the execution of sequences also as part of cognitive training.

In our study, the limited autonomy of the robot used was intentionally compensated by researcher involvement to facilitate tailored interactions for older adults. This approach was crucial in emphasizing user agency in the current context. However, in future practical applications robots are expected to have enough autonomy to follow the users’ commands, reflecting a human dimension that considers the personal factors impacting long-term interactions with technology [91].

7 Conclusions and Future Work

With aging, people experience changes in cognitive, motor, and perceptual abilities, which are then reflected in different aspects of their life. As a consequence, older adults, in particular those in institutions such as nursing and care homes, may face limited opportunities to exert and self-express their agency. The lack of agency can also impair one’s self-identity and personal values manifestation. In parallel, these factors also bring the possibility of social disruption and the risk of isolation. Robotics have been previously explored as a way to foster recreational and social activities among older adults. Still, in most approaches, the interaction is led by the robot itself. In this work, we approach this challenge by placing the agency on the human side, enabling the person to command the robot’s behavior, express their own preferences, and find new valences for its use in social interactions.

In our experiment, over time, we observed participants reshaping the activity, making it more oriented to their values and interests. In particular, final interactions between groups showed affection and play, challenging one another, through the recording of personalized messages.

Future research should empower older adults by prioritizing their input and aligning with their values. Our study shed light on the effects of increased agency, shaping two main future directions.

Exploring Value-Sensitive HRI and Unscripted Interactions: The primary aim of our experimental study was to evaluate the appropriation of robotic engagement and interaction among older adults. We posit that our primary insight falls upon humans to delve into and delineate high-level objectives they wish the robot to achieve [92], as there is currently a conspicuous absence of such human-driven decision-making. Empirical data from our investigation highlighted the intriguing trend of participants initiating unscripted communication as the primary function they sought to explore, over and above the predetermined set of tasks the robot was initially programmed to undertake. This finding insinuates a compelling avenue for future inquiry: employing robots as extensions of older adults’ intentions. This approach opens up possibilities where individuals may transcend the predefined robotic behavior, commanding the robot to venture into spaces beyond the individual’s physical reach or presence. This underlines the profound implications of our study for the future landscape of HRI. It suggests that older individuals can employ robots to articulate their intentions and direct the robot to act in alignment with their intrinsic values and preferences.

Merging Human Values and Robotic Behaviour: Adjusting a robot’s agency and autonomy has been researched in multiple contexts, from industry to classrooms [93], but is still under-explored with older adults. The future will crucially need to balance human leadership and robot autonomy, particularly in social contexts with older people. This interplay can be achieved by merging human values and intentions with robotic behaviour, aligning robotic actions with human directives, and preserving the necessary robot autonomy for effective task execution. Humans should have agency oversight over robots through the definition of high-level goals while robots autonomously plan and execute lower-level actions, allowing human guidance to coexist with robot autonomy in handling complex tasks. This could involve developing new interfaces, methods, or training that enable humans, especially older people, to interact with robots.