Purpose The development of assistive technologies that support people in social interactions has attracted increased attention in HCI. This paper presents a systematic review of studies of Socially Assistive Systems targeted at older adults and people with disabilities. The purpose is threefold: (1) Characterizing related assistive systems with a special focus on the system design, primarily including HCI technologies used and user-involvement approach taken; (2) Examining their ways of system evaluation; (3) Reflecting on insights for future design research. Methods A systematic literature search was conducted using the keywords “social interactions” and “assistive technologies” within the following databases: Scopus, Web of Science, ACM, Science Direct, PubMed, and IEEE Xplore. Results Sixty-five papers met the inclusion criteria and were further analyzed. Our results showed that there were 11 types of HCI technologies that supported social interactions for target users. The most common was cognitive and meaning understanding technologies, often applied with wearable devices for compensating users’ sensory loss; 33.85% of studies involved end-users and stakeholders in the design phase; Four types of evaluation methods were identified. The majority of studies adopted laboratory experiments to measure user-system interaction and system validation. Proxy users were used in system evaluation, especially in initial experiments; 42.46% of evaluations were conducted in field settings, primarily including the participants’ own homes and institutions. Conclusion We contribute an overview of Socially Assistive Systems that support social interactions for older adults and people with disabilities, as well as illustrate emerging technologies and research opportunities for future work.
The domain of Human–Computer Interaction (HCI) has a long-standing history of developing assistive technologies for well-being . According to the International Organization for Standardization (ISO) , assistive technology (AT) or assistive products include “devices, equipment, instruments and software especially produced or generally available, used by or for persons with disability.” In addition to persons with disability, the World Health Organisation (WHO) classifies five primary types of people who most need assistive technology . These are (1) people with disabilities, (2) older adults, (3) people with noncommunicable diseases, (4) people with mental health conditions, and (5) people with gradual functional decline . In our study, assistive technology addresses older adults and people with disabilities, aiming at maintaining or improving their functioning and independence, helping them achieve physical and mental health . The developments in multisensory techniques, computer vision, and wearable technology have introduced various emerging assistive technologies to improve the life quality of older adults and people with disabilities. These assistive systems aim at fulfilling the essential needs of specific users in a broad scope of usage contexts and scenarios such as smart homes and healthcare systems for older adults , screen reader software and braille displays for blind people , assistive communication systems for children with autism spectrum disorder (ASD) , gesture-recognition systems for deaf people , as well as smart wheelchair systems for people with physical disabilities .
In many cases, studies involving assistive technologies focus on enabling fundamental capabilities or “survival” skills of specific user groups (e.g., facilitating visually impaired users to navigate a digital map ([9, 10]), or increasing the mobility of people with physical disabilities ([11, 12])). Relatively fewer studies ([13, 14]) have focused on the assistive technologies’ values of enhancing target users’ social interaction qualities. Nonetheless, neither people nor technology can exist in a social vacuum . As social beings, people have an inherent desire to communicate and maintain social relationships with others . Based on Abraham Maslow’s hierarchy , once people’s basic needs (i.e., survival needs) are satisfied, they will strive to satisfy higher-level needs, such as communication, love and sense of belonging in social circumstances. Insufficient social ties and communication can cause undesirable consequences, such as loneliness, depression, and social isolation in special-need users, such as older adults  and people with disabilities . Thus, there is a substantial and rapidly growing demand for assistive technologies that can support social interactions.
An increasing number of existing examples have shown that assistive systems can satisfy users with special needs in social interactions. For instance, for older adults, there is a greater focus on healthy aging to help them maintain personal relationships and avoid experiencing feelings of loneliness . As an example, a view-sharing system provides the continuous real-time changes of the shared outside views to promote social interactions between older adults living in a care home and people in local communities around . An electronic picture frame could monitor older adults at home with unobtrusive sensors, and collect data about their health . Through this way, older adults can maintain social connectedness with their caregivers. Other examples can be found regarding Socially Assistive Systems for people with disabilities, such as a haptic display that can convey facial movements to help blind people understand interaction partners’ facial expressions and emotions through vibration feedback ; a wearable device that allows blind people to perceive and react to gaze from conversation partners through haptic and visual feedback ([14, 24]); and a nonverbal communication application that supports real-time social distance regulation for children with ASD .
Most of the systematic reviews introduce assistive technologies focusing on solving special-need users’ basic needs in their daily living activities, such as dementia care , body rehabilitation , and navigation assistive technologies . Only a few examples of the systematic reviews could be found to study Socially Assistive Systems, however, they all focused on a specific type of Socially Assistive Systems, and only for older adults. ([29,30,31]) reviewed Social Assistive Robots (SAR) that were designed for older adults. Other reviews summarized ICT technologies regarding how to reduce older adults’ social isolation and enhance their communication ([32, 33]). These reviews have been published recently, but do not take into account all different types of socially assistive technologies with diverse user groups. Socially Assistive Systems are helpful for all special-needs users, including older adults and people who have disabilities. Since research in this field is of growing interest, and every year a lot of new studies are published, there is a great need for writing a systematic review, classifying the existing studies, identifying the promising trends, and guiding the future design research. The generated knowledge will contribute to the HCI community.
To summarize, in this systematic review, we answer two primary research questions:
RQ1: Which kinds of assistive technologies are available to support target users’ social interactions, including perceiving social signals, and how to involve target users in their system design?
RQ2: Which types of the evaluation methods are used in Socially Assistive Systems and how to apply each method in system evaluation?
A literature search was conducted in the following six databases: Scopus, Web of Science, ACM, Science Direct, PubMed, and IEEE Xplore. These databases were chosen because they provide full-text journals and conference proceedings of the most important conferences involving assistive technologies, social interactions, and their relations.
To seek out articles, we selected papers across two categories of Medical Subject Heading (MeSh) terms: “social interactions” and “assistive technologies”. According to MeSh terms, one of the synonyms of “social interactions” is “interpersonal relations”, which refers to “The reciprocal interaction of two or more persons”. Accordingly, several approaches can be used to allow or promote social interactions among people such as face-to-face conversations and exchanging messages though social media. In this review, we identified that only social interactions by means of assistive technology were considered. More specifically, it refers to any information systems that help people develop positive social interactions with each other . Synonyms and spelling variations of “social interactions” and “assistive technologies” were used in several combinations and modified for the databases. Some search terms used in this search strategy were also derived from previous studies regarding social interactions  and assistive technologies ([26, 36]). Table 1 illustrates the example search strategy that was used for the ACM digital library. The searches were performed on article titles, keywords and abstracts. The search strategies for the other five databases resemble this. Relevant articles published in the past 22 years (January 2000–July 2021) were collected. Articles included refereed journal papers and peer reviewed articles that published in conference proceedings. Only English articles are included.
The selection process was conducted according to the guidelines of The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement : (1) a computerized search strategy (Table 1) was performed from October 2019 to July 2021; (2) SQ removed duplicates, screened titles and abstracts of the remaining articles; (3) Two independent coders (i.e., SQ and PCA) screened, analyzed and evaluated the full-text articles. After that, they should reach a consensus to decide which articles fit the inclusion criteria. Discrepancies were resolved by discussions between the two coders.
The articles that met at least one of the following inclusion criteria were included:
Articles presented system design of Socially Assistive Systems;
Articles included system evaluation of Socially Assistive Systems.
The articles that met at least one of the following exclusion criteria were excluded:
Reviews and books;
No information systems;
Information assistive systems, but not for social interactions;
Information systems are not designed for older adults and people with disabilities;
Articles are not written in English;
Articles are less than four pages;
Duplicate reports of the same study in different sources;
Among different kinds of Socially Assistive Systems, robotic systems include a large number of studies, such as Socially Assistive Robots for older adults ([29, 30, 38]) and children . Some existed review papers have reported such field, so we did not include robotic systems in our systematic review.
After discussions of the authors, a template was determined for data extraction. Based on the research questions, we extracted data from two primary aspects: system design and evaluation. In more detail, the template included the following categories (see Appendix for summary of paper lists and features):
HCI technologies include wearable technologies, multimedia technologies, and other HCI technologies that support social interactions, such as virtual reality (VR) and augmented reality (AR) technologies;
Social signal perception refers to the system design that help target users to perceive nonverbal signals, such as facial expressions and head pose;
User involvement refers to whether and how the users and stakeholders involve in the design procedure, such as target users, caregivers, experts and families;
Evaluation consists of evaluation type and time span. Four evaluation types are identified: (1) laboratory experiment, (2) field experiment, (3) qualitative study in artificial setting, and (4) qualitative study in field setting. Time span includes hours/days/weeks/months.
Figure 1 shows an overview of the results during the different stages of selecting articles. Initially, 1463 articles were identified according to the search strategies. After title and abstract screening and removing duplicates, 165 articles remained. Next, we selected full-text articles according to the inclusion and exclusion criteria. In total, 56 articles were included, which directly related to Socially Assistive Systems. We manually searched the references for included articles, and nine articles were selected after hand searching of references. Finally, 65 articles were considered for the systematic review. The primary features of each study are summarized in the Appendix.
Since a great proportion of Socially Assistive Systems adopted Information and Communication Technology (ICT) for its capacity to socially connect people, the main categories of the HCI technologies used in the reviewed studies are classified based on a taxonomy of ICT proposed by Inaba and Squicciarini . Seven out of 13 technology areas in the original taxonomy were removed, mainly because they are underlying technologies that are not very relevant to human interaction (e.g. Large-capacity and high-speed storage). Besides, we add five kinds of emerging technologies in the HCI community because they are not included in this taxonomy but repeatedly mentioned in the reviewed articles. Finally, the established classification includes 11 technologies (Fig. 2): (1) social network and communication, (2) sensor and device network, (3) cognition and meaning understanding, (4) human interface, (5) electronic measurement, (6) wearable technologies, (7) multimedia technologies, (8) Virtual Reality (VR), (9) Ambient Intelligence (AmI), (10) Augmented Reality (AR) and (11) others. Table 2 shows the reviewed studies that applied each type of technology. Since some proposed solutions are the synthesis of multiple technologies, we mainly assign the studies to the technology areas specifically mentioned by the researchers. Usually, less than three core technologies were extracted from each study. A Venn diagram uses the circles to show the relationships among multiple HCI technologies among reviewed articles (Fig. 3).
Cognition and meaning understanding technologies were mentioned in 18 studies. It is a subset of the broader field of artificial intelligence that simulates the functions of the human brain, including computer vision technologies, natural language processing and pattern recognition . Computer vision technologies represented the largest proportion in this area, and most target users were people with visual impairments. They were mainly used to capture, track and recognize non-verbal social cues such as eye contact, facial expressions and head gestures ([41, 42, 44,45,46,47,48,49,50, 52, 53]). One study also applied computer vision to assist people with memory loss to identify social partners . Pattern recognition technologies were applied to support blind users on social networks by recognizing digital images and generating descriptions . Printed patterns such as text and codes could also be recognized for further social contacts, which could be used by older adults . In addition, voice recognition is a very useful technology for deaf people, but only one study explored in this area .
Wearable technologies represented the third largest proportion (16 studies). They were often applied with computer vision technologies to capture visual cues and provide haptic or audio feedback. A typical form is using smart glasses alone ([14, 42, 47, 53, 59, 61]) or with smart belts ([45, 49, 50]). Smart bands and watches are also very common solutions ([57, 62, 63]). Apart from collecting user data, they were mainly used as a personal reminder to improve social skills for the people with memory loss or children with ASD. A smart vest was proposed in  to provide tactile feedback for the user with deaf-blindness. Only one study addressed the esthetic values and socio-cultural aspects of wearable technologies .
Social network and communication technologies were used in 15 studies. They were mainly embedded in software applications to assist tele-communications, which is effective to reduce distance barrier and social isolation. Therefore, the great majority of the target users in this category were older adults and people who had motion disabilities. Typical communication technologies were applied to establish direct social contact including video/audio chatting ([66,67,68,69, 73,74,75,76]) and messaging ([23, 54, 66, 68, 69, 75]). Eight studies developed online platforms where people could create their own profiles, build relationships with other users, and engage in social activities together ([21, 66,67,68,69,70,71]). In  a case is also mentioned aiming to enhance the social connectedness of nursing home residents by transferring real-time photos of outdoor sceneries.
Multimedia technologies were applied in 13 studies. They mainly refer to the technologies that use text, graphic, animation, and sound to deliver information. Seven studies ([21, 72, 74, 78,79,80,81]) presented media playing systems to provide meaningful content for older adults as social stimulus. Multimedia technologies were also frequently used to record digital content, which could be shared by older adults or people with language impairments to enhance their social connectedness through self-disclosure ([55, 67]). The recorded media content could also be used to improve the social skills of people with ASD ([51, 77]). Two studies developed multimedia applications to interpret sign languages for deaf people ([56, 82]).
Human interface technologies were specially mentioned in 13 studies. Haptic interfaces were explored in four studies mainly for people with visual impairments. They presented a mapping between visual cues and vibrotactile representations with devices such as smart belts and ergonomic mesh chairs ([65, 83,84,85]). Four studies integrated social application with tangible interfaces to increase the attractiveness and reduced the technological barriers for older adults and children ([21, 76, 86, 87]). Multi-touch interfaces were increasingly used to encourage co-located collaboration. They were used in three studies to support relationships between people with dementia and caregivers or enhance the social skills of children with ASD ([80, 81, 89]). One study developed two applications with semi-transparent video interfaces to assist deaf people in local group meetings and remote personal meetings .
Sensor and device network technologies were applied in 11 studies. Apart from the multi-device systems that were used with wearable technologies ([14, 39, 44, 61]), a large proportion of the solutions were distributed or multi-agent systems that were developed mainly based on a client/server architecture. The client devices could interact with each other online or offline, and their status could also reflect on the server device. They were usually designed for communities such as care homes or classrooms to enhance social awareness of older adults or children with ASD ([87, 91, 92]).
Virtual reality (VR) technologies were adopted in four studies. Two studies developed immersive therapeutic VR systems for children with ASD ([25, 96]), and one study designed an online VR platform for people living with spinal cord injury . In addition, one study reported the positive effect of VR applications on older adults suffering from social isolation .
Ambient intelligence (AmI) technologies were used in four studies. As a key technology in the ambient assisted living environment, they were mainly applied to enhance the social awareness and interconnectedness between older adults who live alone and their friends, family members or caregivers with ambient displays or ambient lightings ([22, 79, 91, 98]).
Electronic measurement technologies were adopted in four studies. They refer to the technologies that collect, process and analyze electronic signals such as radio waves, Wi-Fi signals and ultrasonic waves. The applications were designed for people with physical disabilities or visual impairments. They were primarily implemented in wearable devices or mobility aids to measure the presence, location, distance or velocity of users’ social partners ([55, 93, 99]). One study presented two new methods for real-time sonification of head movements and head gestures .
Augmented reality (AR) technologies were applied in two studies. One study developed a mobile AR game application for children with ASD to improve their social skills . The other study proposed technology mediated sight to highlight meaningful daily information for people with visual impairments .
Only one study did not directly involve any type of technology mentioned above. It proposed a human-avatar taxonomy to support social communication disorder evaluation .
Social signals perception
Social signals are defined as “the expression of ones attitude towards social situation and interplay” , which are shown through a variety of verbal and nonverbal cues. Vinciarelli et al.  presented the taxonomy of social signals, including (1) face and eye behavior, (2) gestures and postures, (3) vocal behavior, (4) space and environment, as well as (5) physical appearance. According to the taxonomy of social signals, we classify the Socially Assistive Systems which are used to support social signals perception. In our review, 27 out of 65 studies (41.54%) presented Socially Assistive Systems to perceive (1) face and eye behavior (14 studies), (2) gesture and postures (five studies), (3) vocal behavior (five studies), as well as (4) space and environment (four studies) (Fig. 4).
Face and eye behavior A total of 14 studies ([14, 41,42,43,44,45, 49,50,51,52,53, 57, 83, 84]) explored Socially Assistive Systems to perceive face and eye behavior of interaction partners. Among them, the majority of these systems supported identifying face behavior. Seven studies addressed people with visual impairments to perceive typical facial expressions ([41,42,43,44,45, 53, 84]), facial movements , and head pose  of their conversation partners or pedestrians. In , a sharing system was targeted at children with ASD, to help them recognize facial expressions;  presented a mobile phone-based app to identify people’s face, assisting people with dementia to remember the names and relationships of their interaction partners. Three studies ([14, 49, 50]) were regarding Socially Assistive Systems for gaze perception;  used a wearable device to help a blind person perceive eye gaze from a sighted person in dyadic conversations. The prototype of  simulated eye gaze of blind people, allowing them to establish “eye contact” with sighted conversation partners. In , researchers developed a multimodal assistive system to inform a blind person in real-time whenever someone was looking at her.
Gesture and postures Five studies presented systems for gesture and postures perception. Head pose estimation technology was used to help people with visual impairments ([41, 46, 47, 60]) and children with ASD  to perceive typical head gestures/movements, such as nodding and shaking. For example,  developed a Social-Aware Assistant (SAA) to convey the head nodding of conversation partners by using a vibratory belt, to enhance face-to-face interaction between blind and sighted people.
Vocal behavior Five studies ([56, 65, 82, 90, 94]) presented systems that could support deaf people to perceive vocal behaviors. These systems could transfer the voice to sign language ([56, 90, 94]), or sign language to the voice , so as to establish communication between deaf and hearing people. In , researchers developed a Tactile Board focusing on translating the voice into vibrotactile signs, in order to facilitate communication for deafblind people.
Space and environment Four studies ([25, 85, 93, 99]) described the systems for social distance perception. In ([85, 93]), researchers developed Socially Assistive Systems for blind people, to help them identify the distance of interaction partners and initiate social contact;  was regarding providing a real-time support for the proximity regulation of children with ASD, to adjust their social behavior to be more in line with local cultural norms. In , a social following control system supported a good conversation distance between the wheelchair user with physical disability and a companying person.
Involvement of end-users and stakeholders While it is conventional for researchers and developers to validate their designed assistive technologies with end-users, it is not a common case for related research to feature end-user involvement in the design phase. In our review, 22 out of 65 (33.85%) studies reported end-user involvement in the design stage. Among these studies, nine (40.91%) also reported the involvement of stakeholders in addition to the end-user group. These stakeholders included professionals who provided care or services to the target groups, such as caregivers (e.g., [21, 55]), educators (e.g., ), therapist (e.g., ), etc. On the other hand, these stakeholders also included domain experts who had specialized in certain aspects of the target design challenge, such as psychologist, technologist, designers, ethicist (e.g., see ). For instance, while targeting the special learners as their end user group,  also involved the educators and therapists in their participatory design activities;  conducted expert interviews to integrate domain experts’ perspectives in design. Similarly,  included two caring staff members in their participatory prototyping with 10 developmentally disabled adults.
The benefits of involving stakeholders in addition to the end users can be generalized two-fold. First, these stakeholders possess rich practical experiences or scientific understandings about the design context, which could compensate the knowledge and expertise of the developers. Second, for some targeted end-users, such as people with dementia, or children with ASD, it can be difficult for them to fully express their tacit needs and experiences. With the stakeholders’ help, the development team could better discern the target group’s needs and experiences.
User-centered approach and participatory design In general, two types of end-user involvement can be recognized from the reviewed papers: 16 out of 22 studies (72.73%) described a user-centered approach, and six out of 22 studies (27.27%) highlighted a participatory design (or codesign) approach. The major difference between the two concerns the role of the end-users. In a user-centered approach, end-users are involved as the providers of the contextual information, which helps the designers to better understand the design challenge and requirements. Whereas in a participatory design approach, in addition to providing contextual information, end-users also actively share the role of designers to formulate the design solution together. In user-centered development, various techniques have been used to understand the needs, attitudes, or experiences of the end users, including interviews (e.g., [65, 75]), ethnographic studies (e.g., ), focus group (e.g., [22, 80]), trial and customization (e.g., ), feedback sessions (e.g., ), etc.. Whereas in a participatory design approach, besides the above mentioned techniques to gather user-centered insights, the end-users are also invited to participatory, or co-design workshops, to generate design concepts or solutions together with a group of stakeholders (e.g., see [73, 91, 92]). The outcome of the participatory design workshops could take the form of some low-threshold design artefacts (like sketches, drawings, or collages), as well as mockups (e.g., see ), or low-fidelity prototypes (e.g., see ).
End-user involvement in different stages of design The reported user and stakeholder involvements also took place in different stages along the design process, from rather early moments of ideation all the way to the later stages which overlapped with design validation, For example,  utilized a focus group in an early stage to identify difficulties of deaf people in using assistive technologies. Whereas, addressing a similar context,  involved deaf participants in a session after the ideation to gather their feedback on the design concept. Moreover, several studies also featured end user and stakeholder involvements across multiple stages along their design process. For example, in  (targeted at developmentally disabled adults), the researchers first performed card sorting with design partners to gather user requirements. Subsequently, role play sessions were conducted with two participants to build a lo-fi prototype. Finally, the hi-fi prototype was built together with target users and their caring staff members.
Engagement with real-world contexts In addition to including end users and stakeholders in design, seven out of 22 papers (31.82%) also highlighted their in-depth engagement with the real-world contexts of their target group. For example, as reported in , a two-week design ethnography study was conducted, in which the researchers shadowed three participating children to gather field insights into their daily life. Similarly, in , the researchers observed nine classrooms to gain on-the-ground knowledge about the special education context. For another example, in , the researchers conducted an observational study to closely understand how a game is learnt and played by older adults in a senior center.
Summary for RQ1
In response to RQ1, we summarize the findings related to system design in the following.
Eleven types of HCI technologies were identified that supported social interactions for target users. The top three frequently used HCI technologies in Socially Assistive Systems were (1) cognitive and meaning understanding technologies (17.82%), (2) wearable technologies (15.84%), as well as (3) social network and communication technologies (14.85%). Among them, the most common was cognitive and meaning understanding technologies, which were often applied with wearable devices for compensating vision, hearing, and memory loss. Among different types of social signals, the majority of Socially Assistive Systems were designed for target users to perceive face and eye behaviors of their interaction partners. Nearly one-third of the analyzed studies involved end-users and stakeholders in the system design stage. Among reviewed studies, there were two types of end-user involvement: (1) user-centered approach and (2) participatory design.
We categorize system evaluations regarding Socially Assistive Systems based on (1) whether they were experimental studies or qualitative studies and (2) whether they were conducted in the lab or in the field. Thus, evaluation methods for Socially Assistive Systems are classified into four categories (Table 3): (1) Laboratory experiment (27 studies), (2) field experiment (10 studies), (3) qualitative study in artificial setting (15 studies), and (4) qualitative study in field setting (21 studies). Figure 5 shows the pie chat of four evaluation methods used for target users.
This category consists of two sub-categories: (1) laboratory experiment for user system interaction (18 studies), and (2) laboratory experiment for system validation (9 studies).
Laboratory experiment for user system interaction.
In this category, user experiments were conducted under highly controlled condition, focusing on how to evaluate perceptions or experience of users towards Socially Assistive Systems. A total of 18 studies were targeted at people with visual impairments ([14, 41, 42, 49, 60, 84]), older adults ([70, 73, 79, 81, 98]), and children with ASD ([25, 87, 96]). Other four studies were regarding people with memory loss , spinal cord injury , Social Communication Disorders (SCD) , and deafness .
In most situations, researchers involved representative users that directly reported their user experience and perceptions in laboratory experiments ([41, 60, 70, 71, 73, 79, 81, 84, 87, 94, 96]). For example,  investigated how to enhance intergenerational interactions between older adults and their grandchildren while online platform. Grandparents directly reported their user experiences with the platform, such as engagement, social connectedness and social presence. In certain cases, representative users were difficult to study and report directly about their own behaviors, attitudes and intentions, so proxy users provide a feasible solution for running such user experiments ([14, 25, 42, 49, 98, 100]). In , researchers tested blindfolded participants rather than real blind participants in their initial study. In , researchers evaluated whether immersive VR applications could provide effective real-time support for improving social distance for children with ASD. In this case, their mothers were treated as proxy users, to report their children’s user experience of using immersive VR applications.
In this category, only three studies clearly mention that they received an approval from the ethics committee ([57, 60, 84]);  was a clinical trial to investigate people with memory loss. Both participants and their caregivers were recruited from the university, and the Institutional Review Board (IRB) of the university approved this study. In , researchers recruited a total of eight blind participants to attend the study. The research protocol was approved by an IRB. Similarly, in , researchers followed ethical standards provided by the American Psychological Association (APA). The ethics committee of the university approved the research protocol. Most studies (N = 15; 88.24%) used the consent forms for older adults and people with disabilities. The written consent was not always suitable. As reported in , the participants with dementia were first asked to provide a written consent, and if not feasible, verbal consent was obtained and witnessed by a neutral third party. Besides, in , all elderly participants were required to provide an approval by their family doctor to allow them to attend the study.
Laboratory experiment for system validation
In this category, studies focused on evaluating system validation. Nine studies were targeted at people with visual impairments ([44,45,46,47, 83, 85, 93]), physical disability , and low vision, Alzheimer’s disease, and autism spectrum disorder .
The majority of studies ([43,44,45,46,47, 83, 85, 93, 99]) tested the recognition rate of the target system. For example, in , researchers conducted a single trial to test the system accuracy of identifying facial features. Additionally,  investigated how to maintain a good conversation distance between the wheelchair and the companying person. Researchers tested the wheelchair control system to examine the feasibility of the pose detection algorithm.
Among nine studies, six of them used proxy users in the preliminary experiment ([46, 47, 83, 85, 93, 99]). Generally, researchers tested system function with non-disabled participants instead of representative participants who had disabilities, such as people with visual impairments or physical disability. None of nine studies clearly reported an ethical protocol.
According to , field experiment refers to studies using an experimental design in a real-life environment. The experimenter still needs to manipulate at least one independent variable. A total of 10 studies used field experiments to evaluate Socially Assistive Systems, which were targeted at older adults ([66, 69, 74, 75, 78, 97]), people with deafness ([56, 82]), physical disabilities , and visual impairments .
These studies were conducted in various field settings. The most common was the participants’ own homes and their living communities ([66, 69, 74, 75, 82, 97]). Other places included institutions ([52, 75, 78]) and realistic situations for testing purposes ([53, 56]). For example, in , researchers evaluated 19 older adults in homes to use a communication system prototype over 10 weeks. Similarly, researchers visited older adults in their own homes to investigate Uniper-Care Technology (UCT) for enhancing social connectivity and entertainment. Most of these field experiments (N = 8; 88.89%) lasted for a long period of time, from several weeks ([56, 66, 69, 74, 75, 97]) to several months .
Two out of 10 studies clearly report that they were approved from the ethical committee ([52, 69]). For example,  recruited elderly participants through elderly associations and retirement homes. The study was approved by the ethical committee at each of the researchers’ universities. Other studies ([53, 66, 74, 75, 78, 82, 97]) mentioned that they used an consent form in their experiment. For example, in , a responsible relative signed the consent form to agree the participants with dementia to attend the study. Additionally, the nursing home manager and the activity facilitators consented to their participation in the study.
Qualitative study in artificial setting
The category of qualitative study in artificial setting is referred to studies which conducted qualitative evaluation of the design outcome (e.g., concepts or prototype systems) in a setting that was not the naturalistic environment lived in by the end-users. This category was targeted at various end-user groups, including older adults with needs for socializing activities ([22, 54, 68, 72]) or living with dementia ([57, 78, 80]), children with ASD , as well as people with visual impairments ([50, 58, 59, 95]), hearing loss ([64, 90]), or speech impairments .
Multiple types of such artificial settings could be seen in this category. First, five studies created ad-hoc settings for a group session: such as focus groups, or participatory design workshops ([50, 64, 72, 80, 90]). They often aimed for eliciting thick discourses and in-depth reflections from the target group (and their related stakeholders). For example,  introduced an evaluation approach which combined iterative participatory design workshops and a public exhibition, involving both the target group (people with hearing impairments) and domain experts. This subset of studies also featured a relatively close participation of the end-users and related stakeholders in the process: that was, participants were not only asked to evaluate a developed system, but also invited to participate in the iteration or modification of the evaluated design. For instance, in , focus groups were organized for the target group (people with dementia) and their caregivers to assess the designed interface and select relevant design elements.
The second subset in this category concerns four studies introducing a new form of social event or activity (e.g., a game) for end-users, which was outside their daily routines ([68, 77, 78, 95]). For example,  developed an audio-based augmentation system to enhance both blind and sighted players’ experiences in badminton. The study organized several game sessions for the target group to gather their experiences of the system. Similarly,  introduced a new social game to older adults to study their behaviors in, and responses to this newly learnt activity.
The last subset in this category is associated with five studies that adopted a simulation, or in-lab setting to qualitatively evaluate the developed systems with users via methods such as interviews ([22, 54, 57, 58, 67]). As an example,  constructed a simulation environment of home to invite the target group (older adults) to experience the developed system which was aimed for enhancing their social connectedness. For another example, an AI application developed for people with visual impairments to interact with social network photos was evaluated in  through interviews conducted in an in-lab environment. Among all studies from this category, only a few (five out of 15) provide information in regard to the ethical conduct with the target group. Namely, all three studies involving older adults with dementia briefly described the acquisition of participation consent (from responsible relatives  or caregivers and end-users ), or ethic approval from relevant authority . One study supporting socialization of older adults described the reception of end-user consents . One study targeted at people with hearing impairments mentioned the involvement of an ethicist in workshops .
Qualitative study in field setting
There are 21 studies in total in this category. We found the great majority of these studies were conducted out of age-related concerns. Sixteen studies were targeted at older people ([21, 22, 61, 66, 75, 76, 78, 86, 91]) and children ([13, 51, 55, 62, 77, 89, 92]). Two studies aimed at designing communication systems for the people with disabilities, and they still recruited teenagers as their participants ([23, 88]). Only three studies did not specially mention the age of their target users, which focused on people with vision impairments ([58, 63, 95]).
Most studies were carried out in institutions, such as care centers ([21, 61, 62, 75, 78, 86, 91]) and schools ([13, 23, 55, 77, 89, 92]). Three studies on people with dementia ([61, 78, 86]) and one study on children with ASD  mentioned that their participants were directly recruited by institutions according to research needs. The participants of two studies were recommended by institutions and selected by researchers ([55, 75]). Two studies recruited participants through independent research or social organizations ([66, 88]). In addition, our review shows that the evaluation with older adults were usually in the form of group activities organized and facilitated by caregivers ([76, 78, 86, 91]), while other studies in real-world setting were conducted with much less external assistance. It might because older people have more difficulties in accepting and using new technologies than the younger generations.
In spite of the advantages of field setting, researchers often have ethical challenges to collect data in participants’ daily life. To address this, signing consent is a common way to inform the participants of ethical issues ([63, 66, 88]). Besides, many researchers drafted research protocols that needed to be approved by related institutions or stakeholders before field trials ([13, 62, 63, 75, 88]). To protect the participants’ privacy in using communication applications, their real identities and content of conversations was only accessible to them and their social partners ([22, 66, 75, 89]). Given the specialty of children with ASD, one study mentioned that the participants were not informed of the purpose of the study to avoid stressing them . Another study confined the enrolled students in a separated area to avoid disturbing the non-participants .
Overall, the time span of evaluating Socially Assistive Systems ranged from several hours to over one year (Fig. 6). Most studies (N = 45; 60%) report several hours, while 16 studies report several months and 11 studies took several weeks. Two studies took several days and one study lasted for over one year. Short-term studies (i.e., time span in hours or days) were the majority (N = 47; 62.66%). Most of them (N = 25) were the laboratory experiments. Long-term studies (N = 28; 37.34%) included studies of time span in weeks (N = 11), months (N = 16), and years (N = 1). In long-term studies, most of them belong to qualitative study in field setting (N = 15) and field experiment (N = 8).
Summary for RQ2
In response to RQ2, the findings regarding evaluation methods are summarized as follows.
Among four types of system evaluation methods, the most common was laboratory experiment for evaluating user system interaction and system validation. In most situations, researchers evaluated target users about their behaviors, attitudes and intentions. However, proxy users (e.g., blindfolded participants, young proxies, non-disabled participants) were still adopted in evaluations, especially for testing system function in an initial stage. Some 42.47% evaluations were conducted in field settings. The typical locations were the participants’ own home and institutions. Field settings were often used for older adults and children, which could indicate an age-related issue. In qualitative studies, researchers created ad-hoc settings or in-lab settings to qualitatively evaluate the developed systems. Among all system evaluation studies, only five studies clearly reported they received an ethic approval from relevant authority (e.g., university ethical committee). The rest of the studies primarily used the consent forms for the participants.
Short-term evaluations were the majority (62.66%), primarily including (1) laboratory experiment, (2) qualitative study in artificial setting, and (3) qualitative study in field setting. Long-term evaluations (37.34%) mostly consisted of qualitative study in field setting and field experiment.
Based on an analysis of 65 papers related to system design and evaluation, we present insights from the following aspects: (1) HCI technologies, (2) user/stakeholder involvement in the design phase, (3) involving proxy users in laboratory experiments, and (4) engaging field setting in system evaluation. Finally, due to the impact of the coronavirus pandemic, we also identify a promising domain of promoting the online social experience.
Insights for HCI technologies
The studies reviewed in this paper have presented a wide range of systems designed to socially assist users. Various HCI technologies have been developed and applied in these systems to serve different social purposes. In this review, we summarized 11 types of technologies, and they are often adopted in combinations to construct systems.
Cognition and meaning understanding technologies are often applied with wearable technologies to directly compensate for users’ sensory loss in their daily lives ([42, 44, 45, 47, 57]). Our results show that all of the reviewed studies applied such technologies to detect visual cues, which uncovers the need to design systems for other social signals.
Wearable technologies are useful to follow users’ daily activities and continuously provide social support. However, wearing such devices is often associated with a higher risk of privacy violations and stigmatization. Our results show that none of the studies specially considered privacy issues, and only  specialized on dignify users. Hence, more considerations in these aspects should be encouraged when designing related systems.
Social network and communication technologies are useful to assist people with mobility issues because they are more likely to feel isolated. They can be applied with the combinations with multimedia technologies ([66, 67]) to promote content sharing, human interface technologies  to improve accessibility, or VR  to create immersive social environments. Besides, although rarely investigated in our reviewed papers,  revealed a promising domain to enhance the online social experience for those who have difficulties in face-to-face communications.
Sensor and device network technologies Apart from the systems that mainly addressed individual users, sensor and device network technologies can realize bi-directional interactions between multiple people with certain devices, which is suitable to be applied in institutions or communities. They can collaborate with wearable technologies, electronic measurement technologies or Ambient Intelligence (AmI) technologies to assist daily interaction, enhance social awareness and mutual bonding.
Emerging technologies The studies reviewed indicate that some emerging technologies such as AmI, VR, AR were explored much less than most of other technologies, but they showed great potential and worth digging deeper. For example, AmI was applied in only four studies so far, and all of them were designed for older adults through visual feedbacks. We believe that AmI can also be applied for other user groups with limited mobilities and more forms of feedback can be developed. VR technologies are also only adopted in four studies, and all of them used VR for therapeutic interventions, which means more diverse forms of VR applications can be explored for other purposes. Although AR technology was applied in only two studies, it was proved to be effective in training the social skills of children with ASD and providing useful social information for people with visual impairments. We believe that more AR applications can be developed to overcome various social barriers, and the potential of using AR to assist other user groups (e.g., older adults and deaf people) can also be explored.
Insights for user/stakeholder involvement in the design phase
First, the roles of the involved end-users and stakeholders are changing from simply “context providers” toward more of “co-designers”, as their involving approach is changing from a traditional “user-centered design (UCD)” perspective toward a “participatory design (PD)” perspective. This perhaps echoes the earlier discussion in HCI encompassing the nuanced differences between UCD and PD ([103, 104]). In short, while PD is a kind of UCD, it aims to evolve the landscape of design with more emphases on the empowerment of the users (and stakeholders) and the humility of designers. The benefits of carefully supporting users and stakeholders’ roles as co-designers is twofold. First, it enables the users (or stakeholders) to not only tell what they feel/do in the past, but also actively shape what they desire in the future. Second, it allows the design team to gain deeper empathy and knowledge concerning the target end-users. That is because users and stakeholders are the experts of their own lived experiences, and interacting with them closely as collaborators (rather than subjects) helps designers to establish a better understanding about their tacit needs and latent desires . We argue that these two benefits are especially meaningful to Socially Assistive Systems, which fundamentally aim to empower the users with dignity.
The second trend observed by us, concerns the objectives of user/stakeholder involvement, which have been changing from solely need-finding around utility and efficiency, more toward meaning making in regard to sociocultural aspects. For example, in , the researchers set out to support people with hearing loss beyond functional needs. Instead, they explored the aesthetic and socio-cultural needs (e.g., self-expression) of this group, by designing the form and interaction of smart jewels. More examples can be found. For instance, more and more recent studies have speculated how “others” would look at the end-users using the system (social perception), rather than only how the system functions. And this is indeed an important dimension of sociocultural needs that can considerably determine the adoption of Socially Assistive Systems.
Last but not the least, we observe a trend of user/stakeholder involvement being conducted in a more ecological way. By this we mean the user involvement activities tend to be carried out more systematically (e.g., in multiple stages of design), with heavier entanglement with users’ real-world (material and cultural) context. For instance, ethnographic methods  or observational techniques ([68, 92]) were carried out in the context, to make the user involvement more structured and systematic, which could help yield more authentic and ecological understandings than utilizing workshops or interviews solely.
Involving proxy users in laboratory experiments
Most studies tested representative users and reported their feelings and perceptions towards Socially Assistive Systems. Generally, this is an important rule for design researchers and practitioners to recruit representative users in system evaluation . However, sometimes it becomes problematic for investigating assistive systems that support social interactions. Example difficulties include: (1) not easy to get touch with such participants, even contact with institutions; (2) difficulty to find sufficient number of the participants to run a rigorous experimental studies; (3) difficulty to get verbal feedback from such participants (e.g., people with dementia or ASD). Due to these reasons, proxy users, or so called alternative users are still used in system evaluation. There are two primary types of proxy users. The first type refers to able-bodied people who simulate a given disability in certain situation . The most common is testing blindfolded participants as a replacement of real blind people (e.g., [14, 42, 49]). In , researchers evaluated a prototype namely Social Glasses by recruiting both blind and blindfolded participants in laboratory experiments. They used this way to achieve a large N and high statistical power in quantitative analysis. Generally, proxy users are used in the preliminary experiments ([46, 47, 83, 85, 93, 99]). Such studies reported work in progress of the systems. Proxy users could participate in these preliminary evaluations to examine the system feasibility. The second type of proxy users indicates stakeholders who are most familiar with target users, such as family members, caregivers, experts. The typical example is internet proxy user. Older adults’ grand children could be treated as proxy users to do online shopping on behave of older adults . In our reviewed papers, in , caregivers were treated as proxy users. Researchers examined caregivers rather than older adults themselves regarding how Socially Assistive Systems could support older adults to establish social connectedness.
Engaging field setting in system evaluation
Although the laboratory experiments are still the mainstream in system evaluation, we found a considerable number of studies conducted in field settings, which were primarily targeted at older adults ([21, 22, 61, 66, 69, 74,75,76, 78, 86, 91, 97]) and children ([13, 51, 52, 55, 62, 77, 89, 92]). This might be explained by the age-related issue. The special cognitive stage of older adults and children could bring challenges for researchers to make them follow the experimental procedures, so conducting field experiments or qualitative studies in the field settings became a more acceptable form for older adults and children. Numerous studies have proved that older adults and children are vulnerable to their surrounding changes, especially for those with dementia or ASD ([108, 109]). In the field settings, the participants are more likely to perform naturally in their familiar environments, which can ensure the validity of such research by reducing “the Hawthorne Effect” . Apart from the age-related issues, field setting provides an appropriate way to investigate the effect of Socially Assistive Systems on the participants’ social relationships and feelings, as well as daily social interaction, which needs to be evaluated in relatively long term. The majority of field experiments (6/11) were conducted in the participants’ own home. Different from field experiments, most qualitative studies (13/20) in the field settings were conducted in institutions, such as care centers and schools. This is partly because these locations are common social environments. Another important reason is the accessibility to the participants and external support. For example, some older adults have difficulties in accepting and using new technologies, which need external supports from institutions.
Promoting online social experience
In recent days, due to the risk of the COVID-19 pandemic, older adults and people with disabilities are more vulnerable than other people. They have to limit in-person interactions with others as much as possible, and face-to-face communication becomes quite rare. During the isolation period, anxiety, depression, poor sleep quality and physical inactivity have been reported . In this context, we could identify a promising domain that enhance the online social experience for older adults and people with disabilities. For instance, using an online social exergame to attract young people and their older family adults to play together, is not only able to promote physical activity, but also to increase the fun of games and their positive emotions . In this situation, social exergaming is helpful to increase bonds with family members and reduce social anxiety. Additionally, in our reviewed papers,  presented a multimedia story-telling severce to support people with speech and hearing impairments to interact socially when living alone.
This study has several limitations. Firstly, we only searched research papers in English language and it is possible that valuable findings are reported in other languages. Secondly, the research papers were limited to search from six databases (i.e., Scopus, Web of Science, ACM, Science Direct, PubMed, and IEEE Xplore). In order to increase completeness, further more databases in social science could be added, such as ASSIA and EBSCO. Thirdly, this study is limited by the search terms used and the time period of papers published, although the focus on the past 21 years could largely guarantee this systematic review covered the most recent research studies. Fourth, we discussed the general evaluation methods of Socially Assistive Systems in this review, but still lack detailed discussions regarding data gathering in system evaluation. We could analyze date gathering methods of Socially Assistive Systems evaluation in our future work .
This paper provides an overview of Socially Assistive Systems for older adults and people with disabilities. For this systematic review, we analyzed 65 papers from two major aspects: system design and evaluation.
In system design, our results indicated 11 types of HCI technologies that supported social interactions for target users. These technologies were often adopted in combinations to construct systems. The most common was cognitive and meaning understanding technologies, often applied with wearable devices for compensating users’ sensory loss. Some systems adopted emerging technologies such as AmI, VR, AR, which revealed great potential and worth in-depth investigation. Nearly one-third of the studies involved end-users and stakeholders in the design phase. User/stakeholders involvement were observed changing from “context providers” to “co-designers”; from merely utility-needs finding to meaning making in regard to sociocultural aspects.
In system evaluation, we identified four types of evaluation methods. The majority of studies adopted the laboratory experiments to measure user system interaction and system validation. Due to the difficulty of finding and taking target users to a specific location to participate in the study, proxy users were still used in system evaluation, especially in initial experiments. Some 42.46% of all evaluations were conducted in the field settings, primarily including the participants’ own home and institutions. In these settings, the participants could feel safe, more likely to perform naturally in their familiar environments and reduce “the Hawthorne Effect”. Finally, due to the impact of the coronavirus pandemic, we identify the research opportunity of designing Socially Assistive Systems that support online social experience.
Availability of data and material
Full search strategy available from the authors on request.
Ambient assisted living;
American psychological association;
Autism spectrum disorder;
Information and communications technology;
Institutional review board;
Medical subject heading;
Social assistive robot;
Socially assistive system;
Tinetti, M.E., et al.: Home-based multicomponent rehabilitation program for older persons after hip fracture: a randomized trial. Arch. Phys. Med. Rehabil. 80(8), 916–922 (1999)
International Organization for Standardization: ISO 9999:2016 Assistive products for persons with disability—Classification and terminology. International Organization for Standardization, Geneva (2016)
World Health Organization: Assistive technology. Available: https://www.who.int/news-room/fact-sheets/detail/assistive-technology. Accessed 27 Mar 2021
Kim, S.C., Jeong, Y.S., Park, S.O.: RFID-based indoor location tracking to ensure the safety of the elderly in smart home environments. Pers. Ubiquitous Comput. 17(8), 1699–1707 (2013)
Prescher, D., Weber, G., Spindler, M.: “A tactile windowing system for blind users”. In: Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility, pp. 91–98 (2010)
Sampath, H., Agarwal, R., Indurkhya, B.: “Assistive technology for children with autism—Lessons for interaction design”. In: ACM International Conference Proceeding Series, pp. 325–333 (2013)
Sarji, D.K.: HandTalk: assistive technology for the deaf. Computer (Long. Beach. Calif) 41(70), 84–86 (2008)
Simpson, R.C., Levine, S.P.: “Adaptive shared control of a smart wheelchair operated by voice control”. In: IEEE International Conference on Intelligent Robots and Systems, vol. 2, pp. 622–626 (1997)
Brule, E., Bailly, G., Brock, A., Valentin, F., Denis, G., Jouffrais, C.: “MapSense: Multi-sensory interactive maps for children living with visual impairments”. In: Conference on Human Factors in Computing Systems - Proceedings, pp. 445–457 (2016)
Ducasse, J., Brock, A.M., Jouffrais, C.: “Accessible interactive maps for visually impaired users”. Mobility of Visually Impaired People: Fundamentals and ICT Assistive Technologies, Springer International Publishing, pp. 537–584 (2017)
Routhier, F., et al.: Mobility and participation of people with disabilities using mobility assistive technologies: Protocol for a mixed-methods study. JMIR Res. Protoc. 8(4), e12089 (2019)
Sehgal, M., Jacobs, J., Biggs, W.S.: Mobility assistive device use in older adults. Am. Fam. Physician 103(12), 737–744 (2021)
Escobedo, L., et al.: “MOSOCO: a mobile assistive tool to support children with autism practicing social skills in real-life situations”. In: Conference on Human Factors in Computing Systems—Proceedings, pp. 2589–2598 (2020)
Qiu, S., Hu, J., Han, T., Osawa, H., Rauterberg, M.: An evaluation of a wearable assistive device for augmenting social interactions. IEEE Access 8, 164661–164677 (2020)
Shinohara, K., Wobbrock, J.O.: “In the shadow of misperception: assistive technology use and social interactions”. In: Conference on Human Factors in Computing Systems—Proceedings, pp. 705–714 (2011)
Baumeister, R.F., Leary, M.R.: The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychol. Bull. 117(3), 497–529 (1995)
Runco, M.A.: Personality and motivation. Creativity, vol. 1, pp. 265–302 (2014)
Singh, A., Misra, N.: Loneliness, depression and sociability in old age. Ind. Psychiatry J. 18(1), 51–55 (2009)
Naraine, M.D., Lindsay, P.H.: Social inclusion of employees who are blind or low vision. Disabil. Soc. 26(4), 389–403 (2011)
WHO: World report on ageing and health 2015. Geneva: World Health Organization (2015)
Li, C., Kang, K., Lin, X., Hu, J., Hengeveld, B., Hummels, C.: Promoting older residents’ social interaction and wellbeing: a design perspective. Sustainability 12(7), 2834 (2020)
Dadlani, P., Sinitsyn, A., Fontijn, W., Markopoulos, P.: Aurama: caregiver awareness for living independently with an augmented picture frame display. AI Soc. 25(2), 233–245 (2010)
Magee, J.J., Betke, M.: “Automatically generating online social network messages to combat social isolation of people with disabilities.” In: International Conference on Universal Access in Human-Computer Interaction, pp. 684–693 (2013)
Qiu, S., Hu, J., Han, T., Osawa, H., Rauterberg, M.: Social glasses: simulating interactive gaze for visually impaired people in face-to-face communication. Int. J. Hum. Comput. Interact. 36(9), 839–855 (2020)
Boyd, l.E., et al.: “VrSocial: Toward immersive therapeutic VR systems for children with autism”. In: Conference on Human Factors in Computing Systems—Proceedings, pp. 1–12 (2018)
Ienca, M., et al.: Intelligent assistive technology for alzheimer’s disease and other dementias: a systematic review. J. Alzheimer’s Dis. 56(4), 1301–1340 (2017)
Wang, Q., Markopoulos, P., Yu, B., Chen, W., Timmermans, A.: Interactive wearable systems for upper body rehabilitation: a systematic review. J. Neuroeng. Rehabil. 14(1), 1–21 (2017)
Fernandes, H., Costa, P., Filipe, V., Paredes, H., Barroso, J.: A review of assistive spatial orientation and navigation technologies for the visually impaired. Univ. Access Inf. Soc. 18(1), 155–168 (2019)
Bemelmans, R., Gelderblom, G.J., Jonker, P., de Witte, L.: Socially assistive robots in elderly care: a systematic review into effects and effectiveness. J. Am. Med. Dir. Assoc. 13(2), 114-120.e1 (2012)
Kachouie, R., Sedighadeli, S., Khosla, R., Chu, M.T.: Socially assistive robots in elderly care: a mixed-method systematic literature review. Int. J. Hum. Comput. Interact. 30(5), 369–393 (2014)
Erich, F., Hirokawa, M., Suzuki, K.: “A systematic literature review of experiments in socially assistive robotics using humanoid robots”. https://arxiv.org/abs/1711.05379 (2017)
Chen, Y.R., Schulz, P.J.: The effect of information communication technology interventions on reducing social isolation in the elderly: a systematic review. J Med. Internet Res. 18(1), e4596 (2016)
Pedrozo Campos Antunes, T., et al.: Assistive technology for communication of older adults: a systematic review. Aging Ment. Health 23(4), 417–427 (2011)
Henry, S.: Participatory design: theory & techniques. Raleigh, NC: Bookmasters (1990)
Pino, M., Boulay, M., Jouen, F., Rigaud, A.S.: “‘Are we ready for robots that care for us?’ Attitudes and opinions of older adults toward socially assistive robots”. Front. Aging Neurosci. 7
Brandt, Å., Samuelsson, K., Töytäri, O., Salminen, A.L.: Activity and participation, quality of life and user satisfaction outcomes of environmental control systems and smart home technology: a systematic review. Disabil. Rehabil. Assist. Technol. 6(3), 189–206 (2011)
Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G.: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann. Intern. Med. 151(4), 264–269 (2009)
Martinez-Martin, E., Escalona, F., Cazorla, M.: Socially assistive robots for older adults and people with autism: an overview. Electronics 9(2), 367 (2020)
Kabacińska, K., Prescott, T.J., Robillard, J.M.: Socially assistive robots as mental health interventions for children: a scoping review. Int J of Soc Robot. 13, 919–935 (2021)
Inaba, T., Squicciarini, M.: “ICT: a new taxonomy based on the international patent classification”. OECD Science, Technology and Industry Working Papers (2017)
Rahman, A.K.M.M., Anam, A.I., Yeasin, M.: EmoAssist: emotion enabled assistive tool to enhance dyadic conversation for the visually impaired. IEEE Trans. Affect. Comput. 76(6), 7699–7730 (2017)
Anam, A.I., Alam, S., Yeasin, M.: “Expression: a dyadic conversation aid using google glass for people with visual impairments”. In: UbiComp 2014—Adjunct Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 211–214 (2014)
Zhao, X., Zou, J., Li, H., Dellandrea, E., Kakadiaris, I.A., Chen, L.: Automatic 2.5-D facial landmarking and emotion annotation for social interaction assistance. IEEE Trans. Cybern. 46(9), 2042–2055 (2016)
Buimer, H.P., Bittner, M., Kostelijk, T., van der Geest, T.M., Vanwezel, R.J.A., Zhao, Y.: “Enhancing emotion recognition in VIPs with haptic feedback” In: Communications in Computer and Information Science, pp. 157–163 (2016)
Krishna, S., Panchanathan, S.: “Assistive technologies as effective mediators in interpersonal social interactions for persons with visual disability”. In: International Conference on Computers for Handicapped Persons, pp. 316–323 (2010)
Terven, J.R., Salas, J., Raducanu, B.: “Robust head gestures recognition for assistive technology”. In: Mexican Conference on Pattern Recognition, pp. 152–161 (2014)
Hermann, T., Neumann, A., Zehe, S.: “Head gesture sonification for supporting social interaction”. In: ACM International Conference Proceeding Series, pp. 82–89 (2012)
Qiu, S., Han, T., Rauterberg, M., Hu, J.: Impact of simulated gaze gestures on social interaction for people with visual impairments. Adv. Transdisciplinary Eng. 7, 249–258 (2018)
Qiu, S., Rauterberg, M., Hu, J.: “Designing and evaluating a wearable device for accessing gaze signals from the sighted”. In: International Conference on Universal Access in Human-Computer Interaction, pp. 454–464 (2016)
Sarfraz, M.S., Constantinescu, A., Zuzej, M., Stiefelhagen, R.: A multimodal assistive system for helping visually impaired in social interactions. Informatik-Spektrum 40(6), 540–545 (2017)
Voss, C., et al.: “Superpower Glass: delivering unobtrusive eal-Time social cues in wearable systems”. In: UbiComp 2016 Adjunct—Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 1218–1226 (2016)
Hsieh, Y.H., et al.: Communicative interaction with and without eye-gaze technology between children and youths with complex needs and their communication partners. Int. J. Environ. Res. Public Health 18(10), 5134 (2021)
Lee, K., Sato, D., Asakawa, S., Kacorri, H., Asakawa, C.: “Pedestrian detection with wearable cameras for the blind: a two-way perspective”. In: Conference on Human Factors in Computing Systems - Proceedings, pp. 1–12 (2020)
Wang, N., Yu, K., Li, J., Zhang, R., Ren, F.: “Readful-U: improving reading experience and social interaction for low vision elders”. In: Conference on Human Factors in Computing Systems—Proceedings, pp. 80–85 (2016)
Black, R., Waller, A., Tintarev, N., Reiter, E., Reddington, J.: “A mobile phone based personal narrative system” In: ASSETS’11: Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 171–178 (2011)
Abdallah, E.E., Fayyoumi, E.: Assistive technology for deaf people based on android platform. Procedia Comput. Sci. 94, 295–301 (2016)
McCarron, H.R., Zmora, R., Gaugler, J.E.: “A web-based mobile app with a smartwatch to support social engagement in persons with memory loss: pilot randomized controlled trial”. J. Med. Internet Res. 21(6) (2019)
Wu, S., Wieland, J., Farivar, O., Schiller, J.: “Automatic alt-text: computer-generated image descriptions for blind users on a social network service”. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, pp. 1180–1192 (2017)
Zolyomi, A., Shukla, A., Snyder, J.: “Technology-mediated sight: a case study of early adopters of a low vision assistive technology”. In: ASSETS 2017—Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 220–229 (2017)
Meza-de-Luna, M.E., Terven, J.R., Raducanu, B., Salas, J.: A Social-Aware Assistant to support individuals with visual impairments during social interaction: a systematic requirements analysis. Int. J. Hum. Comput. Stud. 122, 50–60 (2019)
Bellini, G., et al.: Understanding social behaviour in a health-care facility from localization data: a case study. Sensors 21(6), 1–22 (2021)
Bonnin, F., Gonzalez, V.M.: “TEA band: an interactive system to aid students with autism improve their social interactions”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10289 LNCS, pp. 621–635 (2017)
Fleury, A., Wu, G., Chau, T.: A wearable fabric-based speech-generating device: system design and case demonstration. Disabil. Rehabil. Assist. Technol. 14(5), 434–444 (2019)
Marti, P., Recupero, A.: “Is deafness a disability?: designing hearing AIDS beyond functionality”. In: Proceedings of the 2019 Creativity and Cognition, pp. 133–143 (2019)
Theil, A., et al.: “Tactile board: a multimodal augmentative and alternative communication device for individuals with Deafblindness”. In: ACM International Conference Proceeding Series, pp. 223–228 (2002)
Garattini, C., Wherton, J., Prendergast, D.: Linking the lonely: an exploration of a communication technology designed to support social interaction among older adults. Univers. Access Inf. Soc. 11(2), 211–222 (2012)
Hine, N., Arnott, J.L.: “Assistive social interaction for non-speaking people living in the community”. In: Proceedings of the fifth international ACM conference on assistive technologies, pp. 162–169 (2002)
Shim, N., Baecker, R., Birnholtz, J., Moffatt, K.: “TableTalk Poker: an online social gaming environment for seniors”. In: Future Play 2010: Research, Play, Share—International Academic Conference on the Future of Game Design and Technology, pp. 98–104 (2010)
Sauvé, L., Renaud, L., Kaufman, D., Duplàa, E.: “Can digital games help seniors improve their quality of life?”. In: Communications in Computer and Information Science, vol. 739, pp. 179–192 (2017)
Baez, M., Ibarra, F., Far, I.K., Ferron, M., Casati, F.: “Online group-exercises for older adults of different physical abilities”. In: 2016 International Conference on Collaboration Technologies and Systems (CTS), pp. 524–533 (2017)
Tamplin, J., Loveridge, B., Clarke, K., Li, Y., Berlowitz, D.J.: “Development and feasibility testing of an online virtual reality platform for delivering therapeutic group singing interventions for people living with spinal cord injury”. J. Telemed. Telecare (2019)
Papa, F., Cornacchia, M., Sapio, B., Nicolò, E.: Engaging technology-resistant elderly people: empirical evidence from an ICT-enabled social environment. Informatics Heal. Soc. Care 42(1), 43–60 (2017)
Fuchsberger, V., Sellner, W., Moser, C., Tscheligi, M.: “Benefits and hurdles for older adults in intergenerational online interactions”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7382 LNCS, no. PART 1, pp. 697–704 (2012)
Isaacson, M., Cohen, I., Shpigelman, C.N.: “Leveraging emotional wellbeing and social engagement of the oldest old by using advanced communication technologies: a pilot study using uniper-care’s technology”. In: IET Conference Publications, vol. 2019, no. CP755 (2019)
Yurkewich, A., Stern, A., Alam, R., Baecker, R.: A field study of older adults with cognitive impairment using tablets for communication at home: closing technology adoption gaps using InTouch. Int. J. Mob. Hum. Comput. Interact. 10(2), 1–30 (2018)
Wu, J., Koon, L.C.: “Tangible social-media application for the elderly”. In: i-CREATe 2012—6th International Convention on Rehabilitation Engineering and Assistive Technology, pp. 1–4 (2012)
Boyd, L.E., Ringland, K.E., Haimson, O.L., Fernandez, H., Bistarkey, M., Hayes, G.R.: Evaluating a collaborative iPad game’s impact on social relationships for children with autism spectrum disorder. ACM Trans. Access. Comput. 7(1), 1–18 (2015)
Nijhof, N., Van Hoof, J., Van Rijn, H., Van Gemert-Pijnen, J.E.W.C.: The behavioral outcomes of a technology-supported leisure activity in people with dementia. Technol. Disabil. 25(4), 263–273 (2013)
Tapia, J.M., Gutierrez, F.J., Ochoa, S.F.: “Using smart TV applications for providing interactive ambient assisted living services to older adults”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10069 LNCS, pp. 514–524 (2016)
Purves, B.A., Phinney, A., Hulko, W., Puurveen, G., Astell, A.J.: Developing CIRCA-BC and exploring the role of the computer as a third participant in conversation. Am. J. Alzheimers. Dis. Other Demen. 30(1), 101–107 (2015)
Astell, A.J., et al.: Using a touch screen computer to support relationships between people with dementia and caregivers. Interact. Comput. 22(4), 267–275 (2010)
Vincent, C., Deaudelin, I., Hotton, M.: Pilot on evaluating social participation following the use of an assistive technology designed to facilitate face-to-face communication between deaf and hearing persons. Technol. Disabil. 19(4), 153–167 (2007)
Bala, S., McDaniel, T., Panchanathan, S.: “Visual-to-tactile mapping of facial movements for enriched social interactions”. In: 2014 IEEE International Symposium on Haptic, Audio and Visual Environments and Games, HAVE 2014—Proceedings, pp. 82–87 (2014)
McDaniel, T., Tran, D., Devkota, S., DiLorenzo, K., Fakhri, B., Panchanathan, S.: “Tactile facial expressions and associated emotions toward accessible social interactions for individuals who are blind”. In: MAHCI 2018—Proceedings of the 2018 Workshop on Multimedia for Accessible Human Computer Interface, co-located with MM 2018, pp. 25–32 (2018)
McDaniel, T.L., Villanueva, D., Krishna, S., Colbry, D., Panchanathan, S.: “Heartbeats: a methodology to convey interpersonal distance through touch”. In: Conference on Human Factors in Computing Systems—Proceedings, pp. 3985–3990 (2010)
Feng, Y., van Reijmersdal, R., Yu, S., Hu, J., Rauterberg, M., Barakova, E.: “Using observational engagement assessment method VC-IOE for evaluating an interactive table designed for seniors with dementia.” In: International Conference on Smart Health, pp. 26–37 (2017)
Brok, J.C.J., Barakova, E.I.: “Engaging autistic children in imitation and turn-taking games with multiagent system of interactive lighting blocks”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6243 LNCS, pp. 115–126 (2010)
Nguyen, T., Garrett, R., Downing, A., Walker, L., Hobbs, D.: An interfacing system that enables speech generating device users to independently access and use a mobile phone. Technol. Disabil. 20(3), 225–239 (2008)
Hourcade, J.P., Bullock-Rest, N.E., Hansen, T.E.: Multitouch tablet applications and activities to enhance the social skills of children with autism spectrum disorders. Pers. Ubiquitous Comput. 16(2), 157–168 (2012)
Miller, D., Gyllstrom, K., Stotts, D., Culp, J.: “Semi-transparent video interfaces to assist Deaf persons in meetings.” In: Proceedings of the Annual Southeast Conference, vol. 2007, pp. 501–506 (2007)
Nazzi, E., Sokoler, T.: “TwitterIDo: what if my shopping bag could tell my friends I’m out shopping”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9194, pp. 512–523 (2015)
Hirano, S.H., Yeganyan, M.T., Marcu, G., Nguyen, D.H., Boyd, L.A., Hayes, G.R.: Evaluation of a system to support classroom activities for children with autism”. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp. 1633–1642 (2011)
Gilfeather-Crowley, P., Smith, C.E., Youtsey, S.: “Connecting visually-impaired people to friends through wireless sensor networks.” In: Conference Proceedings—IEEE International Conference on Systems, Man and Cybernetics, pp. 3224–3229 (2011)
Gugenheimer, J. et al.: “The impact of assistive technology on communication quality between deaf and hearing individuals”. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, pp. 669–682 (2017)
Kim, S., Lee, K.P., Nam, T.J.: “Sonic-badminton: audio-augmented Badminton game for blind people”. In: Conference on Human Factors in Computing Systems—Proceedings, pp. 1922–1929 (2016)
E. Bekele et al.: “Multimodal adaptive social interaction in virtual environment (MASI-VR) for children with Autism spectrum disorders (ASD)” In: 2016 IEEE Virtual Reality (VR), pp. 121–130 (2016)
Lin, C.X., Lee, C., Lally, D., Coughlin, J.F.: “Impact of virtual reality (VR) experience on older adults’ well-being”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10927 LNCS, pp. 89–100 (2018)
Davis, K., Owusu, E.B., Marcenaro, L., Feijs, L., Regazzoni, C., Hu, J.: Effects of ambient lighting displays on peripheral activity awareness. IEEE Access 5, 9318–9335 (2017)
Pingali, T.R., Lemaire, E.D., Baddour, N.: Ultrasonic Tethering to Enable Side-by-Side Following for Powered Wheelchairs. Sensors (Basel) 19(1), 109 (2019)
Johnson, E., Hervás, R., de la Franca, C., Mondéjar, T., Ochoa, S.F., Favela, J.: Assessing empathy and managing emotions through interactions with an affective avatar. Health Inf. J. 24(2), 182–193 (2018)
Vinciarelli, A., Pantic, M., Bourlard, H.: Social signal processing: Survey of an emerging domain. Image Vis. Comput. 27(12), 1743–1759 (2009)
Allen, M.: The SAGE encyclopedia of communication research methods. Sage Publications (2017)
Sanders, E.B., Stappers, P.J.: Co-creation and the new landscapes of design. Taylor Fr 4(1), 5–18 (2008)
Steen, M.: Human-centered design as a fragile encounter. Des. Issues 28(1), 72–80 (2012)
Visser, F.S., Stappers, P.J., van der Lugt, R., Sanders, E.B.-N.: Contextmapping: experiences from practice. CoDesign 1(2), 119–149 (2005)
Sears, A., Hanson, V.L.: “Representing users in accessibility research”. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 2235–2238 (2011)
Dolničar, V., Grošelj, D., Hrast, M.F., Vehovar, V., Petrovčič, A.: The role of social support networks in proxy Internet use from the intergenerational solidarity perspective. Telemat. Inf. 35(2), 305–317 (2018)
Carstensen, L.L., Erickson, R.J.: Enhancing the social environments of elderly nursing home residents: are high rates of interaction enough? J. Appl. Behav. Anal. 19(4), 349–355 (1986)
Zaccaria, K.J., Lagace, D.C., Eisch, A.J., McCasland, J.S.: Resistance to change and vulnerability to stress: autistic-like features of GAP43-deficient mice. Genes Brain Behav. 9(8), 985–996 (2010)
McCarney, R., Warner, J., Iliffe, S., Van Haselen, R., Griffin, M., Fisher, P.: The hawthorne effect: a randomised, controlled trial. BMC Med. Res. Methodol. 7(1), 1–8 (2007)
Sepúlveda-Loyola, W., et al.: Impact of social isolation due to COVID-19 on health in older people: mental and physical effects and recommendations. J. Nutr. Heal. Aging 24(9), 938–947 (2020)
Kaisar, E., Ding, R.B., Han, T., Qiu, S.: “NEONEO balance ball: designing an intergenerational interaction exergame for in-home balance training”. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12787 LNCS, pp. 78–89 (2021)
Qiu, S., An, P., Kang, K., Hu, J., Han, T., Rauterberg, M.: “A review of socially assistive systems: data gathering methods for people with special needs,” pp. 1–39, manuscript (2021)
We would like to thank Bin Yu from Philips Design and Yudan Ma from Eindhoven University of Technology for their suggestions and comments. This work is supported by the [Shanghai Pujiang Program] under Grant [2020PJC071]; [Shanghai Jiao Tong University] under Grant [WF220543011].
This work is supported by the [National Social Science Fund] under Grant [18BRK009]; [Shanghai Pujiang Program] under Grant [2020PJC071]; [Shanghai Jiao Tong University] under Grant [WF220543011].
Conflict of interest
The authors declare that they have no conflict of interest/competing interests.
Consent for publication
All authors read and approved the manuscript for publication.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised due to update in author affiliation.
About this article
Cite this article
Qiu, S., An, P., Kang, K. et al. Investigating socially assistive systems from system design and evaluation: a systematic review. Univ Access Inf Soc (2021). https://doi.org/10.1007/s10209-021-00852-w
- Assistive technology
- Socially assistive system
- Social interaction
- Older adults
- People with disabilities