Keywords

1 Introduction

This paper reports from a diagnostic evaluation study of public home care service for the elderly by use of a video consultation system that builds on existing and familiar equipment in the home, i.e., the television. The study is a part of a larger collaborative change experiment, a longitudinal study set to last for 3 years. The motivation of the experiment is to study remote caring for active elderly people by transferring selected home care services to delivery through ICT. A challenge in the conventional home care service is that many active elderly are unable to start their day until the home care nurse has been on the daily visit, e.g., to give medications. The introduction of ICT-supported delivery of home care services allows active elderly people to receive home care services at more suitable and fixed times.

However, there is still a need to further explore how the care givers and care receivers experience ICT-supported care, especially bringing attention to usability according to the system’s ability to perform selected care tasks by real users within their specific user context [1]. In our prior work [2], we presented findings from a usability testing study of the video consultation system. The study was conducted in controlled environments in our demonstration apartment in a building for care homes as the third phase of the long-term collaborative change experiment later illustrated in Fig. 1. This paper presents findings from the fourth step of the collaborative change experiment, where the video consultation system has been refined and moved from controlled environments into the homes of the elderly home dwellers. Hence, this paper reports from a diagnostic evaluation of the video consultation system after being adopted into real environments.

Fig. 1.
figure 1

Overview of the steps included in the collaborative change experiment

Full size image

The contribution of this paper adds to the existing HCI literature on caregiving through television by presenting findings from a longitudinal study of a video consultation system deployed and integrated into the work routines of the home care workers and as a part of the daily life activities of elderly home dwellers. Our research discusses both practical and methodical challenges when moving a system from controlled settings into real environments, and during the diagnostic evaluation, we emphasize on capturing the two-sided user experience of care including both the care givers (home care nurses) and care receivers (elderly home dwellers). Our results present users’ assessment of system quality as well as discovered usability issues during delivery of home care via video consultation. Altogether, our diagnostic evaluation involved 34 participants, namely 16 home care workers and 18 elderly home dwellers.

The paper is organized as follows. Section 2 presents an overview of related work within the HCI-community. Section 3 briefly describes the longitudinal collaborative change experiment. Section 4 gives and overview of sessions and distribution of participants as well as our methodic approach, while Sect. 5 presents the results. In Sect. 6, we discuss usability issues based on our results and relate these to the previous work within HCI. Section 7 concludes the paper.

2 Related Work

Home telecare is a popular topic within the HCI community. However, few studies have addressed usability issues of incorporated telecare technologies between the home care nurses and their elderly care receivers directly. Milligan, Roberts and Mort [3] have studied how the move of telecare technologies affects the context of the private space, as well as the user experience of the care service. They state that “Telecare affects the nature of care interactions within the home; hence the widespread adoption of these technologies is likely to have a significant impact on the broader landscape of care.” [3, p. 349]. The authors pinpoint that homes that are transformed into institutional context – with all kinds of medical devices and public regulations – conflicts with the gain of staying in the home for these home dwellers. They argue that designers have addressed these issues in recent time by developing and integrating telecare devices that match the layout of the home in order integrate assistive technologies in a subtle manner by providing “invisible” [p. 353] support in the home. Doyle, Bailey and Scanaill [4] confirm how few studies have looked into use of technology in practice, and further add that independent living technologies need to be moved into the homes of elderly people. This way, it can be tested in real environment in order to assess the real value of the design and to study the impact that the technology could have on their lives. Goodman-Deane and Lundell [5] continue the discussion on how the design of technology should meet the needs of the elderly by emphasizing the importance on capturing the user needs of “real older people, including “baby boomers” still in employment, frail older people with disabilities and the full range in between [6, p. 3]”. Blythe, Monk and Doughty [6] explore the needs of the elderly people, and how these provide design implications for HCI. Their study is based on findings from structured interviews with health professionals and elderly people. The authors express concern about technologies used for monitoring bring very little attention to the social context of the home.

Other HCI studies report findings from collaborative or interactive services where elderly people use the television from their home as a platform to receive telecare or similar services [7]. Several studies have made contributions that concern age-related challenges when designing for the elderly generation [8–12]. Others have provided new knowledge on how to develop interfaces usable for older people, e.g., [13, 14]. For instance, Baunstrup and Larsen [13] point out that the television has evolved from a one-way monologue into a communications platform by offer increased dialogue-based services. They also argue that an iTV provides more “complex interaction paradigm” [13, p. 13] since it usually involves additional equipment such as set-top box, additional monitor and media streaming device. Other research contributions emphasize the importance of studying elderly people who already master the interface in the search for compensatory strategies that may be generally applicable to this user group in order to improve the user experience [15]. They point out that previous studies, e.g., [12], mainly deal with physical, sensory and cognitive limitations that come with aging, while they them-selves believe that one should also include aspects of “privacy, acceptability, stigma, control, trust, choice and social alienation” [15, p. 614] into the design process. Specifically, they believe that privacy and trust to be key elements when HCI research enters private homes and communities.

3 Collaborative Change Experiment

We have designed a collaborative change experiment consisting of five stepwise activities. Through these five activities, we aim to experiment with alternative solutions to the existing routines in the delivery of home care services. We emphasize that our goal is not to bring in a permanent change, but rather to explore underlying issues and gain a deeper insight that may contribute to a future permanent change. To address the inter-dependency, we have designed our collaborative change experiment in such a way that it captures usability issues on both sides of the service. Two traditional task-centered user evaluations make out the key activities in our change experiment: the usability testing reported in [2], and the diagnostic evaluation presented in this paper. Common for both of these activities is that they have been expanded from a traditional user-observer setup to a parallel experiment where we have users and observers on both sides of the service simultaneously. In addition to these two main activities, we have supplemented the collaborative change experiment with three supporting activities that we believe helps strengthen the design process, as well as make it more coherent. Through these auxiliary activities, we (1) address some of the challenges that are not directly covered by traditional usability testing, and we (2) gain important input that contribute directly towards the facilitation of the usability testing and diagnostic evaluation. Figure 1 illustrates the five activities and their order. A comprehensive description of the collaborative change experiment was previously presented in [2].

4 Diagnostic Evaluation

The diagnostic evaluation consisted of three different sessions and involved 34 participants altogether. The first session was held in the office of the home care workers over two days and included 12 home workers and administrators. The second session was conducted at the local care home with 14 elderly participants. The third session was a two-way test where we were present on both sides of the interaction and captured issues from both perspectives. This session involved 4 home care workers and 4 elderly participants. Table 1 gives an overview of the three sessions and the distribution of participants. The reason for the number of participant being considerably lower in the third test was that it required parallel presences and had to be coordinated beforehand so we could be ready in the homes of the elderly participants for the call. Nevertheless, all three sessions followed the same methodical procedure and used the same criteria and scale for the tests. Participants were asked to give an evaluation of how they perceived the quality of the system from a technical perspective by grading the image clarity, sound quality and light conditions. For each conversation, we kept track of usability issues by counting five particular values: number of attempts to establish the call, numbers of words repeated during the conversation, number of image freezes, number of drops in image quality, and number of instances with choppy sound.

Table 1. Overview of the three sessions in the diagnostic evaluation
Full size table

In order to capture a broader frame with environmental factors, e.g., lightning and tidiness, all participants, were equipped with 40-inch televisions with wide-angle HD-camera, as well as all necessary network facilities. The nurses called each home dweller, and they had to actively answer the call by using a remote controller in order to establish the connection.

The empirical setting for our study was a local care building with 87 care homes for elderly. The elderly participants were located in their private homes while the nurses were at their homecare service office. Figure 2 demonstrates how two of the home care nurses used the setup in their office, while Fig. 3 shows the setup at the homes of the 4 home dwellers that participated in session 3.

Fig. 2.
figure 2

Two home care workers talking to an elderly from their office

Full size image
Fig. 3.
figure 3

The television setup in the apartments of the four elderly participants in session 3

Full size image

5 Result

5.1 System Quality

One half of the evaluation consisted of participants assessing image clarity, sound quality and light conditions. Table 2 summarizes the average score reported by each group. As we can see from the table, the assessment of image clarity, sound quality and light conditions were very similar across the three groups. All three usability metrics had low internal standard deviation (σ image  = 0.37, σ sound  = 0.16, σ light  = 0.17), which indicates that the quality of image, sound and light were considered similar equal between the three groups. The average score given by each group for the three quality metrics (σ session1  = 8.33, σ session2  = 7.95, σ session3  = 8.12) suggests that the system is – from a technical perspective – capable of delivery all tasks listed in our previous paper [2], and that the overall assessment of the system itself was positive. Figure 4 illustrates the grades given on image clarity, sound quality and light conditions by each of the 34 participants that contributed to the three tests. The lowest grade given at any point was 5 out of 10. This suggest that there were no incidents at any time during the 34 tests – involving 95 min with real delivery of home care services – where the image, sound or light were considered to be worse than 5 out of 10.

Table 2. Average assessment of image, sound and light quality
Full size table
Fig. 4.
figure 4

Overview of grading on image, sound and light qualities from all participants

Full size image

5.2 Usability Issues

The second part consisted of tracking usability issues that arose during the conversation between the elderly person and the home care nurse. We kept track of five values in order to count the numbers of issues during each conversation. Number of attempts to establish the call helped us record how many times a home care nurse had to call before the elderly participant answered the call. Number of words repeated during the conversation gave us an idea of how many times there was a miscommunication and one of the parties had to repeat a word or sentence. We only counted those issues that happened due to technical problems (e.g., choppy sound) as elderly people struggling to hear everything is still an issue with physical presence. Number of image freezes provided a count on how many times the screen froze or went black. While this often accompanied sound issues, there were instances of only image freezes without any sound issues. Number of drops in image quality was an indicator of how often the quality of the image would drop to a low-resolution, pixelated and blurry image. This usually happed when the Internet connection was slow or unstable. Number of in-stances with choppy sound captured all occurrences of unclear or dropping sound. Table 3 gives an overview of the average number of issues per conversation we registered in each session. The conversation time is included for all session in order to give an idea of error counts not only per conversation, but also per minute.

Table 3. Average issue count for the three sessions
Full size table

Table 3 presents the average number of issues registered per conversation. The number registered for each issue is fairly consistent between session 1 and 2 – with the biggest difference being 0.30 for sound chops. We registered a higher number of issues on per conversation during session 3, but that was only due to longer conversation time on average. In fact, the numbers of issues we registered during session 3 were lower than the number we registered during session 1 and 2. Since we were present at both sites simultaneously during session 3, the eight participants only yielded four conversations. Compared to session 1 and 2 with a total of 26 participants, we ended up with a much smaller selection this might have affected the number of errors we registered on average. However, being able to capture such issues on both sides of the interaction simultaneously allowed us to capture usability issues more accurately, thus we should expect the number to rise with additional two-sided evaluations. The average conversation time was 02:44 min, and the average numbers of errors per conversation was 8.10 (or 2.97errors/minute). As the deviation in errors/minute was low between these three sessions (σ = 0.16), we consider the diagnostic evaluation to have captured a representative understanding of usability issues experienced in real-life use rather than in laboratorial settings. As we can see from Fig. 5, the registered issues are for most part evenly distributed. The thick red polygon indicates the overall distribution, and on average, sound chops seem to be the biggest problem (25.0 %). This is one of the most common symptoms of Internet connectivity issues and was not unexpected.

Fig. 5.
figure 5

Distribution of issues for each session and overall aggravated distribution

Full size image

5.3 Beyond the Usability Issues

As we can see in Table 2, the overall assessment of the system quality was positive. However, the use of the system is a part of a larger context, in which we have to consider the task, the flow, the complexity, as well as the very presence of the technology in their homes. First of all, usability issues were not symmetrical, i.e., they were not experienced the same on both sides. Session 3 revealed big differences in both what issued that occurred on each side, as well as how the elderly participant experienced them. Issues due to network instability may result in sound or image issues on both sides, however we registered that the symptoms were rarely similar or never happened simultaneously. The grading assessment was also different during the two-sided observation in session 3. The fact that the number of issues we registered per minute was very similar between the two groups (session 1 and 2) suggests that over time, the issues converge towards very similar levels despite there being individual anomalies. Nevertheless, the type of issue did not matter as much as how disturbing it was to the established flow. For instance, one person in the middle of an exercise may not be bothered if the image quality drops for a few seconds as long as they can still see the main movements and hear the instructions. However, if the sound drops just as the health care worker is about to give instruction on taking medicine, then it becomes a serious issue. Ultimately, it was not the actions of the users but rather the connectivity that decided the frequency and duration of these issues. While the users themselves reported high scores on the isolated technical capabilities of the system, once we counted the issues that arose during conversations, we saw how the experience and quality of the service was greatly reduced due to connectivity issues.

6 Discussion

The results display usability issues of home telecare in real environments. Doyle et al. [4] recommend studying technology use in practice by moving out of the comfortable computer laboratory. Our experiences as we moved from controlled to real test environments were that the underlying contextual factors became more evident compared to the technical capabilities of the system, and the contextual factors had a high impact on the results.

6.1 Timing and User Context

We experienced less control as facilitators when testing in real environment concerning both practical and methodical issues, e.g., we did not know if the participants were awake and ready; or even at home at the time the video consultation was supposed to find place. Neither did we have control over the infrastructure, e.g. the network capacity in the private spaces nor if the camera was still connected to the television. Thus, the results were more affected by the users’ context than isolated technical capabilities, and the timing of the call had great impact on the user’s ability to incorporate ICT-supported care into their daily life activities. Moreover we lost the ability to observe every usability issue at close up in the move from controlled to real environment. Therefore, we had to translate the users’ problems as they reported them. In our previous usability study [2], we observed that the fixed position of the television screen and the camera within the living room were found helpful to compensate the participants’ decline motoric skills [2]. However, in real environment, the fixed position were not experienced as flexible in use for elderly users who were unexpected bedridden or for those doing their morning bath or eating breakfast in the kitchen. All participants had various mobility issues, which resulted in extra time spent moving from one place to another in the home. Thus, we support Dolye et al. [4] who stress the importance of testing in real environment. It is only then, when real users adopt the technology into their everyday life activities, researchers can gain knowledge of the sustainability of their design, and whether it can work in everyday practice with regards to necessary infrastructure, design and simplicity of use.

Another experience we made according to the user context was our interruption of the television domain. As we found it constructive to build the home telecare services on existing and familiar technology in the home by using the participants own television as a platform for the HD camera. This among others to avoid according to Milligan et al. [3] to transform the home with alienating telecare technologies that can affect the participant’s own perspective of the home, as well as the user’s experiences of the care service. However, we found it in some occasions difficult when disturbing the participants while watching television as e.g. one participant expressed: “Oh no, you just called me while the reporter interviewed Petter Northug (a famous ski athlete in Norway) – can you please switch the channel back.” The care giver may in traditional home care service use such an occasion to share the highlights in the news together with the care receiver in order to small talk. Thus, a care giver has to know the care receiver well in order to make a conversation beyond the set care tasks and context when moving the care interaction into a video conference. However, this could also imply that the participants may demand the home care nurses not to call them during their favorite television programs.

6.2 Complexity

A number of HCI researchers have pinpoint age-related challenges when designing for the elderly [8–14]. However, building services on familiar technology such as the television still require additional equipment for additional services - which are making the mastery of the television increasing in complexity. Our study participants have all a large and modern television in their living room, and a set up box for addition channels. Thus, the participants had to master an additional remote controller, and navigate to the correct hdmi source for video conference as we added the HD camera to their television. Moreover, the participants had to switch back to the correct hdmi source in order to return to their television services. This was experienced as troublesome for all of our elderly end-users. Moreover, a common user challenge for the majority of our participants was to locate the remote controller in order to operate the video conference. A universal or an integrated remote controller to the TV – chair may be a solution to this usability problem. Thus, the HD camera should be designed for manual operation as a backup option for lost remote controller. However, the participants need training and practice in order to fully see the potential and understand different use of a familiar technology – as well as they need to be allowed to get the necessary time in order to answer the video call.

6.3 Privacy and Trust

Niman et al. [16] emphasized the importance of including privacy and trust as central aspect of the design process [13], which are especially relevant for HCI researchers moving their test environment into private spaces. Two participants in our study asked if we as researchers could see them 24/7. Moreover, one of the participants pulled out the hdmi cable as she did not want us to see her half naked sitting at the kitchen table. Installing cameras in the homes of the elderly address both privacy and trust issues – and it is essential that we as researchers are aware of informing the participants thorough, and that we show humility and respect when entering private spaces. Moreover, we support that privacy and trust should be included in design, e.g. in our study we rather should have used a camera that had a “curtain” in front of the camera lens (when not in use) to avoid the end-users feeling that their privacy was intruded.

Additionally to contextual factors that were found present during diagnostic evaluation in real environment, we as well as experienced usability issues concerning deterioration of our participants health condition. We have over time experienced that the participants within our empirical setting have had increased need for health care services as a result of decreased functional capability. Blythe et al. [6] argue for the importance of capturing the broader range of user needs among elderly including users with high to low functional capabilities. However, home telecare is not for every elder, and merely for selected services that are experienced appropriate delivered as remote care. Thus, for elderly with complex health care needs, home telecare as delivery of health care services is not an option. However, if the home care nurses are able to reduce their overall work load by delegating some of their services via video conferences to those elderly who are more or less independent living, then the nurses can allocate their time more efficiently by providing traditional health care services to those care receivers who need them the most, and are depended on extended and local care in the home.

7 Conclusion

In this paper, we have studied the two-sided interaction of care between elderly home dwellers and home care nurses by use of video conference equipment in real environment. As we moved from controlled to real test environments, several contextual factors became evident and had a high impact on the usability issues. Despite positive experiences with the system’s technical capabilities and potential, the circumstantial issues became predominant. These issues are as follows: First, the timing of the telecare call was essential in order for the participants to receive remote care as part of their everyday life activities. Second, adding services and additional equipment to the familiar television platform increased the complexity of its use for the participants. Third, privacy and trust were central concerns among the participants included in our study –these concerns should be supported in design of home telecare systems. Finally, home telecare is not for every elder home dweller, and merely for selected services that are experienced appropriate delivered as remote care. Thus, home telecare such as video conference requires a certain degree of functional capabilities in order to be experienced as appropriate and useful for the elderly participants.