Abstract
Assistive technologies, such as telecare monitoring applications installed in the home, are being promoted to help reduce pressure on health care systems caused by an aging population and as such promise a large market for new products. However, despite many projects undertaken by commercial companies, and despite significant investments both by the companies and by national and international funding programs in the EU, such systems are not widespread. This paper reports on a retrospective study of the development of one early system, HandyHelper. We were interested in what challenges the development team faced and why the system is no longer on the market. Qualitative research methods were applied, including document analysis and interviews of key people involved in its development. Even though the system worked technically, the input of older users was sought, and a pilot installation was run, the development was stopped. The findings from a thematic analysis point to complex issues. Some problems were avoidable, e.g., by providing more support for new users, though other problems point to inherent tensions, such as the different needs of sensor-based security features and interactive services aimed at the older users. Yet other aspects are outside of the developer’s control, such as available public funding. We summarize these findings and suggest lessons learnt for future projects.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
In the media, there are repeatedly reports about the aging population in Europe. Due to demographic changes, including a decrease in the number of children being born and an increase in life expectancy, there will be fewer young people to support and care for the older people. This was first brought to the awareness of many people in 2006, when the European Union (EU) issued a policy document highlighting the longer-term problems [11]. In an effort to deal with these challenges, Ambient Assisted Living (AAL) came into being: “the use of technology to: extend the time people can live in their preferred environment by increasing their autonomy, self-confidence and mobility; support maintaining health and functional capability of the elderly individuals, promote a better and healthier lifestyle for individuals at risk; enhance the security, to prevent social isolation and to support maintaining the multifunctional network around the individual; support carers, families and care organizations; to increase the efficiency and productivity of used resources in the aging societies.” [1]. Thus, AAL technologies, now referred to as Active and Assisted Living, promise benefits for countries that financially support the care of older people, but also aim to increase quality of life by allowing people to live at home longer. Given the demographic changes, it also promises a large commercial market for new products.
This policy document encouraged some companies to develop products in this area. To further stimulate investment, funding programs supported the development of this type of systems, including national programs such as the Austrian bmvitFootnote 1 programme “benefit”, the German bmbfFootnote 2 focus on “Altersgerechte Assistenzsysteme” and the British “Preventative Technology” Grant for telecare services, as well as the European-wide Ambient Assisted Living Joint Program (AAL-JP). More than 600 million EUR was invested by the EU AAL-JP alone in order “to support projects developing ICT solutions for aging well with a 2–3 years to market time horizon” [12, p. 7].
Despite the number of projects completed and the money and effort invested, there are few systems on the market, as evidenced by the fact that funding schemes continue to encourage development in this area. For example in 2014, new programs were launched, including “The Long Term Care Revolution National Challenge” (part of innovate UK) and a new phase of the AAL-JP which specifically aims to get more systems to market. The report evaluating 5 years of the EU AAL-JP gives some indication of the situation: It was concluded that “there is a need for greater weight on issues such as integration, scalability, and overcoming barriers to market entry rather than technology development per se.” [12, p. 16]. The report also found “a majority of projects integrate users in some form, most commonly in the requirements and testing phases” [12, p. 10]. Thus, since user-centered methods that work more generally for software development fail to reach the goals in this area, it is worthwhile to study the development of this type of system to understand what happens in detail during the development of systems that aim to go to market, but fail.
There are different aspects that can be investigated to understand the problems. Some researchers have looked at what aspects of these systems are important to users [18] and barriers to their adoption [5]; others have taken a more technical perspective [10]. In other application areas, development processes have been studied more generally to gain an understanding about decision-making during software development projects [16]. Looking at the development process of individual systems could give detailed information to help understand the issues faced by developers. However, despite the challenges and lack of successful systems, few studies have investigated the development of projects in the area of AAL to understand the problems that arise.
Thus, we chose to study the development lifecycle of one specific system entitled HandyHelper, an early project addressing some of the aims of AAL developed in one EU country and in part funded by the national AAL program starting just one year after the EU policy document that made the general public aware of the problems. HandyHelper is interesting because it is a system that worked technically and made it to market, though is no longer sold. Furthermore, at least superficially, it meets the standards of development of this type of project at the time, as it included users in both the requirements and testing phases, as described in the evaluation report of the AAL-JP mentioned previously. We conducted a retrospective analysis of the development to see what worked and what did not, to identify the main issues and themes, and to analyze how these related to each other and to the final outcomes of the project. Topics that emerged included how “users” were considered and involved, but also technical challenges and aspects related to including both services and monitoring features in a single system.
This paper first describes the background and research methods. After that, the project is presented, before discussing what happened. In order to maintain anonymity, the country and company where the study was carried out are not identified, the details of the system are described only sketchily, published articles that could identify the project are not included in the references, and the names of all companies, products, funded projects and persons have been changed.
2 Background
Assistive technologies (AT) have been promoted as the solution to the increasing burden of care for an aging population in many countries. AT aims to assist people with special needs. In European countries, AAL is used to describe AT specifically aimed at needs relating to aging. There is a range of AT systems for older people, such as: systems to support independent living at home, e.g., reminding people to take medication; telehealth systems enabling contact to healthcare professionals, e.g., monitoring blood pressure; and telecare monitoring systems that monitor activities of daily living (ADLs) and react to detected exceptions to ADL routines, e.g., fall detection. These systems can be targeted at older people living at home or those living in care facilities.
Even though various funding programs, such as those mentioned in the introduction, exist to encourage development of these technologies, these systems are still not yet widespread. In order to understand why, some studies have investigated the systems themselves and their adoption [5]. Some barriers to user adoption of, or withdrawal from, telehealth and telecare services that have been identified include the level of technical competence required, threat to independence and identity, and fear of disruption of current health care services [28]. In telecare monitoring systems, the threat is particularly great as there is a tendency to add functions not originally planned that give more control to carers and may reduce the independence of people using these technologies [9], particularly if there is no possibility to adapt the system [23]. This is concerning, as impersonal monitoring systems support independent living by increasing security and may be attractive to people not otherwise open to care [9]. This indicates that it is worthwhile to study the development process where the functionality is determined and designed.
One characteristic that can make the development of AAL systems challenging is that the systems are targeted at older people. The diversity of older people provides challenges for developers [21]. It may be difficult for younger developers to understand less technical older users. There are also widespread preconceptions of older people [31] that can inadvertently affect the design and choice of functions. To avoid such misunderstandings, and increase the benefits and acceptance of the system, a number of projects have tried to use a participatory approach and include older users directly [21]. Besides participatory design, there are also other user-centered design (UCD) approaches such as personas, where “actual users do not play a big role during the design phase” [4]. These are important as time constraints can make it hard for developers in companies to include users [20]. However, just applying a UCD method may not be sufficient to ensure systems meet end user needs [16]. This suggests it could be useful to study the development processes of an AAL system to see what actually happens in more detail in relation to user needs.
Outside of AT, there are studies of what happens in design meetings of interactive systems to gain insight, for example into how and when artifacts and user data are introduced [25], and how certain methods are used [4]. We hypothesize that looking at the development processes might especially be revealing for this type of AT project, for a number of reasons such as the challenges of developing for older people noted above, the use of sensors, e.g., light or movement sensors, which change the role of the user by virtue of their passive sensing capabilities, but also due to other characteristics of developing these systems. This could help identify issues for further study about what factors in the development process contribute to the low success in getting to market and identify points that can help in future projects.
3 Methods
The approach for this research was a retrospective ethnographically informed study using a mix of established qualitative methods. Since the project has been completed, documents from the time of the project were most important. Data collection involved finding documents related to the company and the project, including meeting minutes, description of the technical implementation of certain parts, scientific evaluations of the project, conference articles written by people involved in the development, newspaper articles, promotional materials and financial reports for the company. In addition, screen shots and functionality of different versions of the emerging system were examined. Some documents that were not specific to the project, but which mention this system, were also studied, i.e., reviews of AAL systems, trade magazine articles about AAL and brochures from carer organizations. In all, more than 60 documents, including two formative evaluation reports with 20 and 120 pages, respectively, were studied.
To complement this documentary archive, semi-structured interviews were held with people involved in the various phases of the development of the HandyHelper. Five people were selected to reflect the different perspectives during the development. All people agreed to be interviewed. This included the manager responsible for the development, a technical project leader, a service developer who worked for a different company, a care worker from the assisted living facility who was involved in the pilot and a researcher involved in the first evaluation (see Table 2). Interviews ranged from 1 to 2-1/2 h, except with the researcher, which was more informal. Audio recordings were made of the interviews, and signed consent forms were obtained from the people interviewed. Interviews were supplemented with e-mails with some interviewees to follow-up on specific issues.
The available data were then analyzed. Based on the information available, the authors reconstructed what happened as a chronological account, which could in part be validated in the interviews. The data were then analyzed using the qualitative method thematic analysis [6]. Where the notes taken during interviews were not clear or complete, these were supplemented using the audio recordings, which were also used to get verbatim quotes relating to topics that were of particular interest. These notes and the documents were coded first manually on paper. Then, the interviews and key documents were later re-coded manually in Atlas.TI ™ to get a more detailed view of codes and how they were related. In line with an iterative coding approach, analysis was started before data collection was complete: to identify which themes were most common at which point in the project, which themes were most common in which document and what further data collection was needed to fill in gaps.
In the following, words in quotations are taken verbatim from an interview or document. Ellipses are used when a portion of the original has been left away.
4 Project
HandyHelper was a product developed in a European country with nationalized health care aimed at allowing older people to stay in their homes longer. The development extended over four years starting in 2007, a year after the EU position paper that initiated AAL. The development was undertaken by a company, called GellIT in this article, and involved a number of different phases, funding sources and partners. The development has now been concluded. In the following, the development is described during the various phases from the initial development to the final version (see overview in Table 1). It includes information about the way the different types of users were involved and considered in the project, but also the difficulties the team faced.
4.1 Initial development: prototype and show home
The project was initiated based on the EU report on aging, which showed the increasing costs for caring for the growing older population. On the advice of an external IT consultant who said that this was an area with promise, GellIT decided to branch out into this new business area and self-financed the initial development. They started with a large number of developers (over 20) to see what was technically possible. Users were not included, though the basic goal, which was to keep older people in their own homes longer, was adopted from the EU report. To this purpose, they developed a sensor-based system that included a number of security features, such as turning off the stove. In addition, some services, such as to remind people to take their medicine, were added (as represented in Fig. 1). Interaction with the system, for example to configure it or use the services, was via the TV using a remote control for user input. In just a couple of months, a good prototype was up and running and was installed and tested in the home of the project manager. Once it was clear the system worked, a new business area was officially created in the company, and the consultant (Gabriel) was hired as the manager to lead it.
HandyHelper included both security features and other services that varied by phase
A show home was then setup, where the public could come and see the system working. It was estimated that close to 1000 people visited, which the company interpreted as demonstrating the potential of the system. In retrospect, however, the project leader recalled how difficult it was to explain what the system really did to the older people visiting the show home: “I remember when I started with the topic, it was hard for me to grasp it… If I involve someone and they explain what it is about, they can’t articulate it, because they don’t understand the topic. How can someone develop it who can’t articulate what it is about?”
The next step was to further investigate the needs of older people, and to this end a local university was invited to conduct an evaluation study in the show home. The costs of the evaluation were covered by a small grant from university funds. The report (unpublished) was based on input from approximately 100 people aged from 55 to 90 and included both qualitative and quantitative results. The report showed the great diversity of the potential users, for example: living situation (whether they lived alone or with family), level of computer use, the range of physical and cognitive limitations they experienced. The findings provided mixed feedback about the system. For example, while people reported preferring the TV remote as the interface to control the system, many also found the remote difficult to use. There were also mixed opinions about the value of the system. Some of the older people (over 80) felt it was too late for them: “It doesn’t pay off 5 min before dying!”, while some of the younger ones (under 60) thought it was too early for them. Though the security features were rated as most important, some of the “convenience” services, such as an easy way to return missed calls, were also important to more than half of the people. The report also included suggestions for further services. One feature suggested was to shop for and order groceries. Participants were also asked about costs and almost 40 % were willing to pay at most 50 EUR a month. In fact, concerns about the costs were mentioned most often, followed by usability.
Just one and a half years after starting the development, a version was available for sale. However, only a single installation was purchased—for a person living in an assisted living facility. It was purchased by the local social services to understand the potential of the system. The evaluation was positive and in a TV program featuring the system, this first older user said: “It is unimaginable to be without a computer, Google and search possibilities. And now this is a sensible next step. The system is so simple to use that with the user handbook after a short introduction everyone can use all the features.”
4.2 Intelli project: adding intelligent monitoring
Around the same time, the GellIT experienced some financial difficulties in their established business due to the general market situation. Following on the success and publicity from the show home, they secured a national grant for the development of AAL in collaboration with an academic research organization to add intelligence to the sensor data and adding movement sensors to enhance security, e.g., to check if there is human activity in the flat. In this article, this is referred to as the Intelli project. The project focus was based on the report from the university evaluation carried out in the show home after the initial development, which had shown there was great interest in security features and indicated a need because less than 25 % of people interested in the system lived in assisted living facilities where someone would check on them regularly. The grant enabled the company to access valuable skills via its external partners, for example on how to find patterns in the sensor data, and also to halve the internal development team to 10 people.
The goal of the Intelli project was to collect data from various sensors and system components to see if there was any activity, including movement sensors and sensors on light switches and the refrigerator. If there was no activity within some designated time period, an alarm was to be generated automatically. This was combined with a green button that allowed the person to send an “all clear” before an alarm went out. The challenges at this point were more of a technical nature, such as generating profiles from the sensor data, than trying to understand user needs.
In a publication about the project, the focus is clearly on assisted living facilities, rather than on private homes as in the previous stage. One project member said this change was due to the complexity of installing the system and effort maintaining it, which would be higher since individual homes have a greater diversity and distance between them. To ensure the new intelligence features worked reliably, long-term tests were conducted by installing the system in the home of an older family member of someone in the team. These tests lasted several months and started during the development. The intelligent monitoring was also included in two units of the pilot described below.
4.3 PAAL project: adding services and pilot study
In parallel to the Intelli project, GellIT led a consortium of nine other partners and secured another national research grant (for what is here called the PAAL project) for the development of AAL in order to integrate additional services and to pilot the system in a new assisted living facility being built by one municipality, which provided additional financial support. The PAAL project ran for 12 months during which time the system was to be provided at no cost for the residents of the 25 units.
An overview of the PAAL consortium is shown in Fig. 2. One partner was responsible for carrying out a market analysis to find out which services the future residents needed and wanted. Several of the project partners were to develop services that would be available during the pilot—those services that were deemed useful from the pilot would then be included in a final version of the system. Other partners were responsible for installing the required infrastructure. Yet others were organizations responsible for caring for, managing housing for, and answering emergency calls from the older people using the system. The final evaluation was to be done by a university partner and included specialists in social policy. One service developer mentioned that the PAAL project had good project leadership, and thought that was probably because it was managed by a company rather than an academic partner.
Consortium for PAAL project
The market analysis was conducted with older people and relatives, including approximately one-third of the future residents of the new housing facility, and highlighted again the truly diverse needs of older people using this type of system. For example, when asked about the grocery shopping idea that was suggested in the show home during the initial development, the market analysis cites that their interest ranged from “I think I will use that” and “Good to have such an offer in case it is needed in the future “to “We don`t need that”. Indeed, the report noted that many of the functions explored in the analysis were deemed by the future residents as “not needed yet”, although they liked knowing they were there for later. Topics where the analysis indicated more agreement were the importance of security and social contact, as well as a concern for the costs: “people are afraid of high additional costs”. GellIT provided information from their analysis to the partners, though stipulated that they were not allowed to talk to the older people in advance (to be discussed later)—something one service developer found frustrating.
In retrospect, the team thought the most valuable input came from a group of experts including carer organizations, computer scientists and politicians. Although carers generally were not very interested in technology, the minutes of meetings show that the partners that were care-service providers saw the benefits of this type of system and helped explain the needs of the diverse community of older people that they supported. A few functions were also developed specifically for the carers, such as one to help plan outings by allowing the residents to sign up using the system.
Although having the pilot at a new facility seemed ideal, in practice it brought new problems that increased the overall effort. Initially, there were some technical problems primarily related to the infrastructure and its instability. Also a large number of the residents moved in at the same time and were faced not only with this system, but also with a household full of unfamiliar devices. As a result, the introduction of the HandyHelper was delayed, shortening the pilot to 6 months from the planned 8 months. One service developer felt carers could have done more to get the older people start using it after the training session. However, due to the delay starting the pilot, the carer was unfortunately on vacation for the first couple of weeks, and her substitute was not familiar with the system. Another project member mentioned families also did not help and in some cases even discouraged the older people from asking them for help by saying “if you don’t understand this, you really are old”. The technician who came to the assisted living facility at least once a week during the pilot for the PAAL project was often called upon to help people in other ways, e.g., changing light bulbs. The support costs mounted.
In the end, there were a number of usability issues. The remote control interface needed for the services proved quite problematic and meant additional steps just to turn on the TV. Few of the services, even those that were suggested in the evaluation at the show home and evaluated by the market analysis at the start of the project, were actually used. One of the complaints made by many residents was that an LED light was on all of the time. Surprisingly, some people were just disturbed by the light at night, some worried about the electricity usage and even unplugged it, and others commented that it reminded them of the sensors “watching” them, aspects that related to the security features of the system. The technician then put tape over the LED, which reportedly “solved” the problem.
A qualitative scientific evaluation was also conducted by the one academic project partner to evaluate the changing living situation of the residents after moving to the assisted living facility with AAL. Interviews were conducted before the people moved in and were repeated four months afterward. The evaluation included almost half of the residents, who differed in: age (60–85), living situation (whether they lived alone or as a couple), physical and cognitive disabilities, financial means available and how comfortable they felt with the idea of the AAL system. The findings from the evaluation noted that the views toward the system changed after people moved in. Although some people rejected it from the outset, most did not show any initiative in trying the system and learning to use it. Others mentioned that it was complicated to use. One important point was that the evaluators thought that better support would have been needed, something people at GellIT later agreed with. The report also suggested that the future residents should have been included more in the development, so that the resulting system would be more suited to their cognitive abilities, but also to increase their motivation to use it. The evaluation also showed that, although the security features were valued, the personal advantage of many features was not clear to them. As an example, the feature for grocery shopping was not used, as the new residents enjoyed walking the short distance to the shop, where they might run into someone they knew.
A further review was recommended again after 12 months. However by then, the PAAL project had run a year and the free trial period for the residents was over. The residents were not willing to pay the price, which was just slightly lower than the (subsidized) standard carer fee in the facility, for features that were not yet used. Thus, the control consoles were removed from all units. Referring to 80-year-olds, one team member said “they were not used to using a menu on the TV… even this simple extension was so new to them, they didn’t grasp it”. Several interviewees mentioned that people were overwhelmed with the number of services in the pilot. However, some security features that ran with hardware alone, such as sensors to check if something on the stove overheats, were left in place.
4.4 Final version: Thornhill care home
During the pilot, GellIT also won a paid commercial contract to install the system in all units of another new assisted living facility in a different location (called Thornhill in this article). Based on experiences at the start of the pilot for the PAAL project, though not using the 4-month pilot evaluation results, the system was completely redesigned (decentralized) to be more robust with respect to instability of the infrastructure, but also to allow remote maintenance, something not considered by the academic partners in the Intelli project. Having personally been on-site regularly, the technical project leader felt what needed to be done was clear. The pilot demonstrated that changes were needed both in the technical side to make the system more stable and to improve the user interface. For this stage, the team was reduced to a core team of two developers only, plus the electrical contractors responsible for installing the infrastructure and components during the construction of the facility.
The situation in the new facility was different. The carers would not have regular contact with residents, and so some services, e.g., for planning outings, were removed. Other services, such as calling the elevator, were added. In addition, GellIT switched from the TV remote to a tablet interface to control the system.
A sociologist who worked at Thornhill followed the introduction of the system to evaluate the acceptance. Initially, there were again technical problems with the infrastructure as well as some usability problems, despite the new design.
Shortly afterward, the development of the HandyHelper was stopped. According to a technical project leader, this related to the long lead times with new care homes and the fact that many contractors did not have the necessary know-how. It took well over a year and many meetings from the time a contract was sold until completion. Furthermore, HandyHelper required a reliable communication network—in practice, problems were experienced in all locations that were time-intensive to diagnose and fix. At the price older users were willing to pay, the company calculated it would require a very large number of installations to cover the costs for the company—though they also had troubles finding reliable contractors to cover the geographical spread. Instead, the company expanded into technology aimed at younger user groups.
4.5 The situation now
Now, a couple of years later, the HandyHelper system that was installed in Thornhill continues to be maintained. Both security features, such as the intelligence of whether someone is active, and other services, e.g., calling the elevator, are still used by the residents. One team member from GellIT felt usability is often overestimated: the system in Thornhill is actively used, even though there were lots of complaints from the older users at the beginning about the usability.
Interestingly, reflecting on the project, all of the interviewees had different views with regard to the source of failure. Gabriel, the project leader, said “it is not age related … it is more a matter of character … with technological innovations … you always have 15–20 % that refuse to use it”. He saw so many older people who were excited about the technology. In the end, it just took too long before something was sold. He felt the security features were key—other features/services could be added later. The carer who worked at the home during the pilot for the PAAL project thought that the system provided interesting services and that it was a shame the system was no longer available. However, she attributed many of the problems related to the age of the users: “This generation is pretty far away from computers and electronics”. She also mentioned that they tended to be frugal. Dan, the evaluator, felt a lot of public money was invested into the system, and some problems could have been prevented, as aspects like the usability had been mentioned in the evaluation at the show home. Carl, the technical project leader, mentioned problems with academic partners, who did not respect the needs of having the system work long term.
Looking back, many interviewees agreed that funding programs are valuable if governments want companies to invest in new areas such as AAL. Gabriel, the project manager, saw real advantages: the administration effort was relatively low and he found good partners through the grant consortia. Bill, a service developer, felt the grants helped, though he also felt that funding criteria, such as giving precedence to certain countries or requiring a large number of partners, were problematic and did not necessarily lead to the best partners being included. Furthermore, Bill felt it could complicate development if one partner was responsible for collecting the requirements and others then do not have contact with users.
Even after the development of HandyHelper was stopped, the system continued to have an impact. In parallel to the redevelopment for Thornhill, the version of the HandyHelper system developed in the PAAL project was installed in an assisted living unit in another location to evaluate the system for wider use. By the time this test ended, the system was no longer available for sale, and so the facility copied it. Even after the development was halted, social ministers from another state visited the Thornhill facility to see the system, as it was seen as a role model. Bill, a service developer, felt even at the time of the interview that HandyHelper was one of the top two systems of its type. He mentioned recently trying to find if there was another system like it, however, could not find much, even though there is a real need.
4.6 Putting HandyHelper in context
We were curious to see how the experiences during the development of HandyHelper compared to those of other projects during the same period of time (reflecting similar state of the art of both AAL understanding and technology components). For this purpose, we reviewed the list of projects from the first AAL-JP which ran from 2008 and 2013 [2]. AAL-JP projects were chosen, because these projects have a certain standard, since they were judged good enough to receive EU funding, and because the funding requires certain deliverables that provide access to information about all projects of interest. Furthermore, the AAL-JP provides access to information about development done in other countries than the HandyHelper. All AAL-JP projects that had similar attributes to HandyHelper were studied. Based on comments of interviewees about companies versus academic organizations, the list was reduced to projects where the consortium was headed by a company. Since research focused on a monitoring project based on sensors and some aspects discussed related to these sensors, only projects including sensors for monitoring were considered. In order to study the whole development including evaluation methods, it was necessary that the project was completed at the time the analysis was carried out.
In the time frame 2008-2013, three completed projects funded by the AAL-JP that fit the criteria [2] were identified, referred to in the following only as project 1, 2 and 3. All lasted 24 months, so similar to the time for the Intelli and PAAL projects of HandyHelper, though shorter if the initial development and extended show home period of HandyHelper are taken into account. They included between 7 and 12 partners, so comparable to GellIT and the nine partners who worked on the PAAL project. Of the AAL-JP projects analyzed, only one, project 3, is commercially available, though it has changed significantly since that time.
The documents for the three projects that fit all the criteria were studied. This included the deliverables for AAL-JP grant, supplemented with web pages and scientific publications where these could be found, and as with HandyHelper represents the development perspective. With the exception of one project, relatively little documentation could be found, so that a thematic analysis would have been incomplete. Thus, the analysis focused on those details that could be reconstructed: features that were included, devices used for user interface, which user groups were included during the development, how users were involved, what phases they were involved in, user support and the length of the pilot.
The projects share many similarities with HandyHelper, both in terms of features included and methods applied during the development, as shown in Table 3. One project made similar choices in terms of using a TV and remote control. Methods applied included talking to users in a show home, workshops with experts, a pilot with pre- and post-interviews, methods which were also used during the HandyHelper development. HandyHelper had a more extensive user analysis and evaluation than some of the other projects. The only project reporting the amount of support during the pilot provided less frequent support than HandyHelper, once every 2 weeks rather than weekly. The evaluation methods of the pilot done for HandyHelper most closely matched those of project 3, which is now commercially available. Only one project mentioned problems encountered—as with HandyHelper, there were also some technical problems. Problems reported included: speaker interference, range of wireless components, battery life, need for a redundant internet connection and problems with power supply interruption.
Furthermore, this review of other projects demonstrated how difficult it can be to get detailed information about completed projects even if deliverables are officially public, which underlines the value of studying the development of the HandyHelper.
5 Reflecting on the development process
HandyHelper aimed to put a system on the market. Although the system was sold, it is no longer available on the market. We set out to see what worked and what did not, to identify the main issues and themes, and to analyze how these related to each other and to the final outcomes of the HandyHelper, an early monitoring system for older people.
As a first step, the themes from the retrospective interviews were identified. Top among these was the innovation factor. Still a major theme, though mentioned much less frequently, was the expected issue of having older users. The same themes were also prevalent in the documents related to the development of the HandyHelper studied.
For the analysis, the themes were then grouped by the effects they had on the development (see Table 4): the methods applied and the effort to get it working. These are described in more detail in the following, before the problems are discussed.
5.1 The methods applied
When developing an innovative technology, such as this type of monitoring system was when HandyHelper was started, trying to find out what features are needed is difficult. Usability is also often an issue, particularly when dealing with an older and very diverse user group such as the one referred to in this paper. What methods were used? Do these stand up to “standards”? What problems were encountered? What worked?
If UCD is considered “the active involvement of users for a clear understanding of user and task requirements, iterative design and evaluation, and a multi-disciplinary approach” [22], then the project applied accepted user-centered design (UCD) methods. Input was received before deployment through methods such as the show home, meetings with carer organizations and a market analysis with older people. In addition, multiple iterations were carried out, each with some type of user evaluation afterward. Still, success was limited, in particular as relates to usability, in terms of ease of use.
Although the requirements for the first prototype did not come directly from users, they were based on the user needs described by the EU position paper. Input for subsequent development came from older users through the show home and market analysis or from professional carers who have seen the needs of people in assisted living. An iterative process allowed for suggestions to be implemented. Even looking back, the interviewees involved saw few faults with the methods applied; in fact, Carl, Bill and Anna specifically said the methods were good. Indeed, suggestions of the users, such as the grocery shopping service, were actually listened to and integrated into a later iteration. As pointed out by Gabriel, basing the first prototype on EU documents allowed them to develop a working system quickly, so they could get user feedback early on.
If this is compared to project 2 financed by the AAL-JP described previously, project 2 included more people during user needs analysis. The user needs analysis of this other project included a far larger number of subjects (300 vs 100), most of these being patients. However, in that project the subjects only completed a questionnaire and did not have access to a system and so may have had a limited understanding. The interviews in the HandyHelper show home allowed more meaningful input, which is especially important, when we consider Gabriel’s comment about the difficulty of explaining this type of system even in a show home, where features can be demonstrated. Interestingly, the successful feature, calling the lift, was not from evaluation, rather was thought up by the technical project leader who was at the pilot site regularly.
The evaluation methods used during the development of HandyHelper stand up to standards recommended in the literature. Compared to standards set by review articles [3], the evaluation methods were unusually good: The initial study in the show home included more than 100 people, and both qualitative and quantitative methods were used. Furthermore, the evaluation was carried out early enough for the results to be used for further development. The evaluator, Dan, criticized that although feedback related to functionality was considered, comments related to usability and pricing were not given enough consideration in the next development phase. Indeed, these aspects were identified again in the market analysis of the PAAL project later, and in the evaluation report following the pilot.
The pilot length was comparable to that of AAL-JP project 3 described previously, which had pilots lasting anywhere between 0.5 and 8 months versus 6 months during the PAAL project of HandyHelper. However, HandyHelper included many more users. Thus, it is comparable or better than the three AAL-JP projects studied. It can certainly be faulted that the evaluation report was not used for the redesign, even if the system was in the end adopted by the users. The focus of the redesign was technical, and the final version again had some usability problems.
The project structure limited the ability of some partners to apply UCD during the funded PAAL project. The grants favor projects with a large number of partners. With lots of partners, it was not desirable or practical for each to talk to the future residents individually, so, like Bill, most partners received the analysis results collected by another partner and did not have direct contact with the future residents to really understand their needs. At the few meetings, they did have contact with other partners, the care services providers had a lot of influence when it came to deciding what was needed. Documents show that the functionality was set at these meetings with experts.
5.2 The hidden effort to get it working
In the end, a surprising amount of time and effort was needed to get the system to work. Even though a first version of the system was up and running quickly, it took a lot of effort to test the system long term and then get it working with realistic infrastructure constraints, so much, that in the end, the company gave up.
Although the extended pilot for the PAAL project with a large number of users over 6 months seemed long, it proved insufficient for a system with many features that needed to address a range of needs. The market analysis and the evaluation report indicated some of these features might be needed later, though they were not needed at the time the users had to decide. The residents had just moved in, they enjoyed getting their groceries and the social contact it gave them and so did not use the grocery shopping service suggested during the initial evaluation at the show home. In fact, others have shown that usage of this type of system changes over time [30]. Even with people aging, it could be years, however, before people need those functions they mentioned in the market analysis as wanting in the future.
There was also an issue of support in everyday use. During the pilot, the residents needed support on-site on a regular basis to get started. The non-technical carers were not particularly helpful in answering the questions users had—the problem was compounded by Anna’s absence the first weeks of the pilot. This puts a burden on the technical support of the company not common with other products.
Families also provided little support and in some cases even discouraged use. It was reported that family members were not involved with the system, and that in some cases actually discouraged enquiries by indicating how old the people were if they had troubles using it. Thus, the older people were afraid to ask their family members for help. The lack of family support again had an impact on the company—people turned for help to the company technician instead, also regarding aspects unrelated to the system, which further increased costs.
6 Challenges encountered
Although HandyHelper worked technically, it was not really a success. The system did successfully address some goals of AAL, such as enhancing security, and may support people staying in assisted living longer before moving to a higher level of care, thus saving resources overall. However, it did not fulfill its original goal of keeping people in their own homes longer. Although it aimed to go to market and was even briefly available, it was discontinued because it was not successful commercially.
Looking at the project, the situation is very complex. There were aspects that could be criticized or improved, though no obvious mistake. Perhaps surprisingly, despite the challenges mentioned about working with older people, for example their diversity and lack of computer experience, this was not one of the main problems. The complex technologies also presented challenges; however, these could in the end be managed. Instead, underlying tensions help explain the problems. These also present potential pitfalls for other projects of this type:
-
1.
The way the meaning of “the user” subtly changed as the target group changed from private homes to care homes.
-
2.
The difficulty of getting meaningful input from all user groups: on one hand, the diverse group of older users and on the other, the carers, who have very different experiences and goals.
-
3.
The balance between usability and technical reliability, because the system integrated both services and security features—and this, while the need to make a profit was always present.
-
4.
The trade-offs with funding with an innovative project like this, on the one hand providing valuable know-how and on the other requiring more overhead and coordination.
-
5.
How long to evaluate a system, long enough to get substantial input, though not so long that the benefits of the monitoring functions are forgotten.
These aspects are discussed in the following, to provide more understanding about what happened.
6.1 The shifting shape of “the users”
The project sets out to support older people living independently, as illustrated by the people included in the show home evaluation; however, the target group was changed due to initial difficulties. For one, Gabriel mentioned how difficult it was talking with older people about the system to get their input. For another, the poor sales initially came at a time when the company was by chance experiencing a financial downturn in their core business. This led to GellIT focusing on assisted living facilities such as the one that purchased the first system and then found it easier to get input from professional carers. The strategic shift changed the notion of who the “user” was and the situation they were in, which in turn had implications for the development. Thinking of them simply as “users” or “older people,” hid some of the subtleties that were relevant to the choice of functionality.
Other systems may aim at both user groups. This raises some questions: Who are the users in this type of project: Are they people living at home alone or residents in a care facility or are they professional carers or concerned relatives? What does it mean with regard to the functions chosen?
At the time of the show home, the word “user” referred to an older person living in their own home. Their goal was to get additional support in this environment in order to allow them to live at home longer.
The shift to assisted living facilities also meant the goals shifted. The “user” needs were gathered in meetings with carer organizations, rather than asking the future residents directly. These carers could provide input about the diverse needs of older people, including those with disabilities who did not take part in the evaluation at the show home. Still, this was the view of people who were not older themselves and who would be using the system in a different way. The benefits were now framed as being about reduced care effort. This resulted in some features that, as mentioned by Anna, were about simplifying things for her, for example organizing transport to medical appointments and planning field trips. At the same time, the facilities with carers were now instrumental to the sales, and so their needs counted more from a commercial perspective. This is similar to what is reported elsewhere, where the needs of the family carers, who wanted the system, moved into the foreground [19].
At the same time, “resident older users” were interested in different features. For example, the security features much prized at home became less important in the safer assisted living environment where a carer was on-site every day. Similarly, the grocery shopping service was developed in the PAAL project based on the feedback of older people at the show home—even though the market analysis indicated that many of the future residents of the care home did not need, or yet see the need, for such a service. In the final evaluation of the pilot of the PAAL project, one person said they would only use these services when they really could not do things independently. This may relate to not wanting to lose their physical abilities, but also to wanting to preserve their independence.
More generally, the language of documents such as the 2006 EU policy document [11] makes use of terms like “people” and “elderly individuals”; this creates an impression that the “older people” that assistive technologies seek to address are a homogeneous group. In any case, the problem of the diversity of older people, mentioned in the background discussion and also seen in the pilot evaluation, continued to play out in these projects. “Older people” are a hugely diverse group, who may share chronological age, but who will have very diverse cognitive and physical health statuses and life situations. Not only do these differ greatly from one person to another, but they can also change and evolve over time. For example, some older people have more physical limitations and are grateful they can turn the lights on more easily, others suffer from cognitive limitations and want help with their memory, while yet others suffer more from social isolation. Providing for the needs of all individuals with the one solution is an unrealistic goal and even if achievable would be expensive. The approach that the system studied for this paper chose to provide a little support for a wider user group rather than more help for a smaller subgroup of users could be criticized as not really meeting anyone’s needs.
6.2 The difficulty of getting input
Given that we have the two user groups, the older users and the carers, there is always an issue of how to balance their influence. The methods used with the different user groups were also different—older users were consulted in the show home and during the market analysis, whereas carers were involved in meetings directly relating to the pilot. Were the people who decided on the functionality really like the older people who would be using it? Which methods provided reliable input?
6.2.1 Show home
The show home generated a lot of ideas about useful features. Although some functions suggested were included, some of their feedback, such as usability and costs identified in the show home, were not acted on and were also an issue with the final project.
Although the show home was designed to get input from a broad group of people, the people who participated did not represent a cross section of the entire target population. In practice, the show home probably attracted more technophiles, as indicated by 41 % of the men in the evaluation wanting to definitely have the system. In addition, merely getting to the show home required a certain level of mobility. The evaluation report specifically stated that there were relatively few people with physical impairments.
The show home also illustrated the difference between what people say and what they actually want or do and highlights the ways in which life context matters for this as well. For example, based on recommendations from the evaluation in the show home, the project implemented the grocery shopping service and the simplified TV remote interface. In practice, the grocery shopping service was not used and some older users actually preferred the more complicated TV remote. In the end, the tablet interface in Thornhill proved to be the best option. This seems to support Bill’s view that evaluations are not always of value. Since a pilot was conducted to see what people actually do, they were able to rectify some previous decisions based on the input from the show home.
Another aspect of this type of evaluation that proved problematic was that it encouraged people to come up with a wish list of functions. Though Gabriel valued the evaluation in the show home, it can be argued that this was based on their imagined rather than actual needs and resulted in a large number of services being added during the PAAL project. Even Bill saw the amount of functionality as an overkill. In the end, people did not use many of the services, including the service for communicating Bill developed and the much praised grocery shopping service. This is critical, as others have found that having technologies that fit the person’s needs is particularly important for aging in place [26]. The resulting function-creep actually made the system harder to use and thus less desirable.
6.2.2 Other ways of getting input
Having a person already living in a facility evaluates a system over a longer time, seems it could overcome the problems of having new residents. However, the person testing the first purchased version knew Google and seemed unusually experienced with technology. Still, this review led the municipality to invest in the pilot project. In reality, however, he was not representative of the more typical cross section of people in assisted living facilities—at that time, only just over 40 % of people from 55 to 74 were using PCs in the country where HandyHelper was developed [32]. In the pilot study, most of the residents involved in the pilot were between 70 and 80 years old, and many were not computer users.
It proved hard to get input from the older users. A system like this is based on sensors. This means no direct interaction is required from the inhabitants being sensed. Furthermore, the sensors are often embedded into the home in unobtrusive ways. In this sense, we can say they are “intangible”. Such intangibility provides additional difficulties when working with older users. While in the literature, participatory design is promoted and user design the ideal [29], here the team found it difficult to discuss the system with users, even though Dan, the evaluator, was experienced working with older people. Although other projects have successfully used a participatory approach with older users, in their reports they also specifically mention the difficulties of working with intangible concepts [21]. The situation was exacerbated with HandyHelper, which was not only largely intangible but also innovative, so there was little to compare it to. Gabriel, the manager, felt it would have taken much longer to involve users earlier, which was unacceptable when the company was having financial problems. This is in accord with findings of other researchers, who have also found that working with users is, in practice, difficult due to economic pressures [20].
Apart from communicating about technology, it may be hard for some older people to anticipate and/or articulate their needs. Some comments in the evaluation report from the show home, e.g., the comment mentioned earlier about not being worthwhile getting the system “5 min before dying”, sound almost despairing: that they do not know—or maybe did not really want to think about—what might be coming. Even when people find themselves suddenly having difficulties, they may not really know what help they need.
Professional carers were easier to work with; however, even the carers were susceptible to stereotypes. Anna emphasized the difficulties of technology and older people where, in reality, some people in the pilot did have a computer. Having seen the breadth of disabilities that can occur, the needs the carers described in meetings about requirements may have been closer to those in a nursing home rather than those of the new residents involved in the pilot. Having lived independently before their arrival, the new residents were keen to do as much as possible as long as they could. And so the features recommended by carers were not of interest to the people in the pilot. Still, unlike some older users, the professional carers seemed to be more generally positive about the advantages the technology can provide.
However, some of the services suggested by the carers in the facility could be considered almost coercive by the user group for which the system was originally intended. For example, Anna described a function that would have allowed her to push a button and check whether a person was there, though due to other problems was never turned on. Since the LED reportedly made the users worry about surveillance, they surely would have worried about the idea of Anna remotely checking on them. This is not an isolated case—others also report that there is generally a tendency to add functions not originally planned in this type of AAL systems that give more control to carers and may actually reduce the independence of people using these technologies [9].
6.2.3 Pilot
The pilot served to give valuable input from the point of view of the older users on the final version; however, it had its limits. Even if accurate information can be obtained from carers about features, other aspects important to acceptance by the older people, such as the LED and user interface, have to be checked directly with users. But in practice, the differences between how care were organized from one assisted living facility to another affected which services made sense for the carers. The differences in the amount and type of care provided at individual homes promise to be at least as high.
6.3 The conflicting needs of security versus services
One of the major tensions related to the fact that both security features and services were included, for example security features such as a water stop or activity monitoring and services like the photo album or grocery shopping service. The security features required little interaction from the older users, but needed to be reliable, whereas the services needed to be usable. These tensions are illustrated in Fig. 3.
Tensions between parts of the system: what is key? Who uses it?
6.3.1 Security features
From the point of view of the initial target group, older people living independently, the security features were seen as the key benefit. In practice, during the pilot, some of the older people worried about surveillance, in part due to the sensors, while others forgot that the security features were there and hence also the benefits they provided. Adding “intelligence” increased the security, but at the same time increased the fears about privacy, even though no cameras were used. One person who had a version of the pilot without the “intelligence” was reportedly afraid to undress except in the bathroom. Others have also reported users being afraid of surveillance by strangers [18]. In a private home, installing the system would provide additional autonomy, though in the care facility the aspects of reduced privacy outweighed the benefits of the additional security provided by the motion sensors. Others have found that having some sort of monitoring is in conflict with maintaining autonomy and privacy, which are core values of older people and elicit strong feelings in many [27]. Since it relates to fundamental values, methods such as value-sensitive design may also help to consider these aspects during the design [15].
From a developer point of view, the security features involved hardware: sensors and infrastructure. Since these were the core features, the system was built around these features and focused on reliability. During requirements gathering, it was hard to explain what a system like this could do to the older people with less technical background. The intelligence made the project particularly innovative, because although motion monitoring is used very often, few projects actually use this information to raise an alarm [7] as was the case here. However, the algorithms developed for the Intelli project were complex and required assistance from external specialists. In the end, the infrastructure, required for the sensors and also raising alarms, was one of the major sources of problems.
These same aspects, infrastructure needs and the complexity of the algorithms, raise more fundamental questions when older people in their own homes are considered. If there were infrastructure problems in a new care home with a large budget, will the domestic infrastructure for people still living in their homes of many years be sufficient? And, can the algorithms provide an accurate assessment with sufficient notice for someone to respond in a timely way?
Furthermore, for the security functions, as described elsewhere [15], those watched are actually indirect stakeholders, in this case the older users who pay for the system. In future work, consideration could be given for if and how the older person might also want to engage with their own data, a challenge mentioned also elsewhere [13].
The security features also meant that someone had to respond to the alarms, entailing the inclusion of additional stakeholders each with their own additional requirements. These become the users who interact more directly with the system features—the families and carers answering alarms. However, while their needs and suggestions often took precedence, other work would suggest that more attention also needs to be paid to how to support these stakeholders in dealing with interpreting data or responding to alerts in a way that does not overburden them and add to their workload [24].
6.3.2 Services
On the other hand, for the older users, it is the services they interact with directly on a regular basis, rather than the more intangible sensor-based security features that monitor events in the background. Such interaction was unavoidable. Even turning on the TV required interacting with the system, and so if they had difficulties, they were faced with these again and again. However, for the carers and families who were interested in the security, these features were largely unimportant, hence their lack of interest helping the older users learns to use them. Age acted to further decrease motivation of the older people to learn the system—in the pilot evaluation, one person said he did not understand why they installed technology when older people have decreasing memory and difficulties with using technology. Research indicates additional training and more focus on the services requiring active engagement may have been able to increase the positive attitude of the older users toward the system [27].
For the developers, the services entailed an entirely different set of challenges than the security features. They were software based and required a high degree of usability. Furthermore, the services differed from one care facility to another and also required a high level of modularity. It was through the services that additional partners came in. Still, the services cannot be viewed independently of the security—as others have pointed out, if the system can have different configurations, whether for the different facilities or differing needs of individual users, it is harder to ensure the dependability of the security features [18].
The services also raise questions about the long-term sustainability. Most of the services, and even some of the security features developed for this project, are already available individually, e.g., in smartphones. As more older people begin to use the stand-alone technologies, some of these individual services offered and maybe even comprehensive solutions such as HandyHelper may lose favor.
6.4 The trade-offs with funding
Another large tension was the funding. Funding provided money which was essential, though added complexity due to new partners—both in the organization and by increasing the number of services in the system. The conditions of the grants required innovation to be added at each step and introduced additional deadlines. At the same time, the grants provided access to partners who were important for getting the system to work. Systems such as this are, however, also associated with long-term costs for the users, and some public funding models actually discourage investment in them. These are discussed in the following.
6.4.1 Using grants to fund projects
The grants used for funding had an effect on the development. For example, because the grant evaluation criteria included the number and quality of the partners, a lot of partners were included in the PAAL project. Some of these partners provided services that helped offer innovation—another criterion of the grant issuing body. Some aspects related to funding are shown in Fig. 4 and discussed in more detail below.
Aspects related to funding
Innovation is an important aspect for grants generally. Many of the new services added for the PAAL project were mentioned at the show home and were expected to make the system more attractive. At the same time, the sheer number of unrelated services made the system harder for the older people to use with the linear menu structure of the TV remote and to get an overview of what the system could do. In turn, the usability was one of the negative points mentioned in the final evaluation of the pilot, something that is common even with the current state of the art [27].
At the same time, having more partners meant that the system was also more complex and added additional costs of managing the development and integrating the services. So, to a certain degree, the focus put on partners by the grant issuing body reduced the chance of the system succeeding by increasing the number of services. Furthermore, it also creates an a priori commitment to create certain types of features and functions appropriate to the partners, and matching the proposal.
On the other hand, the partners included in the two grants were felt by the manager to provide valuable support in getting the necessary know-how to get this type of a complex system running. One partner provided invaluable know-how about patterns during the Intelli project and the partners that were carer organizations provided valued input with regard to the needs during the PAAL project.
In retrospect, it would have been possible to reduce the number of services, despite this. In fact, one partner conducted a market analysis with future residents to see what they needed. The diversity of the group and the fact they were just moving into a care facility made it difficult to get a clear answer of what to leave away. At the time of the pilot for the PAAL project, all participants would have the same functions, so the decision was to include lots of different features to meet all needs. Current technologies, such as Android, make it easier to install and uninstall services and would allow users to choose and change the functionality according to their individual needs, with the aforementioned risks relating to the security features.
6.4.2 Effects of biased public subsidies
It is also relevant to consider how care is funded in general. In the country in which the project was developed, care facilities are subsidized, and people receive a monthly allowance based on the level of care needed. For residents of care facilities, the cost of having a professional carer on-site every day is as little as 70 EUR per month. Currently, technology is not subsidized, even though it could be used to delay entry into a care facility for someone who does not need 100 % security. A standard emergency button, using a landline, costs 20 EUR per month. HandyHelper offered more: it was “intelligent” and did not rely solely on a button being pressed and so could provide security 24 hours a day even in cases someone was unable to press the button, or if they forgot there was a button for help. However, this additional security entails additional running costs, such as for user support and paying the service providers. To the older users, these costs seem expensive compared to the subsidized carer, who provides some security and also provides social contact. In addition to these unsubsidized running costs, there are initial costs for the installation of systems like HandyHelper. Since many were aware the system may not suffice for long, this can be a big hurdle. As long as these costs are not subsidized, people will be less likely to adopt the technology that could save costs on carers and care facilities.
6.5 How long to evaluate
A smaller issue is the length of the pilot. Due to the funding, it was not only possible, but even financially advantageous to run a long pilot. This enabled the older users to have HandyHelper installed and to be able to use it during the pilot for free. Initially, the new residents said they would be willing to pay for additional security. However, over time the focus moved away from the security features mentioned in the initial evaluation. Instead, the older users put more focus on the usability of those services they actively used. Based only on the services, the cost did not seem justified, at least at this point, where people had been living at home independently just a few months previously. What considerations are there with regard to the length of the pilot? What difference would it have made in this project?
Could the pilot haven been shortened and still have brought the same results? At the evaluation four months after the system was installed, the problems were already evident. One of the important outcomes of the pilot in this project was to recognize the need for technical changes to support a more robust system and to permit remote maintenance. If the pilot had been shorter this might not have been apparent and might still have been seen as start-up troubles. Furthermore, since the system had to be completely rewritten, it had to be clear that the costs of support continued to be high enough to warrant the costs of the change. Although more recent publications suggest some of the technical changes made for Thornhill should be standard for this type of technology [8], at the time of the development this technology was relatively new and this may not have been clear. Or, it may have related to having an academic partner, who did not consider that the system needed to be supported long term as indicated by Carl, the technical project leader.
Should the pilot instead have been longer, in order to understand the long-term needs and demonstrate the value of the system as suggested by other projects [30]? Of the AAL-JP projects studied that ran pilots, only one had a longer pilot running up to 8 months with a small number of users rather than the 6 months pilot carried out as part of the PAAL project of the HandyHelper. Their system was indeed sold commercially. The documentation of that project indicates that the older users were still aware of the security features in this system during the evaluation; however, the people for this other system still lived independently in which case these features have a greater importance.
6.6 Some promising approaches?
With these complex tensions, it almost seems like there are no solutions. As a developer, it is important to understand this complex situation. In retrospect, there are a few aspects that worked for HandyHelper, both in terms of design decisions and methods that can be singled out and used as a starting point (see Table 5). In isolation, each is associated with complexities and requires care in applying. Together they form the basis of the following recommendations for developers of this type of system.
For this type of system, a modular structure is important. Looking at this project, it can be seen that it was important to be able to adapt services to fit the specific and diverse needs of the older people and different environments—even if only security features are included. In addition, the system has to fit the specific needs of people now and evolve to fit their needs later. One solution might be to allow the older users to add their own features, which could also help make the system seem less coercive [23]. However, care must be taken that the reliability of the core security functions is still ensured.
Having more features is less important than having the right ones. In this project, having fewer functions or just not having a LED light would have made a difference. Furthermore, the services that were appreciated, such as calling the elevator, were very different from the needs discussed in the AAL position papers and valued by grant reviewers. These features need to be found to kick-start engagement with the system. Participatory design may be difficult with “intangible” sensors and older, less technical users, though their values, such as privacy, need to be considered. The show home, meetings with professional carers and the pilot provided valuable input, though they have different strengths: the show home generates ideas that then need to be tested in the pilot.
Although functionality was given priority, the usability of the services was essential, both for the acceptance and with respect to the support needed after installation. To help ensure usability, and that the system will meet the technical needs long term, a long pilot provides invaluable feedback. For this, public funding may be advantageous. However, if the older users only make the decision whether to keep the system at the end of the pilot, effort must be invested to ensure they can use the system and appreciate its value at that point.
The complete costs are often more than expected. If the system is to be used, some sort of support will be needed. A single training session does not suffice, and neither professional carers nor family members can be depended on to take this over. Having some sort of remote access may help to reduce service visits. Having partners in a variety of locations or having lots of installations together, for example in a single facility, can help reduce costs, also with respect to problems with components and infrastructure. In the end, the costs must be considered, as the older users consider the relative cost-benefit to other options, some of which may be subsidized by the state.
7 Conclusion
We set out to understand what happened in detail during the development of one specific project, to examine what problems arise. This project highlighted some interesting aspects that are likely to be relevant to other projects of this type. Looking at the development of the HandyHelper system, a complex picture emerges. Even with extensive evaluations and input from carers and older people, the system was not commercially successful, even though it worked technically. This is particularly interesting since the AAL-JP is now focusing on integrating existing technologies into new platforms and commercializing these for AAL.
The findings from this study make it clear that successful development and deployment of an AAL-type system is not trivial. The differences between older people are large and complicate the possibilities for success, be it the needs of those living at home versus those new to assisted living, or the individual differences they experience in daily life. Even with sufficient subsidies to develop the system, it was surprisingly hard to turn a profit. These challenges promise to be greater more generally in AAL, due to the limited infrastructure often available where older people already live, the diverse care structures and the complex analysis of diverse sensor data that is needed to provide reliable results with sufficient notice for people responding to the alarms to be able to help.
In the end, there are no easy answers. Companies work under financial pressures. Even though the initial version was available quickly, it took a lot of effort to get the system working in a realistic environment. Decisions made in the HandyHelper development to reduce the financial pressures immediately, such as getting funding or switching from individual homes to care facilities, ended up introducing new tensions and had a negative impact. However, there are also challenges intrinsic to this type of system: monitoring features and interactive services have competing requirements that need to be balanced by development teams to be successful.
The methods were comparable to those used in other AAL projects of the same time, though were not without problems when it came to working with users. It can be difficult working with users with sensor-based systems like this that are “intangible”. Professional carers can give valuable input; however, they cannot replace older users and ensure acceptance. Carer input is still needed to understand care structures, which can have a significant impact on the services needed and the design of those services. An extended pilot provides invaluable input for the final release, but even six months can be too short for older people with changing needs, especially if they have recently moved to a care facility. Garnering technical support from family and carers could perhaps have prevented some of the frustrations experienced during this pilot, although is unlikely to cover all support needs.
Developers need to be very aware throughout the development. In this system, “users” were not only older people, but also carers and people responding to the alarms. It is possible to be user-centered and yet shift the focus away from the older users whose acceptance is ultimately central to success. Furthermore, as goals change in projects, the way the word “user” shifts may not be obvious: from a person living on their own at home to a person living in a supervised home, where different features will be needed and valued. Further, the term “user” hides the diversity of the older population. The “right” features need to be found to kick-start engagement, but these also need to be flexible enough to adapt as the individual people face new challenges over time.
Even though the HandyHelper system is no longer sold, some still consider it a role model. The experiences in Thornhill demonstrate that initial usability problems do not necessarily mean the system will not be used in the long term. It raises questions about funding models which fund care facilities, but not technology that might help keep older people in their homes longer, and grants requiring a high number of partners in projects that actually increase complexity.
Those looking at this project will hopefully be able to use the lessons learned from it as a basis to develop useful and usable systems to support older people and carers alike.
More generally, the findings from the study raise questions about some of the implicit assumptions underpinning this type of AAL agenda at a policy level: that older people are a problem, that they are the key “users,” that they can be asked to define their own needs, and that technology is the solution to the problem. The experiences reported in this project point to some of the limitations of such “modernist” [17] technology-driven assumptions and, in line with Fitzpatrick et al. [14], suggest that we may need to reimagine the aims of such systems to better support the diversity of aging experiences and infrastructures.
Notes
Austrian Ministry for Transport, Innovation and Technology.
German Federal Ministry of Education and Research.
References
AAL Objectives: http://www.aal-europe.eu/about/objectives. Accessed 7 Nov 2007
AAL Joint Programme: Catalogue of projects 2013, Brussels (2013)
Barlow, J., Singh, D., Bayer, S., Curry, R.: A systematic review of the benefits of home telecare for frail elderly people and those with long-term conditions. J. Telemed. Telecare 13(4), 172–179 (2007). doi:10.1258/135763307780908058
Blomquist, A., Arvola, M.: Personas in action: ethnography in an interaction design team. NordiCHI 2002. ACM, pp. 197–200 (2002)
Blythe, M.A., Monk, A.F., Doughty, K.: Socially dependable design: the challenge of ageing populations for HCI. Interact. Comput. 17(6), 672–689 (2005)
Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006). doi:10.1191/1478088706qp063oa
Brownsell, S., Bradley, D., Blackburn, S., Cardinaux, F., Hawley, M.: A systematic review of lifestyle monitoring technology. J. Telemed. Telecare 17(4), 185–189 (2011). doi:10.1258/jtt.2010.100803
Doyle, J., Bailey, C., Scanaill, C.N., van den Berg, F.: Lessons learned in deploying independent living technologies to older adults’ homes. Univ. Access Inf. Soc. 13, 191–204 (2014). doi:10.1007/s10209-013-0308-1
Draper, H., Sorell, T.: Telecare remote monitoring and care. Bioethics (2012). doi:10.1111/j.1467-8519.2012.01961.x
Eichelberg, M., Büsching, F., Steen, E.-E., Helmer, A., Thiel, A., Hein, A., et al.: A technical platform for environments for ageing—lessons learned from three field studies. Inf. Health Soc. Care 39(3–4), 272–293 (2014). doi:10.3109/17538157.2014.931853
European Commission: The demographic future of Europe—from challenge to opportunity (2006)
European Commission: Final evaluation of the ambient assisted living joint programme (2013)
Fitzpatrick, G.: New challenges for health IT—design fit for life. In: Yoxall, A. (ed) Proceedings of the First European Conference on Design 4 Health 2011. Sheffield Hallam University, Art and Design Research Centre, Sheffield, pp 121-135 (2011)
Fitzpatrick, G., Huldtgren, A., Malmborg, L., Harley, D., Ijsselsteijn, W.: Design for agency, adaptivity and reciprocity: reimagining AAL and telecare agendas. In: Wulf, V., Schmidt, K., Randall, D. (eds.) Designing Socially Embedded Technologies in the Real-World, pp. 305–338. Springer, London (2015)
Friedman, B., Kahn, P.H.J., Borning, A.: Value sensitive design and information systems. In: Zhang, Y., Galletta, D.F. (eds.) Human-Computer Interaction in Management Information Systems: Foundations, pp. 348–372. M.E.Sharpe, New York (2006)
Friess, E.: Personas and decision making in the design process: an ethnographic case study. CHI 2012. ACM, pp. 1209–1218 (2012)
Greenhalgh, T., Procter, R., Wherton, J., Sugarhood, P., Shaw, S.: The organising vision for telehealth and telecare: discourse analysis. BMJ Open (2012). doi:10.1136/bmjopen-2012-001574
Greenhalgh, T., Wherton, J., Sugarhood, P., Hinder, S., Procter, R., Stones, R.: What matters to older people with assisted living needs? A phenomenological analysis of the use and non-use of telehealth and telecare. Soc. Sci. Med. 93, 86–94 (2013). doi:10.1016/j.socscimed.2013.05.036
Haslwanter J.D.H., Fitzpatrick, G.: The development of a sensor-based system for older people: a case study. HCI 2013. British Computing Society, Swinton, UK- Art. 11 (2013)
Lievesley, M.A., Yee, J.S.R.: Surrogate users: a pragmatic approach to defining user needs. CHI EA ‘07. ACM, pp. 1789–1794 (2002)
Lindsay, S., Jackson, D., Schofield, G., Olivier, P.: Engaging older people using participatory design. CHI 2012. ACM, pp. 1199–1208 (2002)
Mao, J.-Y., Vredenburg, K., Smith, P.W., Carey, T.: The state of user-centered design practice. Commun. ACM 48(3), 105–109 (2005)
Mort, M., Roberts, C., Callen, B.: Ageing with telecare: care or coercion in austerity?. Health Illn., Sociol (2012). doi:10.1111/j.1467-9566.2012.01530.x
Murphy, D.R., Meyer, A.N.D., Russo, E., Sittig, D.F., Wei, L., Singh, H.: The burden of inbox notifications in commercial electronic health records. JAMA Intern. Med. (2016). doi:10.1001/jamainternmed.2016.0209
Nielsen, J., Bodker, M.: The elephant in the room: ambiguity and temporary closure in a design process. OZCHI 2010. ACM, pp 65–71 (2010)
Peek, S.T.M., Wouters, E.J.M., van Hoof, J., Luijkx, K.G., Boeije, H.R., Vrijhoef, H.J.M.: Factors influencing acceptance of technology for aging in place: a systematic review. Int. J. Med. Inf. 83(4), 235–248 (2014). doi:10.1016/j.ijmedinf.2014.01.004
Sallinen, M., Hentonen, O., Karki, A.: Technology and active agency of older adults living in service house environment. Disabil. Rehabil. Assist. Technol. 10(1), 27–31 (2015). doi:10.3109/17483107.2013.836685
Sanders, C., Rogers, A., Bowen, R., Bower, P., Hirani, S., Cartwright, M., et al.: Exploring barriers to participation and adoption of telehealth and telecare within the Whole System Demonstrator trial: a qualitative study. BMC. Health Serv. Res. 12, 220 (2012). doi:10.1186/1472-6963-12-220
Sanders, L., Stappers, P.J.: From designing to co-designing to collective dreaming: three slices in time. Interactions 21(6), 24–33 (2014)
Spellerberg, A., Schelisch, L.: Ein dreiviertel Jahr mit PAUL: Assisted Living in Kaiserslautern, pp. 393–397. VDE, Berlin (2009)
Tornstam, L.: Gerotranscendence: A Developmental Theory of Positive Aging. Springer, New York (2005)
UN United Nations Economic Commission for Europe: Computer use by age and sex. http://w3.unece.org/pxweb/dialog/varval.asp?ma=01_GEICT_ComputerUse_r&path=../database/STAT/30-GE/09-Science_ICT/&lang=1&ti=Computer+use+by+age+and+sex. Accessed 24 May 2014
Acknowledgments
Open access funding provided by University of Applied Sciences Upper Austria. The authors are grateful for the support of the people who were willing to be interviewed about the project and provided their insight into what happened. We thank also the reviewers for their valuable comments, which really helped to improve the paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Hallewell Haslwanter, J.D., Fitzpatrick, G. Why do few assistive technology systems make it to market? The case of the HandyHelper project. Univ Access Inf Soc 16, 755–773 (2017). https://doi.org/10.1007/s10209-016-0499-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10209-016-0499-3