HomeSense is an ambient health and wellness monitoring platform for community dwelling older adults living independently in their own home. It was developed by researchers in the CREATE Health Research Collaborative at the University of South Florida. Since its inception in the field in 2016, HomeSense has had 21 installations and collected more than 10 million hours of sensor data, with an average of 6500 sensor events per day per installation.
HomeSense supports healthy living for older adults who wish to “age in place” in their own home through: 1) daily activity visualization (i.e. the ability to see an older adult move about their home in real/near-real time) and summary visualization of activity trends over time (i.e. the ability to see and compare hygiene activities, sleep, or time spent outside the home at present, to levels yesterday, a week ago, a month ago) or 3 months ago; 2) periodic reporting for case management; 3) custom real-time notifications of behaviors that may signal adverse events (i.e. increased bathroom usage that may be an early indicator of a urinary tract infection); and 4) advanced analytics that allow us to examine long-term health and behavior trends that may signal system deficits, changes in habits, and/or risk or resilience over time.
In the rest of this section, we outline the system design philosophy and development, community testbed and participant recruitment, and capabilities and services of HomeSense that enable its use in basic and applied research in ambient health and wellness monitoring.
Early system design and development
Under supervision of the USF Institutional Board (PRO 00020982), HomeSense was developed in 4 phases. Using a community-based participatory research (CBPR) approach , Phase I involved targeted focus groups and one-on-one discussion with key informants (N = 68) such as older adults, family members (i.e. adult children and spouses), providers (i.e. primary and specialty care physicians, home health service agencies), informal caregivers, members of senior advocacy groups, and payers around system needs, preferences, and abilities in utilizing an ambient sensing platform to improve health and wellness for community residing seniors. The salient themes from these discussions clearly indicated that older adults are not open to the use of cameras and microphones as sensing/monitoring in their homes. A small number of participants (N = 4) — primarily adult children and care providers — also indicated concerns regarding data security. Overwhelmingly, the majority (N = 66) of all key informants in all classes preferred “passive” monitoring technologies which did not require interaction over “active-sensing” where participants would be required to wear or maintain/charge devices, expressing that passive technologies were less intrusive and likely to yield more consistent and robust results over time.
Informed by Phase I, Phase II involved the identification of activities of daily living and home-based behaviors that: 1) aligned with onset and progression of chronic conditions such as depression and heart disease that impacted older adults in late life; 2) could signal risk for adverse events such falls or social isolation; and 3) could signal resilience and/or protection from adverse events or disease progression.
Building on information learned in Phase I and II of the project, Phase III involved development and testing of the ambient sensing platform in a simulated home environment. Known as the “Living Lab” and supported through space donation of The Concordia Village in Tampa, CREATE Health maintains a senior living apartment within Concordia’s senior-focused residential housing complex in Tampa Bay. This facility includes a variety of housing options such as independent living apartments, short-stay rehab beds, assisted living, long-term nursing care and a specialized memory care unit. Comprised of a four-room apartment (one bedroom, one bathroom, living room and kitchen) the Living Lab serves as a test-bed for technologies developed by CREATE Health prior to field use/deployment. In the first step of Phase III of the HomeSense project, example open/close (magnetic contact) sensors, passive infrared motion sensors, energy sensors, pressure sensors, water sensors and environmental sensors from a variety of manufactures were tested to identify devices that were most effective and reliable in capturing information related to the health and behavior domains identified as important in Phases I and II of the project (i.e. bathing and hygiene, feeding and nutrition, general routine, treatment plan compliance, social connectedness, sedentary behavior and sleep, toileting and TV use). Once complete, in Step 2, the long-term operation and functionality of the HomeSense system were tested and validated by a member of the research team who resided in the Living Lab full time for 8 weeks to ensure the system accurately detected the physical state of the living environment and correctly reflected activities of an occupant. Finally, in Step 3, seniors were invited to interact with the HomeSense system in the Living Lab environment and additional refinements to system function, layout, outcomes and visualizations were made based on their suggestions and feedback. In Phase IV, the fully developed HomeSense platform was deployed in the homes of older adults residing in The Villages.
Community test bed and participant recruitment
Core to understanding the factors for successful aging is the capacity to conduct pilot trials of novel interventions as well as large-scale, population studies in environments focused on the needs of aging populations. Located in central Florida, The Villages is an engaged community, focused on the needs of aging populations. To this end, USF’s CREATE Health lab and The Villages have developed a unique partnership over the past 6 years, based upon a mutually recognized goal of improved health for The Villages and the advancement of knowledge about successful aging. In only 30 years, The Villages has become the largest planned older-adult community in the U.S. The Villages is home to more than 120,000 residents over the age of 55 and supports active lifestyles through more than 3000 resident clubs and 800 daily recreational activities. The Villages comprises a well-defined and socially cohesive population, representing older-adults from all 50 states and 14 different countries. Its neighborhood characteristics foster an environment rich with opportunity for social cohesion and social connectedness, while promoting successful aging and improved quality of life among its residents. While many residents report they are happier and more active than other U.S. communities, they share similar health challenges and medical conditions as all aging adults including cardiac and respiratory diseases, cancer and diabetes. The average Hierarchal Condition Category (HCC) prospective risk scores which reflect population health risk/likelihood of needing care for Medicare beneficiaries in Lake (0.97), Marion (0.98) and Sumter (0.92) counties (which comprise The Villages community) closely mirror the national average (1.00)  and given the similarities between The Villages residents and national samples, demonstrate that what is learned here has application for other communities.
Researchers from The CREATE Health lab have had a long-standing relationship with The Villages community that has included the provision of health education events and prior research collaborations [48,49,50]. These collaborations have resulted in the accumulation of a test-bank containing self-reported general health, demographic and contact information for more than 10,000 older adults. These individuals also expressly consented to being approached regarding participation in future studies examining health and wellness issues. Participants likely to meet the study’s eligibility criteria (community dwelling, single adults over the age of 55 who speak English, and live in a pet free home that has internet access within The Villages, Florida were selected at random from amongst test-bank enrollees. They were initially contacted by phone about potential study participation. During this screening call, study aims and requirements were explained to participants, eligibility/enrollment criteria were tentatively verified, and an appointment was set for an in-home visit. During the in-home visit, study goals and needs were recapped, cognitive ability to make decisions was verified by a score of 24 or more on the Mini Mental Status Examination (a standardized survey for the measurement of cognitive health ), and informed consent was obtained.
In the 3 years since project inception, 21 older adults in single-resident homes have participated in the study, with 9 participants currently engaged. Study participation includes the completion of a comprehensive health assessment at enrollment, a willingness to have the passive home sensing system installed, and engagement in biweekly telephone screenings that collect on-going information about health using standardized assessment tools. Study participation is considered complete at 6 months of engagement, a target that has been reached by 19 of the participants (2 exited the study early as a result of relocation). Due to its varied benefits, many participants choose to continue with the study beyond the initial 6-month enrollment period and retained the sensor system in their homes even after the enrollment period. In total 15 participants have crossed the 1-year threshold, 7 have been with program 18 months or longer and 4 have stayed with the study since inception, providing 3 years of continuous home-based behavior data.
Demographics for study participants are shown in Table 1. Study participants range in age from 68 to 89 years, with a mean age at study start of 75.6 years. Six of the 21 participants are male, and 15 are female. Clinical characteristics for the sample are reported in Table 2. Participants reported an average of 2 co-morbid conditions at enrollment with 7 participants noting 3 or more comorbidities. Co-morbid conditions varied by participant and included diseases such as hypertension, diabetes, arthritis, osteoporosis, hearing impairment, depression, early Alzheimer’s and Parkinson’s disease. Not surprisingly, participants reported taking an average of 3.63 medications at the time of study enrollment, ranging from 0 to 8. High average Mini Mental Status Exam scores indicate good cognitive health within the sample. 33% of participants were found to be at high risk for nutritional deficits, and 38% scored as at risk for alcohol misuse as measured by the Nutritional Screening Checklist and Alcohol Use Disorders Identification Test respectively. Scores on the Desmond Fall Risk Questionnaire, ECOG and Timed Up and Go Tests indicated mobility risks in several program participants (for additional details on screening tools, see Section 3). No concerns regarding privacy were raised by participants at the time of enrollment and capture rate (the rate of participants initially contacted about study participation versus those who ultimately met screening criteria and agreed to participate in the study) ranged between 30 and 40%.
As part of HomeSense, wireless sensing devices (i.e. magnetic contact sensors, passive infrared motion sensors, energy sensors, pressure sensors, water sensors and environmental sensors) are placed throughout a home in a standard array by members of the CREATE Health research team. An example of such a deployment is shown in Fig. 1. These devices collect information such as room entry/exit, electrical consumption/use of targeted devices (e.g. microwave, TV), luminance, humidity, temperature, toilet usage, location of occupant, and behavior (e.g., fridge opening) where appropriate. While the number of sensing devices varies by the number of rooms in a home, most deployments utilize between 16 and 20 sensing devices to capture 90% home coverage. Houses are mapped at the consent/enrollment visit to facilitate seamless, efficient, and unobtrusive integration into home environments, and systems can be deployed, verified, and rendered fully operational within 14 days.
As shown in Fig. 2, an array of networked wireless Z-Wave devices are installed in each house. A Raspberry Pi connected to the internet acts as a remote gateway (serving as a small computing device) and monitors the Z-Wave communication network. It stores and relays relevant events and information to USF’s Health Insurance Portability and Accountability Act (HIPAA) compliant secure server cluster. In case of internet outages, data from the homes is stored locally on the gateways and sent on to the main server once connectivity is restored. Data that comes in to the main server from participants’ homes is permanently stored and backed up in relational databases which support administrative tasks such as sensor configuration, device tracking, and notification set up. A collection of data analytics tools continuously process the collected data and support the HomeSense services including alerts and notifications, case management reporting, participant communications, and the web-enabled user interfaces for participants and authorized caregivers which are further discussed in subsequent sections.
Web-enabled user interfaces
Sensor data is sent from remote gateways in the participants’ homes to the main-server where it is processed to facilitate daily activity monitoring and summary, and identification of changes over time. A variety of password-protected user profiles (i.e. participant, family member, health provider, paid case manager, and administrator) are available within the HomeSense web portal. These profiles are fully customizable and ensure the user/older-adult maintains control of what information they do or do not share and with whom. Once logged in, a customizable landing page provides users a brief summary of daily activity and home-based information from the prior day (i.e. sleep and wake times, home temperature, time spent outside the home); and a daily activity visualization screen shown in Fig. 3 facilitates the ability to see an older adult as they move about their home. Developed in conjunction with user feedback collected in Phase I and III of the HomeSense development process, a refined set of sensor data is shown across user classes (i.e. informal caregiver versus paid case manager) to maximize utility and ease of use for the target user group. Additionally, processed data is used to create customizable tables designed to summarize change over time. By default, data points such as sleep or bathroom usage are shown comparing today’s behaviors to 1, 7, 30 and 90 days prior.
We have observed that interfaces such as the daily and summary activity visualizations can help alleviate caregiver burden and engage older adults, encouraging them to be more actively involved in management of their health. Utilizing this tool, older adults can gauge parameters such as the amount of time they spend sedentary and the number of times they have missed their medication and actively take intervention steps to change and improve their overall health. For caregivers, being able to visualize care recipients at home while out shopping or engaged in other activities, results in increased independence and lowers burden since they can be outside of the home but still be aware of what their care recipient is doing. This increased freedom can increase social engagement, reduce loneliness and social isolation and lead to increases/improvements in mental and physical health long term.
Case management reporting
Reporting tools for case managers shown in Fig. 4 enable paid health providers to readily identify and prioritize clients in need of support/touch points. These reports allow case managers to easily identify frequency and changes in activities such as toileting, hygiene, sleep, physical activity and nutritional behaviors. Among other examples, use of these reporting tools can help case managers track and identify patients who have missed medications, failed to rise or who have experienced changes in their sleep and/or activity patterns that may signal the onset of, or risk for adverse events. Case managers supported by computer-aided decision making tools can identify patients outside of their routines who may be in need of intervention or services – all without the need to manually review data patterns and/or physically check in with or on patients across great distances who may not be in need of support. As such, HomeSense serves as a tool to strategically extend provider reach and expand caseloads in ways that still identify and prioritize patients in need of additional services while respecting the privacy of those who don’t.
HomeSense data can also be used to examine differences in home-based behaviors over time across age and clinical sub-groups to facilitate case managers’ understanding of the ways in which advancing age and/or clinical conditions may impact routine and quality of life over time, and to facilitate service and treatment planning. Using Welch’s t-test for unequal sample sizes and the Wilcoxon Rank Sum Test for continuous variables, Table 3 identifies home-based activity data over 180 days for the 15 program participants enrolled in the study 12 months or more. Results indicate significant differences in the amount of time spent doing laundry (70+ =80mins; <70 = 41mins; p = .04) and in the average number of sleep disruptions experienced over-night (70 + =0.8; <70 = 0.5; p = .04). Given the negative impacts of sleep disruption on physical and mental health , reasons for increased sleep disruption in those 70+ should be explored, and where appropriate, medications or lifestyle changes to better control factors such as co-morbid conditions that may improve sleep patterns should be encouraged to preserve long term health and quality of life. While potentially an artifact of machine run times, increased time spent in routines such as doing laundry may also be an early cue that additional help may be needed at home and options for supplements such as housekeeping services may reduce burdens and keep older adults living longer at home.
Differences in home-based behaviors are also seen across clinical sub-groups and can be used as a tool to enhance treatment planning and quality of life. For example, preliminary results in Table 3 illustrate that those with osteoporosis spend less time in bed (osteoporosis =7.3 h/day; no osteoporosis = 8.4 h/day; p = .05) and experience a greater number of sleep disruptions (osteoporosis =1.4/evening; no osteoporosis = 0.5/evening; p = .04). Additionally, people with osteoporosis spend more time using their televisions (osteoporosis =3.9 h/day; no osteoporosis = 2.5 h/day; p = .04) and more time away from home (osteoporosis =4.5 h/day; no osteoporosis = 3.6 h/day; p = .03). These findings are consistent with commonly reported disease symptoms such as frequent pain and trouble getting comfortable  which can lead to behaviors such as rising from bed during the night to watch TV - adversely impacting sleep quality, sleep consistency and the body’s ability to produce essential hormones such as melatonin which are believed to help regulate sleep and reduce free radicals in the body, leading to poorer health over time . For patients such as these, specialized pain clinics and physical therapy may be helpful in increasing night time comfort; additionally, dietary supplements such as synthetic melatonin may be helpful in re-regulating the sleep cycle.
Notifications: A mechanism for health interventions in the short term
The main goal of the HomeSense system is to help seniors age safely in place while optimizing their health and wellness. To do this, HomeSense strategically deploys standard sensor arrays to facilitate data collection in 8 health domains including feeding/nutrition, bathing/hygiene, toileting, sedentary behavior and sleep, general routine, treatment plan compliance, social connectedness and TV usage. Layered on top of this is a fully customizable notification system that facilitates web and mobile alerting to notify interested parties of changes in physical activity, sleeping habits, time spent outside the home, sedentary behavior, nutritional habits, socialization patterns, toileting or hygiene patterns in real/near-real time. Additionally, the notification system can be set up to send alerts when participants are in the bathroom too long, if they exit their home late at night, if they altered/missed their medications, if they failed to rise, and/or if they failed to fulfill obligations at home. These notifications are either rule-based (e.g., notify if an occupant exits the home after 11 pm; notify if an occupant has not gotten up by 11 am) or based on understanding of previously learned/observed participant behavior (i.e. notify if an occupant has a 20% increase in sedentary behavior or toilet usage). Once triggered, notifications appear on the daily activity visualization screen, alerting users that something is outside the norm. In addition, a screenshot of the daily activity screen and an explanation of the notification, as shown in Fig. 5, is sent to users via email facilitating easy access and understanding without the need for additional login or software. Notifications can be sent to select users (e.g. changes in hygiene or sleep habits to health care providers, late-night exist seeking to a child/informal caregiver) and/or to multiple users (e.g. changes in nutritional habits to adult children and health care providers) facilitating strategic communication across occupants’ social/health networks.
Communication with participants
An important research component of an ambient home health and wellness monitoring platform is the ability to recognize events relevant to health and wellness. Algorithms developed for such purposes require ground truth to be validated. Traditional methods like manual activity logging unnecessarily burden users and are ineffective . As a result, we have developed a mobile app which allows us to send custom questions to, and receive answers from our study participants in real time, allowing us to validate the algorithms with minimal burden. An example question sent to a participant regarding the time they went to bed is shown in Fig. 6. We have successfully used this mobile app to validate activity detection algorithms involving four activities involving leaving and returning home, and going to bed at night and waking up in the morning. During a 5 month study, 518 questions (236 true negatives) were sent to two participants who responded to 72.2% of the questions. Their responses demonstrated 90% and 98% accuracy with no significant variation by question type or participant. 88% of the true negative questions were responded to as a “No, I did not perform this activity in this time range” with no significant variation by participant showing strong evidence that there was no social desirability or habituation bias in the validation process . This tool can also be harnessed to administer well-established, short, high salience instruments such as the Patient Health Questionaire-2, a screening tool for depression symptomology  to participants in real time to explore a variety of health outcomes such as mental well-being.
Data visualization for understanding health and wellness related behaviors
An effective way to explore trends in daily activity patterns using sensor data is through data visualization. To this end, we developed a visualization dashboard, HomeSense-Dash, which contains dynamic and flexible sensor data visualizations for varying time durations and sensor devices. The example in Fig. 7 highlights select sensors which represent the general daily activity of a participant over a 24-h period. The participant wakes up around 6:00 am noted by the disengagement of the pressure mat under his bed. After brief activity in the living room, kitchen and bathroom observed by the PIR motion sensors, he leaves the house returning again around noon observed using contact sensors placed on doors leading to the outside and absence of motion inside the home. After a brief period in the house, presumably for lunch based on the level of activity in the kitchen observed using motion sensors and interactions with the fridge and pantry observed by contact sensors placed on fridge and pantry doors, the participant again leaves returning home around 3:00 pm and settling in for the evening. In the evening, he spends time in the living room on his recliner observed using a pressure mat under the chair cushions and in the study watching TV observed using smart switches monitoring energy usage. He also interacts with the fridge and pantry around 8:00 pm presumably for dinner, and he eventually goes to bed around 10:00 pm. Throughout the day, the participant’s toilet usage is observed using motion sensors in the bathroom and water sensors placed in the toilet reservoir.
HomeSense-Dash is also used for more detailed exploration of specific activities. For example, Fig. 8 shows a polar graph of the times of day one of our participants opens their medicine cabinet over several months. The axis represent midnight, 6:00 am, noon, and 6:00 pm similar to a 24-h analog clock. The participant takes medication 3 times a day: morning, late afternoon and evening. It is clear that compliance with the morning and late afternoon medicine routine exhibits much greater time variability than the medication routine in the evening time frame. This kind of data can be very informative in identifying additional targets for utilizing tools such as care plans and health education to increase compliance with proper medication routines. In cases where patients are managing time-sensitive medications like those for blood pressure, this reporting offers significant potential to assist in the maintenance and achievement of good health over time.
In addition to exploring individual sensor data, we have developed algorithms which combine information from individual sensors to extract higher level information such as when participants leave and return home, what time they go to sleep and wake up, how much time they spend in their home engaged in sedentary activities such as watching TV, and how active they are moving about in their home . Visualizations based on the outputs of such algorithms elucidate participants’ daily routines at home and reveal patterns that remain stable as well as vary over time. For example, Figs. 9 and 10 show participant status as active, sedentary, in-bed and out-of-home over a 3 month period for two participants (A and B). The time durations during which the sensor system could not estimate the status of the participant is categorized as unobservable.
Figure 9 shows that Participant A exhibits stable activity patterns over time. The participant tends to leave their home each day in the late morning and early afternoon and usually returns home before 5 pm; we note that the participant was away from home on the evening of Feb 16. Sleep patterns for this participant are also fairly routine and stable. Typically the participant wakes up around 7 am and goes to bed around 10 pm. On January 10th, February 24th and February 27th the participant appears to have had more restless sleep – and has spent the early morning hours out of bed. As reported in their biweekly health assessments (see Section 3) these dates coincide with worry for a family member (Jan 10th) and onset and worsening of a cold (February 24 and 27th), illustrating the ways in which stress and physical body strain change behaviors at home.
Examining home-based behaviors across participants also yields potential for significant insights around the impact of disease conditions and medications on home-based behavior patterns, Fig. 10 demonstrates home-based activity patterns for Participant B over the same time period. The participant generally leaves home twice a day – once around lunch time and again in the late afternoon. S/he has fairly consistent sleep patterns, going to bed around 9 pm each evening and rising around 7 am. By contrast to participant A however, participant B consistently has interruptions in their sleep as noted by the green and yellow colored breaks in the blue colored time-in-bed status during the night. When examining the medical history of both participants, it is noted that Participant B self-reports a diagnosis of Irritable Bowel Syndrome (IBS), a condition known to impact sleep patterns. Given the importance of consistent sleep for immune, physical and social health in late life, these type of visualizations yield an opportunity to explore the potential impact IBS has on the sleep patterns, and to explore the potential positive benefits of changes in medications or dosing could have on the patient’s sleep and quality of life. While beyond the scope of the present IRB, given that participant B initially declined medication for IBS under the auspice that it was not problematic enough to interfere with their daily life, health coaching supported by these types of data-driven outcomes may yield substantive change in outlook and willingness to engage in active treatment.
Ambient sensors create a data-rich environment, but are not without challenges. For example, ambient sensors installed in private homes are relatively inaccessible requiring coordination between the research team and the participant in cases where maintenance is needed. Given the complex nature of participants’ lives and varied commitments, it can often take days or weeks to service or replace malfunctioning devices. Since double sensing of every event in the home is cost-prohibitive and may feel unnecessarily intrusive, extracting reliable and accurate information requires the ability to adequately compensate for occasionally missing or inaccurate data. To this end, we have developed a number of data imputation tools for dealing with missing or inaccurate data. For example, pressure sensors used to monitor participants sleeping patterns may malfunction, or heavy objects placed on the beds can create inaccurate data that is not representative of the behavior of interest. Accurate information related to understanding sleeping patterns through bed sensors must therefore be supplemented with information from other ambient sensors. Figure 11 shows such an example where the data from a bed pressure sensor is imputed with the information from motion sensors in the network to estimate time in bed.
In addition to the challenges associated with sensor failures, ambient data can also be skewed by the presence of long term visitors who may alter a participant from their normal routine (e.g. eating more meals out) and/or alter patterns of behavior, device usage, and time spent in various activities or areas within their home (e.g. guests often increase the use of guest bathroom spaces within a home). Identification of such time periods is critical for accurate assessment of the participant’s health and wellness as these periods may not signal adverse deviations in health, but rather only signal changes in the number/type of residents within a home. Using features from ambient sensor data, we group days into different categories to identify those representative of the participant’s normal activities and those that may be skewed by the presence of visitors and/or sensor system related issues. Figure 12 shows a cluster plot of the days for a 9-month period clustered into 3 categories where the x and y-axis are the two most significant principal components which are linear combinations of the original features . The red cluster shows days that are representative of the participant’s normal routine when home alone, the green cluster shows days where the participant had visitors and the blue cluster represents days where there were visitors and one or more sensors malfunctioned.