1 Introduction

With the revolution era in the Fourth Industrial Revolution “Industry 4.0”, Internet of Things (IoT) enabled technologies have been widely used in transforming many traditional practices or scenarios to more connected automatic and intelligent forms. Remarkably, due to the rapid proliferation of wearable devices and smartphones, utilising IoT and wearable intelligence into pervasive healthcare fields becomes a popular research topic, where conventional hub-based healthcare systems is upgrading to more self-empowered and personlised healthcare systems (PHS). This trend is well aligned with recent concept of “Healthcare 4.0” spill-out from Industry 4.0 that is a continuous process of re-shaping the entire healthcare value chain to be more precise, preventive, predictive and personlised, with significant benefits to elderly home care and dementia care. Especially considering that dementia is the leading contributor to disability amongst older people and causes significant morbidity as well as personal and family burden (Wimo et al., 2006), development of effective strategies to older citizens with dementia has become an international priority, particularly in developed countries due to dramatic demographic changes in the last decades.

Typically, the motivation of utilizing modern ICT in upgrading conventional healthcare systems offers promising approaches for efficiently and precisely delivering medical healthcare services to dementia patients , named as E-health (Burkhard, 2010), such as electronic record systems, telemedicine systems, personalised devices for diagnosis, etc. (James et al., 2001; Qi etal., 2020;; Sun & Fang, 2010). However, due to the continuous increase in life expectancy, there are rapidly growing population who are over 80 years old in developed countries. This fact raises some important issues on economic viability of traditional dementia care systems. There is an urgent need to develop more coordinated ICT solutions to provide high-quality and patient-centered health services for patients with dementia.

Targeting at this need, wearable intelligence solutions have been developed for people with dementia, to prevent disease onset and progression, and maintain patient independence in daily life. Successful application of these wearable intelligence technologies to dementia care could contribute to faster and safer preventive care, further reduce the overall costs and enhance sustainability (Sebestyen et al., 2014). It will potentially provide highly customized services for older people in future.

However, considering difficulties on developing cheap and precise medical wearable sensors, the application of wearable technology to dementia care still remains many challenges (Costigan et al., 2002; Pappas et al., 2001; Veltink et al., 1996), including heterogeneity of wearable devices connected (Li et al., 2010; Mozer, 1998; Thatte et al., 2012) and high demand for interoperability (Barthel et al., 2013; Ovengalt et al., 2016; Qiet al., 2019; Yang et al., 2018). Its successful applications to dementia care require an interoperable IoT system enabling full self-empowerment of data and knowledge standards and sound foundations for clinical decision-making (Qi et al., 2020). The above-mentioned challenges and requirements provide a plenty of opportunities to study, design and develop new concepts, algorithms, models and applications of wearable intelligence technologies in dementia care.

In an effect to understand advance of wearable intelligence technologies in dementia care, we conducted a survey on reviewing multimodal wearable intelligence technologies for dementia care in Healthcare 4.0. We undertook an extensive literature review from 2010 to 2020 by reviewing relevant papers from three main academic databases (ACM digital library, IEEE Xplore and Science-Direct). Key search terms included: ‘wearable computing’ (WC), ‘wearable intelligence’ (WI) and ‘dementia’ (D), ‘Healthcare’ (H). Also, the research projects related to IoT, e-health, smart healthcare, etc., were examined by searching from the portal of EU, TSB and EPSRC funded projects. Our review focused on identifying the breadth and diversity of existing research in multimodal wearable intelligence for dementia care, including technologies, applications and case studies.

The rest of the paper is organized as follows. Section II presents the background and current research of wearable intelligence in dementia care. Section III reviews key enabling technologies related to multimodal wearable intelligence for dementia care. Section IV describes several main applications and case studies related to multimodal wearable intelligence for dementia care. Section V discusses research challenges and future trends. The conclusion is given in Section VI.

2 Current Research of Wearable Intelligence for Dementia Care

As the leading contributor to disability amongst older people, dementia causes significant morbidity as well as personal and family burden, older people with dementia usually suffer from progressive cognitive decline and deterioration in their capacity for living independently. For instance, dementia patients have general confusion and on-going disorientation in time and place, and become apathetic or display a lack of initiative in significant events like family birthdays or anniversaries. These initial symptoms of memory loss and confusion can be transient and may be accompanied by periods of normal and lucid behavior. In the long term, these memory losses associated behavioral and emotional changes in the dementia elderly exhibit strong effects on their living independence and mental health, further leading to huge emotional and financial burden on their families and communities. Enabling effective dementia care services to older people with dementia requires a large amount of high-quality long-term care from professional home care to institutional nursing care facilities. Yet, existing worldwide healthcare systems have been subject to a long history of increasing expenditure on maintaining these high level services. One key strategy is to empower citizens and patients to self-manage their own health and disease, by utilising innovative ICT techniques, including new diagnostics, sensors and devices, co-operative ICTs, mobile or portable new tools, etc. Therefore, innovative ICT technologies and tools like wearable intelligence for supporting dementia care are particularly important (Prince et al., 2013).

As mentioned in last section, Fig. 1 demonstrates the number of Journal and Conference articles related to wearable intelligence and dementia from 2010 to 2021 in three key research databases through searching keywords “Wearable Computing & Healthcare”, “Wearable Intelligence & Healthcare”, “Wearable Computing & Dementia” and “Wearable Intelligence & Dementia”. Notably, in the IEEE explore and ACM library, we searched these keywords in all metadata; but in Science-direct site, these keywords were only searched in title, abstract and author keywords.

Fig. 1
figure 1

Number of Journal and Conference articles related to wearable intelligence and dementia from 2004 to 2021 (Wearable Computing, wearable intelligence, Dementia and Healthcare)

Full size image

The results in Fig. 1 first show that the wearable computing and wearable intelligence technologies have been widely used in ‘Healthcare’ where there are totally 6589 publications in these three research datasets. But wearable computing and wearable intelligence technologies have not been used in dementia care in a large-scale, with only 333 publications in these three research datasets. Also, from the engineering perspective, the research work related to two key words ‘wearable computing’ and ‘wearable intelligence’ are popular in IEEE and ACM publications. It implies that utilising wearable intelligence for dementia care is still in the early stage in comparing to general healthcare applications and waiting for further study. There are many challenges and needs to explore new concepts, algorithms, models and applications in wearable intelligence technology in dementia care in dementia research.

New wearable intelligence technologies are drastically changing the nature of traditional assistive applications or tools for dementia care. Existing digital memory assistive solutions for dementia care are still reliant on external prospective memory aids or game-based memory rehabilitation therapies. Many research work that are built upon the vision that wearable intelligence have a great potential to enhance dementia care services to older people. Modern wearable intelligence technologies, such as unobtrusive life-logging data sensing, multi-modal data aggregation, intelligent reasoning and user-centered interaction have a great prospective in supplementing highly effective and personalised dementia care services to dementia people by overcoming the limitations of traditional memory assistive techniques, such as limited effects in prospective memory aids, tedious content of memory training programmes, unintuitive user interface design for older people. In order to better access wearable intelligence for dementia care in this survey, a framework is built in Fig. 2 for illustrating the key technical implementation of a multi-model wearable intelligence system for dementia care.

Fig. 2
figure 2

Key technical implementation of wearable intelligence system in dementia care

Full size image

The concept of this framework is traced from ‘personlised virtual coaching’, originally targeting at revolutionising the worldwide healthcare systems and community towards the challenges of ageing society. Many EU projects have started the movement in this direction to design and develop various ‘Virtual Coach’ systems for empowering elderly people to self-management of their chronic diseases and promoting them a healthy lifestyle. The most relevant projects include A2E2 (Merilahti et al., 2010), which is a support action for producing a behavioural and motivational enrichment platform for helping elderly people to find the right balance between activity and rest in daily life, P-Wheel (Siewiorek & Smailagic, 2008), and Mem-Exerciser (Lee & Dey, 2007), of all which have shown a high level of interest in helping people whose cognitive abilities are impaired by natural ageing, disease or trauma. But to our knowledge, the research work involved in these projects do not focus on the collection of, access to, analysis of and utilisation of a multi-dimensional personal experience related data or information from the dementia care perspective to support individualized dementia care.

In Healthcare 4.0, towards the more concrete and reinforced concept of “Personalised dementia care”, wearable intelligence technologies will enable providing personalised, intelligent and integrative dementia care services including memory training, guidance for daily difficulties and recommendation for healthy lifestyle. More precisely, we have come up with a number of observations in this field, including:

  • Existing ICT technologies in dementia care mostly focus on the prospective memory tasks but fail to cover both retrospective memory and prospective memory support. Future research in wearable intelligence technologies will be trending towards providing both retrospective memory and prospective memory support to older people with dementia.

  • Lacking automated and intelligent tools for providing personalised dementia support services makes existing dementia supportive applications have limited impacts in user’s daily life. Wearable intelligence technologies will potentially offer a wide range of services including training, rehabilitation, guidance and recommendation to people with dementia.

  • User-centered interaction technology will be a pillar in the dementia supportive systems. Factors such as stigma associated with using the device, general attitudes toward a device, such as perceived usefulness and perceived ease of use, as well as the reliability of the device, can all be barriers to acceptance and use by users.

3 Key Enabling Technologies for Wearable Intelligence in Dementia Care

As shown in Fig. 2, from information processing perspective, key techniques for implementing wearable intelligence in dementia care could be built on a classic tri-phase model of personal information, encompassing the central phases of registration, storage and retrieval, which are respectively implemented by ICT techniques on unobtrusive wearable context monitoring, multi-dimensional data repositories and model repositories, knowledge discovery and adaptive recommender for intelligent reasoning.

3.1 Unobtrusive Wearable Context Monitoring

Wearable context monitoring usually focuses on developing a personalised, highly unobtrusive mobile and wearable system. To date, the specific needs of dementia care patient groups have not been sufficiently identified in the literature.

Typical wearable context monitoring system design started from many mobile and wearable accessory designs, including smart eyeglasses (Kong et al., 2016), smart bracelets (Angelini et al., 2013), and some clip-on or necklace loggers (Baek et al., 2013) and smartphones (N. Armstrong et al., 2010). In the work (Kong et al., 2016), smart glasses with image-based appliance selection using user contextual information has been utilised for controlling home appliances for elderly people. In the work (Angelini et al., 2013), a smart bracelet is designed that aiming at enhancing the life of elderly people, including monitoring the health status and alerting them about abnormal conditions, reminding medications and facilitating the everyday life in many outdoor and indoor activities. Also, many new smart garments developments are also developed recently by researchers; e.g. smart watches (Fleury et al., 2010), smart garments from the EU SimpleSkin project (Ahmad et al., 2016; Amft, 2016; Cheng et al., 2016; Hassib et al., 2016; Martindaley et al., 2016).

The above cost-effective wearable devices have provided essential guidelines and benefits to the end-users. But the well-designed wearable intelligence systems for dementia people should evolve the solution across development cycles to maximize effectiveness and user compliance. Their goal is to evolve a minimal set of personal devices for the virtual memory coach, e.g. a smartphone and a pair of smart eyeglasses, or a smart garment to provide the user with an unobtrusive system that fits personal needs. Some researchers begin to focus on developing monitoring and interaction technologies that can capture user context, i.e. a user's situation, with all relevant aspects, including activity and behaviour , emotion (Sano & Picard, 2013), location (Seon-Woo Lee & Mase, 2002), and social interaction (Paradiso et al., 2010), as well as provide interaction continuously and unobtrusively.

While some aspects of user context, such as outdoor location, is conveniently available from smartphones, details of complex behaviour patterns, such as diet, hygiene, housework, are all still challenging to identify due to the variability of the involved behaviour patterns. Nevertheless, effective dementia care should depend on memory training and reminding on everyday routines. Complex behaviour patterns further include arousal and social interaction, which are relevant components of everyday life and can contribute to some training based memorable events. Also, particular intersections of behaviour and other context information may be helpful to derive interesting, i.e. memorizing moments from the user's experience. For example, a walk on the street and meeting a good friend there, and have a longer conversation may be such a memorizing moment that could be exploited.

3.2 Multi-Dimensional Data Repository and Model Repository

Multi-dimensional data repositories and model repositories in wearable intelligence systems show in Fig. 2 concentrate on research and development of robust methods and tools to manage the storage, retrieving, protection and utilisation of multidimensional life-logging data captured from wearable devices.

We reviewed the prior work in the literature, including data storage model, RDF data repository, data processing models, and security framework. The methods and models investigated in this section build robust and fast data repositories holding multidimensional life-logging data or information of individual users. These information include their activities, behavior, life style, geography environment, etc. The typical data repositories approaches mainly include ontologies based approaches (LOINC, 2018; MedDRA, 2018; SNOMEDCT, 2018) and semantic based approaches (Belleau et al., 2008; HCLSKB, 2018; NIH, 2018; PubMed, 2018; WHO. World Health Organization, 2018; Vandervalk et al., 2008; Zaveri et al., 2013). Ontology data repositories usually reply on controlled vocabularies of scientific terminology to assist in large-scale data annotation, including the basic terms and relations in a domain of interest, and rules how to combine these terms and relations (Tables 1 and 2).

Table 1 Unobtrusive wearable and context monitoring
Full size table
Table 2 Multi-dimensional data repository and model repository
Full size table

In the past years, there have been some healthcare domain ontologies (LOINC, 2018; MedDRA, 2018; SNOMEDCT, 2018), which were established for describing anatomical parts and their relationships in biomedicine or specific terms used in Electronic Health Records or rehabilitation domain. SNOMED CT (SNOMEDCT, 2018) is a well-established and widely-used ontology by many researchers for representation of clinical concepts, terms and relationships in healthcare. Logical Observation Identifiers Names and Codes (LOINC, 2018) aims at offering a universal code system for laboratory test and clinical observations. MedDRA (Medical Dictionary for Regulatory Activities) (MedDRA, 2018) provides a comprehensive, standardised and specific terminology to help share regulatory information for medical products.

Semantic data repository is built on the web source linked data approach and help the health style related data searching and processing from rich resource in the internet for further intelligent recommendations. The model repository were used to store a variety of data analysis or processing method like image analysis, pattern recognition, for extracting the most useful memory cues of constructing individual experience. For instance, over 50 different semantic based datasets have been built by the WHO’s Global Health Observatory for monitoring statistical data and analysis of environment health, health systems, HIV/AIDS, etc. Also, many health institutions or organisations such as PubMed (PubMed, 2018), or US National Library of Medicine (NIH, 2018) provide selective connections to enormous health database repositories. The only issue is that they usually contain data in proprietary formats like PDF or spreadsheets, with some difficulties on further data processing.

But some recent attempts have been made on utilising Semantic Web to decrease complexity of data processing and sort out some classical integration problems. Vandervalk et.al (Vandervalk et al., 2008) developed a decentralized web service framework called CardioSHARE, providing a SPARQL endpoint that enables querying transparently resources in “deep web” from distributed and independent sources. Zaveri et.al (Zaveri et al., 2013) have created one dataset containing about 3 million triples by converting WHOs GHO datasets to RDF using RDF Data Cube Vocabulary. The dataset is published as Linked Data using the OntoWiki platform, with an SPARQL endpoint for querying the data. Additionally, an open-source project called Bio2RDF (Belleau et al., 2008) is established by leveraging Semantic Web technologies to provide the large networks of Linked Data from a diverse set of heterogeneously formatted sources obtained from multiple data providers. The entire Bio2RDF system provides a SQPRQL endpoint that can be used to query around 11 billion triples from 35 different datasets from clinical trials, PubMed and other large data set providers from the biomedical domain.

Another research issue in a multi-dimensional data repository is on developing a unified approach towards trust, security and privacy co-analysis, design, implementation and verification for ensuring these personal data processed and handled in compliance with user needs and rights in autonomous services without human intervention. Some researchers have attentions on solving the research challenges in data confidentiality and authentication (Wenjing Lou et al., 2004), access control (Goyal et al., 2006), privacy and trust (Dwyer et al., 2007) among users and their data, and the enforcement of security and privacy policies. In particular, the end-to-end approach to security is usually proposed as an extending security mechanism from the device to the platform to the application, in a seamless and fully integrated manner. Also, the processing and analysis of large volume of multidimensional personal information can pose difficult privacy, security and trust issues. The suitable solutions here should be independent of the exploited platform and able to guarantee: confidentiality, access control, and privacy for users and things, trustworthiness among devices and users, compliance with defined security and privacy policies.

3.3 Knowledge Discovery and Adaptive Recommender for Intelligent Reasoning

Knowledge discovery for intelligent reasoning in Fig. 2 focus on study, develop and integrate data analysis and intelligent reasoning technologies, for supporting the extraction of accurate information and providing intelligent personalised recommendations to the users. Actually, a vast amount of approaches (Bharucha et al., 2009; Bizer et al., 2009; Boger et al., 2005, 2006; Braziunas & Boutilier, 2010; Chen et al., 2005; Limaye et al., 2010; Liu, 2010; Mulwad et al., 2010; Schmachtenberg et al., 2014) have been proposed which combine methods from data mining and knowledge discovery with Semantic Web data, including Semantic Web Technologies, Link Open Data (LOD), machine learning technologies.

LOD (Bizer et al., 2009; Schmachtenberg et al., 2014) is an open, interlinked collection of datasets in machine-interpretable form, covering multiple domains from life sciences to government data. In LOD, it is possible to harness knowledge vault at various steps of the knowledge discovery process. A large number of approaches have been presented for extracting the schema of the tables, and mapping it to existing ontologies and LOD. Mulwad et al. (Mulwad et al., 2010) have made great effects for interpreting tabular data using LOD from independent domain. Several methods have been proposed by them that use background knowledge from the Linked Open Data cloud to infer the semantics of column headers, table cell values and relations between columns and represent the inferred meaning as graph of RDF triples. Additionally, Liu et al. (Liu, 2010) presented a new learning-based semantic search algorithm to recommend relevant and connected Semantic Web terms and ontologies for the given data. Limaye et al. (Limaye et al., 2010) suggested a YAGO based probabilistic graphical model for simultaneously choosing entities for cells, types for columns and relations for column pairs.

With respect to intelligent reasoning, some work (Bharucha et al., 2009; Boger et al., 2005, 2006) have made use of hybrid, contextualized and adaptive recommender algorithms to assist people with dementia in their daily routines by providing suggestions and context specific information. These recommendations can be intra- or intersubjective, i.e. they can be derived from past actions of one user or can be adapted from successful strategies of others. To generate and decide on the ideal recommendation for each user at a given situation, different aspects have to be considered. Intra- and intersubjective recommendations rely on various data sources, including internal analysis of personal life pattern and rich resources from internet. Adaptive recommendation algorithms will select appropriate sources to adopt to different user role.

The intelligent recommendations (Braziunas & Boutilier, 2010; Chen et al., 2005) can be adapted to different user groups like patients, family members, caregivers etc. For each recommendation, their respective context is considered to determine the necessary adaption. Additionally, feedback and reinforcement learning relies on feedback from all user groups to improve and adapt to requirement changes. Feedback cycles can be installed on different abstraction levels, from the lower end of information extractions up to the recommendation process are calculated. At the beginning of the processing chain, the data fusion can learn which data sources are best suited for any given user group. Similarly, the information extraction and event detection can employ feedback about events and patterns are well received by the users into the machine learning algorithms.

3.4 Multimodal Adaptive User Interface and Personalization

Multimodal adaptive user interface and personalization in Fig. 2 focus on studying existing user interaction techniques, which can be specifically applied into wearable and mobile devices towards older people with dementia, for easy acquisition of intelligent recommendation or guidance, and accessible interaction with memory training applications.

As shown in Table 3, some researchers focus on designing and developing a tangible interaction technique (Hock et al., 2018; Sakai et al., 2012; Schelle et al., 2015; Seymour et al., 2017) for mobile devices with touchscreen, like iPad or Microsoft Surface, etc., which supports the facilitation of intervention or games based memory training applications like reminiscence activity with visual, audio or other tangible artefacts e.g. photographs, videos. Also, some memory cues based interactive games (Sisarica et al., 2013; Westphal et al., 2017) like puzzle touch or mapping learning are developed to improve the effectiveness of psychosocial interventions that foster the well-being and quality of life of people with dementia.

Table 3 User interface and personalization
Full size table

Regarding the interactive interface of mobile device, some work were carried out in the fields of developing and integrating multimodal adaptive user interface (Hoey et al., 2012; Yu & Ballard, 2004) exploiting user preferences, stress, emotions and previous behaviour, and study different modes in receiving and communicate information like guidance or recommendation. The design of user interface is based on identified user requirements from pilot cases, and should support collaboration among elderly and caregivers, and provide personalization tools that allow elderly to customize the user interface and behaviour of their applications. Another important issue in user interaction is to investigate and develop innovative user-interaction methods (Locher & Tröster, 2007; Schwartz, 2008) driving the utilisation of smart textile in wearable context monitoring. Considering the main needs of warning and reminder functions in wearable intelligence system, vibration feedbacks will be used as a key interaction channel to be used in smart clothing, that were embedded in several places of clothes, and would vibrate to indicate the wearer if there are some accidents or reminding things.

4 Wearable Intelligence Applications and Case Studies in Dementia Care

This section reviews some successful m-health platforms and case studies related to dementia care, and also gives a discussion on how these m-health platforms impacting on wearable intelligence applications in dementia care for dementia patients.

4.1 SMART4: Mobile Health Platform

SMART4 (SMART4MD project, 2018) is a EU funded project aiming at developing an innovative mobile healthcare platform that is specifically targeted to patients with mild dementia with an improved quality of life. The tool enables patients to adhere to their treatment and share data with their carers and doctors; and also allows carers to monitor patients more easily and share their own well-being with doctors. The utilization of this tool demonstrates a great success in slowing the dement patients’ cognitive and functional decline, avoid carers getting exhausted and reduced costs of emergency care.

The key features of the SMART4 platform include: 1) a powerful customization engine to enhance accessibility when patients faced by cognitive decline. 2). Quality of life and health tracking for people with dementia and all other conditions being managed. 3) Activity, medicine and appointment reminders to assistant dementia patients’ independent living and better health management 4) Cost-effective visualization tools to help check health progress and medicine compliance.

From end-user perspective, SMART4 platform offers build-in planner to help dementia patient organize their day and send reminders to take medications, attend appointments and perform day-to-day tasks that keep them independent for longer. Healthcare personals can easily track the progress of their dementia patients’ health through SMAT4MD’s secure data sharing features.

4.2 MARIO: Service Robot

MARIO (MARIO Project, 2018) project aims at developing a companion robot that builds resilience and reduces loneliness and isolation in older people with dementia. They designed and developed a service robot, which offers effective and efficient intervention by mitigating simple changes in self-perception and mediating brain stimulation towards dementia patients. Technically, they adopt state-of-the art flexible, modular, low cost robotic platforms for providing real and affordable solutions for people with dementia. A novel semantic method (Casey et al., 2016) is developed for generating and integrating knowledge graphs extracted from multiple natural language sources. This method enables human-robot spoken dialogue interaction in MARIO. The service robot has been verified and tested by a large group of dementia patients. The experimental outcome demonstrates that the robot is able to prompt and remind them with various daily activities like eating, drinking and when to go shopping, and social events, family birthdays and anniversaries.

4.3 CAREGIVERPRO-MMD: Gamification Health Platform

CAREGIVERSPRO-MM(CAREGIVERSPRO-MMD Project, 2018) project aims at facilitating people with dementia or chronic diseases to reduce frequency of visiting care institute using self-management tools/systems and thus improve their daily activities and quality of life. The key feature of project mainly includes: 1) providing new services for people living with dementia and their caregivers; 2) user-centric design for patients with mild to moderate dementia; 3) show the clinical and social benefits for patients and caregivers, as well as financial benefits for the healthcare system. A social platform is established based on Gamification Engine Architecture to monitor users’ behaviour and provide non-pharmacological interventions. The proposed framework has five layers: socialization, education/training, treatment adherence, monitoring and non-pharmacological interventions, which will increase the interest of end-users in using the game-based platform.

4.4 DEM@CARE: Remote Health Management Solution

DEM@CARE (DEM@CARE, 2018) project has developed a closed-loop management solution for people with early or mild-stage dementia through multi-parametric remote monitoring and individually tailored analysis of physiological, behavioural and lifestyle measurements. The loop covers people with dementia, their informal caregivers, and dementia clinicians. The project is categorized into three specific situations. Dem@Lab assesses the cognitive state of participants. Dem@Nursing improves care by monitoring behavioural patterns and symptoms in nursing homes. Dem@Home has demonstrated a positive impact on people with dementia living independently at home. For activity recognition, with ambient sensors, wearable cameras and fixed cameras, a hybrid framework is proposed between knowledge-driven and probabilistic-driven methods for event representation and recognition with an approach that capable to handle noise and ambiguity.

4.5 PRODEMOS

PRODEMOS (PRODEMOS, 2018) is an project aiming at designing and developing an evidence-based dementia prevention strategy using mobile health accessible to those at increased risk of dementia without taking preventive medicine. This project will finally implement a flexible, adaptable m-health platform in a culturally appropriate form in the worldwide. The most important feature of this platform is its evidence based, which means that they will collect a purposive sample of potential participants in both settings in semi-structured interviews to explore cultural, socio-economic and educational barriers and facilitators of dementia prevention. The main topics in this project include some general concept of self-management of risk factors to prevent dementia, and views on the acceptability and usability of m-Health platform prototype for sustained lifestyle behaviour change.

4.6 AI-Mind

AI-Mind (AI-Mind, 2021) project is aiming to build a European platform to shorten dementia risk prediction. Normally, the evolution from MCI to dementia is five years with clinical follow-up. AI-Mind will shorten this journey to one week by integrating AI-based tools into clinical practices, using data from the connector, advanced cognitive tests, genetic biomarkers to important textual variables. The project is a significant breakthrough in dementia assessment, which will improve the healthcare system and boost the innovation. The consortium plans to deliver a medical device of class 2b that can reach TRL7 by the end of the project.

4.7 Brain Health Toolbox

Brain Health Toolbox (Brain Health Toolbox, 2019) is a project funded by the European Union that is to create a strategy to develop accurate dementia prediction and effective prevention. The project makes use of disease models and predictive tools to conduct prevention, preventive treatment trials, and connect non-pharmacological and pharmacological methods. Disease models and prediction tools are multi-dimensional with a wide range of risk factors and biomarker types. Machine learning is explored to analyse the most important factors to an individual's overall risk level. The Brain Health Toolbox will cover all patients in the preclinical stage of disease, and it will provide tools for personalised decision-making to prevent dementia.

4.8 Demo

DEMO (DEMO, 2016) is also an EU funded project that aims to find new biomarkers in brain scans to diagnose dementia related neurodegenerative diseases through Magnetic Resonance Imaging (MRI). The project to improve this process by identifying and developing quantitative imaging biomarkers that are easy to see on MRI scans, and thus using them to simulate the possible progression of the disease. By using historical and longitudinal data from patients with dementia, the project makes use of machine learning approaches to discover changes in brain structure and function, providing predictive capabilities for the appearance and progression of the disease. The outcomes not only can patients be diagnosed at earlier time as well as to provide a more accurate and objective prognosis, but the intervention for dementia can also be evaluated.

The listed projects imply that a large number of e-health related research works have been carried out in aiming at different techniques and end-users of dementia care, from improved quality of life, assistant robot to self-management. Apart from above projects, some other dementia related projects (MinD, 2018; RAMCIP, 2018) are also represented. As shown in Table 4, we match these projects with our targeted wearable intelligence techniques in dementia care for dementia patient.

Table 4 Project and applications related to wearable intelligence for dementia care
Full size table

5 Research Challenges and Future Trends

Although empowering the utility of multimodal wearable intelligence enabled technologies for dementia care has huge potential benefits, it is still widely agreed that the wearable intelligence technologies are in their infancy and face many challenges. Future efforts are still required to address these challenges and examine of availability of existing wearable intelligence technologies to ensure a good fit in the dementia care.

5.1 Expectation of Wearable Intelligence System for Dementia Care

Personalised

The information collected from various wearable devices is expected to be a life-long experience record of the individual older people with dementia that offers useful input to generate effectively personlised memory training programmes and useful recommendations to maximize their independence in daily life.

Unobtrusiveness

The dementia care system typically has a set of unobtrusive wearable accessories and devices for long-term and continuous monitoring life-logging personal information of old people with dementia. These wearable accessories and devices are expected to be comfortable and miniaturized to elderly users, enabling accurately recognizing individual user’s information like activities, behavior.

Intelligent

The knowledge discovery and intelligent reasoning components of the dementia care system will offer the power of intelligently personlised advice and guidance through applying the predictive models and data analysis algorithms to life-long experience related information. The use of these personlised machine-generated advice will improve older people with dementia’s ability to remain active and independent in daily life. These are expected to improve their quality of life by offering healthier lifestyle, reliable activity management, active social participation and better overall wellness.

Integrative

The wearable intelligence system for dementia care will be an integrated interoperable mobile wearable solution, which include the design and development of hardware, algorithm, software, interface and applications. The integrative approach will not only provide independent living solutions and tutoring care for people with dementia, but also enable their caregivers or relatives to easily access and monitor multiple aspects that are influential to these elderly people’s overall wellbeing.

Intuitive

The multi-modal adaptive user-centered interface of the dementia care system will offer the intuitive and tangible interactions to older people with dementia, for effectively accessing their memory training applications and easily acquiring and understanding the personal recommendations with memory cues.

Affordable

Wearable intelligence system should feature the adaptation of a range of latest wearable intelligence technology with the aim of improving the effectiveness of the systems and enhancing elder people‘s independence and engagement. Its success will significantly reduce the burden of caregivers and the financial pressure of elderly people and their families. This will provide a new dimension in healthcare and will enhance the sustainability of healthcare systems for dealing with challenges such as the increasingly ageing population in Europe.For many elderly people with dementia, they still provide meaningful contributions to their communities – financial, provision of social support and care, volunteering and what may be described as ‘social glue’. However, loneliness and social isolation, common in later life, are risk factors and may lead to their physical functional decline and emotion changes. Wearable intelligence system will enable these individuals to live independently and to remain physically and mentally active as they age.

Meanwhile, as a result of this ageing of the population, healthcare and associated social welfare costs are growing exponentially and they will soon become unsustainable unless we change the way in which people are supported. In many cases, there is a need to shift dementia care from caregiver-oriented passive care to self-management lead active ageing.

5.2 Technical Challenges

Cost Effective and Non-obtrusive Wearable Sensing

In the last decade, advanced wearable sensing technologies have been increasingly invented and proven its popularity among general users, their majority usages are limited in many fitness and wellbeing applications, but not specific disease monitoring and care. One key issue is that due to low-cost, these wearable products are mostly designed as consumer electronics that only enable capturing raw sensing data with simple processing. Owning to diversity of individual life pattern (normal person, elderly people and people with dementia) and environmental noises, personal data from these wearable products exhibits remarkable uncertainty in the natural environment such as battery, capacity issues and placed positions. The results are widely divergent when the mobile phone is put in the pants pocket from handbags. It is necessary to design cost-effective and non-obtrusive wearable sensing devices for older people.

Notably, advance wearable sensing devices are limited in terms of their size, fast response, continuous monitoring capability, wireless data transmission, and non-obstructive user experience. There is usually a tradeoff between high quality and low-cost of developing useful and reliable wearable sensing technologies. The idea candidate of future sensing technologies for wearable intelligence enabled dementia care should be a tiny sensor into personal daily use items, including but not limited to clothing, watches, glasses, shoes, belts, and so on. Also, many non-obstructive sensing devices used in chronic diseases monitoring are key to success of future dementia care and will potentially bring a lot of convenience to patients with dementia.

Secured and Trustful Mobile Health System

Future system design of wearable intelligence technologies in Healthcare 4.0 is shifting from open, small, and single loop to closed, large and multiple loops. It means that the entire wearable intelligence system should be well-connected, mobile, secured and highly trustful. Especially towards people with dementia, their health information (e.g., phenomena, health condition, emergency) is relatively higher sensitive than normal users, any inappropriate disclosure may violate user privacy and even result in property loss. Thus, how to design appropriate security and privacy protections in multimodal wearable intelligence system for dementia care is a challenging issue. Meanwhile, the costs of security protections vary with end-users’ diverse needs and may impact their user experiences on wearable health applications. Complicated encryption technique might be a useful solution of offering users more security guarantees but with higher computational overheads and latency than lightweight ones. To satisfy dementia’ patentors specific requirements on security and privacy protection, quality of protection has become a emerging security concept in multimodal wearable intelligence systems.

Effective Data Validation

Targeting at completed free-living environments, multimodal wearable intelligence systems for dementia care will collect massive personal health data from diverse wearable device. These heterogeneous and raw data exhibits remarkable uncertainty due to multiple environment noises and impacts. It is important to validate these data with an improved reliability before utilising them in decision making. How to effective validate these health data requires intensive experimental verification with statistical analysis and probably advanced intelligent algorithms.

Intelligent Data Analytic

Towards multimodal data analysis, a large number of traditionally machine learning models have been proposed in dementia diagnosis and care. One typical challenge in data analytic is how to get large-scale well-labelled multimodal data, as machine learning approaches require sufficiently large number of samples for model training, in which supervised learning methods need to be set appropriate categories ahead of time, and each sample needs to be labelled. Secondly, running machine learning models in large-scale datasets require huge computational resources in remote servers. How to choose or design lower complexity of models or algorithms for supporting multimodal data analytic is an important question. Lastly, only a few current attentions have been devoted to training model with raw data from free-living environment. It is worthy investigating new algorithms or models applied in real life with many uncertainties.

6 Future Research Trends

Lifelogging Mode

One important feature of multimodal wearable intelligence environment for future dementia care in Healthcare 4.0 is that the collection of long-term life-logging personal health data becomes possible, named as lifelogging mode. Considering limited memory and power resource in most affordable wearable devices, lifelogging data will not be milliseconds-based raw sensory signal, but like more fragmented window set in minutes/hours. The lifelogging mode leads to transformation of time-series sensing data processing to longitudinal data analysis. Thus, how to effectively design and transfer available machine learning models into life-logging health related data, how to explore new feasible algorithms for training these life-logging data set, what kind of features in these life-logging data potentially leads to the best accuracy, etc. are all valuable research topics in this area.

Free-Living Environment

Another key feature of multimodal wearable intelligence for dementia care in Healthcare 4.0 is to target at providing support or services to dementia patient in completely free-living environment. This follows a global trend of population ageing where the transformation of traditional hospital-based healthcare services to patient empowered home based healthcare services. Thus, one possible future research direction is to explore how to achieve high accuracy and stability of health data acquisition with multimodal wearable intelligence technologies in free-living environments. Also, it will be interesting to study how to overcome the barriers of short-battery or poor capacity and time-consuming of running machine learning algorithms.

High Volume and Multimodality of Data

The heterogeneous devices connected in future wearable intelligence system will be driving major expansion in big data of dementia patients’ health information. They contain not only a sheer volume of long-term personal lifestyle associated data, but also complex and rich context of other health information. So future research work on how to explore these high volume and multimodality of health related data under wearable intelligence platforms for bringing intelligence for more solid clinical decision-making and policy formulation will be significance.

Security and Privacy

Towards providing dementia care in Healthcare 4.0, future multimodal wearable intelligence systems should be based on a heterogeneous and distributed network to store and manage dementia patient health data. Typically, security and privacy in any wearable intelligence related networking architecture will be naturally inherited to distributed mobile-cloud systems or applications. Compared to existing wearable devices or service providers with data protection schemes on their standalone server like Fitbit, etc. protecting privacy and security in future multimodal wearable intelligence systems will be potentially more difficult as the number of potential attack vectors on wearable intelligence entities is much larger. Future research work on how to protect security and privacy needs to be carried out in this field.

7 Future Multimodal Wearable Intelligence System for Dementia Care in Health 4.0

7.1 Multiple Supportive Functionalities

The future wearable intelligence system for dementia care could be designed as a personalised coach that can provide multiple supportive functionalities to older people with dementia in daily life, rather than a single-function aid for certain circumstance. Traditionally, memory supportive strategies have been widely developed as single-function aids like memory rehabilitation or memory reminder to dementia people for a long time but are still very poorly explored in how to maximize its functionalities for a better quality of life of dementia people, both in the degree of impact and the scale of implication. Principally, the cognitive and memory training in dementia people is a complex field of work, which will concern different brain systems and pathways (Gates et al., 2011); for instance, prospective memory is tending to encoding and storing new information like a learning process as opposed to retrospective memory of retrieving and recalling of already stored information in the brain. Cueing strategies are considered as effective priming technique for facilitation of information retrieval; whereas other strategies like chunking, loci, and mnemonics play the role of enhancing encoding and storing. Thus, very limited attempts have been successfully made towards both retrospective memory and prospective memory supporting so far. Wearable Intelligence System will be the first trial for designing and developing a wearable intelligent system to cover both retrospective memory and prospective memory supporting to people with dementia, including a wide range of functionalities of like memory training, guidance in daily difficulties, recommendation for health lifestyle, for helping them undertake daily tasks and preserving their physical and emotion well-beings. These innovative services will be provided in a timely, automatic, intelligent, and flexible manner to old people with dementia; and are expected to add extremely useful values to enhance their living independence and quality of life.

7.2 Personalisation

Another new potential feature of wearable intelligence system is personalization, which are reflected by offering specific memory training programs, customized guidance and recommendations regarding personal life style to older people with dementia. Traditional memory training or supportive tools are relatively disappointing and easily frustrating to users due to dull and tedious content or ignorant and rigid task reminder, like the Method of Loci and Lumosity. The design of these tools is not customized to end users in terms of their interest, emotion, lifestyles or circumstances. Wearable Intelligence System will build its memory services upon a large collection of multi-dimensional experience based lifelogging personal data or information. It can provide more interesting and personalised programs and recommendations to dementia patients. This feature will significantly empower and motivate the targeted users to use wearable intelligence system in their daily life, which will bring a great convenience and help to them.

7.3 Training and Rehabilitation

Finally, wearable intelligence system could be an innovative way for memory training and cognitive rehabilitation for dementia people, which will rely on the extraction and utilization of salient memory cues through aggregating multi-dimensional personal lifelogging data for triggering their remembering. As indicated before, the memory deterioration of dementia people is a very complex filed of work, and hardly solved by single type of training or intervention strategy. Especially, assessing the effects of these strategies needs large volume of training data, testing and validation. Wearable Intelligence System will stand as an opportunity to clinicians and researchers to explore new approaches of memory training and cognitive rehabilitation, and of assess generalization by testing cognitive, behavioral, quality of life, functional, mood, and psychological wellbeing outcomes.

7.4 Technology Innovation

Technology innovation is one of the keys to the success of the wearable intelligence system for dementia care. The design and implementation of wearable intelligence system requires active involvement of the end-users, the collaboration and exploration between different technique partners. The key technology innovation will potentially cover four aspects.A wearable intelligence system could feature an innovative textile based unobtrusive wearable context monitoring technique for automatic collection of multi-dimensional personal life-logging data and information. Historically, the innovation of unobtrusive sensing technologies and wearable devices is closely coupled with the advancements in electronics (Zheng et al., 2014). For instance, the core technologies of electrocardiogram (ECG) devices have evolved from water buckets and bulky vacuum tubes, bench-top, and portable devices with discrete transistors, to the recent clothing and small gadgets based wearable devices with integrated circuits (Zheng et al., 2014). There is a clear trend that the wearable devices are getting smaller, lighter, less obtrusive and more comfortable to wear. Thus, wearable intelligence system will develop an innovative textile wearable device which is capable of monitoring a variety of personal physiological parameters and environment factors, including place and location, people interacted, daily activities, behaviour, objects, temperature, etc.

Wearable intelligence system could also generate some effective lifelogging context-based knowledge discovery and adaptive recommender approaches, for supporting the extraction of useful memory cue and provides intelligent personalised, context-dependent recommendations to older people with dementia. In light of life-logging data mining, a large number of knowledge discovery approaches and intelligent recommendation methods have been used in the social media based user-generated content, like Park et.al proposes a news recommendation system that exploits a graph-based model to analyse the relationship held among user comments (Park et al., 2012); Shepitsen et.al identify correlations among groups of tags by means of a hierarchical clustering algorithm for enhance the quality of tag-based recommended system (Shepitsen et al., 2008). But these techniques are hardly utilised into processing multidimensional life-logging data, due to the diversity and uncertainty of data. Hence, wearable intelligence system could develop a semantic interpretation based knowledge discovery approach to effectively aggregate these multi-dimensional lifelogging data as a meaningful context. This approach will perform semantic analysis and enrichment of lifelogged data, construct a concept space by indexing life-logged data, and select concepts with high-level semantic features as useful memory cues. Though this approach, Wearable Intelligence System will have huge advantages on the quality, accuracy, and comprehensiveness of extracting and selecting memory cues from multi-dimensional personal life-logging data over existing knowledge discovery approaches, especially for multimedia data.

With respect to intelligent reasoning, wearable Intelligence system could utilise some hybrid, contextualized and adaptive recommender algorithms to assist people with dementia in their daily routines by providing suggestions and context specific information. The recommendation algorithm will rely on two types of data resources: personal life-logging data and web-based information. The former one will provide the knowledge of user’s daily life-style or environments to recommendation algorithms; the latter one will ensure sufficient source of information for searching by recommendation algorithms. In fact, making use of the rich information source will allow recommendation algorithms to response the high quality of feedbacks and suggestions. Also, research also suggests many people and patients are actively involved in Internet discussions to discuss their health status (Natalie Armstrong & Powell, 2009; Lorig et al., 2002), hence the internet resource is an effective means of searching suitable suggestions on health style to old people with dementia.

Thus, the recommender algorithms in wearable intelligence System will reason the daily request from the old people with dementia by offering the most useful, personalised, clear feedbacks, in terms of their different circumstances. Wearable Intelligence System also features some tangible, personalised and adaptive user-centered interaction approaches to older people with dementia, for effectively accessing IMC memory training applications and easily acquiring and understanding the personal recommendations with memory cues. Towards ageing population, especially dementia people, cognitive changes and perceptual abilities decline can strongly influence their attitudes on accepting new technologies (Fisk et al., 2009).

To reduce these problems, touch screen interfaces are more frequently being used to assist the technology experience of older adults as they require direct input, require large button targets and eliminate the need for multi-components (Jin et al., 2007). These features mean that items are larger on the screen, making them (a) easier to see and (b) easier to select accurately. Furthermore, the use of virtual buttons on the screen means that older users do not require as much strength to select a target, and they also do not have to divide their attention between the keypad and the screen. Thus, wearable intelligence system should design and develop a tangible interaction interface for mobile devices with touchscreen, like iPad or Microsoft Surface, which can support the facilitation of memory training programs or applications with visual, audio or other tangible artefacts e.g. photographs, videos. Meanwhile, Wearable Intelligence System could also develop multimodal adaptive user interface for mobile devices with capability of exploiting user preferences, stress, emotions and behavior. These innovative user-center interaction approach will strongly empower old people with dementia to use mobile devices or touch screen, and to improve the effectiveness of psychosocial interventions that foster the well-being and quality of life of people with dementia.

8 Conclusions

Focusing on improving quality of dementia care in healthcare 4.0, we give a systematic review on advanced multimodal wearable intelligence techniques for dementia care. Wearable intelligence enabled technology in dementia care will enable faster and safer preventive care, lower overall cost, improved patient-centered practice and enhanced sustainability. In this survey, we proposed one framework for reviewing the current research of multimodal wearable intelligence techniques, and key enabling technologies, major applications, and successful case studies in dementia care, and finally points out future research trends and challenges. We address some fundamental problems related to human factors, intelligence design and implementation, and security, social, and ethical issues. It will be helpful to researchers with different backgrounds in further exploring wearable intelligence for dementia care in Healthcare 4.0.