Clinically-validated technologies for assisted living

Abstract

One of the most important lifestyle risk factors for many chronic conditions in the older age, low physical activity has shown to have significant impact on the sustainability of national welfare in many developed countries. Technology-based assisted living solutions can effectively be used to enable older adults to optimise their health-related quality of life, as well as to promote an active and healthy longevity. This paper describes vINCI—an interdisciplinary research project to actively support assisted living for older adults via state-of-the-art assistive technologies—which seamlessly deploys an ambient intelligence environment to integrate wearable devices, networking, software, and personalised services. It entails clinical validation and feedback at home and residential care facilities via a cloud microservices platform. Underpinned by blockchain technologies, multiple wearable devices, apps, and cameras securely capture the anonymised facets of different life events, whilst machine learning models create individualised user profiles to analyse any decrease in the perceived health-related quality of life typically associated with old age. Two controlled pilots are being conducted with 80 participants at older adult facilities in Romania and Cyprus. By incorporating clinical validation and feedback from specialised practitioners, the vINCI technologies enable older adults not only to self-evaluate their physical activity level, but also to change their behaviours and lifestyle in the long-term.

1 Introduction

It has become clear that the European Union (EU) is turning grey. According to the EU’s Directorate-General for Economic and Financial Affairs (2018), the total number of EU citizens will slightly increase from 511 million in 2016 to 520 million by 2070, while the working-age population (15–64 years) will decrease significantly from 333 million in 2016 to 292 million by 2070. That is to say, the number of people aged 65 and above is estimated to double over the next 5 years and the demand for Older Adult (OA) care in the EU will face increasing pressure. This surge in ageing population, incidence of chronic conditions, and memory-related medical problems associated with old age, will impose a substantial increase in expenditures with public social and health services.

The EU, and arguably the rest of the world, will soon need to introduce innovative ambient intelligence technological solutions to relieve some of its healthcare resources and automate, as much as possible, OA care processes. Furthermore, OAs across Europe critically require such solutions to enable them to monitor their Daily Level of Activity (DLA) and keep their independence. Old age is particularly associated with an increased need for social and medical assistance and home and residential care, as well as with changes in the structure of family and community life. Therefore, an integrated platform for promoting active ageing, person-centred and event-driven applications, in a manner that reflects the older subject’s lifestyle, needs, DLA, usual habits, and preferences, has increasingly become a necessity for OA independence.

According to the UN’s Department of Economic and Social Affairs (2015), Quality of Life (QoL) represents an individual’s perception of their position in life in the context of their culture, value systems, and their goals, expectations, standards, and concerns. It comprises a broad range of parameters reflecting individual physical health, psychological state, personal beliefs, social relationships, and even the main features of the surrounding environment and their own person’s relations with it. Assisted Living (AL) technologies can arguably have positive impacts on different dimensions of a subject’s health and QoL (Chen and Chiu 2018; Koumakis et al. 2019): by enabling the pervasive collection of data, they can be used as facilitators for operational optimisation of care services (Zambrano-Montenegro et al. 2019; Mendes et al. 2017).

To this end, this paper describes the vINCI (clinically-Validated INtegrated Support for Assistive Care and Lifestyle Improvement: the Human Link) platform—a clinically-validated ambient intelligence framework—where multiple wearable and ambient devices work together to create an aggregated solution able to capture the various facets of events leading to the decrease in the perceived QoL, as associated with old age. Moreover, we describe here the way different vINCI system components engage in the communication, interaction, and exploitation of data and how they interoperate over using specific open middleware and software services.

The vINCI evidence-based Internet of Things (IoT) middleware delivers non-intrusive DLA monitoring and support for OAs to augment professional healthcare provisioning. By integrating proven business and open-data Machine Learning (ML) models with innovative user-driven IoT devices in so-called kits, vINCI aims to assist both informal and formal caregivers and to provide smart care support at home, residential care facilities, and outdoors. vINCI arguably empowers OAs to keep their independence.

To verify, test (clinical validation), and identify vINCI’s added value, two multi-disciplinary controlled pilots involving OAs are being deployed in controlled environments for real-life use cases in Romania and Cyprus. The vINCI consortium has endeavoured to make data and ML models compliant with the EU General Data Protection Regulation (GDPR). The ultimate vINCI goal is to demonstrate a systematic approach to improve QoL for OAs, whilst ensuring the highest level of quality control, automated monitoring, and data governance. This article significantly extends our initial work (Drǎghici et al. 2019; Spinsante et al. 2019) by comprehensively describing the vINCI cloud microservices platform, with blockchain-enabled privacy, ML-based models and profiles, mobile application and controlled pilots held in Romania and Cyprus.

In the following sections a typical scenario addressed by the project is described, with the framework proposed to effectively assess and evaluate the related parameters. The latter effectively measures and critically assesses the subject’s conditions in relation to their health-related QoL. The paper is organised as follows: Sect. 2 provides the interdisciplinary background to technologies and AL. Section 3 describes the integration of devices and systems to generate a positive impact on health-related QoL. Section 4 presents the technologies integrated within the framework, and how they address the different aspects of interest within the vINCI architecture, subsequently presented in Sect. 5. Section 6 describes the vINCI mobile app and associated market research, followed by the personalised ML-based models in Sect. 7. Section 8 reports preliminary outcomes from the two activated pilots and discusses behaviour models that will enable the clinically-validated services supported by vINCI. Finally, Sect. 9 concludes the paper by summarising the overall aims of the project and the research steps that are being undertaken to fulfil them.

2 Background

Originally coined by the European Commission at the turn of the century, ambient intelligence entails technology-underpinned environments which are unobtrusive, interconnected, adaptable, dynamic, embedded, and intelligent to improve QoL (Sadri 2011; Cook et al. 2009). It has been widely regarded as a key factor to actively support AL for the OA and, ultimately, qualitatively enhance active ageing (Hawes et al. 2003). Through intelligent systems, sensing devices, and clinically-validated practices, technology solutions for AL now enable proactive and preventative approaches to active ageing, thus promoting well-being and transforming each OA into the main driver of their own health and life management (Van Den Broek et al. 2010; Ruan 2019; Zambrano-Montenegro et al. 2019).

On the one hand, AL is widely associated with residential care facilities and their inherent philosophy of care for OAs. It encompasses accommodation arrangements with support staff and meals, typically coupled with assistance for routine activities such as bathing and dressing. Diverse studies have historically shown that AL improves the QoL in OAs (Mitchell and Kemp 2000; Ball et al. 2000), nurturing their autonomy, privacy, and independence.

On the other hand, Merriam-Webster (2020) defines assistive as “providing aid or assistance” and, more specifically, “designed or intended to assist disabled persons”. Consequently, vINCI is developing ambient intelligent assistive technologies and services for AL, which will not only be clinically validated by practitioners, but also aid OAs in residential care and outdoors. vINCI technologies and associated services seamlessly monitor health, environment, and DLA, and then naturally interact with OAs and practitioners.

One of the most relevant features of the proposed vINCI platform is the promotion of Physical Activity (PA) in older adults, which is recognised to be one of the most important drivers of healthy ageing and well-being. The benefits of PA have been extensively documented especially in younger population and recently with a growing interest among older age groups. Physical inactivity is the fourth leading cause of death worldwide (Kohl III et al. 2012) and is considered to determine 9% of premature mortality, while elimination of physical inactivity would remove 6–10% of major non-communicable diseases such as coronary heart disease, type 2 diabetes mellitus, and breast and colon cancers (Lee et al. 2012). PA is the only intervention unfailingly demonstrated to attenuate functional decline among seniors, as evidenced in several studies (Pahor et al. 2006; Nelson et al. 2004; Cress et al. 1999; Miszko et al. 2003). There is evidence and also growing interest in research on other benefits of PA in old age, such as protective effects on cognitive function (Fleckenstein et al. 2015; Forte et al. 2013; Phillips et al. 2014; Kirk-Sanchez and McGough 2014), negative correlation with depressive symptoms and positive effects on well-being (Du et al. 2015; Yoshida et al. 2015; Black et al. 2015; Vagetti et al. 2014), reduced fear of falling and improved balance (Howe et al. 2011; Kendrick et al. 2014; Medical Advisory Secretariat 2008; Greenwood-Hickman et al. 2015), and prevention of fractures by increasing muscle strength, balance and bone mineral density (Moreira et al. 2014; Borer 2005; Marcus et al. 1992). Regular aerobic PA combined with resistance exercises have been recommended as beneficial for older adults who are frail or at high risk of frailty (Liu and Fielding 2011; Paterson et al. 2007).

Nonetheless, reliable data on PA levels among general older populations is insufficient. The Eurobarometer study reported that 2/3 of the adult populations of the European countries are insufficiently active for optimal health benefits (Sjöström et al. 2006). A recent systematic review on prevalence of various types of PA in older adults, and the proportion of older people whose PA levels meet the guidelines, has also indicated that a substantial number of older adults do not engage in adequate PA to endorse their health (Sun et al. 2013). For maximum and optimum benefits, PA interventions should be individualised and centred on specific components to enhance functional capacity and maintain independence (Paterson et al. 2007). Effectively increasing the PA level constitutes a challenging task, especially in elderly, since intrinsic and extrinsic limiting factors are accumulating with age. Some of the most important impediments of PA in seniors are: pain, discomfort and mobility limitations related to various comorbidities, fear of falling, lack of motivation, depression, failing to understand the benefits, access difficulties (such as environmental barriers), social influences (social awkwardness, feelings of imperilment, etc.)

The vINCI platform aims to tackle such a challenging task, by increasing the older adults’ awareness of their own health-related QoL, through the delivery of personalised and timely advice and information, and by supporting their engagement in PA through, for example, social interactions with peers, to fulfil common goals and share attained results.

2.1 Contribution

While the healthcare literature has previously reported the use of technologies such as sensors (Alemdar and Ersoy 2010), distributed systems (González-Vélez et al. 2009), and activity detection (Ni et al. 2015), the vINCI approach is arguably novel as it systematically integrates the different data streams from distinct devices in order to expose and then create adequate interactions with the sensed data. Iteratively, the sensed data is used to monitor individual conditions by constructing personalised models, which is further used by trained medical practitioners to evaluate the living context against a set of potentially negative conditions. The vINCI technology in fact affords the possibility of identifying the user’s context data (i.e. location) by exploiting specific devices (i.e. smart watches). Should a user be detected to exhibit an unexpected behaviour and he/she is unable to call for help or if a user is disoriented, then the designed system detects such behaviour and acts accordingly.

The vINCI technology is designed to be provisioned in two forms: for private and for business end users. The former are the elderly and their formal or informal caregivers, who can benefit from the services and functionalities supported by vINCI, in terms of QoL improvement (direct benefit) and reduction of the assistance burden (indirect benefit). The latter are intended as care institutions, nursing homes and communities, that provide assistance services in the market and can afford the use of expensive devices such as the depth cameras required for motion and gestures monitoring.

For the private individuals/end-users, the technology basically comes in the form of a mobile app. Within the vINCI context, we will use the the FDA’s Center for Devices and Radiological Health and Center for Biologics Evaluation and Research (2019) definition: a Mobile Application, or simply an app, is defined as a software application that can be executed (run) on a mobile platform (i.e., a handheld commercial off-the-shelf computing platform, with or without wireless connectivity). Currently, the vINCI project intends to only distribute the platform and does not intend its platform to be used or classified as a medical device.

Arguably, the added value of vINCI with respect to other platforms developed within research projects focused on AL is its clinically-supported validation of the algorithms enabling the detection of anomalous evolving conditions in OAs, based on the observations regarding their DLA status, social behaviour, and health-related parameters.

3 Target scenario

3.1 vINCI aims and expected audience

Following a user-centred design paradigm, the vINCI technology builds upon a close collaboration with potential users: intake questionnaires completed by OAs (persons over 65 years old), have allowed us to understand their needs and possible support solutions. Unlike other assistive technologies for AL typically found in the literature (Rashidi and Mihailidis 2013; Calvaresi et al. 2017), vINCI does not make the tacit assumption that it is possible to predict the medical conditions of a person. Let us face it, all OAs suffer from various medical conditions, that is a fact of life associated with old age. Instead, the focus is on how to better assist OAs and, by continuous monitoring them, to find out—by means of so-called Anomaly Detection (AD) approaches—when a serious deterioration of their usual conditions appears, and then suggest them to visit their General Practitioner (GP). So, the whole project evolves around ML models, by which the platform learns how the OA’s normal (meaning usual) conditions look like. It triggers advice so that the GP should be visited (no direct conclusions about an individual’s health status is drawn).

Within vINCI, there is significant work underway to incorporate inbuilt evidence-based, clinically-validated functionalities which will eventually create standards for development and design of app frameworks and their benchmarking within the AL domain. More specifically, the vINCI app is arguably useful for personal monitoring: the app gathers basic individual details via simple questionnaires and then acts as gateway for secure data communications coming from the wearable and ambient sensors—possibly integrated within the platform—and selected according to the specific individualised needs and preferences. At the same time, a web-based dashboard is available, where the users can securely access their personal profile (how they are performing), but also authorised family members and caregivers can be involved, as monitoring friends.

The whole technology is flexible and cost-efficient, as an individual is able to select what (if any) hardware to use (for example, the user could buy the Smart Insole (SI) and the watch, but opt out of the use of a Fitbit¹ wrist band). The vINCI mobile app itself is also sufficient for the basic monitoring (to input answers to the questionnaires, for example, and receive feedback advice). From a business perspective, we see the potential for partners to invest a little more in medical monitoring technology. For such a partner, the option to buy a depth sensor camera might be feasible, if that camera can monitor an entire room full of people to be cared for. Consider the case of a residential care facility for OAs: the management could install such a camera in the living room, to monitor all residents and have an objective way to gather the DLA (or, the opposite, sedentary behaviours, which in medical terms is the first symptom associated prior to depression, a common problem associated with old age) each resident is having.

3.2 Basic workflow

vINCI aims to connect multiple innovative user-oriented devices together, in different combinations according to the identified user’s needs and expectations: smart watches to locate the user and monitor outdoor activities, smart insoles that count the steps and monitor the level of indoor PA, depth cameras that record DLA, and tablets on which the users can answer surveys and questions about their mood and disposition. All the data are aggregated, anonymised, and analysed by a web platform to generate and continuously update a complete overview of a subject’s health status, QoL and DLA, to also support smart care for OA in clinics.

As shown in Fig. 1, the vINCI platform gathers data from heterogeneous sources:

• Clinical questionnaires filled in by individuals: an app developed for a tablet used by a patient in a care facility and the operators will send data to the collecting module using any available internet connection, either Wi-Fi or Long Term Evolution LTE;

• Smartwatch: the data produced by the smartwatch is obtained via an Application Programming Interface (API) available at the company proprietary platform (an appropriate interface will be implemented in the collecting module);

• SI: the data originated from the SI will be obtained first through a gateway represented by a smartphone with an internet connection. The smartphone will gather data from the SI locally, over a Bluetooth Low Energy BLE link, and relay it to the platform, by exploiting a Wi-Fi connection;

• Depth camera: the RGB-D data generated by the camera are locally processed by an edge computing device connected to it (typically through a cable), then the pre-processed information is sent to the remote vINCI collecting module over an internet connection (Wi-Fi).

When the smartwatch or the smart insole are registered into the system through the vINCI app, they are associated to the person to whom the corresponding account belongs. The same person is also requested to upload a picture of his/her face which will be used for identification purposes. On the other hand, when the RGB-Depth camera captures a picture of the subject’s face, during the initial configuration phase, the same is sent to the backend where the corresponding stored picture of the subject is searched and matched to the one received. From that moment on, the backend system will be able to properly associate all the sensor-generated data to the right user.

Fig. 1

High-Level overview of the vINCI Platform. Different devices produce data streams as distinct sources (smart watches, smart insoles, cameras, etc) which are securely collected and processed by the vINCI platform, which in turn enables services (medication alerts, PA programs, etc.) via ML models

The ultimate goal is to analyse the data collected from different devices, process it and use the results to improve the health-related QoL of the monitored subjects, thus enabling their active and healthy ageing. The involvement of the user (possibly together with a relative or a caregiver) and the integration of different devices into the platform take place according to a defined workflow, to ensure proper user’s engagement and awareness about the technology. Suitable questionnaires to identify and understand users’ needs, developed based on vINCI partners’ experience, on the objectives of the vINCI project, and on the outcomes of a thorough review of the relevant literature, are administered to the elderly and their informal/formal caregivers. A typical workflow should include the following steps:

• 1: When enrolling into vINCI, the user (or a relative) is presented first with the graphical user interface of the vINCI digital application. Personal data for user registration is entered, through a Web-based form, accessible from various devices. The personal data requested by the registration process includes name, age, any known prior clinical conditions. This is the kind of baseline data used to generate the first profile of the user. Then, the QoL before and after the subject uses the vINCI technology for a various period of time is compared. At this step, the user and the relative (or caregiver) receive detailed information and explanations about the type of technologies the system requires to use, and the type of impact they will produce on the user’s daily life, in order to remove possible barriers or difficulties in accepting and incorporating the vINCI technology into the senior’s life.

• 2: Next, the user is presented with a questionnaire based on the WHOQoL-Bref (Programme on Mental Health 1996) to determine their perceived health-related QoL index. The administration of the questionnaire through the app, as shown in Fig. 2a, will need to be repeated periodically throughout time, to re-evaluate the status of the subject (and adjust accordingly the different scales for vINCI monitoring and alert triggering). The vINCI platform offers individualised specific feedback for each person in the form of written text on the tablet screen. When the overall health-related QoL and/or a specific domain are sub-optimal, a specific feedback will be displayed with advice for improvement.

• 3: From this point on, when outdoor, the user will wear a suitable smart watch (e.g. the CMD One smartwatch) see Fig. 2b, and the SI when indoor, as shown in Fig. 2c. Based on the selection of the technologies performed by the user, home monitoring by RGB-Depth cameras when performing exercise sessions can be possible, in semi-controlled configurations to avoid occlusions.

By means of activity tracking algorithms based on RGB-D frames, the system will track the person while at home, to monitor the DLA and detect possible sedentary behaviours. Next, direct feedback with specific recommendations will be provided. These recommendations will be constructed based on scientific evidence and existing clinical guidelines for OAs. The piloting and testing of the vINCI technology is being performed in a clinical settings on a sample of OAs admitted to the National Institute of Gerontology and Geriatrics “Ana Aslan” (NIGG)—a vINCI partner—as patients for various chronic conditions. In order to measure their sense of activity and social levels, each patient is periodically asked to complete the D-VAMS2, a nonverbal mood assessment instrument (Barrows and Thomas 2018), and the International Physical Activity Questionnaire (IPAQ) short form for OAs questionnaire (Hagströmer et al. 2006).

Fig. 2

Illustration of the key technologies an OA is accessing as part of the vINCI pilots: a the vINCI mobile application; b wearable device: smartwatch; and c SI

The aforementioned technologies, i.e. depth cameras, watch, SI, electronic questionnaires, all serve to get data related to a monitored subject. However, in order to correctly interpret that data, appropriate clinically-validated models are mandatory. In the planned clinical pilots the project will collect the actual clinical facts about the user (how the user is feeling), and they will be mapped against the sensed data collected from the sensors, to construct supervised ML models. From that moment on, the user will be continuously monitored via the sensor technologies integrated within the platform. The clinical validation pilot study is designed as a randomised controlled study and will include a study and control group. The vINCI parameters will be compared between groups, and before and after the subjects use the vINCI technology for a period of time. The primary outcome is the impact of the vINCI technology on the health-related QoL and PA levels.

At some interval (after some days or months), the user is asked to re-enter data into the questionnaire, and re-visit the clinical facility to be examined again (Hagströmer et al. 2006). This data is used to update and adjust the trained model of the user profile.

4 Technologies

Different components of the vINCI technology are integrated to offer a platform for early identification of a decreasing QoL and well-being in a step-wise manner. One of the most important features of the vINCI technology is that a user can independently identify risky conditions within his/her own status by receiving a direct feedback with specific recommendations. According to the operation workflow detailed in Sect. 3.2, OAs are able to periodically self-complete a questionnaire that assesses their QoL across multiple domains. The scoring algorithm generates a direct feedback: if the evaluated QoL index is optimal, the received feedback will be delivered in the form of a message like:

“Congratulations! Your quality of life is very good! You can take this test again after 6 months or any time your status changes”.

If the QoL scores below the optimal level, the evaluation of the psychological well-being, subjective and objective assessment of the PA levels is offered in a step-wise process, exploiting the integrated consumer technologies. Because mood disorders and the user mood at the time of completion can influence the QoL score, the next step consists in evaluating the psychological well-being. When the QoL and the psychological assessment results are sub-optimal, the user is then invited to further proceed with assessment of PA, through the vINCI platform, by receiving a proper alert from the vINCI app, as shown in Fig. 3.

Fig. 3

Alert issued by the vINCI app to remind the user to stay active

About the data input by sensing devices, the following applies to the SI, the depth camera and the smartwatch.

4.1 Smart Insole (SI)

Despite the common usage of sensors available onboard smartphones to detect and identify different activities (Pires et al. 2018), in the framework of the vINCI project it has been decided to resort to the use of external and specific sensing devices, based on the analysis of the final user requirements and expectations.

The original plan of activities of the vINCI project was designed around the idea of providing the users with smart shoes, i.e. shoes specifically designed for OAs by one of the consortium partners, and additionally equipped with electronic components, in order to be able to capture data about the user’s mobility, in daily situations in the wild  (Montanini et al. 2018). However, once the project started and following initial discussions with NIGG experts, it was decided to test a SI, instead of a smart shoe, i.e. a removable part of the shoe, equipped with smart sensing technology that could be inserted into preferred shoes of an OA. In order to accommodate such a request from the vINCI partners, it was decided to address the design of a SI that could be also re-used to engineer a smart shoe, as originally planned by the companies participating to the project.

The decision to have a in-house design of the SI was also motivated by the fact that, performing a deep and careful analysis of the devices available on the market, a few potential candidates were found, which were however not suitable for the project. As a matter of fact, devices such as Digitsole SI² are designed for runners and people interested in monitoring their performances during physical activities, and they require a mandatory calibration procedure, with the user running on a treadmill, at different paces, for a specific amount of time. This would not be possible for the older adults targeted by the project and NIGG’s team.

The following most relevant requirements and critical constraints were collected after the initial discussions with users:

• The insole should not require the user to wear anything applied on the ankle, in order to avoid causing potential problems during walking, possibly leading to a fall;

• The insole should be as much easy to use as possible, with minimal intervention requested to the user; it should be possible to wear the SI the same way as a normal one;

• The electronics onboard the insole should be less power demanding as possible, in order to allow the battery to be recharged not so frequently, and able to sustain the data transmission during usage;

• It was recommended to use a rechargeable battery, possibly by a USB cable, as it happens with most of modern battery-powered devices.

The vINCI SI exploits a Bluetooth Low Energy (BLE) communication interface, suitable for an indoor scenario. A proper transceiver is embedded within the electronics, to convert the data generated from the sensors into packets to be transmitted to a BLE receiver, such as a smartphone, for further relaying to a remote server.

Each user is provided with a single SI, shown in Fig. 4a, equipped with three e-textile sensing spots buried within the SI and connected to an embedded electronic board, enabled with BLE, so that the device can be battery-operated for a reasonable and effective amount of time. The sensing device exploits the radio interface to transmit the data generated by the onboard firmware to the remote server, through a data aggregator (a smartphone or a tablet). Each message transmitted by the device onboard the insole contains a numerical value describing the activity state recognised by the device itself, i.e.: non-contact, running, walking, standing (Spinsante and Scalise 2018). Each state is identified by a different numerical label, sent to the remote server at a state change. When the SI is worn by the subject, as shown in Fig. 4b), an initial time interval is used to set the proper condition label; from that moment on, every time the condition changes (e.g. from standing to walking, and back to standing) a corresponding packet is sent to the remote server. That is to say, by collecting the data packets transmitted over a given amount of time and checking the timestamps applied by the server, it can be possible to draw the time distribution of the PA performed by the subject, i.e. how much time has been spent in each state over the whole observed time, as shown in a simplified fashion in Fig. 5.

Fig. 4

a The SI used within the vINCI project, b The SI inserted in the user’s shoe

Fig. 5

Pictorial representation of the time spent by a user in a give status (walking, standing, sitting, not wearing) over a certain observed period

4.2 RGB-Depth camera

The use of sensors and cameras has been widely investigated within the literature, in different applications dedicated to fall detection (Boudouane et al. 2020), activity tracking (Alemdar and Ersoy 2017), and in general motion recognition (Gasparrini et al. 2014; Montanini et al. 2018; Spinsante et al. 2018).

The purpose of the RGB-Depth camera in the vINCI architecture is to record and send RGB and Depth (D) images, and related JSON objects with information about the registered human skeletons, to the cloud. Within the vINCI platform, the information provided by the RGB-D device is used by several microservices, such as the Camera Fitness Microservice and the Camera Movement Microservice, along with the Camera Storage Server, the Activity Recognition and the Face Recognition applications. The former service is designed to monitor the physical exercises a person is performing in front of the camera; the latter is designed to monitor the movements of the older adult in the living environment where the sensor is located.

Under these conditions, we approached the problem from the perspective of image processing. But these images are not the initial frames, but a way to compress a video clip. The basic unit for generating such an image is the skeleton, and a sequence of at least six skeletons can generate an image that in the literature is called SkeleMotion (Caetano et al. 2019). An innovative method, covering the shortcomings that existed until then, based on the calculation of two metrics that have an important role in the process of movement, namely, orientation and magnitude. Moreover, to get the best results, we combined these metrics with different time distances.

The size of the matrix S, a chain c of concatenated skeleton joint coordinates from frame t corresponding to the SkeleMotion image, is $C \times T \times 3$, where C is the number of joints and T is the number of frames. To keep the connection between the body wrists, we made a deep traversal of the skeleton, starting from the point corresponding to the torso, obtaining a vector of 37 wrists (C). The number of frames corresponding to the final image will always be $100\; (T)$. The third dimension of the 3D matrix, the one resulting from calculating the orientation (direction of the joints) and magnitude (speed of movement), is 3 because there are 3 coordinate axes corresponding to the 3 colour channels. The value of each cell in the motion structure ${\mathcal {D}}$ matrix is calculated as a time distance using Eq. (1), the orientation and magnitude being calculated as the difference of the values between two frames that are at distance d.

$$\begin{aligned} {\mathcal {D}}_{(c,t)}=S_{(c,t+d)}-S_c \end{aligned}$$

(1)

Equation (2) provides the mathematical support for implementing the calculation of magnitude M and Eq. (3) for calculating the orientation $\theta$. The former is calculated as the square root of the sum of the squares of the temporal distances corresponding to each coordinate axis:

$$\begin{aligned}&M_{(c,t)}=\sqrt{({\mathcal {D}}_{(c,t)}^x )^2+({\mathcal {D}}_{(c,t)}^y )^2+({\mathcal {D}}_{(c,t)}^z )^2 } \end{aligned}$$

(2)

$$\begin{aligned}&\quad \theta _{(c,t)}=(\theta _{(c,t)}^{xy},\theta _{(c,t)}^{yz},\theta _{(c,t)}^{zx}) \nonumber \\&\quad \quad \quad \quad \qquad \qquad \qquad where: \nonumber \\&\quad \theta _{(c,t)}^{xy}=\arctan {(({\mathcal {D}}_{(c,t)}^y)/({\mathcal {D}}_{(c,t)}^x ))} \nonumber \\&\quad \theta _{(c,t)}^{yz}=\arctan {(({\mathcal {D}}_{(c,t)}^z)/({\mathcal {D}}_{(c,t)}^y ))} \nonumber \\&\quad \theta _{(c,t)}^{zx}=\arctan {(({\mathcal {D}}_{(c,t)}^x)/({\mathcal {D}}_{(c,t)}^z ))} \end{aligned}$$

(3)

For sitting and standing activities, the unbounded calculation in Eq. (3) for the orientation can generate errors, as there are no spatial differences. Therefore, it is necessary to set a limit value for which the orientation will be calculated, otherwise it will be 0.

The specific use of RGB-D sensors enables the availability of D measurements which in turn allows to support applications dedicated to remote motion and gestures monitoring (such as tele-rehabilitation). Furthermore, the privacy-preserving D stream allows to extract relevant information without infringing the subject’s privacy otherwise revealed by RGB frames.

Data provided by the depth sensor depends on the configuration according to which the sensor is used. If the depth sensor is located in front view with respect to the subject, by extracting the skeleton (or body joints’ coordinates) collecting and processing them, it is possible to obtain an objective representation of a movement performed by the subject, and compare the way the movement is executed, in different moments.

Figure 6 schematically shows the way the RGB-Depth camera is connected to the remote server within the vINCI architecture. The proposed solution includes several components: the sensor, which periodically transmits data about people captured in the field of view, and some specialised microservices deployed in the cloud, each with its own responsibilities.

The sensor chosen for the vINCI implementation is the Orbbec Persee³ depth camera, which, with an integrated mini PC, allows the periodic sending to the cloud of data about people in the room, such as synchronised D and RGB frames, along with JSON files that contain details about each person’s position inside the room, and their body coordinates (skeletons). The Orbbec Persee depth camera is capable of capturing depth images at a resolution of $640 \times 480$ pixels, as well as RGB images at a resolution of $1280 \times 720$ pixels. To facilitate the identification of the body posture captured in the images, thus increasing the usefulness of the data extracted by the camera, it was decided to use an external Software Development Kit (SDK), namely Nuitrack, that allows the identification of the position of each person in each frame, together with their body coordinates.

Fig. 6

RGB-Depth camera integration into vINCI architecture: proposed solution

Basically, using the depth images captured by the camera, Nuitrack is able to extract the coordinates of 19 important joints in the human body, such as the head, shoulders, hips, knees, wrists, etc. These coordinates are represented by the distance in centimetres of each joint relative to the X, Y and Z axes, as well as their Cartesian coordinates, in the form of percentages, relative to the depth image and the RGB image. In addition to the coordinates of a particular joint, Nuitrack also offers a confidence level, which can vary in value from 0 to 1, where 1 represents safe captured coordinates and 0 represents estimated coordinates. The totality of the 19 joints for a captured body represents a skeleton.

The data captured by the depth camera (RGB and D frames, and skeletons) is then sent to the Camera Storage Server in the cloud, at intervals of approximately 5 s, using a WebSocket. The storage is performed in separate folders, such as /depth and /rgb in which the images are stored, and in another folder, /skeleton, to store the JSON files containing the body coordinates of the persons. Their correlation is done using the timestamp to which they were sent to the cloud, included in the file name, thus eliminating the risk of duplicates.

The next step is executed by the Person Matching Service, which has the same volume as the Camera Storage Server from the previous step, mounted in the container on which it runs. This microservice listens for events in the file system to start data processing. Specifically, when a new JSON file is saved to disk, a file system event is received which triggers the processing. The JSON file is correlated with the depth images on the disk and attempts are made to identify the posture of the person described by the skeleton in the JSON file. RGB frames are collected from RGB-Depth cameras and sent to the backend in order to feed the facial recognition algorithm integrated into the Person Matching Service microservice. This way, each skeleton is properly associated to the right subject captured within a frame. Once the identification of the skeleton has been performed, the RGB frames are discared and not stored at the backend, in order to avoid GDPR-related issues. As a final step, the Person Matching Service sends the processed data to the Gateway to be stored and displayed. Specifically, the person’s skeleton along with additional information are stored in the database, and the depth image is displayed in real time within the user interface. Figure 7 shows on the left a sample depth frame captured by the camera, as well as an example of coordinates for a particular joint captured using the Nuitrack SDK, on the right side.

Fig. 7

a Sample depth frame and b joints coordinates

4.3 Smart watch

As a product in the consumer electronics market, the HumanLink⁴ smartwatch aims to be used by patients suffering from Alzheimer’s disease and their families. Thanks to the onboard SIM card, the smartwatch can exploit the cellular network to establish and maintain an internet connection on the move, in order to transfer data to a proprietary server.

Although this is very useful as it gives the ability to send an SOS signal in case of emergency, the watch battery life is limited to a maximum of 4 days. The Human Link application has several useful features that include: (i) remote device disconnection; (ii) finding device mechanism; (iii) setting up to 6 geo-fence areas; (iv) setting up alarms for trespassing of the geo-fence zones; (v) setting up to 3 device alarms. The application has the capability to track up to 26 watches. Some of the GPS-based functionalities, such as tracking and geo-fencing, are shown in the sample screenshots of Fig. 8.

Fig. 8

Human Link mobile app services: a current location, b geo-fence setup

Unlike the other devices in the system, the watch data is not sent directly by the watches to the vINCI platform, but to an external server. An API developed on purpose allows the vINCI platform microservice to recover the watch data, that are organised as a JSON with predefined structure. The same watch microservice has a cron which runs every 30 seconds and calls the API to retrieve the data for a specified time interval, for all of the devices.

The vINCI platform users will have to wear a smartwatch device capable of tracking their movements. All the captured data is sent to the Connected Medical Devices (CMD) server by making use of 2G network services. To obtain this data inside the vINCI application, a microservice that handles communication with the Cloud service has been developed, so that data can be viewed and refreshed in real-time. To such an aim, a synchronisation mechanism had to be considered. Since at the moment there is not any way to be notified when new records are being stored on a private cloud, a cron job that asks for new data about each alert type, at a fixed period of time, has to be executed. The data is validated at the microservice level to ensure that it concerns only the devices registered into the platform and then it is mapped to the internal format and sent to the IO server for storage. The chosen format has a unique id for each record, a data field that holds the alert payload in the specific format coming the server, and a timestamp that shows the time on the import.

The vINCI platform stores and interprets data received from the watch, while the native watch application is used for the input of the information and the setup of the device. From the application it is possible to create safety areas (geo-fences), call the watch and send audio messages. All the alerts are received in the vINCI app from the vINCI Platform. In the absence of an active GPS connection, the watch uses cell towers or Wi-Fi signals to determine its current position. Because the service exposed by vINCI platform can only send data for a particular alert type, the cron job has to run one REST call per type every 30 s. At the moment, there are not any payload dimension concerns, since data are requested only for a 30–second time frame, and the number of registered watches is reasonable.

5 The vINCI Architecture

In this section, we present the architecture and the infrastructure designed to support network analytics, upon which the different services provided by the vINCI platform are built. Merging different devices’ technologies and integrating them into one unified architecture brings big challenges. It assumes a new service platform with high capability to deal with the complexity of the network infrastructure, the difficulty of interoperability between different service platforms of the respective devices, and the diversity of APIs in use.

5.1 Available approaches and their evaluation against vINCI requirements

During the past few years, the number of devices that can have internet access has been consistently increasing. More and more devices and sensors are interconnected; they can be easily controlled remotely and the multitude of data types being generated lays the groundwork for outstanding opportunities in terms of innovative products and services. IoT Platform-as-a-Service (PaaS) providers ensure that all data collected by sensors or other similar devices is received and sent to other services where it can be stored, viewed, analysed and used to generate a response for other devices, in a highly available, scalable and secure manner. Different providers also offer distinct SDKs that help developers to quickly connect hardware devices to their platforms.

Naturally, there are many suppliers on the market that provide powerful IoT PaaS services, such as Amazon AWS IoT ⁵, Microsoft Azure ⁶, Google Cloud Platform ⁷, or IBM Watson Internet of Things. ⁸ Adhering to similar architectural principles, they use components like: message broker, rule engine module, security and identity module, a module that knows the state of sensors or connected devices. Each provider supports bidirectional communication between hardware devices and platforms, but with different implementations. Amazon uses a message queue to send messages to a device that is subscribed to a certain topic. On the other hand, Azure provides two endpoints that are used to send and receive data. All mentioned platforms use the HTTP and MQTT protocols. ⁹ Another important aspect of IoT platforms is given by the SDK languages support. IBM Watson IoT and Azure offer SDKs for Java, C#, Python, NodeJS and C, while AWS only for C and NodeJS.

Nevertheless, despite their undeniable strengths, the above-mentioned solutions suffer from a common condition. Clients who desire to use their services, besides having to program their hardware in the supplier’s paradigm, must also possess knowledge on how to create their own piece of application to interpret the raw data received. There are other several other projects which try to address these issues, such as Kaa IoT8, a flexible, multi-purpose, open-source middleware that offers features similar to AWS or Azure IoT. The main advantage is given by the possibility to be hosted in private, hybrid or public clouds. Kaa provides the possibility to store data on Apache Cassandra and MongoDB, but comes up short in regard to filling the client-technology gap.

The problems described above have been previously discussed by Gartner, an industry analyst (Hung 2016). They argue that, because of the existent gaps, companies prefer to develop an in-house container service. It is also predicted that this approach will fail to meet expectations, leading to a major shift towards high-productivity and high-control PaaS options—which is why we propose to build the vINCI collecting IoT platform around several IoT PaaS principles.

5.2 Cloud Microservices

Depicted in Fig. 9, the vINCI Cloud Microservices platform aims to offer extendable services for OA monitoring and to provide technological assistance for them. The innovative nature of the vINCI project resides in its modern, well-built and orchestrated microservices architecture, with a substantial number of cloud-native components. The platform is deployed in Rancher in a Kubernetes environment. The admins can see detailed information about the applications in the system and configure them inside the Dashboard using the Spring Cloud Configuration, while normal users, after registering the devices, can see the recorded data and other insights in their dashboard, in the dedicated widgets. In order to provide a cloud-native high-performance server-side stack with suitable test coverage, each microservice has been developed using JHipster (Sasidharan and Kumar N 2020).

Fig. 9

The vINCI Cloud Microservices platform. It is constituted mainly of the basic JHipster components (Gateway and Dashboard), the IO-Server for data communications, and additional vINCI microservices: the watch responsible for the watch devices business; the shoe in control of the smart shoe devices; two microservices that communicate with the camera: camera movement focused on the person’s movements and camera fitness intended upon the fitness exercises; and the survey microservice, responsible for the data received from the survey Android application installed on patient’s tablets

The Gateway is one of the three base components of any JHipster microservice architecture. It handles all the HTTP requests that come from the dashboard, and redirects requests to the microservices in the application to serve them. With the help of the JHipster Registry, the Gateway knows which microservices are up and running and knows how to reach them. It will proxy all the requests to the microservices using their names. As Fig. 9 shows, besides the obvious communication with the ReactJS dashboard, the Gateway communicates directly with the IO-Server and the camera movement microservices. It also takes care of authentication, user management business, and has a connection to a PostgreSQL database, where it stores information about the registered users, user alerts, about their devices and the devices alerts.

The Dashboard is a ReactJS client-side application which is deployed together with the Gateway. It makes REST API calls using the endpoints provided by the gateway to insert and fetch information in order to display it to the users. If logged in as a normal user, the dashboard provides an account section and several widgets with data about every device the user owns. However, an admin has access to more sections in the dashboard: a section where she can see all the data in the system and an administration section. In the administration section, the dashboard displays the swagger API documentation for each endpoint in the system, the user management rubric, logs and metrics panels and more.

The IO-Server is an important microservice in the platform. As well as most of the applications in the system, it is written in Java Spring. The IO-Server communicates with the other services which receive data from the devices, and it has now its own PostgreSQL database where the data gathered from specific microservices is stored in separate tables. It provides a REST API for inserting and querying data. The API can be used by the device specialised microservices to send new data to the IO-Server, or by the Gateway and the dashboard to get data. New entries can be also inserted manually through the dashboard for testing purposes.

Any hardware device (e.g., watches, SI) is first registered in vINCI with a unique ID. All data sent by the sensors could first be received using the MQTT protocol by a device gateway which forwards all data to a cloud gateway. Because vINCI provides multiple services/kits, for every end-user we will define a list of sensors that the vINCI platform needs to manage, and will select which services/kits they want to use and which sensors will provide all data needed for each service. Also, every sensor can be used by multiple services. Compared to other solutions, the main goal of vINCI is to ensure high availability, scalability and also further development. To meet these requirements, vINCI’s model is implemented using a microservices-based architecture, which has the following advantages: every microservice can be independently developed from others; microservices are usually faster and less expensive to develop than regular monolithic services; each component has its own database, depending on service needs; a microservices-based architecture ensures high availability and scalability.

Each microservice developed has been packaged in a Docker container (Merkel 2014). This ensures a faster launch of microservices in production, a better use of hardware resources, and better isolation from the host machine’s operating system. All containers are managed by Kubernetes (Brewer 2015), an open-source orchestrator for clusters of containers. Microservices and Kubernetes enable two main features: the ability to add on-demand new services for new users and the possibility of offering customers the opportunity to create their own services that can better meet their own needs.

5.3 Secure communications and privacy

Another key factor in this scalable and high-availability system is secure communications, thus vINCI uses the HTTPS and MQTT protocols. All data transmitted from sensors to device gateways employs MQTT, whereas for communication from the device gateways to our service, HTTPS is employed. Furthermore, communication between microservices is asynchronous, event-based and uses the Command Query Responsibility Segregation (CQRS) pattern (Hunkeler et al. 2008). Because vINCI could need to store high volumes of data, its storage layer must have the ability to scale, perform, and offer continuous uptime. The best database in the market that can meet these requirements is Apache Cassandra, as it is designed to scale almost linearly while replicating data automatically in one or more data centres to deal with failover situations. After data is received and stored in a database, it must be processed and analysed. vINCI can perform two types of processing: batch processing and real-time processing. Real-time processing can be used to detect the occurrence of important events, whereas batch processing can be used to analyse all received data within a given time frame from multiple sensors and predict a possible future evolution.

For both types of processing (batch and real-time), vINCI will use Apache Spark (Meng et al. 2016), a fast and general use engine for large-scale data processing. The main advantage of Spark is given by the ability to do all processing in-memory and the support for sophisticated analytics and real-time stream processing using multiple languages like Java, Scala, Python. The different devices (sensors, gateway) and related microservices are being defined for development in the next months of project activities.

Data gathered and handled by the vINCI platform belongs to the category of sensitive data relating to person health. Collecting, analysing and storing of such data must be implemented with special care, since it may significantly affect the privacy of a person. For this reason, many regulatory activities have been carried out in recent years to ensure the highest possible level of privacy and security during the collection and processing of health data. The best-known data protection laws at international level are Health Insurance Portability and Accountability Act (HIPPA) in US and the EU GDPR. To meet the requirements, the vINCI platform incorporates a number of mechanisms such as firewall, user/device authentication and communication service based on HTTPS protocol, to ensure data security and confidentiality.

As illustrated in Fig. 10, the vINCI platform uses blockchain—a proven technology for healthcare applications (Kuo et al. 2017)—in order to ensure privacy while preserving provenance and openness. Blockchain technologies have demonstrated adequate scalability (Leal et al. 2020) and an intrinsic ability to support dynamic source reputation (Hirtan et al. 2020) and consistent quality of service (Leal et al. 2021) in complex sensor networks. Within the vINCI platform, patients are recognised as owners of their own data, with full control over it. Management of permissions and access control to collecting health information and using the personal data stored in the platform is performed using blockchain, which keeps security policies applicable to the data of a given patient.

Fig. 10

The privacy-preserving layer of the vINCI architecture uses blockchain technologies

For this purpose, the blockchain collaborates with the vINCI platform (Storage Stack), hardware devices (through the API Gateway), and the vINCI Digital Caregiver app. The entities (the patient, patient’s relative, the caregiver etc.), with granted proper permissions, can deliver data to the platform or access health records only when their identities and cryptographic keys have been verified by the blockchain. The blockchain allows users (the older adult and/or her/his family) to easily grant, modify or revoke access to their data. In this way, patients’ data are securely shareable and retrievable for careers/healthcare providers who have been granted access to it.

An additional advantage of blockchain is the use of unified access rules to data collected by sensors operating in different technologies. This is essential to ensure efficient services in a heterogeneous IoT environment (Mongay Batalla et al. 2016). The vINCI blockchain-based data access control system is being developed using Hyperledger Fabric (Dhillon et al. 2017) framework that allows implementing role-based blockchains with strong support for privacy and confidentiality of blockchain transactions.

6 The vINCI Digital Caregiver application

The solution proposed in vINCI is to create a support system for the elderly in order to bring together the different services described in previous sections. From the user’s side, the solution consists of an Android tablet application, portable and accessible to anyone, no matter where they are. Illustrated in Fig. 11, the vINCI mobile app, a.k.a the vINCI Digital Caregiver, is designed to run either on mobile tablets, personal laptops or desktop computers. The connectivity between the application and other vINCI modules is to be provided using conventional broadband network technologies: Wi-Fi or LTE for tablets and Ethernet or Wi-Fi for laptops and desktop computers. The vINCI Digital Caregiver will have direct access to the vINCI platform, without the need to use any gateway or other type of edge node. Because the target group of users consists of the elderly, the application needs a simple and concise interface so that each feature can be easily found. With age, vision problems can occur, so font and graphical elements size are large enough to be easily read.

Fig. 11

Screenshot of the vINCI mobile app, a.k.a The vINCI Digital Caregiver

The user interacts directly with the tablet to access the Android application. By the app, requests are issued to the Gateway, to display and process data. In its turn, the Gateway redirects each request to the associated microservice. All data that must be stored for processing is stored in the database corresponding to the gateway.

6.1 Market research

It is possible to compare the The vINCI Digital Caregiver app with similar ones in the market. There are currently many applications that offer activity monitoring services, medical advice or support to well-being in order to improve a specific QoL aspect. But most of them are geared around a single goal, providing facilities for a unique problem. Examples of applications include monitoring the steps taken in a day and recording DLA, offering only general suggestions on possible exercise programs, without providing health advice. Another category includes applications providing general medical advice, or suggestions based on certain symptoms introduced by the user, but without the supervision and control performed by a specialised medical staff. Some applications give the possibility of entering the prescription manually by the user (if he/she received one from the doctor), and get notifications at the moments when medication has to be taken. But all these advice are offered on the basis of general statistics, the recommendations not being individualised and controlled by a specialised framework. Finally, most applications on the market do not combine both the healthcare-related part and the PA monitoring part.

Among the applications available for mobile devices (phone and tablet), which aim to provide a platform for monitoring the user’s health, it is possible to mention: Senior Safety App¹⁰, WebMD¹¹ and Senior Fitness.¹²

Senior Safety App is an application designed to be used by the elderly to monitor their DLA, but especially their safety. It is installed on the senior’s device, but the caregiver is the recipient of alerts, if something unusual happened in the senior’s DLA, and can view the person’s location through a web application in the browser. The important feature of this application is the capability to issue alerts to the assistant and, if necessary, to call the emergency service. In addition to these alerts, the assistant can see at any time the location of the elderly person and the route travelled up to that time. The Senior Safety App is dedicated to seniors and creates a connection with their caregivers and family members, to give them extra safety. However, the app does not focus on providing personalised medical support, ensuring an active lifestyle and tailoring medical advice according to the user’s needs.

The WebMD application focuses on analysing the symptoms of a possible health condition. It is intended for people of all ages, not just the elderly. It offers the possibility to manually record data about the user’s medical treatment and to set alarms to remind the timely intake of medications. WebMD provides a questionnaire through which the user enters biological data and symptoms felt, ultimately generating a list of possible matching diseases, sorted by relevance. If the user acknowledges to suffer from a certain ailment, a service is available to find a nearby medical consultant. The service also locates nearby pharmacies and provides suggestions on where to find medicines at the best prices. In addition to hints for possible conditions, WebMD also has a section that presents medical articles under various points of interest in order to treat some diseases.

However, the advice offered by the WebMD application to the user is not specific or individualised. There is no constant monitoring service for the user DLA and status, nor is there any special treatment created for the user by qualified personnel. It is an application that can be used in the initial phases of diagnosis, particularly for common symptoms. It is a useful support, as it provides substantial medical information from which the user becomes aware of his current health status, but the app is not linked to any specific doctor or nurse. It does not receive data about the users’ QoL based on their activity.

Finally, Senior Fitness is an application dedicated to performing daily physical activities. For each day there is a series of sport exercises designed especially for the elderly, to promote an active lifestyle. Each exercise contains a description and a video with an example of the movements to perform, for easier understanding. Once the user has performed the movement, he/she can check it and the next one appears. At the end of the program, the activity is marked in a calendar, so the users can track their daily progress. The weak point of this application is that the activities are predefined, they are not customised to the needs and capabilities of each individual. There is no way to control training and check the PA performed by the OA. It is exclusively an application that provides sample physical exercises especially for the elderly but does not provide any way to check the user’s health.

All the above mentioned applications are designed to bring small improvements to the user’s QoL, but they have limitations that do not make them suitable to truly ensure a long-term physical and mental well-being. None of them continuously monitors the user DLA through several devices, for data verification and validation. For this reason, no personalised and constantly updated treatment is provided by qualified personnel. The vINCI app aims to ensure both an environment of interaction between the elderly and the caregiver, through constant and controlled monitoring, as well as the possibility of periodic verification of the subject’s condition. This ensures personalised assistance from several perspectives of an active and healthy lifestyle.

7 Patient profile model

An important objective of the vINCI project is to construct/collate personalised ML-based models for continuous intelligent monitoring of behavioural and physical markers of the older adult’s health-related QoL. This includes constructing an evidence-based model (lifestyle profile) of the older adult, by collecting and aggregating sensed data. The profile will be made available to caregivers and out-patients clinics as medical records, and will provide support for individualisation of medical QoL interventions. For vINCI care, the profile will be input to provide personalised support for daily/medical activities. Finally, the profile will be used as evidence to evaluate the impact of vINCI on the perceived QoL level, allowing a proper adjustment (if needed) of the intervention support being provided by caregivers.

Before enrolling into vINCI professional caregiving environment, the participants are first interviewed by a professional caregiver, with a series of questions, to construct QoL profile, and establish a clear medical ground for delivering the most appropriate vINCI support (appropriate IoT kits for the prescribed QoL intervention). Next, in time, as the user starts using more vINCI, his/her data from the sensing device is put to use, such that vINCI automatically learns their personal preferences (e.g. daily walking routine). This personalised model is further used to establish a normal from a stressful activity.

The conceptual model of the patient profile is done by analysing the captured information, using different analysis techniques, knowledge representation techniques, ontologies that can provide descriptions of lifestyle (Bǎjenaru et al. 2019), changes in clinical and behavioural dynamics of the patient and another complex information about that person. This process may contributes to improve the diagnosis and management of geriatric syndrome in primary care, as well as to increased levels of well-being, QoL and perceived health. Building a formal ML model of the patient’s lifestyle and of the behavioural, clinical and biological dynamics of the patient to be integrated in the computer-based systems, represents the next step in the modelling process of the patient profile. This formal model is used by the proposed platform to help the physician to select and prioritise therapeutic interventions that are best suited for a given patient. Also, vINCI models, annotates, and extracts knowledge to facilitate multi-stakeholder data-centric smart follow-ups to support the QoL evaluation and interventions.

The vINCI patient profile is composed of Static and Dynamic Profiles. Shown in Fig. 12, the Static Profile is populated from the data which the user completes and remains unchanged (forever or for a long period of time), such as: name, address, phone number etc. The Dynamic Profile, on the other hand, constitutes of all the data that comes from the devices, which is processed, analysed and transformed into something meaningful for the patient, as shown in Fig. 13.

Fig. 12

The Static Profile diagram

Fig. 13

The Dynamic Profile diagram

Every piece of data transmitted from the devices arrives at the microservices with a format including the following fields: deviceUUID, timestamp and data. Then, it is stored as an object with the following fields: Id, deviceId, timestamp, data. The deviceId is the identifier of the device stored in the gateway database and mapped to the deviceUUID sent by the device. The data field contains the specific data received from each device, which is a JSON serialised into a string. This string needs then to be deserialised and parsed in order to extract relevant information. The presented format applies to the insole, watch and camera services, while the microservice managing the surveys receives the data in a more detailed format. The data from the watch is a JSON that contains, among other information, the coordinates of the location of the watch and the time when the watch was in that location.

The SI Android application can connect to multiple insoles which send data to it. Then, the application transfers the data to the cloud in the form of JSON containing the following information: number of steps, activity type: (WALKING, STANDING or NOT WORN), time when the activity has been recorded. Currently, the data that reaches the cloud is translated into a dashboard that shows the subject number of steps per day. This dashboard helps understanding how active and physically healthy the monitored person is.

The camera sends to the cloud a JSON with a skeleton—an object that contains the cartesian coordinates for each joint—describing the position of a given person body. There can be more skeletons in the JSON if more than one subject is detected in the image. The camera also connects to a WebSocket opened by the Camera Storage Server and besides the JSONs, also sends images (D and RGB) to it regularly. The images and the JSONs are used by the Activity Recognition Python application, that exploits an ML algorithm to determine whether the subject is mostly SITTING or STANDING. These results are used to describe the sedentary of the person and to decide if he/she is active enough or not. The Camera Movement microservice listens to the file system events and fetches both images and JSONs, and sends them to the gateway. The latest image is displayed in a widget also in the person’s dashboard. Images from the Storage Server are also used by the Face Recognition application to map images to subjects, and to send the list of detected persons to the gateway.

The main target group of vINCI is constituted mainly by OAs with and without Mild Cognitive Impairments, which are clinically evaluated first through the DSM5 and ICD-11 criteria (Bǎjenaru et al. 2012). The profile model is used as evidence to evaluate the impact of vINCI on the perceived QoL level, allowing a proper adjustment of the technologies (calibration) and the intervention support provided by caregivers. With personalised healthcare, professionals can still get an early health system in which disease is approached as soon as possible, rather than a “late illness” model, where the focus is mainly on diagnosis and treatment.

8 Preliminary outcomes and quantitative results

To verify and test through a clinical validation the vINCI technology, two multi-disciplinary controlled pilots (in Romania and Cyprus) are being deployed in controlled environments, for real-life use cases involving older adults across Europe.

In Romania, the pilot study involves 30 persons, 65 years of age and older, 15 ones for control group and 15 ones for test group. Before entering the Pilot Study a total of 140 patients have been screened using Recruitment Form and Digital Skills Questionnaire. All consecutive patients 65 years of age and older, admitted to Geriatrics and Gerontology Inpatients Department at NIGG, on referral from general practitioners or other specialist for various chronic or subacute conditions, are considered for inclusion in the Pilot Study and then evaluated against exclusion criteria. The pilot study in Cyprus aims to explore the acceptability of vINCI technology by OA and the potential impact on their the QoL. In order to achieve this aim, the technology will be used by 20 older people; the participants stay in nursing homes or attend day centres.

In order to get the personalisation of healthcare, various factors, such as the individual’s lifestyle, surrounding situations, device capabilities, etc., should be considered (Christensen et al. 2009). The OA is much more likely to suffer from health deterioration than other population groups. Multiple chronic diseases and cognitive, physical and mobility impairments, typically lead to the necessity of increased access to healthcare services, personal assistance, environmental changes (such as adapting the home to a specific dysfunctionality), or even a growing number of nursing homes and better ageing-specialised caretakers. In vINCI, OAs are seen from a new and comprehensive perspective that encompasses not only the improvement or preservation of their health, but also a continuous and personalised assistive care, active and preventative support from a broader range of healthcare specialists and informal caretakers. In this context, the challenges remain to develop technologies that meet the needs of OAs, accommodate their cognitive and perceptual declines, capitalise on their intact abilities, support them in performing everyday activities, and protect their privacy, independence and security (Drǎghici et al. 2019).

8.1 User needs perspective

The vINCI user model demonstrates that the technical, demographic, health issues and QoL domains must be considered to adequately capture the complete pattern of vINCI users (Bǎjenaru et al. 2018; Dobre et al. 2019). For this reason, for all the patients of the vINCI study, the following data is collected as baseline: identification data, socio-cultural data, medical data. vINCI development approach focuses on end users: their requirements, needs, and wishes represent the backbone of the design and development of the vINCI platform, and, at the same time, the information provided by the User Needs Questionnaire (UNQ) outlines the profile of the elderly.

The UNQ questionnaire is divided into two parts. The former (part A), regarding some demographic data, daily activity and interest in new technologies, requires as input the level of familiarity with various technology terms related to health (geo-location, digital communication, use of Internet etc.). The answer is in the form of multiple choices and corresponds to an aspect that can be found in the vINCI platform. The latter (part B) gives to developers a much more detailed picture of user preferences and needs regarding the vINCI environment. They collect information about the level of acceptance of some proposed technical solutions, appearance, functionalities and interaction.

8.2 UNQ Part A: demographic, daily activity, and interest in new technologies

As detailed in Table 1, in the study dedicated to a total of 80 elderly (62 from NIGG and 18 from Cyprus— UNRF), the majority of participants in both groups are women 75% (79.03%—Romania and 61%—Cyprus), have a high level of education, 23.75% higher education and 43.75% secondary education (from total participants) and only 5 are single (58.75% are married from total; 51.61%—Romania and 83.33%—Cyprus).

Both OA groups have a good level of social interaction (daily frequency— 62.75% from total), but most individuals are not so familiar with technologies and applications, revealing the lack of expertise with technology. Such issue should be seriously considered during the design of the overall system and, moreover, for the user interfaces, particularly in connection with usability, learnability, and user satisfaction. Those factors that are crucial for the success and adoption of the vINCI system. The results described in Table 2 shows that 64.52% of the elderly of Romania and 61.11% for Cyprus have a good social relationship with other people. However, for 37.25% of the respondents, there was some degree of isolation, with no regular interaction with other people. Thus, the lack of social interaction is most often correlated with the lack of PA, or a poor emotional state, with negative consequences on the health and QoL of the OA.

Table 1 Demographics of the OA participants from Romania and Cyprus

Table 2 OA interaction

Communications between elderly with other people give us an interesting view about social interaction and rate of adoption of new means of communication. Around 90.32% of the respondents in Romania and 94.44% in Cyprus selected classic mobile phone option, indicating a certain reluctance to use alternative means. Only 22.50% of the elderly use smart devices and applications. The obtained results indicate very clearly a difficult transition to modern means of communication. To improve this tendency, vINCI platform designers must implement a minimalist user interface interfaces, which will ensure a smooth transition to smart communication devices within the platform (e.g. smart applications on tablet).

An important issue for older people relates to the degree of safety they perceive when living alone. A low degree of personal safety can contribute to the isolation of older people, may induce fear of carrying out certain daily activities alone and may create a permanent need for long-term care. The majority of the elderly participating in this study, 53.23% for Romania and 61.11% for Cyprus, consider frequent interaction with the caregiver an important factor in ensuring an increased level of personal safety. The technology can create a permanent bridge of communication between the two actors, thus replacing the lack of physical presence of the caregiver (formal/informal) near the patient. In this regard, the developers of the vINCI platform must provide secure channels of communication and appropriate tools to connect, whenever necessary, the patient and the caregiver.

An improved interaction between the patient and the caregiver does not strictly refer only to the communication (audio–video) on demand, being necessary the existence of some ways by which the situations of danger or those that require special attention can be detected automatically. Thus, the vINCI platform should offer the possibility of establishing a quick way of communicating between the caregiver and the patient, in order to provide an appropriate intervention that confers confidence and safety to the elderly.

8.3 UNQ Part B: user preferences, needs and requirements regarding the vINCI environment

Answers about the users’ preferences regarding a series of functionalities based on intelligent technologies, specific to health monitoring or medical assistance systems, are collected under this category. A high degree of utility from the user perspective leads to the formalisation of user requirements, that are the basis for designing the vINCI platform. The participants in the two pilot studies have the possibility to indicate on a scale from 1 to 5 (1—not useful at all, 2—not very useful, 3—indifferent, 4—partial useful, 5—very useful), how useful they consider a system that can offer functionalities described through a list.

Monitoring the health parameters through intelligent technologies (wearable sensors, IoT, etc.) is an important step in the early detection of health problems. Thus, the health parameters are automatically monitored at home without influencing the patient’s daily activities. A major advantage of remote health monitoring is the increased comfort for the patient and a reduction of medical service needs. Sensors embedded in the environment are used to collect behaviour and activity patterns for the purpose of detecting health changes. Early detection is the key to promoting health and independence for elderly. Data from sensors can be automatically analysed on a daily basis, looking for changes within individual data patterns. If a change is detected, an alert can be sent to clinicians, caregivers, family, etc. The implementation of such technologies (sensors, smart devices, wireless communication methods) is considered very useful by 40.32% of Romania pilot participants, and by 61.11% of Cyprus pilot participants.

The systems for monitoring daily activities or health parameters through sensors, allow the collection of several categories of personal information such as: demographic data, location and moving data, activities recognition associated data, and health related data. There are situations when users are reluctant to provide sensitive data about them to third parties. In order to have an image on the categories of information that can be collected within the vINCI platform, the usefulness of the function of collecting personal information for the aforementioned categories has been evaluated. The complete results obtained in the two pilot studies are shown in Fig. 14.

Fig. 14

User reluctance to provide personal sensitive data to third parties (1—not reluctant at all, 5—very reluctant)

The results of the questionnaire applied to the group of the elderly from Romania and Cyprus reflect their skills, their expectations and their ideas of being monitoring by intelligent devices like sensors, in order to detect changes in PA, social interactions, and mood, as markers of their overall health-related QoL.

9 Conclusion

The main purpose of vINCI technology is to offer an easy-to-use instrument for older people that will enable them to change and optimise their lifestyle in order to ensure an active longevity. One of the most important lifestyle risk factors for many chronic conditions in older age is low PA. The vINCI technology enables elderly not only to independently self-evaluate their PA level as well as their health- related QoL and well-being, but to also change their behaviour and health-related lifestyle, by providing specialised medical recommendations.

In the QoL research, health is one of the most important social factor nominated by the older population in setting their living standards (Lamboy et al. 2007). Thus, based on this empirical evidence, the vINCI project includes health among the QoL dimensions and the factors that decisively influence QoL. The objective of vINCI is to enhance OA active ageing and, as a result, their health-related QoL through technology, in a stepwise iterative manner, by building services tailored to the subjects and able to adapt to their evolving status.