Keywords

1 Introduction

Ambient Assisted Living (AAL) are the Ambient Intelligence based technologies for the support of daily activities to elders. Nowadays elders, often find it difficult to keep track of their cognitive and functional abilities required for remaining independent in their homes. Traditional rehabilitation is an example of a common activity elders may require and that usually implies they move to the rehabilitation clinics, which is the main reason for treatment discontinuation. Tele-rehabilitation is a solution that not only may help elders but also their family members and health professionals to monitor elder’s treatment [20].

Recently the domain of HCI began to grow rapidly with new forms of input (GPS, accelerometers, motion sensors, etc.) and new output devices (mobile phones, tablets, projectors, wearables, etc.) emerging in quick succession [1]. These has reveled a new paradigm of interaction which is “natural interaction”, where the use of these ubiquitous sensors allow to interact with the user via gestures and pointing. An example of technology is the Kinect which has the capability to interpret three-dimensional human body movements in real-time and makes the human body the controller. These gaming technologies has been used and adapted by researchers and therapists to build assistive systems.

The purpose of this paper is to present a tele-rehabilitation system that uses the motion-tracking sensor of the Kinect, to allow the elderly users natural interaction, combined with a set of external sensors as a form of input. The elders interact with the system in a 3D environment, where they perform multiple movement combinations without the need of an attached device or a controller thanks to the Kinect. Information fusion techniques are applied to the data extracted to give a more precise feedback to the user regarding the rehabilitation exercises and a case study for AAL is presented. The paper is organized as follows: Sect. 2 presents the analysis of some studies and AAL applications for the elderly people. Section 3 describes the system model and design. Finally, in Sect. 4, a case study is presented and finally some conclusions are given and future improvements are proposed.

2 Related Work

Ambient Intelligence (AmI) refers to a vision in which people are empowered by an electronic environment that is sensitive and responsive to their needs, and is aware of their presence. Its target is improving quality of life by creating the desired atmosphere and functionality through intelligent and inter-connected systems and services. Inside AmI, Ambient Assisted Living (AAL) is an emergent area that provides useful mechanisms that allows tracking elders through sensoring, for example, using mobile devices, that not only work like communication devices, but also are equipped with several sensors like accelerometer, gyroscope, proximity sensors, microphones, GPS system and camera. Ambient Assisted Living (AAL) can be defined as the use of information and communication technologies (ICT) in a persons daily living and working environment to enable them to stay active longer, remain socially connected and live independently into old age [19].

Gamification dwells on established approaches like serious games and is defined as an “umbrella term for the use of video game elements to improve user experience and user engagement in non-game services and applications” [11]. We used gamification to motivate the patients, which means better adherence to the treatments and faster results.

Several other applied games have been proposed for rehabilitation activities. However, the problem with the sensors precision is still not solved by any of them and it has received less attention. The combination of observations from a number of different sensors provide a solid and complete description of the task, so information fusion techniques match perfectly in this solution.

Information fusion focused in sensors has become increasingly relevant during the last years due to its aim to combine observations from a number of different sensors to provide a solid and complete description of an environment or process of interest. The information fusion systems are characterized by its robustness, increased confidence, reduced ambiguity and uncertainty, and improved resolution.

The Data Fusion Model maintained by the JDL Data Fusion Group is the most widely used method for categorizing data fusion-related functions. They proposed a model of six levels, of which the first is related to information ex- traction, and the last with the extraction of knowledge. The JDL model was never intended to decide a concrete order on the data fusion levels. Levels are not alluded to be processed consecutively, and it can be executed concurrently [8]. Although the JD data fusion model has been criticized, still constitutes a reference to design and build systems to obtain information from the data in complex systems and generate knowledge from the extracted information.

After years of intensive research that is mainly focused on low-level information fusion (IF), the focus is currently shifting towards high-level information fusion [7]. Compared to the increasingly mature field of low-level IF, theoretical and practical challenges posed by high-level IF are more difficult to handle.

Some of the applications that involve high-level IF are:

  • Defense [1, 2, 9, 12, 21]

  • Computer and Information Security [10, 13]

  • Disaster Management [22–25]

  • Fault Detection [3–5]

  • Environment [15, 16, 18]

But these contributions lack of a well-defined spatio-temporal constraints on relevant evidence and suitable models for causality [6].

Our proposed model provides the big picture about risk analysis for that employee at that place in that moment in a real world environment. Our contribution is to build a causality model for accidents investigation by means of a well-defined spatio-temporal constraints on offshore oil industry domain. We use ontological constraints in the post-processing mining stage to prune resulting rules.

3 Model

In this section more details about the Knowledge Retrieval Model are provided. First a detailed description of the proposed architecture, domain ontology and reasoning process described by means of inductive learning process.

3.1 System Architecture

The architecture of our fusion framework is depicted in Fig. 1. This architecture was extended from our previous work [26–29]. Following we give some new details of the architecture.

The system developed has a hierarchical architecture with the following layers: Services layer, Context Acquisition layer, Context Representation layer, Context Information Fusion layer and Infrastructure layer. The hierarchical architecture reflects the complex functionality of the system as shown in the following brief description of the functionality of particular layers:

  • Infrastructure Layer. The lowest level of the movement management architecture is the Sensor Layer which represents the variety of physical and logical sensor agents producing sensor-specific information. Kinect sensor and external arduino sensors used Fig. 5:

    1. 1.

      Microcontroller ESP8266 HUZZAH

    2. 2.

      Accelerometer and gyroscope MPU-6050

    3. 3.

      Temperature sensor DS18B20

    4. 4.

      Capacitive Touch

  • Context Acquisition: The link between sensors (lowest layer) and the representation layer.

  • Context Representation: This is where the low-level information fusion occurs by means of an ontology.

  • Context Information Fusion layer: This layer takes the information of Kinect sensor and other contextual information related to the user as well as external sensors and transforms it into a standard format. This is where the high-level information fusion occurs. It is here where reasoning about context and trained neural network occurs. Extended description is given in next section.

  • Service Layer. This layer interacts with the variety of users of the system (elders/health personnel/caregivers/ family members) and therefore needs to address several issues (who can access the information and to what degree of accuracy), privacy and security of interactions between users and the system.

Fig. 1.
figure 1

Architecture.

Full size image

3.2 Ontology

Normally, ontology represents a conceptualization of particular domains. In our case, we will use the ontology for representing the contextual information of the ambient assisted living environment. Ontologies are particularly suitable to project parts of the information describing and being used in our daily life onto a data structure usable by computers.

Using ontologies provides an uniform way for specifying the model’s core concepts as well as an arbitrary amount of subconcepts and facts, altogether enabling contextual knowledge.

An ontology is defined as “an explicit specification of a conceptualization” [14]. An ontology created for a given domain includes a set of concepts as well as relationships connecting them within the domain. Collectively, the concepts and the relationships form a foundation for reasoning about the domain. A comprehensive, well-populated ontology with classes and relationships closely modeling a specific domain represents a vast compendium of knowledge in the domain.

Fig. 2.
figure 2

Ontology hierarchical concepts.

Full size image
Fig. 3.
figure 3

GUI- Telecare System - in use

Full size image
Fig. 4.
figure 4

GUI- Telecare System - no use

Full size image
Fig. 5.
figure 5

Temperature sensor, micro-controller, accelerometer and gyroscope, temperature sensor and capacitive touch sensor

Full size image

Furthermore, if the concepts in the ontology are organized into hierarchies of higher-level categories, it should be possible to identify the category (or a few categories) that best classify the context of the user. Within the area of computing, the ontological concepts are frequently regarded as classes that are organized into hierarchies. The classes define the types of attributes, or properties common to individual objects within the class. Moreover, classes are interconnected by relationships, indicating their semantic interdependence (relationships are also regarded as attributes).

We built a domain ontology for the Ambient Assisted Living environment (AAL) [17]. We also obtain the inferences that describe the dynamic side and finally we group the inferences sequentially to form tasks. Principal concepts of the ontology can be checked at Fig. 2.

3.3 Data Fusion Model

JDL model of six levels, of which the first is related to information extraction, and the last with the extraction of knowledge. These levels are characterize as:

  • Level 0 (Source Preprocessing/Subject Assessment): Estimation and prediction of observable states of the signal or object, based in the association and characterization of data at a signal level.

  • Level 1 (Object Assessment): In this level, objects are identified and located. Hence, the object situation by fusing the attributes from diverse sources is reported.

  • Level 2 (Situation Assessment): The goal of this level is construct a picture from incomplete information provided by level 1, that is, to relate the reconstructed entity with an observed event.

  • Level 3 (Impact Assessment): Estimation and prediction of the effects that would have the actions provided by participants, taking into account the information extracted at lower levels.

  • Level 4 (Process Refinement): Modification of data capture systems (sensors) and processing the same, to ensure the targets of the mission.

  • Level 5 (User or Cognitive Refinement): Modification of the way that people react from the experience and knowledge gained.

There is a need of a fusion framework to combine data from multiples sources to achieve more specific inferences. A fusion system must satisfy the user’s functional needs and extend their sensory capabilities (Fig. 3).

4 Final Remarks

Although we have introduced and presented the novel tele-rehabilitation solution, it must be pointed out that, this approach is currently deployed as part of a larger activity tracking system that includes also outdoor sensors. The first functional prototype of the system is currently developed (Fig. 4).