1 Introduction

The healthcare systems in Europe and the most developed countries in the world are faced with challenging transition processes due to demographic change. The increase of the elderly population and the expectation of life brings about serious risks, with some profound socio and economic impact on our societies. To reduce the burden of social care, integration of existing unobtrusive, easy-to-use and transparent monitoring systems are proven to be an acceptable solution by the elderly and their careers. The ultimate goal of the research is to replace the carer with a virtual intelligent carer capable of interpretation and understanding of their activities.

Many technological advances in monitoring systems are developed. Ambient Intelligence has been adopted as a term referring to environments that are sensitive and responsive to the presence of people and it is a candidate to become the next wave of computing. Indeed, this novel computing approach is aimed to extend the ubiquitous vision by incorporating intrinsic intelligence in pervasive systems. This idea enables the study, design and development of paradigms for smart environments that not only react to human events through sensing, interpretation and service provision but also learn and adapt their operation and services to the users over time. These paradigms employ contextual information when available, and offer unobtrusive and intuitive interfaces to their users.

The guest editors of this special issue have organised annual workshops under the same topic for many years. The papers presented in this special issue are mainly from the papers presented during the 4th International Workshop on Human Behaviour Monitoring, Interpretation and Understanding (NOTION). The workshop was part of the 2019 Pervasive Technologies Related to Assistive Environments (PETRA) conference.

This special issue aims to encompass valuable research and achievement in the monitoring, interpretation and understanding of human behaviour through smart technologies. There are many readily available monitoring systems to monitor people in their home environment. However, there are challenges ahead with respect to interpretation and understanding of the sensory data and linked that with the actual activity representing a specific behaviour.

Following the special issue call for papers, 34 manuscripts were received. Among them, 17 papers were accepted subject to major revisions. After the next rounds of the revisions, 13 papers were finally accepted for publication. The guest editors wish that this collection of papers demonstrate the state of the art in the area of ambient intelligence applications in human behaviour analysis from different perspectives, and can effectively contribute to the body of knowledge in this subject area.

2 Contributions of this issue

A brief review of the gathered papers in this special issue shows the wide range of approaches, methodologies and techniques that the authors have taken in human behaviour monitoring and detection.

In the first paper by D. Hooda and R. Rani entitled “Ontology Driven Human Activity Recognition in Heterogeneous Sensor Measurements” Hooda and Rani (2020), an ontology-based knowledge model is proposed for human activity recognition. Sensor measurements and activity recognition collectively contribute to develop the proposed ontology and to conceptualise the activity recognition process. The highlight of this research work is that the proposed model can effectively capture the semantic relationships between the low level (simple) and high level (inferred) activities.

The next paper is “A novel feature selection method based on comparison of correlations for human activity recognition problems” by Tsanousa et al. Tsanousa et al. (2020), with a focus on feature selection in human activity recognition. Their method is based on recordings of triaxial accelerometers and gyroscopes and creating a subset of features to optimises the performances of different activity classification algorithms.

Hirt et al.’s paper entitled “Stress Generation and Non-intrusive Measurement in Virtual Environments Using Eye Tracking” Hirt et al. (2020) checks if and how stress can be measured in a non-intrusive way, such as while the users explore a virtual environment. The authors have evaluated this idea by using eye tracking devices to detect stress levels and measuring the individual’s pupil diameter and pulses.

Next, there is a paper by Makri et al., entitled “Human behaviour in multimodal interaction: main effects of civic action and interpersonal and problem-solving skills” Makri et al. (2020). The focus of this paper is on multi-modal interaction and its effects on the human behaviour during meta-cognitive skill training. The authors have designed a user-system evaluation by sensing the before-and-after interaction of the users with the system, integrating a range of factors including dialogue system performance, metacognitive-related and individual-and-community level-related attributes.

Staffa et al., have contributed to this special issue by a paper entitled “Working together: a DBN approach for Individual and Group Activity Recognition” Rossi et al. (2020). Unlike the other works that have focused on individual activity recognition, the authors have addressed the problem of group activity recognition. Their methodology first examines each individual actions by correlating ambient sensor data, then this information are aggregated using a Hierarchical Deep Belief Network.

“Thumbs Up, Thumbs Down: Non-verbal Human-Robot Interaction through Real-time EMG Classification via Inductive and Supervised Transductive Transfer Learning” Kobylarz et al. (2020) is a paper by Kobylarz et al., in which a transfer learning method for gesture classification is proposed. The signs of ’thumbs up’, ’thumbs down’, and ’relax’ are used as means of non-verbal human-computer interactions. Using Electromyography data and relying on Inductive and Transductive Transfer Learning, the authors have compared the efficiency and limitations of different employed classification methods such as Random Forest and Support Vector Machine.

This special issue also includes a paper entitled “Examination of Electrodermal and Cardio-Vascular Reactivity in Virtual Reality Through a Combined Stress Induction Protocol” by Kerous et al. Kerous et al. (2020). Through sensing body reactions to different types and levels of stress, the authors aimed at finding the optimal combination of social and task-related stress to be used to consistently and reliably induce a stressful reaction.

The paper entitled “Human Behaviour Sensing: Challenges and Approaches” by Liu et al. Liu et al. (2020) is a comprehensive survey on the challenges and the future outlooks known for the thing-centric approaches in activity recognition, as opposed to the others’ human-centric approach which are usually associated with privacy concerns. This includes capturing environmental information such as the envoronmental vibrations or signals to indirectly infer the human activities.

The paper by Liouane et al., entitled “An Intelligent Knowledge System for Designing, Modelling, and Recognising the Elderly Behaviour in Smart Space” (Liouane et al. 2020) has focused on semantic understanding and prediction of the elderly behaviours. This has been achieved by proposing a descriptive language, a hybrid Markov model, and developing a knowledge-base developed by processing data coming from ambient sensors.

In the paper by Mohmed et al., entitled “Enhanced Fuzzy Finite State Machine for Human Activity Modelling and Recognition” Mohmed et al. (2020), the different uncertainties naturally embedded in classifying human activities have lead the authors to proposes an enhanced Fuzzy Finite State Machine (FFSM) model of activity recognition, by incorporating the learning capabilities of Long Short-Term Memory and Convolutional Neural Network into the classical FFSM. Through experiments based on sensory data and public datasets, the machine learning techniques are employed to develop a fuzzy knowledge-base that governs the transitions between the FFSM’s states, leading to an improved classification accuracy in recognising the daily activities.

The above papers are followed by “Analyzing the Kinematic and Kinetic Contributions of the Human Upper Body’s Joints for Ergonomics Assessment”, a paper by Menychtas et al. (2020). Having evidences of chronic musculoskeletal injuries caused by problematic body postures, the authors have focused on assessing the extent of these injuries by sensing the body’s joint angles and torques as an effective ergonomics evaluation method. By developing an upper human body model, they have shown how sensing the joint angles and torques can identify which joints are under the most strain while performing a task.

In the paper entitled “Designing Assistive Technology for Getting More Independence for Blind People when Performing Everyday Tasks: An Auditory-Based Tool as a Case Study”, Leporini et al. (2020) have shown how simple auditory-based tools can effectively support the visually-impaired individuals in their independent living tasks, namely on checking if their rooms’ lights are on or off. In a broader view, they have conducted an empirical study leading to recommendations for designing a wider range of assistive technology tools for supporting visually-impaired individuals.

This special issue finally reaches to the last paper by Blanco et al. entitled “Analysis of the Interaction between Elderly People and a Simulated Virtual Coach” Justo Blanco et al. (2020). The authors employed conversational agents to coach elderly people, namely when they interact with a Wizard of Oz driven, simulated system. Helped by natural language processing, facial expression and gestures detection techniques, an analysis of the human behaviour is proposed with the aim of promoting healthy and independent living for the ageing population.