Modeling Rehabilitation Dataset to Implement Effective AI Assistive Systems

The demand for automated systems monitoring and supporting patients in their home-based recovery programs is substantial. While emerging technologies have been proposed as potential solutions to enhance at-home patient care, limited systems are in place due to their challenges in oﬀering real-time monitoring and corrective feedback. Most proposed methodologies provide an overall measure or score for the executed movement. The proposed work involves the adaptation of an existing published dataset to cater to the needs of a system capable of remotely assisting patients, eﬀectively acting as a virtual physical therapist able to provide corrective feedback. A dataset containing a set of three simple exercises for shoulder rehabilitation was processed. Each movement was meticulously annotated for temporal and categorical motion domains to monitor exercise execution in terms of the range of motion completeness and to evaluate compensatory movement patterns. This work carries substantial signiﬁcance by oﬀering a standardized and easily accessible model for human movement data, thus fostering the advancement of digital assistive systems designed to support home-based rehabilitation programs.


Introduction
Home-based rehabilitation heavily relies on the exercise programs prescribed by physiotherapists, yet patient adherence to these regimens often falls short [1].An emerging field in the realm of rehabilitation technologies is dedicated to the advancement of digital biofeedback systems [2,3].These systems utilize external sensors to collect motion data, which is then meticulously analyzed and conveyed to the user, with the primary aim of delivering support and motivation throughout the home-based rehabilitation process.
The assessment of human motion is an intricate endeavor with wide-ranging implications, spanning from enhancing athletic performance to aiding in diagnostic and therapeutic applications within the field of rehabilitation.Researchers and innovators have invested significant time and effort in developing vision tools designed to monitor and evaluate motion, particularly in remote settings, to address the inherent complexities of this task [4].
The development of digital assistive systems for precise human motion evaluation has posed a formidable challenge in the field of artificial intelligence (AI) research [5].This challenge is amplified by the diverse range of movements exhibited by individuals, variations in physical capabilities, and the inherent subjectivity involved in the algorithmic approaches used to assess human motion.Constructing an automated system capable of addressing these intricacies is a substantial undertaking, demanding a profound understanding of the diverse spectrum of human movements, physical abilities, and the subjective nature of human motion assessment [6][7][8].
While most existing computational approaches to automatic human motion assessment involve comparing exercise movements to predefined templates or ground truth, the goal for digital assistive rehabilitation systems is to accurately identify and count each exercise repetition to provide assistance and feedback [9,10].These systems must also categorize motion patterns as correct or incorrect, evaluating factors such as speed, direction, quality, and compensatory patterns to replicate the nuanced work of an in-person physical therapist [11].
Existing repositories of rehabilitation datasets present limitations due to their narrow focus and constraints in data modeling.With the objective of providing continuous monitoring and assistance, the present study describes the adaptation of an existing published dataset to cater to the requirements of a digital assistive system capable of remotely aiding patients, effectively simulating the role of a virtual physical therapist [12,13].This innovative approach to labeling human movement data ensures a more effective and personalized rehabilitation experience by tracking the validity of exercise repetitions and providing corrective feedback for any erroneous movement patterns.This new dataset empowers systems not only to track the validity of exercise repetitions but also to offer corrective feedback to rectify any erroneous movement patterns [11].
This paper underscores the pivotal importance of assessing the quality of human motion within the context of rehabilitation.The innovative rehabilitation data modeling presented here sets the stage for the development of more effective and personalized rehabilitation programs, thereby offering vital support to individuals on their path to recovery.In Section 2, we conduct a comparative analysis with other existing studies and datasets.Section 3 outlines the methodology employed for dataset modeling, Section 4 presents the dataset obtained, and finally, Section 5 and Section 6, provide concluding remarks that underscore the valuable contributions of this work.

Related Study
Recent advancements in research have ventured into the realm of mathematical modeling and evaluation techniques for the analysis of human motion in the context of physical therapy.These innovative methods can be broadly categorized into three distinct areas: comparison, categorization, and scoring [14].
Comparison techniques involve assessing a patient's exercise performance by contrasting their motion sequences with predefined template motion sequences.These templates are typically derived from accurate executions of exercises by healthy individuals, clinicians, or patients under clinical supervision.Categorization methods, on the other hand, are designed to classify patient activities into discrete groups, such as correct and incorrect movement patterns.Scoring approaches aim to provide automated evaluations of a patient's motion using personalized clinical or non-clinical scoring systems, utilizing both continuous and categorical scales.
Unlike comparative and scoring analyses, categorization methods play a crucial role in delivering real-time patient feedback and guidance during rehabilitation sessions.The selection of the most suitable computational technique for monitoring and assessing human movements significantly impacts the collection and organization of datasets.
Several datasets have been published in the field of rehabilitation research.Datasets like SPHERE [15], MSR-3D [16], and UTD-MHAD [17] have been compiled and used for monitoring applications, primarily offering graphical or statistical comparisons of patient actions without providing definitive patient evaluations.For example, the SPHERE dataset includes examples of both normal and physically impaired movements for activities like walking, ascending stairs, and sit-to-stand transitions.The KIMORE [18] and UI-PRMD [19] datasets have been designed to develop an evaluation system that assigns a score to exercise performance.
Conversely, other datasets like the Toronto Rehab Stroke Pose (TRSP) [20], Kinect 3D Active (K3Da) [21], and IntelliRehabDS (IRDS) [22] were created to classify movements into categorical classes.For instance, the TRSP dataset contains pose estimations of stroke patients and healthy subjects performing upper limb rehabilitation movements, annotated for regular and compensatory patterns of motion during an assisted-robot rehabilitation session.The IRDS dataset contains gestures of simple rehabilitation movements performed by both healthy and pathological subjects in various conditions (sitting or standing), labeled for correct and incorrect executions.
This work aims to describe a methodology for modeling a rehabilitation dataset that can be adopted to support AI-driven digital assistive systems, capable of assisting and monitoring home programs and providing feedback to improve patient performance.Given the limitations of currently available published datasets in terms of 3 Materials & Methods

Original Dataset
In pursuit of the primary goal of this study to model a dataset supporting AI-driven digital assistive system, a subset of the freely available IRDS dataset underwent manipulation.This selected subset encapsulates repetitions of shoulder movements executed by 29 subjects, comprising 15 patients and 14 healthy controls.Recorded using a Microsoft Kinect One sensor at 30 frames per second, the dataset provides 3D coordinates of 25 body joints and corresponding depth maps for each frame.Each movement is meticulously annotated with exercise type (shoulder flexion, shoulder abduction, shoulder forward elevation), the limb side involved (right, left, both sides), the subject's position (sitting or standing), and the correctness of pattern motion (correct or incorrect).Shoulder flexion and abduction exercises comprises gesture performed separately with the left body side and the right body side, the shoulder forward elevation is simultaneously performed with both sides.
Table 1 summarizes for each exercise the gesture index.What sets the IRDS apart from other datasets is its inclusion of natural motions performed by both patients and healthy subjects, encompassing correct and incorrect patterns.Moreover, the dataset features patients with diverse pathological conditions, such as stroke and spinal cord injury, offering a valuable resource for enhancing the monitoring and assistance of upper limb rehabilitation across various populations, including those with neurological and musculoskeletal disorders.
While the current published form of the IRDS dataset serves as a valuable resource for categorizing exercises and assessing their correctness, it falls short of addressing the needs of digital assistive rehabilitation systems.These systems require a finer evaluation, recognizing repetitions of each exercise and assessing them for the completeness of range of motion (ROM) while analyzing incorrect patterns of motion, as suggested by recent works [24,25].The ultimate aim is to provide a dataset that facilitates the evaluation of whether an exercise is performed correctly according to its prescription or not.
This work outlines the methods employed to model the IRDS dataset, supporting a movement classification approach for the evaluation of human motions.This approach enables real-time feedback to patients regarding the performance of their prescribed shoulder rehabilitation exercises, allowing for a more detailed analysis of each exercise repetition [23].

Data Modeling
The dataset aims to provide a comprehensive evaluation of ROM and compensatory movement patterns exhibited by patients during shoulder exercise rehabilitation sessions.
To enhance the granularity of the analysis, we associated categorical classes of ROM with specific exercise phases in an individual's performance.This enables a detailed examination of each movement, considering both the completeness and correctness of its execution.
In the IRDS dataset, each frame, capturing both correct and incorrect movements, underwent labeling for ROM execution classes.An expert in human physiology assessed depth images to assign classes such as complete concentric phase (up), complete eccentric phase (down), incomplete eccentric phase (incomplete down), and incomplete concentric phase (incomplete up).These labels serve three main objectives in rehabilitation evaluation: segmenting exercise routines into repetitions, assessing each repetition's completion status and providing feedback for performance enhancement, indicating which phase of the repetition is considered invalid-concentric, eccentric, or totally incomplete.
Furthermore, incorrect movements were labeled to distinguish various classes of ROM, allowing for a nuanced evaluation of repetitions not only in terms of motion completeness but also in terms of the percentage of correct and incorrect patterns.This approach provides insights into the quality of motion and facilitates tracking advancements during the rehabilitation process.
The methodology is based on the hypothesis that separating the evaluation of ROM and the recognition of compensatory movements into distinct stages can enhance the accuracy of AI-assisted rehabilitation systems.This separation reduces the complexity of each task by minimizing the number of classes, thereby potentially improving overall system performance.
Formally, we consider a set of input-output pairs D = {(x i , y i )} N i=1 where x i ∈ R n are the N samples of the input feature space obtained from the 3D video motion analysis and y i ∈ C = {C j |1 ≤ j ≤ j} are the class of labels to which these features correspond.The evaluation look for a decision function f : R n → C which given a sample, that in this work contains features of body joints' 3D pose data, determines the output class that includes the kind of execution performed: x → y = f (x, w).Throughout the following, let X(z) = {X t , m t , c t } denote a labeled sequence of N frames of the considered set of the three shoulder exercises z ∈ {S1, S2, S3} , where  The comprehensive dataset composition is outlined in Table 2, providing a detailed breakdown of ROM classes for both the healthy and patient categories.
The distribution of data for each ROM class was systematically examined within each patient and healthy subject category for various gesture types.Figure 2 depicts a bar plot that distinguishes between patient and healthy category IDs for each gesture type.

Discussion
Ensuring the effectiveness of at-home rehabilitation programs relies on providing both qualitative and quantitative evaluations of exercises to patients.Recent advancements in machine learning and computer vision techniques have sparked increased interest in automating the assessment of rehabilitation exercises [26].However, significant challenges and unanswered questions still exist before we can widely deploy datasets to develop AI-assisted digital systems for both home-based and clinic-based settings [27,28].This paper's contribution lies in presenting a dataset focusing on shoulder rehabilitation exercises.Specifically, a subset of the dataset, involving exercises performed by 29 subjects, was meticulously modeled to assess repetition movements at a granular level.This modeling aims to recognize and differentiate between ideal ROM execution and the identification of incorrect patterns of movement.
To achieve this, a comprehensive set of key ROM poses was meticulously labeled for both correct and incorrect movement patterns at each frame of the exercises.The proposed method offers several advantages for future AI applications in exercise assessment and feedback assistance.
A notable aspect of this method is its applicability in clinical practice.Associating specific categories of motion with each phase of exercise performance enables the system to provide real-time feedback that simulates the expertise of a rehabilitation specialist.During the exercise, camera-captured data is processed by a 3D pose estimation algorithm, extracting features that are fed into an AI system for frame-byframe motion analysis according to predefined evaluation rules.The system, trained to identify correct concentric and eccentric ROM poses, can discern individual repetitions within a continuous motion sequence, evaluating them in terms of completeness [11,29].Additionally, the system can recognize incorrect patterns of motion, offering detailed feedback on deviations from correct execution and quantifying the overall exercise performance quality by analyzing pose validity frame by frame.
One strength of this dataset is its generalizability across various subject characteristics, such as age, sex, and physical impairment conditions, and exercise settings (e.g., sitting or standing) and the presence of both correct and incorrect movement patterns.However, it is worth noting that some limitations exist, such as limited subject positioning in front of the camera.To enhance the algorithm's generalizability for rehabilitation exercises, it is important to consider greater variability in subject positioning, including various distances and angles in front of the camera (e.g., sagittal, frontal, and lateral views) [30].
Furthermore, it Is worth noting that the data reveals a notable class imbalance, especially within the category of patients experiencing incomplete execution of ROM.There is a hypothesis that suggests these individuals might encounter physical impairments that impede their ability to attain the full ROM.However, given the absence of specific medical condition data for each subject, a more comprehensive analysis of the underlying factors cannot be carried out at this time.
Future efforts should aim to expand this dataset, serving as a crucial starting point for further research in the development and testing of AI-assisted digital systems for rehabilitation purposes.

Conclusions
The development of digital systems designed to support and motivate patients during home rehabilitation is a prominent focus in rehabilitation technologies.Consequently, creating approaches for the automated assessment of patient performance and functional recovery represents a vital challenge for researchers.
Despite a wealth of previous literature and research on computational approaches for motion evaluation, there has been a relatively limited exploration into modeling rehabilitation datasets for accomplishing the double objective of quality performance evaluation and the recognition of specific types of movement errors to give corrective feedback.This represents a small step toward advancing the development of future systems that offer remote assistance in rehabilitation procedures, which are poised to become ubiquitous and play a pivotal role in augmenting traditional approaches to the evaluation of home rehabilitation programs.

Fig. 1 :
Fig. 1: A chronological sequence of left-side shoulder flexion, labeled within the context of ROM classes, captured in depth images of Patient ID 215.

Fig. 2 :
Fig. 2: Distribution of ROM classes for gesture type and healthy and patients category of subjects.8

Table 1 :
[23]ription of gesture categories included in the new dataset., a subset of the freely available IRDS was chosen for manipulation in alignment with our objectives.The goal is to implement a set of categorical classes of movements associated with the corresponding exercise phase, distinguishing each exercise repetition in finer terms of the quality of the motion pattern[23].

Table 2 :
Description of the new dataset in terms of samples for each ROM class category along the various gesture types.