Introduction

Markerless motion capture (MMC) technology has been developed to avoid the need for marker placement during tracking and analyzing human movement [1]. By elimination of the time-consuming marker placement procedure, motion capturing experiment can be performed in a more convenient way [2]. Without the constraints that brought by body markers on movement, the development of MMC technology allows the capture of a more lifelike human motion in the environment, in a more natural way, and with the feature that it uses more portable and low-cost sensors compared to marker-based multi-camera systems [3], MMC in turn creates the potential of additional applications.

Previous studies have been conducted to compare the accuracy of MMC and body-marker-based analysis systems. Bonnechere et al. [4] compared the measuring accuracy of full body scanning by Microsoft Kinect 3D scanner software versus that of a high-resolution stereophotogrammetric system, which is a marker-based system in the healthy population. They concluded that Kinect is a reliable markerless tool that is suitable for use as a fast estimator of morphology. Schmitz et al. [5] validated the accuracy of Kinect in measuring knee joint angle of a jig by comparing its measurement using a digital inclinometer that acted as a ground-truth, and they reported that the performance of the Kinect system was satisfactory in terms of knee flexion and abduction. The accuracy of using a smartphone as a measurement system for joint angle has been reviewed by Mourcou et al. [6], who concluded that smartphone applications are reliable for clinical measurements of joint position and range of motion (ROM).

Earlier in 2006, Mündermann et al. [7] described several methods of MMC video processing modules including background separation, visual hull, and iterative closest point methods, etc., and pointed out that MMC has the potential to achieve a level of accuracy that facilitates the biomechanics research of normal and pathological human movement. Together with the reliable performance of MMC technology in the measurement of joint angle and body movement as reflected by [5, 6], it is suggested that the MMC system can be further applied to the rehabilitation field to measure patients’ motor function. However, the actual application of MMC technology for clinical measurement in rehabilitation is still at a preliminary stage. Most of the extant studies have focused on calibration of the MMC system or on validating the MMC system only on healthy persons. Applied research on the actual use of MMC technology in measurements in patient groups has been very diverse: Vivar and the teams [8] applied MMC technology in people with Parkinson’s disease (PD) to detect and classify their tremor level, while Gritsenko et al. [9] used Kinect as the MMC system to measure the shoulder ROM for women breast cancer patients after surgery. Instead of applying MMC technology in adults, Chin et al. [10] assessed the level of proprioceptive ability in children with cerebral palsy by using Kinect as the MMC system to measure the arm position of both healthy children and children with unilateral spastic cerebral palsy (USCP). These researchers found significant differences between the proprioceptive ability of the typically developing children and the children with USCP, as measured by Kinect, thus suggesting that MMC technology has the potential to be useful as a clinical measurement tool for proprioception.

Despite these trials, however, studies on the applications of MMC technology in clinical evaluation are still preliminary and limited in number, and it remains inconclusive how MMC technology can benefit therapists, patients, or the healthcare system, in terms of measuring patients’ conditions. Review studies have been conducted on the use of MMC technology in rehabilitation training, but not in regard to its use in clinical measurement including application of MMC technology in clinical assessment and detection of kinematic parameters that assist in disease diagnosis [11]. Mousavi Hondori and Khademi [12] reviewed the clinical impact of Kinect in rehabilitation, but their study did not cover other types of MMC technology. Therefore, to investigate the current uses of MMC technology as an assessment tool in the healthcare field, in this review we put less focus on the engineering components and attempted primarily to determine the current evidence for using MMC as a measurement tool, in order to further explore the potential benefits of MMC technology in rehabilitation evaluations. In this paper, we define clinical measurement as identification and measurement of movement kinematics in a clinical population [13], while MMC technology include systems and methods that capture and analysis movements without the need of marker placement, including video-based analysis. This systematic review further investigated: (1) the types of patients to whom MMC technology has been applied; (2) the contents of the MMC measurements; (3) the types of MMC systems used; and (4) the efficacy of these MMC systems as measurement tools.

Methods

Search strategy

A systematic computerized literature search was conducted by one of the authors (WTL) in PubMed, Medline, CINAHL, CENTRAL, EMBASE, and IEEE. Only peer-reviewed articles were included. The search keywords used in each database were “Markerless Motion Capture OR Motion Capture OR Motion Capture Technology OR Markerless Motion Capture Technology OR Computer Vision OR Video-based OR Pose Estimation AND Assessment OR Clinical Assessment OR Clinical Measurement OR Assess.” A manual search was also conducted that included searching Google Scholar using the same keywords, and the reference lists of the previous systematic reviews were also screened. The published data were not limited, and the last search took place on March 6, 2023.

Inclusion criteria

Studies were included if they met certain inclusion criteria. Specifically, the studies had to: (1) be peer-reviewed; (2) apply MMC technology for measurement; (3) involve subjects with symptomatic conditions; (4) have any quantitative study design except systematic reviews; (5) include at least one assessment item for clinical evaluation; and (6) be published in English.

Exclusion criteria

Studies were excluded if they met any one of the following exclusion criteria: (1) studying only healthy persons; (2) focusing only on calibration of the MMC system; (3) applying MMC technology only in rehabilitation training; or (4) not reporting results of an assessment evaluation.

Data extraction

The information we assessed included: (1) the types of MMC systems used in the studies; (2) the conditions of the participants that underwent the measurement, such as diagnoses or disabilities; and (3) the contents of the measurements conducted. The interpretations of the studies’ results were extracted and are presented in a summary table (Table 1). The contents of the measurement included the body functions or body parts that were measured, and the context in which the assessment was conducted.

Table 1 Details of the selected studies
Full size table

Results

Literature search and study characteristics

A total of 4283 articles were identified, 278 of which were selected for full-text reading after removal of duplicates and irrelevancies, according to their abstracts (Fig. 1). After next excluding 213 articles on the basis of the inclusion and exclusion criteria, 65 studies remained and were included in the final review (Fig. 1). More than 40% of the studies applied MMC technology to assess a patient population with PD (n = 28) [8, 14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40]. Two other diseases that had commonly been measured by the MMC system were cerebral palsy (CP) (n = 6) [10, 41,42,43,44,45] and stroke (n = 6) [46,47,48,49,50,51]. Four other studies focused on children with autism spectrum disorder (ASD) (n = 4) [52,53,54,55] while there are two studies focused on patients with schizophrenia (n = 2) [56, 57] and patients with dementia (n = 2) [58, 59] respectively. The rest of the studies were conducted on various other diseases: Fragile X syndrome (FXS) [60], chronic neck pain [61], breast cancer [9], spinal cord injury (SCI) [62], amyotrophic lateral sclerosis (ALS) [63], adhesive capsulitis of shoulder (AC) [64], dystrophinopathy [65] and neuromotor diseases [66]. There were also studies that had been conducted on wheelchair users (n = 2) [67, 68], people awaiting total knee arthroplasty (TKR) [69], patients with gait disturbance [70], patients with neurodevelopment disorders (NDD) [71], patients with tremor [72], patients with Duchenne muscular dystrophy (DMD) [73], patients with cervical dystonia (CD) [74] and patients with a variety of diagnoses [75]. Table 1 summarizes the 65 selected studies.

Fig. 1
figure 1

Flow chart for selection of the studies for this review

Full size image

Body function/body part being measured

Of the 28 studies that assessed the PD population by using MMC technology, fourteen measured the hand’s motor conditions to classify or to predict the severity of PD [8, 14,15,16,17,18,19,20,21,22,23,24, 39, 40]. These fourteen studies used the PD features of bradykinesia and tremor, as reflected during hand movements such as a finger-tapping exercise, to train machine-learning models to serve as classifiers. Of the remaining fourteen studies, four focused on using whole-body motion to classify PD [25,26,27, 38], and the other ten measured gait features to detect gait disorder in people with PD [28,29,30,31,32,33,34,35,36,37]. The measured body function for the CP population by the MMC system included gait pattern, trunk mobility, general body movement, fidgety movements, and the level of proprioceptive ability [10, 41,42,43,44,45]. The six studies on stroke survivors applied MMC technology to measure their upper limb movement, including their motor function, movement velocity, and joint angle [46,47,48,49] as well as lower limb movement gait parameters and walking pattern [50, 51]. The studies that worked on the ASD population mainly focused on prediction of diagnosis of ASD by children’s behavioral patterns [52,53,54,55]. The measured areas in the studies that applied MMC technology in patients with other types of diseases varied, and the details are listed in the summary table (Table 1).

Details of measurement and efficacy

The applications of the MMC systems in measurement were classified into several categories. Sixteen out of the 65 selected studies used MMC technology in symptoms identification in disease populations [8, 14, 15, 17, 21, 25, 30,31,32, 36, 39, 40, 50, 53, 54, 72]. Butt et al. attempted to distinguish patients with PD from healthy subjects by features of their hand movements, reporting that their Leap Motion Controller (LMC) system together with the machine-learning models did not provide a reliable measurement for the PD symptoms [15]. Fifteen studies focused on comparing the movement patterns of the disease populations and a healthy population, with all of them reporting a significant difference in at least one of the measured parameters including gait parameters, hand movement patterns, head movement patterns and general body movements [16, 20, 26, 28, 41, 49, 52, 55,56,57, 60, 61, 66, 70, 71]. Fifteen studies applied MMC technology to detect and identify movement limitations or specific movement patterns of patients with certain diseases, and significant parameters that indicate movement abnormity including bradykinesia, shuffling gait, abnormal walking pattern, and tremor were identified [9, 19, 24, 29, 33,34,35, 37, 42,43,44, 51, 59, 63, 73]. Two studies used the MMC system to measure range of motion (ROM), and they suggested MMC could be an alternative to the goniometer as a tool for ROM assessment [64, 75]. Three studies used the MMC system as a tool to analyze the wheelchair propulsion skills of wheelchair users [62, 67, 68]. Ten studies correlated or compared the MMC measurements with clinical assessment scales [18, 23, 27, 38, 46,47,48, 58, 65, 74]. Among the other three studies, one applied MMC technology in a comparison with the optic marker system [45], one used it to measure leg length [69], and one used it as a tool to assess proprioception [10]. Only one study reported unsatisfactory results, claiming that the use of the MMC system alone to measure leg length was not accurate [69]. The details are listed in the summary table (Table 1).

Types of MMC systems

Twenty studies used Kinect in their research, thus making Kinect the most popular MMC system used in the selected studies [9, 10, 30, 31, 43, 45,46,47,48,49, 58, 59, 62,63,64,65, 67, 68, 70, 75]. Sixteen studies used camera including RGB camera, digital video camera, GoPro camera and webcam [16,17,18, 20, 24, 25, 29, 32, 33, 35, 37, 41, 51, 54, 55, 71], while twelve studies analyzed patients’ movement by using smartphone or mobile tablet videos [14, 19, 21, 22, 27, 36, 38, 44, 50, 57, 72, 73]. Six studies performed the motion analysis from YouTube video or video recordings that captured by nonspecific capturing device [23, 34, 52, 53, 66, 74]. Five studies used the leap motion controller (LMC), an optical hand-tracking module [8, 15, 17, 39, 40]. The rest of the studies applied the BioStage™ System (Organic Motion Inc., N.Y., USA) (n = 3) [56, 60, 69]; the DARI Motion platform’s motion capture system (n = 1) [26]; the 4DBODY System (n = 1) [42], and a nonspecific customized motion capture system (n = 1) [61]. Table 2 describes and compares the characteristics of these seven types of MMC systems in terms of their mechanisms, set-up procedures, relative costs, the body part(s) that can be captured, and the systems’ methods of data extraction and analysis.

Table 2 Comparison of the MMC systems
Full size table

Discussion

Our results revealed that most of the research applications of an MMC system for measurement were with patient groups with physical disabilities, and more than half of the studies assessed the PD and CP populations. A possible reason for this trend could be that both PD and CP have obvious and well-defined physical signs and symptoms and abnormal movements. PD is characterized by the presence of tremor, bradykinesia, and rigidity during movement [76], whereas patients with cerebral palsy demonstrate spasticity, ataxia, rigidity in movement, and the like [77]. The characteristic types of movement in these two groups of patients might be especially favorable for detection and analysis by the MMC system because of the significant homogeneity in the patients’ movement patterns. Applications of an MMC system for measurement in other kinds of physical disabilities have been limited, and that was the case in this review, but the heterogeneous disease types that were evaluated in the selected studies suggest the possibility of a high variety of generalized uses of MMC technology in assessing different types of patients.

In addition to the use of MMC systems in applications involving physical disabilities that demonstrate observable physical symptoms, it was noteworthy that such systems were also applied in patients with mental illness and NDD, in an attempt to deduce the presence of movement markers for mental disorder and the behavior associated with NDD. Experimental use of MMC technology in patients with mental illness and NDD suggests an entirely new trend for the application of MMC technology in the clinical field. Heretofore, motion tracking has been used in targeted patients with physical disabilities, because the analysis of their movements can provide necessary information and data about their level of impairment, and that in turn can indicate their recovery progress. However, although clinical observations have demonstrated that there is a difference between the movement patterns of patients with mental illness and those of healthy individuals, application of motion capture systems to assess the physical ability of patients with mental illness is still quite limited [78]. Since traditional marker-based systems for motion analysis are time-consuming to set up given that it requires calibration procedure and attachment of markers on the body, using the traditional motion analysis marker systems might not be cost-effective to study the motion dysfunctions in patients with mental illness whose cognitive functions are predominantly affected. In fact, previous studies on motion detection of patients with mental illness adopted the fuzzy movement method, and precise actions and movement patterns have been less emphasized [79]. Therefore, the development of MMC technology allows motion capture in a more cost-effective way, and that improvement may facilitate future scientific investigations of movement patterns and motor functions in patients with mental illness. Identifying the risk of NDD by extracting the children’s behavioral features with the help of computer-vision technology also proposed a new direction of early screening of NDD [80], in which children’s developmental conditions can be closely monitored in their familiar environment without the need of attachment of markers on the infants’ body. Similarly, the studies that have applied the MMC system to compare the motion patterns of a disease population and a healthy population provide evidence for the technology’s use to identify biomarkers for certain diseases. MMC technology may also contribute to the development and use of big data for future AI screening for diseases, based on body movements. The combination of MMC technology and a machine-learning algorithm in classification of CP in infants by Nguyen-Thai et al. [44] is one of the good examples that demonstrates how MMC technology can help in the preliminary screening of diseases. Compared with screening methods for traditional diseases, which depend heavily on behavioral observations by parents or on subjective self-reported questionnaires [81], MMC technology, which identifies symptoms via movement detection, could be a more objective method for early screening for diseases, facilitating early identification of a disease and thus improving the prognosis for rehabilitation, as well as providing a tool for evaluation before and after rehabilitation.

In contrast to using MMC technology for symptoms identification or for detection of differences in movement patterns between disease groups and their healthy counterparts, other studies applied MMC technology as a direct clinical measurement tool. Although the use of the MMC system to measure leg length was found to be inaccurate [69], the use of Kinect to measure ROM was found to be reliable [64, 75]. These findings suggest that MMC technology might have the potential to serve as an alternative clinical assessment tool. MMC technology also provides a new approach to assessing functional or cognitive abilities, such as objectively evaluating proprioception, which previously relied heavily on manual evaluations by rehabilitation therapists. However, future studies on the measurement accuracy and the validity of MMC technology as a clinical measurement tool are warranted.

Microsoft Kinect, the most commonly used MMC system in the studies in this review, is a relatively low-cost, commercially available system that captures and analyzes whole-body movement. Kinect enables the capture of real-time whole body gross movements, but it appears to be less sensitive in tracking fine hand movements [82]. Although many of the studies used Kinect in their MMC measurements, the system has been out of production since 2017 and was no longer supported by the Xbox Series X, as announced by Microsoft [83]. Future rehabilitation assessors that wish to use MMC technology may have to consider using other kinds of MMC systems, or the newly developed Azure Kinect. Our review also found that the most recent studies adopted the use of camera, smartphone, or video clips from the internet in conjunction with pose estimation algorithms and motion analysis algorithm, which has been rapidly developed in the recent years, to capture images and analyze motion. Human pose estimation method is a way of identifying and classifying human joints position using computer vision, for example, the open-source libraries OpenPose and PoseNet for human pose estimation are widely adopted in motion analysis [84]. With the development of human pose estimation database containing various types of movement datasets, accuracy of pose estimation from video clips can be further enhanced by using a large set of training data. This facilitates the use computer vision methods for motion analysis in video clips captured by portable and low-cost camera rather than using specific sensors in the traditional way. The use of machine-learning algorithms allows meaningful information such as kinematic data to be extracted directly from regular videos, thus making the use of MMC technology much easier in motion capturing in a natural environment without the need to buy any extra hardware devices. Human pose estimation technology such as Convolutional Pose Machines (CPM) and convolution neural network (CNN) based methods which allow extraction of human movement information directly from video clips have been repeatedly tested by researchers [85, 86] while human pose estimation application on analyzing movement in the disease populations were reported to be useful by the studies in our review [14, 16,17,18,19,20,21,22,23,24,25, 27, 29, 32,33,34,35,36,37,38, 41, 44, 50,51,52,53,54,55, 57, 66, 71,72,73,74]. Given that such trajectory extraction method is in rapid evaluation and is becoming more mature for promising identification of posture [87,88,89], using hand-held camera or smartphone as the MMC system would be especially beneficial for understanding the motor performance of individuals in their daily living tasks, hence providing valuable information on levels of impairment and on the constraints that patients might encounter in their activities of daily living in their real-life environment. It is understandable that individuals, particularly young children and older people, might behave differently when they are placed for motion capturing in an unfamiliar laboratory or a simulated environment, thus risking the possibility that the motion analysis might not truly reflect the individuals’ actual movement patterns [90]. The use of a smartphone camera combined with an algorithm for analysis could provide a solution to that problem and suggests the feasibility of assessing patients’ daily movements through an MMC combination of a smartphone and an advanced algorithm. Since it does not require additional hardware for motion capturing, such a system would further broaden MMC technology for measurement and clinical assessment in the field of rehabilitation.

Limitations of the current MMC technology’s applications for clinical measurement

Although the use of MMC system in motion capturing is becoming more common in movement measurement and helps us extend the application of MMC technology to clinical use, the technologies used for analyzing movement and distinguishing motor patterns are not yet generalized. Extracting and processing the data from MMC devices video files is still complicated and time-consuming, preventing the approach from being user-friendly for therapists to adopt as a quick clinical measurement tool. Further investigation is needed in order to design and develop a platform or software that can accurately analyze the movement patterns from videos in a more user-friendly and accurately way so as to further extend its application by clinicians. Although most of the studies that we included reported detecting a significant difference between the motor parameters of healthy control groups and those of disease populations, and while the identification of physical symptoms by the MMC system was also reported to be mostly effective, the sample sizes adopted by these studies were too small. A reliable AI tool for disease screening and classification will need to be trained and tested from a large set of data, to provide better specificity and sensitivity. In order to make use of MMC technology-assisted AI screening and early detection of diseases, a larger database that records movement patterns of both the disease population and the healthy population must be developed. Research on the development and selection of a suitable machine-learning or deep-learning model for classification is also needed. Ultimately, a cost-effective and accurate method for early patient screening will help therapists to identify individuals at risk and involve them in further, in-depth assessments, so that subsequent interventions can be made as early as possible. Moreover, it has been suggested that telerehabilitation could incorporate the use of MMC technology as a measurement tool for assessing and monitoring patients’ prognosis and recovery, thus offering an objective and precise evaluation of patients’ rehabilitation progress.

Conclusions

This review explored the current uses of MMC technology to perform assessments in clinical situations. Most of the studies in the review combined MMC technology and a classification algorithm to perform symptoms identification for disease populations or to detect the differences in movement between disease groups and their healthy counterparts. Findings from these studies revealed a potential use of MMC technology for detecting and identifying disease signs and symptoms. The method might also contribute to early screening by using AI and big data to screen for diseases that lead to physical or mental disabilities. Further studies are warranted to develop and integrate MMC system in a platform that can be user-friendly and accurately analyzed by clinicians to extend the use of MMC technology in clinical measurement.