Introduction

Exoskeletons, defined as mechatronic wearable robots, have brought about the possibility of reconstructing and enhancing human mobility [1,2,3,4]. These devices can be separated into three different categories: assistive exoskeletons, rehabilitation exoskeletons, and augmentation exoskeletons. Exoskeletons from the first two categories are mainly used in the field of medical rehabilitation [5, 6]. These exoskeletons assist/replace the impaired parts of the users, allowing users to live/behave like healthy people while wearing the devices. Alternatively, rehabilitation exoskeletons aim to facilitate rehabilitation treatments and gait training for individuals with disabilities. The third category, augmentation exoskeletons, are mainly used in military [7, 8] and industrial fields [9]. The goal of these exoskeletons is to enable performance of tasks that an able-bodied individual normally would be incapable of, or to reduce the effort required for tasks (less metabolic cost). Moreover, in urban areas, augmentation exoskeletal robots provide ample opportunity to aid professionals such as firefighters [10] and maintenance workers [11], thereby helping them to work more efficiently.

Biomechanical research shows that the legs tend to generate different amounts of mechanical energy during different movement tasks such as walking, ascending ramps or stairs [12]. Human movement patterns vary across the differing tasks [13], requiring the exoskeleton to adapt the gait assistance or assist control to the movement scenario. In addition, the terrain parameters related to each motion scenario also affect the effectiveness of exoskeleton assistance [14]. However, existing researches on exoskeleton control systems have predominantly focused on delivering precise assistance during level treadmill walking [15,16,17,18]. It is vital that the control strategy of the exoskeletons should be properly adapted to and transitioned between different urban terrain environments. Therefore, to employ adaptive assistance strategies in real time according to the urban multi-terrain environments and to provide the necessary assistance torque in an accurate and timely manner, the LLAE must first perform terrain parameter recognition.

The three typical types of terrain for human mobility in urban environments are flat ground, ramps, and stairs. To recognize these terrains, several kinds of terrain recognition methods have been proposed based on different kinds of sensors, such as visible light-based [19, 20], depth-sensing [21,22,23,24] and range-based [25, 26].

Krausz et al.[19] utilized 2D images to detect the edges of rising stair, while Laschowski et al.[20] employed RGB cameras alongside deep convolutional neural networks (CNN) to recognize environment, both yielding terrain classification accuracies greater than 90%. However, they were not tested outdoors, and the dark or bright environment could affect the accuracy of the algorithms.

Qian et al.[24] offset the depth camera point cloud map to a constant ground coordinate system from the camera coordinate system in real-time. They classified the image data using CNN with an average environmental classification accuracy (flat ground and decline stairs) of over 98%. Stairs parameters were investigated by Zhao et al.[21] using a depth camera to enable detection and modeling of stairs and to obtain the height and depth of stairs. However, the algorithm requires point cloud processing which involves high computational complexity.

Range-based recognition methods typically use infrared range sensors or laser range sensors, both of which have advantages and disadvantages. Infrared range sensors emit invisible light and measure over relatively long distances[25], while laser distance sensors emit visible light and have a relatively high accuracy[26]. But both types are all vulnerable to obstacles or weather.

Based on the above challenges, two main issues are considered in this paper: (1) simplifying the algorithm output and reducing the complexity and (2) improving the portability and anti-interference. We proposed an NNR-based TS parameter recognition algorithm for an adaptive assistance strategy in homemade LLAE in urban multi-terrain environments. Here are our main contributions.

1) We defined the terrain parameters into a unified representation: TS, which indicate ramps slope (the angle of ramp inclination) or stairs slope (the angle between the inclined plane of the stair and the horizontal plane). This parameter can provide enough information for the exoskeleton controller.

2) The algorithmic model proposed in this paper relies only on the basic human parameters and IMUs, which is more portable and anti-interference.

The current paper is structured as follows. In the next section, the NNR-based TS parameter recognition algorithm is described in detail. In the subsequent section, the performance of the TS recognition algorithm is verified using the K-fold cross-validation method and wearing our homemade LLAE. Finally, the conclusions and future research directions are presented.

Methods

Fig. 1
figure 1

The relationship between the algorithm in this paper and the LLAE assistance strategy for urban multi-terrain environments

Full size image

This paper proposed an algorithmic framework for TS parameter recognition in LLAE systems used in urban multi-terrain environments, with the aim of providing enough information for the exoskeleton controller, reducing computational complexity and improving immunity to interference. Figure 1 illustrates the relationship between the proposed algorithm in this paper and the LLAE assistance strategy for urban multi-terrain environments. The NNR-based TS parameter recognition algorithm utilizes basic parameters of the human body and the lower limb joint motion posture, which is measured by the IMUs integrated in the LLAE, is used as input to build a recognition model that outputs the TS. Combined with the gait phase recognition algorithm, this enables the LLAE to adaptively employ assistance strategies on various terrains and deliver the required torque assistance to the user’s body accurately and promptly.

Fig. 2
figure 2

Definition of “TS”

Full size image

To solve the algorithm output redundancy problem, we simplified all parameters of the individual terrain as “TS.” Since walking on a flat ground has a similar state of motion to walking on a ramp, this paper considered the flat ground as a ramp with a slope of 0\(^\circ \) and refers to the angle of ramp inclination or the angle between the inclined plane of the stair and the horizontal plane as “TS,” as shown in Fig. 2. Analysis of human knee moment curves variations in ramps (Fig. 3a) and stairs (Fig. 3b) terrain showed that a consistent correlation between knee moment and TS. This observation suggests that an isometric adjustment can be applied at the knee moments based on changes in TS to achieve the purpose of gait control and posture adjustment by the LLAE system. Thus, the single parameter of “TS” is capable of providing adequate information for exoskeleton control in urban multi-terrain environments.

Fig. 3
figure 3

Relationship between human knee joint moments and TS: a ramps terrain; b stairs terrain

Full size image
Fig. 4
figure 4

Construction of a NNR-based TS parameter recognition model

Full size image

A fully connected neural network was utilized to construct a recognition model for recognizing TS parameters, as depicted in Fig. 4. The neural network architecture consists of three layers, namely an input layer, a hidden layer, and an output layer. The basic parameters of the human body such as thigh length, calf length, foot length and hip width,and the lower limb joint motion posture such as thigh angle, calf angle, foot angle, hip coronal rotation angle, are taken as inputs to the input layer while height and weight are also added to the inputs to improve the recognition accuracy of the algorithm. Following the acquisition of TS parameters, The type of terrain can be recognized by analyzing the data on the motion posture of the person’s feet at that moment, as shown in Fig. 5 and calculated using (1).

$$\begin{aligned} \begin{aligned} terrain={\left\{ \begin{array}{ll} ascending\, stairs,\,\theta>0\,and\,\theta _{f}>0\,and\,\theta _{h}\approx 0\\ descending\, stairs,\,\theta<0\,and\,\theta _{f}<0\,and\,\theta _{h}\approx 0\\ ascending\,ramps,\,\theta >0\,and\,\theta _{h}\approx \theta \\ descending\,ramps,\,\theta <0\,and\,\theta _{h}\approx \theta ,\\ \end{array}\right. }\\ \end{aligned} \end{aligned}$$
(1)

where \(\theta \) is the TS, \(\theta _{f}\) and \(\theta _{h}\) are the angles between the front or rear foot and the ground when the front foot touches the ground respectively. Based on this determination, the current movement can be classified into four walking tasks: ascending ramps, descending ramps, ascending stairs, and descending stairs.

The model building and training process is described as follows:

(1) Data pre-processing. Kalman filtering [27] of data to reduce the effects of noise. As the range of data varies across dimensions, to mitigate the effect of biased data on the results, data normalization is performed, see (2):

$$\begin{aligned} x^{\prime }=\frac{x-\min }{\max -\min } \end{aligned}$$
(2)

where max is the maximum value of this dimension, min is the minimum value of this dimension, x is the original data and \(x' \) is the normalized result.

(2) Motion posture data pre-processing. The labels represent the theoretical TS output values. Since this separation has already been performed in the database, it only required extracting the labels from the database.

(3) Dividing the dataset. The data from 10 individuals are divided into 5 segments, each containing data from two individuals. The K-fold cross-validation method (K=5) is employed for training and validation. In each training session, four data segments are utilized as the training set, while one data segment is designated as the validation set.

Fig. 5
figure 5

The relationship between human feet motion posture and TS

Full size image

(4) Model building. The NNR model is constructed using a sequential structure, consisting of a 4-layer structure: a 13-dimensional input layer, two hidden layers with 64 and 32 nodes respectively, and a one-dimensional output layer.

(5) Model training. The optimization iterator is set with an iteration time interval of 0.001s, and the loss function used is the mean squared error (MSE). The training process involve 500 epochs, and the early termination condition is defined as the loss function not decreasing for 100 consecutive epochs.

Experiments and results

Motion posture database construction

Table 1 The basic parameters of the participants
Full size table

Ten healthy adults without motor impairment were recruited to conduct motion posture database construction experiments. The basic parameters of the participants are shown in the Table 1.

Fig. 6
figure 6

Data acquisition environments: a ramp environment; b stair environment

Full size image

The participants were instructed to conduct sessions in three different terrains: flat ground, ramps, and stairs. The flat ground walking session took place on the floor. For ramps and stairs sessions, custom-built simulated platforms were utilized, as depicted in Fig. 6a, b. The simulated ramp platforms were made using stainless steel square tubes that were welded together. The side length of these platforms was approximately 1.5 m. These platforms were covered with approximately 2 cm thick wooden boards and non-slip simulated turf to ensure safety and stability. Three ramp platforms were created, each with slopes of \( \pm \)5\(^\circ \), \( \pm \)10\(^\circ \), and \( \pm \)15\(^\circ \). On the other hand, the simulated stair platform was constructed from four wooden boxes, each with a thickness of 1.5 cm, 60 cm in width and 17 cm in height. By adjusting the relative distances between the boxes, stairs with slopes of \( \pm \)20\(^\circ \), \( \pm \)25\(^\circ \) and \( \pm \)30\(^\circ \) can be simulated. Experimental TS can be achieved as shown in Table 2.

Table 2 Statistics of TS parameters for experimental terrains
Full size table
Fig. 7
figure 7

Motion posture acquisition module and Gait recognition module: a motion posture acquisition module; b gait recognition module; c the position of the sticking point

Full size image

The participants wore a motion posture acquisition module and a gait recognition module during the experiment, as depicted in Fig. 7a and b, the position of the sticking point is shown in Fig. 7c. The motion posture acquisition module consisted of seven Xsens-IMUs and a USB receiver. Six Xsens-IMUs were attached to the outer side of the thighs, calves and feet, while one Xsens-IMU was placed to the tailbone to capture real-time posture data during participants movement. The gait recognition module was used to detect the moment of the foot touches the ground. A computer was used to receive feedback data from both modules simultaneously, using the participants posture data corresponding to the moment of the front foot touch the ground as the single frame of the data set for the NNR model. To achieve the synchronization between the motion posture acquisition module and the gait recognition module, the IMUs of the gait recognition module and the Xsens-IMUs attached to the feet were physically integrated. Subsequently, the synchronization was achieved by comparing the acceleration signals of both.

Fig. 8
figure 8

The process of construction a motion pose database: a initial state; b walking on flat ground; c ascending ramps; d descending ramps; e ascending stairs; f descending stairs

Full size image

Each participant completed the sessions in 5 different walking tasks at a self-selected rate: walking horizontally, ascending ramps, descending ramps, ascending stairs, and descending stairs. Before starting the session, each participant’s initial state needed to be calibrated. During this calibration phase, the participant was required to maintain an upright body posture with their feet together and wait for the data recording process to be completed, as illustrated in Fig. 8a. For each session on flat ground (Fig. 8b) and ramps (Fig. 8c, d) terrain, the body posture data at the moment of the front foot touch the ground was selected as the single frame data set. For stairs terrain (Fig. 8e, f), the posture data when participants’ front foot touched the second step was selected as the single frame data set. Ten sessions were carried out for each terrain.

Table 3 Statistics of data collected from the experiment
Full size table
Fig. 9
figure 9

The mean of the motion posture data from Subject 3: a angles of left leg and hip coronal rotation; b angles of right leg

Full size image

After the experiment, a total of data were collected as shown in Table 3. Where a negative stair TS indicates that the subject is descending the stair. A total of 1300 sets of motion posture data were obtained for different TS. As an example, the mean of the 10 lower limb joint angles collected from subject 3 is shown in Fig. 9a, b.

K-fold cross-validation of NNR-based TS parameter recognition model

This paper used K-fold cross-validation (K = 5) method to validate a NNR-based TS parameter recognition model. The validation process is shown in Fig. 10.

Table 4 shows the detailed results of the 5 cross-validations. To assess the performance of the NNR model during training, two evaluation metrics were used: the MSE and the mean absolute error (MAE) of the validation set. The results showed that the MSE of the model for the five validations was 10.433, indicating a small difference between the predicted and actual values of the model; the MAE on the validation set was 2.09\(^\circ \), demonstrating that the NNR model effectively identify TS.

The predicted data at k = 1 were also analyzed, and the predictions are shown in Fig. 11a. The x-axis represents the actual TS values and the y-axis represents the predicted TS values. It can be observed that the predicted results are more concentrated and close to the true values in the interval from \(-15^\circ \) to 15\(^\circ \) in the ramp terrain, while the predicted results are more scattered in the stair terrain. The MAE of the predicted results is calculated and shown in Fig. 11b, with MAE below 2.487\(^\circ \) for flat ground and ramps terrain. From Fig. 11b, it can be seen that for descending stairs terrain the MAE stays within 1.776\(^\circ \), but for ascending stairs terrain, there is a small increase in MAE with a maximum of 4.016\(^\circ \). This result shows better prediction accuracy for both flat and sloping terrain NNR model. In summary, the MAE across all terrain was 2.09\(^\circ \), indicating that the NNR model performs well in TS recognition and can provide sufficient data to support the subsequent design of the exoskeleton controller.

Fig. 10
figure 10

The process of K-fold cross-validation

Full size image

Performance of TS parameter recognition algorithms on LLAE

Introduction to the exoskeleton system

The homemade LLAE system is shown in Fig. 12. The motor is a TQ-Group frameless motor ILM50x14, which is powered by 48 V, has a rated speed of 3870 rpm, a rated torque of 0.54Nm and a peak torque of 1.75 Nm at the rated current of the motor driver (15A) and is amplified by a harmonic reducer with a reduction ratio of 1:50, resulting in a maximum system output torque of approximately 65Nm. Assuming a maximum wearer weight of 80 kg and a maximum assist ratio of 30%, the maximum required torque is 28.1Nm, thus leaving a large margin for the system to meet this requirement. The exoskeleton system has been verified to have a maximum following speed of 2 m/s, which meets the human walking speed requirement. The execution system consists of a main control board and two node boards. The main control board is a BeagleBone Black Wireless and the node boards are homemade boards. The node boards are connected to the main control board via CAN bus and communicate using the CANopen protocol at a frequency of 1 MHz.

Table 4 K-fold test for NNR model
Full size table

Experiments and results

Ten healthy subjects (age: 22\(^{\sim }\)25 years, height: 1.59\(^{\sim }\)1.75m, weight: 59\(^{\sim }\)68 kg) without motor impairment were recruited to wear the LLAE to verify the performance of the TS parameter recognition algorithm.

Two ramps and two stairs have been selected on campus for this section, including a 5\(^\circ \) barrier-free access (Fig. 13a), a 9\(^\circ \) car park access (Fig. 13b), a 23\(^\circ \) outdoor stair (Fig. 13c) and a 30\(^\circ \) outdoor stair (Fig. 13d).

During this experiment, the LLAE was in a zero moment following state. Prior to each session, participants first underwent a brief period of acclimation to the exoskeleton until it was felt that the exoskeleton did not impede the movement of the participants’ lower limbs. The IMUs integrated into the LLAE device acquired the subjects’ motion posture data during walking. The installation position of the IMUs is shown in Fig. 12, where the IMU measuring the hip rotation angle is installed in the main control box. The exoskeleton robot detects the human motion posture and recognizes the TS parameters and feeds the data to the computer via a wireless module for easy observation. The calculation process of the TS parameter recognition algorithm in the LLAE system is shown in Fig. 14.

For each session, subjects ascended a ramp (Fig. 13a, b) or stair (Fig. 13c, d) at a self-selected comfortable speed, executing a slow turn after reaching the top, then descending and finally stopping at the starting point. Ten sessions were carried out in each terrain.

Experimental results were recorded for 10 subjects and the actual TS values of the terrain and the predicted TS values are shown in Fig. 15a, the MAE of the predictions was visualized in Fig. 15b. From Fig. 15b, it can be seen that the MAE of the TS parameter recognition algorithm on LLAE was 3.73\(^\circ \), slightly higher than the results obtained from testing the NNR model alone (2.09\(^\circ \)). The increase in MAE could be attributed to the fact that most of the IMUs were mounted on the exoskeleton, which differs from the location used in the original dataset. The average error in TS parameter recognition on ramps terrain was 3.11\(^\circ \), compared to 4.34\(^\circ \) on stair terrain, indicating that the TS parameter recognition algorithm has better recognition on tamps terrain.

In conclusion, the NNR-based TS parameter recognition algorithm continued to demonstrate effectiveness on LLAE.

Fig. 11
figure 11

TS prediction results at k = 1: a The actual TS values and the predicted TS values; b The MAE of TS predictions

Full size image
Fig. 12
figure 12

LLAE system

Full size image
Fig. 13
figure 13

The exoskeleton wearing experiments: a Ramp 5\(^\circ \); b Ramp 9\(^\circ \); c Stair 23\(^\circ \); d Stair 30\(^\circ \)

Full size image

Discussion and conclusion

This paper proposed and developed the algorithm for TS parameter recognition. We simplified all parameters of the individual terrain as “TS,” employed a NNR model that utilized the basic parameters of human body and lower limb joint motion posture data as input to predict the TS parameters. In terms of accurately recognizing TS parameters, the model was cross-validated using K-fold with an average error of 2.09\(^\circ \), and further validated by exoskeleton wearing experiments with an average error of 3.73\(^\circ \). Comparing this paper’s algorithm with the current commonly used terrain recognition algorithms, as shown in Table 5, it can be seen that most of the current algorithms are based on RGB cameras, depth cameras, or laser sensors, which are of high computational complexity, low portability, and susceptible to line-of-sight interference or illumination. Moreover, most of the algorithms only achieve terrain classification, and a few of them can achieve parameter recognition, but only for a single terrain [20, 22, 28], and cannot be applied to a variety of terrains. In contrast, the algorithms in this paper are adapted for parameter recognition in multi-terrain environments and are similar or higher in accuracy than the above mentioned methods, and only rely on the IMU on the exoskeleton device, which improves portability while avoiding the effects of external line-of-sight obstruction or illumination.

Fig. 14
figure 14

The calculation process of the TS parameter recognition algorithm in the LLAE system

Full size image
Fig. 15
figure 15

TS prediction results: a The actual TS values of the terrain and the predicted TS values; b The MAE of the predictions

Full size image
Table 5 Comparison of terrain recognition methods
Full size table

In fact, the terrain parameter identification algorithm proposed in this paper has some limitations. The prediction MAE for ramps terrain in the exoskeleton wearing experiment was 3.11\(^\circ \), while the prediction MAE for stair terrain was 4.34\(^\circ \), which was slightly higher than ramps terrain. The reason for the difference in predictions on stair terrain versus ramps terrain may be due to the greater amplitude of human lower limb movement in stair terrain, which could potentially impact the measurements from the IMUs. In addition, the angle of the human rear foot when walking on ramps terrain is more consistent with the TS, so NNR model may give greater weight to this dimension of data, which is one of the reasons for the variability in TS prediction under stair terrain versus ramps terrain. In future work, we may need to increase the number of IMUs or combine other algorithms to reduce the error in stair terrain parameter identification to further improve the accuracy and practicality of the algorithm in this paper.

In addition to this, we will also explore the possibilities of the present algorithm when applied to discontinuous motion scenarios such as squatting or jumping [30, 31]. These types of movements pose unique challenges in terms of terrain parameter recognition. This could involve designing algorithms that can effectively handle the rapid changes in motion and posture during activities like squatting or jumping. Additionally, new techniques may need to be investigated to accurately recognize terrain parameters in such dynamic and non-continuous motions. By addressing these limitations and expanding the algorithm’s capabilities to cover a wider range of motions, we can enhance the overall accuracy and applicability of the TS parameter recognition algorithms in real-world scenarios.

In summary, we proposed a NNR-based TS parameter recognition algorithm. The algorithm relies only on the basic human parameters and IMU, and refers to all parameters of the individual terrain as “TS.” According to the experimental results, our algorithm outperforms other methods in terms of feasibility of TS parameter recognition, with simpler output and lower computational complexity. To further complete the LLAE-assisted strategy for urban multi-terrain environments, the design of an adaptive oscillator-based gait phase recognition will follow.