Unsupervised early prediction of human reaching for human–robot collaboration in shared workspaces
This paper focuses on human–robot collaboration in industrial manipulation tasks that take place in a shared workspace. In this setting we wish to predict, as quickly as possible, the human’s reaching motion so that the robot can avoid interference while performing a complimentary task. Given an observed part of a human’s reaching motion, we thus wish to predict the remainder of the trajectory, and demonstrate that this is effective as a real-time input to the robot for human–robot collaboration tasks. We propose a two-layer framework of Gaussian Mixture Models and an unsupervised online learning algorithm that updates these models with newly-observed trajectories. Unlike previous work in this area which relies on supervised learning methods to build models of human motion, our approach requires no offline training or manual labeling. The main advantage of this unsupervised approach is that it can build models on-the-fly and adapt to new people and new motion styles as they emerge. We test our method on motion capture data from a human-human collaboration experiment to show the early prediction performance. We also present two human–robot workspace sharing experiments of varying difficulty where the robot predicts the human’s motion every 0.1 s. The experimental results suggest that our framework can use human motion predictions to decide on robot motions that avoid the human in real-time applications with high reliability.
KeywordsHuman motion prediction Human–robot collaboration Human–robot manipulation Learning
Prediction of human motion is useful for human–robot interaction, especially for a human and robot collaborating in a shared workspace. Previous work (e.g., Mainprice and Berenson 2013) have shown that early prediction of human reaching motion can help the robot plan its trajectory while avoiding the workspace that the human is going to occupy, which results in a more fluid collaboration in the shared workspace. In this paper, we work on a similar problem as in Mainprice and Berenson (2013), however, we focus more on the algorithm for early prediction of human reaching motion as well as experiments which demonstrate how real-time early prediction can help human–robot collaboration in a shared workspace.
Our framework (see Fig. 1) consists of a two-layer library of Gaussian Mixture Models (GMMs) which model the human reaching motions. The reason to use GMMs as elements of the library is that they can be used as generative models (via Gaussian Mixture Regression) and they can describe arbitrary trajectories. The first layer of our framework is a library of human palm motion, which contains a set of GMMs for human’s palm position. The second layer is composed of a set of GMMs for the human’s arm joint center positions. By using our proposed unsupervised online learning algorithm, our framework can iteratively cluster trajectories based on a geometric similarity measure, and each cluster of trajectories corresponds to a GMM that models those trajectories.
To maintain the motion libraries in our framework, our unsupervised online learning algorithm uses each observed trajectory to determine whether to update the parameters for an existing GMM (using the incremental EM algorithm introduced in Calinon and Billard (2007) or to initialize a new GMM if none of the existing GMMs can “explain” the new trajectory. Thus we can build models on-the-fly (initialize a new GMM) or adapt (update an existing GMM’s parameters) to new people and new motion styles. As our approach can build new GMM models, the framework can handle noise (i.e., atypical reaching motions) by building a new GMM model for the atypical trajectory and adding a membership-proportional prior for each GMM in the library.
An unsupervised learning algorithm for human motion recognition allows us to building the library of reaching motions online.
A two-layer framework that considers different representations of the human arm allows us to recognize and predict reaching motion.
A previous version of this work appears in Luo and Berenson (2015) and Luo et al. (2016), where we tested our method on motion-capture data recorded during a human-human collaboration experiment and a simple real-time human–robot collaboration experiment.The expanded version presented here presents more details about the proposed framework, expands on related work, and presents a more realistic human–robot collaboration experiment to show that our framework can be used in a more complicated environment.
The remainder of this paper is organized as follows: Sect. 2 will introduce related work. Section 3 will present the proposed unsupervised online learning algorithm. Section 4 will show how to use the two-layer framework for human reaching motion early prediction and how to maintain motion libraries in the framework. Section 5 presents the results for human workspace sharing data. Section 6 shows the experimental results for two real-time human–robot collaboration experiments. Finally we conclude in Sect. 7.
2 Related work
Our work contributes to the field of human motion prediction for human–robot collaborations. Most previous work in this area uses supervised learning for human motion recognition and prediction. For example, in Xia et al. (2012), Zhao et al. (2012), Zhang and Parker (2011), the authors propose different types of feature representations of human motions for use inside a supervised learning framework. While these works aim to find underlying features of human motion, another class of work aims to design the underlying models of human motions. For example, Sung et al. (2012b) used a two-layered maximum-entropy Markov model (MEMM) for human activity detection from RGBD images. Koppula et al. (2013) used a Markov random field (MRF) to model both human activities and object affordances. Koppula and Saxena (2016, 2013) used Conditional Random Fields (CRFs) for similar applications. Unlike our work, these works recognize or predict action labels rather than human trajectories. Recently, the work of Koppula and Saxena (2016, 2013) has been extended in Jiang and Saxena (2014) to predict high-dimensional trajectories rather than action labels. However, this still differs from the work presented in this paper because they aim to predict future human actions rather than predict the remainder of a trajectory. In our work, we recognize the observed part of a human’s motion and then predict the remainder of this trajectory. The early prediction of human motion is useful for a robot to react quickly to human motion in a human-robot collaboration task.
The problem of early motion prediction has been studied in previous work. Mainprice and Berenson (2013) used GMMs to model human reaching motions and GMR to predict the remainder of an observed trajectory. Recently Mainprice et al. (2015) explored using Inverse Optimal Control (IOC) to learn a cost function under which demonstrated trajectories are optimal and used that cost function to do iterative re-planning to predict human reaching motions. In related work, Perez-D’Arpino and Shah (2015) used additional task-level information as a prior for early human reaching motion prediction. However, the above methods are all supervised learning algorithms, which require an offline training process and a batch of labeled training data. Unlike these previous works, we consider unsupervised online learning, which requires no manually-labeled data and no offline training process.
Unsupervised human motion prediction has been explored in Weinrich et al. (2013), for the purpose of predicting the trajectories of pedestrians. We are interested in prediction of human reaching motions, which are more challenging to predict because they are more high-dimensional and execute much faster than pedestrian motion. Kulić et al. (2011) proposed an online method for incremental learning of full-body motion primitives. They segmented the human motion into several motion primitives and then used a Hidden Markov Model (HMM) to model both the structure of the primitives and each motion primitive. Unlike their method, we model sets of trajectories using a library of GMMs because we are interested in modeling human reaching motion, which is not clearly separable into primitives. Calinon and Billard (2007) proposed a GMM-based incremental learning of gestures for humanoid robot imitation. The incremental training of a GMM is done by the human manually moving the robot. We use the same incremental EM method proposed in their work as part of our algorithm. However, unlike their work, our framework is given motions corresponding to different tasks and can cluster the motions into different classes. Unsupervised online learning GMMs have been studied in speech recognition Barras et al. (2004), Zhang and Scordilis (2008). Unlike these works, which rely on a well-trained background GMM, our proposed unsupervised online learning algorithm requires no offline training.
We intend to use our framework for human-robot collaboration. Ravichandar and Dani (2015) proposed an algorithm to infer human intention by modeling human motion for human-robot collaboration tasks. Unlike their work which focused on predicting a goal location, we focused on predicting the remaining part of the human trajectory including the positions of arm joint centers which can then be used to extract a goal location. Maeda et al. (2017) proposed a method to learn probabilistic movement primitives for human-robot collaborative tasks. In their work, they learned motion primitives for both a human and robot together and predicted robot trajectories after observing the human motions. Unlike their work which required offline training and focused on the motion primitives of collaborations of the human and robot, our approach is an online algorithm and focuses on the prediction of the entire human trajectory in order to infer the workspace which the human is going to occupy.
3 Unsupervised online learning algorithm
In this section we introduce the core component of our framework: the unsupervised online learning algorithm. Figure 2 shows the pipeline of our proposed algorithm. This algorithm is designed to learn GMMs that model trajectories.
As shown in Fig. 2, the algorithm builds and maintains a trajectory library that consists of multiple GMMs where each GMM \(G_i\) represents a class of trajectories. Given a trajectory \(X_j\), the algorithm will first calculate the probabilities of this trajectory given each GMM - \(p(X_j|G_i)\) for \(i = 1,2,\ldots \) and then calculate the posterior probability \(p(G_i|X_j)\) (explained in Sect. 3.2). If all the posterior probabilities are smaller than a specified threshold, the algorithm will use this trajectory \(X_j\) to initialize a new GMM and store it in the trajectory library. If some posterior probabilities are larger than that threshold, the algorithm will classify (maximum a posteriori estimation) this trajectory into a GMM class \(G_k\) with the highest probability \(p(G_k|X_j)\). Then the algorithm will update the parameters of the GMM \(G_k\).
3.1 Gaussian mixture models for trajectories
In this section, we discuss how to use GMM to model trajectory data. As shown in Algorithm 1, each GMM in the trajectory library represents a class of trajectories. \(G_i\) for \(i=1,2,3,\ldots \) represents each GMM in the library. \(X_j\) for \(j=1,2,3,\ldots \) represents a given trajectory. \(X_j\) is an \(L\,\times \,D\) matrix where L is the number of points in a trajectory and D is the number of feature dimensions of the trajectory.
3.2 Classifying trajectories
3.3 Initializing a GMM from single trajectory
3.4 Update GMM parameters
4 Two-layer framework for early prediction of human reaching motion
In the previous section, we introduced our proposed unsupervised online learning algorithm which updates parameters of a GMM or builds a new GMM to update the trajectory library. In this paper, we use this algorithm for human reaching motions. We consider three types of features to represent human motions: (1) palm position (PP), (2) arm joint center positions (AJCP), (3) arm configurations (AC). The feature comparison experiment in Sect. 5.1 will show comparisons between these representations and the reason why we only use the first two feature representations in our framework. In this section, we will introduce how we use our proposed framework to predict human reaching motions and how to use the proposed unsupervised online learning algorithm to maintain the motion libraries in the framework.
Feature representation comparison
4.1 Early prediction of human reaching motion
The purpose of early human motion prediction is to regress the remainder of a human’s trajectory based on the observed part of the trajectory. We decompose the human motion early prediction problem into two steps: (1) human motion early recognition and (2) human motion trajectory regression. As we focus on the application of human motion prediction for human-robot collaboration tasks, we require regressing the whole arm trajectory (not only the palm trajectory) in order to compute the human’s workspace occupancy. However, the results in Table 2 show that the proposed unsupervised online learning algorithm using PP features significantly outperforms the algorithm using AJCP features in the recognition task. As early recognition is vital for the early prediction problem, we propose a two layer framework for human reaching motion early prediction (Fig. 1). The first layer uses PP features and the second layer uses AJCP features. This two-layer framework can take the advantages of PP features (better recognition performance) and can still model the whole arm trajectory. Both layers use the proposed unsupervised online learning algorithm to build their motion libraries. The first layer builds a palm motion library and the second layer builds an arm motion library for each palm motion class as shown in Fig. 1. Note that as an online system, the framework will observe human motion trajectories one-by-one and human motion postures from each trajectory one-by-one. At the beginning of each trajectory, the framework will only do early prediction based on the current learned models. After observing this trajectory, the framework will then update the human motion libraries using the method in Fig. 2 for each layer.
Feature extraction The framework observes the beginning part of the human motion and extracts two types of features: PP and AJCP.
Human motion early recognition The first layer of the framework takes the PP features and uses MAP to estimate the palm motion class ID i (GMM ID in the library). As each GMM in the first layer links to a different human arm motion library in the second layer, we then use the palm motion class ID i found in the first layer to find the corresponding human arm motion library in the second layer. Then the second layer takes the AJCP features and uses MAP to estimate the human arm motion class ID j in this arm motion library.
Human motion trajectory regression The second layer computes the regressed trajectory \(X'\) using the method of Calinon and Billard (2007) with the GMM parameters of the jth human arm motion class in the arm motion library for the ith palm motion class.
Normalize regressed trajectory Move the regressed trajectory such that the beginning posture of the regressed trajectory overlaps with the end posture of the observed trajectory.
4.2 Updating motion libraries in two-layer framework
To update the libraries in the framework, we run Algorithm 1 at both layers as follows: In the first layer of the framework, there is only one palm motion library. Thus we can directly run Algorithm 1 to update this palm motion library. If the algorithm updates one GMM in the palm motion library, then we find the arm motion library which links to this GMM in the second layer and run Algorithm 1 to update this corresponding arm motion library. If the algorithm builds a new GMM in the palm motion library, then we use Algorithm 1 to generate a new arm motion library with a new GMM in the second layer and link it to the new GMM in the palm motion library.
5 Results from human workspace sharing data
The assembly task required the “active” human to move balls between location 2 and location 4 and between location 2 and location 6 as shown in Fig. 4b. We used a VICON system to capture the human motions. Human subjects wore a suit consisting of nine markers and three rigid plates which were placed following the standards used in the field of biomechanics (Wu et al. 2005). Our suit consists of rigid marker plates attached to a belt, a headband and an elbow pad, a marker on the back of the hand, two on each side of the wrist, two on either side of the shoulder, and two markers straddling the sternum and xyphoid process. The VICON system runs at 100 fps. We used recordings from 3 pairs of human subjects doing the assembly task and each pair performed the assembly task 6 times. Thus we have 18 sets of experiment data. There were a total of 254 trajectories captured from the three “active” human subjects. The average number of frames in each trajectory is 107. The algorithm is implemented in MATLAB. The average runtime to process a trajectory (update the parameters or add a new GMM model) is 0.1 s and the average runtime for one call of the prediction process is 0.0036 s.
In the proposed two-layer framework, we setup the parameters as follows: To initialize a new GMM, we set the \(\varDelta =10\) and generate 5 random trajectories for PP, however, we set \(\varDelta =45\) and generate 10 random trajectories for AJCP. We set the threshold as \(-8\) in the first layer and \(-108\) in the second layer. The parameters were found by manual tuning.
Below we compare feature representations and methods in terms of their precision, recall, and accuracy in trajectory prediction. Precision and recall are computed as the average precision and recall over all 4 classes of motions. For the supervised or semi-supervised methods, the number of GMMs is fixed and each GMM has the same label as the training data that trains this GMM. For the unsupervised methods, the number of GMMs is not fixed and we are actually clustering the trajectories. When we compute the precision and recall, the trajectories in the unsupervised GMMs will take the label of the ground truth label of the majority of the trajectories in that GMM. The analysis is set up this way to show that we cluster trajectories for the same task together, even though we do not know what the tasks are.
5.1 Feature comparison experiment
In this section we evaluate which features allow GMMs to model human reaching motions most effectively. We ran leave-one-out experiments to compare different human motion feature representations using supervised GMMs. In each round of the leave-one-out experiment, we used 1 of 18 sets of the experiment data as the testing data and other 17 sets as training data. We considered three features: PP, AJCP and AC. The AJCP are positions for the right arm’s palm, wrist, elbow and shoulder, which are recorded by our motion capture system. In this paper, we only considered the joint angles of the arm in the AC feature, which we obtain through IK on the set of markers. The dimensions of each feature representation are 3, 12, and 9, respectively.
Table 1 shows the performance for each type of feature using supervised GMMs. Both the PP and AJCP outperform the AC and have no significant difference between them. Although the AC tries to reduce the influence of different body types, people with different body type may perform the same motion with different joint angles. Thus we only used PP features and AJCP features in the rest of the experiments.
5.2 Human reaching motion trajectory recognition
Human reaching motion trajectory recognition
UOLA (PP, MLE)
UOLA (AJCP, MLE)
UOLA (PP, uniform)
UOLA (AJCP, uniform)
UOLA (PP, ratio)
UOLA (AJCP, ratio)
Figure 5 shows the performance changes along with the trajectory indices streaming into the system. Here we only considered the PP feature as it always outperforms the AJCP feature. The figure shows that both proposed unsupervised online learning algorithm with ratio prior and uniform prior consistently outperform the baselines. The figure also shows that MAP estimation with ratio prior and uniform prior outperform the MLE estimation and have no significant difference between them.
5.3 Human reaching motion trajectory early recognition
5.4 Human reaching motion trajectory prediction
In this section, we used the same setup as the previous section, however, we focused on the performance of motion trajectory prediction for each method. In the previous experiment we found that the UOLA variants with PP features outperform the UOLA with AJCP features for early recognition. This result shows that PP features are most useful for early recognition, however, since we wish to predict the entire arm’s trajectory, we require a second layer to perform this prediction that uses AJCP features. Thus in this experiment we use both layers of our framework: the first with PP features, and the second with AJCP features. The evaluation method is the DTW distance between the predicted trajectory and the remainder of the given testing trajectory. Figure 7 shows the relationship between the average DTW distance and the percent observed of the trajectory. All of our proposed two-layer framework variants outperform the baselines. The framework using ratio prior outperforms the framework variants using uniform prior and using no prior. Note that our proposed framework with the ratio prior significantly outperforms the baselines when given a small percentage of the observed part of the trajectory (e.g., 20, 30%). This result suggests that, using our proposed framework, the robot is likely to infer the human’s true motion more quickly than the baseline methods.
6 Application to human-robot workspace sharing
In this section, we present results from two human-robot experiments to show that our framework can help robots plan and re-plan their motions in order to avoid disturbing human motions. We used the proposed two layer framework with the ratio prior in these experiments. Note that in all experiments in this section, we ran our two-layer framework with ratio prior. The predicted trajectories are all generated from the second layer of our framework using the AJCP feature.
6.1 Experiment 1
6.1.1 Experimental setup
6.1.2 Experiment 1 results
Of the 300 trials there were three trials that failed (the robot reached for the wrong color once and the robot re-planned correctly but still touched the human twice). There were 106 trials where the robot directly went to the correct target, which means that the framework successfully predicted the human motions using only the first 10 frames. There were 191 trials where the robot re-planned. Figure 10 shows examples of two trials in which re-planning occurs due to incorrect initial predictions. The top row is a re-planned trajectory that is consistent with our findings of average re-planning times (Fig. 11a), while the bottom row shows an example of a later change in prediction resulting in a smaller distance between robot and subject.
6.2 Experiment 2
6.2.1 Experiment setup
6.2.2 Experiment result
Figure 14 shows two different example trials in the experiment. The top row shows that the human reaches towards Goal 3. In this case, the right arm should wipe the table for the human because the left part of the human workspace is free and the left arm should reach to Goal 2. In this trial, as the robot’s initial prediction is correct, the robot can directly execute the correct trajectories as mentioned before. The bottom row shows that the human reaches towards Goal 2. In this case, the robot right arm should not move because the left part of the human workspace will be occupied by the human and the robot left arm should reach to Goal 2. In this trial, the robot’s initial prediction is not correct, so the robot’s right arm moves at the beginning (as shown in Fig. 14h). However, the robot changes the prediction correctly after observing more of the human motion. Thus, the robot re-plans its trajectory; the right arm moves back to the initial position and the left arm moves toward Goal 3.
Of the 780 trials, there were 725 success trials for a success rate of \(93.0\%\). There were four types of failure cases: (1) the robot touched white platform (3 times); (2) the robot reached to the wrong position (29 times); (3) the robot interfered with the human but resolved later (36 times); (4) the robot significantly interfered with the human such that the human could not finish the task (4 times). Note that some trials may exhibit more than one failure case. The first failure case was not caused by our algorithm, but rather an anomaly produced by the interface with the PR2’s controller. For the second failure case, there were only 7 failures caused by an incorrect prediction from our proposed framework. The other 29 failures were caused by the way we computed the predicted goal region relative to that region’s position in the real world. However this problem can be easily solved by calibration of goal positions (discussed in Sect. 6.3). If we ignore this predicted goal computation problem, there were 46 failed trials in total. For the third failure case, the interference is not significant and can be resolved later. Similar interferences between human and human were observed in the previous human workspace sharing data of the human-human collaboration experiment. For the fourth failure, the problem is that the initial prediction is not correct and the robot interferes with the human. The robot continued to change the prediction as the human moved back and forth. The human then ended the trial because they could not complete the task.
There are two reasons why the early prediction performance in the second experiment is not as good as in the first experiment. First, the experimental setup is more challenging, due to more goal regions which are spaced closer to each other. For example, the beginning part of reaching motions towards Goal 2 are similar to the beginning part of the motion towards Goal 1 or Goal 3, requiring more time to observe unambiguous data that would be necessary for a correct prediction. Second, as in the first experiment, we computed the Euclidean distance between the predicted palm position with the center of the goal regions and consider the closest goal region as the predicted goal. This simple method worked well in the first experiment because the subject’s task was to put his or her hand on the center of the goal region. In this experiment however, the subject grasped a cylinder and put it on the center of the goal region. Thus there is a distance between the palm position and the center of the cylinder dependant on how the human handled the cylinder and the human hand size. This problem does not affect the algorithms in our framework but it does influence the computation of the predicted goal. This problem manifested itself in some trials when the human reached towards Goal 1. In such trials, our framework correctly classified the human trajectory into the motion class for Goal 1, but the subject’s palm rested closer to the center of Goal 2 than that of Goal 1 causing an incorrect prediction. This problem can be solved by calibration of the goal position.
However, despite the calibration issues, the recognition performance of our framework is still good. There were 772 correctly recognized trajectories of all 780 trajectories which resulted in \(99.0\%\) recognition accuracy. In fact, only GMM 10 is a confused motion class (4 trajectories for goal 1 and 3 trajectories for goal 2). These trials are the cases in which the human hesitated to resolve a conflicting robot motion, swaying between two otherwise separate areas of the workspace. This is an example of the atypical motions we would expect to observe when working with humans.
We have presented a two-layer framework for unsupervised online human reaching motion recognition and early prediction. The framework consists of a two-layer library of GMMs. The library grows if it cannot “explain” a new observed trajectory by using the proposed unsupervised online learning algorithm. Given an observed portion of a trajectory, the framework can predict the remainder of the trajectory by first determining which GMM it belongs to, and then using GMR to predict the remainder of the trajectory. The proposed unsupervised online learning algorithm requires no offline training process or manual categorization of trajectories. The results show that our framework can generate models on-the-fly and adapt to new people and new motion styles as they arise. Results from two experiments where a human and robot share a workspace show that our framework can be used to decide on robot motions that avoid the human in real-time applications with high reliability. Future work will explore how to use GMR to generate smoother predicted trajectories and explore fast motion planning algorithms for the robot.
Funding was provided by National Science Foundation (Grant No. 1317462).
Supplementary material 1 (mp4 236938 KB)
- Barras, C., Meignier, S., & Gauvain, J. L. (2004). Unsupervised online adaptation for speaker verification over the telephone. In: The speaker and language recognition workshop.Google Scholar
- Bennewitz, M., Burgard, W., & Thrun, S. (2002). Learning motion patterns of persons for mobile service robots. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 4, 3601–3606.Google Scholar
- Bruce, A., & Gordon, G. (2004). Better motion prediction for people-tracking. In: Proceedings of the IEEE International Conference on Robotics and Automation(ICRA).Google Scholar
- Calinon, S. (2009). Robot programming by demonstration: A probabilistic approach. Lausanne: EPFL Press.Google Scholar
- Calinon, S., & Billard, A. (2007). Incremental learning of gestures by imitation in a humanoid robot. In: Proceedings of the ACM/IEEE International Conference on Human-robot Interaction, pp. 255–262.Google Scholar
- Cederborg, T., Li, M., Baranes, A., & Oudeyer, P.Y. (2010). Incremental local online gaussian mixture regression for imitation learning of multiple tasks. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 267–274.Google Scholar
- Jiang, Y., & Saxena, A. (2014). Modeling high-dimensional humans for activity anticipation using gaussian process latent CRFs. In: Robotics: Science and systems.Google Scholar
- Kalakrishnan, M., Chitta, S., Theodorou, E., Pastor, P., & Schaal, S. (2011). STOMP: Stochastic trajectory optimization for motion planning. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 4569–4574.Google Scholar
- Koppula, H. S., & Saxena, A. (2013). Learning spatio-temporal structure from rgb-d videos for human activity detection and anticipation. In: Proceedings of the 30th International Conference on Machine Learning (ICML), pp. 792–800.Google Scholar
- Kulić, D., Ott, C., Lee, D., Ishikawa, J., & Nakamura, Y. (2011). Incremental learning of full body motion primitives and their sequencing through human motion observation. The International Journal of Robotics Research, 31(3), 330–345.Google Scholar
- Luo, R., & Berenson, D. (2015). A framework for unsupervised online human reaching motion recognition and early prediction. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2426–2433.Google Scholar
- Luo, R., Hayne, R., & Berenson, D. (2016). Early prediction of human reaching motion for long-term human-robot collaboration. In: AI for long-term autonomy workshop at IEEE International Conference on Robotics and Automation (ICRA)Google Scholar
- Maeda, G. J., Neumann, G., Ewerton, M., Lioutikov, R., Kroemer, O., & Peters, J. (2017). Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks. Autonomous Robots, 41(3), 593–612.Google Scholar
- Mainprice, J., & Berenson, D. (2013). Human-robot collaborative manipulation planning using early prediction of human motion. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 299–306Google Scholar
- Mainprice, J., Hayne, R., & Berenson, D. (2015). Predicting human reaching motion in collaborative tasks using inverse optimal control and iterative re-planning. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 885–892.Google Scholar
- Nyga, D., Tenorth, M., & Beetz, M. (2011). How-models of human reaching movements in the context of everyday manipulation activities. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 6221–6226.Google Scholar
- Perez-D’Arpino, C., & Shah, J. A. (2015). Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 6175–6182.Google Scholar
- Ravichandar, H. C., & Dani, A. (2015). Human intention inference and motion modeling using approximate em with online learning. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1819–1824.Google Scholar
- Sung, C., Feldman, D., & Rus, D. (2012a). Trajectory clustering for motion prediction. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1547–1552.Google Scholar
- Sung, J., Ponce, C., Selman, B., & Saxena, A. (2012b). Unstructured human activity detection from rgb-d images. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 842–849.Google Scholar
- Weinrich, C., Volkhardt, M., Einhorn, E., & Gross, H. M. (2013). Prediction of human collision avoidance behavior by lifelong learning for socially compliant robot navigation. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 376–381.Google Scholar
- Wu, G., Van der Helm, F. C., Veeger, H. D., Makhsous, M., Van Roy, P., Anglin, C., et al. (2005). ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion part ii: shoulder, elbow, wrist and hand. Journal of Biomechanics, 38(5), 981–992.CrossRefGoogle Scholar
- Xia, L., Chen, C. C., & Aggarwal, J. (2012). View invariant human action recognition using histograms of 3d joints. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 20–27.Google Scholar
- Zhang, H., & Parker, L. E. (2011). 4-dimensional local spatio-temporal features for human activity recognition. In: Proceedings of the IEEE/RSJ international conference on Intelligent robots and systems (IROS), pp. 2044–2049.Google Scholar
- Zhao, Y., Liu, Z., Yang, L., & Cheng, H. (2012). Combing rgb and depth map features for human activity recognition. In: Proceedings of the Asia-Pacific Signal Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1–4.Google Scholar