Temporal-Spatial Local Gaussian Process Experts for Human Pose Estimation

Abstract

Within a discriminative framework for human pose estimation, modeling the mapping from feature space to pose space is challenging as we are required to handle the multimodal conditional distribution in a high-dimensional space. However, to build the mapping, current techniques usually involve a large set of training samples in the learning process but are limited in their capability to deal with multimodality. In this work, we propose a novel online sparse Gaussian Process (GP) regression model combining both temporal and spatial information. We exploit the fact that for a given test input, its output is mainly determined by the training samples potentially residing in its neighbor domain in the input-output unified space. This leads to a local mixture GP experts system, where the GP experts are defined in the local neighborhoods with the variational covariance function adapting to the specific regions. For the nonlinear human motion series, we integrate the temporal and spatial experts into a seamless system to handle multimodality. All the local experts are defined online within very small neighborhoods, so learning and inference are extremely efficient. We conduct extensive experiments on the real HumanEva database to verify the efficacy of the proposed model, obtaining significant improvement against the previous models.