Exploring High-Level Plane Primitives for Indoor 3D Reconstruction with a Hand-held RGB-D Camera

Abstract

Given a hand-held RGB-D camera (e.g. Kinect), methods such as Structure from Motion (SfM) and Iterative Closest Point (ICP), perform poorly when reconstructing indoor scenes with few image features or little geometric structure information. In this paper, we propose to extract high level primitives–planes–from an RGB-D camera, in addition to low level image features (e.g. SIFT), to better constrain the problem and help improve indoor 3D reconstruction. Our work has two major contributions: first, for frame to frame matching, we propose a new scheme which takes into account both low-level appearance feature correspondences in RGB image and high-level plane correspondences in depth image. Second, in the global bundle adjustment step, we formulate a novel error measurement that not only takes into account the traditional 3D point re-projection errors, but also the planar surface alignment errors. We demonstrate with real datasets that our method with plane constraints achieves more accurate and more appealing results comparing with other state-of-the-art scene reconstruction algorithms in aforementioned challenging indoor scenarios.